You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-16 - 13:44

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Thu Apr 16, 2026 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
8019212248,1202sleep08053-21install19:11:170
84082108910,1067sleep231229-21kworker/2:122:34:272
22502210775,1007sleep013818-1kworker/0:2H21:36:570
225402102716,996sleep29353-21kworker/u8:321:37:282
22876252117,490sleep322915-21ntpq21:37:533
21158249911,398sleep021270-21latency_hist19:51:170
24018247612,449sleep224022-21diskstats21:41:352
15625247017,438sleep315632-21ntp_states19:31:543
7683241914,391sleep17723-21ntpq22:31:401
243062364127,213sleep00-21swapper/021:41:410
120923578,339sleep013441-21snmpd23:55:510
418923569,338sleep215044-1kworker/2:0H22:21:362
8188232210,234sleep08246-21cut20:51:360
8188232210,234sleep08246-21cut20:51:360
74962313112,181sleep07499-21cron00:16:140
9442224288,74sleep09525-21switchtime22:36:430
9442224288,74sleep09525-21switchtime22:36:430
1836224216,199sleep00-21swapper/023:56:360
12542224184,140sleep112548-21diskstats00:31:331
215322338,154sleep02236-21wc22:16:170
10406222114,187sleep00-21swapper/000:23:290
29426221018,174sleep329506-21tune2fs23:41:333
2121322105,30sleep00-21swapper/023:15:430
17177219527,151sleep017195-21idleruntime-cro21:21:150
9855219019,22sleep00-21swapper/020:56:360
17214219011,129sleep00-21swapper/019:36:440
22872218930,25sleep00-21swapper/023:21:160
1345221809,158sleep013532-21tune2fs19:26:350
33052179135,22sleep00-21swapper/020:36:340
765521747,19sleep00-21swapper/022:31:390
26562174135,28sleep02657-21sshd20:33:270
533321739,29sleep05458-21timerwakeupswit00:06:450
533321739,29sleep05458-21timerwakeupswit00:06:450
27031217311,140sleep027186-21grep20:06:470
32281217012,140sleep013441-21snmpd20:26:170
1274821707,151sleep00-21swapper/019:22:060
32164216811,23sleep00-21swapper/023:51:150
17600216715,16sleep00-21swapper/021:21:350
30597216513,138sleep0222-21systemd-journal23:46:150
934121620,6sleep30-21swapper/300:21:253
927421629,141sleep09358-21awk19:11:450
4430216119,122sleep34498-21latency_hist00:06:163
4430216119,122sleep34498-21latency_hist00:06:163
18655216119,119sleep00-21swapper/019:41:400
13047216121,64sleep00-21swapper/022:48:590
13047216121,64sleep00-21swapper/022:48:590
2483221569,99sleep00-21swapper/020:01:350
18249215610,130sleep218268-21apt-get19:41:262
6382215217,120sleep26387-21entropy20:46:332
1563421506,130sleep00-21swapper/019:31:560
2214221479,122sleep022183-21grep19:52:340
526821465,11sleep00-21swapper/022:24:000
2987621459,24sleep00-21swapper/022:01:150
17338214412,117sleep117375-21ntpq23:02:161
2315621427,125sleep013441-21snmpd19:56:220
334214122,11sleep00-21swapper/022:09:020
2169021405,29sleep00-21swapper/021:34:310
578821396,31sleep00-21swapper/020:43:290
2817721395,11sleep00-21swapper/021:54:490
2746221395,63sleep00-21swapper/023:36:050
24938213811,39sleep00-21swapper/023:26:370
12859213812,20sleep00-21swapper/000:31:400
2561321340,32sleep125617-21unixbench_singl20:01:471
3252521308,17sleep00-21swapper/020:26:290
14089212984,27sleep00-21swapper/022:51:450
1371221295,33sleep00-21swapper/021:06:580
2442321258,85sleep024459-21sed23:26:160
1157721258,43sleep011705-21ls19:21:180
30104212416,89sleep230229-21/usr/sbin/munin20:16:462
5896212341,57sleep16048-21grep19:02:541
3787212275,34sleep03904-21latency_hist22:21:170
1041221228,103sleep010408-21apt-get22:41:280
1041221228,103sleep010408-21apt-get22:41:280
1659621217,97sleep00-21swapper/023:01:180
1901121208,15sleep00-21swapper/021:26:270
1901121208,15sleep00-21swapper/021:26:270
781721178,93sleep00-21swapper/000:16:300
734021158,34sleep00-21swapper/020:48:250
6434211219,67sleep10-21swapper/100:11:351
4203211211,14sleep10-21swapper/122:21:431
2828721129,23sleep10-21swapper/121:56:161
1861821126,64sleep00-21swapper/023:06:350
1861821126,64sleep00-21swapper/023:06:350
31345211015,76sleep331378-21/usr/sbin/munin20:21:403
631521095,29sleep00-21swapper/022:26:460
1661210820,72sleep11736-21fschecks_time23:56:331
1413421089,16sleep00-21swapper/021:11:180
10081210817,72sleep1222-21systemd-journal19:16:181
6999210721,69sleep1222-21systemd-journal22:31:151
2638521076,28sleep00-21swapper/021:46:460
1225621076,28sleep00-21swapper/021:03:230
3035921067,27sleep00-21swapper/020:17:050
31713210416,73sleep231720-21grep22:06:302
1606921045,28sleep00-21swapper/022:57:300
12228210414,70sleep312244-21/usr/sbin/munin22:46:353
12228210414,70sleep312244-21/usr/sbin/munin22:46:353
8057210344,40sleep18193-21sed19:11:181
1707210321,66sleep11797-21fschecks_time20:31:351
23204210225,57sleep123220-21uniq23:21:331
1137521028,14sleep00-21swapper/000:26:430
3280210113,74sleep33300-21grep20:36:333
2890221005,26sleep00-21swapper/023:39:200
3132929721,30sleep00-21swapper/022:05:030
2527029630,29sleep10-21swapper/121:46:191
35472956,12sleep00-21swapper/000:01:410
35472956,12sleep00-21swapper/000:01:410
2065329416,57sleep120666-21fschecks_count21:31:331
755829219,53sleep37587-21idleruntime-cro20:51:153
376629222,46sleep13094-21/usr/sbin/munin00:01:461
376629222,46sleep13094-21/usr/sbin/munin00:01:461
1670229117,33sleep30-21swapper/319:36:273
288942905,11sleep00-21swapper/020:13:290
2125229016,51sleep321300-21tr23:16:143
245628912,16sleep20-21swapper/223:56:462
1642828920,51sleep316592-21latency_hist23:01:173
691028625,39sleep37052-21unixbench_multi00:11:453
2648228611,47sleep20-21swapper/223:31:352
135752869,18sleep10-21swapper/121:06:471
446928521,30sleep30-21swapper/322:21:493
207228517,50sleep22103-21cron22:16:152
17982850,28sleep21799-21tune2fs20:31:352
481528420,42sleep25073-21modprobe19:01:182
2184328411,20sleep20-21swapper/223:16:382
2363428319,15sleep30-21swapper/319:56:413
1876628318,49sleep118782-21latency_hist21:26:161
883728213,55sleep18838-21munin-node22:36:241
883728213,55sleep18838-21munin-node22:36:241
3252728214,53sleep232573-21/usr/sbin/munin20:26:312
2592528214,19sleep20-21swapper/223:31:152
718228117,41sleep36694-21/usr/sbin/munin19:06:433
1687928124,37sleep316873-21apt-get23:01:293
1639328115,52sleep216427-21apt-key23:01:162
2905728010,17sleep20-21swapper/221:56:412
768527927,25sleep10-21swapper/120:51:171
3218727916,49sleep12-21kthreadd23:51:161
3214327929,30sleep3222-21systemd-journal23:51:143
2585127921,41sleep325897-21/usr/sbin/munin21:46:373
2449227918,29sleep20-21swapper/220:01:162
2019227920,35sleep20-21swapper/219:46:392
1475827923,24sleep30-21swapper/319:31:183
390527827,32sleep10-21swapper/122:21:181
2910127814,33sleep30-21swapper/323:41:173
2839827818,26sleep30-21swapper/321:56:183
944827720,39sleep29467-21fschecks_count00:21:332
2391527722,39sleep124006-21awk19:56:471
1873827614,44sleep318760-21/usr/sbin/munin19:41:443
52827516,28sleep30-21swapper/320:26:443
2860627516,42sleep328652-21grep21:56:303
2690827518,40sleep327027-21sed20:06:463
610627420,37sleep26132-21awk22:26:402
3219827413,16sleep30-21swapper/322:06:433
3198627416,18sleep10-21swapper/122:06:371
2351627420,24sleep30-21swapper/323:21:393
2093527417,24sleep20-21swapper/221:31:382
934627316,26sleep30-21swapper/320:56:153
934627316,26sleep30-21swapper/320:56:153
2673927319,26sleep10-21swapper/121:51:151
2545127313,45sleep225472-21dpkg21:46:242
2475827323,24sleep10-21swapper/123:26:341
513527224,30sleep00-21swapper/019:01:320
3243627216,26sleep20-21swapper/223:51:292
2402327220,22sleep30-21swapper/321:41:363
1491027224,22sleep30-21swapper/321:11:453
1322327217,21sleep30-21swapper/300:31:473
559127126,23sleep30-21swapper/319:02:483
2157127118,30sleep30-21swapper/323:16:313
2094227111,47sleep20-21swapper/223:12:222
1093027120,34sleep210948-21awk21:01:152
976127028,25sleep30-21swapper/320:56:353
920727013,41sleep19209-21sshd20:54:281
920727013,41sleep19209-21sshd20:54:281
315412700,9sleep20-21swapper/223:46:442
2207927015,27sleep10-21swapper/119:52:311
188512707,13sleep10-21swapper/123:06:401
188512707,13sleep10-21swapper/123:06:401
1648927021,24sleep20-21swapper/219:36:172
912426924,24sleep20-21swapper/200:21:172
808826914,19sleep30-21swapper/300:16:373
2358426910,45sleep223635-21processes23:21:422
2027426921,26sleep30-21swapper/321:31:153
2027426921,26sleep30-21swapper/321:31:153
1470526918,30sleep10-21swapper/121:11:391
1222526927,22sleep20-21swapper/200:31:182
481926817,24sleep10-21swapper/120:41:321
2792826812,40sleep327970-21fschecks_count20:11:353
1690426817,33sleep2367-21sshd23:01:442
649826716,24sleep20-21swapper/200:11:362
2182726718,25sleep30-21swapper/321:36:153
1383326721,24sleep10-21swapper/119:26:441
1299526710,40sleep213045-21/usr/sbin/munin21:06:362
1735426625,26sleep217477-21timerwakeupswit19:36:462
365226519,28sleep23695-21gpgv22:21:152
2918726519,19sleep20-21swapper/220:16:182
2901026515,20sleep10-21swapper/121:56:401
2630026518,22sleep30-21swapper/323:31:323
2535126518,26sleep30-21swapper/320:01:463
2286726522,23sleep20-21swapper/219:56:162
1777026514,33sleep317837-21memory21:21:383
1376726521,23sleep20-21swapper/219:26:412
1233126523,22sleep10-21swapper/122:46:371
1233126523,22sleep10-21swapper/122:46:371
1227026523,19sleep10-21swapper/119:21:431
1026226518,32sleep310258-21apt-get19:16:283
623626418,19sleep30-21swapper/322:26:453
3147426417,25sleep20-21swapper/222:06:182
1254926421,27sleep212581-21tune2fs00:31:342
389426313,33sleep24023-21users00:01:472
389426313,33sleep24023-21users00:01:472
2942926316,29sleep329497-21df20:16:313
2563426321,22sleep30-21swapper/323:27:293
2372626319,18sleep10-21swapper/121:41:181
1974326325,19sleep20-21swapper/223:11:172
1974326325,19sleep20-21swapper/223:11:172
123726320,27sleep31261-21munin-run23:56:153
109202639,12sleep10-21swapper/100:26:321
2133126218,22sleep10-21swapper/123:16:161
1722726217,27sleep217196-21latency_hist21:21:172
1484626214,27sleep30-21swapper/322:56:163
1232026220,18sleep30-21swapper/319:21:453
737226110,11sleep30-21swapper/322:31:343
3073326120,23sleep10-21swapper/120:21:171
294726115,21sleep30-21swapper/320:36:173
252792615,12sleep20-21swapper/220:01:452
2049926114,24sleep10-21swapper/119:46:461
1409026122,24sleep314131-21latency_hist21:11:173
1068426118,20sleep20-21swapper/222:41:382
1068426118,20sleep20-21swapper/222:41:382
935926014,20sleep10-21swapper/119:11:461
2290826016,22sleep10-21swapper/123:21:171
216092609,33sleep30-21swapper/319:51:283
1947526016,24sleep20-21swapper/221:26:422
1947526016,24sleep20-21swapper/221:26:422
1923226016,28sleep119289-21irqstats21:26:371
1923226016,28sleep119289-21irqstats21:26:371
1076326018,21sleep20-21swapper/219:16:442
607725914,29sleep30-21swapper/320:46:183
1416625921,21sleep10-21swapper/122:51:461
56172588,12sleep30-21swapper/300:07:133
56172588,12sleep30-21swapper/300:07:133
2629025817,25sleep126287-21apt-get20:06:281
1672625813,27sleep116721-21apt-get19:36:291
1573025817,20sleep10-21swapper/122:56:471
986525713,28sleep19941-21/usr/sbin/munin00:21:441
875025713,26sleep38759-21/usr/sbin/munin19:11:323
48832576,12sleep20-21swapper/200:06:352
48832576,12sleep20-21swapper/200:06:352
42125721,20sleep10-21swapper/123:51:411
340325716,25sleep13451-21if_eth020:36:351
1840625714,26sleep318425-21apt-get23:06:253
1840625714,26sleep318425-21apt-get23:06:253
1110125716,15sleep30-21swapper/322:41:473
1110125716,15sleep30-21swapper/322:41:473
2960925620,16sleep20-21swapper/223:41:362
2835925614,19sleep20-21swapper/221:56:172
1164225615,22sleep111679-21ntp_states21:01:421
1017725620,18sleep20-21swapper/222:41:182
1017725620,18sleep20-21swapper/222:41:182
735625517,17sleep20-21swapper/219:06:472
369625514,26sleep33785-21apt-config22:21:163
3097625516,23sleep230967-21apt-get20:21:312
1149825510,18sleep20-21swapper/221:01:392
1091725515,24sleep310911-21apt-get00:26:303
830225417,18sleep10-21swapper/100:16:431
79625416,23sleep3799-21apt-get22:11:273
32925416,22sleep132499-21/usr/sbin/munin20:26:381
2786625412,25sleep127873-21df_inode23:36:311
1878625418,15sleep20-21swapper/221:26:172
156125412,25sleep31556-21apt-get23:56:273
805825313,23sleep20-21swapper/220:51:262
805825313,23sleep20-21swapper/220:51:262
604725316,23sleep26076-21latency_hist20:46:172
3233325316,19sleep10-21swapper/120:26:181
2661025314,21sleep126644-21/usr/sbin/munin23:31:381
2652025312,23sleep226563-21irqstats20:06:372
2022225317,19sleep30-21swapper/319:46:403
1262525313,22sleep212715-21users19:21:482
914325220,14sleep20-21swapper/219:11:432
1835525212,23sleep118362-21diskstats19:41:331
1121825213,23sleep311216-21apt-get21:01:283
466725112,24sleep14664-21apt-get00:06:261
466725112,24sleep14664-21apt-get00:06:261
3016125115,21sleep130156-21apt-get22:01:281
1360925114,21sleep20-21swapper/222:51:352
80625015,18sleep10-21swapper/122:11:291
2955125011,16sleep10-21swapper/123:41:351
3016524914,20sleep30-21swapper/322:01:303
1589424912,21sleep115913-21apt-get21:16:271
1511924911,23sleep114636-21apt-get19:31:241
1454424715,15sleep20-21swapper/221:11:362
1048824711,20sleep110497-21df_inode22:41:331
1048824711,20sleep110497-21df_inode22:41:331
319224612,17sleep20-21swapper/220:36:282
935424515,15sleep30-21swapper/322:36:423
935424515,15sleep30-21swapper/322:36:423
1647024514,12sleep30-21swapper/321:16:463
2824524410,20sleep228251-21ntp_states23:36:402
1020724414,16sleep10-21swapper/120:56:461
1217124212,13sleep20-21swapper/222:46:342
1217124212,13sleep20-21swapper/222:46:342
279132378,12sleep10-21swapper/120:11:331
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional