You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-14 - 05:02

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7s.osadl.org (updated Thu May 14, 2026 00:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1113629988,976sleep02804-21kworker/u8:000:04:390
21926261011,583sleep021961-21ntpq22:54:530
31287249749,431sleep131302-21awk21:40:001
2535724618,400sleep125441-21perl19:39:471
4424237920,339sleep04391-21apt-get21:59:580
15165228411,259sleep315208-21ntpq20:49:513
11967224148,179sleep112121-21latency_hist22:24:301
15059223246,167sleep114763-21/usr/sbin/munin20:49:491
19283223146,169sleep119434-21latency_hist21:04:311
4031222684,29sleep14244-21date21:59:301
32173222641,31sleep10-21swapper/121:44:491
28158222451,122sleep128195-21ntp_kernel_pll_23:14:531
28158222451,122sleep128195-21ntp_kernel_pll_23:14:531
30598221112,186sleep130664-21grep19:55:021
20321220713,137sleep120338-21latency_hist19:24:291
621421998,37sleep10-21swapper/123:49:421
3184219412,169sleep03275-21sed21:54:570
14997218822,148sleep115000-21cron22:34:271
1108218611,23sleep10-21swapper/121:49:451
23374218214,75sleep023435-21latency22:59:500
9748218116,105sleep10-21swapper/123:59:511
3812218071,19sleep10-21swapper/120:14:361
3812218071,19sleep10-21swapper/120:14:361
11439217117,91sleep10-21swapper/100:05:081
28985217010,121sleep129118-21timerwakeupswit19:49:571
15814216915,92sleep10-21swapper/100:19:381
9024216419,105sleep19077-21modprobe23:59:271
4405216314,133sleep34391-21apt-get21:59:453
27863216310,136sleep10-21swapper/119:47:101
17379216341,20sleep10-21swapper/120:56:361
18114216017,121sleep318247-21tail19:14:583
2187821588,135sleep121982-21latency_hist19:29:301
16888215816,53sleep017005-21fschecks_count19:14:310
887215610,104sleep1982-21fschecks_time20:04:471
5671215512,129sleep25703-21/usr/sbin/munin22:04:282
32238215411,128sleep132306-21awk19:59:581
28527215411,98sleep128530-21cron21:34:271
17417215442,97sleep10-21swapper/122:39:561
3111215210,27sleep10-21swapper/123:39:401
16168214921,45sleep116312-21tail19:10:001
29193214744,88sleep115282-21cyclictest21:34:511
12781214014,113sleep212847-21fschecks_time00:09:462
5164213917,18sleep00-21swapper/020:19:280
1997421397,94sleep10-21swapper/122:49:411
1216221396,40sleep10-21swapper/122:24:381
17640213572,50sleep117407-21/usr/sbin/munin19:14:481
3198213418,99sleep23250-21fschecks_count23:39:452
10324212728,78sleep310350-21awk22:19:273
14975212513,98sleep115126-21unixbench_singl00:14:571
20806212321,85sleep320780-21latency_hist21:09:303
24655212034,51sleep224651-21apt-get23:04:412
13611212022,82sleep10-21swapper/120:44:531
25143211717,78sleep10-21swapper/123:04:561
17350211715,86sleep00-21swapper/022:39:550
678621169,73sleep16848-21latency_hist20:24:291
30847211625,70sleep030994-21unixbench_singl19:55:050
7628211512,83sleep10-21swapper/120:24:581
21247211515,82sleep221392-21tail19:24:592
635211310,68sleep1764-21grep23:29:591
1673521139,65sleep00-21swapper/020:54:530
2569621128,19sleep10-21swapper/121:24:381
9978211131,12sleep10-21swapper/122:15:091
6696211113,82sleep36758-21sendmail_mailst23:49:563
4477211147,49sleep13868-21/usr/sbin/munin20:15:331
5977211043,49sleep10-21swapper/123:49:291
5977211043,49sleep10-21swapper/123:49:291
24443211014,80sleep224474-21/usr/sbin/munin21:19:502
30143210916,78sleep230168-21latency_hist21:39:292
1331921098,86sleep113443-21timerwakeupswit00:09:571
10675210817,71sleep210771-21dpkg00:04:282
30300210731,12sleep10-21swapper/123:20:341
20256210614,76sleep220323-21cat00:34:292
2392721058,19sleep1284-21dbus-daemon21:19:291
11165210519,67sleep20-21swapper/200:04:572
1851521047,22sleep30-21swapper/322:44:473
2133621038,52sleep10-21swapper/121:09:521
13248210312,77sleep013317-21switchtime00:09:560
2020621018,81sleep11-21systemd00:34:281
19662210116,47sleep10-21swapper/121:04:471
498221009,51sleep10-21swapper/122:00:151
12160210048,14sleep112220-21processes20:39:561
3162029920,58sleep031763-21sed23:24:580
271702999,81sleep00-21swapper/023:10:400
234372998,57sleep123078-21/usr/sbin/munin22:59:511
247082989,46sleep10-21swapper/121:19:571
1166829811,72sleep011687-21apt-get20:39:390
1137529819,60sleep111378-21cron20:39:271
75062977,18sleep17501-21apt-get22:09:401
308932978,56sleep130963-21df23:24:421
2869429711,72sleep228715-21netstat19:49:502
1064229721,59sleep20-21swapper/222:19:422
1632729623,39sleep10-21swapper/122:36:331
28592958,54sleep12875-21munin-run23:39:281
28592958,54sleep12875-21munin-run23:39:281
2474229516,65sleep324753-21awk23:04:463
2372929517,63sleep323738-21df_inode19:34:453
3073429424,54sleep330843-21timerandwakeup19:55:043
2211529438,42sleep10-21swapper/119:29:421
3082029313,22sleep230844-21ntp_states21:39:532
2395929134,14sleep10-21swapper/119:34:511
1781729133,15sleep10-21swapper/100:24:551
1482429118,58sleep014826-21sshd19:05:100
1924429013,61sleep019282-21sed21:04:300
1904029013,51sleep018922-21/usr/sbin/munin00:29:460
1332829015,60sleep013362-21cut20:44:470
797528916,15sleep30-21swapper/323:54:483
676328932,14sleep10-21swapper/122:05:001
2974628915,58sleep029755-21/usr/sbin/munin23:20:210
2259028916,38sleep30-21swapper/321:14:423
1869728919,53sleep218400-21/usr/sbin/munin22:44:502
396428811,63sleep33971-21/usr/sbin/munin20:14:463
2632528813,63sleep026410-21timerandwakeup21:24:570
1879628813,56sleep018834-21ntp_kernel_err22:44:520
210402876,43sleep10-21swapper/100:34:561
2958828629,39sleep329601-21apt19:54:273
1350128615,54sleep013624-21wc22:29:300
85712855,11sleep10-21swapper/120:29:421
2726428510,57sleep227328-21/usr/sbin/munin19:44:572
1404528514,57sleep114120-21idleruntime19:04:471
1145528522,32sleep30-21swapper/320:39:293
241762840,9sleep30-21swapper/323:01:583
630228317,50sleep26389-21fschecks_time23:49:472
447928215,16sleep20-21swapper/223:44:302
447928215,16sleep20-21swapper/223:44:302
1584328214,54sleep315862-21cpu00:19:443
540328119,30sleep00-21swapper/023:45:000
540328119,30sleep00-21swapper/023:45:000
324182816,62sleep00-21swapper/021:44:560
3217628017,45sleep031668-21/usr/sbin/munin19:59:570
189702806,40sleep10-21swapper/122:44:571
819427916,40sleep28351-21unixbench_multi22:10:012
279727918,40sleep02856-21sendmail-msp23:39:270
279727918,40sleep02856-21sendmail-msp23:39:270
226832795,41sleep10-21swapper/122:58:091
273012786,38sleep10-21swapper/123:12:191
2049727817,31sleep00-21swapper/000:34:430
918027717,38sleep30-21swapper/320:29:563
1556127712,51sleep315574-21uname22:34:543
3036527626,28sleep30-21swapper/321:39:383
2986827619,27sleep20-21swapper/223:20:272
2261827628,26sleep30-21swapper/319:29:563
2054927525,20sleep30-21swapper/322:49:563
1810727515,17sleep30-21swapper/320:59:493
663627417,25sleep00-21swapper/022:04:590
1715827420,26sleep00-21swapper/000:24:290
1036527422,34sleep00-21swapper/022:19:280
798027315,41sleep38040-21/usr/sbin/munin22:09:583
302127317,39sleep0363-21snmpd20:09:590
302127317,39sleep0363-21snmpd20:09:590
2182527321,34sleep321851-21munin-run19:29:283
1864227319,33sleep20-21swapper/221:00:002
493827218,23sleep30-21swapper/323:44:513
493827218,23sleep30-21swapper/323:44:513
441727212,17sleep00-21swapper/023:44:280
2547827222,36sleep025457-21latency_hist21:24:290
420827124,27sleep20-21swapper/220:15:262
267562717,11sleep00-21swapper/019:44:390
2342327117,25sleep00-21swapper/019:34:290
1679427123,29sleep316885-21nin-node19:14:303
1575227118,23sleep00-21swapper/019:09:500
1504927116,25sleep00-21swapper/022:34:300
1485727119,23sleep00-21swapper/000:14:560
39727012,42sleep3431-21/usr/sbin/munin23:29:533
347927017,25sleep30-21swapper/323:39:513
289327021,28sleep30-21swapper/323:39:303
289327021,28sleep30-21swapper/323:39:303
2110727012,41sleep221187-21timerandwakeup00:34:572
1553727025,21sleep20-21swapper/220:49:582
996226922,24sleep30-21swapper/323:59:573
2902526922,28sleep329052-21munin-run23:19:273
2902526922,28sleep329052-21munin-run23:19:273
2502526925,26sleep00-21swapper/023:04:510
2399926916,29sleep20-21swapper/223:00:002
2211626926,24sleep20-21swapper/219:29:432
177542690,7sleep30-21swapper/300:24:523
1376926925,25sleep20-21swapper/220:44:582
1102826919,34sleep311069-21/usr/sbin/munin22:19:523
588026811,14sleep00-21swapper/020:19:520
1585426817,26sleep20-21swapper/219:09:522
155762689,15sleep00-21swapper/022:34:550
1269526827,25sleep30-21swapper/322:24:563
776026717,25sleep20-21swapper/223:54:392
3244026714,28sleep00-21swapper/023:29:420
1196226719,24sleep30-21swapper/320:39:503
76626617,30sleep2889-21/usr/sbin/munin23:30:002
1902526614,30sleep00-21swapper/019:19:440
144326615,24sleep00-21swapper/020:04:580
87482657,11sleep00-21swapper/022:14:110
273292658,37sleep30-21swapper/319:44:583
1431426519,25sleep214351-21ntp_states19:04:522
1111326515,25sleep30-21swapper/300:04:383
1983526415,28sleep00-21swapper/021:04:500
103126412,39sleep0363-21snmpd21:49:400
3255026316,24sleep20-21swapper/221:44:582
1818626318,21sleep20-21swapper/222:44:302
1712526316,27sleep317244-21users20:55:003
127726311,25sleep20-21swapper/220:04:562
3150226215,21sleep30-21swapper/323:24:573
264392626,47sleep00-21swapper/019:43:460
2563426214,27sleep20-21swapper/219:39:532
2229926221,20sleep00-21swapper/021:14:270
1767126215,34sleep217753-21sed00:24:522
1449026214,23sleep30-21swapper/300:14:473
113372625,11sleep20-21swapper/220:39:002
29482619,11sleep20-21swapper/221:54:512
2681126114,31sleep226888-21/usr/sbin/munin23:09:582
1700226117,25sleep20-21swapper/220:54:592
897826018,17sleep20-21swapper/220:29:512
3202326016,28sleep332115-21tune2fs21:44:473
3117226012,22sleep00-21swapper/021:39:590
2947026021,22sleep30-21swapper/323:20:163
1910026015,22sleep20-21swapper/219:19:462
1569926012,34sleep215742-21ntpq22:34:572
754525918,18sleep00-21swapper/023:54:290
3230725913,23sleep20-21swapper/219:59:592
2896125916,17sleep00-21swapper/021:34:460
287625919,23sleep20-21swapper/223:39:292
287625919,23sleep20-21swapper/223:39:292
260225914,28sleep32614-21/usr/sbin/munin20:09:503
260225914,28sleep32614-21/usr/sbin/munin20:09:503
2393725920,19sleep20-21swapper/221:19:302
2267725915,24sleep00-21swapper/021:14:480
1930525918,25sleep319389-21/usr/sbin/munin19:19:513
1687225916,26sleep316866-21apt-get22:39:423
136325921,21sleep30-21swapper/320:04:573
5012589,34sleep31-21systemd20:04:293
2890125817,24sleep028984-21sed19:49:560
2857825817,21sleep30-21swapper/321:34:293
2571925816,26sleep325715-21apt-get21:24:403
2181125819,22sleep20-21swapper/222:54:512
2116625817,17sleep20-21swapper/221:09:492
2109425819,17sleep00-21swapper/019:24:570
1318025811,20sleep30-21swapper/300:09:553
1006225814,28sleep010213-21awk23:59:580
63962575,16sleep20-21swapper/220:20:062
383925720,22sleep30-21swapper/321:59:283
2812625714,27sleep028157-21/usr/sbin/munin23:14:520
2812625714,27sleep028157-21/usr/sbin/munin23:14:520
202482578,12sleep00-21swapper/022:49:490
1592725715,28sleep215932-21entropy00:19:472
1454925720,17sleep30-21swapper/319:04:573
857025614,24sleep08563-21apt-get20:29:410
2890125616,17sleep30-21swapper/321:34:453
2753125617,23sleep00-21swapper/021:29:500
2688925615,24sleep30-21swapper/323:09:593
2568025617,24sleep325687-21ntp_states19:39:543
1194025619,18sleep00-21swapper/022:24:290
727025516,17sleep30-21swapper/322:09:303
2491925512,23sleep224959-21munin-run19:39:282
2365925513,25sleep023654-21apt-get19:34:420
22542556,10sleep20-21swapper/221:53:582
2232725514,23sleep322425-21users22:55:013
148062559,35sleep3363-21snmpd22:31:593
2773425414,16sleep20-21swapper/221:29:562
1970425413,26sleep219747-21irqstats21:04:482
1921725411,18sleep30-21swapper/300:29:493
1324225413,26sleep313238-21apt-get20:44:423
932825314,19sleep20-21swapper/222:14:502
685025315,17sleep20-21swapper/220:24:302
2619425322,18sleep20-21swapper/221:24:552
251682535,10sleep30-21swapper/321:21:043
142882539,16sleep20-21swapper/200:14:422
646525218,17sleep20-21swapper/222:04:572
300852527,21sleep20-21swapper/219:54:382
1997025213,23sleep219967-21apt-get22:49:382
755725110,16sleep00-21swapper/020:24:570
546925114,21sleep35472-21apt-get20:19:393
17432516,11sleep30-21swapper/321:50:003
148202519,11sleep20-21swapper/222:32:102
784625010,16sleep00-21swapper/023:54:450
702025014,20sleep37017-21apt-get20:24:393
2775525010,24sleep227749-21apt-get23:14:422
2775525010,24sleep227749-21apt-get23:14:422
2152925012,23sleep321611-21mailstats21:09:583
129972509,24sleep20-21swapper/220:44:292
204942497,19sleep30-21swapper/300:34:423
1534624912,18sleep00-21swapper/020:49:560
1011724914,18sleep00-21swapper/020:34:420
3215224812,19sleep20-21swapper/223:29:282
1878424815,17sleep20-21swapper/219:19:292
123202439,14sleep20-21swapper/222:24:472
233924210,18sleep20-21swapper/220:09:402
233924210,18sleep20-21swapper/220:09:402
1739424012,15sleep00-21swapper/000:24:420
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional