You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-16 - 09:47

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Thu Jul 16, 2026 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
307412209118,2053sleep030734-21apt-get23:18:540
12553214359,1408sleep210410-21kworker/u8:200:04:132
2160276418,728sleep32275-21/usr/sbin/munin20:04:113
23979268414,653sleep123980-21sshd22:55:581
1767425059,481sleep117682-21ntpq00:19:061
1767425059,481sleep117682-21ntpq00:19:061
12143240415,323sleep212341-21rm00:03:402
1588239012,223sleep21602-21fschecks_count20:03:582
9750233514,303sleep09809-21/usr/sbin/munin20:29:070
2590222990,7sleep30-21swapper/321:19:153
1974622960,15sleep215470-21cyclictest19:19:082
17929225650,186sleep217986-21latency19:14:002
17929225650,186sleep217986-21latency19:14:002
936224174,88sleep2999-21latency21:44:012
23619223515,202sleep123687-21fschecks_time21:13:571
23619223515,202sleep123687-21fschecks_time21:13:571
31014223045,71sleep20-21swapper/219:54:012
4597222817,127sleep24689-21fschecks_time23:38:582
6390222474,77sleep20-21swapper/220:19:002
4185221825,37sleep20-21swapper/223:38:382
4185221825,37sleep20-21swapper/223:38:382
2616022138,125sleep226263-21grep23:03:582
31835221215,178sleep331848-21uname21:39:023
24797220955,92sleep224830-21sendmail21:18:382
24797220955,92sleep224830-21sendmail21:18:382
2408222039,87sleep224262-21gpgv19:33:422
25770219823,104sleep225784-21idleruntime-cro23:03:382
20835219817,165sleep020880-21/usr/sbin/munin21:04:070
21376219523,146sleep20-21swapper/219:24:582
9822219318,35sleep29967-21unixbench_multi23:54:082
9822219318,35sleep29967-21unixbench_multi23:54:082
21621218711,161sleep221725-21latency_hist00:33:402
32074218616,153sleep232150-21sendmail_mailtr21:39:092
20519218617,150sleep220555-21fschecks_count21:04:012
6011218516,16sleep20-21swapper/221:59:112
27949218510,92sleep227993-21unin-run21:28:392
27678218015,150sleep227717-21idleruntime19:44:002
27678218015,150sleep227717-21idleruntime19:44:002
11428218012,80sleep20-21swapper/220:34:092
9742217810,153sleep29781-21cat22:13:402
19330217319,128sleep20-21swapper/220:59:042
16898217312,148sleep216957-21awk00:18:382
15715216316,132sleep015727-21fschecks_count00:13:560
15850216215,89sleep215881-21tune2fs20:49:112
1848421599,137sleep218544-21cat22:39:122
3001021567,92sleep2364-21snmpd19:49:532
2241621569,85sleep222488-21cat00:34:092
22345215622,113sleep10-21swapper/119:28:411
2617215313,85sleep22664-21grep21:49:062
15685215213,58sleep215827-21unixbench_singl22:29:312
15207214731,98sleep215233-21munin-run20:48:392
19329214215,110sleep319356-21fschecks_count19:18:583
2555421368,115sleep118934-21kworker/u8:119:37:411
13045213615,93sleep113104-21sendmail_mailqu00:04:321
506021299,59sleep20-21swapper/220:14:072
23329212615,22sleep20-21swapper/221:13:412
628821258,17sleep20-21swapper/223:43:592
24361212313,95sleep323854-21apt-get19:33:513
18271212316,62sleep117763-21apt-get22:38:501
3136211822,77sleep03188-21fschecks_count20:08:560
21380211652,48sleep20-21swapper/222:48:522
11569211615,85sleep211543-21/usr/sbin/munin22:18:532
30054211429,72sleep230067-21awk23:14:112
14801211438,47sleep114943-21grep19:04:111
324211211,80sleep10-21swapper/123:24:071
21024211116,82sleep010737-1kworker/0:1H19:23:510
14586210911,79sleep214747-21gpgv22:28:422
11332210932,64sleep211423-21cat22:18:412
1181621085,94sleep00-21swapper/000:01:200
27999210714,79sleep328127-21sed21:28:403
17746210772,22sleep217823-21apt-config22:38:392
2380121068,63sleep00-21swapper/021:14:000
2380121068,63sleep00-21swapper/021:14:000
13169210611,80sleep313194-21/usr/sbin/munin22:23:543
20072210510,32sleep220215-21date00:28:402
20072210510,32sleep220215-21date00:28:402
12161210512,78sleep112100-21latency_hist20:38:401
8895210412,46sleep20-21swapper/223:53:392
28947210312,75sleep129096-21cat23:13:421
2485721039,62sleep00-21swapper/019:34:330
1892821039,23sleep00-21swapper/022:39:220
7785210213,72sleep17798-21/usr/sbin/munin22:04:111
18133210226,56sleep20-21swapper/220:54:122
7693210114,73sleep17790-21fschecks_time23:48:581
24586210011,70sleep024628-21fschecks_count22:58:550
18407210015,10sleep20-21swapper/200:22:462
18407210015,10sleep20-21swapper/200:22:462
469329915,65sleep34751-21/usr/sbin/munin23:38:593
1369929919,63sleep013681-21latency_hist20:43:400
560529818,65sleep15670-21latency21:59:021
306829815,64sleep03141-21/usr/sbin/munin23:33:590
306829815,64sleep03141-21/usr/sbin/munin23:33:590
140732979,68sleep20-21swapper/200:08:532
2476529616,62sleep20-21swapper/219:34:272
242729627,12sleep20-21swapper/223:30:552
242729627,12sleep20-21swapper/223:30:552
211392968,28sleep221174-21sed22:48:402
1184229617,60sleep111543-21/usr/sbin/munin22:19:001
355629524,59sleep20-21swapper/220:09:072
26492955,77sleep00-21swapper/020:07:110
187212958,17sleep30-21swapper/322:39:173
2320829411,16sleep20-21swapper/222:54:012
50182938,71sleep25080-21sendmail-msp21:58:382
197042936,12sleep30-21swapper/320:59:093
1172629214,66sleep3364-21snmpd00:00:103
825629111,24sleep28330-21mailstats20:24:082
2844929110,15sleep20-21swapper/221:28:512
2194329123,51sleep321957-21/usr/sbin/munin00:33:583
1090829113,64sleep310986-21cut20:33:583
325552898,16sleep20-21swapper/223:24:002
779928818,13sleep20-21swapper/222:04:122
3073828818,53sleep20-21swapper/223:18:522
261602887,13sleep20-21swapper/219:38:592
261602887,13sleep20-21swapper/219:38:592
1313528811,17sleep20-21swapper/220:39:102
3096328723,13sleep20-21swapper/221:34:532
972928623,14sleep20-21swapper/220:29:052
2594728614,53sleep025948-21/usr/sbin/munin19:38:530
2594728614,53sleep025948-21/usr/sbin/munin19:38:530
324032858,16sleep20-21swapper/219:58:532
2637328511,16sleep20-21swapper/221:23:382
2146628516,51sleep321559-21grep22:48:573
157072858,14sleep20-21swapper/219:08:512
1050028518,35sleep30-21swapper/323:58:413
1050028518,35sleep30-21swapper/323:58:413
2397628417,52sleep124081-21latency_hist19:33:411
1863228412,57sleep318751-21latency_hist20:58:413
120128412,56sleep11272-21sensors21:44:081
2933528313,54sleep029413-21/usr/sbin/munin19:49:020
192628312,16sleep10-21swapper/120:04:051
279992828,20sleep228035-21ntpq23:09:032
2769128217,46sleep027707-21fschecks_count23:08:570
248528016,43sleep32500-21uname21:49:033
1761728015,32sleep30-21swapper/300:19:043
1761728015,32sleep30-21swapper/300:19:043
41727917,40sleep1449-21latency_hist21:43:401
2306927916,47sleep023152-21mailstats19:29:080
124832798,13sleep00-21swapper/000:03:430
40627813,30sleep00-21swapper/019:59:080
3144327818,43sleep031550-21sort19:54:110
2976227819,37sleep329888-21wc21:33:413
2067727815,48sleep320708-21/usr/sbin/munin00:29:013
2020327812,31sleep00-21swapper/022:44:020
1604127812,15sleep10-21swapper/120:49:181
1541627822,30sleep10-21swapper/100:13:411
172962778,54sleep10-21swapper/120:53:541
1703327725,33sleep017122-21grep19:13:420
68902769,14sleep00-21swapper/022:03:530
3090927610,32sleep10-21swapper/123:18:591
2645427617,40sleep326033-21/usr/sbin/munin23:04:033
2582227624,21sleep10-21swapper/123:03:411
1570727613,44sleep30-21swapper/300:13:553
3049727519,33sleep329977-21/usr/sbin/munin21:34:093
2949127514,31sleep30-21swapper/319:49:053
2103827518,39sleep321020-21apt-get19:23:583
1673627511,44sleep016767-21/usr/sbin/munin22:34:010
2319427416,30sleep10-21swapper/100:38:401
2898527316,40sleep129057-21grep21:29:111
978227223,17sleep30-21swapper/322:13:413
3008727211,40sleep130128-21fschecks_count21:33:591
3157427120,32sleep031568-21apt-get21:38:550
1545627113,41sleep015583-21cut19:08:410
1320327117,38sleep113213-21/usr/sbin/munin22:23:571
66802700,6sleep00-21swapper/022:03:400
651027021,23sleep00-21swapper/023:44:050
6462700,7sleep30-21swapper/321:43:493
2674327023,32sleep00-21swapper/021:23:570
2611527017,39sleep126159-21sort19:38:581
2611527017,39sleep126159-21sort19:38:581
2269027025,19sleep30-21swapper/319:28:593
2050427013,43sleep020560-21apt-config19:23:390
1900827017,25sleep30-21swapper/319:18:413
1900827017,25sleep30-21swapper/319:18:413
1772727019,35sleep117745-21cron22:38:381
1537727015,25sleep00-21swapper/000:13:400
904726822,25sleep00-21swapper/022:09:100
3042326820,34sleep130460-21systemctl23:18:381
2360326813,39sleep323612-21diskstats21:13:553
2360326813,39sleep323612-21diskstats21:13:553
1781726814,38sleep017894-21/usr/sbin/munin00:19:090
1781726814,38sleep017894-21/usr/sbin/munin00:19:090
1701026821,29sleep317031-21nin-node19:13:413
1542026815,31sleep115561-21gpgv20:48:421
639326719,32sleep36394-21sshd22:00:583
2663326715,26sleep00-21swapper/023:04:080
1563026712,14sleep10-21swapper/100:13:531
1462826718,30sleep30-21swapper/320:44:123
1125326718,24sleep00-21swapper/022:18:390
941126614,27sleep10-21swapper/123:53:591
941126614,27sleep10-21swapper/123:53:591
1538326619,18sleep20-21swapper/219:08:392
1503026518,29sleep115117-21/usr/sbin/munin22:29:171
1441826514,36sleep014493-21cmp22:28:400
1204826517,24sleep10-21swapper/100:03:391
743926419,30sleep17452-21idleruntime-cro20:23:391
1231326424,18sleep30-21swapper/322:19:103
3205226318,21sleep30-21swapper/323:23:413
284452635,12sleep10-21swapper/119:44:391
284452635,12sleep10-21swapper/119:44:391
2727426318,20sleep10-21swapper/121:24:101
1940326316,22sleep00-21swapper/020:59:060
1579626318,26sleep10-21swapper/119:08:591
1551626313,32sleep30-21swapper/322:29:283
1006726316,31sleep30-21swapper/322:13:573
662526214,30sleep30-21swapper/320:19:053
314226213,22sleep10-21swapper/123:34:001
314226213,22sleep10-21swapper/123:34:001
3258926112,20sleep10-21swapper/119:59:001
3226026116,30sleep332271-21sort23:23:493
2790226116,30sleep327950-21ntp_states19:44:053
2790226116,30sleep327950-21ntp_states19:44:053
197832615,13sleep10-21swapper/120:59:111
623026015,29sleep16024-21/usr/sbin/munin23:43:581
2244826011,33sleep322453-21ntp_states21:09:063
1249326018,27sleep312009-21apt-get00:03:503
967525913,29sleep09744-21/usr/sbin/munin23:54:060
967525913,29sleep09744-21/usr/sbin/munin23:54:060
821025915,25sleep08314-21grep23:49:090
1622925916,18sleep00-21swapper/022:33:400
75642585,10sleep30-21swapper/323:48:443
499425816,22sleep30-21swapper/320:14:043
457125817,17sleep10-21swapper/120:13:501
2249525811,24sleep10-21swapper/121:09:081
1819325815,20sleep10-21swapper/119:14:071
1819325815,20sleep10-21swapper/119:14:071
176612587,17sleep00-21swapper/019:13:540
176612587,17sleep00-21swapper/019:13:540
1277925819,24sleep312817-21ntp_kernel_pll_20:39:033
502925712,18sleep00-21swapper/020:14:050
46912576,39sleep00-21swapper/021:54:250
41652575,10sleep30-21swapper/323:38:223
41652575,10sleep30-21swapper/323:38:223
384125711,18sleep10-21swapper/120:09:101
1461925724,17sleep30-21swapper/319:04:093
137925718,20sleep00-21swapper/023:28:510
137925718,20sleep00-21swapper/023:28:510
927225617,24sleep19267-21apt-get20:28:521
903625615,21sleep30-21swapper/320:28:413
6252566,39sleep00-21swapper/021:43:430
32542567,37sleep00-21swapper/021:50:490
3248725620,18sleep30-21swapper/319:58:583
3155125613,22sleep10-21swapper/119:54:121
2862725620,19sleep00-21swapper/021:29:010
2642925616,15sleep00-21swapper/021:23:410
223282566,39sleep00-21swapper/022:49:390
2056125613,26sleep120562-21munin-run19:23:401
1922025616,18sleep00-21swapper/000:24:060
1922025616,18sleep00-21swapper/000:24:060
1394725614,24sleep013943-21apt-get20:43:520
2183625513,23sleep021854-21apt-get00:33:510
2081625512,29sleep020892-21systemctl00:29:060
206325512,21sleep30-21swapper/323:29:103
206325512,21sleep30-21swapper/323:29:103
1407425513,26sleep314068-21apt-get00:08:543
752425419,19sleep30-21swapper/322:04:093
615425414,20sleep00-21swapper/020:18:490
2904625414,18sleep10-21swapper/119:48:551
238612546,39sleep00-21swapper/022:54:270
2213625418,19sleep00-21swapper/021:08:590
1956825413,25sleep119666-21users00:24:111
1956825413,25sleep119666-21users00:24:111
1705825413,22sleep00-21swapper/020:53:410
969725314,23sleep19741-21systemd-run22:13:391
806925311,18sleep00-21swapper/020:24:020
450225313,25sleep04497-21apt-get23:38:520
356525314,18sleep10-21swapper/121:53:411
3105825313,22sleep331099-21irqstats19:54:023
3057825313,26sleep030618-21/usr/sbin/munin21:34:100
2603225312,19sleep10-21swapper/123:03:491
2125425314,24sleep321332-21users21:04:143
1876225313,23sleep318757-21apt-get00:23:523
1876225313,23sleep318757-21apt-get00:23:523
138025313,24sleep11375-21apt-get23:28:521
138025313,24sleep11375-21apt-get23:28:521
1239825311,24sleep112392-21apt-get20:38:531
41432525,35sleep30-21swapper/320:11:303
403225215,16sleep10-21swapper/121:54:011
2749625211,24sleep027490-21apt-get19:43:540
2749625211,24sleep027490-21apt-get19:43:540
2044725214,25sleep320558-21sed22:44:093
176042524,11sleep30-21swapper/322:37:023
495825113,15sleep10-21swapper/123:39:051
1738025114,17sleep00-21swapper/020:53:570
99892509,15sleep10-21swapper/122:13:551
3228825011,22sleep032355-21df23:23:540
2556425013,22sleep125575-21ntp_states21:19:091
2070225014,19sleep10-21swapper/121:04:041
2030725016,16sleep10-21swapper/100:28:511
1182825011,22sleep011841-21irqstats22:18:590
1990524911,16sleep10-21swapper/122:43:551
390924811,22sleep33973-21idleruntime21:53:593
2457924811,22sleep324584-21diskstats22:58:543
280372475,16sleep30-21swapper/323:09:043
77972466,18sleep10-21swapper/120:23:551
2958524614,16sleep30-21swapper/323:14:043
1582224612,20sleep315829-21df_inode20:49:003
216132456,14sleep10-21swapper/122:48:591
1211224312,15sleep00-21swapper/020:38:390
272472429,16sleep30-21swapper/319:43:403
272472429,16sleep30-21swapper/319:43:403
228012428,14sleep30-21swapper/322:53:423
92482417,15sleep30-21swapper/320:28:493
60472416,17sleep30-21swapper/323:43:513
291292419,17sleep00-21swapper/023:13:490
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional