You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-26 - 17:18

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Tue May 26, 2026 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
27775246057,385sleep227924-21grep07:49:142
11608245449,377sleep211738-21grep08:39:412
939022766,258sleep29508-21sed10:14:132
3068226716,236sleep13132-21ps08:14:121
2153226620,184sleep22239-21fschecks_time09:54:022
11011225825,211sleep111050-21cut08:39:281
18318225449,188sleep218404-21ls07:19:142
31425225255,140sleep231452-21awk08:03:432
31425225255,140sleep231452-21awk08:03:432
31474224548,95sleep231488-21fschecks_count09:44:012
17051223755,138sleep217055-21cron08:58:432
10582222651,116sleep20-21swapper/212:03:442
19227222442,161sleep20-21swapper/212:29:072
19379221411,186sleep3363-21snmpd09:04:093
7925220811,149sleep28073-21grep10:09:492
1147220740,69sleep21235-21fschecks_time08:09:052
3704220644,119sleep20-21swapper/209:59:032
6670220326,120sleep26731-21systemd-run10:08:442
28475220355,24sleep20-21swapper/209:34:042
25255220048,104sleep20-21swapper/209:24:022
17573219271,62sleep217637-21latency12:24:032
10363218912,161sleep210430-21apt-config08:38:442
10363218912,161sleep210430-21apt-config08:38:442
5501217716,106sleep25478-21latency_hist08:23:462
122221769,121sleep20-21swapper/211:33:452
6995217535,126sleep26998-21cron08:28:432
27396217416,113sleep227406-21modprobe11:13:432
16205216414,134sleep216236-21fschecks_count07:14:022
14079216418,129sleep114086-21sshd08:49:011
8166216310,136sleep17663-21/usr/sbin/munin11:54:141
8166216310,136sleep17663-21/usr/sbin/munin11:54:141
21490215730,74sleep221618-21unixbench_multi07:29:162
10552215628,107sleep310036-21apt-get10:18:543
22196214915,25sleep20-21swapper/210:54:132
12712214743,92sleep212777-21grep08:44:052
3003214625,78sleep23051-21ntp_offset08:14:092
11510214521,102sleep111649-21grep12:04:161
19504214349,52sleep20-21swapper/209:04:122
13384214220,101sleep313408-21idleruntime-cro10:28:433
1091721427,127sleep05587-1kworker/0:0H08:39:190
25970213713,112sleep226047-21mailstats07:44:142
1524421377,120sleep2363-21snmpd07:11:192
2796421357,117sleep2363-21snmpd09:33:352
24853213513,24sleep20-21swapper/211:04:062
18210213316,103sleep118218-21vmstat12:24:141
23704213252,22sleep20-21swapper/209:19:022
30905212815,87sleep330939-21fschecks_count11:24:013
17014212836,78sleep217032-21modprobe12:23:422
29352212711,52sleep20-21swapper/207:54:152
1288521248,24sleep212958-21awk12:09:132
2304521238,103sleep2363-21snmpd09:15:352
2304521238,103sleep2363-21snmpd09:15:352
1990521208,96sleep20-21swapper/210:48:562
1692121209,100sleep217004-21fschecks_time10:39:032
11219212019,17sleep00-21swapper/010:19:280
32123211944,58sleep20-21swapper/211:28:442
26741211623,69sleep326773-21idleruntime-cro07:48:433
21405211617,73sleep321555-21grep12:34:253
9509211516,82sleep39609-21unixbench_singl10:14:143
20658211516,86sleep020632-21apt-get12:34:060
19562211543,15sleep20-21swapper/207:24:072
23120211224,62sleep323137-21fschecks_count10:59:033
22755211255,19sleep222791-21ntpq07:34:082
2407021105,12sleep20-21swapper/211:00:472
1855421105,48sleep20-21swapper/209:02:432
15367211044,28sleep215570-21apt-config12:18:462
266321096,94sleep00-21swapper/011:37:590
20808210936,46sleep320805-21apt-get07:28:563
31918210856,16sleep231439-21apt-get08:03:522
31918210856,16sleep231439-21apt-get08:03:522
3115021088,63sleep231193-21date09:43:442
1330121077,66sleep2363-21snmpd10:27:412
170021065,47sleep20-21swapper/209:53:092
170021065,47sleep20-21swapper/209:53:092
146221069,57sleep20-21swapper/211:33:572
1176521055,66sleep20-21swapper/212:04:372
7690210436,51sleep20-21swapper/211:53:592
7690210436,51sleep20-21swapper/211:53:592
1523321047,41sleep20-21swapper/210:33:592
13051210421,67sleep113083-21proc_pri07:04:121
14051210331,21sleep214065-21netstat10:29:062
22008210223,48sleep022027-21apt-get09:13:550
22008210223,48sleep022027-21apt-get09:13:550
716929914,67sleep06692-21apt-get10:08:540
331529815,60sleep20-21swapper/211:39:062
2673429712,22sleep326791-21df09:29:003
1868729411,67sleep018807-21latency_hist09:03:460
155752949,66sleep315220-21apt-get12:18:473
1543229428,39sleep015452-21munin-run07:13:430
1326929436,37sleep013434-21grep07:04:150
48062928,11sleep00-21swapper/011:44:040
328129219,54sleep02573-21/usr/sbin/munin08:14:150
2247029216,55sleep122551-21fschecks_time07:34:031
152829219,56sleep31589-21/usr/sbin/munin08:09:143
1739129118,59sleep317431-21latency_hist07:18:433
3163029031,30sleep10-21swapper/108:03:471
2599529011,66sleep326046-21cat11:08:473
3136628916,59sleep031368-21sshd08:03:010
3136628916,59sleep031368-21sshd08:03:010
2866028912,60sleep028665-21diskstats07:54:000
2212028913,58sleep022263-21latency_hist07:33:460
2011428919,54sleep1225-21systemd-journal12:33:421
1033428917,47sleep110354-21apt-get08:38:431
1033428917,47sleep110354-21apt-get08:38:431
1912828817,21sleep00-21swapper/012:29:050
429828719,52sleep34319-21munin-run11:43:433
37428711,63sleep3390-21open_inodes11:29:103
1907528720,41sleep019223-21unixbench_singl10:44:180
1560328716,55sleep315626-21netstat10:34:083
747628635,39sleep213944-21cyclictest08:29:032
1890828616,17sleep20-21swapper/210:44:162
182842855,13sleep10-21swapper/108:59:191
154682859,23sleep215415-21munin-run08:53:462
2829128415,48sleep328306-21ls07:53:433
2642928417,50sleep326479-21unin-run09:28:443
2127728427,37sleep20-21swapper/212:34:242
1245928412,56sleep012499-21fschecks_count12:09:030
1205528411,61sleep00-21swapper/012:08:220
105752847,66sleep2363-21snmpd10:18:572
1043328425,30sleep30-21swapper/308:38:453
1043328425,30sleep30-21swapper/308:38:453
2279328325,32sleep30-21swapper/310:58:463
1523128320,37sleep015227-21apt-get10:33:570
279492825,67sleep00-21swapper/009:33:250
209822829,18sleep20-21swapper/209:09:132
93428112,56sleep1946-21ntpq09:49:081
93428112,56sleep1946-21ntpq09:49:081
2037028116,29sleep00-21swapper/010:49:110
1556028120,40sleep315721-21cut07:13:463
86812807,64sleep00-21swapper/011:55:140
86812807,64sleep00-21swapper/011:55:140
35482809,52sleep10-21swapper/111:39:131
1498528029,30sleep00-21swapper/010:33:450
1290528013,50sleep012928-21/usr/sbin/munin08:44:120
1284028012,54sleep112882-21ps12:09:121
1133528020,39sleep311425-21grep12:04:143
2139927910,12sleep30-21swapper/309:09:183
1056127922,34sleep00-21swapper/008:38:460
1056127922,34sleep00-21swapper/008:38:460
2030427820,42sleep120341-21sed10:49:081
1088427823,33sleep310879-21apt-get08:38:573
692627713,55sleep00-21swapper/008:27:520
2408327620,29sleep30-21swapper/309:19:113
2018327618,36sleep120206-21latency_hist09:08:431
1589727618,41sleep315904-21/usr/sbin/munin08:54:243
1187527618,29sleep10-21swapper/110:23:451
66562757,56sleep0225-21systemd-journal10:08:330
58302759,44sleep10-21swapper/110:04:271
582127531,12sleep20-21swapper/211:48:182
3175827517,24sleep30-21swapper/309:44:063
3162527511,47sleep031723-21users11:24:140
2741027517,39sleep30-21swapper/311:13:443
2612527517,28sleep00-21swapper/007:44:160
730727431,11sleep20-21swapper/211:52:302
730727431,11sleep20-21swapper/211:52:302
2922527417,39sleep129351-21/usr/sbin/munin07:54:141
2769027410,44sleep20-21swapper/211:13:592
1840627424,30sleep318559-21grep07:19:153
1406627430,26sleep30-21swapper/310:29:073
720227321,34sleep37188-21apt-get10:09:013
673327320,26sleep30-21swapper/310:08:453
2927027321,13sleep20-21swapper/211:18:552
2744827332,23sleep00-21swapper/011:13:450
2139627310,28sleep10-21swapper/107:29:151
1786927317,31sleep017933-21idleruntime07:19:040
168932738,45sleep00-21swapper/010:39:010
1638927320,34sleep116405-21/usr/sbin/munin07:14:051
1391627316,40sleep314071-21latency_hist07:08:463
778527219,28sleep30-21swapper/311:54:043
778527219,28sleep30-21swapper/311:54:043
2494227229,11sleep20-21swapper/207:40:302
2401827226,22sleep00-21swapper/009:19:080
124372729,45sleep10-21swapper/108:43:561
1239327221,31sleep012412-21kernelversion10:24:040
1554827120,22sleep30-21swapper/308:53:473
2710727015,17sleep00-21swapper/007:49:000
2537927013,41sleep025435-21interrupts09:24:040
2085527022,30sleep120882-21awk09:09:081
1571727011,13sleep00-21swapper/012:18:480
3057926917,27sleep30-21swapper/311:23:453
2111926919,22sleep00-21swapper/009:09:150
1010526918,25sleep30-21swapper/311:59:173
2264626815,20sleep30-21swapper/309:14:143
2264626815,20sleep30-21swapper/309:14:143
1815126821,28sleep00-21swapper/012:24:130
1799726818,30sleep30-21swapper/312:24:123
1713126822,24sleep00-21swapper/008:58:450
1406026815,38sleep014141-21/usr/sbin/munin12:14:030
1296126820,23sleep30-21swapper/312:09:143
3207926710,45sleep132122-21systemd11:28:431
1503026714,28sleep10-21swapper/107:09:161
1309126726,23sleep20-21swapper/207:04:132
856826614,27sleep10-21swapper/110:13:471
337126616,29sleep00-21swapper/009:58:440
1874226622,27sleep10-21swapper/110:44:111
1527926614,32sleep115365-21apt-config12:18:451
6542657,17sleep00-21swapper/008:07:250
6542657,17sleep00-21swapper/008:07:250
501226529,22sleep04924-21gpgv10:03:450
2842826515,34sleep328574-21unixbench_singl11:14:163
2271226518,29sleep022774-21sendmail-msp10:58:430
1499526521,22sleep10-21swapper/110:33:461
146526520,19sleep30-21swapper/311:33:583
76026418,28sleep1768-21kworker/u8:208:08:431
76026418,28sleep1768-21kworker/u8:208:08:431
61452649,31sleep30-21swapper/311:49:003
61452649,31sleep30-21swapper/311:49:003
588726428,19sleep30-21swapper/308:24:033
427826423,25sleep10-21swapper/108:19:011
2872926414,21sleep10-21swapper/109:34:111
2462626425,18sleep00-21swapper/007:39:150
1986626413,35sleep119959-21timerandwakeup07:24:141
1966826413,37sleep319771-21cut10:48:453
341926318,23sleep10-21swapper/109:58:451
2120826316,23sleep10-21swapper/112:34:231
935626212,35sleep1363-21snmpd11:59:031
2817526210,12sleep00-21swapper/011:14:130
276552625,45sleep10-21swapper/111:13:501
1220926221,21sleep10-21swapper/110:24:011
79626112,35sleep0820-21users11:29:150
745126118,21sleep00-21swapper/011:53:450
745126118,21sleep00-21swapper/011:53:450
726026114,26sleep17279-21apt-get08:28:541
3035126116,16sleep30-21swapper/307:59:043
3035126116,16sleep30-21swapper/307:59:043
2882026118,20sleep30-21swapper/309:34:133
171392616,43sleep00-21swapper/007:15:370
2509526011,17sleep10-21swapper/111:04:131
2352226012,19sleep10-21swapper/110:59:131
2124626013,24sleep30-21swapper/310:53:463
1619226015,30sleep115756-21/usr/sbin/munin08:54:341
792925920,19sleep00-21swapper/008:29:130
3229325916,24sleep332387-21memory08:04:583
3229325916,24sleep332387-21memory08:04:583
2854025911,33sleep028634-21memory09:34:060
2369825912,25sleep00-21swapper/007:38:470
1778425919,24sleep117791-21df_inode07:19:011
1307725919,22sleep30-21swapper/308:44:153
859925819,24sleep38560-21apt-get08:33:443
83822586,16sleep10-21swapper/110:12:461
492225812,26sleep30-21swapper/310:03:443
2957525813,24sleep00-21swapper/011:19:060
2221825812,31sleep122009-21/usr/sbin/munin09:14:041
2221825812,31sleep122009-21/usr/sbin/munin09:14:041
202025811,34sleep12021-21sshd08:10:001
3265225713,29sleep032669-21munin-run09:48:430
2614025715,26sleep126134-21apt-get11:08:571
2454125720,15sleep30-21swapper/307:39:143
1736625715,26sleep317363-21apt-get08:58:563
467525613,25sleep14681-21diskstats11:44:011
381925616,18sleep30-21swapper/308:18:103
242102567,37sleep00-21swapper/011:02:350
16132566,37sleep00-21swapper/009:52:020
16132566,37sleep00-21swapper/009:52:020
85525515,24sleep30-21swapper/309:49:063
85525515,24sleep30-21swapper/309:49:063
281812555,16sleep10-21swapper/107:52:201
2420425515,19sleep10-21swapper/109:19:131
186425518,23sleep11903-21ntpq11:34:081
1401725522,18sleep30-21swapper/312:14:023
923425414,19sleep00-21swapper/010:14:110
515925412,24sleep00-21swapper/008:20:010
509925412,27sleep15320-21cat10:03:461
2558325412,21sleep10-21swapper/107:44:041
2519525415,24sleep125225-21munin-run07:43:441
1708725413,23sleep117181-21memory10:39:071
202672537,16sleep00-21swapper/007:25:300
1576025316,22sleep115746-21apt-get12:19:011
573725212,23sleep15732-21apt-get08:23:571
2671825212,24sleep126712-21apt-get09:28:581
241325210,20sleep30-21swapper/309:54:073
2011025213,21sleep320162-21users07:24:163
177512525,11sleep30-21swapper/310:39:373
926525114,20sleep09259-21apt-get11:58:580
38482519,17sleep00-21swapper/011:39:160
3049025114,15sleep00-21swapper/009:39:140
3013125114,20sleep10-21swapper/107:58:541
2926425113,23sleep129267-21apt-get11:18:541
1936725113,18sleep30-21swapper/312:29:123
1579625116,22sleep115873-21switchtime10:34:141
1421125118,18sleep00-21swapper/008:49:040
641625012,25sleep16463-21latency11:49:061
641625012,25sleep16463-21latency11:49:061
189262507,16sleep00-21swapper/009:03:580
120725012,22sleep11221-21munin-run11:33:441
113325013,22sleep01140-21diskstats08:09:030
3119424916,16sleep30-21swapper/309:43:453
3110224814,18sleep10-21swapper/109:43:431
540624714,16sleep30-21swapper/311:44:153
418124711,20sleep30-21swapper/308:18:553
313882479,21sleep131383-21apt-get09:43:561
2906724712,15sleep30-21swapper/307:54:123
2543624716,14sleep10-21swapper/109:24:051
2539624410,16sleep00-21swapper/011:04:160
2153724414,15sleep10-21swapper/110:54:001
223552396,16sleep30-21swapper/307:33:573
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional