You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-27 - 21:00

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7s.osadl.org (updated Wed May 27, 2026 12:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3512461112,4583sleep1313-21apt-get10:49:091
27654240215,3938sleep316219-1kworker/3:0H08:49:153
1423327006,682sleep314277-21ntpq09:49:033
13379244940,394sleep313470-21xargs11:29:093
11313243320,396sleep111377-21/usr/sbin/munin09:39:081
31702240917,313sleep331717-21latency_hist12:28:413
2819123077,243sleep328227-21ntpq07:09:043
2819123077,243sleep328227-21ntpq07:09:043
319522287101,102sleep30-21swapper/312:28:533
21244224268,91sleep321310-21fschecks_time08:28:593
7914223257,15sleep30-21swapper/311:13:423
24484222722,189sleep324507-21idleruntime-cro10:23:393
4156222114,130sleep34181-21awk07:34:043
12680221729,176sleep312698-21netstat09:44:023
10516221414,84sleep30-21swapper/309:38:413
25802220731,117sleep325816-21acpi08:43:503
2323220420,106sleep32407-21fschecks_time07:28:593
474822020,12sleep20-21swapper/211:03:392
29267220015,164sleep329281-21fschecks_count08:54:033
22708219916,171sleep322744-21grep08:33:563
9380219712,126sleep39426-21modprobe11:18:393
1662821879,156sleep30-21swapper/308:14:053
316218313,153sleep232294-21apt-get10:48:502
22654217710,61sleep30-21swapper/311:59:013
1982521778,157sleep319897-21wc11:49:083
15992217072,16sleep30-21swapper/308:13:413
2616216812,32sleep30-21swapper/309:13:013
434921618,77sleep30-21swapper/309:18:433
434921618,77sleep30-21swapper/309:18:433
30384216017,123sleep00-21swapper/007:14:060
19006215712,129sleep019016-21/usr/sbin/munin11:48:410
30015215356,81sleep330061-21grep07:13:563
12395215324,103sleep212410-21fschecks_count09:43:572
27729215143,27sleep30-21swapper/312:15:133
22214215118,115sleep122301-21switchtime10:14:091
27933214714,114sleep227974-21latency08:49:302
15171214714,119sleep115249-21latency_hist09:53:411
29544214314,113sleep329633-21fschecks_time10:38:583
16152214312,108sleep30-21swapper/309:54:123
2372214259,68sleep30-21swapper/310:54:083
26403214055,71sleep326441-21fschecks_count10:28:563
14423214069,38sleep30-21swapper/311:33:583
18542213864,57sleep30-21swapper/310:03:533
9782213317,36sleep29816-21fschecks_count11:18:582
2213621339,109sleep322195-21date11:58:413
373021325,68sleep30-21swapper/310:59:013
11142213111,107sleep011189-21tr09:39:030
13507213017,89sleep30-21swapper/308:04:043
733821285,28sleep30-21swapper/309:28:283
20568212847,21sleep327580-21cyclictest10:09:063
1381212713,93sleep11537-21unixbench_multi07:24:121
14690212520,36sleep114683-21apt-get08:08:551
3457212424,70sleep112492-21ntpd10:58:501
13255212419,88sleep113295-21ps11:29:071
27875212114,77sleep30-21swapper/310:33:533
17392212112,92sleep317439-21ntp_offset09:59:043
16658212018,78sleep30-21swapper/311:39:103
12901211919,26sleep10-21swapper/108:03:421
3098221168,92sleep30-21swapper/310:43:503
3098221168,92sleep30-21swapper/310:43:503
3131821155,99sleep00-21swapper/012:24:140
21322211411,88sleep021377-21idleruntime08:29:010
2756521135,11sleep00-21swapper/010:33:250
18637211321,69sleep018740-21fschecks_time10:03:580
5131211215,82sleep25088-21latency_hist07:38:432
20637211113,78sleep020700-21/usr/sbin/munin10:09:070
806621090,6sleep00-21swapper/011:13:420
101721097,73sleep30-21swapper/312:33:503
7640210822,17sleep30-21swapper/309:28:523
9766210710,15sleep30-21swapper/307:53:423
28941210716,71sleep128977-21/usr/sbin/munin12:18:581
17672210717,59sleep10-21swapper/109:59:101
1302221078,13sleep20-21swapper/211:29:002
27034210515,70sleep227134-21users07:04:112
6616210418,32sleep30-21swapper/307:43:323
31000210423,49sleep230994-21apt-get10:43:532
31000210423,49sleep230994-21apt-get10:43:532
5816210318,64sleep05821-21ntp_states07:39:050
26326210323,18sleep325452-21/usr/sbin/munin12:09:113
3238121028,28sleep30-21swapper/309:03:573
12235210118,46sleep312364-21sort07:59:113
5851210010,23sleep30-21swapper/311:04:103
3864210013,23sleep13871-21diskstats07:33:571
1107421006,10sleep30-21swapper/307:56:133
107612999,32sleep310779-21cpu09:38:533
59562985,83sleep00-21swapper/011:04:310
178282985,11sleep30-21swapper/311:44:283
3227829618,56sleep00-21swapper/009:03:500
2913029620,59sleep029224-21apt-config07:13:410
2913029620,59sleep029224-21apt-config07:13:410
2556229623,52sleep225623-21fschecks_time12:08:582
2009229615,65sleep020115-21/usr/sbin/munin08:24:240
194929620,60sleep01956-21modprobe07:28:390
2402429520,54sleep024116-21fschecks_time12:03:590
14602945,28sleep30-21swapper/310:51:583
80992936,61sleep10-21swapper/107:47:471
251702926,71sleep325206-21ntpq10:24:043
215629218,39sleep10-21swapper/110:54:021
739529122,38sleep07398-21true09:28:410
3169229120,49sleep131799-21grep07:18:591
2680329133,36sleep00-21swapper/007:04:090
1470829118,56sleep014745-21ntp_kernel_err11:34:030
2465429018,52sleep224258-21/usr/sbin/munin08:39:042
19322895,26sleep30-21swapper/307:28:263
2592528718,43sleep226015-21grep08:44:012
241228713,62sleep02454-21idleruntime07:29:000
198128715,57sleep22067-21fschecks_time10:53:592
82182867,66sleep38249-21date07:48:413
3201628611,60sleep30-21swapper/309:03:403
297122869,14sleep30-21swapper/312:19:123
1971428619,48sleep119228-21/usr/sbin/munin11:49:061
101602868,59sleep30-21swapper/311:19:073
787128513,54sleep27912-21latency_hist11:13:412
140912850,5sleep00-21swapper/009:48:590
481528418,39sleep14874-21sshd09:19:011
1154528412,55sleep211614-21memory11:24:032
1154528412,55sleep211614-21memory11:24:032
580328313,53sleep30-21swapper/309:23:393
1373328323,31sleep10-21swapper/108:04:101
918828212,33sleep00-21swapper/009:33:540
2406028223,26sleep10-21swapper/108:38:431
216302826,14sleep30-21swapper/310:13:503
196092829,13sleep30-21swapper/308:23:553
536728116,30sleep10-21swapper/107:38:541
2734028121,36sleep20-21swapper/212:14:552
3130028024,31sleep10-21swapper/108:59:101
1791428013,52sleep117967-21idleruntime11:44:301
1725728013,25sleep00-21swapper/009:59:010
718527914,41sleep07281-21/usr/sbin/munin07:44:020
336327916,29sleep10-21swapper/109:14:041
336327916,29sleep10-21swapper/109:14:041
245092798,14sleep30-21swapper/308:39:013
43627849,13sleep30-21swapper/307:23:423
3208227811,53sleep232161-21mailstats07:19:092
180602786,13sleep30-21swapper/308:18:543
180602786,13sleep30-21swapper/308:18:543
94527713,45sleep032758-21/usr/sbin/munin10:50:050
322527715,46sleep03345-21latency_hist10:58:410
2893527614,45sleep128984-21wc08:53:411
237972767,13sleep00-21swapper/008:36:320
1541027614,30sleep20-21swapper/209:53:552
3137527513,46sleep231494-21/usr/sbin/munin08:59:112
2688527524,30sleep10-21swapper/107:04:101
2484327510,13sleep20-21swapper/212:04:132
2451027512,50sleep024526-21sh10:23:400
2438427510,33sleep30-21swapper/312:04:063
1698027514,36sleep217129-21grep08:14:132
3061127415,41sleep230669-21/usr/sbin/munin12:24:002
3022327413,17sleep10-21swapper/107:14:031
2687927414,48sleep11-21systemd12:13:411
2474527411,50sleep124761-21open_inodes08:39:071
112262746,12sleep30-21swapper/311:23:523
112262746,12sleep30-21swapper/311:23:523
186527310,16sleep00-21swapper/009:09:070
1308627313,41sleep013214-21/usr/sbin/munin09:44:110
2859027215,42sleep128730-21unixbench_singl10:34:111
1151527219,26sleep10-21swapper/111:24:021
1151527219,26sleep10-21swapper/111:24:021
631927115,42sleep26331-21systemd11:08:412
895127015,42sleep19015-21latency_hist09:33:421
739927014,42sleep17429-21cut09:28:421
1839627010,26sleep10-21swapper/108:19:531
1839627010,26sleep10-21swapper/108:19:531
938426926,22sleep10-21swapper/109:33:591
2135226924,27sleep10-21swapper/110:13:401
860726812,15sleep10-21swapper/107:49:001
772626812,25sleep00-21swapper/009:28:570
3230626814,22sleep10-21swapper/109:03:551
3128026815,37sleep231318-21munin-run07:18:402
2635526826,22sleep30-21swapper/307:04:003
1745726816,23sleep20-21swapper/209:59:072
1676826814,31sleep10-21swapper/109:58:421
127882686,11sleep00-21swapper/008:02:520
303626713,35sleep23043-21/usr/sbin/munin09:13:572
303626713,35sleep23043-21/usr/sbin/munin09:13:572
2555526718,21sleep20-21swapper/210:24:112
1734426717,24sleep20-21swapper/211:43:422
1674326714,40sleep016767-21sed09:58:410
1394926719,33sleep214031-21cut09:48:582
1215826717,35sleep212233-21grep07:59:102
3205526627,21sleep00-21swapper/012:28:580
292622667,11sleep00-21swapper/012:19:050
2763626612,35sleep20-21swapper/207:08:422
2763626612,35sleep20-21swapper/207:08:422
2677326615,37sleep226293-21/usr/sbin/munin10:29:042
2092126614,26sleep20-21swapper/208:28:422
1666726610,40sleep016691-21ntp_states08:14:060
155126611,18sleep10-21swapper/112:34:081
1487226617,30sleep114943-21/usr/sbin/munin11:34:081
1001626620,29sleep210017-21sshd07:54:012
67732659,40sleep16541-21/usr/sbin/munin11:09:001
768726415,34sleep27714-21df09:28:552
3076126416,28sleep20-21swapper/210:43:412
2944826421,22sleep00-21swapper/010:38:510
2138426415,36sleep021459-21switchtime11:54:100
1750526417,31sleep117532-21munin-run08:18:401
536826318,26sleep25435-21df07:38:552
2809026319,21sleep10-21swapper/107:09:021
2809026319,21sleep10-21swapper/107:09:021
1859026313,36sleep118617-21df10:03:551
1287826310,37sleep012962-21fschecks_time11:28:580
86426220,28sleep2902-21/usr/sbin/munin07:24:012
3230126218,26sleep132340-21apt-config10:48:411
3230126218,26sleep132340-21apt-config10:48:411
3051226216,18sleep10-21swapper/108:58:421
2126726220,30sleep221302-21ps11:54:082
1006826214,30sleep20-21swapper/209:34:122
838226112,33sleep18443-21/usr/sbin/munin11:14:011
319932617,12sleep00-21swapper/009:03:210
1639926114,20sleep00-21swapper/011:39:070
743026010,25sleep20-21swapper/209:28:432
3219926011,33sleep232222-21irqstats12:29:002
3165926019,24sleep10-21swapper/112:28:391
3002326013,16sleep10-21swapper/110:39:081
269226011,23sleep20-21swapper/207:29:062
2146026021,17sleep10-21swapper/111:54:111
209026013,27sleep10-21swapper/109:09:121
1042426019,27sleep010469-21/usr/sbin/munin07:54:120
56942595,16sleep00-21swapper/009:22:150
44825919,23sleep00-21swapper/007:23:430
2396725914,29sleep223993-21munin-run08:38:402
2172325914,31sleep221797-21grep10:13:572
2017625916,22sleep20-21swapper/210:08:572
463225818,20sleep20-21swapper/209:18:582
342325814,24sleep00-21swapper/009:14:070
342325814,24sleep00-21swapper/009:14:070
3011125812,23sleep20-21swapper/212:23:392
2762125814,21sleep00-21swapper/012:15:090
2722625811,32sleep126749-21apt-get12:13:451
2708725814,25sleep127114-21munin-run08:48:401
2411825810,35sleep124176-21if_eth012:04:001
1924925816,25sleep219246-21apt-get11:48:512
1778425815,17sleep00-21swapper/008:18:430
1778425815,17sleep00-21swapper/008:18:430
1255725817,19sleep00-21swapper/011:28:420
1255725817,19sleep00-21swapper/011:28:420
1027525816,25sleep110371-21sed11:19:091
2542925715,24sleep125553-21timerwakeupswit10:24:101
843525614,21sleep00-21swapper/007:48:510
287732565,42sleep00-21swapper/008:52:150
2608325617,15sleep00-21swapper/010:28:410
234422565,10sleep10-21swapper/112:01:051
2219725614,15sleep00-21swapper/011:58:420
2137825613,26sleep121427-21irqstats08:29:021
1879325619,17sleep20-21swapper/208:20:022
1879325619,17sleep20-21swapper/208:20:022
3107025519,20sleep00-21swapper/010:43:550
3107025519,20sleep00-21swapper/010:43:550
2602325513,24sleep026083-21sensors_temp12:09:080
1444125514,24sleep014451-21munin-run08:08:410
1231025513,26sleep112305-21apt-get09:43:521
2635425411,29sleep026412-21cat08:44:110
24812545,40sleep20-21swapper/209:11:152
241625412,25sleep22559-21latency_hist12:38:412
2290925413,23sleep022937-21munin-run10:18:400
223625415,23sleep12231-21apt-get07:28:531
1952325412,26sleep019619-21/usr/sbin/munin11:49:010
1103325412,25sleep211125-21/usr/sbin/munin09:39:012
1854125313,24sleep218536-21apt-get10:03:522
1591325313,24sleep115907-21apt-get11:38:521
1433725314,25sleep214330-21apt-get11:33:532
896725215,15sleep20-21swapper/207:49:092
857525214,19sleep20-21swapper/211:14:062
2787325210,24sleep227869-21apt-get10:33:522
2428325212,24sleep024277-21apt-get08:38:540
2322525211,24sleep223219-21apt-get10:18:542
176882526,18sleep20-21swapper/211:43:502
1028625212,19sleep10-21swapper/107:54:091
942925112,22sleep09460-21munin-run11:18:400
730525118,20sleep20-21swapper/207:44:042
510025118,17sleep10-21swapper/111:03:551
418625113,22sleep00-21swapper/010:59:100
2638825113,18sleep10-21swapper/110:28:541
2176725112,19sleep20-21swapper/208:29:112
103925113,21sleep01036-21apt-get12:33:510
3195025012,22sleep131947-21apt-get12:28:511
39802499,25sleep04035-21if_eth007:34:010
3249724915,20sleep232555-21if_eth009:04:002
2975724913,19sleep20-21swapper/208:54:152
2376324910,21sleep123832-21sensors_temp10:19:091
2091124911,20sleep10-21swapper/108:28:411
1322424910,26sleep213274-21fschecks_count08:03:592
1549924811,21sleep215590-21users08:09:122
152932489,23sleep015367-21find08:09:100
2787724711,20sleep027944-21df10:33:540
2918224513,16sleep00-21swapper/008:54:000
969624210,16sleep00-21swapper/011:18:520
229382376,12sleep20-21swapper/211:59:082
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional