You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-28 - 07:15

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7s.osadl.org (updated Fri Aug 28, 2026 00:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3640285018,818sleep128863-21kworker/u8:123:24:121
511527287,689sleep311544-1kworker/3:1H20:05:023
968260513,575sleep01056-21/usr/sbin/munin21:34:210
12730257014,536sleep312886-21grep22:09:573
2409238314,253sleep32559-21grep19:54:323
25629230718,269sleep225677-21unin-run00:34:012
284392304134,151sleep328486-21latency_hist21:19:023
323782273114,54sleep331749-21/usr/sbin/munin21:29:323
3179226869,174sleep33332-21unixbench_singl21:39:333
27563224313,216sleep027628-21awk21:14:310
6845223819,101sleep30-21swapper/321:54:033
9126223017,199sleep19161-21grep21:59:301
29358221638,25sleep30-21swapper/323:04:123
27353221513,104sleep327443-21latency_hist22:59:033
11262221210,188sleep311265-21acpi20:24:113
21048219421,55sleep321180-21latency_hist19:14:043
24529219219,158sleep324532-21cron19:24:013
14843218916,156sleep314861-21cpuspeed23:59:183
4182218213,147sleep33971-21/usr/sbin/munin21:44:263
12027218235,20sleep30-21swapper/323:49:253
16567218115,148sleep316654-21timerandwakeup20:39:343
3019721797,161sleep3374-21sshd21:24:113
845721760,117sleep30-21swapper/323:39:033
1802521769,149sleep318178-21latency_hist22:29:033
2784321749,16sleep30-21swapper/322:59:233
5987217311,141sleep35412-21/usr/sbin/munin23:29:353
719217114,140sleep3875-21grep23:14:333
719217114,140sleep3875-21grep23:14:333
24953216622,23sleep30-21swapper/322:49:303
30534216412,19sleep30-21swapper/319:41:163
6795216113,133sleep30-21swapper/320:09:223
20371216116,69sleep30-21swapper/320:54:033
20371216116,69sleep30-21swapper/320:54:033
26077216018,28sleep30-21swapper/319:29:013
164482158120,16sleep30-21swapper/322:24:023
11050215525,23sleep30-21swapper/320:24:033
3665215310,112sleep311544-1kworker/3:1H23:24:273
21058214915,118sleep021130-21unin-run22:39:010
17511214710,22sleep30-21swapper/300:05:083
1948321469,111sleep319558-21cat19:09:033
12812214411,23sleep30-21swapper/320:29:123
10837214411,121sleep310929-21cat22:04:363
3657214119,102sleep23639-21apt-get23:24:212
401214012,116sleep3437-21cat19:49:243
20152213717,25sleep320190-21grep00:14:283
3893213614,16sleep30-21swapper/319:59:333
22039213510,18sleep30-21swapper/300:19:333
23190213212,24sleep323230-21ntp_kernel_err00:24:253
2456213193,21sleep31833-21/usr/sbin/munin23:20:153
2085621295,11sleep30-21swapper/322:36:303
1549721286,41sleep30-21swapper/320:37:073
4527212723,84sleep04541-21apt-get20:04:020
3092821279,22sleep330931-21apt-get23:09:123
13300212717,93sleep013341-21proc_pri20:29:290
30173212415,87sleep130336-21unixbench_multi19:39:331
2374521247,17sleep30-21swapper/321:04:173
6959212326,68sleep06978-21apt-get23:34:130
22189212394,13sleep30-21swapper/320:59:163
14850212222,27sleep10-21swapper/120:34:311
2417821218,100sleep324209-21/usr/sbin/munin22:49:023
2948321198,66sleep30-21swapper/321:19:353
6282211811,91sleep26395-21unixbench_singl21:49:342
28555211815,78sleep10-21swapper/119:34:321
23273211636,49sleep123268-21apt-get19:19:141
17969211615,85sleep217990-21awk20:44:292
2663321157,14sleep30-21swapper/321:11:483
2663321157,14sleep30-21swapper/321:11:483
1823821146,13sleep30-21swapper/322:29:153
8896211322,70sleep29041-21unixbench_singl20:14:332
24831211312,83sleep224840-21/usr/sbin/munin22:49:252
1321021136,14sleep30-21swapper/323:54:183
32347211210,44sleep30-21swapper/319:49:033
1362621116,44sleep30-21swapper/322:14:173
910121107,62sleep30-21swapper/321:59:283
2691921095,11sleep30-21swapper/319:29:383
1880621097,50sleep30-21swapper/320:49:023
2943210813,72sleep02968-21munin-run19:59:010
2176421085,11sleep30-21swapper/320:57:393
947621066,11sleep30-21swapper/323:39:363
26141210619,66sleep20-21swapper/222:54:212
1111421065,10sleep30-21swapper/323:45:463
831921056,11sleep30-21swapper/321:58:433
21770210513,76sleep021814-21awk22:39:260
16918210513,74sleep117000-21memory00:04:571
924221045,10sleep30-21swapper/320:16:403
7464210416,70sleep27513-21ntpq21:54:262
4560210429,42sleep14628-21cat20:04:031
2686121045,10sleep30-21swapper/300:34:593
26592210414,77sleep026673-21cat22:54:320
2551121046,11sleep30-21swapper/300:32:293
2795321036,60sleep30-21swapper/319:34:183
1929321035,10sleep30-21swapper/300:12:433
16431210315,17sleep10-21swapper/120:39:321
1141021038,15sleep00-21swapper/020:24:210
2231221025,11sleep30-21swapper/322:39:573
12004210215,68sleep00-21swapper/022:09:140
31108210011,74sleep231117-21df_inode19:44:182
226072995,42sleep30-21swapper/319:14:403
226072995,42sleep30-21swapper/319:14:403
206629917,58sleep21833-21/usr/sbin/munin23:20:052
20602995,41sleep30-21swapper/321:38:083
302329819,59sleep23070-21cat19:59:032
399629710,70sleep20-21swapper/221:44:172
3083429713,65sleep030987-21latency_hist19:44:040
213542957,69sleep10-21swapper/100:19:191
3148429428,45sleep031506-21apt21:29:010
1401829427,48sleep214050-21proc_pri22:14:292
2989829311,64sleep129964-21mailstats23:04:341
97112925,11sleep20-21swapper/220:19:102
552429219,51sleep15555-21fschecks_count23:29:211
396529220,54sleep00-21swapper/021:44:110
2505329216,59sleep025138-21apt-key21:09:020
2422729213,61sleep024276-21grep22:49:030
2126629213,62sleep121355-21timerwakeupswit20:54:341
1784729217,58sleep017871-21munin-plugin-st00:09:000
2551529117,48sleep20-21swapper/219:24:322
1605929121,53sleep216123-21unin-run00:04:022
2727428915,55sleep127301-21munin-run22:59:011
1960828616,49sleep219033-21/usr/sbin/munin20:49:302
1960828616,49sleep219033-21/usr/sbin/munin20:49:302
1888128617,50sleep019008-21/usr/sbin/munin22:29:330
997828524,31sleep20-21swapper/222:04:032
3148428516,55sleep131557-21grep23:09:311
2319328312,49sleep223282-21/usr/sbin/munin22:44:242
43628217,28sleep10-21swapper/123:14:271
43628217,28sleep10-21swapper/123:14:271
2914128211,57sleep229160-21open_inodes21:19:302
279852820,6sleep20-21swapper/222:59:252
1455228217,46sleep014684-21latency_hist23:59:030
289112810,6sleep00-21swapper/021:19:230
1792428111,17sleep00-21swapper/020:44:270
737628025,23sleep20-21swapper/220:09:332
1162228023,32sleep211819-21apt-config22:09:032
1737427820,37sleep217509-21unixbench_singl00:05:072
882427713,47sleep18895-21sed20:14:321
2600527720,38sleep126022-21uniq00:34:201
2413427720,34sleep124104-21latency_hist00:29:041
1335227715,46sleep113505-21latency_hist22:14:031
875927617,24sleep00-21swapper/021:59:210
2828027622,36sleep228304-21netstat19:34:252
2039727616,29sleep00-21swapper/020:54:040
2786627521,27sleep00-21swapper/022:59:240
2027427515,39sleep119667-21/usr/sbin/munin00:14:301
365527423,31sleep00-21swapper/023:24:190
2304127414,45sleep223193-21date19:19:032
2304127414,45sleep223193-21date19:19:032
1665727414,29sleep00-21swapper/020:39:350
1941827321,34sleep2289-21dbus-daemon19:09:022
1941127324,31sleep119413-21cron19:09:011
181122739,14sleep00-21swapper/000:09:120
1516227312,41sleep014767-21/usr/sbin/munin23:59:270
3221127213,45sleep032246-21munin-run23:14:020
2768927213,41sleep127778-21fschecks_time22:59:211
2255627214,29sleep00-21swapper/020:59:250
1823227215,35sleep118329-21fschecks_time00:09:221
1201527211,45sleep20-21swapper/222:09:202
823827112,42sleep08242-21diskstats20:14:190
2118827110,17sleep00-21swapper/019:14:050
2118827110,17sleep00-21swapper/019:14:050
144727120,29sleep20-21swapper/219:54:032
1019327115,38sleep210232-21fschecks_count23:44:202
2602327016,31sleep20-21swapper/200:34:212
2522427022,23sleep20-21swapper/221:09:042
2137127018,32sleep221365-21apt-get22:39:162
1186827014,23sleep00-21swapper/023:49:220
1048427013,40sleep110631-21awk20:19:331
381626914,36sleep20-21swapper/219:59:322
3069926919,25sleep20-21swapper/223:09:032
2490226918,28sleep10-21swapper/119:24:191
1717626913,24sleep20-21swapper/222:24:342
717926822,22sleep00-21swapper/020:09:310
698626811,12sleep10-21swapper/123:34:181
50626816,21sleep00-21swapper/023:14:300
50626816,21sleep00-21swapper/023:14:300
2388526826,20sleep20-21swapper/219:19:322
1678126813,16sleep10-21swapper/122:24:251
1259426816,32sleep20-21swapper/220:29:032
2258626716,25sleep10-21swapper/120:59:261
1971526717,25sleep00-21swapper/019:09:120
31052666,13sleep10-21swapper/123:23:481
2149926611,42sleep121548-21wc19:14:101
2149926611,42sleep121548-21wc19:14:101
2036726611,12sleep20-21swapper/200:14:312
1886526627,21sleep00-21swapper/019:05:010
176522668,39sleep00-21swapper/022:25:130
1564426620,20sleep00-21swapper/022:19:310
222826513,28sleep10-21swapper/119:54:301
114522655,11sleep00-21swapper/022:08:410
1059426521,23sleep10-21swapper/122:04:301
615426421,19sleep10-21swapper/121:49:331
329526412,35sleep03400-21sed23:24:030
3217726427,19sleep20-21swapper/221:29:272
2379626417,27sleep10-21swapper/121:04:191
2191126415,33sleep222038-21sed00:19:322
1905926416,25sleep10-21swapper/120:49:171
1905926416,25sleep10-21swapper/120:49:171
1381826416,21sleep00-21swapper/022:14:220
1948126323,18sleep20-21swapper/200:14:032
306026212,18sleep20-21swapper/221:39:322
2925326213,34sleep029362-21latency_hist19:39:030
255302626,11sleep10-21swapper/100:32:431
199202626,13sleep20-21swapper/222:34:232
1192226217,31sleep111941-21kernelversion23:49:231
982026116,23sleep00-21swapper/023:44:010
3224226117,29sleep132302-21/usr/sbin/munin21:29:301
2135826114,28sleep221507-21/usr/sbin/munin20:54:352
1872326122,19sleep20-21swapper/222:29:312
1835926117,29sleep118447-21/usr/sbin/munin22:29:211
178426111,36sleep2850-21/usr/sbin/munin21:34:362
726326014,27sleep17375-21/usr/sbin/munin20:09:321
3059426015,28sleep130623-21/usr/sbin/munin21:24:251
2557226018,24sleep125534-21apt-get21:09:361
2557226018,24sleep125534-21apt-get21:09:361
2268226018,23sleep10-21swapper/122:44:031
130492605,13sleep20-21swapper/223:54:052
618625911,34sleep2367-21snmpd20:07:192
530625913,20sleep00-21swapper/020:05:110
2718025916,23sleep20-21swapper/221:14:212
2481725916,27sleep024812-21apt-get19:24:140
1759325912,28sleep117595-21/usr/sbin/munin20:44:201
1363825915,27sleep113661-21ntp_states23:54:281
325892588,27sleep00-21swapper/019:49:160
2383425812,20sleep10-21swapper/122:44:361
220825817,24sleep10-21swapper/121:39:031
1914325815,24sleep00-21swapper/020:49:200
1447425817,16sleep00-21swapper/020:34:220
1334325814,18sleep10-21swapper/120:29:301
901925714,24sleep10-21swapper/123:39:261
895425711,30sleep28532-21/usr/sbin/munin23:39:232
609025718,17sleep10-21swapper/120:06:091
327572575,10sleep00-21swapper/021:30:300
2895125724,19sleep10-21swapper/121:19:251
2748025715,21sleep00-21swapper/019:32:090
2104025710,30sleep021064-21ntp_states20:54:290
3235025616,18sleep00-21swapper/019:49:040
2646325618,22sleep10-21swapper/119:29:281
2502125618,17sleep00-21swapper/022:49:310
2226925611,29sleep222273-21diskstats20:59:192
2113125616,20sleep10-21swapper/122:39:021
1479325614,25sleep114787-21apt-get23:59:161
1211725614,26sleep212125-21ntpq23:49:282
834225513,22sleep18369-21munin-run21:59:011
832325514,24sleep28455-21latency_hist23:39:032
674125514,28sleep26789-21latency_hist23:34:022
375425517,19sleep00-21swapper/021:44:020
3248725512,26sleep232483-21apt-get23:14:162
3248725512,26sleep232483-21apt-get23:14:162
3153525514,25sleep20-21swapper/221:29:032
2936125516,20sleep00-21swapper/023:04:130
2572125511,21sleep00-21swapper/000:34:030
24192555,11sleep10-21swapper/121:39:111
974225413,25sleep09736-21apt-get20:19:160
708825413,25sleep07082-21apt-get21:54:160
3104025413,24sleep131107-21df19:44:171
2509825312,16sleep10-21swapper/122:49:321
1640225316,23sleep216447-21unin-run22:24:012
1355725313,23sleep113551-21apt-get22:14:151
1291525314,23sleep212921-21diskstats20:29:192
608425217,18sleep00-21swapper/021:49:320
1061725214,17sleep00-21swapper/023:44:300
3155925111,18sleep20-21swapper/223:09:322
1445825114,21sleep214464-21df_inode20:34:202
1056125115,18sleep10-21swapper/123:44:271
2257425012,22sleep022583-21munin-run00:24:020
1981725010,22sleep119826-21diskstats19:09:221
1969025011,23sleep019686-21apt-get00:14:130
139725015,18sleep10-21swapper/119:54:021
2948524914,20sleep229488-21apt-get19:39:132
2557124913,18sleep00-21swapper/021:09:350
2557124913,18sleep00-21swapper/021:09:350
2290824916,20sleep20-21swapper/200:24:202
543924812,18sleep00-21swapper/023:29:160
400024812,18sleep10-21swapper/121:44:201
2731924810,20sleep127364-21irqstats21:14:241
263432487,14sleep20-21swapper/219:29:122
2282124810,21sleep022816-21apt-get00:24:140
2019724810,22sleep220203-21ntp_states19:09:312
1557924811,22sleep115598-21ntp_states22:19:281
2334424710,15sleep00-21swapper/019:19:180
1603424711,22sleep216041-21df_inode20:39:202
1182224711,21sleep111893-21sensors_temp20:24:301
2550824611,21sleep225020-21apt-get21:09:082
2550824611,21sleep225020-21apt-get21:09:082
2649324513,16sleep00-21swapper/000:34:320
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional