You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-03 - 16:26

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7s.osadl.org (updated Mon Aug 03, 2026 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1243729336,916sleep012478-21ntpq08:27:570
22876269622,656sleep122922-21/usr/sbin/munin07:18:021
22876269622,656sleep122922-21/usr/sbin/munin07:18:021
1637244263,77sleep01666-21munin-plugin-st11:22:310
325122313173,78sleep032517-21cron11:17:310
9218228918,256sleep19286-21/usr/sbin/munin10:02:471
11497228525,249sleep0364-21snmpd08:24:020
17283228035,206sleep017310-21df10:27:490
17283228035,206sleep017310-21df10:27:490
24779227416,241sleep224747-21latency_hist10:52:342
1762822715,251sleep00-21swapper/008:43:070
2206522528,232sleep022134-21basename10:42:510
19415225012,131sleep00-21swapper/010:33:060
19415225012,131sleep00-21swapper/010:33:060
3761224919,161sleep00-21swapper/009:43:040
897922388,217sleep08946-21latency_hist10:02:340
4215221816,182sleep14363-21unixbench_singl11:28:011
948222116,171sleep0364-21snmpd11:45:250
24994220814,148sleep025004-21acpi10:52:440
4616219716,119sleep04707-21switchtime08:03:020
20883218727,74sleep00-21swapper/007:12:440
667621816,161sleep00-21swapper/011:37:500
992221706,153sleep00-21swapper/008:21:480
7797216835,25sleep00-21swapper/008:13:040
13744216526,119sleep30-21swapper/311:58:033
13590216412,138sleep113665-21sensors_temp11:58:011
25350216338,109sleep00-21swapper/007:27:340
25178216371,24sleep00-21swapper/009:08:030
14317216218,130sleep314462-21unixbench_singl08:33:013
24302216170,35sleep024272-21latency_hist09:07:340
27859216016,128sleep327890-21awk11:02:323
22783215829,87sleep022831-21ntp_kernel_pll_12:27:560
30271215514,88sleep00-21swapper/007:42:480
141421537,50sleep00-21swapper/009:37:420
24539215220,116sleep10-21swapper/109:07:471
5704214630,12sleep00-21swapper/009:51:230
27893214631,98sleep00-21swapper/011:02:330
2529821457,126sleep025358-21date12:37:330
1870721447,15sleep00-21swapper/007:07:430
614214224,98sleep3639-21idleruntime-cro07:52:323
13985214159,41sleep00-21swapper/010:17:520
3183213920,97sleep33213-21munin-run11:27:313
22593213825,91sleep00-21swapper/007:17:570
22593213825,91sleep00-21swapper/007:17:570
2086421369,19sleep00-21swapper/012:22:470
8475213418,100sleep28494-21fschecks_count11:42:522
15931213111,108sleep016004-21latency_hist12:07:330
22167212945,72sleep022244-21grep08:58:050
2292212727,54sleep02300-21uname11:22:560
19842212713,81sleep00-21swapper/008:52:440
6901212527,50sleep07020-21latency_hist08:12:350
3249721259,77sleep032638-21sed07:48:050
12864212425,78sleep212766-21latency_hist11:57:342
11106212426,12sleep00-21swapper/011:51:220
27858212113,81sleep00-21swapper/009:17:520
27858212113,81sleep00-21swapper/009:17:520
18771211825,26sleep00-21swapper/008:48:000
2475211719,84sleep22548-21grep11:23:022
10574211715,63sleep010721-21latency_hist10:07:340
18977211621,77sleep319116-21tail08:48:033
18896211618,87sleep00-21swapper/012:16:130
13666211620,73sleep213036-21/usr/sbin/munin11:58:022
28153211515,48sleep00-21swapper/011:02:490
15842211537,42sleep015878-21ntpq10:22:560
14376211524,64sleep10-21swapper/110:18:011
26548211412,81sleep326563-21/usr/sbin/munin09:12:583
1576211339,52sleep00-21swapper/007:53:050
29773211233,37sleep00-21swapper/011:07:500
21187211221,73sleep321225-21latency07:12:593
14714210816,72sleep214810-21fschecks_time12:02:512
6739210717,72sleep36867-21timerwakeupswit09:53:043
27754210718,73sleep127862-21timerandwakeup07:33:041
11290210715,65sleep110484-21/usr/sbin/munin08:23:101
1327821055,38sleep00-21swapper/010:13:440
24414210318,68sleep124420-21ntp_states12:32:581
11459210336,37sleep30-21swapper/311:52:423
32323210215,72sleep232418-21mailstats07:48:032
2818821005,35sleep00-21swapper/007:34:480
1076521008,12sleep20-21swapper/210:07:492
10269210030,41sleep310428-21latency_hist08:22:353
295932998,53sleep029632-21ntpq09:22:570
295932998,53sleep029632-21ntpq09:22:570
510929815,30sleep20-21swapper/211:32:512
510929815,30sleep20-21swapper/211:32:512
1052229627,46sleep310570-21cat10:07:333
621029443,40sleep0364-21snmpd11:36:360
621029443,40sleep0364-21snmpd11:36:360
2635729314,63sleep226432-21cat09:12:532
1748129317,52sleep317625-21head08:43:063
1645129314,23sleep116535-21memory12:07:541
1187629322,18sleep00-21swapper/011:52:560
2602329214,15sleep10-21swapper/109:12:361
1359929240,36sleep00-21swapper/008:32:420
70732906,73sleep00-21swapper/009:54:020
27592909,45sleep02825-21wc07:57:580
40882896,70sleep00-21swapper/011:28:000
2110228920,53sleep321143-21sh08:57:323
1891928919,51sleep118982-21fschecks_time10:32:551
1891928919,51sleep118982-21fschecks_time10:32:551
848228819,55sleep28532-21wc08:17:342
79832880,11sleep318529-21cobalt_printf11:42:323
3034428813,61sleep230353-21/usr/sbin/munin07:42:502
1960528832,28sleep20-21swapper/208:52:342
2522028712,60sleep325296-21systemd-run12:37:323
2168528615,55sleep121702-21cut08:57:541
1750528632,35sleep00-21swapper/012:12:340
894228517,50sleep19021-21switchtime11:43:031
269692856,64sleep00-21swapper/009:13:030
2559328531,14sleep00-21swapper/007:27:460
2010628521,43sleep220192-21munin-node07:12:332
1670428522,39sleep316718-21apt-get10:27:323
1761928421,48sleep117675-21processes07:03:001
1761928421,48sleep117675-21processes07:03:001
1267328423,35sleep20-21swapper/208:28:032
339028327,27sleep10-21swapper/109:42:561
3075128313,50sleep130246-21/usr/sbin/munin07:43:011
2253128319,46sleep222626-21timerandwakeup10:43:012
231702820,6sleep30-21swapper/312:28:023
2002228227,36sleep120067-21munin-run07:12:321
112028122,18sleep10-21swapper/109:37:311
849528020,37sleep38594-21fschecks_time11:42:533
789428028,29sleep27928-21munin-run11:42:322
2802828013,48sleep327645-21/usr/sbin/munin09:17:553
2802828013,48sleep327645-21/usr/sbin/munin09:17:553
2502728012,55sleep225099-21mailstats09:08:012
404227917,36sleep10-21swapper/108:02:471
2062527917,46sleep320705-21date12:22:343
1723127929,26sleep00-21swapper/007:02:510
1723127929,26sleep00-21swapper/007:02:510
158212797,57sleep015943-21sed08:38:030
2304127814,52sleep223169-21sed12:28:022
1846527814,17sleep20-21swapper/212:13:042
1212027816,40sleep00-21swapper/010:12:350
3175027733,12sleep00-21swapper/009:28:580
3175027733,12sleep00-21swapper/009:28:580
265432770,8sleep30-21swapper/310:57:343
244202777,56sleep0364-21snmpd10:48:320
2537527618,31sleep10-21swapper/110:52:571
2383027621,25sleep20-21swapper/207:22:352
2383027621,25sleep20-21swapper/207:22:352
2237727621,30sleep30-21swapper/310:42:583
2384127520,29sleep10-21swapper/110:47:561
130572748,49sleep00-21swapper/011:57:440
1292227415,39sleep313027-21timerandwakeup10:13:023
2985327329,11sleep00-21swapper/007:40:580
243082735,12sleep20-21swapper/207:22:552
2021627318,25sleep30-21swapper/307:12:343
1010727318,40sleep110198-21dpkg08:22:331
2870127217,26sleep10-21swapper/111:03:021
2334827225,29sleep30-21swapper/309:02:573
2216927226,26sleep10-21swapper/112:27:331
193472729,18sleep00-21swapper/012:17:510
2969927124,19sleep20-21swapper/211:07:472
1522427118,31sleep215371-21grep10:22:342
10572715,54sleep00-21swapper/009:36:410
2586627023,28sleep20-21swapper/207:27:562
2355327014,37sleep323560-21/usr/sbin/munin10:47:503
2167927017,31sleep30-21swapper/310:42:343
1612327016,24sleep20-21swapper/210:23:022
354426915,37sleep33553-21df_inode11:27:473
558226810,14sleep10-21swapper/108:07:441
534626822,28sleep30-21swapper/308:07:333
3206826822,24sleep30-21swapper/309:32:333
3206826822,24sleep30-21swapper/309:32:333
306992687,17sleep10-21swapper/111:09:121
3158426724,21sleep30-21swapper/307:47:353
2889226717,31sleep329042-21grep11:03:043
2538126719,20sleep10-21swapper/107:27:351
2086226724,25sleep120006-21apt-get07:12:411
2051626724,21sleep20-21swapper/208:53:032
181726716,36sleep31865-21ntpq09:37:583
838926617,19sleep20-21swapper/209:58:032
744826619,23sleep10-21swapper/109:57:341
461726614,32sleep24621-21diskstats09:47:502
3145126619,24sleep20-21swapper/209:28:032
3145126619,24sleep20-21swapper/209:28:032
3002926618,20sleep30-21swapper/307:42:343
2227926618,18sleep20-21swapper/207:17:342
2067126612,39sleep120738-21memory10:37:551
1907626620,23sleep30-21swapper/312:17:333
1735526625,17sleep20-21swapper/208:43:052
1044026619,30sleep110532-21grep11:48:001
529226522,18sleep10-21swapper/111:32:541
529226522,18sleep10-21swapper/111:32:541
313802659,14sleep20-21swapper/211:12:522
2936026516,19sleep20-21swapper/207:38:012
2461226519,20sleep30-21swapper/309:07:493
2325526516,32sleep223347-21/usr/sbin/munin09:02:562
202262656,48sleep00-21swapper/010:37:340
1282026517,23sleep10-21swapper/111:57:331
1140326513,32sleep111520-21sort10:08:041
1124726521,25sleep20-21swapper/211:52:332
1051326516,24sleep30-21swapper/308:22:483
3133226414,24sleep10-21swapper/111:12:511
298726411,30sleep30-21swapper/309:42:463
2786426416,29sleep228002-21unixbench_singl07:33:052
129026414,23sleep20-21swapper/211:18:052
970526321,19sleep20-21swapper/210:03:002
732126315,32sleep17357-21/usr/sbin/munin08:12:521
2921926311,25sleep20-21swapper/209:22:462
2921926311,25sleep20-21swapper/209:22:462
2764426322,23sleep20-21swapper/209:17:422
2764426322,23sleep20-21swapper/209:17:422
2466726319,22sleep20-21swapper/212:33:032
2131426318,28sleep321316-21ntp_states12:22:583
205026313,28sleep10-21swapper/109:38:041
120026322,24sleep31315-21sed09:37:333
1048226318,25sleep20-21swapper/208:22:422
853326216,19sleep30-21swapper/308:17:353
1899726224,19sleep10-21swapper/112:17:311
1747126222,18sleep30-21swapper/312:12:333
1720326212,17sleep30-21swapper/310:27:443
1720326212,17sleep30-21swapper/310:27:443
153826221,18sleep20-21swapper/209:37:532
272526110,34sleep30-21swapper/307:57:573
2667126114,25sleep20-21swapper/210:57:502
2520426122,17sleep20-21swapper/210:52:542
227926116,27sleep10-21swapper/107:57:341
2042226119,21sleep10-21swapper/108:53:021
184892619,11sleep20-21swapper/210:32:272
184892619,11sleep20-21swapper/210:32:272
1461626118,25sleep314613-21apt-get12:02:443
2848226015,22sleep30-21swapper/307:37:343
1489126015,27sleep114985-21memory12:02:541
322702596,11sleep30-21swapper/311:14:313
3084825915,27sleep130852-21diskstats09:27:501
3084825915,27sleep130852-21diskstats09:27:501
2975725914,17sleep10-21swapper/109:23:021
2975725914,17sleep10-21swapper/109:23:021
1627825911,30sleep216375-21perl12:07:512
1450625918,19sleep30-21swapper/310:18:033
785625818,24sleep10-21swapper/111:42:311
3231925817,26sleep232320-21http09:32:482
321425816,21sleep20-21swapper/207:58:082
2931625813,21sleep30-21swapper/309:22:513
2931625813,21sleep30-21swapper/309:22:513
282425811,21sleep10-21swapper/111:23:051
2708025814,20sleep10-21swapper/110:58:021
1987025820,24sleep119957-21sed12:18:041
1968125818,19sleep20-21swapper/210:33:082
1968125818,19sleep20-21swapper/210:33:082
1847725820,19sleep30-21swapper/307:07:343
1847725820,19sleep30-21swapper/307:07:343
168925815,20sleep30-21swapper/311:22:343
514825718,19sleep30-21swapper/309:48:033
2139425720,24sleep20-21swapper/208:57:432
1360525715,25sleep213600-21apt-get08:32:442
1019925713,25sleep29992-21apt-get08:22:342
66725615,20sleep10-21swapper/107:52:341
666225622,16sleep20-21swapper/209:53:032
357225615,22sleep20-21swapper/209:43:012
2969525613,26sleep329692-21apt-get11:07:443
2297825615,24sleep122973-21apt-get09:02:471
91125514,25sleep3907-21apt-get07:52:483
518325514,26sleep35241-21/usr/sbin/munin11:32:523
518325514,26sleep35241-21/usr/sbin/munin11:32:523
2898725516,18sleep10-21swapper/107:37:521
2559625518,19sleep30-21swapper/307:27:493
2083725516,19sleep20-21swapper/210:38:002
1381925513,24sleep113876-21irqstats08:32:511
1093025517,23sleep30-21swapper/310:07:543
607225413,27sleep35565-21/usr/sbin/munin08:08:023
1649425413,19sleep30-21swapper/308:42:343
1518225413,24sleep215179-21apt-get08:37:442
2766925313,23sleep127664-21apt-get09:17:451
2766925313,23sleep127664-21apt-get09:17:451
14862535,36sleep10-21swapper/111:20:341
501725214,22sleep15075-21/usr/sbin/munin09:48:011
2814925213,22sleep228143-21apt-get11:02:472
2296925214,18sleep10-21swapper/112:28:011
2012925214,15sleep30-21swapper/310:37:333
2012925214,15sleep30-21swapper/310:37:333
633225111,21sleep20-21swapper/208:08:052
37262517,35sleep10-21swapper/108:02:151
2093725112,24sleep220946-21/usr/sbin/munin12:22:492
1820825111,15sleep10-21swapper/110:29:031
1820825111,15sleep10-21swapper/110:29:031
1365725113,15sleep20-21swapper/210:17:332
97825015,20sleep2989-21df_inode07:52:512
827625014,21sleep18388-21cat09:58:021
40432508,15sleep20-21swapper/208:02:482
347625013,22sleep23543-21df11:27:462
2141125012,21sleep321414-21apt-get08:57:443
1670025015,19sleep10-21swapper/110:27:301
120342509,18sleep30-21swapper/308:27:443
2450324911,22sleep324509-21ntp_states07:23:003
2250524913,21sleep30-21swapper/307:17:513
2250524913,21sleep30-21swapper/307:17:513
1673924911,22sleep116731-21apt-get08:42:481
1038924914,16sleep30-21swapper/311:47:583
896724812,20sleep38977-21munin-run10:02:333
2198924810,15sleep10-21swapper/110:42:481
1364724810,22sleep113656-21munin-run10:17:321
2692324714,18sleep126869-21latency_hist07:32:351
1977024711,19sleep219776-21ntp_states12:18:012
176762477,13sleep20-21swapper/207:03:012
176762477,13sleep20-21swapper/207:03:012
1528424611,21sleep115388-21grep08:37:521
121362456,21sleep10-21swapper/108:27:501
182822449,18sleep10-21swapper/108:47:451
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional