You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-22 - 23:27

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Sat Aug 22, 2026 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
133142633235,6285sleep012668-21kworker/0:111:04:580
84772204710,2025sleep24952-1kworker/2:1H12:35:122
25044274813,719sleep125051-21/usr/sbin/munin08:19:391
25044274813,719sleep125051-21/usr/sbin/munin08:19:391
507427247,664sleep05110-21ntpq07:14:450
22510259514,558sleep022657-21grep08:09:530
3062923430,8sleep20-21swapper/208:34:532
1531223390,167sleep02539-21cyclictest07:49:210
1531223390,167sleep02539-21cyclictest07:49:210
62932319128,163sleep06344-21cpuspeed07:19:380
1320223059,163sleep013234-21munin-run09:24:220
8885230099,186sleep08921-21ntp_kernel_pll_12:35:300
28177224051,162sleep028324-21unixbench_singl10:09:520
9975222933,176sleep010132-21unixbench_multi10:55:050
3877221646,22sleep00-21swapper/008:54:230
3877221646,22sleep00-21swapper/008:54:230
8994221010,177sleep08763-21latency_hist10:54:240
8994221010,177sleep08763-21latency_hist10:54:240
8104220216,146sleep08175-21latency07:24:430
3267322028,41sleep032690-21munin-run08:44:210
15554219311,129sleep015000-21/usr/sbin/munin09:29:510
1005321918,170sleep010088-21awk09:14:210
200321835,166sleep00-21swapper/008:49:140
1926218211,157sleep11945-21apt-get12:14:321
19405217529,120sleep119551-21unixbench_multi07:59:531
2562217211,106sleep00-21swapper/010:34:220
2562217211,106sleep00-21swapper/010:34:220
2078621697,21sleep00-21swapper/008:04:520
567521689,146sleep05680-21cpu08:59:360
27125216613,111sleep027263-21head11:49:520
2958121648,13sleep10-21swapper/111:59:381
29991215912,104sleep029486-21/usr/sbin/munin11:59:490
1166021527,104sleep00-21swapper/007:34:560
14785214715,65sleep00-21swapper/009:29:220
381521426,13sleep00-21swapper/012:19:440
875021409,117sleep18762-21kernelversion12:35:271
1414221385,11sleep30-21swapper/307:44:403
1414221385,11sleep30-21swapper/307:44:403
23863213620,91sleep323939-21wc11:39:503
1181621360,73sleep111820-21unixbench_singl07:34:561
1607321356,91sleep00-21swapper/007:49:580
1525321319,87sleep015393-21latency_hist11:14:230
7321212710,26sleep00-21swapper/012:29:530
5093212323,28sleep00-21swapper/012:24:360
30056212282,27sleep00-21swapper/010:16:260
30056212282,27sleep00-21swapper/010:16:260
28725212218,88sleep1231-21systemd-journal10:14:211
20375212253,42sleep00-21swapper/009:44:530
27954211918,60sleep00-21swapper/008:29:230
25683211910,97sleep025811-21latency_hist10:04:230
1170211514,84sleep31145-21/usr/sbin/munin07:04:353
417321147,92sleep04329-21latency_hist10:39:230
18722211429,22sleep00-21swapper/009:39:520
2428211213,65sleep30-21swapper/312:14:503
52621095,71sleep00-21swapper/012:09:420
52621095,71sleep00-21swapper/012:09:420
2295121095,70sleep00-21swapper/011:39:020
2790821088,14sleep02539-21cyclictest11:54:320
2178421089,52sleep00-21swapper/009:49:490
21476210843,50sleep021518-21date11:34:220
1151621086,67sleep00-21swapper/009:18:080
3238421078,63sleep032464-21apt-config12:09:220
17074210738,30sleep00-21swapper/011:19:360
7602210616,74sleep27646-21ntpq09:04:462
26007210616,75sleep126087-21tune2fs10:04:401
1190021068,88sleep0367-21snmpd11:01:390
10714210617,68sleep010869-21latency_hist07:34:240
6979210538,36sleep00-21swapper/010:46:410
20542210520,14sleep10-21swapper/108:04:451
12481210517,73sleep312517-21fschecks_count11:04:413
9233210423,62sleep38749-21apt-get10:54:313
546121049,56sleep00-21swapper/008:59:230
29602210418,62sleep00-21swapper/008:34:240
2661721046,14sleep20-21swapper/211:49:422
6961210314,71sleep26990-21munin-run09:04:222
498321039,83sleep05074-21sed10:39:510
17022210315,73sleep317031-21ntpq09:34:463
21326210214,66sleep221216-21/usr/sbin/munin09:49:392
1825021026,11sleep00-21swapper/007:57:210
17033210210,55sleep017075-21uname09:34:470
1250721027,53sleep00-21swapper/007:39:370
1771210122,64sleep11859-21ls07:04:501
28575210016,63sleep327909-21/usr/sbin/munin11:54:513
3104029915,70sleep331059-21apt-get12:04:323
2951029923,37sleep20-21swapper/211:59:352
2553429834,16sleep00-21swapper/008:19:500
2553429834,16sleep00-21swapper/008:19:500
1716829817,59sleep117257-21fschecks_time11:19:411
542629713,63sleep21-21systemd08:59:212
2099429720,59sleep121012-21latency_hist09:49:221
484429622,49sleep24909-21awk10:39:492
39592965,12sleep00-21swapper/007:13:400
1969529629,43sleep319691-21apt-get09:44:353
1623229613,68sleep316335-21grep11:14:543
1623229613,68sleep316335-21grep11:14:543
639829414,61sleep36461-21/usr/sbin/munin10:44:513
30212945,55sleep00-21swapper/012:16:410
2404529417,63sleep324119-21systemd-run09:59:213
2404529417,63sleep324119-21systemd-run09:59:213
3264629311,60sleep232354-21apt-get12:09:242
3264629311,60sleep232354-21apt-get12:09:242
322432937,46sleep00-21swapper/010:24:410
3059829312,67sleep330601-21cpu10:19:363
3024029321,48sleep330384-21grep11:59:523
129672939,69sleep10-21swapper/107:39:501
1898629249,31sleep019005-21kernelversion07:59:430
3241929113,24sleep332511-21memory10:24:443
2926229119,57sleep129283-21ls11:59:221
690529015,53sleep26921-21/usr/sbin/munin12:29:432
245332905,37sleep00-21swapper/011:44:070
2958728918,52sleep329666-21sort10:14:513
2958728918,52sleep329666-21sort10:14:513
249212899,23sleep00-21swapper/011:44:390
2911028821,47sleep10-21swapper/110:14:411
2911028821,47sleep10-21swapper/110:14:411
308462855,11sleep00-21swapper/008:36:060
2769628517,56sleep327755-21latency_hist11:54:233
270362855,69sleep00-21swapper/010:07:290
188942857,35sleep00-21swapper/011:24:450
934928420,41sleep39414-21switchtime09:09:523
251428425,41sleep22027-21apt-get08:49:322
251428425,41sleep22027-21apt-get08:49:322
246252835,67sleep00-21swapper/008:19:100
241632830,8sleep30-21swapper/308:14:523
1520128313,54sleep215229-21munin-run11:14:212
323912826,67sleep20-21swapper/208:40:442
2339328218,45sleep223494-21timerandwakeup09:54:522
2339328218,45sleep223494-21timerandwakeup09:54:522
8482819,59sleep2849-21sshd10:27:582
1763128115,50sleep317712-21cat11:19:513
1741628114,46sleep117475-21/usr/sbin/munin07:54:441
879128024,25sleep20-21swapper/210:54:222
879128024,25sleep20-21swapper/210:54:222
829528024,27sleep10-21swapper/107:24:481
72312807,33sleep00-21swapper/009:04:360
3078228012,45sleep00-21swapper/010:19:410
68827810,52sleep1748-21latency12:09:551
68827810,52sleep1748-21latency12:09:551
2901327821,35sleep229009-21apt-get10:14:342
2901327821,35sleep229009-21apt-get10:14:342
2850327815,43sleep227909-21/usr/sbin/munin11:54:502
1923327822,33sleep30-21swapper/307:59:513
1870927813,42sleep118797-21fschecks_time11:24:421
307027716,46sleep13120-21latency10:34:431
307027716,46sleep13120-21latency10:34:431
1815127714,46sleep218152-21/usr/sbin/munin09:39:382
452727626,28sleep30-21swapper/307:14:283
1840527615,42sleep218428-21awk07:59:212
30122756,13sleep00-21swapper/010:34:420
257502755,13sleep20-21swapper/208:19:522
257502755,13sleep20-21swapper/208:19:522
248527521,36sleep22510-21wc07:09:222
2249927528,27sleep30-21swapper/309:54:203
1374127520,29sleep10-21swapper/111:09:231
354527414,45sleep13612-21df12:19:371
2856627426,28sleep328607-21ls08:29:473
115527418,29sleep30-21swapper/312:10:063
115527418,29sleep30-21swapper/312:10:063
71927315,41sleep1729-21find08:44:301
44127319,37sleep332688-21apt-get08:44:243
1838127319,28sleep20-21swapper/211:24:222
153327320,35sleep21680-21awk08:45:202
406527225,25sleep24140-21munin-node07:14:232
292927212,36sleep32776-21/usr/sbin/munin10:34:413
292927212,36sleep32776-21/usr/sbin/munin10:34:413
253902720,9sleep30-21swapper/311:44:493
2535427216,40sleep325391-21sed08:19:453
2535427216,40sleep325391-21sed08:19:453
1632427218,28sleep30-21swapper/309:34:223
2002927121,28sleep20-21swapper/208:04:222
1707727117,23sleep10-21swapper/109:34:481
762527029,22sleep20-21swapper/207:24:242
1919527013,35sleep318599-21/usr/sbin/munin11:24:523
2481226919,30sleep124831-21apt-get11:44:321
2439826910,46sleep124399-21cron08:16:201
2383426928,30sleep00-21swapper/009:56:350
2383426928,30sleep00-21swapper/009:56:350
2036626921,27sleep220387-21fschecks_count08:04:422
491026823,21sleep30-21swapper/310:39:503
3254426821,27sleep10-21swapper/110:24:461
1671826829,28sleep00-21swapper/011:18:510
1671826829,28sleep00-21swapper/011:18:510
1575926810,40sleep115828-21latency11:14:441
1575926810,40sleep115828-21latency11:14:441
1567726814,14sleep20-21swapper/207:49:442
1147026819,25sleep20-21swapper/207:34:542
3190626715,36sleep232055-21latency_hist10:24:232
3178626721,29sleep131819-21/usr/sbin/munin08:39:461
2151026721,28sleep10-21swapper/109:49:421
1318426719,32sleep313199-21modprobe09:24:213
853226619,22sleep30-21swapper/312:35:173
694126621,21sleep10-21swapper/109:04:211
2773726615,37sleep327780-21latency10:09:423
26532660,8sleep10-21swapper/110:34:221
26532660,8sleep10-21swapper/110:34:221
1511026624,26sleep20-21swapper/209:29:412
1412526621,26sleep10-21swapper/111:09:401
98282656,11sleep10-21swapper/109:11:271
888226518,31sleep38989-21readlink10:54:233
888226518,31sleep38989-21readlink10:54:233
699126520,25sleep30-21swapper/309:04:233
477826523,22sleep10-21swapper/107:14:381
293826516,31sleep32982-21ntpq08:49:583
293826516,31sleep32982-21ntpq08:49:583
2615026518,26sleep30-21swapper/310:04:423
2436826522,24sleep30-21swapper/309:59:373
1690926522,24sleep10-21swapper/107:54:231
1179126515,27sleep20-21swapper/211:00:202
250426415,33sleep22523-21modprobe10:34:202
250426415,33sleep22523-21modprobe10:34:202
2444426419,30sleep124450-21diskstats09:59:391
1402526414,23sleep30-21swapper/311:09:383
1251126424,23sleep10-21swapper/109:19:511
964926318,26sleep10-21swapper/110:54:581
67892638,43sleep10-21swapper/107:19:511
590226320,26sleep25921-21apt-get10:44:332
497426322,22sleep20-21swapper/207:14:432
446426315,34sleep14500-21/usr/sbin/munin10:39:401
283926312,19sleep10-21swapper/108:49:561
283926312,19sleep10-21swapper/108:49:561
2345726313,36sleep223477-21fschecks_count08:14:392
2338126310,24sleep10-21swapper/111:39:401
1462326323,21sleep10-21swapper/107:44:511
1462326323,21sleep10-21swapper/107:44:511
288726220,24sleep22928-21fschecks_count10:34:402
288726220,24sleep22928-21fschecks_count10:34:402
2332126220,23sleep10-21swapper/109:54:511
2332126220,23sleep10-21swapper/109:54:511
216526219,26sleep00-21swapper/007:05:580
1282926212,34sleep212852-21ntp_states11:04:482
1154626217,22sleep30-21swapper/307:34:553
696426115,31sleep37088-21sed07:19:533
60592616,43sleep20-21swapper/212:25:352
2800226117,21sleep20-21swapper/210:09:502
273126115,28sleep32750-21apt-get07:09:323
2488326112,35sleep224910-21sh11:44:372
1879526124,19sleep10-21swapper/109:39:531
1557626112,35sleep215671-21perl11:14:412
1557626112,35sleep215671-21perl11:14:412
100026116,27sleep30-21swapper/308:45:093
2776226013,29sleep127763-21cron08:29:201
2650926021,21sleep30-21swapper/308:24:383
2313226022,19sleep10-21swapper/108:14:221
2081826017,26sleep320946-21grep11:29:523
1575426015,23sleep30-21swapper/307:49:493
1394126015,28sleep113980-21wc09:24:511
873625919,23sleep10-21swapper/110:54:211
873625919,23sleep10-21swapper/110:54:211
485125917,21sleep20-21swapper/212:24:222
270125914,19sleep20-21swapper/212:14:532
2675625919,21sleep10-21swapper/111:49:451
2064825917,21sleep30-21swapper/308:04:473
179942599,23sleep30-21swapper/307:54:543
1381125918,18sleep20-21swapper/207:44:232
757025820,24sleep17599-21memory09:04:451
2308125822,20sleep30-21swapper/308:14:203
2022125816,22sleep10-21swapper/111:29:381
2017025818,24sleep220166-21apt-get11:29:352
1169425820,16sleep20-21swapper/209:19:232
2377625720,23sleep223862-21mailstats11:39:492
2180125720,17sleep20-21swapper/211:34:402
1034025713,26sleep310335-21apt-get09:14:363
517925618,16sleep10-21swapper/112:24:411
489125620,17sleep30-21swapper/312:24:233
3146825613,27sleep131517-21/usr/sbin/munin12:04:471
2592325617,22sleep225993-21/usr/sbin/munin10:04:372
1993325616,22sleep20-21swapper/211:29:232
1471225620,18sleep20-21swapper/207:44:522
1471225620,18sleep20-21swapper/207:44:522
1069225611,28sleep30-21swapper/307:34:233
664025513,25sleep36635-21apt-get12:29:363
409825514,26sleep14093-21apt-get08:54:361
287712559,12sleep10-21swapper/108:29:531
2500425513,20sleep20-21swapper/209:59:522
1852725520,21sleep318575-21ntpq09:39:473
154125514,24sleep31613-21memory10:29:453
154125514,24sleep31613-21memory10:29:453
1387525520,19sleep20-21swapper/209:24:482
1326225520,19sleep20-21swapper/209:24:242
1303925515,21sleep30-21swapper/307:39:523
1016225513,24sleep210283-21users07:29:542
824925413,24sleep38286-21ntp_states07:24:463
671025415,18sleep10-21swapper/112:29:381
2659325418,17sleep20-21swapper/208:24:412
1789425416,20sleep217908-21munin-run09:39:222
3133425318,17sleep20-21swapper/212:04:442
3050225315,21sleep130612-21timerwakeupswit08:34:531
2171725316,22sleep321784-21df11:34:373
1084125318,20sleep30-21swapper/310:59:363
405525214,23sleep14064-21munin-run07:14:221
2223125216,19sleep20-21swapper/208:09:472
2101325217,16sleep20-21swapper/209:49:232
981325111,24sleep29888-21sensors10:55:032
872925121,17sleep20-21swapper/207:24:532
777725114,16sleep30-21swapper/309:04:513
308272514,15sleep30-21swapper/308:35:523
2958925114,16sleep20-21swapper/208:34:222
1091725118,16sleep10-21swapper/110:59:381
900025017,18sleep29040-21if_eth009:09:432
620025014,22sleep16294-21memory10:44:451
509125012,21sleep35088-21apt-get12:24:353
456825017,16sleep30-21swapper/308:54:503
1391625014,21sleep313932-21ntp_states09:24:493
1168125017,18sleep10-21swapper/109:19:221
3189624813,19sleep10-21swapper/110:24:221
1547324613,17sleep10-21swapper/109:29:481
1002724613,13sleep10-21swapper/107:29:531
227732438,26sleep10-21swapper/111:36:451
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional