You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-19 - 22:22

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Sat Sep 19, 2026 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
19813284228,794sleep319873-21/usr/sbin/munin07:03:103
20727280314,770sleep120760-21irqstats08:48:031
9923278916,754sleep210048-21/usr/sbin/munin11:38:132
25641268217,648sleep225633-21apt-get10:47:582
15003255115,518sleep215019-21/usr/sbin/munin10:13:042
2874224670,12sleep320700-21cobalt_printf10:57:513
31166235216,317sleep00-21swapper/009:22:430
2686023387,15sleep00-21swapper/012:33:100
23558229083,26sleep00-21swapper/007:13:120
23558229083,26sleep00-21swapper/007:13:120
13568228078,148sleep013691-21dpkg08:27:420
13568228078,148sleep013691-21dpkg08:27:420
7256225616,224sleep27303-21/usr/sbin/munin08:07:412
27739225275,75sleep00-21swapper/007:28:020
17576224918,218sleep117623-21systemd-run10:22:411
32597223996,28sleep00-21swapper/011:08:150
265632233104,23sleep00-21swapper/007:23:130
2784222611,128sleep00-21swapper/011:17:440
30442222212,152sleep030550-21latency_hist07:37:430
26423222113,190sleep026470-21modprobe09:07:400
1270222209,140sleep012788-21rm10:07:410
10789221622,161sleep00-21swapper/011:42:550
17352221286,31sleep016844-21apt-get08:37:520
18385221147,107sleep013726-1kworker/0:2H12:07:420
20851220212,125sleep020998-21unixbench_multi12:13:110
3811217911,91sleep03869-21if_err_eth009:38:010
1992521769,104sleep00-21swapper/012:12:430
1992521769,104sleep00-21swapper/012:12:430
22778217510,127sleep022922-21awk08:53:150
7760217477,80sleep00-21swapper/011:33:020
2745721697,119sleep027506-21latency10:53:040
14486216870,17sleep00-21swapper/010:12:420
14486216870,17sleep00-21swapper/010:12:420
498921665,86sleep00-21swapper/011:23:090
7631215968,79sleep07636-21diskstats08:08:000
5761215430,22sleep00-21swapper/008:02:430
7404215210,124sleep07541-21unixbench_multi09:48:110
1197821527,128sleep00-21swapper/008:22:420
32992149108,23sleep00-21swapper/011:18:040
19414214825,103sleep00-21swapper/010:27:570
5766214333,97sleep05856-21awk09:43:100
2987321437,122sleep00-21swapper/011:00:400
16127214311,115sleep016253-21timerwakeupswit11:58:130
32002214218,17sleep00-21swapper/009:23:110
2666021417,123sleep0367-21snmpd10:49:370
20945214145,15sleep00-21swapper/010:32:530
8225213515,105sleep28348-21cat08:08:142
24285213515,11sleep00-21swapper/012:24:050
205121358,15sleep00-21swapper/009:32:560
31574213333,83sleep131582-21modprobe11:07:391
1940212927,23sleep02049-21timerandwakeup07:48:130
175221265,13sleep00-21swapper/009:32:320
7345212525,16sleep00-21swapper/008:07:430
13973212512,43sleep113998-21grep11:53:011
31756212412,93sleep031949-21apt-config11:07:420
2118721249,97sleep00-21swapper/007:08:030
2118721249,97sleep00-21swapper/007:08:030
24543212220,21sleep00-21swapper/010:43:070
32490212169,17sleep032509-21uname07:43:040
458521190,8sleep30-21swapper/311:22:593
20974211966,18sleep021040-21awk08:48:100
15569211714,89sleep115570-21cpu08:32:561
1884521165,99sleep00-21swapper/010:24:050
2277221135,37sleep00-21swapper/012:19:330
32714211222,74sleep3356-21latency_hist09:27:423
3165321125,73sleep00-21swapper/007:39:000
3165321125,73sleep00-21swapper/007:39:000
10359211221,72sleep310378-21cron08:17:403
2860211117,77sleep12983-21latency_hist11:17:451
2824821115,11sleep00-21swapper/009:12:530
1510221106,93sleep00-21swapper/008:30:250
3534210913,79sleep33650-21timerwakeupswit07:53:133
3446210917,76sleep23533-21switchtime07:53:122
29563210916,78sleep129570-21/usr/sbin/munin07:33:081
2995210714,75sleep32514-21apt-get11:17:513
23004210710,57sleep00-21swapper/010:38:100
17012210731,37sleep30-21swapper/312:02:533
15832210717,67sleep115439-21/usr/sbin/munin11:58:061
983821066,85sleep00-21swapper/008:13:130
1621121068,15sleep00-21swapper/008:33:120
1269921068,85sleep012710-21needreboot11:48:040
20569210521,64sleep320663-21fschecks_time08:48:013
1510921057,41sleep00-21swapper/011:57:120
24947210412,77sleep125014-21systemctl07:18:101
24947210412,77sleep125014-21systemctl07:18:101
24823210426,62sleep20-21swapper/212:27:512
2393121047,14sleep00-21swapper/008:58:030
2026210417,56sleep12045-21apt-get09:32:531
931821038,55sleep00-21swapper/011:37:590
24225210218,27sleep30-21swapper/307:17:433
24225210218,27sleep30-21swapper/307:17:433
21143210213,75sleep121185-21sort10:33:031
6528210019,65sleep36573-21cat11:28:083
1954221009,79sleep019648-21timerandwakeup08:43:130
3095229913,70sleep131030-21sed07:38:041
3095229913,70sleep131030-21sed07:38:041
1965029918,61sleep119794-21sed08:43:141
99192986,13sleep00-21swapper/009:58:000
99192986,13sleep00-21swapper/009:58:000
2687829811,73sleep126944-21idleruntime09:08:021
360129721,26sleep30-21swapper/309:37:523
164022975,11sleep00-21swapper/010:17:590
2994529618,56sleep329961-21fschecks_count09:18:013
1091329615,64sleep110998-21fw_packets11:43:001
885529511,69sleep38996-21latency_hist08:12:433
248302958,12sleep00-21swapper/007:18:050
248302958,12sleep00-21swapper/007:18:050
71682935,10sleep00-21swapper/011:29:550
304162939,59sleep30-21swapper/311:03:013
258842935,12sleep00-21swapper/012:29:470
2381029315,63sleep323886-21systemctl12:23:123
148552935,11sleep00-21swapper/010:13:020
1122829315,55sleep310660-21/usr/sbin/munin08:18:123
2619429216,56sleep325982-21/usr/sbin/munin07:23:043
276822915,11sleep00-21swapper/009:08:450
1794629118,52sleep317839-21/usr/sbin/munin10:22:593
1137129115,63sleep211379-21acpi10:02:512
2953929021,51sleep329597-21sendmail09:17:413
91902895,10sleep00-21swapper/009:53:200
39862895,10sleep00-21swapper/007:55:520
2338328916,52sleep223465-21mailstats07:13:102
2338328916,52sleep223465-21mailstats07:13:102
2071228814,58sleep120624-21latency_hist07:07:441
1882628815,52sleep118882-21/usr/sbin/munin12:08:031
1882628815,52sleep118882-21/usr/sbin/munin12:08:031
1766728816,52sleep317596-21latency_hist10:22:433
1348428618,53sleep213507-21grep08:27:402
1348428618,53sleep213507-21grep08:27:402
95228523,15sleep20-21swapper/211:12:412
2492528510,17sleep10-21swapper/112:27:581
1357628516,52sleep313619-21ntp_kernel_pll_10:08:133
1357628516,52sleep313619-21ntp_kernel_pll_10:08:133
1987428419,48sleep329249-1kworker/3:1H10:28:083
1690128421,31sleep20-21swapper/208:37:422
649628319,48sleep26527-21/usr/sbin/munin11:28:072
2574228317,51sleep325814-21wc09:03:113
2625128217,29sleep10-21swapper/110:48:131
1473528222,42sleep214875-21sed08:28:372
1366028216,46sleep213159-21/usr/sbin/munin10:08:162
1366028216,46sleep213159-21/usr/sbin/munin10:08:162
826828120,34sleep28262-21apt-get09:52:572
525728119,33sleep10-21swapper/109:42:591
436028126,25sleep20-21swapper/211:22:422
2222528015,37sleep222264-21munin-run10:37:402
2210428014,48sleep122117-21df_inode08:52:591
168082800,6sleep20-21swapper/210:18:082
1399928021,31sleep20-21swapper/211:53:022
1230927919,47sleep312312-21acpi11:47:513
1212927912,52sleep30-21swapper/311:47:433
732127820,40sleep37400-21find09:48:103
2491627814,49sleep324922-21ntp_states07:18:083
2491627814,49sleep324922-21ntp_states07:18:083
1049927818,34sleep10-21swapper/111:42:401
2042827618,26sleep30-21swapper/312:13:023
1968327623,29sleep20-21swapper/210:28:032
1289427516,41sleep212977-21switchtime11:48:102
1738327423,30sleep217407-21/usr/sbin/munin12:03:042
1600027418,34sleep116025-21munin-run10:17:401
340227321,27sleep20-21swapper/209:37:432
2682827318,36sleep226851-21ntp_states12:33:082
99127215,28sleep30-21swapper/309:28:113
570927223,30sleep25727-21modprobe08:02:412
265327213,44sleep32781-21sed11:17:433
2401627213,43sleep224061-21/usr/sbin/munin08:58:052
1363527210,50sleep113636-21/usr/sbin/munin11:52:411
1065927220,34sleep110678-21apt-get08:17:541
389927127,26sleep20-21swapper/209:38:032
289392710,7sleep20-21swapper/207:32:432
2516627122,26sleep10-21swapper/109:02:571
1766027125,25sleep30-21swapper/308:38:073
1413827121,36sleep114159-21fschecks_count08:28:241
47427020,29sleep20-21swapper/209:27:532
3231427013,39sleep132347-21fschecks_count07:43:011
1974227014,39sleep019792-21/usr/sbin/munin07:03:070
2398126914,16sleep10-21swapper/112:23:141
961526814,30sleep30-21swapper/311:38:063
797926815,37sleep2289-21dbus-daemon09:52:412
640126817,24sleep30-21swapper/308:03:063
2184226813,40sleep121870-21cut12:18:011
213902688,41sleep20-21swapper/207:08:092
213902688,41sleep20-21swapper/207:08:092
2073126815,27sleep10-21swapper/110:32:421
1514326822,30sleep315207-21modprobe11:57:403
1223126814,30sleep30-21swapper/308:22:573
1223126814,30sleep30-21swapper/308:22:573
458126711,29sleep20-21swapper/211:22:592
337026717,18sleep10-21swapper/107:53:111
3247426712,38sleep332082-21/usr/sbin/munin11:08:103
2727426717,28sleep10-21swapper/110:53:011
2143026710,13sleep30-21swapper/307:08:103
2143026710,13sleep30-21swapper/307:08:103
1930326720,34sleep218907-21/usr/sbin/munin08:43:072
1681826721,29sleep20-21swapper/212:02:432
1116426723,26sleep211226-21grep08:18:112
108426730,24sleep21056-21latency_hist07:47:422
94122667,19sleep30-21swapper/309:56:133
2462326620,28sleep324693-21/usr/sbin/munin10:43:103
2416926626,25sleep30-21swapper/308:58:103
1907026625,19sleep30-21swapper/312:08:103
1907026625,19sleep30-21swapper/312:08:103
1612226623,19sleep30-21swapper/308:33:113
453026516,35sleep14566-21cut07:57:591
3111526513,32sleep20-21swapper/209:22:412
1062526519,22sleep30-21swapper/309:58:153
1062526519,22sleep30-21swapper/309:58:153
1045826519,22sleep20-21swapper/208:17:432
827926420,23sleep30-21swapper/311:33:133
746526418,21sleep10-21swapper/111:32:431
1608026421,23sleep30-21swapper/310:17:423
1197526420,31sleep112063-21sed10:03:101
1129426412,37sleep211388-21grep11:43:092
2874126319,27sleep20-21swapper/209:13:122
180426314,30sleep20-21swapper/207:48:112
3272726217,32sleep3343-21awk07:43:113
2763126215,23sleep10-21swapper/107:27:591
2676126219,22sleep20-21swapper/207:23:152
2648326219,23sleep30-21swapper/312:33:013
2384826218,24sleep30-21swapper/310:42:423
1042526212,28sleep10-21swapper/109:58:131
1042526212,28sleep10-21swapper/109:58:131
6662616,11sleep10-21swapper/111:09:291
3138626121,26sleep231531-21unixbench_singl07:38:142
3138626121,26sleep231531-21unixbench_singl07:38:142
2663426116,25sleep10-21swapper/107:23:141
2659626121,22sleep10-21swapper/112:33:031
2648826115,33sleep226546-21cat09:07:422
2175726113,21sleep20-21swapper/212:17:582
1942426122,24sleep219449-21fschecks_count07:03:012
1383726123,22sleep113943-21wc10:08:191
1383726123,22sleep113943-21wc10:08:191
99912605,12sleep10-21swapper/108:13:141
2836426019,18sleep10-21swapper/109:13:021
1888326020,17sleep20-21swapper/212:08:052
1888326020,17sleep20-21swapper/212:08:052
1767426014,31sleep117797-21cat12:03:111
493625920,19sleep10-21swapper/109:42:421
3177625915,23sleep20-21swapper/209:23:052
2654825919,19sleep30-21swapper/309:07:433
2212525917,25sleep30-21swapper/308:53:013
2028325916,22sleep10-21swapper/112:13:001
1919625918,21sleep20-21swapper/210:27:422
993325816,22sleep20-21swapper/209:58:022
993325816,22sleep20-21swapper/209:58:022
2370125811,33sleep1367-21snmpd10:41:221
871925716,19sleep10-21swapper/109:53:081
820825720,21sleep28278-21/usr/sbin/munin11:33:122
3084925715,22sleep30-21swapper/307:38:023
3084925715,22sleep30-21swapper/307:38:023
2487625713,27sleep124910-21munin-run09:02:411
784325620,20sleep10-21swapper/111:33:031
608625620,16sleep10-21swapper/108:03:001
3169025614,23sleep131775-21memory09:23:041
2801725621,17sleep30-21swapper/307:28:093
2455725615,19sleep20-21swapper/210:43:092
2138025617,25sleep121389-21ntpq07:08:081
2138025617,25sleep121389-21ntpq07:08:081
1804725619,17sleep10-21swapper/110:23:011
1219625614,23sleep312301-21users10:03:123
557225519,18sleep30-21swapper/309:43:053
30922555,10sleep30-21swapper/309:34:533
2931425513,23sleep329371-21irqstats07:33:023
255825511,20sleep20-21swapper/211:17:422
75242549,21sleep10-21swapper/108:07:531
2805325414,23sleep20-21swapper/209:12:432
2340325418,17sleep20-21swapper/212:23:022
2018625415,24sleep320221-21munin-run08:47:403
1603725417,21sleep30-21swapper/311:58:123
1404925414,24sleep314005-21apt-get08:28:183
432625316,18sleep10-21swapper/111:22:411
2417625314,16sleep10-21swapper/110:43:001
2226225317,19sleep30-21swapper/312:18:113
2037025318,20sleep20-21swapper/212:13:012
643325212,20sleep20-21swapper/209:47:402
485125216,14sleep10-21swapper/111:23:061
2772125218,19sleep30-21swapper/310:53:123
256722526,35sleep20-21swapper/207:22:062
256722526,35sleep20-21swapper/207:22:062
173225213,20sleep31780-21ntp_states07:48:083
451725112,23sleep34521-21df_inode07:57:573
2909425112,22sleep229096-21apt-get07:32:522
1369625117,16sleep10-21swapper/108:27:441
1369625117,16sleep10-21swapper/108:27:441
3158525014,22sleep231643-21apt-config11:07:402
2294825011,22sleep222990-21ntp_offset10:38:082
215132505,16sleep20-21swapper/208:49:412
205225012,15sleep10-21swapper/107:48:141
2485024912,20sleep324917-21df12:27:563
1851124911,24sleep2367-21snmpd08:41:012
1738524913,20sleep117378-21apt-get08:37:571
78802486,34sleep10-21swapper/109:51:231
300792489,17sleep10-21swapper/111:02:431
2244824811,21sleep122480-21ntp_states07:12:441
2244824811,21sleep122480-21ntp_states07:12:441
2142424812,19sleep321433-21ntp_states10:33:093
2718824711,20sleep227255-21df10:52:582
96424610,13sleep10-21swapper/109:28:091
2876224510,18sleep20-21swapper/210:57:552
2601024511,17sleep30-21swapper/310:48:073
221182449,19sleep222124-21diskstats08:53:002
303152399,14sleep10-21swapper/109:18:151
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional