You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-27 - 18:52

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Mon Jul 27, 2026 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
13378280117,766sleep013472-21/usr/sbin/munin10:28:210
1902257323,528sleep01904-21cron09:52:570
29372245119,417sleep129436-21/usr/sbin/munin11:18:191
2664124499,424sleep226712-21ls11:08:272
41722351107,177sleep20-21swapper/208:18:002
2275023110,8sleep00-21swapper/010:57:590
2275023110,8sleep00-21swapper/010:57:590
154012302115,162sleep2225-21systemd-journal07:12:572
154012302115,162sleep2225-21systemd-journal07:12:572
44572295104,174sleep24523-21fschecks_time08:18:162
109512286111,158sleep210973-21munin-run12:02:572
7006227554,204sleep27143-21unixbench_multi11:48:322
7006227554,204sleep27143-21unixbench_multi11:48:322
27854227290,79sleep227195-21/usr/sbin/munin07:49:022
27854227290,79sleep227195-21/usr/sbin/munin07:49:022
17755227286,171sleep217937-21latency_hist10:43:002
14436226485,24sleep20-21swapper/212:13:182
17244225982,70sleep217365-21latency_hist12:22:592
9691225295,60sleep29718-21munin-run10:17:572
30846224441,26sleep20-21swapper/211:23:172
20736222426,36sleep20-21swapper/209:08:182
10612222116,186sleep210690-21timerandwakeup10:18:262
506022099,144sleep25077-21kernelversion11:43:392
18350220616,98sleep218389-21latency10:43:242
2165122000,28sleep221656-21unixbench_singl07:28:302
12607220020,160sleep212587-21latency_hist08:42:592
12607220020,160sleep212587-21latency_hist08:42:592
6522219611,170sleep16596-21find08:23:281
1669521929,167sleep216786-21sed07:13:292
9447218513,153sleep09494-21latency_hist11:57:580
24192218515,146sleep20-21swapper/209:18:232
52621838,94sleep2644-21sed11:28:292
26714217715,146sleep326807-21sort11:08:283
32548217136,55sleep232594-21ntp_offset08:03:232
1221021718,147sleep212332-21wc10:23:302
5413216514,132sleep35469-21tune2fs10:03:173
21189216285,26sleep20-21swapper/212:33:282
3022021608,140sleep2364-21snmpd07:57:172
352321577,53sleep20-21swapper/209:57:592
25236215310,50sleep20-21swapper/207:43:002
8179215223,107sleep20-21swapper/208:28:302
12452215130,110sleep2364-21snmpd12:07:212
5894214912,45sleep20-21swapper/208:23:092
2898521489,121sleep229099-21latency_hist11:18:002
6929214714,116sleep17003-21find11:48:311
6929214714,116sleep17003-21find11:48:311
202402145119,14sleep212-21rcu_sched10:48:292
27378214476,17sleep20-21swapper/209:28:282
25542214212,105sleep20-21swapper/209:23:202
25819214130,99sleep225867-21ntpq07:43:252
25819214130,99sleep225867-21ntpq07:43:252
25081214124,98sleep20-21swapper/211:03:272
25081214124,98sleep20-21swapper/211:03:272
1347321418,51sleep213596-21cat08:43:322
1347321418,51sleep213596-21cat08:43:322
602421375,120sleep014782-1kworker/0:2H11:47:490
25890213717,99sleep325964-21switchtime09:23:293
26096213522,95sleep126203-21grep09:23:311
123821347,15sleep20-21swapper/208:08:112
31894213314,50sleep20-21swapper/208:02:582
19905213214,96sleep020051-21unixbench_multi07:23:290
1917421309,45sleep219197-21cut09:03:182
9464212922,83sleep00-21swapper/008:33:080
30984212819,11sleep20-21swapper/209:39:222
2353621288,45sleep223546-21munin-run09:17:592
14495212754,25sleep20-21swapper/208:48:162
27218212617,83sleep30-21swapper/307:48:103
27218212617,83sleep30-21swapper/307:48:103
19492212615,97sleep319534-21latency07:23:193
10602212618,11sleep20-21swapper/208:36:012
28552212418,33sleep00-21swapper/007:52:590
28552212418,33sleep00-21swapper/007:52:590
8079212321,82sleep17602-21apt-get11:53:071
1327212321,89sleep11419-21tune2fs08:08:181
20141211917,83sleep020183-21sort10:48:270
11708211913,87sleep311217-21/usr/sbin/munin12:03:263
5122211886,14sleep20-21swapper/210:03:002
1885021149,90sleep218991-21modprobe12:27:592
1548621139,17sleep20-21swapper/210:33:322
1809321119,84sleep20-21swapper/212:23:272
16102211126,58sleep116179-21fschecks_time08:53:361
391211012,85sleep2411-21latency_hist09:47:582
1804221109,16sleep20-21swapper/207:18:222
15467211019,65sleep115424-21latency_hist07:13:001
15467211019,65sleep115424-21latency_hist07:13:001
30746210923,61sleep30-21swapper/311:23:103
16055210921,50sleep20-21swapper/208:53:332
23503210819,11sleep20-21swapper/207:37:032
2189210831,49sleep12186-21apt-get09:53:101
808021078,15sleep20-21swapper/211:53:082
30452210618,13sleep20-21swapper/211:22:362
19222210621,32sleep20-21swapper/207:23:102
1805821068,37sleep20-21swapper/208:58:292
10914210625,62sleep311017-21cat08:37:593
6399210510,80sleep36406-21df_inode11:48:163
6399210510,80sleep36406-21df_inode11:48:163
4742210517,72sleep34260-21apt-get11:43:083
3353210533,45sleep20-21swapper/211:38:222
3353210533,45sleep20-21swapper/211:38:222
29491210520,11sleep20-21swapper/209:35:272
28400210513,78sleep128408-21acpi09:33:081
1821210518,12sleep20-21swapper/209:51:542
17941210516,72sleep317956-21/usr/sbin/munin12:23:243
2357210418,12sleep20-21swapper/211:33:542
22460210418,11sleep20-21swapper/210:55:152
22460210418,11sleep20-21swapper/210:55:152
6951210318,48sleep20-21swapper/210:08:172
14071210218,10sleep20-21swapper/210:29:032
11826210113,73sleep111840-21uname10:23:211
3274829917,62sleep3371-21timerandwakeup08:03:283
2018429914,61sleep10-21swapper/110:48:281
1515929929,50sleep315243-21/usr/sbin/munin10:33:283
131502996,14sleep20-21swapper/212:08:222
31962986,83sleep00-21swapper/009:54:380
2409329815,11sleep20-21swapper/211:00:522
2409329815,11sleep20-21swapper/211:00:522
1036429820,52sleep110505-21sed11:58:281
896229730,48sleep18973-21modprobe08:32:571
763129714,68sleep07714-21rm11:52:580
763129714,68sleep07714-21rm11:52:580
253262979,17sleep00-21swapper/009:23:100
1601929714,69sleep016039-21fschecks_count12:18:170
2515329617,60sleep325252-21timerandwakeup11:03:283
2515329617,60sleep325252-21timerandwakeup11:03:283
2590229517,63sleep125417-21/usr/sbin/munin07:43:281
2590229517,63sleep125417-21/usr/sbin/munin07:43:281
57742949,14sleep20-21swapper/210:03:242
1127829320,52sleep310771-21apt-get08:38:083
1127829320,52sleep310771-21apt-get08:38:083
1898729220,58sleep019036-21latency_hist07:23:000
444729113,63sleep04578-21unixbench_multi09:58:300
117329111,65sleep31317-21latency_hist11:32:583
547329013,59sleep05301-21/usr/sbin/munin10:03:180
2515229011,65sleep025196-21systemd-run07:42:580
2154528913,56sleep321619-21/usr/sbin/munin10:53:183
2154528913,56sleep321619-21/usr/sbin/munin10:53:183
2975828817,54sleep329865-21expr07:54:443
3039028719,51sleep130541-21latency_hist07:58:011
2367128624,39sleep023832-21grep10:58:280
2367128624,39sleep023832-21grep10:58:280
1320128420,44sleep113294-21fschecks_time10:28:181
265028313,54sleep02651-21sshd11:37:440
1957028318,30sleep00-21swapper/009:03:280
128352830,5sleep30-21swapper/312:08:143
274728212,56sleep32868-21latency_hist11:37:593
1619428220,36sleep316207-21uname12:18:203
3208228114,18sleep00-21swapper/011:27:590
157332819,17sleep30-21swapper/307:13:023
1646028011,51sleep016499-21ntpq07:13:230
1306028012,49sleep113149-21load12:08:211
2213227926,30sleep30-21swapper/307:33:003
1559227910,57sleep1364-21snmpd12:17:411
2527427713,49sleep025417-21unixbench_singl11:03:290
2527427713,49sleep025417-21unixbench_singl11:03:290
3261227621,36sleep10-21swapper/108:03:261
280027616,47sleep32897-21sed09:53:283
2491127618,38sleep124956-21awk11:03:221
2491127618,38sleep124956-21awk11:03:221
2430527625,26sleep30-21swapper/311:02:593
2430527625,26sleep30-21swapper/311:02:593
2151027620,18sleep10-21swapper/107:28:301
1767627614,18sleep10-21swapper/107:18:131
1718427617,29sleep10-21swapper/110:38:311
1276627617,15sleep00-21swapper/007:03:190
735327518,26sleep10-21swapper/110:08:281
38382759,56sleep0292-21systemd-logind08:15:120
209227518,40sleep12250-21sed11:33:291
1951027516,39sleep019572-21/usr/sbin/munin12:28:260
1332727511,30sleep30-21swapper/308:43:293
1332727511,30sleep30-21swapper/308:43:293
1256527517,46sleep012598-21basename08:42:570
1256527517,46sleep012598-21basename08:42:570
916627417,40sleep09316-21latency_hist08:33:000
2893127416,33sleep328401-21/usr/sbin/munin09:33:263
249427416,39sleep02529-21munin-run08:12:580
1571927426,27sleep10-21swapper/108:52:591
342627215,39sleep13552-21sed08:13:271
291012720,9sleep30-21swapper/311:18:003
1933327226,28sleep319386-21apt-config10:47:583
1537327220,26sleep315382-21modprobe12:14:573
1008327219,29sleep30-21swapper/311:58:223
778627117,28sleep30-21swapper/308:28:193
2033927124,23sleep10-21swapper/112:32:591
1969127114,38sleep119751-21/usr/sbin/munin07:23:251
1345227124,18sleep30-21swapper/307:03:303
2190727020,19sleep10-21swapper/109:12:571
730526915,36sleep07464-21latency_hist08:27:590
276382697,19sleep00-21swapper/011:13:140
2240926912,41sleep122476-21fschecks_time07:33:181
2183626919,30sleep31-21systemd12:37:573
290782678,19sleep00-21swapper/007:54:000
254926717,26sleep30-21swapper/308:13:013
1151026722,22sleep10-21swapper/112:03:201
974426613,39sleep39769-21munin-run10:17:583
3252726615,28sleep00-21swapper/011:28:190
3079826610,39sleep330907-21users09:38:313
2949426616,25sleep30-21swapper/311:18:213
2673726619,32sleep026747-21apt-key07:47:590
2673726619,32sleep026747-21apt-key07:47:590
1800426623,23sleep10-21swapper/108:58:281
1703126620,30sleep017183-21sed10:38:300
1593426616,20sleep10-21swapper/112:18:141
3190926523,18sleep30-21swapper/308:03:003
3185826518,32sleep131890-21munin-run08:02:571
3131426520,26sleep00-21swapper/009:43:000
2818726517,23sleep00-21swapper/009:32:580
2322526511,40sleep323226-21cron09:14:573
1013126511,41sleep010141-21ntp_states11:58:230
530126419,29sleep10-21swapper/111:43:461
2734426416,32sleep127377-21/usr/sbin/munin09:28:271
2192126417,21sleep00-21swapper/012:37:580
464026311,21sleep10-21swapper/108:18:191
39532635,19sleep30-21swapper/311:39:123
39532635,19sleep30-21swapper/311:39:123
1484826317,25sleep00-21swapper/012:13:280
1037726317,18sleep00-21swapper/010:18:200
850826214,21sleep10-21swapper/110:13:161
484626220,25sleep30-21swapper/308:18:263
2104126215,21sleep00-21swapper/007:28:210
1316526222,19sleep10-21swapper/107:03:281
114072626,14sleep00-21swapper/010:23:000
818326017,20sleep00-21swapper/010:12:590
1802426018,25sleep118092-21swap12:23:261
1524626014,27sleep015350-21sed10:33:290
977125914,22sleep00-21swapper/010:17:590
2236725917,25sleep10-21swapper/109:13:181
2196025912,21sleep30-21swapper/309:12:593
1754525918,20sleep30-21swapper/310:42:573
155752594,47sleep00-21swapper/008:51:450
1310325913,28sleep313099-21apt-get10:28:123
1255225915,23sleep00-21swapper/012:07:590
2376025815,27sleep023779-21apt-get09:18:090
2159125812,33sleep0364-21snmpd09:08:530
1451125813,18sleep30-21swapper/307:08:223
1451125813,18sleep30-21swapper/307:08:223
1069225816,23sleep310851-21grep10:18:273
424625716,23sleep10-21swapper/111:42:571
424625716,23sleep10-21swapper/111:42:571
3162625713,18sleep00-21swapper/009:43:160
2866125712,21sleep10-21swapper/107:53:001
2772525710,19sleep10-21swapper/111:13:191
1948225716,18sleep10-21swapper/110:48:001
1854425710,34sleep018584-21ps10:43:310
1603925719,23sleep10-21swapper/110:37:581
850725615,19sleep30-21swapper/311:53:253
421925614,25sleep34224-21ntp_states09:58:253
2990625612,19sleep00-21swapper/011:18:300
2976625611,20sleep00-21swapper/009:37:590
232002567,37sleep00-21swapper/009:14:380
2266825615,18sleep10-21swapper/110:57:571
2266825615,18sleep10-21swapper/110:57:571
1979425616,20sleep30-21swapper/309:03:313
64025517,22sleep3713-21df09:48:113
414025514,21sleep10-21swapper/109:58:231
1930525514,24sleep119358-21irqstats09:03:211
1467625514,20sleep00-21swapper/007:08:270
1467625514,20sleep00-21swapper/007:08:270
72525415,25sleep1728-21/usr/sbin/munin09:48:131
3260525413,18sleep00-21swapper/008:03:250
3155325413,24sleep131548-21apt-get09:43:131
3114125419,21sleep10-21swapper/111:23:241
2420125414,23sleep324259-21memory07:38:213
2028725411,18sleep30-21swapper/312:32:573
1652925416,17sleep30-21swapper/308:53:463
1233325414,21sleep30-21swapper/310:23:313
1007825414,24sleep110085-21df_inode10:18:131
695825315,22sleep36962-21diskstats10:08:183
478625313,23sleep04823-21ntp_states08:18:230
3045125314,24sleep130459-21ntp_states09:38:241
284725311,17sleep30-21swapper/308:13:133
2680925310,17sleep00-21swapper/011:08:290
2519725318,17sleep10-21swapper/107:42:591
2203225312,26sleep122242-21cat12:37:591
1780225316,18sleep30-21swapper/308:58:223
1659825319,22sleep016696-21wc08:53:470
1301825317,22sleep212536-21/usr/sbin/munin07:03:252
1075225316,23sleep110779-21munin-run08:37:571
43225216,18sleep10-21swapper/111:28:281
2767425216,19sleep10-21swapper/107:48:571
2231825214,22sleep322313-21apt-get07:33:123
1521425214,21sleep315312-21users08:48:293
1409025214,16sleep10-21swapper/108:47:341
1409025214,16sleep10-21swapper/108:47:341
1293025213,22sleep012939-21df_inode08:43:170
1293025213,22sleep012939-21df_inode08:43:170
2396825111,21sleep10-21swapper/109:18:181
3169925012,20sleep331762-21fw_packets09:43:193
2745925011,26sleep327556-21switchtime09:28:293
2629625016,18sleep10-21swapper/111:08:181
255152508,16sleep30-21swapper/307:43:173
255152508,16sleep30-21swapper/307:43:173
2423825011,22sleep324244-21ntp_states09:18:243
1128225015,15sleep10-21swapper/108:38:101
1128225015,15sleep10-21swapper/108:38:101
2383624911,20sleep123889-21users10:58:291
2383624911,20sleep123889-21users10:58:291
2276824911,24sleep022779-21ntpq07:33:260
1286424912,21sleep012872-21diskstats12:08:160
60712489,23sleep36127-21fw_packets08:23:183
343824810,16sleep00-21swapper/011:38:240
343824810,16sleep00-21swapper/011:38:240
1983824812,20sleep319857-21apt-get10:48:103
158452476,14sleep10-21swapper/107:13:041
572824612,18sleep10-21swapper/110:03:231
1767324613,16sleep30-21swapper/307:18:113
174742467,14sleep00-21swapper/012:23:090
1821224310,15sleep00-21swapper/008:58:310
310232428,20sleep031046-21ntp_states07:58:240
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional