You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-14 - 15:30

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7s.osadl.org (updated Sat Mar 14, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
307962617815,6150sleep122639-21kworker/1:210:55:301
2236327939,770sleep018671-21kworker/u8:008:48:580
4190272316,689sleep14317-21/usr/sbin/munin11:13:391
4190272316,689sleep14317-21/usr/sbin/munin11:13:391
1523524937,472sleep215255-21apt-get10:08:182
20723238417,318sleep220737-21fschecks_count12:08:272
17906234311,316sleep317945-21ntpq11:58:353
17906234311,316sleep317945-21ntpq11:58:353
11307230310,280sleep211329-21fschecks_count08:13:342
11307230310,280sleep211329-21fschecks_count08:13:342
2237422669,241sleep222435-21if_eth012:13:282
28618224631,131sleep20-21swapper/212:33:312
315242240122,59sleep20-21swapper/207:33:412
315242240122,59sleep20-21swapper/207:33:412
7017222114,150sleep27064-21ntp_kernel_pll_11:23:332
7017222114,150sleep27064-21ntp_kernel_pll_11:23:332
10426220713,172sleep210065-21apt-get09:53:202
10426220713,172sleep210065-21apt-get09:53:202
2851220431,115sleep20-21swapper/209:28:372
4163220310,181sleep013441-21snmpd07:49:500
629219516,60sleep2628-21apt-key09:23:122
29159219545,101sleep229211-21ls07:28:122
437219417,164sleep0474-21sed07:38:350
27477218110,158sleep227517-21uname09:03:422
1298121769,17sleep20-21swapper/208:18:312
520217416,56sleep20-21swapper/207:38:372
28566217211,144sleep028644-21find10:48:470
28566217211,144sleep028644-21find10:48:470
24621216930,120sleep20-21swapper/210:38:232
14194216854,98sleep214255-21find10:03:382
14194216854,98sleep214255-21find10:03:382
18159216389,52sleep20-21swapper/210:18:122
19500216298,43sleep20-21swapper/212:03:352
19500216298,43sleep20-21swapper/212:03:352
24493215748,30sleep20-21swapper/207:13:182
604821538,67sleep26146-21date07:58:122
3279215018,106sleep20-21swapper/211:13:112
30466214757,30sleep20-21swapper/210:54:092
13344214683,44sleep20-21swapper/211:43:362
12248214315,20sleep20-21swapper/209:58:292
12248214315,20sleep20-21swapper/209:58:292
31936214218,25sleep10-21swapper/110:58:451
31936214218,25sleep10-21swapper/110:58:451
2874213947,48sleep20-21swapper/207:48:102
3976213712,109sleep24015-21proc_pri11:13:362
3976213712,109sleep24015-21proc_pri11:13:362
26791213743,79sleep20-21swapper/207:18:392
19164213710,100sleep20-21swapper/210:18:402
1159213616,96sleep20-21swapper/209:24:092
5823213548,72sleep25866-21fschecks_time09:38:312
24182213517,102sleep024224-21awk12:18:350
20640213521,26sleep30-21swapper/312:08:243
17078213113,99sleep116780-21/usr/sbin/munin10:13:301
17622213015,27sleep20-21swapper/211:58:302
17622213015,27sleep20-21swapper/211:58:302
1689021266,37sleep20-21swapper/211:53:582
31296212541,26sleep20-21swapper/210:58:212
31296212541,26sleep20-21swapper/210:58:212
20616212529,59sleep220738-21cat10:23:412
10170212540,13sleep20-21swapper/211:33:352
2227821237,13sleep20-21swapper/208:48:152
18744212214,81sleep018569-1kworker/0:2H08:38:120
14180212112,96sleep114193-21open_files10:03:371
14180212112,96sleep114193-21open_files10:03:371
30796211914,89sleep130930-21unixbench_multi09:13:421
16133211941,52sleep20-21swapper/208:28:332
10119211919,81sleep10-21swapper/111:33:341
2663121187,94sleep20-21swapper/210:43:402
2663121187,94sleep20-21swapper/210:43:402
23133211716,43sleep223158-21awk07:08:332
1897821178,71sleep20-21swapper/208:38:222
2163211514,82sleep32231-21/usr/sbin/munin07:43:393
19727211522,71sleep119879-21sed08:38:411
372721145,97sleep00-21swapper/009:31:460
26139211420,15sleep20-21swapper/212:23:422
2792721138,18sleep20-21swapper/210:48:202
2792721138,18sleep20-21swapper/210:48:202
21076211215,59sleep20-21swapper/208:43:392
21076211215,59sleep20-21swapper/208:43:392
570321116,87sleep00-21swapper/009:38:250
2592821099,17sleep20-21swapper/208:58:372
9480210829,48sleep09629-21grep09:48:420
9480210829,48sleep09629-21grep09:48:420
11003210811,64sleep211049-21ntp_kernel_pll_09:54:052
23866210725,20sleep20-21swapper/212:18:292
7706210613,79sleep17714-21sh09:43:421
23336210310,66sleep10-21swapper/110:33:331
17383210317,66sleep016780-21/usr/sbin/munin10:13:370
10782210316,70sleep010970-21cat08:13:120
10782210316,70sleep010970-21cat08:13:120
17871210233,21sleep20-21swapper/208:33:372
24085210119,61sleep323847-21/usr/sbin/munin08:53:333
1731210027,61sleep21738-21cat11:08:112
16041210013,73sleep016101-21/usr/sbin/munin08:28:310
1243229712,71sleep112453-21kernelversion09:58:321
1243229712,71sleep112453-21kernelversion09:58:321
947729618,58sleep19559-21fschecks_time08:08:291
947729618,58sleep19559-21fschecks_time08:08:291
2264929615,66sleep122700-21cat08:49:111
1244429624,49sleep012466-21modprobe08:18:090
1244429624,49sleep012466-21modprobe08:18:090
956029516,56sleep20-21swapper/208:08:302
956029516,56sleep20-21swapper/208:08:302
790329517,64sleep17935-21idleruntime-cro11:28:081
790329517,64sleep17935-21idleruntime-cro11:28:081
50752948,47sleep20-21swapper/211:18:202
50752948,47sleep20-21swapper/211:18:202
41272948,69sleep20-21swapper/209:33:262
41272948,69sleep20-21swapper/209:33:262
2146329410,45sleep20-21swapper/210:28:212
82992938,46sleep20-21swapper/208:03:372
44612936,52sleep2281-21dbus-daemon07:53:112
168512937,12sleep10-21swapper/111:53:401
1494329310,70sleep315069-21sed08:23:413
296612925,44sleep20-21swapper/207:28:322
1502129211,56sleep115126-21timerandwakeup11:48:401
887929113,62sleep18891-21fschecks_count09:48:271
887929113,62sleep18891-21fschecks_count09:48:271
326112915,11sleep30-21swapper/309:18:453
69042885,11sleep20-21swapper/207:58:402
289642886,12sleep20-21swapper/209:08:342
2035128815,60sleep020395-21sh12:08:090
2035128815,60sleep020395-21sh12:08:090
1512828825,17sleep20-21swapper/211:48:412
1100028822,46sleep011027-21grep11:38:090
87992879,55sleep20-21swapper/211:28:382
2540928714,59sleep025413-21diskstats12:23:270
44742869,63sleep34616-21latency_hist07:53:123
2657628612,57sleep026617-21ntpq10:43:380
2657628612,57sleep026617-21ntpq10:43:380
1615128622,47sleep315236-21/usr/sbin/munin10:08:403
143082858,16sleep20-21swapper/208:23:242
249242845,47sleep20-21swapper/208:55:302
2457828419,44sleep324682-21munin-node07:13:193
1265228416,50sleep20-21swapper/211:43:112
2304628311,15sleep20-21swapper/210:33:262
2192228317,52sleep322126-21latency_hist08:48:123
2192228317,52sleep322126-21latency_hist08:48:123
2279828217,50sleep322826-21latency_hist10:33:113
3145728115,20sleep20-21swapper/209:18:112
1657828129,35sleep316704-21timerwakeupswit11:53:383
1561928116,45sleep113441-21snmpd08:28:111
1285028017,42sleep012910-21/usr/sbin/munin08:18:290
83722785,42sleep20-21swapper/209:46:262
221562786,60sleep00-21swapper/012:13:180
772527721,42sleep37766-21/usr/sbin/munin09:43:453
327127715,39sleep33403-21tail09:28:513
3250127715,36sleep332520-21apt-get07:38:223
3094727718,35sleep130944-21apt-get07:33:231
3094727718,35sleep130944-21apt-get07:33:231
2883327717,41sleep30-21swapper/309:08:313
2346427717,32sleep30-21swapper/312:18:093
164827718,29sleep30-21swapper/309:24:303
1931027617,42sleep018370-21/usr/sbin/munin10:18:420
189027414,26sleep00-21swapper/007:43:320
309227323,24sleep10-21swapper/109:28:441
301372734,50sleep00-21swapper/010:53:510
243527324,29sleep30-21swapper/311:08:363
2283127321,23sleep00-21swapper/010:33:120
2157927323,28sleep00-21swapper/007:03:350
1263027318,32sleep312136-21/usr/sbin/munin09:58:393
1263027318,32sleep312136-21/usr/sbin/munin09:58:393
795527217,28sleep30-21swapper/311:28:103
795527217,28sleep30-21swapper/311:28:103
618027213,42sleep36239-21/usr/sbin/munin09:38:403
2968627242,17sleep30-21swapper/312:38:083
2748527218,33sleep11-21systemd10:48:111
2748527218,33sleep11-21systemd10:48:111
2831627114,21sleep10-21swapper/107:23:381
2663827116,42sleep326678-21ps10:43:413
2663827116,42sleep326678-21ps10:43:413
1337227122,30sleep313474-21/usr/sbin/munin08:18:403
1294427122,28sleep00-21swapper/011:43:260
2227827019,20sleep10-21swapper/112:13:271
3137826917,34sleep0222-21systemd-journal09:18:090
2794626931,28sleep20-21swapper/212:30:402
2176126918,27sleep10-21swapper/110:28:321
1580626915,23sleep00-21swapper/010:08:370
1507226915,38sleep315221-21date10:08:113
2846926812,38sleep028593-21sed07:23:410
1711526819,25sleep30-21swapper/308:33:103
140632689,45sleep10-21swapper/108:23:101
102026814,38sleep1505-21apt-get09:23:231
982426716,37sleep39829-21entropy11:33:273
292432677,13sleep00-21swapper/009:08:400
2836826714,25sleep00-21swapper/009:08:120
2693826718,28sleep00-21swapper/007:18:410
2598526719,21sleep10-21swapper/107:18:091
1780726712,31sleep10-21swapper/111:58:331
1780726712,31sleep10-21swapper/111:58:331
1137026719,24sleep10-21swapper/109:54:121
2619126616,15sleep10-21swapper/108:58:401
2491326612,39sleep124914-21sshd12:21:301
2396226623,23sleep10-21swapper/108:53:311
161826612,40sleep11645-21grep07:43:251
1005226621,25sleep30-21swapper/309:53:093
1005226621,25sleep30-21swapper/309:53:093
856226421,25sleep00-21swapper/011:28:320
816626414,21sleep30-21swapper/311:28:193
3263726421,25sleep30-21swapper/311:03:113
3263726421,25sleep30-21swapper/311:03:113
2977126419,24sleep00-21swapper/007:28:340
2851126413,24sleep00-21swapper/012:33:290
2499026418,29sleep30-21swapper/310:38:323
1729526417,15sleep30-21swapper/310:13:363
1571926416,26sleep00-21swapper/011:53:110
596826326,23sleep05993-21munin-run07:58:100
3018326313,36sleep330190-21sh09:13:283
2021026316,27sleep10-21swapper/110:23:321
2015026316,21sleep00-21swapper/010:23:310
53826217,26sleep10-21swapper/109:23:111
247826219,27sleep00-21swapper/011:08:370
2274826213,35sleep322784-21/usr/sbin/munin08:49:133
167926211,27sleep01710-21munin-run11:08:090
652626121,22sleep10-21swapper/107:58:311
2756126118,23sleep127595-21munin-run07:23:101
2687826113,19sleep00-21swapper/012:28:240
2552126113,18sleep00-21swapper/010:38:410
2520126116,22sleep00-21swapper/008:58:110
2319626111,34sleep023219-21ntp_states07:08:350
217512616,11sleep00-21swapper/008:47:410
217512616,11sleep00-21swapper/008:47:410
1876826122,19sleep30-21swapper/310:18:333
1718526119,19sleep10-21swapper/108:33:121
1069526119,26sleep310714-21munin-run08:13:113
1069526119,26sleep310714-21munin-run08:13:113
1056826116,23sleep00-21swapper/009:53:230
703126014,28sleep37178-21awk07:58:413
557826013,29sleep35636-21/usr/sbin/munin11:18:383
557826013,29sleep35636-21/usr/sbin/munin11:18:383
2544626015,27sleep025440-21apt-get08:58:230
2112626016,20sleep00-21swapper/012:08:370
82372596,17sleep00-21swapper/009:44:380
570825919,21sleep10-21swapper/111:18:401
570825919,21sleep10-21swapper/111:18:401
2410525913,24sleep10-21swapper/107:13:121
2101325912,19sleep10-21swapper/108:43:381
2101325912,19sleep10-21swapper/108:43:381
1970425913,31sleep119801-21date10:23:111
124142595,42sleep00-21swapper/011:41:040
1125025913,28sleep311256-21df_inode08:13:273
1125025913,28sleep311256-21df_inode08:13:273
631725821,17sleep10-21swapper/109:38:421
327642585,42sleep00-21swapper/009:20:440
2921325814,17sleep30-21swapper/307:28:133
2832225821,17sleep30-21swapper/309:08:113
2761125817,26sleep30-21swapper/307:23:123
2727125822,15sleep10-21swapper/112:28:331
1371725815,27sleep013710-21apt-get10:03:240
1371725815,27sleep013710-21apt-get10:03:240
133112586,41sleep00-21swapper/010:01:380
133112586,41sleep00-21swapper/010:01:380
1199325822,17sleep30-21swapper/311:38:403
297972577,17sleep10-21swapper/107:28:351
2243525716,21sleep30-21swapper/307:08:103
2126225714,30sleep221274-21fschecks_count07:03:292
201492576,11sleep00-21swapper/008:41:590
17342578,24sleep00-21swapper/009:24:320
1171625713,25sleep10-21swapper/111:38:341
85882565,17sleep00-21swapper/008:03:430
50025616,25sleep1600-21perl11:03:291
482425614,25sleep14908-21cut07:53:301
3174125612,18sleep00-21swapper/007:33:430
3174125612,18sleep00-21swapper/007:33:430
2856025618,22sleep128557-21apt-get09:08:211
2792125615,25sleep127445-21apt-get10:48:151
2792125615,25sleep127445-21apt-get10:48:151
1587525619,19sleep10-21swapper/110:08:381
1417425615,18sleep10-21swapper/111:48:111
137792567,38sleep30-21swapper/311:43:453
677425512,25sleep06836-21/usr/sbin/munin07:58:380
3161325517,17sleep30-21swapper/307:33:423
3161325517,17sleep30-21swapper/307:33:423
2073925513,23sleep320843-21users10:23:423
2063825513,24sleep120633-21apt-get12:08:221
155422555,38sleep00-21swapper/008:27:500
1117825514,26sleep111167-21apt-get08:13:251
1117825514,26sleep111167-21apt-get08:13:251
827425413,17sleep10-21swapper/108:03:361
3140525415,18sleep00-21swapper/010:58:320
3140525415,18sleep00-21swapper/010:58:320
287712546,38sleep30-21swapper/307:23:493
2560525410,17sleep10-21swapper/112:23:321
2471825414,24sleep024713-21apt-get07:13:240
1940125414,22sleep319437-21ntp_states08:38:353
1936125414,19sleep30-21swapper/312:03:323
1936125414,19sleep30-21swapper/312:03:323
1777925420,18sleep317382-21/usr/sbin/munin08:33:343
1767825414,18sleep10-21swapper/108:33:321
412525314,22sleep04121-21apt-get09:33:240
412525314,22sleep04121-21apt-get09:33:240
391425315,20sleep30-21swapper/311:13:333
391425315,20sleep30-21swapper/311:13:333
2753625313,24sleep327662-21sed12:28:393
2590125319,18sleep10-21swapper/110:43:101
2438525314,19sleep30-21swapper/310:38:123
2378225310,26sleep323849-21df12:18:263
2312525314,24sleep023129-21diskstats10:33:280
1448325313,23sleep314490-21df_inode11:48:253
1058425311,24sleep310525-21apt-get09:53:263
550525214,23sleep05554-21ntp_states11:18:350
550525214,23sleep05554-21ntp_states11:18:350
3107325215,23sleep331094-21latency_hist10:58:103
2796325213,16sleep30-21swapper/310:48:293
2796325213,16sleep30-21swapper/310:48:293
2709425218,21sleep127106-21fschecks_count09:03:331
2125525217,19sleep10-21swapper/107:03:281
1379625214,24sleep313800-21diskstats10:03:273
784325116,17sleep30-21swapper/308:03:263
459125111,24sleep14592-21ntp_states09:33:411
459125111,24sleep14592-21ntp_states09:33:411
351325112,23sleep03508-21apt-get11:13:220
351325112,23sleep03508-21apt-get11:13:220
2853225113,19sleep10-21swapper/112:33:301
2283425115,21sleep322843-21df_inode07:08:263
196625112,23sleep11961-21apt-get11:08:221
174125113,17sleep30-21swapper/311:08:123
710025012,22sleep07106-21ntp_states11:23:350
710025012,22sleep07106-21ntp_states11:23:350
179512509,16sleep00-21swapper/011:58:360
179512509,16sleep00-21swapper/011:58:360
1729925013,14sleep00-21swapper/011:58:110
10692504,35sleep10-21swapper/107:39:501
2553724915,17sleep30-21swapper/312:23:303
2039624918,18sleep10-21swapper/112:08:101
2039624918,18sleep10-21swapper/112:08:101
495924710,22sleep34985-21irqstats07:53:323
404924717,15sleep30-21swapper/309:33:153
404924717,15sleep30-21swapper/309:33:153
316024611,14sleep10-21swapper/107:48:221
2675224410,17sleep30-21swapper/309:03:103
82622439,18sleep18271-21df_inode11:28:251
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional