You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-03 - 14:39

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7s.osadl.org (updated Wed Jun 03, 2026 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1870228587,835sleep318745-21ntpq20:49:053
25006276714,733sleep01113-21kworker/u8:122:53:320
25006276714,733sleep01113-21kworker/u8:122:53:320
22228269216,658sleep022236-21/usr/sbin/munin00:28:360
4494251112,484sleep14502-21ntpq20:03:471
12392241936,357sleep312483-21fschecks_time22:13:363
19313241222,372sleep119432-21/usr/sbin/munin22:33:471
19313241222,372sleep119432-21/usr/sbin/munin22:33:471
4853240830,363sleep24938-21users20:03:522
2255223939,372sleep322599-21ntpq00:28:433
27423228616,253sleep327510-21wc21:18:193
5462279102,63sleep30-21swapper/323:18:373
19902227018,234sleep319916-21fschecks_count20:53:333
22962225316,220sleep022957-21apt-get21:03:310
23048224014,209sleep123100-21fschecks_count21:03:361
29971223791,25sleep30-21swapper/319:43:193
10987222315,192sleep111104-21latency_hist23:53:181
11512219812,82sleep311552-21latency23:53:523
11512219812,82sleep311552-21latency23:53:523
2476219431,74sleep30-21swapper/319:58:323
20682219110,167sleep320762-21cat22:38:463
20682219110,167sleep320762-21cat22:38:463
27746218625,139sleep00-21swapper/021:18:360
23847218229,60sleep30-21swapper/322:48:473
23847218229,60sleep30-21swapper/322:48:473
22643217612,141sleep322657-21modprobe19:13:163
29867217415,140sleep329661-21/usr/sbin/munin23:08:363
25509216915,138sleep325585-21awk22:53:473
25509216915,138sleep325585-21awk22:53:473
32121216613,63sleep332157-21ntpq19:48:393
9847216011,47sleep30-21swapper/323:48:403
8316216086,23sleep30-21swapper/323:43:353
9271215894,21sleep30-21swapper/320:18:463
9271215894,21sleep30-21swapper/320:18:463
27927215873,37sleep327949-21kernelversion21:18:393
27082215717,125sleep327171-21awk19:29:223
27082215717,125sleep327171-21awk19:29:223
25837215413,128sleep325950-21latency_hist21:13:183
17744215213,56sleep30-21swapper/320:48:163
12010215014,110sleep30-21swapper/322:13:153
12010215014,110sleep30-21swapper/322:13:153
17219214816,120sleep317272-21sort00:13:173
31560214722,21sleep30-21swapper/319:48:193
1160214717,55sleep30-21swapper/319:53:403
1673214515,115sleep01744-21latency_hist23:23:180
32701214410,121sleep332748-21ntpq21:33:403
1426214414,50sleep30-21swapper/321:38:333
427721428,70sleep34340-21latency_hist21:48:173
427721428,70sleep34340-21latency_hist21:48:173
7127213512,52sleep30-21swapper/320:13:293
7042213410,105sleep30-21swapper/323:38:433
7443212510,15sleep30-21swapper/321:58:183
18641212424,11sleep30-21swapper/300:17:093
13865212310,39sleep30-21swapper/322:18:283
18242211919,81sleep117758-21apt-get20:48:261
11583211912,13sleep30-21swapper/320:28:183
6821211818,12sleep30-21swapper/320:13:013
4979211810,87sleep30-21swapper/323:33:263
8572211718,10sleep30-21swapper/321:58:593
32499211719,78sleep232551-21sendmail23:18:162
2336321179,40sleep30-21swapper/321:03:423
12813211629,74sleep312806-21apt-get23:58:323
5556211515,87sleep15568-21acpi20:08:271
5509211515,12sleep30-21swapper/321:49:473
5509211515,12sleep30-21swapper/321:49:473
441421155,31sleep30-21swapper/323:29:233
31394211425,67sleep131450-21idleruntime23:13:351
3077221149,38sleep30-21swapper/321:28:443
23730211410,91sleep323778-21munin-run19:18:173
24078211329,72sleep323990-21/usr/sbin/munin19:18:293
2562121126,13sleep30-21swapper/319:24:303
19759211222,68sleep219907-21unixbench_singl00:18:492
19759211222,68sleep219907-21unixbench_singl00:18:492
16488211211,83sleep00-21swapper/020:43:330
30934211116,70sleep031076-21grep19:43:480
19892211122,67sleep019985-21/usr/sbin/munin19:03:370
1543421117,37sleep30-21swapper/322:23:283
1543421117,37sleep30-21swapper/322:23:283
20315211016,80sleep220356-21if_eth022:38:372
20315211016,80sleep220356-21if_eth022:38:372
1178021095,12sleep30-21swapper/320:28:223
26678210828,68sleep326786-21perl22:58:353
26678210828,68sleep326786-21perl22:58:353
2463421086,11sleep30-21swapper/300:34:083
22553210818,12sleep30-21swapper/321:01:443
13980210822,67sleep214045-21uniq22:18:392
29207210711,20sleep30-21swapper/321:23:313
19105210465,23sleep30-21swapper/322:33:453
19105210465,23sleep30-21swapper/322:33:453
28746210314,66sleep30-21swapper/323:03:483
18728210320,61sleep118767-21cron00:18:161
21936210219,18sleep20-21swapper/200:28:182
21936210219,18sleep20-21swapper/200:28:182
2502629918,61sleep025042-21cat00:38:160
1195929915,68sleep212022-21/usr/sbin/munin20:28:382
695129812,71sleep00-21swapper/023:38:400
285729828,11sleep30-21swapper/323:24:183
2668729816,63sleep226741-21if_eth019:29:132
2668729816,63sleep226741-21if_eth019:29:132
2796929521,10sleep30-21swapper/319:33:303
2123629415,66sleep30-21swapper/300:23:473
2123629415,66sleep30-21swapper/300:23:473
2847229310,63sleep00-21swapper/023:03:410
201092939,20sleep00-21swapper/022:38:310
201092939,20sleep00-21swapper/022:38:310
76012926,11sleep00-21swapper/021:58:190
711829219,12sleep30-21swapper/321:55:363
2461329215,54sleep124503-21/usr/sbin/munin21:08:361
2266229218,12sleep30-21swapper/322:44:553
2266229218,12sleep30-21swapper/322:44:553
1460029220,11sleep30-21swapper/320:37:593
1748629120,11sleep30-21swapper/322:29:053
1748629120,11sleep30-21swapper/322:29:053
318302909,18sleep331906-21find23:13:453
2789829013,61sleep128032-21latency_hist23:03:171
2789829013,61sleep128032-21latency_hist23:03:171
2661429012,64sleep126686-21fschecks_time19:29:121
2661429012,64sleep126686-21fschecks_time19:29:121
2321729012,64sleep023244-21grep22:48:310
2321729012,64sleep023244-21grep22:48:310
1609729018,12sleep30-21swapper/320:41:583
1609729018,12sleep30-21swapper/320:41:583
1258229013,60sleep112753-21sed23:58:181
1258229013,60sleep112753-21sed23:58:181
2015928919,11sleep30-21swapper/300:21:253
2015928919,11sleep30-21swapper/300:21:253
1544828918,10sleep30-21swapper/300:06:033
1122828819,54sleep011288-21/usr/sbin/munin22:08:440
1122828819,54sleep011288-21/usr/sbin/munin22:08:440
41012878,14sleep30-21swapper/320:03:373
2643628622,39sleep026431-21apt-get19:28:270
2643628622,39sleep026431-21apt-get19:28:270
229928614,18sleep20-21swapper/223:23:412
191128612,61sleep11923-21cpu23:23:311
1041228615,11sleep30-21swapper/322:07:383
492028524,33sleep00-21swapper/021:48:430
492028524,33sleep00-21swapper/021:48:430
2462028518,44sleep124696-21sshd19:18:431
110628530,31sleep20-21swapper/221:38:152
1082328513,56sleep110832-21df_inode22:08:331
1082328513,56sleep110832-21df_inode22:08:331
2934028416,44sleep029505-21grep19:38:440
173842840,6sleep20-21swapper/220:43:502
133772840,6sleep10-21swapper/120:33:321
364928339,23sleep20-21swapper/223:28:372
305628317,39sleep13135-21fschecks_time21:43:351
305628317,39sleep13135-21fschecks_time21:43:351
2116928313,54sleep021311-21latency_hist20:58:180
728628211,56sleep07287-21sshd21:57:420
728628211,56sleep07287-21sshd21:57:420
616928117,30sleep00-21swapper/020:08:460
2504528126,26sleep10-21swapper/100:38:171
379228015,49sleep13881-21ls20:03:181
2977428015,50sleep229866-21fschecks_time23:08:352
87527918,39sleep0894-21apt-get19:53:290
65927927,28sleep20-21swapper/219:53:182
765127817,16sleep10-21swapper/121:58:321
1965627816,43sleep119758-21/usr/sbin/munin00:18:481
1965627816,43sleep119758-21/usr/sbin/munin00:18:481
713127719,26sleep20-21swapper/220:13:322
2111927717,34sleep10-21swapper/119:08:181
1538427729,30sleep215394-21sh20:38:432
1538427729,30sleep215394-21sh20:38:432
822227512,38sleep28313-21cut23:43:342
199912756,11sleep00-21swapper/000:19:160
199912756,11sleep00-21swapper/000:19:160
696127420,23sleep20-21swapper/220:13:202
2668127421,28sleep00-21swapper/021:13:480
2369927422,27sleep20-21swapper/219:18:162
880327319,31sleep10-21swapper/123:43:451
654527316,42sleep16607-21grep21:53:441
654527316,42sleep16607-21grep21:53:441
54252736,14sleep10-21swapper/123:33:401
1861727311,32sleep018635-21fschecks_count22:33:340
1861727311,32sleep018635-21fschecks_count22:33:340
1788427318,33sleep118065-21gpgv20:48:181
448127214,23sleep00-21swapper/020:03:460
2396627214,36sleep224028-21latency00:33:382
820427111,44sleep08211-21diskstats23:43:320
789227119,23sleep20-21swapper/223:43:182
2973527118,32sleep129810-21/usr/sbin/munin21:23:461
2666527115,27sleep20-21swapper/222:58:342
2666527115,27sleep20-21swapper/222:58:342
1760627118,34sleep117691-21awk22:29:071
1760627118,34sleep117691-21awk22:29:071
1170427114,26sleep00-21swapper/023:53:570
1170427114,26sleep00-21swapper/023:53:570
3182426910,15sleep00-21swapper/019:48:320
2604826916,31sleep026213-21gpgv19:28:180
2008326917,25sleep20-21swapper/220:53:362
198832699,13sleep10-21swapper/120:53:311
1102126917,30sleep20-21swapper/222:08:382
1102126917,30sleep20-21swapper/222:08:382
45226823,26sleep20-21swapper/223:18:362
2292526821,31sleep022949-21awk22:48:150
2292526821,31sleep022949-21awk22:48:150
2261926814,37sleep222683-21ps00:28:462
1832926817,24sleep00-21swapper/022:33:180
1543726823,24sleep10-21swapper/122:23:301
1543726823,24sleep10-21swapper/122:23:301
1002826816,25sleep00-21swapper/020:23:190
1002826816,25sleep00-21swapper/020:23:190
2377026713,40sleep223831-21switchtime22:48:462
2377026713,40sleep223831-21switchtime22:48:462
227126723,23sleep20-21swapper/219:58:192
1156426720,33sleep211582-21wc20:28:172
159632667,15sleep00-21swapper/020:40:180
159632667,15sleep00-21swapper/020:40:180
1345426622,21sleep00-21swapper/020:33:360
975726511,42sleep29760-21sshd20:20:312
975726511,42sleep29760-21sshd20:20:312
3098026522,26sleep20-21swapper/221:28:512
2478826514,32sleep024805-21cut21:08:390
1523726520,24sleep10-21swapper/122:23:181
1492426517,27sleep00-21swapper/000:03:500
1208226514,35sleep00-21swapper/020:28:400
1131426518,29sleep20-21swapper/223:53:472
1131426518,29sleep20-21swapper/223:53:472
3087126416,21sleep10-21swapper/121:28:501
2812826415,22sleep10-21swapper/121:18:451
2616126411,39sleep126227-21fschecks_time21:13:371
1958326415,28sleep00-21swapper/020:53:180
151962649,39sleep015236-21sed22:23:170
1178826423,23sleep10-21swapper/120:28:291
2217226325,18sleep219478-21kworker/2:122:43:452
2217226325,18sleep219478-21kworker/2:122:43:452
2051126215,20sleep30-21swapper/319:03:483
186382625,46sleep00-21swapper/000:17:060
966526121,20sleep10-21swapper/122:03:451
2995626118,19sleep10-21swapper/123:08:381
273326112,24sleep00-21swapper/021:43:180
2176126116,27sleep121870-21perl22:43:361
2176126116,27sleep121870-21perl22:43:361
1987626122,21sleep20-21swapper/219:03:352
1500126120,21sleep10-21swapper/100:03:511
35872605,13sleep00-21swapper/020:01:150
3023726015,32sleep030311-21switchtime23:08:450
295762606,11sleep10-21swapper/119:38:491
2165426019,19sleep10-21swapper/119:08:381
1343726018,26sleep213446-21/usr/sbin/munin20:33:342
896525918,21sleep20-21swapper/222:03:172
3202225914,33sleep132074-21latency19:48:371
2944725918,20sleep10-21swapper/123:08:181
2905625914,29sleep229102-21grep19:38:382
2270825918,19sleep20-21swapper/219:13:192
1674825915,21sleep00-21swapper/022:28:170
1674825915,21sleep00-21swapper/022:28:170
1669525918,27sleep216721-21munin-run22:28:152
1669525918,27sleep216721-21munin-run22:28:152
1653225920,21sleep10-21swapper/100:08:471
1646325914,20sleep00-21swapper/000:08:460
1591625917,20sleep20-21swapper/200:08:322
1483225912,30sleep214854-21/usr/sbin/munin00:03:482
449825814,18sleep20-21swapper/221:48:282
449825814,18sleep20-21swapper/221:48:282
3168525813,24sleep00-21swapper/023:13:420
126825811,23sleep10-21swapper/119:53:421
740725716,18sleep20-21swapper/221:58:172
740725716,18sleep20-21swapper/221:58:172
548925717,20sleep00-21swapper/023:33:430
3085825714,18sleep00-21swapper/021:28:490
2502525712,17sleep10-21swapper/122:53:331
2502525712,17sleep10-21swapper/122:53:331
2409025714,17sleep00-21swapper/019:18:300
2140625717,25sleep021402-21apt-get20:58:300
2067725715,18sleep00-21swapper/000:23:350
2067725715,18sleep00-21swapper/000:23:350
2010225714,25sleep120103-21apt-get22:38:271
2010225714,25sleep120103-21apt-get22:38:271
1711025711,24sleep20-21swapper/222:28:562
1711025711,24sleep20-21swapper/222:28:562
1375625714,30sleep113761-21bash23:59:151
2591625614,25sleep225947-21munin-run19:28:162
1827325612,26sleep218267-21apt-get20:48:312
26022558,36sleep1363-21snmpd21:42:141
231112558,16sleep10-21swapper/119:13:221
197142555,13sleep20-21swapper/222:37:152
197142555,13sleep20-21swapper/222:37:152
1751825513,17sleep00-21swapper/022:29:060
1746225513,25sleep217455-21apt-get00:13:332
136512559,24sleep10-21swapper/122:18:181
1023825514,26sleep010257-21apt-get20:23:270
721425416,18sleep10-21swapper/120:13:351
452325412,25sleep14518-21apt-get21:48:311
452325412,25sleep14518-21apt-get21:48:311
2507825413,17sleep20-21swapper/222:53:342
2507825413,17sleep20-21swapper/222:53:342
2378125416,18sleep00-21swapper/019:18:180
2296425413,23sleep223031-21df21:03:332
1371525415,20sleep00-21swapper/023:58:540
1364025414,23sleep013650-21munin-run22:18:170
1275425413,27sleep212760-21ntp_states22:13:432
1064125413,19sleep10-21swapper/120:23:401
500525311,24sleep24998-21apt-get23:33:292
357525313,26sleep03662-21ls21:43:460
357525313,26sleep03662-21ls21:43:460
322125315,18sleep00-21swapper/023:28:190
2841325312,25sleep228423-21munin-run19:38:172
236992539,27sleep123769-21systemctl22:48:451
236992539,27sleep123769-21systemctl22:48:451
908125214,20sleep10-21swapper/120:18:401
908125214,20sleep10-21swapper/120:18:401
848125213,18sleep00-21swapper/020:18:190
3151625212,17sleep20-21swapper/219:48:182
2837525212,26sleep228409-21irqstats23:03:392
2773025212,17sleep20-21swapper/221:18:342
2452025214,23sleep224522-21apt-get21:08:292
2190225219,18sleep10-21swapper/100:28:171
2190225219,18sleep10-21swapper/100:28:171
1623725214,23sleep116246-21munin-run20:43:181
1623725214,23sleep116246-21munin-run20:43:181
103125213,24sleep11163-21timerwakeupswit23:18:471
91782518,17sleep00-21swapper/022:03:280
3230125114,22sleep132295-21apt-get21:33:301
2447525114,18sleep20-21swapper/219:18:402
224492518,16sleep10-21swapper/100:28:411
1345425112,27sleep213573-21timerwakeupswit23:58:522
1009225113,16sleep00-21swapper/023:48:450
345825012,22sleep03452-21apt-get23:28:310
343825015,18sleep20-21swapper/221:43:442
343825015,18sleep20-21swapper/221:43:442
2965325012,16sleep20-21swapper/221:23:432
26812507,16sleep20-21swapper/221:43:162
941024912,16sleep20-21swapper/222:03:382
355424913,22sleep13648-21grep23:28:361
2919924911,22sleep029202-21apt-get21:23:280
2692424911,21sleep226977-21users21:13:502
2167624912,15sleep10-21swapper/120:58:391
3237824812,19sleep00-21swapper/021:33:330
2657924814,17sleep00-21swapper/022:58:280
2657924814,17sleep00-21swapper/022:58:280
2145924812,20sleep021465-21df_inode19:08:330
711724710,19sleep17190-21/usr/sbin/munin23:38:451
104222479,21sleep210572-21awk23:48:512
302052469,18sleep10-21swapper/119:43:291
301324611,21sleep23082-21sensors_temp19:58:472
226742449,16sleep20-21swapper/221:03:162
283442417,15sleep10-21swapper/119:38:161
283072307,15sleep00-21swapper/019:37:470
255502308,11sleep10-21swapper/119:23:351
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional