You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-13 - 14:37

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Fri Mar 13, 2026 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
101162386718,3836sleep19922-21kworker/1:021:27:351
9452252815,496sleep09459-21cat23:08:320
4975244146,346sleep35103-21grep21:08:483
23416235813,333sleep023470-21sed22:08:360
2233523488,276sleep322354-21modprobe20:23:143
2233523488,276sleep322354-21modprobe20:23:143
5861234410,320sleep35875-21grep21:13:323
4720231024,268sleep04800-21grep21:08:450
1789229327,251sleep31922-21unixbench_singl20:58:473
6355226711,215sleep36368-21fschecks_count22:58:343
7364226617,232sleep17431-21/usr/sbin/munin21:18:311
7364226617,232sleep17431-21/usr/sbin/munin21:18:311
13142224249,64sleep30-21swapper/323:18:403
13142224249,64sleep30-21swapper/323:18:403
3625223811,212sleep23696-21latency_hist00:33:162
2179022357,215sleep321847-21if_eth022:03:353
5744222080,85sleep30-21swapper/319:28:443
13759221772,59sleep30-21swapper/321:38:363
10166220911,48sleep30-21swapper/321:28:143
31423220527,121sleep331438-21latency_hist00:18:153
386321998,18sleep30-21swapper/300:33:293
3134219529,109sleep33171-21ps21:03:433
29644219552,125sleep30-21swapper/322:28:383
960921910,75sleep330879-21cyclictest23:08:343
23623219040,130sleep323689-21wc22:08:433
804721747,36sleep313441-21snmpd19:37:563
25143217414,145sleep225144-21sshd20:29:352
31446216913,142sleep031373-21latency_hist00:18:160
20773216827,42sleep30-21swapper/323:43:353
366321678,105sleep311159-21kworker/u8:222:48:453
12488216716,85sleep30-21swapper/321:33:433
1993421626,15sleep00-21swapper/021:58:190
2756321608,137sleep327651-21cat20:38:363
25695216049,98sleep325706-21sort20:33:273
32028215911,136sleep332059-21fschecks_count20:53:343
2681421598,87sleep30-21swapper/322:18:453
25147215820,13sleep30-21swapper/322:13:403
17098215513,119sleep30-21swapper/323:33:143
31173215248,92sleep331187-21fschecks_count19:08:333
6032214842,47sleep30-21swapper/322:58:173
26647214810,127sleep013441-21snmpd00:02:380
1574021479,80sleep30-21swapper/321:43:443
25022214529,54sleep325149-21apt-config23:58:173
13343214518,100sleep00-21swapper/021:38:160
7917214411,84sleep30-21swapper/321:18:453
7917214411,84sleep30-21swapper/321:18:453
5352214320,101sleep20-21swapper/219:28:352
22561214035,37sleep30-21swapper/323:48:443
2354321387,12sleep00-21swapper/020:23:520
2354321387,12sleep00-21swapper/020:23:520
380121339,107sleep00-21swapper/021:06:350
103821339,68sleep30-21swapper/300:23:403
903721326,29sleep30-21swapper/323:07:383
31754213120,24sleep00-21swapper/000:18:330
29111213160,56sleep328817-21/usr/sbin/munin20:43:373
29111213160,56sleep328817-21/usr/sbin/munin20:43:373
13657213127,45sleep313441-21snmpd19:53:433
24260212812,68sleep324327-21fschecks_time20:28:353
1936821267,11sleep30-21swapper/321:56:443
1934212630,74sleep30-21swapper/319:18:183
1091921269,62sleep310958-21ps21:28:423
11480212510,65sleep30-21swapper/319:48:303
11480212510,65sleep30-21swapper/319:48:303
2905121245,10sleep30-21swapper/322:27:493
26343212443,63sleep330879-21cyclictest23:59:023
25403212221,28sleep00-21swapper/022:13:450
1098212114,25sleep01137-21memory00:23:410
30834211910,95sleep330946-21sed19:08:173
2058321195,29sleep30-21swapper/320:15:553
2058321195,29sleep30-21swapper/320:15:553
1999321185,10sleep30-21swapper/323:40:203
1190721170,15sleep012-21rcu_sched21:33:190
27667211614,82sleep127795-21/usr/sbin/munin00:03:481
24918211624,67sleep125052-21unixbench_singl20:28:491
2044421168,96sleep320491-21ntpq21:59:033
3246921155,30sleep30-21swapper/322:38:193
24175211513,83sleep024245-21/usr/sbin/munin20:28:320
17748211520,78sleep217762-21needreboot23:33:372
1588721155,29sleep30-21swapper/320:00:253
31946211416,81sleep032013-21/usr/sbin/munin20:53:310
4406211323,69sleep14438-21grep22:53:151
2217211310,88sleep12338-21cat22:43:461
52021119,90sleep3616-21cat00:23:163
8873210954,16sleep30-21swapper/319:38:453
373210911,14sleep10-21swapper/122:38:371
1533421095,28sleep30-21swapper/323:25:313
1533421095,28sleep30-21swapper/323:25:313
13271210629,59sleep113240-21/usr/sbin/munin19:53:331
1544221045,88sleep00-21swapper/023:26:490
1544221045,88sleep00-21swapper/023:26:490
18063210313,73sleep018150-21fschecks_time20:08:330
2586210246,40sleep30-21swapper/300:28:373
19634210213,70sleep119705-21/usr/sbin/munin20:13:351
19634210213,70sleep119705-21/usr/sbin/munin20:13:351
2910210120,61sleep02889-21latency_hist22:48:170
5283210017,66sleep25372-21apt-config00:38:172
30766210018,65sleep230936-21sed00:13:462
20535210016,65sleep220557-21/usr/sbin/munin21:59:062
1925629945,14sleep30-21swapper/320:13:173
1765729915,63sleep117361-21/usr/sbin/munin23:33:361
1756729844,37sleep30-21swapper/323:33:343
1530429815,64sleep215324-21/usr/sbin/munin19:58:422
1764829716,67sleep017650-21sshd21:49:350
1685929615,65sleep216901-21proc_pri20:03:432
16692967,12sleep30-21swapper/322:43:323
1452629520,54sleep214533-21modprobe19:58:142
1274629523,46sleep212868-21unixbench_singl21:33:462
1197729512,68sleep111994-21apt21:33:251
3246729416,61sleep032645-21date19:13:180
3076129435,38sleep130793-21munin-run19:08:151
2091029413,67sleep20-21swapper/223:43:382
277729312,66sleep02784-21open_files00:28:420
238629313,15sleep30-21swapper/319:18:343
1948429320,56sleep019536-21http21:58:150
3139229218,59sleep231412-21/usr/sbin/munin22:33:412
172229112,64sleep11723-21sshd19:16:341
1481629124,43sleep114835-21apt-get19:58:251
2260129019,11sleep30-21swapper/320:23:263
2260129019,11sleep30-21swapper/320:23:263
1474929015,54sleep114755-21ntp_states23:23:431
1474929015,54sleep114755-21ntp_states23:23:431
1466228926,42sleep214267-21/usr/sbin/munin23:23:402
1466228926,42sleep214267-21/usr/sbin/munin23:23:402
2293828816,44sleep123086-21latency_hist22:08:171
214802885,11sleep10-21swapper/123:44:341
2698228722,49sleep30-21swapper/300:03:303
1871628714,54sleep018737-21latency_hist23:38:150
1114928717,53sleep210642-21apt-get23:13:262
1114928717,53sleep210642-21apt-get23:13:262
3218828621,49sleep30-21swapper/322:38:153
2960728630,25sleep00-21swapper/019:03:340
1765228620,12sleep30-21swapper/320:08:123
410528511,60sleep04146-21ntpq19:23:400
371928519,11sleep30-21swapper/319:23:313
1236428519,11sleep30-21swapper/323:17:273
1236428519,11sleep30-21swapper/323:17:273
3066228418,11sleep30-21swapper/320:48:373
3066228418,11sleep30-21swapper/320:48:373
1979628419,36sleep00-21swapper/020:13:380
1979628419,36sleep00-21swapper/020:13:380
178452847,67sleep30-21swapper/321:52:043
431828316,20sleep20-21swapper/221:08:352
2636228312,19sleep00-21swapper/022:18:330
971528212,55sleep09862-21latency_hist19:43:180
296442828,15sleep30-21swapper/300:11:253
1279828216,51sleep112900-21apt-key19:53:161
1279828216,51sleep112900-21apt-key19:53:161
3267228117,39sleep232740-21grep19:13:202
2737828129,30sleep00-21swapper/020:38:330
2328528015,45sleep123408-21timerwakeupswit20:23:491
2328528015,45sleep123408-21timerwakeupswit20:23:491
2176428018,43sleep221859-21grep20:18:492
2176428018,43sleep221859-21grep20:18:492
2406827927,29sleep10-21swapper/123:53:431
1638727921,38sleep116418-21modprobe21:48:141
45942780,6sleep10-21swapper/100:33:471
2951127812,47sleep013441-21snmpd20:43:460
2951127812,47sleep013441-21snmpd20:43:460
123927811,53sleep11300-21processes00:23:461
2913427724,28sleep20-21swapper/222:28:162
348027619,42sleep13539-21cat19:23:171
2544227625,29sleep10-21swapper/123:58:391
2331527620,30sleep10-21swapper/123:53:141
106392767,11sleep30-21swapper/319:43:463
1959027517,30sleep219739-21apt-config21:58:172
133727520,27sleep20-21swapper/220:58:382
826127415,36sleep28272-21/usr/sbin/munin21:18:482
826127415,36sleep28272-21/usr/sbin/munin21:18:482
245727412,45sleep12604-21cat21:03:171
1644927416,43sleep016526-21timerandwakeup23:28:470
959127317,36sleep10-21swapper/119:42:341
255432730,7sleep10-21swapper/122:13:451
3170327215,39sleep10-21swapper/120:53:161
3170327215,39sleep10-21swapper/120:53:161
1118227212,44sleep111147-21apt-get23:13:461
1118227212,44sleep111147-21apt-get23:13:461
2494327121,29sleep224930-40mandb23:58:162
936927015,41sleep19396-21grep23:08:291
1815127020,31sleep118203-21interrupts20:08:351
82826920,25sleep10-21swapper/120:58:161
704726923,30sleep1281-21dbus-daemon21:18:151
445926917,25sleep00-21swapper/022:53:180
3226626919,28sleep10-21swapper/122:38:171
3220626919,30sleep232226-21cut20:53:372
2840326917,36sleep028475-21cat22:23:460
273542698,14sleep20-21swapper/200:03:412
264426917,35sleep22709-21find19:18:412
20672695,11sleep00-21swapper/019:18:190
17602698,49sleep00-21swapper/019:17:020
1367126920,33sleep213758-21fschecks_time21:38:352
968426814,41sleep21-21systemd19:43:162
595426817,34sleep05966-21modprobe22:58:140
3010526818,30sleep030253-21sed22:28:470
2905126818,27sleep20-21swapper/220:43:362
2905126818,27sleep20-21swapper/220:43:362
2791226818,23sleep10-21swapper/122:23:351
1859426819,35sleep118605-21netstat21:53:391
1303426824,25sleep013161-21latency_hist19:53:190
1303426824,25sleep013161-21latency_hist19:53:190
395126721,22sleep00-21swapper/000:33:340
1103026718,28sleep10-21swapper/121:28:441
753226621,26sleep07684-21sed19:33:450
322242665,12sleep10-21swapper/119:11:381
2637326612,22sleep10-21swapper/122:18:341
1704726620,25sleep20-21swapper/221:48:402
1665926620,20sleep00-21swapper/020:03:370
2632926515,19sleep00-21swapper/020:33:490
733726412,37sleep013441-21snmpd21:18:260
733726412,37sleep013441-21snmpd21:18:260
637326416,28sleep00-21swapper/022:58:350
1958726417,32sleep219710-21timerwakeupswit23:38:462
1250526419,21sleep00-21swapper/023:18:170
1250526419,21sleep00-21swapper/023:18:170
840026321,24sleep10-21swapper/123:03:441
735026318,17sleep20-21swapper/219:33:432
588026320,22sleep10-21swapper/119:28:461
2596226313,29sleep2281-21dbus-daemon22:18:152
1951826317,24sleep10-21swapper/123:38:451
147382635,11sleep00-21swapper/021:41:560
2964726216,30sleep00-21swapper/000:11:260
2766226216,24sleep10-21swapper/120:38:381
1714926211,26sleep10-21swapper/123:33:161
141862627,12sleep00-21swapper/019:53:520
122726219,20sleep00-21swapper/020:58:360
2579126112,34sleep125797-21/usr/sbin/munin20:33:341
1040826115,26sleep00-21swapper/019:43:430
502826018,20sleep10-21swapper/119:28:171
320002605,44sleep00-21swapper/022:35:490
2930426020,23sleep10-21swapper/120:43:431
2930426020,23sleep10-21swapper/120:43:431
2695826017,21sleep00-21swapper/000:03:270
2263926013,23sleep00-21swapper/023:48:450
105372605,12sleep20-21swapper/219:43:452
466425916,27sleep14687-21ntp_states21:08:421
310562594,17sleep20-21swapper/219:08:202
2653225916,18sleep20-21swapper/220:33:512
2544325912,28sleep225415-21apt-get23:58:402
243812595,19sleep10-21swapper/122:12:571
1419525920,17sleep10-21swapper/121:38:461
1282925918,18sleep10-21swapper/123:18:341
1282925918,18sleep10-21swapper/123:18:341
1274525912,23sleep20-21swapper/223:18:312
1274525912,23sleep20-21swapper/223:18:312
1147925915,25sleep211473-21apt-get19:48:292
1147925915,25sleep211473-21apt-get19:48:292
1111625918,20sleep20-21swapper/221:28:452
506525813,29sleep15096-21/usr/sbin/munin22:53:391
334825818,18sleep20-21swapper/221:03:462
3011625820,22sleep20-21swapper/219:03:442
2853025816,26sleep228524-21apt-get00:08:292
2675525815,21sleep20-21swapper/222:18:442
2080725815,25sleep020876-21memory23:43:360
2026125817,21sleep00-21swapper/023:43:160
2004625817,25sleep120031-21apt-get21:58:321
1521425818,21sleep00-21swapper/021:43:330
1274125816,25sleep012735-21apt-get23:18:290
1274125816,25sleep012735-21apt-get23:18:290
992825715,26sleep19923-21apt-get19:43:271
633225712,32sleep26411-21sed21:13:442
3165725715,25sleep00-21swapper/020:53:150
3165725715,25sleep00-21swapper/020:53:150
3034925713,20sleep10-21swapper/100:13:371
303112576,16sleep10-21swapper/122:29:081
2424925715,18sleep00-21swapper/023:53:460
2309225718,21sleep20-21swapper/220:23:462
2309225718,21sleep20-21swapper/220:23:462
2105525710,20sleep10-21swapper/120:18:281
2105525710,20sleep10-21swapper/120:18:281
130725721,21sleep20-21swapper/222:43:142
694525616,27sleep06946-21sendmail-mta21:16:570
324925620,17sleep10-21swapper/121:03:451
3166525613,25sleep131663-21apt-get00:18:261
2169325611,30sleep221788-21fschecks_time22:03:342
2084125617,17sleep20-21swapper/220:18:162
2084125617,17sleep20-21swapper/220:18:162
913025519,19sleep00-21swapper/023:08:160
684425524,16sleep20-21swapper/222:58:452
3068225517,20sleep20-21swapper/222:33:172
2890025515,25sleep128914-21netstat00:08:411
2217125517,20sleep00-21swapper/022:03:440
936525415,22sleep29362-21apt-get23:08:262
371825413,24sleep23712-21apt-get19:23:302
1794225414,24sleep217936-21apt-get20:08:272
1676325415,18sleep10-21swapper/121:48:351
160125413,13sleep20-21swapper/222:43:312
698225319,18sleep10-21swapper/119:33:341
361725216,18sleep20-21swapper/222:48:432
2108025213,23sleep021074-21apt-get20:18:310
2108025213,23sleep021074-21apt-get20:18:310
980625118,17sleep20-21swapper/221:23:492
849725120,17sleep20-21swapper/223:03:452
647025114,19sleep10-21swapper/122:58:361
3174525116,18sleep20-21swapper/220:53:172
2350825112,23sleep223515-21memory22:08:382
2335425112,23sleep223366-21munin-run23:53:152
176552506,33sleep10-21swapper/120:08:141
49824915,17sleep00-21swapper/022:38:400
1616024915,18sleep00-21swapper/020:03:170
50124811,19sleep2519-21munin-run00:23:152
231824812,21sleep12385-21df00:28:311
888424713,18sleep00-21swapper/021:23:270
520224713,20sleep21-21systemd22:53:442
2782624710,18sleep20-21swapper/220:38:432
206624711,22sleep22075-21unin-run00:28:162
581224614,15sleep00-21swapper/019:28:450
3081124614,16sleep20-21swapper/220:48:402
1597924613,16sleep20-21swapper/223:28:362
1520024611,21sleep215206-21df_inode21:43:312
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional