You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-20 - 09:04

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #4, slot #1

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1s.osadl.org (updated Mon Jul 20, 2026 00:44:20)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3091821000,3sleep130043-21/usr/sbin/munin00:23:471
3091821000,3sleep130043-21/usr/sbin/munin00:23:471
3091821000,3sleep130043-21/usr/sbin/munin00:23:471
133822960,2sleep10-21swapper/120:43:451
133822960,2sleep10-21swapper/120:43:451
303612770,3sleep20-21swapper/220:03:382
303612770,3sleep20-21swapper/220:03:382
160282770,2sleep116027-21head22:18:401
160282770,2sleep116027-21head22:18:401
93012720,3sleep20-21swapper/220:33:412
93012720,3sleep20-21swapper/220:33:412
57602710,2sleep05761-21cat23:18:370
57602710,2sleep05761-21cat23:18:370
57602710,2sleep05761-21cat23:18:370
255662630,2sleep125565-21grep21:18:351
239092630,2sleep122417-40mandb00:04:101
239092630,2sleep122417-40mandb00:04:101
88572600,3chrt0-21swapper/223:23:502
88572600,3chrt0-21swapper/223:23:502
88572600,3chrt0-21swapper/223:23:502
254752580,1sleep025476-21latency_hist21:18:310
39902560,3sleep30-21swapper/320:18:473
39902560,3sleep30-21swapper/320:18:473
39902560,3sleep30-21swapper/320:18:473
17592500,3chrt11523-21cyclictest00:33:391
17592500,3chrt11523-21cyclictest00:33:391
17592500,3chrt11523-21cyclictest00:33:391
319482490,2chrt0-21swapper/121:33:461
319482490,2chrt0-21swapper/121:33:461
319482490,2chrt0-21swapper/121:33:461
25432490,3sleep20-21swapper/200:33:492
25432490,3sleep20-21swapper/200:33:492
25432490,3sleep20-21swapper/200:33:492
168032470,3sleep20-21swapper/222:18:502
168032470,3sleep20-21swapper/222:18:502
101382450,2sleep110139-21uname23:28:431
101382450,2sleep110139-21uname23:28:431
305872440,2sleep10-21swapper/120:03:431
305872440,2sleep10-21swapper/120:03:431
11535994437,3cyclictest0-21swapper/000:03:400
11535994437,3cyclictest0-21swapper/000:03:400
177042430,3sleep017862-21tr22:23:390
177042430,3sleep017862-21tr22:23:390
118432430,2sleep011945-21/usr/sbin/munin23:33:420
81862420,2sleep30-21swapper/323:23:433
81862420,2sleep30-21swapper/323:23:433
81862420,2sleep30-21swapper/323:23:433
11535994133,6cyclictest0-21swapper/000:03:300
326542400,3sleep00-21swapper/020:08:520
326542400,3sleep00-21swapper/020:08:520
326542400,3sleep00-21swapper/020:08:520
1153599403,33cyclictest23080-21if_wlan000:03:420
1153599403,33cyclictest23080-21if_wlan000:03:420
11543993729,6cyclictest550-21cron23:53:261
11543993729,6cyclictest550-21cron23:53:261
1156699352,21cyclictest22473-21chrt00:03:303
1156699352,21cyclictest22473-21chrt00:03:303
1156699352,21cyclictest22473-21chrt00:03:303
11535993529,5cyclictest1354-21netstat21:38:450
11535993529,5cyclictest1354-21netstat21:38:450
11535993529,5cyclictest1354-21netstat21:38:450
288482340,2sleep30-21swapper/300:18:423
288482340,2sleep30-21swapper/300:18:423
288482340,2sleep30-21swapper/300:18:423
11543993326,5cyclictest21023-21gltestperf23:58:391
11543993326,5cyclictest21023-21gltestperf23:58:391
11543993226,4cyclictest10215-21fschecks_count22:03:371
11543993224,6cyclictest24068-21mailstats22:38:471
11543993224,6cyclictest24068-21mailstats22:38:471
11543993224,6cyclictest24068-21mailstats22:38:471
11535993223,7cyclictest8822-21needreboot21:58:450
11535993223,7cyclictest8822-21needreboot21:58:450
7432310,3sleep20-21swapper/223:03:442
7432310,3sleep20-21swapper/223:03:442
7432310,3sleep20-21swapper/223:03:442
118302310,2chrt11831-21cut22:08:311
118302310,2chrt11831-21cut22:08:311
11543993125,4cyclictest4670-21kworker/u9:0-xprtiod23:48:261
11543993125,4cyclictest4670-21kworker/u9:0-xprtiod23:48:261
11543993125,4cyclictest4670-21kworker/u9:0-xprtiod23:48:261
11543993124,5cyclictest18950-21fschecks_time19:33:401
11543993124,5cyclictest18950-21fschecks_time19:33:401
11535993126,3cyclictest3196-21df_inode20:18:380
11535993126,3cyclictest3196-21df_inode20:18:380
11535993126,3cyclictest3196-21df_inode20:18:380
11535993124,6cyclictest7030-21ntp_offset21:53:450
11535993124,6cyclictest7030-21ntp_offset21:53:450
169262300,3sleep20-21swapper/220:53:432
169262300,3sleep20-21swapper/220:53:432
169262300,3sleep20-21swapper/220:53:432
11543993025,4cyclictest999-21gltestperf21:38:401
11543993025,4cyclictest999-21gltestperf21:38:401
11543993025,4cyclictest999-21gltestperf21:38:401
11543993022,6cyclictest21857-21unixbench-2d23:58:481
11543993022,6cyclictest21857-21unixbench-2d23:58:481
11543993022,6cyclictest21857-21unixbench-2d23:58:481
11535993023,5cyclictest2850-21sshd00:36:170
11535993023,5cyclictest2850-21sshd00:36:170
11535993023,5cyclictest2850-21sshd00:36:170
11535993023,5cyclictest11523-21cyclictest19:25:390
11535993023,5cyclictest11523-21cyclictest19:25:390
11535993023,5cyclictest11523-21cyclictest19:25:390
11535993020,7cyclictest14968-21unixbench-2d22:13:470
11535993020,7cyclictest14968-21unixbench-2d22:13:470
11535993020,7cyclictest14968-21unixbench-2d22:13:470
11543992921,6cyclictest2859-21latency_hist20:18:371
11535992921,6cyclictest27810-21cron19:58:250
82102280,3sleep20-21swapper/220:29:432
82102280,3sleep20-21swapper/220:29:432
11543992823,3cyclictest0-21swapper/121:13:251
11543992823,3cyclictest0-21swapper/121:13:251
11543992822,5cyclictest6786-21kernelversion21:53:411
11543992822,5cyclictest6786-21kernelversion21:53:411
11543992822,5cyclictest14459-21sshd19:20:001
11543992822,4cyclictest572-21cups-browsed21:43:461
11543992822,4cyclictest572-21cups-browsed21:43:461
11543992821,5cyclictest8490-21gltestperf21:58:401
11543992820,6cyclictest550-21cron22:38:251
11543992820,6cyclictest550-21cron22:38:251
11543992820,6cyclictest550-21cron22:38:251
11543992818,8cyclictest13454-21cut23:38:391
11543992818,8cyclictest13454-21cut23:38:391
11535992823,3cyclictest0-21swapper/019:23:250
11535992822,4cyclictest0-21swapper/021:23:250
11535992822,4cyclictest0-21swapper/021:23:250
11535992819,7cyclictest5195-21apache222:05:200
11535992819,7cyclictest5195-21apache222:05:200
61592270,3sleep30-21swapper/323:18:403
11543992722,4cyclictest11523-21cyclictest19:23:481
11543992722,4cyclictest11523-21cyclictest19:23:481
11543992722,4cyclictest11523-21cyclictest19:23:481
11543992722,3cyclictest0-21swapper/123:08:481
11543992722,3cyclictest0-21swapper/123:08:481
11543992721,5cyclictest9711-21cut23:28:401
11543992721,5cyclictest9711-21cut23:28:401
11543992721,5cyclictest25206-21mailstats19:48:441
11543992721,5cyclictest25206-21mailstats19:48:441
11543992720,5cyclictest28893-21netstat00:18:441
11543992720,5cyclictest28893-21netstat00:18:441
11543992720,5cyclictest28893-21netstat00:18:441
11543992719,6cyclictest27219-21timerandwakeup19:53:481
11535992722,3cyclictest0-21swapper/000:28:380
11535992721,4cyclictest1926-21cpu23:08:370
11535992721,4cyclictest1926-21cpu23:08:370
11535992721,4cyclictest1926-21cpu23:08:370
11535992721,4cyclictest14678-21systemd-journal22:47:160
11535992721,4cyclictest14678-21systemd-journal22:47:160
11535992721,4cyclictest14678-21systemd-journal22:47:160
11535992720,5cyclictest14553-21sshd23:39:160
11535992720,5cyclictest14553-21sshd23:39:160
11535992718,7cyclictest9976-21kernelversion23:28:420
11535992718,7cyclictest9976-21kernelversion23:28:420
11535992718,7cyclictest2868-21idleruntime-cro20:18:250
11535992718,7cyclictest2868-21idleruntime-cro20:18:250
11535992718,7cyclictest16370-21unixbench_singl23:43:490
11535992718,7cyclictest16370-21unixbench_singl23:43:490
11535992718,7cyclictest16370-21unixbench_singl23:43:490
11552992617,7cyclictest351rcuc/220:13:282
11552992617,7cyclictest351rcuc/220:13:282
11552992617,7cyclictest351rcuc/220:13:282
11543992621,4cyclictest5631-21latency_hist23:18:351
11543992621,4cyclictest5631-21latency_hist23:18:351
11543992621,4cyclictest5631-21latency_hist23:18:351
11543992621,4cyclictest27267-21cut22:48:401
11543992621,4cyclictest27267-21cut22:48:401
11543992621,4cyclictest27267-21cut22:48:401
11543992621,3cyclictest30403-21cron22:58:261
11543992621,3cyclictest30403-21cron22:58:261
11543992621,3cyclictest30403-21cron22:58:261
11543992620,3cyclictest0-21swapper/120:58:501
11543992620,3cyclictest0-21swapper/120:58:501
11543992619,5cyclictest6496-21proc_pri23:18:461
11543992619,5cyclictest6496-21proc_pri23:18:461
11543992619,5cyclictest6496-21proc_pri23:18:461
11543992619,5cyclictest29128-21cron21:28:251
11543992619,5cyclictest29128-21cron21:28:251
11543992619,5cyclictest29128-21cron21:28:251
11543992619,5cyclictest19389-21/usr/sbin/munin22:28:431
11543992619,5cyclictest19389-21/usr/sbin/munin22:28:431
11543992618,6cyclictest5586-21unixbench-2d21:48:481
11543992618,6cyclictest5586-21unixbench-2d21:48:481
11543992618,6cyclictest1092-21threads23:03:491
11543992618,6cyclictest1092-21threads23:03:491
11543992618,6cyclictest1092-21threads23:03:491
11543992617,6cyclictest3012-21latency_hist20:18:331
11535992621,3cyclictest0-21swapper/000:08:440
11535992621,3cyclictest0-21swapper/000:08:440
11535992621,3cyclictest0-21swapper/000:08:440
11535992619,5cyclictest26013-21latency_hist19:53:280
11535992619,5cyclictest26013-21latency_hist19:53:280
11535992619,5cyclictest0-21swapper/023:53:380
11535992619,5cyclictest0-21swapper/023:53:380
11535992618,6cyclictest6070-21idleruntime-cro21:53:250
11535992618,6cyclictest6070-21idleruntime-cro21:53:250
11535992617,7cyclictest2625-21needreboot23:08:440
11535992617,7cyclictest2625-21needreboot23:08:440
11535992616,8cyclictest9306-21latency20:33:410
11535992616,8cyclictest9306-21latency20:33:410
11535992616,8cyclictest32617-21needreboot00:28:420
11535992616,8cyclictest32617-21needreboot00:28:420
11535992616,8cyclictest32617-21needreboot00:28:420
11552992517,7cyclictest351rcuc/223:29:342
11552992517,7cyclictest351rcuc/223:29:342
11543992520,3cyclictest0-21swapper/100:33:361
11543992520,3cyclictest0-21swapper/100:33:361
11543992520,3cyclictest0-21swapper/100:33:361
11543992519,5cyclictest817-21sshd20:11:161
11543992519,5cyclictest817-21sshd20:11:161
11543992519,5cyclictest817-21sshd20:11:161
11543992519,4cyclictest12260-21sendmail_mailtr23:33:471
11543992519,3cyclictest0-21swapper/120:18:441
11543992519,3cyclictest0-21swapper/120:18:441
11543992519,3cyclictest0-21swapper/120:18:441
11543992518,4cyclictest0-21swapper/121:28:421
11543992518,4cyclictest0-21swapper/121:28:421
11543992518,4cyclictest0-21swapper/121:28:421
11535992521,3cyclictest9781-21gltestperf23:28:400
11535992521,3cyclictest9781-21gltestperf23:28:400
11535992520,3cyclictest0-21swapper/021:28:470
11535992520,3cyclictest0-21swapper/021:28:470
11535992520,3cyclictest0-21swapper/021:28:470
1153599251,9cyclictest204872timerandwakeup22:28:470
1153599251,9cyclictest204872timerandwakeup22:28:470
11535992518,5cyclictest13924-21sshd20:45:170
11535992518,5cyclictest13924-21sshd20:45:170
11535992518,4cyclictest0-21swapper/021:33:450
11535992518,4cyclictest0-21swapper/021:33:450
11535992518,4cyclictest0-21swapper/021:33:450
122892240,1sleep112292-21plymouth20:43:251
122892240,1sleep112292-21plymouth20:43:251
11552992415,5cyclictest50462sleep220:23:362
11552992415,5cyclictest50462sleep220:23:362
11552992415,5cyclictest50462sleep220:23:362
11543992420,2cyclictest0-21swapper/119:43:401
11543992420,2cyclictest0-21swapper/119:43:401
11543992419,4cyclictest9018-21apache222:45:221
11543992419,4cyclictest9018-21apache222:45:221
11543992419,4cyclictest9018-21apache222:45:221
11543992419,4cyclictest16067-21idleruntime-cro20:53:261
11543992419,4cyclictest16067-21idleruntime-cro20:53:261
11543992419,3cyclictest0-21swapper/119:38:491
11543992419,3cyclictest0-21swapper/119:38:491
11543992419,3cyclictest0-21swapper/100:38:471
11543992419,2cyclictest0-21swapper/120:23:441
11543992419,2cyclictest0-21swapper/120:23:441
11543992418,5cyclictest25149-21needreboot00:08:491
11543992418,5cyclictest25149-21needreboot00:08:491
11543992418,5cyclictest25149-21needreboot00:08:491
11543992417,5cyclictest0-21swapper/122:13:361
11543992417,5cyclictest0-21swapper/122:13:361
11543992417,5cyclictest0-21swapper/122:13:361
11535992418,4cyclictest0-21swapper/019:28:480
11535992418,4cyclictest0-21swapper/019:28:480
11535992418,4cyclictest0-21swapper/019:28:480
11535992417,5cyclictest18611-21kernelversion20:58:420
11535992417,5cyclictest18611-21kernelversion20:58:420
11535992417,5cyclictest1648-21ntpd20:23:440
11535992417,5cyclictest1648-21ntpd20:23:440
70462230,4sleep17048-21fschecks_count20:28:381
70462230,4sleep17048-21fschecks_count20:28:381
275252230,2sleep20-21swapper/221:23:382
275252230,2sleep20-21swapper/221:23:382
275252230,2sleep20-21swapper/221:23:382
11543992319,3cyclictest32272-21idleruntime-cro23:03:261
11543992319,3cyclictest32272-21idleruntime-cro23:03:261
11543992318,3cyclictest0-21swapper/123:23:431
11543992318,3cyclictest0-21swapper/121:03:501
11543992318,3cyclictest0-21swapper/121:03:501
11543992318,3cyclictest0-21swapper/121:03:501
11543992317,3cyclictest0-21swapper/119:58:411
11535992319,3cyclictest27645-21ntp_offset22:48:450
11535992319,3cyclictest27645-21ntp_offset22:48:450
11535992319,3cyclictest27645-21ntp_offset22:48:450
11535992318,3cyclictest0-21swapper/020:53:470
11535992318,3cyclictest0-21swapper/020:53:470
11535992318,3cyclictest0-21swapper/020:53:470
11535992318,3cyclictest0-21swapper/020:38:500
11535992318,3cyclictest0-21swapper/020:38:500
11535992317,4cyclictest23510-21sh21:13:250
11535992317,4cyclictest23510-21sh21:13:250
11535992317,4cyclictest19445-21ntp_offset19:33:460
11535992317,4cyclictest19445-21ntp_offset19:33:460
11535992317,3cyclictest0-21swapper/000:23:390
11535992317,3cyclictest0-21swapper/000:23:390
11535992316,6cyclictest16022-21gltestperf22:18:400
11535992316,6cyclictest16022-21gltestperf22:18:400
11535992316,5cyclictest28473-21gltestperf00:18:390
11535992316,5cyclictest28473-21gltestperf00:18:390
11535992316,5cyclictest28473-21gltestperf00:18:390
11535992316,5cyclictest21706-21latency_hist21:08:290
11535992316,5cyclictest21706-21latency_hist21:08:290
11535992316,5cyclictest21706-21latency_hist21:08:290
188312220,3sleep20-21swapper/219:33:402
188312220,3sleep20-21swapper/219:33:402
148422220,3chrt0-21swapper/320:48:383
148422220,3chrt0-21swapper/320:48:383
1155299223,14cyclictest224632sleep200:03:302
1155299223,14cyclictest224632sleep200:03:302
1155299223,14cyclictest224632sleep200:03:302
11543992218,2cyclictest0-21swapper/119:28:421
11543992218,2cyclictest0-21swapper/119:28:421
11543992218,2cyclictest0-21swapper/119:28:421
11543992216,4cyclictest0-21swapper/122:23:391
11543992216,4cyclictest0-21swapper/122:23:391
11543992212,8cyclictest14678-21systemd-journal22:18:161
11543992212,8cyclictest14678-21systemd-journal22:18:161
11543992212,8cyclictest14678-21systemd-journal22:18:161
11543992212,8cyclictest14678-21systemd-journal00:18:261
11543992212,8cyclictest14678-21systemd-journal00:18:261
1153599227,13cyclictest23500-21sed22:38:390
1153599227,13cyclictest23500-21sed22:38:390
1153599227,13cyclictest23500-21sed22:38:390
11535992217,3cyclictest0-21swapper/023:23:390
11535992217,3cyclictest0-21swapper/023:23:390
11535992216,4cyclictest3205-21head21:43:430
11535992216,4cyclictest3205-21head21:43:430
11535992216,4cyclictest0-21swapper/022:08:480
11535992216,4cyclictest0-21swapper/022:08:480
11535992216,4cyclictest0-21swapper/022:08:480
11535992216,3cyclictest0-21swapper/019:58:410
11535992215,5cyclictest0-21swapper/020:03:350
11535992215,5cyclictest0-21swapper/020:03:350
11535992215,4cyclictest0-21swapper/022:53:480
11535992215,4cyclictest0-21swapper/022:53:480
11535992215,4cyclictest0-21swapper/022:53:480
1156699213,8cyclictest0-21swapper/300:03:423
1156699213,8cyclictest0-21swapper/300:03:423
1156699213,11cyclictest0-21swapper/319:29:163
1156699213,11cyclictest0-21swapper/319:29:163
1156699213,11cyclictest0-21swapper/319:29:163
11543992116,3cyclictest0-21swapper/120:53:361
11543992116,3cyclictest0-21swapper/120:53:361
11543992116,3cyclictest0-21swapper/120:53:361
11543992115,3cyclictest0-21swapper/120:33:371
11543992115,3cyclictest0-21swapper/120:33:371
1153599218,11cyclictest19374-21grep23:53:410
1153599218,11cyclictest19374-21grep23:53:410
11535992117,2cyclictest0-21swapper/020:53:250
11535992117,2cyclictest0-21swapper/020:53:250
11535992116,3cyclictest0-21swapper/000:14:170
11535992116,3cyclictest0-21swapper/000:14:170
11535992114,5cyclictest32268-21latency_hist23:03:260
11535992114,5cyclictest32268-21latency_hist23:03:260
11535992114,4cyclictest0-21swapper/021:24:050
11535992114,4cyclictest0-21swapper/021:24:050
11535992114,4cyclictest0-21swapper/021:24:050
11535992114,4cyclictest0-21swapper/020:28:380
11535992114,4cyclictest0-21swapper/020:28:380
11535992114,4cyclictest0-21swapper/019:48:360
11535992114,4cyclictest0-21swapper/019:48:360
63672200,3sleep20-21swapper/223:18:432
63672200,3sleep20-21swapper/223:18:432
63672200,3sleep20-21swapper/223:18:432
255992200,3chrt0-21swapper/221:18:362
11543992015,3cyclictest0-21swapper/121:18:401
11543992015,3cyclictest0-21swapper/121:18:401
11535992015,3cyclictest0-21swapper/023:23:480
11535992015,3cyclictest0-21swapper/023:23:250
11535992015,3cyclictest0-21swapper/022:33:480
11535992015,3cyclictest0-21swapper/022:33:480
11535992015,3cyclictest0-21swapper/022:33:480
11535992014,3cyclictest0-21swapper/019:38:410
11535992014,3cyclictest0-21swapper/019:38:410
1156699193,9cyclictest0-21swapper/323:02:173
1156699193,9cyclictest0-21swapper/323:02:173
1156699193,9cyclictest0-21swapper/321:45:093
1156699193,9cyclictest0-21swapper/321:45:093
1155299196,8cyclictest0-21swapper/200:03:512
1155299196,8cyclictest0-21swapper/200:03:512
11535991914,3cyclictest0-21swapper/000:38:460
1156699183,9cyclictest0-21swapper/320:34:163
1156699183,9cyclictest0-21swapper/320:34:163
1156699181,12cyclictest19589-21taskset19:33:493
1156699181,12cyclictest19589-21taskset19:33:493
1156699173,9cyclictest0-21swapper/323:54:163
1156699173,9cyclictest0-21swapper/323:54:163
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional