You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-19 - 11:29

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #4, slot #1

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1s.osadl.org (updated Sun Jul 19, 2026 00:44:19)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
151442880,3chrt0-21swapper/321:13:493
151442880,3chrt0-21swapper/321:13:493
111082830,3sleep20-21swapper/222:31:592
111082830,3sleep20-21swapper/222:31:592
99382810,3chrt0-21swapper/323:53:493
99382810,3chrt0-21swapper/323:53:493
99382810,3chrt0-21swapper/323:53:493
127022650,2sleep10-21swapper/100:03:361
127022650,2sleep10-21swapper/100:03:361
127022650,2sleep10-21swapper/100:03:361
127022650,2sleep10-21swapper/100:03:361
63102640,3sleep30-21swapper/323:43:483
63102640,3sleep30-21swapper/323:43:483
152312570,1sleep015302-21gzip00:08:420
152312570,1sleep015302-21gzip00:08:420
152312570,1sleep015302-21gzip00:08:420
150822560,2sleep30-21swapper/322:43:293
150822560,2sleep30-21swapper/322:43:293
150822560,2sleep30-21swapper/322:43:293
25562540,2sleep129-21ksoftirqd/123:33:491
25562540,2sleep129-21ksoftirqd/123:33:491
76792530,5sleep30-21swapper/319:28:453
76792530,5sleep30-21swapper/319:28:453
76792530,5sleep30-21swapper/319:28:453
160399491,34cyclictest14226-21chrt00:06:000
160399491,34cyclictest14226-21chrt00:06:000
160399491,34cyclictest14226-21chrt00:06:000
160399480,46cyclictest700-21192.168.115.40-manager23:17:190
160399480,46cyclictest700-21192.168.115.40-manager23:17:190
160399480,46cyclictest700-21192.168.115.40-manager23:17:190
160399480,32cyclictest7045-21taskset23:48:340
160399480,32cyclictest7045-21taskset23:48:340
96692470,3sleep20-21swapper/223:53:432
96692470,3sleep20-21swapper/223:53:432
96692470,3sleep20-21swapper/223:53:432
160399471,32cyclictest12920-21chrt22:36:190
160399471,32cyclictest12920-21chrt22:36:190
160399471,30cyclictest20231-21seq22:53:560
160399471,30cyclictest20231-21seq22:53:560
160399471,30cyclictest20231-21seq22:53:560
299652460,3sleep00-21swapper/020:28:390
299652460,3sleep00-21swapper/020:28:390
160399460,44cyclictest700-21192.168.115.40-manager19:45:290
160399460,44cyclictest700-21192.168.115.40-manager19:45:290
160399451,42cyclictest700-21192.168.115.40-manager20:00:140
160399451,29cyclictest9825-21seq21:02:200
160399451,29cyclictest9825-21seq21:02:200
160399451,29cyclictest15825-21taskset19:49:280
160399451,29cyclictest15825-21taskset19:49:280
160399450,43cyclictest700-21192.168.115.40-manager23:12:330
160399450,43cyclictest700-21192.168.115.40-manager23:12:330
160399450,43cyclictest700-21192.168.115.40-manager21:12:460
160399450,43cyclictest700-21192.168.115.40-manager21:12:460
116802450,3sleep00-21swapper/022:33:380
115202450,3sleep30-21swapper/322:33:383
115202450,3sleep30-21swapper/322:33:383
160399440,41cyclictest700-21192.168.115.40-manager23:36:500
160399440,41cyclictest700-21192.168.115.40-manager23:36:500
160399440,41cyclictest700-21192.168.115.40-manager22:39:480
160399440,41cyclictest700-21192.168.115.40-manager22:39:480
160399440,41cyclictest700-21192.168.115.40-manager22:39:480
160399440,32cyclictest32235-21mailstats20:33:450
160399440,27cyclictest7373-21/usr/sbin/munin19:28:410
160399440,27cyclictest7373-21/usr/sbin/munin19:28:410
160399440,27cyclictest7373-21/usr/sbin/munin19:28:410
239262430,2sleep00-21swapper/021:38:420
239262430,2sleep00-21swapper/021:38:420
239262430,2sleep00-21swapper/021:38:420
160399431,40cyclictest0-21swapper/021:18:160
160399431,40cyclictest0-21swapper/021:18:160
160399431,27cyclictest18402-21taskset22:49:200
160399431,27cyclictest18402-21taskset22:49:200
160399431,27cyclictest18402-21taskset22:49:200
160399430,41cyclictest700-21192.168.115.40-manager00:42:210
160399430,41cyclictest700-21192.168.115.40-manager00:42:210
149862430,2sleep114985-21irqstats19:48:401
149862430,2sleep114985-21irqstats19:48:401
160399421,27cyclictest25270-21chrt20:15:570
160399421,27cyclictest25270-21chrt20:15:570
160399420,40cyclictest700-21192.168.115.40-manager00:28:060
160399420,40cyclictest700-21192.168.115.40-manager00:28:060
160399420,39cyclictest700-21192.168.115.40-manager23:41:500
160399420,39cyclictest700-21192.168.115.40-manager23:41:500
160399420,27cyclictest11694-21taskset21:07:070
160399420,27cyclictest11694-21taskset21:07:070
289452410,2sleep30-21swapper/323:18:433
289452410,2sleep30-21swapper/323:18:433
289452410,2sleep30-21swapper/323:18:433
160399411,37cyclictest0-21swapper/000:13:510
160399411,37cyclictest0-21swapper/000:13:510
160399411,35cyclictest0-21swapper/022:07:480
160399411,35cyclictest0-21swapper/022:07:480
160399411,35cyclictest0-21swapper/022:07:480
160399411,27cyclictest7671-21chrt20:53:560
160399411,27cyclictest7671-21chrt20:53:560
160399410,39cyclictest700-21192.168.115.40-manager21:47:170
160399410,39cyclictest700-21192.168.115.40-manager21:47:170
160399410,39cyclictest700-21192.168.115.40-manager21:47:170
160399410,39cyclictest700-21192.168.115.40-manager21:22:160
160399410,39cyclictest700-21192.168.115.40-manager21:22:160
223122400,4sleep30-21swapper/300:28:383
223122400,4sleep30-21swapper/300:28:383
160399402,35cyclictest12558-40mandb00:03:300
160399402,35cyclictest12558-40mandb00:03:300
160399402,35cyclictest12558-40mandb00:03:300
160399402,35cyclictest12558-40mandb00:03:300
160399401,37cyclictest21354-21/usr/sbin/munin21:33:470
160399401,37cyclictest21354-21/usr/sbin/munin21:33:470
160399401,37cyclictest21354-21/usr/sbin/munin21:33:470
160399401,37cyclictest21354-21/usr/sbin/munin21:33:470
160399401,37cyclictest0-21swapper/021:33:170
160399401,37cyclictest0-21swapper/021:33:170
160399401,36cyclictest0-21swapper/023:28:500
160399401,36cyclictest0-21swapper/023:28:500
160399400,38cyclictest700-21192.168.115.40-manager21:27:160
160399400,38cyclictest700-21192.168.115.40-manager21:27:160
27372390,2sleep02738-21/usr/sbin/munin22:08:410
27372390,2sleep02738-21/usr/sbin/munin22:08:410
160399391,35cyclictest0-21swapper/000:23:210
160399391,35cyclictest0-21swapper/000:23:210
160399391,35cyclictest0-21swapper/000:23:210
160399390,37cyclictest700-21192.168.115.40-manager20:07:140
160399390,37cyclictest700-21192.168.115.40-manager20:07:140
160399390,36cyclictest8523-21taskset23:50:320
160399390,36cyclictest8523-21taskset23:50:320
160399390,36cyclictest8523-21taskset23:50:320
160399390,36cyclictest700-21192.168.115.40-manager23:20:190
160399390,36cyclictest700-21192.168.115.40-manager23:20:190
160399390,36cyclictest700-21192.168.115.40-manager23:20:190
182962380,3sleep30-21swapper/300:18:343
182962380,3sleep30-21swapper/300:18:343
160399381,35cyclictest0-21swapper/023:02:190
160399381,35cyclictest0-21swapper/023:02:190
160399381,35cyclictest0-21swapper/022:29:180
160399381,35cyclictest0-21swapper/022:25:180
160399381,35cyclictest0-21swapper/022:25:180
160399381,35cyclictest0-21swapper/022:25:180
160399381,34cyclictest27142-21fschecks_count21:48:400
160399381,34cyclictest27142-21fschecks_count21:48:400
160399381,34cyclictest27142-21fschecks_count21:48:400
160399381,34cyclictest0-21swapper/020:25:450
160399381,34cyclictest0-21swapper/020:25:450
160399381,34cyclictest0-21swapper/019:41:290
160399381,34cyclictest0-21swapper/019:41:290
160399381,34cyclictest0-21swapper/019:41:290
160399380,29cyclictest27304-21taskset20:23:040
160399380,29cyclictest27304-21taskset20:23:040
160399380,29cyclictest27304-21taskset20:23:040
230242370,2sleep023134-21timerandwakeup00:28:480
230242370,2sleep023134-21timerandwakeup00:28:480
1603993731,4cyclictest729-21dbus-daemon22:03:240
1603993731,4cyclictest729-21dbus-daemon22:03:240
1603993731,4cyclictest729-21dbus-daemon22:03:240
160399371,34cyclictest0-21swapper/023:54:500
160399371,34cyclictest0-21swapper/023:54:500
160399371,34cyclictest0-21swapper/023:54:500
160399371,33cyclictest2636-21/usr/sbin/munin20:43:450
160399371,33cyclictest2636-21/usr/sbin/munin20:43:450
160399371,33cyclictest15138-21/usr/sbin/munin22:43:480
160399371,33cyclictest15138-21/usr/sbin/munin22:43:480
160399371,33cyclictest15138-21/usr/sbin/munin22:43:480
160399371,33cyclictest0-21swapper/019:54:480
160399371,26cyclictest0-21swapper/023:05:240
160399371,26cyclictest0-21swapper/023:05:240
37482360,3sleep30-21swapper/323:38:413
37482360,3sleep30-21swapper/323:38:413
1605993631,3cyclictest18863-21kworker/u9:2-xprtiod23:23:251
1605993631,3cyclictest18863-21kworker/u9:2-xprtiod23:23:251
1605993631,3cyclictest18863-21kworker/u9:2-xprtiod23:23:251
160399361,33cyclictest0-21swapper/022:16:480
160399361,33cyclictest0-21swapper/022:16:480
160399361,33cyclictest0-21swapper/021:53:470
160399361,33cyclictest0-21swapper/021:53:470
160399361,32cyclictest31864-21date23:28:280
160399361,32cyclictest31864-21date23:28:280
160399361,32cyclictest25421-21sshd00:35:580
160399361,32cyclictest25421-21sshd00:35:580
160399361,32cyclictest25421-21sshd00:35:580
160399361,32cyclictest2196-21sshd20:40:580
160399361,32cyclictest10285-21sshd19:35:410
160399361,32cyclictest10285-21sshd19:35:410
1605993527,6cyclictest16795-21threads21:18:501
1605993527,6cyclictest16795-21threads21:18:501
160399351,31cyclictest4624-21/usr/sbin/munin20:48:360
160399351,31cyclictest4624-21/usr/sbin/munin20:48:360
160399351,31cyclictest21943-21/usr/sbin/munin20:08:480
160399351,31cyclictest21943-21/usr/sbin/munin20:08:480
1605993429,3cyclictest11767-21gltestperf22:33:381
160399341,30cyclictest7004-21unixbench_multi22:18:450
160399341,30cyclictest7004-21unixbench_multi22:18:450
1605993326,5cyclictest1602-21cyclictest20:58:301
1605993326,5cyclictest1602-21cyclictest20:58:301
1605993325,6cyclictest19868-21sendmail20:03:251
110432330,3chrt0-21swapper/121:03:461
110432330,3chrt0-21swapper/121:03:461
90582320,7sleep19061-21fschecks_count19:33:371
90582320,7sleep19061-21fschecks_count19:33:371
1605993225,6cyclictest7933-21cut22:23:391
1605993225,6cyclictest7933-21cut22:23:391
1605993225,6cyclictest7933-21cut22:23:391
1605993126,4cyclictest12385-21ls19:43:241
1605993126,4cyclictest12385-21ls19:43:241
1605993126,4cyclictest12385-21ls19:43:241
1605993123,6cyclictest13986-21unixbench_multi00:03:501
1605993123,6cyclictest13986-21unixbench_multi00:03:501
1605993123,6cyclictest13986-21unixbench_multi00:03:501
1605993022,6cyclictest2288-21cut22:08:371
180252290,4sleep10-21swapper/119:58:251
160699292,21cyclictest12659-21chrt00:03:322
160699292,21cyclictest12659-21chrt00:03:322
160699292,21cyclictest12659-21chrt00:03:322
160699292,21cyclictest12659-21chrt00:03:322
1605992923,3cyclictest0-21swapper/123:48:251
1605992923,3cyclictest0-21swapper/123:48:251
1605992922,5cyclictest9677-21needreboot23:53:441
1605992922,5cyclictest9677-21needreboot23:53:441
1605992922,5cyclictest9677-21needreboot23:53:441
1605992922,5cyclictest2729-21needreboot22:08:411
1605992922,5cyclictest2729-21needreboot22:08:411
1605992922,5cyclictest24328-21kernelversion20:13:421
1605992922,5cyclictest24328-21kernelversion20:13:421
1605992921,6cyclictest20960-21netstat00:23:441
1605992921,6cyclictest20960-21netstat00:23:441
1605992823,4cyclictest3725-21idleruntime23:38:411
1605992823,4cyclictest3725-21idleruntime23:38:411
1605992823,3cyclictest0-21swapper/122:03:401
1605992823,3cyclictest0-21swapper/122:03:401
1605992823,3cyclictest0-21swapper/100:38:371
1605992823,3cyclictest0-21swapper/100:28:381
1605992823,3cyclictest0-21swapper/100:28:381
1605992822,5cyclictest29541-21needreboot21:53:421
1605992822,5cyclictest29541-21needreboot21:53:421
1605992821,5cyclictest1602-21cyclictest23:28:341
1605992821,5cyclictest1602-21cyclictest23:28:341
1605992820,6cyclictest13431-21cut22:38:371
1605992820,6cyclictest13431-21cut22:38:371
1605992723,2cyclictest0-21swapper/121:33:371
1605992721,4cyclictest15133-21kernelversion00:08:401
1605992721,4cyclictest15133-21kernelversion00:08:401
1605992721,4cyclictest15133-21kernelversion00:08:401
1605992719,6cyclictest23624-21cut23:03:471
1605992719,6cyclictest23624-21cut23:03:471
1605992719,6cyclictest1804-21ls20:38:471
160799264,11cyclictest0-21swapper/300:03:313
160799264,11cyclictest0-21swapper/300:03:313
160799264,11cyclictest0-21swapper/300:03:313
160799264,11cyclictest0-21swapper/300:03:313
1605992621,4cyclictest11239-21wc22:33:251
1605992621,3cyclictest0-21swapper/120:48:351
1605992620,5cyclictest31926-21missed_timers20:33:401
1605992619,5cyclictest18852-21kernelversion00:18:401
1605992619,5cyclictest18852-21kernelversion00:18:401
1605992619,5cyclictest18852-21kernelversion00:18:401
1605992619,5cyclictest18408-21cpuspeed00:18:361
1605992619,5cyclictest18408-21cpuspeed00:18:361
1605992619,4cyclictest0-21swapper/123:08:411
1605992619,4cyclictest0-21swapper/123:08:411
1605992617,7cyclictest19360-21latency_hist21:28:271
1605992617,7cyclictest19360-21latency_hist21:28:271
1605992521,3cyclictest14978-21latency_hist22:43:251
1605992521,3cyclictest14978-21latency_hist22:43:251
1605992521,3cyclictest14978-21latency_hist22:43:251
1605992520,3cyclictest25176-21latency_hist21:43:331
1605992520,3cyclictest25176-21latency_hist21:43:331
1605992520,3cyclictest25176-21latency_hist21:43:331
1605992520,3cyclictest0-21swapper/100:33:451
1605992520,3cyclictest0-21swapper/100:33:451
1605992519,4cyclictest14678-21systemd-journal22:46:581
1605992519,4cyclictest14678-21systemd-journal22:46:581
1605992519,4cyclictest14678-21systemd-journal22:46:581
1605992519,4cyclictest0-21swapper/121:38:361
1605992519,4cyclictest0-21swapper/121:38:361
1605992519,4cyclictest0-21swapper/121:38:361
1605992518,6cyclictest20391-21rt-features21:28:471
1605992518,5cyclictest27124-21ntp_kernel_err23:13:421
1605992518,5cyclictest27124-21ntp_kernel_err23:13:421
1605992518,5cyclictest27124-21ntp_kernel_err23:13:421
1605992518,4cyclictest0-21swapper/120:23:241
1605992518,4cyclictest0-21swapper/120:23:241
1605992518,4cyclictest0-21swapper/120:23:241
1605992511,12cyclictest14678-21systemd-journal21:08:371
1605992511,12cyclictest14678-21systemd-journal21:08:371
1606992418,4cyclictest351rcuc/221:42:502
1606992418,4cyclictest351rcuc/221:42:502
1606992418,4cyclictest351rcuc/221:42:502
1605992420,2cyclictest0-21swapper/122:58:471
1605992420,2cyclictest0-21swapper/122:58:471
1605992420,2cyclictest0-21swapper/120:48:461
1605992420,2cyclictest0-21swapper/120:48:461
1605992419,3cyclictest6119-21grep22:18:361
1605992419,3cyclictest6119-21grep22:18:361
1605992417,5cyclictest31437-21ntp_kernel_err21:58:431
1605992417,5cyclictest31437-21ntp_kernel_err21:58:431
1605992417,5cyclictest31437-21ntp_kernel_err21:58:431
1605992416,5cyclictest0-21swapper/120:23:441
1605992416,5cyclictest0-21swapper/120:23:441
305672230,3sleep30-21swapper/320:28:463
305672230,3sleep30-21swapper/320:28:463
252112230,2sleep30-21swapper/323:08:423
252112230,2sleep30-21swapper/323:08:423
223462230,2sleep10-21swapper/120:08:381
223462230,2sleep10-21swapper/120:08:381
160699236,11cyclictest0-21swapper/223:18:462
160699236,11cyclictest0-21swapper/223:18:462
160699236,11cyclictest0-21swapper/223:18:462
1606992318,3cyclictest351rcuc/223:07:452
1606992318,3cyclictest351rcuc/223:07:452
1605992319,2cyclictest0-21swapper/122:53:401
1605992319,2cyclictest0-21swapper/122:53:401
1605992319,2cyclictest0-21swapper/121:45:471
1605992319,2cyclictest0-21swapper/121:45:471
1605992319,2cyclictest0-21swapper/121:45:471
1605992319,2cyclictest0-21swapper/119:28:491
1605992319,2cyclictest0-21swapper/119:28:491
1605992319,2cyclictest0-21swapper/119:28:491
1605992317,4cyclictest0-21swapper/121:48:401
1605992317,4cyclictest0-21swapper/121:48:401
1605992317,4cyclictest0-21swapper/121:48:401
1605992316,5cyclictest19600-21needreboot22:53:431
1605992316,5cyclictest19600-21needreboot22:53:431
1605992316,5cyclictest19600-21needreboot22:53:431
1605992315,6cyclictest9091-21sort23:53:381
1605992315,6cyclictest9091-21sort23:53:381
1605992218,2cyclictest0-21swapper/120:03:461
1605992218,2cyclictest0-21swapper/120:03:461
1605992218,2cyclictest0-21swapper/119:43:381
1605992218,2cyclictest0-21swapper/119:43:381
1605992217,3cyclictest0-21swapper/122:18:381
1605992217,3cyclictest0-21swapper/122:18:381
1605992217,3cyclictest0-21swapper/100:38:381
1605992217,3cyclictest0-21swapper/100:38:381
1605992216,4cyclictest0-21swapper/122:48:491
1605992216,4cyclictest0-21swapper/120:28:441
1605992216,4cyclictest0-21swapper/120:28:441
160599221,4cyclictest271rcuc/121:13:391
160599221,4cyclictest271rcuc/121:13:391
1605992213,7cyclictest8522-21seq23:50:321
1606992117,3cyclictest351rcuc/222:23:032
1606992117,3cyclictest351rcuc/222:23:032
1605992116,3cyclictest0-21swapper/121:33:461
1605992116,3cyclictest0-21swapper/120:58:461
1605992116,3cyclictest0-21swapper/120:58:461
1605992112,7cyclictest2867-21cut20:43:381
1607992014,4cyclictest191712chrt22:53:383
1607992014,4cyclictest191712chrt22:53:383
1606992016,3cyclictest351rcuc/223:44:552
1606992016,3cyclictest351rcuc/223:44:552
137302200,2sleep20-21swapper/200:03:492
137302200,2sleep20-21swapper/200:03:492
137302200,2sleep20-21swapper/200:03:492
1606991915,3cyclictest351rcuc/220:57:372
1606991915,3cyclictest351rcuc/220:57:372
1606991915,3cyclictest351rcuc/219:47:502
1606991915,3cyclictest351rcuc/219:47:502
1606991914,4cyclictest351rcuc/220:58:412
1606991914,4cyclictest351rcuc/220:58:412
1606991914,3cyclictest351rcuc/222:01:192
1606991914,3cyclictest351rcuc/222:01:192
1606991914,3cyclictest351rcuc/222:01:192
39122181,2sleep20-21swapper/220:43:502
39122181,2sleep20-21swapper/220:43:502
1606991814,3cyclictest351rcuc/221:13:492
1606991814,3cyclictest351rcuc/221:13:492
1606991814,3cyclictest351rcuc/220:38:592
1606991813,4cyclictest351rcuc/222:09:002
1606991813,4cyclictest351rcuc/222:09:002
1606991813,3cyclictest351rcuc/223:25:302
1606991813,3cyclictest351rcuc/223:25:302
1606991812,4cyclictest351rcuc/222:54:582
1606991812,4cyclictest351rcuc/222:54:582
1606991812,4cyclictest351rcuc/222:54:582
1605991814,2cyclictest0-21swapper/123:28:411
1605991814,2cyclictest0-21swapper/123:28:411
160799177,7cyclictest10207-21chrt21:03:363
160799171,13cyclictest31445-21chrt21:58:433
160799171,13cyclictest31445-21chrt21:58:433
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional