You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-25 - 08:50

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #4, slot #1

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1s.osadl.org (updated Sat Jul 25, 2026 00:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1650821440,3sleep20-21swapper/220:16:202
1650821440,3sleep20-21swapper/220:16:202
904721140,3chrt0-21swapper/321:23:423
904721140,3chrt0-21swapper/321:23:423
904721140,3chrt0-21swapper/321:23:423
1879121120,2sleep118794-21sed21:48:451
1879121120,2sleep118794-21sed21:48:451
1879121120,2sleep118794-21sed21:48:451
272042900,4sleep30-21swapper/320:43:513
272042900,4sleep30-21swapper/320:43:513
169652740,6sleep116969-21unixbench-2d23:08:531
169652740,6sleep116969-21unixbench-2d23:08:531
169652740,6sleep116969-21unixbench-2d23:08:531
169652740,6sleep116969-21unixbench-2d23:08:531
235092730,2chrt23512-21switchtime20:33:521
209882650,3chrt0-21swapper/123:20:551
209882650,3chrt0-21swapper/123:20:551
209882650,3chrt0-21swapper/123:20:551
84442640,1chrt0-21swapper/100:13:491
84442640,1chrt0-21swapper/100:13:491
36492610,3sleep30-21swapper/321:08:433
36492610,3sleep30-21swapper/321:08:433
35642610,3sleep125387-21cyclictest22:33:511
35642610,3sleep125387-21cyclictest22:33:511
18682610,2sleep10-21swapper/119:38:401
18682610,2sleep10-21swapper/119:38:401
246602560,2chrt0-21swapper/220:38:432
39332540,2sleep30-21swapper/319:43:413
39332540,2sleep30-21swapper/319:43:413
39332540,2sleep30-21swapper/319:43:413
34772530,1sleep03563-21/usr/sbin/munin22:33:510
34772530,1sleep03563-21/usr/sbin/munin22:33:510
37962510,5sleep13795-40mandb00:03:321
37962510,5sleep13795-40mandb00:03:321
37962510,5sleep13795-40mandb00:03:321
241992510,3sleep20-21swapper/220:38:322
19892490,3sleep30-21swapper/321:03:493
19892490,3sleep30-21swapper/321:03:493
177082490,3chrt0-21swapper/323:13:413
177082490,3chrt0-21swapper/323:13:413
177082490,3chrt0-21swapper/323:13:413
177082490,3chrt0-21swapper/323:13:413
114432460,3sleep30-21swapper/321:28:493
114432460,3sleep30-21swapper/321:28:493
25401994538,3cyclictest0-21swapper/000:03:530
25401994538,3cyclictest0-21swapper/000:03:530
25401994538,3cyclictest0-21swapper/000:03:530
64772440,2sleep30-21swapper/322:43:423
64772440,2sleep30-21swapper/322:43:423
64772440,2sleep30-21swapper/322:43:423
298972430,2sleep30-21swapper/320:53:353
2541099403,26cyclictest3874-21chrt00:03:383
2541099403,26cyclictest3874-21chrt00:03:383
2541099403,26cyclictest3874-21chrt00:03:383
25405993913,19cyclictest5033-21cat00:03:551
25405993913,19cyclictest5033-21cat00:03:551
25405993913,19cyclictest5033-21cat00:03:551
255532380,3chrt0-21swapper/123:33:451
255532380,3chrt0-21swapper/123:33:451
25405993731,4cyclictest572-21cups-browsed23:13:511
25405993731,4cyclictest572-21cups-browsed23:13:511
25405993731,4cyclictest572-21cups-browsed23:13:511
47782360,2sleep14964-21grep19:43:521
47782360,2sleep14964-21grep19:43:521
47782360,2sleep14964-21grep19:43:521
310362360,3chrt0-21swapper/319:28:453
310362360,3chrt0-21swapper/319:28:453
310362360,3chrt0-21swapper/319:28:453
25405993627,7cyclictest16848-21kernelversion21:43:451
25405993627,7cyclictest16848-21kernelversion21:43:451
25405993627,7cyclictest16848-21kernelversion21:43:451
25405993627,7cyclictest16848-21kernelversion21:43:451
25401993528,5cyclictest3884-21netstat21:08:470
25401993528,5cyclictest3884-21netstat21:08:470
242992350,3sleep30-21swapper/320:38:373
25405993425,7cyclictest2710-21fschecks_count22:33:421
25405993425,7cyclictest2710-21fschecks_count22:33:421
100952340,2chrt10094-21irqstats00:18:441
100952340,2chrt10094-21irqstats00:18:441
100952340,2chrt10094-21irqstats00:18:441
63232330,3sleep30-21swapper/319:48:473
63232330,3sleep30-21swapper/319:48:473
25405993327,3cyclictest14032-21kworker/u9:2-rpciod22:03:431
25405993324,7cyclictest23222-21/usr/sbin/munin23:28:481
25405993324,7cyclictest23222-21/usr/sbin/munin23:28:481
25405993323,8cyclictest28891-21fschecks_count19:23:411
25405993323,8cyclictest28891-21fschecks_count19:23:411
25401993326,5cyclictest29207-21latency_hist22:18:300
25401993326,5cyclictest29207-21latency_hist22:18:300
25401993326,5cyclictest27339-21latency_hist22:13:300
25401993326,5cyclictest27339-21latency_hist22:13:300
25401993324,7cyclictest2782-21netstat23:58:480
25401993324,7cyclictest2782-21netstat23:58:480
25401993324,7cyclictest2782-21netstat23:58:480
25401993224,6cyclictest1525-21fschecks_time21:03:450
25401993224,6cyclictest1525-21fschecks_time21:03:450
25405993125,5cyclictest14678-21systemd-journal22:45:581
25405993125,5cyclictest14678-21systemd-journal22:45:581
25405993125,5cyclictest14678-21systemd-journal22:45:581
25405993125,4cyclictest25387-21cyclictest21:38:421
25405993123,6cyclictest22478-21latency_hist20:33:371
25405993123,6cyclictest22478-21latency_hist20:33:371
25401993126,4cyclictest22879-21gltestperf20:33:430
189472310,4sleep30-21swapper/323:13:533
189472310,4sleep30-21swapper/323:13:533
189472310,4sleep30-21swapper/323:13:533
25405993025,3cyclictest25387-21cyclictest21:13:531
25405993025,3cyclictest25387-21cyclictest21:13:531
25405993025,3cyclictest0-21swapper/121:08:511
25405993025,3cyclictest0-21swapper/121:08:511
25405993024,4cyclictest550-21cron21:33:441
25401993024,4cyclictest30823-21cpuspeed23:48:410
25401993024,4cyclictest30823-21cpuspeed23:48:410
25401993024,4cyclictest30823-21cpuspeed23:48:410
25401993024,4cyclictest27564-21ntp_offset19:18:490
25401993024,4cyclictest27564-21ntp_offset19:18:490
25401993023,5cyclictest5127-21sshd19:44:570
25401993023,5cyclictest5127-21sshd19:44:570
25401993023,5cyclictest5127-21sshd19:44:570
25405992923,4cyclictest0-21swapper/119:38:301
25405992922,5cyclictest7625-21latency_hist00:13:371
25405992922,5cyclictest7625-21latency_hist00:13:371
25405992922,5cyclictest7625-21latency_hist00:13:371
25405992920,7cyclictest16228-21latency_hist21:43:351
25405992920,7cyclictest16228-21latency_hist21:43:351
25405992920,7cyclictest16228-21latency_hist21:43:351
25405992920,7cyclictest13292-21latency_hist00:28:381
25405992920,7cyclictest13292-21latency_hist00:28:381
25405992920,7cyclictest13292-21latency_hist00:28:381
25401992921,6cyclictest7690-21fschecks_count19:53:420
25401992921,6cyclictest7690-21fschecks_count19:53:420
108552290,3chrt0-21swapper/322:53:473
108552290,3chrt0-21swapper/322:53:473
108552290,3chrt0-21swapper/322:53:473
25405992823,4cyclictest25387-21cyclictest20:56:361
25405992823,4cyclictest25387-21cyclictest20:56:361
25401992820,6cyclictest17852-21ntp_offset00:38:500
25401992820,6cyclictest17852-21ntp_offset00:38:500
25401992820,5cyclictest31246-21needreboot19:28:490
25401992820,5cyclictest31246-21needreboot19:28:490
25401992820,5cyclictest31246-21needreboot19:28:490
224572280,4sleep122461-21timerandwakeup23:23:521
224572280,4sleep122461-21timerandwakeup23:23:521
224572280,4sleep122461-21timerandwakeup23:23:521
78432270,2sleep30-21swapper/300:13:433
78432270,2sleep30-21swapper/300:13:433
322302271,1sleep00-21swapper/023:51:100
322302271,1sleep00-21swapper/023:51:100
322302271,1sleep00-21swapper/023:51:100
25405992722,3cyclictest6310-21needreboot19:48:481
25405992722,3cyclictest6310-21needreboot19:48:481
25405992722,3cyclictest25387-21cyclictest21:23:421
25405992722,3cyclictest25387-21cyclictest21:23:421
25405992722,3cyclictest25387-21cyclictest21:23:421
25405992721,4cyclictest15325-21gltestperf20:13:431
25405992721,4cyclictest15325-21gltestperf20:13:431
25405992721,4cyclictest0-21swapper/120:48:421
25405992721,3cyclictest0-21swapper/120:08:421
25405992721,3cyclictest0-21swapper/120:08:421
25405992720,5cyclictest27484-21needreboot19:18:481
25405992720,5cyclictest27484-21needreboot19:18:481
25405992720,5cyclictest24598-21fschecks_time20:38:431
25405992720,4cyclictest5273-21ntp_kernel_pll_22:38:461
25405992720,4cyclictest5273-21ntp_kernel_pll_22:38:461
25405992720,4cyclictest5273-21ntp_kernel_pll_22:38:461
25405992718,7cyclictest26830-21cat20:43:441
25405992718,7cyclictest26830-21cat20:43:441
25401992722,3cyclictest1085-21latency_hist21:03:300
25401992722,3cyclictest1085-21latency_hist21:03:300
25401992722,3cyclictest1085-21latency_hist21:03:300
25401992722,3cyclictest0-21swapper/000:33:300
25401992722,3cyclictest0-21swapper/000:33:300
25401992722,3cyclictest0-21swapper/000:33:300
25401992721,4cyclictest10285-21mailstats19:58:510
25401992721,4cyclictest10285-21mailstats19:58:510
25401992719,6cyclictest11868-21gltestperf00:23:450
25401992719,6cyclictest11868-21gltestperf00:23:450
25401992719,6cyclictest11868-21gltestperf00:23:450
25405992620,3cyclictest0-21swapper/122:28:401
25405992620,3cyclictest0-21swapper/122:28:401
25405992618,6cyclictest21764-21latency_hist21:58:321
25405992618,6cyclictest21764-21latency_hist21:58:321
25405992618,6cyclictest21764-21latency_hist21:58:321
25405992616,7cyclictest18917-21latency_hist00:43:381
25405992616,7cyclictest18917-21latency_hist00:43:381
25401992622,3cyclictest547-21rs:main0
25401992622,3cyclictest547-21rs:main0
25401992622,3cyclictest547-21rs:main0
25401992622,2cyclictest0-21swapper/000:23:300
25401992622,2cyclictest0-21swapper/000:23:300
25401992622,2cyclictest0-21swapper/000:23:300
25401992621,4cyclictest28280-21fschecks_count20:48:420
25401992621,3cyclictest0-21swapper/021:33:520
25401992621,3cyclictest0-21swapper/021:33:520
25401992619,4cyclictest0-21swapper/023:13:520
25401992619,4cyclictest0-21swapper/023:13:520
25401992619,4cyclictest0-21swapper/023:13:520
25405992520,3cyclictest0-21swapper/123:03:501
25405992520,3cyclictest0-21swapper/123:03:501
25405992520,3cyclictest0-21swapper/123:03:501
25405992518,5cyclictest25387-21cyclictest00:29:141
25405992518,5cyclictest25387-21cyclictest00:29:141
25405992518,5cyclictest25387-21cyclictest00:29:141
25405992518,5cyclictest11452-21ntp_kernel_pll_21:28:491
25405992518,5cyclictest11452-21ntp_kernel_pll_21:28:491
25405992517,6cyclictest27845-21gltestperf22:13:431
25405992517,6cyclictest27845-21gltestperf22:13:431
25401992521,3cyclictest17669-21cpuspeed23:13:410
25401992521,3cyclictest17669-21cpuspeed23:13:410
25401992521,3cyclictest17669-21cpuspeed23:13:410
25401992521,3cyclictest17669-21cpuspeed23:13:410
25401992521,3cyclictest14818-21latency_hist20:13:420
25401992521,2cyclictest0-21swapper/021:58:520
25401992521,2cyclictest0-21swapper/021:58:520
25401992521,2cyclictest0-21swapper/021:58:520
25401992520,3cyclictest0-21swapper/020:23:420
25401992520,3cyclictest0-21swapper/020:23:420
25401992519,4cyclictest25387-21cyclictest22:23:520
25401992519,4cyclictest25387-21cyclictest22:23:520
25401992519,4cyclictest25387-21cyclictest22:23:520
25401992519,4cyclictest10147-21/usr/sbin/munin22:53:530
25401992519,4cyclictest10147-21/usr/sbin/munin22:53:530
25401992519,4cyclictest10147-21/usr/sbin/munin22:53:530
25401992519,3cyclictest0-21swapper/000:08:450
25401992519,3cyclictest0-21swapper/000:08:450
25401992519,3cyclictest0-21swapper/000:08:450
25401992518,5cyclictest25387-21cyclictest22:38:460
25401992518,5cyclictest25387-21cyclictest22:38:460
25401992518,5cyclictest25387-21cyclictest22:38:460
25401992515,8cyclictest9023-21apache220:55:160
25401992515,8cyclictest9023-21apache220:55:160
136492250,3sleep20-21swapper/220:08:442
136492250,3sleep20-21swapper/220:08:442
316892240,3chrt0-21swapper/322:23:443
316892240,3chrt0-21swapper/322:23:443
316892240,3chrt0-21swapper/322:23:443
25405992419,3cyclictest0-21swapper/122:03:511
25405992419,3cyclictest0-21swapper/122:03:511
25405992419,3cyclictest0-21swapper/122:03:511
25405992419,3cyclictest0-21swapper/119:53:451
25405992419,3cyclictest0-21swapper/119:53:451
25405992418,5cyclictest29949-21latency22:18:451
25405992418,5cyclictest29949-21latency22:18:451
25405992418,5cyclictest29949-21latency22:18:451
25405992418,3cyclictest0-21swapper/119:33:301
25405992418,3cyclictest0-21swapper/119:33:301
25405992418,3cyclictest0-21swapper/119:33:301
25405992416,6cyclictest2870-21ls19:38:511
25405992416,6cyclictest2870-21ls19:38:511
25401992420,2cyclictest0-21swapper/019:43:300
25401992420,2cyclictest0-21swapper/019:43:300
25401992420,2cyclictest0-21swapper/000:33:460
25401992420,2cyclictest0-21swapper/000:33:460
25401992419,3cyclictest0-21swapper/022:58:450
25401992419,3cyclictest0-21swapper/022:58:450
25401992419,3cyclictest0-21swapper/022:58:450
25401992419,3cyclictest0-21swapper/021:48:410
25401992419,3cyclictest0-21swapper/021:48:410
25401992419,3cyclictest0-21swapper/021:48:410
25401992419,3cyclictest0-21swapper/021:48:410
25401992419,3cyclictest0-21swapper/020:38:430
25401992417,5cyclictest25993-21head22:08:430
25401992417,5cyclictest25993-21head22:08:430
25401992417,5cyclictest1421-21needreboot22:28:470
25401992417,5cyclictest1421-21needreboot22:28:470
25401992416,6cyclictest7463-21cut21:18:450
25401992416,6cyclictest7463-21cut21:18:450
25401992416,6cyclictest10696-21sort21:28:420
25401992416,6cyclictest10696-21sort21:28:420
25401992416,6cyclictest10696-21sort21:28:420
25405992319,2cyclictest0-21swapper/121:58:451
25405992319,2cyclictest0-21swapper/121:58:451
25405992318,3cyclictest0-21swapper/121:18:521
25405992318,3cyclictest0-21swapper/121:18:521
25405992317,5cyclictest32025-21cut20:58:441
25405992317,5cyclictest32025-21cut20:58:441
25405992317,5cyclictest32025-21cut20:58:441
25405992317,4cyclictest27164-21fschecks_count23:38:401
25405992317,4cyclictest26219-21latency22:08:451
25405992317,4cyclictest26219-21latency22:08:451
25405992317,4cyclictest0-21swapper/119:58:481
25405992317,4cyclictest0-21swapper/119:58:481
25405992317,3cyclictest0-21swapper/122:48:531
25405992317,3cyclictest0-21swapper/122:48:531
25405992317,3cyclictest0-21swapper/122:48:531
25405992317,3cyclictest0-21swapper/120:18:491
25405992317,3cyclictest0-21swapper/120:18:491
25405992317,3cyclictest0-21swapper/120:18:491
25405992316,5cyclictest31278-21kernelversion23:48:451
25405992316,5cyclictest31278-21kernelversion23:48:451
25405992316,5cyclictest31278-21kernelversion23:48:451
25405992316,4cyclictest0-21swapper/100:34:551
25405992316,4cyclictest0-21swapper/100:34:551
25405992313,8cyclictest13408-21sshd22:59:581
25405992313,8cyclictest13408-21sshd22:59:581
25405992313,8cyclictest13408-21sshd22:59:581
25401992319,2cyclictest0-21swapper/023:23:440
25401992319,2cyclictest0-21swapper/023:23:440
25401992319,2cyclictest0-21swapper/023:23:440
25401992318,3cyclictest5424-21gltestperf21:13:430
25401992318,3cyclictest5424-21gltestperf21:13:430
25401992317,5cyclictest15895-21rt-features20:13:500
25401992317,5cyclictest15895-21rt-features20:13:500
25401992317,4cyclictest0-21swapper/021:28:460
25401992317,4cyclictest0-21swapper/021:28:460
25401992317,3cyclictest0-21swapper/023:53:490
25401992317,3cyclictest0-21swapper/023:53:490
25401992317,3cyclictest0-21swapper/019:23:510
25401992317,3cyclictest0-21swapper/019:23:510
25401992316,5cyclictest867-21rt-features19:33:510
25401992316,5cyclictest867-21rt-features19:33:510
25401992316,5cyclictest867-21rt-features19:33:510
25401992316,5cyclictest19861-21idleruntime-cro21:53:300
25401992316,5cyclictest19861-21idleruntime-cro21:53:300
25401992316,5cyclictest19861-21idleruntime-cro21:53:300
25401992316,4cyclictest0-21swapper/000:13:540
25401992316,4cyclictest0-21swapper/000:13:540
25401992312,10cyclictest31366-21cpuspeed22:23:420
25401992312,10cyclictest31366-21cpuspeed22:23:420
236662230,3sleep20-21swapper/222:03:332
236662230,3sleep20-21swapper/222:03:332
236662230,3sleep20-21swapper/222:03:332
25410992215,5cyclictest431rcuc/323:05:103
25410992215,5cyclictest431rcuc/323:05:103
25410992215,5cyclictest431rcuc/323:05:103
25405992218,2cyclictest0-21swapper/120:23:441
25405992218,2cyclictest0-21swapper/120:23:441
25405992217,3cyclictest0-21swapper/123:43:521
25405992217,3cyclictest0-21swapper/123:43:521
25405992217,3cyclictest0-21swapper/123:43:521
25405992217,3cyclictest0-21swapper/122:23:501
25405992216,4cyclictest12165-21sendmail_mailst20:03:501
25405992216,4cyclictest12165-21sendmail_mailst20:03:501
25405992213,7cyclictest2379-21ls21:03:541
25405992213,7cyclictest2379-21ls21:03:541
25401992218,2cyclictest0-21swapper/021:53:540
25401992218,2cyclictest0-21swapper/021:53:540
25401992218,2cyclictest0-21swapper/021:53:540
25401992217,3cyclictest0-21swapper/023:38:390
25401992217,3cyclictest0-21swapper/022:23:300
25401992217,3cyclictest0-21swapper/020:18:520
25401992217,3cyclictest0-21swapper/020:18:520
25401992217,3cyclictest0-21swapper/020:18:520
25401992217,3cyclictest0-21swapper/019:48:450
25401992217,3cyclictest0-21swapper/019:48:450
25401992216,4cyclictest25387-21cyclictest20:48:360
25401992216,4cyclictest25387-21cyclictest20:48:360
25401992216,4cyclictest10286-21cat22:53:430
25401992216,4cyclictest10286-21cat22:53:430
25401992215,5cyclictest11821-21latency20:03:440
25401992215,5cyclictest11821-21latency20:03:440
25401992215,4cyclictest0-21swapper/023:23:480
25401992215,4cyclictest0-21swapper/023:23:480
25401992215,4cyclictest0-21swapper/023:23:480
25401992213,7cyclictest14678-21systemd-journal20:08:420
25401992212,8cyclictest14678-21systemd-journal22:05:230
176022220,3sleep20-21swapper/223:13:402
176022220,3sleep20-21swapper/223:13:402
176022220,3sleep20-21swapper/223:13:402
176022220,3sleep20-21swapper/223:13:402
2541099212,11cyclictest0-21swapper/300:03:553
2541099212,11cyclictest0-21swapper/300:03:553
2541099212,11cyclictest0-21swapper/300:03:553
25409992112,6cyclictest912-21chrt22:28:422
25409992112,6cyclictest912-21chrt22:28:422
2540599211,7cyclictest1368-21sed23:53:521
2540599211,7cyclictest1368-21sed23:53:521
25405992116,3cyclictest0-21swapper/122:53:521
25405992116,3cyclictest0-21swapper/122:53:521
25405992116,3cyclictest0-21swapper/122:53:521
2540199212,3cyclictest181rcu_preempt22:51:100
25401992116,3cyclictest0-21swapper/021:43:300
25401992116,3cyclictest0-21swapper/021:43:300
25401992116,3cyclictest0-21swapper/021:43:300
25401992116,3cyclictest0-21swapper/020:28:500
25401992116,3cyclictest0-21swapper/020:28:500
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional