You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-21 - 05:24

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #4, slot #1

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1s.osadl.org (updated Tue Jul 21, 2026 00:44:21)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2888321350,3chrt0-21swapper/223:35:412
2888321350,3chrt0-21swapper/223:35:412
2888321350,3chrt0-21swapper/223:35:412
2672321050,2chrt0-21swapper/223:28:502
2672321050,2chrt0-21swapper/223:28:502
2672321050,2chrt0-21swapper/223:28:502
143252890,3sleep30-21swapper/320:03:453
143252890,3sleep30-21swapper/320:03:453
116782840,3sleep20-21swapper/200:18:262
116782840,3sleep20-21swapper/200:18:262
116782840,3sleep20-21swapper/200:18:262
76052810,3chrt0-21swapper/321:13:393
76052810,3chrt0-21swapper/321:13:393
76052810,3chrt0-21swapper/321:13:393
313202810,4chrt0-21swapper/219:23:402
239822660,3sleep30-21swapper/323:23:403
239822660,3sleep30-21swapper/323:23:403
239822660,3sleep30-21swapper/323:23:403
250042640,3sleep30-21swapper/321:58:463
250042640,3sleep30-21swapper/321:58:463
250042640,3sleep30-21swapper/321:58:463
27499996147,10cyclictest6204-21/usr/sbin/munin00:03:410
27499996147,10cyclictest6204-21/usr/sbin/munin00:03:410
27499996147,10cyclictest6204-21/usr/sbin/munin00:03:410
41372600,3chrt0-21swapper/322:28:493
41372600,3chrt0-21swapper/322:28:493
41372600,3chrt0-21swapper/322:28:493
75752560,3sleep20-21swapper/200:03:592
75752560,3sleep20-21swapper/200:03:592
75752560,3sleep20-21swapper/200:03:592
56192560,4sleep20-21swapper/223:58:492
56192560,4sleep20-21swapper/223:58:492
56192560,4sleep20-21swapper/223:58:492
246422560,2sleep10-21swapper/120:33:301
246422560,2sleep10-21swapper/120:33:301
27082540,3chrt0-21swapper/120:58:501
27082540,3chrt0-21swapper/120:58:501
27082540,3chrt0-21swapper/120:58:501
126702540,3sleep20-21swapper/200:18:472
126702540,3sleep20-21swapper/200:18:472
126702540,3sleep20-21swapper/200:18:472
315522520,2sleep131555-21grep22:18:261
315522520,2sleep131555-21grep22:18:261
102922490,2sleep20-21swapper/219:53:442
102922490,2sleep20-21swapper/219:53:442
161462480,3sleep20-21swapper/220:08:462
161462480,3sleep20-21swapper/220:08:462
180502470,3sleep20-21swapper/221:38:512
180502470,3sleep20-21swapper/221:38:512
180502470,3sleep20-21swapper/221:38:512
178622460,3sleep10-21swapper/121:38:501
178622460,3sleep10-21swapper/121:38:501
178622460,3sleep10-21swapper/121:38:501
27500994234,4cyclictest0-21swapper/100:03:341
27500994234,4cyclictest0-21swapper/100:03:341
27500994234,4cyclictest0-21swapper/100:03:341
286382410,2sleep00-21swapper/020:43:390
286382410,2sleep00-21swapper/020:43:390
277952400,4sleep027798-21unixbench-2d20:38:510
277952400,4sleep027798-21unixbench-2d20:38:510
218252400,3sleep00-21swapper/021:48:500
218252400,3sleep00-21swapper/021:48:500
218252400,3sleep00-21swapper/021:48:500
2750399381,26cyclictest6157-21chrt00:03:323
2750399381,26cyclictest6157-21chrt00:03:323
2750399381,26cyclictest6157-21chrt00:03:323
27500993826,10cyclictest819-21fschecks_count23:48:401
27500993826,10cyclictest819-21fschecks_count23:48:401
27500993731,4cyclictest7107-21latency_hist21:13:261
27500993731,4cyclictest7107-21latency_hist21:13:261
27500993731,4cyclictest7107-21latency_hist21:13:261
27500993630,4cyclictest10986-21sort21:23:261
27500993630,4cyclictest10986-21sort21:23:261
27500993630,4cyclictest10986-21sort21:23:261
27499993528,5cyclictest27498-21cyclictest23:43:380
27499993528,5cyclictest27498-21cyclictest23:43:380
27499993528,5cyclictest27498-21cyclictest23:43:380
27499993525,8cyclictest16140-21netstat20:08:460
27499993525,8cyclictest16140-21netstat20:08:460
135422350,2sleep20-21swapper/220:03:372
135422350,2sleep20-21swapper/220:03:372
27500993427,6cyclictest27498-21cyclictest19:33:401
27500993427,6cyclictest27498-21cyclictest19:33:401
27499993426,6cyclictest13228-21fschecks_count21:28:400
27499993426,6cyclictest13228-21fschecks_count21:28:400
27499993421,11cyclictest2059-21kworker/u10:2-rpciod00:04:330
27499993421,11cyclictest2059-21kworker/u10:2-rpciod00:04:330
27499993421,11cyclictest2059-21kworker/u10:2-rpciod00:04:330
210122340,1sleep021040-21latency_hist20:23:370
27500993326,5cyclictest14678-21systemd-journal00:28:391
27500993326,5cyclictest14678-21systemd-journal00:28:391
170872330,2sleep20-21swapper/223:03:492
170872330,2sleep20-21swapper/223:03:492
170872330,2sleep20-21swapper/223:03:492
97912320,2chrt9792-21latency_hist19:53:341
28972320,2sleep20-21swapper/220:58:512
28972320,2sleep20-21swapper/220:58:512
28972320,2sleep20-21swapper/220:58:512
27500993225,5cyclictest5922-21latency_hist19:43:311
27500993225,5cyclictest5922-21latency_hist19:43:311
27500993221,4cyclictest181rcu_preempt23:26:421
27500993221,4cyclictest181rcu_preempt23:26:421
27500993221,4cyclictest181rcu_preempt23:26:421
27500993221,2cyclictest181rcu_preempt23:07:291
27500993221,2cyclictest181rcu_preempt23:07:291
27500993221,2cyclictest181rcu_preempt23:07:291
2750099321,23cyclictest7152-21/usr/sbin/munin00:03:481
2750099321,23cyclictest7152-21/usr/sbin/munin00:03:481
2750099321,23cyclictest7152-21/usr/sbin/munin00:03:481
198292320,3sleep30-21swapper/320:18:443
198292320,3sleep30-21swapper/320:18:443
94382310,2sleep10-21swapper/100:08:491
94382310,2sleep10-21swapper/100:08:491
94382310,2sleep10-21swapper/100:08:491
27500993125,4cyclictest27012-21gltestperf20:38:421
27500993125,4cyclictest27012-21gltestperf20:38:421
27500993125,4cyclictest21862-21processes20:23:471
27500993125,4cyclictest21862-21processes20:23:471
27500993123,6cyclictest30375-21timerandwakeup23:38:491
27500993123,6cyclictest30375-21timerandwakeup23:38:491
27500993111,5cyclictest271rcuc/119:59:101
27500993111,5cyclictest271rcuc/119:59:101
27499993125,4cyclictest6408-21gltestperf19:43:410
27499993125,4cyclictest6408-21gltestperf19:43:410
27499993124,5cyclictest7619-21gltestperf21:13:400
27499993124,5cyclictest7619-21gltestperf21:13:400
27499993124,5cyclictest7619-21gltestperf21:13:400
232782310,3sleep20-21swapper/220:28:402
232782310,3sleep20-21swapper/220:28:402
2750099307,7cyclictest281ktimers/119:29:261
2750099307,7cyclictest281ktimers/119:29:261
2750099303,5cyclictest271rcuc/122:51:351
2750099303,5cyclictest271rcuc/122:51:351
2750099303,5cyclictest271rcuc/122:51:351
27500993025,4cyclictest27498-21cyclictest23:08:511
27500993025,4cyclictest27498-21cyclictest23:08:511
27500993025,4cyclictest27498-21cyclictest23:08:511
27500993023,5cyclictest281ktimers/100:17:031
27500993023,5cyclictest281ktimers/100:17:031
27500993023,5cyclictest281ktimers/100:17:031
27500993017,4cyclictest90-1pr/ttyS022:08:501
27500993017,4cyclictest90-1pr/ttyS022:08:501
27500993016,2cyclictest181rcu_preempt21:53:461
27500993016,2cyclictest181rcu_preempt21:53:461
27500993016,2cyclictest181rcu_preempt21:53:461
27500993016,2cyclictest181rcu_preempt00:23:471
27500993016,2cyclictest181rcu_preempt00:23:471
27499993023,6cyclictest1960-21fschecks_time20:58:400
27499993023,6cyclictest1960-21fschecks_time20:58:400
27499993023,6cyclictest1960-21fschecks_time20:58:400
27499993023,5cyclictest27498-21cyclictest22:18:480
27499993023,5cyclictest27498-21cyclictest22:18:480
27499993023,5cyclictest15912-21cut00:28:400
27499993023,5cyclictest15912-21cut00:28:400
27499993022,6cyclictest4155-21cat23:58:270
27499993022,6cyclictest4155-21cat23:58:270
27499993022,6cyclictest4155-21cat23:58:270
27499993017,10cyclictest23981-21uniq23:23:400
27499993017,10cyclictest23981-21uniq23:23:400
27499993017,10cyclictest23981-21uniq23:23:400
57122290,3sleep20-21swapper/221:08:412
57122290,3sleep20-21swapper/221:08:412
57122290,3sleep20-21swapper/221:08:412
27500992923,4cyclictest25365-21memory20:33:411
27500992923,4cyclictest25365-21memory20:33:411
27500992922,2cyclictest181rcu_preempt21:53:201
27500992922,2cyclictest181rcu_preempt21:53:201
27500992922,2cyclictest181rcu_preempt21:53:201
27500992921,6cyclictest1949-21ntp_kernel_err22:23:451
27500992921,6cyclictest1949-21ntp_kernel_err22:23:451
27500992921,6cyclictest1949-21ntp_kernel_err22:23:451
27500992920,7cyclictest13804-21netstat21:28:451
27500992920,7cyclictest13804-21netstat21:28:451
27500992917,2cyclictest181rcu_preempt22:58:441
27500992917,2cyclictest181rcu_preempt22:58:441
27500992917,2cyclictest181rcu_preempt22:58:441
27500992915,5cyclictest29-21ksoftirqd/121:03:421
27500992915,5cyclictest29-21ksoftirqd/121:03:421
27500992915,5cyclictest29-21ksoftirqd/121:03:421
2750099291,4cyclictest29-21ksoftirqd/123:57:031
2750099291,4cyclictest29-21ksoftirqd/123:57:031
2750099291,4cyclictest29-21ksoftirqd/123:57:031
320942280,4sleep30-21swapper/322:18:403
320942280,4sleep30-21swapper/322:18:403
2750099287,4cyclictest281ktimers/122:34:311
2750099287,4cyclictest281ktimers/122:34:311
2750099287,4cyclictest281ktimers/122:34:311
2750099286,3cyclictest271rcuc/123:18:171
2750099286,3cyclictest271rcuc/123:18:171
2750099286,3cyclictest271rcuc/123:18:171
2750099285,5cyclictest281ktimers/122:45:371
2750099285,5cyclictest281ktimers/122:45:371
27500992823,3cyclictest20809-21latency_hist20:23:261
27500992823,3cyclictest20809-21latency_hist20:23:261
27500992820,6cyclictest30744-21gltestperf20:48:401
27500992820,6cyclictest30744-21gltestperf20:48:401
27500992820,6cyclictest30744-21gltestperf20:48:401
27500992816,2cyclictest181rcu_preempt22:38:421
27500992816,2cyclictest181rcu_preempt22:38:421
27500992813,2cyclictest181rcu_preempt20:03:511
27500992813,2cyclictest181rcu_preempt20:03:511
27500992810,16cyclictest281ktimers/123:44:331
27500992810,16cyclictest281ktimers/123:44:331
27500992810,16cyclictest281ktimers/123:44:331
27499992824,3cyclictest10537-21ntp_offset19:53:460
27499992824,3cyclictest10537-21ntp_offset19:53:460
27499992823,3cyclictest0-21swapper/022:08:470
27499992823,3cyclictest0-21swapper/022:08:470
27499992822,4cyclictest16015-21cron23:03:260
27499992822,4cyclictest16015-21cron23:03:260
27499992822,4cyclictest16015-21cron23:03:260
27499992811,4cyclictest181rcu_preempt00:18:460
27499992811,4cyclictest181rcu_preempt00:18:460
27499992811,4cyclictest181rcu_preempt00:18:460
260342280,1sleep126047-21latency_hist22:03:311
260342280,1sleep126047-21latency_hist22:03:311
260342280,1sleep126047-21latency_hist22:03:311
58112270,3sleep30-21swapper/319:43:273
58112270,3sleep30-21swapper/319:43:273
2750099277,10cyclictest29-21ksoftirqd/121:45:301
2750099277,10cyclictest29-21ksoftirqd/121:45:301
2750099276,4cyclictest281ktimers/120:14:271
2750099276,4cyclictest281ktimers/120:14:271
2750099276,4cyclictest281ktimers/120:14:271
27500992723,3cyclictest25907-21head23:28:411
27500992723,3cyclictest25907-21head23:28:411
27500992723,3cyclictest25907-21head23:28:411
27500992722,3cyclictest0-21swapper/120:43:401
27500992722,3cyclictest0-21swapper/120:43:401
27500992721,4cyclictest27498-21cyclictest22:57:371
27500992721,4cyclictest27498-21cyclictest22:57:371
27500992720,5cyclictest90-1pr/ttyS023:21:591
27500992720,5cyclictest90-1pr/ttyS023:21:591
27500992720,5cyclictest90-1pr/ttyS023:21:591
27500992720,5cyclictest32511-21fschecks_time20:53:401
27500992720,5cyclictest32511-21fschecks_time20:53:401
27500992718,2cyclictest181rcu_preempt23:38:401
27500992717,7cyclictest4848-21latency_hist22:33:331
27500992717,7cyclictest4848-21latency_hist22:33:331
27500992717,7cyclictest4848-21latency_hist22:33:331
27500992712,2cyclictest181rcu_preempt21:18:371
27500992712,2cyclictest181rcu_preempt21:18:371
27500992712,2cyclictest181rcu_preempt21:18:371
27499992723,3cyclictest10966-21gltestperf22:48:410
27499992723,3cyclictest10966-21gltestperf22:48:410
27499992723,3cyclictest10966-21gltestperf22:48:410
27499992722,4cyclictest544-21lldpd20:53:430
27499992722,4cyclictest544-21lldpd20:53:430
27499992722,3cyclictest0-21swapper/019:38:400
27499992722,3cyclictest0-21swapper/019:38:400
27499992721,4cyclictest14678-21systemd-journal20:30:400
27499992721,4cyclictest14678-21systemd-journal20:30:400
27499992721,4cyclictest14123-21gltestperf00:23:410
27499992721,4cyclictest14123-21gltestperf00:23:410
27499992720,5cyclictest29276-21/usr/sbin/munin23:38:380
27499992720,5cyclictest29276-21/usr/sbin/munin23:38:380
27499992720,5cyclictest29276-21/usr/sbin/munin23:38:380
27499992720,5cyclictest27498-21cyclictest20:38:200
27499992720,5cyclictest27498-21cyclictest20:38:200
204212270,2chrt0-21swapper/200:38:492
2750099266,4cyclictest281ktimers/121:28:401
2750099266,4cyclictest281ktimers/121:28:401
27500992622,3cyclictest90-1pr/ttyS019:23:381
27500992621,3cyclictest271rcuc/100:38:341
27500992621,3cyclictest271rcuc/100:38:341
27500992621,3cyclictest271rcuc/100:38:341
27500992619,5cyclictest9436-21sshd19:51:411
27500992616,2cyclictest181rcu_preempt22:03:421
27500992616,2cyclictest181rcu_preempt22:03:421
27500992615,2cyclictest181rcu_preempt19:53:411
27500992615,2cyclictest181rcu_preempt19:53:411
27500992612,2cyclictest181rcu_preempt22:18:501
27500992612,2cyclictest181rcu_preempt22:18:501
27499992622,3cyclictest1382-21ntp_offset23:48:470
27499992622,3cyclictest1382-21ntp_offset23:48:470
27499992621,3cyclictest5652-21head22:33:430
27499992621,3cyclictest5652-21head22:33:430
27499992621,3cyclictest5652-21head22:33:430
27499992621,3cyclictest26327-21ntp_offset23:28:460
27499992621,3cyclictest26327-21ntp_offset23:28:460
27499992621,3cyclictest26327-21ntp_offset23:28:460
27499992620,4cyclictest0-21swapper/022:33:400
27499992617,7cyclictest9771-21threads22:43:490
27499992617,7cyclictest9771-21threads22:43:490
27499992614,10cyclictest18348-21grep23:08:410
27499992614,10cyclictest18348-21grep23:08:410
27499992614,10cyclictest18348-21grep23:08:410
86772250,3sleep20-21swapper/221:13:512
86772250,3sleep20-21swapper/221:13:512
86772250,3sleep20-21swapper/221:13:512
2750099258,10cyclictest29-21ksoftirqd/121:35:411
2750099258,10cyclictest29-21ksoftirqd/121:35:411
2750099258,10cyclictest29-21ksoftirqd/121:35:411
27500992521,2cyclictest0-21swapper/100:18:411
27500992521,2cyclictest0-21swapper/100:18:411
27500992521,2cyclictest0-21swapper/100:18:411
27500992520,4cyclictest6386-21gltestperf19:43:411
27500992520,4cyclictest6386-21gltestperf19:43:411
27500992520,3cyclictest0-21swapper/120:08:511
27500992520,3cyclictest0-21swapper/120:08:511
27500992514,3cyclictest181rcu_preempt23:34:411
27500992514,3cyclictest181rcu_preempt23:34:411
27500992512,2cyclictest181rcu_preempt00:38:451
27499992520,3cyclictest0-21swapper/019:28:500
27499992520,3cyclictest0-21swapper/019:28:500
27499992518,5cyclictest27498-21cyclictest23:23:140
27499992518,5cyclictest27498-21cyclictest23:23:140
27499992518,5cyclictest27498-21cyclictest23:23:140
27499992518,5cyclictest14256-21unixbench-2d21:28:500
27499992518,5cyclictest14256-21unixbench-2d21:28:500
27499992515,8cyclictest26359-21kworker/u10:2-rpciod20:48:510
27499992515,8cyclictest26359-21kworker/u10:2-rpciod20:48:510
27499992515,8cyclictest26359-21kworker/u10:2-rpciod20:48:510
302482240,3sleep30-21swapper/322:13:413
302482240,3sleep30-21swapper/322:13:413
27499992418,4cyclictest28115-21latency_hist22:08:370
27499992418,4cyclictest28115-21latency_hist22:08:370
27499992418,4cyclictest0-21swapper/019:58:390
27499992418,4cyclictest0-21swapper/019:58:390
27499992418,4cyclictest0-21swapper/000:13:480
27499992418,4cyclictest0-21swapper/000:13:480
27499992418,4cyclictest0-21swapper/000:13:480
27499992418,3cyclictest0-21swapper/022:33:390
27499992417,5cyclictest22543-21sshd21:53:400
27499992417,5cyclictest22543-21sshd21:53:400
27499992417,5cyclictest22543-21sshd21:53:400
27499992417,5cyclictest15326-21gltestperf21:33:410
27499992417,5cyclictest15326-21gltestperf21:33:410
27499992417,5cyclictest15326-21gltestperf21:33:410
27499992417,5cyclictest10551-21unixbench_singl21:18:510
27499992417,5cyclictest10551-21unixbench_singl21:18:510
27499992417,5cyclictest10551-21unixbench_singl21:18:510
27499992416,6cyclictest14243-21needreboot20:03:430
27499992416,6cyclictest14243-21needreboot20:03:430
27499992317,4cyclictest30275-21gltestperf22:13:410
27499992317,4cyclictest30275-21gltestperf22:13:410
27499992317,3cyclictest0-21swapper/000:38:400
27499992316,5cyclictest22688-21cron20:28:270
27499992316,5cyclictest22688-21cron20:28:270
27499992316,5cyclictest19506-21gltestperf20:18:410
221052230,3chrt0-21swapper/223:18:412
221052230,3chrt0-21swapper/223:18:412
221052230,3chrt0-21swapper/223:18:412
40882220,3chrt0-21swapper/219:38:352
40882220,3chrt0-21swapper/219:38:352
27503992216,4cyclictest223062sleep321:53:313
27503992216,4cyclictest223062sleep321:53:313
27503992216,4cyclictest223062sleep321:53:313
2749999228,12cyclictest0-21swapper/023:39:360
2749999228,12cyclictest0-21swapper/023:39:360
27499992217,3cyclictest0-21swapper/023:03:420
27499992217,3cyclictest0-21swapper/023:03:420
27499992217,3cyclictest0-21swapper/023:03:420
27499992217,3cyclictest0-21swapper/022:28:480
27499992217,3cyclictest0-21swapper/021:58:400
27499992217,3cyclictest0-21swapper/021:58:400
27499992217,3cyclictest0-21swapper/021:58:400
27499992217,3cyclictest0-21swapper/021:43:400
27499992217,3cyclictest0-21swapper/021:43:400
27499992217,3cyclictest0-21swapper/021:13:460
27499992217,3cyclictest0-21swapper/021:13:460
27499992217,3cyclictest0-21swapper/021:13:460
27499992217,3cyclictest0-21swapper/019:23:480
27499992216,4cyclictest7873-21timerandwakeup22:38:500
27499992216,4cyclictest7873-21timerandwakeup22:38:500
27499992216,4cyclictest0-21swapper/023:13:450
27499992216,4cyclictest0-21swapper/023:13:450
27499992216,4cyclictest0-21swapper/023:13:450
27499992216,3cyclictest0-21swapper/021:03:380
27499992216,3cyclictest0-21swapper/021:03:380
27499992216,3cyclictest0-21swapper/021:03:380
27499992216,3cyclictest0-21swapper/019:48:400
27499992216,3cyclictest0-21swapper/019:48:400
27499992215,5cyclictest0-21swapper/022:53:490
27499992215,5cyclictest0-21swapper/022:53:490
27499992215,4cyclictest0-21swapper/000:33:490
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional