You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-21 - 08:28

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #4, slot #1

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1s.osadl.org (updated Sat Mar 21, 2026 00:44:12)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
236722850,4sleep016-21ksoftirqd/021:26:520
236722850,4sleep016-21ksoftirqd/021:26:520
178652810,2chrt17869-21sed22:36:561
178652810,2chrt17869-21sed22:36:561
297902800,3sleep30-21swapper/320:16:523
297902800,3sleep30-21swapper/320:16:523
107202780,2chrt0-21swapper/320:51:543
107202780,2chrt0-21swapper/320:51:543
265522730,2sleep20-21swapper/220:06:582
265522730,2sleep20-21swapper/220:06:582
87902670,3chrt0-21swapper/323:41:463
87902670,3chrt0-21swapper/323:41:463
231802640,3chrt0-21swapper/300:19:093
231802640,3chrt0-21swapper/300:19:093
266722610,2chrt26673-21grep21:36:461
266722610,2chrt26673-21grep21:36:461
213812570,3sleep30-21swapper/319:56:343
213812570,3sleep30-21swapper/319:56:343
175742570,2sleep30-21swapper/322:36:543
175742570,2sleep30-21swapper/322:36:543
310092550,3chrt0-21swapper/300:41:403
310092550,3chrt0-21swapper/300:41:403
238802530,3chrt0-21swapper/220:01:482
238802530,3chrt0-21swapper/220:01:482
123862530,2sleep00-21swapper/023:51:450
123862530,2sleep00-21swapper/023:51:450
228832520,1chrt0-21swapper/019:57:200
228832520,1chrt0-21swapper/019:57:200
291642510,2sleep30-21swapper/300:36:413
318722500,4sleep031875-21fschecks_time23:16:470
318722500,4sleep031875-21fschecks_time23:16:470
249912480,2sleep30-21swapper/321:31:453
249912480,2sleep30-21swapper/321:31:453
166452480,2chrt16643-21sed00:01:490
166452480,2chrt16643-21sed00:01:490
57982470,4sleep05800-21mailstats23:31:540
57982470,4sleep05800-21mailstats23:31:540
517099452,36cyclictest16036-21sed00:01:400
517099452,36cyclictest16036-21sed00:01:400
517099452,36cyclictest16036-21sed00:01:400
89452440,3chrt0-21swapper/320:46:553
89452440,3chrt0-21swapper/320:46:553
5180993932,4cyclictest28978-21cron00:36:341
186592370,3sleep30-21swapper/300:06:513
186592370,3sleep30-21swapper/300:06:513
518299361,25cyclictest16038-21chrt00:01:403
518299361,25cyclictest16038-21chrt00:01:403
518299361,25cyclictest16038-21chrt00:01:403
5170993629,5cyclictest28269-21cut23:06:460
5170993629,5cyclictest28269-21cut23:06:460
5170993627,7cyclictest18351-21gltestperf00:06:480
5170993627,7cyclictest18351-21gltestperf00:06:480
5180993527,6cyclictest16265-21fschecks_count00:01:461
5180993527,6cyclictest16265-21fschecks_count00:01:461
10772350,3sleep20-21swapper/223:21:392
10772350,3sleep20-21swapper/223:21:392
518199342,23cyclictest16014-21chrt00:01:392
518199342,23cyclictest16014-21chrt00:01:392
518199342,23cyclictest16014-21chrt00:01:392
5170993328,3cyclictest90-1pr/ttyS022:39:300
5170993328,3cyclictest90-1pr/ttyS022:39:300
5170993327,5cyclictest30639-21processes23:11:520
5170993326,5cyclictest547-21rs:main0
5170993326,5cyclictest547-21rs:main0
5170993326,5cyclictest547-21rs:main0
5180993224,6cyclictest6977-21fschecks_time23:36:471
5180993224,6cyclictest6977-21fschecks_time23:36:471
5180993223,6cyclictest15744-21latency_hist19:41:341
5180993223,6cyclictest15744-21latency_hist19:41:341
518099322,25cyclictest16730-21memory00:01:501
518099322,25cyclictest16730-21memory00:01:501
5170993225,5cyclictest14426-21gltestperf19:36:470
5170993225,5cyclictest14426-21gltestperf19:36:470
247002320,2sleep30-21swapper/320:01:573
247002320,2sleep30-21swapper/320:01:573
5180993126,4cyclictest5169-21cyclictest20:46:541
5180993126,4cyclictest5169-21cyclictest20:46:541
5180993124,5cyclictest13796-21processes22:26:541
5180993124,5cyclictest13796-21processes22:26:541
517099312,4cyclictest201rcuc/021:25:280
517099312,4cyclictest201rcuc/021:25:280
5170993124,5cyclictest5169-21cyclictest23:51:070
5170993124,5cyclictest5169-21cyclictest22:04:560
5170993124,5cyclictest5169-21cyclictest22:04:560
5170993124,5cyclictest5169-21cyclictest22:04:560
74752300,3sleep30-21swapper/322:11:453
74752300,3sleep30-21swapper/322:11:453
5180993025,3cyclictest4949-21ntp_offset20:36:531
5180993025,3cyclictest4949-21ntp_offset20:36:531
5180993025,3cyclictest4949-21ntp_offset20:36:531
5180993024,3cyclictest0-21swapper/121:16:341
5180993022,6cyclictest19012-21threads00:06:561
5180993022,6cyclictest19012-21threads00:06:561
5180993021,7cyclictest3018-21needreboot20:31:511
5180993021,7cyclictest3018-21needreboot20:31:511
5180993021,7cyclictest2350-21unixbench_multi23:21:571
5180993021,7cyclictest2350-21unixbench_multi23:21:571
5180993020,8cyclictest9324-21processes19:21:531
5170993022,6cyclictest22737-21threads00:16:560
5170993022,6cyclictest22737-21threads00:16:560
5170993022,6cyclictest19663-21kernelversion21:16:480
5170993022,6cyclictest19663-21kernelversion21:16:480
5170993022,6cyclictest10321-21kernelversion20:51:480
5170993022,6cyclictest10321-21kernelversion20:51:480
65902290,3sleep10-21swapper/122:06:571
5180992922,6cyclictest16300-21sshd22:34:101
5180992922,5cyclictest27345-21cut20:11:461
5170992925,3cyclictest90-1pr/ttyS021:00:570
5170992925,3cyclictest90-1pr/ttyS021:00:570
5170992925,3cyclictest5169-21cyclictest20:21:400
5170992925,3cyclictest5169-21cyclictest20:21:400
5170992923,5cyclictest27676-21kernelversion20:11:480
5170992923,4cyclictest5169-21cyclictest21:14:470
83092280,1sleep18475-21timerwakeupswit22:11:561
83092280,1sleep18475-21timerwakeupswit22:11:561
5180992823,4cyclictest1346-21avahi-daemon20:06:561
5180992823,4cyclictest1346-21avahi-daemon20:06:561
5180992819,7cyclictest28143-21fschecks_time23:06:451
5180992819,7cyclictest28143-21fschecks_time23:06:451
5170992824,2cyclictest0-21swapper/022:51:480
5170992824,2cyclictest0-21swapper/022:51:480
5170992823,4cyclictest9608-21idleruntime-cro20:51:340
5170992823,4cyclictest9608-21idleruntime-cro20:51:340
5170992823,4cyclictest142-21systemd-journal22:07:100
5170992821,5cyclictest20239-21sort00:11:470
5170992821,5cyclictest20239-21sort00:11:470
5170992820,6cyclictest201rcuc/022:44:300
5170992820,6cyclictest201rcuc/022:44:300
5170992820,6cyclictest201rcuc/022:44:300
5180992722,4cyclictest547-21rs:main1
5180992722,4cyclictest547-21rs:main1
5180992722,4cyclictest1346-21avahi-daemon21:56:541
5180992722,4cyclictest1346-21avahi-daemon21:56:541
5180992721,3cyclictest0-21swapper/100:26:341
5180992721,3cyclictest0-21swapper/100:26:341
5180992719,6cyclictest20903-21latency_hist21:21:361
5180992719,6cyclictest20903-21latency_hist21:21:361
5180992719,5cyclictest23018-21sshd00:17:091
5180992719,5cyclictest23018-21sshd00:17:091
5180992718,7cyclictest22576-21fschecks_time22:51:481
5180992718,7cyclictest22576-21fschecks_time22:51:481
517099278,4cyclictest181rcu_preempt22:58:010
517099278,4cyclictest181rcu_preempt22:58:010
517099278,4cyclictest181rcu_preempt22:58:010
5170992720,5cyclictest2824-21latency_hist23:26:340
5170992720,5cyclictest2824-21latency_hist23:26:340
5170992719,6cyclictest26851-21cut21:36:480
5170992719,6cyclictest26851-21cut21:36:480
5170992718,7cyclictest25575-21fschecks_time20:06:480
5170992718,7cyclictest25575-21fschecks_time20:06:480
5170992714,7cyclictest90-1pr/ttyS000:30:090
5170992714,4cyclictest181rcu_preempt20:21:530
5170992714,4cyclictest181rcu_preempt20:21:530
5180992621,4cyclictest20447-21ntp_offset19:51:521
5180992621,4cyclictest20447-21ntp_offset19:51:521
5180992621,3cyclictest25521-21ntp_offset21:31:511
5180992621,3cyclictest25521-21ntp_offset21:31:511
5180992620,5cyclictest5413-21kernelversion23:31:491
5180992620,5cyclictest5413-21kernelversion23:31:491
5170992620,4cyclictest1735-21latency_hist21:56:430
5170992620,4cyclictest0-21swapper/019:46:570
5170992620,4cyclictest0-21swapper/019:41:570
5170992620,4cyclictest0-21swapper/019:41:570
5170992620,4cyclictest0-21swapper/019:41:570
5170992618,6cyclictest25525-21sort00:26:450
5170992618,6cyclictest25525-21sort00:26:450
5170992617,7cyclictest0-21swapper/023:11:340
5170992616,2cyclictest181rcu_preempt22:15:290
5170992616,2cyclictest181rcu_preempt22:15:290
5170992615,4cyclictest90-1pr/ttyS019:51:570
5170992615,4cyclictest90-1pr/ttyS019:51:570
5170992614,3cyclictest0-21swapper/019:26:490
5170992614,3cyclictest0-21swapper/019:26:490
5170992612,8cyclictest90-1pr/ttyS000:33:340
5170992611,4cyclictest90-1pr/ttyS022:46:490
5170992611,4cyclictest90-1pr/ttyS022:46:490
5170992610,7cyclictest16-21ksoftirqd/023:46:020
5170992610,7cyclictest16-21ksoftirqd/023:46:020
5180992521,2cyclictest0-21swapper/100:41:341
5180992521,2cyclictest0-21swapper/100:41:341
5180992520,4cyclictest5169-21cyclictest22:06:371
5180992520,4cyclictest5169-21cyclictest22:06:371
5180992520,4cyclictest5169-21cyclictest22:06:371
5180992520,3cyclictest0-21swapper/123:56:481
5180992518,4cyclictest0-21swapper/123:41:521
5180992518,4cyclictest0-21swapper/123:41:521
5170992521,2cyclictest0-21swapper/022:26:470
5170992521,2cyclictest0-21swapper/022:26:470
5170992521,2cyclictest0-21swapper/022:16:560
5170992521,2cyclictest0-21swapper/022:16:560
5170992521,2cyclictest0-21swapper/022:16:560
5170992521,2cyclictest0-21swapper/021:46:340
5170992521,2cyclictest0-21swapper/021:46:340
5170992519,4cyclictest13873-21grep21:01:480
5170992519,4cyclictest13873-21grep21:01:480
517099251,8cyclictest16-21ksoftirqd/020:45:270
517099251,8cyclictest16-21ksoftirqd/020:45:270
517099251,8cyclictest16-21ksoftirqd/020:45:270
5170992518,5cyclictest25286-21kernelversion21:31:480
5170992518,5cyclictest25286-21kernelversion21:31:480
5170992518,5cyclictest10156-21latency_hist19:26:340
5170992518,2cyclictest181rcu_preempt21:46:530
5170992517,3cyclictest181rcu_preempt20:31:500
5170992517,3cyclictest181rcu_preempt20:31:500
5170992513,6cyclictest90-1pr/ttyS020:05:450
5170992513,6cyclictest90-1pr/ttyS020:05:450
5170992512,11cyclictest0-21swapper/021:56:460
5170992512,11cyclictest0-21swapper/021:56:460
518299245,9cyclictest0-21swapper/300:01:483
518299245,9cyclictest0-21swapper/300:01:483
5180992418,4cyclictest20247-21basename22:46:341
5180992418,4cyclictest20247-21basename22:46:341
5180992418,4cyclictest20247-21basename22:46:341
5180992418,4cyclictest0-21swapper/121:41:561
5180992418,4cyclictest0-21swapper/121:41:561
5180992416,6cyclictest24161-21netstat20:01:521
5180992416,6cyclictest24161-21netstat20:01:521
5170992420,3cyclictest29943-21ntp_offset00:36:520
5170992420,3cyclictest29943-21ntp_offset00:36:520
5170992420,2cyclictest0-21swapper/023:36:470
5170992420,2cyclictest0-21swapper/023:36:470
5170992419,3cyclictest0-21swapper/023:26:570
5170992419,3cyclictest0-21swapper/022:31:550
5170992412,7cyclictest90-1pr/ttyS021:10:370
5170992412,7cyclictest90-1pr/ttyS021:10:370
5180992319,3cyclictest29396-21head20:16:471
5180992319,3cyclictest29396-21head20:16:471
5180992319,2cyclictest0-21swapper/119:26:471
5180992319,2cyclictest0-21swapper/119:26:471
5180992317,3cyclictest0-21swapper/123:51:471
5180992317,3cyclictest0-21swapper/123:51:471
5180992314,6cyclictest12039-21cut20:56:481
5180992314,6cyclictest12039-21cut20:56:481
5170992316,5cyclictest605-21cut20:26:470
5170992316,5cyclictest605-21cut20:26:470
5170992315,5cyclictest11835-21ntp_kernel_err22:21:520
5170992315,5cyclictest11835-21ntp_kernel_err22:21:520
5170992315,5cyclictest11835-21ntp_kernel_err22:21:520
5180992217,4cyclictest13420-21unixbench_multi19:31:561
5180992217,4cyclictest13420-21unixbench_multi19:31:561
5180992217,3cyclictest0-21swapper/123:26:571
5180992217,3cyclictest0-21swapper/121:51:581
5180992217,3cyclictest0-21swapper/120:41:461
5180992217,3cyclictest0-21swapper/120:41:461
5180992217,3cyclictest0-21swapper/120:41:461
5180992216,4cyclictest11115-21sshd20:52:101
5180992216,4cyclictest11115-21sshd20:52:101
5180992216,3cyclictest0-21swapper/121:21:551
5180992216,3cyclictest0-21swapper/121:21:551
5180992215,5cyclictest17146-21latency_hist21:11:361
5180992215,5cyclictest17146-21latency_hist21:11:361
518099221,11cyclictest11273-21ntp_states23:46:541
5170992212,3cyclictest181rcu_preempt19:35:270
5170992212,3cyclictest181rcu_preempt19:35:270
5180992117,2cyclictest0-21swapper/120:22:561
5180992117,2cyclictest0-21swapper/120:22:561
5180992116,3cyclictest0-21swapper/123:11:521
5180992116,3cyclictest0-21swapper/121:46:541
5180992116,3cyclictest0-21swapper/100:16:381
5180992116,3cyclictest0-21swapper/100:16:381
5180992115,4cyclictest0-21swapper/119:56:571
5180992115,4cyclictest0-21swapper/119:56:571
5180992115,3cyclictest0-21swapper/121:01:471
5180992115,3cyclictest0-21swapper/121:01:471
5180992114,5cyclictest10886-21sendmail22:21:341
5180992114,5cyclictest10886-21sendmail22:21:341
5180992114,5cyclictest10886-21sendmail22:21:341
5180992113,6cyclictest21601-21grep22:46:561
5180992113,6cyclictest21601-21grep22:46:561
5180992112,7cyclictest142-21systemd-journal20:26:551
5180992112,7cyclictest142-21systemd-journal20:26:551
5180992014,4cyclictest18756-21mailstats19:46:541
5180992012,6cyclictest23834-21mailstats21:26:541
5180992012,6cyclictest23834-21mailstats21:26:541
5180992011,7cyclictest0-21swapper/122:56:531
5180992011,7cyclictest0-21swapper/122:56:531
5180992011,7cyclictest0-21swapper/122:56:531
91262190,3sleep30-21swapper/319:21:493
5182991913,4cyclictest22787-21chrt22:51:493
5182991913,4cyclictest22787-21chrt22:51:493
518199193,9cyclictest0-21swapper/222:48:102
518199193,9cyclictest0-21swapper/222:48:102
518199193,9cyclictest0-21swapper/219:37:092
518199193,9cyclictest0-21swapper/219:37:092
518199191,13cyclictest10411-21taskset20:51:492
518199191,13cyclictest10411-21taskset20:51:492
518099199,8cyclictest28649-21ntp_kernel_pll_23:06:511
518099199,2cyclictest181rcu_preempt22:26:461
518099199,2cyclictest181rcu_preempt22:26:461
5180991914,3cyclictest0-21swapper/119:41:481
5180991914,3cyclictest0-21swapper/119:41:481
5180991914,3cyclictest0-21swapper/119:41:481
518199183,10cyclictest0-21swapper/200:01:562
518199183,10cyclictest0-21swapper/200:01:562
518199181,12cyclictest30946-21chrt21:46:482
518199181,12cyclictest13320-21chrt22:26:482
518199181,12cyclictest13320-21chrt22:26:482
5180991814,2cyclictest0-21swapper/100:29:091
518099181,10cyclictest12211-21ls22:21:561
159812180,3sleep30-21swapper/321:06:493
159812180,3sleep30-21swapper/321:06:493
518299171,12cyclictest12392-21taskset19:31:463
518299171,12cyclictest12392-21taskset19:31:463
518199173,9cyclictest0-21swapper/220:42:102
518199173,9cyclictest0-21swapper/220:42:102
518199173,9cyclictest0-21swapper/220:42:102
518199173,10cyclictest0-21swapper/219:32:582
518199173,10cyclictest0-21swapper/219:32:582
518199172,10cyclictest19067-21chrt22:41:472
518199172,10cyclictest19067-21chrt22:41:472
518199172,10cyclictest19067-21chrt22:41:472
518199172,10cyclictest0-21swapper/223:06:452
518199172,10cyclictest0-21swapper/223:06:452
518199171,11cyclictest31736-21taskset20:21:542
518199171,11cyclictest31736-21taskset20:21:542
518199171,11cyclictest29391-21chrt20:16:472
518199171,11cyclictest29391-21chrt20:16:472
518199171,11cyclictest10276-21chrt19:26:382
518199171,11cyclictest0-21swapper/220:36:472
518199171,11cyclictest0-21swapper/220:36:472
518199171,11cyclictest0-21swapper/220:36:472
518199171,10cyclictest5363-21chrt23:31:482
518199171,10cyclictest5363-21chrt23:31:482
518299163,9cyclictest0-21swapper/323:51:483
518299163,9cyclictest0-21swapper/323:51:483
518299163,9cyclictest0-21swapper/323:07:093
518299163,9cyclictest0-21swapper/320:58:103
518299163,9cyclictest0-21swapper/320:58:103
518299161,11cyclictest0-21swapper/322:21:443
518299161,11cyclictest0-21swapper/319:46:443
518299161,11cyclictest0-21swapper/319:46:443
518299161,11cyclictest0-21swapper/319:46:443
518199163,9cyclictest0-21swapper/200:38:092
518199163,9cyclictest0-21swapper/200:38:092
518199162,10cyclictest0-21swapper/223:21:492
518199162,10cyclictest0-21swapper/223:21:492
518199161,11cyclictest9605-21taskset22:16:472
518199161,11cyclictest9605-21taskset22:16:472
518199161,11cyclictest9605-21taskset22:16:472
518199161,11cyclictest8476-21chrt22:11:562
518199161,11cyclictest8476-21chrt22:11:562
518199161,11cyclictest20985-21chrt00:11:562
518199161,11cyclictest20985-21chrt00:11:562
518199161,11cyclictest0-21swapper/220:26:562
518199161,11cyclictest0-21swapper/220:26:562
518199161,10cyclictest7432-21chrt23:36:512
518199161,10cyclictest7432-21chrt23:36:512
518199161,10cyclictest3839-21taskset22:01:472
518199161,10cyclictest3839-21taskset22:01:472
518199161,10cyclictest3839-21taskset22:01:472
518199161,10cyclictest30536-21chrt23:11:502
518199161,10cyclictest23136-21chrt21:26:462
518199161,10cyclictest23136-21chrt21:26:462
518199161,10cyclictest22635-21chrt22:51:482
518199161,10cyclictest22635-21chrt22:51:482
518199161,10cyclictest0-21swapper/221:06:452
518199161,10cyclictest0-21swapper/221:06:452
518199161,10cyclictest0-21swapper/220:31:462
518199161,10cyclictest0-21swapper/220:31:462
518199161,10cyclictest0-21swapper/219:56:462
518199161,10cyclictest0-21swapper/219:56:462
518199160,12cyclictest15166-21taskset22:31:462
518299153,8cyclictest0-21swapper/321:48:203
518299152,9cyclictest0-21swapper/321:28:103
518299152,9cyclictest0-21swapper/321:28:103
518299152,9cyclictest0-21swapper/321:26:443
518299152,9cyclictest0-21swapper/321:26:443
518299152,11cyclictest0-21swapper/320:26:503
518299152,11cyclictest0-21swapper/320:26:503
518299151,12cyclictest0-21swapper/322:16:553
518299151,12cyclictest0-21swapper/322:16:553
518299151,11cyclictest6100-21chrt22:06:513
518299151,11cyclictest557-21chrt21:51:553
518299151,11cyclictest19969-21chrt00:11:453
518299151,11cyclictest19969-21chrt00:11:453
518299151,10cyclictest30486-21taskset21:46:453
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional