You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-29 - 23:49

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot4s.osadl.org (updated Wed Jul 29, 2026 12:44:14)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
373682422060,2sleep3481ktimers/308:22:553
37857982610,3sleep13785802-21cpuspeed_turbos09:12:521
3664330995242,7cyclictest3760389-21latency_hist08:45:010
3664335995141,7cyclictest3829629-21latency_hist09:55:021
3664335994941,5cyclictest3799951-21latency_hist09:25:001
3664330994840,6cyclictest3725561-21apt-get08:10:010
3664335994724,20cyclictest3804212-21ntpq09:28:031
3664330994641,4cyclictest3659242-21kworker/u8:0+events_unbound08:57:560
3664335994525,17cyclictest3974938-21irqrtprio12:22:561
39654072440,6chrt451irq_work/312:12:563
3664335994426,15cyclictest3682249-21irqrtprio07:27:571
38166442430,3chrt0-21swapper/209:42:582
3664330994335,5cyclictest3924111-21latency_hist11:30:000
3664330994334,6cyclictest3909285-21latency_hist11:15:010
3664330994313,16cyclictest584-21runrttasks09:12:500
3664330994232,7cyclictest278-21systemd-journal11:50:000
3664337994137,3cyclictest3983211-21kworker/u8:0+events_unbound12:28:082
3664335994132,6cyclictest3989341-21/usr/sbin/munin12:38:021
3664335994132,6cyclictest3859831-21latency_hist10:25:001
3664335994123,2cyclictest3679410-21kthreadcore07:22:591
3664335994120,3cyclictest0-21swapper/111:47:531
366433599403,13cyclictest3736874-21irqrtprio08:22:551
3664335994029,9cyclictest3714928-21ntpq07:58:021
3664330994010,8cyclictest3710160-21/usr/sbin/munin07:53:070
3664335993921,2cyclictest3963608-21cron12:10:001
3664335993915,5cyclictest0-21swapper/109:47:531
3664330993931,7cyclictest3824547-21sshd09:48:170
3664330993915,12cyclictest3735278-21/usr/sbin/munin08:18:080
3664330993914,11cyclictest3736738-21/usr/sbin/munin08:22:550
3664330993912,16cyclictest3855551-21tr10:22:520
3664330993910,16cyclictest3764468-21/usr/sbin/munin08:48:030
38263592380,3sleep2371rcuc/209:52:572
3664335993830,6cyclictest3659242-21kworker/u8:0+flush-8:008:32:541
366433599382,15cyclictest3851126-21sed10:17:551
366433599382,15cyclictest3851126-21sed10:17:551
366433599382,14cyclictest581-21snmpd10:58:041
3664335993820,2cyclictest0-21swapper/109:18:091
3664335993819,3cyclictest3910472-21irqrtprio11:17:581
366433099389,14cyclictest3860134-21runrttasks10:26:040
3664330993829,6cyclictest278-21systemd-journal10:50:000
3664330993818,3cyclictest3744721-21/usr/sbin/munin08:28:000
366433099381,35cyclictest1-21systemd11:43:170
3664330993813,12cyclictest1-21systemd08:40:000
3664330993813,11cyclictest3895151-21/usr/sbin/munin11:02:510
3664330993811,14cyclictest3794514-21/usr/sbin/munin09:18:080
3664330993811,13cyclictest3804878-21cron09:30:000
3664330993810,16cyclictest3829163-21/usr/sbin/munin09:53:070
3664330993810,15cyclictest3944533-21/usr/sbin/munin11:52:540
3664330993810,15cyclictest3924808-21/usr/sbin/munin11:32:520
39175612370,3sleep00-21swapper/011:22:580
39175612370,3sleep00-21swapper/011:22:580
3664337993733,3cyclictest3819564-21kworker/u8:2+events_unbound11:13:002
3664335993727,8cyclictest278-21systemd-journal09:55:011
366433099378,17cyclictest3716051-21chrt08:02:020
3664330993729,6cyclictest3690298-21ntpq07:33:040
3664330993715,9cyclictest3890347-21/usr/sbin/munin10:57:540
3664330993714,11cyclictest3696561-21/usr/sbin/munin07:42:510
3664330993713,11cyclictest3704873-21wc07:48:000
3664330993712,12cyclictest3830721-21ls09:57:540
3664330993711,15cyclictest3715581-21sshd07:59:170
3664330993710,15cyclictest584-21runrttasks12:23:310
3664330993710,14cyclictest3903497-21/usr/sbin/munin11:08:010
3664337993623,4cyclictest3973546-21kworker/u8:3+events_unbound12:23:062
366433599363,9cyclictest584-21runrttasks08:52:491
3664335993630,4cyclictest3899884-21apt-get11:07:501
3664335993620,3cyclictest0-21swapper/112:20:001
3664335993620,2cyclictest0-21swapper/107:47:561
3664335993620,13cyclictest0-21swapper/111:40:001
366433099369,15cyclictest3671430-21sshd07:15:450
366433099369,14cyclictest3839264-21/usr/sbin/munin10:03:060
366433099368,8cyclictest584-21runrttasks12:39:090
3664330993612,13cyclictest3954408-21sort12:02:520
3664330993610,7cyclictest3988214-21seq12:33:030
3664330993610,16cyclictest3746252-21/usr/sbin/munin08:32:540
3664330993610,15cyclictest3685526-21/usr/sbin/munin07:28:080
3664330993610,13cyclictest3845202-21/usr/sbin/munin10:12:490
3664337993530,3cyclictest3644694-21kworker/u8:2+events_unbound07:20:462
366433599352,3cyclictest39285362mailstats11:33:071
3664335993517,3cyclictest3819499-21unixbench_multi09:43:091
366433099359,15cyclictest3779948-21/usr/sbin/munin09:03:070
3664330993515,8cyclictest3875212-21/usr/sbin/munin10:42:500
3664330993511,13cyclictest3850393-21find10:17:480
3664330993511,13cyclictest3850393-21find10:17:480
3664330993511,12cyclictest36943671kthreadcore07:38:010
3664330993510,13cyclictest3979908-21/usr/sbin/munin12:27:560
3664330993510,11cyclictest3888559-21/usr/sbin/munin10:52:580
3664335993428,3cyclictest3697122-21irqrtprio07:42:561
3664335993419,2cyclictest3944951-21irqcore11:52:571
3664335993418,6cyclictest3964084-21apt-get12:12:481
366433099349,13cyclictest3668535-21ld07:13:020
366433099348,10cyclictest3964803-21sed12:12:560
3664330993414,9cyclictest3814660-21sshd09:38:170
3664330993413,11cyclictest584-21runrttasks09:30:480
3664330993411,8cyclictest3815277-21find09:42:490
3664330993411,13cyclictest584-21runrttasks10:35:590
3664330993410,8cyclictest3770447-21sed08:55:010
3664330993410,13cyclictest3952953-21/usr/sbin/munin11:58:000
3664330993410,13cyclictest3780887-21taskset09:07:530
3664330993410,13cyclictest3676358-21runrttasks07:20:240
366433099338,14cyclictest3910002-21/usr/sbin/munin11:17:540
366433099338,14cyclictest3726448-21cpuspeed08:12:520
366433099338,14cyclictest3726448-21cpuspeed08:12:520
3664330993312,8cyclictest3963070-21/usr/sbin/munin12:08:100
3664330993311,12cyclictest3865745-21cat10:32:540
3664330993311,11cyclictest3930371-21/usr/sbin/munin11:37:570
3664330993310,10cyclictest584-21runrttasks09:21:450
36596492334,9sleep10-21swapper/107:07:361
36596492334,9sleep10-21swapper/107:07:351
3664335993223,7cyclictest278-21systemd-journal08:10:011
3664335993223,7cyclictest278-21systemd-journal08:10:011
3664335993222,8cyclictest3668536-21ntpq07:13:031
3664335993222,7cyclictest3755431-21latency_hist08:40:011
366433099328,13cyclictest3972738-21/usr/sbin/munin12:18:020
3664330993217,3cyclictest3844115-21ntpq10:08:050
366433599312,10cyclictest581-21snmpd10:47:501
36595632304,9sleep20-21swapper/207:06:282
36595632304,9sleep20-21swapper/207:06:282
3664337992925,3cyclictest3760404-21kworker/u8:1+events_unbound09:24:232
3664335992924,3cyclictest3805617-21cut09:32:531
3664335992923,3cyclictest3899206-21unixbench_singl11:03:081
3664335992920,6cyclictest3874681-21latency_hist10:40:011
366433799289,6cyclictest3899984-21cut11:07:522
3664337992815,2cyclictest161rcu_preempt09:48:022
3664335992824,3cyclictest3887189-21kthreadcore10:52:571
3664335992821,5cyclictest1-21systemd08:10:001
3664335992815,9cyclictest278-21systemd-journal10:33:181
3664337992723,2cyclictest3860584-21fschecks_count10:27:512
3664337992719,6cyclictest3771056-21fschecks_time08:57:542
366433799270,16cyclictest3790032-21cron09:15:002
366433799270,11cyclictest3948777-21apt11:55:002
3664335992723,2cyclictest3860583-21cut10:27:511
3664335992716,7cyclictest278-21systemd-journal10:04:161
366433599271,22cyclictest1-21systemd11:30:011
366433799267,9cyclictest161rcu_preempt08:08:092
3664337992622,2cyclictest3766033-21cpuspeed_turbos08:52:512
3664337992622,2cyclictest0-21swapper/207:20:002
3664337992621,4cyclictest3644694-21kworker/u8:2+events_unbound07:43:082
3664337992619,5cyclictest278-21systemd-journal08:05:002
3664337992618,2cyclictest161rcu_preempt09:37:572
3664337992610,4cyclictest3869540-21grep10:33:072
366433799260,9cyclictest3890300-21tune2fs10:57:532
366433799260,15cyclictest3953713-21munin-run12:00:002
3664335992616,8cyclictest3814065-21ntpq09:38:031
3664335992616,7cyclictest3785084-21cron09:10:001
3664330992616,6cyclictest3664145-21latency_hist07:10:000
3664330992616,6cyclictest3664145-21latency_hist07:10:000
39285342250,5sleep20-21swapper/211:33:072
366433799258,6cyclictest161rcu_preempt11:30:002
366433799254,3cyclictest3710712-21sh07:55:002
366433799252,5cyclictest0-21swapper/210:12:562
3664337992522,2cyclictest3705576-21unixbench_singl07:48:092
3664337992516,7cyclictest278-21systemd-journal11:43:172
3664337992513,2cyclictest161rcu_preempt08:27:542
3664337992512,2cyclictest161rcu_preempt11:22:482
3664337992512,2cyclictest161rcu_preempt11:22:482
366433799251,14cyclictest3973525-21sendmail-msp12:20:012
366433799251,14cyclictest3754648-21needreboot08:38:022
3664337992511,2cyclictest161rcu_preempt12:12:572
366433799250,8cyclictest3794754-21grep09:18:092
366433799250,12cyclictest3959131-21apt-get12:07:502
3664335992520,3cyclictest3686729-21cut07:32:541
3664335992519,4cyclictest0-21swapper/108:05:001
3664335992517,6cyclictest3849705-21latency_hist10:15:001
366433799243,8cyclictest3856052-21grep10:22:562
366433799242,2cyclictest0-21swapper/212:32:572
3664337992421,2cyclictest3913974-21unixbench_multi11:18:102
3664337992420,3cyclictest3869752-21kworker/u8:1+events_unbound11:46:092
3664337992420,3cyclictest3830089-21kworker/u8:0+events_unbound10:00:302
3664337992420,3cyclictest3659242-21kworker/u8:0+events_unbound07:29:262
366433799241,9cyclictest3750490-21idleruntime-cro08:35:002
3664337992411,2cyclictest161rcu_preempt08:17:482
366433799240,8cyclictest3886869-21cut10:52:572
366433799240,5cyclictest3715657-21munin-run08:00:012
3664335992420,2cyclictest0-21swapper/108:28:001
3664335992418,4cyclictest3731203-21apt-get08:17:481
3664335992416,5cyclictest3659242-21kworker/u8:0+flush-8:008:43:041
3664337992319,3cyclictest3760404-21kworker/u8:1+events_unbound09:33:442
3664337992318,3cyclictest3834620-21unixbench_multi09:58:092
366433799231,7cyclictest3776333-21sed09:02:552
366433799231,6cyclictest3932898-21cut11:38:002
366433799231,6cyclictest3843609-21cut10:08:002
366433799231,6cyclictest3783976-21cut09:07:592
3664337992314,2cyclictest161rcu_preempt07:33:092
366433799230,9cyclictest3880489-21grep10:47:522
366433799230,6cyclictest3666638-21tr07:12:582
366433799230,5cyclictest3879810-21grep10:45:012
366433799230,11cyclictest3854863-21munin-run10:20:002
366433799230,11cyclictest3854863-21munin-run10:20:002
3664335992319,2cyclictest3936585-21kthreadcore11:42:591
3664335992319,2cyclictest0-21swapper/107:18:071
3664335992316,5cyclictest3954194-21apt-get12:02:491
3664335992315,6cyclictest3949240-21apt-get11:57:481
3664335992312,8cyclictest3908606-21ntpq11:13:021
366433799225,9cyclictest3756836-21tail08:42:562
366433799222,9cyclictest3764817-21awk08:48:082
366433799222,2cyclictest0-21swapper/207:37:592
366433799221,3cyclictest3989589-21tr12:37:532
3664337992210,3cyclictest161rcu_preempt08:22:582
366433799220,6cyclictest3898742-21taskset11:03:022
366433799220,6cyclictest3870802-21sed10:37:562
366433799220,5cyclictest3800641-21tr09:27:522
3664335992215,5cyclictest3983817-21apt-get12:29:171
3664335992215,5cyclictest3919174-21turbostat.cron11:25:001
3664335992215,5cyclictest3919174-21turbostat.cron11:25:001
3664335992213,6cyclictest0-21swapper/108:47:571
3664335992212,7cyclictest3710715-21latency_hist07:55:011
3664337992116,3cyclictest371rcuc/212:02:542
3664335992117,3cyclictest3779436-21ntp_kernel_pll_09:03:011
3664335992117,2cyclictest0-21swapper/112:35:001
3664335992117,2cyclictest0-21swapper/110:07:511
3664335992116,3cyclictest3695421-21mailstats07:38:081
366433799201,9cyclictest3726453-21sort08:12:532
366433799201,9cyclictest3726453-21sort08:12:522
3664335992016,2cyclictest0-21swapper/108:57:551
3664335992014,4cyclictest0-21swapper/110:42:551
36597182193,9sleep30-21swapper/307:07:543
36597182193,9sleep30-21swapper/307:07:543
39308072144,6sleep30-21swapper/311:37:593
39580312132,7sleep30-21swapper/312:03:063
38701642132,7sleep30-21swapper/310:37:503
38303012134,5sleep30-21swapper/309:57:503
38273882133,6sleep30-21swapper/309:52:593
38197222131,6sleep30-21swapper/309:45:023
38099392132,6sleep30-21swapper/309:35:023
37759422132,7sleep30-21swapper/309:02:523
37596602131,8sleep30-21swapper/308:43:023
37100192132,7sleep30-21swapper/307:53:033
39809272122,7sleep30-21swapper/312:27:583
39746882122,6sleep30-21swapper/312:22:563
39344812122,6sleep30-21swapper/311:42:523
39285942122,6sleep30-21swapper/311:33:103
39092662122,6sleep30-21swapper/311:15:003
38998562122,6sleep30-21swapper/311:07:513
38951872122,6sleep30-21swapper/311:02:523
38839362122,6sleep30-21swapper/310:48:043
38747182122,6sleep30-21swapper/310:40:013
38688352122,7sleep30-21swapper/310:32:593
38558382122,6sleep30-21swapper/310:22:573
37873122122,6sleep30-21swapper/309:12:593
37817422122,6sleep30-21swapper/309:07:583
37704982122,5sleep30-21swapper/308:55:023
37413882122,6sleep30-21swapper/308:27:543
37267902123,6sleep30-21swapper/308:12:573
37267902123,6sleep30-21swapper/308:12:573
37250432122,6sleep30-21swapper/308:08:083
37163792121,8sleep30-21swapper/308:02:543
36962522124,5sleep30-21swapper/307:41:123
36911072122,5sleep30-21swapper/307:35:023
36815332122,6sleep30-21swapper/307:27:523
36763832122,6sleep30-21swapper/307:20:463
36686982123,5sleep30-21swapper/307:13:073
39892012112,6sleep30-21swapper/312:36:043
39888832112,5sleep30-21swapper/312:35:003
39687772112,5sleep30-21swapper/312:15:023
39590872112,6sleep30-21swapper/312:07:363
39494842112,5sleep30-21swapper/311:57:533
39481592112,5sleep30-21swapper/311:53:073
39392302112,5sleep30-21swapper/311:46:373
39199502112,5sleep30-21swapper/311:27:563
39144222112,5sleep30-21swapper/311:20:023
39144222112,5sleep30-21swapper/311:20:013
39124272112,5sleep30-21swapper/311:18:013
38897272112,5sleep30-21swapper/310:55:023
38601492112,5sleep30-21swapper/310:26:173
38501602112,5sleep30-21swapper/310:15:023
38501602112,5sleep30-21swapper/310:15:023
38402002113,5sleep30-21swapper/310:07:043
38349262112,5sleep30-21swapper/310:00:013
38199782112,6sleep30-21swapper/309:45:173
38092282112,5sleep30-21swapper/309:33:053
38004332112,6sleep30-21swapper/309:27:533
37952882112,5sleep30-21swapper/309:20:583
37660582112,6sleep30-21swapper/308:52:533
37607562112,6sleep30-21swapper/308:46:553
37514572112,6sleep30-21swapper/308:37:573
37152272112,5sleep30-21swapper/307:58:093
37020442112,5sleep30-21swapper/307:47:583
36864382112,5sleep30-21swapper/307:32:493
36715242112,5sleep30-21swapper/307:17:003
38848712102,5sleep30-21swapper/310:51:063
38450602102,5sleep30-21swapper/310:11:093
37902622102,5sleep30-21swapper/309:15:043
37458222102,5sleep30-21swapper/308:30:513
37309572102,5sleep30-21swapper/308:15:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional