You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-29 - 00:22

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack5slot4s.osadl.org (updated Fri Aug 28, 2026 12:44:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
382246599205180,22cyclictest4063446-21python311:07:480
382247099177173,2cyclictest0-21swapper/111:07:471
382247199176171,3cyclictest4063447-21sshd11:07:482
387206921320,4sleep3481ktimers/307:57:393
41306782630,6sleep20-21swapper/212:17:382
39431902560,6sleep3451irq_work/309:07:423
3822465994841,5cyclictest275-21systemd-journal09:32:480
3822470994738,7cyclictest3864548-21apt-get07:50:371
3822465994739,5cyclictest4113576-21latency_hist12:00:000
3822465994739,5cyclictest4113576-21latency_hist12:00:000
3822471994637,7cyclictest4053907-21cp11:00:002
3822465994615,18cyclictest573-21runrttasks07:57:310
3822465994434,8cyclictest3989254-21/usr/sbin/munin09:57:480
3822465994333,8cyclictest275-21systemd-journal12:33:480
3822465994233,7cyclictest4053790-21latency_hist11:00:010
382219624228,9sleep10-21swapper/107:08:461
3822465994132,6cyclictest275-21systemd-journal07:36:370
3822465994115,11cyclictest3889675-21apt-get08:17:320
3822470994033,5cyclictest275-21systemd-journal10:01:481
3822470994032,6cyclictest275-21systemd-journal11:38:471
39744812390,2sleep3575-21kworker/3:2+mm_percpu_wq09:42:293
3822470993931,6cyclictest3874180-21latency_hist08:00:001
3822465993935,3cyclictest3475338-21kworker/u8:4+events_unbound07:12:370
40009142380,1sleep1281rcuc/110:07:391
382247199389,14cyclictest3864750-21apt-get07:52:322
3822465993818,16cyclictest3833542-21ntpq07:17:420
3822465993816,10cyclictest3888471-21ntpq08:12:440
3822465993813,14cyclictest3908387-21/usr/sbin/munin08:32:480
382247199374,21cyclictest4114339-21tune2fs12:02:352
382247199374,21cyclictest4114339-21tune2fs12:02:352
3822465993723,9cyclictest3859748-21apt-get07:47:310
3822465993714,9cyclictest3958807-21/usr/sbin/munin09:22:500
3822465993711,16cyclictest4033440-21runrttasks10:37:470
382247199368,14cyclictest3879829-21apt-get08:07:322
3822465993615,6cyclictest4137767-21/usr/sbin/munin12:22:490
3822465993614,5cyclictest4127672-21ntpq12:12:440
3822465993614,10cyclictest3918215-21ntpq08:42:460
3822465993613,8cyclictest4073171-21/usr/sbin/munin11:17:470
3822465993613,6cyclictest3858646-21ntpq07:42:460
3822465993613,11cyclictest3868512-21ntpq07:52:440
3822465993612,8cyclictest573-21runrttasks10:12:310
3822465993612,14cyclictest0-21swapper/008:23:560
3822465993612,14cyclictest0-21swapper/008:23:560
3822471993527,6cyclictest3874304-21gpgv08:00:002
3822465993523,8cyclictest4128814-21apt-get12:17:290
3822465993515,8cyclictest3984832-21/usr/sbin/munin09:52:370
3822465993514,8cyclictest4068121-21ntpq11:12:450
3822465993514,7cyclictest3945020-21/usr/sbin/munin09:12:350
3822465993513,9cyclictest4048107-21ntpq10:52:430
3822465993513,8cyclictest3949537-21sshd09:15:480
3822465993513,6cyclictest407-21mta-sts-daemon12:06:070
3822465993513,5cyclictest3923077-21ntpq08:47:440
3822465993511,9cyclictest4103043-21ntpq11:47:460
3822465993510,11cyclictest4079351-21runrttasks11:26:300
382247199348,13cyclictest4083460-21python311:27:482
3822470993427,5cyclictest275-21systemd-journal08:44:481
3822470993426,6cyclictest275-21systemd-journal09:38:481
3822470993425,6cyclictest275-21systemd-journal10:18:481
3822465993415,6cyclictest3880550-21/usr/sbin/munin08:07:370
3822465993414,5cyclictest4058300-21ntpq11:02:440
3822465993414,5cyclictest4013335-21ntpq10:17:420
3822465993414,4cyclictest4043416-21python310:47:490
3822465993414,10cyclictest4038502-21python310:42:480
3822465993413,6cyclictest573-21runrttasks09:47:310
3822465993413,5cyclictest3997976-21ntpq10:02:430
3822465993412,9cyclictest573-21runrttasks08:54:250
3822465993412,9cyclictest3838504-21ntpq07:22:440
3822465993412,5cyclictest3933230-21ntpq08:57:460
3822465993411,10cyclictest573-21runrttasks10:22:020
3822470993328,3cyclictest3879377-21cron08:05:001
3822470993326,5cyclictest3934481-21aten2_r5power_p09:02:321
3822470993325,6cyclictest3904608-21aten2_r5power_p08:32:341
3822465993327,3cyclictest4084389-21apt-get11:32:310
3822465993325,4cyclictest4074500-21apt-get11:22:320
3822465993319,11cyclictest4003195-21ntpq10:07:440
3822465993314,5cyclictest4023238-21ntpq10:27:470
3822465993314,10cyclictest4118021-21python312:02:490
3822465993314,10cyclictest4118021-21python312:02:490
3822465993313,5cyclictest3938218-21ntpq09:02:450
3822465993313,5cyclictest3843513-21ntpq07:27:450
3822465993312,8cyclictest4025032-21/usr/sbin/munin10:32:370
3822465993312,7cyclictest0-21swapper/009:44:140
3822465993312,7cyclictest0-21swapper/008:36:270
3822465993312,5cyclictest3903329-21ntpq08:27:430
38372952320,2sleep3481ktimers/307:22:403
3822471993224,6cyclictest275-21systemd-journal12:11:482
3822470993228,3cyclictest4034055-21systemd-run10:40:001
3822470993227,3cyclictest3913684-21unixbench_multi08:37:491
3822470993224,6cyclictest4023662-21unixbench_singl10:27:511
3822465993227,3cyclictest3943889-21unixbench_singl09:07:510
3822465993225,3cyclictest4148830-21apt-get12:37:300
3822465993224,3cyclictest4094280-21apt-get11:42:310
3822465993221,7cyclictest3959655-21apt-get09:27:310
3822465993214,4cyclictest3969979-21/usr/sbin/munin09:37:340
3822465993213,5cyclictest4093136-21ntp_offset11:37:450
3822465993212,5cyclictest4142573-21ntpq12:27:470
382247199316,5cyclictest3933284-21ntpq08:57:472
3822471993126,3cyclictest3859979-21tr07:47:322
3822471993110,5cyclictest4086542-21/usr/sbin/munin11:32:382
3822470993126,3cyclictest4043093-21needreboot10:47:421
3822470993126,3cyclictest4038736-21unixbench_multi10:42:491
3822470993126,3cyclictest3885685-21kthreadcore08:12:381
3822470993125,4cyclictest3963943-21unixbench_singl09:27:501
3822470993124,6cyclictest4148183-21unixbench_singl12:32:501
3822470993124,5cyclictest3861921-21kthreadcore07:47:391
3822470993111,11cyclictest0-21swapper/112:30:011
3822465993115,14cyclictest3878726-21ntpq08:02:450
3822465993110,16cyclictest3993815-21kworker/0:2+events11:55:010
382247199309,6cyclictest3914408-21apt-get08:42:322
382247199309,4cyclictest3955419-21/usr/sbin/munin09:22:382
382247199308,11cyclictest4073123-21ntpq11:17:452
382247199306,5cyclictest4112932-21ntpq11:57:502
382247199306,5cyclictest4112932-21ntpq11:57:502
3822470993026,3cyclictest3894009-21unixbench_singl08:17:501
3822470993026,2cyclictest4008838-21cron10:15:001
3822470993025,3cyclictest4054716-21cpuspeed_turbos11:02:321
3822470993025,3cyclictest3983910-21idleruntime-cro09:50:001
3822465993014,14cyclictest3845214-21/usr/sbin/munin07:32:350
382247199297,6cyclictest3998035-21ntpq10:02:452
382247199296,8cyclictest3968564-21systemctl09:32:472
382247199293,15cyclictest0-21swapper/209:17:422
3822471992924,3cyclictest4148111-21unixbench_multi12:32:502
382247199290,27cyclictest4108390-21taskset11:52:482
382247099297,9cyclictest0-21swapper/107:27:371
3822470992924,3cyclictest3884125-21unixbench_singl08:07:501
3822470992922,5cyclictest275-21systemd-journal07:34:371
3822470992921,6cyclictest4118946-21apt-get12:07:301
3822470992921,5cyclictest4138269-21latency_hist12:25:011
382247199287,5cyclictest4137592-21ntpq12:22:472
382247199287,5cyclictest4097976-21ntpq11:42:442
382247199283,9cyclictest4103098-21ps11:47:472
3822471992824,2cyclictest3984377-21aten2_r5power_c09:52:322
382247099286,9cyclictest0-21swapper/112:02:371
382247099286,9cyclictest0-21swapper/112:02:371
382247099286,11cyclictest0-21swapper/108:50:361
3822470992825,2cyclictest4104407-21cut11:52:321
3822470992824,2cyclictest3950417-21irqrtprio09:17:381
3822470992824,2cyclictest0-21swapper/111:47:321
3822470992823,3cyclictest4069900-21irqcore11:17:371
382247199273,9cyclictest4043139-21ld10:47:432
382247199273,9cyclictest3924695-21awk08:52:372
3822471992724,2cyclictest4009553-21aten2_r5power_c10:17:292
3822471992723,3cyclictest3895024-21fschecks_time08:22:352
3822471992723,3cyclictest3895024-21fschecks_time08:22:352
3822471992723,2cyclictest4148915-21aten2_r5power_p12:37:322
382247199270,12cyclictest39744842cut09:42:292
382247099275,9cyclictest0-21swapper/110:52:451
3822470992723,3cyclictest3986499-21kthreadcore09:52:411
3822470992723,3cyclictest3834326-21systemd-run07:20:001
3822470992723,2cyclictest3991437-21kthreadcore09:57:391
3822470992722,3cyclictest3904067-21cron08:30:001
3822470992722,3cyclictest0-21swapper/109:00:011
382247199268,5cyclictest4142487-21ntpq12:27:442
382247199267,5cyclictest3993013-21ntpq09:57:432
382247199265,5cyclictest3973410-21ps09:37:452
382247199263,8cyclictest0-21swapper/207:40:022
3822471992623,2cyclictest3849146-21unixbench_singl07:32:512
3822471992623,2cyclictest3844397-21latency_hist07:30:002
3822471992622,2cyclictest3913679-21unixbench_multi08:37:492
3822471992621,3cyclictest4019444-21aten2_r5power_c10:27:322
382247099269,9cyclictest0-21swapper/109:08:461
3822470992622,3cyclictest0-21swapper/111:42:491
3822470992618,6cyclictest275-21systemd-journal07:38:361
3822470992616,2cyclictest161rcu_preempt07:12:451
382247199255,4cyclictest4064801-21/usr/sbin/munin11:12:382
382247199253,6cyclictest3998749-21apt-get10:05:032
382247199253,5cyclictest4073803-21apt-get11:20:032
382247199253,5cyclictest3938897-21apt-get09:05:032
3822471992511,10cyclictest3850444-21irqcore07:37:372
382247099255,6cyclictest4084571-21cat11:32:321
382247099255,6cyclictest275-21systemd-journal08:21:481
382247099255,6cyclictest275-21systemd-journal08:21:481
382247099255,11cyclictest0-21swapper/111:27:191
382247099255,10cyclictest0-21swapper/111:56:441
382247099255,10cyclictest0-21swapper/111:56:441
3822470992521,3cyclictest0-21swapper/111:00:001
3822470992521,2cyclictest0-21swapper/111:12:311
3822470992520,3cyclictest0-21swapper/110:32:341
3822470992514,3cyclictest161rcu_preempt10:22:351
3822465992512,10cyclictest3822468-21latency_hist07:10:010
382247199247,15cyclictest4037081-21cut10:42:392
382247199244,5cyclictest4033522-21python310:37:472
382247199244,3cyclictest0-21swapper/210:20:182
382247199243,8cyclictest0-21swapper/208:45:392
3822471992411,8cyclictest3899584-21apt-get08:27:322
3822471992411,10cyclictest3893661-21/usr/sbin/munin08:17:482
3822471992410,12cyclictest3878906-21/usr/sbin/munin08:02:492
382247099243,5cyclictest4078257-21ld11:22:451
3822470992420,2cyclictest0-21swapper/108:47:471
3822470992420,2cyclictest0-21swapper/107:42:321
3822470992419,3cyclictest0-21swapper/112:12:331
3822470992419,3cyclictest0-21swapper/109:25:011
382247199239,11cyclictest3979428-21/usr/sbin/munin09:47:312
382247199232,5cyclictest4024563-21apt-get10:32:312
3822471992320,2cyclictest3834966-21cpuspeed_turbos07:22:332
3822471992311,10cyclictest4093248-21/usr/sbin/munin11:37:482
382247099234,10cyclictest0-21swapper/112:37:171
3822470992318,3cyclictest3838985-21unixbench_multi07:22:501
382247199224,10cyclictest0-21swapper/207:17:312
382247199221,5cyclictest3823180-21perl07:12:342
3822471992212,8cyclictest4004643-21/usr/sbin/munin10:12:352
382247099226,4cyclictest0-21swapper/109:32:381
3822470992215,5cyclictest0-21swapper/109:12:411
3822470992213,2cyclictest161rcu_preempt09:41:591
382247199211,5cyclictest3908358-21ntp_states08:32:482
3822471992111,8cyclictest4122562-21/usr/sbin/munin12:07:412
3822471992111,8cyclictest4048185-21ntpq10:52:462
3822471992111,8cyclictest3938400-21/usr/sbin/munin09:02:482
3822471992111,8cyclictest3888663-21/usr/sbin/munin08:12:482
3822470992117,2cyclictest0-21swapper/112:17:341
3822471992011,7cyclictest3944564-21/usr/sbin/munin09:12:312
3822471992010,8cyclictest3963701-21/usr/sbin/munin09:27:492
3822471991913,4cyclictest381ktimers/211:01:412
38178072194,10sleep20-21swapper/207:07:192
399413321810,5sleep30-21swapper/310:01:323
38177782183,8sleep30-21swapper/307:06:563
41275642156,5sleep30-21swapper/312:12:423
39584092144,6sleep30-21swapper/309:22:443
41384222132,6sleep30-21swapper/312:25:023
40232462132,7sleep30-21swapper/310:27:493
39999302132,7sleep30-21swapper/310:07:373
39890012132,6sleep30-21swapper/309:55:023
39794162132,7sleep30-21swapper/309:47:323
39348912132,7sleep30-21swapper/309:02:363
41525042122,6sleep30-21swapper/312:37:423
41359502122,6sleep30-21swapper/312:22:423
41287312122,6sleep30-21swapper/312:16:483
41213282122,6sleep30-21swapper/312:07:413
41142822120,8sleep30-21swapper/312:02:353
41142822120,8sleep30-21swapper/312:02:353
41041162122,6sleep30-21swapper/311:52:283
40988342122,5sleep30-21swapper/311:45:013
40851002122,6sleep30-21swapper/311:32:373
40820962122,6sleep30-21swapper/311:27:403
40743972121,7sleep30-21swapper/311:21:343
40569842122,6sleep30-21swapper/311:02:403
40497532122,6sleep30-21swapper/310:57:383
40347132122,6sleep30-21swapper/310:42:333
40321222122,6sleep30-21swapper/310:37:403
40239172122,5sleep30-21swapper/310:30:013
40182452121,7sleep30-21swapper/310:22:473
39697642122,6sleep30-21swapper/309:37:323
39641802122,5sleep30-21swapper/309:30:023
39596272122,6sleep30-21swapper/309:27:293
39477472122,6sleep30-21swapper/309:12:403
39296862123,5sleep30-21swapper/308:57:363
39193242122,6sleep30-21swapper/308:47:323
39139932122,6sleep30-21swapper/308:40:023
39085412122,6sleep30-21swapper/308:32:523
38985412122,6sleep30-21swapper/308:22:483
38985412122,6sleep30-21swapper/308:22:483
38905572122,6sleep30-21swapper/308:17:393
38868902122,5sleep30-21swapper/308:12:413
38784892122,6sleep30-21swapper/308:02:423
38590182122,6sleep30-21swapper/307:42:513
38493912122,5sleep30-21swapper/307:35:013
38403982122,6sleep30-21swapper/307:27:393
38277852122,6sleep30-21swapper/307:15:033
38235312122,6sleep30-21swapper/307:12:383
41437862112,6sleep30-21swapper/312:31:283
40941552112,6sleep30-21swapper/311:41:163
40891082112,5sleep30-21swapper/311:35:023
40692442112,5sleep30-21swapper/311:17:003
40641562112,5sleep30-21swapper/311:10:023
40637772112,6sleep30-21swapper/311:08:103
40441392112,6sleep30-21swapper/310:50:163
40392112112,5sleep30-21swapper/310:45:023
40099002112,5sleep30-21swapper/310:17:333
40046292112,5sleep30-21swapper/310:12:363
39847252112,5sleep30-21swapper/309:52:373
39496182112,5sleep30-21swapper/309:16:503
39246852112,6sleep30-21swapper/308:52:383
39130452112,5sleep30-21swapper/308:37:413
39002252112,6sleep30-21swapper/308:27:383
38842162112,5sleep30-21swapper/308:08:483
38645312112,5sleep30-21swapper/307:50:383
38642202112,6sleep30-21swapper/307:49:383
38446502112,5sleep30-21swapper/307:30:033
41086012102,5sleep30-21swapper/311:55:013
41086012102,5sleep30-21swapper/311:55:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional