You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-12 - 12:41

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack2slot7s.osadl.org (updated Sat Jul 12, 2025 00:41:00)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
378930399502,42cyclictest3864179-21ssh22:03:100
378930399502,42cyclictest3864179-21ssh22:03:100
3789305994939,6cyclictest3796790-21idleruntime-cro19:35:012
3789305994838,7cyclictest422-21dbus-daemon22:45:022
3789305994739,5cyclictest3923699-21munin-run23:25:012
3789305994732,4cyclictest380439550kworker/u8:6+xprtiod20:35:012
3789304994739,5cyclictest3972527-21systemd-run00:35:011
3789304994739,5cyclictest3972527-21systemd-run00:35:011
3789303994732,11cyclictest12450irq/32-eth023:18:040
3789303994725,13cyclictest12450irq/32-eth023:58:030
378930399471,38cyclictest3856825-21sh21:53:070
3789305994638,5cyclictest1-21systemd23:30:012
378930399462,38cyclictest3882238-21ssh22:27:580
378930399462,38cyclictest151ktimers/000:02:590
3789305994535,6cyclictest3792343-21latency_hist19:20:012
3789305994436,5cyclictest3836464-21systemd-run21:25:022
3789305994435,5cyclictest3949213-21cups-browsed00:28:092
3789305994435,5cyclictest3949213-21cups-browsed00:28:092
3789305994432,9cyclictest3975452-21munin-plugin-st00:40:012
3789303994423,14cyclictest151ktimers/021:07:580
3789305994335,5cyclictest3955500-21unixbench_multi00:08:102
3789304994335,5cyclictest3861837-21systemctl22:00:011
378930399433,31cyclictest3827638-21ssh21:12:570
3789303994323,12cyclictest12450irq/32-eth022:48:060
3789303994230,9cyclictest3842876-21sed21:33:130
3789305994133,5cyclictest3920029-21cron23:20:012
3789305994133,5cyclictest3847338-21idleruntime-cro21:40:012
3789304994132,6cyclictest3834593-21fschecks_time21:22:551
3789303994124,11cyclictest151ktimers/019:33:020
3789305994034,4cyclictest3789302-21cyclictest21:00:012
3789305994033,4cyclictest3797017-21kworker/u8:1-xprtiod19:45:002
3789305994032,5cyclictest3936371-21cut23:42:502
3789305994032,5cyclictest3798337-21latency_hist19:40:022
3789304994032,5cyclictest3969724-21latency_hist00:30:021
3789304994032,5cyclictest3969724-21latency_hist00:30:021
3789304994026,10cyclictest422-21dbus-daemon21:45:021
3789305993933,3cyclictest0-21swapper/222:12:592
3789305993931,5cyclictest3816831-21fschecks_time20:42:542
378930399393,26cyclictest171rcu_preempt00:22:590
378930399393,26cyclictest171rcu_preempt00:22:590
3789303993924,11cyclictest12450irq/32-eth022:12:580
378930399391,32cyclictest151ktimers/019:58:040
3789305993831,3cyclictest0-21swapper/221:13:012
3789305993830,5cyclictest3899792-21aten_r2power_fr22:52:502
3789305993829,6cyclictest3911139-21missed_timers23:07:582
3789305993816,8cyclictest3799815-21kworker/u8:3+events_unbound20:07:592
3789304993831,4cyclictest3949165-21awk00:00:011
3789304993823,3cyclictest171rcu_preempt22:13:021
3789303993827,8cyclictest12450irq/32-eth023:38:040
3789303993826,8cyclictest3925130-21/usr/sbin/munin23:28:110
378930399381,8cyclictest171rcu_preempt20:58:040
3789305993729,6cyclictest351ktimers/220:38:072
3789305993729,5cyclictest3896631-21missed_timers22:47:562
3789305993728,6cyclictest956-21mta-sts-daemon22:25:582
3789305993727,7cyclictest3971935-21unixbench_singl00:33:122
3789305993727,7cyclictest3971935-21unixbench_singl00:33:122
3789305993726,9cyclictest3841491-21sh21:32:472
3789305993722,6cyclictest2016377-21cups-browsed21:45:532
3789304993731,3cyclictest0-21swapper/120:45:011
3789304993730,3cyclictest0-21swapper/121:48:051
3789304993728,6cyclictest421-21avahi-daemon22:47:031
3789304993724,9cyclictest3944047-21/usr/sbin/munin23:53:001
3789303993729,6cyclictest955-21mta-sts-daemon22:38:030
3789303993729,5cyclictest3929744-21sendmail_mailqu23:33:060
3789303993726,8cyclictest3940020-21fschecks_time23:47:510
3789303993726,8cyclictest3940020-21fschecks_time23:47:510
3789303993722,9cyclictest12450irq/32-eth020:27:580
3789305993631,2cyclictest0-21swapper/222:07:582
3789305993631,2cyclictest0-21swapper/222:07:582
3789305993629,4cyclictest3810273-21cron20:20:012
3789305993629,3cyclictest0-21swapper/200:00:022
3789305993628,5cyclictest3950763-21aten_r2power_po00:02:502
3789305993628,5cyclictest3823834-21munin-plugin-st21:05:012
3789305993627,7cyclictest3880068-21munin-plugin-st22:25:012
3789304993631,2cyclictest0-21swapper/100:40:011
3789304993629,5cyclictest0-21swapper/123:12:521
3789304993628,5cyclictest421-21avahi-daemon23:02:491
3789303993628,5cyclictest0-21swapper/022:25:010
3789303993625,8cyclictest0-21swapper/000:32:500
3789303993625,8cyclictest0-21swapper/000:32:500
378930399361,29cyclictest3814192-21sshd20:32:580
3789305993529,3cyclictest0-21swapper/222:05:022
3789305993529,3cyclictest0-21swapper/222:05:022
3789305993528,4cyclictest3885785-21ntp_kernel_err22:32:572
3789305993527,5cyclictest3944690-21head23:53:122
3789305993527,5cyclictest3930981-21cron23:35:012
3789304993530,3cyclictest955-21mta-sts-daemon22:43:301
3789304993530,2cyclictest0-21swapper/100:10:011
3789304993529,3cyclictest0-21swapper/120:07:581
3789304993529,3cyclictest0-21swapper/119:40:011
3789304993527,5cyclictest3845777-21needreboot21:37:581
3789304993526,6cyclictest421-21avahi-daemon23:09:051
3789304993526,6cyclictest2016377-21cups-browsed21:50:171
378930399355,23cyclictest3867677-21ssh22:08:010
378930399355,23cyclictest3867677-21ssh22:08:010
3789303993529,4cyclictest3840104-21systemd-run21:30:020
3789303993527,5cyclictest421-21avahi-daemon23:52:540
3789303993526,4cyclictest0-21swapper/022:42:550
378930399352,29cyclictest3860422-21ssh21:58:040
3789303993520,10cyclictest12450irq/32-eth022:18:010
3789305993428,3cyclictest0-21swapper/223:12:522
3789305993428,3cyclictest0-21swapper/221:20:012
3789305993425,6cyclictest421-21avahi-daemon21:41:062
3789305993425,6cyclictest2016375-21cupsd21:29:002
3789304993429,2cyclictest0-21swapper/122:30:021
3789304993428,3cyclictest0-21swapper/122:40:011
3789304993426,5cyclictest3887339-21systemd-run22:35:021
3789304993426,5cyclictest3876410-21ssh22:20:011
3789304993426,5cyclictest3804283-21systemctl20:00:011
3789303993425,6cyclictest3846300-21timerandwakeup21:38:120
3789303993423,8cyclictest0-21swapper/021:23:080
3789303993420,9cyclictest12450irq/32-eth023:42:570
3789303993417,12cyclictest151ktimers/019:48:000
3789305993328,2cyclictest0-21swapper/222:20:012
3789305993327,3cyclictest0-21swapper/200:20:012
3789304993325,5cyclictest956-21mta-sts-daemon00:18:031
3789304993325,5cyclictest421-21avahi-daemon23:27:461
3789304993325,5cyclictest3949213-21cups-browsed00:13:381
3789304993325,5cyclictest3865494-21munin-run22:05:011
3789304993325,5cyclictest3865494-21munin-run22:05:011
3789304993325,5cyclictest3827473-21latency21:12:541
3789304993325,5cyclictest2016377-21cups-browsed23:16:461
3789304993325,5cyclictest2016377-21cups-browsed20:33:471
3789304993323,7cyclictest3811152-21ntp_kernel_err20:23:011
3789303993327,4cyclictest3854591-21systemd-run21:50:020
3789303993325,5cyclictest3900978-21unixbench_multi22:53:130
3789303993319,9cyclictest151ktimers/021:18:030
378930399331,27cyclictest1-21systemd22:32:570
3789305993224,5cyclictest3861832-21munin-run22:00:012
3789305993224,5cyclictest1-21systemd00:25:022
3789305993224,5cyclictest1-21systemd00:25:022
3789305993221,9cyclictest421-21avahi-daemon23:46:402
3789305993221,9cyclictest421-21avahi-daemon23:46:402
378930499327,5cyclictest261rcuc/119:28:021
3789304993226,3cyclictest0-21swapper/123:22:581
3789304993226,3cyclictest0-21swapper/121:20:011
3789304993224,5cyclictest3936384-21grep23:42:501
3789304993224,5cyclictest3879142-21unixbench-2d22:23:111
3789304993224,5cyclictest3822273-21unixbench_singl20:58:101
3789304993224,5cyclictest2016377-21cups-browsed20:02:561
3789304993223,6cyclictest453-21systemd-logind20:20:021
3789303993225,5cyclictest3958947-21unixbench_multi00:13:120
3789303993225,5cyclictest3911091-21latency23:07:570
3789303993222,7cyclictest3816215-21head20:38:140
378930399321,27cyclictest3922205-21ssh23:22:590
378930399320,29cyclictest3907831-21ssh23:03:080
3789305993125,3cyclictest0-21swapper/223:02:492
3789305993123,5cyclictest1-21systemd19:15:012
3789304993122,6cyclictest422-21dbus-daemon23:50:011
3789304993122,6cyclictest422-21dbus-daemon23:50:011
3789303993123,6cyclictest3955354-21timerandwakeup00:08:090
3789303993123,5cyclictest3962896-21munin-run00:20:010
3789303993118,4cyclictest3969638-21ssh00:30:000
3789303993118,4cyclictest3969638-21ssh00:30:000
3789305993022,5cyclictest3813187-21unixbench_multi20:28:102
3789304993023,4cyclictest3825329-21diskmemload21:30:031
3789304993022,4cyclictest0-21swapper/122:07:581
3789304993022,4cyclictest0-21swapper/122:07:581
3789304993021,6cyclictest421-21avahi-daemon23:32:101
3789303993020,4cyclictest3795519-21kworker/u8:14+xprtiod20:20:020
378930399301,23cyclictest3849563-21ssh21:43:040
3789305992923,3cyclictest0-21swapper/200:10:312
3789305992921,5cyclictest421-21avahi-daemon21:53:132
3789304992923,4cyclictest271ktimers/119:13:001
3789304992923,3cyclictest0-21swapper/120:40:011
3789304992921,5cyclictest3900002-21fschecks_time22:52:541
3789303992923,4cyclictest3916352-21ssh23:15:010
3789303992912,13cyclictest421-21avahi-daemon19:52:590
3789305992822,3cyclictest3784121-21kworker/u8:5-xprtiod19:29:362
3789305992822,3cyclictest0-21swapper/222:38:022
3789305992821,4cyclictest955-21mta-sts-daemon23:36:422
3789305992821,4cyclictest3820735-21unixbench_multi20:53:102
3789304992822,3cyclictest0-21swapper/121:30:011
3789304992821,4cyclictest3823414-21sendmail_mailqu21:03:061
3789304992821,4cyclictest3819855-21fschecks_count20:52:511
3789304992821,4cyclictest3799815-21kworker/u8:3-xprtiod19:45:011
3789304992821,4cyclictest3791325-21fschecks_time19:17:521
3789304992820,5cyclictest491-21gdbus23:37:301
3789304992819,6cyclictest422-21dbus-daemon19:34:171
3789304992811,3cyclictest171rcu_preempt00:22:391
3789304992811,3cyclictest171rcu_preempt00:22:391
378930599278,3cyclictest171rcu_preempt22:58:072
3789305992720,3cyclictest0-21swapper/221:10:012
3789304992721,4cyclictest3789302-21cyclictest00:03:121
3789304992720,4cyclictest3880309-21kworker/u8:5-rpciod22:55:581
3789304992719,5cyclictest3819264-21unixbench_singl20:48:101
378930399276,12cyclictest957-21mta-sts-daemon00:38:010
3789303992721,4cyclictest3822808-21cpuspeed21:02:510
3789303992720,5cyclictest0-21swapper/019:20:000
3789303992720,4cyclictest433-21polkitd22:56:020
378930599264,4cyclictest171rcu_preempt19:58:042
378930599262,20cyclictest3808151-21sshd20:12:572
3789305992620,3cyclictest0-21swapper/219:55:012
378930499265,7cyclictest171rcu_preempt20:12:571
3789304992618,6cyclictest3825331-21/usr/sbin/munin21:10:021
3789304992618,6cyclictest3800133-21aten_r2power_fr19:47:511
378930499260,3cyclictest171rcu_preempt20:27:571
3789303992615,9cyclictest0-21swapper/019:13:090
3789305992516,6cyclictest2016375-21cupsd20:23:092
3789304992510,10cyclictest422-21dbus-daemon19:52:591
3789303992513,10cyclictest0-21swapper/020:48:080
378930399250,18cyclictest3797677-21modprobe19:37:580
3789305992410,6cyclictest36-21ksoftirqd/220:47:582
3789303992416,5cyclictest956-21mta-sts-daemon20:08:010
3789303992412,5cyclictest956-21mta-sts-daemon20:12:570
378930599231,6cyclictest955-21mta-sts-daemon20:05:022
3789304992317,3cyclictest0-21swapper/119:25:001
378930399233,18cyclictest0-21swapper/019:27:570
3789303992315,5cyclictest957-21mta-sts-daemon19:23:060
3789303992310,9cyclictest422-21dbus-daemon20:25:020
378930599222,14cyclictest3793186-21ntpq19:22:562
378930599221,3cyclictest171rcu_preempt19:48:022
3789304992214,5cyclictest3789324-21latency_hist19:10:021
378930399222,15cyclictest3817272-21ntp_states20:43:060
378930399222,15cyclictest3805281-21ntp_states20:03:040
378930399222,14cyclictest3799239-21ntpq19:43:050
3789303992216,3cyclictest0-21swapper/020:51:580
378930399218,9cyclictest0-21swapper/019:10:030
378930699202,10cyclictest0-21swapper/321:13:013
378930699201,9cyclictest0-21swapper/322:08:053
378930699201,9cyclictest0-21swapper/322:08:053
3789305992014,4cyclictest956-21mta-sts-daemon19:10:022
378930699192,8cyclictest0-21swapper/322:43:003
378930699191,9cyclictest0-21swapper/322:12:583
378930699183,9cyclictest0-21swapper/322:38:043
378930699183,8cyclictest0-21swapper/319:52:593
378930699182,8cyclictest0-21swapper/300:18:003
378930699172,9cyclictest0-21swapper/322:03:063
378930699172,9cyclictest0-21swapper/322:03:063
378930699164,8cyclictest0-21swapper/323:38:043
378930699163,8cyclictest0-21swapper/323:22:583
378930699163,8cyclictest0-21swapper/321:58:003
378930699162,8cyclictest0-21swapper/322:47:583
378930699162,8cyclictest0-21swapper/321:48:033
378930699162,8cyclictest0-21swapper/300:32:593
378930699162,8cyclictest0-21swapper/300:32:593
378930699162,5cyclictest0-21swapper/320:38:023
378930699161,8cyclictest0-21swapper/323:03:033
378930699161,8cyclictest0-21swapper/321:37:593
378930699161,8cyclictest0-21swapper/300:03:033
378930699161,7cyclictest0-21swapper/320:23:013
378930699161,10cyclictest0-21swapper/321:42:483
378930699161,10cyclictest0-21swapper/321:28:023
378930699161,10cyclictest0-21swapper/300:38:023
378930699153,9cyclictest0-21swapper/322:57:503
378930699153,7cyclictest0-21swapper/323:32:583
378930699152,8cyclictest0-21swapper/323:12:593
378930699152,8cyclictest0-21swapper/322:27:573
378930699152,8cyclictest0-21swapper/322:22:533
378930699152,8cyclictest0-21swapper/321:53:043
378930699152,8cyclictest0-21swapper/321:23:043
378930699152,8cyclictest0-21swapper/300:27:593
378930699152,8cyclictest0-21swapper/300:27:593
378930699152,6cyclictest0-21swapper/321:07:573
378930699152,5cyclictest0-21swapper/320:12:573
378930699151,9cyclictest0-21swapper/322:30:203
378930699151,9cyclictest0-21swapper/321:32:593
378930699151,8cyclictest0-21swapper/323:28:043
378930699151,7cyclictest0-21swapper/319:13:013
378930699151,7cyclictest0-21swapper/300:22:573
378930699151,7cyclictest0-21swapper/300:22:573
378930699151,6cyclictest0-21swapper/319:27:573
378930699150,8cyclictest0-21swapper/322:18:013
378930699144,7cyclictest0-21swapper/323:08:083
378930699143,6cyclictest0-21swapper/322:53:013
378930699142,8cyclictest0-21swapper/300:00:023
378930699142,7cyclictest0-21swapper/300:12:493
378930699141,8cyclictest0-21swapper/323:42:583
378930699141,8cyclictest0-21swapper/321:17:583
378930699141,7cyclictest0-21swapper/321:02:593
378930699141,6cyclictest0-21swapper/319:22:543
378930699133,7cyclictest0-21swapper/323:47:513
378930699133,7cyclictest0-21swapper/323:47:513
378930699132,7cyclictest0-21swapper/300:06:083
378930699132,5cyclictest0-21swapper/320:27:573
378930699131,8cyclictest0-21swapper/323:17:563
378930699131,8cyclictest0-21swapper/320:15:573
378930699131,6cyclictest0-21swapper/320:52:573
378930699131,6cyclictest0-21swapper/319:57:593
378930699131,6cyclictest0-21swapper/319:43:083
378930699131,5cyclictest0-21swapper/320:57:573
378930699123,6cyclictest0-21swapper/319:48:013
378930699122,7cyclictest0-21swapper/320:32:563
378930699122,5cyclictest0-21swapper/320:02:573
378930699121,7cyclictest0-21swapper/323:52:543
378930699121,7cyclictest0-21swapper/320:07:583
378930699112,5cyclictest0-21swapper/319:40:013
378930699112,5cyclictest0-21swapper/319:32:483
378930699111,7cyclictest0-21swapper/320:50:013
378930699111,6cyclictest0-21swapper/319:17:583
378930699104,5cyclictest0-21swapper/320:43:023
37893069981,4cyclictest0-21swapper/319:10:033
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional