You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-15 - 16:22
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack9slot2.osadl.org (updated Mon Dec 15, 2025 12:43:24)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
284392319251,55sleep30-21swapper/33
285602298269,17sleep20-21swapper/22
285742280248,21sleep10-21swapper/11
273392275248,16sleep00-21swapper/00
2883999182178,2cyclictest0-21swapper/11
2669421820,4sleep32884699cyclictest3
2883599159154,3cyclictest545-21strings0
2884499109105,2cyclictest0-21swapper/22
288469910198,2cyclictest0-21swapper/33
147392590,10sleep121-21ksoftirqd/11
28835994713,20cyclictest27197-21seq0
28835994712,20cyclictest30641-21seq0
60902420,1sleep20-21swapper/22
218912360,1sleep10-21swapper/11
28835993313,18cyclictest6858-21seq0
28835993313,18cyclictest16729-21df0
28835993313,17cyclictest20080-21seq0
28835993311,20cyclictest3369-21cat0
28835993212,17cyclictest340-21systemd-journal0
28835993114,15cyclictest31933-21tr0
28835993112,17cyclictest23848-21seq0
28835993111,18cyclictest4460-21sendmail-mta0
28835993111,18cyclictest15910-21sendmail-mta0
28835993111,17cyclictest317-21cut0
28835993111,17cyclictest13650-21seq0
28835993110,18cyclictest11097-21seq0
2883599307,4cyclictest21501-21cat0
2883599303,25cyclictest18381-21tr0
28835993013,14cyclictest428-21/usr/sbin/munin0
28835993011,16cyclictest31372-21mailstats0
28835993011,16cyclictest26105-21aten2.4-expect0
28835993011,16cyclictest15976-21expr0
2883599300,5cyclictest21690-21apt-get0
2883599300,28cyclictest6821-21ps0
28844992915,6cyclictest28-21ksoftirqd/22
28835992911,15cyclictest12245-21/usr/sbin/munin0
2883599288,3cyclictest28795-21cat0
28835992814,11cyclictest16064-21/usr/sbin/munin0
28835992813,13cyclictest17355-21/usr/sbin/munin0
28846992714,6cyclictest11122-21runrttasks3
2884499270,1cyclictest0-21swapper/22
2883599279,3cyclictest21715-21grep0
2883599279,2cyclictest18629-21processes0
2883599279,15cyclictest29777-21cpuspeed0
2883599277,4cyclictest8527-21seq0
2883599277,3cyclictest3174-21latency_hist0
2883599277,3cyclictest3174-21latency_hist0
2883599277,3cyclictest26142-21cat0
2883599276,6cyclictest517-21dbus-daemon0
28835992714,11cyclictest20142-21/usr/sbin/munin0
28835992712,13cyclictest16691-21unixbench_singl0
28835992710,3cyclictest26602-21cat0
28835992710,15cyclictest9652-21dump-pmu-power0
2884699267,17cyclictest7625-21idleruntime-cro3
2884699262,2cyclictest0-21swapper/33
28846992621,3cyclictest28960-21/usr/sbin/munin3
2884699261,2cyclictest0-21swapper/33
2883599269,14cyclictest6415-21grep0
2883599268,3cyclictest21616-21cut0
2883599266,3cyclictest340-21systemd-journal0
2883599261,8cyclictest5593-21munin-run0
28835992611,12cyclictest11103-21aten2_r9power_v0
28846992512,6cyclictest15753-21runrttasks3
2883599258,3cyclictest5897-21gzip0
2883599258,3cyclictest5200-21sendmail-mta0
2883599258,15cyclictest0-21swapper/00
2883599257,4cyclictest27449-21irqstats0
2883599257,3cyclictest3417-21sendmail-mta0
2883599257,3cyclictest21041-21sendmail-mta0
2883599257,3cyclictest0-21swapper/00
2883599256,5cyclictest517-21dbus-daemon0
2883599256,4cyclictest517-21dbus-daemon0
2883599256,4cyclictest2281-21missed_timers0
2883599255,4cyclictest517-21dbus-daemon0
2883599252,7cyclictest14645-21ls0
2884699245,16cyclictest29922-21sendmail-mta3
2884499249,3cyclictest111rcu_preempt2
2883599248,3cyclictest11549-21sendmail-mta0
2883599248,3cyclictest10769-21cat0
2883599246,3cyclictest517-21dbus-daemon0
2883599242,6cyclictest32665-21apt-get0
2884699236,4cyclictest35-21ksoftirqd/33
2884699234,17cyclictest28430-21sendmail-mta3
2884699234,17cyclictest24054-21sendmail-mta3
2884699233,17cyclictest9257-21sendmail-mta3
2884699233,17cyclictest5200-21sendmail-mta3
2884699233,17cyclictest3746-21cat3
2884699233,17cyclictest19992-21sendmail-mta3
2884699232,15cyclictest0-21swapper/33
2884699231,7cyclictest23470-21apt3
28846992311,6cyclictest0-21swapper/33
2884499238,3cyclictest111rcu_preempt2
2884499236,3cyclictest111rcu_preempt2
28844992318,3cyclictest28-21ksoftirqd/22
28844992312,3cyclictest111rcu_preempt2
28844992311,3cyclictest111rcu_preempt2
28844992310,3cyclictest111rcu_preempt2
2883999234,16cyclictest24219-21perf1
28839992318,3cyclictest6254-21ls1
2883599237,4cyclictest517-21dbus-daemon0
2883599237,3cyclictest23916-21cmp0
2883599234,4cyclictest0-21swapper/00
2883599231,5cyclictest13337-21latency_hist0
2884699229,11cyclictest35-21ksoftirqd/33
2884699225,4cyclictest35-21ksoftirqd/33
2884699224,4cyclictest35-21ksoftirqd/33
2884699223,17cyclictest814-21sendmail-mta3
2884699223,17cyclictest7175-21cat3
2884699223,17cyclictest31763-21sendmail-mta3
2884699223,17cyclictest2884-21sendmail-mta3
2884699223,17cyclictest2884-21sendmail-mta3
2884699223,17cyclictest21041-21sendmail-mta3
2884699223,17cyclictest1862-21sendmail-mta3
2884699223,17cyclictest15173-21sendmail-mta3
28846992212,7cyclictest7345-21apt-get3
2884699221,18cyclictest19877-21munin-run3
2884699221,15cyclictest0-21swapper/33
2884499229,2cyclictest111rcu_preempt2
2884499228,3cyclictest111rcu_preempt2
2884499228,2cyclictest111rcu_preempt2
2884499227,3cyclictest111rcu_preempt2
2884499227,3cyclictest111rcu_preempt2
2884499225,3cyclictest111rcu_preempt2
2884499224,3cyclictest111rcu_preempt2
28844992211,3cyclictest28-21ksoftirqd/22
2883999224,16cyclictest2275-21ls1
28839992212,8cyclictest211ksoftirqd/11
289612210,12sleep121-21ksoftirqd/11
2884699213,16cyclictest27688-21sendmail-mta3
2884699212,13cyclictest0-21swapper/33
2884699211,4cyclictest13730-21diskstats3
28846992114,4cyclictest20816-21df_inode3
2884699211,3cyclictest32097-21munin-run3
2884699211,14cyclictest0-21swapper/33
2884699210,18cyclictest8680-21perf3
2884499218,2cyclictest111rcu_preempt2
2884499218,2cyclictest111rcu_preempt2
2884499218,2cyclictest111rcu_preempt2
2884499217,3cyclictest111rcu_preempt2
2884499217,3cyclictest111rcu_preempt2
2884499217,2cyclictest111rcu_preempt2
2884499216,3cyclictest111rcu_preempt2
2884499215,3cyclictest111rcu_preempt2
2884499214,3cyclictest111rcu_preempt2
2884499213,3cyclictest28-21ksoftirqd/22
28844992115,4cyclictest5598-21sh2
28839992114,4cyclictest31745-21aten2.4-expect1
28839992112,6cyclictest517-21dbus-daemon1
2883599215,3cyclictest9-21ksoftirqd/00
2884699202,16cyclictest29874-21cron3
28846992016,2cyclictest0-21swapper/33
28846992015,3cyclictest3300-21munin-run3
28846992015,3cyclictest12473-21ls3
28846992014,4cyclictest4370-21basename3
28846992014,3cyclictest24995-21cat3
28846992013,4cyclictest26666-21diskstats3
2884699201,17cyclictest21775-21sendmail-mta3
2884699201,16cyclictest18254-21ntp_states3
2884699200,14cyclictest0-21swapper/33
2884499209,2cyclictest111rcu_preempt2
2884499208,3cyclictest111rcu_preempt2
2884499206,3cyclictest111rcu_preempt2
2884499206,3cyclictest111rcu_preempt2
2884499206,3cyclictest111rcu_preempt2
2884499206,2cyclictest111rcu_preempt2
2884499206,2cyclictest111rcu_preempt2
2884499205,3cyclictest111rcu_preempt2
2884499205,2cyclictest111rcu_preempt2
2884499202,2cyclictest111rcu_preempt2
2884499202,2cyclictest111rcu_preempt2
28844992015,3cyclictest28-21ksoftirqd/22
28844992014,4cyclictest31927-21dump-pmu-power2
2884499201,3cyclictest28-21ksoftirqd/22
2884499201,2cyclictest111rcu_preempt2
28844992011,2cyclictest111rcu_preempt2
28839992015,3cyclictest9914-21/usr/sbin/munin1
28839992015,3cyclictest634-21snmpd1
28839992015,3cyclictest4850-21/usr/sbin/munin1
28839992015,3cyclictest22362-21cpu1
28839992015,3cyclictest1281-21aten2_r9power_c1
28839992015,3cyclictest1281-21aten2_r9power_c1
28839992014,4cyclictest21756-21grep1
28839992013,5cyclictest24965-21awk1
28839992013,4cyclictest14045-21idleruntime-cro1
28839992012,5cyclictest6989-21aten2.4-expect1
28839992012,5cyclictest31388-21sendmail-mta1
2884699193,15cyclictest0-21swapper/33
2884699192,2cyclictest35-21ksoftirqd/33
28846991915,2cyclictest8905-21/usr/sbin/munin3
28846991914,3cyclictest3531-21cpuspeed_turbos3
28846991914,3cyclictest22180-21load3
28846991914,3cyclictest1896-21vmstat3
28846991914,3cyclictest14901-21/usr/sbin/munin3
28846991913,4cyclictest5278-21df3
28846991913,4cyclictest24925-21awk3
28846991913,3cyclictest978-21/usr/sbin/munin3
28846991913,3cyclictest12562-21sendmail_mailst3
28846991912,5cyclictest7388-21dump-pmu-power3
2884699190,17cyclictest0-21swapper/33
2884499193,12cyclictest24766-21apt-get2
28844991917,1cyclictest0-21swapper/22
28844991916,2cyclictest0-21swapper/22
28844991915,3cyclictest28-21ksoftirqd/22
28844991915,3cyclictest28-21ksoftirqd/22
28844991914,3cyclictest8653-21ls2
28844991914,3cyclictest28-21ksoftirqd/22
28844991914,3cyclictest26250-21cpu2
28844991914,3cyclictest15105-21tr2
28844991913,4cyclictest25526-21cut2
28844991912,5cyclictest28324-21telnet2
2884499191,16cyclictest517-21dbus-daemon2
2884499190,3cyclictest111rcu_preempt2
28839991916,2cyclictest0-21swapper/11
28839991916,2cyclictest0-21swapper/11
2883999191,5cyclictest0-21swapper/11
28839991915,2cyclictest30962-21munin-node1
28839991915,2cyclictest1512-21/usr/sbin/munin1
28839991915,2cyclictest12009-21/usr/sbin/munin1
28839991914,3cyclictest9669-21idleruntime-cro1
28839991914,3cyclictest8905-21/usr/sbin/munin1
28839991914,3cyclictest31705-21grep1
28839991914,3cyclictest29806-21needreboot1
28839991914,3cyclictest2827-21/usr/sbin/munin1
28839991914,3cyclictest21324-21sed1
28839991914,3cyclictest18361-21/usr/sbin/munin1
28839991914,3cyclictest15560-21/usr/sbin/munin1
28839991913,4cyclictest8662-21grep1
28839991913,4cyclictest7922-21tr1
28839991913,4cyclictest29698-21awk1
28839991913,4cyclictest27034-21sed1
28839991912,4cyclictest19724-21aten2.4-expect1
28839991912,4cyclictest14236-21sort1
28839991911,5cyclictest517-21dbus-daemon1
28839991911,3cyclictest662-21runrttasks1
28835991915,2cyclictest3274-21/usr/sbin/munin0
28835991914,3cyclictest30427-21seq0
28846991815,2cyclictest0-21swapper/33
28846991813,3cyclictest2627-21expr3
28846991813,3cyclictest20557-21open_files3
28846991812,4cyclictest29732-21grep3
2884499187,3cyclictest111rcu_preempt2
2884499186,4cyclictest28-21ksoftirqd/22
28844991816,1cyclictest0-21swapper/22
28844991816,1cyclictest0-21swapper/22
28844991816,1cyclictest0-21swapper/22
28844991812,4cyclictest963-21cron2
28839991816,1cyclictest0-21swapper/11
28839991815,2cyclictest0-21swapper/11
28839991814,3cyclictest16064-21/usr/sbin/munin1
28839991814,2cyclictest3274-21/usr/sbin/munin1
28839991814,2cyclictest32647-21/usr/sbin/munin1
28839991814,2cyclictest25488-21/usr/sbin/munin1
28839991814,2cyclictest16299-21/usr/sbin/munin1
28839991814,2cyclictest10963-21/usr/sbin/munin1
28839991813,3cyclictest25220-21grep1
28839991813,3cyclictest20050-21date1
28839991813,3cyclictest17382-21cut1
28839991813,3cyclictest15746-21seq1
28839991813,3cyclictest1051-21cat1
28839991812,4cyclictest24864-21grep1
28839991812,4cyclictest19611-21ls1
28839991811,5cyclictest27119-21cpuspeed_turbos1
2884499173,3cyclictest111rcu_preempt2
28839991715,1cyclictest0-21swapper/11
28839991714,2cyclictest0-21swapper/11
28839991712,3cyclictest9515-21aten2_r9power_c1
28839991712,3cyclictest5350-21latency_hist1
28839991712,3cyclictest3066-21sed1
28839991712,3cyclictest13928-21ls1
28839991711,4cyclictest28384-21grep1
28839991711,3cyclictest340-21systemd-journal1
2884499164,4cyclictest111rcu_preempt2
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional