You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-15 - 04:48
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack9slot2.osadl.org (updated Mon Dec 15, 2025 00:43:25)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
170662316248,54sleep30-21swapper/33
171152285255,18sleep20-21swapper/22
168952268196,40sleep10-21swapper/11
169342261190,25sleep00-21swapper/00
1739599233227,4cyclictest10489-21sleep3
1738699152144,6cyclictest292-21jbd2/mmcblk0p2-1
1431521490,2sleep31739599cyclictest3
1737899130125,3cyclictest10449-21cron0
173909910598,5cyclictest518-21in:imuxsock2
246602660,4sleep024653-21cron0
81472450,2sleep38148-21unixbench_singl3
316562420,1sleep20-21swapper/22
26482410,2sleep32651-21latency_hist3
241292400,1sleep00-21swapper/00
17395993013,4cyclictest35-21ksoftirqd/33
1739599290,27cyclictest21302-21kworker/3:13
17395992816,2cyclictest35-21ksoftirqd/33
17395992812,14cyclictest35-21ksoftirqd/33
17395992723,3cyclictest341rcuc/33
17395992715,11cyclictest35-21ksoftirqd/33
17395992617,3cyclictest35-21ksoftirqd/33
17395992613,11cyclictest35-21ksoftirqd/33
1739599255,17cyclictest14021-21sendmail-mta3
17395992510,4cyclictest35-21ksoftirqd/33
1738699258,2cyclictest14783-21/usr/sbin/munin1
17386992515,6cyclictest517-21dbus-daemon1
17395992416,3cyclictest35-21ksoftirqd/33
17395992411,11cyclictest35-21ksoftirqd/33
17395992411,11cyclictest35-21ksoftirqd/33
17390992417,4cyclictest24662-21sh2
241602230,1sleep20-21swapper/22
230742230,2sleep30-21swapper/33
1739599233,18cyclictest7974-21sendmail-mta3
1739599232,18cyclictest22719-21cat3
1739599232,18cyclictest0-21swapper/33
17395992311,3cyclictest35-21ksoftirqd/33
1739599230,20cyclictest6078-21munin-run3
1739599229,11cyclictest29459-21forks3
1739599224,16cyclictest30081-21sendmail-mta3
1739599224,16cyclictest28617-21sendmail-mta3
1739599224,16cyclictest26031-21sendmail-mta3
1739599224,16cyclictest12461-21sendmail-mta3
1739599223,17cyclictest4497-21awk3
1739599223,17cyclictest3904-21sendmail-mta3
1739599223,17cyclictest27892-21sendmail-mta3
1739599223,17cyclictest24565-21sendmail-mta3
1739599223,17cyclictest23839-21sendmail-mta3
1739599223,17cyclictest21948-21sendmail-mta3
17395992219,2cyclictest341rcuc/33
1739599221,4cyclictest6820-1kworker/3:0H3
1739599221,4cyclictest5672-21taskset3
17395992214,2cyclictest35-21ksoftirqd/33
1739599220,4cyclictest6820-1kworker/3:0H3
17386992217,3cyclictest657-21/usr/sbin/munin1
17378992215,4cyclictest662-21runrttasks0
29682210,3sleep0101rcuc/00
260392210,2sleep20-21swapper/22
195672210,2sleep10-21swapper/11
1739599218,11cyclictest0-21swapper/33
1739599213,16cyclictest6506-21latency_hist3
17395992115,4cyclictest23101-21memory3
17395992114,5cyclictest32463-21unixbench_singl3
1739599211,3cyclictest6820-1kworker/3:0H3
1739599211,3cyclictest6820-1kworker/3:0H3
1739599211,3cyclictest6820-1kworker/3:0H3
1739599211,18cyclictest31844-21apt-get3
1739599210,19cyclictest0-21swapper/33
1739599210,18cyclictest22670-21munin-run3
17390992115,4cyclictest7858-21sendmail2
17390992112,4cyclictest10721-21apt-get2
17386992114,5cyclictest31328-21grep1
17386992114,4cyclictest662-21runrttasks1
17386992114,4cyclictest31448-21grep1
17386992114,4cyclictest16790-21sed1
17386992114,4cyclictest11919-21cron1
17378992116,3cyclictest6545-21perf0
1739599209,3cyclictest111rcu_preempt3
1739599209,10cyclictest35-21ksoftirqd/33
1739599207,11cyclictest0-21swapper/33
1739599202,16cyclictest12332-21sendmail-mta3
17395992015,3cyclictest7346-21unixbench-2d3
1739599201,17cyclictest0-21swapper/33
1739599200,3cyclictest6820-1kworker/3:0H3
1739599200,18cyclictest14579-21latency_hist3
17390992017,2cyclictest0-21swapper/22
17390992015,3cyclictest517-21dbus-daemon2
17390992015,3cyclictest517-21dbus-daemon2
17390992015,3cyclictest3068-21ntpq2
17390992014,4cyclictest16938-21cut2
17390992013,5cyclictest7994-21seq2
17390992013,5cyclictest21947-21cron2
17390992013,4cyclictest26013-21sh2
17390992013,4cyclictest14539-21basename2
17390992010,6cyclictest26442-21apt-get2
1738699203,10cyclictest17234-21apt-get1
1738699202,16cyclictest5816-21cron1
17386992015,3cyclictest5848-21seq1
17386992015,3cyclictest24229-21/usr/sbin/munin1
17386992015,3cyclictest19694-21sendmail_mailst1
17386992015,3cyclictest18806-21/usr/sbin/munin1
17386992014,4cyclictest21300-21grep1
17386992014,4cyclictest18600-21cron1
17386992013,4cyclictest571-21gdbus1
17386992012,5cyclictest517-21dbus-daemon1
17378992015,4cyclictest24180-21basename0
17378992015,3cyclictest634-21snmpd0
17378992014,4cyclictest7643-21seq0
17378992014,4cyclictest18164-21seq0
17378992014,4cyclictest14559-21seq0
145452200,4sleep014516-21munin-run0
1739599199,3cyclictest111rcu_preempt3
1739599198,3cyclictest111rcu_preempt3
1739599198,2cyclictest111rcu_preempt3
1739599191,4cyclictest32318-21munin-run3
17395991914,3cyclictest24295-21aten2_r9power_e3
17395991914,3cyclictest20212-21aten2_r9power_e3
17395991914,3cyclictest0-21swapper/33
17395991913,4cyclictest16433-21cat3
1739599191,16cyclictest10612-21awk3
1739099199,6cyclictest28471-21apt-get2
1739099198,9cyclictest4226-21acpi2
17390991916,2cyclictest0-21swapper/22
17390991914,3cyclictest9087-21proc_pri2
17390991914,3cyclictest4431-21ls2
17390991914,3cyclictest30895-21ls2
17390991914,3cyclictest30571-21expr2
17390991913,4cyclictest12759-21cat2
17390991913,4cyclictest12759-21cat2
17390991913,4cyclictest10733-21sed2
17390991913,3cyclictest26116-21cut2
17390991911,5cyclictest571-21gdbus2
1739099191,15cyclictest32457-21cat2
1738699199,7cyclictest517-21dbus-daemon1
1738699191,6cyclictest24698-21sendmail-mta1
17386991915,2cyclictest6339-21/usr/sbin/munin1
17386991915,2cyclictest14917-21/usr/sbin/munin1
17386991914,3cyclictest7533-21unixbench-2d1
17386991914,3cyclictest6063-21acpi1
17386991914,3cyclictest29138-21sed1
17386991914,3cyclictest29138-21sed1
17386991914,3cyclictest27202-21sed1
17386991913,4cyclictest29004-21seq1
17386991913,4cyclictest27721-21sed1
17386991913,4cyclictest21615-21sed1
17386991912,4cyclictest517-21dbus-daemon1
17386991911,5cyclictest517-21dbus-daemon1
17386991911,5cyclictest28335-21grep1
17386991911,5cyclictest26753-21sh1
17378991915,2cyclictest26258-21/usr/sbin/munin0
17378991915,2cyclictest24864-21if_err_enp3s00
17378991915,2cyclictest17252-21/usr/sbin/munin0
17378991915,2cyclictest10137-21/usr/sbin/munin0
17378991914,3cyclictest7995-21timerwakeupswit0
17378991914,3cyclictest4295-21/usr/sbin/munin0
17378991914,3cyclictest30811-21sh0
17378991914,3cyclictest21808-21seq0
17378991914,3cyclictest20144-21/usr/sbin/munin0
17378991914,3cyclictest1725-21which0
17378991913,4cyclictest9872-21seq0
17378991913,4cyclictest517-21dbus-daemon0
17378991913,4cyclictest32749-21cat0
17378991913,4cyclictest18885-21awk0
17378991913,4cyclictest16088-21grep0
17378991913,4cyclictest13911-21seq0
17378991913,4cyclictest11338-21which0
17378991913,4cyclictest11338-21which0
17378991913,4cyclictest10409-21seq0
17378991913,3cyclictest7011-21sed0
1737899191,16cyclictest16239-21tr0
1737899190,4cyclictest0-21swapper/00
17395991813,3cyclictest13749-21ntp_kernel_pll_3
1739599180,16cyclictest0-21swapper/33
1739599180,16cyclictest0-21swapper/33
1739099187,2cyclictest23780-21taskset2
17390991816,1cyclictest0-21swapper/22
17390991816,1cyclictest0-21swapper/22
17390991816,1cyclictest0-21swapper/22
17390991816,1cyclictest0-21swapper/22
17390991814,2cyclictest6882-21aten2.4-expect2
17390991814,2cyclictest5190-21ntp_states2
17390991814,2cyclictest2005-21/usr/sbin/munin2
17390991814,2cyclictest16714-21gltestperf2
17390991813,3cyclictest662-21runrttasks2
17390991813,3cyclictest662-21runrttasks2
17390991813,3cyclictest29283-21cut2
17390991813,3cyclictest20781-21gltestperf2
17390991813,3cyclictest1405-21seq2
17390991812,4cyclictest6198-21seq2
17390991812,4cyclictest3423-21sed2
17390991812,4cyclictest23782-21seq2
17390991812,4cyclictest20737-21cut2
17390991812,3cyclictest13080-21ls2
1739099181,13cyclictest17527-21apt-get2
17390991810,5cyclictest517-21dbus-daemon2
1739099180,16cyclictest0-21swapper/22
1738699183,9cyclictest2212-21apt-get1
1738699182,9cyclictest12897-21apt-get1
1738699182,10cyclictest25981-21runrttasks1
1738699182,10cyclictest19311-21chrt1
17386991814,2cyclictest28586-21/usr/sbin/munin1
17386991814,2cyclictest22345-21/usr/sbin/munin1
17386991813,3cyclictest8342-21seq1
17386991813,3cyclictest32660-21/usr/sbin/munin1
17386991813,3cyclictest29548-21awk1
17386991812,4cyclictest9966-21sendmail-mta1
17386991812,4cyclictest5184-21ntp_states1
17386991812,4cyclictest517-21dbus-daemon1
17386991812,4cyclictest3160-21cat1
17386991812,4cyclictest24401-21/usr/sbin/munin1
17386991812,3cyclictest2353-21aten2.4-expect1
17386991811,4cyclictest517-21dbus-daemon1
17386991811,4cyclictest32432-21idleruntime-cro1
17386991811,4cyclictest0-21swapper/11
1738699180,17cyclictest0-21swapper/11
1738699180,16cyclictest0-21swapper/11
1737899181,6cyclictest26031-21apt-get0
17378991814,3cyclictest20835-21/usr/sbin/munin0
17378991814,2cyclictest4039-21/usr/sbin/munin0
17378991814,2cyclictest30513-21ls0
17378991814,2cyclictest23667-21/usr/sbin/munin0
17378991814,2cyclictest2005-21/usr/sbin/munin0
17378991814,2cyclictest18289-21/usr/sbin/munin0
17378991814,2cyclictest15090-21ntp_states0
17378991813,3cyclictest6258-21find0
17378991813,3cyclictest340-21systemd-journal0
17378991813,3cyclictest24763-21expr0
17378991813,3cyclictest24380-21seq0
17378991813,3cyclictest1313-21cat0
17378991813,3cyclictest10779-21cut0
17378991812,4cyclictest517-21dbus-daemon0
17378991812,4cyclictest31878-21grep0
17378991812,4cyclictest31328-21cat0
17378991812,4cyclictest30910-21grep0
17378991812,4cyclictest3065-21ntp_kernel_pll_0
17378991812,4cyclictest29619-21seq0
17378991812,4cyclictest29619-21seq0
17378991812,4cyclictest22317-21cat0
17378991812,4cyclictest14175-21date0
17378991812,4cyclictest11238-21expr0
17390991715,1cyclictest0-21swapper/22
17390991713,2cyclictest662-21runrttasks2
17390991713,2cyclictest19385-21swap2
17390991713,2cyclictest16700-21latency_hist2
17390991713,2cyclictest16167-21latency_hist2
17390991713,2cyclictest12866-21expr2
17390991712,3cyclictest662-21runrttasks2
17390991712,3cyclictest662-21runrttasks2
17390991712,3cyclictest24616-21cat2
17390991712,3cyclictest22503-21uname2
17390991712,3cyclictest11386-21idleruntime2
17390991711,4cyclictest10261-21cut2
17390991710,4cyclictest19137-21grep2
1739099170,16cyclictest0-21swapper/22
1738699172,10cyclictest634-21snmpd1
17386991715,1cyclictest0-21swapper/11
17386991715,1cyclictest0-21swapper/11
17386991715,1cyclictest0-21swapper/11
17386991715,1cyclictest0-21swapper/11
17386991713,2cyclictest10360-21/usr/sbin/munin1
17386991713,2cyclictest10360-21/usr/sbin/munin1
17386991712,3cyclictest6296-21seq1
17386991712,3cyclictest16142-21sed1
17386991711,4cyclictest32583-21/usr/sbin/munin1
17386991710,5cyclictest517-21dbus-daemon1
17378991714,2cyclictest18806-21/usr/sbin/munin0
17378991713,3cyclictest4219-21/usr/sbin/munin0
17378991713,2cyclictest31063-21/usr/sbin/munin0
17378991712,3cyclictest8673-21cat0
17378991712,3cyclictest28109-21seq0
17378991712,3cyclictest26921-21cut0
17378991712,3cyclictest21873-21which0
17378991712,3cyclictest17521-21cat0
17378991711,4cyclictest517-21dbus-daemon0
17378991711,4cyclictest2703-21sh0
17378991711,4cyclictest2193-21cat0
1739099165,9cyclictest19775-21grep2
1739099162,8cyclictest677-21apt-get2
1739099160,15cyclictest0-21swapper/22
17386991611,3cyclictest22069-21awk1
17386991610,4cyclictest517-21dbus-daemon1
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional