You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-24 - 12:18
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack9slot2.osadl.org (updated Tue Mar 24, 2026 00:43:25)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
67322279238,29sleep30-21swapper/33
75502264202,31sleep20-21swapper/22
73942254216,25sleep10-21swapper/11
75132249183,35sleep00-21swapper/00
197142940,2sleep0778999cyclictest0
67422520,2sleep30-21swapper/33
237692470,2sleep00-21swapper/00
7789993714,4cyclictest325-21systemd-journal0
198802370,1sleep30-21swapper/33
187822350,2sleep00-21swapper/00
7789993213,3cyclictest5744-21seq0
7789993213,3cyclictest16416-21seq0
36792280,2sleep0778999cyclictest0
780099272,6cyclictest2-21kthreadd3
778999250,23cyclictest13381-21kworker/0:10
38052250,2sleep10-21swapper/11
780099240,21cyclictest20956-21idleruntime-cro3
779899231,4cyclictest7550-21munin-run2
779899231,20cyclictest18406-21munin-run2
7798992310,11cyclictest29289-21perf2
7794992313,6cyclictest7617-21apt-get1
7800992215,4cyclictest30947-21sendmail3
7800992215,3cyclictest1450-21sh3
779899224,17cyclictest29120-21awk2
7798992216,4cyclictest32497-21df2
779899221,4cyclictest25704-21munin-run2
296882220,1sleep30-21swapper/33
296882220,1sleep30-21swapper/33
780099213,15cyclictest12794-21perf3
7800992118,2cyclictest0-21swapper/33
7800992115,4cyclictest29138-21sed3
7800992113,5cyclictest512-21dbus-daemon3
7800992113,5cyclictest512-21dbus-daemon3
779899212,17cyclictest22882-21gpg-connect-age2
779899212,17cyclictest21679-21procmail2
7798992116,3cyclictest27068-21dump-pmu-power2
7798992116,3cyclictest25026-21munin-run2
779899211,18cyclictest13467-21munin-run2
779899211,17cyclictest9758-1kworker/2:0H2
779899211,17cyclictest9247-21latency_hist2
7789992116,3cyclictest1447-21sh0
7789992113,5cyclictest10313-21grep0
74532210,3sleep09-21ksoftirqd/00
116312210,1sleep00-21swapper/00
7800992017,2cyclictest0-21swapper/33
7800992015,3cyclictest3985-21sed3
7800992015,3cyclictest11764-21aten2_r9power_p3
7800992014,4cyclictest453-21grep3
7800992014,4cyclictest31124-21tr3
7800992014,4cyclictest17449-21memory3
7800992013,5cyclictest512-21dbus-daemon3
7800992013,4cyclictest4248-21sed3
7800992013,4cyclictest16872-21idleruntime-cro3
7800992012,5cyclictest512-21dbus-daemon3
7800992011,5cyclictest27060-21sendmail3
7800992011,5cyclictest17561-21dump-pmu-power3
779899206,3cyclictest23039-21latency_hist2
779899206,3cyclictest23039-21latency_hist2
7798992015,3cyclictest31726-21cpu2
7798992015,3cyclictest30382-21cstates2
7798992015,3cyclictest28-21ksoftirqd/22
7798992015,3cyclictest18396-21cat2
7798992015,3cyclictest17015-21timerwakeupswit2
7798992015,3cyclictest1691-21gltestperf2
7798992015,3cyclictest10709-21sendmail_mailst2
7798992014,4cyclictest7951-21grep2
7798992014,4cyclictest21120-21df_inode2
7798992014,3cyclictest3292-21sshd2
7798992012,5cyclictest19166-21/usr/sbin/munin2
779899201,17cyclictest512-21dbus-daemon2
779899201,17cyclictest31299-21cups-browsed2
779899201,17cyclictest31299-21cups-browsed2
779899201,17cyclictest22527-1kworker/2:1H2
7798992011,5cyclictest13035-21apt-get2
779899200,4cyclictest7591-1kworker/2:2H2
779899200,19cyclictest0-21swapper/22
779499204,11cyclictest25092-21apt-get1
7794992016,2cyclictest12629-21/usr/sbin/munin1
7794992016,2cyclictest12629-21/usr/sbin/munin1
7794992015,3cyclictest32070-21/usr/sbin/munin1
7794992015,3cyclictest21441-21ls1
7794992015,3cyclictest21441-21ls1
7794992015,3cyclictest21196-21cut1
7794992015,3cyclictest14656-21/usr/sbin/munin1
7794992014,4cyclictest21635-21sh1
7794992013,5cyclictest9058-21sed1
778999203,15cyclictest13492-21idleruntime-cro0
7789992015,3cyclictest1749-21aten2_r9power_e0
7789992014,4cyclictest25820-21wc0
7789992014,4cyclictest14207-21sed0
778999200,17cyclictest2854-21apt-get0
7800991916,2cyclictest0-21swapper/33
7800991916,2cyclictest0-21swapper/33
7800991916,2cyclictest0-21swapper/33
7800991914,3cyclictest11604-21sed3
7800991913,4cyclictest9973-21expr3
7800991913,4cyclictest8966-21awk3
7800991913,4cyclictest5873-21mii-tool3
7800991913,4cyclictest512-21dbus-daemon3
7800991913,4cyclictest21863-21sed3
7800991913,4cyclictest12665-21grep3
7800991912,5cyclictest10785-21sendmail-mta3
7800991912,4cyclictest1092-21apt-key3
779899199,6cyclictest1356-21apt-get2
779899192,16cyclictest26841-21cat2
7798991917,1cyclictest0-21swapper/22
7798991915,3cyclictest17540-21df2
7798991914,3cyclictest4568-21ntp_kernel_pll_2
7798991914,3cyclictest23979-21nfs4_client2
7798991914,3cyclictest20949-21sh2
7798991914,3cyclictest1868-21sendmail_mailst2
7798991914,3cyclictest15372-21latency2
7798991914,3cyclictest14704-21nfs4_client2
7798991914,3cyclictest14418-21latency2
7798991914,3cyclictest12501-21cstates2
7798991914,3cyclictest10349-21nfs4_client2
7798991914,3cyclictest10018-21apt-key2
779899191,3cyclictest7178-21cron2
7798991913,4cyclictest30140-21cat2
7798991913,4cyclictest30140-21cat2
7798991913,4cyclictest11874-21ntpq2
779899190,17cyclictest29824-21awk2
779499192,15cyclictest2636-21grep1
7794991916,2cyclictest0-21swapper/11
7794991916,2cyclictest0-21swapper/11
7794991915,3cyclictest21-21ksoftirqd/11
7794991915,3cyclictest21-21ksoftirqd/11
7794991915,2cyclictest8898-21swap1
7794991914,3cyclictest6914-21/usr/sbin/munin1
7794991914,3cyclictest27481-21/usr/sbin/munin1
7794991914,3cyclictest25265-21/usr/sbin/munin1
7794991914,3cyclictest24847-21pmu-power1
7794991914,3cyclictest18174-21which1
7794991914,3cyclictest17490-21ntp_offset1
7794991914,3cyclictest13244-21cpuspeed1
7794991914,3cyclictest12529-21expr1
7794991913,4cyclictest12953-21sed1
779499190,17cyclictest325-21systemd-journal1
779499190,17cyclictest0-21swapper/11
7789991916,2cyclictest0-21swapper/00
7789991916,2cyclictest0-21swapper/00
7789991915,2cyclictest17799-21/usr/sbin/munin0
7789991915,2cyclictest16688-21/usr/sbin/munin0
7789991915,2cyclictest11822-21/usr/sbin/munin0
7789991914,3cyclictest9259-21apt-get0
7789991914,3cyclictest6699-21basename0
7789991914,3cyclictest5052-21idleruntime-cro0
7789991914,3cyclictest32101-21ntp_states0
7789991914,3cyclictest19632-21sh0
7789991913,4cyclictest668-21runrttasks0
7789991912,5cyclictest30766-21cat0
7789991912,5cyclictest30766-21cat0
7789991912,4cyclictest30203-21ntp_kernel_pll_0
7789991912,4cyclictest30203-21ntp_kernel_pll_0
778999191,16cyclictest0-21swapper/00
299252190,2sleep10-21swapper/11
299252190,2sleep10-21swapper/11
7800991816,1cyclictest0-21swapper/33
7800991816,1cyclictest0-21swapper/33
7800991813,3cyclictest24326-21expr3
7800991813,3cyclictest17845-21aten2.4-expect3
7800991812,4cyclictest29839-21cat3
7800991812,4cyclictest2637-21seq3
7800991812,4cyclictest12648-21head3
7800991812,4cyclictest10142-21mii-tool3
7800991812,3cyclictest16683-21aten2.4-expect3
7800991811,5cyclictest3475-21idleruntime-cro3
7800991811,4cyclictest5168-21basename3
7800991811,4cyclictest26556-21if_enp3s03
7800991811,4cyclictest26556-21if_enp3s03
7800991811,4cyclictest13980-21pmu-power3
7800991810,5cyclictest22079-21grep3
780099180,16cyclictest0-21swapper/33
7798991814,2cyclictest31509-21tr2
7798991814,2cyclictest14258-21aten2_r9power_c2
7798991813,3cyclictest6587-21unixbench_multi2
7798991813,3cyclictest4945-21hostname2
7798991813,3cyclictest3166-21expr2
7798991813,3cyclictest30937-21sh2
7798991813,3cyclictest27933-21cat2
7798991813,3cyclictest27933-21cat2
7798991813,3cyclictest17620-21open_inodes2
7798991812,4cyclictest27867-21wc2
7798991812,4cyclictest22294-21expr2
779499182,11cyclictest0-21swapper/11
7794991816,1cyclictest0-21swapper/11
7794991816,1cyclictest0-21swapper/11
7794991816,1cyclictest0-21swapper/11
7794991816,1cyclictest0-21swapper/11
7794991816,1cyclictest0-21swapper/11
7794991816,1cyclictest0-21swapper/11
7794991815,2cyclictest0-21swapper/11
7794991814,2cyclictest8955-21/usr/sbin/munin1
7794991814,2cyclictest4853-21/usr/sbin/munin1
7794991814,2cyclictest3212-21/usr/sbin/munin1
7794991814,2cyclictest23244-21/usr/sbin/munin1
7794991813,3cyclictest6105-21cat1
7794991813,3cyclictest2768-21cat1
7794991813,3cyclictest2411-21sed1
7794991813,3cyclictest20787-21/usr/sbin/munin1
7794991813,3cyclictest10987-21acpi1
7794991812,4cyclictest24273-21cat1
7794991812,4cyclictest24273-21cat1
779499181,15cyclictest15643-21sendmail1
779499180,17cyclictest0-21swapper/11
779499180,17cyclictest0-21swapper/11
779499180,17cyclictest0-21swapper/11
779499180,16cyclictest0-21swapper/11
7789991814,2cyclictest9927-21munin-node0
7789991814,2cyclictest25265-21/usr/sbin/munin0
7789991814,2cyclictest25072-21/usr/sbin/munin0
7789991814,2cyclictest1425-21aten2_r9power_p0
7789991813,3cyclictest9950-21find0
7789991813,3cyclictest8823-21date0
7789991813,3cyclictest32351-21cat0
7789991813,3cyclictest23687-21diskstats0
7789991813,3cyclictest23367-21expr0
7789991813,3cyclictest23367-21expr0
7789991813,3cyclictest22078-21grep0
7789991813,3cyclictest1923-21cat0
7789991813,3cyclictest16974-21expr0
7789991813,3cyclictest14142-21expr0
7789991813,3cyclictest12643-21expr0
7789991812,4cyclictest31551-21hostname0
7789991812,4cyclictest28669-21wc0
7789991812,4cyclictest16013-21cat0
7789991811,5cyclictest512-21dbus-daemon0
7789991810,6cyclictest325-21systemd-journal0
780099179,5cyclictest11139-21grep3
780099173,9cyclictest7815-21apt-get3
780099173,8cyclictest12648-21apt-get3
780099173,8cyclictest12648-21apt-get3
780099173,10cyclictest6933-21apt-get3
7800991715,1cyclictest0-21swapper/33
7800991715,1cyclictest0-21swapper/33
7800991715,1cyclictest0-21swapper/33
7800991715,1cyclictest0-21swapper/33
7800991712,3cyclictest26147-21tr3
7800991712,3cyclictest25936-21sed3
7800991712,3cyclictest19982-21awk3
7800991712,3cyclictest14256-21expr3
7800991710,4cyclictest1762-21sed3
780099170,15cyclictest9249-21munin-run3
780099170,15cyclictest0-21swapper/33
780099170,15cyclictest0-21swapper/33
7798991714,2cyclictest0-21swapper/22
7798991712,3cyclictest9548-21if_enp3s02
7798991712,3cyclictest8937-21sed2
7798991712,3cyclictest668-21runrttasks2
7798991712,3cyclictest23061-21basename2
7798991712,3cyclictest23061-21basename2
7798991711,4cyclictest14601-21date2
7798991711,4cyclictest14601-21date2
779899170,2cyclictest0-21swapper/22
7794991715,1cyclictest0-21swapper/11
7794991715,1cyclictest0-21swapper/11
7794991713,2cyclictest7895-21sh1
7794991713,2cyclictest15079-21/usr/sbin/munin1
7794991712,3cyclictest9927-21munin-node1
7794991712,3cyclictest668-21runrttasks1
7794991712,3cyclictest668-21runrttasks1
7794991712,3cyclictest4694-21ls1
7794991712,3cyclictest29061-21grep1
7794991712,3cyclictest27239-21hostname1
7794991712,3cyclictest25411-21ls1
7794991712,3cyclictest1786-21entropy1
7794991711,4cyclictest21-21ksoftirqd/11
7794991711,4cyclictest16649-21ntp_states1
7794991710,4cyclictest23453-21grep1
778999179,5cyclictest22042-21cpu0
7789991714,2cyclictest0-21swapper/00
7789991712,3cyclictest6236-21grep0
7789991712,3cyclictest5816-21idleruntime0
7789991712,3cyclictest512-21dbus-daemon0
7789991712,3cyclictest30402-21ls0
7789991712,3cyclictest21758-21if_enp2s00
7789991712,3cyclictest21758-21if_enp2s00
7789991712,3cyclictest20981-21expr0
7789991712,3cyclictest16184-21grep0
7789991712,3cyclictest14743-21uname0
7789991712,3cyclictest12727-21aten2_r9power_p0
7789991712,3cyclictest12727-21aten2_r9power_p0
7789991711,3cyclictest8025-21cpuspeed0
778999170,15cyclictest0-21swapper/00
780099168,5cyclictest0-21swapper/33
7800991610,4cyclictest20865-21aten2_r9power_e3
7794991614,1cyclictest0-21swapper/11
7794991611,3cyclictest29141-21/usr/sbin/munin1
778999169,4cyclictest29444-21tr0
7789991611,3cyclictest28562-21grep0
7789991610,4cyclictest8892-21sed0
778999160,13cyclictest27997-21ntp_states0
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional