You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-28 - 19:23
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack9slot2.osadl.org (updated Tue Apr 28, 2026 12:43:25)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
273522252185,25sleep00-21swapper/00
274632242173,24sleep30-21swapper/33
273732240167,24sleep10-21swapper/11
271592228191,24sleep20-21swapper/22
2768399216210,4cyclictest32350-21idleruntime-cro3
2767299184178,4cyclictest32323-21sendmail0
1467621660,2sleep20-21swapper/22
1467621660,2sleep20-21swapper/22
2767599111106,4cyclictest32341-21perf1
2767999106102,3cyclictest32352-21sendmail2
4802620,1sleep30-21swapper/33
113432620,3sleep111348-21cpuspeed_turbos1
169222590,2sleep10-21swapper/11
287742540,2sleep10-21swapper/11
158312530,2sleep30-21swapper/33
115672470,1sleep30-21swapper/33
16872460,2sleep00-21swapper/00
232562380,1sleep20-21swapper/22
48212370,1sleep20-21swapper/22
104012360,1sleep00-21swapper/00
27675993413,4cyclictest19983-21sendmail1
27675993314,3cyclictest10993-21sed1
27679993214,3cyclictest3330-21diskstats2
2767599322,14cyclictest6932-21basename1
2767999311,14cyclictest8152-21grep2
2767599318,7cyclictest2134-21unixbench-2d1
27675993114,3cyclictest28649-21needreboot1
27675993113,3cyclictest26673-21sh1
27679993014,3cyclictest29537-21ntp_kernel_pll_2
27679993014,3cyclictest29537-21ntp_kernel_pll_2
2767999292,16cyclictest21113-21latency_hist2
27679992915,2cyclictest3277-21processes2
2767599282,8cyclictest4905-21apt-get1
2767999271,7cyclictest24192-21apt-get2
97592250,2sleep00-21swapper/00
2767999252,7cyclictest14058-21apt-get2
27679992516,5cyclictest31209-21apt-get2
2767999243,6cyclictest8558-21apt-get2
27679992416,1cyclictest0-21swapper/22
195922240,3sleep00-21swapper/00
27683992318,3cyclictest30573-21apt-key3
27683992318,3cyclictest30573-21apt-key3
27683992317,4cyclictest14675-21sed3
27683992317,4cyclictest14675-21sed3
2767999231,19cyclictest14485-21perf2
27679992311,10cyclictest12722-21sort2
27679992311,10cyclictest12722-21sort2
27675992317,3cyclictest7023-21grep1
27675992314,5cyclictest22136-21apt-get1
269792230,2sleep00-21swapper/00
112152230,2sleep00-21swapper/00
2767999223,17cyclictest27577-21latency_hist2
2767999223,17cyclictest13636-21latency_hist2
27679992216,4cyclictest10977-21which2
2767999221,18cyclictest11580-21munin-run2
2767299222,17cyclictest12870-21munin-run0
2767299220,9cyclictest179132latency_hist0
59072210,7sleep228-21ksoftirqd/22
29182210,3sleep02767299cyclictest0
27683992115,4cyclictest3669-21users3
27683992115,4cyclictest13872-21timerandwakeup3
27683992114,4cyclictest28483-21seq3
27683992114,4cyclictest24408-21cpuspeed_turbos3
2767999212,17cyclictest26751-21latency_hist2
2767999211,5cyclictest8305-21awk2
27679992114,4cyclictest14052-21awk2
2767999211,3cyclictest3471-21munin-run2
2767999211,18cyclictest7628-21procmail2
2767999211,17cyclictest1876-21cut2
2767999210,19cyclictest6679-21munin-run2
27675992115,4cyclictest17125-21cat1
27675992115,3cyclictest26432-21mailstats1
27675992115,3cyclictest25063-21sh1
27672992115,4cyclictest25535-21idleruntime-cro0
27672992115,4cyclictest24018-21cat0
27672992115,4cyclictest21888-21idleruntime-cro0
27672992114,4cyclictest19639-21grep0
85802200,5sleep10-21swapper/11
27683992015,3cyclictest32195-21interrupts3
27683992014,4cyclictest7582-21cron3
27683992014,4cyclictest3881-21awk3
27683992014,4cyclictest29915-21sh3
27683992014,4cyclictest29915-21sh3
27683992014,4cyclictest13715-21if_enp3s03
27683992014,4cyclictest12612-21sed3
27683992012,5cyclictest1668-21fw_packets3
2767999209,9cyclictest166772chrt2
2767999204,14cyclictest537-21gdbus2
27679992015,3cyclictest9048-21basename2
27679992015,3cyclictest5102-21apt-get2
27679992015,3cyclictest32545-21which2
27679992015,3cyclictest29643-21sh2
27679992015,3cyclictest1509-21cut2
27679992014,4cyclictest31803-21memory2
27679992014,4cyclictest30471-21expr2
27679992014,4cyclictest29845-21seq2
2767999201,3cyclictest29695-21munin-run2
2767999201,17cyclictest28659-1kworker/2:0H2
2767999201,17cyclictest19051-21latency_hist2
2767999201,17cyclictest12401-21latency_hist2
2767999201,17cyclictest0-21swapper/22
2767599202,5cyclictest21-21ksoftirqd/11
27675992015,3cyclictest9921-21apt-get1
27675992015,3cyclictest4858-21seq1
27675992015,3cyclictest27768-21ls1
27675992015,3cyclictest24833-21irqstats1
27675992014,4cyclictest32338-21grep1
27675992014,4cyclictest3150-21cat1
27675992014,4cyclictest20882-21cat1
27672992016,2cyclictest21354-21/usr/sbin/munin0
27672992015,3cyclictest17224-21/usr/sbin/munin0
27672992014,4cyclictest2717-21sed0
27672992013,5cyclictest512-21dbus-daemon0
27672992013,4cyclictest5114-21sh0
27672992012,5cyclictest2790-21switchtime0
2767299201,17cyclictest15407-21sed0
213262200,2sleep00-21swapper/00
172312200,2sleep20-21swapper/22
27683991916,2cyclictest0-21swapper/33
27683991916,2cyclictest0-21swapper/33
27683991916,2cyclictest0-21swapper/33
27683991916,2cyclictest0-21swapper/33
27683991916,2cyclictest0-21swapper/33
27683991915,3cyclictest35-21ksoftirqd/33
27683991915,2cyclictest25975-21if_err_enp3s03
27683991915,2cyclictest24963-21sshd3
27683991914,3cyclictest8277-21sh3
27683991914,3cyclictest31786-21cat3
27683991914,3cyclictest29636-21sh3
27683991914,3cyclictest29630-21gltestperf3
27683991914,3cyclictest2130-21unixbench-2d3
27683991914,3cyclictest20044-21ntp_kernel_pll_3
27683991914,3cyclictest16235-21sed3
27683991913,4cyclictest887-21wc3
2767999193,13cyclictest23518-21grep2
27679991916,2cyclictest0-21swapper/22
27679991915,2cyclictest22576-21/usr/sbin/munin2
27679991914,3cyclictest705-21cat2
27679991914,3cyclictest18243-21telnet2
27679991913,4cyclictest22802-21seq2
27679991913,4cyclictest16041-21ntp_kernel_pll_2
27679991913,4cyclictest11874-21sed2
27679991913,3cyclictest20793-21cat2
2767999190,3cyclictest25175-1kworker/2:1H2
2767999190,17cyclictest20957-21latency_hist2
27675991916,2cyclictest0-21swapper/11
27675991916,2cyclictest0-21swapper/11
27675991915,2cyclictest19321-21/usr/sbin/munin1
27675991914,3cyclictest31667-21sh1
27675991914,3cyclictest26582-21apt-get1
27675991914,3cyclictest26541-21seq1
27675991914,3cyclictest13109-21acpi1
27675991913,4cyclictest31518-21date1
27675991913,4cyclictest31518-21date1
27675991913,4cyclictest29472-21cat1
27675991913,4cyclictest29472-21cat1
27675991913,4cyclictest21416-21hostname1
27675991913,4cyclictest21128-21cat1
27675991913,4cyclictest15581-21seq1
27675991913,4cyclictest15581-21seq1
27675991913,4cyclictest13887-21sed1
27675991912,4cyclictest20026-21date1
2767599190,17cyclictest6125-21idleruntime-cro1
2767299199,6cyclictest512-21dbus-daemon0
27672991916,2cyclictest0-21swapper/00
27672991916,2cyclictest0-21swapper/00
27672991914,3cyclictest3432-21sed0
27672991914,3cyclictest32584-21/usr/sbin/munin0
27672991914,3cyclictest25984-21sh0
27672991914,3cyclictest17509-21sh0
27672991914,3cyclictest13148-21expr0
27672991914,3cyclictest11986-21/usr/sbin/munin0
27672991914,3cyclictest11986-21/usr/sbin/munin0
27672991913,4cyclictest27916-21ntp_states0
27672991913,4cyclictest26708-21grep0
27672991913,4cyclictest1135-21grep0
27672991912,5cyclictest9182-21grep0
2767299191,16cyclictest18922-21munin-run0
302982181,2sleep30-21swapper/33
27683991816,1cyclictest0-21swapper/33
27683991816,1cyclictest0-21swapper/33
27683991815,2cyclictest0-21swapper/33
27683991815,2cyclictest0-21swapper/33
27683991814,2cyclictest8077-21cpuspeed3
27683991814,2cyclictest4522-21missed_timers3
27683991813,3cyclictest9498-21latency_hist3
27683991813,3cyclictest5745-21ntpq3
27683991813,3cyclictest5047-21grep3
27683991813,3cyclictest20460-21expr3
27683991812,4cyclictest4185-21uname3
27683991812,4cyclictest2410-21seq3
27683991812,3cyclictest5728-21sh3
27683991811,5cyclictest300-21seq3
27683991811,4cyclictest28006-21cpu3
27683991811,4cyclictest17236-21aten2.4-expect3
2767999182,14cyclictest7193-21grep2
27679991816,1cyclictest0-21swapper/22
27679991816,1cyclictest0-21swapper/22
27679991814,2cyclictest27985-21awk2
27679991814,2cyclictest20010-21cat2
27679991813,3cyclictest27516-21expr2
2767999180,17cyclictest0-21swapper/22
2767999180,17cyclictest0-21swapper/22
2767999180,16cyclictest29230-21munin-run2
27675991816,1cyclictest0-21swapper/11
27675991816,1cyclictest0-21swapper/11
27675991816,1cyclictest0-21swapper/11
27675991815,2cyclictest0-21swapper/11
27675991814,2cyclictest15012-21/usr/sbin/munin1
27675991814,2cyclictest12003-21apt1
27675991814,2cyclictest12003-21apt1
27675991814,2cyclictest11822-21/usr/sbin/munin1
27675991813,3cyclictest668-21runrttasks1
27675991813,3cyclictest393-21expr1
27675991813,3cyclictest3744-21/usr/sbin/munin1
27675991813,3cyclictest32011-21/usr/sbin/munin1
27675991813,3cyclictest2620-21cut1
27675991813,3cyclictest17224-21/usr/sbin/munin1
27675991812,4cyclictest32305-21cat1
27675991812,4cyclictest17085-21ls1
27675991811,5cyclictest9103-21cut1
2767599180,17cyclictest0-21swapper/11
2767299187,9cyclictest29114-21/usr/sbin/munin0
27672991816,1cyclictest0-21swapper/00
27672991816,1cyclictest0-21swapper/00
27672991815,2cyclictest0-21swapper/00
27672991814,2cyclictest22576-21/usr/sbin/munin0
27672991814,2cyclictest11822-21/usr/sbin/munin0
27672991813,3cyclictest668-21runrttasks0
27672991813,3cyclictest668-21runrttasks0
27672991813,3cyclictest4886-21/usr/sbin/munin0
27672991813,3cyclictest30674-21threads0
27672991813,3cyclictest30674-21threads0
27672991813,3cyclictest28634-21/usr/sbin/munin0
27672991813,3cyclictest28634-21/usr/sbin/munin0
27672991813,3cyclictest27344-21sed0
27672991813,3cyclictest24898-21sed0
27672991813,3cyclictest21310-21ls0
27672991813,3cyclictest18026-21/usr/sbin/munin0
27672991813,3cyclictest13847-21wc0
27672991812,4cyclictest7454-21latency_hist0
27672991812,4cyclictest28056-21cat0
27672991812,4cyclictest15642-21cat0
27672991811,5cyclictest512-21dbus-daemon0
27672991811,4cyclictest329-21cat0
27672991811,4cyclictest31334-21unixbench_singl0
278042170,1sleep00-21swapper/00
2768399172,12cyclictest19064-21sh3
27683991713,2cyclictest26564-21/usr/sbin/munin3
27683991713,2cyclictest23590-21apt-config3
27683991712,3cyclictest668-21runrttasks3
27683991712,3cyclictest3153-21cat3
27683991712,3cyclictest27814-21cat3
27683991712,3cyclictest25708-21memory3
27683991712,3cyclictest16708-21cat3
27683991712,3cyclictest12621-21if_err_enp3s03
27683991712,3cyclictest12621-21if_err_enp3s03
27683991711,4cyclictest7829-21uname3
27683991711,4cyclictest21853-21head3
27683991711,4cyclictest15947-21cat3
2768399170,16cyclictest0-21swapper/33
27679991715,1cyclictest0-21swapper/22
27679991715,1cyclictest0-21swapper/22
2767999170,2cyclictest111rcu_preempt2
27675991713,2cyclictest7847-21/usr/sbin/munin1
27675991713,2cyclictest6353-21/usr/sbin/munin1
27675991713,2cyclictest2871-21interrupts1
27675991713,2cyclictest27749-21cat1
27675991712,3cyclictest6450-21interrupts1
27675991712,3cyclictest3280-21cat1
27675991712,3cyclictest2219-21basename1
27675991712,3cyclictest20020-21cut1
27675991711,4cyclictest14418-21cat1
27672991715,1cyclictest0-21swapper/00
27672991712,3cyclictest4719-21latency_hist0
27672991712,3cyclictest14998-21awk0
27672991712,3cyclictest14998-21awk0
27672991711,4cyclictest32032-21grep0
2767299170,16cyclictest0-21swapper/00
27683991612,2cyclictest8976-21/usr/sbin/munin3
27683991612,2cyclictest10588-21/usr/sbin/munin3
27683991611,3cyclictest16513-21if_enp2s03
2767299162,11cyclictest31237-21aten2_r9power_c0
27672991611,3cyclictest5143-21sed0
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional