You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-27 - 05:07

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3s.osadl.org (updated Mon Jul 27, 2026 00:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
31732032142102,36sleep30-21swapper/33
317316728869,12sleep20-21swapper/22
317329028253,10sleep10-21swapper/11
317326927258,7sleep00-21swapper/00
3173622992523,1cyclictest0-21swapper/33
3173622992522,1cyclictest0-21swapper/33
36254402241,17sleep20-21swapper/22
3173622992422,1cyclictest0-21swapper/33
3173622992421,1cyclictest0-21swapper/33
3173622992320,1cyclictest0-21swapper/33
3173622992319,2cyclictest0-21swapper/33
3173622992313,6cyclictest3201742-21sleep3
317362299227,12cyclictest0-21swapper/33
3173622992219,1cyclictest0-21swapper/33
3173622992211,8cyclictest0-21swapper/33
317362299218,2cyclictest0-21swapper/33
3173622992119,1cyclictest0-21swapper/33
3173622992119,1cyclictest0-21swapper/33
3173622992119,1cyclictest0-21swapper/33
3173622992119,1cyclictest0-21swapper/33
3173622992118,1cyclictest0-21swapper/33
317362299211,16cyclictest0-21swapper/33
317362299211,15cyclictest3322881-21taskset3
33861272201,4sleep13301239-21diskmemload1
317362299205,12cyclictest0-21swapper/33
317362299205,12cyclictest0-21swapper/33
3173622992018,1cyclictest0-21swapper/33
3173622992018,1cyclictest0-21swapper/33
3173622992017,2cyclictest0-21swapper/33
3173622992016,2cyclictest0-21swapper/33
3173622992016,2cyclictest0-21swapper/33
3173622992016,2cyclictest0-21swapper/33
3173622992015,2cyclictest0-21swapper/33
3173622992014,3cyclictest0-21swapper/33
3173622992012,5cyclictest0-21swapper/33
317362299201,16cyclictest531rcuc/33
3173622992011,6cyclictest34233982sleep33
317362299200,16cyclictest34940672sleep33
35440402191,6sleep00-21swapper/00
3173622991917,1cyclictest0-21swapper/33
3173622991917,1cyclictest0-21swapper/33
3173622991916,2cyclictest0-21swapper/33
3173622991916,2cyclictest0-21swapper/33
3173622991916,1cyclictest0-21swapper/33
3173622991916,1cyclictest0-21swapper/33
317362299191,15cyclictest0-21swapper/33
317362299191,15cyclictest0-21swapper/33
317362299191,14cyclictest0-21swapper/33
317362299191,13cyclictest0-21swapper/33
35864312181,13sleep23586441-21ssh2
33158322181,7sleep03301239-21diskmemload0
317362299188,7cyclictest0-21swapper/33
3173622991816,1cyclictest0-21swapper/33
3173622991816,1cyclictest0-21swapper/33
3173622991815,1cyclictest0-21swapper/33
3173622991815,1cyclictest0-21swapper/33
3173622991814,2cyclictest0-21swapper/33
317362299181,14cyclictest0-21swapper/33
3173622991811,3cyclictest0-21swapper/33
36249272170,2sleep20-21swapper/22
35431582171,3sleep20-21swapper/22
34240262171,2sleep00-21swapper/00
34212702171,5sleep23301239-21diskmemload2
33885582171,3sleep20-21swapper/22
33582772171,3sleep00-21swapper/00
33427992170,2sleep00-21swapper/00
33184742171,3sleep10-21swapper/11
31956662171,6sleep03195813-21grep0
317362299172,12cyclictest0-21swapper/33
3173622991714,2cyclictest0-21swapper/33
3173622991714,1cyclictest0-21swapper/33
3173622991713,2cyclictest0-21swapper/33
3173622991713,2cyclictest0-21swapper/33
317362299171,13cyclictest0-21swapper/33
317362299171,13cyclictest0-21swapper/33
317362299171,12cyclictest0-21swapper/33
317362299170,13cyclictest0-21swapper/33
317362299170,13cyclictest0-21swapper/33
36584212161,4sleep23658435-21ssh2
36163242161,2sleep00-21swapper/00
34456982161,7sleep03445712-21ssh0
34296792161,10sleep13301239-21diskmemload1
33625672161,5sleep13362805-21cat1
32543732162,3sleep13254370-21apt-get1
31790402161,6sleep23179069-21munin-node2
3173622991614,1cyclictest0-21swapper/33
3173622991614,1cyclictest0-21swapper/33
3173622991614,1cyclictest0-21swapper/33
317362299161,12cyclictest0-21swapper/33
35160642151,2sleep20-21swapper/22
34885192151,10sleep13488663-21sensors_temp1
34463372150,2sleep20-21swapper/22
34029682151,2sleep20-21swapper/22
33025902151,8sleep23304125-21cut2
32435762151,9sleep03243632-21munin-run0
317362299152,9cyclictest3212494-21taskset3
36564572140,1sleep00-21swapper/00
35977772142,5sleep23598764-21users2
35771652141,2sleep00-21swapper/00
35771652141,2sleep00-21swapper/00
35167512141,8sleep03301239-21diskmemload0
34126452141,7sleep03301239-21diskmemload0
33788082144,2sleep1171rcu_preempt1
33590272141,6sleep23301239-21diskmemload2
32217832147,3sleep20-21swapper/22
3173622991411,1cyclictest0-21swapper/33
317362299140,12cyclictest0-21swapper/33
36512222131,7sleep13651239-21latency_hist1
36434982130,2sleep00-21swapper/00
36240982131,7sleep13301239-21diskmemload1
35957972131,2sleep10-21swapper/11
35674372132,7sleep00-21swapper/00
35096412131,8sleep13509654-21rm1
35014772131,7sleep03301239-21diskmemload0
34027362131,8sleep03301239-21diskmemload0
33809832131,8sleep23382519-21grep2
33775862131,3sleep23377601-21ssh2
33069852130,2sleep10-21swapper/11
32846242133,6sleep2555-21dbus-daemon2
32124732132,5sleep23212491-21apt-get2
36677172121,7sleep23667732-21ssh2
36392812121,4sleep10-21swapper/11
36259452121,7sleep03626125-21if_err_enp1s00
36167862121,7sleep03616798-21sh0
36146462121,7sleep13614661-21ssh1
36045132121,7sleep13604528-21ssh1
35510022121,3sleep00-21swapper/00
35405222121,7sleep13540582-21ssh1
35011942120,6sleep272-21kcompactd02
34923472121,6sleep03301239-21diskmemload0
34644582121,3sleep23301239-21diskmemload2
34571262121,2sleep1171rcu_preempt1
34563242121,7sleep23456334-21ssh2
34553882121,7sleep03301239-21diskmemload0
34452082121,7sleep13445437-21/usr/sbin/munin1
34327582121,6sleep03432782-21kthreadcore0
34298442121,7sleep23429858-21ssh2
34147212121,6sleep13414736-21ssh1
33691062120,3sleep03369121-21ssh0
33533672120,4sleep00-21swapper/00
33351942121,7sleep23335207-21rm2
33020772126,2sleep00-21swapper/00
32491142121,6sleep23248891-21sendmail-msp2
32444022121,3sleep03245101-21cut0
31915452121,5sleep23191564-21apt-get2
31814412121,7sleep13181610-21/usr/sbin/munin1
36604572111,5sleep00-21swapper/00
36516472111,3sleep10-21swapper/11
36414242110,1sleep00-21swapper/00
36109442111,4sleep23612131-21memory2
36056262111,5sleep03301239-21diskmemload0
35796682112,4sleep13579684-21apt-get1
35600072111,6sleep20-21swapper/22
35512782111,2sleep10-21swapper/11
35190022111,3sleep20-21swapper/22
34971102111,5sleep23497176-21ntp_states2
34816302110,1sleep10-21swapper/11
34130152112,4sleep13301239-21diskmemload1
33783282111,6sleep03378342-21ssh0
33430302111,3sleep20-21swapper/22
33277262111,4sleep03327755-21apt-get0
33269642111,6sleep13326978-21ssh1
33189402110,5sleep014-21ksoftirqd/00
32965472111,6sleep03296671-21fschecks_time0
32910472111,5sleep13291300-21df1
32556392111,6sleep23257304-21kthreadcore2
32494112111,5sleep13249620-21hddtemp_smartct1
32426262111,5sleep23242684-21ntp_states2
32168002111,6sleep13217021-21unixbench_singl1
32072612111,5sleep03207514-21df0
31912742111,5sleep23191301-21munin-run2
35795972101,3sleep03301239-21diskmemload0
35776342101,2sleep10-21swapper/11
35776342101,2sleep10-21swapper/11
35765162101,5sleep23301239-21diskmemload2
35765162101,5sleep23301239-21diskmemload2
35623882101,5sleep23562539-21cstates2
35623592102,3sleep10-21swapper/11
35588202100,6sleep13558834-21ssh1
35523592100,1sleep20-21swapper/22
35326112101,4sleep23301239-21diskmemload2
35282152101,4sleep13528293-21irqstats1
35171282101,4sleep13301239-21diskmemload1
34816692101,2sleep00-21swapper/00
34804062101,2sleep20-21swapper/22
34745592101,3sleep20-21swapper/22
34722292101,3sleep03301239-21diskmemload0
34674502101,4sleep13467529-21irqstats1
34640882101,3sleep10-21swapper/11
34581742101,4sleep03301239-21diskmemload0
34372492100,1sleep10-21swapper/11
34371632100,2sleep20-21swapper/22
34126202101,2sleep20-21swapper/22
34024422101,4sleep13301239-21diskmemload1
33941482101,5sleep13394162-21ssh1
33886252101,5sleep03388636-21ssh0
33691342101,3sleep20-21swapper/22
33608622100,1sleep10-21swapper/11
33530272101,2sleep10-21swapper/11
33158042101,3sleep20-21swapper/22
32962602101,4sleep13296279-21apt-get1
32805602100,3sleep10-21swapper/11
32485902101,4sleep172-21kcompactd01
32284952101,5sleep23228713-21ls2
31743352101,5sleep13174710-21grep1
3667242290,1sleep10-21swapper/11
3643235290,5sleep23643273-21fschecks_count2
3639488291,2sleep20-21swapper/22
3631455290,1sleep10-21swapper/11
3596691291,4sleep0563-21systemd-logind0
3592631291,4sleep23592726-21wc2
3559216291,2sleep00-21swapper/00
3524621291,4sleep03524634-21ssh0
3524066291,2sleep10-21swapper/11
3500216291,4sleep03500227-21ssh0
3490367291,2sleep20-21swapper/22
3414685291,4sleep03414720-21dpkg0
3384886291,2sleep00-21swapper/00
3350375291,3sleep23350389-21ssh2
3336756291,4sleep13336978-21ssh1
3323856291,2sleep20-21swapper/22
3291559291,4sleep03291828-21cut0
3279569291,4sleep03279793-21grep0
3275021291,4sleep23275107-21sed2
3259606290,3sleep00-21swapper/00
3233966291,4sleep03235391-21cut0
3197563291,4sleep23199174-21kthreadcore2
3197048291,4sleep03197270-21cat0
3196758291,3sleep13196766-21apt-get1
3174117291,4sleep03174333-21iwlist0
3533668280,4sleep03533683-21ssh0
3497764281,1sleep10-21swapper/11
3327760281,3sleep13327764-21/usr/sbin/munin1
3291022280,2sleep20-21swapper/22
3290688280,1sleep20-21swapper/22
3280538280,2sleep00-21swapper/00
3265725281,3sleep13267335-21grep1
3265337281,3sleep00-21swapper/00
3217723280,2sleep10-21swapper/11
3207227281,3sleep23207009-21sendmail-msp2
3201983280,2sleep10-21swapper/11
3191841280,1sleep10-21swapper/11
3296827271,2sleep20-21swapper/22
3290441270,1sleep10-21swapper/11
3286543271,2sleep00-21swapper/00
3269849271,2sleep00-21swapper/00
3264311270,1sleep10-21swapper/11
3259079270,1sleep00-21swapper/00
3253761270,1sleep00-21swapper/00
3248666270,1sleep20-21swapper/22
3238951271,2sleep10-21swapper/11
3237985270,1sleep20-21swapper/22
3228987271,2sleep00-21swapper/00
3223215270,3sleep10-21swapper/11
3212465270,1sleep00-21swapper/00
3202554271,2sleep23203660-21kthreadcore2
3186311270,2sleep10-21swapper/11
3185711270,1sleep00-21swapper/00
3279345261,2sleep20-21swapper/22
3275343260,1sleep13275340-21apt-get1
3270057260,2sleep10-21swapper/11
3269402260,1sleep20-21swapper/22
3262469260,2sleep20-21swapper/22
3235392261,2sleep10-21swapper/11
3228186260,2sleep10-21swapper/11
3226809261,2sleep20-21swapper/22
3223662260,2sleep00-21swapper/00
3207226260,2sleep10-21swapper/11
3206645260,4sleep00-21swapper/00
3186310261,2sleep00-21swapper/00
3177890260,2sleep20-21swapper/22
3221585250,1sleep00-21swapper/00
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional