You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-01 - 06:44

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3s.osadl.org (updated Fri May 01, 2026 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
207659121360,8sleep014-21ksoftirqd/00
1910565212064,52sleep30-21swapper/33
1910795211092,12sleep10-21swapper/11
191054428060,14sleep00-21swapper/00
191072627658,12sleep20-21swapper/22
21260442330,3sleep244-21ksoftirqd/22
1910939992116,2cyclictest0-21swapper/33
1910939992115,4cyclictest0-21swapper/33
191093999211,17cyclictest0-21swapper/33
1910939992016,2cyclictest0-21swapper/33
191093999201,16cyclictest0-21swapper/33
191093999201,15cyclictest66950irq/132-enp1s0-TxRx-33
1910939992011,5cyclictest0-21swapper/33
191093999201,14cyclictest2183901-21taskset3
22084812191,5sleep02208683-21users0
191093999198,7cyclictest1982706-21sleep3
1910939991916,1cyclictest0-21swapper/33
1910939991912,5cyclictest0-21swapper/33
191093999191,15cyclictest0-21swapper/33
191093999191,15cyclictest0-21swapper/33
191093999191,14cyclictest0-21swapper/33
191093999191,14cyclictest0-21swapper/33
191093999191,14cyclictest0-21swapper/33
191093999191,14cyclictest0-21swapper/33
1910939991910,6cyclictest0-21swapper/33
191093999190,16cyclictest0-21swapper/33
23688172183,4sleep12368842-21apt-get1
23654142181,2sleep00-21swapper/00
22862522181,14sleep22286266-21ssh2
22509912181,5sleep22251229-21users2
19621562181,5sleep01963674-21ps0
191093999189,6cyclictest1988035-21turbostat3
191093999183,13cyclictest0-21swapper/33
191093999182,11cyclictest1988327-21taskset3
191093999181,4cyclictest0-21swapper/33
191093999181,14cyclictest0-21swapper/33
191093999181,14cyclictest0-21swapper/33
191093999181,14cyclictest0-21swapper/33
191093999181,14cyclictest0-21swapper/33
191093999181,14cyclictest0-21swapper/33
191093999181,14cyclictest0-21swapper/33
191093999181,14cyclictest0-21swapper/33
191093999181,14cyclictest0-21swapper/33
191093999181,14cyclictest0-21swapper/33
191093999181,14cyclictest0-21swapper/33
191093999181,13cyclictest0-21swapper/33
191093999180,2cyclictest0-21swapper/33
191093999180,15cyclictest66950irq/132-enp1s0-TxRx-33
191093999180,14cyclictest0-21swapper/33
191093999180,14cyclictest0-21swapper/33
191093999180,14cyclictest0-21swapper/33
237583021711,3sleep22375841-21ssh2
23578582171,3sleep00-21swapper/00
23359592171,3sleep20-21swapper/22
23295452179,4sleep22041846-21diskmemload2
22419682171,5sleep22242019-21ntpq2
218519921711,3sleep02185213-21ssh0
21435252171,3sleep10-21swapper/11
1910939991713,2cyclictest2014509-21taskset3
1910939991713,2cyclictest0-21swapper/33
1910939991711,4cyclictest0-21swapper/33
191093999171,13cyclictest0-21swapper/33
191093999171,12cyclictest1950999-21taskset3
191093999170,14cyclictest0-21swapper/33
191093999170,14cyclictest0-21swapper/33
191093999170,14cyclictest0-21swapper/33
191093999170,13cyclictest0-21swapper/33
191093999170,13cyclictest0-21swapper/33
23301212162,3sleep02041846-21diskmemload0
22962922160,8sleep02041846-21diskmemload0
1910939991612,3cyclictest0-21swapper/33
191093999161,13cyclictest0-21swapper/33
191093999161,12cyclictest0-21swapper/33
191093999160,3cyclictest1916152-21taskset3
191093999160,13cyclictest0-21swapper/33
191093999160,13cyclictest0-21swapper/33
23568542150,2sleep20-21swapper/22
22333642151,7sleep12233463-21ntpq1
22110522151,10sleep12211064-21ssh1
21771422151,4sleep02041846-21diskmemload0
21086212151,3sleep10-21swapper/11
21002702150,1sleep20-21swapper/22
20924472151,9sleep22041846-21diskmemload2
20421172156,6sleep12042130-21sh1
19159082151,10sleep21915914-21fwupd2
191093999159,3cyclictest1939908-21taskset3
1910939991512,2cyclictest0-21swapper/33
1910939991512,1cyclictest0-21swapper/33
191093999150,13cyclictest0-21swapper/33
191093999150,13cyclictest0-21swapper/33
191093999150,12cyclictest0-21swapper/33
23434302143,5sleep22343459-21apt-get2
23369372141,9sleep02336948-21ssh0
23219662141,3sleep22041846-21diskmemload2
22674612141,7sleep12267522-21ntpq1
22285082141,9sleep02228509-21sendmail-mta0
22185062140,9sleep22041846-21diskmemload2
20301322141,4sleep12030137-21sensors_temp1
1910939991411,1cyclictest0-21swapper/33
23429542130,2sleep20-21swapper/22
23368512134,4sleep1331ktimers/11
22947612131,7sleep12041846-21diskmemload1
22687372131,2sleep20-21swapper/22
22048232131,4sleep22206429-21cut2
21577522131,4sleep12157764-21ssh1
21174792131,8sleep22117489-21ssh2
20749762131,8sleep22074989-21ssh2
19942002131,4sleep11994911-21cut1
19353942131,8sleep01936018-21tr0
19168182131,8sleep21917899-21cut2
1910939991311,1cyclictest0-21swapper/33
191093999130,2cyclictest0-21swapper/33
191093999130,10cyclictest0-21swapper/33
191093999130,10cyclictest0-21swapper/33
23898092121,7sleep22389887-21sed2
23104322125,2sleep2431ktimers/22
23022572124,4sleep12302254-21sh1
22701822120,1sleep00-21swapper/00
22626412121,5sleep20-21swapper/22
22520392121,7sleep02252050-21ssh0
22285102121,7sleep12228522-21ssh1
22260132120,2sleep20-21swapper/22
22182642121,7sleep12218274-21ssh1
22004072121,7sleep22200422-21ssh2
21992232121,7sleep02199334-21needreboot0
21864542121,5sleep22186489-21ssh2
21667702121,7sleep12041846-21diskmemload1
21171042121,7sleep02115122-21kworker/u16:3+rpciod0
21079262121,6sleep02041846-21diskmemload0
20506562122,5sleep12050677-21apt-get1
20485362121,7sleep22048548-21ssh2
19926602121,7sleep21992675-21plymouth2
19152002121,5sleep11915248-21ntpq1
23870342113,4sleep12388765-21cut1
23406862110,2sleep00-21swapper/00
23285342111,2sleep10-21swapper/11
22888162111,5sleep22288838-21apt-get2
22861752111,7sleep00-21swapper/00
22615552111,3sleep00-21swapper/00
22598512111,7sleep12259862-21ssh1
22441822111,5sleep00-21swapper/00
22356902111,2sleep02235701-21ssh0
21660512111,6sleep22166063-21ssh2
21602352110,1sleep20-21swapper/22
21483162111,5sleep12148373-21ntpq1
21441492111,5sleep02144164-21ssh0
21302252111,5sleep02131179-21memory0
20934332111,6sleep02093451-21ssh0
20818832111,3sleep20-21swapper/22
20754982110,2sleep10-21swapper/11
20586202111,7sleep02041846-21diskmemload0
20557072111,3sleep20-21swapper/22
20330672111,6sleep12034892-21cut1
20139172111,6sleep02014044-21wc0
20096972111,4sleep12009716-21apt-get1
20040752111,5sleep12004123-21munin-run1
23896832101,5sleep02389742-21ntpq0
23841612101,4sleep12041846-21diskmemload1
23813312101,2sleep20-21swapper/22
23667142101,3sleep20-21swapper/22
23652302101,3sleep10-21swapper/11
23211362101,6sleep02321150-21ssh0
23113792102,5sleep02041846-21diskmemload0
23030042101,4sleep02041846-21diskmemload0
22973182101,4sleep22297342-21apt-get2
22805992101,5sleep12280810-21grep1
22787342100,1sleep20-21swapper/22
22760032100,1sleep10-21swapper/11
22734242101,5sleep02275108-21grep0
22508902100,7sleep12250989-21ls1
22370512101,4sleep22237063-21ssh2
21952372102,3sleep12195261-21apt-get1
21938682100,4sleep00-21swapper/00
21844212101,4sleep22041846-21diskmemload2
21776122101,2sleep10-21swapper/11
21765282101,4sleep10-21swapper/11
21759962101,4sleep22041846-21diskmemload2
21681462101,3sleep00-21swapper/00
21508812101,4sleep20-21swapper/22
21346492101,5sleep22134660-21sh2
21192642101,5sleep02120201-21kthreadcore0
21094902101,3sleep20-21swapper/22
20911322101,2sleep00-21swapper/00
20733612100,1sleep00-21swapper/00
20647162101,2sleep20-21swapper/22
20196142101,5sleep02019778-21/usr/sbin/munin0
20104402101,5sleep22012093-21cut2
19816782101,4sleep11981728-21ntpq1
19664782100,2sleep10-21swapper/11
19562582101,4sleep01956276-21apt-get0
19475922101,5sleep01949412-21cut0
2384110291,3sleep00-21swapper/00
2374624291,3sleep02041846-21diskmemload0
2348619291,3sleep02041846-21diskmemload0
2348467291,3sleep12348477-21ssh1
2341984291,2sleep10-21swapper/11
2318512291,4sleep12318568-21ntpq1
2308582291,2sleep10-21swapper/11
2244001291,3sleep12041846-21diskmemload1
2217910291,3sleep02041846-21diskmemload0
2190753291,5sleep12190876-21ps1
2158329291,2sleep00-21swapper/00
2142940291,4sleep02142953-21ssh0
2135924291,2sleep20-21swapper/22
2127157291,4sleep12127186-21apt-get1
2123160291,4sleep12041846-21diskmemload1
2115774291,2sleep10-21swapper/11
2098268291,3sleep10-21swapper/11
2084553291,2sleep10-21swapper/11
2080258291,5sleep12080319-21gcc1
2064293290,5sleep02064304-21ssh0
2058983291,4sleep12059009-21ssh1
2048982290,1sleep00-21swapper/00
2039765291,4sleep12040682-21missed_timers1
2015705291,4sleep22017369-21cut2
2004741291,4sleep12004896-21/usr/sbin/munin1
1999562291,5sleep22000770-21cut2
1979094291,4sleep21980862-21cut2
1972774291,4sleep01973088-21grep0
1968475291,4sleep01970261-21kthreadcore0
1961864291,4sleep21962077-21hddtemp_smartct2
1956538291,4sleep21956645-21sort2
1940734291,4sleep21941359-21cat2
1929999291,3sleep11930070-21irqstats1
1928238291,4sleep21928287-21ntpq2
1916215291,4sleep11916817-21ntp_192.168.1141
2356981281,3sleep12356995-21ssh1
2036719280,4sleep00-21swapper/00
2025746280,2sleep00-21swapper/00
2025166280,1sleep00-21swapper/00
2024465280,3sleep12024505-21ntpq1
2015570281,4sleep12015704-21grep1
2004897280,4sleep22004934-21kthreadcore2
1973089281,3sleep11974779-21grep1
1957651281,3sleep11959407-21cut1
1955977280,1sleep00-21swapper/00
1945574280,3sleep21945593-21apt-get2
1939825280,1sleep20-21swapper/22
1934222280,2sleep00-21swapper/00
1911105281,2sleep01911124-21apt-get0
2025380270,1sleep20-21swapper/22
2008719271,3sleep02008926-21unixbench-2d0
1993187270,1sleep00-21swapper/00
1977597270,1sleep00-21swapper/00
1945112270,2sleep1171rcu_preempt1
1936019270,4sleep11937730-21kthreadcore1
2034893260,2sleep22035162-21strings2
2031684261,2sleep00-21swapper/00
2030707260,1sleep20-21swapper/22
1997568260,2sleep00-21swapper/00
1994912261,2sleep20-21swapper/22
1992227261,2sleep10-21swapper/11
1987671260,4sleep10-21swapper/11
1982958260,2sleep00-21swapper/00
1977073260,4sleep20-21swapper/22
1971763260,1sleep10-21swapper/11
1955703260,1sleep20-21swapper/22
1944381261,2sleep00-21swapper/00
1930071261,2sleep20-21swapper/22
1928143260,2sleep10-21swapper/11
1919741261,2sleep00-21swapper/00
2037136250,1sleep20-21swapper/22
1999280250,2sleep00-21swapper/00
1986850250,1sleep20-21swapper/22
1971721250,1sleep20-21swapper/22
1950067250,1sleep10-21swapper/11
1927481250,1sleep00-21swapper/00
1950915240,1sleep10-21swapper/11
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional