You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-09 - 04:42

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot3s.osadl.org (updated Thu Jul 09, 2026 00:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1674365212554,67sleep30-21swapper/33
199692821170,3sleep00-21swapper/00
167914029179,8sleep20-21swapper/22
167914428062,12sleep10-21swapper/11
167892827656,14sleep00-21swapper/00
20971292290,2sleep20-21swapper/22
192598122110,7sleep11925995-21ssh1
167939399212,13cyclictest1928289-21taskset3
1679393992112,6cyclictest0-21swapper/33
167939399211,16cyclictest0-21swapper/33
167939399211,16cyclictest0-21swapper/33
167939399202,15cyclictest0-21swapper/33
1679393992012,5cyclictest0-21swapper/33
167939399201,16cyclictest0-21swapper/33
167939399201,15cyclictest66950irq/132-enp1s0-TxRx-33
167939399201,15cyclictest0-21swapper/33
167939399201,15cyclictest0-21swapper/33
167939399201,15cyclictest0-21swapper/33
18996542191,4sleep11807145-21diskmemload1
18634822191,4sleep10-21swapper/11
167939399199,6cyclictest66950irq/132-enp1s0-TxRx-33
1679393991912,5cyclictest0-21swapper/33
1679393991912,5cyclictest0-21swapper/33
167939399191,15cyclictest0-21swapper/33
167939399191,15cyclictest0-21swapper/33
167939399191,15cyclictest0-21swapper/33
167939399191,14cyclictest66950irq/132-enp1s0-TxRx-33
167939399191,14cyclictest0-21swapper/33
167939399191,14cyclictest0-21swapper/33
167939399191,14cyclictest0-21swapper/33
167939399191,14cyclictest0-21swapper/33
167939399191,14cyclictest0-21swapper/33
167939399191,14cyclictest0-21swapper/33
167939399190,16cyclictest66950irq/132-enp1s0-TxRx-33
167939399190,15cyclictest0-21swapper/33
19929952182,3sleep01993005-21ssh0
1679393991815,1cyclictest0-21swapper/33
167939399181,14cyclictest0-21swapper/33
167939399181,14cyclictest0-21swapper/33
167939399181,14cyclictest0-21swapper/33
167939399181,14cyclictest0-21swapper/33
167939399181,13cyclictest0-21swapper/33
167939399180,15cyclictest0-21swapper/33
167939399180,13cyclictest0-21swapper/33
20633562171,2sleep00-21swapper/00
20280362171,4sleep02028038-21ssh0
19102932171,3sleep00-21swapper/00
1679393991713,2cyclictest0-21swapper/33
167939399171,13cyclictest0-21swapper/33
167939399171,13cyclictest0-21swapper/33
167939399171,13cyclictest0-21swapper/33
167939399171,13cyclictest0-21swapper/33
167939399171,13cyclictest0-21swapper/33
167939399171,13cyclictest0-21swapper/33
167939399171,13cyclictest0-21swapper/33
167939399171,13cyclictest0-21swapper/33
167939399171,13cyclictest0-21swapper/33
167939399171,13cyclictest0-21swapper/33
167939399171,13cyclictest0-21swapper/33
167939399170,13cyclictest0-21swapper/33
167939399170,13cyclictest0-21swapper/33
21161652161,4sleep21807145-21diskmemload2
19966012161,4sleep11996625-21apt-get1
18656892161,9sleep21807145-21diskmemload2
18484982161,10sleep11807145-21diskmemload1
17865402161,4sleep11786758-21df_inode1
17608242160,8sleep11761898-21grep1
16796502161,10sleep21679853-21diskstats2
167939399162,9cyclictest1776042-21taskset3
167939399161,3cyclictest0-21swapper/33
167939399161,3cyclictest0-21swapper/33
167939399161,3cyclictest0-21swapper/33
1679393991612,2cyclictest0-21swapper/33
167939399161,13cyclictest0-21swapper/33
167939399161,13cyclictest0-21swapper/33
167939399161,12cyclictest1723144-21chrt3
167939399160,13cyclictest0-21swapper/33
20728162151,4sleep02072827-21ssh0
20538772150,1sleep20-21swapper/22
20110092151,3sleep00-21swapper/00
19836242150,1sleep00-21swapper/00
19249952151,5sleep01925007-21ssh0
18462682151,7sleep01846335-21ntp_states0
18254312151,10sleep21826480-21cut2
17215322151,6sleep21722235-21missed_timers2
167939399153,8cyclictest0-21swapper/33
167939399150,13cyclictest0-21swapper/33
167939399150,12cyclictest0-21swapper/33
167939399150,12cyclictest0-21swapper/33
167939399150,12cyclictest0-21swapper/33
21604572141,2sleep20-21swapper/22
21233242141,9sleep12123335-21ssh1
20868892141,3sleep11807145-21diskmemload1
20868892141,3sleep11807145-21diskmemload1
20292692140,2sleep10-21swapper/11
19447112141,3sleep01807145-21diskmemload0
19354652141,9sleep21807145-21diskmemload2
18656662141,3sleep01865676-21ssh0
18210362141,9sleep21821047-21ssh2
167939399141,8cyclictest1749804-21taskset3
1679393991412,1cyclictest0-21swapper/33
1679393991410,2cyclictest0-21swapper/33
167939399140,2cyclictest0-21swapper/33
21517842130,10sleep01807145-21diskmemload0
21395832131,7sleep02139681-21awk0
21071262130,2sleep1171rcu_preempt1
20981622131,7sleep11807145-21diskmemload1
20891062131,8sleep02089120-21ssh0
20891062131,8sleep02089120-21ssh0
20818392130,8sleep22081853-21ssh2
20818392130,8sleep22081853-21ssh2
20454792131,6sleep20-21swapper/22
20439752131,8sleep12043989-21ssh1
20098422131,7sleep21807145-21diskmemload2
19950662131,3sleep11995077-21ssh1
19359542131,2sleep10-21swapper/11
19185202131,3sleep11918533-21ssh1
18549132131,8sleep21855050-21mailstats2
167939399138,3cyclictest1785895-21chrt3
167939399130,2cyclictest0-21swapper/33
167939399130,10cyclictest0-21swapper/33
21520932121,7sleep12152161-21rm1
21500452121,3sleep22150059-21ssh2
21243722121,5sleep21807145-21diskmemload2
21075682121,6sleep22107579-21ssh2
20044542121,5sleep20-21swapper/22
19374062121,5sleep11807145-21diskmemload1
18986612121,7sleep21898673-21rm2
18843862121,6sleep11807145-21diskmemload1
18843862121,6sleep11807145-21diskmemload1
18767502121,6sleep01876883-21diskstats0
18767502121,6sleep01876883-21diskstats0
18386412121,7sleep21838652-21ssh2
18229002121,5sleep10-21swapper/11
18070342122,7sleep0106713-21snmpd0
17938082121,7sleep01795227-21cut0
17327842121,7sleep21732856-21/usr/sbin/munin2
17189092121,7sleep01719840-21cat0
16846022121,7sleep11684978-21grep1
21615102111,5sleep22161586-21irqstats2
21423662111,4sleep21807145-21diskmemload2
21101152111,6sleep12110134-21/usr/sbin/munin1
21088892111,6sleep02108904-21ssh0
20800162111,6sleep22080030-21ssh2
20783112111,3sleep10-21swapper/11
20547462111,4sleep10-21swapper/11
20362202111,6sleep02036236-21ssh0
20307842111,5sleep22030947-21diskstats2
20135962111,4sleep22013599-21apt-get2
20102702111,4sleep11807145-21diskmemload1
19868102111,6sleep21986823-21ssh2
19625662111,5sleep01962838-21df_inode0
19584142111,3sleep20-21swapper/22
19505652111,3sleep00-21swapper/00
19236092111,3sleep20-21swapper/22
18937612112,4sleep01893782-21apt-get0
18903632111,4sleep21807145-21diskmemload2
18563012110,2sleep00-21swapper/00
18560042110,1sleep10-21swapper/11
18503122111,5sleep21850371-21munin-run2
18405952111,4sleep01807145-21diskmemload0
18304392111,5sleep11807145-21diskmemload1
17914362111,6sleep01791659-21latency_hist0
17760142111,5sleep01776033-21apt-get0
17724962111,6sleep01774225-21cut0
17707252111,4sleep21770744-21apt-get2
17655432111,5sleep21765778-21df_inode2
17485742110,7sleep01748632-21/usr/sbin/munin0
17389932111,3sleep00-21swapper/00
17083852111,6sleep11708896-21kthreadcore1
17030792111,5sleep11703153-21irqstats1
21680212101,4sleep11807145-21diskmemload1
21579632101,4sleep02157977-21ssh0
21525172101,5sleep12152763-21expr1
21334322101,4sleep11807145-21diskmemload1
21324052101,5sleep02132416-21ssh0
21097132100,5sleep00-21swapper/00
20904682101,5sleep02090582-21cut0
20799882101,3sleep00-21swapper/00
20648662101,5sleep22065035-21ssh2
20644032102,3sleep134-21ksoftirqd/11
20573392101,5sleep22058922-21grep2
20535242100,1sleep00-21swapper/00
20306572101,5sleep12030783-21cut1
20189362100,1sleep10-21swapper/11
19931142100,2sleep20-21swapper/22
19801762101,5sleep11981543-21cut1
19771422101,2sleep20-21swapper/22
19688122101,2sleep20-21swapper/22
19677692101,6sleep11967782-21ssh1
19538702101,4sleep11953494-21apt-get1
19503122101,4sleep21807145-21diskmemload2
19189442101,4sleep21807145-21diskmemload2
19184122101,3sleep01807145-21diskmemload0
19081052101,4sleep21807145-21diskmemload2
19063222101,3sleep10-21swapper/11
18916972101,4sleep11807145-21diskmemload1
18916852101,5sleep01891694-21sh0
18820182101,3sleep20-21swapper/22
18820182101,3sleep20-21swapper/22
18743672101,4sleep01807145-21diskmemload0
18736702101,3sleep11807145-21diskmemload1
18712232101,5sleep21871952-21ls2
18391492101,3sleep10-21swapper/11
18135862100,1sleep10-21swapper/11
17959742101,5sleep11796152-21sed1
17922152101,5sleep21792317-21sort2
17812612101,6sleep21781483-21cstates2
17599842101,5sleep20-21swapper/22
17026002101,5sleep21702714-21cpuspeed_turbos2
2162636291,4sleep02164266-21cut0
2141759291,2sleep10-21swapper/11
2133392291,2sleep20-21swapper/22
2124997291,2sleep00-21swapper/00
2039088291,3sleep01807145-21diskmemload0
2027301290,2sleep20-21swapper/22
2018521291,2sleep00-21swapper/00
1976782291,3sleep11807145-21diskmemload1
1976213291,3sleep01807145-21diskmemload0
1960091291,4sleep01960102-21ssh0
1950825291,3sleep11807145-21diskmemload1
1941938291,2sleep20-21swapper/22
1928174290,1sleep00-21swapper/00
1829210290,5sleep01829427-21timerwakeupswit0
1823041291,2sleep00-21swapper/00
1812962291,3sleep01807145-21diskmemload0
1812726291,2sleep20-21swapper/22
1797018291,4sleep21797068-21cpuspeed_turbos2
1796981291,4sleep11796736-21sendmail-msp1
1785540291,4sleep01785807-21sed0
1780670290,4sleep20-21swapper/22
1760339290,4sleep21760542-21diskstats2
1739282291,3sleep10-21swapper/11
1731180290,1sleep00-21swapper/00
1719841291,5sleep11720426-21kthreadcore1
1714498291,4sleep01716196-21cut0
1713373291,4sleep11713496-21perl1
1706140291,4sleep01706513-21grep0
1697351291,3sleep01697370-21apt-get0
1692113290,1sleep20-21swapper/22
2056164281,2sleep10-21swapper/11
1781239280,2sleep10-21swapper/11
1776039280,2sleep10-21swapper/11
1775323280,5sleep11733603-21kworker/1:1+mm_percpu_wq1
1770316280,3sleep00-21swapper/00
1755021281,2sleep11755040-21apt-get1
1749183280,1sleep20-21swapper/22
1745944281,3sleep11747634-21cut1
1724820281,3sleep11726476-21kthreadcore1
1724820281,3sleep11726476-21kthreadcore1
1723582281,3sleep01723578-21apt-get0
1723582281,3sleep01723578-21apt-get0
1712730280,1sleep20-21swapper/22
1698920281,3sleep21700610-21cut2
1692107281,2sleep139542-21systemd-journal1
1686866280,2sleep00-21swapper/00
1679565280,3sleep01679584-21apt-get0
1760254270,2sleep00-21swapper/00
1759902270,1sleep00-21swapper/00
1754630270,1sleep10-21swapper/11
1751697270,3sleep01753449-21cut0
1749782270,2sleep20-21swapper/22
1736108271,2sleep20-21swapper/22
1734660270,4sleep11736107-21kthreadcore1
1702192270,4sleep1106713-21snmpd1
1684979271,3sleep21686092-21kthreadcore2
1680139271,2sleep10-21swapper/11
1806857260,4sleep20-21swapper/22
1806817260,1sleep10-21swapper/11
1796876260,2sleep00-21swapper/00
1765514261,2sleep10-21swapper/11
1728777261,2sleep10-21swapper/11
1713497261,2sleep20-21swapper/22
1692111261,2sleep00-21swapper/00
1786759250,1sleep20-21swapper/22
1744008250,3sleep00-21swapper/00
1723348250,1sleep20-21swapper/22
1723348250,1sleep20-21swapper/22
1712165250,1sleep00-21swapper/00
1739304240,1sleep20-21swapper/22
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional