You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-22 - 03:24

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack3slot3s.osadl.org (updated Wed Apr 22, 2026 00:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1590271212162,55sleep30-21swapper/33
207190121100,2sleep2421rcuc/22
159018528567,12sleep20-21swapper/22
159010128554,12sleep10-21swapper/11
159024827454,14sleep00-21swapper/00
1590454992410,10cyclictest1905268-21turbostat3
159045499221,17cyclictest0-21swapper/33
159045499211,17cyclictest0-21swapper/33
1590454992111,6cyclictest1972360-21taskset3
1590454992111,6cyclictest1972360-21taskset3
159045499211,16cyclictest0-21swapper/33
1590454992010,7cyclictest1789274-21taskset3
159045499200,16cyclictest1878903-21taskset3
20051442190,2sleep00-21swapper/00
19363772199,7sleep11931805-21/usr/sbin/munin1
18676272191,3sleep11867641-21ssh1
159045499199,7cyclictest0-21swapper/33
159045499192,11cyclictest1603313-21taskset3
1590454991917,1cyclictest0-21swapper/33
159045499191,15cyclictest0-21swapper/33
159045499191,15cyclictest0-21swapper/33
159045499191,14cyclictest0-21swapper/33
159045499191,14cyclictest0-21swapper/33
159045499191,14cyclictest0-21swapper/33
159045499191,14cyclictest0-21swapper/33
159045499190,7cyclictest1937380-21taskset3
159045499190,16cyclictest66950irq/132-enp1s0-TxRx-33
159045499187,8cyclictest1699475-21turbostat3
1590454991815,1cyclictest0-21swapper/33
159045499181,4cyclictest1753649-21taskset3
1590454991813,2cyclictest0-21swapper/33
159045499181,14cyclictest66950irq/132-enp1s0-TxRx-33
159045499181,14cyclictest66950irq/132-enp1s0-TxRx-33
159045499181,14cyclictest0-21swapper/33
159045499181,14cyclictest0-21swapper/33
159045499181,14cyclictest0-21swapper/33
159045499181,14cyclictest0-21swapper/33
159045499181,14cyclictest0-21swapper/33
159045499181,14cyclictest0-21swapper/33
159045499181,14cyclictest0-21swapper/33
159045499181,14cyclictest0-21swapper/33
159045499181,13cyclictest0-21swapper/33
159045499181,13cyclictest0-21swapper/33
159045499181,13cyclictest0-21swapper/33
159045499181,13cyclictest0-21swapper/33
159045499180,15cyclictest66950irq/132-enp1s0-TxRx-33
159045499180,15cyclictest0-21swapper/33
20249962171,3sleep00-21swapper/00
19573192171,4sleep11720810-21diskmemload1
19188502171,3sleep10-21swapper/11
17550812171,3sleep00-21swapper/00
159045499171,13cyclictest0-21swapper/33
159045499171,13cyclictest0-21swapper/33
159045499171,13cyclictest0-21swapper/33
1590454991710,4cyclictest66950irq/132-enp1s0-TxRx-33
159045499170,14cyclictest0-21swapper/33
159045499170,14cyclictest0-21swapper/33
159045499170,14cyclictest0-21swapper/33
159045499170,13cyclictest0-21swapper/33
159045499170,13cyclictest0-21swapper/33
159045499170,13cyclictest0-21swapper/33
20860832161,3sleep00-21swapper/00
19545262161,4sleep01954540-21ssh0
19236942161,10sleep21924652-21cat2
18354382161,6sleep01835713-21df_inode0
17537842161,3sleep20-21swapper/22
17359632161,4sleep01720810-21diskmemload0
159045499168,5cyclictest1683486-21turbostat3
1590454991614,1cyclictest0-21swapper/33
1590454991612,1cyclictest0-21swapper/33
159045499161,13cyclictest0-21swapper/33
159045499160,13cyclictest0-21swapper/33
159045499160,12cyclictest0-21swapper/33
20717312150,2sleep00-21swapper/00
19919832152,3sleep11720810-21diskmemload1
19907352151,4sleep21720810-21diskmemload2
19447642151,2sleep00-21swapper/00
18575012151,4sleep01857561-21ssh0
18433892151,4sleep21720810-21diskmemload2
18174632151,2sleep20-21swapper/22
17549862151,3sleep11720810-21diskmemload1
159045499153,7cyclictest16890872sleep33
1590454991512,2cyclictest0-21swapper/33
1590454991512,1cyclictest0-21swapper/33
159045499151,12cyclictest0-21swapper/33
159045499150,13cyclictest0-21swapper/33
159045499150,13cyclictest0-21swapper/33
159045499150,12cyclictest0-21swapper/33
20551932141,3sleep12056801-21kthreadcore1
19985642141,3sleep11998574-21ssh1
19885392141,9sleep01988599-21cut0
19399902141,8sleep21720810-21diskmemload2
18783182140,1sleep10-21swapper/11
18758802141,3sleep01720810-21diskmemload0
18108342144,6sleep01812535-21cut0
17262152140,2sleep20-21swapper/22
17039402141,9sleep01704057-21/usr/sbin/munin0
159045499141,2cyclictest0-21swapper/33
159045499141,11cyclictest1645921-21turbostat3
159045499140,11cyclictest0-21swapper/33
159045499140,11cyclictest0-21swapper/33
19966292130,2sleep20-21swapper/22
19741192131,3sleep21720810-21diskmemload2
19741192131,3sleep21720810-21diskmemload2
19739582131,4sleep11973973-21ssh1
19739582131,4sleep11973973-21ssh1
19233892131,8sleep01923545-21expr0
19224252132,6sleep21720810-21diskmemload2
19222502131,6sleep00-21swapper/00
19107132131,6sleep10-21swapper/11
18863392131,5sleep00-21swapper/00
18794562131,2sleep20-21swapper/22
18332852131,7sleep01720810-21diskmemload0
18057332131,8sleep21805747-21ssh2
17727642131,8sleep01772776-21ssh0
17630592130,2sleep10-21swapper/11
17050812131,8sleep11705099-21apt-get1
16458962131,8sleep01645980-21sort0
16301992131,3sleep11630218-21apt-get1
159045499130,1cyclictest0-21swapper/33
20718792121,3sleep11720810-21diskmemload1
20296112121,7sleep22031338-21cut2
20279622121,5sleep02027990-21apt-get0
20260302121,6sleep21720810-21diskmemload2
20104382121,7sleep12010707-21df1
20058512121,5sleep22005909-21ntpq2
20015492121,5sleep10-21swapper/11
19827282121,6sleep10-21swapper/11
19817742121,7sleep21981785-21ssh2
19564192121,7sleep21956432-21ssh2
18706642121,3sleep21870678-21ssh2
18578932121,6sleep11858121-21users1
18537692121,6sleep21854158-21cut2
18170012123,4sleep1331ktimers/11
17806722121,7sleep2431ktimers/22
17359742121,3sleep11735989-21ssh1
17358402121,6sleep21720810-21diskmemload2
17037712121,5sleep11703828-21ntpq1
16568332121,7sleep21657020-21expr2
16150742121,7sleep2802-21snmpd2
15956652120,1sleep00-21swapper/00
20503792111,3sleep11720810-21diskmemload1
20494972111,5sleep02049589-21ntpq0
20145692111,4sleep02014620-21ssh0
20103762111,6sleep22010390-21ssh2
19807852110,7sleep01720810-21diskmemload0
19632792112,3sleep01720810-21diskmemload0
19049962110,7sleep01905011-21ssh0
18975102111,6sleep21898884-21grep2
18957572111,6sleep01895768-21ssh0
18512552111,3sleep20-21swapper/22
18494082111,5sleep01720810-21diskmemload0
18312192111,5sleep21831302-21find2
18261712111,5sleep11720810-21diskmemload1
18063182111,3sleep00-21swapper/00
17956432111,5sleep11795692-21ssh1
17818722110,2sleep10-21swapper/11
17782212111,6sleep01778284-21ld0
17700842111,3sleep10-21swapper/11
17441812111,3sleep10-21swapper/11
17429662111,3sleep20-21swapper/22
16994662111,5sleep21699514-21munin-run2
16825092111,5sleep21682562-21ntpq2
16668472110,1sleep10-21swapper/11
16608492111,5sleep11660895-21ntpq1
15906522111,5sleep11590870-21df_inode1
20883602101,4sleep11720810-21diskmemload1
20806512101,3sleep20-21swapper/22
20693892101,3sleep20-21swapper/22
20682142101,4sleep11720810-21diskmemload1
20680502101,5sleep01720810-21diskmemload0
20610052101,5sleep02061016-21ssh0
20525602100,2sleep20-21swapper/22
20442012101,3sleep01720810-21diskmemload0
20278702101,4sleep11720810-21diskmemload1
20258692101,3sleep10-21swapper/11
20002812101,4sleep01720810-21diskmemload0
19637682101,3sleep20-21swapper/22
19576162100,1sleep10-21swapper/11
19457622100,2sleep10-21swapper/11
19360282101,4sleep01936088-21ntpq0
19290592101,3sleep10-21swapper/11
19120472101,4sleep21720810-21diskmemload2
19052572100,6sleep01905514-21ssh0
18968782101,3sleep10-21swapper/11
18945212101,4sleep21720810-21diskmemload2
18937972101,2sleep10-21swapper/11
18874052101,4sleep11720810-21diskmemload1
18701422101,4sleep01720810-21diskmemload0
18693112101,4sleep21720810-21diskmemload2
18504362101,2sleep10-21swapper/11
18338542101,3sleep10-21swapper/11
18232522101,4sleep21720810-21diskmemload2
17956972101,4sleep21795754-21ntpq2
17612192101,3sleep00-21swapper/00
17387142101,4sleep01738733-21apt-get0
17292322101,3sleep10-21swapper/11
17273012101,4sleep01720810-21diskmemload0
16843572101,5sleep21686032-21kthreadcore2
16823182100,5sleep01682459-21ntpq0
16735922101,5sleep21673661-21/usr/sbin/munin2
16555652101,5sleep01655611-21ntpq0
16394132101,6sleep01635507-21/usr/sbin/munin0
16289732100,2sleep00-21swapper/00
16139282101,4sleep11613947-21apt-get1
16086152101,3sleep10-21swapper/11
16032912101,3sleep20-21swapper/22
2037008291,3sleep12037228-21hddtemp_smartct1
2036581290,3sleep20-21swapper/22
1972023291,3sleep01720810-21diskmemload0
1972023291,3sleep01720810-21diskmemload0
1840397291,3sleep11720810-21diskmemload1
1822462291,3sleep01720810-21diskmemload0
1800228291,3sleep10-21swapper/11
1789407291,2sleep20-21swapper/22
1789036291,2sleep00-21swapper/00
1772629291,3sleep21720810-21diskmemload2
1761191291,4sleep21761203-21sh2
1715780291,4sleep01716039-21df_inode0
1710144291,4sleep01710175-21munin-run0
1694690290,1sleep00-21swapper/00
1694400291,3sleep11694419-21apt-get1
1693178291,4sleep01693232-21ld0
1673055291,3sleep01672560-21apt-get0
1619255291,4sleep11619449-21expr1
1598024291,4sleep21598227-21diskstats2
2061883281,4sleep22061894-21ssh2
1940480281,2sleep20-21swapper/22
1796990281,3sleep01720810-21diskmemload0
1788302281,3sleep11720810-21diskmemload1
1719667281,3sleep21719788-21ntpq2
1693233281,3sleep11693308-21head1
1688515280,1sleep00-21swapper/00
1678434280,3sleep11678453-21apt-get1
1661006280,1sleep00-21swapper/00
1646475281,3sleep21646685-21hddtemp_smartct2
1639598281,3sleep21639647-21ntpq2
1635075280,1sleep00-21swapper/00
1629787280,1sleep10-21swapper/11
1624834280,1sleep20-21swapper/22
1623605280,1sleep20-21swapper/22
1619233281,2sleep01619252-21apt-get0
1613650280,2sleep00-21swapper/00
1597939281,2sleep01597958-21apt-get0
1715135270,1sleep10-21swapper/11
1711368271,2sleep00-21swapper/00
1693978270,1sleep20-21swapper/22
1677993270,1sleep10-21swapper/11
1672352270,1sleep00-21swapper/00
1671354271,3sleep20-21swapper/22
1662138270,2sleep00-21swapper/00
1646786271,3sleep00-21swapper/00
1641682271,2sleep10-21swapper/11
1635506270,2sleep10-21swapper/11
1608314270,1sleep00-21swapper/00
1595367270,1sleep20-21swapper/22
1720562260,1sleep10-21swapper/11
1715253260,1sleep20-21swapper/22
1706490261,2sleep21708250-21kthreadcore2
1703830260,1sleep20-21swapper/22
1683739260,2sleep10-21swapper/11
1671608260,3sleep10-21swapper/11
1667150260,1sleep20-21swapper/22
1656374260,1sleep10-21swapper/11
1650905260,1sleep10-21swapper/11
1645553260,4sleep20-21swapper/22
1634262260,2sleep21634309-21ntpq2
1609442261,2sleep20-21swapper/22
1608095260,1sleep10-21swapper/11
1598511261,2sleep10-21swapper/11
1655962250,1sleep20-21swapper/22
1617812250,1sleep00-21swapper/00
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional