You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-27 - 02:27

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack3slot3s.osadl.org (updated Thu Aug 27, 2026 00:43:53)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
16835262142125,10sleep10-21swapper/11
1683776212660,61sleep30-21swapper/33
168360629055,11sleep20-21swapper/22
168363927762,6sleep00-21swapper/00
168401399231,19cyclictest0-21swapper/33
1684013992211,8cyclictest0-21swapper/33
168401399215,12cyclictest0-21swapper/33
19978052201,15sleep01997817-21ssh0
19809882201,2sleep0171rcu_preempt0
1684013992016,2cyclictest0-21swapper/33
1684013992012,5cyclictest1821089-21chrt3
168401399201,15cyclictest0-21swapper/33
21033052191,12sleep11812149-21diskmemload1
18253182192,9sleep21812149-21diskmemload2
168401399199,6cyclictest1889634-21turbostat3
168401399192,4cyclictest66950irq/132-enp1s0-TxRx-33
168401399191,7cyclictest2093692-21taskset3
1684013991915,2cyclictest0-21swapper/33
1684013991911,5cyclictest0-21swapper/33
168401399191,14cyclictest0-21swapper/33
1684013991910,6cyclictest1701744-21turbostat3
168401399190,2cyclictest0-21swapper/33
168401399189,6cyclictest1696434-21turbostat3
168401399181,9cyclictest2008861-21chrt3
1684013991815,1cyclictest0-21swapper/33
1684013991815,1cyclictest0-21swapper/33
1684013991814,2cyclictest1811858-21taskset3
1684013991814,2cyclictest0-21swapper/33
1684013991814,2cyclictest0-21swapper/33
168401399181,14cyclictest0-21swapper/33
168401399181,14cyclictest0-21swapper/33
168401399181,14cyclictest0-21swapper/33
168401399181,14cyclictest0-21swapper/33
168401399181,14cyclictest0-21swapper/33
168401399181,14cyclictest0-21swapper/33
168401399181,14cyclictest0-21swapper/33
168401399181,14cyclictest0-21swapper/33
168401399181,13cyclictest0-21swapper/33
168401399181,13cyclictest0-21swapper/33
168401399181,13cyclictest0-21swapper/33
168401399181,13cyclictest0-21swapper/33
168401399180,14cyclictest0-21swapper/33
168401399180,14cyclictest0-21swapper/33
168401399180,14cyclictest0-21swapper/33
20257092171,4sleep21812149-21diskmemload2
1684013991714,1cyclictest0-21swapper/33
1684013991711,3cyclictest0-21swapper/33
168401399171,13cyclictest0-21swapper/33
168401399171,13cyclictest0-21swapper/33
168401399171,13cyclictest0-21swapper/33
168401399171,13cyclictest0-21swapper/33
168401399170,14cyclictest0-21swapper/33
168401399170,14cyclictest0-21swapper/33
168401399170,13cyclictest0-21swapper/33
168401399170,13cyclictest0-21swapper/33
21219462161,3sleep20-21swapper/22
20926552161,2sleep10-21swapper/11
20530542160,1sleep00-21swapper/00
20235602161,4sleep11812149-21diskmemload1
20096272161,9sleep00-21swapper/00
19764192161,11sleep21977835-21grep2
19043492161,2sleep20-21swapper/22
18180892161,4sleep01812149-21diskmemload0
168401399161,3cyclictest0-21swapper/33
1684013991613,2cyclictest0-21swapper/33
168401399160,13cyclictest0-21swapper/33
21725182151,4sleep21812149-21diskmemload2
20955552151,4sleep21812149-21diskmemload2
20530132151,10sleep12053027-21ssh1
20503072151,7sleep10-21swapper/11
20158632150,4sleep01812149-21diskmemload0
18957502150,3sleep11879870-21kworker/u16:2+rpciod1
18192002150,4sleep21812149-21diskmemload2
18183032150,3sleep10-21swapper/11
1684013991512,2cyclictest0-21swapper/33
1684013991512,2cyclictest0-21swapper/33
1684013991512,1cyclictest0-21swapper/33
168401399151,12cyclictest2129212-21chrt3
168401399151,12cyclictest1743441-21chrt3
168401399151,11cyclictest0-21swapper/33
168401399151,11cyclictest0-21swapper/33
168401399150,12cyclictest0-21swapper/33
168401399150,12cyclictest0-21swapper/33
21550612141,2sleep20-21swapper/22
21331082140,9sleep02132784-21/usr/sbin/munin0
21197932141,8sleep01812149-21diskmemload0
20004802140,3sleep21812149-21diskmemload2
19898342141,8sleep11812149-21diskmemload1
19276742141,4sleep21928449-21ssh2
19073102141,6sleep21907573-21df2
18774342141,9sleep11877605-21/usr/sbin/munin1
17434922140,1sleep00-21swapper/00
17387462140,4sleep10-21swapper/11
168401399141,2cyclictest0-21swapper/33
168401399141,2cyclictest0-21swapper/33
1684013991411,2cyclictest0-21swapper/33
168401399141,11cyclictest0-21swapper/33
168401399140,2cyclictest0-21swapper/33
168401399140,2cyclictest0-21swapper/33
168401399140,2cyclictest0-21swapper/33
168401399140,12cyclictest0-21swapper/33
168401399140,11cyclictest17908472sleep33
168401399140,11cyclictest0-21swapper/33
168401399140,11cyclictest0-21swapper/33
21736052130,2sleep10-21swapper/11
21586562132,5sleep02158661-21apt-get0
21318132131,2sleep10-21swapper/11
21212912131,7sleep11812149-21diskmemload1
21124262131,7sleep01812149-21diskmemload0
21121482131,7sleep21812149-21diskmemload2
20708172131,8sleep22070905-21irqstats2
20683622131,2sleep10-21swapper/11
20434272131,3sleep22043438-21ssh2
20156382131,8sleep22015651-21ssh2
19374762131,2sleep21937617-21ssh2
19374762131,2sleep21937617-21ssh2
19234992131,7sleep21812149-21diskmemload2
19004222131,8sleep01901898-21grep0
18989532131,8sleep11899170-21smartctl1
18922912131,3sleep21893813-21ssh2
18532822131,8sleep01812149-21diskmemload0
18476512131,8sleep21848084-21cut2
16969762130,8sleep21696476-21sendmail-msp2
16883812131,8sleep11688532-21/usr/sbin/munin1
21561402121,8sleep02156151-21ssh0
21435842121,7sleep12145047-21cut1
21336342121,7sleep20-21swapper/22
20878142121,5sleep00-21swapper/00
20781172121,5sleep11812149-21diskmemload1
20745602121,6sleep01812149-21diskmemload0
20168202121,7sleep12016834-21ssh1
20088902121,5sleep20-21swapper/22
19731822121,7sleep21973194-21rm2
19570992121,2sleep20-21swapper/22
19495162121,2sleep10-21swapper/11
18724422121,6sleep21872468-21dump-pmu-power2
18704372121,7sleep01870449-21rm0
18459582121,3sleep11845971-21ssh1
17756202121,4sleep01760757-21kworker/0:1+events0
17494302121,5sleep11749553-21diskstats1
17230592121,5sleep11723077-21apt-get1
21749852111,3sleep00-21swapper/00
21485942110,5sleep20-21swapper/22
21396292111,2sleep10-21swapper/11
21304452111,6sleep02130456-21ssh0
21294102111,4sleep21812149-21diskmemload2
21130112111,6sleep12113025-21ssh1
21028272111,4sleep21812149-21diskmemload2
20869882111,6sleep02086999-21ssh0
20580462111,3sleep20-21swapper/22
20346282111,6sleep21812149-21diskmemload2
19903982111,3sleep20-21swapper/22
19844502111,4sleep01984474-21apt-get0
19753752111,4sleep11812149-21diskmemload1
19667902111,6sleep21967023-21cat2
19665762111,3sleep10-21swapper/11
19565872111,6sleep11956597-21ssh1
19409122111,3sleep10-21swapper/11
19409122111,3sleep10-21swapper/11
19300772111,4sleep01812149-21diskmemload0
19154472111,6sleep11915717-21ssh1
18690352111,5sleep11863856-21sendmail-msp1
18448152111,4sleep21812149-21diskmemload2
18360602110,1sleep00-21swapper/00
18018182111,6sleep04192558-21rs:main0
17873032111,6sleep01788958-21kthreadcore0
17813932111,6sleep11782783-21kthreadcore1
17544962111,4sleep11754515-21apt-get1
17544962111,4sleep11754515-21apt-get1
17357672111,6sleep11737361-21cat1
21646732101,4sleep12164685-21ssh1
21507062101,5sleep12152053-21tr1
21033442101,3sleep00-21swapper/00
20856062100,1sleep20-21swapper/22
20829082102,4sleep12083049-21/usr/sbin/munin1
20623332103,4sleep00-21swapper/00
20525012101,4sleep01812149-21diskmemload0
20495692101,4sleep21812149-21diskmemload2
20359182101,4sleep12036184-21df1
20324872101,3sleep00-21swapper/00
20243502100,2sleep00-21swapper/00
19815682101,3sleep10-21swapper/11
19752132101,3sleep00-21swapper/00
19664152101,3sleep00-21swapper/00
19501742100,1sleep00-21swapper/00
19416082101,3sleep00-21swapper/00
19295302101,3sleep10-21swapper/11
19137112101,3sleep10-21swapper/11
19132772101,5sleep01812149-21diskmemload0
18951892100,2sleep00-21swapper/00
18859212101,4sleep01886175-21users0
18804652101,3sleep00-21swapper/00
18618052101,4sleep21812149-21diskmemload2
18615352101,5sleep01861549-21ssh0
18558652101,4sleep11855979-21perl1
18384422101,5sleep01838595-21cpuspeed_turbos0
18350392101,3sleep21812149-21diskmemload2
18346792101,3sleep10-21swapper/11
18059602101,4sleep21801602-21sendmail-msp2
17968712101,4sleep21797081-21hddtemp_smartct2
17962042100,1sleep10-21swapper/11
17956952101,4sleep21795887-21users2
17758242101,5sleep11776033-21hddtemp_smartct1
17702972101,3sleep21770316-21apt-get2
17650412102,3sleep01765059-21apt-get0
17177582101,5sleep01717537-21sendmail-msp0
2164242291,4sleep22164256-21sshd-auth2
2141476291,4sleep02141734-21df0
2067090291,3sleep21812149-21diskmemload2
2036481291,3sleep02036820-21ssh0
2033471291,4sleep12033481-21ssh1
2006744291,3sleep11812149-21diskmemload1
1998473291,3sleep11812149-21diskmemload1
1947794291,2sleep21812149-21diskmemload2
1939966291,4sleep01939978-21sh0
1939966291,4sleep01939978-21sh0
1915742291,2sleep00-21swapper/00
1889324291,3sleep11812149-21diskmemload1
1888803292,3sleep21812149-21diskmemload2
1879300290,3sleep20-21swapper/22
1846940290,1sleep10-21swapper/11
1825815291,2sleep10-21swapper/11
1821105290,2sleep0171rcu_preempt0
1807735291,4sleep11809038-21grep1
1800630290,5sleep01800726-21ps0
1764449290,1sleep00-21swapper/00
1764449290,1sleep00-21swapper/00
1762946291,4sleep139542-21systemd-journal1
1762946291,4sleep139542-21systemd-journal1
1733524291,4sleep01733764-21cat0
1729644291,4sleep11731324-21grep1
1727131291,4sleep21723058-21/usr/sbin/munin2
1718296291,5sleep11718535-21kthreadcore1
1712799291,4sleep11713012-21hddtemp_smartct1
1712508290,4sleep21712527-21apt-get2
1701246291,4sleep0205-21jbd2/sda2-80
1693262292,3sleep01695000-21kthreadcore0
1691940291,4sleep2431ktimers/22
1684471291,4sleep21684698-21basename2
1807158280,3sleep01807177-21apt-get0
1801849281,3sleep11801847-21apt-get1
1800729281,3sleep11800855-21ls1
1789952280,1sleep20-21swapper/22
1785444281,3sleep21780562-21sendmail-msp2
1770322281,4sleep01770583-21df_inode0
1759780281,2sleep20-21swapper/22
1759780281,2sleep20-21swapper/22
1748059281,3sleep21748122-21/usr/sbin/munin2
1744030281,2sleep10-21swapper/11
1732487281,3sleep20-21swapper/22
1727789280,2sleep00-21swapper/00
1706949281,3sleep10-21swapper/11
1685066281,3sleep01686645-21kthreadcore0
1807324270,3sleep20-21swapper/22
1785334271,2sleep00-21swapper/00
1775533270,2sleep20-21swapper/22
1775024270,1sleep10-21swapper/11
1759141270,1sleep00-21swapper/00
1759141270,1sleep00-21swapper/00
1749841271,2sleep00-21swapper/00
1739047271,2sleep20-21swapper/22
1733184270,1sleep00-21swapper/00
1722676270,1sleep20-21swapper/22
1707246271,2sleep21707224-21/usr/sbin/munin2
1707240270,2sleep10-21swapper/11
1706683270,1sleep00-21swapper/00
1706621270,1sleep20-21swapper/22
1696692271,2sleep10-21swapper/11
1689221271,2sleep10-21swapper/11
1786373261,2sleep10-21swapper/11
1748673260,2sleep00-21swapper/00
1711912260,1sleep00-21swapper/00
1791327250,1sleep00-21swapper/00
1767410250,1sleep20-21swapper/22
1765062250,1sleep10-21swapper/11
1754518250,2sleep20-21swapper/22
1754518250,2sleep20-21swapper/22
1749554250,1sleep20-21swapper/22
1737363250,1sleep20-21swapper/22
1712328240,1sleep00-21swapper/00
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional