You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-19 - 14:27

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3s.osadl.org (updated Tue May 19, 2026 12:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1573875211953,60sleep30-21swapper/33
156938328154,14sleep00-21swapper/00
156958628055,15sleep10-21swapper/11
157430227659,12sleep20-21swapper/22
18928312220,2sleep30-21swapper/33
157471599221,17cyclictest2042984-21taskset3
157471599220,20cyclictest0-21swapper/33
157471599211,17cyclictest66950irq/132-enp1s0-TxRx-33
157471599210,19cyclictest0-21swapper/33
18312722201,8sleep01831286-21ssh0
157471599201,16cyclictest1923755-21taskset3
157471599200,17cyclictest0-21swapper/33
20452292191,7sleep22045520-21grep2
19927732191,13sleep01705735-21diskmemload0
157471599197,9cyclictest1764919-21turbostat3
1574715991912,5cyclictest0-21swapper/33
157471599191,15cyclictest66950irq/132-enp1s0-TxRx-33
157471599191,15cyclictest0-21swapper/33
157471599191,15cyclictest0-21swapper/33
157471599191,15cyclictest0-21swapper/33
157471599191,14cyclictest66950irq/132-enp1s0-TxRx-33
157471599191,14cyclictest0-21swapper/33
157471599191,14cyclictest0-21swapper/33
157471599191,14cyclictest0-21swapper/33
1574715991910,6cyclictest1966167-21taskset3
18571962181,3sleep00-21swapper/00
157471599189,6cyclictest66950irq/132-enp1s0-TxRx-33
157471599189,4cyclictest18247662sleep33
157471599181,5cyclictest0-21swapper/33
157471599181,15cyclictest66950irq/132-enp1s0-TxRx-33
157471599181,15cyclictest0-21swapper/33
157471599181,14cyclictest0-21swapper/33
157471599181,14cyclictest0-21swapper/33
157471599181,13cyclictest1838355-21taskset3
157471599181,13cyclictest0-21swapper/33
157471599181,13cyclictest0-21swapper/33
1574715991810,6cyclictest0-21swapper/33
157471599180,15cyclictest0-21swapper/33
157471599180,15cyclictest0-21swapper/33
157471599180,15cyclictest0-21swapper/33
157471599180,14cyclictest0-21swapper/33
157471599178,6cyclictest1636138-21turbostat3
157471599178,6cyclictest1636138-21turbostat3
1574715991713,2cyclictest0-21swapper/33
1574715991712,3cyclictest1748514-21taskset3
1574715991711,4cyclictest0-21swapper/33
157471599171,13cyclictest0-21swapper/33
157471599171,13cyclictest0-21swapper/33
157471599171,13cyclictest0-21swapper/33
157471599171,13cyclictest0-21swapper/33
157471599171,12cyclictest1619860-21taskset3
157471599170,14cyclictest0-21swapper/33
20012262161,3sleep20-21swapper/22
19522462161,8sleep11952379-21diskstats1
19167542161,4sleep10-21swapper/11
18551472161,4sleep11705735-21diskmemload1
17628892161,2sleep10-21swapper/11
17281982169,3sleep11705735-21diskmemload1
157471599164,1cyclictest0-21swapper/33
1574715991612,3cyclictest0-21swapper/33
1574715991612,3cyclictest0-21swapper/33
157471599161,13cyclictest0-21swapper/33
157471599161,12cyclictest0-21swapper/33
157471599160,13cyclictest0-21swapper/33
157471599160,13cyclictest0-21swapper/33
157471599160,13cyclictest0-21swapper/33
157471599160,13cyclictest0-21swapper/33
157471599160,13cyclictest0-21swapper/33
157471599160,13cyclictest0-21swapper/33
157471599160,13cyclictest0-21swapper/33
157471599160,13cyclictest0-21swapper/33
157471599160,13cyclictest0-21swapper/33
157471599160,13cyclictest0-21swapper/33
157471599160,12cyclictest0-21swapper/33
20235682151,9sleep22023625-21ntpq2
20003162151,3sleep01705735-21diskmemload0
19752442151,4sleep01975255-21ssh0
19577122151,2sleep20-21swapper/22
19492872150,1sleep00-21swapper/00
19435892151,4sleep11705735-21diskmemload1
17442482152,6sleep21744302-21ntpq2
17400342152,8sleep11740060-21apt-get1
17271862153,6sleep21727235-21ntpq2
1574715991512,2cyclictest0-21swapper/33
157471599150,14cyclictest0-21swapper/33
157471599150,13cyclictest0-21swapper/33
157471599150,12cyclictest0-21swapper/33
20536172141,9sleep12053699-21sh1
19252302141,8sleep11705735-21diskmemload1
18911312141,8sleep11705735-21diskmemload1
18051082141,2sleep20-21swapper/22
17814012141,8sleep21705735-21diskmemload2
17225162141,8sleep01705735-21diskmemload0
17186342141,7sleep11718695-21ntpq1
17138812141,4sleep11713893-21ssh1
17116152141,2sleep20-21swapper/22
16727012141,2sleep10-21swapper/11
16101222141,3sleep11610238-21irqstats1
157471599140,2cyclictest0-21swapper/33
157471599140,12cyclictest0-21swapper/33
157471599140,11cyclictest0-21swapper/33
20292982131,7sleep02030954-21tr0
20188612131,8sleep21705735-21diskmemload2
20113042131,2sleep10-21swapper/11
20089752131,2sleep10-21swapper/11
19741702131,8sleep11974181-21ssh1
19659942131,3sleep01966005-21ssh0
18735322131,2sleep00-21swapper/00
18637112131,8sleep11863721-21ssh1
18637112131,8sleep11863721-21ssh1
18554382131,8sleep21855455-21ssh2
18298122131,3sleep11829824-21ssh1
16090472131,8sleep01609048-40tracker-extract0
15917492131,6sleep11591803-21ntpq1
1574715991311,1cyclictest0-21swapper/33
157471599130,10cyclictest15979082sleep33
20259322124,2sleep1171rcu_preempt1
19583492124,4sleep01793719-21kworker/u16:1+nfsiod0
19436752121,7sleep21943879-21expr2
19161452120,7sleep21705735-21diskmemload2
18680482121,7sleep11868387-21cat1
18395042121,5sleep00-21swapper/00
18113872121,6sleep11811627-21memory1
17902352121,5sleep10-21swapper/11
17626622121,7sleep21762676-21ssh2
17057212121,5sleep21705748-21munin-run2
17018982121,7sleep01703619-21grep0
16354702121,7sleep21635724-21head2
16354702121,7sleep21635724-21head2
15828342121,7sleep21584010-21cut2
20516912111,5sleep01705735-21diskmemload0
19993212111,7sleep11999334-21sh1
19911052111,5sleep11705735-21diskmemload1
19856922111,5sleep21985704-21/usr/sbin/munin2
19838532111,3sleep20-21swapper/22
19429772111,3sleep20-21swapper/22
19263332111,6sleep21926339-21ssh2
18989302111,4sleep21705735-21diskmemload2
18840042112,4sleep11705735-21diskmemload1
18716352111,5sleep21871690-21ntpq2
18593222111,6sleep21860767-21tr2
18593222111,6sleep21860767-21tr2
18400392111,7sleep11705735-21diskmemload1
18333902111,5sleep21833656-21df2
18221312111,3sleep00-21swapper/00
18190502110,1sleep20-21swapper/22
18118352111,5sleep01811898-21ntpq0
18033802111,6sleep11803446-21modprobe1
17984262111,4sleep11705735-21diskmemload1
17973272111,5sleep01705735-21diskmemload0
17907042112,4sleep21790728-21apt-get2
17818092111,5sleep01781868-21munin-run0
17717012111,3sleep10-21swapper/11
17646792110,6sleep014-21ksoftirqd/00
17559762111,7sleep01755978-21ssh0
17465042111,5sleep01705735-21diskmemload0
17390882111,4sleep10-21swapper/11
16993792111,5sleep21699426-21ntpq2
16993792111,5sleep21699426-21ntpq2
16903952111,5sleep21690511-21irqstats2
16843562111,7sleep21684430-21date2
16674012111,5sleep21667450-21ntpq2
16620802111,5sleep11662131-21ntpq1
16421652111,5sleep21642281-21irqstats2
16359542111,4sleep172-21kcompactd01
16359542111,4sleep172-21kcompactd01
16090092110,2sleep10-21swapper/11
15973322111,6sleep11597478-21sort1
15753862111,5sleep11575457-21irqstats1
20599852101,4sleep01705735-21diskmemload0
20531072101,5sleep22053258-21cut2
20486342101,2sleep12048806-21cut1
20432582100,6sleep02043270-21ssh0
20417312101,2sleep10-21swapper/11
20340232101,4sleep11705735-21diskmemload1
20331362101,4sleep21705735-21diskmemload2
20170852101,5sleep01705735-21diskmemload0
20080792100,6sleep02008090-21ssh0
19813912101,4sleep11981453-21ld1
19673812101,6sleep21967383-21sh2
19660062100,2sleep11966020-21ssh1
19344612101,5sleep11934475-21ssh1
19339152100,5sleep20-21swapper/22
19251672101,4sleep01705735-21diskmemload0
19100662101,4sleep01910281-21diskstats0
19056852101,4sleep11905738-21ssh1
19010322101,2sleep10-21swapper/11
18980952101,3sleep01705735-21diskmemload0
18926202101,5sleep21892830-21date2
18821832101,4sleep01705735-21diskmemload0
18818962101,2sleep20-21swapper/22
18659752101,4sleep01705735-21diskmemload0
18659752101,4sleep01705735-21diskmemload0
18442132101,3sleep01705735-21diskmemload0
18416902101,2sleep10-21swapper/11
18290342101,4sleep21705735-21diskmemload2
18240132100,1sleep10-21swapper/11
18059702101,3sleep00-21swapper/00
17892092100,1sleep20-21swapper/22
17737982101,4sleep11774062-21df1
17652642103,2sleep0221rcuc/00
17651852100,1sleep20-21swapper/22
17553892101,3sleep20-21swapper/22
17540292101,2sleep10-21swapper/11
17387692101,2sleep20-21swapper/22
17386882101,5sleep01738699-21ssh0
17275642103,3sleep00-21swapper/00
17202922101,3sleep20-21swapper/22
16886662101,5sleep21688712-21ntpq2
16580072101,4sleep01658026-21apt-get0
16528452101,4sleep11652969-21diskstats1
16414532101,4sleep21641477-21munin-run2
16102392100,6sleep21611341-21grep2
2043232290,1sleep20-21swapper/22
2010252291,2sleep20-21swapper/22
1977731291,4sleep01977813-21irqstats0
1973951291,3sleep20-21swapper/22
1942907291,3sleep01705735-21diskmemload0
1942074291,3sleep10-21swapper/11
1932927291,3sleep01705735-21diskmemload0
1901896291,3sleep01705735-21diskmemload0
1901438291,4sleep21705735-21diskmemload2
1890378290,4sleep00-21swapper/00
1847935291,2sleep20-21swapper/22
1813262291,3sleep21705735-21diskmemload2
1789803291,4sleep01789816-21sh0
1712102291,2sleep00-21swapper/00
1694104291,4sleep11694180-21pmu-power1
1674275290,5sleep21674488-21hddtemp_smartct2
1647830291,4sleep11648059-21cut1
1645827291,4sleep01645883-21ntpq0
1640290291,4sleep01640428-21nfs4_client0
1635123291,4sleep01635173-21ntpq0
1635123291,4sleep01635173-21ntpq0
1616486291,4sleep01618232-21cut0
1602671291,3sleep21602719-21ntpq2
1598563291,3sleep11598582-21apt-get1
1582554291,4sleep01582717-21/usr/sbin/munin0
2025630281,3sleep02025641-21ssh0
1663332280,3sleep11663351-21apt-get1
1652703281,3sleep00-21swapper/00
1648060281,4sleep21649410-21kthreadcore2
1636384281,2sleep10-21swapper/11
1626259281,3sleep1321rcuc/11
1625419280,1sleep10-21swapper/11
1624804280,1sleep00-21swapper/00
1620957281,3sleep21622332-21cat2
1609005280,3sleep290-21kswapd02
1596864280,1sleep20-21swapper/22
1587081280,1sleep10-21swapper/11
1703620270,4sleep10-21swapper/11
1695292271,2sleep01695311-21apt-get0
1695292271,2sleep01695311-21apt-get0
1683442271,3sleep01683487-21/usr/sbin/munin0
1676084270,3sleep11677809-21grep1
1674577271,3sleep01676082-21kthreadcore0
1668636270,2sleep00-21swapper/00
1668633270,1sleep20-21swapper/22
1630683270,1sleep00-21swapper/00
1615347270,3sleep11615531-21sed1
1609960270,4sleep00-21swapper/00
1579743270,1sleep00-21swapper/00
1691642261,2sleep00-21swapper/00
1689600260,1sleep00-21swapper/00
1684615260,2sleep10-21swapper/11
1662912260,1sleep20-21swapper/22
1652669260,4sleep20-21swapper/22
1615048260,1sleep20-21swapper/22
1597684260,1sleep00-21swapper/00
1592602260,1sleep00-21swapper/00
1592522260,1sleep20-21swapper/22
1575458261,2sleep20-21swapper/22
1695313250,2sleep10-21swapper/11
1695313250,2sleep10-21swapper/11
1667214250,1sleep00-21swapper/00
1645885250,2sleep10-21swapper/11
1625477250,1sleep20-21swapper/22
1598533250,1sleep00-21swapper/00
1679673240,1sleep10-21swapper/11
1647302240,1sleep00-21swapper/00
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional