You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-12 - 08:31

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3s.osadl.org (updated Tue May 12, 2026 00:43:53)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
277795621270,3sleep12777962-21needreboot1
2712657212061,55sleep30-21swapper/33
2711763211192,10sleep10-21swapper/11
271278428163,12sleep20-21swapper/22
271259526956,7sleep00-21swapper/00
29450662231,7sleep02843317-21diskmemload0
271303499221,8cyclictest3187948-21taskset3
2713034992117,2cyclictest0-21swapper/33
271303499211,3cyclictest0-21swapper/33
271303499211,16cyclictest0-21swapper/33
271303499211,16cyclictest0-21swapper/33
271303499211,15cyclictest2902927-21chrt3
271303499210,18cyclictest0-21swapper/33
2713034992017,1cyclictest0-21swapper/33
271303499201,16cyclictest2933593-21taskset3
271303499199,8cyclictest2943164-21chrt3
271303499199,7cyclictest3071379-21taskset3
271303499192,14cyclictest66950irq/132-enp1s0-TxRx-33
2713034991916,1cyclictest0-21swapper/33
2713034991916,1cyclictest0-21swapper/33
271303499191,15cyclictest0-21swapper/33
271303499191,14cyclictest0-21swapper/33
271303499191,14cyclictest0-21swapper/33
271303499191,14cyclictest0-21swapper/33
271303499191,14cyclictest0-21swapper/33
271303499191,14cyclictest0-21swapper/33
271303499191,14cyclictest0-21swapper/33
31893032182,4sleep00-21swapper/00
28666752181,3sleep02866689-21ssh0
28590542181,4sleep12859068-21ssh1
271303499181,14cyclictest66950irq/132-enp1s0-TxRx-33
271303499181,14cyclictest0-21swapper/33
271303499181,14cyclictest0-21swapper/33
271303499181,14cyclictest0-21swapper/33
271303499181,14cyclictest0-21swapper/33
271303499181,14cyclictest0-21swapper/33
271303499181,14cyclictest0-21swapper/33
271303499181,14cyclictest0-21swapper/33
271303499181,14cyclictest0-21swapper/33
271303499181,14cyclictest0-21swapper/33
271303499181,14cyclictest0-21swapper/33
271303499181,14cyclictest0-21swapper/33
271303499181,13cyclictest0-21swapper/33
271303499181,13cyclictest0-21swapper/33
271303499180,2cyclictest0-21swapper/33
271303499180,15cyclictest0-21swapper/33
271303499180,15cyclictest0-21swapper/33
271303499180,14cyclictest0-21swapper/33
271303499180,14cyclictest0-21swapper/33
31390892171,3sleep20-21swapper/22
29529702171,4sleep22843317-21diskmemload2
29529702171,4sleep22843317-21diskmemload2
28515712171,3sleep00-21swapper/00
28515712171,3sleep00-21swapper/00
271303499171,4cyclictest0-21swapper/33
271303499171,4cyclictest0-21swapper/33
271303499171,4cyclictest0-21swapper/33
2713034991712,3cyclictest0-21swapper/33
271303499171,13cyclictest0-21swapper/33
271303499171,13cyclictest0-21swapper/33
271303499171,12cyclictest0-21swapper/33
271303499170,14cyclictest0-21swapper/33
271303499170,14cyclictest0-21swapper/33
271303499170,13cyclictest0-21swapper/33
31960832161,10sleep02843317-21diskmemload0
31417292161,12sleep13142024-21kthreadcore1
31291732161,2sleep20-21swapper/22
31291732161,2sleep20-21swapper/22
30960202161,3sleep00-21swapper/00
30103362161,2sleep10-21swapper/11
29851792160,1sleep10-21swapper/11
29335562161,4sleep02843317-21diskmemload0
2713034991613,1cyclictest0-21swapper/33
271303499161,13cyclictest0-21swapper/33
271303499161,11cyclictest0-21swapper/33
271303499160,13cyclictest0-21swapper/33
271303499160,13cyclictest0-21swapper/33
271303499160,13cyclictest0-21swapper/33
271303499160,13cyclictest0-21swapper/33
271303499160,13cyclictest0-21swapper/33
271303499160,13cyclictest0-21swapper/33
30391052151,10sleep03039150-21kthreadcore0
30391052151,10sleep03039150-21kthreadcore0
30365822151,3sleep02843317-21diskmemload0
29939862151,9sleep02843317-21diskmemload0
29596192151,3sleep02843317-21diskmemload0
29071772151,5sleep02907235-21ntpq0
28986742151,4sleep22898728-21ntpq2
28168932151,10sleep02816963-21fschecks_count0
2713034991513,1cyclictest0-21swapper/33
271303499151,2cyclictest0-21swapper/33
271303499150,12cyclictest0-21swapper/33
271303499150,12cyclictest0-21swapper/33
271303499150,12cyclictest0-21swapper/33
30510672142,6sleep13051125-21ntpq1
30435582141,2sleep10-21swapper/11
30435582141,2sleep10-21swapper/11
30216292140,3sleep20-21swapper/22
29842342141,4sleep22984244-21ssh2
28501342141,9sleep12850144-21ssh1
28501342141,9sleep12850144-21ssh1
27941452141,5sleep22794268-21find2
271303499141,2cyclictest0-21swapper/33
2713034991411,1cyclictest0-21swapper/33
271303499140,12cyclictest0-21swapper/33
271303499140,12cyclictest0-21swapper/33
271303499140,11cyclictest0-21swapper/33
31803452130,2sleep20-21swapper/22
31554892131,8sleep13155499-21ssh1
31113912131,7sleep22843317-21diskmemload2
31031212130,3sleep10-21swapper/11
30793492131,7sleep12843317-21diskmemload1
30680062132,6sleep03068063-21ntpq0
29411762131,6sleep22941242-21ntp_states2
28997022131,7sleep02899716-21ssh0
28938012131,5sleep20-21swapper/22
27469932131,2sleep00-21swapper/00
271303499131,1cyclictest0-21swapper/33
271303499130,11cyclictest0-21swapper/33
31787652121,7sleep02843317-21diskmemload0
31648582121,3sleep03164910-21idleruntime-cro0
31404982121,3sleep03140533-21kthreadcore0
31194102121,7sleep13119421-21ssh1
31185362122,5sleep23118589-21ntpq2
31134632121,3sleep03113474-21ssh0
30949522121,3sleep20-21swapper/22
30948522121,6sleep12843317-21diskmemload1
30870402121,6sleep22843317-21diskmemload2
30811212121,6sleep13081340-21hddtemp_smartct1
30809862121,7sleep03081119-21ssh0
30289942121,5sleep10-21swapper/11
29438802121,6sleep12843317-21diskmemload1
29028972121,5sleep22843317-21diskmemload2
29016752121,6sleep12843317-21diskmemload1
28684972120,3sleep12868509-21ssh1
28666942121,7sleep22866702-21ssh2
28205222121,5sleep12820663-21ntpq1
27686032122,5sleep22768622-21apt-get2
27565672121,5sleep22756656-21ntpq2
31954952110,3sleep20-21swapper/22
31356512111,5sleep13135702-21ntpq1
31264362111,6sleep03122876-21/usr/sbin/munin0
31264362111,6sleep03122876-21/usr/sbin/munin0
31016572111,6sleep090-21kswapd00
30700222111,4sleep22843317-21diskmemload2
30612572111,4sleep22843317-21diskmemload2
30608012110,2sleep13060812-21ssh1
30538512110,1sleep20-21swapper/22
30388112111,6sleep23038494-21/usr/sbin/munin2
30388112111,6sleep23038494-21/usr/sbin/munin2
30364532111,4sleep12843317-21diskmemload1
30306392111,5sleep23031266-21cut2
30202012111,4sleep02843317-21diskmemload0
30037072111,3sleep10-21swapper/11
29955702111,6sleep12843317-21diskmemload1
29673972111,3sleep20-21swapper/22
29633012111,6sleep12964866-21kthreadcore1
29353942111,3sleep20-21swapper/22
29269632111,7sleep02843317-21diskmemload0
29114642111,5sleep22911486-21apt-get2
29103712111,4sleep12843317-21diskmemload1
28835742111,4sleep22843317-21diskmemload2
28730312111,4sleep12873213-21strings1
28690262111,5sleep02843317-21diskmemload0
27673462111,5sleep12767395-21ntpq1
31968782100,1sleep10-21swapper/11
31901682101,5sleep13190212-21basename1
31901322101,3sleep20-21swapper/22
31719752101,2sleep10-21swapper/11
31707312100,5sleep03170742-21ssh0
31705172101,4sleep22843317-21diskmemload2
31537342101,4sleep02843317-21diskmemload0
31534302100,3sleep23153441-21ssh2
31320062102,4sleep00-21swapper/00
31294002101,4sleep12843317-21diskmemload1
31294002101,4sleep12843317-21diskmemload1
31134752101,3sleep10-21swapper/11
30723272101,4sleep03072359-21ssh0
30607012101,4sleep02843317-21diskmemload0
30543322101,2sleep00-21swapper/00
30204582101,4sleep22843317-21diskmemload2
30135492101,5sleep13013651-21ps1
30018562101,5sleep20-21swapper/22
30012872101,5sleep03001297-21ssh0
29922142101,3sleep22843317-21diskmemload2
29786322101,1sleep10-21swapper/11
29775262101,4sleep02977538-21ssh0
29767972101,3sleep20-21swapper/22
29583122101,4sleep12843317-21diskmemload1
29539762101,4sleep20-21swapper/22
29513422100,1sleep00-21swapper/00
29513422100,1sleep00-21swapper/00
29507992100,2sleep10-21swapper/11
29507992100,2sleep10-21swapper/11
29191792101,4sleep12843317-21diskmemload1
29156982101,4sleep02915765-21ntpq0
28831672101,4sleep02843317-21diskmemload0
28759812101,2sleep20-21swapper/22
28589752101,4sleep22843317-21diskmemload2
28574432101,5sleep02857454-21sh0
28398142101,4sleep02841608-21cut0
28271022100,2sleep10-21swapper/11
28056802101,5sleep12805814-21grep1
27792782101,3sleep12779297-21apt-get1
27728162101,5sleep12772931-21/usr/sbin/munin1
27582212101,4sleep02758431-21hddtemp_smartct0
27351712101,4sleep12735312-21ntpq1
27318132101,5sleep22732399-21kthreadcore2
27289772101,5sleep02729843-21memory0
27132182101,4sleep02713237-21apt-get0
27132182101,4sleep02713237-21apt-get0
3179683290,1sleep10-21swapper/11
3162826291,2sleep20-21swapper/22
3157375291,4sleep13158365-21ssh1
3144965291,2sleep20-21swapper/22
3118648291,4sleep03118705-21/usr/sbin/munin0
3097201291,3sleep23097479-21ssh2
3072627291,4sleep23072686-21fschecks_count2
3070867291,3sleep12843317-21diskmemload1
3027925291,3sleep02843317-21diskmemload0
3010158291,3sleep23010168-21ssh2
3010132291,3sleep02843317-21diskmemload0
2986157290,1sleep00-21swapper/00
2962472291,3sleep02962498-21apt-get0
2925333291,3sleep12925343-21ssh1
2891935291,3sleep02891947-21ssh0
2890535291,3sleep10-21swapper/11
2882572291,3sleep12843317-21diskmemload1
2850455291,3sleep20-21swapper/22
2850455291,3sleep20-21swapper/22
2842613290,6sleep12842872-21unixbench_multi1
2835372291,5sleep12836811-21/usr/sbin/munin1
2817164291,4sleep22817720-21cut2
2810898290,1sleep20-21swapper/22
2799693291,4sleep12799966-21grep1
2779571293,3sleep00-21swapper/00
2778726291,2sleep00-21swapper/00
2763254291,3sleep02763273-21apt-get0
2754021291,5sleep02755801-21kthreadcore0
2747777291,4sleep22747843-21/usr/sbin/munin2
2722135291,4sleep02723925-21cut0
2714035291,4sleep12715722-21grep1
2714035291,4sleep12715722-21grep1
2929889281,3sleep12929150-21ssh1
2925824281,2sleep20-21swapper/22
2837701280,1sleep00-21swapper/00
2827558281,2sleep00-21swapper/00
2805961281,3sleep12806151-21cstates1
2799327281,2sleep20-21swapper/22
2794951280,2sleep10-21swapper/11
2762719280,1sleep10-21swapper/11
2752147280,1sleep00-21swapper/00
2743617281,4sleep12745396-21kthreadcore1
2737849281,3sleep12739593-21cut1
2725361281,2sleep20-21swapper/22
2725295280,1sleep10-21swapper/11
2843162270,1sleep20-21swapper/22
2828127271,3sleep12828162-21kthreadcore1
2827298271,3sleep02827333-21idleruntime-cro0
2822219270,2sleep20-21swapper/22
2812599271,3sleep02814357-21kthreadcore0
2805939270,2sleep00-21swapper/00
2805667270,2sleep00-21swapper/00
2802787271,3sleep20-21swapper/22
2794675270,1sleep00-21swapper/00
2785138270,3sleep12785503-21grep1
2783348271,3sleep22783400-21/usr/sbin/munin2
2778885270,1sleep20-21swapper/22
2751445271,2sleep12751530-21ps1
2730743270,1sleep10-21swapper/11
2725937270,2sleep20-21swapper/22
2829270261,2sleep20-21swapper/22
2816103260,1sleep20-21swapper/22
2815456261,2sleep10-21swapper/11
2800067260,1sleep00-21swapper/00
2789387260,1sleep00-21swapper/00
2773613260,4sleep00-21swapper/00
2768227260,1sleep20-21swapper/22
2757402260,4sleep10-21swapper/11
2745397261,2sleep20-21swapper/22
2741327260,1sleep20-21swapper/22
2736271260,4sleep00-21swapper/00
2788952250,1sleep20-21swapper/22
2758509250,2sleep20-21swapper/22
2718233250,1sleep20-21swapper/22
2718233250,1sleep20-21swapper/22
2833208240,1sleep20-21swapper/22
2736610240,1sleep00-21swapper/00
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional