You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-11 - 01:11

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3s.osadl.org (updated Fri Apr 10, 2026 12:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
166618421420,1sleep172-21kcompactd01
1467037211859,55sleep30-21swapper/33
146688627952,7sleep00-21swapper/00
146692227860,12sleep10-21swapper/11
146218827859,12sleep20-21swapper/22
19580512270,2sleep20-21swapper/22
1467283992623,1cyclictest0-21swapper/33
146728399244,8cyclictest0-21swapper/33
18426152230,2sleep30-21swapper/33
146728399222,16cyclictest17101862chrt3
146728399221,9cyclictest1633806-21turbostat3
1467283992212,6cyclictest16576332chrt3
146728399221,17cyclictest0-21swapper/33
146728399211,17cyclictest0-21swapper/33
146728399202,4cyclictest19673692sleep33
146728399202,4cyclictest19673692sleep33
1467283992011,6cyclictest0-21swapper/33
146728399201,15cyclictest1947887-21taskset3
146728399201,15cyclictest0-21swapper/33
146728399201,14cyclictest18078092sleep33
18081962190,7sleep134-21ksoftirqd/11
146728399194,13cyclictest0-21swapper/33
146728399193,13cyclictest0-21swapper/33
1467283991916,1cyclictest0-21swapper/33
146728399191,15cyclictest0-21swapper/33
146728399191,15cyclictest0-21swapper/33
146728399191,15cyclictest0-21swapper/33
146728399191,15cyclictest0-21swapper/33
146728399191,14cyclictest0-21swapper/33
146728399191,14cyclictest0-21swapper/33
146728399191,14cyclictest0-21swapper/33
146728399191,14cyclictest0-21swapper/33
146728399191,14cyclictest0-21swapper/33
146728399191,14cyclictest0-21swapper/33
146728399191,14cyclictest0-21swapper/33
146728399190,1cyclictest0-21swapper/33
18418222180,6sleep1331ktimers/11
17912402181,2sleep00-21swapper/00
1467283991816,1cyclictest0-21swapper/33
1467283991816,1cyclictest0-21swapper/33
146728399181,14cyclictest0-21swapper/33
146728399181,14cyclictest0-21swapper/33
146728399181,14cyclictest0-21swapper/33
146728399181,14cyclictest0-21swapper/33
146728399181,14cyclictest0-21swapper/33
146728399181,14cyclictest0-21swapper/33
146728399181,13cyclictest0-21swapper/33
146728399181,13cyclictest0-21swapper/33
146728399181,13cyclictest0-21swapper/33
146728399181,13cyclictest0-21swapper/33
146728399181,13cyclictest0-21swapper/33
146728399181,11cyclictest0-21swapper/33
146728399180,15cyclictest0-21swapper/33
19134012171,7sleep01913413-21ssh0
18429572171,12sleep21842971-21ssh2
17977902171,5sleep21598628-21diskmemload2
16308452171,3sleep00-21swapper/00
146728399171,13cyclictest0-21swapper/33
146728399171,13cyclictest0-21swapper/33
146728399171,13cyclictest0-21swapper/33
146728399171,13cyclictest0-21swapper/33
146728399171,13cyclictest0-21swapper/33
146728399171,12cyclictest1598297-21taskset3
146728399170,13cyclictest0-21swapper/33
18525882161,10sleep11598628-21diskmemload1
17977642161,3sleep00-21swapper/00
17817182161,4sleep01781729-21ssh0
146728399168,5cyclictest1517688-21turbostat3
146728399162,9cyclictest66950irq/132-enp1s0-TxRx-33
1467283991613,1cyclictest0-21swapper/33
1467283991611,3cyclictest0-21swapper/33
146728399160,13cyclictest0-21swapper/33
146728399160,13cyclictest0-21swapper/33
19650442151,2sleep00-21swapper/00
19650442151,2sleep00-21swapper/00
18351192150,2sleep00-21swapper/00
18338792150,2sleep20-21swapper/22
17621682151,3sleep11598628-21diskmemload1
17549422151,2sleep00-21swapper/00
17436902151,9sleep21743791-21ntpq2
16975892151,10sleep01699306-21cut0
16517012151,9sleep01651971-21df0
16031802151,10sleep21603241-21collect22
16031802151,10sleep21603241-21collect22
15828112151,4sleep11582830-21apt-get1
15060292151,9sleep21506077-21ntpq2
146728399151,3cyclictest0-21swapper/33
1467283991513,1cyclictest0-21swapper/33
1467283991512,2cyclictest0-21swapper/33
146728399151,11cyclictest0-21swapper/33
146728399150,13cyclictest0-21swapper/33
146728399150,12cyclictest0-21swapper/33
146728399150,12cyclictest0-21swapper/33
146728399150,12cyclictest0-21swapper/33
146728399150,12cyclictest0-21swapper/33
19037282141,7sleep11598628-21diskmemload1
18586262142,8sleep01598628-21diskmemload0
17997582140,3sleep11598628-21diskmemload1
16327582142,6sleep21598628-21diskmemload2
16235182140,10sleep01623533-21ssh0
15715842141,9sleep11571784-21cut1
14948062141,8sleep01494854-21ntpq0
1467283991410,2cyclictest0-21swapper/33
146728399140,11cyclictest0-21swapper/33
146728399140,11cyclictest0-21swapper/33
19478972132,7sleep01947907-21ssh0
19104142131,7sleep11598628-21diskmemload1
18739162131,8sleep21875699-21cut2
18280782130,2sleep10-21swapper/11
18160002131,6sleep20-21swapper/22
18076442131,7sleep21598628-21diskmemload2
17889592130,2sleep10-21swapper/11
17285522131,3sleep11728563-21ssh1
16752622131,8sleep01675277-21ssh0
19486272121,6sleep20-21swapper/22
19409362120,3sleep21940949-21ssh2
19162302120,9sleep10-21swapper/11
19071372121,7sleep21907165-21apt2
19057972120,1sleep20-21swapper/22
18994522121,7sleep01901160-21cut0
18809762121,8sleep11881113-21cut1
18456362121,6sleep01845897-21df0
18266052121,5sleep00-21swapper/00
17732122121,7sleep21773226-21ssh2
17613602121,5sleep21761420-21ntpq2
17452582121,6sleep11598628-21diskmemload1
17209162121,3sleep21720931-21ssh2
16997312121,7sleep21699868-21gcc2
16753852121,5sleep20-21swapper/22
16660452120,8sleep21666060-21ssh2
15395612122,5sleep21539580-21apt-get2
15095022121,7sleep01511131-21latency0
15009372120,7sleep11500354-31find1
14789222121,5sleep21478970-21ntpq2
14788702121,7sleep11474798-21/usr/sbin/munin1
19667442110,2sleep20-21swapper/22
19667442110,2sleep20-21swapper/22
19413752111,4sleep11598628-21diskmemload1
19308672111,5sleep20-21swapper/22
19251672111,5sleep11925283-21irqstats1
18946352111,5sleep11598628-21diskmemload1
18807052111,6sleep01880973-21df_inode0
18787602111,4sleep11598628-21diskmemload1
18688632111,3sleep10-21swapper/11
18624902111,7sleep21862505-21ssh2
18420112111,7sleep01842023-21rm0
18272892111,3sleep20-21swapper/22
18232892111,5sleep11598628-21diskmemload1
17803762111,4sleep21598628-21diskmemload2
17703582111,5sleep11770491-21mailstats1
17614782111,5sleep01761544-21ntp_states0
17554322111,3sleep20-21swapper/22
17535212111,5sleep11716895-21kworker/1:0+events1
17371952111,3sleep00-21swapper/00
17208492111,3sleep10-21swapper/11
16950002112,6sleep01695014-21sh0
16944622111,6sleep11694472-21ssh1
16844732111,7sleep01684487-21ssh0
16839352111,3sleep20-21swapper/22
16692922111,4sleep11669316-21apt-get1
16661722111,4sleep01666183-21ssh0
16593902111,4sleep11598628-21diskmemload1
16485242111,5sleep11598628-21diskmemload1
16484012110,3sleep00-21swapper/00
16388192111,5sleep11639048-21users1
16219442111,3sleep11621957-21ssh1
15932772111,5sleep01593301-21munin-run0
15762072111,6sleep01576261-21ntpq0
15393002111,6sleep21539312-21sh2
15393002111,6sleep21539312-21sh2
15383362111,6sleep01538390-21ntpq0
15383362111,6sleep01538390-21ntpq0
15129042111,5sleep11513113-21hddtemp_smartct1
15003802111,6sleep00-21swapper/00
19666342101,5sleep11598628-21diskmemload1
19666342101,5sleep11598628-21diskmemload1
19591452101,2sleep00-21swapper/00
19511692101,4sleep11598628-21diskmemload1
19241472101,3sleep00-21swapper/00
18865382101,2sleep20-21swapper/22
18796082101,3sleep00-21swapper/00
18631972101,4sleep21598628-21diskmemload2
18594722101,4sleep11598628-21diskmemload1
18523992101,6sleep21598628-21diskmemload2
18167912101,4sleep11598628-21diskmemload1
17837742101,2sleep20-21swapper/22
17828542101,3sleep11598628-21diskmemload1
17381502101,3sleep11598628-21diskmemload1
17300292101,3sleep00-21swapper/00
17261122101,4sleep21726261-21nfs4_client2
17145352101,6sleep01716146-21cut0
17105172101,2sleep10-21swapper/11
17087302101,2sleep20-21swapper/22
16934392101,3sleep21598628-21diskmemload2
16822612101,4sleep11682318-21ntpq1
16474112101,3sleep21598628-21diskmemload2
16394782101,3sleep01598628-21diskmemload0
16230372100,1sleep20-21swapper/22
16152042100,1sleep20-21swapper/22
16139152101,3sleep00-21swapper/00
16053962101,3sleep00-21swapper/00
16053962101,3sleep00-21swapper/00
15989852101,3sleep11599009-21apt-get1
15989852101,3sleep11599009-21apt-get1
15888292101,5sleep11590476-21cut1
15543482101,4sleep11554394-21ntpq1
15542892101,4sleep01554346-21ntpq0
15449372101,4sleep21545193-21df2
15232892101,4sleep11523308-21apt-get1
15117602101,4sleep11511977-21users1
1933339290,2sleep00-21swapper/00
1896380291,2sleep20-21swapper/22
1894254291,4sleep01894279-21ssh0
1817036291,5sleep00-21swapper/00
1772223291,2sleep00-21swapper/00
1741336291,4sleep01743024-21cut0
1734893291,2sleep20-21swapper/22
1711475291,4sleep01711489-21ssh0
1702254291,3sleep11598628-21diskmemload1
1659351290,1sleep20-21swapper/22
1634054290,1sleep20-21swapper/22
1631112291,3sleep11598628-21diskmemload1
1613467291,4sleep11613480-21ssh1
1593924292,4sleep21594093-21if_err_enp1s02
1587213291,4sleep01587466-21head0
1577741291,4sleep01577766-21/usr/sbin/munin0
1576263291,4sleep11576316-21ntp_states1
1570965291,4sleep21571210-21users2
1565263291,4sleep21565317-21ntpq2
1538617291,4sleep11538841-21unixbench_multi1
1538617291,4sleep11538841-21unixbench_multi1
1532996291,4sleep11528613-21/usr/sbin/munin1
1497216291,4sleep21498944-21grep2
1495596291,3sleep272-21kcompactd02
1485479290,4sleep01485498-21apt-get0
1480711291,4sleep01481374-21cut0
1942232281,3sleep11942243-21ssh1
1929681281,3sleep01929695-21ssh0
1922501281,3sleep21922515-21ssh2
1869054281,3sleep01869067-21ssh0
1806756281,3sleep01806769-21ssh0
1598381280,1sleep10-21swapper/11
1598378281,4sleep01598380-21modprobe0
1577999281,3sleep21578392-21cat2
1571787281,3sleep21571592-21apt-get2
1561189280,3sleep11561208-21apt-get1
1555885281,3sleep21555932-21grep2
1544915281,2sleep11544934-21apt-get1
1528685281,3sleep01528904-21diskstats0
1512610281,2sleep21512629-21apt-get2
1506506280,5sleep00-21swapper/00
1484323280,5sleep21484403-21proc_pri2
1468201281,3sleep21469883-21cut2
1593022270,1sleep20-21swapper/22
1587519270,4sleep20-21swapper/22
1582412270,1sleep10-21swapper/11
1561491271,3sleep00-21swapper/00
1560766270,1sleep10-21swapper/11
1539860270,3sleep11540072-21smartctl1
1533655270,1sleep20-21swapper/22
1528235270,1sleep00-21swapper/00
1524483271,2sleep20-21swapper/22
1522998270,4sleep00-21swapper/00
1522005271,2sleep21522054-21ntpq2
1495633270,1sleep10-21swapper/11
1484979270,1sleep10-21swapper/11
1473025270,4sleep00-21swapper/00
1471387270,1sleep00-21swapper/00
1467450270,2sleep10-21swapper/11
1566560261,2sleep00-21swapper/00
1560882260,1sleep00-21swapper/00
1555083260,1sleep20-21swapper/22
1539553261,2sleep00-21swapper/00
1518394260,1sleep10-21swapper/11
1507257260,1sleep20-21swapper/22
1506665260,4sleep10-21swapper/11
1490362260,1sleep10-21swapper/11
1549935251,2sleep00-21swapper/00
1512759250,1sleep00-21swapper/00
1485747240,1sleep20-21swapper/22
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional