You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-17 - 03:01

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack3slot3s.osadl.org (updated Thu Sep 17, 2026 00:43:53)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
163764621480,2sleep01637642-21apt-get0
1524596213055,70sleep30-21swapper/33
152470428455,11sleep20-21swapper/22
152460428454,12sleep10-21swapper/11
152457928357,14sleep00-21swapper/00
152509599239,10cyclictest1753176-21turbostat3
152509599237,12cyclictest0-21swapper/33
152509599237,12cyclictest0-21swapper/33
17917022221,17sleep11653170-21diskmemload1
152509599221,16cyclictest66950irq/132-enp1s0-TxRx-33
152509599221,16cyclictest0-21swapper/33
152509599220,19cyclictest0-21swapper/33
152509599202,13cyclictest1998032-21chrt3
1525095992017,1cyclictest0-21swapper/33
1525095992012,7cyclictest0-21swapper/33
152509599201,14cyclictest66950irq/132-enp1s0-TxRx-33
18798162192,6sleep01653170-21diskmemload0
1525095991916,1cyclictest0-21swapper/33
152509599191,15cyclictest0-21swapper/33
152509599191,15cyclictest0-21swapper/33
152509599191,14cyclictest0-21swapper/33
152509599191,14cyclictest0-21swapper/33
152509599191,14cyclictest0-21swapper/33
152509599191,14cyclictest0-21swapper/33
152509599191,14cyclictest0-21swapper/33
152509599191,14cyclictest0-21swapper/33
152509599191,14cyclictest0-21swapper/33
152509599191,13cyclictest0-21swapper/33
19094012181,4sleep21653170-21diskmemload2
17429422181,5sleep01653170-21diskmemload0
17429422181,5sleep01653170-21diskmemload0
15308202181,11sleep21532064-21users2
152509599187,8cyclictest1928537-21turbostat3
1525095991815,1cyclictest0-21swapper/33
152509599181,14cyclictest66950irq/132-enp1s0-TxRx-33
152509599181,14cyclictest66950irq/132-enp1s0-TxRx-33
152509599181,14cyclictest0-21swapper/33
152509599181,14cyclictest0-21swapper/33
152509599181,14cyclictest0-21swapper/33
152509599181,14cyclictest0-21swapper/33
152509599181,14cyclictest0-21swapper/33
152509599181,14cyclictest0-21swapper/33
152509599181,14cyclictest0-21swapper/33
152509599181,14cyclictest0-21swapper/33
152509599181,14cyclictest0-21swapper/33
152509599181,14cyclictest0-21swapper/33
152509599181,13cyclictest0-21swapper/33
152509599181,13cyclictest0-21swapper/33
152509599181,13cyclictest0-21swapper/33
152509599181,13cyclictest0-21swapper/33
152509599180,14cyclictest66950irq/132-enp1s0-TxRx-33
152509599180,14cyclictest0-21swapper/33
19605622171,11sleep11960714-21grep1
18028442170,3sleep21802858-21ssh2
17251952170,2sleep10-21swapper/11
16617542171,11sleep21661785-21ssh2
1525095991715,1cyclictest0-21swapper/33
152509599171,13cyclictest0-21swapper/33
152509599171,13cyclictest0-21swapper/33
152509599171,13cyclictest0-21swapper/33
152509599171,13cyclictest0-21swapper/33
1525095991710,4cyclictest0-21swapper/33
152509599170,14cyclictest0-21swapper/33
152509599170,13cyclictest0-21swapper/33
152509599170,13cyclictest0-21swapper/33
188712421610,2sleep10-21swapper/11
18678012162,2sleep2171rcu_preempt2
18611632160,2sleep00-21swapper/00
18354452161,3sleep10-21swapper/11
17287522161,8sleep21728972-21hddtemp_smartct2
17193652161,2sleep00-21swapper/00
16868252161,9sleep01653170-21diskmemload0
15646042161,5sleep11564699-21cut1
152509599164,7cyclictest0-21swapper/33
152509599164,7cyclictest0-21swapper/33
152509599162,9cyclictest1537782-21taskset3
1525095991612,2cyclictest0-21swapper/33
152509599160,12cyclictest0-21swapper/33
19534232151,3sleep01953436-21sh0
19281872150,3sleep00-21swapper/00
18931182150,3sleep20-21swapper/22
18482892151,6sleep21845499-21/usr/sbin/munin2
17098552150,1sleep20-21swapper/22
17028622152,8sleep01653170-21diskmemload0
16952012153,7sleep11695199-21apt-get1
15325432151,8sleep01532540-21apt-get0
15287542151,4sleep21529263-21grep2
152509599153,8cyclictest0-21swapper/33
152509599151,2cyclictest0-21swapper/33
152509599151,2cyclictest0-21swapper/33
152509599150,13cyclictest0-21swapper/33
152509599150,12cyclictest0-21swapper/33
152509599150,12cyclictest0-21swapper/33
152509599150,12cyclictest0-21swapper/33
152509599150,12cyclictest0-21swapper/33
19729332141,8sleep01972707-21sendmail-msp0
18791442140,2sleep2421rcuc/22
18427022141,7sleep20-21swapper/22
18178942141,4sleep11817908-21ssh1
18172842140,1sleep00-21swapper/00
17856552140,2sleep10-21swapper/11
17601152141,9sleep21760125-21ssh2
17277082141,2sleep00-21swapper/00
16874442141,5sleep11687675-21kthreadcore1
16469842141,8sleep11642671-21sendmail-msp1
152509599141,3cyclictest0-21swapper/33
1525095991410,2cyclictest0-21swapper/33
152509599140,12cyclictest0-21swapper/33
152509599140,11cyclictest0-21swapper/33
19897902130,2sleep1171rcu_preempt1
19870862130,8sleep11653170-21diskmemload1
19652642131,7sleep21965867-21cut2
19257212130,2sleep20-21swapper/22
18763512130,2sleep20-21swapper/22
18722802131,8sleep00-21swapper/00
18344932130,8sleep01653170-21diskmemload0
18089382131,8sleep11808950-21rm1
18016222131,8sleep11801633-21ssh1
17934362131,8sleep01653170-21diskmemload0
17772832131,3sleep01777296-21ssh0
17583862131,7sleep00-21swapper/00
17458302131,8sleep21747106-21cut2
17340632131,6sleep01653170-21diskmemload0
17193402131,3sleep11719350-21ssh1
17084082131,8sleep01703104-21sendmail-msp0
15950732130,1sleep10-21swapper/11
15792942130,1sleep10-21swapper/11
15792942130,1sleep10-21swapper/11
152509599134,5cyclictest0-21swapper/33
19188732120,2sleep10-21swapper/11
18871642121,5sleep01887195-21ssh0
18781182121,7sleep11878129-21ssh1
18087422120,2sleep0171rcu_preempt0
17827482126,2sleep20-21swapper/22
17691542126,2sleep00-21swapper/00
17607032121,6sleep11653170-21diskmemload1
17019662121,8sleep21701979-21ssh2
16265122121,6sleep01621625-21sendmail-msp0
15798212123,5sleep1483560-21systemd-journal1
15302532121,7sleep01530299-21systemd-run0
152509599120,1cyclictest0-21swapper/33
19969142110,4sleep00-21swapper/00
19885642111,2sleep20-21swapper/22
19730672111,5sleep21972707-21sendmail-msp2
19543432111,4sleep11653170-21diskmemload1
19469002111,3sleep00-21swapper/00
19462312111,3sleep10-21swapper/11
19338722111,3sleep10-21swapper/11
19297142111,2sleep00-21swapper/00
18591952111,5sleep21653170-21diskmemload2
18341502111,6sleep21834162-21ssh2
18269182111,4sleep01653170-21diskmemload0
18253812111,3sleep20-21swapper/22
17696102111,3sleep20-21swapper/22
17686622110,2sleep10-21swapper/11
17531462111,5sleep11753214-21munin-run1
17439222111,4sleep21653170-21diskmemload2
17439222111,4sleep21653170-21diskmemload2
17080522111,4sleep11703104-21sendmail-msp1
16922482111,3sleep20-21swapper/22
16794612111,6sleep01681015-21kthreadcore0
16763952111,2sleep0106713-21snmpd0
16587982111,2sleep10-21swapper/11
16484872111,6sleep01648199-21/usr/sbin/munin0
16420192111,4sleep21642225-21users2
16271712111,5sleep21627429-21df_inode2
16113832111,5sleep01611642-21df_inode0
15844892110,1sleep00-21swapper/00
15772872111,5sleep01578450-21memory0
15772872111,5sleep01578450-21memory0
15377602112,3sleep287-1kworker/2:1H-kblockd2
20056682101,4sleep11653170-21diskmemload1
19971852101,2sleep20-21swapper/22
19865972101,4sleep01653170-21diskmemload0
19785792101,2sleep10-21swapper/11
19725652101,2sleep10-21swapper/11
19646072101,5sleep01964617-21ssh0
19607152104,3sleep21960955-21timerwakeupswit2
19569722104,2sleep00-21swapper/00
19482052101,4sleep21948419-21diskstats2
19288942101,3sleep20-21swapper/22
19205802101,3sleep10-21swapper/11
19182882101,3sleep00-21swapper/00
19168732101,4sleep21917075-21users2
19115162100,3sleep10-21swapper/11
19009472100,2sleep21900958-21ssh2
18955922101,4sleep11895588-21apt-get1
18790782101,5sleep11879142-21fschecks_count1
18674452101,4sleep01653170-21diskmemload0
18624422101,4sleep11862665-21hddtemp_smartct1
18616992101,4sleep11653170-21diskmemload1
18520512101,4sleep11653170-21diskmemload1
18454012101,4sleep01653170-21diskmemload0
18447252101,2sleep00-21swapper/00
18000362100,2sleep00-21swapper/00
17929342101,4sleep21653170-21diskmemload2
17850372100,1sleep00-21swapper/00
17771492101,2sleep10-21swapper/11
17752792101,3sleep20-21swapper/22
17529392101,4sleep01653170-21diskmemload0
17440012101,4sleep11653170-21diskmemload1
17440012101,4sleep11653170-21diskmemload1
17286482101,3sleep11653170-21diskmemload1
16860122100,1sleep20-21swapper/22
16846912101,4sleep11653170-21diskmemload1
16675212101,4sleep01653170-21diskmemload0
16607942101,2sleep20-21swapper/22
16486512101,4sleep11648767-21irqstats1
16168862101,4sleep01617095-21hddtemp_smartct0
16165952101,4sleep11616614-21apt-get1
16054912101,5sleep10-21swapper/11
15693332101,4sleep01569352-21apt-get0
15690622101,4sleep01569118-21munin-run0
15593232101,4sleep01559392-21irqstats0
15471322101,5sleep11547248-21gzip1
15294262100,2sleep10-21swapper/11
2004988291,4sleep02004998-21ssh0
2002882291,2sleep20-21swapper/22
1944543291,2sleep20-21swapper/22
1909274291,4sleep01909286-21ssh0
1900643291,3sleep01653170-21diskmemload0
1838144291,4sleep10-21swapper/11
1820732291,4sleep11820878-21head1
1813377291,4sleep21814978-21cut2
1810627290,1sleep20-21swapper/22
1725011291,4sleep21725025-21ssh2
1716644291,2sleep20-21swapper/22
1678049291,2sleep20-21swapper/22
1668865290,1sleep10-21swapper/11
1653470291,2sleep00-21swapper/00
1643380291,4sleep01643471-21irqstats0
1632111291,4sleep01632331-21wc0
1606133291,2sleep00-21swapper/00
1596187291,5sleep21597745-21kthreadcore2
1595583291,3sleep11595602-21apt-get1
1584486291,3sleep21579601-21sendmail-msp2
1563828291,4sleep11563884-21munin-run1
1530304291,5sleep10-21swapper/11
1676123281,3sleep10-21swapper/11
1647558281,3sleep2482473-21in:imuxsock2
1636468281,3sleep11636622-21systemctl1
1611359280,2sleep20-21swapper/22
1607475281,4sleep11609175-21kthreadcore1
1600840281,3sleep21600895-21cpuspeed_turbos2
1586685282,2sleep11587222-21kthreadcore1
1586685282,2sleep11587222-21kthreadcore1
1574590280,2sleep20-21swapper/22
1574590280,2sleep20-21swapper/22
1563462281,3sleep21558597-21sendmail-msp2
1558055280,1sleep20-21swapper/22
1553122280,1sleep20-21swapper/22
1548318281,3sleep01548320-21apt-get0
1543107281,3sleep01543099-21apt-get0
1636623270,3sleep21636851-21grep2
1627169271,3sleep11627166-21apt-get1
1626877271,2sleep11627076-21latency_hist1
1617168272,2sleep20-21swapper/22
1616011270,1sleep10-21swapper/11
1610685270,1sleep20-21swapper/22
1605023271,2sleep00-21swapper/00
1595878271,2sleep01596091-21smartctl0
1595079270,1sleep20-21swapper/22
1590940271,3sleep01592438-21cut0
1589948270,1sleep00-21swapper/00
1589948270,1sleep00-21swapper/00
1574134270,1sleep10-21swapper/11
1568937270,1sleep20-21swapper/22
1558383270,1sleep00-21swapper/00
1552981270,1sleep10-21swapper/11
1542413271,2sleep10-21swapper/11
1541909271,3sleep00-21swapper/00
1648768261,2sleep20-21swapper/22
1637900260,3sleep10-21swapper/11
1637337260,1sleep00-21swapper/00
1623649260,2sleep21623967-21kthreadcore2
1585412261,2sleep20-21swapper/22
1585412261,2sleep20-21swapper/22
1574035260,1sleep20-21swapper/22
1557834260,3sleep10-21swapper/11
1543073260,1sleep20-21swapper/22
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional