You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-16 - 12:53

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack3slot3s.osadl.org (updated Mon Mar 16, 2026 00:43:53)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1443913211853,60sleep30-21swapper/33
144387528656,11sleep10-21swapper/11
144392828060,14sleep00-21swapper/00
144387627659,11sleep20-21swapper/22
15970552470,5sleep2421rcuc/22
144436499271,23cyclictest0-21swapper/33
144436499261,14cyclictest0-21swapper/33
144436499261,14cyclictest0-21swapper/33
144436499235,13cyclictest0-21swapper/33
144436499234,15cyclictest0-21swapper/33
18033572221,3sleep00-21swapper/00
144436499226,14cyclictest0-21swapper/33
144436499224,14cyclictest0-21swapper/33
144436499223,14cyclictest1610457-21turbostat3
144436499221,16cyclictest0-21swapper/33
144436499220,2cyclictest0-21swapper/33
144436499220,11cyclictest0-21swapper/33
17411462211,3sleep20-21swapper/22
144436499214,13cyclictest0-21swapper/33
144436499213,14cyclictest0-21swapper/33
144436499211,4cyclictest0-21swapper/33
144436499211,16cyclictest1793428-21taskset3
144436499211,16cyclictest0-21swapper/33
144436499211,16cyclictest0-21swapper/33
144436499210,3cyclictest0-21swapper/33
144436499210,3cyclictest0-21swapper/33
144436499210,18cyclictest0-21swapper/33
144436499210,11cyclictest0-21swapper/33
19229262201,5sleep20-21swapper/22
19229262201,5sleep20-21swapper/22
17863232201,6sleep01786348-21apt-get0
144436499208,2cyclictest0-21swapper/33
144436499202,13cyclictest1575358-21turbostat3
144436499201,3cyclictest0-21swapper/33
1444364992013,4cyclictest0-21swapper/33
144436499201,16cyclictest0-21swapper/33
144436499201,15cyclictest1819836-21chrt3
144436499201,15cyclictest0-21swapper/33
144436499201,14cyclictest0-21swapper/33
18980262191,3sleep10-21swapper/11
144436499199,8cyclictest0-21swapper/33
144436499193,14cyclictest0-21swapper/33
144436499191,7cyclictest1516265-21turbostat3
144436499191,5cyclictest66950irq/132-enp1s0-TxRx-33
144436499191,5cyclictest0-21swapper/33
144436499191,4cyclictest0-21swapper/33
1444364991913,3cyclictest0-21swapper/33
144436499191,15cyclictest0-21swapper/33
144436499191,15cyclictest0-21swapper/33
144436499191,15cyclictest0-21swapper/33
144436499191,14cyclictest0-21swapper/33
144436499191,14cyclictest0-21swapper/33
144436499191,14cyclictest0-21swapper/33
144436499191,14cyclictest0-21swapper/33
144436499191,14cyclictest0-21swapper/33
144436499191,14cyclictest0-21swapper/33
144436499190,15cyclictest0-21swapper/33
19424852181,2sleep00-21swapper/00
17071792181,5sleep00-21swapper/00
16954142181,3sleep10-21swapper/11
16616612181,3sleep00-21swapper/00
144436499188,7cyclictest1634630-21taskset3
144436499187,2cyclictest0-21swapper/33
1444364991811,5cyclictest0-21swapper/33
144436499181,15cyclictest0-21swapper/33
144436499181,14cyclictest0-21swapper/33
144436499181,14cyclictest0-21swapper/33
144436499181,14cyclictest0-21swapper/33
144436499181,14cyclictest0-21swapper/33
144436499181,14cyclictest0-21swapper/33
144436499181,13cyclictest0-21swapper/33
18729032171,3sleep00-21swapper/00
18269302171,2sleep20-21swapper/22
17329232170,2sleep00-21swapper/00
17329232170,2sleep00-21swapper/00
144436499178,6cyclictest1532576-21turbostat3
144436499171,13cyclictest0-21swapper/33
144436499170,5cyclictest0-21swapper/33
144436499170,1cyclictest0-21swapper/33
144436499170,1cyclictest0-21swapper/33
18561912160,1sleep00-21swapper/00
1444364991613,1cyclictest0-21swapper/33
144436499161,12cyclictest0-21swapper/33
144436499160,6cyclictest0-21swapper/33
144436499160,13cyclictest0-21swapper/33
144436499160,13cyclictest0-21swapper/33
16815832154,7sleep11682264-21tr1
16278692151,4sleep11575354-21diskmemload1
15369812150,7sleep21532822-21/usr/sbin/munin2
144436499150,13cyclictest0-21swapper/33
144436499150,12cyclictest0-21swapper/33
19352812141,7sleep21935313-21ssh2
17758262140,2sleep20-21swapper/22
17431582141,4sleep01575354-21diskmemload0
17061422141,9sleep2802-21snmpd2
144436499143,7cyclictest0-21swapper/33
144436499140,11cyclictest15002032sleep33
19169492131,7sleep11575354-21diskmemload1
19169492131,7sleep11575354-21diskmemload1
18789332131,3sleep01575354-21diskmemload0
18703092131,7sleep11575354-21diskmemload1
18644752131,3sleep11864486-21ssh1
18536662130,1sleep00-21swapper/00
18531242131,6sleep11575354-21diskmemload1
18476032131,7sleep20-21swapper/22
18044622131,7sleep21805333-21tr2
17939162131,8sleep11575354-21diskmemload1
17668462137,3sleep01766860-21ssh0
17236442131,6sleep11575354-21diskmemload1
17000202131,8sleep11701634-21cut1
16695102131,8sleep01669524-21ssh0
16600482131,5sleep20-21swapper/22
16510982131,7sleep11575354-21diskmemload1
15826312131,8sleep21575354-21diskmemload2
14616192131,6sleep21461699-21ntpq2
19425562121,7sleep11942567-21ssh1
18629722122,6sleep2802-21snmpd2
18460352121,2sleep00-21swapper/00
18019082121,6sleep11575354-21diskmemload1
17742262122,5sleep11575354-21diskmemload1
17651532121,6sleep21575354-21diskmemload2
17571292121,7sleep01757140-21ssh0
17228162121,7sleep01722828-21ssh0
16771722121,7sleep01677186-21ssh0
16688302121,5sleep20-21swapper/22
16535812121,3sleep20-21swapper/22
16289572121,7sleep01630359-21cut0
15830182121,6sleep11575354-21diskmemload1
19222872110,2sleep00-21swapper/00
19222872110,2sleep00-21swapper/00
19199332116,2sleep1321rcuc/11
19199332116,2sleep1321rcuc/11
18727232111,6sleep21872735-21ssh2
18469112111,5sleep11575354-21diskmemload1
18378452111,6sleep01837853-21sh0
18342792111,5sleep21834349-21ntpq2
18254922110,5sleep01825550-21ntpq0
18130842111,7sleep01813168-21/usr/sbin/munin0
18091172111,3sleep00-21swapper/00
17908502111,6sleep21790932-21ssh2
17819952111,4sleep21575354-21diskmemload2
17660452111,4sleep11575354-21diskmemload1
17439212111,6sleep11745640-21kthreadcore1
17318762112,5sleep21731889-21ssh2
17318762112,5sleep21731889-21ssh2
17234542111,3sleep20-21swapper/22
17129202111,3sleep20-21swapper/22
16963172111,6sleep01696331-21ssh0
16864732111,6sleep01686483-21ssh0
16763812111,5sleep21676442-21ntpq2
16363342111,6sleep11575354-21diskmemload1
16107682111,4sleep21575354-21diskmemload2
16083632111,2sleep00-21swapper/00
16010292111,3sleep00-21swapper/00
16009492110,4sleep10-21swapper/11
15913842111,6sleep11575354-21diskmemload1
15275212111,5sleep11527773-21df1
15099512110,3sleep00-21swapper/00
15008332111,5sleep01501045-21hddtemp_smartct0
14942592112,5sleep01494455-21unixbench-2d0
14446562111,4sleep21444675-21apt-get2
19323882101,3sleep10-21swapper/11
19284162101,5sleep01930140-21tr0
19134532101,4sleep01913684-21users0
19134532101,4sleep01913684-21users0
19044942101,2sleep20-21swapper/22
18999422102,4sleep1802-21snmpd1
18976432101,3sleep21575354-21diskmemload2
18903682101,2sleep00-21swapper/00
18889542101,2sleep10-21swapper/11
18879952100,2sleep20-21swapper/22
18802272101,3sleep20-21swapper/22
18797972101,4sleep11575354-21diskmemload1
18282452101,3sleep11575354-21diskmemload1
18207392101,2sleep20-21swapper/22
18125702101,5sleep11812650-21tr1
18116072101,3sleep11575354-21diskmemload1
18010942101,5sleep20-21swapper/22
17834882101,3sleep01575354-21diskmemload0
17774672100,3sleep10-21swapper/11
17587292101,3sleep20-21swapper/22
17512232101,4sleep11575354-21diskmemload1
17508962101,4sleep21750899-21ssh2
17415972101,3sleep11575354-21diskmemload1
17135292101,3sleep11575354-21diskmemload1
17128932101,2sleep00-21swapper/00
16956772101,3sleep20-21swapper/22
16880732102,5sleep21444351-21cyclictest2
16804752101,4sleep11575354-21diskmemload1
16601862101,4sleep11575354-21diskmemload1
16535552101,3sleep00-21swapper/00
16437732101,3sleep20-21swapper/22
16427062101,4sleep11575354-21diskmemload1
16354322101,3sleep20-21swapper/22
16195802101,3sleep20-21swapper/22
16162122101,3sleep00-21swapper/00
16161732101,3sleep11575354-21diskmemload1
16076712101,2sleep10-21swapper/11
15907362100,3sleep20-21swapper/22
15850202101,4sleep01585092-21irqstats0
15737662101,5sleep11574285-21strings1
15537702101,4sleep11553789-21apt-get1
15537702101,4sleep11553789-21apt-get1
15104002101,5sleep21510564-21vmstat2
14795342101,4sleep21479744-21hddtemp_smartct2
14780802100,1sleep00-21swapper/00
14571032101,4sleep11457114-21apt-get1
1927062290,1sleep20-21swapper/22
1915329291,3sleep21575354-21diskmemload2
1915329291,3sleep21575354-21diskmemload2
1907001291,4sleep01907015-21ssh0
1896493291,4sleep01896507-21ssh0
1844948291,2sleep20-21swapper/22
1836087291,3sleep11575354-21diskmemload1
1775544291,3sleep00-21swapper/00
1738365291,2sleep00-21swapper/00
1724697291,4sleep11724953-21ssh1
1724697291,4sleep11724953-21ssh1
1667675291,3sleep11575354-21diskmemload1
1638960291,4sleep01640701-21grep0
1583402291,2sleep00-21swapper/00
1571949291,4sleep01573765-21cut0
1565454294,2sleep20-21swapper/22
1553263291,4sleep21553514-21tail2
1553263291,4sleep21553514-21tail2
1547987291,4sleep11548232-21users1
1538163290,3sleep10-21swapper/11
1527206290,2sleep20-21swapper/22
1521262291,4sleep01521507-21users0
1516237291,1sleep00-21swapper/00
1510237290,1sleep10-21swapper/11
1501130291,4sleep21502636-21ps2
1500540291,3sleep11500559-21apt-get1
1490432291,4sleep21490962-21cut2
1488647291,4sleep01488694-21ntpq0
1488465291,4sleep21488598-21netstat2
1468593291,2sleep20-21swapper/22
1463223290,4sleep11462727-21apt-get1
1619422281,3sleep01619436-21ssh0
1608485280,2sleep20-21swapper/22
1569125280,5sleep11569266-21mailstats1
1559343280,4sleep21559358-21munin-run2
1558720281,3sleep11558967-21/usr/sbin/munin1
1548870281,2sleep00-21swapper/00
1548870281,2sleep00-21swapper/00
1543977281,3sleep21544093-21irqstats2
1539066281,3sleep21540783-21grep2
1533888281,4sleep01535618-21grep0
1533099281,3sleep11533166-21fschecks_count1
1527499280,2sleep00-21swapper/00
1522169280,2sleep10-21swapper/11
1517354281,3sleep20-21swapper/22
1499246281,3sleep11499293-21ntpq1
1484313281,3sleep01484480-21latency_hist0
1483618280,4sleep10-21swapper/11
1461810281,3sleep11461892-21pmu-power1
1456000281,3sleep01456088-21ntpq0
1575054270,1sleep20-21swapper/22
1569303271,2sleep00-21swapper/00
1564351270,1sleep00-21swapper/00
1564320270,1sleep20-21swapper/22
1556085271,3sleep01557894-21tr0
1545193271,3sleep01547015-21kthreadcore0
1542962270,1sleep00-21swapper/00
1527068270,1sleep00-21swapper/00
1521606270,1sleep10-21swapper/11
1500017270,1sleep00-21swapper/00
1495234270,2sleep20-21swapper/22
1494700270,1sleep10-21swapper/11
1473896270,2sleep10-21swapper/11
1472943271,2sleep00-21swapper/00
1468059270,1sleep20-21swapper/22
1459633271,2sleep00-21swapper/00
1453615271,2sleep20-21swapper/22
1559592260,4sleep10-21swapper/11
1532322260,1sleep20-21swapper/22
1516016260,1sleep10-21swapper/11
1511187260,1sleep20-21swapper/22
1484481260,2sleep10-21swapper/11
1474499261,2sleep21476027-21kthreadcore2
1473565260,1sleep10-21swapper/11
1464412261,2sleep00-21swapper/00
1449529260,1sleep10-21swapper/11
1449493260,1sleep00-21swapper/00
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional