You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-13 - 06:53

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack3slot3s.osadl.org (updated Mon Apr 13, 2026 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1174982212260,58sleep30-21swapper/33
117508928770,11sleep20-21swapper/22
117501128159,15sleep10-21swapper/11
117505427757,14sleep00-21swapper/00
13903902330,2sleep30-21swapper/33
14551022240,4sleep20-21swapper/22
1175503992316,3cyclictest1268815-21chrt3
117550399230,20cyclictest0-21swapper/33
117550399230,17cyclictest15490432sleep33
1175503992219,1cyclictest0-21swapper/33
1175503992211,1cyclictest0-21swapper/33
1175503992117,2cyclictest0-21swapper/33
117550399211,3cyclictest0-21swapper/33
1175503992110,7cyclictest1515099-21taskset3
117550399201,3cyclictest0-21swapper/33
13300202192,8sleep0171rcu_preempt0
1175503991917,1cyclictest0-21swapper/33
1175503991916,1cyclictest0-21swapper/33
1175503991916,1cyclictest0-21swapper/33
117550399191,16cyclictest0-21swapper/33
117550399191,15cyclictest0-21swapper/33
117550399191,15cyclictest0-21swapper/33
117550399191,14cyclictest15250042sleep33
117550399191,14cyclictest0-21swapper/33
117550399191,14cyclictest0-21swapper/33
117550399191,14cyclictest0-21swapper/33
117550399191,14cyclictest0-21swapper/33
117550399191,14cyclictest0-21swapper/33
117550399191,14cyclictest0-21swapper/33
117550399191,14cyclictest0-21swapper/33
117550399191,14cyclictest0-21swapper/33
117550399191,14cyclictest0-21swapper/33
117550399190,16cyclictest0-21swapper/33
13385592181,3sleep10-21swapper/11
1175503991815,1cyclictest0-21swapper/33
1175503991815,1cyclictest0-21swapper/33
117550399181,4cyclictest0-21swapper/33
117550399181,14cyclictest0-21swapper/33
117550399181,14cyclictest0-21swapper/33
117550399181,14cyclictest0-21swapper/33
117550399181,14cyclictest0-21swapper/33
117550399181,14cyclictest0-21swapper/33
117550399181,14cyclictest0-21swapper/33
117550399181,14cyclictest0-21swapper/33
117550399180,15cyclictest0-21swapper/33
117550399180,15cyclictest0-21swapper/33
16712792170,4sleep11305944-21diskmemload1
15688982171,3sleep00-21swapper/00
15609662171,3sleep20-21swapper/22
14737192171,3sleep00-21swapper/00
117550399177,7cyclictest1436680-21taskset3
117550399171,4cyclictest0-21swapper/33
1175503991714,1cyclictest0-21swapper/33
117550399171,3cyclictest1193398-21taskset3
1175503991712,3cyclictest0-21swapper/33
1175503991712,2cyclictest0-21swapper/33
1175503991711,3cyclictest66950irq/132-enp1s0-TxRx-33
117550399171,13cyclictest0-21swapper/33
117550399171,13cyclictest0-21swapper/33
117550399170,14cyclictest0-21swapper/33
117550399170,14cyclictest0-21swapper/33
16372512161,9sleep00-21swapper/00
16198162160,1sleep10-21swapper/11
16098942161,5sleep21609954-21ntpq2
15002842161,2sleep00-21swapper/00
14550302161,4sleep11305944-21diskmemload1
14285122161,3sleep20-21swapper/22
13999582161,9sleep00-21swapper/00
13820142161,8sleep00-21swapper/00
12745612161,5sleep21274664-21perl2
117550399165,8cyclictest0-21swapper/33
1175503991614,1cyclictest0-21swapper/33
1175503991613,1cyclictest0-21swapper/33
1175503991613,1cyclictest0-21swapper/33
117550399160,13cyclictest0-21swapper/33
117550399160,13cyclictest0-21swapper/33
16463102151,2sleep20-21swapper/22
15927662151,4sleep11305944-21diskmemload1
14884502151,4sleep11305944-21diskmemload1
14727732152,7sleep21305944-21diskmemload2
14628562151,9sleep11305944-21diskmemload1
14176172151,6sleep1802-21snmpd1
13561422151,10sleep11305944-21diskmemload1
12162102156,5sleep21217933-21cut2
1175503991511,2cyclictest0-21swapper/33
117550399150,13cyclictest0-21swapper/33
117550399150,13cyclictest0-21swapper/33
117550399150,12cyclictest0-21swapper/33
117550399150,12cyclictest0-21swapper/33
16222172141,8sleep21305944-21diskmemload2
15541282141,9sleep11305944-21diskmemload1
14779092141,8sleep21477959-21ntpq2
14095622141,4sleep21409573-21ssh2
13768032141,7sleep11376945-21diskstats1
13739842140,1sleep00-21swapper/00
13650642141,3sleep01365074-21ssh0
13063112143,5sleep21306339-21apt-get2
1175503991411,2cyclictest0-21swapper/33
117550399140,2cyclictest0-21swapper/33
117550399140,12cyclictest0-21swapper/33
117550399140,12cyclictest0-21swapper/33
16743052131,3sleep21674316-21ssh2
15684702130,5sleep20-21swapper/22
15426022131,7sleep11305944-21diskmemload1
15326332131,3sleep21532646-21ssh2
14348012131,8sleep00-21swapper/00
13990722131,6sleep20-21swapper/22
13933312131,8sleep21393342-21ssh2
13930402131,8sleep01393054-21ssh0
12630342130,1sleep10-21swapper/11
11978832131,8sleep21197924-21ntpq2
117550399130,1cyclictest0-21swapper/33
16642922120,2sleep10-21swapper/11
16582722122,5sleep01658302-21apt-get0
15807792121,7sleep21582416-21tr2
15749512121,5sleep21575047-21ntpq2
15514752121,7sleep11305944-21diskmemload1
14994392121,4sleep01499452-21ssh0
14694722120,5sleep00-21swapper/00
14281412121,6sleep11305944-21diskmemload1
13488542120,3sleep01348868-21ssh0
13209702121,6sleep01305944-21diskmemload0
13115892121,7sleep11311600-21ssh1
12689302121,5sleep20-21swapper/22
11758442121,5sleep21175863-21apt-get2
16641982112,2sleep21664209-21ssh2
16578802111,6sleep01658020-21date0
16571772112,4sleep11305944-21diskmemload1
16490872111,6sleep11649241-21latency_hist1
16325432111,7sleep21633091-21cat2
16023092111,4sleep01305944-21diskmemload0
15958392110,1sleep00-21swapper/00
15955712111,4sleep20-21swapper/22
15761702111,7sleep11576181-21ssh1
15361842111,7sleep00-21swapper/00
15338822111,4sleep11305944-21diskmemload1
15327392110,7sleep01532754-21ssh0
15252762111,4sleep11305944-21diskmemload1
15232882110,5sleep00-21swapper/00
15173072111,4sleep11305944-21diskmemload1
15081342111,4sleep11305944-21diskmemload1
14893832111,3sleep20-21swapper/22
14878752111,5sleep00-21swapper/00
14789472111,7sleep10-21swapper/11
14386992111,3sleep20-21swapper/22
14365432110,6sleep11305944-21diskmemload1
14191822111,5sleep20-21swapper/22
14071462111,6sleep11305944-21diskmemload1
14003262111,4sleep11305944-21diskmemload1
13910742111,4sleep11305944-21diskmemload1
13843872111,4sleep20-21swapper/22
13633052111,4sleep11363365-21ntpq1
13568142111,4sleep21305944-21diskmemload2
13400322111,6sleep21340046-21ssh2
13213952111,5sleep11305944-21diskmemload1
12899692111,5sleep21289989-21munin-run2
12315972111,5sleep01231713-21irqstats0
12149242111,6sleep01215098-21grep0
11897982111,6sleep01191542-21tr0
16551012101,3sleep20-21swapper/22
16463882101,2sleep00-21swapper/00
16306182101,2sleep00-21swapper/00
16296042101,4sleep11305944-21diskmemload1
16286942101,4sleep21305944-21diskmemload2
16123182101,3sleep11305944-21diskmemload1
16110922101,3sleep00-21swapper/00
16032372101,2sleep20-21swapper/22
16030222101,4sleep11305944-21diskmemload1
15751072101,4sleep01575163-21ntpq0
15579112101,3sleep00-21swapper/00
15449012101,4sleep21545035-21diskstats2
15153432101,3sleep21305944-21diskmemload2
15133812101,4sleep01513449-21ntpq0
15054552100,1sleep20-21swapper/22
14932472101,6sleep11494976-21grep1
14714532101,4sleep11305944-21diskmemload1
14604212101,5sleep01460482-21ntpq0
14592722101,4sleep21460263-21memory2
14537462100,1sleep00-21swapper/00
14451242101,3sleep00-21swapper/00
14258912101,5sleep01348660-21kworker/u16:0+flush-8:00
14185212101,5sleep01418534-21ssh0
14028982101,4sleep01403140-21df_inode0
13733422101,4sleep11305944-21diskmemload1
13655442101,6sleep21305944-21diskmemload2
13546002101,5sleep01354667-21ssh0
13448332101,4sleep11305944-21diskmemload1
13389442101,3sleep00-21swapper/00
13226692101,3sleep20-21swapper/22
13105632101,4sleep01310627-21ntpq0
13009072101,4sleep11301160-21df1
12996832101,4sleep21299740-21ntp_states2
12946192101,5sleep01294812-21grep0
12854742101,5sleep11286638-21tr1
12205982101,4sleep11220801-21diskstats1
12032572101,5sleep21203303-21ntpq2
1672271291,3sleep00-21swapper/00
1636663290,1sleep10-21swapper/11
1614345290,2sleep00-21swapper/00
1584883291,3sleep11305944-21diskmemload1
1579431291,4sleep01579445-21ssh0
1568867291,3sleep11305944-21diskmemload1
1548852291,2sleep00-21swapper/00
1535948291,2sleep21535976-21apt-get2
1519823291,4sleep21521545-21tr2
1496561291,4sleep21496574-21ssh2
1443667291,3sleep11305944-21diskmemload1
1429862291,4sleep21429995-21diskstats2
1375038291,3sleep21305944-21diskmemload2
1341550291,2sleep20-21swapper/22
1330201291,2sleep20-21swapper/22
1329223291,3sleep11305944-21diskmemload1
1283698291,4sleep11283755-21ntp_states1
1263603291,3sleep00-21swapper/00
1253515291,4sleep11253952-21kthreadcore1
1237880291,4sleep01239616-21grep0
1225793291,2sleep11225812-21apt-get1
1208558291,3sleep11208603-21ntpq1
1183187291,2sleep10-21swapper/11
1294937280,2sleep10-21swapper/11
1279873281,3sleep21280067-21gltestperf2
1257953281,3sleep21257955-21modprobe2
1257896280,3sleep00-21swapper/00
1246763280,1sleep10-21swapper/11
1236690280,4sleep20-21swapper/22
1232865281,3sleep20-21swapper/22
1225815281,3sleep21225971-21cstates2
1197971280,2sleep00-21swapper/00
1300885270,2sleep00-21swapper/00
1300593270,1sleep00-21swapper/00
1295115270,1sleep20-21swapper/22
1284364270,1sleep00-21swapper/00
1275580271,2sleep10-21swapper/11
1274280270,2sleep00-21swapper/00
1273812270,1sleep00-21swapper/00
1264306271,3sleep11265989-21kthreadcore1
1263099270,1sleep20-21swapper/22
1258842270,3sleep01260205-21cut0
1252361270,1sleep20-21swapper/22
1247643270,2sleep10-21swapper/11
1246419271,3sleep00-21swapper/00
1246062271,3sleep20-21swapper/22
1241773270,4sleep10-21swapper/11
1215186270,2sleep10-21swapper/11
1214658270,1sleep10-21swapper/11
1204503271,2sleep01204522-21apt-get0
1204084270,1sleep00-21swapper/00
1199175270,2sleep10-21swapper/11
1188170270,1sleep20-21swapper/22
1188063270,4sleep00-21swapper/00
1180743270,1sleep00-21swapper/00
1175814270,2sleep11175288-21apt-get1
1305888260,1sleep20-21swapper/22
1300530260,1sleep10-21swapper/11
1289734260,1sleep00-21swapper/00
1272831261,2sleep10-21swapper/11
1267979261,2sleep20-21swapper/22
1251795261,2sleep00-21swapper/00
1231714261,2sleep10-21swapper/11
1224519260,2sleep00-21swapper/00
1220148260,4sleep00-21swapper/00
1209281260,1sleep20-21swapper/22
1198763260,4sleep10-21swapper/11
1193515260,2sleep10-21swapper/11
1192448260,2sleep20-21swapper/22
1225782250,1sleep00-21swapper/00
1224844251,1sleep20-21swapper/22
1210205240,1sleep20-21swapper/22
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional