You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-14 - 13:37

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot3s.osadl.org (updated Tue Jul 14, 2026 00:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1129023213680,52sleep30-21swapper/33
125705021260,3sleep2421rcuc/22
112893328149,12sleep20-21swapper/22
112879228063,11sleep10-21swapper/11
112868728056,7sleep00-21swapper/00
112920099231,19cyclictest0-21swapper/33
112920099211,3cyclictest0-21swapper/33
1129200992112,6cyclictest0-21swapper/33
112920099211,16cyclictest66950irq/132-enp1s0-TxRx-33
112920099211,16cyclictest0-21swapper/33
112920099211,16cyclictest0-21swapper/33
112920099211,16cyclictest0-21swapper/33
112920099211,15cyclictest0-21swapper/33
11767252201,5sleep11177210-21/usr/sbin/munin1
1129200992017,2cyclictest0-21swapper/33
1129200992017,1cyclictest0-21swapper/33
112920099201,15cyclictest0-21swapper/33
112920099201,15cyclictest0-21swapper/33
112920099201,15cyclictest0-21swapper/33
112920099201,15cyclictest0-21swapper/33
112920099201,15cyclictest0-21swapper/33
15633342190,3sleep10-21swapper/11
14402802198,3sleep11441449-21cut1
14402802198,3sleep11441449-21cut1
13549312191,14sleep11356655-21kthreadcore1
112920099194,12cyclictest0-21swapper/33
112920099191,14cyclictest0-21swapper/33
112920099191,14cyclictest0-21swapper/33
112920099191,14cyclictest0-21swapper/33
112920099191,14cyclictest0-21swapper/33
112920099190,16cyclictest0-21swapper/33
15801162182,5sleep01256812-21diskmemload0
14540542181,8sleep21256812-21diskmemload2
14298082181,2sleep00-21swapper/00
13938222181,8sleep21393837-21ssh2
12739352181,4sleep10-21swapper/11
112920099182,12cyclictest1413672-21taskset3
112920099182,11cyclictest1291930-21taskset3
1129200991812,3cyclictest0-21swapper/33
112920099181,14cyclictest0-21swapper/33
112920099181,14cyclictest0-21swapper/33
112920099181,14cyclictest0-21swapper/33
112920099181,14cyclictest0-21swapper/33
112920099181,14cyclictest0-21swapper/33
112920099181,14cyclictest0-21swapper/33
112920099181,14cyclictest0-21swapper/33
112920099181,14cyclictest0-21swapper/33
112920099181,14cyclictest0-21swapper/33
112920099181,14cyclictest0-21swapper/33
112920099181,14cyclictest0-21swapper/33
112920099181,14cyclictest0-21swapper/33
112920099181,14cyclictest0-21swapper/33
112920099181,13cyclictest0-21swapper/33
112920099181,13cyclictest0-21swapper/33
112920099181,13cyclictest0-21swapper/33
112920099181,13cyclictest0-21swapper/33
112920099181,13cyclictest0-21swapper/33
112920099181,13cyclictest0-21swapper/33
112920099181,12cyclictest0-21swapper/33
112920099180,15cyclictest0-21swapper/33
112920099180,14cyclictest0-21swapper/33
13948862171,3sleep10-21swapper/11
13246032172,4sleep11256812-21diskmemload1
1129200991713,2cyclictest0-21swapper/33
112920099171,13cyclictest0-21swapper/33
112920099171,13cyclictest0-21swapper/33
112920099171,13cyclictest0-21swapper/33
112920099171,13cyclictest0-21swapper/33
112920099170,14cyclictest0-21swapper/33
112920099170,14cyclictest0-21swapper/33
16124242161,4sleep21256812-21diskmemload2
15956922160,3sleep21595710-21ssh2
14965952161,5sleep01256812-21diskmemload0
14963592161,4sleep11256812-21diskmemload1
14738812161,2sleep10-21swapper/11
14370182165,2sleep1171rcu_preempt1
13775402161,2sleep10-21swapper/11
13344212160,1sleep20-21swapper/22
13006172161,4sleep01300858-21expr0
12924492161,5sleep21292530-21irqstats2
112920099165,8cyclictest1251570-21turbostat3
112920099161,12cyclictest0-21swapper/33
112920099161,12cyclictest0-21swapper/33
112920099161,12cyclictest0-21swapper/33
16230372151,3sleep21256812-21diskmemload2
16060162151,4sleep21256812-21diskmemload2
15766222151,8sleep11576866-21/usr/sbin/munin1
14720772151,2sleep00-21swapper/00
14565912151,9sleep10-21swapper/11
14079362152,6sleep21408939-21memory2
13446332151,3sleep01344871-21kthreadcore0
12896392151,2sleep20-21swapper/22
11943242151,9sleep11194527-21diskstats1
112920099152,8cyclictest1173318-21taskset3
1129200991512,1cyclictest0-21swapper/33
112920099150,12cyclictest0-21swapper/33
112920099150,12cyclictest0-21swapper/33
15372652145,5sleep01537346-21kthreadcore0
15260772141,9sleep11526090-21ssh1
15234122140,3sleep01256812-21diskmemload0
14975842141,3sleep21256812-21diskmemload2
14873972141,9sleep01487527-21ssh0
14234162141,3sleep21424994-21kthreadcore2
14136382141,5sleep2421rcuc/22
13954652141,7sleep00-21swapper/00
13593062141,4sleep21359317-21ssh2
12822982141,9sleep21282312-21ssh2
12735062141,4sleep01273520-21ssh0
11293832141,7sleep21129638-21df2
112920099148,3cyclictest1230347-21chrt3
112920099148,3cyclictest1141544-21chrt3
1129200991412,1cyclictest0-21swapper/33
1129200991412,1cyclictest0-21swapper/33
112920099141,10cyclictest12460702sleep33
16041602131,3sleep11604174-21ssh1
15689932131,7sleep01256812-21diskmemload0
15596242131,7sleep01552295-40find0
15326632131,8sleep11532677-21ssh1
15090262131,8sleep11509036-21ssh1
15005812130,4sleep0221rcuc/00
14385512131,7sleep21256812-21diskmemload2
14374582131,3sleep01437471-21ssh0
14136262131,4sleep11413637-21ssh1
13428882131,7sleep21256812-21diskmemload2
13069012131,5sleep20-21swapper/22
12156852131,6sleep11215756-21irqstats1
11948092131,8sleep01195903-21grep0
112920099130,1cyclictest0-21swapper/33
112920099130,10cyclictest0-21swapper/33
16050642121,7sleep01605079-21ssh0
15596512120,6sleep21552189-40sort2
15592322121,7sleep11552295-40find1
15510062121,7sleep01551018-21rm0
15323742120,1sleep20-21swapper/22
14824172121,3sleep21482431-21ssh2
14729182121,7sleep21472932-21ssh2
14648212121,7sleep01464837-21ssh0
14648212121,7sleep01464837-21ssh0
14549522121,7sleep01454965-21rm0
14204152121,6sleep01256812-21diskmemload0
12899862121,7sleep11289999-21ssh1
12758872121,6sleep11276732-21kthreadcore1
12557042121,5sleep11255848-21ntp_192.168.1141
16226142111,4sleep01256812-21diskmemload0
15973482111,4sleep01256812-21diskmemload0
15727272110,7sleep21573533-21grep2
15009762111,5sleep21501060-21irqstats2
14883732111,3sleep20-21swapper/22
14745862111,5sleep01474719-21ssh0
14401982111,5sleep01440279-21irqstats0
14401982111,5sleep01440279-21irqstats0
13780002110,2sleep20-21swapper/22
13664292110,6sleep01366441-21ssh0
13659402111,2sleep20-21swapper/22
13604472110,1sleep00-21swapper/00
13427112110,1sleep00-21swapper/00
13316542111,6sleep11331665-21ssh1
13248632111,6sleep21256812-21diskmemload2
13084212111,5sleep11308432-21ssh1
12975742111,3sleep00-21swapper/00
12722672111,3sleep20-21swapper/22
12644212110,1sleep10-21swapper/11
12624202111,5sleep01256812-21diskmemload0
12466772111,4sleep11246880-21diskstats1
12416312111,5sleep21241754-21perl2
12361342111,6sleep21236194-21grep2
12204172111,4sleep21220436-21apt-get2
11732962111,4sleep21173315-21apt-get2
11418792111,5sleep01141882-21apt-get0
11373672111,6sleep21138963-21cut2
11371172111,6sleep11137365-21kthreadcore1
16210772101,2sleep10-21swapper/11
16149222101,2sleep00-21swapper/00
15894352102,4sleep11589454-21apt-get1
15857642101,2sleep00-21swapper/00
15808782101,4sleep11581092-21diskstats1
15487432101,2sleep20-21swapper/22
15483212101,5sleep11548575-21ssh1
15432922101,4sleep21256812-21diskmemload2
15424202100,6sleep11256812-21diskmemload1
15335162101,4sleep01533530-21ssh0
15139332101,3sleep00-21swapper/00
14890802101,4sleep11256812-21diskmemload1
14632162101,5sleep21454613-21kworker/2:1+events2
14632162101,5sleep21454613-21kworker/2:1+events2
14451162101,4sleep21256812-21diskmemload2
14451162101,4sleep21256812-21diskmemload2
14102372101,4sleep11256812-21diskmemload1
14055342100,7sleep00-21swapper/00
14028912101,3sleep10-21swapper/11
14016212101,6sleep20-21swapper/22
13846212101,3sleep00-21swapper/00
13839652101,4sleep11383972-21ssh1
13815042101,4sleep21382825-21ssh2
13754392101,4sleep01256812-21diskmemload0
13695532101,2sleep10-21swapper/11
13440212100,2sleep10-21swapper/11
13235322101,4sleep01256812-21diskmemload0
13138422103,3sleep11256812-21diskmemload1
13092232101,4sleep21256812-21diskmemload2
13091962102,4sleep04192556-21in:imuxsock0
12966612101,3sleep11256812-21diskmemload1
12831502101,4sleep01256812-21diskmemload0
12335272101,4sleep11234760-21memory1
11994822101,4sleep11199494-21apt-get1
11616022101,5sleep21161720-21grep2
11471202101,4sleep01147379-21df_inode0
1612213291,2sleep10-21swapper/11
1564077291,4sleep21565390-21ssh2
1519836291,5sleep21521507-21kthreadcore2
1514410291,3sleep21514421-21ssh2
1504816291,4sleep11505015-21sed1
1474721291,3sleep11256812-21diskmemload1
1464635291,2sleep10-21swapper/11
1464635291,2sleep10-21swapper/11
1426448291,4sleep11426517-21rm1
1401087290,6sleep01401099-21sh0
1350132291,3sleep21256812-21diskmemload2
1340995291,2sleep10-21swapper/11
1331793291,4sleep01331804-21ssh0
1279250291,4sleep01279319-21ssh0
1246881291,4sleep21247001-21perl2
1231887291,4sleep01233526-21cut0
1230877291,3sleep21230896-21apt-get2
1225664290,1sleep10-21swapper/11
1225368291,3sleep072-21kcompactd00
1214709290,1sleep10-21swapper/11
1211660291,5sleep01213386-21kthreadcore0
1209947291,2sleep20-21swapper/22
1204736291,5sleep01204893-21cstates0
1200135291,4sleep21201714-21cut2
1193970291,4sleep21194028-21sd-rmrf2
1189574291,4sleep01190143-21grep0
1188276290,1sleep00-21swapper/00
1183775291,4sleep21183560-21sendmail-msp2
1183494291,4sleep01183573-21munin-run0
1182804291,4sleep11183049-21awk1
1178532290,3sleep20-21swapper/22
1162795291,3sleep01162814-21apt-get0
1157562291,3sleep11157565-21apt-get1
1156583291,4sleep11156836-21awk1
1152317290,3sleep21152329-21apt-get2
1129358291,3sleep01129377-21apt-get0
1581093280,1sleep20-21swapper/22
1256611280,1sleep00-21swapper/00
1256453280,2sleep20-21swapper/22
1241907281,3sleep01242669-21grep0
1236112280,2sleep10-21swapper/11
1225668280,4sleep00-21swapper/00
1220149281,3sleep01220287-21latency_hist0
1183811281,3sleep11184065-21df1
1162425280,2sleep00-21swapper/00
1141849280,4sleep20-21swapper/22
1141567281,3sleep01-21systemd0
1247154271,3sleep01247932-21kthreadcore0
1242670271,2sleep10-21swapper/11
1241033270,1sleep00-21swapper/00
1215206271,3sleep21215321-21cpuspeed_turbos2
1194528271,3sleep20-21swapper/22
1193679270,1sleep10-21swapper/11
1175004271,3sleep00-21swapper/00
1173025270,3sleep21-21systemd2
1172748270,1sleep10-21swapper/11
1168056270,3sleep01167770-21apt-get0
1167585270,1sleep10-21swapper/11
1167532270,4sleep20-21swapper/22
1151834270,1sleep10-21swapper/11
1147095270,2sleep20-21swapper/22
1225665261,2sleep21225448-21sendmail-msp2
1204705261,2sleep10-21swapper/11
1204701261,1sleep20-21swapper/22
1199777261,2sleep00-21swapper/00
1157243260,4sleep00-21swapper/00
1141638260,2sleep10-21swapper/11
1129908261,2sleep10-21swapper/11
1223119250,1sleep10-21swapper/11
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional