You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-17 - 08:12

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3s.osadl.org (updated Fri Jul 17, 2026 00:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
44841321360,2sleep2448297-21ssh2
22802212552,67sleep30-21swapper/33
2287628353,11sleep20-21swapper/22
2257928163,12sleep10-21swapper/11
2245127662,7sleep00-21swapper/00
4795082340,3sleep0150759-21diskmemload0
2171512330,3sleep00-21swapper/00
23085993014,5cyclictest0-21swapper/33
23085992713,5cyclictest0-21swapper/33
23085992623,1cyclictest0-21swapper/33
23085992521,2cyclictest0-21swapper/33
2308599232,17cyclictest0-21swapper/33
2308599232,17cyclictest0-21swapper/33
5016782222,6sleep1501772-21ssh1
2308599221,7cyclictest168435-21taskset3
2308599221,17cyclictest0-21swapper/33
23085992111,6cyclictest0-21swapper/33
2308599211,16cyclictest0-21swapper/33
23085992110,2cyclictest237807-21turbostat3
2308599210,13cyclictest0-21swapper/33
2308599202,15cyclictest66950irq/132-enp1s0-TxRx-33
2308599202,13cyclictest316431-21taskset3
23085992013,4cyclictest0-21swapper/33
2308599201,1cyclictest0-21swapper/33
2308599201,17cyclictest0-21swapper/33
2308599201,16cyclictest0-21swapper/33
2308599201,14cyclictest0-21swapper/33
2308599200,3cyclictest774362sleep33
2769442191,7sleep2277089-21/usr/sbin/munin2
2308599198,8cyclictest510124-21chrt3
2308599194,2cyclictest0-21swapper/33
2308599191,15cyclictest0-21swapper/33
2308599191,15cyclictest0-21swapper/33
2308599191,14cyclictest0-21swapper/33
2308599191,14cyclictest0-21swapper/33
2308599191,14cyclictest0-21swapper/33
2308599191,14cyclictest0-21swapper/33
2308599191,14cyclictest0-21swapper/33
2308599191,14cyclictest0-21swapper/33
2308599191,14cyclictest0-21swapper/33
2308599191,14cyclictest0-21swapper/33
2308599190,17cyclictest0-21swapper/33
4998442181,2sleep00-21swapper/00
4098532181,3sleep00-21swapper/00
3729172189,1sleep1171rcu_preempt1
2308599181,5cyclictest0-21swapper/33
2308599181,5cyclictest0-21swapper/33
2308599181,4cyclictest0-21swapper/33
23085991814,2cyclictest0-21swapper/33
23085991812,3cyclictest0-21swapper/33
2308599181,14cyclictest0-21swapper/33
2308599181,14cyclictest0-21swapper/33
2308599181,14cyclictest0-21swapper/33
2308599181,14cyclictest0-21swapper/33
2308599181,13cyclictest0-21swapper/33
2308599181,13cyclictest0-21swapper/33
2308599181,13cyclictest0-21swapper/33
2308599181,13cyclictest0-21swapper/33
2308599181,13cyclictest0-21swapper/33
4590592171,11sleep2460613-21ssh2
4557822171,2sleep00-21swapper/00
2308599172,9cyclictest98682-21taskset3
2308599171,4cyclictest0-21swapper/33
2308599171,3cyclictest0-21swapper/33
23085991712,3cyclictest0-21swapper/33
23085991712,2cyclictest0-21swapper/33
23085991711,4cyclictest0-21swapper/33
2308599171,13cyclictest0-21swapper/33
23085991710,4cyclictest0-21swapper/33
2308599170,14cyclictest0-21swapper/33
2308599170,14cyclictest0-21swapper/33
2308599170,14cyclictest0-21swapper/33
2308599170,13cyclictest0-21swapper/33
2262462171,4sleep0150759-21diskmemload0
2092772170,3sleep20-21swapper/22
1847842170,2sleep00-21swapper/00
1143592172,4sleep2114378-21apt-get2
992082161,11sleep299631-21cut2
4497292161,11sleep1450600-21cut1
4082242161,2sleep10-21swapper/11
3318792161,3sleep20-21swapper/22
2708592161,2sleep10-21swapper/11
2532442161,4sleep0150759-21diskmemload0
23085991610,3cyclictest0-21swapper/33
2308599160,14cyclictest0-21swapper/33
1998192160,10sleep20-21swapper/22
983662154,7sleep298432-21sendmail-msp2
5160342150,2sleep0221rcuc/00
4740632151,2sleep00-21swapper/00
4457032150,2sleep10-21swapper/11
4363472150,2sleep10-21swapper/11
4178722150,1sleep10-21swapper/11
4113442150,2sleep20-21swapper/22
2308599150,13cyclictest0-21swapper/33
2308599150,12cyclictest0-21swapper/33
2308599150,12cyclictest0-21swapper/33
2308599150,12cyclictest0-21swapper/33
1748202151,3sleep0150759-21diskmemload0
571102141,4sleep057212-21perl0
5167682141,8sleep1150759-21diskmemload1
4019622140,2sleep10-21swapper/11
3334882140,2sleep00-21swapper/00
3215892141,3sleep0150759-21diskmemload0
2722852141,2sleep0150759-21diskmemload0
2380992141,2sleep20-21swapper/22
2308599143,7cyclictest0-21swapper/33
2201492141,3sleep1220159-21ssh1
1768312141,8sleep20-21swapper/22
5158552131,7sleep2150759-21diskmemload2
4915222131,8sleep1491533-21ssh1
4725702131,3sleep1472584-21ssh1
4355622131,7sleep0150759-21diskmemload0
4277562131,8sleep1427768-21ssh1
4022062131,3sleep0402220-21ssh0
3825972131,3sleep00-21swapper/00
3658062131,8sleep0365820-21ssh0
3511432131,7sleep1351411-21df1
3424302133,4sleep2342434-21apt-get2
3419802131,3sleep2342263-21cut2
3149522131,6sleep00-21swapper/00
3138432130,3sleep2150759-21diskmemload2
3075962131,6sleep1307354-21sendmail-msp1
2951262131,6sleep10-21swapper/11
2308599131,1cyclictest0-21swapper/33
2289642131,8sleep1228979-21ssh1
1998612131,7sleep1150759-21diskmemload1
869482121,7sleep186990-21ntp_kernel_pll_1
5003382121,6sleep2150759-21diskmemload2
4300582121,7sleep2431163-21cut2
4270762121,4sleep00-21swapper/00
410572122,4sleep041076-21apt-get0
3908042120,2sleep00-21swapper/00
3733062122,4sleep2150759-21diskmemload2
3408922120,8sleep1340903-21ssh1
3227362121,6sleep1150759-21diskmemload1
305342121,5sleep130787-21df1
2978922121,6sleep0150759-21diskmemload0
2608882121,7sleep1260903-21ssh1
2608882121,7sleep1260903-21ssh1
2542992120,7sleep2150759-21diskmemload2
2347712121,6sleep1234785-21ssh1
2103412124,4sleep123076-21cyclictest1
1983542121,7sleep0198684-21ssh0
1914902120,2sleep00-21swapper/00
1850602121,2sleep1185074-21ssh1
1563142121,6sleep00-21swapper/00
724782111,3sleep20-21swapper/22
4744482111,6sleep2474461-21ssh2
4258662111,2sleep20-21swapper/22
4202982110,7sleep2420331-21ls2
4165202111,5sleep0416775-21users0
410792114,3sleep141248-21expr1
3998282111,3sleep20-21swapper/22
3932902111,3sleep20-21swapper/22
3849332111,6sleep1150759-21diskmemload1
3422872110,7sleep0342051-21sendmail-msp0
3408332111,5sleep0150759-21diskmemload0
3314602111,4sleep10-21swapper/11
2906262110,1sleep20-21swapper/22
2897202110,3sleep2289734-21ssh2
2805922111,3sleep00-21swapper/00
2732542111,6sleep1273359-21fschecks_time1
2556032111,3sleep00-21swapper/00
2556032111,3sleep00-21swapper/00
2381272111,6sleep0238138-21ssh0
2336942111,4sleep2150759-21diskmemload2
2191312110,2sleep20-21swapper/22
2088872110,7sleep0150759-21diskmemload0
1573462111,6sleep1157358-21ssh1
1405372111,4sleep0140557-21apt-get0
1091722111,6sleep10-21swapper/11
986532100,1sleep00-21swapper/00
568122100,2sleep20-21swapper/22
5055412101,3sleep0150759-21diskmemload0
4805662101,4sleep1150759-21diskmemload1
4437692100,4sleep2431ktimers/22
3656272101,3sleep20-21swapper/22
3654612101,3sleep1150759-21diskmemload1
3586642100,1sleep00-21swapper/00
3232182101,3sleep2150759-21diskmemload2
3067682101,2sleep20-21swapper/22
3040692101,4sleep0150759-21diskmemload0
3007132101,5sleep1302308-21kthreadcore1
2894002100,3sleep10-21swapper/11
2702592101,4sleep2150759-21diskmemload2
2528472101,4sleep1150759-21diskmemload1
2448532101,3sleep10-21swapper/11
2339792102,4sleep0234225-21sed0
232692100,6sleep123370-21expr1
232472101,4sleep023266-21apt-get0
2259102101,4sleep2150759-21diskmemload2
1907202101,4sleep1190947-21users1
1863402101,4sleep2150759-21diskmemload2
1754412101,3sleep10-21swapper/11
1637682101,4sleep2163951-21strings2
1601382101,5sleep1160254-21perl1
1588062100,1sleep20-21swapper/22
1445542101,4sleep1144606-21ntp_states1
1417752101,5sleep2143433-21cut2
1091512101,3sleep00-21swapper/00
90083291,4sleep191838-21cut1
83540291,4sleep284750-21kthreadcore2
77734291,3sleep177753-21apt-get1
51051291,4sleep151052-2191
508509290,2sleep20-21swapper/22
497605290,1sleep10-21swapper/11
485201291,4sleep0486797-21grep0
484009291,2sleep20-21swapper/22
475326291,3sleep2150759-21diskmemload2
463767290,2sleep10-21swapper/11
443291291,4sleep0443302-21ssh0
385980291,4sleep1386241-21cstates1
384066290,3sleep20-21swapper/22
375859291,4sleep0375872-21sh0
35769291,3sleep035788-21apt-get0
357313291,1sleep20-21swapper/22
347270291,3sleep1347281-21ssh1
34660291,4sleep234798-21sed2
30525291,3sleep030531-21apt-get0
286623291,2sleep00-21swapper/00
255519291,4sleep2106713-21snmpd2
255519291,4sleep2106713-21snmpd2
182194291,2sleep20-21swapper/22
150380290,4sleep172-21kcompactd01
146305291,4sleep0146580-21cut0
135847290,6sleep2136735-21cut2
134648290,1sleep20-21swapper/22
119618291,2sleep10-21swapper/11
103923291,3sleep1103920-21apt-get1
93441280,2sleep00-21swapper/00
92393281,3sleep292520-21find2
87920281,3sleep087976-21unin-run0
78270281,3sleep277735-21/usr/sbin/munin2
68723281,3sleep170407-21kthreadcore1
66707280,1sleep10-21swapper/11
66312280,1sleep20-21swapper/22
56498281,3sleep256575-21systemd-run2
53666280,1sleep00-21swapper/00
457936281,2sleep00-21swapper/00
40664281,2sleep20-21swapper/22
165442280,2sleep00-21swapper/00
150455281,4sleep2116341-21sshd2
124448280,1sleep00-21swapper/00
104506281,3sleep20-21swapper/22
98659271,2sleep198678-21apt-get1
94028271,3sleep195511-21ps1
77225270,1sleep10-21swapper/11
77158270,1sleep00-21swapper/00
71856272,3sleep2431ktimers/22
63074271,3sleep00-21swapper/00
56826270,3sleep10-21swapper/11
51191270,1sleep00-21swapper/00
45661270,1sleep20-21swapper/22
28171270,1sleep20-21swapper/22
139938270,1sleep00-21swapper/00
135280270,2sleep10-21swapper/11
125401271,3sleep1126538-21grep1
125330270,3sleep00-21swapper/00
118409271,3sleep00-21swapper/00
103896270,1sleep00-21swapper/00
82383260,1sleep00-21swapper/00
67267261,2sleep00-21swapper/00
56241260,4sleep10-21swapper/11
46597260,1sleep20-21swapper/22
35760261,2sleep10-21swapper/11
132077261,2sleep10-21swapper/11
130064261,2sleep00-21swapper/00
124996260,2sleep20-21swapper/22
119619260,2sleep20-21swapper/22
114649250,2sleep10-21swapper/11
109424250,1sleep20-21swapper/22
88212240,1sleep00-21swapper/00
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional