You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-28 - 19:57

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot3s.osadl.org (updated Thu May 28, 2026 12:43:53)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
211339821380,2sleep00-21swapper/00
224154821240,3sleep1321rcuc/11
2063452212163,53sleep30-21swapper/33
206310128857,11sleep20-21swapper/22
206337028453,11sleep10-21swapper/11
206340328152,7sleep00-21swapper/00
206357799231,16cyclictest0-21swapper/33
249585722216,3sleep12492611-21/usr/sbin/munin1
206357799220,18cyclictest0-21swapper/33
2063577992117,2cyclictest0-21swapper/33
206357799211,17cyclictest0-21swapper/33
24140572201,3sleep20-21swapper/22
206357799201,16cyclictest24193802sleep33
206357799201,16cyclictest2108369-21chrt3
206357799201,15cyclictest0-21swapper/33
25453972191,4sleep20-21swapper/22
22877612190,3sleep22287768-21ssh2
21359842191,5sleep22136175-21grep2
206357799191,15cyclictest0-21swapper/33
206357799191,14cyclictest0-21swapper/33
206357799191,14cyclictest0-21swapper/33
206357799191,14cyclictest0-21swapper/33
206357799191,14cyclictest0-21swapper/33
206357799191,13cyclictest2277976-21chrt3
206357799190,15cyclictest66950irq/132-enp1s0-TxRx-33
24401462181,4sleep02194196-21diskmemload0
21728382181,6sleep22172886-21munin-run2
2063577991815,1cyclictest0-21swapper/33
2063577991815,1cyclictest0-21swapper/33
2063577991815,1cyclictest0-21swapper/33
2063577991814,2cyclictest0-21swapper/33
206357799181,14cyclictest0-21swapper/33
206357799181,14cyclictest0-21swapper/33
206357799181,14cyclictest0-21swapper/33
206357799181,14cyclictest0-21swapper/33
206357799181,14cyclictest0-21swapper/33
206357799181,14cyclictest0-21swapper/33
206357799181,14cyclictest0-21swapper/33
206357799181,14cyclictest0-21swapper/33
206357799181,14cyclictest0-21swapper/33
206357799181,14cyclictest0-21swapper/33
206357799180,1cyclictest0-21swapper/33
206357799180,14cyclictest66950irq/132-enp1s0-TxRx-33
206357799180,14cyclictest0-21swapper/33
206357799180,14cyclictest0-21swapper/33
25519932171,2sleep00-21swapper/00
23431672171,2sleep20-21swapper/22
22268932171,5sleep02194196-21diskmemload0
22196982171,4sleep02194196-21diskmemload0
20976152171,4sleep272-21kcompactd02
206357799172,9cyclictest2299976-21taskset3
2063577991713,2cyclictest0-21swapper/33
2063577991712,3cyclictest0-21swapper/33
206357799171,13cyclictest2167504-21turbostat3
206357799171,13cyclictest0-21swapper/33
206357799171,12cyclictest0-21swapper/33
206357799170,14cyclictest0-21swapper/33
206357799170,14cyclictest0-21swapper/33
206357799170,14cyclictest0-21swapper/33
206357799170,14cyclictest0-21swapper/33
206357799170,14cyclictest0-21swapper/33
206357799170,14cyclictest0-21swapper/33
206357799170,14cyclictest0-21swapper/33
206357799170,14cyclictest0-21swapper/33
206357799170,14cyclictest0-21swapper/33
24722472161,3sleep10-21swapper/11
24542852160,1sleep00-21swapper/00
24542852160,1sleep00-21swapper/00
23874022161,4sleep02194196-21diskmemload0
23656312161,10sleep172-21kcompactd01
23236602161,4sleep02194196-21diskmemload0
22790712161,4sleep02194196-21diskmemload0
22371192161,3sleep20-21swapper/22
206357799161,12cyclictest2124512-21turbostat3
206357799161,12cyclictest0-21swapper/33
206357799161,12cyclictest0-21swapper/33
206357799161,12cyclictest0-21swapper/33
206357799160,14cyclictest0-21swapper/33
206357799160,13cyclictest0-21swapper/33
206357799160,13cyclictest0-21swapper/33
206357799160,13cyclictest0-21swapper/33
206357799160,13cyclictest0-21swapper/33
25415352151,9sleep12194196-21diskmemload1
25415352151,9sleep12194196-21diskmemload1
25343472150,1sleep20-21swapper/22
25241122151,4sleep12194196-21diskmemload1
251825121510,2sleep20-21swapper/22
25084282151,10sleep22194196-21diskmemload2
24619312151,4sleep02461992-21ssh0
24223592151,9sleep02194196-21diskmemload0
24120402157,3sleep10-21swapper/11
23921332151,7sleep12392193-21ntpq1
23419402151,2sleep10-21swapper/11
22883902150,1sleep00-21swapper/00
22824342150,4sleep10-21swapper/11
22378672151,2sleep10-21swapper/11
22088532151,2sleep00-21swapper/00
21828802151,9sleep22183124-21users2
21666752150,1sleep20-21swapper/22
206357799151,8cyclictest21837812sleep33
206357799151,8cyclictest21837812sleep33
2063577991512,2cyclictest0-21swapper/33
206357799151,11cyclictest21296822sleep33
206357799150,13cyclictest0-21swapper/33
206357799150,13cyclictest0-21swapper/33
206357799150,13cyclictest0-21swapper/33
206357799150,12cyclictest0-21swapper/33
206357799150,12cyclictest0-21swapper/33
206357799150,11cyclictest2091894-21chrt3
24397182141,2sleep20-21swapper/22
23866102141,9sleep12386624-21ssh1
23518492141,4sleep12194196-21diskmemload1
23237402141,9sleep22323754-21ssh2
22915442141,9sleep20-21swapper/22
22487932141,8sleep02250436-21cut0
20765092141,8sleep02076528-21apt-get0
206357799147,5cyclictest2076257-21turbostat3
2063577991411,1cyclictest0-21swapper/33
2063577991410,2cyclictest0-21swapper/33
25586162131,8sleep12558630-21ssh1
25532882131,5sleep10-21swapper/11
25435182130,2sleep20-21swapper/22
25435182130,2sleep20-21swapper/22
23694232131,8sleep22369434-21ssh2
23598552131,8sleep02359866-21ssh0
22984352131,7sleep12194196-21diskmemload1
22435022131,3sleep22243514-21ssh2
22294352131,7sleep22229482-21awk2
21736132131,5sleep12173852-21kthreadcore1
206357799138,3cyclictest0-21swapper/33
25240452121,7sleep02194196-21diskmemload0
25142652121,7sleep12514330-21ssh1
25071462121,4sleep02507157-21ssh0
24792662121,5sleep12479364-21ntpq1
24750422121,6sleep02194196-21diskmemload0
24656842121,7sleep22465697-21ssh2
24495852121,7sleep12450722-21cat1
24495852121,7sleep12450722-21cat1
24410082121,7sleep22442607-21grep2
24384502120,3sleep02438464-21ssh0
24022072121,7sleep12402221-21ssh1
23509792121,5sleep02194196-21diskmemload0
23431412122,6sleep02343151-21ssh0
23337342121,7sleep22333744-21ssh2
23154802122,5sleep02194196-21diskmemload0
22788512121,5sleep22194196-21diskmemload2
22647772121,7sleep12264883-21expr1
22647772121,7sleep12264883-21expr1
22513382121,7sleep22251488-21/usr/sbin/munin2
22272112122,5sleep12194196-21diskmemload1
22138602124,4sleep12215526-21cut1
25607892111,6sleep02560802-21ssh0
25380042111,6sleep02539695-21grep0
25380042111,6sleep02539695-21grep0
25274702111,6sleep12527481-21ssh1
24917092111,2sleep00-21swapper/00
24601152111,4sleep10-21swapper/11
24402132111,4sleep12194196-21diskmemload1
24311072111,6sleep02431291-21latency_hist0
24201072111,4sleep10-21swapper/11
24053162111,4sleep02194196-21diskmemload0
23985912111,6sleep02400303-21grep0
23933992112,4sleep22194196-21diskmemload2
23930782111,4sleep02194196-21diskmemload0
23697142111,5sleep02369780-21munin-run0
23581082110,2sleep12358122-21ssh1
23298292111,6sleep12326448-21/usr/sbin/munin1
23230152111,4sleep10-21swapper/11
23154122111,3sleep20-21swapper/22
23092652111,7sleep12309400-21tune2fs1
23068122112,4sleep22194196-21diskmemload2
22973032112,4sleep02194196-21diskmemload0
22724402111,3sleep20-21swapper/22
22724402111,3sleep20-21swapper/22
22688552110,5sleep02268953-21ntpq0
22688552110,5sleep02268953-21ntpq0
22618022111,6sleep12194196-21diskmemload1
22437822111,4sleep02194196-21diskmemload0
22107322111,3sleep12210746-21ssh1
22078372111,5sleep22207901-21ntpq2
22009742111,4sleep12194196-21diskmemload1
21997792110,1sleep20-21swapper/22
21694322110,7sleep12171213-21cut1
21455512111,5sleep22145768-21users2
21039342111,3sleep20-21swapper/22
20823462111,5sleep12082417-21irqstats1
20679712111,6sleep0555-21dbus-daemon0
20637712111,5sleep22064023-21df2
25618632101,3sleep20-21swapper/22
25342262101,4sleep02194196-21diskmemload0
25220002101,4sleep22522793-21memory2
25175262101,6sleep02517540-21ssh0
24889672101,3sleep20-21swapper/22
24713782101,2sleep00-21swapper/00
24557202101,3sleep20-21swapper/22
24557202101,3sleep20-21swapper/22
24291202100,3sleep20-21swapper/22
24024652101,4sleep20-21swapper/22
23847042101,4sleep22194196-21diskmemload2
23775112101,3sleep20-21swapper/22
23764752101,4sleep02194196-21diskmemload0
23758782101,3sleep10-21swapper/11
23582182101,4sleep22194196-21diskmemload2
23440012101,4sleep22344270-21df_inode2
23316012101,5sleep02331615-21ssh0
23014012101,2sleep00-21swapper/00
22818632101,6sleep12194196-21diskmemload1
22523162101,4sleep12194196-21diskmemload1
22341522101,5sleep02234283-21grep0
22194192100,1sleep20-21swapper/22
21944442101,3sleep02194196-21diskmemload0
21898212101,5sleep22191496-21cut2
21790182101,5sleep02180605-21tr0
21665442101,5sleep02166598-21ntpq0
21518152101,4sleep22152018-21diskstats2
21197082101,5sleep22119869-21grep2
21092922101,5sleep12110190-21kthreadcore1
21062302101,4sleep02107306-21memory0
20877372101,5sleep12087773-21kthreadcore1
20855352100,1sleep00-21swapper/00
20735252101,5sleep22071118-21/usr/sbin/munin2
2507263291,2sleep10-21swapper/11
2497857290,1sleep20-21swapper/22
2494163291,4sleep02495855-21cut0
2490174291,3sleep10-21swapper/11
2480550290,2sleep20-21swapper/22
2466393291,4sleep22466404-21ssh2
2437456291,3sleep12194196-21diskmemload1
2428360291,4sleep12428372-21ssh1
2410998291,4sleep22411009-21ssh2
2306853291,4sleep12306866-21ssh1
2261257291,4sleep22261270-21ssh2
2260307292,3sleep00-21swapper/00
2162722291,4sleep12162854-21grep1
2157778292,3sleep22159455-21kthreadcore2
2123492291,4sleep02123543-21ntpq0
2119477291,4sleep12119707-21df_inode1
2098304291,4sleep12098428-21perl1
2097599290,4sleep12097532-31find1
2088775291,4sleep2421rcuc/22
2185235281,3sleep02186985-21kthreadcore0
2185235281,3sleep02186985-21kthreadcore0
2168327281,3sleep02168362-21kthreadcore0
2146398280,3sleep22146417-21apt-get2
2134239280,1sleep20-21swapper/22
2114057280,3sleep00-21swapper/00
2097588280,4sleep02097532-31find0
2076784281,3sleep20-21swapper/22
2071139281,3sleep10-21swapper/11
2071117281,2sleep00-21swapper/00
2194176271,4sleep10-21swapper/11
2188493270,1sleep20-21swapper/22
2188493270,1sleep20-21swapper/22
2188149271,3sleep10-21swapper/11
2188149271,3sleep10-21swapper/11
2180606271,3sleep10-21swapper/11
2178018270,1sleep00-21swapper/00
2157076270,2sleep10-21swapper/11
2146698271,3sleep10-21swapper/11
2130103270,2sleep00-21swapper/00
2128826271,3sleep20-21swapper/22
2118158271,2sleep10-21swapper/11
2102967270,1sleep20-21swapper/22
2098043270,3sleep00-21swapper/00
2087163272,3sleep0151ktimers/00
2086802270,1sleep20-21swapper/22
2063749270,2sleep10-21swapper/11
2173119261,2sleep20-21swapper/22
2162061260,4sleep00-21swapper/00
2152278261,2sleep12152740-21kthreadcore1
2151731260,1sleep00-21swapper/00
2150546261,2sleep00-21swapper/00
2141633261,2sleep00-21swapper/00
2137452260,3sleep12139244-21tr1
2134187261,2sleep10-21swapper/11
2129689260,1sleep00-21swapper/00
2129492260,4sleep10-21swapper/11
2110193261,2sleep20-21swapper/22
2108382260,1sleep10-21swapper/11
2077455261,2sleep10-21swapper/11
2193975250,3sleep00-21swapper/00
2141223250,1sleep10-21swapper/11
2140769250,1sleep00-21swapper/00
2118447250,1sleep20-21swapper/22
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional