You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-14 - 14:33

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3s.osadl.org (updated Thu May 14, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2325747212951,73sleep30-21swapper/33
2325747212951,73sleep30-21swapper/33
232592228968,11sleep10-21swapper/11
232592228968,11sleep10-21swapper/11
232597328354,10sleep20-21swapper/22
232597328354,10sleep20-21swapper/22
232591627858,14sleep00-21swapper/00
232591627858,14sleep00-21swapper/00
25215182300,2sleep30-21swapper/33
232620399221,17cyclictest2606630-21chrt3
232620399211,17cyclictest0-21swapper/33
232620399201,15cyclictest66950irq/132-enp1s0-TxRx-33
232620399200,18cyclictest0-21swapper/33
27832532191,9sleep22456825-21diskmemload2
27363532191,3sleep22737298-21cut2
232620399197,7cyclictest2781632-21taskset3
2326203991916,1cyclictest0-21swapper/33
232620399191,15cyclictest0-21swapper/33
232620399191,14cyclictest2672584-21taskset3
232620399191,14cyclictest0-21swapper/33
232620399191,14cyclictest0-21swapper/33
232620399191,14cyclictest0-21swapper/33
232620399191,13cyclictest2583590-21chrt3
232620399190,16cyclictest66950irq/132-enp1s0-TxRx-33
232620399190,12cyclictest0-21swapper/33
27776682186,8sleep02456825-21diskmemload0
26166962181,2sleep10-21swapper/11
232620399181,14cyclictest0-21swapper/33
232620399181,14cyclictest0-21swapper/33
232620399181,14cyclictest0-21swapper/33
232620399181,14cyclictest0-21swapper/33
232620399181,14cyclictest0-21swapper/33
232620399181,14cyclictest0-21swapper/33
232620399181,13cyclictest0-21swapper/33
232620399181,13cyclictest0-21swapper/33
232620399181,13cyclictest0-21swapper/33
232620399181,13cyclictest0-21swapper/33
232620399181,13cyclictest0-21swapper/33
232620399181,13cyclictest0-21swapper/33
232620399181,13cyclictest0-21swapper/33
232620399180,15cyclictest66950irq/132-enp1s0-TxRx-33
232620399180,15cyclictest0-21swapper/33
232620399180,14cyclictest0-21swapper/33
232620399180,14cyclictest0-21swapper/33
232620399180,14cyclictest0-21swapper/33
27863592172,11sleep02786462-21perl0
25901452171,3sleep10-21swapper/11
25560022172,4sleep02456825-21diskmemload0
25316232171,2sleep20-21swapper/22
25049692171,6sleep22456825-21diskmemload2
232620399179,4cyclictest23605912sleep33
232620399176,8cyclictest2381675-21turbostat3
232620399171,7cyclictest2582194-21taskset3
232620399171,14cyclictest0-21swapper/33
232620399171,12cyclictest0-21swapper/33
232620399171,12cyclictest0-21swapper/33
2326203991710,3cyclictest2397674-21taskset3
232620399170,3cyclictest0-21swapper/33
232620399170,14cyclictest0-21swapper/33
232620399170,14cyclictest0-21swapper/33
232620399170,14cyclictest0-21swapper/33
232620399170,13cyclictest2359872-21chrt3
232620399170,13cyclictest0-21swapper/33
27515832161,3sleep20-21swapper/22
27155882161,2sleep10-21swapper/11
26589892161,4sleep02456825-21diskmemload0
26317322160,3sleep20-21swapper/22
24624732161,2sleep20-21swapper/22
232620399161,3cyclictest0-21swapper/33
232620399161,3cyclictest0-21swapper/33
2326203991612,2cyclictest0-21swapper/33
232620399161,13cyclictest0-21swapper/33
232620399161,12cyclictest0-21swapper/33
232620399161,12cyclictest0-21swapper/33
232620399160,14cyclictest0-21swapper/33
232620399160,13cyclictest0-21swapper/33
232620399160,13cyclictest0-21swapper/33
27845172151,3sleep12456825-21diskmemload1
26939342152,8sleep02694129-21sed0
26640912151,7sleep22456825-21diskmemload2
26511162151,3sleep12456825-21diskmemload1
25658742151,2sleep20-21swapper/22
25162142151,2sleep10-21swapper/11
25154092153,8sleep22490866-21kworker/u16:2+rpciod2
24987172150,2sleep10-21swapper/11
24796282151,4sleep22456825-21diskmemload2
24745312151,5sleep12474566-21/usr/sbin/munin1
24715152151,3sleep22456825-21diskmemload2
24710152151,3sleep10-21swapper/11
2326203991511,2cyclictest0-21swapper/33
232620399150,13cyclictest0-21swapper/33
232620399150,12cyclictest0-21swapper/33
232620399150,12cyclictest0-21swapper/33
232620399150,12cyclictest0-21swapper/33
27744362141,3sleep22456825-21diskmemload2
27594732141,2sleep10-21swapper/11
27239992141,6sleep22724014-21ssh2
26237322141,3sleep22456825-21diskmemload2
24830362141,8sleep22483172-21gltestperf2
232620399141,9cyclictest2344461-21chrt3
2326203991412,1cyclictest0-21swapper/33
2326203991411,2cyclictest0-21swapper/33
232620399140,2cyclictest0-21swapper/33
232620399140,12cyclictest0-21swapper/33
232620399140,12cyclictest0-21swapper/33
28090862131,7sleep12456825-21diskmemload1
27753832130,2sleep10-21swapper/11
27400362131,2sleep00-21swapper/00
27250122131,8sleep12725027-21ssh1
27062892131,5sleep02456825-21diskmemload0
26936412131,8sleep22693713-21tr2
25900582130,6sleep20-21swapper/22
25372442131,6sleep12537299-21ntpq1
25307912131,7sleep12456825-21diskmemload1
24277032131,8sleep22429057-21/usr/sbin/munin2
24075062131,8sleep22407606-21/usr/sbin/munin2
28080332121,7sleep22456825-21diskmemload2
28016162120,8sleep22801627-21ssh2
27505702121,7sleep02456825-21diskmemload0
27334962121,7sleep02456825-21diskmemload0
27083892121,7sleep22708400-21ssh2
27069802121,3sleep12706991-21ssh1
26846062121,7sleep20-21swapper/22
26392242121,6sleep12456825-21diskmemload1
26259262121,6sleep02626189-21df0
26165212121,3sleep22616532-21ssh2
25755152121,7sleep22575754-21cat2
25729082121,8sleep12572913-21rm1
25633542121,7sleep02563364-21ssh0
25563942121,5sleep10-21swapper/11
25487542121,5sleep00-21swapper/00
25162832121,5sleep22516311-21apt-get2
25152022121,6sleep00-21swapper/00
24962142121,7sleep22496225-21ssh2
23660232121,5sleep22366226-21diskstats2
23538822121,5sleep22353938-21ntpq2
23396602125,3sleep22340211-21cut2
28080092111,4sleep00-21swapper/00
27687192111,6sleep02768734-21ssh0
27681622111,6sleep12456825-21diskmemload1
27607792111,3sleep20-21swapper/22
27498332112,4sleep12456825-21diskmemload1
27324662111,6sleep12732479-21ssh1
27237752111,7sleep02723787-21rm0
26740402111,3sleep10-21swapper/11
26665892111,5sleep12456825-21diskmemload1
26416562111,4sleep0151ktimers/00
26405232111,4sleep22456825-21diskmemload2
26217802110,1sleep10-21swapper/11
25991092111,6sleep02599120-21sh0
25838602111,6sleep02583917-21ssh0
25481602111,5sleep12456825-21diskmemload1
25465672110,1sleep20-21swapper/22
25229742111,3sleep00-21swapper/00
25143092111,2sleep12514312-21ssh1
24557172111,5sleep22455806-21ntpq2
24345022110,1sleep20-21swapper/22
24326582111,6sleep02434371-21latency0
24307702111,5sleep12430980-21hddtemp_smartct1
24238652111,5sleep12423914-21ntpq1
24009982111,5sleep12402089-21memory1
23969412111,5sleep12396990-21ntpq1
27989372101,3sleep10-21swapper/11
27929472101,4sleep22456825-21diskmemload2
27924082101,3sleep10-21swapper/11
27768342101,4sleep02456825-21diskmemload0
27525732101,3sleep00-21swapper/00
27524052101,4sleep22456825-21diskmemload2
27418582101,6sleep12741869-21sh1
27277852101,2sleep2171rcu_preempt2
26987322101,3sleep10-21swapper/11
26914452101,2sleep10-21swapper/11
26904872101,2sleep20-21swapper/22
26859002101,5sleep02687127-21kthreadcore0
26839812101,3sleep00-21swapper/00
26838072101,3sleep10-21swapper/11
26733502102,3sleep22456825-21diskmemload2
26731342100,2sleep00-21swapper/00
26675632101,5sleep02667575-21ssh0
26588222101,4sleep22456825-21diskmemload2
26491812101,4sleep12649193-21ssh1
26476872101,2sleep20-21swapper/22
26315762101,3sleep10-21swapper/11
26162622101,2sleep00-21swapper/00
26082262100,1sleep20-21swapper/22
26000572101,2sleep10-21swapper/11
25971022101,2sleep20-21swapper/22
25821182101,5sleep12456825-21diskmemload1
25740222101,4sleep02456825-21diskmemload0
25498272101,3sleep20-21swapper/22
25395302101,3sleep00-21swapper/00
25297812100,1sleep00-21swapper/00
25055462101,4sleep12456825-21diskmemload1
24987972101,4sleep02456825-21diskmemload0
24891752100,1sleep10-21swapper/11
24803842100,1sleep00-21swapper/00
24643542101,6sleep10-21swapper/11
24636922101,4sleep02456825-21diskmemload0
24451472101,5sleep22445193-21ntpq2
24251532101,4sleep02425172-21apt-get0
24129382101,5sleep02412968-21ntpq0
23928612101,4sleep02392880-21apt-get0
23916772101,6sleep00-21swapper/00
23806152105,2sleep00-21swapper/00
23647412101,4sleep12364794-21ntp_states1
23599982100,1sleep00-21swapper/00
23497712101,4sleep12349789-21apt-get1
23432662101,6sleep02343326-21pmu-power0
23303042100,5sleep12330445-21ntpq1
2798592291,3sleep02798644-21ntpq0
2716548291,3sleep22456825-21diskmemload2
2716244291,3sleep02456825-21diskmemload0
2650618291,4sleep02650629-21ssh0
2621628291,3sleep02621779-21ntpq0
2605890291,2sleep00-21swapper/00
2597433291,4sleep12597444-21ssh1
2572025291,3sleep22456825-21diskmemload2
2562486290,5sleep12562543-21ntpq1
2533302290,5sleep22533381-21ssh2
2499540290,1sleep00-21swapper/00
2490681291,2sleep00-21swapper/00
2472378291,4sleep02472389-21ssh0
2452057291,4sleep12452181-21perl1
2435826291,4sleep20-21swapper/22
2426027291,4sleep12427702-21tr1
2414788291,4sleep12416032-21cut1
2414716290,1sleep00-21swapper/00
2412758291,4sleep22412796-21memory2
2404967291,4sleep02406713-21cut0
2402699290,2sleep20-21swapper/22
2387562291,3sleep22387582-21apt-get2
2383977291,4sleep02385778-21grep0
2377190291,4sleep12378706-21grep1
2356619291,4sleep12358391-21cut1
2337799291,3sleep12337837-21ntpq1
2330446291,4sleep22330497-21ld2
2580725280,1sleep00-21swapper/00
2447014281,3sleep12447942-21tr1
2441646281,3sleep12441892-21kthreadcore1
2419786281,3sleep02419789-21acpi0
2382192281,2sleep22381674-21apt-get2
2371269280,2sleep10-21swapper/11
2366510281,4sleep12367788-21kthreadcore1
2354668280,4sleep00-21swapper/00
2348482280,2sleep20-21swapper/22
2343867280,1sleep10-21swapper/11
2337667280,1sleep00-21swapper/00
2453253271,3sleep02455017-21kthreadcore0
2450820271,3sleep20-21swapper/22
2421302271,3sleep20-21swapper/22
2419084270,1sleep20-21swapper/22
2406714271,3sleep10-21swapper/11
2397760270,1sleep20-21swapper/22
2365434270,1sleep20-21swapper/22
2360094270,1sleep20-21swapper/22
2349308270,1sleep00-21swapper/00
2451304260,1sleep00-21swapper/00
2445256261,2sleep00-21swapper/00
2435804260,2sleep10-21swapper/11
2412975260,2sleep10-21swapper/11
2387858261,2sleep10-21swapper/11
2378707261,2sleep20-21swapper/22
2376007260,1sleep00-21swapper/00
2375347260,2sleep20-21swapper/22
2366438260,2sleep00-21swapper/00
2349457260,1sleep10-21swapper/11
2331463260,2sleep22331751-21ld2
2330776260,3sleep02331023-21awk0
2440858250,2sleep00-21swapper/00
2399275250,1sleep00-21swapper/00
2387207250,1sleep10-21swapper/11
2360614250,2sleep00-21swapper/00
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional