You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-03 - 09:50

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack3slot3s.osadl.org (updated Fri Apr 03, 2026 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2368499211456,54sleep30-21swapper/33
236854628868,11sleep20-21swapper/22
236860927955,18sleep00-21swapper/00
236355827853,12sleep10-21swapper/11
28483802360,2sleep10-21swapper/11
236891399232,17cyclictest27510612sleep33
236891399221,18cyclictest0-21swapper/33
2368913992118,1cyclictest0-21swapper/33
236891399211,16cyclictest66950irq/132-enp1s0-TxRx-33
236891399211,15cyclictest2849428-21chrt3
236891399201,15cyclictest0-21swapper/33
25930862191,5sleep02499651-21diskmemload0
2368913991916,1cyclictest0-21swapper/33
2368913991915,2cyclictest0-21swapper/33
236891399191,2cyclictest0-21swapper/33
236891399191,15cyclictest0-21swapper/33
236891399191,15cyclictest0-21swapper/33
236891399191,15cyclictest0-21swapper/33
236891399191,15cyclictest0-21swapper/33
236891399191,15cyclictest0-21swapper/33
236891399191,15cyclictest0-21swapper/33
236891399191,15cyclictest0-21swapper/33
236891399191,15cyclictest0-21swapper/33
236891399191,14cyclictest2381527-21turbostat3
236891399191,14cyclictest0-21swapper/33
236891399191,14cyclictest0-21swapper/33
236891399190,16cyclictest0-21swapper/33
27446772181,5sleep02744949-21df_inode0
26194192180,9sleep12499651-21diskmemload1
25152022181,3sleep00-21swapper/00
236891399182,11cyclictest2622485-21taskset3
236891399182,10cyclictest2456875-21taskset3
2368913991812,5cyclictest0-21swapper/33
2368913991812,4cyclictest0-21swapper/33
2368913991812,3cyclictest0-21swapper/33
236891399181,14cyclictest2759593-21taskset3
236891399181,14cyclictest0-21swapper/33
236891399181,14cyclictest0-21swapper/33
236891399181,14cyclictest0-21swapper/33
236891399181,14cyclictest0-21swapper/33
236891399181,14cyclictest0-21swapper/33
236891399181,14cyclictest0-21swapper/33
236891399181,14cyclictest0-21swapper/33
236891399181,14cyclictest0-21swapper/33
236891399181,14cyclictest0-21swapper/33
236891399181,14cyclictest0-21swapper/33
236891399181,14cyclictest0-21swapper/33
236891399181,14cyclictest0-21swapper/33
236891399181,14cyclictest0-21swapper/33
236891399181,13cyclictest0-21swapper/33
236891399181,13cyclictest0-21swapper/33
236891399181,13cyclictest0-21swapper/33
236891399181,13cyclictest0-21swapper/33
236891399181,13cyclictest0-21swapper/33
236891399180,14cyclictest0-21swapper/33
28016842175,7sleep02801706-21/usr/sbin/munin0
27258962171,3sleep00-21swapper/00
26106102171,5sleep12499651-21diskmemload1
25442522172,7sleep02241913-21kworker/0:2+events0
25041322171,5sleep02504233-21ntpq0
2368913991715,1cyclictest0-21swapper/33
236891399171,13cyclictest0-21swapper/33
236891399170,14cyclictest0-21swapper/33
236891399170,14cyclictest0-21swapper/33
28313922161,4sleep22499651-21diskmemload2
26717822160,2sleep10-21swapper/11
25875622161,9sleep12499651-21diskmemload1
25868002161,3sleep00-21swapper/00
25171462168,4sleep22517262-21sed2
2368913991613,2cyclictest0-21swapper/33
236891399161,13cyclictest0-21swapper/33
2368913991610,3cyclictest2413611-21taskset3
236891399160,13cyclictest0-21swapper/33
236891399160,13cyclictest0-21swapper/33
236891399160,13cyclictest0-21swapper/33
28493952151,6sleep20-21swapper/22
28379902151,4sleep12499651-21diskmemload1
28117662151,2sleep20-21swapper/22
27933452150,1sleep00-21swapper/00
27331862151,3sleep12499651-21diskmemload1
26545722151,2sleep00-21swapper/00
26022512151,2sleep00-21swapper/00
25764212151,2sleep00-21swapper/00
23964232151,2sleep10-21swapper/11
236891399158,4cyclictest2456271-21chrt3
236891399152,2cyclictest0-21swapper/33
2368913991512,2cyclictest0-21swapper/33
236891399150,12cyclictest0-21swapper/33
236891399150,12cyclictest0-21swapper/33
236891399150,12cyclictest0-21swapper/33
236891399150,12cyclictest0-21swapper/33
236891399150,12cyclictest0-21swapper/33
27882872141,4sleep12788526-21ssh1
27433572140,3sleep12499651-21diskmemload1
27289822140,3sleep02499651-21diskmemload0
27261112141,4sleep12726122-21ssh1
26019202140,6sleep22499651-21diskmemload2
25944842141,9sleep22594498-21ssh2
23873732141,9sleep12387417-21fschecks_count1
236891399147,4cyclictest2494280-21chrt3
2368913991411,2cyclictest0-21swapper/33
28313342131,8sleep12831346-21sh1
27838952131,7sleep22784150-21users2
27509922131,8sleep12499651-21diskmemload1
26984122131,8sleep02698419-21ssh0
26841362131,4sleep12684172-21kthreadcore1
26641192131,2sleep22499651-21diskmemload2
26306492131,2sleep10-21swapper/11
25429292131,3sleep22542943-21ssh2
28609222121,5sleep00-21swapper/00
28550662120,5sleep10-21swapper/11
28462702126,3sleep2421rcuc/22
27970942121,7sleep22797166-21grep2
27862532121,7sleep12499651-21diskmemload1
27774042120,6sleep244-21ksoftirqd/22
27672282121,4sleep00-21swapper/00
27505402121,5sleep20-21swapper/22
27008792120,2sleep12499651-21diskmemload1
26734502121,3sleep22673463-21ssh2
26096672121,8sleep02609851-21wc0
25611672122,4sleep22561193-21apt-get2
25277712121,7sleep12529435-21tr1
25154252121,6sleep12499651-21diskmemload1
24558972121,7sleep22456089-21sed2
24187842120,1sleep10-21swapper/11
24030602121,5sleep02403079-21apt-get0
236891399120,9cyclictest0-21swapper/33
28659222111,5sleep12499651-21diskmemload1
28480682111,7sleep02848082-21ssh0
28214862111,5sleep12499651-21diskmemload1
28041102111,4sleep12499651-21diskmemload1
27540082111,7sleep02755585-21cut0
27351152111,4sleep20-21swapper/22
27217112111,5sleep22722483-21memory2
26821222111,4sleep12499651-21diskmemload1
26490042111,5sleep12649225-21hddtemp_smartct1
26268942111,6sleep02626990-21ps0
26185092111,3sleep00-21swapper/00
26108252111,4sleep22499651-21diskmemload2
26033252111,5sleep12499651-21diskmemload1
25762412111,4sleep22499651-21diskmemload2
25692252110,7sleep02569238-21rm0
25681222111,6sleep12499651-21diskmemload1
25499732111,3sleep20-21swapper/22
25494542111,3sleep10-21swapper/11
25233062111,4sleep12499651-21diskmemload1
25180922111,5sleep02518172-21irqstats0
24948512111,5sleep02495064-21hddtemp_smartct0
24876912111,5sleep22487745-21ntpq2
24876912111,5sleep22487745-21ntpq2
24715722111,5sleep02471620-21ntpq0
24681322111,7sleep12469424-21grep1
24622182111,5sleep22462479-21df_inode2
24555672111,4sleep02455608-21ntpq0
24449622111,5sleep22445011-21ntpq2
24395132111,5sleep22439601-21ntpq2
24233552111,5sleep22423405-21ntpq2
24140062111,4sleep02414266-21df_inode0
24081652110,2sleep10-21swapper/11
28638562101,3sleep22499651-21diskmemload2
28493682101,3sleep00-21swapper/00
28382572101,5sleep22499651-21diskmemload2
28378792101,2sleep00-21swapper/00
28301462101,4sleep02830160-21ssh0
28058392101,4sleep12805863-21apt-get1
27942272101,5sleep22794241-21ssh2
27760742101,3sleep0221rcuc/00
27680532101,4sleep12499651-21diskmemload1
27660322101,4sleep22766130-21ntpq2
27604362101,5sleep22760451-21ssh2
27359642101,3sleep20-21swapper/22
27146522101,4sleep22499651-21diskmemload2
27138722101,4sleep12713930-21ntpq1
27074752101,2sleep00-21swapper/00
26981922101,3sleep20-21swapper/22
26886122100,2sleep20-21swapper/22
26775652101,4sleep02678785-21memory0
26713092100,5sleep02667924-21kworker/u16:3+xprtiod0
26623382101,3sleep12499651-21diskmemload1
26579712101,6sleep02658419-21cut0
26554532101,2sleep20-21swapper/22
26458792100,2sleep00-21swapper/00
26452282101,4sleep12499651-21diskmemload1
26449842101,3sleep22499651-21diskmemload2
26378792101,2sleep00-21swapper/00
26312102101,3sleep20-21swapper/22
26224712101,2sleep20-21swapper/22
26138352101,3sleep20-21swapper/22
25785772101,4sleep12578841-21df1
25749672101,4sleep12499651-21diskmemload1
25581582101,4sleep22499651-21diskmemload2
25568102100,2sleep0171rcu_preempt0
25424462101,3sleep00-21swapper/00
25406012101,4sleep12499651-21diskmemload1
25345512100,1sleep20-21swapper/22
25053342101,4sleep22499651-21diskmemload2
24986542101,4sleep22498700-21ntpq2
24608362101,4sleep12460924-21ntpq1
24512222100,2sleep2171rcu_preempt2
24302182101,4sleep02430237-21apt-get0
24238082101,5sleep02423973-21vmstat0
23817982101,4sleep02381817-21apt-get0
23744762101,2sleep00-21swapper/00
2821741290,3sleep20-21swapper/22
2820928291,3sleep00-21swapper/00
2811483291,4sleep02811497-21ssh0
2783463291,3sleep02783523-21ntpq0
2770901291,4sleep12771112-21ssh1
2753562291,3sleep12499651-21diskmemload1
2741796291,4sleep02741807-21sh0
2714370291,3sleep00-21swapper/00
2701599291,4sleep22702519-21grep2
2698326291,2sleep10-21swapper/11
2683509291,2sleep00-21swapper/00
2682435291,4sleep22499651-21diskmemload2
2637733291,3sleep12499651-21diskmemload1
2584372291,4sleep22584386-21ssh2
2557645290,4sleep12499651-21diskmemload1
2533129291,4sleep02533141-21ssh0
2521821291,2sleep20-21swapper/22
2494561291,3sleep12494580-21apt-get1
2493311291,4sleep02493357-21ntpq0
2482304291,4sleep22482359-21ntpq2
2474022291,4sleep02475753-21grep0
2462763291,5sleep12463825-21cut1
2460931290,4sleep22460980-21ntpq2
2448097291,4sleep12449915-21cut1
2446807291,4sleep02448095-21cat0
2437133291,4sleep02438921-21grep0
2427748291,4sleep12428794-21memory1
2408619291,2sleep20-21swapper/22
2499949280,2sleep10-21swapper/11
2487595281,3sleep02487633-21ntpq0
2487595281,3sleep02487633-21ntpq0
2483579281,3sleep10-21swapper/11
2483579281,3sleep10-21swapper/11
2477869280,2sleep10-21swapper/11
2469425281,3sleep22471249-21kthreadcore2
2451577281,3sleep10-21swapper/11
2412768281,3sleep02412822-21ntpq0
2403082281,3sleep12403304-21expr1
2402669280,1sleep10-21swapper/11
2388057281,3sleep02389758-21grep0
2375069280,1sleep10-21swapper/11
2461044270,3sleep00-21swapper/00
2440870270,2sleep10-21swapper/11
2429823270,1sleep00-21swapper/00
2419294270,2sleep10-21swapper/11
2397708270,2sleep00-21swapper/00
2391935270,1sleep20-21swapper/22
2386630270,1sleep10-21swapper/11
2381795270,1sleep20-21swapper/22
2377538271,3sleep20-21swapper/22
2373863270,1sleep00-21swapper/00
2494296260,3sleep20-21swapper/22
2494166260,2sleep10-21swapper/11
2482254260,2sleep10-21swapper/11
2476801260,1sleep20-21swapper/22
2467186260,1sleep00-21swapper/00
2445620260,1sleep00-21swapper/00
2439716261,2sleep10-21swapper/11
2435029260,4sleep10-21swapper/11
2433402261,2sleep20-21swapper/22
2399819260,2sleep20-21swapper/22
2397376260,1sleep00-21swapper/00
2393230261,2sleep20-21swapper/22
2373841260,4sleep10-21swapper/11
2373837260,1sleep20-21swapper/22
2482416250,2sleep00-21swapper/00
2425235250,1sleep20-21swapper/22
2419028250,1sleep20-21swapper/22
2407791250,1sleep20-21swapper/22
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional