You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-05 - 02:28

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3s.osadl.org (updated Tue May 05, 2026 00:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2987862212054,61sleep30-21swapper/33
298785128555,11sleep20-21swapper/22
298804628365,12sleep10-21swapper/11
298786427960,14sleep00-21swapper/00
298832799290,18cyclictest0-21swapper/33
298832799261,3cyclictest0-21swapper/33
298832799258,4cyclictest0-21swapper/33
298832799258,4cyclictest0-21swapper/33
2988327992521,2cyclictest0-21swapper/33
298832799250,13cyclictest0-21swapper/33
2988327992111,6cyclictest3325720-21taskset3
298832799210,1cyclictest0-21swapper/33
33499772201,3sleep03118976-21diskmemload0
298832799202,15cyclictest66950irq/132-enp1s0-TxRx-33
2988327992016,2cyclictest0-21swapper/33
2988327992015,2cyclictest0-21swapper/33
2988327992012,5cyclictest0-21swapper/33
298832799201,15cyclictest0-21swapper/33
298832799201,15cyclictest0-21swapper/33
2988327992010,7cyclictest3408188-21taskset3
34000152191,5sleep20-21swapper/22
298832799198,6cyclictest2993566-21taskset3
298832799191,7cyclictest3234807-21taskset3
2988327991915,2cyclictest0-21swapper/33
298832799191,15cyclictest66950irq/132-enp1s0-TxRx-33
298832799191,15cyclictest0-21swapper/33
298832799191,15cyclictest0-21swapper/33
298832799191,15cyclictest0-21swapper/33
298832799191,14cyclictest30703192chrt3
298832799191,14cyclictest0-21swapper/33
298832799191,14cyclictest0-21swapper/33
298832799191,14cyclictest0-21swapper/33
298832799191,14cyclictest0-21swapper/33
298832799191,14cyclictest0-21swapper/33
298832799191,14cyclictest0-21swapper/33
298832799191,14cyclictest0-21swapper/33
2988327991910,5cyclictest34340392sleep33
2988327991815,1cyclictest0-21swapper/33
298832799181,14cyclictest0-21swapper/33
298832799181,14cyclictest0-21swapper/33
298832799181,14cyclictest0-21swapper/33
298832799181,13cyclictest0-21swapper/33
298832799181,13cyclictest0-21swapper/33
298832799181,13cyclictest0-21swapper/33
298832799181,13cyclictest0-21swapper/33
298832799181,13cyclictest0-21swapper/33
298832799181,13cyclictest0-21swapper/33
298832799180,1cyclictest0-21swapper/33
298832799180,15cyclictest0-21swapper/33
34691382171,4sleep03469150-21ssh0
33661632170,1sleep20-21swapper/22
33495632170,3sleep20-21swapper/22
32581892171,3sleep10-21swapper/11
31678452173,4sleep03118976-21diskmemload0
298832799171,4cyclictest0-21swapper/33
2988327991714,1cyclictest0-21swapper/33
298832799171,13cyclictest0-21swapper/33
298832799171,13cyclictest0-21swapper/33
298832799171,13cyclictest0-21swapper/33
298832799170,14cyclictest66950irq/132-enp1s0-TxRx-33
34502902161,3sleep20-21swapper/22
34425972161,2sleep00-21swapper/00
34352772161,2sleep00-21swapper/00
32606422161,10sleep23260863-21df_inode2
32344882161,4sleep03118976-21diskmemload0
32242522161,2sleep00-21swapper/00
31517892161,4sleep13118976-21diskmemload1
298832799167,5cyclictest3044588-21taskset3
2988327991611,4cyclictest0-21swapper/33
298832799161,13cyclictest0-21swapper/33
298832799161,12cyclictest0-21swapper/33
298832799161,12cyclictest0-21swapper/33
298832799160,13cyclictest0-21swapper/33
298832799160,13cyclictest0-21swapper/33
33774362150,6sleep23118976-21diskmemload2
33603522155,4sleep0151ktimers/00
33187132151,3sleep03318725-21ssh0
33008692151,10sleep23118976-21diskmemload2
33008692151,10sleep23118976-21diskmemload2
32898872151,9sleep13289928-21ntpq1
31179572151,5sleep13118012-21gcc1
30334792154,7sleep23033728-21expr2
2988327991512,2cyclictest0-21swapper/33
2988327991511,2cyclictest0-21swapper/33
298832799151,12cyclictest0-21swapper/33
298832799150,13cyclictest0-21swapper/33
298832799150,13cyclictest0-21swapper/33
298832799150,13cyclictest0-21swapper/33
298832799150,12cyclictest0-21swapper/33
298832799150,12cyclictest0-21swapper/33
298832799150,10cyclictest30990772sleep33
34024452148,3sleep03402458-21ssh0
33432122141,8sleep23118976-21diskmemload2
33013652141,8sleep03118976-21diskmemload0
33013652141,8sleep03118976-21diskmemload0
32916632140,1sleep20-21swapper/22
32766282141,9sleep13277079-21dpkg1
31599732141,4sleep23159987-21ssh2
2988327991411,1cyclictest0-21swapper/33
298832799141,11cyclictest0-21swapper/33
298832799140,12cyclictest0-21swapper/33
34273912131,7sleep13118976-21diskmemload1
34254162131,3sleep03118976-21diskmemload0
34169332131,8sleep03118976-21diskmemload0
34091142131,8sleep03118976-21diskmemload0
33870262137,3sleep20-21swapper/22
33591022131,4sleep23359113-21ssh2
33404712131,2sleep13118976-21diskmemload1
32647372131,8sleep13264797-21ntpq1
32594082131,7sleep03118976-21diskmemload0
32275292131,6sleep13227656-21irqstats1
32245732131,8sleep13224587-21ssh1
32169502130,2sleep10-21swapper/11
31991052131,7sleep00-21swapper/00
31570822132,5sleep13157141-21ntpq1
30773592131,8sleep13079067-21tr1
29961712131,8sleep22996388-21/usr/sbin/munin2
29926342130,8sleep22992682-21ntpq2
34533912121,8sleep13453405-21ssh1
34368012121,7sleep23436766-21/usr/sbin/munin2
34367752121,6sleep10-21swapper/11
33944762121,6sleep13394737-21df1
33761562121,6sleep03118976-21diskmemload0
33585312121,3sleep13358538-21ssh1
33520652121,6sleep13118976-21diskmemload1
33271122121,5sleep10-21swapper/11
33099762121,5sleep20-21swapper/22
32259122121,3sleep23225922-21ssh2
31992062121,6sleep13118976-21diskmemload1
31420042121,7sleep13118976-21diskmemload1
31318992121,5sleep13131956-21ntpq1
30230782121,7sleep23024252-21cat2
34515772111,3sleep10-21swapper/11
33919912111,5sleep13118976-21diskmemload1
33331572110,7sleep23333161-21sh2
33277242112,4sleep03118976-21diskmemload0
33274322115,2sleep00-21swapper/00
33246302111,7sleep23118976-21diskmemload2
33189912111,3sleep10-21swapper/11
32763112111,3sleep00-21swapper/00
32658582111,6sleep03265872-21ssh0
32511282111,5sleep20-21swapper/22
32509252111,3sleep10-21swapper/11
32500252111,5sleep00-21swapper/00
32396402111,5sleep03239703-21ntpq0
32353802111,6sleep10-21swapper/11
32332012111,3sleep23233215-21ssh2
32068092115,3sleep03206867-21ssh0
31820062111,5sleep13182060-21ntpq1
31671732113,4sleep2431ktimers/22
31665942111,4sleep10-21swapper/11
31405992111,6sleep03140597-21mailstats0
30930412111,5sleep13093156-21irqstats1
30552862110,7sleep03055818-21grep0
30441112111,6sleep03044202-21tr0
30428592111,4sleep13042915-21ntp_states1
30053422111,5sleep03005391-21ntpq0
34687422101,6sleep23468753-21ssh2
34674972101,4sleep13118976-21diskmemload1
34587342101,5sleep03458744-21ssh0
34530712101,3sleep272-21kcompactd02
34527662101,4sleep03118976-21diskmemload0
34342072101,5sleep13434221-21ssh1
34335382101,5sleep23433549-21ssh2
34256812101,4sleep23425693-21ssh2
34072242101,4sleep23407283-21ssh2
33909882101,2sleep00-21swapper/00
33837572101,4sleep13118976-21diskmemload1
33830582101,4sleep03118976-21diskmemload0
33823512101,3sleep20-21swapper/22
33761252101,5sleep13376135-21ssh1
33652372101,4sleep13365285-21ntpq1
33368562101,5sleep03336940-21ssh0
33352572100,5sleep13118976-21diskmemload1
33118942101,5sleep23312456-21kthreadcore2
33074012101,3sleep10-21swapper/11
32994402101,4sleep13118976-21diskmemload1
32994402101,4sleep13118976-21diskmemload1
32825002100,1sleep10-21swapper/11
32770802100,5sleep23276641-21apt-get2
32747132101,2sleep20-21swapper/22
32597122101,2sleep20-21swapper/22
32409802101,2sleep20-21swapper/22
32178372101,4sleep03118976-21diskmemload0
32151372101,4sleep23215194-21ntpq2
32080072101,3sleep13118976-21diskmemload1
31932802101,3sleep03118976-21diskmemload0
31838672101,4sleep03118976-21diskmemload0
31759442101,4sleep03118976-21diskmemload0
31529102101,4sleep03152939-21apt-get0
31517492101,4sleep23118976-21diskmemload2
31365622101,5sleep23136688-21irqstats2
31353722100,2sleep20-21swapper/22
31254432101,4sleep20-21swapper/22
31243652100,1sleep10-21swapper/11
31126202101,4sleep03112670-21ntpq0
31076412101,6sleep23107867-21unixbench_multi2
31032822101,4sleep13103543-21df_inode1
31018412101,4sleep13101982-21ntpq1
30982182101,4sleep03098342-21perl0
30931572101,5sleep23093192-21kthreadcore2
30763202101,4sleep03076339-21apt-get0
30709962101,4sleep23071248-21df2
30654172101,4sleep23065436-21apt-get2
30588932101,6sleep23058945-21pmu-power2
30431872101,4sleep23043431-21users2
30281002101,3sleep03027587-21apt-get0
30071392100,6sleep23008103-21tr2
29999872101,4sleep03000037-21ntpq0
3416345291,2sleep20-21swapper/22
3403014291,4sleep13403274-21df1
3366684290,5sleep03366695-21ssh0
3289767291,4sleep03289886-21ssh0
3283048291,3sleep03118976-21diskmemload0
3202622290,2sleep20-21swapper/22
3200231291,4sleep23200245-21ssh2
3191889291,3sleep13118976-21diskmemload1
3185943291,2sleep20-21swapper/22
3179938290,6sleep23181692-21cut2
3175623291,4sleep13175636-21ssh1
3169645290,1sleep20-21swapper/22
3135069291,3sleep03118976-21diskmemload0
3124352291,2sleep00-21swapper/00
3087562291,4sleep23088483-21tr2
3083311290,6sleep00-21swapper/00
3076597291,4sleep23076652-21grep2
3070849291,4sleep03070452-21apt-get0
3064115291,4sleep03064168-21modprobe0
3022129290,3sleep00-21swapper/00
3012428291,4sleep13013887-21cat1
3011855291,3sleep03011874-21apt-get0
3006573291,4sleep13006591-21apt1
3001549290,5sleep23001250-21/usr/sbin/munin2
3413830281,3sleep13415402-21latency1
3306917281,3sleep03307020-21ps0
3149371281,3sleep03149381-21ssh0
3115924280,1sleep20-21swapper/22
3108994280,1sleep20-21swapper/22
3096069281,3sleep03096601-21needreboot0
3070081280,1sleep00-21swapper/00
3061708280,1sleep10-21swapper/11
3049972280,4sleep23050770-21grep2
3049972280,4sleep23050770-21grep2
3049694281,3sleep03049817-21fschecks_time0
3049694281,3sleep03049817-21fschecks_time0
3033973281,3sleep13034219-21kthreadcore1
3028135281,3sleep23028389-21sh2
3028113280,2sleep10-21swapper/11
3113892270,2sleep00-21swapper/00
3113435270,1sleep10-21swapper/11
3097712270,4sleep20-21swapper/22
3091830270,1sleep10-21swapper/11
3091368271,3sleep00-21swapper/00
3086361270,4sleep10-21swapper/11
3081641270,1sleep20-21swapper/22
3070974270,2sleep10-21swapper/11
3070395270,1sleep10-21swapper/11
3053347270,3sleep10-21swapper/11
3053347270,3sleep10-21swapper/11
3046084271,2sleep10-21swapper/11
3044378270,2sleep20-21swapper/22
3043746270,1sleep00-21swapper/00
3038375270,1sleep00-21swapper/00
3027293270,1sleep10-21swapper/11
3022171270,1sleep20-21swapper/22
3022042270,1sleep10-21swapper/11
3011581270,1sleep00-21swapper/00
2993595271,3sleep02993607-21cron0
2988501270,2sleep10-21swapper/11
3103827261,2sleep00-21swapper/00
3065028260,1sleep20-21swapper/22
3054735260,4sleep10-21swapper/11
3022797260,3sleep03022970-21gltestperf0
3012020260,1sleep20-21swapper/22
3006207260,1sleep10-21swapper/11
2993612261,2sleep10-21swapper/11
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional