You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-07 - 19:46

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3s.osadl.org (updated Fri Aug 07, 2026 12:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
339452821270,1sleep03394526-21ssh0
3082418211960,54sleep30-21swapper/33
308235229363,12sleep20-21swapper/22
308245028455,9sleep10-21swapper/11
308265527656,14sleep00-21swapper/00
31074772330,2sleep30-21swapper/33
33080162240,2sleep2421rcuc/22
308283399221,18cyclictest0-21swapper/33
3082833992118,1cyclictest0-21swapper/33
308283399211,16cyclictest3142454-21taskset3
308283399211,16cyclictest0-21swapper/33
308283399208,8cyclictest3503332-21sleep3
308283399208,3cyclictest33142382sleep33
3082833992015,3cyclictest66950irq/132-enp1s0-TxRx-33
308283399201,15cyclictest0-21swapper/33
308283399201,15cyclictest0-21swapper/33
308283399201,14cyclictest0-21swapper/33
35132062191,14sleep03514261-21kthreadcore0
34656132191,3sleep03465627-21ssh0
308283399198,7cyclictest3458941-21turbostat3
308283399194,12cyclictest3291143-21chrt3
3082833991915,2cyclictest3579202-21taskset3
308283399191,15cyclictest0-21swapper/33
308283399191,15cyclictest0-21swapper/33
308283399191,14cyclictest66950irq/132-enp1s0-TxRx-33
308283399191,14cyclictest0-21swapper/33
308283399191,14cyclictest0-21swapper/33
308283399191,14cyclictest0-21swapper/33
308283399191,14cyclictest0-21swapper/33
308283399191,14cyclictest0-21swapper/33
308283399191,14cyclictest0-21swapper/33
34770542181,9sleep03476687-21sendmail-msp0
32700742181,2sleep10-21swapper/11
31589372181,7sleep13158609-21/usr/sbin/munin1
308283399181,5cyclictest66950irq/132-enp1s0-TxRx-33
3082833991815,1cyclictest0-21swapper/33
308283399181,14cyclictest0-21swapper/33
308283399181,14cyclictest0-21swapper/33
308283399181,14cyclictest0-21swapper/33
308283399181,14cyclictest0-21swapper/33
308283399181,14cyclictest0-21swapper/33
308283399181,14cyclictest0-21swapper/33
308283399181,14cyclictest0-21swapper/33
308283399181,13cyclictest0-21swapper/33
308283399181,13cyclictest0-21swapper/33
308283399181,13cyclictest0-21swapper/33
308283399181,13cyclictest0-21swapper/33
308283399180,16cyclictest0-21swapper/33
308283399180,15cyclictest0-21swapper/33
308283399180,15cyclictest0-21swapper/33
32878372171,3sleep20-21swapper/22
308283399172,10cyclictest3163925-21taskset3
308283399171,3cyclictest0-21swapper/33
308283399171,3cyclictest0-21swapper/33
3082833991712,3cyclictest0-21swapper/33
308283399171,13cyclictest0-21swapper/33
308283399171,13cyclictest0-21swapper/33
308283399171,13cyclictest0-21swapper/33
3082833991710,5cyclictest0-21swapper/33
308283399170,15cyclictest0-21swapper/33
308283399170,15cyclictest0-21swapper/33
308283399170,13cyclictest0-21swapper/33
34495462161,3sleep10-21swapper/11
32983292161,11sleep03298340-21ssh0
32436222161,11sleep03243636-21ssh0
32371942160,1sleep00-21swapper/00
308283399167,6cyclictest3179357-21turbostat3
308283399162,2cyclictest0-21swapper/33
308283399161,4cyclictest0-21swapper/33
308283399161,14cyclictest0-21swapper/33
308283399161,14cyclictest0-21swapper/33
3082833991611,3cyclictest0-21swapper/33
308283399161,12cyclictest0-21swapper/33
308283399161,12cyclictest0-21swapper/33
308283399160,13cyclictest0-21swapper/33
35636012151,2sleep00-21swapper/00
34566962151,3sleep03210939-21diskmemload0
34048192151,4sleep13210939-21diskmemload1
33059652151,2sleep00-21swapper/00
32694952150,2sleep20-21swapper/22
32280622151,2sleep10-21swapper/11
31431892151,10sleep03143256-21tune2fs0
308283399151,3cyclictest0-21swapper/33
308283399151,10cyclictest3142931-21chrt3
308283399151,10cyclictest0-21swapper/33
308283399150,13cyclictest0-21swapper/33
308283399150,12cyclictest0-21swapper/33
308283399150,12cyclictest0-21swapper/33
35257542141,8sleep13525919-21sed1
35197872141,9sleep23519800-21ssh2
35017822141,2sleep10-21swapper/11
34834322140,2sleep10-21swapper/11
33904462141,2sleep10-21swapper/11
33506592141,2sleep10-21swapper/11
32994832141,5sleep23299496-21ssh2
32336202141,3sleep13210939-21diskmemload1
31895562140,2sleep00-21swapper/00
31166852141,9sleep03116862-21diskstats0
3082833991412,1cyclictest0-21swapper/33
308283399140,2cyclictest0-21swapper/33
308283399140,12cyclictest0-21swapper/33
35817122130,2sleep20-21swapper/22
35574182131,8sleep20-21swapper/22
34920932130,1sleep20-21swapper/22
34738652131,8sleep23473879-21ssh2
34063122131,2sleep00-21swapper/00
34026912131,3sleep03402706-21ssh0
33431132130,1sleep10-21swapper/11
33428452131,7sleep03210939-21diskmemload0
33139052131,3sleep23313920-21ssh2
32557312131,7sleep03255733-21apt-get0
32440062131,2sleep13210939-21diskmemload1
32260192134,5sleep03226031-21sh0
32180442131,4sleep13218058-21ssh1
32180442131,4sleep13218058-21ssh1
31749302131,8sleep13175494-21cut1
35636132121,7sleep13563628-21ssh1
35464162120,2sleep10-21swapper/11
35294312121,7sleep03529445-21ssh0
35284522121,3sleep23528467-21ssh2
35000222121,7sleep03500036-21ssh0
34948202122,4sleep23494844-21apt-get2
34872342121,7sleep13488841-21tr1
34555282121,7sleep13455538-21ssh1
34484552121,6sleep20-21swapper/22
34303272121,7sleep13430342-21ssh1
34215262121,7sleep23323024-21kworker/u16:1+xprtiod2
34215262121,7sleep23323024-21kworker/u16:1+xprtiod2
34047322121,7sleep23404745-21ssh2
33338202120,7sleep13210939-21diskmemload1
32444942120,2sleep20-21swapper/22
32163412121,5sleep20-21swapper/22
32163412121,5sleep20-21swapper/22
31639042122,4sleep23163922-21apt-get2
31585992121,7sleep23158384-21sendmail-msp2
35800022111,3sleep10-21swapper/11
35665002111,5sleep23568029-21cut2
35455532110,6sleep03210939-21diskmemload0
35352952111,4sleep23210939-21diskmemload2
35351532111,3sleep00-21swapper/00
35040202111,6sleep13504171-21tail1
34859782111,3sleep00-21swapper/00
34830182111,3sleep20-21swapper/22
34681742112,4sleep13468198-21apt-get1
34671182110,6sleep272-21kcompactd02
34643412111,3sleep10-21swapper/11
34492162110,1sleep00-21swapper/00
34136372110,1sleep10-21swapper/11
34063562111,4sleep23210939-21diskmemload2
33940692111,4sleep20-21swapper/22
33843072111,6sleep03384567-21head0
33794402111,5sleep00-21swapper/00
33693262111,2sleep00-21swapper/00
33682282111,4sleep13210939-21diskmemload1
33504882111,3sleep00-21swapper/00
33281542111,6sleep03328958-21kthreadcore0
33232482111,3sleep00-21swapper/00
33058212111,3sleep10-21swapper/11
32795272111,6sleep13279539-21sh1
32367412110,7sleep23236755-21ssh2
32251962110,1sleep20-21swapper/22
31836922111,5sleep13183763-21ntp_states1
31219272111,4sleep13121946-21apt-get1
31091702111,5sleep03109952-21memory0
35804012101,3sleep00-21swapper/00
35709212101,2sleep00-21swapper/00
35557412101,4sleep13210939-21diskmemload1
35551082101,3sleep23210939-21diskmemload2
35461702101,2sleep20-21swapper/22
35309142101,5sleep13530950-21kthreadcore1
35197732101,3sleep10-21swapper/11
35036452101,2sleep20-21swapper/22
34751712101,4sleep03210939-21diskmemload0
34553832101,3sleep20-21swapper/22
34380332101,4sleep23210939-21diskmemload2
34379132101,3sleep10-21swapper/11
34296302101,3sleep00-21swapper/00
34214542101,2sleep00-21swapper/00
34214542101,2sleep00-21swapper/00
34191552101,3sleep10-21swapper/11
34191552101,3sleep10-21swapper/11
33847542101,4sleep23210939-21diskmemload2
33798522102,3sleep13379866-21apt-get1
33794552100,6sleep23379593-21latency_hist2
33672212101,3sleep20-21swapper/22
33596882101,4sleep13210939-21diskmemload1
33584502101,5sleep03358466-21sh0
33433902101,4sleep23343404-21ssh2
33342082100,4sleep23210939-21diskmemload2
33233062100,3sleep10-21swapper/11
33184872101,5sleep23319347-21kthreadcore2
33163782101,5sleep03210939-21diskmemload0
33162942101,3sleep10-21swapper/11
32985562101,3sleep10-21swapper/11
32895742101,4sleep13210939-21diskmemload1
32874952101,2sleep00-21swapper/00
32780072101,6sleep03278233-21ssh0
32644242101,5sleep00-21swapper/00
32613312101,4sleep13210939-21diskmemload1
32609172101,2sleep20-21swapper/22
32532902100,3sleep00-21swapper/00
32526072101,2sleep10-21swapper/11
32113342101,4sleep03211366-21ssh0
32113342101,4sleep03211366-21ssh0
31953832100,4sleep20-21swapper/22
31949962100,2sleep20-21swapper/22
31901512101,3sleep00-21swapper/00
31429252101,4sleep23142928-21apt-get2
31165722101,5sleep20-21swapper/22
31165662100,1sleep10-21swapper/11
31118272101,5sleep13112075-21kthreadcore1
30997332100,6sleep23099874-21switchtime2
30955382101,4sleep13095796-21df1
30951382100,4sleep10-21swapper/11
30835482101,5sleep13083943-21kthreadcore1
3571834290,1sleep10-21swapper/11
3554546291,4sleep03554558-21sh0
3503673291,3sleep00-21swapper/00
3437184291,3sleep03210939-21diskmemload0
3429342290,3sleep20-21swapper/22
3378339291,2sleep10-21swapper/11
3354462291,4sleep23356036-21tr2
3349602291,2sleep20-21swapper/22
3278795291,4sleep23278809-21ssh2
3255235291,2sleep20-21swapper/22
3201181291,3sleep23200415-21sendmail-msp2
3200659290,1sleep00-21swapper/00
3184930291,2sleep10-21swapper/11
3179633291,3sleep03179652-21apt-get0
3179220290,3sleep272-21kcompactd02
3174107291,4sleep03174307-21latency_hist0
3158890291,4sleep03158384-21sendmail-msp0
3148668291,4sleep13149154-21kthreadcore1
3137316291,3sleep03137395-21munin-run0
3102345290,5sleep03104027-21tr0
3091784291,4sleep23093490-21kthreadcore2
3210140281,3sleep13210287-21find1
3199255280,4sleep13199370-21grep1
3185504281,3sleep23186791-21grep2
3184659281,3sleep23184895-21latency_hist2
3169676281,3sleep20-21swapper/22
3157942280,1sleep20-21swapper/22
3153368280,2sleep10-21swapper/11
3137803281,3sleep0205-21jbd2/sda2-80
3132353280,3sleep13132372-21apt-get1
3116281281,3sleep03116360-21munin-run0
3100804280,1sleep10-21swapper/11
3095222281,3sleep03095288-21sendmail-msp0
3205989271,3sleep00-21swapper/00
3205463270,1sleep10-21swapper/11
3169118270,2sleep10-21swapper/11
3168844270,1sleep10-21swapper/11
3142359270,1sleep10-21swapper/11
3137598271,2sleep20-21swapper/22
3132001270,1sleep10-21swapper/11
3127140270,2sleep00-21swapper/00
3112077271,2sleep20-21swapper/22
3106050270,2sleep10-21swapper/11
3105603270,1sleep20-21swapper/22
3095511271,2sleep00-21swapper/00
3087616270,1sleep20-21swapper/22
3210809260,1sleep20-21swapper/22
3200143260,4sleep00-21swapper/00
3194756260,1sleep10-21swapper/11
3178790261,2sleep00-21swapper/00
3157945260,1sleep00-21swapper/00
3152890260,1sleep00-21swapper/00
3149157261,2sleep20-21swapper/22
3132921261,2sleep23133787-21kthreadcore2
3131907260,1sleep20-21swapper/22
3122497261,2sleep20-21swapper/22
3106072261,2sleep23106069-21apt-get2
3086861261,2sleep00-21swapper/00
3168783250,3sleep00-21swapper/00
3142908250,1sleep10-21swapper/11
3126563250,2sleep00-21swapper/00
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional