You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-06 - 11:48

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack3slot3s.osadl.org (updated Mon Jul 06, 2026 00:43:53)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
327950322890,1sleep03279502-21ssh0
2972302214048,87sleep30-21swapper/33
297242928854,7sleep00-21swapper/00
297219528858,12sleep20-21swapper/22
297231028253,10sleep10-21swapper/11
31672882333,14sleep03167315-21/usr/sbin/munin0
31804012310,2sleep10-21swapper/11
297267699241,19cyclictest0-21swapper/33
2972676992319,1cyclictest0-21swapper/33
2972676992217,2cyclictest0-21swapper/33
2972676992117,2cyclictest0-21swapper/33
33922522201,6sleep13392286-21ssh1
32975842201,2sleep10-21swapper/11
297267699209,7cyclictest33652562sleep33
2972676992017,2cyclictest0-21swapper/33
2972676992015,2cyclictest0-21swapper/33
32174992191,7sleep13218073-21cut1
31252372190,6sleep13125250-21sh1
297267699194,3cyclictest66950irq/132-enp1s0-TxRx-33
2972676991916,1cyclictest0-21swapper/33
2972676991915,2cyclictest0-21swapper/33
2972676991915,2cyclictest0-21swapper/33
2972676991912,4cyclictest0-21swapper/33
297267699191,15cyclictest3199478-21chrt3
2972676991911,5cyclictest3079371-21taskset3
297267699191,14cyclictest0-21swapper/33
297267699191,14cyclictest0-21swapper/33
297267699191,14cyclictest0-21swapper/33
297267699191,14cyclictest0-21swapper/33
297267699191,14cyclictest0-21swapper/33
297267699191,14cyclictest0-21swapper/33
297267699191,14cyclictest0-21swapper/33
297267699191,14cyclictest0-21swapper/33
32337742181,7sleep13233864-21grep1
297267699182,12cyclictest0-21swapper/33
297267699181,5cyclictest0-21swapper/33
2972676991815,1cyclictest0-21swapper/33
2972676991815,1cyclictest0-21swapper/33
2972676991814,2cyclictest0-21swapper/33
2972676991813,3cyclictest66950irq/132-enp1s0-TxRx-33
2972676991813,2cyclictest0-21swapper/33
2972676991811,4cyclictest3016416-21taskset3
297267699181,14cyclictest0-21swapper/33
297267699181,14cyclictest0-21swapper/33
297267699181,14cyclictest0-21swapper/33
297267699181,14cyclictest0-21swapper/33
297267699181,13cyclictest0-21swapper/33
297267699181,13cyclictest0-21swapper/33
297267699181,13cyclictest0-21swapper/33
297267699180,16cyclictest0-21swapper/33
34475462171,4sleep13100407-21diskmemload1
34375742171,10sleep23437654-21ntp_states2
33718592171,3sleep00-21swapper/00
33467402171,3sleep10-21swapper/11
33320482171,3sleep00-21swapper/00
32493252171,5sleep03100407-21diskmemload0
31977632171,5sleep23100407-21diskmemload2
2972676991714,1cyclictest0-21swapper/33
2972676991713,1cyclictest0-21swapper/33
297267699171,13cyclictest3248243-21taskset3
297267699171,13cyclictest0-21swapper/33
297267699171,13cyclictest0-21swapper/33
297267699171,13cyclictest0-21swapper/33
297267699171,12cyclictest0-21swapper/33
2972676991710,5cyclictest0-21swapper/33
297267699170,14cyclictest0-21swapper/33
297267699170,14cyclictest0-21swapper/33
297267699170,14cyclictest0-21swapper/33
297267699170,14cyclictest0-21swapper/33
297267699170,13cyclictest0-21swapper/33
34483452161,11sleep03448347-21ssh0
33475442160,2sleep00-21swapper/00
33212792161,3sleep13100407-21diskmemload1
32734672160,3sleep10-21swapper/11
32732172161,2sleep00-21swapper/00
32315662162,5sleep13100407-21diskmemload1
30116112161,10sleep13011615-21find1
297267699166,6cyclictest3053266-21turbostat3
297267699162,9cyclictest3058783-21taskset3
297267699161,9cyclictest3064019-21taskset3
2972676991614,1cyclictest0-21swapper/33
2972676991613,2cyclictest0-21swapper/33
2972676991613,1cyclictest0-21swapper/33
2972676991610,4cyclictest33155952sleep33
297267699160,13cyclictest0-21swapper/33
297267699160,13cyclictest0-21swapper/33
33630382151,4sleep13100407-21diskmemload1
33630382151,4sleep13100407-21diskmemload1
32738442151,9sleep23100407-21diskmemload2
31896362151,9sleep10-21swapper/11
31894162152,5sleep23100407-21diskmemload2
31650942151,2sleep00-21swapper/00
31237602151,4sleep23100407-21diskmemload2
2972676991512,1cyclictest0-21swapper/33
2972676991511,2cyclictest0-21swapper/33
297267699151,12cyclictest0-21swapper/33
297267699151,11cyclictest2995080-21chrt3
297267699150,12cyclictest0-21swapper/33
34228612141,5sleep23422875-21ssh2
34156802140,3sleep23100407-21diskmemload2
34057652141,8sleep00-21swapper/00
33141552140,3sleep00-21swapper/00
33061802140,2sleep20-21swapper/22
32417212141,8sleep13100407-21diskmemload1
32144432141,4sleep23100407-21diskmemload2
31499092141,3sleep20-21swapper/22
31399042141,4sleep13139918-21ssh1
29778962141,8sleep02978118-21munin-node0
297267699141,9cyclictest2972879-21chrt3
2972676991412,1cyclictest0-21swapper/33
33908092130,1sleep20-21swapper/22
33838492131,8sleep13381441-21sendmail-msp1
33669702131,8sleep13368607-21grep1
33621982131,8sleep23362211-21ssh2
33621982131,8sleep23362211-21ssh2
33565122131,6sleep13356549-21munin-run1
33565122131,6sleep13356549-21munin-run1
33299032131,2sleep03100407-21diskmemload0
33049422130,3sleep03100407-21diskmemload0
32898962131,8sleep13289905-21sh1
32571362131,8sleep13257149-21ssh1
31586982131,2sleep20-21swapper/22
31433302131,6sleep13144884-21cut1
31415882130,2sleep03100407-21diskmemload0
31314982131,7sleep23100407-21diskmemload2
31156832131,6sleep03100407-21diskmemload0
30901992130,9sleep13090196-21apt-get1
30891052131,8sleep23089256-21grep2
30698002131,5sleep03070748-21kthreadcore0
30269502130,1sleep10-21swapper/11
297267699130,11cyclictest0-21swapper/33
34068232121,3sleep23406831-21ssh2
33961082121,6sleep03396383-21grep0
33705342121,7sleep23370547-21rm2
33238452122,5sleep23323841-21apt-get2
33156472121,2sleep03315660-21ssh0
33133472121,7sleep23313359-21rm2
32635602121,6sleep23100407-21diskmemload2
32586672121,7sleep03260158-21tr0
32395822121,7sleep014-21ksoftirqd/00
31903892121,6sleep00-21swapper/00
31744602121,7sleep23174474-21ssh2
31650032120,2sleep10-21swapper/11
31145082121,7sleep23100407-21diskmemload2
30587562121,7sleep13058773-21basename1
30480752121,7sleep23048061-21sendmail-msp2
34173252111,4sleep03417543-21hddtemp_smartct0
34088332111,4sleep13408855-21apt-get1
33806072110,2sleep00-21swapper/00
33735572111,4sleep23373570-21apt-get2
33291092115,3sleep10-21swapper/11
33168562111,6sleep23100407-21diskmemload2
33142162111,4sleep10-21swapper/11
32906862111,4sleep23290710-21apt-get2
32877732110,7sleep03287784-21ssh0
32805622110,6sleep13100407-21diskmemload1
32496482111,4sleep23100407-21diskmemload2
32305552110,2sleep03230565-21ssh0
32229582111,7sleep03222965-21ssh0
31928802111,5sleep03194367-21ps0
31573632111,3sleep10-21swapper/11
31476102110,1sleep00-21swapper/00
31091542111,4sleep13109169-21ssh1
31056002111,5sleep13100407-21diskmemload1
31049422111,3sleep20-21swapper/22
30849372111,4sleep03084955-21apt-get0
30683082111,6sleep13063997-21/usr/sbin/munin1
30640132111,4sleep23064016-21apt-get2
30380572111,5sleep23038211-21grep2
30116172111,4sleep23011634-21apt-get2
29958892111,5sleep02995608-21apt-get0
34395722100,1sleep00-21swapper/00
34333662100,1sleep10-21swapper/11
34292322101,5sleep03429380-21ntp_192.168.1140
34169902103,2sleep10-21swapper/11
34135232101,4sleep03100407-21diskmemload0
34064252101,1sleep10-21swapper/11
33922122101,2sleep20-21swapper/22
33907852101,3sleep00-21swapper/00
33796262101,4sleep13100407-21diskmemload1
33645062101,4sleep03100407-21diskmemload0
33645062101,4sleep03100407-21diskmemload0
33545872101,2sleep00-21swapper/00
33545872101,2sleep00-21swapper/00
33534652100,2sleep20-21swapper/22
33534652100,2sleep20-21swapper/22
33471902100,1sleep23347201-21ssh2
33396942101,5sleep20-21swapper/22
32957702101,4sleep03100407-21diskmemload0
32868562101,3sleep20-21swapper/22
32628732101,4sleep13100407-21diskmemload1
32570782101,5sleep03257095-21rm0
32470402100,3sleep23247043-21ssh2
32379522101,2sleep20-21swapper/22
32262892101,5sleep23227881-21kthreadcore2
32244812101,3sleep20-21swapper/22
32144562101,4sleep03100407-21diskmemload0
32086952101,4sleep13100407-21diskmemload1
32054502101,3sleep10-21swapper/11
32029292101,5sleep23204452-21latency2
31815952101,4sleep23100407-21diskmemload2
31653662100,1sleep20-21swapper/22
31571222101,3sleep00-21swapper/00
31348932101,6sleep23136252-21cut2
31334752101,4sleep13100407-21diskmemload1
31311862101,4sleep03100407-21diskmemload0
31175642101,4sleep03117589-21apt-get0
30435412101,5sleep13043642-21tr1
30430152101,4sleep23042739-21apt-get2
30322602101,4sleep03032316-21munin-run0
3447185291,2sleep20-21swapper/22
3430128290,3sleep13100407-21diskmemload1
3425454291,4sleep23425588-21diskstats2
3337152290,4sleep10-21swapper/11
3303449290,5sleep13303681-21timerwakeupswit1
3280185291,2sleep20-21swapper/22
3204631292,4sleep03204714-21/usr/sbin/munin0
3192135291,3sleep13191796-21apt-get1
3181940290,2sleep00-21swapper/00
3167316291,3sleep13167320-21apt-get1
3108548291,2sleep00-21swapper/00
3100315290,1sleep20-21swapper/22
3095434291,4sleep03095690-21df0
3095137291,4sleep03095337-21sed0
3072444291,4sleep23073147-21ls2
3069255291,4sleep13069516-21df_inode1
3068158291,5sleep03068303-21threads0
3057610291,5sleep13057754-21mailstats1
3053548291,3sleep03053544-21apt-get0
3053251291,4sleep03053308-21pluginstate0
3033274290,5sleep03034891-21grep0
3031369290,1sleep20-21swapper/22
3027858291,4sleep13028199-21cat1
3020707291,4sleep23020843-21netstat2
2985403290,3sleep10-21swapper/11
2973144291,4sleep12973269-21/usr/sbin/munin1
3095691280,1sleep10-21swapper/11
3062945281,3sleep03063092-21find0
3058775280,2sleep20-21swapper/22
3037791281,3sleep03037838-21grep0
3022086280,4sleep23022163-21expr2
3022059280,2sleep0171rcu_preempt0
3007679281,3sleep03009345-21grep0
3005899280,1sleep00-21swapper/00
3003306280,5sleep13004975-21/usr/sbin/munin1
3000858281,3sleep21-21systemd2
2990353281,3sleep01-21systemd0
2986252281,3sleep22987493-21kthreadcore2
2978133281,3sleep12979601-21kthreadcore1
2972857280,2sleep20-21swapper/22
3085762271,2sleep10-21swapper/11
3084667271,2sleep20-21swapper/22
3079091270,1sleep20-21swapper/22
3075313271,2sleep00-21swapper/00
3053808270,3sleep20-21swapper/22
3047635270,1sleep00-21swapper/00
3042563270,1sleep10-21swapper/11
3036867271,3sleep23037111-21/usr/sbin/munin2
3017612271,3sleep03019219-21kthreadcore0
3016393270,1sleep00-21swapper/00
3006392270,2sleep20-21swapper/22
3006375271,2sleep10-21swapper/11
3001429271,2sleep23001643-21smartctl2
3000740270,1sleep10-21swapper/11
2993833271,2sleep20-21swapper/22
2984910270,1sleep20-21swapper/22
2977584270,1sleep00-21swapper/00
3094867260,1sleep20-21swapper/22
3080529261,2sleep00-21swapper/00
3079974261,2sleep10-21swapper/11
3048294260,1sleep10-21swapper/11
3037199260,1sleep10-21swapper/11
3021025261,2sleep10-21swapper/11
2992137260,2sleep10-21swapper/11
2991196250,2sleep00-21swapper/00
3074745240,1sleep10-21swapper/11
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional