You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-01 - 23:31

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot3s.osadl.org (updated Wed Jul 01, 2026 12:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
295889422810,1sleep12958893-21awk1
2805725212854,69sleep30-21swapper/33
280573029669,12sleep20-21swapper/22
280572928462,12sleep10-21swapper/11
280551728161,14sleep00-21swapper/00
28478742380,2sleep00-21swapper/00
30912812260,2sleep00-21swapper/00
280591999235,15cyclictest3133523-21taskset3
280591999221,17cyclictest0-21swapper/33
31994432211,8sleep03199453-21ssh0
2805919992112,5cyclictest0-21swapper/33
280591999211,16cyclictest3072910-21taskset3
280591999211,16cyclictest0-21swapper/33
280591999211,15cyclictest0-21swapper/33
325638022014,2sleep30-21swapper/33
32401432200,1sleep10-21swapper/11
280591999202,14cyclictest0-21swapper/33
2805919992014,4cyclictest66950irq/132-enp1s0-TxRx-33
280591999201,15cyclictest0-21swapper/33
280591999201,15cyclictest0-21swapper/33
280591999201,14cyclictest0-21swapper/33
280591999199,7cyclictest3197653-21chrt3
2805919991917,1cyclictest0-21swapper/33
2805919991915,2cyclictest0-21swapper/33
280591999191,15cyclictest0-21swapper/33
280591999191,14cyclictest0-21swapper/33
280591999191,14cyclictest0-21swapper/33
280591999191,14cyclictest0-21swapper/33
280591999191,14cyclictest0-21swapper/33
280591999190,17cyclictest0-21swapper/33
280591999190,15cyclictest31506532sleep33
280591999190,14cyclictest2896906-21taskset3
2805919991815,1cyclictest0-21swapper/33
2805919991815,1cyclictest0-21swapper/33
280591999181,4cyclictest31090072chrt3
280591999181,14cyclictest66950irq/132-enp1s0-TxRx-33
280591999181,14cyclictest0-21swapper/33
280591999181,14cyclictest0-21swapper/33
280591999181,14cyclictest0-21swapper/33
280591999181,14cyclictest0-21swapper/33
280591999181,14cyclictest0-21swapper/33
280591999181,14cyclictest0-21swapper/33
280591999181,13cyclictest0-21swapper/33
280591999181,13cyclictest0-21swapper/33
280591999181,13cyclictest0-21swapper/33
280591999180,16cyclictest0-21swapper/33
280591999180,14cyclictest0-21swapper/33
280591999180,14cyclictest0-21swapper/33
280591999180,14cyclictest0-21swapper/33
280591999180,14cyclictest0-21swapper/33
31354852171,12sleep03137126-21grep0
2805919991713,3cyclictest0-21swapper/33
280591999171,13cyclictest0-21swapper/33
280591999171,13cyclictest0-21swapper/33
280591999171,13cyclictest0-21swapper/33
280591999171,12cyclictest0-21swapper/33
280591999170,14cyclictest0-21swapper/33
280591999170,14cyclictest0-21swapper/33
32818782161,4sleep12933777-21diskmemload1
31725322161,3sleep20-21swapper/22
30853962161,7sleep23087050-21grep2
30332652161,11sleep0106713-21snmpd0
30178172161,10sleep13017986-21diskstats1
29136512161,3sleep10-21swapper/11
280591999161,3cyclictest0-21swapper/33
280591999161,13cyclictest0-21swapper/33
280591999161,12cyclictest0-21swapper/33
280591999161,12cyclictest0-21swapper/33
280591999161,11cyclictest0-21swapper/33
280591999160,13cyclictest0-21swapper/33
280591999160,13cyclictest0-21swapper/33
280591999160,12cyclictest0-21swapper/33
32549862151,3sleep02933777-21diskmemload0
31908742151,3sleep02933777-21diskmemload0
31466912151,4sleep03146869-21ssh0
29988352151,3sleep02933777-21diskmemload0
29675662151,2sleep20-21swapper/22
29501962150,3sleep00-21swapper/00
280591999158,4cyclictest2855071-21chrt3
2805919991512,1cyclictest0-21swapper/33
280591999151,12cyclictest0-21swapper/33
280591999150,13cyclictest0-21swapper/33
280591999150,13cyclictest0-21swapper/33
280591999150,13cyclictest0-21swapper/33
280591999150,12cyclictest0-21swapper/33
32811392141,5sleep23281153-21ssh2
32493422144,2sleep2171rcu_preempt2
32330222140,2sleep10-21swapper/11
32145122141,9sleep23214523-21ssh2
31973892140,2sleep10-21swapper/11
31068552140,2sleep20-21swapper/22
30653192141,3sleep03065330-21ssh0
30563932141,4sleep13056404-21ssh1
28979512141,8sleep02898570-21cut0
28768602141,8sleep12876930-21irqstats1
280591999149,3cyclictest0-21swapper/33
280591999141,8cyclictest2824377-21taskset3
280591999140,12cyclictest0-21swapper/33
280591999140,12cyclictest0-21swapper/33
32812062131,3sleep03281217-21ssh0
32725302131,3sleep03272532-21ssh0
32590382133,2sleep20-21swapper/22
32333602131,7sleep02933777-21diskmemload0
31754192131,6sleep00-21swapper/00
31655022131,3sleep13165512-21ssh1
31508602131,5sleep13150980-21diskstats1
30727422130,2sleep12933777-21diskmemload1
30228582131,3sleep23022869-21ssh2
30023872131,8sleep03003960-21cut0
29996332131,5sleep20-21swapper/22
29634512131,7sleep22963531-21ssh2
28344562131,8sleep12834704-21expr1
32649322122,6sleep0106713-21snmpd0
32487032121,5sleep02933777-21diskmemload0
32406832120,1sleep20-21swapper/22
32303662120,8sleep23230377-21ssh2
32250262122,4sleep02933777-21diskmemload0
32167952121,7sleep23216761-21/usr/sbin/munin2
31735752120,2sleep10-21swapper/11
31497022121,7sleep23149716-21ssh2
31251112121,5sleep02933777-21diskmemload0
31188972121,7sleep23120533-21cut2
31157452121,6sleep02933777-21diskmemload0
31089912121,5sleep23109006-21ssh2
30923842120,2sleep13092405-21apt-get1
30821632121,7sleep23082174-21ssh2
30667842121,7sleep13066796-21ssh1
30492452121,6sleep02933777-21diskmemload0
30262702121,2sleep10-21swapper/11
30259522121,2sleep23025977-21apt-get2
30251082121,7sleep03025118-21ssh0
30064652122,4sleep22933777-21diskmemload2
29840972121,7sleep10-21swapper/11
29399342121,7sleep12933777-21diskmemload1
28921052121,6sleep00-21swapper/00
28763762121,7sleep22876478-21grep2
28554122121,4sleep00-21swapper/00
32572692111,3sleep20-21swapper/22
32487162111,6sleep13248728-21ssh1
32133282111,3sleep10-21swapper/11
32052172111,4sleep22933777-21diskmemload2
31901962110,6sleep12933777-21diskmemload1
31894732111,2sleep20-21swapper/22
31656542112,4sleep22933777-21diskmemload2
31326562110,6sleep02933777-21diskmemload0
31084122111,5sleep02933777-21diskmemload0
31075042111,3sleep10-21swapper/11
30980592111,6sleep23098070-21ssh2
30655372111,5sleep22933777-21diskmemload2
30425292111,5sleep13042800-21df_inode1
30418292111,4sleep10-21swapper/11
30156332111,6sleep03015647-21ssh0
29924302111,6sleep02992691-21sed0
29588552110,7sleep02958890-21ls0
29479182111,6sleep22947931-21sh2
29390122111,4sleep02933777-21diskmemload0
29340922110,1sleep20-21swapper/22
29183322112,4sleep12918334-21apt-get1
29084382111,5sleep12908550-21grep1
28816222111,4sleep22881641-21apt-get2
28554132111,5sleep12855673-21/usr/sbin/munin1
32722992101,4sleep12933777-21diskmemload1
32708672101,2sleep20-21swapper/22
32635942101,2sleep10-21swapper/11
32548752101,2sleep10-21swapper/11
32238352101,4sleep02933777-21diskmemload0
32217112100,2sleep10-21swapper/11
32089642101,4sleep02933777-21diskmemload0
32051002101,3sleep00-21swapper/00
31990922101,2sleep20-21swapper/22
31810842101,3sleep22933777-21diskmemload2
31754502101,5sleep13175629-21expr1
31734692101,4sleep02933777-21diskmemload0
31575542100,1sleep00-21swapper/00
31531782100,6sleep23154640-21cat2
31502562101,4sleep13150345-21munin-run1
31399522101,2sleep10-21swapper/11
31399152101,4sleep22933777-21diskmemload2
31269322101,5sleep23128195-21ssh2
31222402101,2sleep10-21swapper/11
31001432101,4sleep02933777-21diskmemload0
30826722100,1sleep00-21swapper/00
30813472101,2sleep10-21swapper/11
30731882101,3sleep02933777-21diskmemload0
30578802101,3sleep00-21swapper/00
30570612101,3sleep20-21swapper/22
30480832101,4sleep22933777-21diskmemload2
30151832101,2sleep20-21swapper/22
30039622101,4sleep12933777-21diskmemload1
29981872101,2sleep10-21swapper/11
29896282100,1sleep10-21swapper/11
29845512101,4sleep22984815-21df2
29835612101,4sleep02933777-21diskmemload0
29815122100,1sleep20-21swapper/22
29591992100,1sleep00-21swapper/00
29588942101,2sleep10-21swapper/11
29565482100,1sleep20-21swapper/22
29472612101,3sleep10-21swapper/11
29235702101,3sleep12923589-21apt-get1
29137562101,5sleep22915345-21kthreadcore2
28454772101,6sleep22846202-21kthreadcore2
28437722101,5sleep02843893-21sendmail_mailst0
28364522101,5sleep22838208-21kthreadcore2
28213752101,4sleep12822596-21memory1
28101302101,4sleep12810183-21ntp_states1
28101302101,4sleep12810183-21ntp_states1
3206605291,4sleep13206617-21rm1
3160728291,5sleep03162379-21cut0
3126220291,4sleep13126931-21kthreadcore1
3114226291,4sleep13114237-21ssh1
3089048290,1sleep10-21swapper/11
3067568291,4sleep23067672-21tr2
3040314291,2sleep00-21swapper/00
3039254291,4sleep23039267-21ssh2
3015506290,4sleep10-21swapper/11
2972773291,4sleep12972784-21ssh1
2967334291,3sleep00-21swapper/00
2893562291,4sleep22895256-21tr2
2886863291,3sleep22886866-21apt-get2
2881773291,3sleep02881913-21diskstats0
2866197291,5sleep22866420-21idleruntime2
2850692291,4sleep12850756-21kernelversion1
2844646290,1sleep20-21swapper/22
2823677291,3sleep22823691-21fwupdmgr2
2811126291,4sleep12811633-21cut1
2811074291,4sleep02811125-21sqv0
2811074291,4sleep02811125-21sqv0
2806106291,4sleep22806363-21df2
2806106291,4sleep22806363-21df2
2933623281,3sleep02933625-21sshd-session0
2933574280,2sleep10-21swapper/11
2918172280,1sleep00-21swapper/00
2902657281,3sleep22902676-21apt-get2
2902452280,3sleep02902424-21sendmail-msp0
2891828281,3sleep12891898-21latency_hist1
2891683280,1sleep00-21swapper/00
2881225280,2sleep00-21swapper/00
2872488281,3sleep22874162-21cut2
2865585281,3sleep21-21systemd2
2853970280,3sleep02854143-21strings0
2844920281,2sleep12844939-21apt-get1
2830337281,3sleep12831987-21grep1
2829221280,2sleep00-21swapper/00
2828556280,1sleep00-21swapper/00
2827211281,3sleep22827888-21cat2
2813969281,3sleep02814039-21kthreadcore0
2930250271,3sleep20-21swapper/22
2927265270,2sleep02927556-21strings0
2924144271,3sleep20-21swapper/22
2922257270,4sleep20-21swapper/22
2908552271,2sleep20-21swapper/22
2906853270,4sleep10-21swapper/11
2902076270,1sleep20-21swapper/22
2898572271,2sleep10-21swapper/11
2892673271,3sleep12892106-21/usr/sbin/munin1
2886325270,1sleep10-21swapper/11
2871681271,2sleep10-21swapper/11
2871118270,2sleep00-21swapper/00
2870593270,1sleep10-21swapper/11
2864958271,2sleep00-21swapper/00
2834435270,2sleep00-21swapper/00
2834114270,1sleep20-21swapper/22
2828626270,1sleep10-21swapper/11
2823470270,1sleep00-21swapper/00
2912736260,1sleep00-21swapper/00
2866720261,2sleep00-21swapper/00
2865255260,1sleep10-21swapper/11
2859975260,1sleep20-21swapper/22
2854996260,1sleep20-21swapper/22
2813711261,2sleep20-21swapper/22
2913102250,1sleep00-21swapper/00
2839670250,1sleep10-21swapper/11
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional