You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-24 - 14:39

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack3slot3s.osadl.org (updated Wed Jun 24, 2026 12:43:53)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2964916211859,54sleep30-21swapper/33
296504129060,11sleep20-21swapper/22
296473028953,11sleep10-21swapper/11
296024228354,7sleep00-21swapper/00
33168112231,7sleep23316824-21sh2
296521399231,17cyclictest3233807-21taskset3
296521399222,16cyclictest0-21swapper/33
2965213992111,6cyclictest3123809-21taskset3
2965213992017,1cyclictest0-21swapper/33
296521399201,15cyclictest66950irq/132-enp1s0-TxRx-33
296521399201,15cyclictest66950irq/132-enp1s0-TxRx-33
296521399201,15cyclictest0-21swapper/33
33162152192,4sleep03092905-21diskmemload0
296521399191,14cyclictest0-21swapper/33
296521399191,14cyclictest0-21swapper/33
296521399191,14cyclictest0-21swapper/33
296521399191,14cyclictest0-21swapper/33
32574382181,6sleep20-21swapper/22
30200172181,6sleep13020140-21diskstats1
296521399184,10cyclictest0-21swapper/33
296521399183,13cyclictest66950irq/132-enp1s0-TxRx-33
296521399181,15cyclictest0-21swapper/33
296521399181,15cyclictest0-21swapper/33
296521399181,14cyclictest3382235-21taskset3
2965213991811,4cyclictest0-21swapper/33
296521399181,14cyclictest0-21swapper/33
296521399181,14cyclictest0-21swapper/33
296521399181,14cyclictest0-21swapper/33
296521399181,14cyclictest0-21swapper/33
296521399181,13cyclictest0-21swapper/33
296521399181,13cyclictest0-21swapper/33
296521399180,15cyclictest0-21swapper/33
296521399180,14cyclictest0-21swapper/33
33423832171,3sleep20-21swapper/22
33423832171,3sleep20-21swapper/22
32256922171,3sleep10-21swapper/11
31900582171,3sleep00-21swapper/00
31268102171,3sleep00-21swapper/00
31007762171,2sleep00-21swapper/00
2965213991711,3cyclictest0-21swapper/33
2965213991711,3cyclictest0-21swapper/33
2965213991711,3cyclictest0-21swapper/33
296521399171,13cyclictest0-21swapper/33
296521399171,13cyclictest0-21swapper/33
296521399171,13cyclictest0-21swapper/33
296521399171,13cyclictest0-21swapper/33
296521399171,13cyclictest0-21swapper/33
296521399171,13cyclictest0-21swapper/33
296521399171,13cyclictest0-21swapper/33
296521399171,13cyclictest0-21swapper/33
296521399171,13cyclictest0-21swapper/33
296521399171,13cyclictest0-21swapper/33
296521399171,12cyclictest0-21swapper/33
296521399170,14cyclictest66950irq/132-enp1s0-TxRx-33
296521399170,14cyclictest0-21swapper/33
296521399170,14cyclictest0-21swapper/33
296521399170,14cyclictest0-21swapper/33
296521399170,13cyclictest0-21swapper/33
296521399170,13cyclictest0-21swapper/33
33902192161,4sleep03092905-21diskmemload0
32746382160,2sleep20-21swapper/22
31435252161,8sleep10-21swapper/11
31435252161,8sleep10-21swapper/11
296521399168,5cyclictest3082408-21turbostat3
2965213991613,1cyclictest0-21swapper/33
296521399161,14cyclictest0-21swapper/33
296521399161,12cyclictest0-21swapper/33
2965213991610,4cyclictest0-21swapper/33
296521399160,14cyclictest0-21swapper/33
296521399160,13cyclictest0-21swapper/33
296521399160,13cyclictest0-21swapper/33
296521399160,13cyclictest0-21swapper/33
34075292151,4sleep23407540-21ssh2
33981472150,7sleep23398159-21ssh2
33820982150,2sleep10-21swapper/11
33574412151,4sleep13357453-21ssh1
32981252151,4sleep13092905-21diskmemload1
32899352151,4sleep13092905-21diskmemload1
32412092151,9sleep23092905-21diskmemload2
31838032151,6sleep13183815-21sh1
31247072158,4sleep2431ktimers/22
31170052151,2sleep10-21swapper/11
31071942151,2sleep00-21swapper/00
31006012151,8sleep10-21swapper/11
296521399157,5cyclictest30670092chrt3
2965213991513,1cyclictest0-21swapper/33
2965213991510,3cyclictest0-21swapper/33
296521399150,13cyclictest0-21swapper/33
296521399150,13cyclictest0-21swapper/33
296521399150,13cyclictest0-21swapper/33
296521399150,12cyclictest0-21swapper/33
33090762141,3sleep23092905-21diskmemload2
32988872141,9sleep03298898-21ssh0
32649792141,3sleep03264990-21ssh0
32417252141,9sleep13092905-21diskmemload1
32060042141,4sleep13206083-21cut1
31996942141,9sleep23199708-21ssh2
31488222141,8sleep03092905-21diskmemload0
31488222141,8sleep03092905-21diskmemload0
31350682141,3sleep13092905-21diskmemload1
31190652141,10sleep00-21swapper/00
31096742141,9sleep23109677-21cron2
296521399145,4cyclictest30146622sleep33
296521399141,2cyclictest29992002sleep33
296521399141,2cyclictest29992002sleep33
2965213991410,2cyclictest0-21swapper/33
296521399140,2cyclictest0-21swapper/33
296521399140,2cyclictest0-21swapper/33
34241292131,5sleep03424138-21pool-udisksd0
34166422131,8sleep13416654-21ssh1
34150662131,5sleep20-21swapper/22
33636292131,8sleep13363714-21cut1
33564212131,3sleep23356425-21ssh2
33334042131,7sleep13092905-21diskmemload1
31651812130,1sleep20-21swapper/22
31407242131,3sleep03140734-21ssh0
296521399138,3cyclictest3066521-21chrt3
296521399130,2cyclictest30140862sleep33
296521399130,1cyclictest0-21swapper/33
34413182121,2sleep20-21swapper/22
33927512121,5sleep13392786-21ssh1
33596522121,5sleep23359786-21diskstats2
33507912121,7sleep03350803-21ssh0
33396932120,1sleep00-21swapper/00
33396932120,1sleep00-21swapper/00
33242062121,7sleep13324216-21ssh1
32855722121,7sleep23286275-21ssh2
32758522120,2sleep03092905-21diskmemload0
32344872120,2sleep10-21swapper/11
31910782120,2sleep20-21swapper/22
31747962121,5sleep20-21swapper/22
31655782121,2sleep03165588-21ssh0
31646502124,4sleep13164759-21sensors_temp1
31595202121,5sleep10-21swapper/11
31560382121,7sleep03151947-21/usr/sbin/munin0
31202192121,7sleep13121893-21ssh1
31173002121,6sleep23092905-21diskmemload2
30777452121,5sleep13077955-21hddtemp_smartct1
30343012121,6sleep13034371-21ntp_states1
29989402121,5sleep22999199-21df_inode2
29989402121,5sleep22999199-21df_inode2
29889452121,7sleep12989544-21cut1
34312062110,1sleep13092905-21diskmemload1
34061192111,2sleep1171rcu_preempt1
34059652111,3sleep00-21swapper/00
33976622112,5sleep0151ktimers/00
33902302111,5sleep13390244-21ssh1
33749722110,1sleep10-21swapper/11
33594652111,4sleep03359490-21apt-get0
33513992113,4sleep00-21swapper/00
33502422111,3sleep20-21swapper/22
33326712111,6sleep23332684-21ssh2
33185242111,4sleep23092905-21diskmemload2
33088892111,4sleep03092905-21diskmemload0
33031892111,6sleep13304822-21kthreadcore1
32583532110,7sleep13258367-21ssh1
32495942111,3sleep20-21swapper/22
32461342111,5sleep13247378-21memory1
32421422110,1sleep00-21swapper/00
32193742111,6sleep03220907-21cut0
32159412111,4sleep13092905-21diskmemload1
32023802112,4sleep03202453-21irqstats0
31760362111,3sleep10-21swapper/11
31738822111,4sleep03092905-21diskmemload0
31403122111,4sleep23092905-21diskmemload2
31319872110,4sleep20-21swapper/22
30766152111,5sleep03076799-21users0
29880452110,1sleep10-21swapper/11
29766702111,5sleep12976728-21ntp_states1
34346092101,4sleep13434744-21perl1
34328012101,2sleep20-21swapper/22
34304222101,4sleep03092905-21diskmemload0
34248952101,4sleep13092905-21diskmemload1
34159592101,4sleep03092905-21diskmemload0
33839572101,3sleep20-21swapper/22
33811932100,1sleep00-21swapper/00
33728362101,3sleep00-21swapper/00
33324022101,4sleep03092905-21diskmemload0
33178342101,5sleep10-21swapper/11
32983192100,4sleep20-21swapper/22
32907672101,3sleep03092905-21diskmemload0
32817132101,4sleep03092905-21diskmemload0
32580612101,4sleep03092905-21diskmemload0
32505062101,3sleep00-21swapper/00
32319182100,2sleep20-21swapper/22
32314632100,1sleep00-21swapper/00
32150682101,2sleep00-21swapper/00
32081962101,2sleep20-21swapper/22
32006702101,6sleep03200680-21ssh0
31985372100,1sleep10-21swapper/11
31915822101,2sleep10-21swapper/11
31832492101,3sleep20-21swapper/22
31768982101,4sleep03176910-21ssh0
31598702101,2sleep20-21swapper/22
31444702100,6sleep23146019-21grep2
31444702100,6sleep23146019-21grep2
31089882100,1sleep10-21swapper/11
30976142101,3sleep23092905-21diskmemload2
30709852101,1sleep2171rcu_preempt2
30669822101,2sleep00-21swapper/00
30188582101,5sleep03019050-21sed0
30112962101,2sleep20-21swapper/22
29975242101,5sleep02997663-21ntp_192.168.1140
29975242101,5sleep02997663-21ntp_192.168.1140
29779182101,4sleep22977937-21apt-get2
29654202101,4sleep22965422-21apt-get2
3439140291,4sleep03092905-21diskmemload0
3417630291,2sleep20-21swapper/22
3384354291,2sleep20-21swapper/22
3374278291,2sleep20-21swapper/22
3343062291,3sleep10-21swapper/11
3341758290,2sleep10-21swapper/11
3341758290,2sleep10-21swapper/11
3282968291,2sleep20-21swapper/22
3282099291,4sleep13282110-21ssh1
3273552291,2sleep10-21swapper/11
3264857291,2sleep20-21swapper/22
3259913291,2sleep10-21swapper/11
3225657291,4sleep23225667-21ssh2
3210078291,2sleep20-21swapper/22
3131639291,2sleep10-21swapper/11
3115167290,1sleep00-21swapper/00
3092885291,4sleep23093152-21sed2
3092067291,4sleep03092207-21cat0
3083237291,4sleep13083781-21cut1
3082395291,4sleep03082613-21latency_hist0
3072242291,4sleep13072452-21expr1
3040797291,3sleep23040816-21apt-get2
3040505291,5sleep03040619-21latency_hist0
3030945291,4sleep23032539-21tr2
3026457291,4sleep13028121-21cut1
3004676291,4sleep23005219-21cut2
2983433291,4sleep02983536-21perl0
2969636291,4sleep02969843-21cat0
3323049280,1sleep00-21swapper/00
3063735280,4sleep03065467-21grep0
3056503281,2sleep10-21swapper/11
3055858280,1sleep10-21swapper/11
3042578281,3sleep03044333-21grep0
3040425280,1sleep20-21swapper/22
3034982280,1sleep00-21swapper/00
3020424281,3sleep03021842-21kthreadcore0
3009360280,2sleep00-21swapper/00
3008004280,1sleep10-21swapper/11
3003750281,3sleep10-21swapper/11
3003750281,3sleep10-21swapper/11
3003565280,2sleep00-21swapper/00
3003565280,2sleep00-21swapper/00
2993649280,2sleep10-21swapper/11
2993649280,2sleep10-21swapper/11
2983172280,2sleep20-21swapper/22
2973438280,5sleep22974757-21kthreadcore2
3083782271,3sleep23085423-21kthreadcore2
3082203270,3sleep20-21swapper/22
3046737271,2sleep03048130-21kthreadcore0
3046311270,1sleep10-21swapper/11
3045395270,1sleep10-21swapper/11
3025357271,3sleep23025470-21fschecks_time2
3020141271,3sleep20-21swapper/22
3015162272,2sleep23016338-21grep2
3009933271,3sleep13011294-21kthreadcore1
3008848270,1sleep00-21swapper/00
2994831271,2sleep20-21swapper/22
2994831271,2sleep20-21swapper/22
2993224270,1sleep20-21swapper/22
2988384270,1sleep00-21swapper/00
2979168270,3sleep02980841-21cut0
2977706270,3sleep12977692-21sendmail-msp1
2970334270,2sleep10-21swapper/11
3092669260,4sleep10-21swapper/11
3082686260,2sleep00-21swapper/00
3066390260,4sleep20-21swapper/22
3056799260,2sleep00-21swapper/00
3051823261,2sleep00-21swapper/00
3036193261,2sleep10-21swapper/11
3025094260,2sleep03025090-21apt-get0
3014590260,1sleep10-21swapper/11
3074350250,1sleep20-21swapper/22
3071655250,3sleep10-21swapper/11
3061617250,2sleep10-21swapper/11
3056525250,1sleep20-21swapper/22
3055450250,2sleep20-21swapper/22
3050664251,2sleep20-21swapper/22
2977449250,3sleep00-21swapper/00
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional