You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-26 - 22:32

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3s.osadl.org (updated Sun Apr 26, 2026 12:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
206381922880,2sleep02063817-21ntp_states0
1867230211857,56sleep30-21swapper/33
186711928756,10sleep10-21swapper/11
186645028452,10sleep20-21swapper/22
186719427654,16sleep00-21swapper/00
21953252221,16sleep02195336-21sshd0
1867603992215,4cyclictest2322667-21taskset3
186760399221,17cyclictest1918525-21taskset3
186760399221,17cyclictest0-21swapper/33
186760399221,16cyclictest0-21swapper/33
186760399221,15cyclictest2300817-21chrt3
1867603992118,2cyclictest0-21swapper/33
186760399211,17cyclictest0-21swapper/33
22540022201,8sleep02254147-21/usr/sbin/munin0
20982502201,6sleep12098309-21ntpq1
1867603992017,1cyclictest0-21swapper/33
186760399201,16cyclictest66950irq/132-enp1s0-TxRx-33
186760399201,16cyclictest2021857-21taskset3
186760399201,15cyclictest66950irq/132-enp1s0-TxRx-33
186760399201,15cyclictest1952308-21taskset3
186760399200,17cyclictest66950irq/132-enp1s0-TxRx-33
186760399200,17cyclictest0-21swapper/33
186760399200,17cyclictest0-21swapper/33
19167032191,6sleep11916751-21ntpq1
1867603991915,2cyclictest0-21swapper/33
1867603991915,2cyclictest0-21swapper/33
186760399191,15cyclictest66950irq/132-enp1s0-TxRx-33
186760399191,14cyclictest2212015-21taskset3
186760399191,14cyclictest0-21swapper/33
186760399191,14cyclictest0-21swapper/33
186760399191,14cyclictest0-21swapper/33
186760399191,14cyclictest0-21swapper/33
22210842180,1sleep10-21swapper/11
20220492181,3sleep20-21swapper/22
19014472182,7sleep11901432-31find1
1867603991814,2cyclictest66950irq/132-enp1s0-TxRx-33
1867603991813,3cyclictest0-21swapper/33
1867603991813,3cyclictest0-21swapper/33
1867603991812,4cyclictest0-21swapper/33
1867603991811,4cyclictest2013393-21taskset3
186760399181,14cyclictest0-21swapper/33
186760399181,14cyclictest0-21swapper/33
186760399181,14cyclictest0-21swapper/33
186760399181,14cyclictest0-21swapper/33
186760399181,14cyclictest0-21swapper/33
186760399181,14cyclictest0-21swapper/33
186760399181,14cyclictest0-21swapper/33
186760399181,14cyclictest0-21swapper/33
186760399181,14cyclictest0-21swapper/33
186760399181,14cyclictest0-21swapper/33
186760399181,14cyclictest0-21swapper/33
186760399180,16cyclictest0-21swapper/33
186760399180,14cyclictest0-21swapper/33
186760399180,14cyclictest0-21swapper/33
186760399178,5cyclictest19486562sleep33
1867603991715,1cyclictest0-21swapper/33
1867603991711,5cyclictest0-21swapper/33
186760399171,13cyclictest0-21swapper/33
186760399171,13cyclictest0-21swapper/33
186760399171,12cyclictest1998442-21turbostat3
186760399170,14cyclictest0-21swapper/33
186760399170,13cyclictest0-21swapper/33
186760399170,13cyclictest0-21swapper/33
23076792161,5sleep11998436-21diskmemload1
22646472161,3sleep00-21swapper/00
21959252161,3sleep10-21swapper/11
21690372161,10sleep21998436-21diskmemload2
19233082161,3sleep20-21swapper/22
1867603991612,3cyclictest0-21swapper/33
1867603991611,3cyclictest0-21swapper/33
186760399161,13cyclictest0-21swapper/33
186760399160,13cyclictest0-21swapper/33
186760399160,13cyclictest0-21swapper/33
186760399160,12cyclictest0-21swapper/33
23495902150,2sleep20-21swapper/22
23495902150,2sleep20-21swapper/22
23348142151,4sleep12334844-21apt-get1
23348142151,4sleep12334844-21apt-get1
23232702151,4sleep21998436-21diskmemload2
22806272151,4sleep02280640-21ssh0
22650042151,3sleep21998436-21diskmemload2
22122662150,4sleep00-21swapper/00
21619972151,9sleep21998436-21diskmemload2
20981492151,8sleep02098247-21ntpq0
20730912151,2sleep10-21swapper/11
20671032151,3sleep11998436-21diskmemload1
20472272152,10sleep01998436-21diskmemload0
19757542151,9sleep11975802-21ntpq1
19232862151,4sleep11923305-21apt-get1
1867603991512,2cyclictest0-21swapper/33
1867603991511,2cyclictest0-21swapper/33
186760399151,12cyclictest0-21swapper/33
186760399150,13cyclictest0-21swapper/33
186760399150,12cyclictest0-21swapper/33
186760399150,12cyclictest0-21swapper/33
23496582141,8sleep01998436-21diskmemload0
23496582141,8sleep01998436-21diskmemload0
23278462141,4sleep12329377-21ssh1
23159882146,3sleep11998436-21diskmemload1
23065372141,2sleep00-21swapper/00
22673312141,9sleep22267493-21/usr/sbin/munin2
22477712141,4sleep02247783-21ssh0
22364422141,3sleep11998436-21diskmemload1
22143692140,2sleep20-21swapper/22
22033892146,4sleep22203402-21sh2
21884522141,3sleep00-21swapper/00
20464812141,7sleep12046530-21ntp_states1
20290412141,7sleep02029100-21ntpq0
18768802141,5sleep11878662-21kthreadcore1
186760399147,5cyclictest1912342-21taskset3
186760399140,2cyclictest0-21swapper/33
186760399140,12cyclictest0-21swapper/33
186760399140,11cyclictest0-21swapper/33
23578912131,8sleep11998436-21diskmemload1
23337852131,8sleep22333799-21ssh2
23189292131,7sleep12320611-21kthreadcore1
22917082130,9sleep11998436-21diskmemload1
22290492130,1sleep00-21swapper/00
22243502131,8sleep12224324-21/usr/sbin/munin1
21881692131,6sleep10-21swapper/11
21632662131,2sleep00-21swapper/00
21542402131,8sleep1255-21systemd-journal1
21445022131,3sleep12144514-21ssh1
20917852131,5sleep20-21swapper/22
19014862134,5sleep0151ktimers/00
22993892121,5sleep21998436-21diskmemload2
22470082121,5sleep20-21swapper/22
22225832120,2sleep20-21swapper/22
22128252121,8sleep12212838-21ssh1
21270892121,7sleep22127104-21ssh2
21106642121,6sleep10-21swapper/11
20983112121,6sleep22098367-21ntpq2
20834682121,7sleep02083482-21ssh0
20732832121,7sleep02073295-21ssh0
20584802121,3sleep12058494-21ssh1
20385422121,2sleep00-21swapper/00
19389742121,7sleep01938989-21latency_hist0
18908652121,5sleep01890915-21munin-run0
18718812121,5sleep01871927-21ntpq0
186760399120,9cyclictest0-21swapper/33
22991682111,6sleep02299179-21ssh0
22879512110,2sleep00-21swapper/00
22646082111,5sleep11998436-21diskmemload1
22505522111,6sleep12250632-21kthreadcore1
22334802111,5sleep22233583-21irqstats2
22232092111,3sleep00-21swapper/00
22053722110,2sleep10-21swapper/11
21947562113,4sleep22194765-21sh2
21769872111,5sleep22177245-21users2
21436922111,4sleep01998436-21diskmemload0
21357342110,3sleep12135748-21ssh1
21346922111,6sleep00-21swapper/00
21252922111,5sleep11998436-21diskmemload1
21079612111,5sleep01998436-21diskmemload0
20834292110,4sleep10-21swapper/11
20562242111,4sleep21998436-21diskmemload2
20498562111,5sleep21998436-21diskmemload2
20404982111,2sleep20-21swapper/22
20322952111,4sleep11998436-21diskmemload1
20248652111,3sleep20-21swapper/22
20244452111,3sleep00-21swapper/00
20161472110,2sleep10-21swapper/11
20075372111,4sleep00-21swapper/00
20047522111,2sleep21998436-21diskmemload2
20045912111,4sleep11998436-21diskmemload1
19974362111,5sleep21997482-21ntpq2
19706462111,5sleep11970858-21users1
19131262111,5sleep01913206-21irqstats0
19113232111,5sleep01911371-21ntpq0
18746082111,5sleep21875017-21diskstats2
18722502111,5sleep21872494-21users2
23580992101,6sleep22358102-21ssh2
23574472101,3sleep01998436-21diskmemload0
23401752101,5sleep02340187-21ssh0
23401752101,5sleep02340187-21ssh0
23347352101,2sleep20-21swapper/22
23347352101,2sleep20-21swapper/22
23162092100,2sleep00-21swapper/00
23140082101,3sleep21998436-21diskmemload2
23007842101,3sleep22300806-21apt-get2
22887562101,4sleep21998436-21diskmemload2
22811702101,2sleep20-21swapper/22
22753822101,6sleep12275399-21grep1
22711722101,3sleep172-21kcompactd01
22549082101,4sleep21998436-21diskmemload2
22418302101,4sleep11998436-21diskmemload1
22393952100,1sleep00-21swapper/00
22285662101,5sleep22228620-21ntpq2
22036632101,5sleep00-21swapper/00
21810642101,5sleep22181296-21df_inode2
21636912101,6sleep12163778-21cpuspeed_turbos1
21620362101,2sleep10-21swapper/11
21540512101,2sleep20-21swapper/22
21516722101,3sleep00-21swapper/00
21416792101,4sleep20-21swapper/22
21332532101,5sleep22133317-21ntp_states2
21264192101,2sleep00-21swapper/00
21189922101,2sleep10-21swapper/11
21181482101,4sleep01998436-21diskmemload0
21167222101,3sleep20-21swapper/22
21070342101,2sleep20-21swapper/22
20921222101,4sleep12092132-21ssh1
20854042101,4sleep02085431-21apt-get0
20752262101,3sleep20-21swapper/22
20662282100,4sleep21998436-21diskmemload2
20411252101,5sleep11998436-21diskmemload1
20130282101,2sleep20-21swapper/22
20075702101,4sleep11998436-21diskmemload1
19882912101,5sleep21988313-21/usr/sbin/munin2
18811782101,5sleep21882853-21kthreadcore2
18810002101,5sleep11881174-21kthreadcore1
2347679291,4sleep12347736-21ssh1
2347679291,4sleep12347736-21ssh1
2334131291,2sleep00-21swapper/00
2297595291,3sleep11998436-21diskmemload1
2266809291,4sleep01998436-21diskmemload0
2177909291,4sleep02177916-21ssh0
2172312291,2sleep10-21swapper/11
2168770291,3sleep01998436-21diskmemload0
2080922291,4sleep22076770-21/usr/sbin/munin2
2053676291,4sleep02054832-21ssh0
1997834291,4sleep01998051-21users0
1993366293,3sleep11993411-21cpu1
1985144291,4sleep11986294-21memory1
1912708291,4sleep21912924-21diskstats2
1906091291,4sleep11906138-21/usr/sbin/munin1
1901642291,4sleep21-21systemd2
1895896291,3sleep2802-21snmpd2
2321750281,3sleep02321805-21ssh0
2004522281,3sleep02004539-21ssh0
1988872281,4sleep11990557-21grep1
1981127281,3sleep21981178-21ntpq2
1972337281,3sleep21973653-21grep2
1955220280,3sleep01955239-21apt-get0
1950210281,4sleep01950423-21ls0
1944587280,2sleep20-21swapper/22
1938708280,1sleep20-21swapper/22
1933553280,2sleep00-21swapper/00
1933385280,2sleep20-21swapper/22
1928330280,2sleep00-21swapper/00
1907838281,3sleep10-21swapper/11
1895252280,1sleep10-21swapper/11
1889817281,3sleep21889874-21ntpq2
1970203271,2sleep20-21swapper/22
1965553270,1sleep10-21swapper/11
1960820270,2sleep20-21swapper/22
1959938270,1sleep20-21swapper/22
1959929270,4sleep10-21swapper/11
1949934271,2sleep20-21swapper/22
1949912270,2sleep10-21swapper/11
1949362270,1sleep10-21swapper/11
1948960270,3sleep01949030-21/usr/sbin/munin0
1944203270,1sleep20-21swapper/22
1944006270,1sleep10-21swapper/11
1943945270,1sleep00-21swapper/00
1933470270,1sleep10-21swapper/11
1922801270,1sleep00-21swapper/00
1921905271,2sleep20-21swapper/22
1912267270,1sleep20-21swapper/22
1906798270,1sleep00-21swapper/00
1892663271,3sleep01894433-21tr0
1890654270,1sleep10-21swapper/11
1875710271,2sleep00-21swapper/00
1872699270,1sleep10-21swapper/11
1988014260,2sleep00-21swapper/00
1987182260,4sleep20-21swapper/22
1983423261,2sleep00-21swapper/00
1981910260,4sleep00-21swapper/00
1981032261,1sleep10-21swapper/11
1965813260,4sleep00-21swapper/00
1936028261,2sleep10-21swapper/11
1918244260,2sleep10-21swapper/11
1906656260,1sleep20-21swapper/22
1972056250,1sleep00-21swapper/00
1970301250,1sleep00-21swapper/00
1885220250,3sleep00-21swapper/00
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional