You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-22 - 00:12

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack3slot3s.osadl.org (updated Thu May 21, 2026 12:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2117777214082,54sleep30-21swapper/33
211306129461,9sleep00-21swapper/00
211795029170,12sleep20-21swapper/22
211789129062,11sleep10-21swapper/11
21950452310,4sleep20-21swapper/22
211817999245,14cyclictest0-21swapper/33
211817999232,16cyclictest21237052sleep33
2118179992318,2cyclictest0-21swapper/33
2118179992318,2cyclictest0-21swapper/33
211817999222,14cyclictest2250057-21turbostat3
2118179992219,1cyclictest0-21swapper/33
211817999220,18cyclictest66950irq/132-enp1s0-TxRx-33
25139752211,3sleep10-21swapper/11
25061252210,2sleep30-21swapper/33
25058482211,11sleep12505862-21ssh1
211817999215,13cyclictest0-21swapper/33
2118179992119,1cyclictest0-21swapper/33
211817999211,7cyclictest2381386-21taskset3
2118179992116,2cyclictest0-21swapper/33
2118179992112,6cyclictest2152107-21taskset3
211817999211,16cyclictest66950irq/132-enp1s0-TxRx-33
211817999211,16cyclictest66950irq/132-enp1s0-TxRx-33
23915272201,3sleep00-21swapper/00
211817999206,11cyclictest0-21swapper/33
211817999204,13cyclictest0-21swapper/33
211817999204,13cyclictest0-21swapper/33
211817999203,14cyclictest0-21swapper/33
2118179992018,1cyclictest0-21swapper/33
2118179992017,1cyclictest0-21swapper/33
2118179992016,2cyclictest0-21swapper/33
2118179992016,2cyclictest0-21swapper/33
2118179992016,1cyclictest0-21swapper/33
2118179992015,2cyclictest0-21swapper/33
211817999201,15cyclictest66950irq/132-enp1s0-TxRx-33
211817999201,15cyclictest0-21swapper/33
211817999201,14cyclictest0-21swapper/33
2118179991916,1cyclictest0-21swapper/33
2118179991915,2cyclictest0-21swapper/33
2118179991915,2cyclictest0-21swapper/33
2118179991915,2cyclictest0-21swapper/33
2118179991915,2cyclictest0-21swapper/33
2118179991914,2cyclictest0-21swapper/33
2118179991912,4cyclictest2523645-21turbostat3
2118179991911,4cyclictest0-21swapper/33
211817999191,14cyclictest0-21swapper/33
211817999191,14cyclictest0-21swapper/33
23013012181,8sleep02301315-21ssh0
22280962181,13sleep22228127-21perf2
2118179991816,1cyclictest0-21swapper/33
211817999181,5cyclictest0-21swapper/33
2118179991815,2cyclictest0-21swapper/33
2118179991815,1cyclictest0-21swapper/33
2118179991815,1cyclictest0-21swapper/33
2118179991815,1cyclictest0-21swapper/33
2118179991815,1cyclictest0-21swapper/33
2118179991815,1cyclictest0-21swapper/33
2118179991815,1cyclictest0-21swapper/33
2118179991814,2cyclictest0-21swapper/33
2118179991814,2cyclictest0-21swapper/33
2118179991814,2cyclictest0-21swapper/33
211817999181,14cyclictest0-21swapper/33
211817999181,14cyclictest0-21swapper/33
211817999181,14cyclictest0-21swapper/33
26113082171,3sleep00-21swapper/00
24718032171,5sleep12471910-21cut1
2118179991714,2cyclictest0-21swapper/33
2118179991714,2cyclictest0-21swapper/33
211817999171,13cyclictest0-21swapper/33
211817999170,14cyclictest0-21swapper/33
211817999170,14cyclictest0-21swapper/33
25331162161,10sleep12533343-21df_inode1
24576132161,4sleep02457676-21ntpq0
24134592162,10sleep22413597-21/usr/sbin/munin2
23879152161,4sleep12250051-21diskmemload1
2118179991614,1cyclictest0-21swapper/33
211817999161,3cyclictest0-21swapper/33
2118179991611,3cyclictest21956952sleep33
211817999161,12cyclictest0-21swapper/33
211817999160,13cyclictest0-21swapper/33
211817999160,13cyclictest0-21swapper/33
25745232150,1sleep00-21swapper/00
25669022150,1sleep00-21swapper/00
25582572151,4sleep10-21swapper/11
24949022151,2sleep00-21swapper/00
24437692157,1sleep2171rcu_preempt2
24261262154,2sleep2171rcu_preempt2
23976872150,1sleep10-21swapper/11
23445112151,10sleep02344525-21ssh0
22686802151,8sleep12268796-21perl1
22682782151,4sleep02268292-21ssh0
21569202151,10sleep12152168-31find1
21265722152,8sleep22128249-21tr2
211817999157,5cyclictest21310742sleep33
2118179991511,3cyclictest0-21swapper/33
211817999151,11cyclictest0-21swapper/33
26032252140,3sleep20-21swapper/22
25027202141,8sleep22250051-21diskmemload2
24854292141,9sleep02485440-21ssh0
24695202141,4sleep12250051-21diskmemload1
24517962142,7sleep02250051-21diskmemload0
24508452140,2sleep10-21swapper/11
24087322140,1sleep10-21swapper/11
23822792145,5sleep12382285-21sh1
23730962141,3sleep02250051-21diskmemload0
23574712141,9sleep02359109-21tr0
22594902141,8sleep22250051-21diskmemload2
22563882141,3sleep00-21swapper/00
21522342140,6sleep00-21swapper/00
211817999140,11cyclictest0-21swapper/33
26185602131,4sleep12618571-21ssh1
26018942131,5sleep10-21swapper/11
25858362131,6sleep02586008-21diskstats0
25835892131,7sleep22250051-21diskmemload2
25571492130,8sleep22250051-21diskmemload2
25285652131,8sleep22528627-21/usr/sbin/munin2
25211652131,8sleep12521176-21ssh1
24178702131,5sleep00-21swapper/00
23876432131,8sleep02387655-21sh0
23386742131,3sleep12338686-21ssh1
23264092131,3sleep02326423-21ssh0
23183802131,7sleep12250051-21diskmemload1
23111022131,4sleep22311104-21ssh2
23096432131,8sleep02309654-21sh0
23035882132,5sleep12303608-21apt-get1
22999402131,6sleep10-21swapper/11
22902272130,2sleep20-21swapper/22
22828192131,5sleep10-21swapper/11
22828192131,5sleep10-21swapper/11
22754972131,8sleep22275511-21ssh2
22576802131,8sleep12257691-21ssh1
22271232131,8sleep12227178-21ntpq1
22014042131,6sleep22201426-21apt-get2
26163412121,7sleep02616410-21ssh0
25747912121,5sleep10-21swapper/11
25677462121,7sleep22567962-21cut2
25589662121,5sleep02250051-21diskmemload0
25241942121,4sleep12524205-21ssh1
24938942120,2sleep10-21swapper/11
24856102120,1sleep20-21swapper/22
24806742121,7sleep12481951-21tr1
24791842120,2sleep2171rcu_preempt2
24781342121,7sleep02250051-21diskmemload0
24491302121,7sleep22449265-21cat2
24413992121,6sleep02250051-21diskmemload0
24334502121,7sleep12433462-21ssh1
24313552121,7sleep22431598-21unixbench_singl2
23646652121,7sleep23671894-21systemd-journal2
22443422120,1sleep00-21swapper/00
22438312120,8sleep20-21swapper/22
21853982121,5sleep10-21swapper/11
21704742121,7sleep22172178-21cut2
21632302121,8sleep12163233-21cat1
26038752111,7sleep12604912-21grep1
25599292111,6sleep12560038-21ps1
25478362110,1sleep20-21swapper/22
25475212111,6sleep12547535-21ssh1
25395712111,4sleep20-21swapper/22
25385702111,3sleep00-21swapper/00
25204892111,4sleep00-21swapper/00
25155452111,6sleep22515192-21/usr/sbin/munin2
25050672111,6sleep02505078-21ssh0
24676272111,4sleep20-21swapper/22
24419192110,3sleep10-21swapper/11
24156882111,5sleep02250051-21diskmemload0
24065062110,1sleep00-21swapper/00
24003112111,5sleep290-21kswapd02
23987812111,4sleep20-21swapper/22
23872842113,4sleep22387485-21sed2
23724972111,5sleep20-21swapper/22
23637402110,4sleep10-21swapper/11
23544242111,4sleep22250051-21diskmemload2
23356202114,2sleep22250051-21diskmemload2
23211862111,4sleep12250051-21diskmemload1
23181722110,1sleep00-21swapper/00
23164432111,6sleep22316578-21/usr/sbin/munin2
22950962111,5sleep22295316-21hddtemp_smartct2
22814152111,6sleep22281476-21ntpq2
22814152111,6sleep22281476-21ntpq2
22639142112,4sleep02250051-21diskmemload0
22555872110,1sleep20-21swapper/22
22350292111,6sleep02236724-21cut0
21950092111,3sleep10-21swapper/11
21803542111,7sleep12180580-21basename1
21407762111,6sleep02140845-21/usr/sbin/munin0
21297662111,5sleep12129806-21ntpq1
26125952101,5sleep22612680-21kernelversion2
25999722101,3sleep22250051-21diskmemload2
25856812101,4sleep22250051-21diskmemload2
25840832101,3sleep10-21swapper/11
25835622101,4sleep02250051-21diskmemload0
25741882101,4sleep22250051-21diskmemload2
25314542101,3sleep00-21swapper/00
25135542101,4sleep02250051-21diskmemload0
25128682104,3sleep20-21swapper/22
24597992101,4sleep12250051-21diskmemload1
24573952101,4sleep22457547-21ntpq2
24242722100,1sleep10-21swapper/11
24173782100,2sleep10-21swapper/11
23815732101,4sleep22250051-21diskmemload2
23693812101,4sleep12369433-21ntpq1
23545712100,6sleep12250051-21diskmemload1
23463752101,3sleep20-21swapper/22
23346612101,4sleep12334891-21users1
23328672101,4sleep02333983-21memory0
23253692101,5sleep22325420-21ntpq2
22860032101,4sleep12250051-21diskmemload1
22818412100,1sleep00-21swapper/00
22818412100,1sleep00-21swapper/00
22658332101,3sleep10-21swapper/11
22496842100,1sleep20-21swapper/22
22367262101,4sleep12237885-21memory1
22107052101,5sleep02210794-21ntpq0
22107052101,5sleep02210794-21ntpq0
21965812101,5sleep22196892-21kthreadcore2
21957692101,5sleep02195966-21grep0
21893402101,4sleep02189431-21ntpq0
21675012101,4sleep02167540-21ntpq0
21571472100,6sleep02152168-31find0
21466702100,1sleep00-21swapper/00
21224072101,4sleep02122466-21ntpq0
2600601290,1sleep00-21swapper/00
2590259291,4sleep12590477-21unixbench-2d1
2541763291,2sleep00-21swapper/00
2493434291,4sleep22493679-21grep2
2426607290,2sleep00-21swapper/00
2379807291,2sleep00-21swapper/00
2353171291,2sleep00-21swapper/00
2292332291,2sleep02250051-21diskmemload0
2243708291,4sleep12243830-21sed1
2228966291,5sleep22230525-21kthreadcore2
2211109291,4sleep22211121-21wc2
2211109291,4sleep22211121-21wc2
2200587291,4sleep12200773-21users1
2176124291,4sleep02177915-21cut0
2174903291,4sleep22175019-21irqstats2
2174155290,1sleep00-21swapper/00
2131460291,4sleep22131634-21cat2
2126022291,4sleep02126245-21basename0
2118370291,4sleep12118546-21expr1
2461902281,3sleep02462307-21ssh0
2219539281,3sleep02219801-21kthreadcore0
2180053281,2sleep20-21swapper/22
2152230280,3sleep12152168-31find1
2151978280,1sleep20-21swapper/22
2135454281,4sleep12135698-21sed1
2249708270,1sleep10-21swapper/11
2249687270,1sleep00-21swapper/00
2223025271,2sleep00-21swapper/00
2222556270,1sleep10-21swapper/11
2222425270,1sleep20-21swapper/22
2217062270,1sleep10-21swapper/11
2211911271,3sleep22211807-21apt-get2
2207303271,2sleep10-21swapper/11
2207303271,2sleep10-21swapper/11
2206419270,1sleep10-21swapper/11
2184891270,1sleep00-21swapper/00
2169087270,2sleep10-21swapper/11
2168290270,1sleep10-21swapper/11
2163242271,3sleep20-21swapper/22
2160123271,3sleep00-21swapper/00
2142645271,3sleep22144348-21cut2
2142283271,3sleep12142644-21kthreadcore1
2140243270,4sleep12117813-21kworker/1:2+events1
2136402271,2sleep22136421-21apt-get2
2132418271,3sleep02134163-21cut0
2234311261,2sleep20-21swapper/22
2233226260,1sleep00-21swapper/00
2228359260,1sleep10-21swapper/11
2217302260,2sleep00-21swapper/00
2201393260,3sleep00-21swapper/00
2200454261,2sleep00-21swapper/00
2186792260,2sleep20-21swapper/22
2179189260,1sleep10-21swapper/11
2163585260,1sleep20-21swapper/22
2157215260,2sleep20-21swapper/22
2122754261,2sleep22122845-21timerandwakeup2
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional