You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-25 - 11:44

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot3s.osadl.org (updated Wed Mar 25, 2026 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
235571021320,2sleep1321rcuc/11
2085687212152,64sleep30-21swapper/33
208556528959,11sleep10-21swapper/11
208575028655,12sleep20-21swapper/22
208547127955,6sleep00-21swapper/00
208596399222,15cyclictest2408884-21taskset3
208596399222,15cyclictest2408884-21taskset3
22248752210,1sleep20-21swapper/22
2085963992112,5cyclictest0-21swapper/33
208596399211,16cyclictest2486154-21taskset3
21514372201,8sleep22151559-21grep2
208596399206,10cyclictest2406029-21chrt3
2085963992016,2cyclictest0-21swapper/33
208596399201,16cyclictest0-21swapper/33
208596399201,15cyclictest2311122-21taskset3
208596399201,15cyclictest2235226-21taskset3
208596399201,15cyclictest0-21swapper/33
208596399201,15cyclictest0-21swapper/33
208596399201,15cyclictest0-21swapper/33
2085963992010,5cyclictest25184492sleep33
25026312191,8sleep22502646-21ssh2
22882042191,7sleep22289790-21cut2
208596399194,12cyclictest0-21swapper/33
208596399191,15cyclictest0-21swapper/33
208596399191,15cyclictest0-21swapper/33
208596399191,14cyclictest66950irq/132-enp1s0-TxRx-33
208596399191,14cyclictest2317303-21taskset3
208596399191,14cyclictest0-21swapper/33
208596399191,14cyclictest0-21swapper/33
208596399191,14cyclictest0-21swapper/33
208596399191,14cyclictest0-21swapper/33
208596399191,14cyclictest0-21swapper/33
208596399191,14cyclictest0-21swapper/33
208596399191,14cyclictest0-21swapper/33
2085963991910,6cyclictest2276309-21taskset3
208596399190,16cyclictest0-21swapper/33
208596399190,16cyclictest0-21swapper/33
208596399185,11cyclictest0-21swapper/33
2085963991816,1cyclictest0-21swapper/33
208596399181,4cyclictest2449428-21taskset3
2085963991814,2cyclictest0-21swapper/33
208596399181,14cyclictest0-21swapper/33
208596399181,14cyclictest0-21swapper/33
208596399181,14cyclictest0-21swapper/33
208596399181,13cyclictest0-21swapper/33
208596399181,13cyclictest0-21swapper/33
208596399181,13cyclictest0-21swapper/33
208596399180,15cyclictest0-21swapper/33
208596399180,14cyclictest0-21swapper/33
208596399180,14cyclictest0-21swapper/33
25736222172,4sleep22573638-21apt-get2
25639762171,3sleep00-21swapper/00
25533662171,3sleep00-21swapper/00
23658542171,5sleep02366132-21kthreadcore0
22654792171,11sleep12265534-21ntpq1
208596399172,11cyclictest2086138-21chrt3
2085963991712,3cyclictest0-21swapper/33
208596399171,14cyclictest0-21swapper/33
208596399171,13cyclictest0-21swapper/33
208596399170,14cyclictest0-21swapper/33
208596399170,14cyclictest0-21swapper/33
25644882161,3sleep20-21swapper/22
25482932162,10sleep22549582-21grep2
23377532161,3sleep20-21swapper/22
208596399161,13cyclictest0-21swapper/33
208596399161,13cyclictest0-21swapper/33
208596399160,13cyclictest0-21swapper/33
208596399160,13cyclictest0-21swapper/33
208596399160,12cyclictest0-21swapper/33
208596399160,12cyclictest0-21swapper/33
25195072150,2sleep00-21swapper/00
25090632151,4sleep12217027-21diskmemload1
24441372151,10sleep22445072-21cut2
24426372151,4sleep02442650-21ssh0
23209512150,1sleep10-21swapper/11
2085963991512,2cyclictest0-21swapper/33
2085963991512,1cyclictest0-21swapper/33
208596399151,11cyclictest0-21swapper/33
208596399151,11cyclictest0-21swapper/33
208596399150,13cyclictest0-21swapper/33
208596399150,12cyclictest0-21swapper/33
208596399150,12cyclictest0-21swapper/33
24061692141,4sleep22406180-21ssh2
23867152141,9sleep12386772-21/usr/sbin/munin1
22234052141,8sleep12217027-21diskmemload1
208596399149,3cyclictest0-21swapper/33
2085963991412,1cyclictest0-21swapper/33
208596399140,2cyclictest0-21swapper/33
208596399140,2cyclictest0-21swapper/33
208596399140,11cyclictest0-21swapper/33
25106972131,6sleep22217027-21diskmemload2
24766922130,2sleep00-21swapper/00
24653352131,2sleep20-21swapper/22
23987102131,8sleep02217027-21diskmemload0
23545522131,5sleep00-21swapper/00
22855262131,9sleep02285540-21ssh0
22513052133,6sleep22251320-21sh2
22443292131,8sleep12244387-21fschecks_count1
22336712131,5sleep20-21swapper/22
22250132131,8sleep02225025-21ssh0
208596399138,3cyclictest2206165-21chrt3
2085963991310,1cyclictest0-21swapper/33
208596399130,11cyclictest0-21swapper/33
25815812121,7sleep12217027-21diskmemload1
25652422121,7sleep02565459-21hddtemp_smartct0
25569692122,7sleep12217027-21diskmemload1
25385062121,7sleep00-21swapper/00
25294552120,8sleep12529467-21sh1
25209692121,3sleep20-21swapper/22
24987942121,6sleep12217027-21diskmemload1
24776302121,7sleep22477852-21sed2
24602932121,7sleep22460305-21rm2
24570592121,7sleep02457070-21ssh0
24215392121,6sleep12421663-21ntpq1
24160242121,7sleep22416026-21ssh2
24160242121,7sleep22416026-21ssh2
24016932121,7sleep02403371-21cut0
23573962121,3sleep22358999-21kthreadcore2
23545772121,5sleep20-21swapper/22
23444122121,6sleep22217027-21diskmemload2
23292432121,2sleep10-21swapper/11
22858442121,8sleep12217027-21diskmemload1
22119352121,6sleep12211954-21apt-get1
22105752121,5sleep02210664-21ntpq0
22015492121,5sleep22201761-21hddtemp_smartct2
25804812110,1sleep00-21swapper/00
25721252111,6sleep22217027-21diskmemload2
25551752111,4sleep12217027-21diskmemload1
25452312111,4sleep22217027-21diskmemload2
25290782110,7sleep22529089-21ssh2
24930862111,7sleep12493103-21ssh1
24840172111,6sleep22484024-21ssh2
24757992111,4sleep12217027-21diskmemload1
24663862111,4sleep12217027-21diskmemload1
24595392111,7sleep12459545-21sh1
24502752111,3sleep00-21swapper/00
24417652111,4sleep12217027-21diskmemload1
24399272111,4sleep20-21swapper/22
24336032111,3sleep10-21swapper/11
24159992111,6sleep02416010-21ssh0
24159992111,6sleep02416010-21ssh0
24067532111,5sleep12217027-21diskmemload1
23987372110,5sleep20-21swapper/22
23814842110,6sleep12217027-21diskmemload1
23812952111,4sleep20-21swapper/22
23623442111,2sleep00-21swapper/00
23616062111,4sleep12217027-21diskmemload1
23447652111,5sleep02217027-21diskmemload0
23361762111,3sleep00-21swapper/00
23136282111,6sleep22313760-21ssh2
23033062111,3sleep20-21swapper/22
22955642111,4sleep12217027-21diskmemload1
22953012111,3sleep00-21swapper/00
22762962110,1sleep20-21swapper/22
22573642111,6sleep12257535-21ssh1
22414782111,4sleep12217027-21diskmemload1
22125392111,3sleep20-21swapper/22
22066322111,6sleep12206616-21/usr/sbin/munin1
22056352111,6sleep12205781-21sort1
21956642110,1sleep10-21swapper/11
21400442111,5sleep22140526-21/usr/sbin/munin2
21098332111,5sleep02110046-21hddtemp_smartct0
21095652111,6sleep22109718-21cstates2
20979852111,5sleep02098229-21users0
20912352111,4sleep12091256-21munin-run1
25699442100,4sleep12217027-21diskmemload1
25442062100,2sleep00-21swapper/00
25389592102,3sleep12217027-21diskmemload1
25359442101,2sleep20-21swapper/22
25266312101,6sleep02526645-21ssh0
25182292101,3sleep12217027-21diskmemload1
24919802101,2sleep00-21swapper/00
24909752101,4sleep22491039-21ntpq2
24846052102,3sleep12217027-21diskmemload1
24662262101,2sleep00-21swapper/00
24495992101,3sleep12217027-21diskmemload1
24333692101,3sleep22217027-21diskmemload2
24310582100,1sleep00-21swapper/00
24175172101,2sleep20-21swapper/22
24152222100,1sleep10-21swapper/11
24152222100,1sleep10-21swapper/11
23903452101,3sleep02217027-21diskmemload0
23882972101,4sleep22217027-21diskmemload2
23814182101,3sleep02217027-21diskmemload0
23360372101,2sleep10-21swapper/11
23202382101,3sleep00-21swapper/00
23110672101,2sleep20-21swapper/22
23103352101,3sleep00-21swapper/00
22761872101,4sleep12217027-21diskmemload1
22757722101,4sleep02217027-21diskmemload0
22690782101,3sleep00-21swapper/00
22495412101,5sleep02249553-21sh0
22422572101,2sleep20-21swapper/22
22394562101,4sleep02217027-21diskmemload0
22306602100,1sleep00-21swapper/00
22164022101,4sleep02216646-21users0
22107202101,4sleep22210776-21ntpq2
22053422101,4sleep02205396-21ntpq0
22010022101,5sleep2255-21systemd-journal2
21778422101,6sleep22178277-21needreboot2
21676672101,4sleep12167717-21ntpq1
21352812101,4sleep22135331-21ntpq2
21047122101,4sleep22104782-21irqstats2
21043422101,5sleep02104501-21diskstats0
20950942101,6sleep22096887-21grep2
2530218291,3sleep12217027-21diskmemload1
2509103291,3sleep02509113-21ssh0
2501407291,4sleep02501420-21ssh0
2417430291,3sleep00-21swapper/00
2399316291,4sleep12399327-21ssh1
2369510291,4sleep22369567-21ssh2
2343858291,4sleep12344114-21head1
2327616291,2sleep20-21swapper/22
2322034291,4sleep02322284-21df_inode0
2310146290,1sleep10-21swapper/11
2303831291,3sleep00-21swapper/00
2303346291,3sleep12217027-21diskmemload1
2286545291,3sleep22217027-21diskmemload2
2267674291,4sleep22267688-21ssh2
2259820291,3sleep22217027-21diskmemload2
2257800291,2sleep00-21swapper/00
2231618290,1sleep10-21swapper/11
2163577291,3sleep02163596-21apt-get0
2162629291,4sleep1321rcuc/11
2159785291,4sleep02161569-21cut0
2141934291,4sleep22142195-21df_inode2
2135237291,4sleep12135275-21ntpq1
2128060291,5sleep02129730-21latency0
2125314291,4sleep02125537-21cat0
2124354291,4sleep12124399-21ntpq1
2119782291,3sleep22119801-21apt-get2
2478363281,3sleep02478574-21hddtemp_smartct0
2370291281,3sleep12370302-21ssh1
2200994281,3sleep11-21systemd1
2195497280,2sleep20-21swapper/22
2192510281,3sleep00-21swapper/00
2184849281,3sleep22184696-21apt-get2
2173953280,1sleep00-21swapper/00
2141911283,2sleep02141912-21/usr/sbin/munin0
2138239281,3sleep12140043-21kthreadcore1
2130762280,1sleep10-21swapper/11
2114402280,1sleep10-21swapper/11
2098909281,3sleep02099064-21cstates0
2098909281,3sleep02099064-21cstates0
2097834280,1sleep10-21swapper/11
2185813271,3sleep02186873-21kthreadcore0
2176024271,3sleep12177839-21kthreadcore1
2173822270,4sleep10-21swapper/11
2172899271,2sleep20-21swapper/22
2158189270,2sleep20-21swapper/22
2153144271,3sleep12154044-21cut1
2142472271,2sleep10-21swapper/11
2137112271,2sleep00-21swapper/00
2125847270,2sleep20-21swapper/22
2125269270,1sleep20-21swapper/22
2119651270,1sleep10-21swapper/11
2200973260,1sleep00-21swapper/00
2186875260,3sleep1321rcuc/11
2183638260,2sleep12183686-21ntpq1
2183220260,2sleep02183632-21strings0
2174834261,2sleep00-21swapper/00
2157324260,4sleep00-21swapper/00
2148071261,2sleep02149724-21kthreadcore0
2135394260,3sleep00-21swapper/00
2108973260,1sleep10-21swapper/11
2103723260,4sleep10-21swapper/11
2103723260,4sleep10-21swapper/11
2102786260,2sleep22102899-21strings2
2102786260,2sleep22102899-21strings2
2090544261,2sleep00-21swapper/00
2086116260,2sleep22086135-21apt-get2
2184601250,1sleep20-21swapper/22
2167719250,1sleep20-21swapper/22
2152660250,2sleep20-21swapper/22
2147531250,2sleep10-21swapper/11
2119019250,1sleep00-21swapper/00
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional