You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-04 - 19:53

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack3slot3s.osadl.org (updated Thu Jun 04, 2026 12:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1835875212953,70sleep30-21swapper/33
18366342110102,5sleep20-21swapper/22
183651328668,12sleep10-21swapper/11
183642827562,7sleep00-21swapper/00
23216002261,9sleep02321731-21ntpq0
183687499233,15cyclictest1972943-21taskset3
183687499212,16cyclictest0-21swapper/33
183687499211,7cyclictest2026681-21taskset3
183687499211,7cyclictest1854828-21chrt3
183687499211,5cyclictest1946186-21taskset3
183687499211,15cyclictest0-21swapper/33
183687499211,15cyclictest0-21swapper/33
19895632201,4sleep20-21swapper/22
1836874992017,1cyclictest0-21swapper/33
183687499201,16cyclictest66950irq/132-enp1s0-TxRx-33
183687499201,16cyclictest0-21swapper/33
183687499201,15cyclictest66950irq/132-enp1s0-TxRx-33
183687499201,15cyclictest0-21swapper/33
1836874992010,7cyclictest2202489-21chrt3
21341142191,8sleep02133815-21/usr/sbin/munin0
21341142191,8sleep02133815-21/usr/sbin/munin0
20856252191,12sleep12085683-21ntpq1
183687499193,13cyclictest66950irq/132-enp1s0-TxRx-33
1836874991912,4cyclictest0-21swapper/33
183687499191,15cyclictest0-21swapper/33
183687499191,14cyclictest0-21swapper/33
183687499191,14cyclictest0-21swapper/33
183687499191,14cyclictest0-21swapper/33
183687499191,14cyclictest0-21swapper/33
183687499191,14cyclictest0-21swapper/33
183687499191,14cyclictest0-21swapper/33
183687499191,14cyclictest0-21swapper/33
183687499191,14cyclictest0-21swapper/33
1836874991910,7cyclictest1999814-21chrt3
20567252181,8sleep12058395-21kthreadcore1
183687499189,7cyclictest2046138-21sleep3
1836874991815,2cyclictest0-21swapper/33
1836874991815,1cyclictest0-21swapper/33
1836874991813,3cyclictest0-21swapper/33
1836874991812,3cyclictest0-21swapper/33
1836874991811,4cyclictest0-21swapper/33
183687499181,14cyclictest0-21swapper/33
183687499181,14cyclictest0-21swapper/33
183687499181,14cyclictest0-21swapper/33
183687499181,14cyclictest0-21swapper/33
183687499181,14cyclictest0-21swapper/33
183687499181,14cyclictest0-21swapper/33
183687499181,14cyclictest0-21swapper/33
183687499181,14cyclictest0-21swapper/33
183687499181,13cyclictest0-21swapper/33
183687499181,13cyclictest0-21swapper/33
183687499181,13cyclictest0-21swapper/33
183687499180,15cyclictest0-21swapper/33
23080572171,4sleep01967206-21diskmemload0
21835082171,2sleep20-21swapper/22
21763452171,9sleep20-21swapper/22
21728902171,10sleep12172955-21ntpq1
21555892171,11sleep22155720-21wc2
21467482171,5sleep22146812-21ntpq2
20455112170,8sleep02045525-21ssh0
20455112170,8sleep02045525-21ssh0
20016072171,3sleep00-21swapper/00
183687499171,4cyclictest0-21swapper/33
1836874991714,1cyclictest0-21swapper/33
1836874991712,3cyclictest0-21swapper/33
183687499171,14cyclictest0-21swapper/33
183687499171,13cyclictest0-21swapper/33
183687499171,13cyclictest0-21swapper/33
183687499171,12cyclictest0-21swapper/33
183687499170,14cyclictest0-21swapper/33
23236942161,6sleep12323706-21rm1
22087242161,5sleep21967206-21diskmemload2
20784202161,2sleep20-21swapper/22
20217502165,7sleep22023433-21cut2
20195712161,6sleep22019582-21ssh2
18673572162,7sleep11869084-21tr1
1836874991611,3cyclictest0-21swapper/33
183687499160,13cyclictest0-21swapper/33
183687499160,12cyclictest0-21swapper/33
183687499160,12cyclictest0-21swapper/33
183687499160,12cyclictest0-21swapper/33
23150122151,9sleep01967206-21diskmemload0
23076782153,2sleep2171rcu_preempt2
22728332150,3sleep10-21swapper/11
22307592152,3sleep20-21swapper/22
22025022151,3sleep00-21swapper/00
21766772151,3sleep00-21swapper/00
21666992151,8sleep20-21swapper/22
21237272151,2sleep10-21swapper/11
21076962151,3sleep20-21swapper/22
20977242151,6sleep01967206-21diskmemload0
19819362151,2sleep10-21swapper/11
19398912151,3sleep20-21swapper/22
18969682151,5sleep01897028-21ntp_offset0
1836874991513,1cyclictest0-21swapper/33
1836874991511,2cyclictest0-21swapper/33
183687499151,12cyclictest0-21swapper/33
183687499150,12cyclictest0-21swapper/33
183687499150,12cyclictest0-21swapper/33
22265492141,7sleep01967206-21diskmemload0
22256312141,9sleep22225689-21ntpq2
22214292145,4sleep11967206-21diskmemload1
22126562143,5sleep02212681-21apt-get0
22103712141,8sleep00-21swapper/00
21169682141,2sleep00-21swapper/00
21072302141,8sleep22107420-21wc2
21071102141,2sleep10-21swapper/11
20726642140,2sleep10-21swapper/11
20529422141,8sleep10-21swapper/11
18495302143,6sleep21849596-21http2
1836874991411,1cyclictest0-21swapper/33
183687499140,2cyclictest0-21swapper/33
183687499140,2cyclictest0-21swapper/33
183687499140,12cyclictest0-21swapper/33
183687499140,11cyclictest0-21swapper/33
22906012131,8sleep11967206-21diskmemload1
22717472131,5sleep20-21swapper/22
22655142131,4sleep02265737-21basename0
21500152131,5sleep00-21swapper/00
21303242131,2sleep10-21swapper/11
21149962131,8sleep12115011-21ssh1
21046462131,8sleep02104659-21ssh0
20619032131,7sleep00-21swapper/00
20442102131,2sleep11967206-21diskmemload1
20442102131,2sleep11967206-21diskmemload1
20171762131,3sleep02017187-21ssh0
19235282131,8sleep01923576-21ntpq0
19182632131,7sleep21918313-21ntpq2
183687499138,3cyclictest19248312sleep33
183687499130,1cyclictest0-21swapper/33
23072102121,3sleep10-21swapper/11
22980902121,6sleep11967206-21diskmemload1
22872532121,6sleep20-21swapper/22
22793352121,2sleep0171rcu_preempt0
22624592120,2sleep00-21swapper/00
22395972121,7sleep12241155-21tr1
22078502123,4sleep12207909-21ntpq1
21920372121,5sleep10-21swapper/11
21323962121,6sleep20-21swapper/22
21252682121,5sleep02125401-21diskstats0
20962382121,5sleep20-21swapper/22
20863182121,5sleep00-21swapper/00
20716642121,6sleep21967206-21diskmemload2
20343402121,3sleep01967206-21diskmemload0
20335672121,7sleep12033596-21cat1
20293302120,7sleep22029551-21hddtemp_smartct2
20265042121,4sleep12026515-21ssh1
20097922121,7sleep11967206-21diskmemload1
19731862124,4sleep21973197-21sh2
19618882121,5sleep11961937-21munin-run1
19510322120,1sleep10-21swapper/11
19093502121,3sleep10-21swapper/11
18710262121,8sleep21871036-21ner-fs-32
18379302121,7sleep01839627-21cut0
23330942111,5sleep11967206-21diskmemload1
23314732111,3sleep00-21swapper/00
22540782111,5sleep11967206-21diskmemload1
22515832111,7sleep22251787-21needreboot2
22449352110,1sleep00-21swapper/00
22185252111,3sleep20-21swapper/22
22133092111,5sleep12214811-21kthreadcore1
21949072111,3sleep20-21swapper/22
21839672111,5sleep11967206-21diskmemload1
21833922111,4sleep01967206-21diskmemload0
21570382111,4sleep11967206-21diskmemload1
21512162111,6sleep00-21swapper/00
21496202110,1sleep10-21swapper/11
20698832111,3sleep00-21swapper/00
20610372112,3sleep21967206-21diskmemload2
20264892110,2sleep00-21swapper/00
20081742112,4sleep01967206-21diskmemload0
20003602111,3sleep20-21swapper/22
19941882111,5sleep11994323-21diskstats1
19846272111,5sleep21984708-21munin-run2
19833422111,3sleep00-21swapper/00
19622512111,5sleep11962435-21diskstats1
19621692110,7sleep01962247-21grep0
19526932114,3sleep21953563-21kthreadcore2
19340832111,4sleep21934171-21ntpq2
19182102111,2sleep1171rcu_preempt1
18880062111,5sleep11888077-21irqstats1
18765822110,1sleep10-21swapper/11
18608402111,5sleep11861043-21diskstats1
18542182111,5sleep11854272-21ntpq1
23141732101,4sleep10-21swapper/11
22994012101,2sleep00-21swapper/00
22810902101,4sleep12281104-21ssh1
22625422101,4sleep21967206-21diskmemload2
22541742101,5sleep01967206-21diskmemload0
22379042101,3sleep10-21swapper/11
21921542101,2sleep2171rcu_preempt2
21670762101,3sleep00-21swapper/00
21418982101,3sleep10-21swapper/11
21418982101,3sleep10-21swapper/11
21413562101,4sleep21967206-21diskmemload2
21413562101,4sleep21967206-21diskmemload2
21215902101,3sleep20-21swapper/22
21131702101,5sleep02113181-21ssh0
20977362101,3sleep10-21swapper/11
20813362101,3sleep20-21swapper/22
20801662101,5sleep02080181-21ssh0
20689832101,3sleep10-21swapper/11
20498562100,3sleep00-21swapper/00
20465122101,5sleep22046589-21ssh2
19913552101,5sleep00-21swapper/00
19718472101,4sleep11971910-21ntp_states1
19559472101,5sleep01956164-21users0
19454002101,5sleep21945563-21/usr/sbin/munin2
18554282101,4sleep01855447-21apt-get0
18541672101,5sleep01854216-21ntpq0
18447302101,4sleep11844940-21hddtemp_smartct1
2333452291,3sleep22333465-21ssh2
2319154291,5sleep22320822-21kthreadcore2
2316289290,1sleep20-21swapper/22
2296600291,4sleep21967206-21diskmemload2
2287652291,2sleep00-21swapper/00
2275440291,4sleep22277104-21kthreadcore2
2262120291,2sleep10-21swapper/11
2245993291,2sleep20-21swapper/22
2232606290,5sleep02234209-21latency0
2200664291,3sleep11967206-21diskmemload1
2193742291,2sleep00-21swapper/00
2159933291,3sleep11967206-21diskmemload1
2037748291,3sleep22037783-21ssh2
2037748291,3sleep22037783-21ssh2
2016942291,4sleep12016953-21ssh1
2008402291,2sleep20-21swapper/22
1990714291,4sleep11990727-21ssh1
1948103291,4sleep01949924-21cut0
1946190291,3sleep21946209-21apt-get2
1898717291,4sleep01899022-21grep0
1882204292,2sleep20-21swapper/22
1876878290,3sleep01876897-21apt-get0
1871576290,4sleep21871595-21apt-get2
1972902280,2sleep01967206-21diskmemload0
1956838281,3sleep01956863-21grep0
1934889283,2sleep0251irq_work/00
1925350281,3sleep21925796-21kthreadcore2
1902556281,3sleep21902666-21timerandwakeup2
1893146281,3sleep11893284-21grep1
1866657281,3sleep01867354-21kthreadcore0
1865188280,5sleep21860756-21/usr/sbin/munin2
1861327280,4sleep01862587-21kthreadcore0
1856039281,3sleep11857686-21kthreadcore1
1850371280,5sleep21850500-21gltestperf2
1849492280,1sleep00-21swapper/00
1960231271,3sleep21960791-21needreboot2
1941595271,3sleep01943275-21kthreadcore0
1934286270,3sleep11934371-21ps1
1925134270,3sleep11925347-21if_enp1s01
1920851271,3sleep11921584-21kthreadcore1
1919496270,1sleep20-21swapper/22
1908858271,2sleep20-21swapper/22
1908480270,1sleep10-21swapper/11
1908413270,1sleep20-21swapper/22
1887000270,1sleep00-21swapper/00
1860216270,1sleep20-21swapper/22
1967086260,1sleep20-21swapper/22
1951471260,2sleep10-21swapper/11
1945587260,1sleep10-21swapper/11
1940264260,4sleep00-21swapper/00
1939614260,2sleep10-21swapper/11
1929515260,3sleep00-21swapper/00
1918078261,2sleep00-21swapper/00
1913810260,4sleep00-21swapper/00
1904579261,2sleep00-21swapper/00
1899023261,2sleep10-21swapper/11
1891887261,2sleep00-21swapper/00
1888078261,2sleep20-21swapper/22
1886971260,1sleep10-21swapper/11
1881652260,1sleep20-21swapper/22
1876900261,2sleep10-21swapper/11
1876329260,1sleep00-21swapper/00
1841878260,1sleep20-21swapper/22
1892850240,2sleep20-21swapper/22
1836832240,1sleep10-21swapper/11
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional