You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-16 - 17:38

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3s.osadl.org (updated Sat May 16, 2026 12:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
307310021230,5sleep03069523-21kworker/u16:1-writeback0
307310021230,5sleep03069523-21kworker/u16:1-writeback0
2846909212152,62sleep30-21swapper/33
284726528872,10sleep20-21swapper/22
284723928664,12sleep10-21swapper/11
284724328458,6sleep00-21swapper/00
284773499249,9cyclictest3004217-21taskset3
2847734992317,4cyclictest0-21swapper/33
33049342222,8sleep23305033-21ntpq2
2847734992118,2cyclictest0-21swapper/33
2847734992111,6cyclictest66950irq/132-enp1s0-TxRx-33
284773499211,16cyclictest3232960-21taskset3
284773499211,14cyclictest0-21swapper/33
31312252201,8sleep03131236-21ssh0
2847734992017,1cyclictest0-21swapper/33
284773499201,15cyclictest0-21swapper/33
284773499201,13cyclictest0-21swapper/33
32206712191,6sleep03220732-21ntpq0
284773499199,6cyclictest2881841-21turbostat3
284773499191,7cyclictest3209764-21taskset3
2847734991915,3cyclictest66950irq/132-enp1s0-TxRx-33
284773499191,2cyclictest0-21swapper/33
284773499191,15cyclictest3299188-21taskset3
284773499191,15cyclictest0-21swapper/33
284773499191,15cyclictest0-21swapper/33
284773499191,14cyclictest0-21swapper/33
284773499191,14cyclictest0-21swapper/33
284773499191,14cyclictest0-21swapper/33
284773499191,14cyclictest0-21swapper/33
32732612181,3sleep00-21swapper/00
31701212181,13sleep03170185-21ssh0
30047202181,3sleep20-21swapper/22
290876621811,4sleep22908841-21tr2
284773499181,4cyclictest3306156-21taskset3
2847734991814,2cyclictest0-21swapper/33
2847734991812,4cyclictest0-21swapper/33
2847734991811,5cyclictest32670512chrt3
284773499181,14cyclictest0-21swapper/33
284773499181,14cyclictest0-21swapper/33
284773499181,14cyclictest0-21swapper/33
284773499181,14cyclictest0-21swapper/33
284773499181,14cyclictest0-21swapper/33
284773499181,14cyclictest0-21swapper/33
284773499181,14cyclictest0-21swapper/33
284773499181,14cyclictest0-21swapper/33
284773499181,14cyclictest0-21swapper/33
284773499181,13cyclictest0-21swapper/33
30546162171,2sleep20-21swapper/22
29140722170,4sleep12914325-21df1
2847734991712,3cyclictest0-21swapper/33
2847734991711,4cyclictest2908741-21chrt3
284773499171,13cyclictest0-21swapper/33
284773499171,13cyclictest0-21swapper/33
284773499171,13cyclictest0-21swapper/33
284773499171,13cyclictest0-21swapper/33
284773499171,12cyclictest0-21swapper/33
284773499170,15cyclictest0-21swapper/33
284773499170,15cyclictest0-21swapper/33
284773499170,14cyclictest0-21swapper/33
284773499170,14cyclictest0-21swapper/33
284773499170,14cyclictest0-21swapper/33
284773499170,14cyclictest0-21swapper/33
284773499170,13cyclictest0-21swapper/33
31552762160,3sleep20-21swapper/22
31145552161,3sleep00-21swapper/00
30219882161,3sleep03023597-21cut0
30177362161,4sleep12978136-21diskmemload1
2847734991613,1cyclictest0-21swapper/33
2847734991612,2cyclictest0-21swapper/33
284773499161,14cyclictest0-21swapper/33
2847734991611,3cyclictest0-21swapper/33
284773499160,13cyclictest0-21swapper/33
284773499160,13cyclictest0-21swapper/33
284773499160,13cyclictest0-21swapper/33
284773499160,13cyclictest0-21swapper/33
284773499160,12cyclictest0-21swapper/33
33148812150,3sleep244-21ksoftirqd/22
32328792150,1sleep00-21swapper/00
32049242150,1sleep00-21swapper/00
31373382151,9sleep22978136-21diskmemload2
31286272155,4sleep1331ktimers/11
31219932150,3sleep10-21swapper/11
31044802151,3sleep20-21swapper/22
30802212150,3sleep20-21swapper/22
30802212150,3sleep20-21swapper/22
30694172151,4sleep03069428-21sh0
30615512151,3sleep12978136-21diskmemload1
30361502151,3sleep20-21swapper/22
30123492151,4sleep02978136-21diskmemload0
30009422151,3sleep22978136-21diskmemload2
29873462151,2sleep10-21swapper/11
2847734991513,1cyclictest0-21swapper/33
2847734991512,1cyclictest0-21swapper/33
284773499151,12cyclictest0-21swapper/33
284773499150,12cyclictest2902800-21chrt3
284773499150,12cyclictest0-21swapper/33
284773499150,11cyclictest0-21swapper/33
32576742142,8sleep02978136-21diskmemload0
32218442141,3sleep12978136-21diskmemload1
31895742141,8sleep02978136-21diskmemload0
31800522141,9sleep02978136-21diskmemload0
31552102141,4sleep12978136-21diskmemload1
30265792140,2sleep20-21swapper/22
29512032141,3sleep12951195-21fwupd1
28814622140,9sleep12881412-31find1
28735522140,2sleep244-21ksoftirqd/22
28712782140,1sleep10-21swapper/11
2847734991411,2cyclictest0-21swapper/33
284773499140,2cyclictest0-21swapper/33
284773499140,13cyclictest0-21swapper/33
284773499140,12cyclictest0-21swapper/33
33324782131,7sleep12978136-21diskmemload1
32741622131,7sleep20-21swapper/22
32367182131,2sleep10-21swapper/11
31974962131,3sleep13197505-21ssh1
31624042131,5sleep00-21swapper/00
31565012131,7sleep02978136-21diskmemload0
31478772131,7sleep02978136-21diskmemload0
31211532131,7sleep22978136-21diskmemload2
31118082131,8sleep23111822-21ssh2
30803502130,3sleep1321rcuc/11
30803502130,3sleep1321rcuc/11
30513362131,8sleep03051398-21ntpq0
29912942131,6sleep02991351-21ntpq0
29087442130,2sleep10-21swapper/11
2847734991311,1cyclictest0-21swapper/33
32861752121,6sleep13287781-21sed1
32746082121,6sleep13274619-21ssh1
32409262121,3sleep0802-21snmpd0
32159902120,3sleep03216001-21ssh0
32142592121,4sleep10-21swapper/11
31616252121,6sleep13161676-21ntpq1
30629992121,6sleep22978136-21diskmemload2
30270722121,6sleep13027083-21ssh1
30119192121,5sleep10-21swapper/11
29854222124,4sleep22985432-21ssh2
29854222124,4sleep22985432-21ssh2
33277852111,6sleep03329536-21grep0
33230612110,3sleep00-21swapper/00
33065152111,4sleep10-21swapper/11
33001402111,3sleep00-21swapper/00
32919242111,6sleep03292101-21latency_hist0
32813362111,3sleep20-21swapper/22
32570662111,3sleep10-21swapper/11
32406222111,3sleep20-21swapper/22
32256532111,6sleep13225687-21kthreadcore1
32228802110,7sleep23222890-21ssh2
31819782111,4sleep22978136-21diskmemload2
31734382111,2sleep20-21swapper/22
31449532111,5sleep13145023-21ntp_states1
31061742112,4sleep02978136-21diskmemload0
30975622111,3sleep10-21swapper/11
30865022111,5sleep12978136-21diskmemload1
30459292111,4sleep23045940-21sshd2
30435072111,7sleep13043518-21ssh1
30368042111,6sleep13036807-21ssh1
30007792111,4sleep12978136-21diskmemload1
29784672111,5sleep02978136-21diskmemload0
29784672111,5sleep02978136-21diskmemload0
29558502111,6sleep22955918-21pmu-power2
28607112111,5sleep22860927-21diskstats2
33207542101,6sleep23321611-21missed_timers2
33190612101,5sleep13320749-21ssh1
33148032101,4sleep02978136-21diskmemload0
33068062101,2sleep00-21swapper/00
33003422101,4sleep12978136-21diskmemload1
32964432100,5sleep23296491-21ntpq2
32898282101,2sleep20-21swapper/22
32818282101,3sleep00-21swapper/00
32803422101,2sleep10-21swapper/11
32586352101,3sleep00-21swapper/00
32480952101,4sleep02978136-21diskmemload0
32479682101,4sleep22978136-21diskmemload2
32319672101,4sleep22978136-21diskmemload2
32156722101,2sleep20-21swapper/22
32049862101,4sleep12978136-21diskmemload1
32048962102,3sleep22978136-21diskmemload2
31988742101,5sleep23198878-21ssh2
31720242101,3sleep10-21swapper/11
31616782101,2sleep20-21swapper/22
31381892102,3sleep12978136-21diskmemload1
31378142100,2sleep00-21swapper/00
31217472101,3sleep00-21swapper/00
30963722100,1sleep00-21swapper/00
30939102101,4sleep23093969-21ntpq2
30871932101,3sleep22978136-21diskmemload2
30641332101,5sleep23064299-21cstates2
30640992101,4sleep10-21swapper/11
30561272101,6sleep03056177-21kthreadcore0
30544602101,2sleep10-21swapper/11
30355002101,2sleep00-21swapper/00
29938572101,2sleep20-21swapper/22
29798302101,4sleep12981480-21cut1
29798302101,4sleep12981480-21cut1
29730202100,5sleep22973038-21apt-get2
29720702102,4sleep0151ktimers/00
29504582101,4sleep22950514-21ntp_states2
29415482100,6sleep02943054-21cut0
29140502101,4sleep02914069-21apt-get0
29037082101,4sleep02903921-21hddtemp_smartct0
28996192101,6sleep10-21swapper/11
28871732101,4sleep12887221-21munin-run1
28813522101,5sleep0255-21systemd-journal0
28753222101,4sleep02875368-21ntpq0
28495112100,6sleep02851315-21cut0
3330871291,2sleep20-21swapper/22
3309226291,2sleep10-21swapper/11
3263094290,5sleep20-21swapper/22
3258983291,3sleep12978136-21diskmemload1
3256358290,3sleep22978136-21diskmemload2
3246870291,4sleep12978136-21diskmemload1
3191871291,4sleep03192687-21grep0
3188749291,4sleep13188760-21ssh1
3188050290,5sleep23188062-21ssh2
3179661291,3sleep12978136-21diskmemload1
3145971291,3sleep22978136-21diskmemload2
3111847291,3sleep12978136-21diskmemload1
3099353291,4sleep13100961-21tr1
3086147291,2sleep00-21swapper/00
3044190291,2sleep00-21swapper/00
3017899291,2sleep20-21swapper/22
3013982291,5sleep03015618-21kthreadcore0
3000913291,3sleep02978136-21diskmemload0
2976993291,4sleep12977110-21ld1
2963581291,5sleep02965324-21kthreadcore0
2959714291,4sleep12960891-21memory1
2952234291,4sleep02953060-21grep0
2924701291,4sleep12924954-21df1
2907437291,3sleep12907485-21ntpq1
2898636292,4sleep02899615-21kthreadcore0
2896910291,4sleep22896998-21ps2
2887455290,5sleep22887611-21cstates2
2866247291,4sleep22866317-21fschecks_time2
3124019281,3sleep22978136-21diskmemload2
2968249281,4sleep22969576-21grep2
2967345281,4sleep1802-21snmpd1
2955798281,3sleep12955843-21ntpq1
2946813281,4sleep02946888-21kernelversion0
2945738280,2sleep20-21swapper/22
2931769281,3sleep22933563-21kthreadcore2
2929386280,1sleep00-21swapper/00
2924397281,3sleep22924642-21latency_hist2
2924203281,3sleep12924204-21pool-udisksd1
2919099280,1sleep20-21swapper/22
2892939281,3sleep12893051-21/usr/sbin/munin1
2892544280,1sleep00-21swapper/00
2887433280,3sleep10-21swapper/11
2886290280,1sleep20-21swapper/22
2880822281,3sleep22880948-21run-parts2
2853313281,3sleep00-21swapper/00
2851890280,2sleep20-21swapper/22
2967676271,2sleep12967695-21apt-get1
2940217270,1sleep10-21swapper/11
2929498270,1sleep20-21swapper/22
2924349270,1sleep00-21swapper/00
2908142270,1sleep20-21swapper/22
2882706271,2sleep02884220-21kthreadcore0
2865966270,2sleep00-21swapper/00
2865549270,1sleep10-21swapper/11
2855263270,2sleep20-21swapper/22
2852864270,1sleep10-21swapper/11
2978101260,1sleep00-21swapper/00
2961810260,4sleep00-21swapper/00
2945737260,4sleep10-21swapper/11
2940290260,1sleep00-21swapper/00
2934844260,4sleep00-21swapper/00
2930580261,2sleep10-21swapper/11
2902930260,2sleep20-21swapper/22
2892166260,1sleep00-21swapper/00
2867168261,2sleep10-21swapper/11
2860042260,1sleep10-21swapper/11
2966235251,1sleep20-21swapper/22
2957400250,1sleep20-21swapper/22
2909034250,1sleep00-21swapper/00
2861139250,1sleep00-21swapper/00
2935687240,1sleep20-21swapper/22
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional