You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-08 - 15:14

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack3slot3s.osadl.org (updated Wed Jul 08, 2026 12:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2566640211962,53sleep30-21swapper/33
256695428362,14sleep00-21swapper/00
256675828063,11sleep10-21swapper/11
256661927962,11sleep20-21swapper/22
2567094992416,5cyclictest66950irq/132-enp1s0-TxRx-33
27552602221,16sleep22756876-21cut2
256709499222,14cyclictest2720177-21taskset3
256709499221,18cyclictest0-21swapper/33
256709499211,15cyclictest0-21swapper/33
29427112201,2sleep10-21swapper/11
28693692201,8sleep12694840-21diskmemload1
256709499206,11cyclictest0-21swapper/33
256709499201,15cyclictest0-21swapper/33
256709499201,15cyclictest0-21swapper/33
2567094992010,7cyclictest0-21swapper/33
2567094991912,4cyclictest30019312sleep33
256709499191,15cyclictest66950irq/132-enp1s0-TxRx-33
256709499191,15cyclictest66950irq/132-enp1s0-TxRx-33
256709499191,15cyclictest0-21swapper/33
256709499191,15cyclictest0-21swapper/33
256709499191,14cyclictest0-21swapper/33
256709499191,14cyclictest0-21swapper/33
256709499191,14cyclictest0-21swapper/33
256709499191,14cyclictest0-21swapper/33
256709499191,14cyclictest0-21swapper/33
256709499191,14cyclictest0-21swapper/33
256709499190,16cyclictest0-21swapper/33
30022022180,1sleep20-21swapper/22
29036012180,3sleep00-21swapper/00
28510012181,3sleep00-21swapper/00
26833852181,5sleep12683438-21ntp_states1
25713172181,5sleep22571370-21ntp_states2
2567094991815,2cyclictest0-21swapper/33
256709499181,14cyclictest0-21swapper/33
256709499181,14cyclictest0-21swapper/33
256709499181,14cyclictest0-21swapper/33
256709499181,14cyclictest0-21swapper/33
256709499181,14cyclictest0-21swapper/33
256709499181,14cyclictest0-21swapper/33
256709499181,14cyclictest0-21swapper/33
256709499181,14cyclictest0-21swapper/33
256709499181,14cyclictest0-21swapper/33
256709499181,14cyclictest0-21swapper/33
256709499181,14cyclictest0-21swapper/33
256709499181,14cyclictest0-21swapper/33
256709499181,13cyclictest0-21swapper/33
256709499181,13cyclictest0-21swapper/33
256709499181,13cyclictest0-21swapper/33
2567094991810,4cyclictest28957012sleep33
256709499180,14cyclictest0-21swapper/33
29793122171,3sleep00-21swapper/00
29029692171,3sleep10-21swapper/11
27601082171,3sleep10-21swapper/11
27427202171,2sleep20-21swapper/22
256709499176,8cyclictest2673900-21turbostat3
256709499171,13cyclictest0-21swapper/33
256709499171,13cyclictest0-21swapper/33
256709499171,13cyclictest0-21swapper/33
256709499171,13cyclictest0-21swapper/33
256709499171,13cyclictest0-21swapper/33
256709499171,13cyclictest0-21swapper/33
256709499171,12cyclictest0-21swapper/33
256709499170,14cyclictest0-21swapper/33
256709499170,14cyclictest0-21swapper/33
256709499170,13cyclictest0-21swapper/33
30048612161,7sleep13005781-21cut1
29953602161,2sleep20-21swapper/22
29847732161,2sleep10-21swapper/11
29617542161,2sleep10-21swapper/11
29439212161,2sleep00-21swapper/00
28449782161,2sleep20-21swapper/22
28103252160,3sleep10-21swapper/11
27329672161,7sleep02733239-21ssh0
256709499169,4cyclictest2576432-21taskset3
256709499161,3cyclictest0-21swapper/33
2567094991611,3cyclictest0-21swapper/33
2567094991611,3cyclictest0-21swapper/33
2567094991611,3cyclictest0-21swapper/33
256709499161,12cyclictest0-21swapper/33
256709499161,11cyclictest0-21swapper/33
256709499160,13cyclictest0-21swapper/33
256709499160,13cyclictest0-21swapper/33
256709499160,13cyclictest0-21swapper/33
256709499160,13cyclictest0-21swapper/33
29682792151,3sleep20-21swapper/22
29682792151,3sleep20-21swapper/22
29284042151,2sleep00-21swapper/00
29202982150,2sleep00-21swapper/00
29082862151,10sleep02908368-21grep0
28933562152,3sleep2171rcu_preempt2
28090172151,2sleep20-21swapper/22
27954782151,9sleep02795746-21df0
27936222151,2sleep10-21swapper/11
27787902151,9sleep22779011-21diskstats2
26808602151,10sleep22682558-21cut2
26112352154,7sleep22611480-21expr2
256709499151,4cyclictest0-21swapper/33
2567094991512,2cyclictest0-21swapper/33
2567094991511,2cyclictest0-21swapper/33
256709499151,11cyclictest2616147-21chrt3
30109572141,3sleep23010959-21ssh2
28706542141,7sleep12870920-21df1
28458112141,1sleep10-21swapper/11
26741782141,8sleep22674435-21df2
25722592141,4sleep12572312-21sh1
2567094991412,1cyclictest0-21swapper/33
2567094991411,2cyclictest0-21swapper/33
2567094991411,1cyclictest0-21swapper/33
30446442131,7sleep03044654-21ssh0
29856952131,6sleep22694840-21diskmemload2
29287862131,7sleep12928800-21ssh1
29260982131,6sleep20-21swapper/22
28590222131,4sleep22859033-21sh2
28292982131,2sleep00-21swapper/00
27946102131,5sleep00-21swapper/00
27688382130,6sleep244-21ksoftirqd/22
27178722131,8sleep12717886-21ssh1
27020432131,4sleep12702057-21ssh1
26957912131,8sleep20-21swapper/22
26944982130,1sleep20-21swapper/22
26383502131,8sleep12639963-21grep1
25745202130,2sleep10-21swapper/11
2567094991311,1cyclictest0-21swapper/33
2567094991310,1cyclictest0-21swapper/33
30275772121,3sleep13027587-21ssh1
30132262121,6sleep03014178-21cut0
29952092121,7sleep02995220-21sh0
29940992120,2sleep10-21swapper/11
29780232121,6sleep02978033-21ssh0
29700002120,6sleep02694840-21diskmemload0
29700002120,6sleep02694840-21diskmemload0
29520342121,7sleep02694840-21diskmemload0
29465532121,6sleep22947263-21cut2
29348762121,7sleep02934886-21ssh0
28858492121,4sleep00-21swapper/00
28624912121,6sleep22862527-21fschecks_count2
28281322121,5sleep20-21swapper/22
28273512120,5sleep10-21swapper/11
28200162121,7sleep02694840-21diskmemload0
28195722121,2sleep20-21swapper/22
28178732121,5sleep12694840-21diskmemload1
28027472121,5sleep20-21swapper/22
27842522120,2sleep10-21swapper/11
27766572121,7sleep12776666-21ssh1
27684952121,7sleep12768505-21ssh1
27519262121,7sleep22751938-21rm2
27253812121,7sleep10-21swapper/11
26578852120,1sleep10-21swapper/11
26484932121,7sleep12648892-21cut1
26101222121,7sleep12610245-21/usr/sbin/munin1
26101222121,7sleep12610245-21/usr/sbin/munin1
25723142121,7sleep22572524-21munin-node2
30342492111,4sleep22694840-21diskmemload2
30291332111,4sleep03029155-21apt-get0
30186012111,4sleep10-21swapper/11
30158032111,4sleep23016613-21memory2
30042352111,4sleep03004261-21apt-get0
30010612110,2sleep03001072-21ssh0
29965722111,6sleep12996935-21cut1
29449232111,6sleep22944936-21diskmemload2
29345802111,4sleep22694840-21diskmemload2
29193262112,4sleep22694840-21diskmemload2
29177712110,5sleep134-21ksoftirqd/11
29090742111,4sleep12694840-21diskmemload1
28868082112,5sleep22852482-21kworker/u16:1+xprtiod2
28687892111,5sleep02694840-21diskmemload0
28508612110,5sleep20-21swapper/22
28436002110,7sleep12843611-21ssh1
28426502111,5sleep02694840-21diskmemload0
28287792111,5sleep12829040-21df1
27919842111,3sleep20-21swapper/22
27705302111,5sleep02770214-21/usr/sbin/munin0
27672832111,4sleep02694840-21diskmemload0
27416052111,5sleep02694840-21diskmemload0
27368002110,7sleep12736806-21ssh1
27341442110,1sleep20-21swapper/22
27188302111,6sleep02694840-21diskmemload0
27100232111,3sleep10-21swapper/11
26901482111,5sleep02690362-21hddtemp_smartct0
26793882111,5sleep02679407-21apt-get0
26633892111,6sleep02663552-21latency_hist0
25806402112,5sleep12582246-21kthreadcore1
30422692101,2sleep13042280-21ssh1
30269182101,2sleep00-21swapper/00
30214022101,2sleep20-21swapper/22
29783632101,4sleep12694840-21diskmemload1
29698612101,3sleep10-21swapper/11
29698612101,3sleep10-21swapper/11
29615262101,3sleep20-21swapper/22
29592442101,4sleep02694840-21diskmemload0
29504322100,6sleep12694840-21diskmemload1
29040742101,4sleep22904105-21ssh2
29004822100,1sleep20-21swapper/22
28927972101,3sleep12694840-21diskmemload1
28872262101,3sleep02694840-21diskmemload0
28789422105,2sleep10-21swapper/11
28766472101,3sleep00-21swapper/00
28706012101,5sleep22870613-21ssh2
28617042101,2sleep10-21swapper/11
28608852101,3sleep00-21swapper/00
28359182101,3sleep22694840-21diskmemload2
28203722101,2sleep00-21swapper/00
28114842101,2sleep00-21swapper/00
27775692101,2sleep20-21swapper/22
27600072101,3sleep00-21swapper/00
27452642101,3sleep12745293-21apt-get1
27284402100,1sleep10-21swapper/11
27255152101,4sleep02694840-21diskmemload0
27173632101,4sleep22694840-21diskmemload2
27113342101,3sleep20-21swapper/22
27013242101,4sleep02694840-21diskmemload0
26864052101,5sleep12688135-21kthreadcore1
26533662101,4sleep02653489-21diskstats0
26529292101,5sleep22653071-21latency_hist2
26112102101,3sleep12611228-21apt-get1
3037483291,3sleep23037506-21apt-get2
3034577291,2sleep10-21swapper/11
2976034290,1sleep20-21swapper/22
2929526291,4sleep12929195-21/usr/sbin/munin1
2778682291,2sleep00-21swapper/00
2752353290,3sleep00-21swapper/00
2725248291,2sleep20-21swapper/22
2706815291,3sleep02694840-21diskmemload0
2693849291,4sleep12689855-21/usr/sbin/munin1
2668915291,2sleep20-21swapper/22
2663681290,4sleep22663700-21apt-get2
2663147290,3sleep272-21kcompactd02
2642499291,4sleep1205-21jbd2/sda2-81
2637459291,3sleep00-21swapper/00
2600854291,3sleep22601057-21diskstats2
2600527291,4sleep1555-21dbus-daemon1
2590209291,2sleep20-21swapper/22
2579781290,6sleep02579773-21apt-get0
2802067280,2sleep10-21swapper/11
2653199281,2sleep20-21swapper/22
2647490280,1sleep00-21swapper/00
2647006280,4sleep22647244-21sed2
2632495281,3sleep02632615-21fschecks_time0
2621697281,2sleep22621716-21apt-get2
2616444280,2sleep10-21swapper/11
2610791280,1sleep20-21swapper/22
2610791280,1sleep20-21swapper/22
2586385281,3sleep10-21swapper/11
2579273280,1sleep00-21swapper/00
2684918270,3sleep20-21swapper/22
2674163270,2sleep10-21swapper/11
2670741271,3sleep02672472-21kthreadcore0
2668611270,1sleep00-21swapper/00
2652716270,1sleep00-21swapper/00
2642199270,1sleep20-21swapper/22
2637058271,4sleep20-21swapper/22
2632205270,2sleep10-21swapper/11
2626937270,2sleep20-21swapper/22
2626916271,2sleep12626935-21apt-get1
2626644270,1sleep00-21swapper/00
2626395270,1sleep10-21swapper/11
2620260270,2sleep02620433-21strings0
2616164270,1sleep00-21swapper/00
2610621270,1sleep00-21swapper/00
2610621270,1sleep00-21swapper/00
2600745271,2sleep00-21swapper/00
2600123270,1sleep20-21swapper/22
2600123270,1sleep20-21swapper/22
2600106270,1sleep00-21swapper/00
2600106270,1sleep00-21swapper/00
2595462270,2sleep10-21swapper/11
2595462270,2sleep10-21swapper/11
2590509271,2sleep12590721-21smartctl1
2589838270,1sleep00-21swapper/00
2584410270,1sleep20-21swapper/22
2567256270,1sleep00-21swapper/00
2684614261,2sleep00-21swapper/00
2678896260,1sleep00-21swapper/00
2673663260,1sleep10-21swapper/11
2663674261,2sleep10-21swapper/11
2659055261,2sleep10-21swapper/11
2616466260,2sleep22616626-21cstates2
2585291260,3sleep2431ktimers/22
2627199250,1sleep00-21swapper/00
2594916250,1sleep00-21swapper/00
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional