You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-10 - 11:09

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack3slot3s.osadl.org (updated Mon Aug 10, 2026 00:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2837642213162,65sleep30-21swapper/33
2836624210990,10sleep20-21swapper/22
283750629485,6sleep10-21swapper/11
283261127459,7sleep00-21swapper/00
2837866992212,7cyclictest2871306-21sleep3
31302252211,5sleep03125003-21sendmail-msp0
2837866992119,1cyclictest0-21swapper/33
283786699211,16cyclictest0-21swapper/33
283786699211,16cyclictest0-21swapper/33
2837866992017,1cyclictest0-21swapper/33
283786699201,3cyclictest0-21swapper/33
283786699201,16cyclictest0-21swapper/33
283786699201,15cyclictest3222149-21taskset3
283786699201,14cyclictest0-21swapper/33
283786699201,14cyclictest0-21swapper/33
33140552191,9sleep13314875-21grep1
32857942190,7sleep134-21ksoftirqd/11
32690282193,11sleep23269022-21apt-get2
32689922191,9sleep13266705-40find1
31936442191,3sleep13193657-21sh1
31826042191,2sleep00-21swapper/00
283786699192,14cyclictest66950irq/132-enp1s0-TxRx-33
283786699191,15cyclictest0-21swapper/33
283786699191,14cyclictest0-21swapper/33
283786699191,14cyclictest0-21swapper/33
283786699191,14cyclictest0-21swapper/33
283786699191,14cyclictest0-21swapper/33
283786699191,13cyclictest2886998-21taskset3
283786699190,16cyclictest0-21swapper/33
32284202181,2sleep20-21swapper/22
29345592181,12sleep22934337-21sendmail-msp2
283786699189,6cyclictest3245321-21taskset3
283786699189,6cyclictest2860855-21turbostat3
2837866991812,3cyclictest66950irq/132-enp1s0-TxRx-33
283786699181,14cyclictest66950irq/132-enp1s0-TxRx-33
283786699181,14cyclictest0-21swapper/33
283786699181,14cyclictest0-21swapper/33
283786699181,14cyclictest0-21swapper/33
283786699181,14cyclictest0-21swapper/33
283786699181,14cyclictest0-21swapper/33
283786699181,14cyclictest0-21swapper/33
283786699181,14cyclictest0-21swapper/33
283786699181,13cyclictest0-21swapper/33
283786699181,13cyclictest0-21swapper/33
283786699181,13cyclictest0-21swapper/33
283786699181,13cyclictest0-21swapper/33
283786699181,13cyclictest0-21swapper/33
283786699181,13cyclictest0-21swapper/33
283786699181,13cyclictest0-21swapper/33
283786699181,13cyclictest0-21swapper/33
283786699180,15cyclictest0-21swapper/33
283786699180,14cyclictest0-21swapper/33
31142722171,2sleep20-21swapper/22
305137421710,3sleep23051388-21ssh2
30082012171,5sleep23008214-21ssh2
283786699178,6cyclictest2881845-21turbostat3
283786699171,4cyclictest0-21swapper/33
2837866991714,1cyclictest0-21swapper/33
2837866991712,3cyclictest0-21swapper/33
283786699171,13cyclictest0-21swapper/33
283786699171,13cyclictest0-21swapper/33
283786699171,13cyclictest0-21swapper/33
283786699170,14cyclictest0-21swapper/33
283786699170,14cyclictest0-21swapper/33
283786699170,14cyclictest0-21swapper/33
283786699170,14cyclictest0-21swapper/33
33032902161,4sleep12965764-21diskmemload1
32289732161,3sleep00-21swapper/00
316630321612,1sleep10-21swapper/11
29814212161,2sleep10-21swapper/11
283786699162,9cyclictest2845317-21taskset3
283786699161,3cyclictest0-21swapper/33
283786699161,3cyclictest0-21swapper/33
283786699161,12cyclictest0-21swapper/33
283786699160,13cyclictest29236962chrt3
283786699160,13cyclictest0-21swapper/33
283786699160,13cyclictest0-21swapper/33
283786699160,13cyclictest0-21swapper/33
283786699160,13cyclictest0-21swapper/33
33104532151,2sleep00-21swapper/00
32226502151,4sleep13222679-21apt-get1
30949162151,9sleep02965764-21diskmemload0
30090262151,2sleep00-21swapper/00
29172822150,1sleep20-21swapper/22
29070752151,9sleep22907147-21sed2
28668372151,9sleep22866906-21irqstats2
283786699150,13cyclictest0-21swapper/33
283786699150,12cyclictest0-21swapper/33
32100912141,2sleep00-21swapper/00
31752362141,8sleep02965764-21diskmemload0
31239582141,9sleep13123971-21ssh1
31047862140,6sleep02965764-21diskmemload0
30806262141,9sleep23080637-21ssh2
30195822141,3sleep13019698-21perl1
30180592144,4sleep0151ktimers/00
29079902141,8sleep01-21systemd0
283786699149,3cyclictest29554822sleep33
2837866991412,1cyclictest0-21swapper/33
283786699140,2cyclictest0-21swapper/33
33314182131,8sleep03331649-21basename0
32907752131,8sleep03290920-21/usr/sbin/munin0
32868162131,5sleep10-21swapper/11
32544712131,7sleep12965764-21diskmemload1
32188762131,3sleep02965764-21diskmemload0
32155202131,8sleep23217137-21cut2
32024162130,3sleep02965764-21diskmemload0
31848802131,3sleep23184893-21ssh2
31431192131,6sleep03143135-21ssh0
31236272131,8sleep03123640-21ssh0
30992932131,7sleep23098899-21/usr/sbin/munin2
30619252131,6sleep12965764-21diskmemload1
30613552131,5sleep20-21swapper/22
30419662131,8sleep23041981-21ssh2
29927282131,5sleep02992763-21ssh0
29815692133,2sleep0171rcu_preempt0
29722482130,3sleep02972263-21ssh0
283786699138,4cyclictest2929311-21chrt3
2837866991311,1cyclictest0-21swapper/33
33305212121,7sleep20-21swapper/22
33210532121,7sleep23321066-21ssh2
32709162121,3sleep03272146-21grep0
32633532121,2sleep10-21swapper/11
32016942121,7sleep23201711-21sh2
31600022121,7sleep23160017-21ssh2
31393632121,7sleep13139375-21ssh1
31118692121,7sleep13107687-21/usr/sbin/munin1
30817352123,6sleep00-21swapper/00
30624902121,7sleep03062504-21ssh0
30431712121,7sleep13043183-21sh1
29293322121,5sleep02929351-21apt-get0
29188422122,4sleep22918845-21apt-get2
29095932121,7sleep12911276-21cut1
28840592120,7sleep0221rcuc/00
33384112111,6sleep13338421-21ssh1
33193722111,6sleep02965764-21diskmemload0
33109142111,5sleep22965764-21diskmemload2
33103102110,7sleep12965764-21diskmemload1
32840272111,3sleep00-21swapper/00
32657172111,6sleep23265730-21ssh2
32643542111,4sleep02965764-21diskmemload0
32463112111,6sleep23246320-21sh2
32101622111,5sleep12965764-21diskmemload1
32094832111,6sleep23209690-21grep2
31935272110,7sleep02965764-21diskmemload0
31710772111,6sleep23172703-21kthreadcore2
31209112111,6sleep23121135-21timerwakeupswit2
31157362111,4sleep00-21swapper/00
30882572111,3sleep20-21swapper/22
30777132110,1sleep00-21swapper/00
30762192110,2sleep13076394-21strings1
30706812110,6sleep22965764-21diskmemload2
30675922111,5sleep03067659-21ntp_states0
30343672111,4sleep12965764-21diskmemload1
30280242111,5sleep20-21swapper/22
30251162111,3sleep20-21swapper/22
30167002111,4sleep12965764-21diskmemload1
29927002111,3sleep20-21swapper/22
29920722111,4sleep22965764-21diskmemload2
29596172110,7sleep02959705-21ps0
29403422111,2sleep20-21swapper/22
29231602111,5sleep12923404-21users1
29135442110,1sleep00-21swapper/00
29122772111,5sleep02912329-21ntp_states0
28558502111,4sleep22855869-21apt-get2
28380272111,4sleep12838046-21apt-get1
33272592101,4sleep02965764-21diskmemload0
32922002100,1sleep20-21swapper/22
32835242101,4sleep22965764-21diskmemload2
32493132101,4sleep03249527-21df_inode0
32455512100,3sleep10-21swapper/11
32367212101,4sleep12965764-21diskmemload1
32035882101,4sleep12965764-21diskmemload1
31956762101,5sleep23195790-21wc2
31743892101,4sleep12965764-21diskmemload1
31659962101,3sleep22965764-21diskmemload2
31592462100,1sleep00-21swapper/00
31578852101,4sleep12965764-21diskmemload1
31502952101,4sleep23150310-21ssh2
31483172101,3sleep10-21swapper/11
31409152100,6sleep23140928-21sh2
31345822101,2sleep0171rcu_preempt0
31325252100,1sleep10-21swapper/11
31050572101,3sleep10-21swapper/11
30907162101,5sleep13092152-21cut1
30899152101,4sleep23089617-21sendmail-msp2
30646042101,5sleep13066162-21grep1
30506702100,6sleep03050681-21ssh0
30369532101,4sleep03037221-21df0
30335532100,1sleep00-21swapper/00
30193112101,4sleep03019580-21df_inode0
30177732101,5sleep23017786-21ssh2
30046982101,4sleep13005634-21memory1
29915782101,4sleep12965764-21diskmemload1
29905102101,4sleep02965764-21diskmemload0
29663002101,4sleep12966515-21diskstats1
29660732100,1sleep20-21swapper/22
29502672101,3sleep20-21swapper/22
29441792101,4sleep12944371-21users1
29135142100,1sleep10-21swapper/11
29027572101,4sleep12902813-21munin-run1
28818262101,4sleep22881882-21munin-run2
28720712102,4sleep0205-21jbd2/sda2-80
28601412101,5sleep10-21swapper/11
28526682101,5sleep22850339-21sendmail-msp2
28505752101,3sleep00-21swapper/00
28505582101,4sleep12850339-21sendmail-msp1
3331819291,4sleep23333346-21cut2
3322165291,3sleep13321802-21apt-get1
3303115290,4sleep02965764-21diskmemload0
3300840291,4sleep23300852-21ssh2
3249154291,2sleep20-21swapper/22
3244605291,2sleep00-21swapper/00
3239478291,3sleep22965764-21diskmemload2
3231504291,4sleep03231116-21sendmail-msp0
3213811291,2sleep10-21swapper/11
3184342290,1sleep10-21swapper/11
3166177291,3sleep02965764-21diskmemload0
3130356291,3sleep22965764-21diskmemload2
3085333291,2sleep10-21swapper/11
3052266291,3sleep12965764-21diskmemload1
2999218291,2sleep10-21swapper/11
2975805291,5sleep20-21swapper/22
2965749291,4sleep12965978-21ssh1
2950536291,4sleep02950559-21forks0
2903049291,3sleep12903052-21apt-get1
2898715291,4sleep02900338-21cut0
2892580290,1sleep10-21swapper/11
2887334291,2sleep20-21swapper/22
2868043291,4sleep02869772-21kthreadcore0
2865428291,4sleep02865673-21uptime0
2960783280,2sleep00-21swapper/00
2940074281,3sleep00-21swapper/00
2934583281,3sleep02934586-21apt-get0
2931147281,3sleep22932882-21kthreadcore2
2891195281,3sleep12887318-21/usr/sbin/munin1
2889922280,1sleep00-21swapper/00
2871567281,3sleep22871342-21sendmail-msp2
2871232280,1sleep10-21swapper/11
2845589281,3sleep12845800-21hddtemp_smartct1
2843059281,3sleep22843573-21kthreadcore2
2843001280,1sleep00-21swapper/00
2955481271,2sleep20-21swapper/22
2949859270,1sleep10-21swapper/11
2944934270,4sleep20-21swapper/22
2939285270,1sleep10-21swapper/11
2928675270,1sleep20-21swapper/22
2924063270,2sleep00-21swapper/00
2912881270,1sleep20-21swapper/22
2902707270,1sleep20-21swapper/22
2886921270,1sleep10-21swapper/11
2881484270,4sleep10-21swapper/11
2881473270,1sleep00-21swapper/00
2871561271,2sleep10-21swapper/11
2865965270,1sleep20-21swapper/22
2861111270,2sleep10-21swapper/11
2857851271,3sleep02859600-21kthreadcore0
2850113270,1sleep00-21swapper/00
2842993270,1sleep20-21swapper/22
2965645260,1sleep20-21swapper/22
2955485260,2sleep10-21swapper/11
2954956260,1sleep10-21swapper/11
2945628260,2sleep00-21swapper/00
2929354260,2sleep12929489-21tr1
2924625261,2sleep10-21swapper/11
2897410260,4sleep20-21swapper/22
2892574260,2sleep00-21swapper/00
2921466250,1sleep00-21swapper/00
2882097250,2sleep20-21swapper/22
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional