You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-30 - 03:03

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack3slot3s.osadl.org (updated Thu Apr 30, 2026 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3846512212364,55sleep30-21swapper/33
385114928567,10sleep20-21swapper/22
385125128363,16sleep10-21swapper/11
385148127350,6sleep00-21swapper/00
40671662500,2sleep20-21swapper/22
41164232261,21sleep23981727-21diskmemload2
385162499222,16cyclictest0-21swapper/33
385162499221,18cyclictest0-21swapper/33
385162499221,17cyclictest0-21swapper/33
385162499220,18cyclictest66950irq/132-enp1s0-TxRx-33
38685962211,14sleep13868637-21ntpq1
3851624992118,1cyclictest0-21swapper/33
385162499211,16cyclictest4056511-21chrt3
385162499211,16cyclictest3954016-21chrt3
385162499211,16cyclictest0-21swapper/33
385162499211,16cyclictest0-21swapper/33
385162499211,16cyclictest0-21swapper/33
385162499211,16cyclictest0-21swapper/33
385162499211,16cyclictest0-21swapper/33
385162499210,17cyclictest0-21swapper/33
235432211,16sleep023556-21sh0
41897162201,16sleep14189730-21ssh1
41822122200,12sleep24182228-21ssh2
414646022012,4sleep04146470-21ssh0
3851624992011,6cyclictest4085997-21taskset3
385162499201,15cyclictest12494-21taskset3
408390121910,6sleep03981727-21diskmemload0
385162499191,15cyclictest0-21swapper/33
385162499191,14cyclictest0-21swapper/33
385162499191,14cyclictest0-21swapper/33
385162499191,14cyclictest0-21swapper/33
38568742181,5sleep03856922-21munin-run0
3851624991814,2cyclictest0-21swapper/33
3851624991813,3cyclictest71182chrt3
3851624991812,3cyclictest0-21swapper/33
385162499181,14cyclictest0-21swapper/33
385162499181,14cyclictest0-21swapper/33
385162499181,14cyclictest0-21swapper/33
385162499181,14cyclictest0-21swapper/33
385162499181,14cyclictest0-21swapper/33
385162499181,13cyclictest4147050-21chrt3
385162499181,13cyclictest0-21swapper/33
3851624991810,5cyclictest41813732sleep33
3851624991810,5cyclictest0-21swapper/33
3851624991810,5cyclictest0-21swapper/33
385162499180,16cyclictest0-21swapper/33
385162499180,15cyclictest0-21swapper/33
385162499180,14cyclictest0-21swapper/33
385162499180,14cyclictest0-21swapper/33
41174092170,13sleep04117422-21sh0
40315672171,5sleep23981727-21diskmemload2
39882542171,3sleep20-21swapper/22
39690592171,4sleep23969884-21memory2
39556962171,6sleep13955369-21/usr/sbin/munin1
385162499176,8cyclictest3864257-21turbostat3
3851624991712,4cyclictest0-21swapper/33
3851624991711,3cyclictest0-21swapper/33
385162499171,13cyclictest0-21swapper/33
385162499171,13cyclictest0-21swapper/33
385162499171,13cyclictest0-21swapper/33
385162499171,13cyclictest0-21swapper/33
385162499171,13cyclictest0-21swapper/33
385162499171,13cyclictest0-21swapper/33
385162499170,14cyclictest0-21swapper/33
385162499170,14cyclictest0-21swapper/33
385162499170,13cyclictest0-21swapper/33
385162499170,13cyclictest0-21swapper/33
489252161,4sleep03981727-21diskmemload0
432042161,10sleep03981727-21diskmemload0
41653492161,5sleep23981727-21diskmemload2
41089302160,3sleep10-21swapper/11
40416142161,10sleep24042621-21grep2
40387482161,2sleep20-21swapper/22
385162499164,1cyclictest0-21swapper/33
385162499161,12cyclictest3917508-21chrt3
385162499161,12cyclictest0-21swapper/33
385162499161,12cyclictest0-21swapper/33
385162499160,14cyclictest0-21swapper/33
385162499160,13cyclictest0-21swapper/33
385162499160,13cyclictest0-21swapper/33
372222161,4sleep23981727-21diskmemload2
114442161,3sleep10-21swapper/11
8012151,4sleep1816-21ssh1
41548602151,3sleep20-21swapper/22
41196632151,8sleep00-21swapper/00
41157692151,2sleep10-21swapper/11
41097312150,1sleep20-21swapper/22
39717762151,9sleep23971892-21irqstats2
39659972151,9sleep13966255-21df_inode1
385162499151,3cyclictest0-21swapper/33
3851624991512,1cyclictest0-21swapper/33
385162499151,11cyclictest0-21swapper/33
385162499150,12cyclictest0-21swapper/33
385162499150,12cyclictest0-21swapper/33
385162499150,12cyclictest0-21swapper/33
385162499150,12cyclictest0-21swapper/33
385162499150,12cyclictest0-21swapper/33
385162499150,12cyclictest0-21swapper/33
385162499150,12cyclictest0-21swapper/33
385162499150,11cyclictest0-21swapper/33
222872151,2sleep20-21swapper/22
41765382141,2sleep00-21swapper/00
40835492140,2sleep20-21swapper/22
40388272141,9sleep14038841-21ssh1
39900792141,9sleep03990100-21apt0
39398972141,4sleep03940142-21tr0
94092131,6sleep19439-21munin-run1
603182131,3sleep160328-21ssh1
469202130,8sleep23981727-21diskmemload2
41702852131,2sleep00-21swapper/00
41337792131,6sleep10-21swapper/11
41187472131,5sleep10-21swapper/11
40793312131,3sleep04079341-21ssh0
40755972131,8sleep14077409-21tr1
40702792135,3sleep23981727-21diskmemload2
40328632131,8sleep04033021-21grep0
40163492131,4sleep04016923-21cut0
39946632131,8sleep03981727-21diskmemload0
39772352131,8sleep03978737-21grep0
385162499130,1cyclictest0-21swapper/33
268802131,2sleep20-21swapper/22
574162121,7sleep057430-21ssh0
41808302121,5sleep10-21swapper/11
41633212121,7sleep04163374-21ssh0
41556832121,6sleep23981727-21diskmemload2
41308762121,3sleep00-21swapper/00
41289132121,7sleep03981727-21diskmemload0
41057222121,6sleep03981727-21diskmemload0
40992732121,7sleep14100895-21kthreadcore1
40977282121,3sleep04097738-21ssh0
40924032121,7sleep23981727-21diskmemload2
40849212121,6sleep13981727-21diskmemload1
40597202121,3sleep14059731-21ssh1
40486972121,7sleep14048708-21sh1
40169252122,6sleep14018568-21cut1
38747862121,7sleep23874787-21sshd2
38747862121,7sleep23874787-21sshd2
38632602121,5sleep03863309-21ntpq0
341142121,6sleep134163-21systemd-run1
22912121,7sleep23937-21cut2
207652120,7sleep03981727-21diskmemload0
115222121,5sleep20-21swapper/22
543592111,5sleep254425-21ntp_states2
41924312111,1sleep00-21swapper/00
41886612110,7sleep04188674-21sh0
41777182111,5sleep13981727-21diskmemload1
41655402111,6sleep14165543-21ssh1
41444582111,3sleep20-21swapper/22
41302852114,4sleep13851619-21cyclictest1
40810782111,3sleep20-21swapper/22
40649282111,5sleep14065564-21memory1
40573292111,6sleep04057371-21perf0
40152792110,5sleep10-21swapper/11
39444312110,1sleep20-21swapper/22
39328182111,5sleep03932867-21ntpq0
39183342111,6sleep03918387-21fschecks_count0
39168212111,6sleep03916984-21/usr/sbin/munin0
355052111,3sleep10-21swapper/11
129352111,5sleep00-21swapper/00
81622101,4sleep03981727-21diskmemload0
613822101,3sleep20-21swapper/22
442552101,3sleep20-21swapper/22
41924582101,3sleep20-21swapper/22
41815832101,3sleep00-21swapper/00
41759882101,3sleep20-21swapper/22
41544752101,4sleep03981727-21diskmemload0
41440682101,2sleep10-21swapper/11
41390012101,4sleep04139117-21ntpq0
41349212101,2sleep20-21swapper/22
41318542101,4sleep23981727-21diskmemload2
41280822101,3sleep20-21swapper/22
41187052101,4sleep23981727-21diskmemload2
41096682101,4sleep03981727-21diskmemload0
41036502101,4sleep23981727-21diskmemload2
40974652101,3sleep20-21swapper/22
40933842101,4sleep03981727-21diskmemload0
40577122101,6sleep00-21swapper/00
40300842100,3sleep00-21swapper/00
40210012100,2sleep20-21swapper/22
40152672100,1sleep00-21swapper/00
40133862100,3sleep20-21swapper/22
40064972101,2sleep20-21swapper/22
40052182101,3sleep00-21swapper/00
39956442101,4sleep13981727-21diskmemload1
39904402101,3sleep20-21swapper/22
39769212100,1sleep10-21swapper/11
39657122101,4sleep13965761-21munin-run1
39541632101,5sleep13954217-21ntpq1
39379592101,5sleep13938100-21ntpq1
39379592101,5sleep13938100-21ntpq1
39346192101,5sleep23935664-21kthreadcore2
39346192101,5sleep23935664-21kthreadcore2
39274172101,4sleep23927465-21ntpq2
38791222101,4sleep13879243-21ntpq1
345352101,3sleep03981727-21diskmemload0
337792101,3sleep00-21swapper/00
223262101,3sleep10-21swapper/11
203542101,3sleep20-21swapper/22
193602101,3sleep10-21swapper/11
60306290,1sleep00-21swapper/00
50780291,4sleep152350-21cut1
48311290,6sleep148324-21rm1
4187756291,2sleep20-21swapper/22
4168153292,3sleep14168163-21ssh1
4167379291,4sleep24167393-21ssh2
4157503291,4sleep04157513-21ssh0
4156434291,2sleep10-21swapper/11
4152600291,3sleep10-21swapper/11
4140334291,3sleep13981727-21diskmemload1
4097792291,3sleep13981727-21diskmemload1
4091010290,1sleep10-21swapper/11
4072174290,3sleep10-21swapper/11
4072112291,4sleep04072123-21ssh0
4067335290,1sleep00-21swapper/00
4059011291,2sleep20-21swapper/22
4055004291,2sleep20-21swapper/22
4054449292,2sleep10-21swapper/11
4047017291,2sleep00-21swapper/00
4025581291,4sleep14027259-21cut1
4005608291,2sleep10-21swapper/11
3982106291,3sleep10-21swapper/11
3975506290,6sleep13975619-21ps1
3966533291,4sleep03967309-21cut0
3959368291,3sleep23959409-21ntpq2
3934339291,5sleep03934502-21expr0
3934339291,5sleep03934502-21expr0
3911170291,4sleep23911226-21ld2
3897297291,4sleep03898985-21cut0
3885898291,3sleep03886150-21df0
3874944291,4sleep13874964-21munin-run1
3874944291,4sleep13874964-21munin-run1
3868639291,3sleep23868686-21ntpq2
3857987291,4sleep23858990-21memory2
38079291,3sleep13981727-21diskmemload1
3955003280,1sleep20-21swapper/22
3949421280,1sleep00-21swapper/00
3944709280,3sleep13944728-21apt-get1
3927520281,4sleep00-21swapper/00
3911518281,3sleep03911676-21sed0
3895372280,1sleep00-21swapper/00
3891220281,3sleep23891427-21cstates2
3885375280,1sleep00-21swapper/00
3882491281,3sleep13884341-21kthreadcore1
3973017271,3sleep0221rcuc/00
3961015271,3sleep00-21swapper/00
3954820270,1sleep00-21swapper/00
3944731271,2sleep23944886-21cstates2
3933761270,1sleep10-21swapper/11
3923390270,1sleep10-21swapper/11
3912979270,3sleep23913129-21kthreadcore2
3906777270,1sleep10-21swapper/11
3906595270,1sleep20-21swapper/22
3902918271,2sleep00-21swapper/00
3901234270,2sleep20-21swapper/22
3891198270,2sleep10-21swapper/11
3890612270,1sleep20-21swapper/22
3886151271,3sleep13886191-21fschecks_count1
3885329270,1sleep20-21swapper/22
3879972270,1sleep00-21swapper/00
3870162271,3sleep00-21swapper/00
3870162271,3sleep00-21swapper/00
3856923271,3sleep13857984-21kthreadcore1
3852090271,3sleep20-21swapper/22
3980493260,2sleep23980669-21strings2
3961852261,2sleep20-21swapper/22
3940144260,2sleep13941810-21kthreadcore1
3933605260,4sleep20-21swapper/22
3921971261,2sleep10-21swapper/11
3917583260,2sleep20-21swapper/22
3917408260,4sleep10-21swapper/11
3912020260,1sleep10-21swapper/11
3875837261,2sleep20-21swapper/22
3853515261,2sleep10-21swapper/11
3955358250,1sleep00-21swapper/00
3901569250,2sleep10-21swapper/11
3864317250,2sleep00-21swapper/00
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional