You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-29 - 12:46

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3s.osadl.org (updated Sun Mar 29, 2026 00:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3178182211859,55sleep30-21swapper/33
317821028463,12sleep10-21swapper/11
317795928253,12sleep20-21swapper/22
317806927951,6sleep00-21swapper/00
36515482500,2sleep30-21swapper/33
35694202221,17sleep23569433-21sh2
317840499226,13cyclictest0-21swapper/33
317840499221,16cyclictest33837612sleep33
317840499212,15cyclictest3657331-21taskset3
317840499211,17cyclictest0-21swapper/33
317840499211,17cyclictest0-21swapper/33
317840499211,16cyclictest0-21swapper/33
3178404992111,5cyclictest35800562sleep33
317840499211,12cyclictest0-21swapper/33
34223882201,3sleep00-21swapper/00
3178404992015,3cyclictest0-21swapper/33
3178404992012,5cyclictest0-21swapper/33
317840499201,14cyclictest0-21swapper/33
3178404992010,7cyclictest3491248-21chrt3
33348632190,7sleep2431ktimers/22
33348632190,7sleep2431ktimers/22
317840499191,15cyclictest0-21swapper/33
317840499191,15cyclictest0-21swapper/33
317840499191,14cyclictest0-21swapper/33
317840499191,14cyclictest0-21swapper/33
317840499191,14cyclictest0-21swapper/33
317840499191,14cyclictest0-21swapper/33
317840499191,14cyclictest0-21swapper/33
317840499191,14cyclictest0-21swapper/33
317840499191,14cyclictest0-21swapper/33
317840499191,14cyclictest0-21swapper/33
317840499190,15cyclictest66950irq/132-enp1s0-TxRx-33
34665532181,3sleep00-21swapper/00
34051792181,3sleep20-21swapper/22
34037752181,3sleep00-21swapper/00
3178404991812,4cyclictest0-21swapper/33
3178404991812,3cyclictest0-21swapper/33
317840499181,14cyclictest0-21swapper/33
317840499181,14cyclictest0-21swapper/33
317840499181,14cyclictest0-21swapper/33
317840499181,14cyclictest0-21swapper/33
317840499181,14cyclictest0-21swapper/33
317840499181,14cyclictest0-21swapper/33
317840499181,13cyclictest3552269-21taskset3
317840499181,13cyclictest0-21swapper/33
317840499181,13cyclictest0-21swapper/33
317840499181,13cyclictest0-21swapper/33
317840499181,13cyclictest0-21swapper/33
317840499180,4cyclictest35432652sleep33
317840499180,14cyclictest0-21swapper/33
36415522171,5sleep10-21swapper/11
35192962171,3sleep00-21swapper/00
35192962171,3sleep00-21swapper/00
33307912171,2sleep10-21swapper/11
33307912171,2sleep10-21swapper/11
317840499171,4cyclictest0-21swapper/33
317840499171,14cyclictest0-21swapper/33
317840499171,13cyclictest3259914-21chrt3
317840499171,13cyclictest3259914-21chrt3
317840499171,13cyclictest0-21swapper/33
317840499171,13cyclictest0-21swapper/33
317840499171,13cyclictest0-21swapper/33
317840499171,13cyclictest0-21swapper/33
317840499170,14cyclictest0-21swapper/33
317840499170,14cyclictest0-21swapper/33
317840499170,13cyclictest0-21swapper/33
36419462160,12sleep03641957-21ssh0
36147572162,4sleep23308489-21diskmemload2
34240402165,7sleep23425135-21tr2
34240402165,7sleep23425135-21tr2
34206792161,3sleep20-21swapper/22
33075402161,5sleep23307593-21ntp_states2
3178404991612,2cyclictest0-21swapper/33
317840499161,13cyclictest0-21swapper/33
317840499161,12cyclictest0-21swapper/33
317840499160,14cyclictest0-21swapper/33
317840499160,13cyclictest32923612sleep33
317840499160,13cyclictest0-21swapper/33
36593292151,10sleep03659344-21ssh0
36167942151,2sleep20-21swapper/22
35974312151,4sleep03597433-21ssh0
35969232151,4sleep13308489-21diskmemload1
35283012151,9sleep13528333-21munin-run1
35094222151,2sleep20-21swapper/22
35079952157,2sleep13308489-21diskmemload1
34589712156,5sleep13460114-21cut1
33488342157,4sleep13348843-21/usr/sbin/munin1
33240752151,6sleep03324089-21sh0
317840499159,4cyclictest0-21swapper/33
317840499151,2cyclictest0-21swapper/33
3178404991511,2cyclictest0-21swapper/33
317840499150,13cyclictest0-21swapper/33
317840499150,13cyclictest0-21swapper/33
317840499150,12cyclictest0-21swapper/33
317840499150,12cyclictest0-21swapper/33
36044592142,4sleep1171rcu_preempt1
35866302140,3sleep13308489-21diskmemload1
35602012141,9sleep13308489-21diskmemload1
35263582142,2sleep0171rcu_preempt0
35100122143,2sleep0171rcu_preempt0
34848282141,3sleep03484839-21ssh0
34462102141,9sleep13308489-21diskmemload1
33879422141,8sleep23308489-21diskmemload2
33607902140,2sleep00-21swapper/00
33426482141,8sleep13308489-21diskmemload1
32872522140,2sleep20-21swapper/22
32229652140,2sleep10-21swapper/11
317840499149,3cyclictest32549012sleep33
317840499149,3cyclictest0-21swapper/33
317840499148,3cyclictest32967062sleep33
3178404991411,1cyclictest0-21swapper/33
317840499141,10cyclictest3249917-21turbostat3
317840499140,12cyclictest0-21swapper/33
317840499140,11cyclictest0-21swapper/33
36762472131,8sleep03676258-21ssh0
36167632131,6sleep03616771-21ssh0
35780192131,4sleep03578034-21ssh0
35609802131,8sleep23560992-21sh2
34578592131,7sleep23308489-21diskmemload2
34379932130,1sleep00-21swapper/00
34123752130,2sleep10-21swapper/11
33765042131,8sleep13376515-21ssh1
33683892130,4sleep13368400-21ssh1
36744842120,7sleep13308489-21diskmemload1
36666102120,2sleep03341009-21kworker/u16:3+rpciod0
36508412121,2sleep13650852-21ssh1
36251962121,7sleep13625226-21sh1
36151002120,8sleep10-21swapper/11
35866682121,5sleep00-21swapper/00
35637562120,3sleep03563769-21ssh0
35523512120,1sleep20-21swapper/22
35263852121,5sleep20-21swapper/22
35172772120,5sleep10-21swapper/11
35172772120,5sleep10-21swapper/11
34923642121,6sleep23308489-21diskmemload2
34149752121,7sleep13415060-21cut1
33950392120,1sleep20-21swapper/22
33942622120,3sleep13394270-21ssh1
33847052125,4sleep03178397-21cyclictest0
33679142121,3sleep03367928-21ssh0
33435952121,7sleep03343649-21rm0
33260472121,6sleep23326086-21turbostat2
33163592121,5sleep13308489-21diskmemload1
32924742121,6sleep03292523-21munin-run0
32807852121,5sleep23280833-21ntpq2
32595182121,5sleep13259567-21ntpq1
32595182121,5sleep13259567-21ntpq1
32551762121,7sleep23255209-21perf2
32383112121,5sleep03238364-21ntp_states0
31953472121,5sleep23195400-21ntpq2
36768322112,5sleep2431ktimers/22
36586722111,4sleep13308489-21diskmemload1
36564512111,3sleep20-21swapper/22
36504082111,4sleep23308489-21diskmemload2
36159372111,5sleep03308489-21diskmemload0
35680672111,2sleep10-21swapper/11
35633072110,5sleep00-21swapper/00
35536712111,4sleep13308489-21diskmemload1
34824752111,6sleep13482486-21ssh1
34813692111,6sleep23308489-21diskmemload2
34811402111,6sleep03481153-21ssh0
34731732111,4sleep23308489-21diskmemload2
34564672110,2sleep10-21swapper/11
34360392111,3sleep23436279-21strings2
34310372110,2sleep00-21swapper/00
34310372110,2sleep00-21swapper/00
34233122111,4sleep13308489-21diskmemload1
34233122111,4sleep13308489-21diskmemload1
34131032111,4sleep23308489-21diskmemload2
34053642111,4sleep10-21swapper/11
33868142111,4sleep13308489-21diskmemload1
33605432111,4sleep13308489-21diskmemload1
33580012113,5sleep23358213-21ssh2
33498772111,3sleep20-21swapper/22
33442852111,7sleep03344425-21cut0
33337912111,6sleep03333803-21ssh0
33337912111,6sleep03333803-21ssh0
33141082111,3sleep00-21swapper/00
33034322111,2sleep10-21swapper/11
32541842111,5sleep03254235-21ntpq0
32392472111,5sleep13239295-21munin-run1
32276352111,4sleep13227688-21ntpq1
31968512111,5sleep13196870-21apt-get1
36678542100,1sleep20-21swapper/22
36662092100,4sleep10-21swapper/11
36412072101,3sleep20-21swapper/22
36327712101,4sleep13308489-21diskmemload1
36322672101,3sleep20-21swapper/22
36320902101,2sleep00-21swapper/00
36047512101,5sleep03604761-21ssh0
36045272101,5sleep23604541-21ssh2
35953622101,3sleep23308489-21diskmemload2
35780352101,3sleep10-21swapper/11
35777012100,2sleep23577715-21ssh2
35453192101,3sleep00-21swapper/00
35448852101,3sleep20-21swapper/22
35441122100,1sleep10-21swapper/11
35362992101,3sleep23308489-21diskmemload2
35360822101,4sleep03308489-21diskmemload0
35352752101,2sleep10-21swapper/11
35173332101,4sleep23308489-21diskmemload2
35173332101,4sleep23308489-21diskmemload2
35019912101,5sleep03502014-21cron0
34936892101,3sleep23308489-21diskmemload2
34936022100,4sleep13308489-21diskmemload1
34916352101,3sleep10-21swapper/11
34729572101,3sleep13308489-21diskmemload1
34582562101,3sleep23308489-21diskmemload2
34567112101,2sleep00-21swapper/00
34471872101,2sleep20-21swapper/22
34100812101,4sleep03410128-21ssh0
33937862101,3sleep00-21swapper/00
33700802101,2sleep20-21swapper/22
33355232100,6sleep23335675-21grep2
33175312101,4sleep13308489-21diskmemload1
33131422101,4sleep23308489-21diskmemload2
33031512101,4sleep13303175-21munin-run1
32967942100,4sleep03296843-21ntpq0
32826102101,5sleep13282932-21cat1
32759132101,4sleep13276105-21users1
32649312100,7sleep23264979-21/usr/sbin/munin2
32607862101,4sleep072-21kcompactd00
32288662101,5sleep13229025-21cstates1
32270722101,5sleep23227491-21grep2
32037042101,5sleep03205481-21kthreadcore0
31908852100,1sleep10-21swapper/11
31859362101,4sleep03186192-21df0
31830212101,6sleep13183261-21unixbench_singl1
3643131290,3sleep00-21swapper/00
3473256291,3sleep03308489-21diskmemload0
3446953291,3sleep03446967-21ssh0
3439349290,1sleep10-21swapper/11
3377045290,1sleep00-21swapper/00
3375919291,4sleep23375932-21ssh2
3303410291,3sleep03303429-21apt-get0
3271419291,3sleep23271438-21apt-get2
3266415291,4sleep23266647-21cat2
3259398291,5sleep03259517-21kworker/u16:00
3259398291,5sleep03259517-21kworker/u16:00
3249367291,4sleep03249532-21vmstat0
3244827291,3sleep13244846-21apt-get1
3240082291,4sleep13241515-21cut1
3234190291,3sleep23234209-21apt-get2
3232869291,3sleep03232920-21ntpq0
3224122291,4sleep03225259-21cut0
3222597291,4sleep23222771-21cat2
3208102290,5sleep23208483-21grep2
3197881291,4sleep23199592-21cut2
3195156291,3sleep03195297-21ntpq0
3182727291,4sleep03182781-21ntpq0
3182579291,4sleep23182668-21ntpq2
3581406281,3sleep23581433-21ssh2
3552227280,3sleep00-21swapper/00
3281576280,1sleep10-21swapper/11
3273524281,3sleep00-21swapper/00
3249890280,1sleep20-21swapper/22
3213604281,3sleep23215285-21kthreadcore2
3213435281,4sleep134-21ksoftirqd/11
3213216281,3sleep03213434-21sshd0
3191245280,2sleep10-21swapper/11
3190089281,3sleep23190233-21mailstats2
3298377271,3sleep03298589-21hddtemp_smartct0
3294445271,3sleep10-21swapper/11
3291543270,4sleep10-21swapper/11
3291278270,2sleep23291423-21strings2
3270877270,1sleep10-21swapper/11
3264876271,2sleep10-21swapper/11
3243871271,3sleep23243958-21timerandwakeup2
3239506270,2sleep00-21swapper/00
3218771271,2sleep03219090-21kthreadcore0
3207023270,2sleep10-21swapper/11
3201955271,3sleep00-21swapper/00
3302806260,1sleep20-21swapper/22
3296905261,2sleep20-21swapper/22
3266966261,2sleep00-21swapper/00
3257066260,2sleep20-21swapper/22
3257066260,2sleep20-21swapper/22
3255084260,4sleep10-21swapper/11
3233653260,1sleep20-21swapper/22
3212434260,4sleep10-21swapper/11
3202204260,4sleep20-21swapper/22
3286284250,1sleep00-21swapper/00
3212027250,1sleep00-21swapper/00
3276946240,1sleep00-21swapper/00
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional