You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-10 - 09:07

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack3slot3s.osadl.org (updated Sun May 10, 2026 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
21635132172115,52sleep30-21swapper/33
216429329263,11sleep20-21swapper/22
216431528555,11sleep10-21swapper/11
216450327957,16sleep00-21swapper/00
26032582251,19sleep22295082-21diskmemload2
216463599227,13cyclictest0-21swapper/33
216463599211,16cyclictest0-21swapper/33
216463599211,15cyclictest66950irq/132-enp1s0-TxRx-33
2164635992017,1cyclictest0-21swapper/33
216463599201,15cyclictest66950irq/132-enp1s0-TxRx-33
216463599201,15cyclictest0-21swapper/33
216463599201,15cyclictest0-21swapper/33
216463599201,15cyclictest0-21swapper/33
216463599201,15cyclictest0-21swapper/33
216463599191,2cyclictest0-21swapper/33
216463599191,15cyclictest0-21swapper/33
216463599191,14cyclictest0-21swapper/33
216463599191,14cyclictest0-21swapper/33
216463599191,14cyclictest0-21swapper/33
216463599191,14cyclictest0-21swapper/33
216463599191,14cyclictest0-21swapper/33
216463599191,14cyclictest0-21swapper/33
216463599191,12cyclictest0-21swapper/33
216463599191,12cyclictest0-21swapper/33
216463599191,12cyclictest0-21swapper/33
26304112181,13sleep02630214-21/usr/sbin/munin0
25604462181,13sleep12560458-21ssh1
22027942181,2sleep30-21swapper/33
2164635991815,1cyclictest0-21swapper/33
2164635991814,2cyclictest0-21swapper/33
216463599181,16cyclictest0-21swapper/33
216463599181,14cyclictest0-21swapper/33
216463599181,14cyclictest0-21swapper/33
216463599181,14cyclictest0-21swapper/33
216463599181,14cyclictest0-21swapper/33
216463599181,14cyclictest0-21swapper/33
216463599181,14cyclictest0-21swapper/33
216463599181,14cyclictest0-21swapper/33
216463599181,13cyclictest0-21swapper/33
216463599181,13cyclictest0-21swapper/33
216463599180,15cyclictest0-21swapper/33
216463599180,14cyclictest0-21swapper/33
25040422171,5sleep12504176-21diskstats1
23636622171,12sleep02363722-21kthreadcore0
216463599178,6cyclictest2273648-21taskset3
216463599171,14cyclictest0-21swapper/33
2164635991711,3cyclictest0-21swapper/33
216463599171,13cyclictest0-21swapper/33
216463599171,13cyclictest0-21swapper/33
216463599171,13cyclictest0-21swapper/33
216463599171,13cyclictest0-21swapper/33
216463599171,13cyclictest0-21swapper/33
216463599171,13cyclictest0-21swapper/33
216463599171,12cyclictest0-21swapper/33
216463599170,14cyclictest0-21swapper/33
216463599170,14cyclictest0-21swapper/33
216463599170,13cyclictest0-21swapper/33
25455422160,1sleep20-21swapper/22
25343402161,9sleep12295082-21diskmemload1
23756112161,10sleep22375675-21ntp_states2
23115312160,10sleep12311544-21sshd1
23114922161,4sleep22295082-21diskmemload2
216463599161,13cyclictest0-21swapper/33
216463599161,12cyclictest0-21swapper/33
216463599160,13cyclictest0-21swapper/33
216463599160,13cyclictest0-21swapper/33
216463599160,13cyclictest0-21swapper/33
216463599160,13cyclictest0-21swapper/33
23762522150,1sleep00-21swapper/00
23361622151,2sleep20-21swapper/22
22843892151,10sleep12284416-21munin-run1
22780142151,4sleep22278062-21ntpq2
216463599159,4cyclictest0-21swapper/33
2164635991512,1cyclictest0-21swapper/33
216463599150,12cyclictest0-21swapper/33
216463599150,12cyclictest0-21swapper/33
216463599150,12cyclictest0-21swapper/33
216463599150,11cyclictest0-21swapper/33
25851852141,3sleep02295082-21diskmemload0
25032472140,2sleep10-21swapper/11
24361022141,1sleep00-21swapper/00
21886962141,3sleep12189389-21kthreadcore1
21846102141,2sleep00-21swapper/00
216463599149,3cyclictest22414102sleep33
2164635991411,2cyclictest0-21swapper/33
2164635991411,2cyclictest0-21swapper/33
216463599141,11cyclictest0-21swapper/33
216463599141,10cyclictest22090202sleep33
26275992130,3sleep12627611-21ssh1
26115922131,8sleep12295082-21diskmemload1
26019702131,3sleep12601983-21ssh1
25938002131,8sleep22593812-21ssh2
25509372131,8sleep02170045-21munin-node0
25436582131,6sleep10-21swapper/11
25343792131,7sleep02295082-21diskmemload0
25174752131,8sleep22517482-21ssh2
25019762131,6sleep20-21swapper/22
24941842131,2sleep20-21swapper/22
24611802131,3sleep12461185-21sshd1
23429772131,3sleep02295082-21diskmemload0
23020452131,8sleep22302057-21ssh2
22144572130,1sleep10-21swapper/11
21699282131,4sleep22170788-21grep2
2164635991310,2cyclictest0-21swapper/33
26261992121,5sleep22626253-21ntpq2
26137252121,5sleep00-21swapper/00
25959142121,7sleep02595925-21ssh0
25706692121,2sleep10-21swapper/11
25612742121,8sleep02295082-21diskmemload0
25190272120,2sleep00-21swapper/00
25041772121,5sleep22504398-21hddtemp_smartct2
24937672121,7sleep02493778-21ssh0
24935732121,2sleep10-21swapper/11
24847162121,5sleep00-21swapper/00
24757532121,7sleep12475766-21ssh1
24287702121,3sleep02428772-21ssh0
22633602121,7sleep12263339-21/usr/sbin/munin1
22514292121,6sleep02251478-21ntpq0
22238092123,5sleep12224622-21missed_timers1
216463599120,2cyclictest0-21swapper/33
26436892111,3sleep20-21swapper/22
26430832111,6sleep12643312-21sh1
26298462111,6sleep02630050-21date0
26194862110,5sleep10-21swapper/11
26179182111,5sleep22617966-21ntpq2
26017672111,3sleep00-21swapper/00
25804192111,7sleep22581404-21grep2
25797902110,2sleep10-21swapper/11
25788722111,4sleep00-21swapper/00
25788722111,4sleep00-21swapper/00
25776412111,6sleep22577652-21ssh2
25776412111,6sleep22577652-21ssh2
25529242111,7sleep12295082-21diskmemload1
25351942111,3sleep20-21swapper/22
25261452111,6sleep22526159-21ssh2
24832332111,7sleep22295082-21diskmemload2
24773762111,3sleep20-21swapper/22
24530992111,4sleep20-21swapper/22
24523992111,4sleep00-21swapper/00
24351442111,6sleep22435155-21ssh2
24127302110,2sleep22412741-21ssh2
24106952111,4sleep10-21swapper/11
24100012111,4sleep20-21swapper/22
24097202111,4sleep02295082-21diskmemload0
24022362111,7sleep12402242-21ssh1
23956022110,1sleep10-21swapper/11
23858062112,4sleep02295082-21diskmemload0
23548122111,5sleep00-21swapper/00
23534102110,2sleep00-21swapper/00
23448122111,4sleep22295082-21diskmemload2
23371982111,5sleep12295082-21diskmemload1
23106572111,2sleep00-21swapper/00
23000242111,5sleep12300090-21ntp_states1
22940942111,5sleep12294143-21ntpq1
22638432111,5sleep02263909-21irqstats0
26459082101,2sleep00-21swapper/00
26343262101,4sleep22295082-21diskmemload2
26056792101,5sleep22606241-21tr2
25959662101,3sleep12295082-21diskmemload1
25792062100,2sleep10-21swapper/11
25792062100,2sleep10-21swapper/11
25685022101,6sleep22568514-21rm2
25616312101,3sleep20-21swapper/22
25265422101,3sleep10-21swapper/11
25038182101,5sleep02295082-21diskmemload0
25004842101,5sleep02500491-21ssh0
24854342101,4sleep12295082-21diskmemload1
24688442101,4sleep22295082-21diskmemload2
24684052101,3sleep02295082-21diskmemload0
24610102101,3sleep20-21swapper/22
24596122101,2sleep00-21swapper/00
24518602101,3sleep10-21swapper/11
24445932101,3sleep20-21swapper/22
24431772101,2sleep00-21swapper/00
24369262101,3sleep10-21swapper/11
24269092100,2sleep20-21swapper/22
24207022101,2sleep00-21swapper/00
24177502101,4sleep12295082-21diskmemload1
24003442102,5sleep0151ktimers/00
23939472101,2sleep20-21swapper/22
23850702101,4sleep22295082-21diskmemload2
23768412101,3sleep10-21swapper/11
23694962101,4sleep12295082-21diskmemload1
23612982101,3sleep20-21swapper/22
23600462101,4sleep12295082-21diskmemload1
23538622101,4sleep12295082-21diskmemload1
23517532101,3sleep22295082-21diskmemload2
23434192101,5sleep12343429-21ssh1
23365882100,5sleep0802-21snmpd0
23277412100,1sleep10-21swapper/11
23271092101,5sleep02327118-21ssh0
23181302102,3sleep02295082-21diskmemload0
23024402101,2sleep00-21swapper/00
22842762100,4sleep00-21swapper/00
22793222101,4sleep22279341-21apt-get2
22580302101,4sleep12258283-21df1
22580082101,4sleep02258027-21apt-get0
22366502102,4sleep12236669-21apt-get1
22330652101,5sleep22234874-21grep2
2636306291,2sleep10-21swapper/11
2610763291,2sleep00-21swapper/00
2568247291,2sleep00-21swapper/00
2550957291,3sleep22295082-21diskmemload2
2543695291,2sleep00-21swapper/00
2526218291,3sleep02295082-21diskmemload0
2514379294,2sleep12516189-21kthreadcore1
2477607291,2sleep00-21swapper/00
2468983291,2sleep10-21swapper/11
2444767291,2sleep10-21swapper/11
2421692291,4sleep12421798-21cut1
2396125291,2sleep20-21swapper/22
2395027290,3sleep00-21swapper/00
2363367291,3sleep22295082-21diskmemload2
2327660291,2sleep20-21swapper/22
2319616291,2sleep20-21swapper/22
2317909291,3sleep12295082-21diskmemload1
2290556291,4sleep02291284-21grep0
2285743291,5sleep12287479-21kthreadcore1
2284672291,2sleep00-21swapper/00
2278474291,4sleep12278691-21users1
2246961291,2sleep20-21swapper/22
2240736291,3sleep22240785-21ntpq2
2220673291,4sleep22220690-21apt2
2219420291,5sleep22219437-21cat2
2214789293,3sleep20-21swapper/22
2209513291,3sleep02209178-21apt-get0
2197510291,3sleep12197559-21ntpq1
2195232291,4sleep22197064-21grep2
2169191291,4sleep02169196-21sensors_temp0
2385004281,3sleep12295082-21diskmemload1
2273043280,3sleep22273201-21sed2
2257750281,3sleep22257984-21sed2
2230733280,2sleep10-21swapper/11
2226584281,3sleep22226015-21/usr/sbin/munin2
2219909280,1sleep10-21swapper/11
2198790280,1sleep20-21swapper/22
2169576281,4sleep2802-21snmpd2
2294755270,1sleep20-21swapper/22
2273536271,3sleep12273537-21pool-udisksd1
2262362271,3sleep22262582-21/usr/sbin/munin2
2257402270,1sleep00-21swapper/00
2256650271,2sleep12256744-21modprobe1
2246082271,2sleep00-21swapper/00
2241407270,1sleep00-21swapper/00
2231891271,3sleep10-21swapper/11
2224797271,3sleep00-21swapper/00
2209118271,3sleep22209119-21sshd2
2198819270,4sleep02198969-21seq0
2192961270,1sleep20-21swapper/22
2186990270,1sleep10-21swapper/11
2181576270,3sleep00-21swapper/00
2172130270,2sleep10-21swapper/11
2284424260,3sleep20-21swapper/22
2278911260,1sleep00-21swapper/00
2273541260,1sleep00-21swapper/00
2219794260,1sleep00-21swapper/00
2204386261,2sleep10-21swapper/11
2203826260,1sleep10-21swapper/11
2192930260,1sleep00-21swapper/00
2187526260,1sleep20-21swapper/22
2182309260,4sleep10-21swapper/11
2182193260,1sleep20-21swapper/22
2168995261,2sleep12169081-21/usr/sbin/munin1
2263505250,1sleep20-21swapper/22
2251480250,1sleep10-21swapper/11
2247655250,1sleep20-21swapper/22
2246130250,1sleep10-21swapper/11
2235799250,1sleep00-21swapper/00
2193968250,1sleep00-21swapper/00
2225805240,1sleep00-21swapper/00
2203842240,1sleep00-21swapper/00
2171917240,1sleep00-21swapper/00
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional