You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-22 - 18:05

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack3slot3s.osadl.org (updated Sun Mar 22, 2026 12:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2391229212162,54sleep30-21swapper/33
239192628454,10sleep20-21swapper/22
239207128163,12sleep10-21swapper/11
239178827854,18sleep00-21swapper/00
239225399291,24cyclictest0-21swapper/33
239225399290,26cyclictest0-21swapper/33
239225399290,20cyclictest0-21swapper/33
239225399281,24cyclictest0-21swapper/33
239225399271,8cyclictest0-21swapper/33
239225399271,23cyclictest0-21swapper/33
239225399261,22cyclictest0-21swapper/33
239225399260,24cyclictest0-21swapper/33
27753732251,5sleep20-21swapper/22
239225399251,20cyclictest0-21swapper/33
239225399250,22cyclictest0-21swapper/33
239225399244,15cyclictest24854362sleep33
239225399241,2cyclictest0-21swapper/33
239225399240,2cyclictest0-21swapper/33
239225399230,2cyclictest0-21swapper/33
239225399230,2cyclictest0-21swapper/33
239225399230,2cyclictest0-21swapper/33
239225399230,1cyclictest0-21swapper/33
239225399230,12cyclictest0-21swapper/33
239225399221,14cyclictest2757536-21chrt3
239225399220,1cyclictest0-21swapper/33
239225399220,1cyclictest0-21swapper/33
28055952211,16sleep02807292-21kthreadcore0
2392253992113,6cyclictest0-21swapper/33
239225399211,16cyclictest0-21swapper/33
239225399211,11cyclictest0-21swapper/33
239225399210,2cyclictest0-21swapper/33
239225399210,2cyclictest0-21swapper/33
239225399210,1cyclictest0-21swapper/33
27317952201,11sleep22731806-21rm2
25988862201,3sleep20-21swapper/22
25687272201,15sleep12570479-21cut1
239225399203,14cyclictest0-21swapper/33
239225399201,7cyclictest2406636-21taskset3
2392253992015,3cyclictest0-21swapper/33
239225399201,15cyclictest0-21swapper/33
239225399201,15cyclictest0-21swapper/33
239225399201,15cyclictest0-21swapper/33
239225399201,15cyclictest0-21swapper/33
239225399200,1cyclictest0-21swapper/33
28640452191,3sleep00-21swapper/00
27766772191,8sleep02776691-21ssh0
27210572191,3sleep00-21swapper/00
239225399199,8cyclictest0-21swapper/33
239225399191,9cyclictest0-21swapper/33
239225399191,9cyclictest0-21swapper/33
239225399191,9cyclictest0-21swapper/33
2392253991915,2cyclictest0-21swapper/33
239225399191,15cyclictest0-21swapper/33
239225399191,14cyclictest0-21swapper/33
239225399191,14cyclictest0-21swapper/33
239225399191,14cyclictest0-21swapper/33
239225399191,10cyclictest0-21swapper/33
239225399181,9cyclictest0-21swapper/33
239225399181,14cyclictest0-21swapper/33
239225399181,14cyclictest0-21swapper/33
239225399181,14cyclictest0-21swapper/33
239225399181,13cyclictest66950irq/132-enp1s0-TxRx-33
239225399181,13cyclictest0-21swapper/33
239225399181,13cyclictest0-21swapper/33
239225399180,14cyclictest0-21swapper/33
28283012171,3sleep00-21swapper/00
27675662171,3sleep10-21swapper/11
27051712171,3sleep10-21swapper/11
26775242171,2sleep10-21swapper/11
26595402171,6sleep20-21swapper/22
26587242171,5sleep12658781-21ntpq1
24894152171,4sleep22489465-21ntpq2
239225399178,2cyclictest0-21swapper/33
239225399176,3cyclictest0-21swapper/33
239225399171,13cyclictest2468837-21turbostat3
239225399171,13cyclictest0-21swapper/33
239225399170,14cyclictest0-21swapper/33
239225399170,14cyclictest0-21swapper/33
239225399170,14cyclictest0-21swapper/33
239225399170,13cyclictest0-21swapper/33
28904642161,2sleep10-21swapper/11
27744872161,4sleep12522368-21diskmemload1
27289992162,7sleep12729059-21ntpq1
26872762161,2sleep00-21swapper/00
26691202161,3sleep12522368-21diskmemload1
25753472161,10sleep22522368-21diskmemload2
239225399166,6cyclictest0-21swapper/33
239225399161,11cyclictest24578062sleep33
239225399160,13cyclictest0-21swapper/33
239225399160,13cyclictest0-21swapper/33
239225399160,13cyclictest0-21swapper/33
28284702151,2sleep10-21swapper/11
27053902151,4sleep22522368-21diskmemload2
26407752151,9sleep22640874-21ntpq2
25529742151,8sleep272-21kcompactd02
25472572151,3sleep12522368-21diskmemload1
25278102151,2sleep00-21swapper/00
24258762151,8sleep172-21kcompactd01
239225399151,2cyclictest0-21swapper/33
28835752141,9sleep22885103-21/usr/sbin/munin2
28626502141,1sleep20-21swapper/22
28609572141,8sleep12522368-21diskmemload1
28555492141,4sleep22855563-21ssh2
28198442141,6sleep20-21swapper/22
28021012141,5sleep12802114-21rm1
27946112147,4sleep02392248-21cyclictest0
27640682141,8sleep22764124-21ntpq2
27566592141,4sleep22756672-21ssh2
27481952141,9sleep22748207-21ssh2
27229702141,7sleep20-21swapper/22
26505422144,6sleep02522368-21diskmemload0
26422092144,2sleep1171rcu_preempt1
26252952141,9sleep12625313-21ssh1
247445521410,1sleep20-21swapper/22
28710462130,1sleep10-21swapper/11
28346112131,6sleep02834670-21ntpq0
27385312131,7sleep12522368-21diskmemload1
27385312131,7sleep12522368-21diskmemload1
26868982130,1sleep20-21swapper/22
26866322132,8sleep1331ktimers/11
26687492131,3sleep02668763-21ssh0
26505692131,5sleep20-21swapper/22
26253142131,7sleep20-21swapper/22
25841522133,4sleep12584171-21apt-get1
25572002131,6sleep10-21swapper/11
25364742131,4sleep02536489-21ssh0
24358592131,2sleep00-21swapper/00
24162532131,8sleep02416980-21tr0
24090342131,8sleep12409122-21ntpq1
24038352130,1sleep20-21swapper/22
28204432121,5sleep00-21swapper/00
28019062122,3sleep2431ktimers/22
27296862120,4sleep00-21swapper/00
26954282121,5sleep10-21swapper/11
26796342121,7sleep22679646-21rm2
26778172121,5sleep00-21swapper/00
26694752120,1sleep20-21swapper/22
26354572121,7sleep22635470-21ssh2
26236862120,8sleep02623760-21sh0
26187262121,6sleep00-21swapper/00
26183962120,2sleep20-21swapper/22
26067352121,7sleep12606745-21ssh1
26058632121,6sleep22605920-21ntpq2
25905672121,5sleep22522368-21diskmemload2
25541672121,5sleep00-21swapper/00
24890492121,7sleep02485353-21/usr/sbin/munin0
24840682122,5sleep02484106-21ntpq0
28903942110,2sleep00-21swapper/00
28793972111,4sleep12522368-21diskmemload1
28710592110,4sleep20-21swapper/22
28553132111,3sleep00-21swapper/00
28466502111,3sleep20-21swapper/22
28453652111,6sleep12845376-21ssh1
28275052111,3sleep20-21swapper/22
27952802111,5sleep02795550-21df_inode0
27948372111,3sleep20-21swapper/22
27854122111,3sleep20-21swapper/22
27675072111,6sleep02767518-21ssh0
27510472111,5sleep02751066-21apt-get0
27489482111,4sleep12522368-21diskmemload1
27480732111,3sleep00-21swapper/00
27341612111,5sleep02734990-21grep0
27341612111,5sleep02734990-21grep0
27133362113,1sleep20-21swapper/22
27072482112,4sleep12707277-21apt-get1
26445412111,3sleep00-21swapper/00
26340302110,1sleep10-21swapper/11
26334022111,3sleep00-21swapper/00
25978392110,3sleep00-21swapper/00
25929602112,5sleep12593197-21cstates1
25814032111,4sleep12522368-21diskmemload1
25720172111,3sleep20-21swapper/22
25621032111,4sleep272-21kcompactd02
25578402112,5sleep02557862-21apt-get0
25397102111,6sleep12539723-21ssh1
25286942111,3sleep20-21swapper/22
24948232111,6sleep02490700-21/usr/sbin/munin0
24624812111,5sleep12462530-21ntpq1
24411612112,4sleep22441208-21ntpq2
24159792111,5sleep22416082-21perl2
28803072101,3sleep00-21swapper/00
28791052102,3sleep22522368-21diskmemload2
28696232101,5sleep02869686-21ntpq0
28529782101,2sleep10-21swapper/11
28369452101,2sleep20-21swapper/22
28337022101,5sleep12834430-21missed_timers1
28184792101,4sleep12522368-21diskmemload1
28111802101,3sleep20-21swapper/22
28104762101,2sleep10-21swapper/11
27934482101,3sleep12522368-21diskmemload1
27847862101,3sleep00-21swapper/00
27773532101,4sleep12522368-21diskmemload1
27580172101,4sleep12522368-21diskmemload1
27395072100,1sleep20-21swapper/22
27395072100,1sleep20-21swapper/22
27213772100,1sleep10-21swapper/11
27071302101,4sleep02522368-21diskmemload0
27035832101,2sleep00-21swapper/00
26956222101,3sleep00-21swapper/00
26927992101,4sleep22693852-21memory2
26544712101,3sleep00-21swapper/00
26513152101,3sleep12522368-21diskmemload1
26167282101,6sleep10-21swapper/11
25903362101,5sleep00-21swapper/00
25723452101,3sleep00-21swapper/00
25642282101,3sleep12522368-21diskmemload1
25461542101,4sleep22522368-21diskmemload2
25380312100,4sleep20-21swapper/22
25310602101,4sleep12531098-21munin-run1
25179332102,4sleep12519600-21grep1
25066582101,4sleep02506677-21apt-get0
25055222101,5sleep12505645-21find1
24983472101,5sleep12499905-21/usr/sbin/munin1
24750042101,5sleep02475187-21grep0
24695392101,4sleep02469662-21diskstats0
24105602101,4sleep12410683-21diskstats1
23964012101,4sleep12396482-21ntpq1
2844340291,4sleep02844350-21ssh0
2608294291,2sleep00-21swapper/00
2575319291,4sleep02575334-21ssh0
2547708291,2sleep00-21swapper/00
2521664291,4sleep22521838-21timerwakeupswit2
2517315291,3sleep02517334-21apt-get0
2500869290,4sleep00-21swapper/00
2490699291,3sleep22490718-21apt-get2
2480271291,4sleep22480339-21forks2
2479576290,1sleep10-21swapper/11
2469663291,4sleep12469872-21hddtemp_smartct1
2464348291,4sleep12465921-21grep1
2458158291,4sleep22458172-21latency_hist2
2452531290,4sleep00-21swapper/00
2452023290,1sleep20-21swapper/22
2447496291,4sleep02447753-21sed0
2444067291,4sleep12445861-21cut1
2441895290,1sleep10-21swapper/11
2435972291,4sleep22436065-21wc2
2430653291,5sleep22430773-21rt-features2
2427077291,4sleep02428714-21cut0
2425860290,4sleep02425793-31find0
2420512290,4sleep10-21swapper/11
2396593292,4sleep02396660-21pmu-power0
2513901281,3sleep02515735-21tr0
2503521281,4sleep00-21swapper/00
2480771281,3sleep12482460-21kthreadcore1
2457502281,3sleep12457762-21sed1
2441275281,4sleep02441367-21grep0
2393018281,3sleep20-21swapper/22
2516717270,1sleep10-21swapper/11
2511006271,3sleep12511015-21switchtime1
2501334270,2sleep20-21swapper/22
2500200271,3sleep20-21swapper/22
2463745270,2sleep00-21swapper/00
2463249270,1sleep20-21swapper/22
2436663270,2sleep10-21swapper/11
2430846271,3sleep10-21swapper/11
2421022270,2sleep20-21swapper/22
2405633271,2sleep20-21swapper/22
2403732270,2sleep00-21swapper/00
2516004260,2sleep20-21swapper/22
2495511260,1sleep10-21swapper/11
2485851260,2sleep10-21swapper/11
2475286261,2sleep20-21swapper/22
2468117261,2sleep20-21swapper/22
2458939261,2sleep00-21swapper/00
2455276261,2sleep00-21swapper/00
2452705260,1sleep10-21swapper/11
2445863260,2sleep20-21swapper/22
2415251260,4sleep20-21swapper/22
2410967261,2sleep00-21swapper/00
2410081260,4sleep00-21swapper/00
2403782260,3sleep10-21swapper/11
2506508240,1sleep20-21swapper/22
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional