You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-08 - 23:25

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot3s.osadl.org (updated Mon Jun 08, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
333038422690,2sleep134-21ksoftirqd/11
317290021360,3sleep0221rcuc/00
308350421320,2sleep20-21swapper/22
2936225211850,61sleep30-21swapper/33
30515152850,2sleep0221rcuc/00
293636828353,10sleep20-21swapper/22
293616928367,10sleep10-21swapper/11
293632627959,14sleep00-21swapper/00
293658399221,19cyclictest0-21swapper/33
293658399221,16cyclictest0-21swapper/33
293658399212,15cyclictest0-21swapper/33
293658399201,15cyclictest66950irq/132-enp1s0-TxRx-33
293658399201,15cyclictest0-21swapper/33
293658399201,15cyclictest0-21swapper/33
32410872192,10sleep23241146-21ntpq2
30461232191,5sleep03046173-21munin-run0
293658399192,12cyclictest3076669-21taskset3
2936583991916,1cyclictest0-21swapper/33
2936583991915,2cyclictest0-21swapper/33
293658399191,15cyclictest0-21swapper/33
293658399191,15cyclictest0-21swapper/33
293658399191,15cyclictest0-21swapper/33
293658399191,14cyclictest0-21swapper/33
293658399191,14cyclictest0-21swapper/33
293658399191,14cyclictest0-21swapper/33
293658399191,14cyclictest0-21swapper/33
293658399191,14cyclictest0-21swapper/33
293658399191,14cyclictest0-21swapper/33
293658399187,8cyclictest3244933-21turbostat3
293658399181,14cyclictest0-21swapper/33
293658399181,14cyclictest0-21swapper/33
293658399181,14cyclictest0-21swapper/33
293658399181,14cyclictest0-21swapper/33
293658399181,14cyclictest0-21swapper/33
293658399181,14cyclictest0-21swapper/33
293658399181,14cyclictest0-21swapper/33
293658399181,14cyclictest0-21swapper/33
293658399181,14cyclictest0-21swapper/33
293658399181,14cyclictest0-21swapper/33
293658399181,14cyclictest0-21swapper/33
293658399181,13cyclictest0-21swapper/33
293658399181,13cyclictest0-21swapper/33
293658399181,13cyclictest0-21swapper/33
293658399181,13cyclictest0-21swapper/33
293658399181,13cyclictest0-21swapper/33
293658399181,13cyclictest0-21swapper/33
293658399181,12cyclictest2960084-21taskset3
293658399180,15cyclictest0-21swapper/33
33258732171,5sleep03325934-21sed0
32798172171,6sleep23280028-21cut2
32613532171,3sleep03261364-21ssh0
31527162171,6sleep03152955-21cat0
2936583991711,3cyclictest3024520-21taskset3
293658399171,13cyclictest0-21swapper/33
293658399171,13cyclictest0-21swapper/33
293658399171,13cyclictest0-21swapper/33
293658399171,13cyclictest0-21swapper/33
293658399170,14cyclictest0-21swapper/33
293658399170,14cyclictest0-21swapper/33
293658399170,14cyclictest0-21swapper/33
293658399170,14cyclictest0-21swapper/33
293658399170,12cyclictest0-21swapper/33
34126052161,3sleep00-21swapper/00
33723252161,10sleep03372557-21df_inode0
33540322160,1sleep20-21swapper/22
31935782161,2sleep13193590-21ssh1
31675542161,3sleep00-21swapper/00
30744052167,3sleep00-21swapper/00
293658399163,1cyclictest0-21swapper/33
2936583991613,1cyclictest0-21swapper/33
293658399161,14cyclictest0-21swapper/33
293658399161,11cyclictest0-21swapper/33
293658399160,13cyclictest29867162chrt3
293658399160,13cyclictest0-21swapper/33
293658399160,13cyclictest0-21swapper/33
293658399160,13cyclictest0-21swapper/33
293658399160,12cyclictest0-21swapper/33
34135312151,2sleep20-21swapper/22
33963652150,1sleep10-21swapper/11
32670642151,2sleep00-21swapper/00
32284242151,4sleep13228440-21ssh1
32259302151,2sleep10-21swapper/11
29929762151,3sleep02992995-21apt-get0
2936583991512,2cyclictest0-21swapper/33
2936583991512,2cyclictest0-21swapper/33
293658399151,11cyclictest0-21swapper/33
293658399150,13cyclictest0-21swapper/33
293658399150,13cyclictest0-21swapper/33
293658399150,12cyclictest0-21swapper/33
293658399150,12cyclictest0-21swapper/33
33192562141,9sleep13319268-21ssh1
32843492140,3sleep13067804-21diskmemload1
32592752140,1sleep20-21swapper/22
32454032141,8sleep23245609-21df2
32113432140,3sleep23067804-21diskmemload2
31935652141,8sleep03067804-21diskmemload0
29408012141,8sleep02940847-21ntpq0
293658399141,8cyclictest3035727-21taskset3
293658399140,12cyclictest0-21swapper/33
34210942131,4sleep13421105-21ssh1
33973592132,5sleep03397383-21apt-get0
33860062131,5sleep20-21swapper/22
33459042131,7sleep03067804-21diskmemload0
33430342131,6sleep23343281-21users2
31916102131,7sleep23067804-21diskmemload2
31842812131,3sleep03067804-21diskmemload0
31765772131,3sleep23176588-21ssh2
31444282131,6sleep03144560-21diskstats0
30611752131,7sleep13061224-21ntpq1
30558592131,6sleep03055915-21ntpq0
29978832130,1sleep20-21swapper/22
293658399133,8cyclictest3040792-21turbostat3
33875502121,3sleep13387552-21ssh1
33714702121,3sleep20-21swapper/22
33560822121,7sleep23357270-21cat2
33134232121,7sleep03314161-21cut0
33120832121,6sleep13067804-21diskmemload1
32950392121,7sleep13295049-21ssh1
32427842121,7sleep13242798-21ssh1
32010272121,6sleep03201039-21ssh0
31997132121,5sleep20-21swapper/22
31485232121,5sleep10-21swapper/11
31358662120,7sleep10-21swapper/11
31268662121,6sleep23067804-21diskmemload2
30987462121,2sleep20-21swapper/22
30918552121,7sleep23091869-21ssh2
30755542121,6sleep13067804-21diskmemload1
30642942121,7sleep23066053-21cut2
30268572121,7sleep03028676-21grep0
30202132121,7sleep03020505-21cut0
30089862121,7sleep23009001-21/usr/sbin/munin2
29983202121,6sleep12998339-21apt-get1
29875792121,4sleep12987066-21apt-get1
29875792121,4sleep12987066-21apt-get1
29587162120,2sleep00-21swapper/00
34211812111,4sleep03067804-21diskmemload0
33622542110,5sleep134-21ksoftirqd/11
33358302110,1sleep00-21swapper/00
33221022114,3sleep23322182-21kthreadcore2
32876872111,6sleep03287712-21apt-get0
32782162111,3sleep20-21swapper/22
32453032111,6sleep03245324-21apt-get0
32290272111,5sleep23229738-21cut2
32263622110,3sleep23226372-21ssh2
32176132111,5sleep00-21swapper/00
31680412110,7sleep13168053-21ssh1
31609712111,5sleep20-21swapper/22
31263252111,4sleep13067804-21diskmemload1
31080592111,4sleep00-21swapper/00
31013822111,5sleep00-21swapper/00
31004102111,5sleep13067804-21diskmemload1
30504712112,4sleep13050519-21ntpq1
30176292111,2sleep00-21swapper/00
29937692111,3sleep12995440-21cut1
29754552111,5sleep02975509-21ntpq0
34213422101,4sleep23067804-21diskmemload2
34123352101,3sleep10-21swapper/11
34026962101,4sleep13067804-21diskmemload1
34016122101,4sleep23401680-21ntp_states2
33957652101,3sleep00-21swapper/00
33879972101,3sleep03067804-21diskmemload0
33722372100,1sleep20-21swapper/22
33710912101,2sleep10-21swapper/11
33622032101,2sleep00-21swapper/00
33527192101,4sleep13067804-21diskmemload1
33521502101,6sleep03352164-21ssh0
33382982100,1sleep10-21swapper/11
33352292101,3sleep23067804-21diskmemload2
33214182101,3sleep10-21swapper/11
33208462101,2sleep20-21swapper/22
33102382101,4sleep23067804-21diskmemload2
33002432101,5sleep23300300-21awk2
32948162101,2sleep20-21swapper/22
32843372100,1sleep00-21swapper/00
32750292101,3sleep13275095-21ntpq1
32691162101,4sleep13067804-21diskmemload1
32587432101,3sleep10-21swapper/11
32418242101,4sleep03067804-21diskmemload0
32313972101,4sleep03232458-21memory0
32260202101,4sleep03067804-21diskmemload0
31947222101,4sleep13067804-21diskmemload1
31858912101,3sleep20-21swapper/22
31828952101,2sleep10-21swapper/11
31546902101,5sleep13156419-21ssh1
31491952101,3sleep23067804-21diskmemload2
31432412100,2sleep20-21swapper/22
31343132100,2sleep23134327-21sh2
31324332101,2sleep10-21swapper/11
31275322101,4sleep03127758-21df_inode0
31255472102,3sleep03067804-21diskmemload0
31168422101,4sleep20-21swapper/22
31155422101,4sleep03067804-21diskmemload0
30858442101,5sleep13086383-21grep1
30842032101,5sleep03084215-21ssh0
30839772101,2sleep10-21swapper/11
30557152101,4sleep23055802-21ntpq2
30387032101,4sleep03039628-21memory0
30357022101,3sleep23035721-21apt-get2
30291682101,5sleep13029224-21cut1
30205062101,5sleep13022189-21cut1
30037442101,4sleep03003947-21diskstats0
30025172101,6sleep02998321-21/usr/sbin/munin0
29917162101,4sleep02991765-21ntpq0
29917162101,4sleep02991765-21ntpq0
29859622101,4sleep22986053-21ntpq2
29807762101,5sleep12980833-21/usr/sbin/munin1
29755112101,6sleep12975602-21proc_pri1
29495032101,5sleep22949652-21ls2
29381042101,5sleep12939849-21kthreadcore1
3394319291,3sleep20-21swapper/22
3378642291,3sleep10-21swapper/11
3369659291,3sleep00-21swapper/00
3309521290,1sleep00-21swapper/00
3301573291,4sleep03067804-21diskmemload0
3301444291,3sleep13067804-21diskmemload1
3271371292,3sleep00-21swapper/00
3262093291,2sleep20-21swapper/22
3253182291,3sleep13067804-21diskmemload1
3216193291,3sleep13067804-21diskmemload1
3208462291,2sleep20-21swapper/22
3207833290,2sleep00-21swapper/00
3207606291,4sleep13207743-21ssh1
3175390290,3sleep10-21swapper/11
3141639291,4sleep03141652-21ssh0
3110851291,4sleep13111020-21grep1
3106232291,2sleep20-21swapper/22
3106174291,3sleep13067804-21diskmemload1
3091735290,1sleep00-21swapper/00
3073629291,3sleep23067804-21diskmemload2
3066700291,4sleep03066794-21ld0
3048354291,4sleep23050178-21grep2
3032297291,4sleep03034116-21cut0
3018417291,3sleep20-21swapper/22
3014345291,5sleep13014362-21acpi1
3002402291,4sleep23002451-21ntpq2
2970957290,4sleep072-21kcompactd00
2965788291,2sleep20-21swapper/22
2955590291,4sleep12955661-21irqstats1
2941815291,4sleep02942170-21sed0
3160254281,2sleep20-21swapper/22
3040133281,3sleep13040393-21unixbench_singl1
3025015281,2sleep23025034-21apt-get2
2971298280,2sleep20-21swapper/22
2970898281,3sleep12970878-31find1
2965531281,2sleep00-21swapper/00
2949774280,1sleep00-21swapper/00
2948307281,4sleep12948432-21cat1
3056576271,1sleep10-21swapper/11
3014060270,1sleep10-21swapper/11
3008376270,1sleep10-21swapper/11
2985300271,2sleep10-21swapper/11
2983534271,3sleep02985297-21kthreadcore0
2977238271,3sleep00-21swapper/00
2965453270,1sleep20-21swapper/22
2944086270,2sleep20-21swapper/22
3063278261,2sleep10-21swapper/11
3061801260,2sleep00-21swapper/00
3061226260,2sleep20-21swapper/22
3045793260,1sleep20-21swapper/22
3009267261,2sleep00-21swapper/00
2991511261,2sleep20-21swapper/22
2991511261,2sleep20-21swapper/22
2981582260,1sleep20-21swapper/22
2965234260,4sleep10-21swapper/11
2955662261,2sleep20-21swapper/22
2941756260,4sleep20-21swapper/22
3041125250,1sleep10-21swapper/11
3030628250,2sleep13030840-21sshd1
3030335250,1sleep20-21swapper/22
3023831251,1sleep20-21swapper/22
3003659250,1sleep20-21swapper/22
2949481240,1sleep10-21swapper/11
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional