You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-03 - 20:49

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3s.osadl.org (updated Wed Jun 03, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
387172821250,4sleep23871729-21kthreadcore2
3720243212060,56sleep30-21swapper/33
372051228266,10sleep10-21swapper/11
372034228252,11sleep20-21swapper/22
372023027955,18sleep00-21swapper/00
57362232,11sleep014-21ksoftirqd/00
372067999231,19cyclictest0-21swapper/33
372067999221,7cyclictest4043033-21taskset3
372067999213,15cyclictest66950irq/132-enp1s0-TxRx-33
372067999211,7cyclictest3928506-21taskset3
372067999211,16cyclictest0-21swapper/33
41541862201,6sleep14154315-21ntpq1
372067999203,14cyclictest66950irq/132-enp1s0-TxRx-33
372067999201,16cyclictest66950irq/132-enp1s0-TxRx-33
372067999201,16cyclictest0-21swapper/33
372067999201,15cyclictest7526-21taskset3
372067999201,15cyclictest66950irq/132-enp1s0-TxRx-33
372067999201,15cyclictest4096733-21taskset3
372067999194,2cyclictest0-21swapper/33
3720679991915,2cyclictest0-21swapper/33
372067999191,14cyclictest3874921-21taskset3
372067999191,14cyclictest0-21swapper/33
372067999191,14cyclictest0-21swapper/33
372067999191,14cyclictest0-21swapper/33
372067999191,14cyclictest0-21swapper/33
372067999191,14cyclictest0-21swapper/33
372067999191,14cyclictest0-21swapper/33
372067999191,14cyclictest0-21swapper/33
372067999190,2cyclictest0-21swapper/33
372067999190,16cyclictest0-21swapper/33
257302191,14sleep125740-21ssh1
40662632181,3sleep10-21swapper/11
3720679991815,2cyclictest0-21swapper/33
3720679991814,2cyclictest0-21swapper/33
3720679991811,5cyclictest0-21swapper/33
3720679991811,4cyclictest3982205-21chrt3
372067999181,14cyclictest0-21swapper/33
372067999181,14cyclictest0-21swapper/33
372067999181,14cyclictest0-21swapper/33
372067999181,14cyclictest0-21swapper/33
372067999181,14cyclictest0-21swapper/33
372067999181,14cyclictest0-21swapper/33
372067999181,13cyclictest0-21swapper/33
372067999181,13cyclictest0-21swapper/33
372067999181,13cyclictest0-21swapper/33
372067999180,16cyclictest0-21swapper/33
40589262171,4sleep23851751-21diskmemload2
40315692171,9sleep24031626-21ntpq2
37549862171,4sleep03754667-31find0
3720679991715,1cyclictest0-21swapper/33
372067999171,14cyclictest0-21swapper/33
372067999171,13cyclictest0-21swapper/33
372067999171,13cyclictest0-21swapper/33
372067999171,13cyclictest0-21swapper/33
3720679991710,4cyclictest0-21swapper/33
372067999170,14cyclictest0-21swapper/33
372067999170,13cyclictest0-21swapper/33
41829312161,2sleep20-21swapper/22
41399812160,2sleep00-21swapper/00
40356722161,10sleep14035709-21perf1
39604322161,4sleep13961520-21memory1
372067999161,13cyclictest0-21swapper/33
372067999161,12cyclictest0-21swapper/33
372067999160,14cyclictest0-21swapper/33
372067999160,13cyclictest0-21swapper/33
372067999160,13cyclictest0-21swapper/33
372067999160,13cyclictest0-21swapper/33
243152161,2sleep00-21swapper/00
41314932151,4sleep03851751-21diskmemload0
41027102151,4sleep23851751-21diskmemload2
40964112150,1sleep20-21swapper/22
3720679991512,2cyclictest0-21swapper/33
3720679991511,2cyclictest0-21swapper/33
372067999151,11cyclictest0-21swapper/33
372067999150,13cyclictest0-21swapper/33
372067999150,12cyclictest0-21swapper/33
372067999150,12cyclictest0-21swapper/33
372067999150,12cyclictest0-21swapper/33
372067999150,12cyclictest0-21swapper/33
224272151,9sleep23851751-21diskmemload2
149132151,4sleep23851751-21diskmemload2
41112302142,4sleep0171rcu_preempt0
40869162140,2sleep10-21swapper/11
40849982141,4sleep24085012-21ssh2
40625472141,7sleep04062654-21perl0
40524132141,9sleep04052425-21rm0
39091952142,6sleep03909391-21users0
38682002141,2sleep23851751-21diskmemload2
38588572141,3sleep03851751-21diskmemload0
372067999141,1cyclictest0-21swapper/33
3720679991411,2cyclictest0-21swapper/33
372067999141,10cyclictest3738616-21chrt3
372067999141,10cyclictest0-21swapper/33
372067999140,12cyclictest0-21swapper/33
372067999140,11cyclictest0-21swapper/33
372067999140,11cyclictest0-21swapper/33
41924402131,8sleep24192443-21ssh2
41913632130,1sleep10-21swapper/11
41677862130,3sleep04167796-21ssh0
41647622131,3sleep04164772-21ssh0
41308182131,3sleep24130830-21ssh2
40965422131,8sleep04096552-21ssh0
40711332131,6sleep14071255-21diskstats1
40438312136,1sleep20-21swapper/22
40429952131,8sleep04043005-21ssh0
40412432131,8sleep14041256-21rm1
40247322131,7sleep03851751-21diskmemload0
40078592131,8sleep04007866-21ssh0
39970112131,7sleep03997135-21cat0
39899612131,8sleep13851751-21diskmemload1
39552662131,3sleep23955277-21ssh2
39373612131,3sleep23937376-21ssh2
39219882131,8sleep23922248-21expr2
38841382131,4sleep00-21swapper/00
38737782131,7sleep13873838-21ntpq1
38130512131,8sleep23813191-21cat2
88592121,7sleep09087-21cstates0
80512121,7sleep13851751-21diskmemload1
41567492121,7sleep04156752-21ssh0
41479942121,7sleep24147997-21ssh2
41237912121,4sleep10-21swapper/11
40939922121,6sleep10-21swapper/11
40262472121,3sleep24026258-21ssh2
40229442121,7sleep14022999-21/usr/sbin/munin1
40159092121,7sleep04015923-21ssh0
39924342122,5sleep23992454-21apt-get2
38409632120,1sleep00-21swapper/00
38033202120,7sleep13803339-21apt-get1
37860372121,5sleep23786069-21ntpq2
37806612121,5sleep13780702-21ntpq1
37712962121,7sleep03771461-21expr0
37656882121,7sleep03765713-21munin-run0
37442092122,5sleep03744228-21apt-get0
37429432121,5sleep23742992-21ntpq2
41840382111,6sleep14184049-21ssh1
41741432115,1sleep2421rcuc/22
41481202111,7sleep00-21swapper/00
41405112111,5sleep23851751-21diskmemload2
40789202110,2sleep00-21swapper/00
40628382111,6sleep24063503-21kthreadcore2
40534742110,3sleep04053488-21ssh0
40514252111,6sleep14051435-21ssh1
40002182110,3sleep14000229-21ssh1
39724902111,6sleep03972501-21ssh0
39632472111,3sleep00-21swapper/00
39550482111,7sleep13955061-21sh1
39473922111,7sleep23851751-21diskmemload2
39186082111,3sleep00-21swapper/00
39102562111,3sleep10-21swapper/11
39017462111,2sleep13901757-21ssh1
38993272111,6sleep03899838-21grep0
38937052110,3sleep13893719-21ssh1
38758912111,5sleep03851751-21diskmemload0
38290192111,5sleep13829070-21ntpq1
38140282111,5sleep13814283-21df1
38086862110,2sleep10-21swapper/11
38074092111,5sleep23807459-21ntpq2
37665202111,6sleep13767942-21cut1
37659482111,4sleep03765967-21apt-get0
7332101,6sleep2786-21kthreadcore2
41847662101,4sleep03851751-21diskmemload0
41679062101,4sleep14167930-21apt-get1
41644242101,3sleep13851751-21diskmemload1
41584862101,4sleep24158564-21munin-run2
41475272101,4sleep13851751-21diskmemload1
41213812101,4sleep03851751-21diskmemload0
41207292101,4sleep13851751-21diskmemload1
41120012101,3sleep20-21swapper/22
41118392101,3sleep10-21swapper/11
41027402101,4sleep13851751-21diskmemload1
41015822101,5sleep04101640-21ntpq0
40852162101,3sleep00-21swapper/00
40752962101,4sleep23851751-21diskmemload2
40578332101,5sleep14057922-21ssh1
40151842101,2sleep20-21swapper/22
40118122101,6sleep14013543-21kthreadcore1
40078792101,4sleep23851751-21diskmemload2
40054012101,5sleep14005452-21sed1
39889172101,4sleep23851751-21diskmemload2
39810712101,4sleep13851751-21diskmemload1
39806242101,4sleep23851751-21diskmemload2
39722392101,6sleep1802-21snmpd1
39716602100,1sleep20-21swapper/22
39656952101,4sleep23851751-21diskmemload2
39538762101,4sleep03851751-21diskmemload0
39452222101,3sleep10-21swapper/11
39388612101,3sleep10-21swapper/11
39301652100,1sleep00-21swapper/00
39299662101,3sleep10-21swapper/11
39206572101,3sleep20-21swapper/22
39184732101,4sleep13851751-21diskmemload1
39056942101,5sleep23907367-21kthreadcore2
39038722101,4sleep23851751-21diskmemload2
38925532101,4sleep23851751-21diskmemload2
38823482101,4sleep13882498-21ntpq1
38654792101,5sleep03865757-21head0
38574812101,2sleep20-21swapper/22
38452972101,4sleep13845386-21ntpq1
38193692100,6sleep13819389-21dpkg1
38190872102,4sleep21-21systemd2
38086642101,4sleep03808683-21apt-get0
37970742101,4sleep23797318-21users2
37546992101,4sleep13754667-31find1
37536332101,5sleep03753680-21ntpq0
37482872101,5sleep13748342-21ntpq1
37282202101,3sleep20-21swapper/22
9088291,4sleep18827-21/usr/sbin/munin1
4190682291,4sleep04190693-21ssh0
4166054291,3sleep23851751-21diskmemload2
4138190290,3sleep10-21swapper/11
4115383291,4sleep24115953-21cut2
4044755291,3sleep23851751-21diskmemload2
4032703290,2sleep00-21swapper/00
3989599291,4sleep03989613-21ssh0
3981111291,4sleep03981121-21ssh0
3947270290,1sleep00-21swapper/00
3938043291,3sleep00-21swapper/00
3888213290,2sleep00-21swapper/00
3883406291,3sleep20-21swapper/22
3868435291,3sleep13851751-21diskmemload1
3856380291,4sleep13856435-21ntpq1
3840524291,4sleep23840688-21vmstat2
3834554291,4sleep03834694-21ntp_192.168.1140
3825009291,5sleep23825169-21cstates2
3792024290,3sleep23792007-21fwupd2
3791976290,1sleep00-21swapper/00
3738879291,3sleep13738898-21apt-get1
3726309292,3sleep10-21swapper/11
3845739280,1sleep00-21swapper/00
3841339281,2sleep20-21swapper/22
3835632280,2sleep20-21swapper/22
3824381280,1sleep00-21swapper/00
3818962280,1sleep00-21swapper/00
3799320281,3sleep03801059-21kthreadcore0
3791366281,3sleep13791414-21ntpq1
3776216280,1sleep20-21swapper/22
3760631281,4sleep23760892-21df_inode2
3760349281,4sleep13760393-21systemd-run1
3749830281,3sleep23750038-21hddtemp_smartct2
3744458280,1sleep20-21swapper/22
3733575280,2sleep20-21swapper/22
3725729280,2sleep10-21swapper/11
3720890280,5sleep23721149-21df_inode2
3848071271,3sleep20-21swapper/22
3841026270,1sleep10-21swapper/11
3830595270,2sleep10-21swapper/11
3825280271,3sleep03825492-21hddtemp_smartct0
3782491271,3sleep13782548-21kthreadcore1
3781565270,1sleep00-21swapper/00
3767943271,3sleep20-21swapper/22
3743947270,1sleep00-21swapper/00
3725964271,2sleep00-21swapper/00
3847217261,2sleep10-21swapper/11
3819940261,2sleep20-21swapper/22
3803610260,2sleep00-21swapper/00
3798365261,2sleep20-21swapper/22
3796762261,2sleep134-21ksoftirqd/11
3793466261,2sleep00-21swapper/00
3777618261,2sleep20-21swapper/22
3761176260,2sleep13762131-21kthreadcore1
3755322260,2sleep20-21swapper/22
3737668260,2sleep00-21swapper/00
3851733250,1sleep00-21swapper/00
3802905250,3sleep10-21swapper/11
3785851250,1sleep00-21swapper/00
3724865250,2sleep00-21swapper/00
3771462240,1sleep10-21swapper/11
3733924240,1sleep10-21swapper/11
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional