You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-19 - 09:04

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack3slot3s.osadl.org (updated Tue May 19, 2026 00:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2487584212451,68sleep30-21swapper/33
248750528768,12sleep10-21swapper/11
248775028064,10sleep20-21swapper/22
248770027858,14sleep00-21swapper/00
29736822241,3sleep02973693-21ssh0
248799799241,20cyclictest0-21swapper/33
26259922230,2sleep00-21swapper/00
2487997992319,2cyclictest0-21swapper/33
248799799221,17cyclictest27883412sleep33
29566282211,2sleep00-21swapper/00
248799799211,16cyclictest2839085-21taskset3
25856372201,15sleep22585662-21basename2
248799799204,13cyclictest2838294-21taskset3
248799799203,14cyclictest66950irq/132-enp1s0-TxRx-33
248799799201,15cyclictest66950irq/132-enp1s0-TxRx-33
248799799201,15cyclictest66950irq/132-enp1s0-TxRx-33
248799799201,15cyclictest0-21swapper/33
248799799201,15cyclictest0-21swapper/33
248799799201,14cyclictest0-21swapper/33
28799492191,9sleep22879960-21ssh2
25747582191,6sleep22574804-21ntpq2
248799799198,8cyclictest2500666-21turbostat3
2487997991916,1cyclictest0-21swapper/33
2487997991916,1cyclictest0-21swapper/33
248799799191,15cyclictest0-21swapper/33
248799799191,15cyclictest0-21swapper/33
248799799191,14cyclictest0-21swapper/33
248799799191,14cyclictest0-21swapper/33
248799799191,14cyclictest0-21swapper/33
248799799191,14cyclictest0-21swapper/33
248799799191,14cyclictest0-21swapper/33
248799799191,14cyclictest0-21swapper/33
248799799191,14cyclictest0-21swapper/33
2487997991910,6cyclictest2548837-21chrt3
248799799190,2cyclictest0-21swapper/33
28713382180,2sleep00-21swapper/00
28635202181,3sleep20-21swapper/22
248799799183,12cyclictest2554290-21taskset3
248799799182,12cyclictest2618610-21turbostat3
248799799181,5cyclictest0-21swapper/33
2487997991812,3cyclictest0-21swapper/33
248799799181,14cyclictest2625888-21taskset3
248799799181,14cyclictest0-21swapper/33
248799799181,14cyclictest0-21swapper/33
248799799181,14cyclictest0-21swapper/33
248799799181,14cyclictest0-21swapper/33
248799799181,14cyclictest0-21swapper/33
248799799181,14cyclictest0-21swapper/33
248799799181,13cyclictest0-21swapper/33
248799799181,13cyclictest0-21swapper/33
248799799181,13cyclictest0-21swapper/33
248799799181,13cyclictest0-21swapper/33
248799799180,14cyclictest0-21swapper/33
29662002171,3sleep20-21swapper/22
29404412172,4sleep02940467-21apt-get0
28866682171,2sleep22886669-21sh2
28023702171,4sleep12618594-21diskmemload1
26850632171,2sleep00-21swapper/00
26739402171,5sleep22673992-21ntpq2
26613942177,7sleep10-21swapper/11
25588512171,5sleep02558907-21ntp_states0
248799799177,6cyclictest2575787-21turbostat3
2487997991711,4cyclictest0-21swapper/33
248799799171,13cyclictest0-21swapper/33
248799799171,13cyclictest0-21swapper/33
248799799171,13cyclictest0-21swapper/33
248799799171,13cyclictest0-21swapper/33
248799799171,11cyclictest2528236-21taskset3
248799799170,15cyclictest0-21swapper/33
248799799170,14cyclictest0-21swapper/33
248799799170,13cyclictest0-21swapper/33
248799799170,13cyclictest0-21swapper/33
28618522161,2sleep10-21swapper/11
27522312161,4sleep12618594-21diskmemload1
27010822161,10sleep02618594-21diskmemload0
26995252161,4sleep12699588-21ssh1
26694142161,5sleep02669495-21munin-run0
26606472161,3sleep20-21swapper/22
26506142161,3sleep10-21swapper/11
248799799163,11cyclictest0-21swapper/33
248799799161,13cyclictest0-21swapper/33
248799799161,13cyclictest0-21swapper/33
248799799160,14cyclictest0-21swapper/33
248799799160,13cyclictest0-21swapper/33
248799799160,13cyclictest0-21swapper/33
248799799160,13cyclictest0-21swapper/33
248799799160,13cyclictest0-21swapper/33
248799799160,12cyclictest0-21swapper/33
29210822151,3sleep22618594-21diskmemload2
28978892151,2sleep20-21swapper/22
28217432151,2sleep20-21swapper/22
28029932151,3sleep22618594-21diskmemload2
27910772151,9sleep12792332-21memory1
27212582151,10sleep12722669-21cut1
27110992151,2sleep10-21swapper/11
26520842151,3sleep22618594-21diskmemload2
2487997991511,2cyclictest0-21swapper/33
248799799150,12cyclictest0-21swapper/33
248799799150,12cyclictest0-21swapper/33
29315772141,9sleep12931600-21munin-run1
29288322141,2sleep20-21swapper/22
28530052141,9sleep22853019-21ssh2
28530052141,9sleep22853019-21ssh2
27872732141,2sleep20-21swapper/22
27455732141,4sleep22745587-21ssh2
26839092141,2sleep20-21swapper/22
26777422141,9sleep12677756-21ssh1
248799799141,8cyclictest2607939-21taskset3
248799799140,2cyclictest0-21swapper/33
248799799140,12cyclictest0-21swapper/33
248799799140,12cyclictest0-21swapper/33
29568882130,2sleep20-21swapper/22
29551302131,3sleep12955140-21ssh1
29477362131,6sleep20-21swapper/22
29368892133,6sleep22936900-21sh2
29310182131,4sleep02931020-21ssh0
28722772134,2sleep1171rcu_preempt1
28362882131,7sleep20-21swapper/22
28054162132,5sleep22805442-21apt-get2
27771362131,8sleep12777147-21ssh1
27551182134,2sleep10-21swapper/11
27514622131,2sleep00-21swapper/00
27502222131,7sleep22750279-21ntpq2
26751732131,8sleep02618594-21diskmemload0
25230352131,8sleep20-21swapper/22
29641102121,6sleep02618594-21diskmemload0
29221132121,6sleep12618594-21diskmemload1
29145562121,7sleep02618594-21diskmemload0
29130922121,2sleep20-21swapper/22
28873202120,7sleep02618594-21diskmemload0
28305552121,7sleep02830566-21ssh0
28197372121,7sleep02819748-21ssh0
27361552121,6sleep12618594-21diskmemload1
27206382120,2sleep00-21swapper/00
27202952121,8sleep22720489-21latency_hist2
27109082120,7sleep02618594-21diskmemload0
26424832121,7sleep02642495-21ssh0
26066242121,5sleep22606671-21ntpq2
26016492120,8sleep02601859-21grep0
25600532121,7sleep02560072-21apt-get0
25163172120,1sleep10-21swapper/11
29588582111,6sleep22960504-21sh2
29397962110,2sleep0171rcu_preempt0
29319682111,4sleep12618594-21diskmemload1
29028452111,7sleep02902859-21ssh0
28969012111,4sleep02618594-21diskmemload0
28937842111,6sleep12893858-21pmu-power1
28727382111,4sleep02872766-21ssh0
28716752111,6sleep22618594-21diskmemload2
28554412111,3sleep10-21swapper/11
28554412111,3sleep10-21swapper/11
28466772111,3sleep20-21swapper/22
28264142111,5sleep12826474-21ntpq1
27951752111,4sleep22618594-21diskmemload2
27883682111,5sleep02788636-21df_inode0
27840352111,5sleep12784093-21ssh1
27785292111,4sleep22618594-21diskmemload2
27697082111,5sleep00-21swapper/00
27685682111,5sleep12618594-21diskmemload1
27672722111,5sleep22767328-21/usr/sbin/munin2
27343912111,3sleep20-21swapper/22
27032402111,2sleep22703248-21ssh2
26918542111,3sleep00-21swapper/00
26911912111,5sleep12691245-21ntpq1
26414122111,4sleep10-21swapper/11
26316502111,5sleep02631706-21ntpq0
26176252111,5sleep12617679-21ntpq1
25866452111,4sleep02586664-21apt-get0
25654042111,5sleep12565423-21apt-get1
25440952111,5sleep02544350-21df0
25120312111,5sleep22512102-21irqstats2
25103172110,1sleep00-21swapper/00
25066532111,5sleep22506769-21irqstats2
24922972111,4sleep22492343-21ntpq2
24881672111,5sleep12488185-21apt-get1
29654442101,5sleep1584-21gdbus1
29453082101,2sleep10-21swapper/11
29208172101,2sleep00-21swapper/00
29038182100,6sleep12903836-21ssh1
28954072101,2sleep20-21swapper/22
28797332101,2sleep10-21swapper/11
28621452101,4sleep02618594-21diskmemload0
28549972101,3sleep02618594-21diskmemload0
28549972101,3sleep02618594-21diskmemload0
28442932100,2sleep02844306-21ssh0
28442162100,2sleep10-21swapper/11
28368882101,3sleep10-21swapper/11
28278402100,1sleep00-21swapper/00
28194322101,4sleep12618594-21diskmemload1
28126832101,3sleep00-21swapper/00
27973522101,5sleep02797426-21kthreadcore0
27768382101,2sleep00-21swapper/00
27624372101,3sleep00-21swapper/00
27546762101,5sleep22754809-21diskstats2
27436922100,5sleep02618594-21diskmemload0
27434582101,3sleep10-21swapper/11
27337992101,4sleep02734024-21users0
27194482101,5sleep10-21swapper/11
27176852101,4sleep02618594-21diskmemload0
27116332101,3sleep20-21swapper/22
26839482104,3sleep12618594-21diskmemload1
26668522101,4sleep22618594-21diskmemload2
26526852101,3sleep02652156-21apt-get0
26445432101,6sleep02644659-21cut0
26420382100,2sleep2171rcu_preempt2
26333362101,5sleep12633347-21sh1
26327682101,6sleep22632782-21ssh2
26195662101,6sleep20-21swapper/22
26065022101,4sleep12606623-21ntpq1
25746162101,4sleep12574757-21ntpq1
25651262102,4sleep12565158-21munin-run1
25493982102,4sleep22549417-21apt-get2
25177912101,5sleep12519472-21kthreadcore1
25168632101,4sleep00-21swapper/00
25018142101,5sleep02503491-21kthreadcore0
24997072101,4sleep02499764-21ntp_states0
2970262291,4sleep12970324-21ld1
2914619291,3sleep12618594-21diskmemload1
2884063291,2sleep10-21swapper/11
2829055290,1sleep20-21swapper/22
2805311291,4sleep12805323-21ssh1
2784921290,1sleep00-21swapper/00
2720590291,3sleep22618594-21diskmemload2
2692531290,1sleep20-21swapper/22
2652810291,4sleep12652870-21ssh1
2623345291,2sleep10-21swapper/11
2597298291,2sleep20-21swapper/22
2591993291,4sleep02592234-21expr0
2591993291,4sleep02592234-21expr0
2564425291,4sleep22564531-21wc2
2559176290,4sleep12559420-21users1
2543792291,4sleep12543934-21cat1
2539280293,3sleep20-21swapper/22
2538174290,6sleep22538314-21gpgv2
2531905291,4sleep02531956-21ntpq0
2511254290,1sleep10-21swapper/11
2500653291,4sleep11-21systemd1
2499604291,4sleep22495562-21/usr/sbin/munin2
2592268281,4sleep22592481-21sort2
2592268281,4sleep22592481-21sort2
2591963280,1sleep10-21swapper/11
2591963280,1sleep10-21swapper/11
2575088282,3sleep00-21swapper/00
2565426280,4sleep22565655-21cat2
2553525280,4sleep02553578-21ntp_states0
2549113281,3sleep2802-21snmpd2
2527874280,1sleep10-21swapper/11
2493264281,1sleep00-21swapper/00
2607917270,2sleep20-21swapper/22
2607298270,1sleep00-21swapper/00
2602216270,1sleep10-21swapper/11
2586102270,1sleep10-21swapper/11
2580969270,1sleep00-21swapper/00
2576036270,2sleep10-21swapper/11
2554728270,2sleep20-21swapper/22
2549137270,1sleep10-21swapper/11
2543657270,2sleep00-21swapper/00
2532826270,1sleep20-21swapper/22
2505852270,1sleep10-21swapper/11
2505808270,1sleep20-21swapper/22
2612742260,1sleep00-21swapper/00
2590732260,2sleep10-21swapper/11
2586637260,1sleep20-21swapper/22
2549688260,1sleep10-21swapper/11
2533476261,2sleep10-21swapper/11
2522833260,2sleep10-21swapper/11
2522768261,2sleep00-21swapper/00
2515648261,2sleep00-21swapper/00
2617570250,1sleep00-21swapper/00
2612084250,1sleep10-21swapper/11
2581600250,1sleep00-21swapper/00
2580814250,1sleep20-21swapper/22
2535732250,1sleep00-21swapper/00
2519475250,1sleep20-21swapper/22
2613482240,1sleep20-21swapper/22
2565702240,1sleep00-21swapper/00
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional