You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-26 - 21:37

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3s.osadl.org (updated Fri Jun 26, 2026 12:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
395433622770,1sleep072-21kcompactd00
3525333213054,71sleep30-21swapper/33
352052128149,18sleep00-21swapper/00
352514728063,11sleep10-21swapper/11
352536327759,11sleep20-21swapper/22
352552799238,12cyclictest3828211-21taskset3
352552799238,12cyclictest3828211-21taskset3
352552799222,7cyclictest39762972chrt3
352552799221,17cyclictest3616063-21taskset3
352552799211,16cyclictest0-21swapper/33
352552799205,12cyclictest0-21swapper/33
352552799201,15cyclictest0-21swapper/33
352552799201,15cyclictest0-21swapper/33
352552799201,15cyclictest0-21swapper/33
352552799195,2cyclictest0-21swapper/33
352552799191,8cyclictest66950irq/132-enp1s0-TxRx-33
3525527991916,2cyclictest0-21swapper/33
352552799191,15cyclictest3803363-21taskset3
352552799191,14cyclictest0-21swapper/33
352552799191,14cyclictest0-21swapper/33
352552799191,14cyclictest0-21swapper/33
352552799191,14cyclictest0-21swapper/33
352552799191,13cyclictest3829815-21chrt3
3525527991910,5cyclictest0-21swapper/33
352552799190,2cyclictest0-21swapper/33
352552799190,16cyclictest0-21swapper/33
39123932181,5sleep03912400-21ssh0
352552799181,5cyclictest0-21swapper/33
3525527991812,4cyclictest66950irq/132-enp1s0-TxRx-33
352552799181,14cyclictest0-21swapper/33
352552799181,14cyclictest0-21swapper/33
352552799181,14cyclictest0-21swapper/33
352552799181,14cyclictest0-21swapper/33
352552799181,14cyclictest0-21swapper/33
352552799181,13cyclictest0-21swapper/33
352552799181,13cyclictest0-21swapper/33
352552799181,13cyclictest0-21swapper/33
352552799180,6cyclictest3711924-21taskset3
40016932171,3sleep00-21swapper/00
38714202171,3sleep20-21swapper/22
38266572171,2sleep10-21swapper/11
38266572171,2sleep10-21swapper/11
36534902171,7sleep13653701-21expr1
3525527991712,3cyclictest0-21swapper/33
3525527991711,3cyclictest0-21swapper/33
352552799171,13cyclictest0-21swapper/33
352552799171,13cyclictest0-21swapper/33
352552799171,13cyclictest0-21swapper/33
352552799171,13cyclictest0-21swapper/33
352552799171,13cyclictest0-21swapper/33
352552799171,13cyclictest0-21swapper/33
352552799171,13cyclictest0-21swapper/33
3525527991710,3cyclictest36480972sleep33
352552799170,14cyclictest66950irq/132-enp1s0-TxRx-33
352552799170,14cyclictest0-21swapper/33
352552799170,14cyclictest0-21swapper/33
352552799170,14cyclictest0-21swapper/33
40009552161,2sleep10-21swapper/11
38697102161,2sleep20-21swapper/22
38213272161,10sleep23653083-21diskmemload2
38213272161,10sleep23653083-21diskmemload2
38195392161,3sleep10-21swapper/11
38122682161,3sleep10-21swapper/11
37453782161,4sleep13653083-21diskmemload1
36618042160,3sleep20-21swapper/22
35483722161,6sleep21-21systemd2
352552799162,9cyclictest3601035-21taskset3
3525527991614,1cyclictest0-21swapper/33
3525527991612,2cyclictest0-21swapper/33
352552799161,13cyclictest0-21swapper/33
352552799161,13cyclictest0-21swapper/33
352552799160,13cyclictest0-21swapper/33
352552799160,13cyclictest0-21swapper/33
352552799160,12cyclictest0-21swapper/33
38453002150,2sleep10-21swapper/11
37838802151,7sleep10-21swapper/11
37433912150,1sleep20-21swapper/22
352552799151,3cyclictest0-21swapper/33
352552799151,2cyclictest0-21swapper/33
3525527991512,2cyclictest0-21swapper/33
352552799151,13cyclictest0-21swapper/33
352552799151,10cyclictest3637359-21chrt3
352552799150,14cyclictest0-21swapper/33
352552799150,13cyclictest0-21swapper/33
352552799150,12cyclictest0-21swapper/33
352552799150,12cyclictest0-21swapper/33
39379792140,8sleep20-21swapper/22
39280522141,4sleep03928059-21ssh0
39179912141,3sleep23653083-21diskmemload2
39120292140,1sleep20-21swapper/22
39109832141,8sleep13653083-21diskmemload1
38314292141,3sleep03832945-21ssh0
37626882141,8sleep13762955-21/usr/sbin/munin1
37530362141,2sleep00-21swapper/00
37211912141,8sleep23721346-21interrupts2
35751552141,7sleep03575365-21hddtemp_smartct0
39711002131,8sleep03971254-21expr0
39351162131,8sleep13935128-21ssh1
39034702131,9sleep00-21swapper/00
39032332131,2sleep20-21swapper/22
38438972131,8sleep23843908-21sh2
37676792131,7sleep03653083-21diskmemload0
37027702131,3sleep23702781-21ssh2
37022392131,2sleep10-21swapper/11
36996112131,6sleep03699681-21awk0
36942462131,2sleep00-21swapper/00
36936832131,8sleep13693692-21sh1
36858922131,6sleep00-21swapper/00
36756162131,3sleep13675627-21ssh1
36321192131,8sleep13632209-21sed1
352552799139,2cyclictest0-21swapper/33
352552799130,2cyclictest0-21swapper/33
352552799130,11cyclictest0-21swapper/33
352552799130,10cyclictest0-21swapper/33
39848472120,2sleep10-21swapper/11
39789372121,5sleep13653083-21diskmemload1
39674192121,3sleep10-21swapper/11
39618472121,7sleep23961860-21sh2
39283652121,7sleep23928375-21ssh2
38882542121,4sleep00-21swapper/00
38464602121,4sleep00-21swapper/00
38463812121,7sleep13846391-21ssh1
38124252120,2sleep20-21swapper/22
38048732121,6sleep03653083-21diskmemload0
37940402121,6sleep03653083-21diskmemload0
37777112121,5sleep20-21swapper/22
37728642121,3sleep13653083-21diskmemload1
37592302121,7sleep03759244-21ssh0
37533852120,1sleep10-21swapper/11
37505082121,7sleep23750627-21ssh2
37352442121,6sleep23735257-21sshd-session2
36932572121,5sleep20-21swapper/22
36776942121,7sleep03677702-21sh0
36671492121,7sleep03667160-21ssh0
36053812121,5sleep23605598-21users2
35696262121,5sleep00-21swapper/00
39548042111,6sleep13954997-21gltestperf1
39374952111,3sleep20-21swapper/22
39350322111,6sleep03935043-21ssh0
39275042110,3sleep13927506-21ssh1
39132292111,5sleep03913493-21df0
38933202111,4sleep23653083-21diskmemload2
38878052111,7sleep00-21swapper/00
38876322111,4sleep10-21swapper/11
38680312111,3sleep10-21swapper/11
38535742110,5sleep20-21swapper/22
38410902111,5sleep03842117-21memory0
38027162110,2sleep00-21swapper/00
37877502111,5sleep10-21swapper/11
37859312110,1sleep00-21swapper/00
37343212111,4sleep03653083-21diskmemload0
37342842110,7sleep13653083-21diskmemload1
36668262111,3sleep10-21swapper/11
36323912111,3sleep00-21swapper/00
36219832111,5sleep03622229-21df_inode0
35618962111,5sleep13563059-21memory1
35430512110,1sleep00-21swapper/00
35329302111,4sleep03532949-21apt-get0
35256792111,4sleep13525698-21apt-get1
40015252100,1sleep20-21swapper/22
39931942101,3sleep20-21swapper/22
39917562101,3sleep03653083-21diskmemload0
39857152101,3sleep20-21swapper/22
39841142101,2sleep00-21swapper/00
39761842101,2sleep20-21swapper/22
39627412100,1sleep00-21swapper/00
39562102101,5sleep03957839-21kthreadcore0
39511712100,6sleep23951185-21ssh2
39510182101,3sleep10-21swapper/11
39432152101,4sleep13653083-21diskmemload1
38882612101,4sleep13888532-21df_inode1
38875432101,2sleep20-21swapper/22
38766992100,1sleep00-21swapper/00
38719932102,4sleep13872120-21ssh1
38685402100,1sleep00-21swapper/00
38618292101,5sleep13861841-21rm1
38345922100,2sleep10-21swapper/11
38273572101,4sleep03653083-21diskmemload0
38273572101,4sleep03653083-21diskmemload0
38129522101,4sleep20-21swapper/22
38035742101,3sleep10-21swapper/11
38010602101,2sleep20-21swapper/22
37937652101,4sleep23653083-21diskmemload2
37751832101,5sleep03775252-21sed0
37682402101,2sleep20-21swapper/22
37595762101,3sleep10-21swapper/11
37591672101,3sleep20-21swapper/22
37379042101,6sleep03738093-21basename0
37267372100,6sleep13726748-21ssh1
37175292101,4sleep13653083-21diskmemload1
37123092101,3sleep20-21swapper/22
37110912101,3sleep10-21swapper/11
37093142101,3sleep23653083-21diskmemload2
36855762101,4sleep23653083-21diskmemload2
36787332101,4sleep13678747-21apt-get1
36753332101,4sleep23653083-21diskmemload2
36611932100,1sleep00-21swapper/00
36586362100,1sleep20-21swapper/22
36530652101,5sleep13653145-21ssh1
36318992101,2sleep20-21swapper/22
36124552101,5sleep23614028-21tr2
35955032101,4sleep03595559-21munin-run0
35542052101,5sleep23554419-21/usr/sbin/munin2
35477342101,4sleep03547982-21users0
35474152101,4sleep13547468-21ntp_states1
3994043290,3sleep10-21swapper/11
3962715290,1sleep20-21swapper/22
3937921291,3sleep03653083-21diskmemload0
3913129291,2sleep10-21swapper/11
3901785291,2sleep10-21swapper/11
3854740290,3sleep00-21swapper/00
3829728291,3sleep23653083-21diskmemload2
3817954290,1sleep00-21swapper/00
3787337291,2sleep20-21swapper/22
3720817291,4sleep03721043-21ssh0
3717937291,3sleep03653083-21diskmemload0
3711181291,2sleep00-21swapper/00
3648116291,3sleep03648118-21apt-get0
3643320291,4sleep13643436-21irqstats1
3641448290,6sleep23641562-21needreboot2
3638295291,4sleep03639878-21grep0
3621942291,3sleep23621961-21apt-get2
3611789291,5sleep03612008-21cat0
3611483290,3sleep10-21swapper/11
3606266291,2sleep20-21swapper/22
3590563290,1sleep20-21swapper/22
3583074290,1sleep10-21swapper/11
3574860290,3sleep13574879-21apt-get1
3564414291,3sleep23564433-21apt-get2
3558834290,1sleep10-21swapper/11
3548706291,5sleep00-21swapper/00
3860463281,3sleep23860470-21ssh2
3642315280,1sleep10-21swapper/11
3626767280,2sleep10-21swapper/11
3590561281,2sleep10-21swapper/11
3583974281,3sleep23580116-21/usr/sbin/munin2
3541268280,1sleep20-21swapper/22
3539579281,3sleep10-21swapper/11
3526292281,3sleep23527781-21kthreadcore2
3652829270,1sleep20-21swapper/22
3647548270,1sleep00-21swapper/00
3632666270,4sleep20-21swapper/22
3621506270,1sleep00-21swapper/00
3621487270,1sleep10-21swapper/11
3621418270,1sleep20-21swapper/22
3610471271,3sleep13610707-21grep1
3605857270,1sleep00-21swapper/00
3601001271,2sleep10-21swapper/11
3590151270,1sleep00-21swapper/00
3585327270,2sleep23585346-21apt-get2
3580189270,4sleep00-21swapper/00
3579698270,1sleep20-21swapper/22
3569621270,1sleep10-21swapper/11
3565777270,4sleep03567465-21kthreadcore0
3560265271,3sleep03561894-21kthreadcore0
3553936271,3sleep00-21swapper/00
3552758271,3sleep13552867-21processes1
3537583270,1sleep20-21swapper/22
3537507270,1sleep10-21swapper/11
3647575260,1sleep20-21swapper/22
3633259261,2sleep10-21swapper/11
3632053260,1sleep00-21swapper/00
3600460260,1sleep10-21swapper/11
3600446260,1sleep20-21swapper/22
3596461261,2sleep00-21swapper/00
3568591261,2sleep10-21swapper/11
3563864260,1sleep20-21swapper/22
3553360260,1sleep20-21swapper/22
3610312250,1sleep00-21swapper/00
3585614250,1sleep10-21swapper/11
3571723250,1sleep20-21swapper/22
3525976250,1sleep00-21swapper/00
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional