You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-02 - 13:03

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3s.osadl.org (updated Thu Jul 02, 2026 00:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
192234921330,2sleep01922351-21tr0
1898624212062,54sleep30-21swapper/33
1898624212062,54sleep30-21swapper/33
189878328655,10sleep10-21swapper/11
189878328655,10sleep10-21swapper/11
189865828466,12sleep20-21swapper/22
189865828466,12sleep20-21swapper/22
189862127652,7sleep00-21swapper/00
189862127652,7sleep00-21swapper/00
189904299221,7cyclictest2343035-21taskset3
189904299211,16cyclictest2308444-21taskset3
189904299211,15cyclictest0-21swapper/33
189904299211,15cyclictest0-21swapper/33
189904299211,15cyclictest0-21swapper/33
1899042992110,7cyclictest1927187-21turbostat3
1899042992011,7cyclictest2026298-21chrt3
189904299201,15cyclictest0-21swapper/33
20927242191,8sleep12092738-21ssh1
189904299197,9cyclictest1904211-21turbostat3
1899042991916,1cyclictest0-21swapper/33
1899042991916,1cyclictest0-21swapper/33
1899042991912,5cyclictest0-21swapper/33
189904299191,15cyclictest0-21swapper/33
189904299191,15cyclictest0-21swapper/33
189904299191,15cyclictest0-21swapper/33
189904299191,14cyclictest0-21swapper/33
189904299191,14cyclictest0-21swapper/33
189904299191,14cyclictest0-21swapper/33
189904299191,14cyclictest0-21swapper/33
189904299191,14cyclictest0-21swapper/33
189904299191,14cyclictest0-21swapper/33
189904299191,13cyclictest2210912-21taskset3
189904299191,13cyclictest0-21swapper/33
189904299190,17cyclictest0-21swapper/33
23532972181,12sleep12026605-21diskmemload1
23071612181,3sleep10-21swapper/11
20433012181,11sleep02043346-21munin-run0
1899042991811,4cyclictest0-21swapper/33
189904299181,14cyclictest0-21swapper/33
189904299181,14cyclictest0-21swapper/33
189904299181,14cyclictest0-21swapper/33
189904299181,14cyclictest0-21swapper/33
189904299181,14cyclictest0-21swapper/33
189904299181,14cyclictest0-21swapper/33
189904299181,14cyclictest0-21swapper/33
189904299181,14cyclictest0-21swapper/33
189904299181,14cyclictest0-21swapper/33
189904299181,14cyclictest0-21swapper/33
189904299181,14cyclictest0-21swapper/33
189904299181,14cyclictest0-21swapper/33
189904299181,14cyclictest0-21swapper/33
189904299181,14cyclictest0-21swapper/33
189904299181,14cyclictest0-21swapper/33
189904299181,13cyclictest0-21swapper/33
189904299181,13cyclictest0-21swapper/33
189904299181,13cyclictest0-21swapper/33
189904299181,13cyclictest0-21swapper/33
189904299180,14cyclictest66950irq/132-enp1s0-TxRx-33
189904299180,14cyclictest66950irq/132-enp1s0-TxRx-33
22927442171,3sleep00-21swapper/00
20681472171,5sleep02026605-21diskmemload0
20425812170,2sleep20-21swapper/22
189904299171,13cyclictest0-21swapper/33
189904299171,13cyclictest0-21swapper/33
189904299171,13cyclictest0-21swapper/33
189904299170,14cyclictest0-21swapper/33
189904299170,14cyclictest0-21swapper/33
189904299170,14cyclictest0-21swapper/33
189904299170,13cyclictest0-21swapper/33
189904299170,13cyclictest0-21swapper/33
189904299170,13cyclictest0-21swapper/33
189904299170,12cyclictest0-21swapper/33
23506542161,1sleep20-21swapper/22
21508182161,2sleep20-21swapper/22
21414112160,1sleep20-21swapper/22
20847722161,2sleep20-21swapper/22
20339372161,2sleep20-21swapper/22
189904299161,3cyclictest0-21swapper/33
189904299161,11cyclictest0-21swapper/33
189904299160,14cyclictest0-21swapper/33
189904299160,13cyclictest0-21swapper/33
23332632150,3sleep12333262-21sshd-session1
23013202151,3sleep12026605-21diskmemload1
22024992151,6sleep12202767-21df_inode1
21833602150,1sleep00-21swapper/00
21661572151,2sleep10-21swapper/11
21055222150,2sleep20-21swapper/22
189904299150,12cyclictest0-21swapper/33
189904299150,11cyclictest0-21swapper/33
23671792141,2sleep10-21swapper/11
23084782141,9sleep22026605-21diskmemload2
22163392144,4sleep1331ktimers/11
21996232141,1sleep10-21swapper/11
21833312141,8sleep12026605-21diskmemload1
21744302141,8sleep12174442-21ssh1
21239872141,9sleep02124001-21ssh0
20581712140,1sleep00-21swapper/00
20479962141,9sleep02048151-21wc0
19489622141,9sleep21949791-21tr2
189904299141,3cyclictest0-21swapper/33
1899042991412,1cyclictest0-21swapper/33
189904299140,2cyclictest0-21swapper/33
189904299140,11cyclictest0-21swapper/33
23776412131,7sleep12377651-21ssh1
23360622131,8sleep02336066-21ssh0
23178342131,5sleep00-21swapper/00
22985952130,1sleep20-21swapper/22
22668582130,8sleep12026605-21diskmemload1
22596762131,2sleep00-21swapper/00
22507262131,7sleep02026605-21diskmemload0
22507262131,7sleep02026605-21diskmemload0
22410482131,8sleep02241059-21ssh0
21916402131,7sleep12026605-21diskmemload1
21723312131,8sleep02169241-21/usr/sbin/munin0
21405492131,7sleep12026605-21diskmemload1
20581452131,3sleep22058156-21ssh2
20545572131,8sleep12056122-21wc1
189904299130,2cyclictest0-21swapper/33
23301592125,4sleep20-21swapper/22
23272072121,5sleep22026605-21diskmemload2
23266222121,7sleep02026605-21diskmemload0
23208522122,5sleep12320862-21apt-get1
23180452120,1sleep10-21swapper/11
23097132121,4sleep02026605-21diskmemload0
22837922121,3sleep22283797-21ssh2
22688832121,3sleep12268893-21cpu1
22594382121,6sleep12026605-21diskmemload1
22530552120,7sleep22254564-21sed2
22502372121,7sleep22250249-21rm2
22502372121,7sleep22250249-21rm2
22352842121,7sleep12235344-21/usr/sbin/munin1
22327652120,2sleep20-21swapper/22
22322552121,4sleep00-21swapper/00
22258902121,2sleep10-21swapper/11
22243732121,7sleep22224386-21ssh2
22158382121,7sleep22215851-21ssh2
21928912121,5sleep20-21swapper/22
21920122121,7sleep02192022-21ssh0
21503892121,7sleep12150403-21ssh1
21192902121,5sleep22119482-21df_inode2
21147142120,6sleep00-21swapper/00
20823022121,7sleep12082315-21ssh1
20750352121,6sleep02026605-21diskmemload0
20502192121,6sleep12026605-21diskmemload1
20353132121,6sleep12026605-21diskmemload1
20107982120,1sleep10-21swapper/11
19995062121,5sleep11999559-21ntp_states1
19795542121,7sleep11979800-21sed1
19586172121,7sleep11958792-21sed1
19483992121,5sleep11948418-21apt-get1
19367252121,5sleep11936778-21ntp_states1
19167232121,5sleep11916780-21munin-run1
23758282111,3sleep20-21swapper/22
23754922110,2sleep02375502-21ssh0
23623092111,5sleep20-21swapper/22
23432822111,4sleep00-21swapper/00
23374962111,3sleep20-21swapper/22
22933792111,6sleep12293407-21latency_hist1
22912652111,3sleep20-21swapper/22
22771132111,4sleep00-21swapper/00
22688962111,5sleep22269160-21df_inode2
22399512111,6sleep12240104-21sed1
22359812111,4sleep22236201-21hddtemp_smartct2
22255062110,1sleep00-21swapper/00
21825802111,2sleep20-21swapper/22
21665232111,6sleep02166533-21ssh0
21597432111,3sleep10-21swapper/11
21540212111,6sleep22155608-21kthreadcore2
21484812110,8sleep02148680-21timerwakeupswit0
21399822110,7sleep02140149-21sensors_temp0
21106502111,5sleep22110417-21sendmail-msp2
21086442111,5sleep00-21swapper/00
21075062111,4sleep12026605-21diskmemload1
21008712111,4sleep02026605-21diskmemload0
20981822111,5sleep22098247-21ntp_states2
20940402112,4sleep12094061-21apt-get1
20927632110,7sleep02026605-21diskmemload0
20856962110,5sleep22085717-21ssh2
20740532111,4sleep10-21swapper/11
20510492111,3sleep20-21swapper/22
20268482111,3sleep00-21swapper/00
20216412111,3sleep20-21swapper/22
20164322111,6sleep12016607-21cstates1
19998542111,5sleep02000070-21users0
19638502111,6sleep01964050-21sed0
19379682111,3sleep10-21swapper/11
19379462111,4sleep01937965-21apt-get0
19222102111,5sleep21922229-21apt-get2
23685332101,2sleep00-21swapper/00
23599582100,2sleep10-21swapper/11
23559882101,5sleep22357620-21cut2
23514362101,5sleep02026605-21diskmemload0
23427262101,2sleep10-21swapper/11
23195282101,4sleep22026605-21diskmemload2
22760272101,5sleep02276041-21ssh0
22658382101,3sleep20-21swapper/22
22609912101,4sleep02026605-21diskmemload0
22511612101,3sleep10-21swapper/11
22511612101,3sleep10-21swapper/11
22111362101,3sleep02026605-21diskmemload0
22075822101,3sleep20-21swapper/22
22018942100,4sleep00-21swapper/00
21943412101,3sleep20-21swapper/22
21685542101,3sleep20-21swapper/22
21564672100,2sleep0171rcu_preempt0
21347612101,3sleep20-21swapper/22
21190792101,4sleep12026605-21diskmemload1
21177542101,2sleep10-21swapper/11
20839682101,4sleep02026605-21diskmemload0
20662662101,3sleep10-21swapper/11
20605762101,4sleep22060590-21apt-get2
20331242101,3sleep10-21swapper/11
20007442101,3sleep00-21swapper/00
19955122101,5sleep21995766-21df2
19850582101,4sleep21985061-21apt-get2
19693582101,4sleep01969615-21df0
19641492101,4sleep01964407-21df_inode0
19539492104,3sleep01954077-21gltestperf0
19473632100,6sleep11947518-21cat1
19274712100,1sleep00-21swapper/00
19175652100,6sleep11918283-21cat1
2354707291,4sleep02354789-21uname0
2298524291,3sleep02026605-21diskmemload0
2281932291,4sleep12281943-21ssh1
2208243291,3sleep02026605-21diskmemload0
2175777290,5sleep22175788-21ssh2
2133771291,3sleep12026605-21diskmemload1
2127408291,3sleep02026605-21diskmemload0
2074872291,2sleep20-21swapper/22
2016036290,2sleep20-21swapper/22
2001497291,4sleep12003095-21cut1
1990567291,4sleep01990777-21hddtemp_smartct0
1973527291,5sleep11973677-21timerandwakeup1
1927731291,4sleep11927834-21perl1
1911455291,4sleep21911523-21sendmail-msp2
1904724291,4sleep11906167-21perl1
2016410281,2sleep00-21swapper/00
2005934280,2sleep20-21swapper/22
1990276280,3sleep11990295-21apt-get1
1989694281,3sleep01989697-21sshd-auth0
1979836281,2sleep00-21swapper/00
1922502281,3sleep11922711-21/usr/sbin/munin1
1916998280,2sleep00-21swapper/00
1911785280,2sleep00-21swapper/00
2026395270,1sleep10-21swapper/11
2026310270,1sleep00-21swapper/00
2016397270,1sleep20-21swapper/22
2015909270,1sleep00-21swapper/00
2011175270,2sleep10-21swapper/11
2010600270,1sleep00-21swapper/00
1990845271,2sleep20-21swapper/22
1984642270,1sleep20-21swapper/22
1974581270,4sleep00-21swapper/00
1964123270,2sleep20-21swapper/22
1964122270,1sleep10-21swapper/11
1954228271,2sleep20-21swapper/22
1953162270,1sleep00-21swapper/00
1943931271,3sleep01945527-21kthreadcore0
1942019270,1sleep20-21swapper/22
1933598271,3sleep20-21swapper/22
1921796270,1sleep20-21swapper/22
1915904270,4sleep21916035-21sensors_temp2
1907378271,3sleep00-21swapper/00
1903938270,1sleep00-21swapper/00
2005451260,1sleep20-21swapper/22
1980657261,2sleep10-21swapper/11
1974883261,2sleep20-21swapper/22
1974296261,2sleep20-21swapper/22
1963533260,1sleep20-21swapper/22
1963512260,4sleep10-21swapper/11
1947924260,4sleep20-21swapper/22
1932704260,3sleep0221rcuc/00
1904054260,1sleep20-21swapper/22
1900866261,2sleep11902581-21grep1
1985037250,1sleep10-21swapper/11
1927447250,1sleep20-21swapper/22
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional