You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-31 - 14:34

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot3s.osadl.org (updated Fri Jul 31, 2026 12:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3481890212261,56sleep30-21swapper/33
348170528769,12sleep20-21swapper/22
348182828558,11sleep10-21swapper/11
348164227555,14sleep00-21swapper/00
36027522460,3sleep1321rcuc/11
3482068992522,1cyclictest0-21swapper/33
38163942240,2sleep03713452-21kworker/u16:0+xprtiod0
3482068992419,2cyclictest0-21swapper/33
3482068992419,2cyclictest0-21swapper/33
348206899241,19cyclictest0-21swapper/33
36858372231,6sleep03609879-21diskmemload0
3482068992318,2cyclictest0-21swapper/33
3482068992318,2cyclictest0-21swapper/33
348206899224,15cyclictest541ktimers/33
348206899222,16cyclictest3541490-21taskset3
3482068992219,1cyclictest0-21swapper/33
3482068992219,1cyclictest0-21swapper/33
3482068992219,1cyclictest0-21swapper/33
3482068992219,1cyclictest0-21swapper/33
3482068992218,2cyclictest0-21swapper/33
3482068992218,2cyclictest0-21swapper/33
3482068992218,2cyclictest0-21swapper/33
3482068992218,2cyclictest0-21swapper/33
3482068992211,7cyclictest3583580-21taskset3
35682472211,5sleep23568511-21df_inode2
348206899213,13cyclictest3897062-21taskset3
3482068992118,1cyclictest0-21swapper/33
3482068992118,1cyclictest0-21swapper/33
3482068992118,1cyclictest0-21swapper/33
3482068992117,2cyclictest0-21swapper/33
3482068992117,2cyclictest0-21swapper/33
3482068992117,2cyclictest0-21swapper/33
3482068992117,2cyclictest0-21swapper/33
3482068992116,2cyclictest0-21swapper/33
36797972201,3sleep10-21swapper/11
348206899207,11cyclictest0-21swapper/33
348206899206,2cyclictest0-21swapper/33
348206899204,13cyclictest0-21swapper/33
3482068992018,1cyclictest0-21swapper/33
3482068992017,1cyclictest0-21swapper/33
3482068992017,1cyclictest0-21swapper/33
3482068992017,1cyclictest0-21swapper/33
3482068992017,1cyclictest0-21swapper/33
3482068992017,1cyclictest0-21swapper/33
3482068992017,1cyclictest0-21swapper/33
3482068992017,1cyclictest0-21swapper/33
3482068992016,2cyclictest0-21swapper/33
3482068992016,2cyclictest0-21swapper/33
3482068992016,2cyclictest0-21swapper/33
3482068992016,2cyclictest0-21swapper/33
3482068992016,2cyclictest0-21swapper/33
3482068992016,2cyclictest0-21swapper/33
3482068992014,3cyclictest0-21swapper/33
38331022191,8sleep03609879-21diskmemload0
37991382190,3sleep10-21swapper/11
348206899194,12cyclictest0-21swapper/33
3482068991916,1cyclictest0-21swapper/33
3482068991916,1cyclictest0-21swapper/33
3482068991916,1cyclictest0-21swapper/33
3482068991916,1cyclictest0-21swapper/33
3482068991916,1cyclictest0-21swapper/33
3482068991915,2cyclictest0-21swapper/33
3482068991915,2cyclictest0-21swapper/33
3482068991915,2cyclictest0-21swapper/33
39736492181,5sleep10-21swapper/11
39474372181,7sleep03609879-21diskmemload0
39159792181,8sleep03917594-21cut0
38931532180,2sleep00-21swapper/00
38082492181,8sleep03808260-21ssh0
38082492181,8sleep03808260-21ssh0
37474562180,3sleep23747471-21ssh2
36692622181,5sleep23609879-21diskmemload2
3482068991815,1cyclictest0-21swapper/33
348206899181,14cyclictest0-21swapper/33
39021212171,2sleep00-21swapper/00
38607892171,11sleep03860804-21ssh0
38406232171,6sleep13840812-21timerandwakeup1
37719792170,3sleep00-21swapper/00
37042012171,7sleep13704216-21ssh1
348206899172,12cyclictest0-21swapper/33
3482068991715,1cyclictest0-21swapper/33
3482068991713,2cyclictest0-21swapper/33
3482068991713,2cyclictest0-21swapper/33
348206899171,12cyclictest0-21swapper/33
39288822161,10sleep23609879-21diskmemload2
35513192161,11sleep20-21swapper/22
3482068991612,2cyclictest0-21swapper/33
3482068991612,2cyclictest0-21swapper/33
3482068991611,2cyclictest0-21swapper/33
348206899160,14cyclictest0-21swapper/33
348206899160,12cyclictest0-21swapper/33
39656312151,4sleep13609879-21diskmemload1
39484342151,4sleep23609879-21diskmemload2
39371232157,4sleep13937136-21sh1
39123282151,1sleep10-21swapper/11
38970282153,5sleep23897052-21apt-get2
38766492151,10sleep13876659-21ssh1
38238632151,4sleep13609879-21diskmemload1
36542492151,4sleep23654326-21irqstats2
348206899150,12cyclictest0-21swapper/33
39558472140,2sleep00-21swapper/00
39369022140,4sleep00-21swapper/00
39288102141,9sleep03928824-21ssh0
39120002140,8sleep03609879-21diskmemload0
38889622141,4sleep23890103-21kthreadcore2
38506132141,9sleep03609879-21diskmemload0
37902052141,9sleep03790217-21ssh0
37293812141,8sleep13729395-21ssh1
35005282141,9sleep23500945-21grep2
3482068991410,2cyclictest0-21swapper/33
39203602131,7sleep23920371-21ssh2
38680112132,3sleep13609879-21diskmemload1
38621722131,2sleep00-21swapper/00
38416972130,1sleep20-21swapper/22
38018442131,7sleep13609879-21diskmemload1
38018442131,7sleep13609879-21diskmemload1
37904412130,4sleep23609879-21diskmemload2
37659172131,7sleep13609879-21diskmemload1
37119132131,8sleep23711923-21ssh2
36932482131,8sleep13693275-21ssh1
36771532131,6sleep23609879-21diskmemload2
36594282131,8sleep13659439-21ssh1
36351412135,5sleep03635153-21rm0
36330492131,8sleep13633054-21rm1
36160052131,8sleep13616017-21sh1
35846752131,8sleep23586264-21cut2
35629542131,6sleep03562973-21apt-get0
39294852121,7sleep13609879-21diskmemload1
38846062121,5sleep00-21swapper/00
38783842121,7sleep23609879-21diskmemload2
38246032120,1sleep00-21swapper/00
37558962120,1sleep00-21swapper/00
37552112120,4sleep20-21swapper/22
37470872120,1sleep10-21swapper/11
37470732121,6sleep03609879-21diskmemload0
37372642120,1sleep20-21swapper/22
37319202121,6sleep13609879-21diskmemload1
37208952120,1sleep00-21swapper/00
36764072121,7sleep03676417-21ssh0
36508182121,6sleep13609879-21diskmemload1
36503882120,9sleep00-21swapper/00
36383162121,6sleep13639956-21tr1
35996762120,2sleep20-21swapper/22
35787322121,6sleep23578487-21sendmail-msp2
35754922121,7sleep13577252-21grep1
35264952120,1sleep00-21swapper/00
39635562111,3sleep20-21swapper/22
39533922111,6sleep13953468-21sh1
39405062111,6sleep13940671-21cat1
39119822110,3sleep20-21swapper/22
38966552111,5sleep13896721-21munin-run1
38879242110,1sleep10-21swapper/11
38852912111,4sleep23609879-21diskmemload2
38768892111,2sleep03876904-21ssh0
38616202110,1sleep10-21swapper/11
38527222112,2sleep1171rcu_preempt1
38345482111,5sleep13609879-21diskmemload1
38326192111,3sleep20-21swapper/22
38164252111,4sleep23609879-21diskmemload2
38090362110,3sleep20-21swapper/22
38090362110,3sleep20-21swapper/22
37993532111,5sleep23609879-21diskmemload2
37898642111,6sleep13789878-21ssh1
37746382111,6sleep13774651-21rm1
37735812111,4sleep23609879-21diskmemload2
37660592110,3sleep00-21swapper/00
37378932111,5sleep03609879-21diskmemload0
37196442111,4sleep23609879-21diskmemload2
37145502111,4sleep10-21swapper/11
37116852111,5sleep03609879-21diskmemload0
37089532111,5sleep13709919-21memory1
36860732111,4sleep20-21swapper/22
36783172111,6sleep13678331-21ssh1
36504152111,5sleep23609879-21diskmemload2
36413732111,4sleep20-21swapper/22
35682462111,5sleep13568243-21apt-get1
35355402111,5sleep03535610-21ntp_states0
35210322111,5sleep23521286-21df2
35203412111,3sleep10-21swapper/11
34947952111,5sleep03495050-21df0
39738662102,3sleep23609879-21diskmemload2
39636392101,3sleep00-21swapper/00
39559292101,3sleep20-21swapper/22
39385762100,1sleep20-21swapper/22
39215342101,2sleep10-21swapper/11
39045272101,4sleep13609879-21diskmemload1
38517102101,3sleep23609879-21diskmemload2
38427442101,2sleep00-21swapper/00
37934762101,5sleep03794643-21cut0
37830312101,4sleep13609879-21diskmemload1
37805972101,3sleep20-21swapper/22
37759952101,4sleep03776077-21irqstats0
37635822101,4sleep23609879-21diskmemload2
37286072101,3sleep00-21swapper/00
37235942101,5sleep23723710-21fschecks_time2
37027252101,3sleep00-21swapper/00
36955572101,2sleep20-21swapper/22
36945152100,6sleep03694526-21ssh0
36665732101,3sleep00-21swapper/00
36597672101,4sleep03609879-21diskmemload0
36275792101,5sleep20-21swapper/22
36262702101,3sleep10-21swapper/11
36248402100,2sleep20-21swapper/22
36239592101,5sleep03623974-21ssh0
36157482101,3sleep00-21swapper/00
36102552101,2sleep20-21swapper/22
36049222101,4sleep13604934-21apt-get1
35830552101,4sleep13583299-21users1
35527592101,4sleep23552967-21hddtemp_smartct2
35515972101,5sleep03551812-21awk0
35478032101,5sleep13548837-21cut1
35469592101,5sleep13547043-21latency_hist1
35460122101,4sleep03546069-21ntp_states0
34990902101,4sleep23499334-21users2
34941082100,1sleep20-21swapper/22
3970701291,4sleep03970765-21ssh0
3867872290,1sleep20-21swapper/22
3858730291,2sleep20-21swapper/22
3825252290,2sleep20-21swapper/22
3815846290,1sleep10-21swapper/11
3749372291,4sleep13749639-21df1
3702696290,1sleep20-21swapper/22
3662475291,4sleep13662386-21/usr/sbin/munin1
3641652291,4sleep03641666-21ssh0
3609864291,4sleep03610089-21cut0
3599669291,3sleep039542-21systemd-journal0
3594732291,5sleep13594780-21fschecks_count1
3593906290,3sleep10-21swapper/11
3589238291,3sleep03589240-21apt-get0
3573053290,3sleep00-21swapper/00
3557793291,4sleep03558011-21df_inode0
3521017291,4sleep13521029-21apt-get1
3515726290,1sleep20-21swapper/22
3510459291,3sleep03510462-21apt-get0
3504184291,4sleep03504381-21sed0
3494166290,3sleep00-21swapper/00
3486645291,4sleep13486862-21users1
3595457282,3sleep00-21swapper/00
3594174280,1sleep20-21swapper/22
3588809280,1sleep00-21swapper/00
3581064281,3sleep03582593-21/usr/sbin/munin0
3573481281,3sleep03573707-21expr0
3554137281,3sleep10-21swapper/11
3541407280,1sleep00-21swapper/00
3536813281,3sleep24192558-21rs:main2
3531238280,1sleep10-21swapper/11
3515751281,3sleep03516010-21df_inode0
3505222280,3sleep13505241-21apt-get1
3494760280,4sleep10-21swapper/11
3487297281,3sleep10-21swapper/11
3482825281,4sleep20-21swapper/22
3604890270,1sleep20-21swapper/22
3599171270,1sleep20-21swapper/22
3583968270,2sleep10-21swapper/11
3577253271,2sleep23577448-21strings2
3567828270,2sleep10-21swapper/11
3564217271,2sleep20-21swapper/22
3536791271,2sleep13536810-21apt-get1
3536492270,1sleep20-21swapper/22
3536322270,1sleep10-21swapper/11
3526281270,2sleep20-21swapper/22
3525172271,2sleep00-21swapper/00
3515313270,4sleep10-21swapper/11
3509802270,1sleep00-21swapper/00
3500312271,3sleep13500524-21if_enp1s01
3487054270,1sleep00-21swapper/00
3557724261,2sleep20-21swapper/22
3552485261,2sleep00-21swapper/00
3546949260,1sleep20-21swapper/22
3512619261,2sleep23514280-21ls2
3509920260,1sleep20-21swapper/22
3557718250,1sleep10-21swapper/11
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional