You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-12 - 04:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3s.osadl.org (updated Sun Apr 12, 2026 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2976693212158,58sleep30-21swapper/33
297653128671,10sleep10-21swapper/11
297656728364,11sleep20-21swapper/22
297666527260,6sleep00-21swapper/00
34075862550,3sleep0221rcuc/00
30581932450,2sleep00-21swapper/00
297688199221,17cyclictest3149129-21taskset3
297688199211,17cyclictest3210559-21taskset3
297688199211,15cyclictest0-21swapper/33
297688199211,15cyclictest0-21swapper/33
2976881992110,6cyclictest31662462sleep33
297688199202,14cyclictest3432835-21turbostat3
297688199201,15cyclictest0-21swapper/33
297688199201,14cyclictest0-21swapper/33
297688199200,17cyclictest0-21swapper/33
2976881991917,1cyclictest0-21swapper/33
297688199191,6cyclictest3465163-21taskset3
2976881991915,2cyclictest3414207-21taskset3
2976881991913,5cyclictest0-21swapper/33
2976881991912,5cyclictest0-21swapper/33
297688199191,15cyclictest0-21swapper/33
297688199191,15cyclictest0-21swapper/33
297688199191,14cyclictest0-21swapper/33
297688199190,15cyclictest3096573-21turbostat3
34680322181,12sleep13107280-21diskmemload1
32312232181,6sleep23231399-21grep2
32098402181,2sleep20-21swapper/22
31840942181,8sleep03184096-21ssh0
297688199181,5cyclictest0-21swapper/33
297688199181,14cyclictest0-21swapper/33
297688199181,14cyclictest0-21swapper/33
297688199181,14cyclictest0-21swapper/33
297688199181,14cyclictest0-21swapper/33
297688199181,14cyclictest0-21swapper/33
297688199181,14cyclictest0-21swapper/33
297688199181,14cyclictest0-21swapper/33
297688199181,14cyclictest0-21swapper/33
297688199181,14cyclictest0-21swapper/33
297688199181,14cyclictest0-21swapper/33
297688199181,14cyclictest0-21swapper/33
297688199181,13cyclictest0-21swapper/33
297688199181,13cyclictest0-21swapper/33
297688199181,13cyclictest0-21swapper/33
297688199181,13cyclictest0-21swapper/33
297688199181,13cyclictest0-21swapper/33
297688199181,13cyclictest0-21swapper/33
297688199180,15cyclictest0-21swapper/33
297688199180,14cyclictest66950irq/132-enp1s0-TxRx-33
34748942171,4sleep23107280-21diskmemload2
34473342171,4sleep03107280-21diskmemload0
33152082171,3sleep10-21swapper/11
32801152171,5sleep13107280-21diskmemload1
297688199177,7cyclictest0-21swapper/33
297688199172,12cyclictest2988854-21taskset3
297688199171,4cyclictest2998950-21chrt3
297688199171,4cyclictest0-21swapper/33
297688199171,14cyclictest0-21swapper/33
297688199171,13cyclictest0-21swapper/33
297688199171,13cyclictest0-21swapper/33
297688199171,12cyclictest0-21swapper/33
297688199170,14cyclictest0-21swapper/33
297688199170,14cyclictest0-21swapper/33
297688199170,13cyclictest0-21swapper/33
34405772162,9sleep13107280-21diskmemload1
33866032161,2sleep00-21swapper/00
33601092161,4sleep03360120-21ssh0
32298682161,2sleep20-21swapper/22
31130662161,2sleep20-21swapper/22
297688199160,13cyclictest0-21swapper/33
297688199160,13cyclictest0-21swapper/33
297688199160,13cyclictest0-21swapper/33
297688199160,13cyclictest0-21swapper/33
34286522151,10sleep03428701-21ssh0
33889062151,10sleep13388917-21ssh1
33695742151,2sleep10-21swapper/11
33604892151,8sleep23107280-21diskmemload2
33337052151,3sleep03107280-21diskmemload0
32100212151,2sleep00-21swapper/00
31846222150,1sleep10-21swapper/11
31659092151,2sleep00-21swapper/00
30057412151,10sleep13005988-21expr1
297688199158,4cyclictest3048202-21chrt3
297688199152,8cyclictest3016638-21taskset3
297688199151,3cyclictest0-21swapper/33
297688199151,2cyclictest0-21swapper/33
297688199150,2cyclictest0-21swapper/33
297688199150,13cyclictest0-21swapper/33
297688199150,13cyclictest0-21swapper/33
297688199150,12cyclictest0-21swapper/33
297688199150,12cyclictest0-21swapper/33
34240182140,1sleep20-21swapper/22
32922642141,3sleep13107280-21diskmemload1
32728792141,4sleep03272895-21ssh0
32537232141,8sleep13107280-21diskmemload1
32461662141,2sleep13107280-21diskmemload1
32379252141,2sleep00-21swapper/00
32098262140,2sleep1321rcuc/11
32023082140,8sleep10-21swapper/11
31249512141,9sleep23124965-21ssh2
31072642140,6sleep00-21swapper/00
297688199144,6cyclictest3067699-21taskset3
2976881991411,2cyclictest0-21swapper/33
2976881991411,2cyclictest0-21swapper/33
297688199140,3cyclictest0-21swapper/33
34580432131,8sleep03458053-21ssh0
34465282131,3sleep20-21swapper/22
34309462131,7sleep13107280-21diskmemload1
34287742131,8sleep23428911-21find2
34231322131,8sleep03423142-21ssh0
33857062130,1sleep10-21swapper/11
33529702133,5sleep23344044-21kworker/u16:0+nfsiod2
33511942131,8sleep03351208-21ssh0
33264762131,3sleep13326487-21ssh1
32986062130,1sleep20-21swapper/22
32911432131,6sleep13107280-21diskmemload1
32753622131,8sleep03276881-21ssh0
32630272131,7sleep13107280-21diskmemload1
32578022131,7sleep23259305-21cut2
32481622131,3sleep03248315-21ssh0
32197372131,2sleep20-21swapper/22
31657222131,6sleep23107280-21diskmemload2
31308872131,8sleep23130901-21ssh2
31249942131,3sleep13125005-21ssh1
30251582131,7sleep13025952-21memory1
297688199139,3cyclictest0-21swapper/33
33961512121,3sleep03396163-21ssh0
33771072121,8sleep23377121-21ssh2
33513202121,7sleep13107280-21diskmemload1
33062412121,4sleep10-21swapper/11
32928252121,7sleep03293226-21cut0
32819562121,3sleep23281967-21ssh2
32726832121,5sleep10-21swapper/11
32610792121,5sleep00-21swapper/00
32373182121,6sleep13107280-21diskmemload1
32305122121,7sleep13230634-21date1
31993972120,8sleep23199929-21missed_timers2
31957782121,6sleep03195811-21ssh0
31919072121,5sleep20-21swapper/22
31521092121,6sleep03152242-21/usr/sbin/munin0
31258492121,7sleep03107280-21diskmemload0
31009182121,5sleep23100967-21ntpq2
30582522121,7sleep13058365-21ps1
30220312121,6sleep03022289-21df_inode0
34723012110,1sleep00-21swapper/00
34672742110,1sleep10-21swapper/11
34148872111,4sleep13107280-21diskmemload1
34033392111,7sleep03403351-21rm0
33863752111,3sleep20-21swapper/22
33780532111,4sleep13107280-21diskmemload1
33389012111,5sleep13340133-21memory1
33315732111,5sleep23331643-21ntp_states2
33198472111,6sleep23321509-21cut2
33140352112,4sleep03314093-21ntpq0
32718462111,6sleep23271857-21ssh2
32483282110,3sleep23248342-21ssh2
32450872111,3sleep20-21swapper/22
32224822114,3sleep03222645-21tail0
31864532111,6sleep23186605-21latency_hist2
31671832111,5sleep13107280-21diskmemload1
31580892111,5sleep13107280-21diskmemload1
31574822111,3sleep20-21swapper/22
31422642111,7sleep13142275-21ssh1
31255132111,4sleep13107280-21diskmemload1
31021812111,3sleep10-21swapper/11
30959642111,5sleep13096181-21users1
30847732111,6sleep13084862-21ntpq1
30741332111,5sleep23074184-21ntpq2
30474682111,5sleep03047514-21ntpq0
30327152114,2sleep00-21swapper/00
30327152114,2sleep00-21swapper/00
30207132112,4sleep03020761-21ntpq0
30167162111,5sleep13016934-21df_inode1
30151442111,6sleep13015198-21ntpq1
29884262111,5sleep02988474-21ntpq0
34662122101,4sleep23107280-21diskmemload2
34564392101,6sleep20-21swapper/22
34469372101,3sleep13107280-21diskmemload1
34387292101,5sleep03438743-21ssh0
34206482101,2sleep10-21swapper/11
33975722100,6sleep13107280-21diskmemload1
33958762101,3sleep20-21swapper/22
33697642100,2sleep00-21swapper/00
33664012101,5sleep23366545-21netstat2
33608882101,4sleep13107280-21diskmemload1
33411242101,2sleep20-21swapper/22
33324892101,3sleep13107280-21diskmemload1
33096972101,3sleep20-21swapper/22
33077052100,2sleep00-21swapper/00
32903972101,2sleep20-21swapper/22
32837902101,4sleep03284014-21hddtemp_smartct0
32215112101,2sleep10-21swapper/11
32204372101,3sleep00-21swapper/00
31768662100,1sleep10-21swapper/11
31504122101,3sleep00-21swapper/00
31487432101,4sleep13107280-21diskmemload1
31486602101,4sleep23107280-21diskmemload2
31425222101,4sleep23107280-21diskmemload2
31167372101,5sleep03116903-21cstates0
31130012101,4sleep13107280-21diskmemload1
31022032101,4sleep23102461-21df_inode2
30754582101,5sleep13075479-21http1
30742902104,3sleep13074417-21sendmail_mailqu1
30688172101,4sleep13068863-21ntpq1
30594252101,5sleep13059496-21sort1
30353512101,4sleep23036635-21memory2
30353512101,4sleep23036635-21memory2
30271112101,5sleep23027145-21basename2
29990912100,1sleep00-21swapper/00
3464478291,4sleep03464491-21ssh0
3457192291,3sleep13107280-21diskmemload1
3437215291,4sleep23437365-21ssh2
3411963291,2sleep20-21swapper/22
3398036291,2sleep20-21swapper/22
3375421291,2sleep00-21swapper/00
3336068291,2sleep00-21swapper/00
3324085290,2sleep00-21swapper/00
3305387291,2sleep20-21swapper/22
3246411290,1sleep00-21swapper/00
3192048290,3sleep10-21swapper/11
3174774291,2sleep00-21swapper/00
3174652291,4sleep23174666-21ssh2
3138357291,3sleep03138468-21ntpq0
3107660291,4sleep03107681-21apt-get0
3091639291,3sleep03091762-21diskstats0
3016636291,4sleep23016637-21apt-get2
3005719291,3sleep03005738-21apt-get0
3000672290,1sleep00-21swapper/00
2999189291,4sleep22999240-21open_inodes2
3191885281,3sleep03191897-21ssh0
3101920281,4sleep13101934-21latency_hist1
3096837280,1sleep00-21swapper/00
3069646280,1sleep00-21swapper/00
3063814280,3sleep23063885-21find2
3053664280,1sleep10-21swapper/11
3039569281,3sleep13041339-21grep1
3033646282,3sleep13035349-21kthreadcore1
3033646282,3sleep13035349-21kthreadcore1
3011046281,2sleep00-21swapper/00
2977618281,3sleep22978768-21cat2
3091472270,2sleep20-21swapper/22
3090979270,3sleep20-21swapper/22
3086409271,2sleep13086462-21fschecks_count1
3085739270,1sleep20-21swapper/22
3085605270,1sleep00-21swapper/00
3080169270,1sleep00-21swapper/00
3042940270,1sleep00-21swapper/00
3032394271,3sleep2802-21snmpd2
3032072270,4sleep00-21swapper/00
3015889270,1sleep20-21swapper/22
3008007272,2sleep22976876-21cyclictest2
2990663271,3sleep02991854-21kthreadcore0
2989116270,1sleep10-21swapper/11
2981258271,3sleep02981371-21ps0
3087207261,2sleep00-21swapper/00
3063461260,3sleep00-21swapper/00
3053601260,4sleep00-21swapper/00
3049219261,2sleep20-21swapper/22
3048111260,1sleep20-21swapper/22
3032288260,4sleep10-21swapper/11
3000985261,2sleep20-21swapper/22
3000373260,2sleep13000392-21apt-get1
2995111261,1sleep10-21swapper/11
2994810260,1sleep10-21swapper/11
2994516260,1sleep20-21swapper/22
2981850260,4sleep10-21swapper/11
3075480250,1sleep20-21swapper/22
3074241250,2sleep00-21swapper/00
3064796250,1sleep20-21swapper/22
3058902250,1sleep20-21swapper/22
3047516250,1sleep10-21swapper/11
3042789250,3sleep20-21swapper/22
2989289250,1sleep20-21swapper/22
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional