You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-10-20 - 00:37

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack3slot3s.osadl.org (updated Tue Sep 29, 2026 00:43:53)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304154021500,4sleep23041534-21cc12
34749012760,2sleep10-21swapper/11
29782272631,53sleep30-21swapper/33
33070032470,5sleep2431ktimers/22
29874802284,15sleep22987489-21cc12
30239682262,9sleep23023979-21cc12
298950399260,18cyclictest0-21swapper/33
30893482253,17sleep13089376-21cc11
30737532251,8sleep13073785-21cc11
30196372251,8sleep13019640-21cc11
298950399250,16cyclictest0-21swapper/33
29833192252,8sleep12983122-21cc11
32282502241,3sleep00-21swapper/00
2989503992420,2cyclictest0-21swapper/33
2989503992418,3cyclictest3024103-21taskset3
2989503992414,6cyclictest0-21swapper/33
298950399240,5cyclictest0-21swapper/33
29853962241,16sleep02985422-21nm0
29965332231,16sleep22996542-21cc12
298950399233,16cyclictest0-21swapper/33
2989503992315,4cyclictest0-21swapper/33
298950399230,5cyclictest0-21swapper/33
298950399230,5cyclictest0-21swapper/33
32668002221,3sleep00-21swapper/00
32633872222,16sleep23265035-21latency2
30877332221,8sleep03087751-21cc10
30563602221,9sleep13056381-21cc11
30095112221,15sleep13009520-21cc11
29966662222,14sleep12996656-21cc11
298950399224,15cyclictest0-21swapper/33
298950399221,17cyclictest0-21swapper/33
298950399221,14cyclictest0-21swapper/33
298950399220,5cyclictest0-21swapper/33
298950399220,15cyclictest0-21swapper/33
30733282211,8sleep03073336-21cc10
30439742211,15sleep13043990-21nm1
30336612213,11sleep03033637-21cc10
30205772211,8sleep03020592-21cc10
298950399213,15cyclictest0-21swapper/33
298950399211,9cyclictest0-21swapper/33
2989503992115,4cyclictest0-21swapper/33
2989503992114,4cyclictest3359567-21taskset3
2989503992112,5cyclictest0-21swapper/33
298950399211,18cyclictest0-21swapper/33
298950399211,17cyclictest0-21swapper/33
298950399211,15cyclictest0-21swapper/33
30951122201,14sleep03095120-21cc10
30319312201,6sleep13031904-21cc11
298950399205,12cyclictest32016812sleep33
298950399203,13cyclictest0-21swapper/33
298950399203,13cyclictest0-21swapper/33
2989503992015,3cyclictest0-21swapper/33
2989503992014,4cyclictest66950irq/132-enp1s0-TxRx-33
2989503992014,4cyclictest66950irq/132-enp1s0-TxRx-33
2989503992014,4cyclictest0-21swapper/33
2989503992012,5cyclictest0-21swapper/33
298950399201,16cyclictest0-21swapper/33
298950399201,15cyclictest0-21swapper/33
298950399200,4cyclictest0-21swapper/33
298950399200,4cyclictest0-21swapper/33
298950399200,16cyclictest0-21swapper/33
298950399200,14cyclictest0-21swapper/33
298950399200,14cyclictest0-21swapper/33
30789662191,13sleep13079016-21cc11
30744542191,11sleep23074462-21cc12
30416722191,11sleep03041654-21cc10
30322322191,13sleep23032231-21cc12
30080722191,11sleep03008086-21cc10
29979832192,10sleep02997972-21cc10
298950399194,10cyclictest3277648-21taskset3
298950399193,13cyclictest0-21swapper/33
298950399192,14cyclictest0-21swapper/33
298950399192,13cyclictest0-21swapper/33
298950399191,15cyclictest0-21swapper/33
298950399191,15cyclictest0-21swapper/33
298950399191,14cyclictest0-21swapper/33
298950399191,14cyclictest0-21swapper/33
298950399191,14cyclictest0-21swapper/33
298950399190,2cyclictest0-21swapper/33
298950399190,1cyclictest0-21swapper/33
298950399190,17cyclictest0-21swapper/33
35256742181,3sleep03186555-21diskmemload0
33501052181,3sleep20-21swapper/22
32110852180,12sleep00-21swapper/00
30721932181,6sleep23072209-21cc12
30517962181,12sleep23051799-21cc12
298950399187,7cyclictest3451036-21turbostat3
298950399181,9cyclictest0-21swapper/33
298950399181,14cyclictest0-21swapper/33
298950399181,13cyclictest3176009-21turbostat3
298950399181,13cyclictest3154885-21turbostat3
298950399181,13cyclictest3128469-21turbostat3
298950399180,3cyclictest0-21swapper/33
298950399180,15cyclictest0-21swapper/33
298950399180,15cyclictest0-21swapper/33
35117782171,7sleep13513222-21grep1
35082412171,4sleep23502860-21sendmail-msp2
33914782171,3sleep00-21swapper/00
33774022172,11sleep13377415-21apt-get1
33774022172,11sleep13377415-21apt-get1
32535132171,4sleep13254981-21grep1
30524712171,11sleep03052497-21nm0
30094472171,9sleep23009437-21cc12
298950399171,13cyclictest3160296-21turbostat3
298950399171,13cyclictest3149628-21turbostat3
298950399171,13cyclictest0-21swapper/33
298950399171,12cyclictest3123191-21turbostat3
298950399170,15cyclictest0-21swapper/33
298950399170,14cyclictest0-21swapper/33
32833732161,2sleep20-21swapper/22
32018502161,2sleep20-21swapper/22
30973102164,6sleep23097334-21cc12
298950399162,9cyclictest3155221-21taskset3
298950399161,12cyclictest3186561-21turbostat3
298950399161,12cyclictest3186561-21turbostat3
34159432151,4sleep03186555-21diskmemload0
33511022151,10sleep13351113-21ssh1
33259932151,4sleep13186555-21diskmemload1
32778032151,8sleep03277936-21diskstats0
32496512151,8sleep20-21swapper/22
32426622151,3sleep20-21swapper/22
298950399150,12cyclictest0-21swapper/33
35351422141,9sleep10-21swapper/11
34926702140,4sleep23492684-21ssh2
34645622141,3sleep20-21swapper/22
34643092141,6sleep00-21swapper/00
34403742141,2sleep10-21swapper/11
33676342141,9sleep03367644-21ssh0
33676342141,9sleep03367644-21ssh0
32885942141,8sleep00-21swapper/00
32741092140,2sleep10-21swapper/11
31181782143,6sleep13118175-21apt-get1
31175282141,8sleep03112611-21sendmail-msp0
298950399140,12cyclictest0-21swapper/33
34922332130,8sleep13186555-21diskmemload1
34846932132,6sleep13186555-21diskmemload1
34412302134,5sleep2431ktimers/22
33762912131,8sleep03376302-21ssh0
33762912131,8sleep03376302-21ssh0
32913842131,6sleep20-21swapper/22
32419612131,9sleep03186555-21diskmemload0
32106162130,9sleep13210627-21ssh1
31495682130,1sleep00-21swapper/00
31181562130,2sleep00-21swapper/00
35355082121,6sleep23186555-21diskmemload2
35231472121,7sleep23523424-21grep2
35170622121,7sleep03517077-21ssh0
34435012121,5sleep03443721-21hddtemp_smartct0
34360342121,7sleep03437434-21cut0
34315772120,1sleep10-21swapper/11
34249342120,1sleep10-21swapper/11
34243502120,1sleep20-21swapper/22
33650082121,5sleep13365077-21ntp_states1
33650082121,5sleep13365077-21ntp_states1
33595472120,1sleep10-21swapper/11
33423352121,5sleep00-21swapper/00
33412142120,3sleep23341225-21ssh2
33176012120,1sleep00-21swapper/00
32571712122,6sleep23257304-21switchtime2
32170532120,1sleep00-21swapper/00
32023062121,5sleep00-21swapper/00
31393892121,5sleep23139648-21df_inode2
35031762111,5sleep13503207-21apt-get1
34977752111,5sleep03498940-21memory0
34869282111,5sleep03487148-21hddtemp_smartct0
34735822111,6sleep03467642-40find0
34571392111,5sleep23186555-21diskmemload2
34437222111,4sleep13186555-21diskmemload1
34269882111,5sleep03427210-21hddtemp_smartct0
34164252110,5sleep10-21swapper/11
34095232111,5sleep13186555-21diskmemload1
34086602111,2sleep20-21swapper/22
34063882111,5sleep03406519-21/usr/sbin/munin0
34009582110,2sleep10-21swapper/11
33619242111,6sleep23363493-21grep2
33619242111,6sleep23363493-21grep2
33578392111,3sleep00-21swapper/00
33359072111,6sleep13336157-21ssh1
33025452111,5sleep03186555-21diskmemload0
32986962111,5sleep23298950-21users2
32846242111,6sleep13284627-21ssh1
32516932111,1sleep10-21swapper/11
32476662111,5sleep03248632-21memory0
32345562111,3sleep20-21swapper/22
32273902111,3sleep10-21swapper/11
32201072112,4sleep23220126-21apt-get2
32136622111,6sleep13215349-21cut1
32012772110,3sleep10-21swapper/11
31807842111,4sleep23176036-21sendmail-msp2
31542042111,5sleep13154441-21/usr/sbin/munin1
35331602101,4sleep03186555-21diskmemload0
35263662101,1sleep10-21swapper/11
35186202101,4sleep23186555-21diskmemload2
35018662101,4sleep13186555-21diskmemload1
35002712101,3sleep20-21swapper/22
34835392101,2sleep20-21swapper/22
34697902101,5sleep2483560-21systemd-journal2
34644702101,5sleep13464478-21sh1
34584662101,3sleep00-21swapper/00
34534182101,5sleep13455094-21sh1
34483262101,3sleep20-21swapper/22
34323072101,3sleep23186555-21diskmemload2
34260652101,4sleep03186555-21diskmemload0
33990502101,5sleep03399063-21ssh0
33900502101,3sleep10-21swapper/11
33826492100,1sleep20-21swapper/22
33826492100,1sleep20-21swapper/22
33758092100,1sleep20-21swapper/22
33758092100,1sleep20-21swapper/22
33690592100,1sleep10-21swapper/11
33690592100,1sleep10-21swapper/11
33575772102,3sleep23186555-21diskmemload2
33518442101,3sleep00-21swapper/00
33339902101,5sleep03186555-21diskmemload0
33330582101,4sleep23186555-21diskmemload2
33267632101,2sleep20-21swapper/22
33186832101,2sleep20-21swapper/22
33042752101,5sleep13305788-21tr1
33010422101,2sleep10-21swapper/11
32942332101,5sleep03294494-21df0
32912812101,4sleep13186555-21diskmemload1
32752212101,4sleep03186555-21diskmemload0
32743972105,2sleep20-21swapper/22
32658552100,1sleep10-21swapper/11
32334172101,2sleep10-21swapper/11
32258842101,3sleep00-21swapper/00
32153502101,5sleep23215792-21cut2
31933942100,6sleep20-21swapper/22
31919432101,4sleep03186555-21diskmemload0
31917552101,4sleep13186555-21diskmemload1
31811502100,1sleep00-21swapper/00
31706472100,2sleep20-21swapper/22
31387412101,4sleep23133778-21sendmail-msp2
31342952101,5sleep13134417-21gltestperf1
3483976291,4sleep03483987-21ssh0
3416058290,3sleep20-21swapper/22
3398410294,2sleep23398422-21sh2
3385639291,2sleep20-21swapper/22
3332862290,1sleep10-21swapper/11
3325034291,2sleep00-21swapper/00
3317025291,2sleep10-21swapper/11
3258586291,2sleep00-21swapper/00
3237849291,5sleep13238625-21kthreadcore1
3209727291,2sleep20-21swapper/22
3186292290,6sleep00-21swapper/00
3186292290,6sleep00-21swapper/00
3155198291,4sleep13155217-21apt-get1
3151931291,4sleep03153704-21grep0
3139388291,2sleep10-21swapper/11
3123460291,2sleep00-21swapper/00
3112838291,3sleep23112611-21sendmail-msp2
3108973291,4sleep10-21swapper/11
3508657281,3sleep03508665-21ssh0
3382731281,3sleep03382741-21ssh0
3382731281,3sleep03382741-21ssh0
3186273280,1sleep10-21swapper/11
3186273280,1sleep10-21swapper/11
3172501281,3sleep03174187-21kthreadcore0
3170467280,6sleep00-21swapper/00
3160994281,3sleep13161110-21irqstats1
3159912281,3sleep23154935-21sendmail-msp2
3133461280,1sleep10-21swapper/11
3130113281,3sleep00-21swapper/00
3112332280,2sleep20-21swapper/22
3108105281,3sleep03108970-21kthreadcore0
3181559270,2sleep20-21swapper/22
3181559270,2sleep20-21swapper/22
3176254271,3sleep10-21swapper/11
3175916270,1sleep10-21swapper/11
3160830270,4sleep00-21swapper/00
3160563270,3sleep23160610-21grep2
3159910271,2sleep00-21swapper/00
3153895270,1sleep20-21swapper/22
3149441270,5sleep10-21swapper/11
3147266271,3sleep20-21swapper/22
3139366270,2sleep00-21swapper/00
3133993271,2sleep0205-21jbd2/sda2-80
3128719270,1sleep20-21swapper/22
3127687271,2sleep20-21swapper/22
3113998271,3sleep13115475-21kthreadcore1
3170996260,1sleep20-21swapper/22
3170434260,1sleep10-21swapper/11
3124465261,2sleep10-21swapper/11
3119548261,2sleep20-21swapper/22
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional