You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-24 - 16:04

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3s.osadl.org (updated Tue Mar 24, 2026 12:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3003267211856,57sleep30-21swapper/33
311154721170,2sleep10-21swapper/11
300330728756,10sleep20-21swapper/22
300332028562,6sleep00-21swapper/00
300328528251,12sleep10-21swapper/11
33508552251,3sleep00-21swapper/00
300364999250,7cyclictest0-21swapper/33
32475702241,2sleep10-21swapper/11
300364999242,8cyclictest3299864-21taskset3
300364999242,8cyclictest3299864-21taskset3
300364999242,18cyclictest0-21swapper/33
31787802231,7sleep03180248-21kthreadcore0
300364999231,8cyclictest0-21swapper/33
300364999231,19cyclictest0-21swapper/33
35010692221,7sleep03501079-21ssh0
3003649992216,3cyclictest3227664-21taskset3
300364999221,5cyclictest3016277-21turbostat3
300364999221,18cyclictest0-21swapper/33
300364999220,13cyclictest0-21swapper/33
34859092210,9sleep134-21ksoftirqd/11
32047962212,7sleep23204932-21perl2
300364999212,16cyclictest0-21swapper/33
300364999211,7cyclictest3048368-21turbostat3
300364999211,4cyclictest0-21swapper/33
300364999211,17cyclictest0-21swapper/33
300364999211,17cyclictest0-21swapper/33
300364999210,18cyclictest0-21swapper/33
33446802201,4sleep23133908-21diskmemload2
300364999205,11cyclictest0-21swapper/33
300364999202,6cyclictest3123208-21turbostat3
300364999202,6cyclictest3123208-21turbostat3
300364999201,7cyclictest3107254-21turbostat3
300364999201,5cyclictest0-21swapper/33
3003649992013,5cyclictest0-21swapper/33
3003649992011,6cyclictest3307400-21taskset3
300364999201,16cyclictest0-21swapper/33
300364999201,15cyclictest3377214-21taskset3
300364999201,15cyclictest0-21swapper/33
300364999201,15cyclictest0-21swapper/33
300364999201,13cyclictest0-21swapper/33
34491812190,2sleep13133908-21diskmemload1
34028392191,7sleep23133908-21diskmemload2
31862552191,9sleep23133908-21diskmemload2
30645962191,2sleep00-21swapper/00
300364999191,7cyclictest3265739-21turbostat3
300364999191,7cyclictest3064347-21turbostat3
300364999191,6cyclictest3493679-21turbostat3
300364999191,6cyclictest3037704-21turbostat3
3003649991916,1cyclictest0-21swapper/33
3003649991915,2cyclictest0-21swapper/33
3003649991914,3cyclictest3150234-21taskset3
300364999191,16cyclictest0-21swapper/33
300364999191,15cyclictest32903442sleep33
300364999191,15cyclictest32903442sleep33
300364999191,15cyclictest0-21swapper/33
300364999191,15cyclictest0-21swapper/33
300364999191,15cyclictest0-21swapper/33
300364999191,15cyclictest0-21swapper/33
300364999191,14cyclictest0-21swapper/33
300364999191,14cyclictest0-21swapper/33
300364999191,14cyclictest0-21swapper/33
300364999190,2cyclictest0-21swapper/33
300364999190,10cyclictest0-21swapper/33
32631852181,7sleep13133908-21diskmemload1
32626182181,2sleep00-21swapper/00
300364999181,7cyclictest0-21swapper/33
300364999181,15cyclictest0-21swapper/33
300364999181,14cyclictest0-21swapper/33
300364999181,14cyclictest0-21swapper/33
300364999181,14cyclictest0-21swapper/33
300364999181,14cyclictest0-21swapper/33
300364999181,13cyclictest0-21swapper/33
300364999181,13cyclictest0-21swapper/33
300364999180,9cyclictest0-21swapper/33
300364999180,4cyclictest3031103-21taskset3
300364999180,1cyclictest0-21swapper/33
300364999180,16cyclictest0-21swapper/33
300364999180,14cyclictest0-21swapper/33
300364999171,13cyclictest0-21swapper/33
300364999171,12cyclictest0-21swapper/33
34784792161,7sleep13480214-21kthreadcore1
34759802161,2sleep10-21swapper/11
32296832161,3sleep10-21swapper/11
30473772161,10sleep23047424-21ntpq2
300364999162,6cyclictest0-21swapper/33
300364999161,3cyclictest0-21swapper/33
300364999160,13cyclictest0-21swapper/33
300364999160,13cyclictest0-21swapper/33
34120772150,1sleep20-21swapper/22
33957672151,2sleep10-21swapper/11
33361402155,2sleep0171rcu_preempt0
33333802151,2sleep20-21swapper/22
32936132151,9sleep13295062-21tr1
32936132151,9sleep13295062-21tr1
32560292151,3sleep13133908-21diskmemload1
32442512150,2sleep2431ktimers/22
32200562151,3sleep13133908-21diskmemload1
31699672151,10sleep03170803-21cut0
31114332150,4sleep03111546-21ntpq0
300364999150,12cyclictest0-21swapper/33
34680512141,9sleep23468241-21sed2
34645472141,8sleep13133908-21diskmemload1
34457182142,7sleep23445771-21ssh2
34033232140,2sleep00-21swapper/00
33868422141,5sleep20-21swapper/22
33758912141,9sleep13375951-21ntpq1
33373282141,4sleep13133908-21diskmemload1
33260752141,3sleep13133908-21diskmemload1
33079302141,7sleep13133908-21diskmemload1
32666852141,6sleep23266767-21/usr/sbin/munin2
32563692141,9sleep03133908-21diskmemload0
31682602141,8sleep13133908-21diskmemload1
31568522141,3sleep23133908-21diskmemload2
30218632140,2sleep10-21swapper/11
3003649991411,2cyclictest0-21swapper/33
300364999140,11cyclictest0-21swapper/33
34923242131,8sleep03492337-21ssh0
33799382130,3sleep23379952-21ssh2
33505252131,3sleep23350527-21ssh2
33422172131,2sleep00-21swapper/00
32990012131,6sleep00-21swapper/00
32990012131,6sleep00-21swapper/00
32914392130,4sleep10-21swapper/11
32914392130,4sleep10-21swapper/11
32615572131,6sleep23261616-21ntpq2
32194452131,1sleep20-21swapper/22
32013252131,5sleep00-21swapper/00
31388382131,8sleep03139043-21ssh0
31230032130,2sleep10-21swapper/11
31230032130,2sleep10-21swapper/11
34817922121,6sleep00-21swapper/00
34641792120,4sleep00-21swapper/00
34322682120,2sleep03379331-21kworker/u16:3+rpciod0
33978832122,4sleep13397912-21apt-get1
33950932121,4sleep03395104-21pool-udisksd0
33879322121,3sleep13387944-21ssh1
33684302121,7sleep03368443-21ssh0
33250042121,7sleep03325014-21ssh0
32548222121,6sleep20-21swapper/22
32394952121,5sleep00-21swapper/00
31766502121,7sleep23176664-21ssh2
31405922121,5sleep10-21swapper/11
31021832121,1sleep20-21swapper/22
30920802121,7sleep23092177-21ps2
30374712120,7sleep03037396-31find0
30263522121,7sleep03021864-21/usr/sbin/munin0
34676392111,3sleep00-21swapper/00
34309872111,4sleep272-21kcompactd02
34149542111,4sleep13133908-21diskmemload1
33822162111,7sleep03383905-21kthreadcore0
33695662111,4sleep13133908-21diskmemload1
33632542111,6sleep23363431-21/usr/sbin/munin2
33535112111,5sleep13133908-21diskmemload1
33324172111,5sleep13133908-21diskmemload1
33087362110,2sleep00-21swapper/00
32827162111,3sleep10-21swapper/11
32809232110,1sleep00-21swapper/00
32368992114,4sleep2431ktimers/22
31943632110,3sleep13194377-21ssh1
31824042111,5sleep13182471-21ntp_states1
31487732111,4sleep13133908-21diskmemload1
31461262111,3sleep20-21swapper/22
31384342111,4sleep23138492-21ntpq2
31184392110,7sleep23118642-21gltestperf2
31184392110,7sleep23118642-21gltestperf2
31009252111,5sleep13100975-21ntpq1
30968572111,5sleep03096876-21apt-get0
30878942111,6sleep03089668-21grep0
30849382111,5sleep23084986-21ntpq2
30740092111,5sleep23074059-21ntpq2
30259222111,5sleep23025973-21ntpq2
30205562111,5sleep13020604-21ntpq1
30171072111,6sleep23018329-21kthreadcore2
30089722111,5sleep13009387-21ntp_states1
35013322101,2sleep20-21swapper/22
34912502101,4sleep23133908-21diskmemload2
34666862101,3sleep23133908-21diskmemload2
34475162100,4sleep0151ktimers/00
34407462101,3sleep00-21swapper/00
34386952101,3sleep13133908-21diskmemload1
34305062101,4sleep13133908-21diskmemload1
34211712101,4sleep13133908-21diskmemload1
34179322102,3sleep20-21swapper/22
34154422101,4sleep03133908-21diskmemload0
34124072106,1sleep00-21swapper/00
33946572100,1sleep20-21swapper/22
33770792101,3sleep00-21swapper/00
33602332101,3sleep00-21swapper/00
33585312101,2sleep10-21swapper/11
33547772101,5sleep23355430-21kthreadcore2
33251262101,5sleep23325137-21ssh2
33186532101,2sleep00-21swapper/00
33117462101,5sleep13313385-21cut1
33004552101,3sleep20-21swapper/22
33004552101,3sleep20-21swapper/22
32920572101,3sleep23133908-21diskmemload2
32920572101,3sleep23133908-21diskmemload2
32884212101,4sleep03288479-21ntpq0
32884212101,4sleep03288479-21ntpq0
32822192101,4sleep23133908-21diskmemload2
32725062101,3sleep10-21swapper/11
32724442101,3sleep00-21swapper/00
32441282101,4sleep03244185-21ntpq0
32363842101,2sleep10-21swapper/11
32292592101,6sleep00-21swapper/00
32283622101,3sleep20-21swapper/22
32203942101,4sleep03133908-21diskmemload0
32009862101,3sleep13133908-21diskmemload1
31941732101,3sleep20-21swapper/22
31769392101,4sleep13133908-21diskmemload1
31590212101,2sleep10-21swapper/11
31563952101,5sleep03156410-21sshd0
31491192101,3sleep00-21swapper/00
31290932101,5sleep03129280-21gltestperf0
31235572101,4sleep13123766-21diskstats1
30315402101,2sleep10-21swapper/11
30274492101,4sleep23027552-21perl2
30271692100,4sleep00-21swapper/00
3500575290,4sleep13133908-21diskmemload1
3476831291,3sleep23477054-21ssh2
3456250291,2sleep00-21swapper/00
3455029290,2sleep20-21swapper/22
3450754291,4sleep13133908-21diskmemload1
3437961290,1sleep20-21swapper/22
3316627290,1sleep20-21swapper/22
3301733291,2sleep20-21swapper/22
3209563291,2sleep00-21swapper/00
3200697291,2sleep20-21swapper/22
3166093291,2sleep00-21swapper/00
3160512290,3sleep20-21swapper/22
3124050291,4sleep03125278-21grep0
3118721291,5sleep03119926-21kthreadcore0
3118721291,5sleep03119926-21kthreadcore0
3116905291,4sleep23116951-21ntpq2
3113335293,3sleep13113430-21kthreadcore1
3079637291,4sleep03079858-21/usr/sbin/munin0
3065166291,4sleep23066350-21cut2
3047329291,3sleep13047375-21ntpq1
3037341291,3sleep13037410-40gst-plugin-scan1
3015271291,3sleep03015318-21ntpq0
3008643290,2sleep20-21swapper/22
3204109281,3sleep13204364-21ssh1
3191945281,3sleep03191956-21ssh0
3129374283,2sleep2401irq_work/22
3096297280,1sleep00-21swapper/00
3086699281,3sleep23086815-21/usr/sbin/munin2
3075254280,2sleep20-21swapper/22
3070219281,3sleep03070433-21smartctl0
3048618281,2sleep00-21swapper/00
3041926280,3sleep13042014-21ntpq1
3015379280,5sleep23015511-21mailstats2
3133780270,1sleep10-21swapper/11
3117797270,1sleep00-21swapper/00
3107118270,1sleep10-21swapper/11
3091867271,2sleep13092079-21smartctl1
3069926270,2sleep10-21swapper/11
3069532270,1sleep10-21swapper/11
3063338270,2sleep10-21swapper/11
3058629270,4sleep20-21swapper/22
3051378270,3sleep20-21swapper/22
3049668271,3sleep13051377-21grep1
3008593270,1sleep00-21swapper/00
3005687271,3sleep10-21swapper/11
3108772261,2sleep20-21swapper/22
3104284261,2sleep00-21swapper/00
3091239260,1sleep10-21swapper/11
3085639260,1sleep10-21swapper/11
3063389261,2sleep20-21swapper/22
3053966261,2sleep10-21swapper/11
3053944260,1sleep00-21swapper/00
3047481260,3sleep00-21swapper/00
3037473260,2sleep290-21kswapd02
3081047250,1sleep00-21swapper/00
3079475250,1sleep10-21swapper/11
3059062250,1sleep00-21swapper/00
3042021250,1sleep20-21swapper/22
3021395250,2sleep00-21swapper/00
3123767240,1sleep20-21swapper/22
3097134240,1sleep20-21swapper/22
3038251240,1sleep00-21swapper/00
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional