You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-08 - 01:51

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot3s.osadl.org (updated Mon Jun 08, 2026 00:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3861252212554,66sleep30-21swapper/33
386115728454,12sleep20-21swapper/22
386131327953,10sleep10-21swapper/11
386097627554,14sleep00-21swapper/00
386145599261,4cyclictest4141482-21chrt3
40312782251,2sleep13991631-21diskmemload1
386145599251,14cyclictest0-21swapper/33
40756002231,8sleep14075610-21ssh1
386145599239,7cyclictest0-21swapper/33
386145599224,14cyclictest41609072sleep33
386145599211,16cyclictest0-21swapper/33
41332882201,8sleep04133298-21ssh0
409211422011,2sleep10-21swapper/11
40149312201,7sleep13991631-21diskmemload1
386145599209,8cyclictest0-21swapper/33
386145599205,11cyclictest3932989-21turbostat3
386145599201,9cyclictest85791-21turbostat3
3861455992014,3cyclictest0-21swapper/33
386145599201,3cyclictest0-21swapper/33
386145599201,16cyclictest0-21swapper/33
386145599201,15cyclictest4091397-21taskset3
386145599201,15cyclictest0-21swapper/33
386145599201,15cyclictest0-21swapper/33
386145599200,1cyclictest0-21swapper/33
1193282201,3sleep20-21swapper/22
405111621912,2sleep13991631-21diskmemload1
40412222191,3sleep14041230-21sh1
386145599193,13cyclictest0-21swapper/33
3861455991914,3cyclictest0-21swapper/33
386145599191,3cyclictest0-21swapper/33
386145599191,15cyclictest0-21swapper/33
386145599191,14cyclictest0-21swapper/33
386145599191,14cyclictest0-21swapper/33
386145599191,14cyclictest0-21swapper/33
386145599191,14cyclictest0-21swapper/33
386145599190,6cyclictest4150714-21taskset3
386145599190,16cyclictest0-21swapper/33
386145599190,12cyclictest0-21swapper/33
1099002191,4sleep2109911-21ssh2
41764842181,2sleep00-21swapper/00
40311192181,6sleep04031136-21ssh0
38956982181,5sleep03895769-21munin-run0
386145599188,7cyclictest4184302-21taskset3
386145599185,9cyclictest3959653-21turbostat3
386145599181,4cyclictest0-21swapper/33
386145599181,3cyclictest0-21swapper/33
386145599181,3cyclictest0-21swapper/33
386145599181,3cyclictest0-21swapper/33
3861455991813,3cyclictest0-21swapper/33
3861455991812,3cyclictest3889985-21taskset3
386145599181,15cyclictest4040232-21taskset3
386145599181,14cyclictest0-21swapper/33
386145599181,14cyclictest0-21swapper/33
386145599181,14cyclictest0-21swapper/33
386145599181,13cyclictest0-21swapper/33
386145599181,13cyclictest0-21swapper/33
386145599181,13cyclictest0-21swapper/33
386145599181,13cyclictest0-21swapper/33
386145599181,13cyclictest0-21swapper/33
386145599180,9cyclictest0-21swapper/33
386145599180,9cyclictest0-21swapper/33
386145599180,15cyclictest0-21swapper/33
386145599180,15cyclictest0-21swapper/33
386145599180,14cyclictest0-21swapper/33
386145599180,10cyclictest0-21swapper/33
82102171,3sleep10-21swapper/11
506572170,4sleep134-21ksoftirqd/11
41755622171,3sleep20-21swapper/22
3861455991714,2cyclictest0-21swapper/33
3861455991714,1cyclictest0-21swapper/33
386145599171,3cyclictest0-21swapper/33
3861455991713,3cyclictest0-21swapper/33
386145599171,13cyclictest0-21swapper/33
386145599170,7cyclictest0-21swapper/33
253892170,3sleep00-21swapper/00
41680612161,2sleep20-21swapper/22
41592132161,4sleep03991631-21diskmemload0
40730252161,10sleep24073081-21ntpq2
386145599162,9cyclictest111411-21taskset3
386145599160,13cyclictest0-21swapper/33
386145599160,12cyclictest0-21swapper/33
314272161,3sleep13991631-21diskmemload1
149912161,2sleep10-21swapper/11
924412150,1sleep10-21swapper/11
611952152,7sleep161416-21hddtemp_smartct1
41418362151,4sleep23991631-21diskmemload2
40656652151,8sleep00-21swapper/00
40573402150,2sleep10-21swapper/11
40064212151,2sleep20-21swapper/22
386145599151,3cyclictest3901053-21turbostat3
386145599151,2cyclictest0-21swapper/33
3861455991512,2cyclictest0-21swapper/33
3861455991512,2cyclictest0-21swapper/33
386145599150,12cyclictest0-21swapper/33
386145599150,12cyclictest0-21swapper/33
386145599150,11cyclictest0-21swapper/33
1507222151,9sleep13991631-21diskmemload1
734332141,9sleep273479-21ntpq2
597182141,2sleep00-21swapper/00
41660162142,6sleep14166068-21ntpq1
41491332140,4sleep10-21swapper/11
41197292141,5sleep24119760-21cpu2
41079742140,3sleep20-21swapper/22
40418042140,8sleep20-21swapper/22
386145599148,3cyclictest39648372sleep33
386145599140,12cyclictest0-21swapper/33
141302141,9sleep014219-21ntpq0
514712131,4sleep051482-21ssh0
41939812130,2sleep2171rcu_preempt2
41873882135,4sleep00-21swapper/00
41864472131,2sleep20-21swapper/22
41604632131,8sleep24160480-21ssh2
41449242131,8sleep04145150-21ssh0
40834512131,6sleep00-21swapper/00
40248132131,3sleep04024824-21ssh0
40232482130,1sleep10-21swapper/11
40178932131,8sleep24018274-21sh2
312022131,8sleep231333-21rt-features2
158362131,2sleep2171rcu_preempt2
1354862131,7sleep13991631-21diskmemload1
1267212131,8sleep2126732-21ssh2
921442120,4sleep20-21swapper/22
857062121,5sleep00-21swapper/00
660332121,7sleep066036-21rm0
531742121,7sleep153692-21cut1
517192121,6sleep23991631-21diskmemload2
433152121,7sleep143329-21ssh1
432532121,7sleep043264-21ssh0
41827632121,5sleep14182819-21ntpq1
41533612122,4sleep13991631-21diskmemload1
41190502121,7sleep14119091-21perf1
41027422122,6sleep04102935-21cstates0
40948192121,7sleep14095656-21cut1
40900182121,5sleep04090082-21ntpq0
40766772121,6sleep00-21swapper/00
40660682121,6sleep23991631-21diskmemload2
40595762121,7sleep23991631-21diskmemload2
40159172121,7sleep04015927-21ssh0
39871132121,7sleep23988083-21cut2
39798902124,4sleep13979942-21collect21
39599492121,7sleep13960194-21expr1
1527962121,6sleep03991631-21diskmemload0
1347862121,7sleep0134788-21ssh0
1341492120,4sleep20-21swapper/22
1264462121,4sleep10-21swapper/11
1059192121,5sleep0107125-21memory0
1007032121,7sleep0100717-21ssh0
946432111,3sleep20-21swapper/22
93982111,4sleep03991631-21diskmemload0
776172111,6sleep177629-21ssh1
41847012111,4sleep03991631-21diskmemload0
41422622111,3sleep00-21swapper/00
41323582111,5sleep24132425-21ntp_states2
41263722111,4sleep13991631-21diskmemload1
41251022111,4sleep00-21swapper/00
41159552110,3sleep00-21swapper/00
41023352111,6sleep04102352-21sh0
40912842111,5sleep23991631-21diskmemload2
40848112111,3sleep20-21swapper/22
40678512111,3sleep10-21swapper/11
40461942111,3sleep20-21swapper/22
40319422112,4sleep23991631-21diskmemload2
40154092111,4sleep23991631-21diskmemload2
40076082111,7sleep03991631-21diskmemload0
39979342111,2sleep00-21swapper/00
39961752111,5sleep23996234-21ntpq2
39703192111,4sleep23970339-21munin-run2
39703192111,4sleep23970339-21munin-run2
39580452111,6sleep03958483-21memory0
39543152111,4sleep13954363-21/usr/sbin/munin1
39440032111,5sleep13944206-21diskstats1
39385932110,5sleep03938612-21apt-get0
39350932111,6sleep23936910-21tr2
39279112110,3sleep00-21swapper/00
39159972111,5sleep13916038-21ntpq1
39124322110,7sleep03912503-21/usr/sbin/munin0
39106872111,5sleep23910733-21ntpq2
39071582111,6sleep13907807-21kthreadcore1
39063502111,5sleep23906369-21munin-run2
38962692111,4sleep23896478-21hddtemp_smartct2
38774282111,5sleep23878283-21memory2
38667192111,5sleep23866988-21ntpq2
362122111,6sleep237220-21tr2
348642110,2sleep034876-21ssh0
1425702111,3sleep00-21swapper/00
945802101,3sleep10-21swapper/11
78782101,3sleep20-21swapper/22
742792101,4sleep00-21swapper/00
684972100,4sleep244-21ksoftirqd/22
587172101,3sleep20-21swapper/22
41935812101,4sleep13991631-21diskmemload1
41754252100,1sleep10-21swapper/11
41685522101,3sleep00-21swapper/00
41504572101,4sleep23991631-21diskmemload2
41424622101,3sleep10-21swapper/11
41334942101,3sleep10-21swapper/11
41167422101,3sleep20-21swapper/22
41072302101,3sleep10-21swapper/11
40993682101,4sleep23991631-21diskmemload2
40830932101,4sleep13991631-21diskmemload1
40518632101,5sleep04052083-21hddtemp_smartct0
40431902101,3sleep00-21swapper/00
40388242101,4sleep04038914-21ntpq0
40067452101,3sleep10-21swapper/11
39980372101,3sleep13991631-21diskmemload1
39373032101,4sleep13937351-21ntpq1
39319962101,4sleep23932046-21ntpq2
39279282101,4sleep13927931-21apt-get1
39273202100,1sleep20-21swapper/22
39213432101,5sleep23921381-21ntpq2
248802101,3sleep13991631-21diskmemload1
238192101,2sleep20-21swapper/22
1451732101,5sleep2145293-21diskstats2
1429542101,4sleep13991631-21diskmemload1
1185482100,1sleep00-21swapper/00
91661291,2sleep00-21swapper/00
84839290,3sleep23991631-21diskmemload2
69653290,6sleep169253-21/usr/sbin/munin1
3970578291,2sleep10-21swapper/11
3948328291,4sleep23948576-21grep2
3944485291,4sleep03945454-21cat0
3921253291,4sleep13921342-21ntpq1
3904704290,1sleep00-21swapper/00
3901754291,4sleep13901870-21irqstats1
3890227291,3sleep23890088-21apt-get2
3885748291,4sleep23887394-21tr2
3885033291,3sleep13885052-21apt-get1
3880272290,5sleep03881244-21cut0
3869552291,4sleep13870905-21cut1
144203291,4sleep23991631-21diskmemload2
125160291,2sleep00-21swapper/00
117827291,3sleep13991631-21diskmemload1
111061291,3sleep13991631-21diskmemload1
3974814280,1sleep00-21swapper/00
3943544280,2sleep272-21kcompactd02
3939363281,3sleep13941033-21grep1
3922900281,3sleep10-21swapper/11
3916888280,1sleep20-21swapper/22
3897881281,3sleep13899695-21cut1
3879777281,3sleep13879990-21df_inode1
3873991280,1sleep00-21swapper/00
3986540270,1sleep10-21swapper/11
3985668270,3sleep23985912-21users2
3979771270,3sleep00-21swapper/00
3970600271,3sleep20-21swapper/22
3970184270,1sleep10-21swapper/11
3970184270,1sleep10-21swapper/11
3958623271,3sleep23958663-21ldconfig2
3953960270,1sleep00-21swapper/00
3917283270,2sleep00-21swapper/00
3896002271,3sleep03896073-21cpuspeed_turbos0
3889924270,1sleep00-21swapper/00
3884681270,1sleep20-21swapper/22
3875689270,3sleep13877426-21kthreadcore1
3861620271,2sleep03861639-21apt-get0
3986289260,3sleep00-21swapper/00
3983112260,3sleep13983314-21kthreadcore1
3981793261,2sleep00-21swapper/00
3979944260,2sleep20-21swapper/22
3965244260,1sleep00-21swapper/00
3965244260,1sleep00-21swapper/00
3964952260,1sleep00-21swapper/00
3964894261,3sleep20-21swapper/22
3891992261,2sleep10-21swapper/11
3874916261,2sleep00-21swapper/00
3865748260,2sleep20-21swapper/22
3958772250,2sleep10-21swapper/11
3949819250,2sleep20-21swapper/22
3933238250,1sleep00-21swapper/00
3926957251,1sleep00-21swapper/00
3910514250,2sleep00-21swapper/00
3862457250,2sleep10-21swapper/11
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional