You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-01 - 00:20

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 100 highest latencies:
System rack3slot3s.osadl.org (updated Mon Jun 30, 2025 12:43:56)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
339533221750,2sleep00-21swapper/011:23:080
30729202146101,15sleep20-21swapper/207:08:072
30728282146100,15sleep30-21swapper/307:06:563
30728412144101,15sleep10-21swapper/107:07:051
333620521330,3sleep1321rcuc/110:45:011
30727762132108,17sleep00-21swapper/007:06:160
331833321240,3sleep134-21ksoftirqd/110:30:261
319509621200,2sleep00-21swapper/009:05:270
351504321140,2sleep00-21swapper/012:33:070
34742322440,2sleep20-21swapper/212:10:172
3073213994331,8cyclictest3417823-21grep11:35:293
3073213993227,3cyclictest0-21swapper/310:10:193
3073213993227,2cyclictest0-21swapper/310:05:133
33279532310,3sleep20-21swapper/210:38:362
3073213993126,3cyclictest0-21swapper/312:05:183
3073213992924,3cyclictest0-21swapper/310:40:223
3073213992923,4cyclictest3491551-21gltestperf12:20:203
30804502280,2sleep243-21ksoftirqd/207:15:152
3073213992723,2cyclictest0-21swapper/309:20:323
3073213992722,3cyclictest0-21swapper/309:55:253
3073213992718,6cyclictest3314518-21tr10:30:163
3073213992623,2cyclictest0-21swapper/308:40:193
3073213992622,2cyclictest0-21swapper/312:15:153
3073213992622,2cyclictest0-21swapper/308:00:303
3073213992617,6cyclictest3240210-21dump-pmu-power09:40:013
33584122250,2sleep1321rcuc/110:59:591
3073213992415,6cyclictest3302090-21cut10:20:203
33962832230,3sleep10-21swapper/111:24:371
3073213992319,3cyclictest3078128-21latency_hist07:15:003
3073213992317,2cyclictest3196722-21gltestperf09:10:173
3073213992312,8cyclictest3222529-21ntpq09:25:243
32314722221,9sleep03231487-21ssh09:33:210
3073213992219,2cyclictest3478394-21unixbench_multi12:10:293
307321399221,8cyclictest3460493-21ssh12:00:213
3073213992218,2cyclictest3145684-21sendmail08:20:003
3073213992218,2cyclictest0-21swapper/310:45:163
307321399221,7cyclictest3404901-21cut11:30:133
3073213992213,7cyclictest0-21swapper/311:15:133
3073213992212,8cyclictest0-21swapper/310:53:183
3073213992212,7cyclictest3242694-21grep09:40:223
307321399215,13cyclictest3446190-21sh11:53:153
307321399213,14cyclictest3228986-21kthreadcore09:30:243
3073213992117,2cyclictest0-21swapper/307:55:263
3073213992116,3cyclictest717-21systemd-logind07:35:003
3073213992116,3cyclictest3516628-21apt-key12:35:013
3073213992112,6cyclictest3322516-21cut10:35:193
307321399211,18cyclictest3516547-21apt-get12:35:023
307321399210,19cyclictest3365177-21sh11:03:343
307321399209,7cyclictest3433794-21cut11:45:213
307321399207,9cyclictest3374418-21perl11:10:183
307321399207,9cyclictest3374418-21perl11:10:173
3073213992016,2cyclictest0-21swapper/308:50:233
3073213992015,3cyclictest3262252-21dump-pmu-power09:54:593
3073213992014,4cyclictest3294972-21kthreadcore10:15:223
3073213992013,5cyclictest3310385-21cut10:25:233
3073213992012,5cyclictest3195981-21diskmemload11:56:343
307321399201,15cyclictest3136341-21irqstats08:10:153
307321399200,17cyclictest3394118-21runrttasks11:20:373
307321399200,17cyclictest3275943-21sh10:02:453
307321399199,6cyclictest733-21gdbus09:05:273
3073213991915,2cyclictest3135450-21apt-key08:10:003
3073213991915,2cyclictest0-21swapper/311:44:203
3073213991915,2cyclictest0-21swapper/311:25:253
3073213991914,3cyclictest0-21swapper/307:25:003
307321399191,16cyclictest3247767-21tr09:45:173
3073213991911,6cyclictest0-21swapper/307:40:153
307321399191,15cyclictest3189799-21runrttasks09:00:213
307321399191,15cyclictest3185917-21awk09:00:003
3073213991910,8cyclictest3366603-21cpuspeed_turbos11:05:143
3073213991910,6cyclictest3504430-21ssh12:25:353
3073213991910,6cyclictest3195981-21diskmemload09:18:413
3073213991910,6cyclictest262-21systemd-journal07:50:203
34030492181,2sleep10-21swapper/111:27:541
33507642181,3sleep10-21swapper/110:54:401
32815402181,7sleep23281790-21sed10:05:262
32476202181,4sleep03247644-21apt-get09:45:150
307321399189,7cyclictest0-21swapper/310:58:143
307321399189,6cyclictest3174628-21memory08:45:193
307321399181,6cyclictest3083876-21cat07:15:193
3073213991815,2cyclictest3100768-21gltestperf07:35:143
3073213991814,2cyclictest709-21polkitd08:35:263
3073213991814,2cyclictest0-21swapper/308:20:243
3073213991814,2cyclictest0-21swapper/307:29:593
32543112171,3sleep10-21swapper/109:49:181
307321399177,6cyclictest733-21gdbus08:25:253
3073213991710,5cyclictest3156178-21cut08:30:153
34863842161,10sleep13486448-21gcc12:15:211
33635812160,4sleep03195981-21diskmemload11:00:560
307321399160,3cyclictest3110551-21/usr/sbin/munin07:45:183
35248162151,10sleep03524830-21ssh12:39:140
34546432151,8sleep23195981-21diskmemload11:58:062
34376902151,8sleep03195981-21diskmemload11:48:220
33570862151,2sleep03195981-21diskmemload10:57:310
35165972141,3sleep13517061-21ssh12:35:011
34708172141,9sleep10-21swapper/112:06:431
34379622141,8sleep13195981-21diskmemload11:48:431
34211382141,3sleep20-21swapper/211:39:422
33057272141,7sleep03195981-21diskmemload10:23:370
32388502141,3sleep13238856-21ssh09:37:271
34983532131,8sleep13498364-21ssh12:23:471
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional