You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-10-19 - 08:41

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack9slot4s (updated Sun Oct 19, 2025 00:59:03)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
168349912141205,6cyclictest0-21swapper/119:25:051
168339910941064,27cyclictest21358-21kworker/0:219:25:040
164442724681,29sleep00-21swapper/019:21:360
165472466418,33sleep10-21swapper/119:22:421
1683399180163,12cyclictest0-21swapper/019:35:010
1559621300,28sleep00-21swapper/021:33:430
133482970,8sleep10-21swapper/100:52:361
88892860,5sleep01683399cyclictest00:14:410
239612680,2sleep10-21swapper/121:05:311
152902520,2sleep115291-21/usr/sbin/munin00:20:301
140162520,3sleep1201rcuc/122:05:191
271332490,3sleep1201rcuc/121:45:231
273382481,4sleep124665-21diskmemload21:11:521
23482471,29sleep02425-21sed23:00:200
299822460,4sleep11683499cyclictest22:55:221
16833994412,20cyclictest0-21swapper/022:03:170
16833994123,7cyclictest30636-21pidof20:00:000
16833994113,16cyclictest3-21ksoftirqd/000:36:390
16834994010,13cyclictest0-21swapper/123:27:101
16833994024,5cyclictest21354-21sendmail-msp21:40:000
16833994016,8cyclictest3-21ksoftirqd/000:47:440
1683499392,16cyclictest22304-21iostat21:40:221
16833993923,5cyclictest19319-21grep22:10:290
16833993923,11cyclictest3872-21ssh23:35:240
16833993922,6cyclictest7141-21hddtemp_smartct23:05:180
16833993921,7cyclictest6475-21timerandwakeup22:30:320
16833993916,8cyclictest3-21ksoftirqd/000:04:320
16833993916,7cyclictest3-21ksoftirqd/020:45:350
297232381,33sleep16204-21gdbus22:21:051
16833993823,5cyclictest20695-21runrttasks20:58:040
16833993821,5cyclictest13722-21diskstats22:05:170
16833993815,7cyclictest3-21ksoftirqd/020:26:360
216452371,3sleep121069-21/usr/sbin/munin21:00:221
16833993723,9cyclictest23578-21hddtemp_smartct22:15:180
16833993722,5cyclictest13530-21irqstats23:45:280
16833993722,10cyclictest20752-21munin-run23:19:590
16833993721,5cyclictest3020-21runrttasks00:24:090
16833993721,5cyclictest22939-21fschecks_time23:55:180
16833993720,6cyclictest8985-21processes23:40:250
16833993716,8cyclictest17789-21hddtemp_smartct20:50:220
16833993716,7cyclictest3-21ksoftirqd/021:22:410
16833993712,9cyclictest3352-21diskmemload00:08:380
16833993711,16cyclictest26367-21ssh22:51:090
58162361,3sleep10-21swapper/123:37:191
284972361,31sleep028547-21iostat22:20:210
1683499366,14cyclictest40950irq/93-i91522:32:411
1683499366,13cyclictest40950irq/93-i91519:58:241
1683499363,5cyclictest0-21swapper/123:23:451
16833993621,5cyclictest30116-21df21:15:170
16833993620,6cyclictest11313-21sort22:35:330
16833993620,5cyclictest9757-21/usr/sbin/munin20:30:280
16833993620,5cyclictest21734-21iostat23:20:220
16833993618,5cyclictest15241-21cron20:45:010
16833993613,8cyclictest3-21ksoftirqd/021:47:210
92532351,29sleep16203-21packagekitd23:07:171
38002351,4sleep13919-21ls21:55:131
1683499357,24cyclictest0-21swapper/121:52:361
1683499355,14cyclictest0-21swapper/122:36:481
16834993516,15cyclictest22-21ksoftirqd/123:14:481
16834993516,15cyclictest22-21ksoftirqd/100:14:521
16834993510,21cyclictest24928-21processes00:30:281
16833993521,4cyclictest2220-21dbus-daemon19:51:140
16833993520,5cyclictest1658-21unixbench_singl22:25:310
16833993516,7cyclictest6593-21ssh21:57:340
16833993514,5cyclictest23024-21aten_r4power_po21:05:140
16833993513,7cyclictest3-21ksoftirqd/020:38:150
16833993513,11cyclictest0-21swapper/021:41:480
16833993512,8cyclictest3-21ksoftirqd/021:53:550
1683399351,16cyclictest18095-21fschecks_time23:50:170
129972350,3sleep0111rcuc/023:10:440
1683499341,15cyclictest14559-21ssh23:45:591
1683399343,8cyclictest11211-21iostat_ios00:50:190
16833993420,4cyclictest25259-21df21:10:150
16833993420,10cyclictest3782-21taskset20:13:270
16833993420,10cyclictest17815-21sh22:42:430
16833993419,4cyclictest31385-21iostat_ios23:30:240
16833993417,5cyclictest29617-21df_inode22:55:160
16833993416,7cyclictest0-21swapper/019:45:300
16833993416,6cyclictest0-21swapper/019:36:540
16833993413,5cyclictest2086-21ntp_states00:40:270
100692341,4sleep010079-21ssh21:27:420
100692341,4sleep010079-21ssh21:27:420
16834993315,15cyclictest22-21ksoftirqd/100:15:091
1683499331,15cyclictest24965-21ssh22:16:121
16833993315,6cyclictest4179-21aten_r4power_po20:15:140
16833993313,6cyclictest3-21ksoftirqd/023:28:060
16833993312,7cyclictest3-21ksoftirqd/020:23:070
1683399331,14cyclictest19243-21sort00:25:130
16833993310,8cyclictest3-21ksoftirqd/021:02:300
16833993310,7cyclictest24184-21df00:30:150
293982321,27sleep129409-21irqstats00:35:241
16834993210,5cyclictest40950irq/93-i91519:40:181
16833993213,7cyclictest10995-21rm00:15:440
16833993213,6cyclictest3-21ksoftirqd/020:00:590
16833993211,6cyclictest19950-21ssh22:45:140
122262321,27sleep112230-21irqstats20:35:211
33732312,3sleep10-21swapper/123:00:441
206312311,6sleep10-21swapper/100:25:411
16834993119,8cyclictest16367-21ssh22:40:381
16834993114,13cyclictest4974-21ssh00:43:541
16834993113,14cyclictest22-21ksoftirqd/123:42:411
16834993113,14cyclictest17520-21cat23:15:301
1683499311,26cyclictest0-21swapper/122:28:151
12652311,4sleep11275-21ssh23:32:521
194112301,5sleep10-21swapper/123:51:021
1683499307,19cyclictest0-21swapper/100:45:571
1683499302,7cyclictest24786-21df22:50:151
1683499301,9cyclictest2326-21tr20:10:131
16834993017,9cyclictest22-21ksoftirqd/121:26:201
16834993017,9cyclictest22-21ksoftirqd/121:26:191
16834993016,5cyclictest22-21ksoftirqd/122:04:431
16834993016,10cyclictest22-21ksoftirqd/100:08:531
16834993015,11cyclictest22094-21ssh22:46:571
1683399302,23cyclictest23761-21perl19:40:170
16833993013,6cyclictest319-21wget20:05:130
167242301,5sleep10-21swapper/120:46:001
324722291,5sleep10-21swapper/120:04:341
1683499299,16cyclictest0-21swapper/100:03:191
1683499296,19cyclictest0-21swapper/121:20:191
1683499294,20cyclictest0-21swapper/121:35:051
1683499294,20cyclictest0-21swapper/121:33:051
16834992917,9cyclictest22-21ksoftirqd/121:17:201
16834992916,9cyclictest22-21ksoftirqd/122:13:261
285092281,5sleep10-21swapper/119:52:391
265232281,5sleep10-21swapper/119:46:301
229312281,5sleep10-21swapper/119:38:221
18882281,5sleep10-21swapper/120:09:261
16834992817,8cyclictest22-21ksoftirqd/123:58:261
114082281,5sleep10-21swapper/120:34:261
74712271,5sleep10-21swapper/120:22:321
211992271,5sleep10-21swapper/119:34:551
204972271,5sleep10-21swapper/120:56:051
148482271,5sleep10-21swapper/120:41:001
16834992616,6cyclictest19013-21latency_hist20:55:001
16834992610,12cyclictest0-21swapper/120:25:321
1683499243,6cyclictest0-21swapper/120:19:101
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional