You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-06-16 - 20:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack9slot4s (updated Mon Jun 16, 2025 12:59:02)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
169089912811221,46cyclictest17244-21cpuspeed_turbos07:25:130
169099911651142,19cyclictest24846-21gzip07:41:131
169099911601143,13cyclictest17133-21wget07:25:121
168212576508,52sleep10-21swapper/107:24:391
167732564517,32sleep00-21swapper/007:24:080
1690899129105,20cyclictest0-21swapper/007:41:120
199602800,6sleep01690899cyclictest10:20:290
309452560,8sleep01690899cyclictest10:50:210
2992425517,6sleep03-21ksoftirqd/009:20:340
107692530,4sleep00-21swapper/008:30:230
254012440,4sleep0111rcuc/009:10:160
276372431,3sleep027644-21diskstats07:50:160
1690999417,5cyclictest81rcu_preempt09:50:311
1690999388,6cyclictest40950irq/93-i91510:35:361
16908993815,5cyclictest32636-21tune2fs10:55:190
89642371,32sleep19005-21ntp_states08:25:241
71662371,4sleep16307-21/usr/sbin/munin08:20:441
1690899371,18cyclictest25595-21grep12:00:320
1690899371,18cyclictest25595-21grep12:00:310
319462360,3sleep1211ktimersoftd/109:25:321
294872351,30sleep129492-21irqstats09:20:241
284902351,4sleep128662-21perl12:10:211
16908993410,13cyclictest14938-21runrttasks10:07:280
231232331,28sleep123128-21irqstats11:55:241
1690999335,4cyclictest81rcu_preempt12:17:451
16909993316,5cyclictest22-21ksoftirqd/109:11:381
16908993316,6cyclictest3-21ksoftirqd/010:48:300
126322331,28sleep012699-21/usr/sbin/munin10:00:330
1690999326,4cyclictest81rcu_preempt12:43:041
1690999326,4cyclictest81rcu_preempt12:43:031
1690999326,4cyclictest81rcu_preempt11:27:401
1690999325,4cyclictest81rcu_preempt10:20:551
1690999323,26cyclictest15182-21cron10:10:011
1690999321,16cyclictest0-21swapper/109:59:351
1690999321,13cyclictest186112latency_hist11:45:011
1690899328,12cyclictest81rcu_preempt09:55:300
1690899324,13cyclictest6204-21gdbus08:16:100
1690899321,19cyclictest8079-21cstates09:50:160
1690899321,19cyclictest23225-21chrt09:05:070
1690899321,13cyclictest4866-21grep09:40:240
1690899321,10cyclictest81rcu_preempt11:28:550
3682310,8sleep11690999cyclictest08:05:001
29662311,25sleep12993-21ntp_states09:35:271
1690999315,4cyclictest81rcu_preempt12:28:231
1690999312,5cyclictest81rcu_preempt07:39:361
1690999311,5cyclictest81rcu_preempt11:24:101
16909993114,12cyclictest16298-21find10:10:291
1690899318,12cyclictest3529-21seq09:36:170
16908993116,5cyclictest3-21ksoftirqd/011:11:410
16908993113,6cyclictest3-21ksoftirqd/009:25:290
56102301,5sleep10-21swapper/109:43:151
298542301,5sleep10-21swapper/110:45:591
280052301,5sleep00-21swapper/010:41:150
21612301,5sleep00-21swapper/008:09:000
1690999305,4cyclictest81rcu_preempt08:47:401
1690999304,5cyclictest81rcu_preempt07:50:181
1690999304,17cyclictest0-21swapper/112:00:221
1690999304,17cyclictest0-21swapper/112:00:221
1690999304,10cyclictest9631-21iostat_ios12:45:201
1690999301,5cyclictest81rcu_preempt12:31:561
1690999301,18cyclictest0-21swapper/108:30:201
1690899302,14cyclictest3020-21runrttasks11:40:500
1690899301,4cyclictest81rcu_preempt11:17:210
1690899301,4cyclictest81rcu_preempt11:07:220
16908993014,6cyclictest3-21ksoftirqd/012:16:360
1690899301,15cyclictest18982-21awk10:20:010
85862291,5sleep00-21swapper/012:42:230
85862291,5sleep00-21swapper/012:42:230
72552291,5sleep00-21swapper/009:45:470
26872291,5sleep00-21swapper/012:25:350
227832291,5sleep00-21swapper/009:02:010
202482291,5sleep00-21swapper/011:47:350
1690999295,4cyclictest81rcu_preempt10:53:451
1690999295,4cyclictest81rcu_preempt10:16:191
1690999294,4cyclictest81rcu_preempt11:06:091
1690999293,4cyclictest81rcu_preempt07:33:301
16909992918,6cyclictest27605-21find10:40:281
1690999291,4cyclictest81rcu_preempt12:21:571
1690999291,4cyclictest81rcu_preempt11:14:351
16909992914,8cyclictest13720-21iostat_ios11:30:231
1690899299,7cyclictest0-21swapper/008:25:410
1690899292,14cyclictest25975-21grep10:35:360
1690899292,14cyclictest13886-21/usr/sbin/munin11:30:270
1690899292,14cyclictest11987-21/usr/sbin/munin12:50:330
1690899291,4cyclictest81rcu_preempt11:20:360
1690899291,4cyclictest81rcu_preempt10:10:140
134952291,5sleep10-21swapper/108:38:321
104252291,5sleep00-21swapper/012:46:410
306282281,5sleep10-21swapper/107:56:501
276922281,5sleep10-21swapper/112:06:451
269802281,4sleep10-21swapper/107:47:561
230732281,5sleep00-21swapper/007:38:280
225102281,5sleep00-21swapper/010:27:520
21912281,5sleep10-21swapper/108:09:171
190532281,5sleep10-21swapper/108:52:311
17492281,5sleep10-21swapper/109:33:271
1690999285,8cyclictest20293-21ntp_states08:55:281
1690999284,4cyclictest81rcu_preempt10:55:051
1690999284,4cyclictest81rcu_preempt08:41:521
1690999284,18cyclictest21660-21diskstats09:00:211
1690999283,4cyclictest81rcu_preempt10:32:151
1690999283,4cyclictest81rcu_preempt10:32:141
1690999281,4cyclictest81rcu_preempt11:17:551
1690999280,4cyclictest81rcu_preempt09:16:341
1690899289,6cyclictest2768-21/usr/sbin/munin11:00:310
1690899286,12cyclictest3020-21runrttasks07:46:480
1690899282,13cyclictest682-21hddtemp_smartct09:30:220
1690899282,13cyclictest18860-21/usr/sbin/munin08:50:360
1690899282,13cyclictest15827-21ntp_states11:35:270
1690899281,17cyclictest6749-21runrttasks12:37:590
16908992811,6cyclictest3-21ksoftirqd/008:11:090
16908992811,6cyclictest20102-21http_loadtime07:30:180
16908992811,4cyclictest32242-21diskstats12:20:150
1690899281,14cyclictest12950-21python08:35:320
16908992810,5cyclictest22029-21sshd11:51:350
73492271,5sleep10-21swapper/109:46:471
3442271,5sleep00-21swapper/008:04:530
276972271,5sleep00-21swapper/012:06:480
221372271,5sleep10-21swapper/111:52:431
1690999274,17cyclictest21304-21diskstats10:25:171
1690999271,4cyclictest81rcu_preempt11:00:491
1690999271,4cyclictest81rcu_preempt08:18:201
1690999271,3cyclictest81rcu_preempt12:51:071
1690899279,15cyclictest20970-21taskset08:57:570
1690899278,16cyclictest23504-21kernelversion10:30:240
1690899278,16cyclictest23504-21kernelversion10:30:240
1690899276,8cyclictest3645-21df12:30:180
1690899272,13cyclictest16281-21runrttasks08:45:240
16908992711,6cyclictest3-21ksoftirqd/008:21:580
1690899271,13cyclictest27388-21/usr/sbin/munin09:15:190
1690899271,13cyclictest23013-21tune2fs11:55:210
1690899271,12cyclictest6204-21gdbus12:13:590
1690999269,4cyclictest2463-21aten_r4power_cu08:10:141
16909992617,6cyclictest22-21ksoftirqd/110:04:161
1690999261,4cyclictest81rcu_preempt11:47:201
16909992610,5cyclictest40950irq/93-i91509:05:271
16908992612,11cyclictest0-21swapper/007:59:090
148532261,4sleep10-21swapper/111:35:011
1690999253,18cyclictest6472-21unixbench_singl12:35:331
1690899259,5cyclictest3-21ksoftirqd/008:43:400
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional