You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-05-05 - 01:41

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack9slot4s (updated Sun May 04, 2025 12:59:00)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
296069912211202,16cyclictest6951-21copy07:45:481
296069911301121,6cyclictest0-21swapper/107:25:121
291572659611,32sleep10-21swapper/107:20:511
291412553508,30sleep00-21swapper/007:20:400
2960599150141,6cyclictest0-21swapper/007:45:480
29605999059,17cyclictest0-21swapper/007:25:120
320692870,9sleep02960599cyclictest11:45:220
29605995914,7cyclictest16442-21hddtemp_smartct12:30:180
316092580,4sleep02960599cyclictest10:20:000
2960599583,16cyclictest24874-21python11:25:260
2960599531,13cyclictest27336-21ntpq08:40:240
205662430,4sleep02960599cyclictest12:40:280
2960699374,4cyclictest81rcu_preempt09:40:521
2960699346,4cyclictest81rcu_preempt12:22:371
66232331,4sleep16724-21users09:10:341
2960699336,4cyclictest81rcu_preempt12:37:581
2960599334,5cyclictest14289-21cron11:00:000
2960599334,26cyclictest3-21ksoftirqd/008:47:360
2960599331,20cyclictest19747-21missed_timers08:20:420
22872331,28sleep11594-21/usr/sbin/munin09:00:231
306562321,27sleep030767-21users11:40:310
2960699325,5cyclictest81rcu_preempt08:27:281
2960699324,4cyclictest81rcu_preempt11:50:581
2960699324,4cyclictest81rcu_preempt08:58:161
2960699324,4cyclictest81rcu_preempt07:43:311
2960699324,11cyclictest3484-21diskstats10:30:151
2960699321,16cyclictest16938-21taskset08:14:581
112512321,4sleep111301-21iostat10:50:211
2960699316,4cyclictest81rcu_preempt12:50:431
2960699315,4cyclictest81rcu_preempt11:02:541
2960699314,4cyclictest81rcu_preempt11:10:341
2960699314,4cyclictest81rcu_preempt10:59:551
2960699313,4cyclictest81rcu_preempt10:09:561
2960699311,17cyclictest13817-21fschecks_time08:05:161
2960599312,24cyclictest16950-21iostat_ios11:05:230
2960599312,17cyclictest0-21swapper/008:18:000
2960599311,4cyclictest81rcu_preempt11:57:120
29605993112,7cyclictest0-21swapper/012:01:500
2960599311,19cyclictest3165-21taskset09:02:270
29605993111,8cyclictest0-21swapper/012:25:180
132582311,5sleep00-21swapper/009:30:160
97362301,5sleep00-21swapper/012:10:530
87362301,25sleep18806-21expr07:50:311
2960699308,7cyclictest0-21swapper/108:37:211
2960699306,4cyclictest81rcu_preempt09:15:381
2960699305,4cyclictest81rcu_preempt12:26:211
2960699305,4cyclictest81rcu_preempt11:18:241
2960699304,4cyclictest81rcu_preempt11:35:101
2960699304,4cyclictest81rcu_preempt08:21:121
2960699301,8cyclictest28646-21latency10:10:221
29606993016,11cyclictest22-21ksoftirqd/111:20:171
2960699301,4cyclictest81rcu_preempt10:29:401
29606993013,4cyclictest22-21ksoftirqd/111:30:121
2960599305,10cyclictest0-21swapper/008:53:080
2960599303,15cyclictest0-21swapper/010:14:080
2960599301,4cyclictest81rcu_preempt08:56:210
2960599301,4cyclictest81rcu_preempt08:32:460
29605993012,6cyclictest23822-21fschecks_count12:50:240
2960599301,26cyclictest12009-21cron12:20:000
2960599301,25cyclictest34652sleep007:38:560
29605993012,5cyclictest0-21swapper/009:55:300
29605993011,6cyclictest12317-21ntp_states08:00:260
2960599301,15cyclictest11913-21grep10:50:300
29605993010,7cyclictest27061-21runrttasks11:31:010
25072301,5sleep00-21swapper/011:54:420
119182305,3sleep111209-21kworker/1:312:19:021
105682301,5sleep00-21swapper/009:21:110
104732301,5sleep00-21swapper/010:49:190
314452291,5sleep10-21swapper/110:18:261
2960699295,4cyclictest81rcu_preempt11:09:141
2960699295,4cyclictest81rcu_preempt09:37:451
2960699295,4cyclictest81rcu_preempt08:50:501
2960699294,4cyclictest81rcu_preempt12:14:241
2960699294,4cyclictest81rcu_preempt11:57:271
2960699294,4cyclictest81rcu_preempt11:40:361
2960699293,4cyclictest81rcu_preempt09:45:591
2960699292,4cyclictest81rcu_preempt12:06:561
29606992919,6cyclictest10104-21/usr/sbin/munin07:55:191
2960699291,7cyclictest22373-21/usr/sbin/munin09:55:301
2960699291,4cyclictest81rcu_preempt11:46:291
2960699291,4cyclictest81rcu_preempt09:32:161
2960699290,4cyclictest81rcu_preempt09:54:551
2960599299,5cyclictest994-21/usr/sbin/munin07:30:310
2960599296,12cyclictest18386-21cpuspeed_turbos11:10:130
2960599292,14cyclictest8442-21ls07:50:240
2960599292,14cyclictest22282-21df_inode11:20:150
2960599292,14cyclictest15727-21/usr/sbin/munin08:10:180
2960599292,12cyclictest0-21swapper/010:00:310
2960599291,4cyclictest81rcu_preempt07:44:360
29605992912,5cyclictest10417-21/usr/sbin/munin07:55:260
2960599291,17cyclictest3485-21diskstats10:30:150
2960599291,15cyclictest7369-21ntpdc12:05:250
29605992910,6cyclictest22260-21latency12:45:220
2960599290,3cyclictest81rcu_preempt12:40:010
258272291,4sleep10-21swapper/110:03:361
128992291,5sleep10-21swapper/108:01:571
2960699285,4cyclictest81rcu_preempt12:01:461
2960699285,4cyclictest81rcu_preempt10:41:581
2960699284,4cyclictest81rcu_preempt08:17:271
2960699283,15cyclictest0-21swapper/112:41:371
2960699281,5cyclictest81rcu_preempt09:05:131
2960599289,6cyclictest21481-21iostat_ios08:25:210
2960599283,13cyclictest0-21swapper/011:00:220
2960599282,4cyclictest81rcu_preempt10:09:410
2960599282,14cyclictest20383-21/usr/sbin/munin11:15:140
2960599282,13cyclictest6854-21runrttasks09:11:450
2960599282,13cyclictest25451-21grep08:35:230
2960599281,3cyclictest81rcu_preempt09:52:410
29605992811,5cyclictest4109-21/usr/sbin/munin09:05:190
29605992811,5cyclictest2220-21dbus-daemon10:21:180
2960599281,14cyclictest5360-21diskstats10:35:170
2960599281,14cyclictest13077-21proc_pri12:20:270
125292281,5sleep10-21swapper/109:27:061
12352281,5sleep10-21swapper/107:31:521
63842271,5sleep10-21swapper/110:35:581
35442271,5sleep10-21swapper/107:39:471
2960699273,9cyclictest0-21swapper/110:45:291
29606992718,5cyclictest17651-21cron12:35:011
29606992717,6cyclictest22-21ksoftirqd/108:31:571
2960699271,14cyclictest0-21swapper/108:40:121
2960599278,15cyclictest11194-21aten_r4power_vo09:25:130
29605992711,5cyclictest0-21swapper/010:27:560
2960599271,14cyclictest9120-21latency_hist09:20:010
2960599271,13cyclictest28719-21timerandwakeup11:35:280
2960599271,11cyclictest17707-21awk09:40:330
29605992710,5cyclictest19465-21open_files09:45:270
2960699268,14cyclictest10122chrt10:24:511
29606992617,6cyclictest22-21ksoftirqd/111:27:371
2960699261,16cyclictest0-21swapper/109:20:331
2960599269,6cyclictest13816-21grep08:05:170
2960599262,12cyclictest15128-21df09:35:140
29605992611,4cyclictest7013-21taskset10:40:280
29606992517,5cyclictest22-21ksoftirqd/108:48:471
2960699248,4cyclictest22212-21iostat12:45:201
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional