You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-25 - 10:18

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack9slot4s (updated Tue Nov 25, 2025 00:59:04)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
289199912551198,42cyclictest29097-21aten_r4power_en19:25:120
289209911781169,6cyclictest0-21swapper/119:45:291
289209911461112,30cyclictest29063-21/usr/sbin/munin19:25:111
271532654607,31sleep00-21swapper/019:20:110
285692506460,31sleep10-21swapper/119:22:001
2891999127102,22cyclictest0-21swapper/019:45:290
37302750,7sleep00-21swapper/021:41:540
150122630,7sleep00-21swapper/000:40:540
29652500,8sleep02891999cyclictest21:05:130
2892099491,10cyclictest22-21ksoftirqd/123:52:031
246292470,7sleep12892099cyclictest23:10:281
2892099463,11cyclictest0-21swapper/122:00:181
2892099461,9cyclictest0-21swapper/100:35:501
2892099452,9cyclictest0-21swapper/100:53:321
2892099441,10cyclictest0-21swapper/123:25:351
2892099436,6cyclictest7039-21ssh00:00:121
2892099432,9cyclictest0-21swapper/123:31:171
28920994324,9cyclictest22-21ksoftirqd/123:04:061
2892099431,9cyclictest0-21swapper/123:15:201
28920994317,5cyclictest22-21ksoftirqd/121:16:121
2891999431,12cyclictest81rcu_preempt22:45:210
26082430,8sleep12892099cyclictest23:21:101
26082430,8sleep12892099cyclictest23:21:091
2892099428,5cyclictest12131-21smartctl21:50:271
2892099427,5cyclictest27457-21mailstats00:20:301
2892099427,5cyclictest21288-21ssh22:34:201
2892099424,11cyclictest0-21swapper/100:26:581
2892099423,10cyclictest0-21swapper/122:12:151
2892099422,18cyclictest0-21swapper/100:09:041
2892099421,9cyclictest0-21swapper/123:55:261
2892099421,9cyclictest0-21swapper/123:55:261
2892099421,9cyclictest0-21swapper/120:35:301
2892099421,13cyclictest22-21ksoftirqd/121:58:241
2892099417,5cyclictest20310-21taskset23:05:491
2892099411,9cyclictest0-21swapper/122:05:171
2892099411,9cyclictest0-21swapper/121:30:201
2892099411,8cyclictest0-21swapper/123:42:201
28920994117,5cyclictest22-21ksoftirqd/122:28:391
28920994116,6cyclictest22-21ksoftirqd/122:52:021
28920994116,5cyclictest22-21ksoftirqd/123:47:491
28920994116,5cyclictest22-21ksoftirqd/122:42:351
28920994116,5cyclictest22-21ksoftirqd/122:15:581
28920994116,5cyclictest22-21ksoftirqd/100:41:531
28920994114,17cyclictest22-21ksoftirqd/121:00:281
28920994114,17cyclictest22-21ksoftirqd/121:00:281
28920994113,3cyclictest22-21ksoftirqd/122:36:411
2892099405,5cyclictest21918-21ssh00:49:191
2892099403,12cyclictest0-21swapper/100:30:261
2892099402,9cyclictest0-21swapper/121:27:301
2892099401,8cyclictest0-21swapper/123:38:001
2892099401,8cyclictest0-21swapper/100:10:521
28920994016,5cyclictest22-21ksoftirqd/122:57:371
28920994016,5cyclictest22-21ksoftirqd/122:45:051
28920994016,5cyclictest22-21ksoftirqd/100:17:571
28920994011,4cyclictest81rcu_preempt21:10:291
2892099397,6cyclictest22-21ksoftirqd/121:48:351
2892099397,6cyclictest22-21ksoftirqd/120:54:081
2892099397,5cyclictest1397-21fschecks_time19:35:181
2892099395,5cyclictest22-21ksoftirqd/119:56:481
2892099392,9cyclictest0-21swapper/121:41:421
2892099392,8cyclictest0-21swapper/121:08:101
28920993915,5cyclictest22-21ksoftirqd/121:22:291
28920993912,4cyclictest22-21ksoftirqd/120:34:151
2891999398,15cyclictest0-21swapper/023:31:380
2891999396,4cyclictest81rcu_preempt22:05:240
268722391,4sleep026882-21ssh23:13:240
263332390,4sleep00-21swapper/000:54:040
2892099388,6cyclictest13448-21smartctl20:05:271
28920993814,7cyclictest22-21ksoftirqd/122:20:141
28920993814,5cyclictest22-21ksoftirqd/121:39:211
2892099376,6cyclictest22-21ksoftirqd/120:12:011
2892099376,5cyclictest32085-21ntpq20:55:251
2892099373,10cyclictest0-21swapper/120:21:211
28920993713,3cyclictest22-21ksoftirqd/120:19:311
28920993712,6cyclictest22-21ksoftirqd/119:50:031
28920993710,4cyclictest22-21ksoftirqd/120:03:261
28919993723,5cyclictest3-21ksoftirqd/023:19:580
2891999371,17cyclictest0-21swapper/000:47:370
2891999371,13cyclictest11899-21ssh21:50:230
168052370,4sleep03-21ksoftirqd/023:02:440
2892099366,6cyclictest22-21ksoftirqd/120:29:011
28920993614,4cyclictest22-21ksoftirqd/120:45:151
28920993614,4cyclictest22-21ksoftirqd/119:40:201
2891999367,5cyclictest81rcu_preempt20:35:290
2891999366,5cyclictest81rcu_preempt23:20:170
2891999366,5cyclictest81rcu_preempt23:20:170
2891999362,22cyclictest31335-21grep22:10:270
2891999361,20cyclictest29207-21cron20:50:000
2891999361,13cyclictest0-21swapper/020:58:530
2892099357,6cyclictest22-21ksoftirqd/120:41:461
2891999357,4cyclictest81rcu_preempt23:44:130
2891999355,4cyclictest81rcu_preempt20:25:290
2891999355,4cyclictest81rcu_preempt00:23:340
2891999354,4cyclictest81rcu_preempt21:36:460
2891999354,15cyclictest24356-21diskstats21:30:160
2891999351,16cyclictest131032wc00:05:300
2891999351,10cyclictest81rcu_preempt19:56:250
222142350,5sleep02891999cyclictest23:08:360
184692351,30sleep018480-21ssh21:57:190
28920993411,5cyclictest22-21ksoftirqd/119:31:231
2891999347,4cyclictest81rcu_preempt21:28:210
2891999346,4cyclictest81rcu_preempt21:24:350
2891999341,8cyclictest81rcu_preempt22:57:340
2891999341,16cyclictest0-21swapper/022:15:430
234042340,3sleep0111rcuc/022:02:270
2891999337,4cyclictest81rcu_preempt23:57:300
2891999337,4cyclictest81rcu_preempt23:57:300
2891999335,4cyclictest81rcu_preempt23:50:140
2891999335,4cyclictest81rcu_preempt23:35:310
2891999335,4cyclictest81rcu_preempt22:40:140
2891999335,4cyclictest81rcu_preempt21:13:290
2891999334,4cyclictest81rcu_preempt22:27:080
2891999333,4cyclictest81rcu_preempt21:15:230
2891999332,4cyclictest81rcu_preempt00:12:560
2891999331,4cyclictest81rcu_preempt22:53:290
2891999331,4cyclictest81rcu_preempt21:45:020
2891999331,14cyclictest26333-21ssh22:39:390
2891999331,12cyclictest81rcu_preempt23:50:010
90902321,5sleep09102-21rm23:28:290
2891999327,13cyclictest9398-21ntp_states00:35:260
2891999326,5cyclictest81rcu_preempt21:02:260
2891999326,5cyclictest81rcu_preempt21:02:250
2891999325,4cyclictest81rcu_preempt22:30:570
2891999325,4cyclictest81rcu_preempt00:25:260
2891999323,13cyclictest31090-21cron20:55:000
2891999321,4cyclictest81rcu_preempt19:51:570
256422321,4sleep025656-21ssh00:19:280
2891999318,4cyclictest81rcu_preempt00:00:050
2891999315,4cyclictest81rcu_preempt00:31:200
2891999314,4cyclictest81rcu_preempt22:21:140
2891999310,4cyclictest81rcu_preempt19:35:440
2891999304,3cyclictest81rcu_preempt20:22:330
2891999301,4cyclictest81rcu_preempt20:03:310
2891999296,11cyclictest14404-21aten_r4power_vo20:10:130
2891999295,12cyclictest3318-21entropy19:40:170
2891999291,15cyclictest27319-21runrttasks20:44:580
2891999281,15cyclictest16264-21wget20:15:130
2891999275,11cyclictest13167-21needreboot20:05:210
2891999271,17cyclictest0-21swapper/019:32:140
2891999271,15cyclictest22213-21fschecks_count20:30:160
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional