You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-18 - 00:50

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack9slot4s (updated Mon Nov 17, 2025 12:59:04)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
104609912371207,20cyclictest0-21swapper/007:25:070
104619911111102,6cyclictest0-21swapper/107:25:091
104619911061096,7cyclictest0-21swapper/107:35:021
103082591510,65sleep00-21swapper/007:24:010
86872563517,30sleep10-21swapper/107:20:091
75492960,4sleep07546-21aten_r4power_vo08:40:160
313032890,10sleep01046099cyclictest09:22:230
158702830,3sleep041ktimersoftd/010:46:520
10460996759,5cyclictest3-21ksoftirqd/007:35:010
214722660,8sleep01046099cyclictest12:33:520
38692490,3sleep03871-21aten_r4power_en10:35:130
104242420,5sleep0111rcuc/010:08:010
1046199411,17cyclictest22886-21diskstats12:35:191
183572400,3sleep00-21swapper/007:40:320
12162390,4sleep0111rcuc/011:05:250
1046099386,4cyclictest81rcu_preempt09:10:060
316782371,32sleep10-21swapper/112:44:491
316782371,32sleep10-21swapper/112:44:481
277712371,4sleep127783-21ssh09:52:161
10460993713,5cyclictest3260-21users11:40:320
78532361,4sleep17867-21ssh09:31:201
215002361,32sleep121510-21ssh12:33:571
138322361,4sleep013860-21ntp_states12:25:270
1046199365,5cyclictest81rcu_preempt10:18:471
1046199361,8cyclictest1113-21tr09:25:131
1046099367,13cyclictest0-21swapper/009:53:540
290752351,5sleep00-21swapper/010:27:060
227192351,29sleep118594-21diskmemload09:46:551
1046199359,13cyclictest0-21swapper/110:24:321
1046199353,5cyclictest81rcu_preempt10:45:131
1046199352,5cyclictest0-21swapper/109:00:291
1046199352,21cyclictest0-21swapper/111:34:161
10461993517,6cyclictest22-21ksoftirqd/112:10:211
10461993517,6cyclictest22-21ksoftirqd/109:35:161
1046099359,13cyclictest1930-21ssh12:13:410
1046099355,5cyclictest81rcu_preempt11:35:210
1046099352,5cyclictest81rcu_preempt11:54:290
10460993510,12cyclictest30659-21entropy08:15:200
48922340,9sleep01046099cyclictest08:30:320
244202341,28sleep124571-21awk10:55:331
231952341,29sleep023236-21iostat12:35:250
125912341,29sleep012605-21ssh11:17:070
1046199342,7cyclictest11217-21/usr/sbin/munin08:50:281
1046099341,20cyclictest2969-21awk08:25:330
90322331,28sleep19046-21ssh11:46:541
319782331,4sleep032022-21iostat10:30:200
302942332,10sleep03-21ksoftirqd/012:09:460
155582331,4sleep115568-21irqstats11:20:221
1046199335,4cyclictest81rcu_preempt09:06:381
1046199332,18cyclictest0-21swapper/110:12:431
1046199332,17cyclictest15944-21ssh12:27:501
1046099334,7cyclictest81rcu_preempt12:45:350
1046099332,4cyclictest81rcu_preempt10:21:200
10460993322,7cyclictest27805-21fschecks_time12:40:180
10460993322,7cyclictest27805-21fschecks_time12:40:170
10460993315,4cyclictest12422-21grep08:50:300
1046099331,4cyclictest81rcu_preempt11:34:570
10460993313,6cyclictest3-21ksoftirqd/012:15:460
1046099331,20cyclictest25456-21rm09:15:550
10460993311,8cyclictest23612-21ssh10:55:190
1046099331,10cyclictest11134-21ssh12:22:510
4522321,27sleep1463-21sh10:30:411
1046199321,8cyclictest6240-21ssh10:36:541
1046199321,7cyclictest3020-21runrttasks09:21:381
1046199321,6cyclictest81rcu_preempt11:15:331
10461993210,5cyclictest22-21ksoftirqd/112:09:331
1046099326,5cyclictest81rcu_preempt09:41:390
1046099326,5cyclictest81rcu_preempt09:41:390
1046099325,4cyclictest81rcu_preempt11:45:390
1046099324,4cyclictest81rcu_preempt11:22:180
1046099323,4cyclictest81rcu_preempt10:40:180
10460993213,6cyclictest21456-21sendmail10:20:000
1046099321,14cyclictest3561-21gzip10:00:320
1046099321,11cyclictest0-21swapper/010:50:210
1046099321,11cyclictest0-21swapper/010:50:210
10460993210,7cyclictest6146-21df09:30:150
48162311,5sleep00-21swapper/009:28:560
42132311,5sleep10-21swapper/110:01:011
244242311,4sleep124431-21irqstats09:15:231
168382311,4sleep116845-21udevd07:38:371
1046199319,18cyclictest0-21swapper/109:41:161
1046199319,18cyclictest0-21swapper/109:41:151
1046199314,4cyclictest81rcu_preempt12:54:591
1046199314,4cyclictest81rcu_preempt08:48:501
10461993116,11cyclictest22-21ksoftirqd/111:58:401
1046099317,4cyclictest81rcu_preempt11:25:510
1046099312,4cyclictest81rcu_preempt11:12:050
10460993116,5cyclictest3-21ksoftirqd/009:47:480
1046099311,4cyclictest81rcu_preempt11:03:090
10460993113,6cyclictest3-21ksoftirqd/009:39:020
10460993111,6cyclictest30981-21/usr/sbin/munin09:55:310
10460993110,6cyclictest17084-21memory11:55:240
26722301,5sleep10-21swapper/111:06:501
1046199306,4cyclictest81rcu_preempt08:16:181
1046199306,20cyclictest0-21swapper/111:35:311
1046199304,5cyclictest81rcu_preempt10:41:351
10461993021,6cyclictest1789-21aten_r4power_po11:40:131
1046199301,4cyclictest81rcu_preempt11:26:581
1046099309,16cyclictest3020-21runrttasks10:11:280
1046099304,4cyclictest81rcu_preempt12:03:010
1046099304,19cyclictest19372-21diskstats07:45:190
1046099302,4cyclictest81rcu_preempt12:52:290
1046099302,15cyclictest27616-21python08:05:300
1046099302,14cyclictest18608-21munin-run09:10:000
1046099302,14cyclictest16554-21kworker/0:307:38:360
1046099301,13cyclictest1052-21awk08:20:330
88272291,5sleep10-21swapper/108:41:291
13352291,5sleep10-21swapper/108:23:101
1046199299,16cyclictest0-21swapper/112:00:201
1046199293,4cyclictest81rcu_preempt08:38:411
1046199293,4cyclictest81rcu_preempt08:33:111
1046199292,14cyclictest624-21ssh09:57:591
1046199291,6cyclictest211ktimersoftd/110:07:091
1046199291,4cyclictest81rcu_preempt08:26:081
1046199291,19cyclictest0-21swapper/111:04:091
10460992912,5cyclictest9625-21df_inode08:45:160
10460992911,5cyclictest21236-21df_inode07:50:150
263712281,5sleep00-21swapper/008:04:230
243602281,5sleep10-21swapper/107:58:121
164252281,5sleep00-21swapper/009:02:050
1046199289,15cyclictest0-21swapper/109:10:571
1046199283,3cyclictest81rcu_preempt11:14:001
10461992817,8cyclictest22-21ksoftirqd/112:15:151
1046199281,6cyclictest4104-21latency_hist12:50:011
10461992814,8cyclictest12633-21munin-run12:25:001
1046099288,6cyclictest6156-21irqstats08:35:250
1046199277,8cyclictest2220-21dbus-daemon07:49:111
1046199273,4cyclictest81rcu_preempt08:14:361
1046199272,21cyclictest0-21swapper/110:50:101
1046199272,21cyclictest0-21swapper/110:50:091
1046199272,11cyclictest27498-21ntp_states10:25:271
1046199271,14cyclictest12838-21grep11:50:311
187292261,5sleep10-21swapper/107:43:131
1046199267,15cyclictest0-21swapper/108:05:291
1046099269,5cyclictest14114-21/usr/sbin/munin08:55:290
1046099261,12cyclictest23050-21cstates07:55:170
1046199254,4cyclictest81rcu_preempt08:04:381
1046199254,3cyclictest81rcu_preempt07:51:371
10461992517,5cyclictest22-21ksoftirqd/108:55:011
1046099251,16cyclictest0-21swapper/008:12:070
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional