You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-03-25 - 03:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack9slot4s (updated Tue Mar 25, 2025 00:59:02)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
4779912841232,39cyclictest697-21wget19:25:090
4789912651235,26cyclictest823-21tr19:25:111
4789911381118,16cyclictest5180-21smartctl21:05:291
4072592546,32sleep10-21swapper/119:24:541
325802506452,38sleep00-21swapper/019:21:500
1354721070,8sleep047799cyclictest21:30:250
54952970,6sleep05496-21fschecks_time22:50:120
477999374,15cyclictest0-21swapper/021:05:280
283692620,6sleep00-21swapper/000:00:220
285192540,3sleep1211ktimersoftd/100:00:241
175102470,7sleep117492-21munin-run23:30:001
175102470,7sleep117492-21munin-run23:30:001
308462431,36sleep130913-21aten_r4power_po20:50:101
183602400,2sleep118359-21missed_timers23:30:221
138472380,7sleep10-21swapper/123:15:201
116482381,4sleep011743-21mailstats19:50:300
477993711,14cyclictest17111-21chrt23:26:060
477993711,14cyclictest17111-21chrt23:26:050
478993612,5cyclictest40950irq/93-i91520:15:251
47799361,21cyclictest9167-21diskstats23:00:170
47899351,10cyclictest23927-21/usr/sbin/munin23:50:161
57412341,29sleep15791-21missed_timers22:50:181
47899345,5cyclictest81rcu_preempt20:40:001
47799348,13cyclictest9997-21timerandwakeup19:45:300
327282341,9sleep00-21swapper/020:55:110
54152331,4sleep15420-21diskstats19:35:141
49612331,28sleep15047-21sort19:35:001
47899334,20cyclictest0-21swapper/100:49:571
478993317,12cyclictest22-21ksoftirqd/122:02:051
47799332,19cyclictest27760-21mailstats22:15:280
477993313,6cyclictest8756-21ntp_states21:15:270
300552331,28sleep130114-21latency_hist20:45:011
261572331,4sleep00-21swapper/020:33:420
261572331,4sleep00-21swapper/020:33:410
47899325,4cyclictest81rcu_preempt21:01:361
47899324,4cyclictest81rcu_preempt23:35:491
241162321,5sleep00-21swapper/020:26:570
47899319,5cyclictest10978-21kworker/1:221:15:261
47899317,5cyclictest81rcu_preempt23:10:141
47899317,4cyclictest81rcu_preempt21:31:521
47799317,13cyclictest22249-21/usr/sbin/munin23:40:250
47799315,12cyclictest2220-21dbus-daemon22:03:380
47799311,8cyclictest0-21swapper/021:50:120
47799311,4cyclictest81rcu_preempt21:42:110
477993111,16cyclictest2220-21dbus-daemon22:10:170
44252311,26sleep14432-21diskstats00:30:121
165772311,6sleep00-21swapper/023:22:050
60422301,5sleep00-21swapper/021:10:110
47899307,4cyclictest81rcu_preempt22:58:371
47899306,4cyclictest81rcu_preempt23:58:581
47899306,4cyclictest81rcu_preempt22:20:271
47899305,4cyclictest81rcu_preempt23:42:111
47899305,4cyclictest81rcu_preempt22:27:341
47899305,4cyclictest81rcu_preempt21:43:281
47899305,4cyclictest81rcu_preempt21:12:421
47899305,4cyclictest81rcu_preempt00:40:541
47899305,4cyclictest81rcu_preempt00:30:001
47899304,6cyclictest19544-21fgrep21:50:241
47899304,4cyclictest81rcu_preempt20:07:521
47899303,23cyclictest0-21swapper/100:21:201
478993022,5cyclictest9544-21cat19:45:211
47899301,4cyclictest81rcu_preempt22:40:021
47899301,16cyclictest25171-21irqstats20:30:171
47899301,16cyclictest25171-21irqstats20:30:171
47799307,11cyclictest27948-21sshd20:37:490
47799302,15cyclictest13921-21/usr/sbin/munin19:55:310
47799301,4cyclictest81rcu_preempt20:20:160
477993011,5cyclictest31944-21ls00:15:000
477993010,7cyclictest0-21swapper/021:39:490
38992301,5sleep00-21swapper/022:40:520
246262301,5sleep00-21swapper/022:09:060
19032301,5sleep00-21swapper/022:35:570
19032301,5sleep00-21swapper/022:35:560
109202301,5sleep00-21swapper/023:07:010
98892291,24sleep110031-21grep21:20:131
98072291,5sleep00-21swapper/021:20:020
55862291,5sleep00-21swapper/000:32:150
48182291,5sleep00-21swapper/019:33:320
47899296,4cyclictest81rcu_preempt19:54:231
47899295,4cyclictest81rcu_preempt00:51:341
47899294,15cyclictest0-21swapper/120:23:291
47899293,4cyclictest81rcu_preempt21:36:251
47899293,4cyclictest81rcu_preempt20:02:491
47899291,4cyclictest81rcu_preempt19:41:511
47899291,13cyclictest0-21swapper/122:30:561
47799296,12cyclictest14543-21taskset23:15:410
47799294,4cyclictest81rcu_preempt19:40:260
47799292,15cyclictest20178-21/usr/sbin/munin20:15:140
47799292,14cyclictest0-21swapper/019:39:120
47799291,4cyclictest81rcu_preempt00:38:120
47799291,13cyclictest3877-21sshd21:03:470
47799290,4cyclictest81rcu_preempt22:25:290
297202291,5sleep00-21swapper/020:41:310
271312291,5sleep00-21swapper/023:56:030
255852291,5sleep00-21swapper/023:54:310
217742291,5sleep00-21swapper/021:55:460
209212291,4sleep00-21swapper/023:36:460
182682291,5sleep00-21swapper/021:48:160
177462291,5sleep00-21swapper/020:06:010
161202291,4sleep00-21swapper/020:03:350
47899287,3cyclictest40950irq/93-i91521:55:131
47899285,4cyclictest81rcu_preempt20:12:241
47899284,4cyclictest81rcu_preempt23:01:541
47899284,4cyclictest81rcu_preempt20:55:371
47899284,3cyclictest81rcu_preempt23:49:411
47899283,9cyclictest16412-21users23:20:311
47899283,18cyclictest27041-21df_inode22:15:151
47899282,4cyclictest81rcu_preempt00:11:161
478992821,5cyclictest22-21ksoftirqd/122:10:221
47799289,7cyclictest17818-21cpuspeed_turbos23:30:120
47799282,13cyclictest1751-21cstates00:20:110
477992812,5cyclictest3-21ksoftirqd/023:10:140
47799281,14cyclictest8258-21fschecks_count00:40:160
47799281,14cyclictest32648-21sort20:55:010
47799281,10cyclictest0-21swapper/000:09:160
477992810,6cyclictest7809-21open_files22:55:260
477992810,5cyclictest31444-21diskstats22:30:100
477992810,5cyclictest19755-21sshd20:12:220
47732281,5sleep10-21swapper/122:48:241
299522281,4sleep00-21swapper/022:20:490
246302281,5sleep10-21swapper/122:09:091
118902281,5sleep00-21swapper/000:52:040
117682281,5sleep10-21swapper/121:25:591
10252281,5sleep00-21swapper/000:16:520
47899271,13cyclictest13248-21iostat19:55:201
241302271,5sleep10-21swapper/120:27:051
118282271,5sleep00-21swapper/021:26:360
11332271,5sleep10-21swapper/100:17:561
109852271,5sleep10-21swapper/123:07:461
47899269,5cyclictest40950irq/93-i91500:35:151
47899262,3cyclictest81rcu_preempt22:39:471
47899262,3cyclictest81rcu_preempt22:39:471
478992617,6cyclictest22-21ksoftirqd/121:48:091
478992617,6cyclictest22-21ksoftirqd/100:05:231
477992617,6cyclictest3-21ksoftirqd/023:47:560
304212261,4sleep00-21swapper/020:47:160
47799252,4cyclictest81rcu_preempt00:28:430
47899241,4cyclictest81rcu_preempt20:47:201
47799241,12cyclictest4689-21taskset22:47:330
47799241,10cyclictest0-21swapper/000:48:420
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional