You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-05-09 - 07:24

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack9slot4s (updated Fri May 09, 2025 00:59:01)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
307749912591237,18cyclictest31000-21wget19:25:131
307739911751139,19cyclictest3-21ksoftirqd/019:25:120
307749911421126,13cyclictest0-21swapper/120:50:011
305212549503,31sleep00-21swapper/019:22:570
306102476381,32sleep10-21swapper/119:23:521
2969621040,8sleep03077399cyclictest20:45:180
30773998165,13cyclictest0-21swapper/020:50:010
30773996119,9cyclictest18225-21iostat_ios21:40:210
110112461,33sleep111098-21cat00:15:011
30773993615,5cyclictest5140-21df19:40:120
266192361,4sleep12376-21NetworkManager22:02:371
3077399347,16cyclictest10869-21fschecks_count19:55:150
3077399346,15cyclictest0-21swapper/000:52:440
30774993315,5cyclictest10878-21fschecks_time19:55:151
3077399332,18cyclictest1448-21proc_pri20:55:290
30773993313,16cyclictest22884-21basename23:20:160
3077399331,19cyclictest16053-21timerandwakeup00:25:290
233052331,26sleep123319-21basename21:55:161
145342331,28sleep014541-21diskstats20:05:160
89112321,5sleep00-21swapper/000:07:360
48982321,4sleep14902-21ntp_states22:30:251
48982321,4sleep14902-21ntp_states22:30:251
30774993215,14cyclictest22-21ksoftirqd/120:59:011
3077499321,4cyclictest81rcu_preempt20:46:391
3077399328,13cyclictest20780-21chrt23:14:420
3077399325,13cyclictest5033-21mailstats22:30:270
3077399325,13cyclictest5033-21mailstats22:30:270
3077399321,16cyclictest21654-21sort20:25:000
3077499317,4cyclictest81rcu_preempt23:05:371
3077499316,4cyclictest81rcu_preempt20:11:031
3077499315,4cyclictest81rcu_preempt20:28:241
3077499313,7cyclictest10245-21ls22:45:191
3077499313,4cyclictest81rcu_preempt22:28:101
3077499311,16cyclictest397-21iostat_ios23:45:211
30774993110,10cyclictest40950irq/93-i91519:50:151
3077399318,10cyclictest0-21swapper/023:44:060
3077399314,14cyclictest0-21swapper/023:45:320
3077399312,17cyclictest0-21swapper/022:15:230
30773993112,6cyclictest3020-21runrttasks20:19:380
30773993112,6cyclictest27700-21diskstats20:40:150
72552301,5sleep00-21swapper/022:35:510
3077499307,4cyclictest81rcu_preempt21:28:101
3077499306,4cyclictest81rcu_preempt00:48:481
3077499306,4cyclictest81rcu_preempt00:16:161
3077499305,4cyclictest81rcu_preempt23:36:291
3077499305,4cyclictest81rcu_preempt23:31:261
3077499305,4cyclictest81rcu_preempt00:42:461
3077499304,4cyclictest81rcu_preempt22:56:301
3077399309,6cyclictest9554-21grep19:50:250
3077399304,4cyclictest81rcu_preempt21:51:360
3077399303,14cyclictest10541-21grep00:10:300
30773993023,4cyclictest3-21ksoftirqd/020:01:360
3077399301,4cyclictest81rcu_preempt22:41:340
3077399301,18cyclictest13039-21runrttasks22:52:190
3077399301,16cyclictest19298-21/usr/sbin/munin00:35:210
30773993010,7cyclictest14062-21iostat_ios22:55:200
249632301,5sleep10-21swapper/120:31:011
81292290,8sleep10-21swapper/122:40:151
58722291,5sleep10-21swapper/121:07:001
3077499296,4cyclictest81rcu_preempt21:35:401
3077499295,4cyclictest81rcu_preempt22:14:551
3077499294,4cyclictest81rcu_preempt00:22:591
3077499293,4cyclictest81rcu_preempt22:16:571
3077499292,4cyclictest81rcu_preempt20:09:481
3077499291,7cyclictest8227-21/usr/sbin/munin21:15:111
3077499291,6cyclictest6846-21awk00:00:351
3077499291,19cyclictest0-21swapper/121:54:581
3077399292,14cyclictest17243-21/usr/sbin/munin23:05:130
30773992912,5cyclictest4964-21/usr/sbin/munin21:05:200
30773992912,5cyclictest21190-21iostat_ios00:40:220
30773992912,5cyclictest19965-21perl21:45:170
3077399291,16cyclictest6251-21/usr/sbin/munin00:00:230
30773992911,5cyclictest25687-21/usr/sbin/munin22:00:210
3077399291,14cyclictest17179-21ls20:10:300
30773992910,6cyclictest24943-21cpuspeed_turbos23:25:170
30773992910,6cyclictest12013-21ntp_states00:15:270
30773992910,6cyclictest0-21swapper/021:57:490
247442291,5sleep10-21swapper/123:25:131
224112291,5sleep00-21swapper/023:17:090
224112291,5sleep00-21swapper/023:17:080
181982291,5sleep00-21swapper/000:31:330
173802291,5sleep00-21swapper/021:39:240
15782291,5sleep00-21swapper/022:21:230
152132291,5sleep00-21swapper/023:00:020
136492291,6sleep00-21swapper/021:29:350
89422281,5sleep10-21swapper/100:07:561
82852281,5sleep10-21swapper/119:48:231
78042281,4sleep00-21swapper/021:12:280
73522281,4sleep10-21swapper/122:36:521
61592281,5sleep10-21swapper/119:40:471
45382281,5sleep10-21swapper/119:38:391
3077499287,6cyclictest16048-21sed00:25:291
3077499287,4cyclictest81rcu_preempt23:55:521
3077499281,13cyclictest20077-21tr20:20:101
30774992811,14cyclictest201rcuc/100:31:181
3077399288,7cyclictest27617-21/usr/sbin/munin22:05:210
3077399286,5cyclictest0-21swapper/000:24:360
3077399282,13cyclictest8215-21munin-node21:15:100
3077399282,13cyclictest28989-21hddtemp_smartct23:35:200
3077399281,4cyclictest81rcu_preempt20:33:370
30773992811,5cyclictest26915-21diskstats23:30:180
30773992811,5cyclictest10717-21irqstats21:20:220
30773992810,14cyclictest11146-21taskset22:47:060
241682281,5sleep00-21swapper/000:49:420
232522281,5sleep00-21swapper/020:27:440
224652281,5sleep10-21swapper/123:17:451
224652281,5sleep10-21swapper/123:17:441
132802281,5sleep10-21swapper/122:54:541
73562271,4sleep10-21swapper/121:10:311
3077499275,4cyclictest81rcu_preempt22:20:141
3077499274,4cyclictest81rcu_preempt23:21:311
3077499273,16cyclictest0-21swapper/121:46:321
3077499272,4cyclictest81rcu_preempt20:41:431
3077499272,11cyclictest3020-21runrttasks21:33:061
3077499271,5cyclictest81rcu_preempt20:35:111
3077499271,4cyclictest81rcu_preempt23:40:121
3077499270,4cyclictest81rcu_preempt23:11:261
3077399279,6cyclictest3568-21irqstats19:35:210
3077399274,11cyclictest0-21swapper/019:33:450
3077399272,13cyclictest7523-21/usr/sbin/munin19:45:240
3077399272,12cyclictest14685-21/usr/sbin/munin21:30:250
30773992717,6cyclictest2779-21cpuspeed_turbos21:00:130
3077399271,4cyclictest81rcu_preempt20:37:490
3077399271,14cyclictest29642-21head22:10:220
30773992711,12cyclictest4299-21kernelversion23:55:220
195742271,5sleep10-21swapper/120:18:421
3077499265,4cyclictest81rcu_preempt23:52:211
3077499265,17cyclictest0-21swapper/122:05:131
3077499262,4cyclictest81rcu_preempt21:22:311
30774992618,5cyclictest22-21ksoftirqd/121:40:131
30774992617,6cyclictest22-21ksoftirqd/120:00:081
3077499261,11cyclictest3076-21hddtemp_smartct21:00:181
3077399261,13cyclictest1798-21grep23:50:110
30773992610,5cyclictest3017-21ntp_states22:25:260
256442261,5sleep10-21swapper/100:50:341
24892261,5sleep10-21swapper/119:32:391
3077499259,5cyclictest40950irq/93-i91500:40:011
3077499243,18cyclictest0-21swapper/123:00:051
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional