You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-23 - 11:30

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack9slot4s (updated Sun Nov 23, 2025 00:59:03)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
40219912471200,27cyclictest4216-21wget19:25:121
40219911021087,9cyclictest30541-21users20:30:301
37562561517,29sleep00-21swapper/019:22:530
39062481417,48sleep10-21swapper/119:24:291
402099208188,16cyclictest0-21swapper/019:25:120
125222990,5sleep0402099cyclictest22:16:520
4020999569,23cyclictest0-21swapper/020:30:310
59542910,5sleep14018-21cyclictest22:10:201
232772850,3sleep0111rcuc/021:20:320
245772660,2sleep024578-21ssh00:44:110
307472650,3sleep130746-21kernelversion00:50:271
124522630,2sleep012451-21diskstats21:10:140
308272530,3sleep00-21swapper/022:35:320
242462410,4sleep0111rcuc/021:55:280
4020993924,4cyclictest3-21ksoftirqd/022:25:110
4020993923,5cyclictest3-21ksoftirqd/000:34:110
195702397,4sleep019652-21timerandwakeup21:50:290
4020993823,5cyclictest3-21ksoftirqd/021:42:530
88492371,32sleep18854-21irqstats21:00:221
4021993721,12cyclictest21820-21/usr/sbin/munin21:20:121
4020993722,5cyclictest3-21ksoftirqd/019:31:200
4020993721,6cyclictest3-21ksoftirqd/023:25:420
4020993720,6cyclictest3-21ksoftirqd/020:24:110
4020993720,6cyclictest3-21ksoftirqd/000:18:480
4020993713,8cyclictest803-21open_files23:45:280
4020993713,7cyclictest3-21ksoftirqd/021:34:220
191342370,4sleep041ktimersoftd/023:30:290
4021993617,15cyclictest22-21ksoftirqd/122:54:461
4020993620,6cyclictest3-21ksoftirqd/023:11:300
4020993620,6cyclictest3-21ksoftirqd/021:36:240
4020993620,5cyclictest3-21ksoftirqd/019:54:510
4020993612,7cyclictest3-21ksoftirqd/020:00:480
275502361,31sleep127561-21ssh23:05:351
402099356,18cyclictest14767-21ssh21:45:300
4020993520,5cyclictest3-21ksoftirqd/021:09:480
4020993520,5cyclictest3-21ksoftirqd/000:52:340
4020993519,6cyclictest3-21ksoftirqd/022:00:540
4020993516,6cyclictest28768-21ssh21:26:310
4020993515,5cyclictest22197-21ntp_states23:00:250
4020993512,7cyclictest3-21ksoftirqd/000:13:580
402099351,22cyclictest4160-21ssh00:22:290
120352351,30sleep111870-21diskmemload21:43:101
4021993415,15cyclictest22-21ksoftirqd/100:08:171
402099341,4cyclictest19748-21hddtemp_smartct00:05:170
4020993411,12cyclictest27394-21ssh23:40:070
180192341,29sleep118033-21ssh00:03:421
402199333,15cyclictest0-21swapper/121:36:441
402199332,27cyclictest0-21swapper/123:50:401
4021993313,16cyclictest2220-21dbus-daemon21:30:121
402099332,10cyclictest8744-21hddtemp_smartct23:20:180
4020993319,10cyclictest0-21swapper/023:54:530
4020993318,5cyclictest3-21ksoftirqd/020:51:300
4020993316,6cyclictest21601-21pidof20:10:000
4020993316,5cyclictest25717-21df_inode00:45:150
4020993315,5cyclictest16068-21gzip22:20:270
4020993314,16cyclictest0-21swapper/020:43:190
4020993312,6cyclictest3-21ksoftirqd/023:19:040
402099331,19cyclictest1332-21ntp_states22:05:250
4020993311,7cyclictest3-21ksoftirqd/019:37:120
4020993310,6cyclictest3-21ksoftirqd/022:50:220
244092330,3sleep124419-21ssh23:36:001
244092330,3sleep124419-21ssh23:35:591
20832331,4sleep12094-21sshd23:46:421
402199329,19cyclictest0-21swapper/100:33:581
402199325,4cyclictest81rcu_preempt20:00:471
402099329,8cyclictest3-21ksoftirqd/022:42:330
402099328,7cyclictest3-21ksoftirqd/000:03:240
402099327,11cyclictest17147-21grep21:15:140
4020993211,7cyclictest3-21ksoftirqd/020:29:380
4020993210,7cyclictest2376-21NetworkManager22:46:150
168342321,4sleep116851-21watchsnmpd23:29:001
78642311,5sleep10-21swapper/123:19:591
402199315,4cyclictest81rcu_preempt00:22:131
402199311,4cyclictest81rcu_preempt22:19:371
402199310,4cyclictest81rcu_preempt00:13:571
402099319,7cyclictest3-21ksoftirqd/023:36:290
402099319,7cyclictest3-21ksoftirqd/023:36:290
402099319,4cyclictest6288-21ntp_states22:10:250
402099318,12cyclictest24022-21fschecks_time20:15:180
4020993116,5cyclictest25258-21if_eth022:30:190
4020993115,6cyclictest0-21swapper/019:44:110
402099311,26cyclictest13180-21ssh23:58:390
4020993110,7cyclictest3-21ksoftirqd/021:00:240
4020993110,7cyclictest3-21ksoftirqd/000:25:540
163222311,4sleep016359-21hddtemp_smartct00:35:180
155052311,4sleep115519-21ssh21:13:181
100022311,4sleep110016-21pidof00:29:001
77202301,5sleep00-21swapper/020:56:250
402199309,15cyclictest0-21swapper/122:07:221
402199306,11cyclictest0-21swapper/121:59:431
402199305,3cyclictest81rcu_preempt19:34:431
402199302,4cyclictest81rcu_preempt21:53:191
4021993017,9cyclictest22-21ksoftirqd/123:58:501
4021993016,10cyclictest22-21ksoftirqd/123:31:141
4021993016,10cyclictest22-21ksoftirqd/121:48:041
4021993014,12cyclictest0-21swapper/122:01:041
402099308,12cyclictest27386-21/usr/sbin/munin23:05:300
402099302,16cyclictest0-21swapper/019:59:260
402099302,16cyclictest0-21swapper/019:59:260
39502301,5sleep00-21swapper/020:46:230
265992301,5sleep10-21swapper/121:25:101
180452301,4sleep118056-21ssh22:55:481
139042301,25sleep12220-21dbus-daemon19:48:291
402199299,16cyclictest0-21swapper/122:35:341
402199296,4cyclictest81rcu_preempt20:29:391
402199293,4cyclictest81rcu_preempt21:08:401
402199292,14cyclictest16100-21df00:35:141
4021992919,5cyclictest10880-21cut19:40:181
4021992917,9cyclictest22-21ksoftirqd/100:45:301
4021992917,9cyclictest22-21ksoftirqd/100:15:561
4021992916,9cyclictest22-21ksoftirqd/123:14:281
4021992916,9cyclictest22-21ksoftirqd/123:00:271
4021992916,9cyclictest22-21ksoftirqd/122:30:201
4021992916,9cyclictest22-21ksoftirqd/100:40:251
402199291,4cyclictest81rcu_preempt20:24:381
402199291,4cyclictest81rcu_preempt19:37:111
4021992910,15cyclictest0-21swapper/123:41:091
4020992914,4cyclictest0-21swapper/022:55:130
402199286,18cyclictest0-21swapper/122:27:241
402199284,5cyclictest81rcu_preempt20:40:581
4021992819,5cyclictest21915-21tr20:10:131
4021992816,9cyclictest22-21ksoftirqd/122:46:141
4021992816,9cyclictest22-21ksoftirqd/122:21:171
4021992816,9cyclictest22-21ksoftirqd/121:16:151
4021992816,8cyclictest22-21ksoftirqd/123:20:371
4021992811,12cyclictest0-21swapper/122:41:421
402099289,9cyclictest0-21swapper/020:35:460
402099288,7cyclictest3-21ksoftirqd/019:45:110
232152281,5sleep00-21swapper/020:12:280
61052271,5sleep10-21swapper/120:54:091
212662271,5sleep10-21swapper/120:06:461
176262271,5sleep10-21swapper/119:58:071
176262271,5sleep10-21swapper/119:58:071
402199262,4cyclictest81rcu_preempt20:17:391
4021992617,5cyclictest7071-21cat20:55:231
4021992612,10cyclictest31392-21cstates20:35:151
402199252,18cyclictest0-21swapper/120:45:131
4021992515,6cyclictest22-21ksoftirqd/119:50:321
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional