You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-18 - 13:20

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack9slot4s (updated Tue Nov 18, 2025 00:59:03)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
156619912311216,12cyclictest0-21swapper/119:25:121
156609911931144,45cyclictest15815-21cut19:25:110
156619910951085,6cyclictest25240-21users19:45:331
154682654608,30sleep10-21swapper/119:23:381
153492464417,32sleep00-21swapper/019:22:260
1566099235215,17cyclictest0-21swapper/019:45:330
74152920,4sleep00-21swapper/023:06:300
315972900,8sleep01566099cyclictest21:17:440
315972900,8sleep01566099cyclictest21:17:440
15660996112,5cyclictest440-21iostat_ios00:40:250
256332580,8sleep01566099cyclictest21:11:060
1566099571,14cyclictest13259-21rm21:32:050
316942490,8sleep11566199cyclictest00:05:261
87622400,3sleep0111rcuc/000:15:220
244732400,8sleep01566099cyclictest21:44:200
179992401,35sleep118092-21cut20:55:171
84122391,33sleep18423-21ssh00:49:141
1566099395,6cyclictest15306-21ssh23:15:170
15660993916,5cyclictest0-21swapper/021:25:580
132332391,34sleep112891-21/usr/sbin/munin00:20:161
9592380,3sleep041ktimersoftd/023:00:190
25432381,33sleep02553-21ssh22:28:340
276802371,32sleep027695-21df_inode00:35:180
1566199377,4cyclictest81rcu_preempt21:41:371
1566099373,20cyclictest15218-21awk20:45:280
240962361,31sleep124114-21df_inode21:10:181
319282351,30sleep131940-21ssh21:18:151
319282351,30sleep131940-21ssh21:18:151
203912351,29sleep120484-21df19:35:141
17482351,29sleep11797-21ssh21:20:241
1566199358,23cyclictest0-21swapper/123:41:581
1566099355,4cyclictest81rcu_preempt22:00:110
1566099354,4cyclictest81rcu_preempt23:24:380
1566099354,4cyclictest81rcu_preempt22:40:150
15660993514,6cyclictest28804-21memory22:55:250
89072341,29sleep18921-21ssh21:27:461
223722341,28sleep122455-21memory21:05:231
1566199348,22cyclictest0-21swapper/100:34:391
15661993423,7cyclictest22-21ksoftirqd/123:30:561
1566199341,11cyclictest15950-21/usr/sbin/munin23:15:281
1566099345,4cyclictest81rcu_preempt00:30:130
1566099344,4cyclictest81rcu_preempt23:33:510
1566099344,4cyclictest81rcu_preempt22:23:080
1566099344,4cyclictest81rcu_preempt22:16:560
1566099341,12cyclictest8160-21unixbench_singl23:40:340
72922331,5sleep17298-21iostat_ios20:25:221
1566199337,5cyclictest81rcu_preempt22:43:221
1566199335,15cyclictest0-21swapper/122:15:161
1566199334,4cyclictest81rcu_preempt23:25:211
1566199332,27cyclictest0-21swapper/123:53:061
1566199331,19cyclictest0-21swapper/123:10:391
1566099335,5cyclictest81rcu_preempt00:13:080
1566099334,4cyclictest81rcu_preempt22:54:370
316012321,3sleep031693-21/usr/sbin/munin00:05:260
173772320,3sleep117375-21grep22:10:161
1566199322,26cyclictest0-21swapper/100:35:471
1566199322,16cyclictest26302-21diskstats00:00:161
1566199321,4cyclictest81rcu_preempt00:44:401
1566099324,14cyclictest0-21swapper/020:37:350
1566099323,4cyclictest81rcu_preempt00:50:380
1566099321,9cyclictest4858-21awk22:30:270
1566099321,9cyclictest4858-21awk22:30:270
15660993213,5cyclictest15911-21diskstats21:35:180
1566099321,12cyclictest0-21swapper/020:26:190
15660993210,8cyclictest13015-21sh00:20:120
96512311,3sleep09805-21unixbench_singl20:30:290
222062311,5sleep10-21swapper/123:55:291
190352311,26sleep019085-21iostat22:45:230
164112311,26sleep116116-21/usr/sbin/munin20:50:171
1566199315,4cyclictest81rcu_preempt20:21:011
1566199314,4cyclictest81rcu_preempt20:38:471
15661993118,10cyclictest22-21ksoftirqd/123:49:211
1566199311,16cyclictest28804-21memory22:55:251
15661993111,4cyclictest81rcu_preempt23:00:191
1566099313,5cyclictest81rcu_preempt00:29:430
1566099312,5cyclictest81rcu_preempt21:50:240
1566099312,15cyclictest8283-21ssh00:49:010
1566099312,15cyclictest11689-21ssh23:45:120
1566099312,13cyclictest28565-21ntp_states19:55:290
1566099311,4cyclictest81rcu_preempt23:39:400
1566099311,15cyclictest13309-21awk20:40:280
1566099311,10cyclictest0-21swapper/021:23:500
46392301,4sleep14713-21memory22:30:251
46392301,4sleep14713-21memory22:30:251
251482301,5sleep10-21swapper/121:45:071
1566199305,21cyclictest0-21swapper/100:17:541
1566199304,4cyclictest81rcu_preempt19:42:201
1566199302,4cyclictest81rcu_preempt21:52:561
1566199301,8cyclictest0-21swapper/121:39:151
15661993018,4cyclictest22-21ksoftirqd/100:25:101
15661993016,11cyclictest22-21ksoftirqd/123:05:291
1566099306,5cyclictest81rcu_preempt00:00:400
1566099306,13cyclictest16620-21http_loadtime20:50:200
1566099304,13cyclictest0-21swapper/020:05:310
1566099301,4cyclictest81rcu_preempt20:18:530
15660993012,6cyclictest30890-21/usr/sbin/munin20:00:300
15660993011,5cyclictest19058-21/usr/sbin/munin19:30:240
1566099301,11cyclictest0-21swapper/019:39:110
15660993010,6cyclictest9917-21python22:35:310
15660993010,4cyclictest3188-21iostat_ios21:55:230
64302291,5sleep00-21swapper/020:23:550
61502291,5sleep10-21swapper/123:39:391
269882291,5sleep00-21swapper/023:26:550
258852291,24sleep125896-21sh22:52:211
25632291,5sleep00-21swapper/020:13:530
237852291,5sleep00-21swapper/019:44:460
231412291,4sleep00-21swapper/021:06:340
1566199299,7cyclictest20487-21sh21:00:211
1566199299,16cyclictest0-21swapper/100:11:171
1566199293,4cyclictest81rcu_preempt19:33:451
1566199293,22cyclictest0-21swapper/123:22:041
1566199292,17cyclictest0-21swapper/122:26:011
15661992916,10cyclictest22-21ksoftirqd/100:50:191
15661992912,4cyclictest3020-21runrttasks19:55:501
1566199291,23cyclictest26934-21diskstats22:20:151
1566199291,15cyclictest11180-21sh21:30:161
1566199290,4cyclictest81rcu_preempt21:59:511
1566099295,4cyclictest81rcu_preempt23:50:340
1566099293,4cyclictest81rcu_preempt23:12:460
1566099292,14cyclictest17268-21df_inode22:10:150
1566099291,7cyclictest81rcu_preempt23:56:520
1566099291,14cyclictest25881-21irqstats21:45:200
1566099291,11cyclictest0-21swapper/022:06:290
119722291,4sleep111982-21ssh22:38:081
116502291,5sleep10-21swapper/122:04:431
23632281,5sleep10-21swapper/120:11:561
1566199288,16cyclictest0-21swapper/122:05:481
1566199284,4cyclictest81rcu_preempt20:02:061
1566199284,18cyclictest26150-21diskstats19:50:171
15661992813,11cyclictest0-21swapper/122:45:081
1566099281,14cyclictest26783-21ps19:50:290
1566099281,13cyclictest18983-21awk20:55:310
136952281,5sleep10-21swapper/120:41:131
15661992716,7cyclictest9637-21sort20:30:291
1566199271,4cyclictest81rcu_preempt20:07:461
1566199271,12cyclictest0-21swapper/120:45:191
1566099272,13cyclictest21694-21latency_hist21:05:010
15661992612,10cyclictest22-21ksoftirqd/120:15:141
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional