You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-12 - 08:29

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack9slot4s (updated Wed Nov 12, 2025 00:59:03)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
146629912661,5cyclictest0-21swapper/119:25:071
146619912511228,20cyclictest0-21swapper/019:25:060
146629911171091,23cyclictest19184-21sed19:35:011
144182562516,31sleep00-21swapper/019:23:070
142942466419,32sleep10-21swapper/119:21:531
3154821230,10sleep01466199cyclictest00:05:520
52212790,3sleep1211ktimersoftd/121:25:161
106452770,4sleep041ktimersoftd/023:10:340
240772710,4sleep0111rcuc/000:32:010
240022700,4sleep123991-21ssh21:45:011
22272570,5sleep12223-21grep23:35:311
173002500,22sleep03-21ksoftirqd/021:37:220
81842491,30sleep08192-21sh00:50:060
1466299461,6cyclictest22-21ksoftirqd/122:00:471
1466299451,7cyclictest22-21ksoftirqd/123:34:471
1466299451,6cyclictest22-21ksoftirqd/123:14:581
1466299441,7cyclictest22-21ksoftirqd/122:57:111
1466299441,7cyclictest22-21ksoftirqd/122:49:571
1466299441,7cyclictest22-21ksoftirqd/122:39:171
1466299441,6cyclictest22-21ksoftirqd/121:20:351
1466299431,7cyclictest22-21ksoftirqd/121:12:481
1466299431,6cyclictest22-21ksoftirqd/123:27:311
1466299431,6cyclictest22-21ksoftirqd/123:23:561
1466299431,6cyclictest22-21ksoftirqd/122:44:061
1466299431,6cyclictest22-21ksoftirqd/122:24:481
1466299431,6cyclictest22-21ksoftirqd/122:17:421
1466299431,6cyclictest22-21ksoftirqd/121:58:251
1466299431,6cyclictest22-21ksoftirqd/121:37:501
1466299427,5cyclictest22-21ksoftirqd/100:06:291
1466299421,7cyclictest22-21ksoftirqd/121:17:381
1466299421,7cyclictest22-21ksoftirqd/100:27:311
1466299421,6cyclictest22-21ksoftirqd/123:52:041
1466299421,6cyclictest22-21ksoftirqd/123:45:441
1466299421,6cyclictest22-21ksoftirqd/123:15:111
1466299421,6cyclictest22-21ksoftirqd/123:15:101
1466299421,6cyclictest22-21ksoftirqd/122:10:321
1466299421,6cyclictest22-21ksoftirqd/121:46:211
1466299421,6cyclictest22-21ksoftirqd/100:39:451
1466299421,6cyclictest22-21ksoftirqd/100:22:001
1466299411,6cyclictest22-21ksoftirqd/121:32:251
1466299411,6cyclictest22-21ksoftirqd/100:32:291
14662994110,9cyclictest22-21ksoftirqd/119:40:231
1466299409,4cyclictest81rcu_preempt00:54:111
1466299408,9cyclictest81rcu_preempt21:53:041
1466299401,7cyclictest22-21ksoftirqd/100:00:281
1466299401,7cyclictest22-21ksoftirqd/100:00:271
1466299401,6cyclictest22-21ksoftirqd/123:03:371
1466299401,6cyclictest22-21ksoftirqd/122:30:001
1466299392,4cyclictest22-21ksoftirqd/123:58:461
1466299391,7cyclictest22-21ksoftirqd/120:10:181
1466299391,6cyclictest22-21ksoftirqd/100:45:291
14662993914,6cyclictest22-21ksoftirqd/100:17:371
14662993824,4cyclictest22-21ksoftirqd/119:45:251
1466299381,6cyclictest22-21ksoftirqd/123:05:301
1466299381,6cyclictest22-21ksoftirqd/122:30:271
1466299381,6cyclictest22-21ksoftirqd/120:35:211
1466299381,6cyclictest22-21ksoftirqd/120:05:161
14662993816,19cyclictest22-21ksoftirqd/100:44:111
1466299371,7cyclictest22-21ksoftirqd/123:44:271
1466299371,7cyclictest22-21ksoftirqd/119:50:131
1466299371,6cyclictest22-21ksoftirqd/100:10:131
14662993713,4cyclictest81rcu_preempt19:35:301
1466299371,19cyclictest22-21ksoftirqd/120:20:181
14661993711,13cyclictest22502-21diskmemload22:51:360
1466299363,4cyclictest81rcu_preempt22:05:531
104492360,8sleep00-21swapper/022:04:230
197352351,5sleep019807-21processes21:00:280
1466299355,9cyclictest22-21ksoftirqd/122:50:141
14662993510,4cyclictest81rcu_preempt20:00:281
1466199356,4cyclictest40950irq/93-i91519:45:240
14661993512,4cyclictest0-21swapper/023:42:510
14661993511,4cyclictest81rcu_preempt22:50:000
1466299348,4cyclictest81rcu_preempt19:55:261
1466199345,4cyclictest81rcu_preempt23:33:160
1466199344,4cyclictest81rcu_preempt21:56:300
1466199344,4cyclictest81rcu_preempt00:00:500
1466199344,4cyclictest81rcu_preempt00:00:490
1466199343,4cyclictest81rcu_preempt23:07:100
1466199341,4cyclictest81rcu_preempt22:32:220
1466199341,21cyclictest21715-21grep21:05:310
1466199341,12cyclictest5104-21vmstat20:20:310
131562341,4sleep014777-21kworker/0:322:06:180
103982341,29sleep010478-21python20:35:310
1466299331,4cyclictest81rcu_preempt20:30:161
14662993311,7cyclictest22-21ksoftirqd/121:09:051
1466199336,4cyclictest81rcu_preempt20:45:120
1466199335,5cyclictest81rcu_preempt22:21:300
1466199333,4cyclictest81rcu_preempt23:22:540
1466199332,5cyclictest81rcu_preempt23:57:250
14661993325,5cyclictest6919-21latency_hist00:15:010
1466199332,4cyclictest81rcu_preempt21:14:290
1466199331,13cyclictest15160-21cat23:50:000
1466299326,5cyclictest81rcu_preempt20:51:511
1466299320,4cyclictest81rcu_preempt21:03:251
1466199324,4cyclictest81rcu_preempt21:15:200
1466199323,4cyclictest81rcu_preempt23:53:400
1466199322,12cyclictest0-21swapper/023:00:000
14661993217,5cyclictest3-21ksoftirqd/019:30:220
1466199321,4cyclictest81rcu_preempt00:27:430
1466199321,4cyclictest81rcu_preempt00:24:090
1466199321,20cyclictest8076-21ssh22:35:170
1466199321,19cyclictest3227-21ssh23:36:450
1466199321,15cyclictest0-21swapper/000:18:200
14661993210,11cyclictest2376-21NetworkManager00:40:440
1466299311,15cyclictest81rcu_preempt20:25:151
1466199316,5cyclictest81rcu_preempt21:43:430
1466199315,5cyclictest81rcu_preempt20:00:580
1466199312,4cyclictest81rcu_preempt22:17:480
1466199312,4cyclictest81rcu_preempt21:46:450
1466199312,18cyclictest3020-21runrttasks23:25:280
1466199311,8cyclictest0-21swapper/021:22:500
1466199311,4cyclictest81rcu_preempt23:00:420
1466199311,4cyclictest81rcu_preempt21:27:460
1466199311,4cyclictest81rcu_preempt19:44:390
14661993112,5cyclictest10685-21awk21:30:240
1466199311,22cyclictest0-21swapper/021:50:080
206202301,5sleep00-21swapper/019:36:590
1466199309,8cyclictest0-21swapper/019:58:080
1466199306,4cyclictest81rcu_preempt00:45:130
1466199303,4cyclictest81rcu_preempt23:18:520
1466199303,4cyclictest81rcu_preempt23:18:520
1466199303,4cyclictest81rcu_preempt00:35:580
14661993023,5cyclictest0-21swapper/020:51:110
14661993019,7cyclictest11474-21wget20:40:180
1466199301,4cyclictest81rcu_preempt22:26:070
1466199301,4cyclictest81rcu_preempt20:11:030
14661993012,5cyclictest16781-21memory22:10:220
1466199300,4cyclictest81rcu_preempt20:56:380
70692291,5sleep00-21swapper/020:26:240
33362291,5sleep00-21swapper/020:16:540
1466299291,4cyclictest81rcu_preempt20:45:121
14662992912,6cyclictest40950irq/93-i91520:55:181
1466199292,4cyclictest81rcu_preempt22:40:100
1466199291,4cyclictest81rcu_preempt20:34:360
263262281,5sleep00-21swapper/019:52:290
1466299282,7cyclictest2108-21df20:15:131
1466199283,18cyclictest31341-21ntp_states20:05:270
14662992719,5cyclictest12118-21needreboot20:40:291
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional