You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-07 - 02:33

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 100 highest latencies:
System rack9slot4s (updated Thu Nov 06, 2025 12:59:04)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
274139911931169,21cyclictest0-21swapper/007:25:040
274149911421111,25cyclictest31850-21apt-get07:35:001
261312587508,64sleep10-21swapper/107:20:201
266622583537,31sleep00-21swapper/007:20:300
2741499203194,6cyclictest0-21swapper/107:25:041
83821230,7sleep02741399cyclictest09:40:250
274139910986,19cyclictest31889-21latency_hist07:35:010
2202121040,3sleep022020-21sh08:32:000
184992860,3sleep041ktimersoftd/009:25:290
112372760,7sleep12741499cyclictest10:25:141
303662750,3sleep041ktimersoftd/009:37:300
121472550,6sleep02741399cyclictest11:00:000
305492460,3sleep041ktimersoftd/010:10:480
236642440,4sleep1211ktimersoftd/110:04:481
223372431,4sleep122351-21ssh12:16:551
27413994224,6cyclictest3-21ksoftirqd/010:44:370
2741499413,6cyclictest21488-21fschecks_time12:50:161
2741499407,5cyclictest81rcu_preempt09:29:161
27414994020,15cyclictest1253-21wget10:15:121
27414994020,15cyclictest1253-21wget10:15:111
27413994019,6cyclictest3-21ksoftirqd/011:31:240
27413994019,6cyclictest3-21ksoftirqd/011:31:240
2741499397,4cyclictest81rcu_preempt09:39:521
27413993922,6cyclictest3-21ksoftirqd/010:17:470
27413993922,6cyclictest3-21ksoftirqd/010:17:470
27413993920,6cyclictest3-21ksoftirqd/012:50:000
27413993822,6cyclictest3-21ksoftirqd/012:09:350
27413993822,6cyclictest3-21ksoftirqd/012:09:350
27413993821,6cyclictest3-21ksoftirqd/012:37:430
27413993821,6cyclictest3-21ksoftirqd/011:11:510
27413993820,6cyclictest3-21ksoftirqd/012:28:270
27413993720,6cyclictest3-21ksoftirqd/009:48:390
27413993718,8cyclictest3-21ksoftirqd/010:52:000
27413993716,17cyclictest25743-21runrttasks09:32:460
27413993619,6cyclictest3-21ksoftirqd/010:49:500
27413993619,6cyclictest3-21ksoftirqd/010:05:540
2741399361,7cyclictest2220-21dbus-daemon07:54:550
2741499358,23cyclictest0-21swapper/111:58:031
2741499352,28cyclictest0-21swapper/112:09:051
2741499352,28cyclictest0-21swapper/112:09:051
2741399352,18cyclictest19263-21/usr/sbin/munin10:00:330
27413993520,6cyclictest3-21ksoftirqd/011:37:250
27413993520,5cyclictest3-21ksoftirqd/011:43:490
27413993520,5cyclictest3-21ksoftirqd/009:01:160
27413993520,5cyclictest3-21ksoftirqd/007:44:430
27413993520,5cyclictest3-21ksoftirqd/007:35:400
27413993519,6cyclictest3-21ksoftirqd/011:20:500
27413993519,6cyclictest3-21ksoftirqd/008:03:310
27413993519,5cyclictest3-21ksoftirqd/008:17:410
27413993518,6cyclictest3-21ksoftirqd/011:45:030
27413993510,6cyclictest0-21swapper/010:39:360
260352351,3sleep00-21swapper/008:44:320
94342341,29sleep09448-21ssh10:23:080
326212341,29sleep13165-21diskmemload12:27:501
2741499346,4cyclictest81rcu_preempt11:53:231
2741499342,8cyclictest10362-21users11:30:311
2741499342,8cyclictest10362-21users11:30:301
27414993416,15cyclictest22-21ksoftirqd/110:36:001
2741399348,12cyclictest5798-21taskset11:25:510
27413993419,5cyclictest3-21ksoftirqd/009:55:450
27413993419,5cyclictest3-21ksoftirqd/007:49:520
27413993418,6cyclictest3-21ksoftirqd/009:22:360
27413993418,5cyclictest3-21ksoftirqd/010:31:100
27413993417,6cyclictest3-21ksoftirqd/008:25:260
107382341,4sleep110751-21sh09:16:581
100682341,29sleep19614-21/usr/sbin/munin09:50:171
34212331,4sleep13457-21grep09:10:111
2741499337,5cyclictest0-21swapper/112:35:451
2741499332,4cyclictest81rcu_preempt09:40:471
27414993323,6cyclictest5433-21ssh12:00:061
2741499331,5cyclictest81rcu_preempt10:11:231
2741399339,13cyclictest3-21ksoftirqd/012:22:310
27413993318,5cyclictest3-21ksoftirqd/011:52:390
27413993317,6cyclictest3-21ksoftirqd/008:12:380
27413993317,5cyclictest3-21ksoftirqd/009:18:120
27413993317,5cyclictest0-21swapper/012:15:180
79232321,4sleep17937-21ssh11:28:591
76312321,27sleep17643-21ssh09:47:291
2741499329,19cyclictest0-21swapper/110:34:011
2741499325,4cyclictest81rcu_preempt10:57:261
2741499324,4cyclictest81rcu_preempt10:43:071
2741499322,4cyclictest81rcu_preempt11:45:081
27413993217,5cyclictest3-21ksoftirqd/010:30:000
27413993216,5cyclictest3-21ksoftirqd/008:38:550
27413993215,6cyclictest3-21ksoftirqd/011:06:060
2741399321,14cyclictest14237-21ssh11:01:070
159282320,4sleep02741399cyclictest12:44:360
103312320,7sleep02741399cyclictest09:50:210
27414993117,5cyclictest22-21ksoftirqd/109:58:281
27414993112,14cyclictest0-21swapper/111:05:041
2741399319,11cyclictest7785-21/usr/sbin/munin07:55:170
2741399317,13cyclictest3-21ksoftirqd/012:32:020
27413993116,5cyclictest3-21ksoftirqd/008:56:260
27413993116,5cyclictest3-21ksoftirqd/008:50:450
27413993116,5cyclictest3-21ksoftirqd/008:21:340
2741399311,26cyclictest27789-21ntp_states11:15:270
2741399311,13cyclictest444-21ssh11:55:040
228812311,5sleep10-21swapper/111:44:431
181982311,4sleep118208-21ssh12:45:581
92052301,5sleep00-21swapper/012:03:490
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional