You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-22 - 01:37

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack9slot4s (updated Fri Nov 21, 2025 12:59:04)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
65939911261103,19cyclictest30944-21snmpd07:45:271
65929911221082,37cyclictest0-21swapper/007:25:100
61362738693,30sleep10-21swapper/107:20:561
61172561516,31sleep00-21swapper/007:20:430
659399115109,3cyclictest0-21swapper/107:25:101
2706921150,3sleep10-21swapper/112:10:141
1908521070,5sleep0659299cyclictest10:55:080
1908521070,5sleep0659299cyclictest10:55:070
263992870,8sleep1659399cyclictest11:01:561
6592998477,4cyclictest0-21swapper/007:45:270
125952820,2sleep012594-21sh11:55:130
122422820,7sleep00-21swapper/012:28:180
290312810,11sleep00-21swapper/008:20:210
171552730,4sleep00-21swapper/011:25:310
231642530,10sleep0659299cyclictest09:18:180
234492501,35sleep123617-21df12:40:131
234492501,35sleep123617-21df12:40:121
257482491,4sleep025765-21sh11:01:000
659399416,5cyclictest81rcu_preempt09:55:341
6592994125,5cyclictest3-21ksoftirqd/010:42:500
6592994123,6cyclictest3-21ksoftirqd/009:36:000
6592994023,6cyclictest3-21ksoftirqd/010:25:310
6592994023,5cyclictest3-21ksoftirqd/010:07:190
6592994016,8cyclictest3-21ksoftirqd/011:15:150
659299391,14cyclictest0-21swapper/009:20:160
175362391,34sleep117547-21ssh11:26:051
6592993822,5cyclictest3-21ksoftirqd/012:18:430
97212370,4sleep0111rcuc/007:30:140
6592993722,5cyclictest3-21ksoftirqd/012:36:410
6592993721,6cyclictest3-21ksoftirqd/010:39:130
6592993721,5cyclictest3-21ksoftirqd/010:03:090
6592993720,6cyclictest3-21ksoftirqd/012:46:400
659299371,25cyclictest0-21swapper/010:18:040
307102371,32sleep130721-21ssh10:33:171
6592993621,5cyclictest3-21ksoftirqd/007:55:500
6592993619,6cyclictest3-21ksoftirqd/007:38:110
6592993619,5cyclictest3-21ksoftirqd/007:52:430
165532361,31sleep114687-21diskmemload10:51:401
659399358,5cyclictest81rcu_preempt12:37:551
659399354,10cyclictest12230-21ssh10:47:211
6592993520,5cyclictest3-21ksoftirqd/011:53:010
6592993520,4cyclictest3-21ksoftirqd/011:45:440
6592993519,6cyclictest3-21ksoftirqd/009:59:260
6592993519,5cyclictest3-21ksoftirqd/011:06:180
6592993519,5cyclictest3-21ksoftirqd/008:50:560
6592993518,6cyclictest3-21ksoftirqd/012:23:170
6592993518,6cyclictest3-21ksoftirqd/009:40:420
659299351,21cyclictest21862-21unixbench_multi11:30:280
229852351,30sleep022991-21diskstats08:05:120
659399341,14cyclictest20493-21/usr/sbin/munin11:30:291
659399341,14cyclictest12897-21sshd11:21:201
659299349,13cyclictest18635-21sh12:00:350
6592993419,5cyclictest3-21ksoftirqd/009:11:090
6592993417,6cyclictest3-21ksoftirqd/011:39:090
6592993417,6cyclictest3-21ksoftirqd/009:50:400
6593993310,19cyclictest0-21swapper/109:46:571
6592993319,4cyclictest3-21ksoftirqd/012:05:560
6592993318,5cyclictest3-21ksoftirqd/008:47:120
6592993318,5cyclictest3-21ksoftirqd/008:15:450
6592993317,5cyclictest3-21ksoftirqd/009:06:590
6592993312,5cyclictest17603-21ssh12:34:060
70602321,4sleep17187-21grep09:35:171
659399328,20cyclictest0-21swapper/109:51:231
659399324,11cyclictest15610-21iostat_ios09:10:181
659399322,25cyclictest29747-21df_inode09:25:141
659399321,9cyclictest3020-21runrttasks11:18:501
659399321,4cyclictest81rcu_preempt09:18:381
659399321,11cyclictest0-21swapper/110:25:261
659299326,15cyclictest40950irq/93-i91510:22:450
6592993219,7cyclictest27701-21ntp_states12:10:260
6592993218,5cyclictest3-21ksoftirqd/011:14:210
6592993217,5cyclictest3-21ksoftirqd/011:23:040
6592993217,5cyclictest3-21ksoftirqd/009:04:190
6592993217,5cyclictest3-21ksoftirqd/008:43:250
6592993217,5cyclictest3-21ksoftirqd/007:41:570
6592993217,5cyclictest3-21ksoftirqd/007:41:560
6592993215,6cyclictest3-21ksoftirqd/012:42:500
6592993215,6cyclictest3-21ksoftirqd/012:42:490
63922321,5sleep10-21swapper/110:07:571
77902311,5sleep10-21swapper/112:23:511
659399317,5cyclictest40950irq/93-i91511:56:071
659399317,10cyclictest0-21swapper/112:02:551
659399314,4cyclictest81rcu_preempt11:46:031
659399313,4cyclictest81rcu_preempt07:57:231
659399312,8cyclictest0-21swapper/107:35:021
659399312,24cyclictest29457-21iostat11:05:211
659399311,4cyclictest81rcu_preempt12:46:081
6593993113,14cyclictest0-21swapper/111:39:081
659299319,11cyclictest11322-21diskmemload10:46:030
659299312,15cyclictest31501-21users08:25:330
6592993116,5cyclictest3-21ksoftirqd/010:51:080
6592993115,5cyclictest3-21ksoftirqd/008:55:080
6592993115,5cyclictest3-21ksoftirqd/008:32:310
6592993114,6cyclictest30835-21iostat11:40:210
6592993114,5cyclictest3-21ksoftirqd/008:01:210
6592993113,5cyclictest8074-21/usr/sbin/munin10:10:230
51062310,3sleep1211ktimersoftd/110:40:191
250712311,5sleep10-21swapper/108:10:161
167852311,4sleep10-21swapper/110:19:041
131732311,5sleep10-21swapper/109:40:471
659399306,4cyclictest81rcu_preempt08:58:091
659399305,4cyclictest81rcu_preempt09:02:381
659399305,10cyclictest2220-21dbus-daemon11:50:541
659399303,24cyclictest0-21swapper/107:51:551
659399302,8cyclictest18047-21df10:20:141
659399302,24cyclictest0-21swapper/110:04:131
659399301,6cyclictest32114-21/usr/sbin/munin08:30:131
659399301,16cyclictest22850-21netstat12:05:261
659399300,4cyclictest81rcu_preempt12:50:541
6592993013,6cyclictest3-21ksoftirqd/012:51:260
44172301,5sleep10-21swapper/111:13:031
21782301,5sleep10-21swapper/111:44:541
15412301,5sleep130944-21snmpd12:16:571
128092301,5sleep10-21swapper/112:29:081
114962301,5sleep10-21swapper/107:35:171
659399294,4cyclictest81rcu_preempt09:22:281
659399294,17cyclictest3841-21diskstats08:40:161
659399292,8cyclictest19821-21kernelversion10:55:211
659399292,8cyclictest19821-21kernelversion10:55:211
659399292,4cyclictest81rcu_preempt08:38:221
659399291,7cyclictest15527-21ssh12:31:031
659399290,4cyclictest81rcu_preempt10:12:441
6592992911,15cyclictest0-21swapper/009:28:270
31112291,5sleep00-21swapper/008:37:260
296402291,5sleep10-21swapper/108:20:351
179332291,5sleep00-21swapper/009:45:420
659399289,15cyclictest0-21swapper/109:31:061
659399283,4cyclictest81rcu_preempt07:41:391
659399283,4cyclictest81rcu_preempt07:41:381
6593992819,5cyclictest9028-21latency_hist08:55:001
6593992815,9cyclictest22-21ksoftirqd/110:35:281
659299281,7cyclictest28347-21ntp_states10:30:270
56382281,5sleep00-21swapper/009:33:510
318792281,5sleep10-21swapper/108:29:181
241472281,5sleep10-21swapper/108:07:171
223772281,5sleep10-21swapper/108:03:251
68022271,5sleep10-21swapper/108:46:301
260572271,5sleep00-21swapper/008:12:370
143272271,5sleep10-21swapper/109:06:251
659399259,5cyclictest40950irq/93-i91508:15:201
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional