You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-19 - 06:47

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack9slot4s (updated Sat Jul 19, 2025 00:59:04)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
39949912641233,21cyclictest0-21swapper/019:28:520
36552564517,30sleep00-21swapper/019:22:060
35872511416,31sleep10-21swapper/119:21:241
399599233227,4cyclictest9341-21kworker/1:319:28:521
148032840,4sleep03-21ksoftirqd/023:26:170
166022760,2sleep116601-21sendmail_mailtr00:35:301
153412720,4sleep00-21swapper/000:35:030
113752710,4sleep00-21swapper/022:50:070
31102690,3sleep00-21swapper/023:48:220
123472570,2sleep012346-21ssh23:57:490
321052500,3sleep132104-21ssh23:10:281
321052500,3sleep132104-21ssh23:10:281
189952490,3sleep041ktimersoftd/022:57:240
44112450,5sleep0399499cyclictest21:35:190
399499456,15cyclictest6204-21gdbus23:15:440
253482440,3sleep10-21swapper/100:45:151
3995994217,15cyclictest22-21ksoftirqd/100:42:141
3995994217,15cyclictest22-21ksoftirqd/100:14:181
3995994215,7cyclictest22-21ksoftirqd/121:33:391
3995994215,15cyclictest22-21ksoftirqd/122:33:541
3995994116,15cyclictest22-21ksoftirqd/100:22:291
304772410,10sleep0399499cyclictest23:43:070
3995994016,7cyclictest22-21ksoftirqd/122:53:461
3995994015,15cyclictest22-21ksoftirqd/121:56:311
3995994015,15cyclictest22-21ksoftirqd/121:16:081
3995994015,15cyclictest22-21ksoftirqd/121:16:071
3994994010,10cyclictest0-21swapper/000:20:260
3995993916,6cyclictest22-21ksoftirqd/123:55:231
3995993914,15cyclictest22-21ksoftirqd/100:31:121
3995993914,15cyclictest22-21ksoftirqd/100:19:211
3995993817,7cyclictest22-21ksoftirqd/122:38:351
3995993817,6cyclictest22-21ksoftirqd/123:05:371
3995993817,6cyclictest22-21ksoftirqd/122:18:151
3995993813,15cyclictest22-21ksoftirqd/122:10:031
399499388,13cyclictest3020-21runrttasks00:11:490
399499382,17cyclictest0-21swapper/022:05:280
399499382,14cyclictest1464-21cron00:55:000
158872381,33sleep115974-21ssh22:20:281
137312380,2sleep113730-21awk23:25:241
68392371,4sleep16924-21df22:45:151
3995993717,6cyclictest22-21ksoftirqd/123:19:441
3995993716,6cyclictest22-21ksoftirqd/123:00:221
3995993716,6cyclictest22-21ksoftirqd/122:59:571
3995993715,7cyclictest22-21ksoftirqd/123:43:571
3995993715,7cyclictest22-21ksoftirqd/122:42:031
3995993715,6cyclictest22-21ksoftirqd/122:01:341
3995993715,6cyclictest22-21ksoftirqd/100:50:151
3995993713,14cyclictest22-21ksoftirqd/123:35:501
3995993712,15cyclictest22-21ksoftirqd/121:45:441
3995993616,6cyclictest22-21ksoftirqd/123:49:551
3995993615,7cyclictest22-21ksoftirqd/123:51:571
3995993615,7cyclictest22-21ksoftirqd/123:51:571
3995993614,7cyclictest22-21ksoftirqd/121:51:141
399499362,29cyclictest7138-21cut19:30:170
399499362,15cyclictest0-21swapper/022:44:440
399499361,5cyclictest12229-21sh22:16:350
399499361,19cyclictest5855-21taskset20:52:120
399499361,11cyclictest7060-21ssh23:52:060
399499361,11cyclictest7060-21ssh23:52:060
287792360,7sleep0399499cyclictest23:07:370
399599358,5cyclictest40950irq/93-i91500:05:011
399599352,15cyclictest19826-21/usr/sbin/munin20:05:261
3995993514,7cyclictest22-21ksoftirqd/100:07:551
3995993514,6cyclictest22-21ksoftirqd/121:37:321
3995993513,7cyclictest22-21ksoftirqd/121:13:271
399499359,8cyclictest20886-21if_err_eth000:40:180
399499358,6cyclictest25204-21irqstats22:30:230
399499355,11cyclictest14856-21runrttasks22:20:110
399499352,14cyclictest0-21swapper/023:35:480
399499352,13cyclictest23618-21perl21:55:160
3994993511,8cyclictest0-21swapper/023:32:570
399499351,14cyclictest6246-21awk22:10:300
399499351,14cyclictest28832-21expr00:15:130
124372351,30sleep012451-21ssh23:24:530
3995993413,7cyclictest22-21ksoftirqd/122:29:161
3995993412,7cyclictest3992-21cyclictest00:25:121
3995993411,8cyclictest12087-21ssh21:43:211
3995993410,10cyclictest22-21ksoftirqd/123:23:131
399499343,16cyclictest0-21swapper/000:45:100
399499342,5cyclictest10976-21diskstats00:30:160
399499342,14cyclictest16332-21df19:55:130
399499342,13cyclictest8987-21df_inode21:40:130
399499342,13cyclictest10959-21/usr/sbin/munin19:40:200
399499341,6cyclictest13561-21aten_r4power_en21:45:140
399499341,14cyclictest31560-21/usr/sbin/munin23:10:210
399499341,14cyclictest31560-21/usr/sbin/munin23:10:200
399599339,20cyclictest11789-21diskmemload22:09:121
399599331,5cyclictest31108-21/usr/sbin/munin20:35:321
3995993312,7cyclictest22-21ksoftirqd/123:32:181
399499339,6cyclictest32152-21sed21:30:210
399499332,12cyclictest0-21swapper/000:25:270
399499331,6cyclictest6945-21diskstats22:45:160
399499331,6cyclictest29395-21grep22:35:150
399499331,6cyclictest19535-21fw_packets00:05:220
399499331,18cyclictest0-21swapper/020:43:250
399499331,18cyclictest0-21swapper/020:43:250
399499331,16cyclictest0-21swapper/021:11:110
399499331,14cyclictest15729-21sh00:00:420
399499331,12cyclictest2343-21irqbalance21:50:370
3994993310,7cyclictest30604-21runrttasks22:02:160
3994993310,6cyclictest19704-21ssh21:17:280
3994993310,6cyclictest19704-21ssh21:17:270
399599327,5cyclictest40950irq/93-i91520:30:301
399599325,6cyclictest27270-21kernelversion21:25:201
3995993212,6cyclictest22-21ksoftirqd/119:46:301
399499327,5cyclictest20594-21ntp_states22:25:230
399499322,5cyclictest17949-21/usr/sbin/munin20:00:140
399499321,6cyclictest28761-21ssh21:26:370
399499321,5cyclictest6491-21cpuspeed_turbos20:55:140
399499321,5cyclictest23616-21ssh23:02:070
399499321,14cyclictest25986-21/usr/sbin/munin20:20:170
399499321,13cyclictest3256-21sed20:45:220
399599318,8cyclictest22-21ksoftirqd/120:42:251
399599318,8cyclictest22-21ksoftirqd/120:42:251
399599318,7cyclictest9203-21sed19:35:241
3995993117,10cyclictest22-21ksoftirqd/120:01:291
3995993112,6cyclictest22-21ksoftirqd/119:31:261
399499311,14cyclictest11408-21taskset21:06:100
399499311,13cyclictest9617-21runrttasks21:02:030
399499311,13cyclictest30692-21taskset20:32:290
399499311,13cyclictest27835-21perl20:25:200
399499311,13cyclictest22125-21fschecks_time20:10:160
399499311,12cyclictest9021-21fw_packets19:35:190
140112311,26sleep014032-21munin-run19:50:010
399599309,7cyclictest23752-21ssh21:21:181
399599307,8cyclictest22-21ksoftirqd/120:26:561
399599301,10cyclictest6525-21cstates20:55:151
399599301,10cyclictest11824-21taskset19:41:451
399499303,16cyclictest0-21swapper/019:51:040
3994993019,7cyclictest24799-21unixbench_singl20:15:280
399499301,16cyclictest0-21swapper/020:05:030
57222291,5sleep10-21swapper/120:50:531
399599298,7cyclictest22-21ksoftirqd/121:09:401
399599291,12cyclictest0-21swapper/120:48:261
257192291,5sleep00-21swapper/021:24:120
399599285,4cyclictest40950irq/93-i91519:55:221
3995992811,4cyclictest23413-21latency_hist20:15:001
399599281,13cyclictest0-21swapper/120:16:001
399499281,16cyclictest0-21swapper/020:36:040
157512281,5sleep10-21swapper/119:53:381
399599275,6cyclictest26951-21seq20:22:371
3995992617,6cyclictest22-21ksoftirqd/121:03:321
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional