You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-16 - 00:45

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack9slot4s (updated Sat Nov 15, 2025 12:59:02)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
309189912811,1243cyclictest0-21swapper/007:25:050
309199911611151,7cyclictest0-21swapper/107:35:001
309199911071094,8cyclictest31011-21sed07:25:011
304052565518,32sleep00-21swapper/007:20:330
3091999150143,5cyclictest0-21swapper/107:25:061
30918999162,26cyclictest0-21swapper/007:35:000
191292880,9sleep03091899cyclictest08:15:230
28062840,7sleep00-21swapper/010:12:030
227472820,4sleep1211ktimersoftd/112:13:231
103502750,2sleep00-21swapper/011:26:360
269792730,4sleep00-21swapper/012:17:320
30919997213,10cyclictest16045-21run-parts11:00:001
30919996915,23cyclictest11749-21taskset09:15:081
30919996915,15cyclictest20886-21runrttasks08:20:471
30919996817,15cyclictest7341-21/usr/sbin/munin09:10:161
30919996813,14cyclictest30798-21/usr/sbin/munin11:15:251
30919996717,16cyclictest30944-21snmpd12:08:451
30919996716,15cyclictest11459-21users10:20:341
30919996715,17cyclictest28361-21ssh11:11:521
30919996714,21cyclictest7937-21ssh09:44:371
30919996713,21cyclictest32519-21kworker/1:210:35:341
30919996713,20cyclictest27817-21ssh12:18:461
30919996712,16cyclictest21852-21irqstats11:05:231
30919996616,14cyclictest3020-21runrttasks12:00:251
30919996615,17cyclictest25088-21taskset10:01:271
30919996615,15cyclictest27759-21taskset10:05:091
30919996613,14cyclictest31871-21kernelversion09:35:221
30919996610,15cyclictest6669-21diskmemload12:45:131
318192650,3sleep0111rcuc/010:42:010
3091999659,13cyclictest30436-21gzip10:40:301
30919996515,16cyclictest689-21fschecks_time10:10:171
30919996515,15cyclictest25548-21sh11:42:361
30919996514,15cyclictest7358-21ssh10:50:241
30919996514,15cyclictest16915-21hddtemp_smartct11:00:211
30919996514,14cyclictest20444-21memory10:30:241
30919996513,16cyclictest19098-21ntp_offset09:55:261
30919996417,16cyclictest27778-21users09:30:321
30919996417,15cyclictest1335-21ssh12:25:101
30919996415,16cyclictest4860-21ssh11:20:381
30919996414,16cyclictest24665-21if_eth008:30:201
30919996414,15cyclictest9324-21ssh12:32:471
30919996414,15cyclictest6669-21diskmemload11:47:521
30919996414,14cyclictest16950-21fschecks_count08:10:181
30919996413,15cyclictest10635-21latency_hist07:55:001
30919996410,15cyclictest14320-21/usr/sbin/munin09:50:281
3091999639,21cyclictest5235-21ssh10:48:211
3091999639,17cyclictest10822-21ssh11:27:171
30919996316,17cyclictest6385-21users11:55:421
30919996315,15cyclictest30741-21chrt12:20:571
30919996315,15cyclictest2580-21chrt08:57:461
30919996315,15cyclictest18838-21fschecks_time08:15:181
30919996315,15cyclictest13327-21ssh12:36:351
30919996314,17cyclictest10100-21ssh09:45:431
30919996314,16cyclictest21308-21cat09:25:011
30919996313,16cyclictest5033-21latency_hist07:40:011
30919996313,14cyclictest13714-21irqstats11:30:261
30919996311,24cyclictest7375-21ssh10:16:391
30919996310,22cyclictest21933-21ssh11:39:231
30919996217,14cyclictest26808-21sh08:35:261
30919996216,15cyclictest11270-21cat07:55:201
30919996215,15cyclictest32439-21aten_r4power_po11:50:161
30919996214,14cyclictest25402-21wget12:50:121
30919996214,13cyclictest5564-21fschecks_time07:40:191
30919996116,16cyclictest18103-21ssh10:28:111
30919996115,16cyclictest23816-21rm09:26:471
30919996114,16cyclictest28125-21expr08:40:141
30919996114,15cyclictest31152-21chrt08:46:471
30919996114,14cyclictest18242-21sh12:41:391
30919996114,14cyclictest1009-21wc08:55:011
30919996113,15cyclictest23636-21chrt08:26:571
30919996014,14cyclictest3866-21awk09:00:251
30919995814,13cyclictest2376-21NetworkManager08:09:341
30919995112,17cyclictest6333-21runrttasks09:06:381
30919994916,14cyclictest13097-21df08:00:141
30919994811,15cyclictest7728-21netstat07:45:241
3091899471,17cyclictest0-21swapper/012:11:100
3091899471,15cyclictest0-21swapper/009:56:580
3091899461,15cyclictest0-21swapper/010:09:030
3091899461,15cyclictest0-21swapper/009:40:100
30918994220,7cyclictest3-21ksoftirqd/011:15:100
3091899375,4cyclictest81rcu_preempt09:36:550
3091899371,6cyclictest7033-21cut10:50:180
3091899361,22cyclictest12406-21df_inode09:15:410
3091899361,17cyclictest21482-21df_inode11:05:150
30918993611,4cyclictest81rcu_preempt11:03:130
30918993611,4cyclictest81rcu_preempt10:45:380
3091899361,14cyclictest0-21swapper/010:33:030
225332361,5sleep042-21kswapd012:46:010
3091899357,15cyclictest11098-21ssh12:00:330
3091899354,4cyclictest81rcu_preempt11:52:200
3091899353,5cyclictest81rcu_preempt09:21:020
3091899351,20cyclictest21940-21fschecks_count09:25:220
3091899351,16cyclictest16285-21df12:40:140
3091899347,4cyclictest81rcu_preempt10:25:370
3091899345,4cyclictest81rcu_preempt12:10:000
3091899344,4cyclictest81rcu_preempt09:48:000
3091899343,5cyclictest81rcu_preempt11:34:310
3091899341,6cyclictest27460-21ssh10:04:540
3091899341,21cyclictest5193-21diskstats11:55:210
3091899341,20cyclictest28209-21ssh10:38:500
295242342,28sleep00-21swapper/012:20:180
33542338,5sleep03363-21diskstats09:00:160
3091899339,4cyclictest81rcu_preempt10:16:360
3091899338,7cyclictest11865-21ssh10:55:170
3091899337,13cyclictest13644-21diskstats09:50:170
3091899333,4cyclictest81rcu_preempt11:11:140
3091899333,4cyclictest81rcu_preempt10:20:040
3091899333,4cyclictest81rcu_preempt09:31:350
3091899331,5cyclictest81rcu_preempt12:50:110
3091899331,4cyclictest81rcu_preempt08:52:090
3091899331,20cyclictest3137-21ssh12:25:490
3091899331,20cyclictest25833-21latency_hist08:35:010
3091899326,5cyclictest81rcu_preempt11:45:400
3091899323,4cyclictest81rcu_preempt12:32:090
3091899322,4cyclictest81rcu_preempt11:44:020
30918993221,7cyclictest11426-21missed_timers07:55:240
3091899321,4cyclictest81rcu_preempt11:24:120
115232322,26sleep011627-21fschecks_time12:35:210
82572311,5sleep00-21swapper/007:45:440
59712311,26sleep06085-21cut07:40:290
45612311,5sleep00-21swapper/007:36:350
3091899313,4cyclictest81rcu_preempt11:38:580
3091899313,13cyclictest26575-21irqstats08:35:210
3091899311,5cyclictest81rcu_preempt09:10:190
3091899306,12cyclictest0-21swapper/008:10:270
3091899291,4cyclictest81rcu_preempt09:05:300
30918992913,4cyclictest81rcu_preempt08:05:200
3091899291,17cyclictest10531-21runrttasks07:54:520
3091899281,4cyclictest81rcu_preempt08:20:460
3091899281,23cyclictest13098-21df_inode08:00:140
3091899281,15cyclictest30406-21/usr/sbin/munin08:45:230
30918992811,5cyclictest1339-21df_inode08:55:150
3091899278,6cyclictest23319-21timerandwakeup08:25:290
3091899273,4cyclictest81rcu_preempt08:40:250
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional