You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-26 - 23:46

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack9slot4s (updated Wed Nov 26, 2025 12:59:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
238689911161078,18cyclictest0-21swapper/007:25:110
238689911161078,18cyclictest0-21swapper/007:25:100
238699910891067,18cyclictest7149-21apt-get08:03:541
234552649602,32sleep10-21swapper/107:21:201
235562479430,33sleep00-21swapper/007:22:240
2386899241211,13cyclictest0-21swapper/008:03:540
2386999139134,3cyclictest0-21swapper/107:25:111
2386999139134,3cyclictest0-21swapper/107:25:101
207622950,3sleep0111rcuc/010:05:120
285852930,4sleep00-21swapper/010:12:560
230242910,8sleep12386999cyclictest10:40:271
282832870,10sleep02386899cyclictest09:39:330
299102770,3sleep0111rcuc/009:40:340
114192490,9sleep12386999cyclictest11:35:281
288382440,4sleep128816-21ssh11:20:231
69312410,9sleep12386999cyclictest09:50:221
23869993914,6cyclictest22-21ksoftirqd/111:17:181
303922381,32sleep130522-21/usr/sbin/munin07:40:131
2386999386,4cyclictest81rcu_preempt10:32:411
23868993720,6cyclictest3-21ksoftirqd/010:40:420
52562360,2sleep05257-21fschecks_time09:15:180
2386999366,5cyclictest81rcu_preempt10:02:241
2386999361,12cyclictest25542ntp_kernel_pll_07:50:501
2386899364,4cyclictest81rcu_preempt12:29:090
2386899362,21cyclictest12227-21ntp_states08:15:250
233222361,31sleep123332-21ssh09:34:231
11062360,8sleep12386999cyclictest11:25:181
11062360,8sleep12386999cyclictest11:25:181
48122351,4sleep14823-21sh12:02:161
48122351,4sleep14823-21sh12:02:151
35032351,30sleep03546-21iostat10:20:260
2386999357,6cyclictest40950irq/93-i91510:10:271
23869993523,9cyclictest30118-1kworker/1:0H12:24:081
2386899351,22cyclictest16469-21ssh09:26:220
23868993512,10cyclictest11266-21ssh12:09:390
82892341,29sleep18299-21rm10:59:251
306532341,4sleep130718-21uname11:55:271
2386999343,17cyclictest11208-21ssh09:55:141
2386999342,27cyclictest0-21swapper/112:29:451
2386999342,24cyclictest40950irq/93-i91512:31:511
2386999341,6cyclictest28643-21ls09:40:011
2386899344,4cyclictest81rcu_preempt12:00:280
2386899344,4cyclictest81rcu_preempt12:00:280
2386899344,4cyclictest81rcu_preempt11:59:420
2386899342,14cyclictest23654-21ssh11:48:400
66722331,5sleep00-21swapper/010:57:010
2386999336,5cyclictest81rcu_preempt12:45:411
2386999335,9cyclictest201rcuc/111:53:091
23869993325,5cyclictest19048-21ssh10:36:001
23869993318,11cyclictest20939-21wget08:40:151
23869993313,6cyclictest22-21ksoftirqd/109:40:531
2386899339,11cyclictest6306-21cpu09:50:130
2386899333,4cyclictest81rcu_preempt12:22:040
23868993324,6cyclictest20889-21ssh12:19:360
23868993313,7cyclictest10893-21ssh09:55:030
23868993311,6cyclictest3999-21grep12:35:160
23868993311,6cyclictest3999-21grep12:35:150
154632331,5sleep115470-21diskstats08:25:151
101752331,5sleep110185-21ssh12:08:021
42472320,4sleep041ktimersoftd/011:28:290
42472320,4sleep041ktimersoftd/011:28:280
33412320,9sleep02386899cyclictest09:13:220
323302321,27sleep032388-21iostat10:50:200
2386999322,16cyclictest5953-21ssh12:36:531
2386999322,16cyclictest5953-21ssh12:36:531
2386999321,7cyclictest13797-21/usr/sbin/munin11:05:281
23869993216,4cyclictest22-21ksoftirqd/109:45:211
2386999321,4cyclictest81rcu_preempt11:49:141
2386999321,4cyclictest81rcu_preempt10:48:181
23869993214,15cyclictest22-21ksoftirqd/110:23:181
2386999321,27cyclictest0-21swapper/111:00:291
2386999321,18cyclictest21733-21grep10:05:271
2386899326,14cyclictest0-21swapper/010:18:210
2386899324,4cyclictest81rcu_preempt11:05:480
2386899323,5cyclictest81rcu_preempt09:30:270
2386899323,4cyclictest81rcu_preempt11:32:470
2386899322,4cyclictest81rcu_preempt11:35:140
23868993222,6cyclictest16344-21ssh10:34:050
2386899321,6cyclictest0-21swapper/011:20:320
136852321,4sleep00-21swapper/009:24:220
114832321,4sleep111494-21ssh12:42:531
98322311,4sleep110005-21perl08:10:161
50102311,5sleep00-21swapper/007:56:140
270722311,26sleep127155-21memory08:55:231
270722311,26sleep127155-21memory08:55:231
2386999316,13cyclictest40950irq/93-i91510:50:251
23869993116,5cyclictest22-21ksoftirqd/110:16:231
2386999311,5cyclictest81rcu_preempt12:10:291
2386999311,16cyclictest8298-21iostat_ios10:25:211
2386899315,13cyclictest18588-21sh11:10:080
2386899313,4cyclictest81rcu_preempt12:50:200
2386899313,21cyclictest31422-21diskstats12:30:150
2386899312,4cyclictest81rcu_preempt10:26:310
2386899312,15cyclictest24657-21/usr/sbin/munin08:50:120
2386899311,5cyclictest81rcu_preempt10:45:250
2386899311,4cyclictest81rcu_preempt12:48:020
2386899311,4cyclictest81rcu_preempt10:39:460
23868993113,6cyclictest8747-21diskstats12:40:170
23868993111,6cyclictest23436-21/usr/sbin/munin11:15:200
2386899311,16cyclictest18040-21mailstats08:30:280
2386899311,12cyclictest16458-21wc12:15:010
23868993111,16cyclictest3020-21runrttasks08:47:400
190092311,5sleep119019-21ssh11:43:541
178732311,5sleep00-21swapper/010:01:220
296462301,4sleep00-21swapper/009:00:330
281832301,5sleep00-21swapper/007:34:110
2386999306,5cyclictest40950irq/93-i91509:16:091
23869993025,3cyclictest137350irq/90-eth009:29:371
23869993015,12cyclictest22-21ksoftirqd/112:15:281
2386999301,4cyclictest81rcu_preempt12:52:191
2386999301,4cyclictest81rcu_preempt08:30:241
2386999301,16cyclictest24657-21aten_r4power_po08:50:111
23869993011,14cyclictest492-21irqstats09:10:231
2386899307,4cyclictest81rcu_preempt11:40:160
227142301,5sleep10-21swapper/111:14:221
167922301,5sleep00-21swapper/008:29:020
303652291,5sleep00-21swapper/007:40:090
284372291,5sleep00-21swapper/011:53:370
2386999294,18cyclictest383-21iostat_ios07:45:211
23869992921,5cyclictest22-21ksoftirqd/109:20:131
23869992917,7cyclictest26768-21cpuspeed_turbos07:30:121
23869992911,6cyclictest40950irq/93-i91509:05:251
2386999290,4cyclictest81rcu_preempt08:36:321
2386899291,4cyclictest81rcu_preempt09:46:010
2386899291,14cyclictest9000-21/usr/sbin/munin11:00:270
148732291,5sleep00-21swapper/008:23:380
113842291,5sleep10-21swapper/108:15:031
90062281,4sleep00-21swapper/008:06:240
316912281,5sleep00-21swapper/009:07:140
2386999282,5cyclictest28471-21/usr/sbin/munin07:35:161
2386999282,4cyclictest81rcu_preempt11:34:591
2386999282,4cyclictest81rcu_preempt09:01:491
23869992820,5cyclictest3996-21cut07:55:151
2386899284,4cyclictest81rcu_preempt07:50:010
2386899282,13cyclictest19523-21iostat08:35:190
2386899281,4cyclictest81rcu_preempt07:37:480
23868992810,5cyclictest21847-21python08:40:300
23869992716,8cyclictest22-21ksoftirqd/108:45:521
2386999271,4cyclictest81rcu_preempt08:06:521
2386899272,13cyclictest10820-21taskset08:10:460
2386899272,12cyclictest26541-21aten_r4power_po08:55:120
2386899272,12cyclictest26541-21aten_r4power_po08:55:120
2386899271,13cyclictest32767-21hwlatdetect07:45:190
2386999269,6cyclictest40950irq/93-i91508:20:261
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional