You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-06 - 00:36

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack9slot4s (updated Sat Jul 05, 2025 12:59:02)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
188889912191184,25cyclictest0-21swapper/007:25:150
188899911731145,21cyclictest14935-21smartctl08:35:301
188899911591131,24cyclictest6204-21gdbus07:25:141
187042576531,30sleep00-21swapper/007:23:410
184432490444,31sleep10-21swapper/107:20:591
188889911794,20cyclictest0-21swapper/008:35:300
2128521080,9sleep01888899cyclictest09:36:300
81342870,4sleep08117-21sendmail08:20:010
62722790,5sleep06274-21aten_r4power_po09:55:120
225112740,2sleep022512-21if_eth012:25:190
274722530,2sleep027473-21cat10:50:160
320082470,3sleep1211ktimersoftd/112:35:191
311672440,3sleep00-21swapper/009:14:070
22512421,11sleep00-21swapper/010:24:340
1888899407,5cyclictest81rcu_preempt09:23:190
1888899391,19cyclictest0-21swapper/012:40:540
182442391,5sleep118315-21grep12:20:291
43392381,14sleep14401-21ssh12:40:171
230352380,22sleep03-21ksoftirqd/011:52:080
1888999388,6cyclictest0-21swapper/110:46:291
1888899371,23cyclictest19514-21date08:50:010
1888899371,23cyclictest19514-21date08:50:000
170202370,3sleep1211ktimersoftd/111:45:331
53042361,4sleep05315-21ssh09:54:090
262462361,31sleep025271-21/usr/sbin/munin11:55:270
1888999361,11cyclictest6774-21/usr/sbin/munin09:55:201
12972361,31sleep11311-21ssh12:03:431
294742351,28sleep10-21swapper/109:11:431
1888999358,23cyclictest0-21swapper/109:32:481
18889993511,11cyclictest0-21swapper/110:18:561
1888899355,17cyclictest0-21swapper/010:12:470
1888999346,4cyclictest81rcu_preempt10:03:471
18889993425,5cyclictest28069-21/usr/sbin/munin10:50:261
1888999341,4cyclictest81rcu_preempt11:01:401
1888899344,5cyclictest81rcu_preempt12:14:430
18888993414,5cyclictest123822latency_hist10:35:010
1888899341,14cyclictest0-21swapper/011:44:010
1888899341,12cyclictest0-21swapper/008:57:560
175142341,29sleep12220-21dbus-daemon08:45:001
252722331,5sleep00-21swapper/007:38:500
244282331,28sleep124442-21ssh11:54:111
235742330,3sleep123588-21ssh12:25:531
1888999337,23cyclictest0-21swapper/110:55:191
1888999331,4cyclictest81rcu_preempt11:28:261
1888899332,5cyclictest81rcu_preempt12:38:200
18888993312,6cyclictest19613-21hddtemp_smartct11:15:210
1888899331,11cyclictest0-21swapper/011:11:370
101332331,29sleep09459-21/usr/sbin/munin09:25:230
45702321,27sleep14580-21ssh09:20:111
326242321,26sleep032695-21iostat10:55:190
296502321,4sleep129661-21ssh12:32:421
1888999327,21cyclictest0-21swapper/111:39:301
1888999325,11cyclictest0-21swapper/111:05:511
1888999322,5cyclictest81rcu_preempt10:35:211
18889993224,5cyclictest7605-21latency_hist10:30:011
1888999321,4cyclictest81rcu_preempt11:30:451
1888999321,13cyclictest0-21swapper/107:31:401
1888899327,11cyclictest2545-21ssh09:17:360
1888899324,4cyclictest81rcu_preempt09:46:180
18888993222,6cyclictest17333-21/usr/sbin/munin10:05:320
1888899322,15cyclictest5539-21/usr/sbin/munin11:00:280
1888899322,15cyclictest25426-21/usr/sbin/munin09:40:330
1888899321,5cyclictest81rcu_preempt10:25:320
1888899321,4cyclictest81rcu_preempt10:35:230
1888899321,14cyclictest2604-21cut08:05:120
133002321,6sleep00-21swapper/010:01:420
269702311,5sleep00-21swapper/007:41:460
1888999319,4cyclictest40950irq/93-i91511:43:501
1888999318,8cyclictest2220-21dbus-daemon07:44:041
1888999315,4cyclictest81rcu_preempt09:02:411
1888999313,4cyclictest81rcu_preempt12:06:011
1888999313,24cyclictest0-21swapper/107:55:131
1888999312,16cyclictest0-21swapper/110:41:391
1888999311,6cyclictest225632sh09:38:211
1888999311,5cyclictest81rcu_preempt12:10:061
1888999311,18cyclictest0-21swapper/111:11:551
1888999311,13cyclictest81rcu_preempt10:13:531
1888899319,5cyclictest15188-21sh12:17:410
1888899313,9cyclictest81rcu_preempt09:33:340
1888899312,13cyclictest21863-21df_inode07:30:160
18888993117,9cyclictest2893-21df12:05:150
1888899311,5cyclictest81rcu_preempt08:44:590
1888899311,4cyclictest81rcu_preempt08:54:380
1888899311,16cyclictest9375-21/usr/sbin/munin12:45:180
311982301,5sleep00-21swapper/007:55:050
288182301,5sleep00-21swapper/010:18:070
1888999307,19cyclictest0-21swapper/109:47:301
1888999304,4cyclictest81rcu_preempt09:44:101
1888999304,4cyclictest81rcu_preempt08:07:211
18889993023,5cyclictest0-21swapper/109:08:101
1888999302,23cyclictest16553-21iostat_ios10:05:211
1888999302,15cyclictest27824-21rm11:57:041
1888999301,4cyclictest81rcu_preempt12:47:331
18889993013,5cyclictest40950irq/93-i91509:25:181
1888999301,24cyclictest14025-21iostat_ios12:50:201
18889993012,13cyclictest2222-21ntp_states09:50:271
1888899306,4cyclictest81rcu_preempt11:20:130
1888899306,12cyclictest0-21swapper/011:35:130
1888899304,19cyclictest30446-21diskstats12:00:160
1888899304,13cyclictest3020-21runrttasks09:08:070
1888899302,14cyclictest24161-21ntp_states09:00:230
1888899301,4cyclictest81rcu_preempt12:32:030
1888899301,4cyclictest81rcu_preempt11:33:190
18888993012,5cyclictest22655-21sh10:45:140
1888899301,16cyclictest13022-21/usr/sbin/munin08:30:280
1888899301,15cyclictest29407-21irqstats11:25:230
116822301,5sleep00-21swapper/011:06:430
97772291,5sleep00-21swapper/008:22:450
1888999295,4cyclictest81rcu_preempt07:46:081
1888999292,6cyclictest31623-21ssh10:20:311
1888999291,4cyclictest81rcu_preempt10:32:371
1888899295,4cyclictest81rcu_preempt08:00:500
1888899291,8cyclictest15196-21ssh12:51:070
18888992911,6cyclictest17149-21aten_r4power_po12:20:130
306812281,5sleep00-21swapper/007:51:160
200592281,4sleep10-21swapper/108:50:201
1888999288,7cyclictest2220-21dbus-daemon08:12:111
1888999284,4cyclictest81rcu_preempt11:23:161
1888999282,4cyclictest81rcu_preempt08:19:311
18889992817,7cyclictest13758-21cron08:35:011
18889992810,15cyclictest0-21swapper/109:15:261
1888899282,13cyclictest28624-21/usr/sbin/munin07:45:300
1888899281,14cyclictest17499-21/usr/sbin/munin10:40:250
193562271,5sleep10-21swapper/108:49:351
193562271,5sleep10-21swapper/108:49:341
1888999276,17cyclictest0-21swapper/112:15:051
1888999276,17cyclictest0-21swapper/111:15:361
1888999274,4cyclictest81rcu_preempt08:22:121
1888999273,9cyclictest23772-21df07:35:171
1888999273,4cyclictest81rcu_preempt08:02:131
1888999272,21cyclictest0-21swapper/107:50:351
1888999271,4cyclictest81rcu_preempt08:56:311
18889992710,5cyclictest3020-21runrttasks08:28:411
1888899272,13cyclictest15838-21/usr/sbin/munin11:45:160
18888992710,12cyclictest1969-21lldpad08:25:330
1888899261,21cyclictest4973-21fschecks_time08:10:170
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional