You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-07 - 12:56

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack9slot4s (updated Sun Dec 07, 2025 00:59:03)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
86679911991177,18cyclictest21444-21ssh21:15:011
86669911851135,32cyclictest9094-21df19:25:150
86679911091085,21cyclictest0-21swapper/119:25:121
81942649602,31sleep10-21swapper/119:20:421
72332562516,30sleep00-21swapper/019:20:170
866699189137,47cyclictest21454-21latency_hist21:15:010
101782960,3sleep110179-21cpu22:10:151
95522910,3sleep041ktimersoftd/021:35:260
10482850,9sleep0866699cyclictest22:34:310
71902830,4sleep10-21swapper/122:40:191
292612810,3sleep1201rcuc/100:44:201
176942770,4sleep0111rcuc/019:45:240
11172760,4sleep00-21swapper/023:07:320
212132590,2sleep021214-21sh00:01:350
8666995822,6cyclictest3-21ksoftirqd/000:06:380
244492580,8sleep0866699cyclictest23:31:160
285852550,8sleep0866699cyclictest22:29:220
33172500,8sleep0866699cyclictest20:35:010
121952490,5sleep041ktimersoftd/019:30:280
233272440,8sleep1866799cyclictest00:04:401
107222421,3sleep010738-21iostat_ios22:10:240
866699404,31cyclictest81rcu_preempt00:31:250
322492391,34sleep132259-21ssh00:13:401
866699381,21cyclictest1237-21ssh23:40:430
189412381,5sleep118952-21sh22:52:191
8667993716,10cyclictest22-21ksoftirqd/100:45:351
8666993714,7cyclictest16504-21munin-run21:10:000
8666993713,8cyclictest26749-21df21:20:130
149912371,32sleep116512-21diskmemload00:29:361
866799368,5cyclictest81rcu_preempt22:25:291
866799365,4cyclictest81rcu_preempt23:42:261
866699365,4cyclictest81rcu_preempt23:23:180
866799352,17cyclictest19989-21df_inode22:20:151
8666993510,4cyclictest81rcu_preempt21:56:090
75432351,3sleep17546-21ssh23:14:451
160622350,8sleep0866699cyclictest23:56:060
8667993419,4cyclictest22-21ksoftirqd/123:30:511
866699347,4cyclictest81rcu_preempt00:21:530
866699341,20cyclictest32609-21ssh00:47:230
181222341,29sleep118133-21sh23:59:081
866799334,4cyclictest81rcu_preempt21:06:351
866799332,5cyclictest81rcu_preempt23:37:461
866799331,4cyclictest81rcu_preempt00:37:071
866699338,5cyclictest81rcu_preempt22:55:440
866699334,4cyclictest81rcu_preempt21:44:500
8666993322,7cyclictest4590-21ssh00:18:270
866699331,6cyclictest22289-21ssh21:15:200
8666993315,5cyclictest27853-21ssh23:35:200
866699331,19cyclictest19158-21diskstats19:50:160
866699331,10cyclictest12961-21ssh00:26:350
8666993310,5cyclictest26775-21fschecks_count20:10:200
3682330,8sleep0866699cyclictest20:25:230
321422331,29sleep132308-21cat20:25:131
214032331,28sleep121430-21fschecks_count22:55:151
161502331,28sleep116310-21unixbench_multi19:40:301
99252321,4sleep19946-21hddtemp_smartct23:50:191
88212321,27sleep18881-21df21:35:131
866799329,19cyclictest0-21swapper/100:19:351
866799325,4cyclictest81rcu_preempt20:44:211
866799325,23cyclictest0-21swapper/121:49:391
866799323,23cyclictest9750-21munin-run22:10:011
8667993215,13cyclictest22-21ksoftirqd/121:53:161
866799321,4cyclictest81rcu_preempt23:28:101
866799321,4cyclictest81rcu_preempt22:46:031
866799321,17cyclictest10537-21ssh23:17:041
8667993210,18cyclictest0-21swapper/100:34:291
866699329,4cyclictest81rcu_preempt21:46:220
866699324,5cyclictest81rcu_preempt22:17:190
866699323,5cyclictest81rcu_preempt22:40:320
866699321,9cyclictest21395-21ssh00:35:270
8666993215,6cyclictest3-21ksoftirqd/022:06:200
866699321,4cyclictest81rcu_preempt23:03:560
866699321,4cyclictest81rcu_preempt21:32:350
74852321,4sleep17496-21sh23:47:401
308622321,4sleep130944-21snmpd21:24:501
297112321,27sleep06204-21gdbus20:17:120
256412321,27sleep12376-21NetworkManager00:06:011
194362321,5sleep00-21swapper/022:53:010
866799318,12cyclictest40950irq/93-i91500:24:051
866799312,25cyclictest0-21swapper/123:21:341
8667993113,13cyclictest2390-21iostat_ios22:35:221
866699316,5cyclictest81rcu_preempt23:11:070
866699316,4cyclictest81rcu_preempt22:47:270
866699315,4cyclictest81rcu_preempt00:11:300
866699313,5cyclictest81rcu_preempt21:27:150
866699313,5cyclictest81rcu_preempt21:27:150
866699312,4cyclictest81rcu_preempt22:00:320
866699312,4cyclictest81rcu_preempt22:00:310
866699312,13cyclictest3175-21ntp_states00:50:250
866699311,14cyclictest6802-21ssh23:46:380
61622311,5sleep00-21swapper/022:39:470
146342311,5sleep00-21swapper/019:37:030
866799307,5cyclictest40950irq/93-i91520:20:011
866799302,16cyclictest0-21swapper/121:55:211
866799301,4cyclictest81rcu_preempt21:31:331
866799301,4cyclictest81rcu_preempt20:53:191
866699307,4cyclictest81rcu_preempt23:28:000
866699306,11cyclictest16512-21diskmemload23:51:270
866699303,14cyclictest0-21swapper/020:57:270
866699302,14cyclictest26962-21ssh21:54:060
8666993014,5cyclictest3-21ksoftirqd/020:20:130
8666993010,6cyclictest21200-21ssh22:20:370
70192301,5sleep00-21swapper/020:44:260
40522301,4sleep14062-21ssh00:51:071
306672301,25sleep130679-21ssh23:05:071
164582301,5sleep00-21swapper/019:41:180
866799295,4cyclictest81rcu_preempt21:03:411
866799294,4cyclictest81rcu_preempt20:57:291
866799294,4cyclictest81rcu_preempt20:47:401
866799294,18cyclictest21951-21diskstats21:15:161
866799294,15cyclictest0-21swapper/119:55:271
866799294,14cyclictest0-21swapper/122:16:191
866799293,22cyclictest0-21swapper/122:31:281
8667992915,4cyclictest40950irq/93-i91521:43:101
8666992911,5cyclictest25696-21unixbench_multi20:05:290
866699291,14cyclictest14134-21grep21:00:310
39562291,4sleep13970-21hwlatdetect20:35:191
316692291,5sleep10-21swapper/120:22:481
266422291,5sleep10-21swapper/123:00:231
10822294,2sleep10-21swapper/121:26:431
10822294,2sleep10-21swapper/121:26:431
866799283,9cyclictest16984-21/usr/sbin/munin19:45:321
866799283,4cyclictest81rcu_preempt20:03:381
8667992817,8cyclictest22-21ksoftirqd/122:00:171
8667992817,8cyclictest22-21ksoftirqd/122:00:171
866699282,13cyclictest26005-21/usr/sbin/munin00:40:230
866699281,8cyclictest0-21swapper/020:50:170
866699281,15cyclictest8217-21grep23:15:160
866699281,14cyclictest8075-21ntp_states20:45:280
8666992810,6cyclictest21134-21fschecks_time19:55:170
31372281,4sleep10-21swapper/120:33:211
261542281,5sleep10-21swapper/120:09:171
194622281,5sleep10-21swapper/119:50:231
866799273,8cyclictest13805-21iostat_ios19:35:211
8666992717,6cyclictest0-21swapper/020:37:560
866699271,13cyclictest23057-21grep20:00:190
866799269,5cyclictest40950irq/93-i91519:30:011
866799251,11cyclictest26792-21taskset20:10:201
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional