You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-29 - 18:04

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack9slot4s (updated Mon Jun 29, 2026 12:59:01)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
315069912741,6cyclictest19164-21perl09:40:141
315069910791070,6cyclictest0-21swapper/107:26:071
314282578532,31sleep00-21swapper/007:24:450
313922562518,29sleep10-21swapper/107:24:241
3150599243221,19cyclictest0-21swapper/007:26:060
3150599238214,4cyclictest0-21swapper/009:40:140
31506992341,230cyclictest0-21swapper/112:25:261
241602510,22sleep03-21ksoftirqd/011:20:120
222432481,5sleep022280-21idleruntime12:40:180
314052470,4sleep1211ktimersoftd/108:45:291
31506994215,4cyclictest22-21ksoftirqd/110:20:241
31506994215,4cyclictest22-21ksoftirqd/110:20:231
206452411,4sleep120680-21ntp_states12:35:261
164232410,5sleep00-21swapper/011:00:010
75942400,9sleep03150599cyclictest12:00:290
275752400,9sleep03150599cyclictest08:35:280
268242401,35sleep126224-21/usr/sbin/munin10:00:221
3150599374,11cyclictest81rcu_preempt09:10:580
3150599371,23cyclictest2695-21snmpd09:28:030
207452361,31sleep120808-21latency_hist08:20:001
110272361,30sleep111045-21iostat_ios12:10:191
3150699357,4cyclictest81rcu_preempt09:00:101
3150699351,6cyclictest22-21ksoftirqd/111:55:181
31506993515,4cyclictest22-21ksoftirqd/110:30:001
165972351,30sleep016643-21/usr/sbin/munin12:25:170
31506993418,12cyclictest21045-21ntp_states11:10:261
3150699341,7cyclictest22-21ksoftirqd/111:45:301
3150599347,6cyclictest286292sort10:05:190
3150599342,16cyclictest17090-21df09:35:130
3150599341,20cyclictest4046-21strings10:25:230
145652341,29sleep114713-21fw_packets12:20:161
3150699332,5cyclictest81rcu_preempt09:35:241
3150599333,15cyclictest10744-21diskstats12:10:140
264622331,4sleep026545-21memory11:25:200
3150699325,5cyclictest81rcu_preempt07:50:181
3150699324,4cyclictest81rcu_preempt11:27:411
31506993218,8cyclictest18936-21iostat_ios11:05:211
3150599322,18cyclictest0-21swapper/012:00:010
3150599322,18cyclictest0-21swapper/011:30:150
31505993217,9cyclictest8131-21iostat_ios07:45:180
31505993212,7cyclictest20900-21memory11:10:230
3150599321,20cyclictest0-21swapper/012:54:080
210432321,5sleep121211-21df08:20:121
83082311,5sleep10-21swapper/112:03:591
3150699317,4cyclictest22-21ksoftirqd/107:30:301
3150699315,4cyclictest81rcu_preempt07:36:341
3150699312,4cyclictest81rcu_preempt07:45:411
31506993118,9cyclictest22-21ksoftirqd/109:20:141
31506993113,6cyclictest22-21ksoftirqd/110:40:181
31506993113,6cyclictest22-21ksoftirqd/108:50:101
3150699311,10cyclictest22-21ksoftirqd/109:10:191
3150599318,5cyclictest81rcu_preempt08:45:120
3150599314,5cyclictest81rcu_preempt09:50:180
3150599312,17cyclictest0-21swapper/008:41:060
3150599311,4cyclictest81rcu_preempt08:10:290
31505993110,9cyclictest0-21swapper/010:45:280
211682311,4sleep121173-21irqstats09:45:191
203402311,5sleep00-21swapper/008:15:490
71112301,5sleep10-21swapper/109:08:441
69902301,5sleep00-21swapper/009:07:290
3150699307,4cyclictest81rcu_preempt10:50:201
3150699304,5cyclictest81rcu_preempt11:35:161
3150699304,4cyclictest81rcu_preempt12:31:041
31506993021,5cyclictest28954-21unixbench_singl11:30:301
31506993020,6cyclictest6613-21find07:40:281
31506993016,10cyclictest22-21ksoftirqd/110:30:131
31506993012,6cyclictest22-21ksoftirqd/108:40:251
3150599304,19cyclictest19695-21munin-run12:35:000
3150599303,4cyclictest81rcu_preempt08:25:250
3150599301,5cyclictest81rcu_preempt09:16:070
3150599301,4cyclictest81rcu_preempt09:33:420
3150599301,4cyclictest81rcu_preempt07:30:210
31505993012,14cyclictest6842chrt10:15:370
3150599301,13cyclictest32698-21perl08:50:150
157542301,5sleep00-21swapper/012:23:070
102982301,5sleep00-21swapper/010:41:230
85382291,5sleep00-21swapper/010:37:460
71652291,5sleep00-21swapper/007:42:530
50502291,5sleep00-21swapper/007:35:360
31832291,5sleep00-21swapper/008:57:140
3150699297,5cyclictest81rcu_preempt10:55:321
3150699297,4cyclictest81rcu_preempt12:45:221
3150699297,4cyclictest81rcu_preempt08:00:171
3150699296,4cyclictest81rcu_preempt09:30:151
3150699295,4cyclictest81rcu_preempt11:52:541
3150699295,4cyclictest81rcu_preempt08:27:141
3150699293,4cyclictest81rcu_preempt09:30:011
31506992914,4cyclictest22-21ksoftirqd/111:40:511
31506992914,11cyclictest22-21ksoftirqd/111:00:101
3150699291,11cyclictest31698-21latency_hist10:15:021
3150599298,7cyclictest32602-21sort11:40:280
3150599294,5cyclictest81rcu_preempt11:15:130
3150599293,14cyclictest81rcu_preempt08:00:010
3150599292,14cyclictest0-21swapper/012:50:000
3150599291,8cyclictest0-21swapper/007:50:230
3150599291,24cyclictest24834-21/usr/sbin/munin09:55:170
3150599291,15cyclictest4161-21/usr/sbin/munin09:00:170
3150599290,4cyclictest81rcu_preempt08:31:150
314832291,5sleep00-21swapper/010:12:540
30362291,5sleep00-21swapper/010:24:140
30362291,5sleep00-21swapper/010:24:130
276612291,5sleep00-21swapper/010:02:100
212482291,5sleep00-21swapper/012:36:450
179892291,5sleep00-21swapper/011:03:060
126812291,5sleep00-21swapper/009:22:500
7902281,5sleep10-21swapper/110:16:451
3150699286,4cyclictest81rcu_preempt12:05:261
3150699286,4cyclictest81rcu_preempt09:55:181
3150699284,4cyclictest81rcu_preempt11:15:131
3150699284,4cyclictest81rcu_preempt08:33:381
3150699284,17cyclictest24281-21diskstats11:20:141
3150699283,9cyclictest26857-21diskstats08:35:161
3150699281,7cyclictest0-21swapper/112:15:121
3150699281,10cyclictest22934-21fschecks_time09:50:161
3150599284,6cyclictest13405-21cpuspeed_turbos08:00:110
3150599283,13cyclictest5998-21ntp_states10:30:250
3150599281,4cyclictest81rcu_preempt12:15:550
3150599281,13cyclictest30089-21fschecks_time11:35:180
196462281,5sleep00-21swapper/011:05:510
127312281,4sleep10-21swapper/107:57:111
3150699274,4cyclictest81rcu_preempt08:05:151
3150699274,11cyclictest81rcu_preempt09:15:191
31506992716,8cyclictest22-21ksoftirqd/110:35:111
31506992712,4cyclictest81rcu_preempt08:55:121
31506992711,4cyclictest22-21ksoftirqd/110:45:191
3150599279,5cyclictest21559-21sendmail_mailtr09:45:260
3150599271,3cyclictest81rcu_preempt12:05:460
3150599271,14cyclictest1513-21if_eth011:45:190
296672271,5sleep10-21swapper/110:08:331
272362271,5sleep10-21swapper/112:55:001
42382261,5sleep00-21swapper/011:50:510
3150699262,9cyclictest17628-21fschecks_time08:10:171
3150599261,17cyclictest0-21swapper/008:05:050
224812261,5sleep00-21swapper/008:23:220
3150699241,9cyclictest21749-21/usr/sbin/munin12:40:101
3150599241,14cyclictest0-21swapper/010:51:230
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional