You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-21 - 05:34

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack9slot4s (updated Fri Nov 21, 2025 00:59:02)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
181109912561,1251cyclictest0-21swapper/019:25:120
181119911551117,20cyclictest28989-21memory19:50:211
179852677630,32sleep00-21swapper/019:24:170
177532565518,31sleep10-21swapper/119:21:511
1811199128120,5cyclictest0-21swapper/119:25:121
198292980,9sleep11811199cyclictest00:24:421
18110999772,6cyclictest0-21swapper/019:50:220
249912860,9sleep01811099cyclictest22:15:220
243042790,4sleep03-21ksoftirqd/000:29:110
18912500,8sleep01811099cyclictest00:05:250
237482460,4sleep1201rcuc/121:40:231
1811199432,33cyclictest40950irq/93-i91522:57:361
106842430,7sleep01811099cyclictest00:48:390
1811199391,30cyclictest40950irq/93-i91523:38:281
1811099392,4cyclictest81rcu_preempt21:54:210
1811099391,29cyclictest40950irq/93-i91522:20:410
99842380,8sleep00-21swapper/022:33:340
1811199381,29cyclictest40950irq/93-i91521:33:021
1811099382,11cyclictest0-21swapper/021:10:260
18110993820,6cyclictest12919-21awk00:50:300
18110993711,13cyclictest21730-21mailstats19:30:270
1811199357,6cyclictest0-21swapper/122:35:131
18111993526,6cyclictest30948-21ssh00:01:511
18110993512,8cyclictest8735-21memory23:05:300
87712341,4sleep18785-21ssh22:31:501
44222341,5sleep14433-21ssh00:41:151
198272341,29sleep119841-21ssh22:43:501
1811199347,6cyclictest259512diskmemload22:08:221
1811199346,5cyclictest124162awk23:10:011
1811199346,24cyclictest0-21swapper/123:30:291
1811099344,4cyclictest81rcu_preempt22:43:110
1811099343,16cyclictest5270-21memory23:35:260
1811099342,13cyclictest0-21swapper/021:45:060
1811199339,13cyclictest40950irq/93-i91500:05:211
1811199336,4cyclictest81rcu_preempt20:45:261
18111993324,5cyclictest8126-21awk21:25:001
18111993323,6cyclictest24569-21/usr/sbin/munin23:55:241
18111993323,6cyclictest24569-21/usr/sbin/munin23:55:241
1811199332,26cyclictest0-21swapper/123:21:581
18111993310,19cyclictest0-21swapper/100:17:061
1811099333,4cyclictest81rcu_preempt22:45:250
1811099331,9cyclictest18896-21ssh00:23:180
1811099331,20cyclictest14620-21taskset20:39:590
1811099331,15cyclictest0-21swapper/022:07:470
23872321,27sleep12485-21/usr/sbin/munin20:05:261
221962321,4sleep122207-21ssh00:26:061
1811199323,23cyclictest17230-21munin-run23:15:001
1811199322,4cyclictest81rcu_preempt21:46:171
1811199320,6cyclictest81rcu_preempt00:13:221
1811099329,4cyclictest81rcu_preempt23:27:130
1811099327,4cyclictest81rcu_preempt22:50:140
1811099323,3cyclictest81rcu_preempt21:41:120
1811099322,4cyclictest81rcu_preempt22:11:170
1811099322,16cyclictest3500-21/usr/sbin/munin20:10:140
1811099322,14cyclictest22084-21munin-run23:20:000
1811099321,5cyclictest81rcu_preempt23:01:310
310672311,5sleep10-21swapper/122:21:131
208972311,5sleep10-21swapper/121:37:351
1811199317,12cyclictest40950irq/93-i91521:15:171
1811199316,4cyclictest81rcu_preempt23:02:011
1811199316,4cyclictest81rcu_preempt19:55:361
1811199315,22cyclictest0-21swapper/122:03:011
18111993121,6cyclictest2640-21/usr/sbin/munin22:25:291
1811199311,7cyclictest14813-21/usr/sbin/munin20:40:311
18111993115,13cyclictest22-21ksoftirqd/123:50:011
1811199311,4cyclictest81rcu_preempt23:54:091
18111993110,17cyclictest22-21ksoftirqd/100:46:571
1811099313,4cyclictest81rcu_preempt21:15:090
1811099312,7cyclictest0-21swapper/021:31:250
1811099312,4cyclictest81rcu_preempt23:45:020
1811099312,17cyclictest0-21swapper/022:00:270
1811099311,4cyclictest81rcu_preempt23:41:280
1811099311,4cyclictest81rcu_preempt00:30:330
18110993112,6cyclictest32619-21sh00:37:140
18110993112,6cyclictest24934-21grep23:55:290
18110993112,6cyclictest24934-21grep23:55:290
111912311,4sleep111205-21ssh23:41:131
317352301,5sleep00-21swapper/019:58:220
278942301,5sleep00-21swapper/019:47:300
278092301,5sleep10-21swapper/100:32:171
1811199303,4cyclictest81rcu_preempt20:29:241
1811199302,4cyclictest81rcu_preempt21:26:191
1811199301,4cyclictest81rcu_preempt19:30:561
18111993011,4cyclictest81rcu_preempt22:45:111
1811099308,11cyclictest6979-21sh21:23:210
1811099303,4cyclictest81rcu_preempt21:58:130
1811099302,10cyclictest0-21swapper/022:25:330
1811099301,9cyclictest31762-21ssh00:03:020
1811099301,8cyclictest4729-21ssh00:41:420
1811099301,6cyclictest0-21swapper/022:35:020
1811099301,5cyclictest81rcu_preempt21:25:130
1811099301,4cyclictest81rcu_preempt23:53:080
1811099301,4cyclictest81rcu_preempt23:24:150
18110993011,5cyclictest31560-21/usr/sbin/munin22:55:280
1811099300,4cyclictest81rcu_preempt20:29:240
50722291,5sleep10-21swapper/120:13:401
300672291,24sleep130105-21ssh00:35:141
255112291,5sleep00-21swapper/021:05:400
237002291,5sleep00-21swapper/021:01:200
218482291,5sleep10-21swapper/123:19:441
198582291,5sleep10-21swapper/120:50:391
1811199298,4cyclictest81rcu_preempt21:10:291
1811199295,4cyclictest81rcu_preempt21:50:541
1811199295,4cyclictest81rcu_preempt20:33:101
1811199295,18cyclictest26089-21diskstats22:50:171
1811199294,4cyclictest81rcu_preempt19:35:371
1811199291,7cyclictest20507-21ssh22:10:311
18111992917,9cyclictest22-21ksoftirqd/100:52:331
1811199291,15cyclictest23171-21ntp_states21:00:241
1811099292,4cyclictest81rcu_preempt00:11:350
18110992921,4cyclictest2012-21smartctl20:05:180
1811099292,13cyclictest14132-21users23:10:310
1811099291,4cyclictest81rcu_preempt19:38:130
18110992911,6cyclictest471-21ntp_states23:30:240
1811099291,14cyclictest25229-21memory19:40:180
1811199288,6cyclictest8850-21sh21:59:001
1811199284,20cyclictest0-21swapper/123:26:461
18111992815,7cyclictest25509-21python22:15:321
1811199281,25cyclictest0-21swapper/120:35:271
1811199281,23cyclictest28119-21sshd19:49:491
1811099289,15cyclictest7652-21fschecks_count20:20:160
1811099284,8cyclictest11335-21diskstats20:30:210
1811099282,4cyclictest81rcu_preempt21:35:090
1811099282,15cyclictest0-21swapper/020:02:420
18110992811,5cyclictest11382-21iostat_ios00:15:230
18110992810,5cyclictest6199-21ntp_states20:15:260
69412271,5sleep10-21swapper/120:18:291
1811199271,6cyclictest24223-21/usr/sbin/munin21:05:261
1811199271,13cyclictest0-21swapper/120:20:161
1811099272,14cyclictest0-21swapper/020:45:220
1811099271,13cyclictest19257-21seq20:50:230
1811199264,16cyclictest25232-21memory19:40:191
1811199262,11cyclictest0-21swapper/120:58:271
18111992616,6cyclictest22-21ksoftirqd/120:00:061
1811099267,6cyclictest21471-21python20:55:340
1811099261,12cyclictest16578-21ls20:45:000
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional