You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-30 - 04:50

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack9slot4s (updated Mon Mar 30, 2026 00:59:02)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
190559912671237,17cyclictest0-21swapper/019:26:570
190569910941085,6cyclictest0-21swapper/119:27:401
186142534489,30sleep00-21swapper/019:20:580
179812490442,32sleep10-21swapper/119:20:221
723421080,5sleep1211ktimersoftd/121:55:281
156832920,8sleep01905599cyclictest00:18:120
40252870,8sleep01905599cyclictest21:52:210
1905699871,29cyclictest11582-21ssh00:47:351
80552820,8sleep01905599cyclictest22:30:160
1905699781,35cyclictest27058-21ssh00:30:131
1905699781,34cyclictest3964-21awk00:40:001
1905699751,32cyclictest24749-21taskset23:20:231
1905699741,33cyclictest26847-21diskmemload00:20:121
1905699721,38cyclictest23256-21sh00:25:271
1905699721,37cyclictest7609-21sh23:35:411
1905699721,37cyclictest26847-21diskmemload22:25:121
1905699721,37cyclictest26847-21diskmemload00:45:001
1905699721,37cyclictest16884-21ssh21:32:131
1905699721,36cyclictest16975-21mailstats22:05:271
1905699721,32cyclictest11807-21sh23:40:241
1905699721,30cyclictest26372-21strings22:15:231
1905699718,27cyclictest17212-21latency_hist22:40:001
1905699712,37cyclictest29324-21perl21:45:171
1905699711,39cyclictest4527-21ssh00:06:011
1905699711,38cyclictest6069-21ssh21:20:371
1905699711,36cyclictest8707-21sh00:10:271
1905699711,35cyclictest10181-21ssh23:05:231
1905699711,33cyclictest11198-21timerwakeupswit20:25:271
1905699701,37cyclictest32399-21uniq22:55:141
1905699701,37cyclictest30316-21sh21:13:011
1905699701,36cyclictest25650-21chrt21:40:511
1905699701,36cyclictest22972-21sh21:39:011
1905699701,36cyclictest18299-21netstat22:40:241
1905699701,35cyclictest27261-21sh22:50:101
1905699701,34cyclictest28237-21/usr/sbin/munin19:45:261
1905699701,34cyclictest12305-21fschecks_count20:30:171
1905699701,33cyclictest6410-21ssh23:01:131
1905699701,33cyclictest26847-21diskmemload22:32:251
1905699701,33cyclictest16293-21ssh23:11:321
1905699701,33cyclictest12007-21sh22:00:251
1905699701,32cyclictest14920-21ssh00:50:501
33482690,3sleep041ktimersoftd/020:05:230
1905699693,33cyclictest1034-21hddtemp_smartct20:00:171
1905699692,34cyclictest3302-21runrttasks22:11:191
1905699692,33cyclictest22861-21if_err_eth022:45:191
1905699692,31cyclictest16009-21df_inode20:40:131
1905699692,30cyclictest9750-21df_inode21:25:171
1905699691,36cyclictest1840-21ssh21:50:191
1905699691,36cyclictest1659-21runrttasks22:23:491
1905699691,35cyclictest6837-21idleruntime20:15:181
1905699691,35cyclictest25369-21cstates21:05:131
1905699691,34cyclictest32178-21cpuspeed_turbos21:15:131
1905699691,34cyclictest30537-21chrt19:50:451
1905699691,33cyclictest21811-21ls23:50:281
1905699682,32cyclictest31678-21if_eth019:55:171
1905699681,35cyclictest19716-21cstates20:50:131
1905699681,34cyclictest3020-21sh23:31:061
1905699681,34cyclictest14178-21fschecks_count20:35:141
1905699681,33cyclictest3302-21runrttasks23:25:181
1905699681,33cyclictest24340-21ls21:00:291
1905699681,33cyclictest23027-21chrt19:30:541
1905699681,33cyclictest21074-21runrttasks23:16:271
1905699681,33cyclictest18130-21ls20:45:171
1905699681,32cyclictest9032-21ntp_states20:20:241
1905699681,30cyclictest26847-21diskmemload00:15:271
1905699671,35cyclictest19317-21runrttasks23:48:371
1905699671,34cyclictest31208-21sh00:00:251
1905699671,34cyclictest24681-21sed19:35:281
1905699671,33cyclictest29367-21sh23:59:121
1905699671,31cyclictest26061-21cat19:40:201
1905699662,30cyclictest3104-21cat20:05:171
1905699661,36cyclictest4431-21/usr/sbin/munin20:10:231
1905699661,31cyclictest22718-21seq20:56:241
294862500,4sleep041ktimersoftd/000:32:090
1905599451,13cyclictest81rcu_preempt23:35:170
1905599413,4cyclictest81rcu_preempt00:50:460
1905599406,5cyclictest0-21swapper/022:51:560
19055994016,5cyclictest0-21swapper/000:21:330
19055994013,13cyclictest19609-21hddtemp_smartct21:35:150
1905599401,11cyclictest0-21swapper/021:13:230
19055993910,9cyclictest0-21swapper/022:15:350
259172381,4sleep025928-21ssh00:29:010
279832370,7sleep01905599cyclictest23:24:240
194382378,3sleep00-21swapper/023:48:480
1905599377,27cyclictest0-21swapper/022:37:150
243182361,30sleep024387-21ntp_kernel_pll_19:35:230
1905599368,24cyclictest0-21swapper/023:43:530
1905599367,26cyclictest1968-21lldpad23:34:050
85062351,30sleep07540-21/usr/sbin/munin00:10:270
1905599356,4cyclictest81rcu_preempt21:57:020
1905599355,5cyclictest81rcu_preempt22:45:280
298142341,29sleep029920-21cut21:45:280
1905599345,4cyclictest81rcu_preempt23:00:100
1905599344,4cyclictest81rcu_preempt00:43:350
1905599344,4cyclictest81rcu_preempt00:00:430
1905599342,4cyclictest81rcu_preempt21:20:360
1905599342,29cyclictest0-21swapper/023:10:350
1905599342,29cyclictest0-21swapper/022:58:350
1905599341,8cyclictest0-21swapper/023:55:130
1905599341,30cyclictest0-21swapper/021:15:550
1905599341,19cyclictest24873-21ssh22:14:300
1905599335,4cyclictest81rcu_preempt23:10:010
1905599334,5cyclictest81rcu_preempt00:05:240
1905599334,4cyclictest81rcu_preempt22:26:460
1905599333,4cyclictest81rcu_preempt22:20:040
1905599333,4cyclictest81rcu_preempt21:28:400
1905599333,19cyclictest0-21swapper/023:19:550
1905599332,28cyclictest0-21swapper/022:03:160
1905599331,29cyclictest2403-21irqbalance00:47:150
1905599331,15cyclictest81rcu_preempt23:29:230
15682330,8sleep01905599cyclictest20:00:270
1905599323,4cyclictest81rcu_preempt23:51:330
1905599323,4cyclictest81rcu_preempt22:41:340
1905599322,4cyclictest81rcu_preempt21:40:120
19055993220,8cyclictest10150irq/91-xhci_hcd19:55:000
1905599321,5cyclictest0-21swapper/022:05:560
1905599321,5cyclictest0-21swapper/020:13:550
19055993214,6cyclictest14546-21diskstats21:30:150
1905599321,29cyclictest0-21swapper/021:05:360
1905599321,29cyclictest0-21swapper/020:53:560
19055993212,7cyclictest12006-21grep20:30:120
19055993124,5cyclictest0-21swapper/020:44:360
1905599312,12cyclictest32411-21irqstats00:35:210
1905599311,4cyclictest0-21swapper/020:26:560
1905599311,14cyclictest31611-21/usr/sbin/munin19:55:150
96602301,5sleep00-21swapper/020:22:220
1905599304,4cyclictest81rcu_preempt20:36:450
1905599303,4cyclictest81rcu_preempt19:32:000
273502291,5sleep00-21swapper/019:45:070
268322291,5sleep00-21swapper/019:41:140
1905599291,4cyclictest81rcu_preempt20:45:140
1905599282,13cyclictest6896-21iostat20:15:200
1905599279,5cyclictest21930-21/usr/sbin/munin20:55:170
1905599262,12cyclictest23748-21hddtemp_smartct21:00:180
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional