You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-15 - 13:16

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack9slot4s (updated Sat Nov 15, 2025 00:59:04)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
281629912901230,41cyclictest0-21swapper/019:25:070
281639912391218,15cyclictest0-21swapper/119:40:011
281639911701161,6cyclictest0-21swapper/119:25:081
280492530487,29sleep00-21swapper/019:24:270
277552468418,33sleep10-21swapper/119:21:241
2816299211186,20cyclictest2105-21sed19:40:010
94572980,3sleep0111rcuc/023:30:010
195082960,9sleep12816399cyclictest22:32:541
75482800,6sleep02816299cyclictest22:54:290
112592720,3sleep00-21swapper/022:24:540
131432570,7sleep02816299cyclictest21:52:410
28162995621,6cyclictest3-21ksoftirqd/000:18:100
155272490,8sleep02816299cyclictest00:10:000
192672410,27sleep041ktimersoftd/023:40:140
28162993922,6cyclictest3-21ksoftirqd/022:30:370
2816299385,5cyclictest81rcu_preempt23:46:100
28162993822,5cyclictest3-21ksoftirqd/000:01:130
28162993821,6cyclictest3-21ksoftirqd/023:18:390
28162993721,5cyclictest3-21ksoftirqd/021:17:150
28162993720,6cyclictest3-21ksoftirqd/021:20:220
2816299369,14cyclictest16246-21/usr/sbin/munin20:15:230
28162993619,6cyclictest3-21ksoftirqd/022:13:290
28162993619,6cyclictest3-21ksoftirqd/022:13:280
28162993619,6cyclictest3-21ksoftirqd/000:32:020
2816299361,23cyclictest14347-21ntpdc20:10:230
255062361,31sleep025516-21ssh22:39:380
233612361,32sleep123371-21ssh00:50:481
28163993517,15cyclictest22-21ksoftirqd/122:55:511
2816299355,5cyclictest81rcu_preempt22:05:130
2816299353,4cyclictest81rcu_preempt22:25:120
28162993520,5cyclictest3-21ksoftirqd/020:47:170
28162993514,9cyclictest111rcuc/000:52:270
2816399341,5cyclictest81rcu_preempt21:18:041
2816399341,4cyclictest81rcu_preempt00:35:241
2816299349,5cyclictest81rcu_preempt22:55:240
2816299346,5cyclictest81rcu_preempt00:20:250
28162993419,5cyclictest3-21ksoftirqd/019:46:170
28162993418,5cyclictest3-21ksoftirqd/019:54:320
28162993417,6cyclictest3-21ksoftirqd/021:35:080
2816299341,15cyclictest0-21swapper/000:42:570
28162993411,19cyclictest15807-21missed_timers21:55:230
257072341,29sleep124862-21/usr/sbin/munin20:40:271
212492340,4sleep1201rcuc/100:15:201
154282341,29sleep115515-21aten_r4power_vo20:15:121
324922331,28sleep132501-21sh00:26:401
2816399337,22cyclictest0-21swapper/123:22:131
2816399332,8cyclictest3020-21runrttasks22:21:331
28163993323,6cyclictest28760-21ssh22:09:221
2816399332,17cyclictest10392-21iostat_ios20:00:211
2816299337,5cyclictest81rcu_preempt23:38:010
2816299335,4cyclictest81rcu_preempt23:03:430
28162993318,5cyclictest3-21ksoftirqd/023:31:140
28162993317,5cyclictest3-21ksoftirqd/023:09:580
28162993317,5cyclictest3-21ksoftirqd/020:36:520
28162993317,5cyclictest3-21ksoftirqd/020:03:260
28162993313,6cyclictest3-21ksoftirqd/022:43:350
273932331,4sleep127403-21ssh00:21:161
241722331,28sleep124186-21ssh22:04:441
172632331,28sleep117274-21ssh21:23:421
6742320,4sleep1211ktimersoftd/121:40:141
308212321,27sleep130835-21ssh21:37:241
295302321,27sleep129615-21cpu22:10:131
295302321,27sleep129615-21cpu22:10:131
2816399329,10cyclictest31635-21ntpq19:30:251
2816399321,9cyclictest16621-21sh21:55:451
2816399321,4cyclictest81rcu_preempt23:28:501
2816399321,18cyclictest24298-21ssh22:37:521
2816299328,20cyclictest23608-21df_inode21:30:180
2816299327,4cyclictest81rcu_preempt00:35:200
28162993218,4cyclictest3-21ksoftirqd/022:18:290
28162993216,6cyclictest3-21ksoftirqd/020:21:490
2816299321,18cyclictest17046-21wc00:10:300
209812320,5sleep1201rcuc/123:08:001
37182311,3sleep126715-1kworker/1:2H22:15:591
299032311,5sleep00-21swapper/020:50:340
289032311,26sleep128912-21df_inode20:50:151
28163993122,5cyclictest12090-21ssh23:32:031
2816399311,4cyclictest81rcu_preempt21:51:041
2816399311,4cyclictest81rcu_preempt21:14:351
28163993110,10cyclictest22-21ksoftirqd/123:11:441
2816399310,4cyclictest81rcu_preempt23:56:501
2816299317,5cyclictest81rcu_preempt23:11:050
2816299317,4cyclictest81rcu_preempt22:01:410
2816299313,4cyclictest81rcu_preempt22:45:530
28162993116,5cyclictest3-21ksoftirqd/000:46:020
28162993115,5cyclictest3-21ksoftirqd/021:44:200
28162993115,5cyclictest3-21ksoftirqd/021:27:590
28162993115,5cyclictest3-21ksoftirqd/019:56:280
28162993112,4cyclictest4186-21cstates21:10:140
28162993112,4cyclictest40950irq/93-i91519:30:250
28162993111,7cyclictest572-21kernelversion21:00:230
21292311,26sleep12136-21diskstats21:05:191
15102313,25sleep126715-1kworker/1:2H22:47:541
36382301,5sleep10-21swapper/119:41:351
36252301,5sleep00-21swapper/021:09:450
2816399307,9cyclictest13337-21rm23:00:151
2816399303,14cyclictest6698-21hddtemp_smartct00:00:171
28163993021,6cyclictest6204-21gdbus19:55:201
2816399301,8cyclictest14618-21ssh00:08:431
28163993017,9cyclictest22-21ksoftirqd/123:37:571
2816399301,4cyclictest81rcu_preempt23:15:441
2816399301,4cyclictest81rcu_preempt19:47:431
2816399300,4cyclictest81rcu_preempt21:49:351
2816299306,20cyclictest0-21swapper/023:53:500
28162993022,4cyclictest12508-21ntp_states20:05:280
2816299301,15cyclictest2158-21/usr/sbin/munin19:40:160
207522301,5sleep00-21swapper/020:28:160
185612301,25sleep118571-21ssh00:12:271
146572301,5sleep10-21swapper/100:41:431
90792291,5sleep00-21swapper/021:48:480
319842291,5sleep00-21swapper/020:57:380
2816399299,16cyclictest0-21swapper/123:46:421
2816399293,4cyclictest81rcu_preempt20:13:211
2816399293,4cyclictest81rcu_preempt00:46:161
28163992919,6cyclictest12992-21grep22:25:301
28163992917,8cyclictest22-21ksoftirqd/100:32:381
28163992917,7cyclictest21684-21iostat_ios20:30:201
28163992916,9cyclictest22-21ksoftirqd/123:54:051
28163992916,9cyclictest22-21ksoftirqd/121:27:561
2816399291,18cyclictest40950irq/93-i91522:50:241
2816299291,8cyclictest3775-21ssh23:56:590
28162992914,5cyclictest3-21ksoftirqd/020:30:490
246432291,5sleep10-21swapper/120:39:431
233162291,5sleep10-21swapper/123:44:231
2816399289,15cyclictest0-21swapper/121:30:591
2816399283,4cyclictest81rcu_preempt20:59:011
2816399281,7cyclictest27518-21grep22:40:321
2816299281,14cyclictest25356-21hddtemp_smartct20:40:180
2816299281,13cyclictest30256-21/usr/sbin/munin00:25:230
133772281,5sleep10-21swapper/120:09:571
77262271,5sleep10-21swapper/119:54:591
282542271,5sleep10-21swapper/120:47:551
2816399273,12cyclictest19499-21df_inode20:25:151
187902271,5sleep10-21swapper/120:22:231
12602271,5sleep10-21swapper/121:00:591
28162992613,4cyclictest81rcu_preempt23:23:560
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional