You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-01 - 04:46

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 100 highest latencies:
System rack9slot4s (updated Tue Jul 01, 2025 00:59:01)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
317049910831054,19cyclictest0-21swapper/019:27:220
299362591544,32sleep00-21swapper/019:20:070
315412561478,41sleep10-21swapper/119:23:511
3170599109101,5cyclictest0-21swapper/119:27:221
3136521020,5sleep041ktimersoftd/023:40:000
53832620,8sleep13170599cyclictest22:30:181
120132580,7sleep10-21swapper/119:55:191
206182470,8sleep03170499cyclictest23:10:180
247452381,33sleep124881-21df20:30:161
3170499364,7cyclictest81rcu_preempt23:53:290
152512361,30sleep114490-21/usr/sbin/munin22:55:271
148912360,8sleep10-21swapper/120:03:371
80502351,4sleep130944-21snmpd00:02:051
3170499351,11cyclictest81rcu_preempt20:53:460
284602351,30sleep128486-21ntp_states23:30:251
250552351,29sleep125074-21ntp_states21:55:281
31705993410,13cyclictest40950irq/93-i91521:30:191
3170499342,14cyclictest7638-21/usr/sbin/munin00:00:280
283022341,27sleep128310-21df_inode22:05:171
59652330,9sleep03170499cyclictest22:30:290
3170599334,4cyclictest81rcu_preempt22:47:551
3170499334,26cyclictest3-21ksoftirqd/022:48:380
31704993314,12cyclictest24881-21df20:30:170
31704993312,10cyclictest3-21ksoftirqd/021:17:350
307112331,27sleep130738-21hddtemp_smartct20:45:171
218102331,9sleep00-21swapper/021:45:560
168632331,5sleep117010-21df21:35:171
3170599326,4cyclictest81rcu_preempt23:59:261
31704993223,6cyclictest17182-21/usr/sbin/munin23:00:250
31704993221,7cyclictest29341-21pidof22:06:000
3170499321,8cyclictest0-21swapper/000:28:430
3170499321,7cyclictest81rcu_preempt21:30:390
3170499321,4cyclictest81rcu_preempt19:52:460
3170499321,26cyclictest2793-21latency_hist23:50:010
3170499321,18cyclictest0-21swapper/019:55:310
232752321,27sleep123316-21iostat20:25:211
204202320,7sleep10-21swapper/123:10:161
184852321,27sleep018546-21df23:05:170
86262311,5sleep00-21swapper/022:39:330
3170599315,4cyclictest81rcu_preempt23:28:391
3170499318,10cyclictest12735-21munin-run00:20:000
3170499312,15cyclictest18814-21df_inode00:35:150
3170499311,4cyclictest81rcu_preempt22:10:220
3170499311,4cyclictest81rcu_preempt00:31:350
31704993114,4cyclictest29550-21grep20:40:290
29232311,5sleep00-21swapper/022:23:530
67492301,5sleep00-21swapper/021:05:540
45612301,5sleep00-21swapper/022:26:170
31852301,5sleep00-21swapper/020:58:010
317572301,5sleep00-21swapper/020:47:530
3170599304,4cyclictest81rcu_preempt21:12:051
3170599304,10cyclictest1509-21memory19:30:031
3170599303,4cyclictest81rcu_preempt00:12:191
3170599303,11cyclictest15372-21iostat_ios00:25:221
3170599302,12cyclictest3020-21runrttasks20:53:131
31705993011,5cyclictest3020-21runrttasks00:06:231
3170499306,10cyclictest9250-21cpu00:10:090
3170499302,15cyclictest2020-21/usr/sbin/munin19:30:140
3170499302,15cyclictest0-21swapper/020:23:590
3170499302,15cyclictest0-21swapper/020:23:590
3170499302,13cyclictest22613-21df_inode00:45:160
3170499301,4cyclictest81rcu_preempt21:54:040
3170499301,27cyclictest0-21swapper/021:00:100
3170499300,4cyclictest81rcu_preempt23:27:020
281682301,5sleep00-21swapper/020:39:540
151802301,5sleep00-21swapper/020:05:030
148662301,5sleep00-21swapper/020:03:210
85482291,5sleep00-21swapper/000:05:460
71732291,5sleep00-21swapper/019:41:350
61212291,5sleep00-21swapper/023:56:480
3170599295,4cyclictest81rcu_preempt22:43:351
3170599294,4cyclictest81rcu_preempt19:39:221
3170599294,17cyclictest12912-21diskstats22:50:191
3170599291,4cyclictest81rcu_preempt23:18:041
31705992914,8cyclictest30133-21iostat_ios23:35:221
3170599291,14cyclictest9483-21df_inode21:15:151
3170599291,11cyclictest21175-21grep00:40:261
31705992910,14cyclictest1434-21cat23:45:211
3170499296,12cyclictest6204-21gdbus21:22:350
3170499295,13cyclictest20369-21sh20:17:000
3170499294,11cyclictest0-21swapper/021:25:180
3170499291,4cyclictest81rcu_preempt19:37:060
3170499291,17cyclictest17609-21ps21:35:300
279432291,5sleep10-21swapper/120:37:341
276012291,5sleep00-21swapper/022:02:270
256782291,5sleep00-21swapper/000:52:400
253542291,5sleep00-21swapper/023:22:170
241852291,5sleep00-21swapper/020:27:270
233182291,5sleep00-21swapper/023:15:510
223322291,4sleep10-21swapper/120:22:431
223322291,4sleep10-21swapper/120:22:421
187492291,5sleep00-21swapper/020:14:430
186122291,5sleep10-21swapper/121:40:061
82482281,5sleep10-21swapper/122:35:391
3170599284,4cyclictest81rcu_preempt23:22:041
3170599284,4cyclictest81rcu_preempt21:21:531
3170599283,3cyclictest81rcu_preempt23:05:451
3170599283,20cyclictest27857-21idleruntime-cro22:05:001
3170599283,10cyclictest0-21swapper/121:01:371
3170599282,4cyclictest81rcu_preempt00:32:061
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional