You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-02 - 03:36

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack9slot4s (updated Wed Jul 02, 2025 00:59:04)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
74559911631141,19cyclictest0-21swapper/121:40:311
74559911211087,30cyclictest30944-21snmpd19:26:101
72672661616,30sleep00-21swapper/019:23:370
71572493448,30sleep10-21swapper/119:22:271
745499192167,8cyclictest0-21swapper/021:40:310
745499110101,6cyclictest0-21swapper/019:26:090
2769121000,5sleep00-21swapper/023:02:120
36332930,4sleep0111rcuc/022:04:420
118802920,4sleep041ktimersoftd/021:39:510
37632910,8sleep1745599cyclictest21:30:291
284322770,4sleep0111rcuc/022:30:140
209652750,4sleep0745499cyclictest22:55:230
745499502,16cyclictest11054-21awk00:25:210
7455994519,15cyclictest18922-21grep22:20:171
7455994519,15cyclictest18922-21grep22:20:161
7455994319,8cyclictest22-21ksoftirqd/100:43:191
7455994316,7cyclictest22-21ksoftirqd/123:07:141
7455994316,15cyclictest22-21ksoftirqd/122:06:131
177742430,4sleep0745499cyclictest21:11:400
7455994216,16cyclictest22-21ksoftirqd/123:35:061
7455994216,15cyclictest22-21ksoftirqd/123:51:051
7455994216,10cyclictest22-21ksoftirqd/121:15:261
7455994215,7cyclictest22-21ksoftirqd/123:32:281
7455994211,21cyclictest22-21ksoftirqd/122:51:481
245152420,4sleep0745499cyclictest23:32:220
7455994117,7cyclictest22-21ksoftirqd/122:26:591
7455994116,15cyclictest22-21ksoftirqd/121:39:341
7455994115,6cyclictest22-21ksoftirqd/100:38:051
7455994114,7cyclictest22-21ksoftirqd/123:49:551
7455994113,17cyclictest22-21ksoftirqd/100:26:481
101562410,3sleep00-21swapper/023:50:290
7455994017,7cyclictest22-21ksoftirqd/121:13:421
7455994016,8cyclictest22-21ksoftirqd/120:28:591
7455994015,15cyclictest22-21ksoftirqd/122:48:361
7455994012,7cyclictest22-21ksoftirqd/100:22:071
7455993918,7cyclictest22-21ksoftirqd/122:59:481
7455993918,7cyclictest22-21ksoftirqd/100:45:321
7455993918,6cyclictest22-21ksoftirqd/123:55:161
7455993917,8cyclictest22-21ksoftirqd/122:12:461
7455993916,7cyclictest22-21ksoftirqd/123:25:341
7455993916,7cyclictest22-21ksoftirqd/121:27:291
7455993914,6cyclictest22-21ksoftirqd/121:48:161
7455993914,15cyclictest22-21ksoftirqd/100:04:261
7455993914,14cyclictest22-21ksoftirqd/123:41:361
7455993914,14cyclictest22-21ksoftirqd/100:31:201
7455993913,16cyclictest22-21ksoftirqd/121:52:141
745499393,7cyclictest18105-21perl19:50:170
139732390,2sleep013972-21cat23:55:180
7455993816,8cyclictest22-21ksoftirqd/121:05:461
7455993816,6cyclictest22-21ksoftirqd/119:30:251
7455993815,8cyclictest22-21ksoftirqd/120:46:361
7455993814,14cyclictest22-21ksoftirqd/123:20:421
7455993814,14cyclictest22-21ksoftirqd/122:19:221
7455993814,14cyclictest22-21ksoftirqd/100:07:021
7455993813,14cyclictest22-21ksoftirqd/122:41:561
7455993813,14cyclictest22-21ksoftirqd/122:35:591
745499388,12cyclictest17301-21diskstats23:25:200
745499382,14cyclictest24313-21ntp_states00:05:280
745499381,20cyclictest29721-21diskmemload00:10:400
745499381,11cyclictest6197-21interrupts00:20:210
7455993716,7cyclictest22-21ksoftirqd/120:09:441
7455993715,8cyclictest22-21ksoftirqd/119:41:181
7455993715,7cyclictest22-21ksoftirqd/122:03:071
7455993713,14cyclictest22-21ksoftirqd/123:16:231
7455993713,14cyclictest22-21ksoftirqd/121:20:561
7455993713,14cyclictest22-21ksoftirqd/100:53:421
7455993713,14cyclictest22-21ksoftirqd/100:18:351
7455993713,14cyclictest22-21ksoftirqd/100:18:351
7455993711,14cyclictest22-21ksoftirqd/122:31:581
745499372,16cyclictest4365-21ssh21:30:590
745499371,13cyclictest9797-21memory22:10:260
240932371,32sleep024107-21hddtemp_smartct22:25:220
7455993616,6cyclictest22-21ksoftirqd/119:35:141
7455993615,7cyclictest22-21ksoftirqd/119:47:351
7455993615,7cyclictest22-21ksoftirqd/119:47:341
7455993614,7cyclictest22-21ksoftirqd/123:12:361
7455993614,7cyclictest22-21ksoftirqd/121:55:491
7455993612,14cyclictest22-21ksoftirqd/123:01:061
198962361,4sleep020019-21ssh22:20:330
198962361,4sleep020019-21ssh22:20:330
7455993514,6cyclictest22-21ksoftirqd/100:12:221
7455993413,7cyclictest22-21ksoftirqd/120:39:111
7455993412,7cyclictest22-21ksoftirqd/120:10:051
7454993423,6cyclictest18982-21ssh21:46:050
745499341,4cyclictest81rcu_preempt00:45:290
124162340,5sleep012417-21fschecks_time19:35:170
7455993318,5cyclictest17747-21aten_r4power_vo19:50:131
745599331,10cyclictest10340-21iostat20:55:191
79402321,4sleep17874-21/usr/sbin/munin20:50:121
7455993210,6cyclictest290872chrt20:20:121
745499323,23cyclictest27487-21iostat_ios21:55:220
7454993220,8cyclictest15698-21tune2fs00:30:180
7454993219,9cyclictest4771-21ssh00:52:450
48632320,3sleep00-21swapper/023:11:590
324692321,9sleep00-21swapper/020:29:300
322132321,9sleep00-21swapper/021:26:320
745599318,5cyclictest0-21swapper/120:00:271
745599312,9cyclictest3020-21runrttasks20:34:111
745499311,4cyclictest81rcu_preempt22:07:260
745499311,4cyclictest81rcu_preempt00:43:190
745499311,24cyclictest13374-21munin-plugin-st21:05:010
203892311,4sleep020406-21diskstats00:35:170
745599302,13cyclictest12245-21iostat_ios21:00:211
745499305,5cyclictest81rcu_preempt21:15:250
745499301,15cyclictest13650-21ssh23:20:390
287922300,7sleep00-21swapper/023:36:350
745599295,5cyclictest40950irq/93-i91519:55:191
745599291,12cyclictest0-21swapper/120:15:111
745499295,4cyclictest81rcu_preempt22:46:590
745499291,4cyclictest81rcu_preempt22:54:100
745499291,4cyclictest81rcu_preempt22:37:410
745499291,4cyclictest81rcu_preempt00:19:400
745499291,4cyclictest81rcu_preempt00:19:390
7454992914,8cyclictest18860-21diskstats00:00:170
7454992911,12cyclictest3020-21runrttasks20:05:090
253722291,4sleep081rcu_preempt21:53:210
745599286,5cyclictest40950irq/93-i91520:40:281
745499282,8cyclictest7006-21python20:45:290
7454992820,5cyclictest2220-21dbus-daemon20:35:070
745499281,4cyclictest81rcu_preempt23:45:210
745499281,4cyclictest81rcu_preempt23:42:020
745499281,4cyclictest81rcu_preempt21:05:020
745499281,4cyclictest81rcu_preempt19:30:250
745499281,3cyclictest81rcu_preempt23:15:410
745499281,11cyclictest0-21swapper/023:07:140
57022281,5sleep00-21swapper/020:43:470
302422281,5sleep00-21swapper/020:20:530
270262281,5sleep00-21swapper/020:15:090
174212281,5sleep00-21swapper/022:18:380
94442271,5sleep00-21swapper/020:53:350
88932271,5sleep00-21swapper/022:43:020
745499274,17cyclictest16780-21python19:45:300
745499274,17cyclictest16780-21python19:45:290
209072271,5sleep00-21swapper/019:56:450
19312271,5sleep00-21swapper/020:34:550
745499261,4cyclictest81rcu_preempt19:41:180
252202261,5sleep00-21swapper/021:20:110
227812261,4sleep00-21swapper/020:01:330
105662261,4sleep00-21swapper/020:55:250
745499251,21cyclictest26422-21users20:10:320
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional