You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-16 - 06:40

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack9slot4s (updated Sun Nov 16, 2025 00:59:03)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
162549912941,1275cyclictest20745-21cat19:35:001
162549911011092,6cyclictest0-21swapper/119:25:071
157722577531,30sleep10-21swapper/119:20:401
158642545499,31sleep00-21swapper/019:21:380
1625399250231,3cyclictest0-21swapper/019:35:010
93732970,5sleep19372-21ssh22:00:331
16253999773,21cyclictest0-21swapper/019:25:070
18322770,3sleep041ktimersoftd/023:34:460
318112700,5sleep00-21swapper/000:39:090
36362530,5sleep01625399cyclictest00:10:010
216822451,25sleep121796-21cut19:35:301
87202420,4sleep11625499cyclictest23:07:361
284332411,11sleep00-21swapper/022:55:020
16253994124,6cyclictest3-21ksoftirqd/023:52:150
16253994123,6cyclictest3-21ksoftirqd/022:16:470
16253994018,7cyclictest3-21ksoftirqd/020:10:260
191632391,4sleep019232-21df22:45:150
16253993923,5cyclictest3-21ksoftirqd/021:27:380
16253993923,5cyclictest3-21ksoftirqd/000:30:470
16253993922,6cyclictest3-21ksoftirqd/023:17:340
16253993921,6cyclictest3-21ksoftirqd/021:48:340
16253993921,6cyclictest3-21ksoftirqd/020:43:230
1625499386,4cyclictest81rcu_preempt00:09:471
16253993822,6cyclictest3-21ksoftirqd/020:37:350
120432381,33sleep112053-21ssh00:51:541
178022370,2sleep117803-21ntp_kernel_pll_23:50:251
1625499374,4cyclictest81rcu_preempt20:10:271
16253993722,5cyclictest3-21ksoftirqd/023:20:400
16253993721,6cyclictest3-21ksoftirqd/021:24:350
16253993714,6cyclictest5533-21cut00:45:330
16253993712,9cyclictest20309-21sh00:26:240
80432360,8sleep01625399cyclictest23:06:360
272982361,4sleep127312-21ssh21:45:361
16253993621,5cyclictest3-21ksoftirqd/019:38:220
16253993619,6cyclictest3-21ksoftirqd/000:42:230
16253993617,12cyclictest21504-21python21:00:290
92632351,4sleep09371-21users22:00:340
322422350,3sleep00-21swapper/022:25:140
312712351,30sleep031282-21iostat_ios21:50:220
1625499358,23cyclictest0-21swapper/123:18:571
1625399352,18cyclictest2396-21users23:00:320
16253993519,6cyclictest3-21ksoftirqd/021:08:510
1625399351,7cyclictest5119-21iostat22:30:200
16253993510,11cyclictest26285-21pidof19:49:590
74812341,30sleep17495-21ssh00:13:501
283122341,4sleep128325-21diskmemload00:01:021
1625499345,5cyclictest81rcu_preempt19:50:131
16253993419,5cyclictest3-21ksoftirqd/000:15:470
16253993412,7cyclictest3-21ksoftirqd/022:54:370
16253993412,7cyclictest3-21ksoftirqd/022:10:330
16253993412,7cyclictest3-21ksoftirqd/020:54:480
16253993411,7cyclictest3-21ksoftirqd/020:58:150
150542341,4sleep015116-21df20:45:170
306062330,3sleep130618-21ssh22:23:251
306062330,3sleep130618-21ssh22:23:251
250662331,3sleep125127-21if_eth019:45:201
1625499335,24cyclictest0-21swapper/121:38:401
1625399339,13cyclictest4908-21diskmemload23:37:030
16253993314,6cyclictest16577-21cron23:50:000
88222321,9sleep00-21swapper/023:40:440
39712321,5sleep10-21swapper/123:35:431
322872321,27sleep132289-21ssh00:39:521
1625499325,4cyclictest81rcu_preempt20:58:161
1625499324,4cyclictest81rcu_preempt20:54:481
1625499321,4cyclictest81rcu_preempt23:02:361
1625499321,27cyclictest0-21swapper/123:32:591
16254993212,16cyclictest22-21ksoftirqd/121:34:561
1625499321,13cyclictest218132missed_timers21:40:261
1625399328,8cyclictest3-21ksoftirqd/021:40:100
1625399322,26cyclictest3810-21aten_r4power_cu00:10:140
16253993216,5cyclictest3-21ksoftirqd/019:41:310
16253993215,5cyclictest40950irq/93-i91522:41:540
1625399321,18cyclictest30057-21sendmail20:00:010
16253993211,7cyclictest3-21ksoftirqd/023:25:420
1625399321,11cyclictest0-21swapper/021:55:390
16253993210,7cyclictest3-21ksoftirqd/020:31:090
16253993210,17cyclictest996-21taskset20:06:220
16253993210,17cyclictest13264-21rm23:12:110
135502321,4sleep113554-21irqstats20:40:231
1625499318,19cyclictest0-21swapper/100:30:361
1625499315,22cyclictest0-21swapper/100:18:451
1625499314,4cyclictest81rcu_preempt21:28:101
1625499313,4cyclictest81rcu_preempt22:10:331
1625499312,7cyclictest0-21swapper/122:54:471
1625499311,4cyclictest81rcu_preempt23:25:421
1625499311,4cyclictest81rcu_preempt21:54:221
1625499311,4cyclictest81rcu_preempt20:08:451
1625399319,7cyclictest3-21ksoftirqd/019:53:270
1625399318,12cyclictest3-21ksoftirqd/021:39:410
1625399312,14cyclictest25038-21iostat_ios21:10:190
1625399311,8cyclictest15907-21ssh00:22:020
16253993113,7cyclictest23164-21chrt23:55:330
16253993113,5cyclictest7141-21munin-run20:25:000
16253993113,15cyclictest3-21ksoftirqd/022:06:570
116772311,5sleep00-21swapper/000:51:190
40152301,5sleep10-21swapper/121:21:481
326312301,5sleep10-21swapper/122:59:221
1625499305,4cyclictest81rcu_preempt21:03:041
1625499303,4cyclictest81rcu_preempt21:56:101
1625499302,6cyclictest0-21swapper/123:21:411
16254993013,13cyclictest0-21swapper/122:37:461
1625499301,24cyclictest12971-21/usr/sbin/munin23:45:251
16254993011,15cyclictest22-21ksoftirqd/100:43:301
1625499300,4cyclictest81rcu_preempt22:26:501
1625399308,7cyclictest3-21ksoftirqd/022:22:060
1625399308,7cyclictest3-21ksoftirqd/022:22:050
16253993010,7cyclictest3-21ksoftirqd/021:15:560
88502291,5sleep00-21swapper/020:28:230
50752291,5sleep00-21swapper/020:18:260
282502291,5sleep00-21swapper/000:00:570
1625499299,16cyclictest0-21swapper/123:40:541
1625499299,16cyclictest0-21swapper/122:34:251
1625499299,16cyclictest0-21swapper/121:13:561
1625499299,16cyclictest0-21swapper/100:47:111
1625499299,16cyclictest0-21swapper/100:28:181
1625499294,4cyclictest81rcu_preempt19:57:191
1625499292,6cyclictest81rcu_preempt23:55:291
16254992916,4cyclictest22-21ksoftirqd/121:15:271
1625399298,5cyclictest24094-21diskmemload22:39:560
1625399296,11cyclictest0-21swapper/020:00:260
1625399294,10cyclictest6204-21gdbus21:33:370
137012291,5sleep10-21swapper/123:12:521
1625499289,15cyclictest0-21swapper/122:45:521
1625499284,18cyclictest22967-21iostat_ios21:05:221
1625499283,5cyclictest0-21swapper/122:44:141
1625499283,19cyclictest11349-21diskstats20:35:181
1625499282,4cyclictest81rcu_preempt20:22:061
1625499281,4cyclictest81rcu_preempt20:32:231
16254992813,11cyclictest0-21swapper/120:00:231
1625499281,23cyclictest13676-21ntp_states22:05:271
16254992810,14cyclictest14095-21runrttasks00:20:211
88052271,5sleep10-21swapper/120:27:561
243442271,5sleep10-21swapper/119:44:271
1625499268,15cyclictest0-21swapper/122:15:261
16254992514,6cyclictest22-21ksoftirqd/120:17:171
1625499246,4cyclictest40950irq/93-i91520:45:151
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional