You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-29 - 12:20

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack9slot4s (updated Sat Nov 29, 2025 00:59:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
46919912021156,26cyclictest23020-21irqstats20:10:221
46919911481131,14cyclictest0-21swapper/119:25:101
46909911401109,21cyclictest0-21swapper/019:25:110
43952650605,30sleep00-21swapper/019:22:310
43952650605,30sleep00-21swapper/019:22:310
46152577531,31sleep10-21swapper/119:24:511
46152577531,31sleep10-21swapper/119:24:501
469099174149,22cyclictest0-21swapper/020:10:220
2065321150,29sleep00-21swapper/021:50:420
299942950,21sleep03-21ksoftirqd/021:27:190
85282850,6sleep0469099cyclictest21:39:080
85282850,6sleep0469099cyclictest21:39:070
189562850,8sleep00-21swapper/020:00:140
125952760,4sleep0111rcuc/021:43:000
125952760,4sleep0111rcuc/021:43:000
193862650,5sleep14688-21cyclictest21:50:161
314752450,5sleep1469199cyclictest00:50:261
4691994322,9cyclictest22-21ksoftirqd/121:56:261
4690994325,7cyclictest28813-21wget22:00:120
209322430,3sleep1201rcuc/120:05:161
309562420,4sleep0111rcuc/022:35:230
4690994124,6cyclictest31855-21ssh23:10:160
4690994025,5cyclictest0-21swapper/000:20:200
4690994024,6cyclictest11407-21ls00:30:010
4690994023,5cyclictest18173-21ssh00:35:560
4690993925,5cyclictest10744-21sort21:05:000
4690993924,5cyclictest6299-21df22:10:140
4690993924,10cyclictest12273-21aten_r4power_en22:50:110
4690993923,5cyclictest27029-21df_inode23:05:150
4690993923,12cyclictest23974-21iostat_ios23:35:220
4690993824,10cyclictest11776-21ntp_states22:15:260
4690993822,6cyclictest24700-21/usr/sbin/munin21:55:210
4690993817,7cyclictest3-21ksoftirqd/023:22:540
4690993816,7cyclictest3-21ksoftirqd/022:33:550
160352381,32sleep016046-21ssh21:46:010
469199375,7cyclictest0-21swapper/123:48:041
4690993723,5cyclictest3020-21runrttasks22:55:140
4690993723,5cyclictest3020-21runrttasks22:55:130
4690993722,5cyclictest5676-21ls20:50:190
4690993722,5cyclictest12081-21latency_hist22:50:020
4690993721,6cyclictest22121-21grep23:00:140
4690993721,6cyclictest12185-21ls19:40:250
469199366,5cyclictest40950irq/93-i91521:15:231
4690993622,5cyclictest21701-21timerandwakeup22:25:270
4690993621,5cyclictest1997-21ntp_kernel_pll_22:05:240
4690993621,5cyclictest0-21swapper/019:30:120
4690993620,6cyclictest0-21swapper/020:20:230
4690993620,4cyclictest30754-21wget00:50:160
4690993613,8cyclictest27124-21ssh00:11:580
156352361,31sleep015646-21ssh00:34:160
469199353,13cyclictest22794-21df_inode21:20:141
469199352,14cyclictest32591-21fschecks_time21:30:171
469199351,14cyclictest23978-21sh22:28:311
469099359,6cyclictest30068-21egrep20:30:150
4690993522,9cyclictest2302-21python20:40:270
4690993522,5cyclictest0-21swapper/023:17:370
4690993522,4cyclictest11187-21ssh23:55:250
4690993519,8cyclictest0-21swapper/022:43:010
4690993515,10cyclictest0-21swapper/020:48:540
4690993513,6cyclictest3-21ksoftirqd/020:05:180
324262351,30sleep132430-21irqstats20:35:231
469199347,23cyclictest0-21swapper/123:28:221
469199347,11cyclictest40950irq/93-i91522:00:121
469199343,14cyclictest11617-21/usr/sbin/munin00:30:321
4691993422,9cyclictest24809-21/usr/sbin/munin20:15:181
4690993421,9cyclictest12680-21ssh21:10:010
4690993421,5cyclictest0-21swapper/021:24:330
4690993421,4cyclictest0-21swapper/000:43:480
4690993420,5cyclictest0-21swapper/021:06:590
4690993419,4cyclictest6347-21sh23:50:250
4690993418,8cyclictest0-21swapper/021:32:040
4690993417,5cyclictest19162-21iostat_ios23:30:190
4690993417,5cyclictest15204-21grep19:50:170
4690993414,6cyclictest3-21ksoftirqd/023:29:350
469199335,13cyclictest40950irq/93-i91521:45:301
469199335,13cyclictest40950irq/93-i91500:47:241
469199335,13cyclictest40950irq/93-i91500:47:231
469199332,5cyclictest0-21swapper/121:12:001
469199332,14cyclictest3295-21ssh22:06:281
469199332,13cyclictest0-21swapper/100:22:401
4691993319,9cyclictest10867-21users21:40:311
4691993319,9cyclictest10867-21users21:40:301
4691993313,4cyclictest22-21ksoftirqd/123:37:521
4690993320,5cyclictest0-21swapper/020:35:150
4690993319,4cyclictest0-21swapper/000:06:100
4690993318,5cyclictest30768-21timerandwakeup00:15:300
4690993312,7cyclictest3-21ksoftirqd/022:24:190
4690993312,7cyclictest3-21ksoftirqd/021:16:140
4690993311,7cyclictest3-21ksoftirqd/000:47:050
4690993311,7cyclictest3-21ksoftirqd/000:47:040
4690993310,7cyclictest4688-21cyclictest19:57:260
33062331,4sleep13389-21memory22:40:221
255082331,28sleep025558-21who20:15:310
187852331,27sleep118804-21diskstats23:30:151
172892331,28sleep117353-21iostat19:55:191
469199326,10cyclictest40950irq/93-i91523:50:141
469199323,5cyclictest40950irq/93-i91522:36:461
469199322,5cyclictest11989-21ssh23:55:571
469199321,16cyclictest0-21swapper/121:28:171
469199321,15cyclictest30944-21snmpd00:07:591
469199321,14cyclictest6803-21ssh00:25:101
469199321,13cyclictest12749-21ssh23:24:031
4691993210,4cyclictest40950irq/93-i91523:01:401
4691993210,18cyclictest0-21swapper/122:10:561
469099329,8cyclictest3-21ksoftirqd/019:49:010
4690993219,5cyclictest0-21swapper/000:04:020
4690993218,6cyclictest0-21swapper/023:48:480
4690993218,6cyclictest0-21swapper/023:42:310
4690993218,5cyclictest0-21swapper/019:35:240
469099321,10cyclictest0-21swapper/020:25:130
43522320,8sleep1469199cyclictest23:15:151
286302320,4sleep1201rcuc/123:40:181
26852321,5sleep10-21swapper/123:13:371
160002321,27sleep116118-21fschecks_time22:20:171
469199318,5cyclictest40950irq/93-i91500:12:271
469199315,4cyclictest40950irq/93-i91500:42:341
469199315,10cyclictest40950irq/93-i91522:56:391
469199315,10cyclictest40950irq/93-i91522:56:381
4691993121,6cyclictest15492-21cut00:00:161
4690993121,8cyclictest0-21swapper/020:57:180
469199309,5cyclictest40950irq/93-i91500:39:311
469199308,5cyclictest40950irq/93-i91522:31:551
469199308,5cyclictest40950irq/93-i91521:35:501
469199308,5cyclictest40950irq/93-i91521:35:491
469199308,5cyclictest40950irq/93-i91519:50:171
469199305,22cyclictest0-21swapper/100:16:511
469199304,6cyclictest13030-21memory22:50:221
469199301,9cyclictest6309-21unixbench_multi20:50:291
4691993010,5cyclictest40950irq/93-i91522:17:551
289142301,5sleep10-21swapper/123:06:371
469199298,5cyclictest40950irq/93-i91520:55:201
469199297,5cyclictest40950irq/93-i91522:45:541
469199297,5cyclictest40950irq/93-i91521:05:221
469199284,22cyclictest0-21swapper/119:44:011
4691992819,5cyclictest22-21ksoftirqd/121:00:211
316152281,5sleep10-21swapper/120:34:341
293762281,5sleep10-21swapper/120:25:501
10882281,5sleep10-21swapper/120:40:011
278132271,5sleep10-21swapper/120:24:031
146762271,5sleep10-21swapper/119:49:341
87022263,19sleep14688-21cyclictest19:31:171
469199262,20cyclictest0-21swapper/119:35:231
469199252,17cyclictest3757-21hddtemp_smartct20:45:191
4691992416,5cyclictest22-21ksoftirqd/120:03:001
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional