You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-02 - 11:17

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack9slot4s (updated Sun Nov 02, 2025 00:59:03)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
65419912341203,21cyclictest0-21swapper/019:25:050
65429912061175,27cyclictest0-21swapper/119:25:011
65429911121090,19cyclictest0-21swapper/119:35:001
54522548500,31sleep00-21swapper/019:20:230
654299234227,5cyclictest0-21swapper/119:25:061
65419911973,42cyclictest10977-21sed19:35:000
107821130,4sleep00-21swapper/023:43:350
27802980,3sleep02341-21/usr/sbin/munin23:45:160
19382940,3sleep03-21ksoftirqd/000:17:500
202522920,5sleep041ktimersoftd/000:03:160
305352810,8sleep1654299cyclictest23:40:171
176942550,5sleep0654199cyclictest21:12:490
176942550,5sleep0654199cyclictest21:12:490
271242490,7sleep1654299cyclictest20:15:291
112862490,4sleep0111rcuc/021:00:200
207392461,4sleep020920-21sed21:15:300
186292450,4sleep041ktimersoftd/022:54:590
186742441,30sleep018840-21perl00:35:210
6541994117,8cyclictest3-21ksoftirqd/021:28:240
6541994024,6cyclictest3-21ksoftirqd/021:40:300
6541994016,8cyclictest3-21ksoftirqd/023:07:020
654299396,4cyclictest81rcu_preempt23:45:121
6541993923,6cyclictest3-21ksoftirqd/022:43:230
6541993922,5cyclictest3-21ksoftirqd/019:47:200
654299386,4cyclictest81rcu_preempt21:28:241
6541993822,6cyclictest3-21ksoftirqd/022:27:530
6541993822,5cyclictest3-21ksoftirqd/021:50:040
6541993822,5cyclictest3-21ksoftirqd/020:42:110
6541993822,5cyclictest3-21ksoftirqd/019:42:320
6541993821,5cyclictest3-21ksoftirqd/020:31:370
6541993816,7cyclictest3-21ksoftirqd/021:36:100
654299375,4cyclictest81rcu_preempt23:07:021
6541993721,6cyclictest3-21ksoftirqd/021:05:080
6541993721,5cyclictest3-21ksoftirqd/020:23:520
197702371,4sleep119780-21ssh00:02:361
6541993621,5cyclictest3-21ksoftirqd/023:28:550
6541993620,6cyclictest3-21ksoftirqd/022:46:320
6541993620,5cyclictest3-21ksoftirqd/022:06:470
6541993620,5cyclictest3-21ksoftirqd/020:39:030
6541993620,5cyclictest3-21ksoftirqd/000:52:390
654299359,22cyclictest0-21swapper/123:26:371
654299359,22cyclictest0-21swapper/122:31:401
654299355,26cyclictest17168-21tune2fs19:50:161
654299353,5cyclictest81rcu_preempt21:36:101
654199359,7cyclictest111rcuc/000:22:310
654199356,14cyclictest23144-21cut20:05:250
6541993520,5cyclictest3-21ksoftirqd/023:56:490
6541993520,5cyclictest3-21ksoftirqd/023:16:450
6541993520,5cyclictest3-21ksoftirqd/023:04:430
6541993520,5cyclictest3-21ksoftirqd/020:51:210
6541993520,5cyclictest3-21ksoftirqd/000:12:460
6541993519,5cyclictest3-21ksoftirqd/000:44:160
6541993518,6cyclictest3-21ksoftirqd/023:14:250
654199351,20cyclictest16841-21ssh22:19:280
654199351,14cyclictest0-21swapper/000:47:440
37122351,4sleep13743-21iostat_ios22:05:231
106162351,30sleep010627-21ssh00:26:160
654299342,19cyclictest0-21swapper/123:53:281
654299341,14cyclictest66582rm00:22:311
654199349,8cyclictest7664-21fschecks_count23:50:160
654199349,12cyclictest31240-21sh22:34:200
654199341,6cyclictest0-21swapper/020:12:140
6541993412,17cyclictest9162-21df_inode20:55:150
654199341,20cyclictest32766-21latency22:35:250
151452341,4sleep115152-21diskstats19:45:181
134512341,4sleep130944-21snmpd21:41:331
102352341,29sleep130944-21snmpd20:56:231
654299336,23cyclictest0-21swapper/122:42:321
654299331,7cyclictest6323-21latency_hist21:35:001
6541993317,6cyclictest3-21ksoftirqd/021:22:320
6541993315,5cyclictest0-21swapper/021:45:360
6541993311,6cyclictest3-21ksoftirqd/021:35:000
654199331,11cyclictest0-21swapper/022:00:180
654299326,4cyclictest81rcu_preempt21:53:431
654299321,4cyclictest81rcu_preempt00:43:591
6542993213,16cyclictest22-21ksoftirqd/121:15:301
654299321,13cyclictest188762ssh21:14:321
654299321,13cyclictest188762ssh21:14:321
654199328,7cyclictest17344-21kernelversion19:50:220
6541993212,10cyclictest13799-21df00:30:160
281472321,5sleep10-21swapper/100:10:461
200372321,5sleep10-21swapper/121:49:071
654299315,4cyclictest81rcu_preempt20:35:341
654299311,8cyclictest17524-21kworker/1:123:38:191
6542993118,10cyclictest22-21ksoftirqd/100:17:061
654199318,7cyclictest3-21ksoftirqd/019:36:550
654199317,8cyclictest3-21ksoftirqd/000:07:040
654199312,15cyclictest3020-21runrttasks21:59:120
6541993116,5cyclictest3-21ksoftirqd/020:19:040
118722311,4sleep111907-21ntp_states19:35:251
118222311,3sleep111832-21ssh22:47:051
106052311,4sleep110615-21ssh23:20:041
654299305,5cyclictest40950irq/93-i91500:30:161
654299301,7cyclictest6645-21memory23:15:231
6542993014,12cyclictest24411-21cut21:20:131
654299301,25cyclictest23946-21ssh00:06:391
6542993012,12cyclictest40950irq/93-i91521:59:441
654299300,4cyclictest81rcu_preempt00:37:011
654199308,6cyclictest3-21ksoftirqd/019:56:440
654199308,12cyclictest3-21ksoftirqd/022:55:420
6541993015,5cyclictest11505-21sh23:20:230
6541993011,6cyclictest3-21ksoftirqd/022:10:580
654199301,15cyclictest26116-21sh23:35:260
229822301,5sleep10-21swapper/123:32:171
124342301,4sleep10-21swapper/121:04:301
66942291,5sleep10-21swapper/120:48:361
654299299,16cyclictest0-21swapper/122:00:531
654299296,4cyclictest81rcu_preempt20:21:101
654299294,4cyclictest81rcu_preempt19:56:441
654299294,3cyclictest81rcu_preempt00:28:261
6542992919,6cyclictest22-21ksoftirqd/100:54:121
6542992917,9cyclictest22-21ksoftirqd/123:55:311
6542992916,5cyclictest22-21ksoftirqd/122:20:281
6542992915,10cyclictest22-21ksoftirqd/122:54:371
654299291,23cyclictest1242-21diskstats23:10:161
6542992911,14cyclictest24382-21ssh22:26:231
654199297,7cyclictest3-21ksoftirqd/020:04:230
654199293,7cyclictest40950irq/93-i91522:20:210
50972291,5sleep00-21swapper/020:45:020
314062291,5sleep10-21swapper/120:27:371
312432291,5sleep00-21swapper/020:25:580
285762291,5sleep10-21swapper/100:45:241
144992291,5sleep10-21swapper/119:43:011
654299289,15cyclictest0-21swapper/122:10:161
654299288,16cyclictest0-21swapper/123:04:241
654299285,4cyclictest81rcu_preempt20:52:101
654299283,18cyclictest3831-21iostat_ios20:40:211
6542992818,6cyclictest12744-21grep22:15:121
654299281,23cyclictest19402-21grep22:55:171
6542992810,14cyclictest32527-21grep22:35:191
654199289,4cyclictest0-21swapper/023:30:150
257972281,5sleep10-21swapper/120:13:081
142952281,5sleep10-21swapper/121:09:191
7882271,5sleep10-21swapper/120:32:191
654299271,4cyclictest81rcu_preempt20:04:231
6542992617,6cyclictest22-21ksoftirqd/120:09:311
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional