You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-09-16 - 22:50

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack9slot4s (updated Tue Sep 16, 2025 12:59:04)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
313679912031180,19cyclictest6559-21smartctl07:40:331
313669911271092,24cyclictest0-21swapper/007:25:080
313679911141093,18cyclictest31529-21aten_r4power_cu07:25:101
310752561517,29sleep00-21swapper/007:22:350
295902514418,32sleep10-21swapper/107:20:071
3136699145121,19cyclictest0-21swapper/007:40:330
2485121450,8sleep03136699cyclictest08:30:150
2442021100,3sleep0111rcuc/011:08:180
1909421070,4sleep03136699cyclictest11:35:350
31366996413,12cyclictest30693-21/usr/sbin/munin11:15:230
3136699612,17cyclictest0-21swapper/011:10:220
31366995915,6cyclictest9593-21diskstats07:50:230
19622560,23sleep03-21ksoftirqd/011:51:460
3136699531,5cyclictest81rcu_preempt12:54:590
3136699531,5cyclictest81rcu_preempt12:54:590
3136699521,5cyclictest23573-21which11:40:290
3136699521,5cyclictest23573-21which11:40:280
31366995210,5cyclictest3020-21runrttasks09:07:260
31366995031,4cyclictest40950irq/93-i91509:34:410
8522450,6sleep03-21ksoftirqd/008:50:310
33612420,5sleep1201rcuc/110:45:331
197732411,36sleep119786-21sh09:22:211
136592391,34sleep113670-21sh12:37:381
3136799389,24cyclictest0-21swapper/110:04:471
3136799388,5cyclictest81rcu_preempt12:51:541
3136799388,5cyclictest81rcu_preempt12:51:531
31367993813,5cyclictest40950irq/93-i91511:57:291
3136699383,5cyclictest81rcu_preempt12:35:180
214582381,33sleep17138-21diskmemload10:30:471
154192380,3sleep1211ktimersoftd/112:05:311
51872372,5sleep10-21swapper/111:21:181
3136699378,12cyclictest492-21ssh10:43:300
3136799363,16cyclictest22-21ksoftirqd/111:07:511
3136799361,19cyclictest19743-21ntp_states08:15:261
3136699367,4cyclictest81rcu_preempt11:02:100
16382360,4sleep00-21swapper/007:30:130
3136799358,23cyclictest0-21swapper/109:49:511
3136799354,4cyclictest81rcu_preempt10:40:231
31367993521,12cyclictest22-21ksoftirqd/112:30:001
3136699354,5cyclictest81rcu_preempt12:47:090
3136699352,20cyclictest15445-21diskstats08:05:150
3136699351,8cyclictest31618-21ssh10:08:310
31366993510,4cyclictest81rcu_preempt10:25:180
31102351,30sleep130944-21snmpd11:53:191
238682351,30sleep123879-21ssh09:26:161
123482352,4sleep112456-21/usr/sbin/munin10:55:251
3136799349,21cyclictest0-21swapper/110:07:441
3136799341,19cyclictest81rcu_preempt09:55:231
3136699342,5cyclictest81rcu_preempt10:14:130
31366993424,6cyclictest5383-21ssh12:29:360
270332341,29sleep127036-21irqstats08:35:211
245372341,3sleep00-21swapper/010:35:030
149882341,5sleep00-21swapper/009:17:250
31367993315,6cyclictest22-21ksoftirqd/112:30:161
3136699338,13cyclictest14107-21ntp_states08:00:240
3136699335,5cyclictest81rcu_preempt12:42:180
3136699335,4cyclictest81rcu_preempt09:20:140
31366993322,6cyclictest19106-21iostat_ios09:55:210
31366993311,5cyclictest22577-21iostat09:25:210
3136799327,4cyclictest0-21swapper/109:19:361
3136799325,17cyclictest0-21swapper/110:17:301
3136799322,5cyclictest81rcu_preempt11:46:321
31367993223,6cyclictest6541-21rm09:42:051
3136799322,26cyclictest0-21swapper/109:31:261
3136799321,16cyclictest7957-21perl09:10:181
3136699327,4cyclictest81rcu_preempt11:25:250
3136699326,4cyclictest81rcu_preempt09:12:350
3136699323,15cyclictest5492-21users09:40:350
31366993212,7cyclictest0-21swapper/012:05:330
97872311,26sleep09811-21ntp_states09:45:300
84222310,3sleep0111rcuc/010:50:500
318742315,4sleep127976-21kworker/1:112:23:041
3136799318,19cyclictest0-21swapper/111:38:171
3136799312,8cyclictest0-21swapper/111:04:481
31367993123,4cyclictest14075-21fschecks_time09:50:151
31367993115,4cyclictest22-21ksoftirqd/111:42:001
31367993115,4cyclictest22-21ksoftirqd/111:41:591
3136799311,11cyclictest26837-21ssh12:17:451
3136799311,11cyclictest26837-21ssh12:17:451
3136699319,11cyclictest0-21swapper/012:23:390
3136699317,4cyclictest81rcu_preempt12:19:300
3136699317,4cyclictest81rcu_preempt12:19:300
3136699316,4cyclictest81rcu_preempt10:59:000
3136699312,14cyclictest0-21swapper/012:30:280
3136699311,4cyclictest81rcu_preempt09:50:410
31366993112,7cyclictest20744-21/usr/sbin/munin10:30:240
3136699311,19cyclictest18597-21taskset08:13:300
31366993111,6cyclictest3020-21runrttasks08:40:010
248382311,26sleep024849-21ssh10:00:410
224352311,5sleep10-21swapper/108:24:081
49652301,5sleep00-21swapper/009:02:040
3136799306,8cyclictest81rcu_preempt08:10:271
3136799305,4cyclictest81rcu_preempt10:54:351
3136799304,4cyclictest81rcu_preempt10:26:121
3136799303,4cyclictest81rcu_preempt08:52:151
3136799303,4cyclictest81rcu_preempt07:55:121
3136799302,8cyclictest0-21swapper/112:11:391
31367993022,5cyclictest22-21ksoftirqd/112:05:011
3136799301,4cyclictest81rcu_preempt11:30:161
3136799300,6cyclictest81rcu_preempt08:46:241
3136699306,4cyclictest81rcu_preempt11:20:450
31366993020,6cyclictest13068-21ssh12:03:430
3136699301,4cyclictest81rcu_preempt08:57:380
31366993013,5cyclictest0-21swapper/011:49:370
31366993011,6cyclictest6829-21users10:15:320
31366993010,7cyclictest0-21swapper/008:29:580
284092301,4sleep128420-21ssh10:38:551
3136799294,18cyclictest7709-21diskstats07:45:181
31367992921,5cyclictest32203-21iostat_ios09:35:221
3136799291,8cyclictest783-21diskstats10:10:151
3136799291,4cyclictest81rcu_preempt07:38:371
3136799291,3cyclictest81rcu_preempt12:49:021
3136799291,16cyclictest0-21swapper/111:27:451
31367992911,13cyclictest2612-21sshd08:25:241
3136699299,7cyclictest14314-21grep11:30:360
3136699294,4cyclictest81rcu_preempt07:35:300
3136699292,14cyclictest10900-21iostat_ios10:20:210
3136699291,15cyclictest9119-21cron12:00:010
31366992910,7cyclictest3239-21runrttasks09:39:190
149322291,5sleep10-21swapper/108:04:281
3136799284,3cyclictest81rcu_preempt10:20:591
31367992821,4cyclictest22-21ksoftirqd/111:15:201
3136799281,14cyclictest10979-21tar07:53:261
3136699282,13cyclictest19840-21memory12:10:240
3136699281,4cyclictest81rcu_preempt10:47:080
3136699281,13cyclictest8649-21grep07:45:320
31366992810,14cyclictest20444-21seq08:18:090
3136799277,6cyclictest81rcu_preempt08:58:391
3136799271,4cyclictest81rcu_preempt08:33:251
31367992713,7cyclictest3537-21/usr/sbin/munin09:00:321
3136799271,14cyclictest0-21swapper/108:40:161
31367992710,4cyclictest0-21swapper/112:40:171
3136699279,5cyclictest21104-21runrttasks08:20:190
3136699272,13cyclictest28616-21df08:40:110
3136699272,12cyclictest30277-21basename08:45:120
3136699271,22cyclictest12643-21users07:55:310
30092271,5sleep10-21swapper/107:30:491
151442271,5sleep10-21swapper/108:05:071
3136799267,15cyclictest0-21swapper/111:10:131
31367992518,4cyclictest22-21ksoftirqd/109:05:141
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional