You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-09-15 - 20:42

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 100 highest latencies:
System rack5slot3s.osadl.org (updated Mon Sep 15, 2025 12:44:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
161912242162,25sleep30-21swapper/307:07:273
161322239163,24sleep20-21swapper/207:06:422
163222204166,25sleep00-21swapper/007:09:100
161112203166,24sleep10-21swapper/107:06:251
2811621130,4sleep121-21rcuc/112:30:191
16489991081,105cyclictest0-21swapper/208:37:292
164899910495,8cyclictest0-21swapper/209:34:532
16489999591,3cyclictest0-21swapper/209:17:072
16489999591,3cyclictest0-21swapper/209:17:072
291852730,3sleep29-21rcu_preempt11:15:212
188202650,5sleep07-21ksoftirqd/009:40:140
300692490,2sleep20-21swapper/212:35:182
16476994011,13cyclictest28770-21munin-run08:50:020
1647699396,12cyclictest12050irq/123-ahci[0011:45:020
1647699354,14cyclictest21540-21munin-run07:20:000
1648299330,21cyclictest3577-21df_inode09:05:151
1648299323,14cyclictest19086-40dnf08:25:351
174322310,1sleep30-21swapper/312:05:213
1648299292,5cyclictest26947-21munin-run12:30:011
1648299292,13cyclictest10918-21munin-run11:50:021
1648299291,9cyclictest0-21swapper/112:12:351
1648299291,9cyclictest0-21swapper/111:53:471
1648299291,9cyclictest0-21swapper/109:55:341
1648299291,7cyclictest0-21swapper/110:00:581
1648299291,17cyclictest7482-21cat10:25:271
1648299291,15cyclictest0-21swapper/111:59:051
1648299291,12cyclictest784-21NetworkManager09:20:451
209262280,3sleep011-21rcuc/008:30:180
1648299287,6cyclictest0-21swapper/111:19:511
1648299285,6cyclictest0-21swapper/109:36:181
1648299281,16cyclictest17156-21strings12:05:161
1648299281,11cyclictest1103-21cpuspeed_turbos09:00:121
1647699285,11cyclictest25061-21awk09:55:020
291342270,3sleep21648999cyclictest08:50:142
1648999279,8cyclictest5798-21sendmail_mailtr11:35:232
1648299279,4cyclictest16895-21perf10:50:021
16482992714,3cyclictest5208-21diskmemload11:20:271
1648299271,17cyclictest0-21swapper/109:41:231
1648299271,17cyclictest0-21swapper/108:49:031
1648299271,10cyclictest0-21swapper/111:38:301
1648299270,25cyclictest0-21swapper/107:10:081
1649399261,9cyclictest0-21swapper/308:02:323
16493992612,4cyclictest13049-21df10:40:143
1649399261,12cyclictest10409-21fschecks_time09:20:173
1648299262,11cyclictest22118-21diskstats07:20:131
1648299261,9cyclictest0-21swapper/110:43:301
1648299261,4cyclictest27049-21kworker/u8:310:14:171
1648299261,17cyclictest0-21swapper/109:27:271
1648299261,16cyclictest0-21swapper/108:12:271
1648299261,16cyclictest0-21swapper/108:12:261
1648299261,16cyclictest0-21swapper/107:45:591
1648299261,16cyclictest0-21swapper/107:40:011
1648299261,11cyclictest25346-21/usr/sbin/munin08:40:181
1648299261,11cyclictest24068-21cut07:25:151
1648299261,11cyclictest16249-21grep09:35:011
1648299260,5cyclictest0-21swapper/111:14:211
1648299260,16cyclictest0-21swapper/111:00:211
1648299260,16cyclictest0-21swapper/109:50:021
1648999253,9cyclictest86550irq/126-enp1s0-11:00:012
1648999251,11cyclictest209282idleruntime08:30:172
1648299252,9cyclictest23118-21diskstats12:20:151
1648299252,11cyclictest2692-21df_inode11:30:151
1648299252,10cyclictest836-21runrttasks08:54:381
1648299251,4cyclictest6871-21kworker/u8:009:18:231
1648299251,4cyclictest6871-21kworker/u8:009:18:221
1648299251,4cyclictest10256-21kworker/u8:110:38:421
1648299251,17cyclictest0-21swapper/112:04:421
1648299251,16cyclictest0-21swapper/108:02:151
1648299251,15cyclictest0-21swapper/110:55:031
1648299251,15cyclictest0-21swapper/107:30:051
1649399241,6cyclictest17544-21/usr/sbin/munin09:35:233
1649399241,11cyclictest0-21swapper/310:30:193
1649399241,10cyclictest0-21swapper/308:07:163
1649399240,14cyclictest0-21swapper/311:05:173
1648299241,9cyclictest15630-21ls08:20:011
1648299241,4cyclictest30007-21sed12:35:171
16482992414,2cyclictest0-21swapper/110:33:021
16482992413,3cyclictest0-21swapper/107:19:581
1648299241,16cyclictest0-21swapper/111:07:481
1648299241,15cyclictest0-21swapper/111:28:561
1648299241,15cyclictest0-21swapper/111:28:561
1648299241,15cyclictest0-21swapper/110:18:211
1648299241,15cyclictest0-21swapper/107:58:401
1648299241,15cyclictest0-21swapper/107:52:301
1648299241,11cyclictest4979-21ntp_states10:20:201
1648299241,11cyclictest4979-21ntp_states10:20:201
1648299240,6cyclictest0-21swapper/110:54:411
1648299240,4cyclictest1738-21kworker/u8:011:40:231
1648299240,15cyclictest0-21swapper/108:39:251
16476992412,7cyclictest19208-21processes08:25:350
1649399235,3cyclictest24825-21grep07:25:223
1649399232,10cyclictest814-21sshd10:16:163
1649399231,9cyclictest23217-21fschecks_count12:20:163
16493992315,2cyclictest0-21swapper/312:35:143
16493992310,3cyclictest15790-21latency10:45:223
1648999234,4cyclictest32511-21/usr/sbin/munin11:25:192
1648999234,4cyclictest32511-21/usr/sbin/munin11:25:192
1648999230,4cyclictest0-21swapper/210:30:192
1648299231,10cyclictest22537-21latency_hist12:20:011
1648299231,10cyclictest10452-21/usr/sbin/munin08:05:231
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional