You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-03 - 21:02

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 100 highest latencies:
System rack5slot3s.osadl.org (updated Wed Dec 03, 2025 12:44:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
154822238163,25sleep10-21swapper/107:06:031
157242236161,25sleep30-21swapper/307:09:113
138302205167,25sleep00-21swapper/007:05:080
138322201164,24sleep20-21swapper/207:05:112
1589599106103,2cyclictest0-21swapper/211:26:062
84632610,5sleep31590199cyclictest08:05:023
116942600,4sleep230-21rcuc/208:10:192
181492590,2sleep00-21swapper/007:15:010
220832570,5sleep01588499cyclictest12:15:160
204612560,4sleep21589599cyclictest08:30:182
174112540,1sleep20-21swapper/207:10:252
238472470,2sleep20-21swapper/208:40:012
156622460,1sleep10-21swapper/108:20:171
1588499381,23cyclictest27565-21kworker/0:012:27:460
1588499377,3cyclictest86850irq/129-enp1s0-09:20:270
1588499377,3cyclictest2686-21/usr/sbin/munin09:05:190
15884993712,12cyclictest17232-21awk08:25:000
15884993616,17cyclictest27291-21chrt08:45:200
15884993611,22cyclictest4291-21df11:35:120
1588499357,3cyclictest22480-21/usr/sbin/munin09:50:150
1588499351,30cyclictest2670-21cut07:50:150
1588499351,10cyclictest18711-21/usr/sbin/munin10:55:140
1588499351,10cyclictest0-21swapper/011:59:440
15884993510,2cyclictest25912-21df07:30:140
1588499347,4cyclictest24398-21/usr/sbin/munin11:05:250
1588499347,13cyclictest9070-21perf09:20:010
1588499342,4cyclictest19362-21/usr/sbin/munin12:10:190
15884993412,19cyclictest1098-21/usr/sbin/munin10:15:210
1588499341,10cyclictest0-21swapper/010:50:000
15884993410,21cyclictest11171-21tail11:50:150
281752330,3sleep31590199cyclictest12:30:113
1588499339,2cyclictest9915-21/usr/sbin/munin10:35:260
1588499339,2cyclictest9915-21/usr/sbin/munin10:35:260
1588499337,11cyclictest0-21swapper/009:05:010
1588499336,12cyclictest30430-21basename08:55:000
1588499331,6cyclictest0-21swapper/010:13:320
1589199321,24cyclictest0-21swapper/112:30:141
1588499327,3cyclictest18085-21/usr/sbin/munin09:40:240
1588499326,4cyclictest848-21sssd_nss11:25:150
1588499326,4cyclictest0-21swapper/012:35:170
1588499325,8cyclictest4848-21gltestperf07:55:150
1588499322,19cyclictest21411-21kworker/0:108:44:490
1588499321,4cyclictest29727-21/usr/sbin/munin11:20:210
15884993213,16cyclictest14758-21seq08:16:540
15884993211,18cyclictest836-21runrttasks10:32:580
1588499321,10cyclictest14634-21snmpd11:33:390
15884993210,5cyclictest0-21swapper/010:00:260
15884993210,3cyclictest17852-21hwlatdetect12:05:220
1588499319,4cyclictest12059-21fgrep08:10:230
1588499319,3cyclictest17146-21gltestperf10:50:160
1588499316,12cyclictest20951-21munin-run07:20:010
1588499312,5cyclictest29090-21/usr/sbin/munin10:05:280
15884993111,17cyclictest21807-21sed09:45:260
1588499310,9cyclictest0-21swapper/011:10:180
1590199301,13cyclictest0-21swapper/307:24:113
1589199301,23cyclictest0-21swapper/109:16:131
1588499308,4cyclictest848-21sssd_nss09:10:210
1588499307,3cyclictest32263-21grep07:45:130
1588499304,7cyclictest0-21swapper/011:40:120
1588499304,7cyclictest0-21swapper/011:40:120
1588499304,5cyclictest848-21sssd_nss09:25:150
1588499304,3cyclictest12050irq/123-ahci[0012:35:010
1588499301,11cyclictest24397-21netstat07:25:210
1588499301,11cyclictest24397-21netstat07:25:210
1588499301,10cyclictest14634-21snmpd10:28:260
15884993010,17cyclictest8371-21idleruntime-cro08:05:000
1589199291,3cyclictest12689-21cut11:55:021
1589199291,22cyclictest0-21swapper/109:48:051
1588499298,3cyclictest25236-21grep12:20:280
1588499297,3cyclictest28787-21sensors11:15:250
1588499297,3cyclictest17446-21/usr/sbin/munin08:25:120
1588499297,3cyclictest17446-21/usr/sbin/munin08:25:120
1588499296,3cyclictest31249-21grep08:55:160
1588499294,3cyclictest88050irq/133-enp2s0-10:40:280
1588499294,3cyclictest86550irq/126-enp1s0-08:05:150
1588499291,9cyclictest15056-21cut12:00:120
1588499291,4cyclictest20915-21/usr/sbin/munin11:00:250
1588499291,3cyclictest29971-21/usr/sbin/munin07:40:190
15884992912,14cyclictest9613-21mailstats11:45:280
1588499291,13cyclictest23820-21munin-run08:40:000
15884992910,3cyclictest28237-21cut07:35:180
1590199285,14cyclictest780-21rngd08:23:363
1590199281,11cyclictest0-21swapper/308:14:483
1590199280,19cyclictest0-21swapper/307:47:443
1589199281,22cyclictest0-21swapper/112:16:511
1588499286,4cyclictest4557-21sendmail_mailst10:20:220
1588499286,4cyclictest24682-21/usr/sbin/munin09:55:190
1588499286,4cyclictest16923-21ntpq09:35:230
1588499283,7cyclictest14212-21fschecks_time09:30:160
1588499283,7cyclictest14212-21fschecks_time09:30:160
1590199271,11cyclictest0-21swapper/307:14:563
1590199270,12cyclictest0-21swapper/310:18:403
15895992724,2cyclictest0-21swapper/208:43:142
1589199271,17cyclictest0-21swapper/111:56:511
15891992711,4cyclictest2339-21perl11:30:161
1588499279,4cyclictest20308-21sed08:30:150
1588499271,8cyclictest22670-21unixbench_multi07:20:260
1590199260,20cyclictest0-21swapper/312:38:153
1589199261,22cyclictest581-21systemd-journal08:00:011
1589199261,16cyclictest0-21swapper/108:42:151
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional