You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-03 - 06:14

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 100 highest latencies:
System rack5slot3s.osadl.org (updated Wed Dec 03, 2025 00:44:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
14942239162,25sleep00-21swapper/019:09:170
14282237164,24sleep20-21swapper/219:08:262
13752237163,59sleep10-21swapper/119:07:451
13772236165,58sleep30-21swapper/319:07:473
1669998785,1cyclictest0-21swapper/222:04:232
16852580,3sleep00-21swapper/021:35:120
165999381,10cyclictest7575-21/usr/sbin/munin20:35:140
1659993712,2cyclictest19825-21sensors_temp21:00:250
1659993712,2cyclictest19825-21sensors_temp21:00:250
1659993712,22cyclictest8848-21/usr/sbin/munin23:05:180
165999371,11cyclictest28192-21sshd23:47:200
165999361,10cyclictest6955-21/usr/sbin/munin19:20:250
1659993610,12cyclictest21811-21munin-run23:35:010
167499350,2cyclictest88250irq/135-enp2s0-00:20:253
165999355,14cyclictest13316-21munin-run19:35:000
1659993512,3cyclictest11107-21cat20:40:240
165999351,13cyclictest2899-21/usr/sbin/munin00:05:290
1659993510,3cyclictest28945-21/usr/sbin/munin20:10:160
167499341,24cyclictest0-21swapper/322:27:513
1659993410,2cyclictest13902-21/usr/sbin/munin00:30:200
167499331,22cyclictest0-21swapper/322:44:393
165999338,3cyclictest7293-21grep23:00:170
165999334,14cyclictest1376-21perf21:35:010
165999331,6cyclictest0-21swapper/021:20:170
1659993313,17cyclictest15246-21seq19:36:460
1659993312,3cyclictest21408-21/usr/sbin/munin22:20:130
165999331,13cyclictest5785-21munin-run21:45:000
165999331,13cyclictest18891-21kworker/0:222:14:310
167499325,11cyclictest5787-21perf21:45:003
167499321,12cyclictest0-21swapper/320:00:313
167499320,26cyclictest0-21swapper/323:53:433
165999322,6cyclictest31926-40dnf20:15:190
165999321,5cyclictest0-21swapper/021:45:140
167499315,8cyclictest86550irq/126-enp1s0-22:50:203
167499314,11cyclictest780-21rngd20:06:333
167499310,24cyclictest0-21swapper/323:37:433
167499310,14cyclictest0-21swapper/319:18:233
165999319,3cyclictest27137-21crond21:20:010
165999319,10cyclictest15233-21perf23:20:010
165999318,3cyclictest836-21runrttasks21:25:160
165999316,5cyclictest0-21swapper/000:25:170
165999314,8cyclictest30325-21cpuspeed_turbos22:40:130
165999313,13cyclictest24307-21munin-run20:00:010
165999313,13cyclictest24307-21munin-run20:00:010
165999312,22cyclictest404-21kworker/0:120:28:110
165999311,5cyclictest14170-21/usr/sbin/munin20:50:140
1659993112,3cyclictest8909-21seq00:15:290
1659993112,3cyclictest8909-21seq00:15:290
1659993112,16cyclictest31911-21sh23:55:220
1659993111,3cyclictest955-21/usr/sbin/munin20:20:130
1659993111,3cyclictest26523-21perf20:05:010
1659993110,3cyclictest27018-21expr20:05:140
1659993110,3cyclictest19468-21seq19:45:250
1659993110,2cyclictest836-21runrttasks20:34:570
165999310,4cyclictest12050irq/123-ahci[0019:20:010
167499305,15cyclictest0-21swapper/321:15:233
167499304,11cyclictest780-21rngd19:36:073
165999309,2cyclictest32736-21df22:45:130
165999308,3cyclictest836-21runrttasks00:23:460
165999308,3cyclictest25419-21seq22:28:030
165999308,3cyclictest12959-21entropy22:00:140
1659993012,3cyclictest30003-21wc22:40:010
165999301,13cyclictest10190-21awk21:55:010
165999301,10cyclictest4361-21ls22:55:010
1659993010,3cyclictest22050-21seq19:54:280
167499295,10cyclictest780-21rngd23:18:313
166499291,21cyclictest0-21swapper/121:20:261
165999299,3cyclictest24271-21cpu23:40:120
165999299,3cyclictest18410-21if_err_enp1s023:25:160
165999298,4cyclictest0-21swapper/020:45:280
165999298,3cyclictest836-21runrttasks21:09:050
165999291,2cyclictest0-21swapper/022:15:220
1659992911,3cyclictest29282-21sed23:50:180
1659992911,3cyclictest17436-21threads00:35:260
165999291,12cyclictest6169-21sshd00:10:200
1659992910,3cyclictest16163-21sort22:05:240
1659992910,3cyclictest11238-21cut21:55:180
167499284,15cyclictest780-21rngd20:45:133
167499282,17cyclictest0-21swapper/320:53:433
167499281,12cyclictest0-21swapper/319:10:153
167499281,11cyclictest0-21swapper/319:47:593
167499280,13cyclictest0-21swapper/321:54:233
1664992818,3cyclictest0-21swapper/100:12:131
165999289,3cyclictest11863-21uname23:10:180
165999289,3cyclictest11863-21uname23:10:180
165999288,3cyclictest836-21runrttasks19:44:190
165999287,3cyclictest10056-21load19:25:190
1659992810,3cyclictest22738-21idleruntime23:35:180
167499276,10cyclictest86650irq/127-enp1s0-19:54:353
167499275,9cyclictest780-21rngd19:40:353
167499275,6cyclictest842-21lldpd23:45:203
167499273,9cyclictest86650irq/127-enp1s0-19:28:073
167499273,15cyclictest0-21swapper/300:18:133
167499273,15cyclictest0-21swapper/300:18:133
167499271,10cyclictest0-21swapper/322:58:233
167499271,10cyclictest0-21swapper/320:35:113
167499270,9cyclictest0-21swapper/322:45:513
167499270,11cyclictest0-21swapper/323:30:153
1664992720,4cyclictest10576-21ntp_states00:20:221
165999278,3cyclictest17764-21cat20:55:240
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional