You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-16 - 22:08

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 100 highest latencies:
System rack5slot3s.osadl.org (updated Sun Nov 16, 2025 12:44:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
56524040,5sleep30-21swapper/307:09:273
3532233161,59sleep00-21swapper/007:06:520
3962214162,39sleep10-21swapper/107:07:261
4862204166,25sleep20-21swapper/207:08:312
2306921760,4sleep070499cyclictest10:25:140
119522800,4sleep339-21rcuc/307:30:273
119522800,4sleep339-21rcuc/307:30:273
257022730,1sleep10-21swapper/108:05:011
718996864,3cyclictest0-21swapper/208:09:362
718995855,2cyclictest0-21swapper/211:30:342
104182550,3sleep30-21swapper/311:10:133
718994535,9cyclictest0-21swapper/208:31:482
718994535,9cyclictest0-21swapper/208:31:482
60122430,2sleep30-21swapper/309:45:203
324332410,1sleep032432-21sed09:30:270
70499397,29cyclictest14354-21munin-run11:20:010
70499371,11cyclictest3994-21munin-run12:10:010
704993710,4cyclictest26795-21/usr/sbin/munin08:05:220
14072360,3sleep070499cyclictest09:35:170
70499359,3cyclictest27142-21/usr/sbin/munin10:35:230
70499358,9cyclictest24266-21gltestperf08:00:190
704993510,3cyclictest30273-21/usr/sbin/munin08:15:250
70499348,4cyclictest14164-21awk12:30:250
70499343,8cyclictest15505-21sort11:20:190
70499341,6cyclictest768-21in:imjournal10:05:010
704993414,3cyclictest17064-21grep08:55:270
704993411,3cyclictest29743-21ntp_kernel_err09:25:240
704993311,3cyclictest26156-21gltestperf11:45:160
704993311,3cyclictest22535-21ntp_states07:55:240
704993311,3cyclictest19913-21/usr/sbin/munin09:05:260
704993310,3cyclictest8619-21/usr/sbin/munin12:20:170
70499329,3cyclictest5779-21/usr/sbin/munin11:00:200
70499328,4cyclictest28137-21/usr/sbin/munin11:50:160
70499322,10cyclictest14634-21snmpd10:12:340
704993212,3cyclictest9184-21seq09:53:310
70499321,10cyclictest15984-21latency_hist10:10:010
72399311,7cyclictest21880-21sadc09:10:013
70499314,12cyclictest31319-21awk10:45:010
70499314,11cyclictest0-21swapper/012:15:010
70499313,2cyclictest86550irq/126-enp1s0-09:10:220
70499312,13cyclictest1627-21unixbench_multi08:20:270
70499311,6cyclictest0-21swapper/009:15:380
70499311,5cyclictest17072-21/usr/sbin/munin07:45:160
704993111,3cyclictest3182-21sed07:15:010
704993110,3cyclictest836-21runrttasks09:01:390
704993110,3cyclictest2809-21cat10:50:270
704993110,3cyclictest24255-21sort11:40:200
704993110,3cyclictest24255-21sort11:40:200
70499309,3cyclictest27580-21ntp_states09:20:240
70499309,3cyclictest22240-21seq10:21:410
70499309,3cyclictest14484-21processes08:50:240
70499302,4cyclictest2960-21latency_hist07:15:010
70499301,6cyclictest11438-21gltestperf12:25:210
70499301,11cyclictest21878-21diskmemload12:36:200
70499301,11cyclictest0-21swapper/010:48:300
704993010,3cyclictest8867-21gltestperf07:25:170
704993010,3cyclictest24946-21/usr/sbin/munin10:30:290
704993010,3cyclictest16248-21timerandwakeup07:40:280
70499296,3cyclictest86750irq/128-enp1s0-12:15:200
70499296,2cyclictest0-21swapper/010:15:210
70499294,4cyclictest0-21swapper/010:55:510
70499291,9cyclictest20150-21/usr/sbin/munin11:30:240
70499291,8cyclictest21171-21/usr/sbin/munin11:35:330
70499291,5cyclictest10484-21/usr/sbin/munin07:30:170
70499291,5cyclictest10484-21/usr/sbin/munin07:30:160
704992910,3cyclictest6491-21munin-run08:35:010
704992910,3cyclictest6491-21munin-run08:35:010
704992910,2cyclictest5193-21kworker/0:211:25:150
718992824,3cyclictest0-21swapper/208:23:412
70499289,3cyclictest5858-21sort09:45:180
70499289,3cyclictest3145-21kernelversion08:25:200
70499288,3cyclictest21324-21cut07:55:010
70499281,8cyclictest0-21swapper/011:07:000
70499281,10cyclictest0-21swapper/009:58:340
704992810,3cyclictest9230-21expr08:40:160
704992810,2cyclictest28788-21seq08:10:180
20102281,3sleep22107-21/usr/sbin/munin10:50:182
70499275,3cyclictest86750irq/128-enp1s0-07:35:210
70499272,4cyclictest30909-21/usr/sbin/munin11:55:200
70499271,4cyclictest0-21swapper/012:00:540
70499271,11cyclictest0-21swapper/011:13:460
288742270,4sleep339-21rcuc/309:25:173
70499261,11cyclictest0-21swapper/009:43:170
136102260,4sleep121-21rcuc/108:50:141
72399251,5cyclictest842-21lldpd09:40:143
712992519,3cyclictest10455-21memory09:55:201
70499250,9cyclictest0-21swapper/008:35:170
38542250,15sleep121878-21diskmemload09:40:191
72399241,4cyclictest14634-21snmpd07:23:173
71899241,6cyclictest32-21ksoftirqd/211:23:462
71299242,3cyclictest12446-40dnf12:25:501
71299241,20cyclictest30985-21cat09:30:121
71899231,9cyclictest17251-21open_inodes10:10:232
71299232,3cyclictest17453-21hddtemp_smartct11:25:191
71299231,3cyclictest19358-21cpuspeed_turbos07:50:131
71299231,3cyclictest18052sleep108:21:311
71299231,19cyclictest4257-21hddtemp_smartct10:55:171
71299231,14cyclictest1682-21sshd12:03:471
72399221,6cyclictest0-21swapper/312:15:353
723992216,3cyclictest27114-21grep09:20:173
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional