You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-05 - 01:02

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack9slot4s (updated Thu Dec 04, 2025 12:59:03)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
38599912541235,15cyclictest19348-21apt-get08:02:291
38589912471183,38cyclictest4232-21cstates07:25:130
38599911801151,24cyclictest0-21swapper/107:25:111
37282561516,30sleep00-21swapper/007:24:160
34842464417,32sleep10-21swapper/107:21:431
385899185161,19cyclictest0-21swapper/008:02:290
3092621110,4sleep041ktimersoftd/011:43:310
67332870,9sleep1385999cyclictest10:45:081
97902700,6sleep00-21swapper/011:20:460
19722640,2sleep00-21swapper/008:40:310
19722640,2sleep00-21swapper/008:40:300
157042590,8sleep0385899cyclictest12:00:310
385899578,12cyclictest0-21swapper/008:05:120
166042480,7sleep0385899cyclictest07:55:200
116422480,2sleep011641-21ssh09:42:280
277172441,25sleep127729-21df_inode11:40:151
188382431,24sleep018841-21ssh12:37:580
91942420,9sleep1385999cyclictest09:00:261
385999411,27cyclictest40950irq/93-i91511:45:371
385899403,4cyclictest81rcu_preempt09:25:180
385899391,17cyclictest0-21swapper/011:06:530
194632390,3sleep119517-21ssh09:50:271
385899381,24cyclictest15607-21ssh10:20:190
123092381,33sleep112323-21ssh11:57:291
385999374,29cyclictest0-21swapper/110:23:401
385899371,5cyclictest2813250ssh12:14:280
165952371,6sleep116606-21sh11:28:431
385999364,4cyclictest81rcu_preempt10:41:121
385999362,18cyclictest0-21swapper/110:50:281
3859993620,4cyclictest22-21ksoftirqd/112:38:391
385999361,9cyclictest30366-21ssh10:01:401
220922360,3sleep00-21swapper/010:26:410
92712351,30sleep19282-21ssh10:14:031
48782351,5sleep14896-21fschecks_count08:50:161
385999352,12cyclictest0-21swapper/112:40:381
385999352,11cyclictest0-21swapper/110:37:531
385899354,4cyclictest81rcu_preempt11:56:470
385899354,4cyclictest81rcu_preempt09:10:040
385899352,12cyclictest12552-21ntp_states10:50:260
385899351,22cyclictest21634-21seq11:00:160
385899351,18cyclictest0-21swapper/010:09:230
83782341,29sleep18391-21sh12:26:451
385999347,23cyclictest0-21swapper/112:21:271
385999342,17cyclictest28515-21munin-run12:15:011
3859993418,5cyclictest22-21ksoftirqd/109:35:311
3859993418,4cyclictest22-21ksoftirqd/111:32:001
385999341,7cyclictest22-21ksoftirqd/109:17:541
385899344,4cyclictest81rcu_preempt10:48:430
149672341,28sleep115017-21iostat12:00:201
12632341,5sleep11297-21iostat08:40:211
12632341,5sleep11297-21iostat08:40:201
385999336,6cyclictest0-21swapper/111:13:441
385899337,4cyclictest81rcu_preempt09:33:000
385899337,11cyclictest32542-21df11:45:160
385899336,4cyclictest81rcu_preempt12:55:000
385899332,4cyclictest81rcu_preempt10:00:010
385899332,4cyclictest81rcu_preempt10:00:000
385899332,13cyclictest28770-21ssh12:15:120
3858993313,16cyclictest3-21ksoftirqd/008:19:220
385899331,21cyclictest28328-21ssh10:33:460
385899331,21cyclictest28328-21ssh10:33:460
205022332,27sleep120566-21iostat08:05:201
139182331,27sleep113932-21ssh09:10:511
111992331,28sleep111220-21open_files07:40:291
105352331,28sleep010599-21sh10:15:160
75312321,4sleep17544-21sh11:52:321
385999327,5cyclictest81rcu_preempt09:32:541
3859993226,4cyclictest137350irq/90-eth010:05:241
385999321,5cyclictest81rcu_preempt09:44:471
3859993215,14cyclictest22-21ksoftirqd/111:18:021
385999321,13cyclictest219002/usr/sbin/munin09:20:191
3859993210,8cyclictest11090-21cpuspeed_turbos12:30:141
385899324,5cyclictest81rcu_preempt08:35:000
385899322,4cyclictest81rcu_preempt11:50:170
385899322,4cyclictest81rcu_preempt10:11:260
385899321,9cyclictest24604-21ssh11:36:200
3858993218,11cyclictest0-21swapper/009:38:450
385899321,5cyclictest81rcu_preempt09:49:420
385899321,4cyclictest81rcu_preempt09:15:590
385899321,19cyclictest11213-21open_files07:40:280
385899321,17cyclictest17779-21/usr/sbin/munin11:30:240
3858993211,7cyclictest0-21swapper/007:39:340
30872322,4sleep081rcu_preempt12:21:030
224372320,3sleep122447-21ssh12:08:111
175222323,3sleep017614-21rm10:55:320
385999314,5cyclictest0-21swapper/107:35:171
3859993125,4cyclictest137350irq/90-eth009:25:021
385999312,24cyclictest25339-21irqstats10:30:211
385999312,24cyclictest25339-21irqstats10:30:211
385999311,9cyclictest23884-21ssh10:29:171
3859993117,4cyclictest22-21ksoftirqd/112:48:421
385999311,4cyclictest81rcu_preempt12:17:361
385899312,15cyclictest0-21swapper/012:29:420
3858993120,7cyclictest10761-21wget09:05:250
3858993118,8cyclictest2220-21dbus-daemon08:45:150
385899311,15cyclictest28536-21grep10:00:150
385899311,15cyclictest15256-21ls07:50:310
317662310,8sleep0385899cyclictest11:10:240
245262311,26sleep124559-21ntp_states08:15:251
139152312,6sleep10-21swapper/107:49:371
385999307,19cyclictest11962-21diskmemload09:45:171
385999303,15cyclictest10532-21diskstats10:15:151
385999302,6cyclictest22-21ksoftirqd/111:07:481
385999301,6cyclictest81rcu_preempt08:45:461
385999301,4cyclictest81rcu_preempt11:21:231
385999301,17cyclictest26512-21sh09:58:061
385999301,17cyclictest26512-21sh09:58:051
385999301,13cyclictest756-21ssh12:53:291
385899307,10cyclictest21944-21sh09:53:270
385899303,4cyclictest81rcu_preempt09:24:150
3858993022,5cyclictest3840-21diskmemload11:15:190
385899302,15cyclictest0-21swapper/012:45:180
385899301,4cyclictest81rcu_preempt09:02:100
385899301,15cyclictest12961-21/usr/sbin/munin11:25:330
3858993010,6cyclictest22134-21df08:10:130
385999293,19cyclictest25927-21diskstats08:20:151
3859992924,3cyclictest137350irq/90-eth011:02:381
385999291,5cyclictest0-21swapper/107:58:021
385999291,4cyclictest81rcu_preempt09:08:581
3859992912,13cyclictest23283-21irqstats11:35:221
385999291,16cyclictest16684-21cstates10:55:161
385899293,14cyclictest0-21swapper/007:45:190
385899293,12cyclictest1909-21aten_r4power_en10:40:130
385899292,23cyclictest0-21swapper/008:28:410
385899291,5cyclictest6929-21fschecks_count07:30:160
385899291,4cyclictest81rcu_preempt12:30:590
385899291,4cyclictest81rcu_preempt08:21:200
3858992911,5cyclictest31843-21/usr/sbin/munin08:35:190
385899291,14cyclictest30167-21irqstats10:35:230
385899291,12cyclictest24151-21sh12:43:410
385999281,3cyclictest81rcu_preempt08:14:351
385899283,4cyclictest81rcu_preempt12:09:290
154542281,3sleep10-21swapper/107:51:041
82392271,5sleep10-21swapper/107:34:501
79892271,4sleep10-21swapper/108:58:041
385899273,6cyclictest0-21swapper/008:53:560
385899271,17cyclictest0-21swapper/008:57:110
385999262,10cyclictest28000-21hddtemp_smartct08:25:171
385999252,10cyclictest29878-21hddtemp_smartct08:30:181
385999251,11cyclictest32527-21users08:35:301
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional