You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-01 - 15:46

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack9slot4s (updated Mon Dec 01, 2025 12:59:03)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
300999912081175,26cyclictest2376-21NetworkManager09:10:301
300999911781145,15cyclictest30311-21basename07:25:131
300989911421101,29cyclictest30294-21wget07:25:130
298732543497,31sleep00-21swapper/007:23:180
283232505459,30sleep10-21swapper/107:20:081
3009899151125,21cyclictest6141-21/usr/sbin/munin09:10:300
2448021260,9sleep03009899cyclictest10:01:570
131442930,5sleep10-21swapper/112:38:261
182002910,8sleep03009899cyclictest10:29:510
166222840,8sleep03009899cyclictest12:08:290
266312820,5sleep126629-21ssh09:30:281
142502810,4sleep00-21swapper/009:51:040
30098996110,13cyclictest5073-21ntp_kernel_pll_07:40:230
178832610,22sleep03-21ksoftirqd/009:21:230
30098995824,6cyclictest27661-21ssh11:45:340
3009899532,20cyclictest31195-21tr11:50:170
3009899511,19cyclictest17974-21ssh12:43:250
3009899511,19cyclictest17974-21ssh12:43:240
3009899491,15cyclictest27286-21sh12:53:370
3009899471,12cyclictest6438-21ssh11:57:430
289422460,7sleep03009899cyclictest10:40:250
3009999421,8cyclictest29851-21ssh10:40:451
3009899401,8cyclictest21953-21timerandwakeup09:25:340
1659024024,4sleep122-21ksoftirqd/112:41:241
1659024024,4sleep122-21ksoftirqd/112:41:231
3009999387,4cyclictest81rcu_preempt12:28:271
114522371,4sleep111470-21ssh12:03:001
76712351,4sleep17681-21ssh12:32:291
30099993517,7cyclictest2220-21dbus-daemon11:18:041
322642341,29sleep132228-21munin-node07:30:041
3009999343,5cyclictest81rcu_preempt10:14:301
30099993411,11cyclictest0-21swapper/109:20:461
3009899348,13cyclictest81rcu_preempt09:39:590
3009899345,4cyclictest81rcu_preempt12:48:590
3009899342,17cyclictest3020-21runrttasks10:57:520
3009899341,7cyclictest1906-21ssh11:20:080
30098993416,11cyclictest17170-21iostat_ios11:35:200
3009899341,5cyclictest81rcu_preempt11:07:260
30098993413,6cyclictest18921-21df_inode10:30:170
247762341,29sleep130944-21snmpd08:32:371
120852340,2chrt12084-21ssh10:55:581
34342331,4sleep13451-21iostat_ios09:40:201
3009999338,13cyclictest40950irq/93-i91509:45:291
3009999336,15cyclictest0-21swapper/109:39:231
3009999335,4cyclictest81rcu_preempt12:46:041
30099993323,6cyclictest19567-21ssh11:04:521
30099993310,19cyclictest0-21swapper/110:08:121
3009899332,14cyclictest31323-21/usr/sbin/munin08:50:160
293822332,6sleep10-21swapper/112:21:031
3009999329,19cyclictest0-21swapper/109:27:531
30099993216,5cyclictest22-21ksoftirqd/111:58:381
3009899329,3cyclictest0-21swapper/012:14:340
3009899328,7cyclictest18783-21ssh09:55:420
30098993222,6cyclictest11168-21ssh12:02:340
3009899322,19cyclictest0-21swapper/012:37:090
3009899322,17cyclictest0-21swapper/008:16:130
3009899321,4cyclictest81rcu_preempt11:44:310
3009899321,4cyclictest81rcu_preempt10:14:330
30098993213,4cyclictest3-21ksoftirqd/010:38:010
3009899321,14cyclictest28556-21ssh09:32:550
254582321,5sleep10-21swapper/110:03:241
240962321,3sleep10-21swapper/111:42:251
188842321,27sleep118894-21ssh09:55:511
154602321,5sleep00-21swapper/008:08:360
120032320,4sleep122-21ksoftirqd/110:22:501
3009999318,5cyclictest81rcu_preempt11:20:111
3009999317,5cyclictest40950irq/93-i91511:29:141
3009999315,4cyclictest81rcu_preempt10:46:491
3009999315,23cyclictest0-21swapper/112:54:141
3009999314,4cyclictest81rcu_preempt07:47:181
30099993122,5cyclictest12963-21ssh09:50:181
3009999311,8cyclictest20814-21ssh12:12:361
3009999311,26cyclictest0-21swapper/112:09:061
3009999311,17cyclictest25554-21ssh10:36:331
3009899319,6cyclictest246002chrt08:30:440
3009899316,4cyclictest81rcu_preempt10:23:460
3009899313,11cyclictest3020-21runrttasks12:17:370
3009899312,14cyclictest1773-21python10:45:300
3009899311,4cyclictest81rcu_preempt11:18:050
3009899311,25cyclictest8851-21ntp_states07:50:250
30098993111,7cyclictest0-21swapper/008:40:030
3009899311,11cyclictest81rcu_preempt08:48:020
3009999309,4cyclictest51352/usr/sbin/munin09:05:301
3009999306,13cyclictest81rcu_preempt11:50:291
3009999305,20cyclictest0-21swapper/110:50:351
3009999304,4cyclictest81rcu_preempt08:26:081
3009999304,4cyclictest81rcu_preempt08:02:101
3009999303,14cyclictest0-21swapper/110:28:081
3009999301,4cyclictest81rcu_preempt11:05:031
30099993014,5cyclictest40950irq/93-i91510:15:131
3009999300,4cyclictest81rcu_preempt11:11:541
3009899307,5cyclictest81rcu_preempt12:30:530
3009899302,15cyclictest0-21swapper/008:35:450
30098993012,6cyclictest1395-21/usr/sbin/munin08:55:280
30098993011,5cyclictest12578-21unixbench_singl09:15:440
3009899301,11cyclictest0-21swapper/009:46:010
30098993010,7cyclictest15915-21ls11:00:220
3009899300,4cyclictest81rcu_preempt12:21:300
265702301,5sleep10-21swapper/112:18:591
126592301,25sleep112671-21ssh09:15:481
321522291,5sleep10-21swapper/108:50:561
3009999292,11cyclictest0-21swapper/111:30:401
3009999291,9cyclictest18926-21ssh10:30:171
3009899296,12cyclictest40950irq/93-i91508:25:010
3009899294,13cyclictest0-21swapper/007:31:360
3009899292,14cyclictest5044-21ssh09:41:420
3009899292,14cyclictest28937-21ssh11:14:320
3009899291,4cyclictest81rcu_preempt09:05:100
3009899291,14cyclictest0-21swapper/011:26:220
191462291,24sleep11956-21minissdpd08:17:491
174602291,4sleep117499-21open_inodes11:35:241
55722281,5sleep10-21swapper/107:40:511
304412281,5sleep10-21swapper/108:47:441
3009999284,4cyclictest81rcu_preempt08:40:121
3009999281,14cyclictest32715-21/usr/sbin/munin08:55:281
30099992810,15cyclictest0-21swapper/111:45:221
3009899283,4cyclictest81rcu_preempt10:51:580
3009899282,4cyclictest0-21swapper/010:15:350
3009899282,13cyclictest13150-21/usr/sbin/munin08:00:270
3009899282,13cyclictest0-21swapper/007:56:420
3009899281,4cyclictest81rcu_preempt10:05:540
30098992814,10cyclictest0-21swapper/012:26:560
3009899281,3cyclictest81rcu_preempt09:04:500
3009899281,14cyclictest16069-21diskstats08:10:190
3009899281,11cyclictest0-21swapper/011:33:210
29552281,5sleep10-21swapper/107:35:221
3009999274,16cyclictest15615-21munin-run08:10:001
3009999271,14cyclictest8851-21ntp_states07:50:261
30099992711,12cyclictest9900-21/usr/sbin/munin07:55:211
3009999271,10cyclictest3525-21sed09:00:291
3009899272,13cyclictest0-21swapper/007:45:140
3009899271,4cyclictest81rcu_preempt08:26:000
210322271,5sleep10-21swapper/108:22:421
3009899269,5cyclictest3025-21latency07:35:220
30099992516,5cyclictest22-21ksoftirqd/108:14:031
264582251,5sleep10-21swapper/108:35:361
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional