You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-08-29 - 12:04

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 100 highest latencies:
System rack9slot4s (updated Fri Aug 29, 2025 00:59:04)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
148819911181079,23cyclictest16750-21pidof19:30:011
148022590519,54sleep00-21swapper/019:24:490
146542462417,30sleep10-21swapper/119:23:151
14880992501,6cyclictest16762-21basename19:30:010
18492910,4sleep041ktimersoftd/021:55:150
59952840,4sleep10-21swapper/121:59:361
199112700,8sleep01488099cyclictest00:27:400
68172660,4sleep06813-21tr22:00:140
305692550,4sleep01488099cyclictest21:17:220
5792490,8sleep01488099cyclictest23:00:270
296402471,29sleep129651-21sh22:56:591
185622450,23sleep122-21ksoftirqd/100:25:421
312942430,4sleep0111rcuc/000:05:330
194132420,3sleep00-21swapper/023:54:010
1488099423,5cyclictest81rcu_preempt00:30:290
14880994125,5cyclictest3-21ksoftirqd/023:39:020
322552391,4sleep132266-21ssh22:25:441
223262390,4sleep041ktimersoftd/023:56:150
50722381,33sleep15083-21ssh00:12:091
50722381,33sleep15083-21ssh00:12:091
14881993720,5cyclictest2220-21dbus-daemon21:45:341
230062361,4sleep123086-21sh21:10:131
198852361,4sleep119949-21ntpq23:20:261
184372361,9sleep10-21swapper/122:11:391
150842362,4sleep10-21swapper/100:22:411
14880993622,4cyclictest2776-21sh00:10:130
14880993622,4cyclictest2776-21sh00:10:130
14880993611,5cyclictest81rcu_preempt23:31:270
14880993611,5cyclictest81rcu_preempt23:31:260
14881993518,6cyclictest22-21ksoftirqd/123:01:431
1488099354,5cyclictest81rcu_preempt23:27:060
1488099351,5cyclictest29894-21ssh00:38:110
1488099351,15cyclictest3347-21taskset00:44:370
1488099351,14cyclictest316242ssh22:25:260
324492341,29sleep132460-21sh00:06:561
1488199349,21cyclictest0-21swapper/100:36:281
1488199343,4cyclictest81rcu_preempt22:49:371
1488199341,19cyclictest8379-21cat20:30:201
1488099344,4cyclictest81rcu_preempt23:42:090
1488099343,4cyclictest81rcu_preempt00:47:320
14880993416,5cyclictest3-21ksoftirqd/022:14:280
1488099341,21cyclictest4544-21/usr/sbin/munin20:20:190
14880993410,5cyclictest81rcu_preempt00:50:330
1488199336,23cyclictest0-21swapper/122:33:201
1488199336,23cyclictest0-21swapper/122:00:471
1488199334,4cyclictest81rcu_preempt21:22:531
14881993313,6cyclictest16868-21sh22:44:231
1488099335,5cyclictest81rcu_preempt22:48:160
1488099333,5cyclictest81rcu_preempt22:30:170
1488099333,4cyclictest81rcu_preempt23:18:570
1488099332,19cyclictest24804-21runrttasks22:51:550
14880993312,7cyclictest3-21ksoftirqd/021:21:360
14880993310,5cyclictest0-21swapper/022:44:010
121842331,27sleep112267-21memory20:40:231
28202321,4sleep12916-21/usr/sbin/munin20:15:231
1488199325,23cyclictest0-21swapper/122:16:091
1488199322,26cyclictest0-21swapper/100:15:471
1488199322,17cyclictest0-21swapper/122:09:261
14881993218,8cyclictest1296-21diskstats23:35:161
1488199321,4cyclictest81rcu_preempt00:42:101
1488199321,4cyclictest81rcu_preempt00:04:211
14881993213,15cyclictest22-21ksoftirqd/123:44:521
1488099328,4cyclictest81rcu_preempt22:37:430
1488099328,11cyclictest27621-21latency19:55:240
1488099322,8cyclictest29688-21cron22:23:590
1488099322,4cyclictest81rcu_preempt23:45:170
1488099322,4cyclictest81rcu_preempt21:50:560
1488099322,25cyclictest20453-21memory21:40:250
14880993216,5cyclictest3-21ksoftirqd/019:39:460
1488099321,4cyclictest81rcu_preempt00:03:110
14880993210,7cyclictest25675-21iostat_ios19:50:230
148682321,4sleep114884-21sh23:49:271
148372321,3sleep02220-21dbus-daemon20:46:360
102842321,4sleep110367-21memory20:35:231
29182310,9sleep01488099cyclictest20:15:240
185002311,5sleep00-21swapper/020:55:340
1488199317,20cyclictest0-21swapper/122:23:311
1488199315,4cyclictest81rcu_preempt21:07:551
1488199315,4cyclictest81rcu_preempt20:53:251
1488199315,22cyclictest0-21swapper/123:56:021
1488199312,15cyclictest29353-21users21:15:371
1488199311,5cyclictest81rcu_preempt21:28:211
1488199311,4cyclictest81rcu_preempt23:08:011
14881993113,14cyclictest22-21ksoftirqd/100:49:351
14881993111,7cyclictest0-21swapper/100:32:471
1488099319,6cyclictest12320-21ssh21:31:480
1488099312,25cyclictest16277-21users21:35:340
1488099311,4cyclictest81rcu_preempt21:25:300
14880993114,6cyclictest3-21ksoftirqd/021:45:570
14880993112,6cyclictest11756-21/usr/sbin/munin22:05:180
1488099311,16cyclictest11303-21ssh23:11:160
14880993110,6cyclictest26293-21ssh21:13:090
88472301,5sleep00-21swapper/000:15:380
73392301,5sleep00-21swapper/020:26:570
237782301,25sleep123790-21ssh23:25:111
177782301,4sleep10-21swapper/123:18:411
173342301,3sleep117357-21munin-node19:30:061
173342301,3sleep117357-21munin-node19:30:051
1488199304,22cyclictest0-21swapper/123:31:211
1488199304,22cyclictest0-21swapper/123:31:201
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional