You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-03 - 23:45

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 100 highest latencies:
System rack9slot4s (updated Wed Dec 03, 2025 12:59:02)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
21329912181185,29cyclictest352-21udevd07:52:411
21319911081092,14cyclictest22257-21kworker/0:307:25:150
19202728683,29sleep10-21swapper/107:23:311
17122608510,30sleep00-21swapper/007:21:200
213199170148,4cyclictest14142-21updatedb.mlocat07:52:420
213299129122,5cyclictest0-21swapper/107:25:151
777721030,4sleep0213199cyclictest10:47:130
62472970,21sleep03-21ksoftirqd/011:19:570
210242960,8sleep0213199cyclictest11:00:310
172202780,5sleep0111rcuc/008:00:250
131362650,4sleep0213199cyclictest10:20:010
213299455,4cyclictest81rcu_preempt09:57:121
2131994326,5cyclictest3-21ksoftirqd/012:41:270
244522421,3sleep024465-21ssh09:57:300
2131994224,6cyclictest3-21ksoftirqd/012:32:380
2131994023,6cyclictest3-21ksoftirqd/012:06:120
2131994022,6cyclictest3-21ksoftirqd/009:35:370
2131993921,6cyclictest3-21ksoftirqd/011:24:220
2132993813,10cyclictest22-21ksoftirqd/110:23:111
213199388,4cyclictest81rcu_preempt10:57:580
2131993821,6cyclictest3-21ksoftirqd/009:54:460
2131993821,5cyclictest3-21ksoftirqd/011:06:520
146232380,3sleep014620-21fschecks_count12:35:180
288152371,4sleep128831-21sh10:35:271
267332371,4sleep126744-21ssh12:14:431
213299378,4cyclictest81rcu_preempt09:10:001
2132993725,7cyclictest7280-21irqstats11:20:221
2132993711,4cyclictest81rcu_preempt10:15:111
213199372,4cyclictest81rcu_preempt09:22:570
2131993720,5cyclictest3-21ksoftirqd/012:22:470
213199371,14cyclictest23741-21runrttasks08:19:110
198272371,4sleep110340-21diskmemload09:52:471
140802370,8sleep0213199cyclictest12:00:320
213299369,23cyclictest0-21swapper/110:50:101
213299363,26cyclictest28838-21diskstats12:50:281
213199366,5cyclictest81rcu_preempt10:37:070
213199364,7cyclictest81rcu_preempt09:30:320
2131993619,6cyclictest3-21ksoftirqd/010:54:300
2131993619,6cyclictest3-21ksoftirqd/010:14:490
2131993619,6cyclictest3-21ksoftirqd/009:26:380
2131993619,6cyclictest3-21ksoftirqd/008:50:470
2131993618,6cyclictest3-21ksoftirqd/011:13:080
66262351,30sleep16640-21ssh12:26:271
317012351,29sleep131797-21sendmail_mailqu11:45:301
213299356,5cyclictest81rcu_preempt12:09:341
213199354,4cyclictest81rcu_preempt10:22:000
213199352,21cyclictest0-21swapper/007:37:050
2131993518,6cyclictest3-21ksoftirqd/010:33:380
2131993517,6cyclictest3-21ksoftirqd/012:45:170
151132351,30sleep115127-21ssh09:47:551
75162341,4sleep17530-21ssh11:54:471
213299345,4cyclictest81rcu_preempt10:50:011
213299343,4cyclictest81rcu_preempt07:38:181
2132993411,4cyclictest81rcu_preempt09:10:231
213199344,4cyclictest81rcu_preempt12:26:120
213199344,4cyclictest81rcu_preempt09:49:420
213199341,7cyclictest9972-21sshd09:06:230
2131993417,6cyclictest3-21ksoftirqd/012:50:020
2131993417,6cyclictest3-21ksoftirqd/010:08:150
2131993410,8cyclictest3-21ksoftirqd/012:12:030
199742340,4sleep0111rcuc/010:25:560
109712341,5sleep00-21swapper/011:57:480
213299332,27cyclictest40950irq/93-i91509:37:191
2132993318,12cyclictest22-21ksoftirqd/111:15:301
2132993315,15cyclictest22-21ksoftirqd/110:01:041
213299331,11cyclictest81rcu_preempt08:48:091
213199339,8cyclictest6735-21/usr/sbin/munin09:00:150
213199334,4cyclictest81rcu_preempt11:46:550
213199334,4cyclictest81rcu_preempt10:40:310
213199333,4cyclictest81rcu_preempt10:01:550
2131993319,5cyclictest3-21ksoftirqd/007:34:440
2131993318,5cyclictest3-21ksoftirqd/011:40:340
2131993316,6cyclictest3-21ksoftirqd/012:15:120
2131993315,6cyclictest3-21ksoftirqd/009:43:280
109932331,27sleep111103-21hddtemp_smartct07:45:161
213299327,21cyclictest0-21swapper/111:05:091
213299326,22cyclictest0-21swapper/112:49:201
2132993225,4cyclictest10387-21proc_pri12:30:271
2132993223,4cyclictest22-21ksoftirqd/111:00:311
213299322,25cyclictest0-21swapper/112:41:471
213299322,25cyclictest0-21swapper/110:28:431
213299321,4cyclictest81rcu_preempt11:32:221
213199323,4cyclictest81rcu_preempt09:15:560
2131993218,5cyclictest3-21ksoftirqd/007:44:580
2131993216,6cyclictest3-21ksoftirqd/008:56:280
2131993215,6cyclictest3-21ksoftirqd/009:14:270
186412321,26sleep018646-21diskstats08:05:150
213299316,4cyclictest81rcu_preempt11:35:251
213299315,4cyclictest81rcu_preempt08:59:551
213299313,13cyclictest0-21swapper/110:44:101
2132993125,4cyclictest137350irq/90-eth010:06:541
2132993122,4cyclictest22-21ksoftirqd/110:59:021
213299311,4cyclictest81rcu_preempt09:40:381
213299311,22cyclictest40950irq/93-i91509:32:401
213299311,17cyclictest6797-21ssh10:12:481
213299311,15cyclictest32284-21cpuspeed_turbos12:20:121
213199314,4cyclictest81rcu_preempt11:50:430
2131993116,5cyclictest3-21ksoftirqd/008:47:280
2131993116,4cyclictest3-21ksoftirqd/007:45:330
2131993115,6cyclictest3-21ksoftirqd/008:26:390
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional