You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-05 - 11:21

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack9slot4s (updated Fri Dec 05, 2025 00:59:03)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
289859911321107,19cyclictest29233-21wget19:25:111
289859911071077,27cyclictest30944-21snmpd20:15:271
289849910971065,22cyclictest22257-21kworker/0:319:25:120
287112565517,31sleep10-21swapper/119:22:451
288432545501,30sleep00-21swapper/019:24:070
2898499157102,36cyclictest17082-21ntp_states20:15:260
43462920,8sleep02898499cyclictest19:40:290
277312900,9sleep02898499cyclictest23:14:320
72702880,7sleep1201rcuc/122:52:481
72702880,7sleep1201rcuc/122:52:481
204372610,22sleep020439-21grep21:25:300
129522540,5sleep1201rcuc/123:32:001
133442461,27sleep013433-21gzip20:05:270
28985994424,8cyclictest22-21ksoftirqd/121:25:291
28985994417,16cyclictest22-21ksoftirqd/121:58:381
28985994326,5cyclictest22-21ksoftirqd/122:15:231
28985994316,11cyclictest22-21ksoftirqd/121:45:341
156112430,3sleep0111rcuc/022:27:590
28985994225,6cyclictest22-21ksoftirqd/121:12:401
28985994125,5cyclictest22-21ksoftirqd/100:02:581
28985994123,4cyclictest22-21ksoftirqd/100:33:531
28985994122,5cyclictest22-21ksoftirqd/123:58:551
286822410,5sleep1201rcuc/121:35:011
28985994025,4cyclictest22-21ksoftirqd/123:25:441
28985994024,6cyclictest22-21ksoftirqd/122:37:531
28985994024,5cyclictest22-21ksoftirqd/100:41:581
28985994023,6cyclictest22-21ksoftirqd/121:17:461
2898599392,8cyclictest3020-21runrttasks00:45:361
28985993925,4cyclictest22-21ksoftirqd/100:37:231
28985993924,4cyclictest22-21ksoftirqd/123:50:161
28985993924,4cyclictest22-21ksoftirqd/121:52:501
28985993924,4cyclictest22-21ksoftirqd/121:36:501
28985993923,6cyclictest22-21ksoftirqd/120:47:191
28985993923,6cyclictest22-21ksoftirqd/119:37:511
28985993923,5cyclictest22-21ksoftirqd/123:44:521
28985993923,5cyclictest22-21ksoftirqd/122:28:501
28985993923,5cyclictest22-21ksoftirqd/122:21:591
28985993923,5cyclictest22-21ksoftirqd/100:11:461
2898599392,25cyclictest0-21swapper/100:06:431
28985993917,6cyclictest22-21ksoftirqd/121:21:121
28985993824,5cyclictest22-21ksoftirqd/123:46:291
28985993824,5cyclictest22-21ksoftirqd/122:07:211
28985993823,4cyclictest22-21ksoftirqd/123:21:261
28985993820,8cyclictest22-21ksoftirqd/123:05:051
28985993820,8cyclictest22-21ksoftirqd/121:44:181
28985993820,5cyclictest22-21ksoftirqd/100:22:061
28985993819,5cyclictest22-21ksoftirqd/100:27:151
28985993817,6cyclictest22-21ksoftirqd/100:16:301
28985993815,6cyclictest22-21ksoftirqd/122:40:161
2898499381,16cyclictest0-21swapper/022:13:380
2898499381,11cyclictest81rcu_preempt00:47:010
256892381,4sleep125793-21aten_r4power_vo20:40:141
68972371,32sleep06204-21gdbus22:19:220
28985993723,5cyclictest22-21ksoftirqd/122:59:081
28985993723,5cyclictest22-21ksoftirqd/122:32:011
28985993721,5cyclictest22-21ksoftirqd/122:12:541
28985993721,5cyclictest22-21ksoftirqd/122:03:441
2898499374,5cyclictest81rcu_preempt21:10:280
2898499374,10cyclictest81rcu_preempt23:26:490
20682371,4sleep12080-21ssh22:47:251
55192361,31sleep05533-21ssh00:31:040
28985993623,4cyclictest22-21ksoftirqd/123:10:131
28985993621,6cyclictest22-21ksoftirqd/120:37:031
28985993613,7cyclictest22-21ksoftirqd/123:15:191
2898499365,4cyclictest9918-21ssh22:21:430
28985993519,6cyclictest22-21ksoftirqd/119:32:061
28985993518,6cyclictest22-21ksoftirqd/120:25:011
28985993511,6cyclictest0-21swapper/119:50:281
2898499354,4cyclictest81rcu_preempt22:59:370
2898499353,5cyclictest81rcu_preempt23:45:110
2898499353,5cyclictest81rcu_preempt23:35:080
2898499351,13cyclictest7901-21sort19:50:280
226142351,29sleep022647-21hddtemp_smartct22:35:180
28985993418,6cyclictest22-21ksoftirqd/119:59:121
2898499346,4cyclictest81rcu_preempt23:20:190
2898499345,5cyclictest81rcu_preempt22:51:080
2898499345,5cyclictest81rcu_preempt22:51:080
2898499345,5cyclictest81rcu_preempt00:15:570
2898499344,4cyclictest81rcu_preempt23:55:030
2898499343,4cyclictest81rcu_preempt21:31:520
2898499343,14cyclictest0-21swapper/000:21:570
2898499342,5cyclictest81rcu_preempt00:05:180
177072341,30sleep017718-21ssh23:03:590
2898599333,26cyclictest3579-21smartctl19:40:171
28985993319,4cyclictest22-21ksoftirqd/123:37:181
28985993319,4cyclictest22-21ksoftirqd/100:52:581
2898499336,4cyclictest81rcu_preempt00:28:350
2898499335,4cyclictest81rcu_preempt21:17:270
2898499334,5cyclictest81rcu_preempt23:41:300
2898499333,4cyclictest81rcu_preempt21:41:430
28984993310,6cyclictest14298-21ntp_states00:40:240
28985993218,4cyclictest22-21ksoftirqd/123:02:291
2898499326,5cyclictest81rcu_preempt23:53:010
2898499325,4cyclictest81rcu_preempt00:01:300
2898499324,4cyclictest81rcu_preempt22:46:490
2898499324,4cyclictest81rcu_preempt21:22:380
2898499323,3cyclictest81rcu_preempt23:18:520
2898499321,4cyclictest81rcu_preempt23:30:200
2898499321,4cyclictest81rcu_preempt00:35:280
208372321,27sleep020964-21df22:00:130
186772321,27sleep018730-21/usr/sbin/munin20:20:200
179342321,5sleep00-21swapper/021:56:270
28985993116,5cyclictest22-21ksoftirqd/119:48:511
2898499319,17cyclictest3020-21runrttasks00:51:200
2898499315,11cyclictest20744-21missed_timers20:25:240
2898499311,4cyclictest81rcu_preempt23:09:410
2898499311,4cyclictest81rcu_preempt22:44:200
2898499311,4cyclictest81rcu_preempt21:45:080
2898499311,4cyclictest81rcu_preempt19:32:320
2898499311,13cyclictest19902-21watchsnmpd00:13:000
149882311,26sleep115059-21memory20:10:241
2898599305,13cyclictest40950irq/93-i91520:20:231
2898499301,4cyclictest81rcu_preempt21:51:260
28984993010,7cyclictest32063-21ssh21:37:590
28984993010,13cyclictest26305-21memory22:05:240
16592301,5sleep10-21swapper/121:00:331
2898599293,13cyclictest0-21swapper/120:00:521
2898599291,9cyclictest32499-21unixbench_multi20:55:311
28985992915,4cyclictest22-21ksoftirqd/120:32:581
2898499292,7cyclictest26522-21ntp_states20:40:280
2898499291,17cyclictest0-21swapper/021:07:320
2898499291,14cyclictest32500-21unixbench_multi20:55:310
2898499291,13cyclictest16164-21wc20:15:020
28984992910,6cyclictest11118-21/usr/sbin/munin20:00:200
289352291,5sleep00-21swapper/020:46:300
201122291,5sleep00-21swapper/022:32:340
307992281,5sleep00-21swapper/020:51:190
2898599284,17cyclictest30136-21memory20:50:251
2898499282,13cyclictest0-21swapper/019:45:260
24872281,5sleep00-21swapper/019:36:360
23612281,5sleep00-21swapper/021:03:330
141632281,5sleep10-21swapper/120:09:351
2898499277,7cyclictest9207-21hddtemp_smartct19:55:190
2898499272,12cyclictest24081-21diskstats20:35:170
219052271,5sleep00-21swapper/020:30:110
2898599262,12cyclictest2376-21NetworkManager21:09:291
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional