You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-03 - 11:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 100 highest latencies:
System rack9slot4s (updated Mon Nov 03, 2025 00:59:04)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
20019911791129,30cyclictest1942-21latency_hist19:25:011
20019911591131,25cyclictest3964-21latency_hist19:30:011
20009911311099,21cyclictest0-21swapper/019:25:070
15272563516,31sleep00-21swapper/019:20:470
292432920,9sleep1200199cyclictest20:35:111
121292840,4sleep00-21swapper/022:51:590
54702820,5sleep05464-21ssh22:45:190
311142820,4sleep03-21ksoftirqd/021:30:430
215222710,3sleep121523-21cpuspeed_turbos21:55:151
167682660,5sleep041ktimersoftd/000:37:330
190162650,4sleep0111rcuc/021:52:050
47542620,22sleep03-21ksoftirqd/000:25:190
71352580,8sleep1200199cyclictest19:35:181
165212540,9sleep0200099cyclictest22:56:210
254352520,2sleep125434-21wc23:05:331
107252510,8sleep0200099cyclictest19:45:170
296422430,7sleep1200199cyclictest21:30:141
52262420,5sleep0111rcuc/020:55:240
115752420,5sleep041ktimersoftd/022:18:140
63892411,6sleep130944-21snmpd00:26:301
2001994114,6cyclictest34242grep21:35:311
200099411,14cyclictest8271-21users21:40:310
87112391,34sleep18782-21if_eth021:05:211
200099384,4cyclictest81rcu_preempt22:35:130
298722371,32sleep029884-21ssh00:17:390
2001993722,4cyclictest22-21ksoftirqd/123:18:541
56552361,31sleep05666-21ssh22:11:390
293202360,8sleep1200199cyclictest23:43:511
279392361,3sleep10-21swapper/121:28:071
200199365,4cyclictest81rcu_preempt00:00:041
2001993612,4cyclictest18703-21unixbench_multi00:05:321
2000993614,7cyclictest22509-21diskstats00:10:190
2000993614,7cyclictest22509-21diskstats00:10:190
200199356,18cyclictest0-21swapper/100:13:221
200199356,18cyclictest0-21swapper/100:13:221
200199355,18cyclictest0-21swapper/122:46:491
200199351,18cyclictest9837-21ssh00:30:241
200099353,5cyclictest81rcu_preempt23:39:440
200099352,15cyclictest19058-21df_inode00:40:150
2000993510,13cyclictest11691-21hddtemp_smartct23:25:210
65302340,3sleep1211ktimersoftd/123:20:151
2001993415,15cyclictest22-21ksoftirqd/100:35:391
200099347,5cyclictest81rcu_preempt23:53:370
200099344,4cyclictest81rcu_preempt21:47:200
200099341,7cyclictest556-21ssh22:06:250
200099341,20cyclictest10459-21ssh23:57:250
103962341,4sleep110409-21sh23:57:211
200199338,22cyclictest0-21swapper/122:30:161
200199332,26cyclictest0-21swapper/121:42:271
200199332,16cyclictest12103-21ntp_states23:25:291
200199331,7cyclictest16323-21cat21:50:001
200199331,17cyclictest5035-21irqstats20:55:211
200099336,5cyclictest81rcu_preempt23:32:520
200099333,4cyclictest81rcu_preempt00:31:130
200099331,7cyclictest24510-21ssh21:58:040
2000993313,6cyclictest26316-21ntp_states20:25:260
200099331,19cyclictest5049-21df_inode19:30:140
2000993310,6cyclictest26421-21udevd22:33:480
2000993310,11cyclictest9407-21proc_pri19:40:270
130152331,3sleep113029-21ssh21:12:281
200199329,13cyclictest40950irq/93-i91522:11:201
200199324,5cyclictest81rcu_preempt21:52:191
2001993217,12cyclictest22-21ksoftirqd/100:52:121
200199321,18cyclictest0-21swapper/122:05:291
200099328,5cyclictest81rcu_preempt22:03:170
200099325,5cyclictest81rcu_preempt23:14:590
200099323,4cyclictest81rcu_preempt21:11:390
200099323,3cyclictest81rcu_preempt23:15:240
200099323,16cyclictest0-21swapper/021:20:580
200199317,6cyclictest20546-21wc20:10:231
200199316,5cyclictest40950irq/93-i91523:50:161
200199316,21cyclictest0-21swapper/100:24:091
200199315,4cyclictest81rcu_preempt21:17:431
200199315,4cyclictest81rcu_preempt20:09:551
2001993121,6cyclictest32321-21ssh22:40:081
200199311,8cyclictest15912-21ssh23:30:111
200199311,17cyclictest12778-21ssh22:52:541
200099319,12cyclictest0-21swapper/021:35:290
200099316,4cyclictest81rcu_preempt23:20:220
200099315,5cyclictest81rcu_preempt00:50:200
200099315,4cyclictest81rcu_preempt23:05:010
200099312,4cyclictest81rcu_preempt23:46:110
2000993113,5cyclictest607-21/usr/sbin/munin22:40:220
200099311,13cyclictest13675-21tr00:00:290
305742301,5sleep00-21swapper/020:36:060
211882301,5sleep00-21swapper/000:09:030
200199308,8cyclictest8563-21taskset22:15:151
200199305,12cyclictest0-21swapper/120:00:531
200199303,4cyclictest81rcu_preempt19:32:271
200199302,7cyclictest31752-21kworker/1:200:16:031
2001993024,4cyclictest137350irq/90-eth022:00:271
200199302,17cyclictest0-21swapper/123:00:581
200199301,4cyclictest81rcu_preempt22:40:001
2001993012,5cyclictest3020-21runrttasks19:48:411
2001993012,15cyclictest0-21swapper/123:46:291
200199301,16cyclictest22911-21ssh00:44:321
2001993011,15cyclictest20658-21latency21:20:261
200099309,16cyclictest0-21swapper/020:10:010
200099304,5cyclictest81rcu_preempt23:07:050
200099304,4cyclictest81rcu_preempt00:46:080
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional