You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-04 - 12:26

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack9slot4s (updated Fri Jul 04, 2025 00:59:04)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
15799911761140,26cyclictest0-21swapper/019:28:030
14062655606,32sleep10-21swapper/119:23:561
13712597552,31sleep00-21swapper/019:23:320
158099235227,5cyclictest0-21swapper/119:28:021
87372950,7sleep00-21swapper/000:28:270
175462950,6sleep081rcu_preempt21:50:300
207112880,4sleep081rcu_preempt23:00:260
305852680,11sleep0111rcuc/020:40:000
157999648,35cyclictest22718-21sh00:42:430
1579996410,6cyclictest9482-21diskmemload23:28:110
157999621,14cyclictest2257-21ssh22:41:240
157999604,9cyclictest19690-21users00:05:320
157999593,7cyclictest17208-21iostat23:30:190
157999592,12cyclictest0-21swapper/021:11:160
157999591,14cyclictest24712-21grep00:45:150
157999584,7cyclictest31690-21memory22:05:230
157999584,47cyclictest15949-21diskstats20:00:190
250832570,34sleep0157999cyclictest22:31:350
157999571,15cyclictest21053-21date21:55:010
157999571,14cyclictest11613-21rm22:50:450
1579995710,6cyclictest22016-21cat20:15:270
157999565,5cyclictest26968-21iostat23:40:220
157999563,7cyclictest20229-21iostat_ios21:20:240
157999562,7cyclictest13952-21hddtemp_smartct00:00:200
157999562,13cyclictest11918-21cut19:50:130
157999561,51cyclictest27736-21/usr/sbin/munin22:35:290
157999554,12cyclictest9204-21/usr/sbin/munin22:15:250
157999552,49cyclictest32450-21cron00:20:000
157999551,40cyclictest11319-21taskset19:45:580
157999551,13cyclictest12069-21ls21:45:190
157999549,5cyclictest3-21ksoftirqd/023:08:440
157999543,47cyclictest0-21swapper/000:24:480
157999531,4cyclictest2376-21NetworkManager00:35:450
1579995313,6cyclictest20366-21cat20:10:290
1579995311,4cyclictest9457-21ntp_states23:55:250
157999524,8cyclictest25039-21iostat_ios21:25:210
157999522,6cyclictest23393-21df_inode00:10:180
157999522,14cyclictest2290-21tune2fs23:15:160
157999521,15cyclictest7644-21ssh23:54:090
157999521,13cyclictest17864-21ssh21:18:040
157999521,12cyclictest18618-21timerandwakeup20:05:300
157999521,12cyclictest18618-21timerandwakeup20:05:300
157999518,11cyclictest2220-21dbus-daemon20:30:500
157999512,12cyclictest9482-21diskmemload22:24:220
1579995113,5cyclictest29748-21df23:10:160
1579995113,5cyclictest29748-21df23:10:150
29462500,5sleep02939-21python21:35:320
157999509,5cyclictest31257-21ssh21:31:200
157999508,6cyclictest5564-21runrttasks20:58:360
157999504,8cyclictest15011-21users19:55:290
157999504,7cyclictest32084-21ntp_states23:45:260
157999502,16cyclictest4967-21/usr/sbin/munin19:30:240
157999502,11cyclictest26739-21iostat_ios22:00:220
157999502,11cyclictest26739-21iostat_ios22:00:210
1579995012,7cyclictest7161-21df_inode23:20:150
1579995011,4cyclictest0-21swapper/000:30:140
157999501,12cyclictest7941-21cron19:39:590
157999494,9cyclictest31389-21memory20:40:240
157999493,4cyclictest15548-21ssh22:55:230
157999492,16cyclictest5599-21chrt22:11:310
1579994911,4cyclictest0-21swapper/023:36:120
1579994910,4cyclictest0-21swapper/021:40:280
157999482,6cyclictest6204-21gdbus21:06:580
157999482,14cyclictest8684-21/usr/sbin/munin19:40:200
1579994815,29cyclictest32141-21cron00:53:000
1579994812,6cyclictest2220-21dbus-daemon22:25:280
1579994812,6cyclictest2019-21/usr/sbin/munin20:50:160
157999481,15cyclictest25837-21/usr/sbin/munin20:25:230
157999471,14cyclictest2220-21dbus-daemon20:22:420
219082461,32sleep121922-21ssh21:21:541
157999463,9cyclictest6187-21diskstats21:00:190
157999463,9cyclictest1858-21munin-run20:50:010
157999463,9cyclictest1858-21munin-run20:50:000
157999463,7cyclictest5467-21diskstats22:45:160
288922420,8sleep1158099cyclictest00:50:011
163162410,10sleep1158099cyclictest22:55:331
1580994114,10cyclictest22-21ksoftirqd/100:10:391
1580993915,7cyclictest22-21ksoftirqd/120:53:311
1580993825,9cyclictest6703-21latency_hist23:20:011
158099373,27cyclictest16538-21/usr/sbin/munin23:30:321
158099371,4cyclictest81rcu_preempt23:46:491
89512361,6sleep10-21swapper/122:48:571
158099365,4cyclictest81rcu_preempt22:07:531
158099364,5cyclictest81rcu_preempt21:42:541
1580993614,8cyclictest22-21ksoftirqd/123:14:171
1580993614,8cyclictest22-21ksoftirqd/123:14:161
158099357,5cyclictest81rcu_preempt21:55:121
1580993517,7cyclictest22-21ksoftirqd/119:45:141
66722341,29sleep16722-21ssh23:20:011
247622341,29sleep16203-21packagekitd23:38:161
174422341,29sleep117453-21ssh00:04:191
162602341,4sleep116342-21memory20:00:251
158099344,8cyclictest0-21swapper/123:02:221
158099342,4cyclictest81rcu_preempt23:05:191
44782330,9sleep1158099cyclictest00:24:181
158099338,13cyclictest0-21swapper/122:27:251
158099336,4cyclictest81rcu_preempt20:17:311
158099335,4cyclictest81rcu_preempt20:14:081
158099335,24cyclictest0-21swapper/100:29:091
158099333,4cyclictest81rcu_preempt23:59:031
1580993322,6cyclictest13583-21cron19:55:001
1580993311,18cyclictest0-21swapper/100:18:361
158099327,21cyclictest0-21swapper/121:19:501
158099322,4cyclictest81rcu_preempt22:02:171
158099322,4cyclictest81rcu_preempt22:02:161
158099321,7cyclictest15159-21runrttasks19:56:541
1580993215,10cyclictest22-21ksoftirqd/100:37:571
1580993210,7cyclictest22-21ksoftirqd/120:25:231
158099314,23cyclictest0-21swapper/122:43:221
158099312,25cyclictest0-21swapper/121:46:031
158099311,9cyclictest15068-21ssh22:21:091
158099311,6cyclictest13024-21ssh00:32:411
1580993116,11cyclictest22-21ksoftirqd/122:16:041
158099311,4cyclictest81rcu_preempt21:31:331
158099311,4cyclictest81rcu_preempt21:13:311
158099311,4cyclictest81rcu_preempt00:50:161
158099311,25cyclictest0-21swapper/100:07:561
158099311,17cyclictest13584-21ssh22:53:351
158099302,14cyclictest28301-21hddtemp_smartct22:35:191
158099301,4cyclictest81rcu_preempt21:50:021
1580993014,4cyclictest22-21ksoftirqd/120:40:011
1580993013,14cyclictest22-21ksoftirqd/123:26:401
73042291,5sleep10-21swapper/121:02:011
203282290,5sleep120326-21ssh00:40:191
158099296,4cyclictest81rcu_preempt20:34:341
158099293,4cyclictest81rcu_preempt19:31:421
158099292,23cyclictest0-21swapper/121:28:001
1580992917,9cyclictest22-21ksoftirqd/121:35:241
158099291,17cyclictest7731-21ssh23:54:181
158099291,14cyclictest8358-21iostat21:05:211
158099291,14cyclictest26334-21ssh22:33:231
158099288,5cyclictest81rcu_preempt23:40:221
1580992819,6cyclictest22-21ksoftirqd/122:10:451
158099281,13cyclictest5822-21latency_hist21:00:011
1580992810,6cyclictest22-21ksoftirqd/120:45:281
1580992810,6cyclictest22-21ksoftirqd/120:45:271
94062271,5sleep10-21swapper/119:40:341
158099278,5cyclictest6339-21cpu19:35:131
244692261,5sleep10-21swapper/120:20:431
188362261,5sleep10-21swapper/120:05:521
188362261,5sleep10-21swapper/120:05:521
1580992617,6cyclictest22-21ksoftirqd/120:40:051
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional