You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-06-07 - 11:18

x86 Intel Atom Z530 @1600 MHz, Linux 3.12.61-rt81 (Profile)

Latency plot of system in rack #1, slot #8
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 100 highest latencies:
System rack1slot8.osadl.org (updated Wed Jun 07, 2023 00:44:00)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
154542712688,17sleep00-21swapper/019:07:590
1569499667557,78cyclictest0-21swapper/019:19:280
1569599609478,33cyclictest0-21swapper/119:19:281
153812588565,16sleep10-21swapper/119:07:181
211999900,4rtkit-daemon2118-21rtkit-daemon22:01:130
15694998315,58cyclictest0-21swapper/023:44:410
1569499822,6cyclictest9819-21ssh00:20:430
211999800,4rtkit-daemon2118-21rtkit-daemon23:45:511
211999800,4rtkit-daemon2118-21rtkit-daemon22:47:281
1569499803,6cyclictest18399-21yumBackend.py22:49:090
211999790,5rtkit-daemon2118-21rtkit-daemon23:17:401
211999790,4rtkit-daemon2118-21rtkit-daemon20:00:311
211999790,4rtkit-daemon2118-21rtkit-daemon20:00:301
15694997954,18cyclictest0-21swapper/022:36:250
211999780,5rtkit-daemon2118-21rtkit-daemon00:05:201
211999780,4rtkit-daemon2118-21rtkit-daemon23:03:251
1569499786,66cyclictest1432-21ssh23:14:340
211999770,5rtkit-daemon2118-21rtkit-daemon00:00:011
211999770,4rtkit-daemon2118-21rtkit-daemon19:30:261
186422770,5sleep118641-21idleruntime20:55:261
1569499776,64cyclictest0-21swapper/000:00:200
1569499775,65cyclictest0-21swapper/022:59:150
1569499772,6cyclictest22753-21latency_hist23:50:020
211999760,6rtkit-daemon0-21swapper/021:55:530
211999760,4rtkit-daemon2118-21rtkit-daemon19:55:201
1569499763,7cyclictest18911-21munin-node20:55:300
1569499753,7cyclictest10064-21munin-node20:30:270
1569499753,64cyclictest23212-21diskmemload21:50:240
1569499753,52cyclictest3424-21ssh21:30:230
1569499752,6cyclictest29108-21latency_hist21:20:010
1569499749,36cyclictest0-21swapper/000:25:210
1569499743,7cyclictest19622-21sh22:50:240
278532730,5sleep027851-21nfs_client22:10:310
1569499737,39cyclictest0-21swapper/022:20:310
1569499737,39cyclictest0-21swapper/022:20:300
1569499732,9cyclictest6693-21crond22:30:000
1569499732,6cyclictest15023-21yum23:35:350
1569499732,53cyclictest27057-21latency_hist22:10:010
211999720,4rtkit-daemon2118-21rtkit-daemon21:49:371
1569499723,8cyclictest5097-21ssh00:13:450
1569499723,44cyclictest12099-21munin-node21:45:320
15694997210,55cyclictest1442-21runrttasks19:35:000
211999710,4rtkit-daemon2118-21rtkit-daemon21:20:481
1569499719,48cyclictest27210-21ssh23:03:090
1569499715,59cyclictest0-21swapper/021:30:020
1569499715,59cyclictest0-21swapper/020:14:020
1569499713,8cyclictest28990-21munin-run19:50:010
1569499713,7cyclictest20930-21grep21:00:290
1569499713,6cyclictest16603-21runrttasks00:32:070
15694997127,38cyclictest0-21swapper/021:37:360
1569499712,9cyclictest30935-21munin-run23:10:010
1569499712,9cyclictest30935-21munin-run23:10:010
1569499712,61cyclictest11537-21ssh21:44:120
1569499712,6cyclictest3287-21latency_hist20:10:010
1569499712,17cyclictest4414-21ssh23:19:250
164542700,4sleep116455-21ssh23:39:241
1569499705,58cyclictest0-21swapper/022:44:470
1569499704,59cyclictest25323-21ssh21:12:530
1569499703,8cyclictest30401-21ssh21:20:330
1569499703,7cyclictest4989-21ssh23:20:200
1569499703,7cyclictest25667-21munin-run19:40:010
1569499702,6cyclictest23623-21ssh23:50:250
1569499702,6cyclictest1442-21runrttasks23:27:200
1569499702,42cyclictest30554-21gltestperf22:15:290
1569499693,7cyclictest32385-21hwlatdetect00:05:220
1569499692,7cyclictest18327-21ssh00:35:230
1569499692,61cyclictest30191-21kworker/0:222:34:200
1569499692,59cyclictest6388-21needreboot00:15:270
1569499683,7cyclictest19156-21munin-node19:20:340
1569499683,6cyclictest27178-21ksmtuned23:55:420
1569499675,55cyclictest0-21swapper/023:30:250
1569499675,48cyclictest0-21swapper/019:11:210
1569499673,8cyclictest27302-21awk19:44:550
1569499672,6cyclictest8718-21if_err_eth020:25:200
1569499663,40cyclictest23204-21crond21:10:010
15694996618,41cyclictest746-21lldpad19:50:280
15694996610,50cyclictest0-21swapper/020:19:300
211999650,4rtkit-daemon2118-21rtkit-daemon20:39:020
1569499646,42cyclictest0-21swapper/020:00:240
1569499646,42cyclictest0-21swapper/020:00:230
1569499643,53cyclictest13953-21sshd20:40:270
1569499642,6cyclictest18171-21latency_hist20:55:000
1569499633,7cyclictest20806-21munin-node19:25:210
1569499632,7cyclictest32298-21munin-run20:00:010
15694996025,29cyclictest0-21swapper/020:20:290
1569499592,7cyclictest15024-21munin-node20:45:160
205782574,13sleep10-21swapper/100:39:371
124332574,13sleep10-21swapper/123:31:431
74662564,13sleep10-21swapper/121:36:221
211999560,4rtkit-daemon2118-21rtkit-daemon22:51:451
96222554,14sleep10-21swapper/120:27:161
253952554,13sleep10-21swapper/122:05:471
15695995516,28cyclictest0-21swapper/123:44:041
116452552,6sleep111691-21ssh00:25:011
304672544,9sleep10-21swapper/123:08:471
304672544,9sleep10-21swapper/123:08:461
211999540,4rtkit-daemon2118-21rtkit-daemon23:28:451
207912544,12sleep10-21swapper/119:25:091
1569599544,44cyclictest0-21swapper/119:14:001
180142534,14sleep10-21swapper/120:53:251
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional