You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-05-17 - 23:16

Freescale i.MX6 Quad @996 MHz, Linux 5.16.2-rt19 (Profile)

Latency plot of system in rack #b, slot #7
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 100 highest latencies:
System rackbslot7 (updated Tue May 17, 2022 12:44:16)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2945999965,17cyclictest29460-21latency_hist07:08:170
2945999932,9cyclictest17633-21/usr/sbin/munin10:28:240
2945999932,26cyclictest17301-21uniq11:23:430
2945999932,17cyclictest5217-21taskset10:06:380
29479999140,34cyclictest29446-21cyclictest12:13:253
2945999913,15cyclictest10878-21latency_hist11:13:150
29459998920,63cyclictest0-21swapper/012:03:480
29459998919,7cyclictest0-21swapper/011:38:350
2945999873,9cyclictest12095-21taskset10:18:320
290192875,15sleep20-21swapper/207:04:362
2945999863,14cyclictest15098-21ssh09:27:080
29479998528,49cyclictest0-21swapper/309:48:143
29472998531,8cyclictest0-21swapper/209:18:152
2945999853,17cyclictest15637-21grep08:03:330
29459998525,54cyclictest0-21swapper/010:28:130
29479998434,43cyclictest5924-21mailstats11:03:503
2945999843,29cyclictest21427-21/usr/sbin/munin09:38:370
2945999843,18cyclictest4149-21sh11:58:130
2945999843,12cyclictest8721-21munin-run07:43:130
2945999842,17cyclictest23340-21/usr/sbin/munin10:38:300
2945999842,17cyclictest23340-21/usr/sbin/munin10:38:300
2945999842,11cyclictest392-21runrttasks12:28:450
29472998329,7cyclictest0-21swapper/207:18:132
2945999834,17cyclictest972-21grep09:58:470
2945999832,18cyclictest25195-21modprobe11:38:130
2945999832,17cyclictest32130-21seq11:48:490
2945999832,17cyclictest32130-21seq11:48:490
2945999832,16cyclictest10468-21fschecks_time09:18:490
2945999832,16cyclictest10468-21fschecks_time09:18:490
2945999832,15cyclictest2563-21df_inode10:58:390
29472998232,9cyclictest25869-21munin-plugin-st10:43:132
29472998232,9cyclictest25869-21munin-plugin-st10:43:132
2945999824,18cyclictest11535-21unixbench_singl07:48:470
2945999823,16cyclictest7847-21latency_hist11:08:150
2945999823,12cyclictest26108-21chrt08:33:470
2945999823,12cyclictest26108-21chrt08:33:470
2945999822,25cyclictest583-21sshd08:58:080
2945999822,17cyclictest19506-21sort12:23:160
2945999822,14cyclictest30035-21latency_hist08:48:160
29459998217,6cyclictest0-21swapper/012:38:160
29459998217,56cyclictest0-21swapper/012:23:240
29459998215,61cyclictest29446-21cyclictest07:08:350
2945999816,10cyclictest0-21swapper/008:28:430
2945999813,16cyclictest8665-21sh10:13:130
2945999813,14cyclictest27773-21grep08:38:470
29459998118,7cyclictest0-21swapper/011:48:130
29459998118,10cyclictest27521-21fschecks_time09:48:510
29459998117,7cyclictest0-21swapper/009:28:370
29479998024,8cyclictest0-21swapper/311:08:553
29479998022,11cyclictest0-21swapper/309:28:123
2945999806,64cyclictest0-21swapper/007:58:170
2945999803,18cyclictest13825-21/usr/sbin/munin07:58:450
2945999803,16cyclictest3539-21grep09:04:170
29459998025,47cyclictest0-21swapper/010:18:150
29459998025,47cyclictest0-21swapper/010:18:150
2945999802,19cyclictest5081-21ssh11:58:420
2945999802,19cyclictest5081-21ssh11:58:420
2945999802,13cyclictest14802-21sed12:14:070
29459998016,8cyclictest25298-21unixbench_singl09:43:500
2947299793,16cyclictest12987-21head10:18:502
2945999793,17cyclictest31490-21latency_hist08:53:160
2945999793,17cyclictest20323-21/usr/sbin/munin08:18:370
2945999793,14cyclictest20045-21modprobe08:18:140
2945999793,13cyclictest5039-21ssh09:08:580
2945999792,17cyclictest12179-21head11:13:510
2945999792,16cyclictest11590-21tail12:08:490
2945999792,13cyclictest14048-21/usr/sbin/munin11:18:420
29459997919,54cyclictest0-21swapper/007:23:480
29459997914,8cyclictest724-21sh10:54:310
29479997819,12cyclictest4860-21modprobe11:03:133
2945999783,17cyclictest32660-21latency_hist09:58:160
2945999783,13cyclictest1063-21grep07:18:350
2945999783,13cyclictest1063-21grep07:18:350
2945999782,19cyclictest392-21runrttasks11:29:570
2945999782,11cyclictest7251-21cpu09:13:300
29459997820,50cyclictest0-21swapper/008:13:140
2945999772,18cyclictest19562-21unixbench_multi09:33:490
2945999772,17cyclictest23301-21idleruntime-cro08:28:140
2945999772,17cyclictest23301-21idleruntime-cro08:28:140
2945999772,16cyclictest1136-21cut08:58:340
2945999772,14cyclictest28778-21latency_hist10:48:150
29479997626,7cyclictest0-21swapper/309:56:593
29479997623,8cyclictest0-21swapper/310:28:353
29479997619,9cyclictest0-21swapper/310:08:303
29467997660,9cyclictest6880-21latency_hist09:13:141
2945999763,16cyclictest5712-21/usr/sbin/munin07:33:400
2945999763,15cyclictest20902-21uname10:33:380
29479997526,6cyclictest0-21swapper/310:53:573
29479997515,14cyclictest0-21swapper/311:33:473
2945999753,17cyclictest392-21runrttasks07:48:150
29459997517,8cyclictest0-21swapper/010:48:470
29479997414,12cyclictest0-21swapper/307:28:363
29479997412,15cyclictest13711-21idleruntime-cro11:18:133
2947299742,23cyclictest7549-21ssh12:03:252
42342730,6sleep30-21swapper/311:58:153
29479997326,7cyclictest0-21swapper/309:08:173
29479997324,6cyclictest0-21swapper/311:23:463
29479997323,8cyclictest0-21swapper/308:48:163
29479997315,11cyclictest0-21swapper/312:18:263
2945999732,18cyclictest5446-21cron07:33:130
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional