You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2021-07-29 - 05:41

Intel(R) Atom(TM) Processor E3950 @ 1.60GHz, Linux 4.19.182-rt74 (Profile)

Latency plot of system in rack #d, slot #8
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -n -p90 -i200 -h400 -a0-2 -t3 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 100 highest latencies:
System rackdslot8.osadl.org (updated Thu Feb 06, 2020 10:49:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
121028690315,23cyclictest1284161-21munin-run08:40:011
121028690315,23cyclictest1284161-21munin-run08:40:001
121028590315,13cyclictest1428017-21perf11:35:010
1210286902514,6cyclictest1424479-21ptp4l-jitter11:30:031
121028690251,21cyclictest393-21systemd-journal12:40:011
121028690251,21cyclictest1612-21systemd-logind07:40:001
1210285902419,3cyclictest1358156-21munin-run10:10:010
121028590241,2cyclictest111rcu_preempt11:15:000
121028590241,11cyclictest1304765-21latency_hist09:05:010
1210287902319,2cyclictest0-21swapper/211:20:012
1210286902313,2cyclictest1382833-21taskset10:40:001
121028690231,19cyclictest393-21systemd-journal12:20:021
121028690231,15cyclictest1432254-21/usr/sbin/munin11:40:101
121028590230,11cyclictest1214348-21perf07:15:010
1210287902214,4cyclictest1464003-21netstat12:15:142
121028690226,3cyclictest14186252missed_timers11:20:151
121028690226,3cyclictest14186252missed_timers11:20:141
1210286902211,9cyclictest21-21ksoftirqd/108:55:041
121028590220,20cyclictest393-21systemd-journal09:00:020
121028590220,20cyclictest393-21systemd-journal07:57:470
121028590220,19cyclictest393-21systemd-journal09:40:110
1210287902116,3cyclictest1324380-21proc_pri09:25:172
1210287902115,4cyclictest1393769-21irqstats10:50:152
1210287902115,4cyclictest1338635-21sed09:45:052
1210287902115,4cyclictest1338635-21sed09:45:042
1210287902113,5cyclictest1304191-21grep09:00:212
121028690219,7cyclictest1509-21runrttasks11:49:331
1210286902116,3cyclictest21-21ksoftirqd/110:00:011
1210286902116,3cyclictest1274246-21grep08:25:111
1210286902116,3cyclictest1247232-21cut07:55:011
1210286902115,4cyclictest21-21ksoftirqd/111:15:011
1210286902115,4cyclictest1264491-21head08:15:041
1210286902114,4cyclictest1272077-21acpi08:25:021
1210286902113,4cyclictest1367705-21df_abs10:20:051
121028690211,2cyclictest111rcu_preempt10:25:181
121028690211,18cyclictest393-21systemd-journal09:44:441
1210286902110,7cyclictest1308827-21diskmemload09:30:091
121028690210,6cyclictest21-21ksoftirqd/111:25:081
121028590219,4cyclictest1431684-21unixbench_singl11:35:220
121028590219,4cyclictest1431684-21unixbench_singl11:35:210
121028590211,9cyclictest1391025-21munin-run10:50:010
1210285902115,4cyclictest1275645-21sshd08:26:360
121028590211,10cyclictest1478123-21acpi12:35:030
121028590210,9cyclictest13645972iostat10:15:100
121028590210,19cyclictest393-21systemd-journal10:25:000
121028590210,12cyclictest1465095-21systemd-udevd12:19:560
1210287902014,4cyclictest1274245-21unixbench_singl08:25:112
1210287902014,4cyclictest1229973-21sendmail_mailtr07:30:172
1210287902014,3cyclictest1358597-21open_files10:10:032
1210287902014,3cyclictest1297147-21cpu08:55:042
1210286902016,2cyclictest21-21ksoftirqd/110:05:011
1210286902015,3cyclictest21-21ksoftirqd/110:50:021
1210286902015,3cyclictest1465243-21/usr/sbin/munin12:20:051
1210286902015,3cyclictest1363371-21grep10:15:051
1210286902014,4cyclictest1330260-21cstates09:35:041
1210286902014,4cyclictest1254281-21wc08:00:161
1210286902014,4cyclictest1254281-21wc08:00:151
1210286902014,3cyclictest1262082-21wc08:10:121
1210286902013,4cyclictest1449625-21head12:00:041
1210286902013,4cyclictest1241001-21ls07:45:101
121028690201,16cyclictest1612-21systemd-logind11:00:001
121028690201,16cyclictest1612-21systemd-logind10:10:001
121028590203,15cyclictest1234850-21perf07:40:010
121028590201,16cyclictest1612-21systemd-logind12:00:010
121028590200,17cyclictest393-21systemd-journal07:25:010
121028790198,8cyclictest1449622-21cpuspeed_turbos12:00:052
1210287901914,3cyclictest1476702-21sendmail_mailst12:30:182
1210287901914,3cyclictest1308076-21sendmail_mailtr09:05:212
1210287901914,3cyclictest1308076-21sendmail_mailtr09:05:202
1210287901914,3cyclictest1244693-21cpuspeed07:50:082
1210287901913,4cyclictest1478778-21hwlatdetect12:35:052
1210287901913,4cyclictest1390177-21processes10:45:172
1210287901913,3cyclictest1409376-21grep11:10:092
1210287901912,4cyclictest1289707-21diskstats08:45:062
1210287901912,4cyclictest1281357-21cpuspeed_turbos08:35:042
1210287901912,4cyclictest1281357-21cpuspeed_turbos08:35:042
121028790190,8cyclictest0-21swapper/209:12:022
121028690198,6cyclictest157250irq/133-enp2s0-08:48:031
1210286901914,3cyclictest1457965-21wc12:10:061
1210286901914,3cyclictest1457965-21wc12:10:051
1210286901914,3cyclictest1448111-21grep11:55:211
1210286901914,3cyclictest1358241-21/usr/sbin/munin10:10:061
1210286901914,3cyclictest1235194-21/usr/sbin/munin07:40:161
1210286901913,4cyclictest1473436-21date12:30:011
1210286901913,4cyclictest1402447-21which11:00:171
1210286901913,4cyclictest1402447-21which11:00:161
1210286901913,4cyclictest1374714-21/usr/sbin/munin10:30:111
1210286901913,4cyclictest1324728-21sed09:25:201
1210286901913,3cyclictest1475524-21awk12:30:111
1210286901912,4cyclictest1441527-21head11:50:051
1210286901912,4cyclictest1339788-21sh09:45:111
1210286901912,4cyclictest1339788-21sh09:45:101
1210286901912,4cyclictest1280176-21cut08:35:011
121028690190,2cyclictest111rcu_preempt07:15:031
121028690190,16cyclictest393-21systemd-journal12:08:391
121028590197,7cyclictest412-21systemd-udevd12:20:120
1210285901915,2cyclictest1395119-21crond10:55:010
1210285901914,3cyclictest1452150-21ls12:00:200
1210285901914,3cyclictest1359706-21cut10:10:080
1210285901914,3cyclictest1329368-21sh09:35:010
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional