You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2021-09-17 - 21:55

Intel(R) Celeron(R) CPU N3350 @ 1.10GHz, Linux 5.9.1-rt18 (Profile)

Latency plot of system in rack #c, slot #4
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the alll highest latencies:
System rackcslot4.osadl.org (updated Fri Sep 10, 2021 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2137699353,12cyclictest906-21apt-get11:30:271
2137699343,11cyclictest14526-21apt-get08:35:221
2137699325,7cyclictest30237-21apt-get09:30:181
2137699322,10cyclictest24629-21apt-get09:10:151
2137699320,11cyclictest17507-21apt-get12:25:181
2137699302,10cyclictest5538-21apt-get08:05:161
2137699300,15cyclictest21786-21/usr/sbin/munin10:50:221
2137699300,10cyclictest4106-21apt-get08:00:171
2137699292,8cyclictest11325-21apt-get10:15:131
2137699284,9cyclictest30992-21sed11:20:251
2137699282,12cyclictest2230-21sed09:40:291
2137699274,9cyclictest4153-21dpkg11:40:021
2137699274,8cyclictest28049-21latency09:20:221
2137699271,12cyclictest20817-21cat10:45:271
2137699264,7cyclictest15346-21grep12:15:261
2137699263,8cyclictest4734-21sed09:50:241
2137699254,6cyclictest23047-21munin-run09:04:591
2137699253,6cyclictest27107-21cat11:05:291
2137699253,6cyclictest17765-21tune2fs10:35:221
2137699253,15cyclictest23225-21apt-esm-hook09:05:191
2137699252,8cyclictest19899-21sed10:40:271
2137699252,7cyclictest19-21ksoftirqd/107:50:181
2137699252,6cyclictest19-21ksoftirqd/107:40:331
2137699244,6cyclictest13793-21sed08:30:341
2137699243,14cyclictest12796-21apt-esm-hook10:20:181
2137699242,8cyclictest9424-21sed08:15:251
2137699242,8cyclictest7453-21cat10:00:211
2137699242,8cyclictest17267-21unixbench_singl12:20:311
2137699242,7cyclictest717-21sed07:45:301
2137699242,7cyclictest19-21ksoftirqd/111:15:261
2137699242,6cyclictest6608-21cat11:45:271
2137699242,6cyclictest23924-21sed07:15:291
2137699241,9cyclictest20168-21mailstats12:30:261
2137699241,9cyclictest20168-21mailstats12:30:261
2137699241,10cyclictest2642-21/usr/sbin/munin07:55:301
2137699240,9cyclictest26279-21/usr/sbin/munin07:25:251
2137699240,9cyclictest19881-21cat08:50:301
2137599249,9cyclictest21910-21apt-get07:10:190
2137599242,15cyclictest32187-21apt-get09:35:180
2137599242,15cyclictest16048-40apt-check12:20:110
21375992410,9cyclictest21763-21apt-get09:00:190
2137699232,7cyclictest19-21ksoftirqd/111:50:261
2137699232,6cyclictest19-21ksoftirqd/108:25:311
2137699232,5cyclictest16966-21sed08:40:331
2137699232,14cyclictest20754-21apt-esm-hook12:35:161
2137699232,14cyclictest14230-21apt-esm-hook10:25:151
2137699231,9cyclictest19-21ksoftirqd/107:35:161
2137699230,9cyclictest19-21ksoftirqd/112:10:251
2137699230,6cyclictest23252-21dpkg07:15:001
2137599238,9cyclictest26065-21apt-get09:15:180
2137599238,9cyclictest26065-21apt-get09:15:170
2137599233,14cyclictest5564-21apt-get09:55:160
2137599232,15cyclictest24849-21apt-get07:20:220
2137599232,15cyclictest20346-21apt-get10:45:160
2137599232,15cyclictest17457-21apt-esm-hook08:45:150
2137599232,14cyclictest5549-21apt-esm-hook08:05:160
21375992314,7cyclictest1386-21cat11:30:310
2137699222,7cyclictest19-21ksoftirqd/109:35:251
2137699222,6cyclictest3713-21sed09:45:271
2137699222,6cyclictest21271-21cat08:55:271
2137699222,6cyclictest19-21ksoftirqd/110:55:261
2137699222,6cyclictest19-21ksoftirqd/108:45:271
2137699222,14cyclictest9942-21apt-get12:00:171
2137699222,14cyclictest9902-21apt-esm-hook10:10:121
2137699222,14cyclictest8469-21apt-esm-hook10:05:151
2137699222,13cyclictest24689-21apt-esm-hook11:00:131
2137699221,6cyclictest9087-21sed11:55:251
2137699221,6cyclictest19-21ksoftirqd/111:35:261
2137699221,6cyclictest19-21ksoftirqd/111:25:191
2137699221,6cyclictest16694-21cat10:30:301
2137699221,5cyclictest6118-21cat09:55:211
2137699220,8cyclictest28538-21switchtime11:10:271
2137699220,7cyclictest29561-21cat09:25:221
2137699220,7cyclictest26594-21latency09:15:231
2137699220,7cyclictest26594-21latency09:15:221
2137699220,7cyclictest10945-21cat08:20:291
2137599229,9cyclictest15985-21apt-get08:40:180
2137599222,14cyclictest7048-21apt-get11:50:140
2137599222,14cyclictest30736-40apt-check09:30:100
2137599222,14cyclictest20295-21apt-get08:55:150
2137599222,14cyclictest16944-21apt-get10:35:180
2137599222,14cyclictest12785-21apt-get10:20:170
2137599222,14cyclictest11354-21apt-esm-hook10:15:130
2137599222,14cyclictest11299-21apt-get08:25:140
2137599220,6cyclictest101rcuc/011:25:010
2137699213,12cyclictest24877-21apt-esm-hook07:20:221
2137699212,13cyclictest11408-21apt-esm-hook12:05:161
2137699211,14cyclictest27768-21apt-esm-hook07:30:191
2137599214,10cyclictest23223-21apt-get10:55:120
2137599213,13cyclictest31875-21apt-get11:25:150
2137599212,14cyclictest27740-21apt-get07:30:190
2137599212,13cyclictest9972-21apt-esm-hook12:00:170
2137599212,13cyclictest8406-21apt-get08:15:120
2137599212,13cyclictest4179-21apt-get11:40:210
2137599212,13cyclictest29900-21snmp_o5powerctl09:25:270
2137599212,13cyclictest29269-21apt-get07:35:140
2137599212,13cyclictest2691-21apt-get09:45:150
2137599212,13cyclictest2361-21apt-get11:35:170
2137599212,13cyclictest18908-21apt-get10:40:150
2137599212,13cyclictest18790-21apt-get12:30:060
2137599212,13cyclictest18790-21apt-get12:30:060
2137599212,13cyclictest12812-21apt-get12:10:160
2137599212,13cyclictest1247-21apt-get09:40:160
2137599212,13cyclictest1201-21apt-get07:50:150
2137599212,12cyclictest23390-21apt-get07:15:190
2137699202,13cyclictest6982-21apt-esm-hook08:10:141
2137599205,10cyclictest21789-21apt-get10:50:160
2137599203,11cyclictest14217-21apt-get10:25:150
2137599203,10cyclictest26112-21apt-get11:05:140
2137599202,13cyclictest24678-21apt-get11:00:140
2137599202,13cyclictest18861-21apt-get08:50:180
2137599202,12cyclictest9891-21apt-get10:10:140
2137599202,12cyclictest28981-21apt-get11:15:150
2137599202,12cyclictest23197-21apt-get09:05:180
2137599202,12cyclictest17537-21apt-esm-hook12:25:180
2137599201,14cyclictest9865-21apt-get08:20:160
2137599201,14cyclictest2645-21apt-get07:55:160
2137599201,14cyclictest26282-21apt-get07:25:210
2137599201,13cyclictest8456-21apt-get10:05:150
2137599201,13cyclictest7034-21apt-esm-hook10:00:180
2137599201,13cyclictest15654-21apt-get10:30:190
2137599201,13cyclictest14009-21apt-get08:35:160
2137599192,13cyclictest13671-21snmp_o5powerctl08:30:310
2137599192,12cyclictest4134-21apt-esm-hook08:00:160
2137599192,12cyclictest27573-21apt-esm-hook11:10:150
2137599192,10cyclictest27525-21apt-get09:20:180
2137599191,14cyclictest24642-21apt-esm-hook09:10:150
2137599191,13cyclictest6971-21apt-get08:10:150
2137599191,13cyclictest32160-21apt-get07:45:160
2137599191,13cyclictest11378-21apt-get12:05:150
2137599191,12cyclictest20725-21apt-get12:35:160
2137599191,12cyclictest14246-21apt-get12:15:140
2137599187,8cyclictest4132-21apt-get09:50:140
2137599184,8cyclictest30727-21apt-get07:40:160
2137599182,10cyclictest5614-21apt-get11:45:160
2137599181,12cyclictest8484-21apt-get11:55:160
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional