You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2021-09-28 - 09:20

Intel(R) Xeon(R) CPU E31220 @ 3.10GHz, Linux 4.19.8-rt6 (Profile)

Latency plot of system in rack #1, slot #0
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 100 highest latencies:
System rack1slot0.osadl.org (updated Tue Sep 28, 2021 00:43:55)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1443221130,1sleep00-21swapper/019:05:180
708921110,19sleep395750irq/24-eno1-rx-23:54:183
1392421080,0sleep20-21swapper/219:05:122
212152620,0sleep20-21swapper/223:13:112
35842590,0sleep10-21swapper/100:13:431
188172590,1sleep018816-21sshd21:42:030
34052580,0sleep00-21swapper/021:52:130
279292580,0sleep027925-21sshd00:01:410
171162580,0sleep00-21swapper/023:52:140
205342570,0sleep20-21swapper/200:12:202
125922570,0sleep30-21swapper/322:18:393
274272560,0sleep10-21swapper/100:21:251
274272560,0sleep10-21swapper/100:21:251
234042560,0sleep30-21swapper/321:51:063
165402560,0sleep10-21swapper/123:04:171
165402560,0sleep10-21swapper/123:04:161
1524125644,7sleep10-21swapper/119:09:051
272582550,0sleep30-21swapper/323:55:593
167592550,0sleep00-21swapper/022:35:550
320142540,0sleep20-21swapper/222:23:152
275492540,0sleep30-21swapper/322:45:243
273462540,2sleep31542499cyclictest21:13:263
14082540,1sleep11407-21grep22:09:071
283542530,0sleep30-21swapper/300:30:083
133772530,0sleep20-21swapper/223:01:052
133772530,0sleep20-21swapper/223:01:052
1509624938,7sleep30-21swapper/319:07:283
35582210,0sleep20-21swapper/200:22:102
35582210,0sleep20-21swapper/200:22:102
159492210,0sleep10-21swapper/121:41:481
85452190,0sleep0111rcuc/000:14:110
66592190,2sleep11541199cyclictest23:54:161
4182190,0sleep10-21swapper/123:45:151
272332190,0sleep00-21swapper/023:41:570
256782190,0sleep20-21swapper/223:27:412
254482190,0sleep10-21swapper/121:56:571
30912180,0sleep00-21swapper/000:25:000
30912180,0sleep00-21swapper/000:25:000
269302180,0sleep20-21swapper/222:02:482
47042170,0sleep14702-21sshd23:25:481
258242170,0sleep20-21swapper/222:08:242
233232170,0sleep30-21swapper/322:42:113
14512170,1sleep31448-21lspci00:10:373
114422170,0sleep10-21swapper/123:20:471
92772160,0sleep29281-21bash00:31:152
84772160,0sleep10-21swapper/121:23:531
41492160,0sleep20-21swapper/223:23:012
211492160,1sleep3391rcuc/323:07:273
79302150,0sleep00-21swapper/023:37:330
35882150,0sleep20-21swapper/221:49:262
32202150,0sleep0111rcuc/021:49:230
261502150,0sleep10-21swapper/100:35:391
251452150,0sleep20-21swapper/222:59:212
123862150,0sleep212346-21sshd21:18:052
123862150,0sleep212346-21sshd21:18:042
268012140,0sleep2311rcuc/222:16:572
1542499142,11cyclictest13593-21sshd21:33:053
86662130,0sleep20-21swapper/221:23:542
274182130,0sleep10-21swapper/121:19:331
274182130,0sleep10-21swapper/121:19:321
256892120,0sleep00-21swapper/023:24:550
231532120,1sleep3909-21NetworkManager22:33:463
15424991210,1cyclictest801-21systemd-logind23:34:213
15424991210,1cyclictest12738-21sshd00:37:193
1542499120,1cyclictest1-21systemd23:10:273
1542499120,11cyclictest820-21dbus-daemon00:00:593
1541699121,1cyclictest131rcu_sched23:33:142
15416991210,1cyclictest20676-21grepconf.sh00:06:432
1541199129,2cyclictest25295-21tty21:54:091
1541199120,11cyclictest2754-21tty22:17:421
1540099120,11cyclictest16817-21cp23:18:300
1542499110,10cyclictest1-21systemd21:40:403
1542499110,10cyclictest0-21swapper/323:22:343
15416991110,0cyclictest669-40kipmi023:05:212
1541699110,10cyclictest0-21swapper/223:53:582
1541699110,10cyclictest0-21swapper/221:13:372
1541199119,1cyclictest0-21swapper/122:29:111
1541199110,10cyclictest0-21swapper/122:34:021
1541199110,10cyclictest0-21swapper/122:22:141
1541199110,0cyclictest0-21swapper/122:03:211
1540099119,1cyclictest13011-21sshd23:09:380
15400991110,0cyclictest0-21swapper/022:50:240
1540099110,10cyclictest13121-21sshd23:57:350
1540099110,10cyclictest0-21swapper/022:25:330
285242100,0sleep10-21swapper/121:45:491
1542499109,0cyclictest0-21swapper/322:56:243
1542499109,0cyclictest0-21swapper/321:37:563
1542499109,0cyclictest0-21swapper/321:26:093
1542499109,0cyclictest0-21swapper/321:26:083
1542499100,0cyclictest0-21swapper/323:01:543
1542499100,0cyclictest0-21swapper/323:01:543
1542499100,0cyclictest0-21swapper/321:55:323
1541699109,0cyclictest20035-21lspci00:37:562
1541699109,0cyclictest0-21swapper/222:47:172
1541699109,0cyclictest0-21swapper/222:37:162
1541699109,0cyclictest0-21swapper/222:32:262
1541699109,0cyclictest0-21swapper/221:42:122
1541699100,9cyclictest25603-21kworker/2:022:44:152
1541699100,6cyclictest29151-21kworker/2:021:28:202
1541699100,6cyclictest29151-21kworker/2:021:28:192
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional