You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2021-10-23 - 06:49

Intel(R) Xeon(R) CPU E3-1220L V2 @ 2.30GHz, Linux 4.9.47-rt37 (Profile)

Latency plot of system in rack #a, slot #0
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the alll highest latencies:
System rackaslot0.osadl.org (updated Sat Oct 23, 2021 00:44:34)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
308082940,0sleep130810-21id22:12:451
194822870,1sleep327329-21diskmemload21:49:463
242532840,0sleep024250-21sshd22:59:030
157162670,0sleep01785-21runrttasks21:36:500
76252630,1sleep37606-21sshd21:26:433
63822600,1sleep010-21rcuc/023:40:530
191525924,18sleep20-21swapper/219:09:292
226142580,1sleep022612-21seq22:15:070
211792580,1sleep00-21swapper/022:11:470
198542580,0sleep30-21swapper/300:35:363
36762560,0sleep20-21swapper/221:57:402
248722550,1sleep21785-21runrttasks21:37:452
31412540,1sleep01545-21nfsd00:24:360
46232530,1sleep30-21swapper/300:34:163
290372530,0sleep00-21swapper/021:31:530
159282530,1sleep21785-21runrttasks23:16:522
35332520,0sleep23510-21sshd23:31:182
208112520,0sleep30-21swapper/321:56:073
14192520,0sleep00-21swapper/023:24:590
103252520,2sleep3221899cyclictest22:10:423
315222510,0sleep20-21swapper/222:34:482
240312510,1sleep324020-21sshd00:13:573
21612510,1sleep30-21swapper/323:37:243
184325130,18sleep00-21swapper/019:08:340
322062500,0sleep032202-21sshd21:19:470
20902500,0sleep02091-21fschecks_time20:15:190
148272500,0sleep10-21swapper/100:22:321
97992490,0sleep00-21swapper/000:12:300
94432490,0sleep327329-21diskmemload23:22:333
56624923,7sleep10-21swapper/119:05:221
305132490,0sleep330512-21if_err_eth022:25:163
251152490,0sleep20-21swapper/222:15:172
163724927,18sleep30-21swapper/319:06:093
126842490,0sleep212676-21sshd21:52:152
86622480,0sleep10-21swapper/100:06:061
293692480,0sleep30-21swapper/323:40:043
154052470,0sleep215408-21bash00:25:422
41972460,0sleep30-21swapper/323:28:183
301042460,0sleep10-21swapper/100:33:341
94082450,0sleep19402-21sshd23:53:441
247532440,0sleep10-21swapper/121:47:041
209282440,0sleep20-21swapper/222:05:292
189622440,0sleep30-21swapper/323:48:253
189622440,0sleep30-21swapper/323:48:253
243672390,0sleep124366-21id22:18:261
158572390,0sleep10-21swapper/121:33:461
42892300,0sleep04287-21bash23:15:410
114862300,0sleep00-21swapper/023:04:040
124432240,1sleep227329-21diskmemload22:04:492
2215992321,1cyclictest15182-21sshd22:20:380
197632230,1sleep219747-21sshd00:13:322
221699229,1cyclictest0-21swapper/123:29:451
2215992220,1cyclictest14855-21sshd23:35:260
99492210,0sleep00-21swapper/023:31:560
2218992115,1cyclictest9203-21sshd23:53:433
2217992119,1cyclictest0-21swapper/223:42:152
221699219,7cyclictest0-21swapper/121:50:121
221699219,6cyclictest0-21swapper/121:26:011
2215992119,1cyclictest22535-21sshd21:43:470
293612190,1sleep229350-21sshd21:41:172
221699198,2cyclictest0-21swapper/121:12:521
221699198,1cyclictest4140-21sshd23:31:201
221699197,2cyclictest23356-21sshd22:58:561
221699191,17cyclictest0-21swapper/123:37:581
221699190,2cyclictest22739-21sshd22:21:241
183532190,0sleep218342-21sshd21:46:262
152612190,0sleep10-21swapper/123:41:461
2216991817,1cyclictest17462-21sshd22:52:071
221699180,9cyclictest28714-21bash22:40:371
221699180,2cyclictest29186-21sshd21:38:091
221699180,1cyclictest16939-21sshd00:10:081
2215991816,1cyclictest13153-21sshd00:25:280
126122180,0sleep30-21swapper/323:57:073
221699179,1cyclictest0-21swapper/123:18:001
221699179,1cyclictest0-21swapper/123:11:181
221699178,2cyclictest20189-21sshd23:23:361
221699178,1cyclictest17059-21bash21:55:441
221699178,1cyclictest16189-21sshd21:43:091
221699178,1cyclictest0-21swapper/122:32:311
221699171,9cyclictest0-21swapper/122:01:461
221699170,16cyclictest0-21swapper/122:49:311
221699170,11cyclictest0-21swapper/123:48:331
221699170,11cyclictest0-21swapper/123:48:331
221699170,11cyclictest0-21swapper/100:02:111
199762170,1sleep00-21swapper/023:11:010
118892170,0sleep20-21swapper/200:19:012
320872160,1sleep027329-21diskmemload22:53:340
2218991614,1cyclictest0-21swapper/323:01:593
221699169,1cyclictest0-21swapper/122:37:361
221699169,1cyclictest0-21swapper/121:20:271
221699169,1cyclictest0-21swapper/100:15:501
221699168,2cyclictest0-21swapper/122:05:471
221699168,2cyclictest0-21swapper/100:35:061
221699168,2cyclictest0-21swapper/100:27:361
221699166,2cyclictest5831-21sshd22:26:001
221699161,14cyclictest0-21swapper/121:18:261
221699160,1cyclictest0-21swapper/123:01:021
221699160,10cyclictest22514-21sshd23:05:071
221599169,3cyclictest0-21swapper/022:00:260
141332160,0sleep10-21swapper/120:40:351
80612150,0sleep339-21ksoftirqd/321:54:593
2218991514,1cyclictest0-21swapper/323:31:273
2218991514,1cyclictest0-21swapper/322:42:293
221699159,1cyclictest0-21swapper/123:55:221
2215991513,1cyclictest9216-21sshd22:07:320
221599150,4cyclictest8979-21sshd00:37:480
221599150,4cyclictest0-21swapper/023:48:160
221599150,4cyclictest0-21swapper/023:48:150
221599150,12cyclictest0-21swapper/021:23:160
221599150,12cyclictest0-21swapper/019:25:170
21972150,0sleep30-21swapper/322:35:063
79862140,0sleep30-21swapper/321:33:003
221899149,4cyclictest0-21swapper/323:12:503
221899149,4cyclictest0-21swapper/322:56:203
221899143,10cyclictest28242-21kworker/3:021:35:123
2218991412,1cyclictest11714-21kworker/3:022:00:203
221899140,2cyclictest28678-21sshd00:20:373
221899140,13cyclictest10568-21sshd22:20:163
221899140,13cyclictest0-21swapper/322:32:233
221899140,13cyclictest0-21swapper/321:41:033
221799143,10cyclictest1631-21sshd22:53:062
221799140,3cyclictest0-21swapper/221:27:442
221599149,4cyclictest29225-21sshd23:05:410
221599149,4cyclictest0-21swapper/022:32:020
221599148,3cyclictest13176-21bash22:36:030
221599142,1cyclictest0-21swapper/022:26:070
221599141,4cyclictest0-21swapper/021:05:000
2215991412,2cyclictest0-21swapper/000:00:190
221599140,3cyclictest23479-21bash21:53:180
221599140,14cyclictest0-21swapper/000:16:340
221599140,13cyclictest20336-21sshd23:51:360
221599140,13cyclictest0-21swapper/021:45:520
221599140,11cyclictest0-21swapper/021:14:120
221599140,10cyclictest0-21swapper/000:30:400
221899139,1cyclictest10678-21sshd00:06:183
221899139,1cyclictest0-21swapper/322:50:223
221899138,2cyclictest8589-21sshd23:16:083
221899137,2cyclictest28202-21sshd00:17:253
221899133,1cyclictest450-21kworker/3:223:09:543
2218991311,1cyclictest19176-21smartctl21:15:203
221899130,1cyclictest0-21swapper/322:19:483
221899130,12cyclictest27329-21diskmemload00:02:383
221899130,12cyclictest15210-21sshd21:24:253
221899130,12cyclictest14101-21sshd00:25:343
221899130,12cyclictest0-21swapper/322:45:343
221899130,12cyclictest0-21swapper/321:12:053
221899130,10cyclictest13618-21sshd22:07:583
221799139,3cyclictest0-21swapper/223:10:192
2217991313,0cyclictest27329-21diskmemload23:27:322
2217991311,1cyclictest29402-21sshd22:37:382
221799130,2cyclictest14359-21id00:35:132
221799130,12cyclictest4326-21bash00:05:392
221799130,12cyclictest30223-21sshd00:20:492
221799130,12cyclictest20498-21sshd22:46:092
221799130,12cyclictest0-21swapper/223:50:212
221799130,12cyclictest0-21swapper/223:45:152
221799130,12cyclictest0-21swapper/223:45:152
221799130,12cyclictest0-21swapper/222:57:192
221799130,12cyclictest0-21swapper/222:10:342
221799130,12cyclictest0-21swapper/221:21:222
221799130,12cyclictest0-21swapper/200:01:012
221799130,10cyclictest0-21swapper/223:22:532
221599138,3cyclictest0-21swapper/022:46:000
221599137,4cyclictest0-21swapper/022:41:050
221599132,9cyclictest0-21swapper/023:25:480
221599131,8cyclictest0-21swapper/019:20:240
221599130,9cyclictest0-21swapper/019:55:150
221599130,12cyclictest17052-21sshd00:06:560
221599130,12cyclictest13627-21bash21:55:280
221599130,12cyclictest0-21swapper/023:56:210
221599130,12cyclictest0-21swapper/021:27:190
221799129,2cyclictest0-21swapper/223:05:362
221799129,2cyclictest0-21swapper/222:25:252
221799129,2cyclictest0-21swapper/222:20:332
221799127,2cyclictest0-21swapper/223:35:072
221799121,10cyclictest0-21swapper/221:16:052
2217991210,1cyclictest9560-21sshd21:30:052
2217991210,1cyclictest12101-21sshd00:31:472
2217991210,1cyclictest0-21swapper/223:02:442
221799120,9cyclictest0-21swapper/223:59:192
221799120,1cyclictest0-21swapper/221:12:562
221799120,12cyclictest0-21swapper/222:40:032
221599129,2cyclictest0-21swapper/019:15:010
221599124,4cyclictest22564-21minicom21:00:000
221599123,5cyclictest0-21swapper/020:40:360
221599123,5cyclictest0-21swapper/019:40:140
221599120,9cyclictest0-21swapper/020:35:140
221699119,1cyclictest0-21swapper/119:15:021
221899109,1cyclictest0-21swapper/321:00:443
221899109,1cyclictest0-21swapper/320:17:093
221899109,1cyclictest0-21swapper/319:50:123
221899109,1cyclictest0-21swapper/319:39:323
221899108,1cyclictest4909-21ntp_states19:15:023
221899108,1cyclictest0-21swapper/320:25:183
221899100,1cyclictest0-21swapper/319:11:183
221899100,10cyclictest0-21swapper/319:30:423
221899100,10cyclictest0-21swapper/319:28:303
221799109,1cyclictest0-21swapper/219:45:142
221799108,1cyclictest0-21swapper/221:05:172
221799100,7cyclictest0-21swapper/219:53:112
221699109,1cyclictest0-21swapper/119:55:211
221599109,1cyclictest0-21swapper/019:30:160
221599104,3cyclictest0-21swapper/020:12:160
221599101,6cyclictest0-21swapper/020:55:000
221599100,9cyclictest6908-21perl20:25:180
221599100,9cyclictest0-21swapper/020:20:160
221599100,7cyclictest0-21swapper/021:05:200
221599100,7cyclictest0-21swapper/019:10:190
22189998,1cyclictest0-21swapper/320:53:043
22189990,9cyclictest0-21swapper/319:42:463
22189990,6cyclictest0-21swapper/319:55:193
22179998,1cyclictest0-21swapper/220:00:482
22179998,1cyclictest0-21swapper/219:25:162
22179998,1cyclictest0-21swapper/219:15:302
22179990,9cyclictest0-21swapper/220:43:192
22179990,9cyclictest0-21swapper/220:38:402
22179990,9cyclictest0-21swapper/220:11:162
22179990,9cyclictest0-21swapper/219:13:182
22179990,8cyclictest0-21swapper/219:36:322
22179990,1cyclictest0-21swapper/221:02:222
22169993,5cyclictest0-21swapper/119:20:451
22169990,9cyclictest0-21swapper/120:18:071
22159994,4cyclictest0-21swapper/020:05:150
22159990,9cyclictest0-21swapper/020:46:120
22159990,9cyclictest0-21swapper/020:31:320
22159990,9cyclictest0-21swapper/019:53:170
22189987,1cyclictest0-21swapper/320:40:093
22189987,1cyclictest0-21swapper/320:09:363
22189981,6cyclictest0-21swapper/321:07:273
22189981,6cyclictest0-21swapper/320:35:223
22189981,6cyclictest0-21swapper/319:45:163
22189981,6cyclictest0-21swapper/319:20:243
22189980,6cyclictest0-21swapper/320:30:223
22179987,1cyclictest0-21swapper/220:54:352
22179981,6cyclictest0-21swapper/219:55:182
22179981,6cyclictest0-21swapper/219:20:262
22179980,8cyclictest0-21swapper/219:42:482
22179980,8cyclictest0-21swapper/219:31:112
22179980,7cyclictest0-21swapper/220:25:212
22179980,1cyclictest0-21swapper/220:06:232
22169987,1cyclictest0-21swapper/120:49:221
22169987,1cyclictest0-21swapper/120:30:191
22169982,4cyclictest0-21swapper/120:20:221
22169982,4cyclictest0-21swapper/119:35:231
22169981,6cyclictest0-21swapper/121:05:241
22169981,6cyclictest0-21swapper/120:35:141
22169981,6cyclictest0-21swapper/120:05:211
22169981,6cyclictest0-21swapper/120:00:211
22169980,7cyclictest0-21swapper/120:52:321
22169980,7cyclictest0-21swapper/119:42:021
22159987,1cyclictest0-21swapper/020:04:250
22159980,5cyclictest19539-21munin-node19:45:230
22159980,5cyclictest0-21swapper/019:35:250
22189975,1cyclictest3781-21apache_accesses20:20:143
22189970,6cyclictest0-21swapper/320:55:203
22189970,6cyclictest0-21swapper/320:10:343
22189970,6cyclictest0-21swapper/320:00:143
22179975,1cyclictest55962chrt20:21:052
22179971,5cyclictest0-21swapper/220:55:242
22179971,5cyclictest0-21swapper/220:30:162
22179971,5cyclictest0-21swapper/220:15:202
22179970,5cyclictest0-21swapper/220:45:222
22169976,1cyclictest0-21swapper/119:51:381
22169976,1cyclictest0-21swapper/119:29:131
22169975,1cyclictest20873-21fschecks_count20:55:221
22169972,5cyclictest0-21swapper/119:14:311
22169971,6cyclictest0-21swapper/119:30:191
22169970,6cyclictest0-21swapper/121:00:181
22169970,6cyclictest0-21swapper/120:25:021
22169970,6cyclictest0-21swapper/120:10:251
22169970,6cyclictest0-21swapper/119:45:151
22189960,5cyclictest0-21swapper/320:45:203
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional