You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2021-05-15 - 07:32

Intel(R) Xeon(R) CPU E3-1220L V2 @ 2.30GHz, Linux 4.9.47-rt37 (Profile)

Latency plot of system in rack #a, slot #0
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the highest latencies:
System rackaslot0.osadl.org (updated Sat May 15, 2021 00:44:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
223892880,1sleep31714-21runrttasks00:25:053
77012630,1sleep17699-21bash00:03:521
168252630,0sleep30-21swapper/321:37:563
168762620,1sleep00-21swapper/022:27:360
116626024,8sleep20-21swapper/219:08:532
92825923,18sleep30-21swapper/319:05:563
124122580,0sleep30-21swapper/321:52:143
236132570,0sleep20-21swapper/221:24:142
290422550,1sleep20-21swapper/223:08:412
260502530,0sleep10-21swapper/122:19:171
180232520,0sleep30-21swapper/323:36:103
95924926,9sleep00-21swapper/019:06:170
87724824,9sleep10-21swapper/119:05:261
104942480,0sleep10-21swapper/122:46:041
104522480,0sleep20-21swapper/221:41:522
51512470,0sleep22438-21munin-node22:40:252
321582470,0sleep20-21swapper/223:48:062
124522470,0sleep10-21swapper/123:45:091
318422450,1sleep21714-21runrttasks21:45:212
21562450,0sleep10-21swapper/122:40:081
114812450,0sleep20-21swapper/221:32:152
63182430,0sleep00-21swapper/023:19:560
215462220,0sleep11714-21runrttasks00:24:591
1543992119,1cyclictest8160-21sshd21:26:501
1543992119,1cyclictest16435-21sshd22:32:241
1543992019,1cyclictest16531-21sshd22:37:171
154399201,1cyclictest23600-21sshd23:12:431
288792180,0sleep20-21swapper/223:33:042
1544991816,1cyclictest40342chrt23:24:272
320792170,1sleep032083-21fschecks_time21:50:190
204632170,1sleep30-21swapper/323:31:453
178922170,0sleep10-21swapper/123:40:561
1545991715,2cyclictest30962-21sshd23:47:563
1543991715,1cyclictest8605-21sshd21:31:461
1542991715,1cyclictest0-21swapper/000:20:350
1545991614,1cyclictest0-21swapper/322:27:363
154399161,11cyclictest0-21swapper/123:56:491
154399160,13cyclictest0-21swapper/123:27:451
154299169,6cyclictest0-21swapper/021:35:150
154299166,6cyclictest15938-21sshd21:57:470
1542991615,1cyclictest25120-21sshd00:15:510
1542991614,1cyclictest15284-21sshd21:42:380
154299160,15cyclictest0-21swapper/022:34:020
83182150,2sleep30-21swapper/300:32:343
154599156,9cyclictest0-21swapper/319:30:173
154599155,9cyclictest1192-21sshd22:59:353
1545991513,2cyclictest0-21swapper/323:57:323
1545991513,1cyclictest0-21swapper/322:21:293
154499150,14cyclictest0-21swapper/223:38:552
154399150,14cyclictest0-21swapper/121:22:581
154299159,4cyclictest0-21swapper/023:58:100
154299154,10cyclictest0-21swapper/000:37:480
1542991514,1cyclictest5112-21sshd22:01:080
1542991512,1cyclictest24806-21sshd23:12:540
1542991511,3cyclictest0-21swapper/022:56:290
154299150,1cyclictest0-21swapper/022:16:030
154299150,14cyclictest23369-21cpuspeed_turbos00:25:130
154299150,14cyclictest0-21swapper/022:51:310
154299150,14cyclictest0-21swapper/021:25:130
116682150,0sleep00-21swapper/023:20:320
236752140,0sleep10-21swapper/123:37:041
154599147,4cyclictest926-21sshd00:02:503
154599144,1cyclictest11976-21rm23:54:523
1545991412,1cyclictest19648-21bash21:58:213
154599140,14cyclictest0-21swapper/300:08:543
154599140,13cyclictest20234-21sshd22:03:313
154599140,13cyclictest0-21swapper/321:19:293
154599140,12cyclictest0-21swapper/300:38:053
154499149,4cyclictest0-21swapper/222:25:482
154499141,12cyclictest8438-21bash23:25:082
154499140,13cyclictest0-21swapper/221:58:482
154399149,4cyclictest0-21swapper/123:30:121
154399149,2cyclictest0-21swapper/100:15:581
154399148,5cyclictest0-21swapper/122:01:301
1543991412,1cyclictest10468-21sshd22:55:511
1543991411,2cyclictest0-21swapper/119:10:111
154399140,13cyclictest14662-21sshd21:42:321
154399140,13cyclictest0-21swapper/100:37:451
154399140,11cyclictest0-21swapper/121:18:051
154399140,10cyclictest0-21swapper/123:00:031
154299149,4cyclictest0-21swapper/022:05:450
154299148,5cyclictest32188-21sshd23:52:590
1542991412,1cyclictest28674-21sshd21:20:080
154299140,3cyclictest4218-21sshd23:00:010
154299140,2cyclictest11720-21bash21:32:170
154299140,13cyclictest0-21swapper/022:43:540
154299140,13cyclictest0-21swapper/022:24:120
154299140,13cyclictest0-21swapper/022:11:080
154299140,13cyclictest0-21swapper/000:00:130
154299140,11cyclictest0-21swapper/023:41:370
154299140,11cyclictest0-21swapper/023:32:210
154299140,11cyclictest0-21swapper/023:08:240
154299140,11cyclictest0-21swapper/022:47:020
154299140,11cyclictest0-21swapper/021:16:350
15132140,0sleep30-21swapper/321:45:293
203952130,1sleep31714-21runrttasks21:23:423
183992130,0sleep30-21swapper/300:15:033
154599139,1cyclictest0-21swapper/323:25:263
154599139,1cyclictest0-21swapper/323:21:323
1545991311,1cyclictest800-21sshd22:49:423
1545991310,2cyclictest0-21swapper/323:01:363
1545991310,2cyclictest0-21swapper/322:41:423
154599130,12cyclictest5864-21sshd21:26:273
154599130,12cyclictest5468-21sshd23:05:073
154599130,12cyclictest14718-21sshd21:13:023
154599130,12cyclictest12312-21sshd23:15:453
154599130,12cyclictest0-21swapper/322:30:463
154599130,12cyclictest0-21swapper/300:10:243
154599130,10cyclictest26641-21diskmemload22:50:323
154499139,3cyclictest19060-21sshd22:08:182
154499139,3cyclictest0-21swapper/222:00:152
154499138,4cyclictest10380-21bash21:17:172
154499138,4cyclictest0-21swapper/200:30:312
154499137,2cyclictest22403-21id22:33:222
154499137,2cyclictest0-21swapper/200:16:102
154499131,11cyclictest0-21swapper/222:36:292
154499130,4cyclictest23939-21sshd22:14:042
154499130,4cyclictest0-21swapper/223:56:242
154499130,1cyclictest0-21swapper/222:24:182
154499130,12cyclictest31936-21sshd23:52:572
154499130,12cyclictest16230-21sshd23:11:332
154499130,12cyclictest12264-21sshd00:14:072
154499130,12cyclictest0-21swapper/223:02:072
154499130,12cyclictest0-21swapper/222:46:202
154499130,12cyclictest0-21swapper/222:17:312
154499130,12cyclictest0-21swapper/221:53:192
154499130,12cyclictest0-21swapper/221:11:582
154499130,12cyclictest0-21swapper/200:25:222
154399139,1cyclictest488-21jbd2/dm-0-821:58:251
154399139,1cyclictest0-21swapper/123:51:491
154399139,1cyclictest0-21swapper/122:13:251
154399131,11cyclictest31214-21sshd00:26:151
1543991311,1cyclictest0-21swapper/121:36:201
154399131,11cyclictest0-21swapper/120:35:141
154399130,3cyclictest0-21swapper/122:06:361
154399130,12cyclictest26641-21diskmemload23:21:471
154399130,12cyclictest22801-21bash22:23:421
154399130,12cyclictest13181-21bash21:12:451
154399130,12cyclictest0-21swapper/123:07:511
154399130,12cyclictest0-21swapper/122:26:171
154399130,12cyclictest0-21swapper/100:11:061
154399130,10cyclictest0-21swapper/123:19:571
154299139,3cyclictest10315-21munin-node23:35:250
154299139,3cyclictest0-21swapper/000:30:160
154299137,3cyclictest0-21swapper/000:05:190
154299134,5cyclictest14754-21munin-run20:45:000
1542991311,1cyclictest26927-21sshd23:27:490
154299131,11cyclictest0-21swapper/023:45:050
154299130,4cyclictest0-21swapper/000:10:590
154299130,2cyclictest0-21swapper/022:39:010
154299130,12cyclictest3014-21sshd21:45:400
154299130,12cyclictest0-21swapper/021:10:340
154299130,12cyclictest0-21swapper/020:10:160
107322130,1sleep30-21swapper/323:10:383
154599129,2cyclictest0-21swapper/322:10:123
154599129,2cyclictest0-21swapper/321:30:343
154599128,2cyclictest0-21swapper/322:05:173
154599128,1cyclictest0-21swapper/323:41:493
154599121,10cyclictest31874-1kworker/3:2H22:40:013
154599121,10cyclictest0-21swapper/300:20:143
154599120,11cyclictest0-21swapper/322:16:353
154599120,10cyclictest0-21swapper/321:42:163
154499121,2cyclictest0-21swapper/221:35:142
154499121,2cyclictest0-21swapper/200:07:512
154499121,10cyclictest0-21swapper/200:00:142
1544991210,1cyclictest22950-21fschecks_time00:20:192
1544991210,1cyclictest1496-21nfsd00:35:122
154499120,12cyclictest0-21swapper/222:50:342
154499120,11cyclictest0-21swapper/223:44:102
154499120,11cyclictest0-21swapper/221:26:062
154499120,11cyclictest0-21swapper/220:43:532
154399129,2cyclictest0-21swapper/122:52:261
154399129,2cyclictest0-21swapper/100:30:061
154399129,2cyclictest0-21swapper/100:05:111
154399121,10cyclictest0-21swapper/121:53:441
154399120,9cyclictest0-21swapper/121:46:011
154599110,10cyclictest0-21swapper/319:15:013
154499119,1cyclictest1496-21nfsd22:55:102
154499119,1cyclictest0-21swapper/223:15:022
154499111,9cyclictest0-21swapper/219:50:142
1544991110,1cyclictest0-21swapper/220:24:362
154499110,2cyclictest0-21swapper/219:40:172
154499110,11cyclictest0-21swapper/220:46:262
154499110,11cyclictest0-21swapper/220:11:492
154499110,11cyclictest0-21swapper/219:47:192
154499110,11cyclictest0-21swapper/219:35:282
154499110,11cyclictest0-21swapper/219:30:142
154399110,10cyclictest4146-21munin-node19:15:011
154399110,10cyclictest0-21swapper/120:23:361
154299119,1cyclictest0-21swapper/019:50:130
154299117,3cyclictest0-21swapper/019:15:010
154299115,2cyclictest0-21swapper/019:55:200
154299113,5cyclictest0-21swapper/019:29:350
154299112,8cyclictest0-21swapper/021:00:350
154299110,10cyclictest0-21swapper/020:50:140
154599108,1cyclictest37-21rcuc/319:55:193
154599100,9cyclictest6238-21perl20:25:173
154599100,9cyclictest0-21swapper/321:05:153
154599100,9cyclictest0-21swapper/319:25:143
154499109,1cyclictest0-21swapper/221:06:392
154499109,1cyclictest0-21swapper/220:57:082
154499108,1cyclictest6216-21fschecks_time20:25:172
154499108,1cyclictest29166-21tune2fs20:05:162
154499100,1cyclictest4062-21diskstats19:15:022
154399109,1cyclictest0-21swapper/120:30:271
154399109,1cyclictest0-21swapper/119:27:571
154399108,1cyclictest18060-21perl20:50:131
154399108,1cyclictest12465-21fschecks_time19:30:151
154399108,1cyclictest0-21swapper/119:45:141
154399100,1cyclictest0-21swapper/120:11:031
154299107,2cyclictest0-21swapper/020:30:150
154299101,8cyclictest0-21swapper/020:55:120
1542991010,0cyclictest0-21swapper/019:35:220
154299100,9cyclictest0-21swapper/020:20:160
154299100,1cyclictest0-21swapper/019:10:160
154299100,10cyclictest0-21swapper/020:27:040
154299100,10cyclictest0-21swapper/019:43:370
154299100,10cyclictest0-21swapper/019:31:180
154299100,10cyclictest0-21swapper/019:21:320
15459998,1cyclictest0-21swapper/320:40:133
15459997,1cyclictest0-21swapper/320:50:133
15459993,3cyclictest0-21swapper/320:04:403
15459990,7cyclictest0-21swapper/320:12:163
15459990,1cyclictest0-21swapper/319:21:263
15449999,0cyclictest0-21swapper/220:51:332
15449998,1cyclictest0-21swapper/220:35:152
15449998,1cyclictest0-21swapper/219:26:312
15449998,1cyclictest0-21swapper/219:12:072
15449990,9cyclictest0-21swapper/220:30:152
15439998,1cyclictest0-21swapper/120:56:361
15439998,1cyclictest0-21swapper/120:44:441
15439998,1cyclictest0-21swapper/120:29:381
15439998,1cyclictest0-21swapper/119:40:171
15439996,2cyclictest0-21swapper/119:50:131
15439996,1cyclictest4993-21kworker/1:121:05:021
15439996,1cyclictest4993-21kworker/1:120:03:531
15439996,1cyclictest4993-21kworker/1:119:55:451
15439990,9cyclictest0-21swapper/119:35:151
15439990,1cyclictest0-21swapper/121:00:471
15429994,3cyclictest15397-21df_abs20:45:170
15429993,3cyclictest29169-21perl20:05:150
15429991,3cyclictest0-21swapper/020:00:200
15459987,1cyclictest3892-21fw_forwarded_lo20:20:163
15459987,1cyclictest0-21swapper/320:18:363
15459983,5cyclictest0-21swapper/321:00:043
15459983,5cyclictest0-21swapper/320:45:503
15459983,5cyclictest0-21swapper/320:33:343
15459983,5cyclictest0-21swapper/320:06:343
15459983,5cyclictest0-21swapper/319:50:543
15459982,3cyclictest0-21swapper/319:11:063
15459981,6cyclictest0-21swapper/320:35:123
15459981,6cyclictest0-21swapper/319:45:233
15459981,6cyclictest0-21swapper/319:40:163
15449988,0cyclictest0-21swapper/220:15:152
15449981,6cyclictest0-21swapper/221:00:212
15449981,6cyclictest0-21swapper/220:00:182
15449981,6cyclictest0-21swapper/219:20:172
15439986,1cyclictest7790-21fschecks_time19:20:171
15439981,6cyclictest0-21swapper/120:15:141
15429987,1cyclictest0-21swapper/021:05:160
15429986,1cyclictest19330-21diskstats19:45:130
15429985,2cyclictest0-21swapper/020:35:190
15429981,7cyclictest0-21swapper/020:15:190
15459975,1cyclictest0-21swapper/320:55:163
15459972,5cyclictest0-21swapper/319:37:453
15449972,2cyclictest0-21swapper/219:55:212
15439971,5cyclictest0-21swapper/120:45:151
15439970,6cyclictest0-21swapper/120:05:111
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional