You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2021-05-15 - 06:35

Intel(R) Xeon(R) CPU E3-1220L V2 @ 2.30GHz, Linux 4.4.39-rt50+ (Profile)

Latency plot of system in rack #7, slot #0
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the highest latencies:
System rack7slot0.osadl.org (updated Sat May 15, 2021 00:44:53)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
110050600,0irq/25-eth00-21swapper/319:05:463
117950570,0irq/26-eth1-rx-0-21swapper/119:07:031
1779224720,0sleep20-21swapper/219:09:312
1779024421,0sleep00-21swapper/019:09:280
351180,0ktimersoftd/32030-21sshd23:56:003
1809599180,0cyclictest28649-21expr20:40:113
18094991817,0cyclictest26-21rcuc/223:56:002
18092991716,0cyclictest0-21swapper/022:14:030
18092991716,0cyclictest0-21swapper/022:14:020
1809599160,0cyclictest0-21swapper/321:44:393
18094991615,0cyclictest271ktimersoftd/200:34:012
18094991514,0cyclictest10447-21sshd22:56:132
18094991514,0cyclictest0-21swapper/221:25:432
1809499150,0cyclictest20593-21sshd23:34:132
1809399150,0cyclictest28726-21sshd00:23:111
1809399150,0cyclictest0-21swapper/120:30:171
191140,0ktimersoftd/10-21swapper/121:20:341
18095991413,0cyclictest4111-21sshd23:28:023
1809599140,0cyclictest0-21swapper/322:06:153
18094991413,0cyclictest8949-21id00:20:412
18094991413,0cyclictest0-21swapper/200:16:432
18093991413,0cyclictest0-21swapper/122:30:091
18093991412,0cyclictest2841-21sshd22:43:091
18092991413,0cyclictest0-21swapper/021:48:150
18095991312,0cyclictest6482-21sshd00:04:453
18095991312,0cyclictest0-21swapper/322:35:033
1809599130,0cyclictest0-21swapper/322:12:323
1809599130,0cyclictest0-21swapper/322:12:313
1809599130,0cyclictest0-21swapper/300:08:413
18094991312,0cyclictest8067-21sshd22:47:522
18094991312,0cyclictest0-21swapper/223:42:412
18094991312,0cyclictest0-21swapper/223:19:072
18094991311,0cyclictest0-21swapper/221:36:522
1809499130,0cyclictest0-21swapper/223:23:382
1809499130,0cyclictest0-21swapper/222:20:252
1809499130,0cyclictest0-21swapper/221:24:582
18093991313,0cyclictest0-21swapper/100:39:331
18093991312,0cyclictest0-21swapper/123:55:321
18093991312,0cyclictest0-21swapper/122:02:161
18093991312,0cyclictest0-21swapper/100:15:171
18093991310,0cyclictest18576-21sshd22:12:441
18093991310,0cyclictest18576-21sshd22:12:441
1809399130,0cyclictest0-21swapper/121:37:191
18092991312,0cyclictest0-21swapper/022:35:240
18092991311,0cyclictest0-21swapper/022:42:570
1809299130,0cyclictest0-21swapper/023:24:320
1809299130,0cyclictest0-21swapper/021:10:580
1809299130,0cyclictest0-21swapper/000:09:300
110050130,0irq/25-eth00-21swapper/122:20:171
18095991212,0cyclictest0-21swapper/322:04:093
18095991211,0cyclictest14785-21sshd22:44:423
18095991211,0cyclictest0-21swapper/321:32:343
18095991210,0cyclictest27523-21sshd21:53:073
18095991210,0cyclictest20992-21sshd22:57:363
18095991210,0cyclictest0-21swapper/320:49:493
1809599120,0cyclictest17311-21sshd22:24:513
1809599120,0cyclictest0-21swapper/323:01:113
1809599120,0cyclictest0-21swapper/322:46:123
1809599120,0cyclictest0-21swapper/321:46:093
1809599120,0cyclictest0-21swapper/300:24:293
18094991212,0cyclictest11081-21sshd22:11:442
18094991212,0cyclictest11081-21sshd22:11:432
18094991212,0cyclictest0-21swapper/222:37:172
18094991212,0cyclictest0-21swapper/200:39:302
18094991211,0cyclictest1468-21bash21:41:242
18094991211,0cyclictest0-21swapper/222:05:272
18094991210,0cyclictest0-21swapper/221:58:362
1809499120,0cyclictest0-21swapper/223:37:132
1809499120,0cyclictest0-21swapper/220:30:122
18093991212,0cyclictest9730-21diskmemload22:59:191
18093991211,0cyclictest10609-21sshd22:19:571
18093991211,0cyclictest0-21swapper/123:47:381
18093991211,0cyclictest0-21swapper/123:08:021
18093991211,0cyclictest0-21swapper/121:57:311
18093991211,0cyclictest0-21swapper/121:34:171
18093991211,0cyclictest0-21swapper/100:28:391
18093991210,0cyclictest7259-21bash21:13:431
18093991210,0cyclictest1452-21nfsd23:26:091
18093991210,0cyclictest0-21swapper/123:12:331
1809399120,0cyclictest0-21swapper/123:30:401
1809399120,0cyclictest0-21swapper/123:20:071
1809399120,0cyclictest0-21swapper/123:17:051
1809399120,0cyclictest0-21swapper/121:54:501
18092991211,0cyclictest2374-21bash21:17:070
18092991211,0cyclictest1128-21sshd23:47:530
18092991211,0cyclictest0-21swapper/022:56:480
18092991211,0cyclictest0-21swapper/022:02:040
18092991210,0cyclictest0-21swapper/022:45:590
18092991210,0cyclictest0-21swapper/000:15:540
1809299120,0cyclictest16216-21sshd23:37:330
1809299120,0cyclictest15975-21sshd23:25:250
1809299120,0cyclictest0-21swapper/023:50:200
1809299120,0cyclictest0-21swapper/019:25:150
1809299120,0cyclictest0-21swapper/000:04:070
110050120,0irq/25-eth00-21swapper/223:08:592
41110,0ktimersoftd/00-21swapper/023:30:260
351110,0ktimersoftd/30-21swapper/319:10:163
271110,0ktimersoftd/20-21swapper/221:45:562
18095991111,0cyclictest0-21swapper/322:26:553
18095991110,0cyclictest30997-21sshd21:12:353
18095991110,0cyclictest0-21swapper/323:31:033
18095991110,0cyclictest0-21swapper/300:10:113
1809599110,0cyclictest0-21swapper/321:25:563
1809599110,0cyclictest0-21swapper/321:22:203
1809599110,0cyclictest0-21swapper/320:10:123
1809599110,0cyclictest0-21swapper/319:38:143
18094991111,0cyclictest0-21swapper/222:29:092
18094991111,0cyclictest0-21swapper/222:15:192
18094991111,0cyclictest0-21swapper/200:12:462
18094991110,0cyclictest0-21swapper/222:03:212
18094991110,0cyclictest0-21swapper/221:11:262
1809499110,0cyclictest0-21swapper/223:28:092
1809499110,0cyclictest0-21swapper/220:23:482
18093991111,0cyclictest0-21swapper/121:46:221
18093991111,0cyclictest0-21swapper/121:26:091
18093991110,0cyclictest0-21swapper/122:36:461
1809399110,0cyclictest19052-21bash22:25:031
1809399110,0cyclictest0-21swapper/121:40:191
1809399110,0cyclictest0-21swapper/121:04:121
1809399110,0cyclictest0-21swapper/119:47:461
1809399110,0cyclictest0-21swapper/119:34:021
1809399110,0cyclictest0-21swapper/119:19:241
1809299118,0cyclictest31746-21sshd21:53:420
18092991111,0cyclictest15655-21sshd21:59:590
18092991111,0cyclictest0-21swapper/000:38:040
18092991110,0cyclictest0-21swapper/022:17:540
1809299110,0cyclictest7728-21sshd00:32:380
1809299110,0cyclictest0-21swapper/021:41:170
41100,0ktimersoftd/09730-21diskmemload23:59:290
41100,0ktimersoftd/024967-21ls19:20:240
41100,0ktimersoftd/013347-21sshd23:13:000
1809599109,0cyclictest0-21swapper/321:57:193
1809599109,0cyclictest0-21swapper/321:16:283
1809599109,0cyclictest0-21swapper/319:27:343
18095991010,0cyclictest0-21swapper/323:35:353
18095991010,0cyclictest0-21swapper/321:05:303
18095991010,0cyclictest0-21swapper/320:23:413
18095991010,0cyclictest0-21swapper/300:29:563
1809499109,0cyclictest22831-21sshd23:01:552
1809499109,0cyclictest0-21swapper/221:51:242
1809499109,0cyclictest0-21swapper/200:08:502
18094991010,0cyclictest271ktimersoftd/221:17:352
18094991010,0cyclictest0-21swapper/223:11:332
18094991010,0cyclictest0-21swapper/222:50:312
18094991010,0cyclictest0-21swapper/221:33:522
18094991010,0cyclictest0-21swapper/221:00:172
18094991010,0cyclictest0-21swapper/219:10:232
1809499100,0cyclictest0-21swapper/219:31:392
1809399109,0cyclictest9730-21diskmemload00:03:261
1809399109,0cyclictest28856-21sshd00:11:211
18093991010,0cyclictest0-21swapper/122:47:561
18093991010,0cyclictest0-21swapper/121:08:291
18093991010,0cyclictest0-21swapper/120:46:251
18093991010,0cyclictest0-21swapper/120:22:291
18093991010,0cyclictest0-21swapper/119:55:051
18093991010,0cyclictest0-21swapper/119:35:201
18093991010,0cyclictest0-21swapper/100:34:061
1809399100,0cyclictest20991-21ls23:50:221
1809399100,0cyclictest0-21swapper/120:02:241
1809299109,0cyclictest27672-21sshd23:43:020
18092991010,0cyclictest0-21swapper/021:24:400
1809299100,0cyclictest16646-21bash21:39:180
1809299100,0cyclictest13184-21bash22:20:130
1599100,0watchdog/00-21swapper/021:32:030
117950100,0irq/26-eth1-rx-28849-21sshd22:50:230
110050100,0irq/25-eth00-21swapper/100:05:381
110050100,0irq/25-eth00-21swapper/000:10:220
4190,0ktimersoftd/09730-21diskmemload23:15:140
4190,0ktimersoftd/014646-21sshd23:05:010
4190,0ktimersoftd/010523-21sshd23:04:300
4190,0ktimersoftd/00-21swapper/022:30:120
4190,0ktimersoftd/00-21swapper/022:05:120
4190,0ktimersoftd/00-21swapper/021:27:070
4190,0ktimersoftd/00-21swapper/000:20:300
180959999,0cyclictest0-21swapper/322:53:593
180959999,0cyclictest0-21swapper/319:40:493
180959998,0cyclictest0-21swapper/323:46:303
180959998,0cyclictest0-21swapper/300:39:393
180959998,0cyclictest0-21swapper/300:18:063
180959992,0cyclictest0-21swapper/319:50:153
180959990,0cyclictest0-21swapper/319:22:203
180949999,0cyclictest0-21swapper/223:50:352
180949999,0cyclictest0-21swapper/220:56:342
180949998,0cyclictest0-21swapper/223:46:392
180949998,0cyclictest0-21swapper/220:06:442
180949998,0cyclictest0-21swapper/219:47:272
180949995,0cyclictest28-21ksoftirqd/200:03:512
180949990,0cyclictest0-21swapper/222:42:502
180939999,0cyclictest0-21swapper/123:43:411
180939999,0cyclictest0-21swapper/120:43:591
180939999,0cyclictest0-21swapper/120:07:091
180939999,0cyclictest0-21swapper/119:21:031
180939998,0cyclictest8456-21sshd22:07:171
180939998,0cyclictest0-21swapper/120:55:361
180939990,0cyclictest0-21swapper/123:37:291
180929999,0cyclictest0-21swapper/022:25:460
180929999,0cyclictest0-21swapper/000:26:130
149990,0migration/018070-21latencyplot19:10:010
11005090,0irq/25-eth00-21swapper/323:42:253
180959988,0cyclictest0-21swapper/320:35:593
180959988,0cyclictest0-21swapper/320:17:243
180959988,0cyclictest0-21swapper/319:32:483
180959987,0cyclictest0-21swapper/323:21:593
180959987,0cyclictest0-21swapper/323:17:273
180959980,0cyclictest0-21swapper/320:30:143
180949988,0cyclictest28-21ksoftirqd/219:15:302
180949988,0cyclictest0-21swapper/220:45:462
180949984,0cyclictest4879-21bash22:31:092
180939988,0cyclictest0-21swapper/120:52:141
180939988,0cyclictest0-21swapper/120:28:041
180939988,0cyclictest0-21swapper/120:18:261
180939988,0cyclictest0-21swapper/120:13:261
180939980,0cyclictest0-21swapper/123:04:371
180929984,0cyclictest0-21swapper/019:35:150
180929980,0cyclictest0-21swapper/020:55:140
180929980,0cyclictest0-21swapper/019:55:220
11005080,0irq/25-eth028106-21sshd23:06:363
11005080,0irq/25-eth020477-21sshd21:15:181
11005080,0irq/25-eth020468-21hddtemp_smartct23:50:183
11005080,0irq/25-eth018872-21sshd00:30:023
11005080,0irq/25-eth01512-21sshd22:32:023
11005080,0irq/25-eth013120-21sshd23:12:583
11005080,0irq/25-eth00-21swapper/322:17:413
11005080,0irq/25-eth00-21swapper/321:35:213
11005080,0irq/25-eth00-21swapper/020:09:290
180959977,0cyclictest0-21swapper/320:53:113
180959970,0cyclictest0-21swapper/320:05:143
180949977,0cyclictest0-21swapper/221:05:512
180949976,0cyclictest28-21ksoftirqd/200:25:082
180949972,0cyclictest0-21swapper/219:20:192
180939977,0cyclictest24872-21sshd22:50:011
180939977,0cyclictest0-21swapper/119:53:481
180939976,0cyclictest20312-21wc19:15:011
180939974,0cyclictest0-21swapper/119:25:231
180929974,0cyclictest32519-21chrt20:46:550
149970,0migration/023456-21dump-pmu-power20:30:000
149970,0migration/022498-21munin-node19:15:250
149970,0migration/015551-21nfsd420:10:220
11005070,0irq/25-eth00-21swapper/020:18:060
259960,0migration/21989-21munin-node20:50:192
180959966,0cyclictest0-21swapper/320:28:193
180959962,0cyclictest0-21swapper/321:00:193
180959962,0cyclictest0-21swapper/320:55:183
180959962,0cyclictest0-21swapper/320:00:163
180959962,0cyclictest0-21swapper/319:55:203
180959962,0cyclictest0-21swapper/319:45:153
180959962,0cyclictest0-21swapper/319:15:013
180949965,0cyclictest5091-21basename19:50:102
180949965,0cyclictest28-21ksoftirqd/220:40:142
180949965,0cyclictest28-21ksoftirqd/220:35:132
180949965,0cyclictest28-21ksoftirqd/220:15:232
180949965,0cyclictest28-21ksoftirqd/219:40:132
180949962,0cyclictest0-21swapper/220:10:202
180949962,0cyclictest0-21swapper/220:00:162
180949962,0cyclictest0-21swapper/219:55:202
180949962,0cyclictest0-21swapper/219:35:162
180949962,0cyclictest0-21swapper/219:25:162
180939963,0cyclictest26444-21diskstats20:35:131
180929963,0cyclictest10420-21fw_forwarded_lo20:00:140
180929962,0cyclictest0-21swapper/021:05:160
180929962,0cyclictest0-21swapper/020:40:150
180929962,0cyclictest0-21swapper/020:35:160
180929962,0cyclictest0-21swapper/020:30:160
180929962,0cyclictest0-21swapper/020:20:170
180929962,0cyclictest0-21swapper/019:40:180
180929962,0cyclictest0-21swapper/019:30:160
149960,0migration/05268-21cpuspeed_turbos19:50:110
149960,0migration/03457-21idleruntime19:45:140
4150,0ktimersoftd/00-21swapper/020:50:170
180949954,0cyclictest28-21ksoftirqd/220:25:002
180939950,0cyclictest0-21swapper/119:40:001
180929954,0cyclictest5265-21munin-node21:00:160
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional