You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2020-12-03 - 05:45

Intel(R) Xeon(R) CPU E3-1220L V2 @ 2.30GHz, Linux 4.4.39-rt50+ (Profile)

Latency plot of system in rack #7, slot #0
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of all highest latencies:
System rack7slot0.osadl.org (updated Wed Dec 02, 2020 12:44:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
117950590,0irq/26-eth1-rx-0-21swapper/307:06:313
110050580,0irq/25-eth00-21swapper/107:06:201
9950520,0irq/24-0000:00:0-21swapper/007:05:350
919524523,0sleep20-21swapper/207:06:062
9950270,0irq/24-0000:00:0-21swapper/007:10:000
9759991512,0cyclictest0-21swapper/209:18:072
9758991512,0cyclictest0-21swapper/111:42:001
9760991413,0cyclictest0-21swapper/310:39:303
9760991412,0cyclictest4098-21sshd12:27:463
976099140,0cyclictest32687-21bash09:58:573
9759991412,0cyclictest0-21swapper/210:11:402
975999140,0cyclictest0-21swapper/212:10:122
975899140,0cyclictest23618-21sshd12:03:221
975899140,0cyclictest0-21swapper/108:45:041
9757991411,0cyclictest0-21swapper/011:23:510
41140,0ktimersoftd/020149-21nfsd411:40:110
9950130,0irq/24-0000:00:0-21swapper/109:25:151
9760991312,0cyclictest1512-21sshd11:01:343
9760991312,0cyclictest0-21swapper/310:12:333
976099130,0cyclictest16880-21sshd12:20:203
976099130,0cyclictest0-21swapper/311:42:273
9759991312,0cyclictest0-21swapper/211:27:182
9759991312,0cyclictest0-21swapper/211:11:442
9759991312,0cyclictest0-21swapper/209:21:002
9759991311,0cyclictest0-21swapper/211:05:342
9759991311,0cyclictest0-21swapper/210:19:062
9759991311,0cyclictest0-21swapper/208:50:082
975999130,0cyclictest32153-21id09:36:282
975999130,0cyclictest0-21swapper/212:00:032
9758991312,0cyclictest0-21swapper/110:19:311
9758991311,0cyclictest0-21swapper/112:15:201
975899130,0cyclictest922-21sshd10:25:411
9757991312,0cyclictest0-21swapper/011:49:310
9757991312,0cyclictest0-21swapper/011:49:310
9757991312,0cyclictest0-21swapper/009:57:460
9757991312,0cyclictest0-21swapper/009:34:310
9757991311,0cyclictest0-21swapper/012:18:400
110050130,0irq/25-eth02185-21cp10:57:231
9760991212,0cyclictest0-21swapper/312:32:183
9760991212,0cyclictest0-21swapper/310:53:463
9760991212,0cyclictest0-21swapper/310:27:463
9760991211,0cyclictest0-21swapper/311:09:213
9760991211,0cyclictest0-21swapper/310:44:233
9760991211,0cyclictest0-21swapper/310:03:293
9760991210,0cyclictest0-21swapper/307:38:523
976099120,0cyclictest995-21diskmemload11:59:183
976099120,0cyclictest0-21swapper/312:10:153
976099120,0cyclictest0-21swapper/311:31:253
976099120,0cyclictest0-21swapper/309:00:323
976099120,0cyclictest0-21swapper/308:24:593
9759991211,0cyclictest0-21swapper/211:39:572
9759991211,0cyclictest0-21swapper/210:57:472
9759991211,0cyclictest0-21swapper/209:53:122
9759991210,0cyclictest0-21swapper/209:10:272
9759991210,0cyclictest0-21swapper/208:33:382
975999120,0cyclictest0-21swapper/212:25:362
975999120,0cyclictest0-21swapper/210:51:382
975999120,0cyclictest0-21swapper/209:56:272
9758991211,0cyclictest8211-21sshd09:10:521
9758991211,0cyclictest13090-21sshd12:10:481
9758991211,0cyclictest10873-21bash09:20:101
9758991211,0cyclictest0-21swapper/111:48:091
9758991211,0cyclictest0-21swapper/111:48:091
9758991211,0cyclictest0-21swapper/111:30:581
9758991211,0cyclictest0-21swapper/110:50:251
975899120,0cyclictest0-21swapper/111:54:191
975899120,0cyclictest0-21swapper/111:16:401
9757991211,0cyclictest0-21swapper/012:29:220
9757991211,0cyclictest0-21swapper/010:40:440
9757991211,0cyclictest0-21swapper/010:05:330
9757991210,0cyclictest13284-21id11:57:180
9757991210,0cyclictest0-21swapper/009:38:430
975799120,0cyclictest0-21swapper/011:25:280
975799120,0cyclictest0-21swapper/011:18:580
975799120,0cyclictest0-21swapper/009:42:540
975799120,0cyclictest0-21swapper/008:40:100
975799120,0cyclictest0-21swapper/008:25:140
41120,0ktimersoftd/021854-21sshd09:48:350
351120,0ktimersoftd/30-21swapper/311:45:293
351120,0ktimersoftd/30-21swapper/311:45:293
110050120,0irq/25-eth030928-21bash12:31:250
110050120,0irq/25-eth029912-21sshd12:22:150
110050120,0irq/25-eth00-21swapper/311:37:073
976099119,0cyclictest0-21swapper/307:25:513
9760991111,0cyclictest0-21swapper/310:22:533
9760991111,0cyclictest0-21swapper/309:23:323
9760991111,0cyclictest0-21swapper/309:11:203
9760991110,0cyclictest0-21swapper/312:36:493
9760991110,0cyclictest0-21swapper/311:28:103
9760991110,0cyclictest0-21swapper/310:08:013
9760991110,0cyclictest0-21swapper/309:43:253
9760991110,0cyclictest0-21swapper/308:17:053
976099110,0cyclictest0-21swapper/309:16:253
9759991111,0cyclictest110050irq/25-eth010:02:362
9759991111,0cyclictest0-21swapper/212:30:082
9759991111,0cyclictest0-21swapper/211:16:152
9759991111,0cyclictest0-21swapper/209:32:182
9759991111,0cyclictest0-21swapper/209:28:072
9759991110,0cyclictest12084-21munin-run10:50:002
9759991110,0cyclictest0-21swapper/208:56:102
9758991110,0cyclictest0-21swapper/112:27:181
9758991110,0cyclictest0-21swapper/112:06:161
9758991110,0cyclictest0-21swapper/110:35:471
975899110,0cyclictest9470-21sshd09:42:211
975899110,0cyclictest0-21swapper/112:20:111
975899110,0cyclictest0-21swapper/110:04:391
975799115,0cyclictest21858-21sshd10:01:510
9757991111,0cyclictest0-21swapper/010:56:540
9757991111,0cyclictest0-21swapper/010:37:230
9757991111,0cyclictest0-21swapper/009:20:220
9757991111,0cyclictest0-21swapper/009:17:280
9757991110,0cyclictest0-21swapper/009:54:300
975799110,0cyclictest7258-21bash12:37:080
975799110,0cyclictest0-21swapper/011:51:080
975799110,0cyclictest0-21swapper/008:06:220
975799110,0cyclictest0-21swapper/007:35:080
41110,0ktimersoftd/0995-21diskmemload10:50:260
41110,0ktimersoftd/012228-21sshd10:45:270
41110,0ktimersoftd/00-21swapper/012:00:530
41110,0ktimersoftd/00-21swapper/011:07:530
41110,0ktimersoftd/00-21swapper/010:29:300
351110,0ktimersoftd/30-21swapper/309:48:353
271110,0ktimersoftd/20-21swapper/209:48:352
271110,0ktimersoftd/20-21swapper/207:12:052
110050110,0irq/25-eth05038-21netstat12:05:100
110050110,0irq/25-eth00-21swapper/109:54:121
110050110,0irq/25-eth00-21swapper/010:34:040
976099109,0cyclictest0-21swapper/310:15:273
976099108,0cyclictest0-21swapper/308:53:123
976099108,0cyclictest0-21swapper/308:14:523
9760991010,0cyclictest5992-21sshd09:37:213
9760991010,0cyclictest0-21swapper/309:31:523
9760991010,0cyclictest0-21swapper/308:42:033
9760991010,0cyclictest0-21swapper/307:54:393
976099100,0cyclictest0-21swapper/312:08:223
975999109,0cyclictest7389-21bash10:26:382
975999109,0cyclictest0-21swapper/211:31:462
975999109,0cyclictest0-21swapper/211:03:052
975999109,0cyclictest0-21swapper/210:07:082
9759991010,0cyclictest995-21diskmemload11:47:442
9759991010,0cyclictest995-21diskmemload11:47:442
9759991010,0cyclictest0-21swapper/210:33:452
9759991010,0cyclictest0-21swapper/210:22:012
9759991010,0cyclictest0-21swapper/209:43:122
9759991010,0cyclictest0-21swapper/209:05:412
9759991010,0cyclictest0-21swapper/207:57:222
9759991010,0cyclictest0-21swapper/207:24:312
975999100,0cyclictest0-21swapper/212:21:052
975999100,0cyclictest0-21swapper/211:43:562
975899109,0cyclictest18493-21bash11:35:211
975899109,0cyclictest0-21swapper/110:09:101
975899109,0cyclictest0-21swapper/109:38:101
9758991010,0cyclictest0-21swapper/112:36:221
9758991010,0cyclictest0-21swapper/112:31:501
9758991010,0cyclictest0-21swapper/111:10:311
9758991010,0cyclictest0-21swapper/109:17:151
975899100,0cyclictest0-21swapper/109:56:331
975799109,0cyclictest0-21swapper/008:38:540
975799109,0cyclictest0-21swapper/007:51:500
9757991010,0cyclictest0-21swapper/011:14:040
9757991010,0cyclictest0-21swapper/010:10:040
9757991010,0cyclictest0-21swapper/009:02:280
9757991010,0cyclictest0-21swapper/008:55:090
975799100,0cyclictest0-21swapper/012:10:100
975799100,0cyclictest0-21swapper/009:13:160
41100,0ktimersoftd/05778-21sshd11:33:450
41100,0ktimersoftd/025110-21bash09:26:390
41100,0ktimersoftd/015028-21munin-node11:35:140
41100,0ktimersoftd/00-21swapper/010:16:520
97609999,0cyclictest0-21swapper/309:06:343
97609999,0cyclictest0-21swapper/307:56:523
97609998,0cyclictest0-21swapper/308:33:483
97609998,0cyclictest0-21swapper/308:01:173
97609990,0cyclictest0-21swapper/310:33:173
97609990,0cyclictest0-21swapper/309:50:083
97609990,0cyclictest0-21swapper/308:25:073
97599999,0cyclictest0-21swapper/212:19:272
97599999,0cyclictest0-21swapper/211:58:252
97599999,0cyclictest0-21swapper/209:00:552
97599999,0cyclictest0-21swapper/208:41:312
97599999,0cyclictest0-21swapper/207:29:382
97599998,0cyclictest29269-21bash12:08:402
97599998,0cyclictest0-21swapper/212:39:122
97599998,0cyclictest0-21swapper/207:37:302
97599990,0cyclictest0-21swapper/208:25:122
97599990,0cyclictest0-21swapper/207:15:262
97589999,0cyclictest0-21swapper/111:57:131
97589999,0cyclictest0-21swapper/111:29:201
97589999,0cyclictest0-21swapper/111:03:291
97589999,0cyclictest0-21swapper/110:11:331
97589999,0cyclictest0-21swapper/108:35:051
97589998,0cyclictest9238-21bash10:22:251
97589998,0cyclictest0-21swapper/110:45:321
97589998,0cyclictest0-21swapper/109:32:421
97589993,0cyclictest0-21swapper/108:44:131
97589990,0cyclictest0-21swapper/107:40:011
97579999,0cyclictest0-21swapper/008:22:100
97579999,0cyclictest0-21swapper/007:29:040
97579999,0cyclictest0-21swapper/007:19:050
97579998,0cyclictest0-21swapper/008:19:580
97579993,0cyclictest0-21swapper/007:55:070
4190,0ktimersoftd/00-21swapper/011:02:560
4190,0ktimersoftd/00-21swapper/010:23:590
35190,0ktimersoftd/30-21swapper/308:56:483
35190,0ktimersoftd/30-21swapper/307:34:123
11005090,0irq/25-eth036-21ksoftirqd/309:28:133
11005090,0irq/25-eth025696-21sshd11:18:363
11005090,0irq/25-eth00-21swapper/310:49:293
97609988,0cyclictest0-21swapper/312:02:123
97609988,0cyclictest0-21swapper/311:53:083
97609988,0cyclictest0-21swapper/311:13:533
97609988,0cyclictest0-21swapper/308:39:303
97609988,0cyclictest0-21swapper/307:21:203
97609987,0cyclictest24727-21bash10:56:003
97609987,0cyclictest0-21swapper/307:41:043
97599988,0cyclictest0-21swapper/211:53:542
97599988,0cyclictest0-21swapper/210:35:212
97599980,0cyclictest0-21swapper/210:42:512
97589987,0cyclictest20-21ksoftirqd/110:33:101
97589985,0cyclictest0-21swapper/110:43:471
97579988,0cyclictest0-21swapper/008:12:030
97579988,0cyclictest0-21swapper/007:23:370
179980,0migration/13377-21bash11:06:281
11005080,0irq/25-eth028896-21sshd12:17:403
11005080,0irq/25-eth022842-21sshd11:22:401
11005080,0irq/25-eth013738-21bash11:21:222
11005080,0irq/25-eth01310-21sshd09:45:361
11005080,0irq/25-eth00-21swapper/007:42:140
97599977,0cyclictest0-21swapper/208:48:512
97599977,0cyclictest0-21swapper/207:41:552
97599976,0cyclictest0-21swapper/208:07:092
97589977,0cyclictest0-21swapper/107:50:361
97589975,0cyclictest0-21swapper/109:09:501
97589974,0cyclictest0-21swapper/107:31:151
97589972,0cyclictest0-21swapper/108:05:131
97589971,0cyclictest0-21swapper/107:48:101
97589970,0cyclictest0-21swapper/108:25:151
97589970,0cyclictest0-21swapper/107:35:121
97579974,0cyclictest0-21swapper/008:00:060
97579973,0cyclictest0-21swapper/008:30:160
97579970,0cyclictest0-21swapper/009:05:070
97579970,0cyclictest0-21swapper/008:50:100
11005070,0irq/25-eth00-21swapper/311:21:403
11005070,0irq/25-eth00-21swapper/007:31:240
97609966,0cyclictest0-21swapper/307:48:583
97609962,0cyclictest0-21swapper/307:15:073
97599966,0cyclictest0-21swapper/207:31:502
97599965,0cyclictest29654-21fschecks_time07:50:052
97599962,0cyclictest0-21swapper/208:20:082
97599962,0cyclictest0-21swapper/208:15:132
97599962,0cyclictest0-21swapper/208:00:092
97589965,0cyclictest3029-21taskset08:01:411
97589965,0cyclictest1620-21runrttasks08:55:541
97589965,0cyclictest11061-21grep08:20:031
97589963,0cyclictest0-21swapper/107:56:011
97589963,0cyclictest0-21swapper/107:12:011
97589962,0cyclictest0-21swapper/109:00:171
97589962,0cyclictest0-21swapper/108:30:091
97589962,0cyclictest0-21swapper/108:15:141
97589961,0cyclictest0-21swapper/107:15:021
97589960,0cyclictest0-21swapper/107:22:241
97579962,0cyclictest0-21swapper/007:45:070
4160,0ktimersoftd/00-21swapper/008:45:050
97609954,0cyclictest788-40kipmi008:45:103
97609954,0cyclictest10199-21diskstats07:10:043
97609950,0cyclictest0-21swapper/308:05:123
97599951,0cyclictest0-21swapper/208:35:072
97599951,0cyclictest0-21swapper/207:45:072
97599950,0cyclictest0-21swapper/208:10:072
97589955,0cyclictest24757-21df_abs08:50:061
97589951,0cyclictest0-21swapper/107:25:021
97589950,0cyclictest0-21swapper/108:10:041
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional