You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2021-10-22 - 01:38

Intel(R) Xeon(R) CPU E3-1220L V2 @ 2.30GHz, Linux 4.4.39-rt50+ (Profile)

Latency plot of system in rack #7, slot #0
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the highest latencies:
System rack7slot0.osadl.org (updated Thu Oct 21, 2021 12:44:55)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
110050660,0irq/25-eth00-21swapper/307:05:263
110050610,0irq/25-eth00-21swapper/107:05:191
117950480,0irq/26-eth1-rx-0-21swapper/207:05:292
110050380,0irq/25-eth00-21swapper/007:07:360
110050270,0irq/25-eth00-21swapper/007:10:000
9950210,0irq/24-0000:00:0-21swapper/307:10:003
30190991514,0cyclictest0-21swapper/212:30:002
30191991413,0cyclictest0-21swapper/309:32:143
3019099140,0cyclictest27414-21ls08:10:212
3019099140,0cyclictest0-21swapper/210:35:062
30191991312,0cyclictest0-21swapper/309:19:273
30191991311,0cyclictest0-21swapper/310:00:283
3019199130,0cyclictest0-21swapper/311:35:163
30190991312,0cyclictest28209-21sshd10:56:072
30190991312,0cyclictest0-21swapper/212:13:442
3019099130,0cyclictest23931-21chrt11:58:242
3019099130,0cyclictest0-21swapper/209:49:332
30189991312,0cyclictest7221-21sshd10:09:491
30189991312,0cyclictest0-21swapper/111:03:081
30189991312,0cyclictest0-21swapper/110:16:061
30189991312,0cyclictest0-21swapper/110:14:001
3018999131,0cyclictest0-21swapper/110:30:161
30188991312,0cyclictest0-21swapper/011:46:520
30188991312,0cyclictest0-21swapper/009:37:290
30188991312,0cyclictest0-21swapper/009:37:280
3018899130,0cyclictest0-21swapper/011:35:250
191130,0ktimersoftd/10-21swapper/112:01:211
351120,0ktimersoftd/30-21swapper/309:14:353
30191991212,0cyclictest0-21swapper/311:14:143
30191991211,0cyclictest0-21swapper/312:23:283
30191991211,0cyclictest0-21swapper/307:50:293
30191991210,0cyclictest0-21swapper/310:22:323
3019199120,0cyclictest0-21swapper/312:00:483
3019199120,0cyclictest0-21swapper/311:03:183
3019199120,0cyclictest0-21swapper/309:50:213
3019199120,0cyclictest0-21swapper/309:29:343
3019199120,0cyclictest0-21swapper/308:53:543
30190991212,0cyclictest0-21swapper/209:59:402
30190991211,0cyclictest0-21swapper/212:17:332
30190991211,0cyclictest0-21swapper/209:52:152
30190991211,0cyclictest0-21swapper/209:42:072
30190991211,0cyclictest0-21swapper/208:55:192
30190991210,0cyclictest0-21swapper/209:24:002
3019099120,0cyclictest0-21swapper/212:20:062
3019099120,0cyclictest0-21swapper/211:24:562
30189991211,0cyclictest110050irq/25-eth009:58:131
30189991211,0cyclictest0-21swapper/110:23:531
30189991210,0cyclictest110050irq/25-eth010:47:001
30189991210,0cyclictest0-21swapper/110:59:111
30189991210,0cyclictest0-21swapper/108:49:541
3018999120,0cyclictest0-21swapper/112:27:081
3018999120,0cyclictest0-21swapper/109:29:591
30188991212,0cyclictest0-21swapper/011:03:530
30188991211,0cyclictest11589-21sshd09:49:410
30188991211,0cyclictest0-21swapper/012:30:330
30188991211,0cyclictest0-21swapper/011:07:500
30188991211,0cyclictest0-21swapper/010:46:490
30188991211,0cyclictest0-21swapper/009:22:360
30188991210,0cyclictest25688-21sshd11:23:520
30188991210,0cyclictest0-21swapper/011:11:480
30188991210,0cyclictest0-21swapper/009:42:140
3018899120,0cyclictest0-21swapper/012:08:500
3018899120,0cyclictest0-21swapper/010:38:340
3018899120,0cyclictest0-21swapper/010:16:300
3018899120,0cyclictest0-21swapper/010:12:190
3018899120,0cyclictest0-21swapper/009:59:110
271120,0ktimersoftd/20-21swapper/212:34:232
3019199119,0cyclictest0-21swapper/311:18:523
30191991111,0cyclictest0-21swapper/311:51:323
30191991111,0cyclictest0-21swapper/311:23:523
30191991111,0cyclictest0-21swapper/311:08:463
30191991110,0cyclictest0-21swapper/310:47:103
30191991110,0cyclictest0-21swapper/310:41:223
3019199110,0cyclictest0-21swapper/308:59:163
3019199110,0cyclictest0-21swapper/308:48:543
3019199110,0cyclictest0-21swapper/308:37:453
3019199110,0cyclictest0-21swapper/307:15:593
30190991110,0cyclictest0-21swapper/210:27:122
3019099110,0cyclictest0-21swapper/212:03:292
3019099110,0cyclictest0-21swapper/209:39:282
3019099110,0cyclictest0-21swapper/209:39:272
30189991111,0cyclictest0-21swapper/110:37:151
30189991111,0cyclictest0-21swapper/109:32:401
30189991110,0cyclictest0-21swapper/111:32:251
30189991110,0cyclictest0-21swapper/109:40:411
30189991110,0cyclictest0-21swapper/109:11:511
3018999110,0cyclictest0-21swapper/111:07:061
3018999110,0cyclictest0-21swapper/108:31:261
3018999110,0cyclictest0-21swapper/108:18:181
3018999110,0cyclictest0-21swapper/107:37:251
3018899119,0cyclictest0-21swapper/010:43:280
30188991111,0cyclictest0-21swapper/011:28:530
30188991111,0cyclictest0-21swapper/009:15:100
30188991110,0cyclictest18056-21sshd09:30:020
30188991110,0cyclictest0-21swapper/012:25:260
30188991110,0cyclictest0-21swapper/010:25:130
3018899110,0cyclictest0-21swapper/011:57:190
3018899110,0cyclictest0-21swapper/010:52:370
191110,0ktimersoftd/10-21swapper/110:00:391
110050110,0irq/25-eth028222-21atenall_r7power09:25:160
110050110,0irq/25-eth019830-21netstat11:50:230
110050110,0irq/25-eth00-21swapper/111:54:431
41100,0ktimersoftd/031546-21bash11:18:050
351100,0ktimersoftd/316890-21sshd10:18:493
3019199109,0cyclictest14639-21sshd09:22:083
3019199109,0cyclictest0-21swapper/312:12:403
3019199109,0cyclictest0-21swapper/311:30:433
3019199109,0cyclictest0-21swapper/308:29:163
30191991010,0cyclictest0-21swapper/312:35:543
30191991010,0cyclictest0-21swapper/310:56:563
30191991010,0cyclictest0-21swapper/309:37:353
30191991010,0cyclictest0-21swapper/309:37:353
30191991010,0cyclictest0-21swapper/307:59:463
30191991010,0cyclictest0-21swapper/307:20:303
3019199100,0cyclictest0-21swapper/311:27:193
3019199100,0cyclictest0-21swapper/310:28:563
3019199100,0cyclictest0-21swapper/308:16:443
3019099109,0cyclictest0-21swapper/212:38:352
3019099109,0cyclictest0-21swapper/210:15:282
30190991010,0cyclictest10102-21sshd10:31:102
30190991010,0cyclictest0-21swapper/211:32:432
30190991010,0cyclictest0-21swapper/210:51:142
30190991010,0cyclictest0-21swapper/210:21:442
30190991010,0cyclictest0-21swapper/209:26:402
30190991010,0cyclictest0-21swapper/209:00:252
3019099100,0cyclictest0-21swapper/211:35:162
3019099100,0cyclictest0-21swapper/210:43:052
3019099100,0cyclictest0-21swapper/208:52:382
3019099100,0cyclictest0-21swapper/207:25:312
3018999109,0cyclictest0-21swapper/111:14:041
3018999109,0cyclictest0-21swapper/110:27:501
30189991010,0cyclictest0-21swapper/111:23:431
30189991010,0cyclictest0-21swapper/109:35:191
30189991010,0cyclictest0-21swapper/109:35:191
30189991010,0cyclictest0-21swapper/109:04:251
30189991010,0cyclictest0-21swapper/107:47:181
3018999100,0cyclictest0-21swapper/110:52:091
30188991010,0cyclictest0-21swapper/010:22:460
30188991010,0cyclictest0-21swapper/010:01:520
30188991010,0cyclictest0-21swapper/008:30:190
3018899100,0cyclictest0-21swapper/012:00:200
191100,0ktimersoftd/10-21swapper/107:18:151
110050100,0irq/25-eth00-21swapper/109:19:111
4190,0ktimersoftd/026196-21cp10:06:420
301919999,0cyclictest0-21swapper/309:06:413
301919999,0cyclictest0-21swapper/308:43:553
301919999,0cyclictest0-21swapper/307:44:473
301919999,0cyclictest0-21swapper/307:26:333
301919998,0cyclictest0-21swapper/311:49:373
301909999,0cyclictest0-21swapper/211:51:392
301909999,0cyclictest0-21swapper/210:02:202
301909998,0cyclictest0-21swapper/211:07:582
301909998,0cyclictest0-21swapper/210:45:272
301909990,0cyclictest0-21swapper/208:18:012
301899999,0cyclictest0-21swapper/109:50:461
301899999,0cyclictest0-21swapper/108:59:401
301899999,0cyclictest0-21swapper/108:39:201
301899999,0cyclictest0-21swapper/108:02:171
301899998,0cyclictest5833-21sshd11:19:381
301899998,0cyclictest0-21swapper/108:50:291
301899992,0cyclictest0-21swapper/108:40:211
301889999,0cyclictest21838-21diskmemload12:10:060
19190,0ktimersoftd/10-21swapper/107:31:301
11005090,0irq/25-eth00-21swapper/111:42:241
11005090,0irq/25-eth00-21swapper/010:31:550
11005090,0irq/25-eth00-21swapper/009:54:450
995080,0irq/24-0000:00:21838-21diskmemload11:30:001
4180,0ktimersoftd/021838-21diskmemload09:12:000
4180,0ktimersoftd/019702-21munin-node09:05:210
4180,0ktimersoftd/00-21swapper/012:15:270
301919988,0cyclictest0-21swapper/310:54:293
301919988,0cyclictest0-21swapper/310:35:533
301919988,0cyclictest0-21swapper/310:30:263
301919988,0cyclictest0-21swapper/309:40:153
301919988,0cyclictest0-21swapper/309:01:553
301919988,0cyclictest0-21swapper/308:30:263
301919988,0cyclictest0-21swapper/307:30:443
301919980,0cyclictest0-21swapper/312:26:483
301909988,0cyclictest0-21swapper/211:25:512
301909988,0cyclictest0-21swapper/211:16:492
301909988,0cyclictest0-21swapper/211:13:262
301909988,0cyclictest0-21swapper/209:32:012
301909988,0cyclictest0-21swapper/208:49:582
301909988,0cyclictest0-21swapper/208:44:372
301909988,0cyclictest0-21swapper/208:39:172
301909988,0cyclictest0-21swapper/207:56:042
301909987,0cyclictest0-21swapper/210:11:172
301899987,0cyclictest0-21swapper/110:43:381
301899987,0cyclictest0-21swapper/109:46:011
301899980,0cyclictest21838-21diskmemload12:37:311
301889988,0cyclictest0-21swapper/008:38:200
301889987,0cyclictest0-21swapper/011:40:230
301889980,0cyclictest0-21swapper/012:35:240
149980,0migration/027391-21cstates12:20:130
11795080,0irq/26-eth1-rx-0-21swapper/011:33:000
11005080,0irq/25-eth021838-21diskmemload11:55:023
11005080,0irq/25-eth020-21ksoftirqd/112:20:181
11005080,0irq/25-eth013085-21sshd12:30:573
4170,0ktimersoftd/00-21swapper/007:55:250
329970,0watchdog/30-21swapper/310:06:193
301919975,0cyclictest17769-21kworker/3:008:20:193
301919973,0cyclictest0-21swapper/310:10:233
301909977,0cyclictest0-21swapper/211:00:052
301909977,0cyclictest0-21swapper/210:05:002
301909977,0cyclictest0-21swapper/209:05:322
301909977,0cyclictest0-21swapper/208:26:162
301909977,0cyclictest0-21swapper/207:49:062
301909977,0cyclictest0-21swapper/207:30:032
301909973,0cyclictest0-21swapper/207:35:202
301909972,0cyclictest0-21swapper/212:05:182
301909970,0cyclictest0-21swapper/207:10:342
301899977,0cyclictest0-21swapper/108:05:111
301899977,0cyclictest0-21swapper/107:56:351
301899977,0cyclictest0-21swapper/107:52:591
301899977,0cyclictest0-21swapper/107:41:361
301889976,0cyclictest0-21swapper/007:50:220
301889975,0cyclictest0-21swapper/007:25:320
301889973,0cyclictest0-21swapper/008:17:140
301889970,0cyclictest25282-21sed10:55:320
301889970,0cyclictest0-21swapper/008:40:210
301889970,0cyclictest0-21swapper/007:40:260
179970,0migration/112640-21bash11:35:031
11795070,0irq/26-eth1-rx-27187-21sshd09:10:492
11795070,0irq/26-eth1-rx-0-21swapper/009:04:420
11005070,0irq/25-eth06939-21munin-node11:40:313
11005070,0irq/25-eth02720-21consoletype12:07:473
11005070,0irq/25-eth024628-21munin-node09:45:203
11005070,0irq/25-eth021711-21sshd09:16:372
11005070,0irq/25-eth020-21ksoftirqd/111:55:201
11005070,0irq/25-eth014399-21sshd12:17:171
11005070,0irq/25-eth01335-21sshd12:07:271
11005070,0irq/25-eth00-21swapper/312:15:173
11005070,0irq/25-eth00-21swapper/309:55:183
11005070,0irq/25-eth00-21swapper/111:45:181
11005070,0irq/25-eth00-21swapper/008:01:370
301919962,0cyclictest0-21swapper/308:05:203
301919962,0cyclictest0-21swapper/308:00:213
301919962,0cyclictest0-21swapper/307:45:203
301919962,0cyclictest0-21swapper/307:35:203
301909965,0cyclictest3433-21expr07:20:162
301909965,0cyclictest26639-21bash11:45:022
301909962,0cyclictest0-21swapper/208:05:212
301909962,0cyclictest0-21swapper/208:00:212
301909962,0cyclictest0-21swapper/207:50:182
301909962,0cyclictest0-21swapper/207:15:012
301909960,0cyclictest0-21swapper/208:33:212
301909960,0cyclictest0-21swapper/208:20:212
301899966,0cyclictest0-21swapper/109:07:071
301899966,0cyclictest0-21swapper/108:20:031
301899965,0cyclictest8443-21sshd12:30:141
301899965,0cyclictest14339-21bash12:10:161
301899962,0cyclictest0-21swapper/108:25:251
301899962,0cyclictest0-21swapper/107:20:201
301899962,0cyclictest0-21swapper/107:10:281
301899961,0cyclictest0-21swapper/107:25:251
301899960,0cyclictest0-21swapper/109:20:171
301889965,0cyclictest12543-21latency_hist08:50:000
301889963,0cyclictest0-21swapper/008:05:180
301889962,0cyclictest0-21swapper/008:50:280
301889962,0cyclictest0-21swapper/008:10:190
301889962,0cyclictest0-21swapper/007:20:200
301889960,0cyclictest0-21swapper/008:55:140
301889960,0cyclictest0-21swapper/008:25:220
301889960,0cyclictest0-21swapper/007:45:000
301889960,0cyclictest0-21swapper/007:30:060
301889960,0cyclictest0-21swapper/007:15:000
259960,0migration/212582-21munin-node07:40:152
11005060,0irq/25-eth04456-21sshd11:40:172
301919951,0cyclictest0-21swapper/308:10:193
301889954,0cyclictest3-21ksoftirqd/008:20:140
11795050,0irq/26-eth1-rx-0-21swapper/108:10:271
11795050,0irq/26-eth1-rx-0-21swapper/007:35:190
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional