You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-11-28 - 01:37

x86 Intel Xeon E3-1220L V2 @2300 MHz, Linux 4.4.39-rt50+ (Profile)

Latency plot of system in rack #7, slot #0
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the alll highest latencies:
System rack7slot0.osadl.org (updated Sun Nov 27, 2022 12:44:59)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
110050600,0irq/25-eth00-21swapper/307:07:023
110050590,0irq/25-eth00-21swapper/107:06:421
1722524721,0sleep00-21swapper/007:07:260
1715324522,0sleep20-21swapper/207:06:332
110050270,0irq/25-eth00-21swapper/307:10:013
17697991716,0cyclictest14336-21sshd12:27:322
110050170,0irq/25-eth06296-21sshd09:43:271
17696991612,0cyclictest0-21swapper/107:10:171
17695991615,0cyclictest110050irq/25-eth009:58:180
110050160,0irq/25-eth020-21ksoftirqd/111:33:251
351150,0ktimersoftd/30-21swapper/312:19:303
17698991512,0cyclictest0-21swapper/309:52:533
17697991514,0cyclictest0-21swapper/210:47:382
110050150,0irq/25-eth01452-21nfsd09:34:371
110050150,0irq/25-eth00-21swapper/111:08:111
17698991413,0cyclictest13487-21sshd12:00:123
17698991413,0cyclictest0-21swapper/310:29:073
17698991411,0cyclictest17924-21bash09:31:033
1769899140,0cyclictest0-21swapper/310:38:253
17696991411,0cyclictest8965-21sshd10:37:591
17696991411,0cyclictest0-21swapper/110:52:021
17698991312,0cyclictest12984-21sed12:10:183
17698991312,0cyclictest0-21swapper/312:33:043
17698991312,0cyclictest0-21swapper/311:20:213
17698991311,0cyclictest0-21swapper/311:50:343
17698991311,0cyclictest0-21swapper/310:43:323
17698991311,0cyclictest0-21swapper/310:21:063
1769899130,0cyclictest9318-21diskmemload09:35:483
1769899130,0cyclictest0-21swapper/311:15:563
17697991312,0cyclictest5072-21bash10:30:402
17697991312,0cyclictest0-21swapper/212:37:592
17697991312,0cyclictest0-21swapper/212:04:112
17697991312,0cyclictest0-21swapper/211:39:542
1769799130,0cyclictest0-21swapper/209:35:442
1769799130,0cyclictest0-21swapper/209:30:222
17696991312,0cyclictest11562-21sshd12:16:571
17696991312,0cyclictest0-21swapper/110:15:131
17696991311,0cyclictest0-21swapper/111:27:491
17696991311,0cyclictest0-21swapper/110:26:081
1769699130,0cyclictest0-21swapper/110:59:221
1769699130,0cyclictest0-21swapper/108:25:191
17695991312,0cyclictest28951-21sshd12:25:350
17695991311,0cyclictest9691-21sshd12:03:100
1769599130,0cyclictest0-21swapper/010:17:200
1769599130,0cyclictest0-21swapper/009:24:180
110050130,0irq/25-eth012335-21sshd10:45:121
17698991211,0cyclictest16027-21sshd12:07:133
17698991211,0cyclictest0-21swapper/311:37:053
17698991211,0cyclictest0-21swapper/310:47:223
17698991211,0cyclictest0-21swapper/309:18:013
17698991210,0cyclictest0-21swapper/312:38:533
1769899120,0cyclictest0-21swapper/311:34:523
1769899120,0cyclictest0-21swapper/311:04:333
1769899120,0cyclictest0-21swapper/310:58:313
1769899120,0cyclictest0-21swapper/310:52:283
1769899120,0cyclictest0-21swapper/308:50:163
1769899120,0cyclictest0-21swapper/307:20:253
17697991211,0cyclictest19210-21kworker/2:010:38:532
17697991211,0cyclictest15090-21sshd11:12:392
17697991211,0cyclictest0-21swapper/210:01:242
17697991210,0cyclictest0-21swapper/212:20:272
17697991210,0cyclictest0-21swapper/210:50:112
17697991210,0cyclictest0-21swapper/210:13:012
1769799120,0cyclictest0-21swapper/212:07:332
1769799120,0cyclictest0-21swapper/211:53:022
1769799120,0cyclictest0-21swapper/211:06:062
1769799120,0cyclictest0-21swapper/210:43:482
1769799120,0cyclictest0-21swapper/209:21:262
17696991212,0cyclictest0-21swapper/112:09:171
17696991211,0cyclictest9318-21diskmemload11:45:151
17696991211,0cyclictest0-21swapper/112:12:401
17696991211,0cyclictest0-21swapper/111:56:031
17696991211,0cyclictest0-21swapper/110:44:221
17696991211,0cyclictest0-21swapper/110:20:411
17696991211,0cyclictest0-21swapper/109:55:561
1769699120,0cyclictest7424-21sshd09:53:441
1769699120,0cyclictest18720-21sshd09:17:401
1769699120,0cyclictest0-21swapper/112:23:071
1769699120,0cyclictest0-21swapper/111:35:441
17695991211,0cyclictest30046-21bash09:42:300
17695991211,0cyclictest0-21swapper/012:36:230
17695991211,0cyclictest0-21swapper/009:45:590
17695991210,0cyclictest0-21swapper/010:35:000
17695991210,0cyclictest0-21swapper/009:29:290
1769599120,0cyclictest0-21swapper/011:56:400
1769599120,0cyclictest0-21swapper/011:23:400
1769599120,0cyclictest0-21swapper/010:04:270
1769599120,0cyclictest0-21swapper/009:10:260
110050120,0irq/25-eth00-21swapper/311:45:543
110050120,0irq/25-eth00-21swapper/311:13:163
110050120,0irq/25-eth00-21swapper/109:47:341
110050120,0irq/25-eth00-21swapper/010:05:120
41110,0ktimersoftd/025965-21sshd09:15:040
41110,0ktimersoftd/00-21swapper/011:03:490
271110,0ktimersoftd/20-21swapper/211:59:522
17698991110,0cyclictest7380-21sshd11:42:253
17698991110,0cyclictest6626-21sshd09:23:113
17698991110,0cyclictest6331-21sshd10:10:273
17698991110,0cyclictest2735-21sshd12:29:473
17698991110,0cyclictest27285-21sshd11:27:323
17698991110,0cyclictest25807-21sshd12:21:533
17698991110,0cyclictest18999-21sshd11:57:183
17698991110,0cyclictest16106-21sshd09:10:333
17698991110,0cyclictest14305-21cp09:47:383
17698991110,0cyclictest0-21swapper/310:08:343
1769899110,0cyclictest0-21swapper/309:40:193
17697991111,0cyclictest0-21swapper/211:44:192
17697991111,0cyclictest0-21swapper/209:56:172
17697991110,0cyclictest9318-21diskmemload12:15:132
17697991110,0cyclictest7448-21sshd10:24:152
17697991110,0cyclictest25827-21sshd10:56:452
17697991110,0cyclictest11439-21sshd11:25:482
17697991110,0cyclictest0-21swapper/212:10:552
17697991110,0cyclictest0-21swapper/211:45:152
17697991110,0cyclictest0-21swapper/210:27:102
17697991110,0cyclictest0-21swapper/209:13:112
17696991111,0cyclictest0-21swapper/109:27:081
17696991111,0cyclictest0-21swapper/109:22:221
17696991111,0cyclictest0-21swapper/108:52:591
17696991111,0cyclictest0-21swapper/107:50:141
17696991110,0cyclictest31171-21sshd12:39:351
17696991110,0cyclictest0-21swapper/112:29:161
17696991110,0cyclictest0-21swapper/111:40:451
1769699110,0cyclictest0-21swapper/108:36:361
1769699110,0cyclictest0-21swapper/108:19:101
1769699110,0cyclictest0-21swapper/107:19:561
1769599119,0cyclictest9318-21diskmemload11:10:050
17695991111,0cyclictest0-21swapper/012:09:540
17695991111,0cyclictest0-21swapper/011:49:480
17695991111,0cyclictest0-21swapper/009:34:150
17695991111,0cyclictest0-21swapper/008:05:150
17695991110,0cyclictest8340-21sshd10:51:300
17695991110,0cyclictest2046-21fw_forwarded_lo10:20:150
17695991110,0cyclictest0-21swapper/011:50:450
17695991110,0cyclictest0-21swapper/010:58:580
17695991110,0cyclictest0-21swapper/009:51:390
1769599110,0cyclictest0-21swapper/011:25:530
1769599110,0cyclictest0-21swapper/010:13:100
1769599110,0cyclictest0-21swapper/007:44:490
1599110,0watchdog/031014-21sshd10:26:390
110050110,0irq/25-eth09318-21diskmemload11:54:461
110050110,0irq/25-eth09318-21diskmemload11:37:070
110050110,0irq/25-eth029379-21sshd10:33:201
110050110,0irq/25-eth00-21swapper/110:13:361
110050110,0irq/25-eth00-21swapper/110:01:241
41100,0ktimersoftd/022465-21bash11:33:520
191100,0ktimersoftd/10-21swapper/107:34:291
1769899109,0cyclictest5330-21sshd11:08:153
1769899109,0cyclictest0-21swapper/310:00:183
1769899109,0cyclictest0-21swapper/309:55:053
1769899109,0cyclictest0-21swapper/307:26:313
17698991010,0cyclictest0-21swapper/307:55:473
17698991010,0cyclictest0-21swapper/307:44:093
17698991010,0cyclictest0-21swapper/307:37:333
17698991010,0cyclictest0-21swapper/307:19:323
1769899100,0cyclictest17710-21sshd10:15:103
1769799109,0cyclictest5000-21sshd09:50:062
1769799109,0cyclictest23984-21sshd10:19:122
1769799109,0cyclictest23001-21id09:48:342
1769799109,0cyclictest22358-21sshd10:08:502
17697991010,0cyclictest428-21sshd09:25:512
17697991010,0cyclictest0-21swapper/211:32:012
17697991010,0cyclictest0-21swapper/211:21:542
1769799100,0cyclictest0-21swapper/212:32:042
1769799100,0cyclictest0-21swapper/211:03:242
1769799100,0cyclictest0-21swapper/207:50:432
1769699109,0cyclictest9318-21diskmemload09:11:551
1769699109,0cyclictest5581-21bash11:18:291
1769699109,0cyclictest10541-21sshd11:12:091
1769699101,0cyclictest0-21swapper/111:20:281
17696991010,0cyclictest9318-21diskmemload11:04:071
17696991010,0cyclictest0-21swapper/108:23:561
17696991010,0cyclictest0-21swapper/107:37:001
1769699100,0cyclictest9318-21diskmemload09:36:261
1769699100,0cyclictest0-21swapper/108:41:011
1769599109,0cyclictest28349-21sshd11:07:140
1769599109,0cyclictest2022-21sshd11:18:070
1769599109,0cyclictest0-21swapper/012:16:480
1769599108,0cyclictest8428-21sshd12:13:160
17695991010,0cyclictest0-21swapper/008:02:080
17695991010,0cyclictest0-21swapper/007:29:570
17695991010,0cyclictest0-21swapper/007:13:340
1769599100,0cyclictest0-21swapper/011:43:430
1769599100,0cyclictest0-21swapper/007:47:020
110050100,0irq/25-eth0414-21bash12:22:390
110050100,0irq/25-eth019752-21sshd10:42:300
110050100,0irq/25-eth018020-21sed10:35:270
4190,0ktimersoftd/00-21swapper/010:48:190
4190,0ktimersoftd/00-21swapper/007:34:280
27190,0ktimersoftd/20-21swapper/207:34:292
176989999,0cyclictest0-21swapper/308:19:013
176989999,0cyclictest0-21swapper/307:52:393
176989999,0cyclictest0-21swapper/307:49:313
176979998,0cyclictest9872-21sshd11:18:572
176979998,0cyclictest30238-21id09:18:542
176979998,0cyclictest25122-21chrt08:31:572
176979998,0cyclictest1199-21sshd09:42:542
176979990,0cyclictest0-21swapper/207:55:232
176969999,0cyclictest110050irq/25-eth012:32:561
176969999,0cyclictest0-21swapper/109:01:261
176969999,0cyclictest0-21swapper/108:06:521
176969999,0cyclictest0-21swapper/107:26:531
176969998,0cyclictest24514-21consoletype10:05:341
176969998,0cyclictest0-21swapper/112:02:331
176969998,0cyclictest0-21swapper/108:10:561
176969998,0cyclictest0-21swapper/107:23:251
176959999,0cyclictest0-21swapper/009:00:540
176959999,0cyclictest0-21swapper/008:38:500
176959999,0cyclictest0-21swapper/007:39:080
176959999,0cyclictest0-21swapper/007:20:260
176959998,0cyclictest0-21swapper/007:15:250
176959996,0cyclictest6277-21sshd12:30:090
176959990,0cyclictest9318-21diskmemload09:37:370
11005090,0irq/25-eth017455-21cp10:32:023
176989988,0cyclictest0-21swapper/309:09:473
176989988,0cyclictest0-21swapper/308:56:333
176989988,0cyclictest0-21swapper/308:48:043
176989988,0cyclictest0-21swapper/308:40:303
176989988,0cyclictest0-21swapper/308:26:003
176989988,0cyclictest0-21swapper/308:12:593
176989988,0cyclictest0-21swapper/308:08:543
176979987,0cyclictest0-21swapper/208:26:302
176979980,0cyclictest0-21swapper/207:25:202
176969988,0cyclictest0-21swapper/108:30:541
176969987,0cyclictest0-21swapper/108:56:071
176969987,0cyclictest0-21swapper/107:56:311
176959988,0cyclictest0-21swapper/007:56:470
11005080,0irq/25-eth02770-21sshd09:26:053
995070,0irq/24-0000:00:16045-21latency_hist08:15:000
176989977,0cyclictest0-21swapper/309:02:493
176989977,0cyclictest0-21swapper/308:22:303
176989977,0cyclictest0-21swapper/308:03:553
176979977,0cyclictest0-21swapper/207:45:352
176979977,0cyclictest0-21swapper/207:20:162
176969977,0cyclictest0-21swapper/108:02:471
176959977,0cyclictest0-21swapper/008:24:540
176959976,0cyclictest4631-21sh09:00:000
176959970,0cyclictest0-21swapper/008:48:510
4160,0ktimersoftd/024027-21sh08:30:180
4160,0ktimersoftd/00-21swapper/008:28:100
176989962,0cyclictest0-21swapper/308:35:173
176989962,0cyclictest0-21swapper/308:30:213
176989962,0cyclictest0-21swapper/307:30:193
176979965,0cyclictest20333-21vmstat08:20:242
176979965,0cyclictest10018-21fw_conntrack08:00:192
176979962,0cyclictest0-21swapper/209:05:182
176979962,0cyclictest0-21swapper/209:00:202
176979962,0cyclictest0-21swapper/208:55:152
176979962,0cyclictest0-21swapper/208:50:222
176979962,0cyclictest0-21swapper/208:40:162
176979962,0cyclictest0-21swapper/208:10:162
176979962,0cyclictest0-21swapper/208:05:192
176979962,0cyclictest0-21swapper/207:15:022
176979962,0cyclictest0-21swapper/207:10:172
176979960,0cyclictest0-21swapper/208:47:372
176969965,0cyclictest32292-21cat07:40:011
176969962,0cyclictest0-21swapper/109:05:181
176969962,0cyclictest0-21swapper/108:45:181
176969962,0cyclictest0-21swapper/107:45:171
176959965,0cyclictest0-21swapper/007:50:100
176959962,0cyclictest788-40kipmi008:40:150
176959962,0cyclictest0-21swapper/009:05:170
176959962,0cyclictest0-21swapper/008:15:180
176959961,0cyclictest0-21swapper/008:50:200
11005060,0irq/25-eth016280-21munin-node08:15:252
249950,0watchdog/20-21swapper/208:35:232
176979950,0cyclictest0-21swapper/207:40:102
176979950,0cyclictest0-21swapper/207:35:492
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional