You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-08-09 - 00:30

x86 Intel Xeon E3-1220L V2 @2300 MHz, Linux 4.4.39-rt50+ (Profile)

Latency plot of system in rack #7, slot #0
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack7slot0.osadl.org (updated Mon Aug 08, 2022 12:44:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
110050660,0irq/25-eth00-21swapper/107:05:241
117950600,0irq/26-eth1-rx-0-21swapper/307:09:213
117950570,0irq/26-eth1-rx-0-21swapper/207:05:002
117950550,0irq/26-eth1-rx-0-21swapper/007:09:510
110050270,0irq/25-eth00-21swapper/107:10:001
117950210,0irq/26-eth1-rx-0-21swapper/007:10:010
3011799191,0cyclictest26896-21id12:21:183
191190,0ktimersoftd/114918-21sshd11:48:061
271180,0ktimersoftd/214929-21sshd11:48:062
3011699160,0cyclictest0-21swapper/209:38:312
3011599160,0cyclictest0-21swapper/111:11:151
3011799150,0cyclictest0-21swapper/311:14:013
30115991512,0cyclictest0-21swapper/111:44:001
110050150,0irq/25-eth03-21ksoftirqd/012:24:480
30117991413,0cyclictest28867-21sshd11:49:403
30116991412,0cyclictest0-21swapper/209:19:212
30116991411,0cyclictest0-21swapper/212:35:202
30115991413,0cyclictest21372-21sshd10:38:081
30115991413,0cyclictest0-21swapper/110:53:351
3011599140,0cyclictest31471-21bash10:21:311
3011599140,0cyclictest2940-21id11:36:061
3011599140,0cyclictest0-21swapper/109:49:001
9950130,0irq/24-0000:00:985-21sshd12:22:032
30117991312,0cyclictest0-21swapper/312:08:293
30117991312,0cyclictest0-21swapper/310:42:393
30117991312,0cyclictest0-21swapper/309:36:473
3011799130,0cyclictest25700-21sshd09:56:113
3011799130,0cyclictest0-21swapper/311:41:463
3011799130,0cyclictest0-21swapper/310:06:103
3011799130,0cyclictest0-21swapper/309:26:403
3011799130,0cyclictest0-21swapper/309:14:563
30116991312,0cyclictest17307-21sshd10:34:102
30116991312,0cyclictest0-21swapper/207:55:162
30116991311,0cyclictest0-21swapper/209:59:402
3011699130,0cyclictest16053-21bash11:16:202
3011699130,0cyclictest0-21swapper/212:28:292
30115991312,0cyclictest28834-21sshd10:00:091
30115991312,0cyclictest24359-21sshd11:59:481
30114991312,0cyclictest0-21swapper/011:53:540
30114991312,0cyclictest0-21swapper/011:36:570
30114991311,0cyclictest25693-21sshd12:28:290
30114991311,0cyclictest0-21swapper/012:13:460
3011499130,0cyclictest31390-21sshd11:10:550
3011499130,0cyclictest21740-21diskmemload12:09:350
3011499130,0cyclictest0-21swapper/009:42:240
3011499130,0cyclictest0-21swapper/009:30:330
110050130,0irq/25-eth01402-21sshd09:25:220
110050130,0irq/25-eth01330-21id10:07:410
41120,0ktimersoftd/024196-21bash09:24:240
41120,0ktimersoftd/01512-21sshd09:55:320
41120,0ktimersoftd/00-21swapper/010:15:370
41120,0ktimersoftd/00-21swapper/009:16:090
30117991211,0cyclictest0-21swapper/311:27:083
30117991210,0cyclictest0-21swapper/311:59:193
30117991210,0cyclictest0-21swapper/310:28:223
30117991210,0cyclictest0-21swapper/308:35:333
3011799120,0cyclictest0-21swapper/308:50:463
3011699129,0cyclictest0-21swapper/212:15:202
3011699129,0cyclictest0-21swapper/210:35:192
3011699125,0cyclictest20588-21bash10:52:082
30116991211,0cyclictest0-21swapper/212:04:122
30116991211,0cyclictest0-21swapper/211:42:222
30116991211,0cyclictest0-21swapper/210:46:082
30116991210,0cyclictest691-21perf08:24:592
30116991210,0cyclictest0-21swapper/209:33:452
3011699120,0cyclictest2013-21rm12:07:542
3011699120,0cyclictest0-21swapper/211:53:452
3011699120,0cyclictest0-21swapper/210:18:432
3011699120,0cyclictest0-21swapper/210:09:392
30115991212,0cyclictest0-21swapper/112:31:031
30115991211,0cyclictest30063-21sshd10:10:421
30115991211,0cyclictest0-21swapper/109:12:591
30115991210,0cyclictest0-21swapper/111:28:471
3011599120,0cyclictest1453-21nfsd11:05:261
3011599120,0cyclictest0-21swapper/109:17:101
30114991210,0cyclictest0-21swapper/012:15:510
30114991210,0cyclictest0-21swapper/009:47:230
30114991210,0cyclictest0-21swapper/007:15:480
3011499120,0cyclictest13814-21sshd10:23:110
3011499120,0cyclictest0-21swapper/011:30:120
3011499120,0cyclictest0-21swapper/010:02:220
271120,0ktimersoftd/20-21swapper/212:30:332
271120,0ktimersoftd/20-21swapper/209:14:362
30117991111,0cyclictest0-21swapper/311:35:023
30117991111,0cyclictest0-21swapper/311:15:103
30117991111,0cyclictest0-21swapper/309:00:533
30117991110,0cyclictest32541-21sshd11:21:413
30117991110,0cyclictest25552-21sshd12:28:273
30117991110,0cyclictest17535-21sshd09:20:123
30117991110,0cyclictest0-21swapper/312:10:343
30117991110,0cyclictest0-21swapper/312:02:063
30117991110,0cyclictest0-21swapper/310:12:203
3011799110,0cyclictest0-21swapper/309:40:023
3011799110,0cyclictest0-21swapper/308:33:293
3011799110,0cyclictest0-21swapper/308:07:343
3011799110,0cyclictest0-21swapper/308:03:433
3011799110,0cyclictest0-21swapper/307:45:573
3011699114,0cyclictest2322-21sshd10:25:252
30116991111,0cyclictest0-21swapper/211:34:282
30116991110,0cyclictest0-21swapper/210:01:522
3011699110,0cyclictest0-21swapper/211:27:082
30115991111,0cyclictest0-21swapper/112:17:201
30115991111,0cyclictest0-21swapper/111:20:181
30115991111,0cyclictest0-21swapper/109:38:591
3011599111,0cyclictest4980-21bash10:25:381
30115991110,0cyclictest0-21swapper/112:22:051
30115991110,0cyclictest0-21swapper/109:44:341
3011599110,0cyclictest23474-21sshd09:59:331
3011599110,0cyclictest0-21swapper/112:04:061
3011599110,0cyclictest0-21swapper/111:19:421
3011499119,0cyclictest0-21swapper/011:43:410
30114991111,0cyclictest0-21swapper/011:22:520
30114991111,0cyclictest0-21swapper/010:47:410
30114991111,0cyclictest0-21swapper/010:36:180
30114991111,0cyclictest0-21swapper/009:52:240
30114991110,0cyclictest0-21swapper/009:00:140
3011499110,0cyclictest0-21swapper/009:11:020
110050110,0irq/25-eth06454-21sshd12:04:550
110050110,0irq/25-eth05005-21head08:30:270
110050110,0irq/25-eth00-21swapper/312:33:353
110050110,0irq/25-eth00-21swapper/112:14:541
110050110,0irq/25-eth00-21swapper/008:30:000
9950100,0irq/24-0000:00:0-21swapper/010:55:250
41100,0ktimersoftd/030175-21bash11:17:570
41100,0ktimersoftd/023508-21sshd10:10:090
41100,0ktimersoftd/00-21swapper/007:45:250
351100,0ktimersoftd/30-21swapper/310:38:073
3011799109,0cyclictest9245-21munin-node09:15:323
3011799109,0cyclictest1875-21sshd11:50:153
3011799109,0cyclictest14980-21sshd10:48:003
3011799109,0cyclictest0-21swapper/310:59:093
3011799109,0cyclictest0-21swapper/307:22:153
30117991010,0cyclictest0-21swapper/309:05:043
30117991010,0cyclictest0-21swapper/308:58:123
30117991010,0cyclictest0-21swapper/308:43:563
30117991010,0cyclictest0-21swapper/307:51:193
30117991010,0cyclictest0-21swapper/307:33:243
30117991010,0cyclictest0-21swapper/307:33:243
30117991010,0cyclictest0-21swapper/307:25:173
3011699109,0cyclictest2499-21sshd09:53:452
3011699109,0cyclictest0-21swapper/210:20:272
3011699109,0cyclictest0-21swapper/210:10:152
3011699109,0cyclictest0-21swapper/208:41:222
3011699108,0cyclictest0-21swapper/211:35:022
3011699106,0cyclictest25416-21sshd09:28:012
3011699105,0cyclictest9376-21bash09:43:572
3011699105,0cyclictest21444-21sshd09:45:162
3011699105,0cyclictest16324-21fw_conntrack11:55:192
3011699105,0cyclictest13567-21bash11:12:332
3011699104,0cyclictest16098-21bash11:09:222
3011699100,0cyclictest0-21swapper/210:55:182
3011599109,0cyclictest12890-21bash09:51:171
3011599109,0cyclictest0-21swapper/110:49:021
30115991010,0cyclictest0-21swapper/109:28:541
30115991010,0cyclictest0-21swapper/109:01:501
3011599100,0cyclictest20490-21bash12:06:191
3011599100,0cyclictest0-21swapper/110:33:081
3011599100,0cyclictest0-21swapper/110:05:091
3011499109,0cyclictest0-21swapper/008:10:030
3011499101,0cyclictest0-21swapper/010:33:190
3011499101,0cyclictest0-21swapper/007:50:290
30114991010,0cyclictest0-21swapper/009:35:190
30114991010,0cyclictest0-21swapper/007:59:570
3011499100,0cyclictest0-21swapper/012:35:120
3011499100,0cyclictest0-21swapper/011:04:460
3011499100,0cyclictest0-21swapper/010:25:220
3011499100,0cyclictest0-21swapper/007:24:370
1499100,0migration/011454-21sshd10:40:280
117950100,0irq/26-eth1-rx-28-21ksoftirqd/209:20:412
117950100,0irq/26-eth1-rx-19549-21sshd09:45:033
117950100,0irq/26-eth1-rx-16389-21kworker/1:112:35:111
117950100,0irq/26-eth1-rx-12381-21sshd09:33:411
110050100,0irq/25-eth07937-21sshd11:33:101
110050100,0irq/25-eth027933-21sshd11:07:030
110050100,0irq/25-eth021740-21diskmemload10:51:320
4190,0ktimersoftd/00-21swapper/011:57:240
4190,0ktimersoftd/00-21swapper/008:04:070
301179999,0cyclictest0-21swapper/310:01:453
301179999,0cyclictest0-21swapper/309:51:463
301179999,0cyclictest0-21swapper/308:11:103
301179998,0cyclictest0-21swapper/310:30:053
301179998,0cyclictest0-21swapper/308:29:533
301179998,0cyclictest0-21swapper/307:43:523
301179992,0cyclictest0-21swapper/307:10:293
301169999,0cyclictest0-21swapper/208:57:582
301169998,0cyclictest0-21swapper/207:50:082
301169998,0cyclictest0-21swapper/207:12:302
301169995,0cyclictest29128-21bash11:00:132
301169994,0cyclictest4048-21sshd12:11:382
301169994,0cyclictest3443-21sshd11:22:042
301169991,0cyclictest0-21swapper/210:40:212
301169990,0cyclictest0-21swapper/208:38:222
301159999,0cyclictest0-21swapper/107:34:431
301159999,0cyclictest0-21swapper/107:34:421
301159998,0cyclictest11230-21sed10:40:261
301159990,0cyclictest0-21swapper/110:57:171
301159990,0cyclictest0-21swapper/108:56:091
301149999,0cyclictest0-21swapper/009:05:560
301149999,0cyclictest0-21swapper/008:39:190
301149999,0cyclictest0-21swapper/007:30:400
301149999,0cyclictest0-21swapper/007:30:400
301149999,0cyclictest0-21swapper/007:26:090
301149998,0cyclictest8586-21sshd12:33:450
301149998,0cyclictest30372-21sshd11:46:110
301149995,0cyclictest0-21swapper/011:25:170
11795090,0irq/26-eth1-rx-18-21rcuc/109:20:591
11795090,0irq/26-eth1-rx-0-21swapper/108:15:221
11005090,0irq/25-eth027976-21sshd10:17:383
301179988,0cyclictest0-21swapper/307:57:143
301169987,0cyclictest0-21swapper/208:08:432
301169983,0cyclictest28-21ksoftirqd/207:25:292
301169983,0cyclictest23147-21sed08:00:272
301169983,0cyclictest20152-21cut09:05:192
301169981,0cyclictest0-21swapper/209:00:192
301169981,0cyclictest0-21swapper/207:15:432
301169980,0cyclictest0-21swapper/208:27:252
301169980,0cyclictest0-21swapper/207:30:262
301169980,0cyclictest0-21swapper/207:30:252
301159988,0cyclictest0-21swapper/107:51:331
301159988,0cyclictest0-21swapper/107:21:561
301159984,0cyclictest0-21swapper/107:25:221
301159983,0cyclictest0-21swapper/109:07:141
301159983,0cyclictest0-21swapper/108:06:271
301159983,0cyclictest0-21swapper/108:01:561
301159982,0cyclictest0-21swapper/108:37:371
301159982,0cyclictest0-21swapper/108:11:001
301149988,0cyclictest0-21swapper/008:19:270
301149984,0cyclictest0-21swapper/008:55:170
301149984,0cyclictest0-21swapper/008:05:270
11795080,0irq/26-eth1-rx-0-21swapper/108:27:111
11005080,0irq/25-eth06368-21sshd10:54:113
11005080,0irq/25-eth04040-21sshd11:00:531
11005080,0irq/25-eth02939-21sshd12:18:433
11005080,0irq/25-eth027246-21sshd10:24:423
11005080,0irq/25-eth024261-21bash09:31:213
11005080,0irq/25-eth021740-21diskmemload11:00:043
11005080,0irq/25-eth020457-21sshd10:16:471
11005080,0irq/25-eth011216-21sshd12:37:343
11005080,0irq/25-eth00-21swapper/311:31:403
11005080,0irq/25-eth00-21swapper/311:07:423
301179977,0cyclictest0-21swapper/308:48:423
301179970,0cyclictest0-21swapper/307:35:293
301169970,0cyclictest0-21swapper/208:50:182
301169970,0cyclictest0-21swapper/208:10:212
301159976,0cyclictest20-21ksoftirqd/112:25:181
301159972,0cyclictest0-21swapper/107:40:541
301149970,0cyclictest0-21swapper/008:40:240
301149970,0cyclictest0-21swapper/008:24:230
301149970,0cyclictest0-21swapper/007:40:220
19170,0ktimersoftd/10-21swapper/107:56:531
11795070,0irq/26-eth1-rx-15133-21df_inode07:45:161
11005070,0irq/25-eth06404-21sshd11:50:361
4160,0ktimersoftd/00-21swapper/008:50:140
329960,0watchdog/331128-21cut08:20:103
301179962,0cyclictest0-21swapper/308:15:163
301179962,0cyclictest0-21swapper/307:15:013
301169965,0cyclictest12524-21cpu07:40:142
301169963,0cyclictest0-21swapper/208:33:432
301169960,0cyclictest0-21swapper/208:19:122
301169960,0cyclictest0-21swapper/207:45:192
301159962,0cyclictest0-21swapper/108:45:191
301159962,0cyclictest0-21swapper/108:40:201
301159962,0cyclictest0-21swapper/108:30:171
301159962,0cyclictest0-21swapper/108:20:161
301159962,0cyclictest0-21swapper/107:15:011
301159961,0cyclictest0-21swapper/107:35:131
301149962,0cyclictest0-21swapper/008:45:200
301149960,0cyclictest0-21swapper/007:35:350
179960,0migration/113819-21nfsd408:50:191
301169950,0cyclictest0-21swapper/208:45:132
301169950,0cyclictest0-21swapper/207:35:412
301169950,0cyclictest0-21swapper/207:20:132
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional