You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-06-06 - 15:48

x86 Intel Xeon E3-1220L V2 @2300 MHz, Linux 4.4.39-rt50+ (Profile)

Latency plot of system in rack #7, slot #0
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 792 highest latencies:
System rack7slot0.osadl.org (updated Tue Jun 06, 2023 12:44:53)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
110050660,0irq/25-eth00-21swapper/307:05:303
110050580,0irq/25-eth00-21swapper/107:07:481
441625227,0sleep00-21swapper/007:05:410
469224623,0sleep20-21swapper/207:09:122
5027991612,0cyclictest0-21swapper/311:10:093
5026991514,0cyclictest0-21swapper/212:11:302
502599152,0cyclictest8499-21sshd11:02:051
5024991513,0cyclictest4377-21sshd10:57:490
110050150,0irq/25-eth00-21swapper/310:01:503
5027991413,0cyclictest12139-21ls09:40:343
5024991411,0cyclictest0-21swapper/009:23:130
502499140,0cyclictest0-21swapper/009:12:330
5027991312,0cyclictest22941-21sshd12:29:363
5027991312,0cyclictest0-21swapper/311:47:263
5027991311,0cyclictest0-21swapper/311:51:573
502799130,0cyclictest0-21swapper/309:11:333
5026991313,0cyclictest10326-21sshd10:31:572
5026991312,0cyclictest19384-21sshd10:36:492
5026991312,0cyclictest10131-21sshd09:57:272
5026991312,0cyclictest0-21swapper/211:12:572
5026991312,0cyclictest0-21swapper/209:45:212
5026991312,0cyclictest0-21swapper/209:42:422
5026991312,0cyclictest0-21swapper/209:14:132
5026991311,0cyclictest21343-21sshd09:16:542
5026991311,0cyclictest0-21swapper/212:33:132
5025991311,0cyclictest0-21swapper/109:26:421
5024991312,0cyclictest0-21swapper/011:42:170
5024991312,0cyclictest0-21swapper/011:06:230
5024991312,0cyclictest0-21swapper/009:25:540
5024991310,0cyclictest0-21swapper/010:51:380
502499130,0cyclictest23174-21cp11:30:110
502499130,0cyclictest0-21swapper/011:56:480
9950120,0irq/24-0000:00:26-21rcuc/209:52:562
9950120,0irq/24-0000:00:0-21swapper/311:04:123
5027991211,0cyclictest11147-21bash12:38:473
5027991211,0cyclictest0-21swapper/310:15:463
502799120,0cyclictest30571-21sshd10:45:373
5026991211,0cyclictest5576-21cp11:49:192
5026991211,0cyclictest28251-21sshd11:44:462
5026991211,0cyclictest25793-21rm11:19:202
5026991211,0cyclictest0-21swapper/210:46:342
5026991210,0cyclictest11792-21sshd09:28:092
5025991212,0cyclictest0-21swapper/112:37:401
5025991212,0cyclictest0-21swapper/108:00:221
5025991211,0cyclictest0-21swapper/112:31:441
5025991211,0cyclictest0-21swapper/112:28:281
5025991211,0cyclictest0-21swapper/109:10:041
5025991210,0cyclictest10255-21bash11:28:461
5025991210,0cyclictest0-21swapper/110:25:131
5025991210,0cyclictest0-21swapper/109:43:531
502599120,0cyclictest0-21swapper/111:24:501
502599120,0cyclictest0-21swapper/110:15:501
502499129,0cyclictest8709-21sshd09:57:150
5024991211,0cyclictest964-21sshd11:27:450
5024991211,0cyclictest0-21swapper/011:18:550
5024991211,0cyclictest0-21swapper/010:12:570
5024991211,0cyclictest0-21swapper/009:42:300
5024991210,0cyclictest0-21swapper/010:28:450
5024991210,0cyclictest0-21swapper/010:00:160
5024991210,0cyclictest0-21swapper/009:31:150
41120,0ktimersoftd/00-21swapper/009:16:010
191120,0ktimersoftd/19745-21sshd12:10:361
191120,0ktimersoftd/10-21swapper/109:54:061
110050120,0irq/25-eth00-21swapper/312:10:533
110050120,0irq/25-eth00-21swapper/110:34:391
110050120,0irq/25-eth00-21swapper/109:22:591
502799119,0cyclictest0-21swapper/308:13:343
5027991111,0cyclictest18151-21sshd12:22:033
5027991111,0cyclictest0-21swapper/312:06:083
5027991111,0cyclictest0-21swapper/311:17:273
5027991111,0cyclictest0-21swapper/310:58:453
5027991110,0cyclictest8499-21sshd10:43:123
5027991110,0cyclictest8499-21sshd10:43:113
5027991110,0cyclictest0-21swapper/311:56:303
5027991110,0cyclictest0-21swapper/311:44:253
5027991110,0cyclictest0-21swapper/310:33:253
5027991110,0cyclictest0-21swapper/309:32:003
5027991110,0cyclictest0-21swapper/308:18:563
5027991110,0cyclictest0-21swapper/307:56:373
502799110,0cyclictest13110-21sshd09:20:083
502799110,0cyclictest0-21swapper/307:26:463
502799110,0cyclictest0-21swapper/307:26:453
5026991111,0cyclictest0-21swapper/210:20:192
5026991110,0cyclictest26235-21bash10:07:042
5026991110,0cyclictest13684-21bash10:28:392
502699110,0cyclictest0-21swapper/212:39:082
502699110,0cyclictest0-21swapper/212:24:032
5025991111,0cyclictest0-21swapper/112:08:501
5025991111,0cyclictest0-21swapper/110:09:131
5025991111,0cyclictest0-21swapper/109:59:331
5025991111,0cyclictest0-21swapper/107:11:491
5025991110,0cyclictest4586-21sshd12:17:061
5025991110,0cyclictest21207-21rm10:21:521
5025991110,0cyclictest18381-21sshd10:48:071
5025991110,0cyclictest0-21swapper/112:20:561
5025991110,0cyclictest0-21swapper/110:52:031
5025991110,0cyclictest0-21swapper/109:35:161
5025991110,0cyclictest0-21swapper/109:18:401
5025991110,0cyclictest0-21swapper/108:14:061
502599110,0cyclictest0-21swapper/110:36:511
502599110,0cyclictest0-21swapper/108:07:131
502599110,0cyclictest0-21swapper/107:52:011
502599110,0cyclictest0-21swapper/107:15:321
502499119,0cyclictest0-21swapper/012:35:150
5024991111,0cyclictest0-21swapper/010:19:330
5024991110,0cyclictest20469-21bash12:29:190
5024991110,0cyclictest13932-21bash12:14:400
5024991110,0cyclictest0-21swapper/012:01:200
5024991110,0cyclictest0-21swapper/011:39:150
5024991110,0cyclictest0-21swapper/011:21:220
5024991110,0cyclictest0-21swapper/011:11:370
5024991110,0cyclictest0-21swapper/007:12:260
502499110,0cyclictest0-21swapper/010:35:230
41110,0ktimersoftd/06366-21sshd11:49:240
41110,0ktimersoftd/00-21swapper/011:51:330
41110,0ktimersoftd/00-21swapper/008:35:010
351110,0ktimersoftd/30-21swapper/309:29:533
110050110,0irq/25-eth0271ktimersoftd/211:00:572
110050110,0irq/25-eth00-21swapper/009:45:470
9950100,0irq/24-0000:00:0-21swapper/309:36:303
502799109,0cyclictest0-21swapper/310:35:533
502799109,0cyclictest0-21swapper/310:09:103
502799109,0cyclictest0-21swapper/308:48:463
5027991010,0cyclictest110050irq/25-eth011:09:133
5027991010,0cyclictest0-21swapper/312:32:503
5027991010,0cyclictest0-21swapper/310:21:503
5027991010,0cyclictest0-21swapper/308:58:323
5027991010,0cyclictest0-21swapper/308:29:153
5027991010,0cyclictest0-21swapper/307:35:073
5027991010,0cyclictest0-21swapper/307:33:023
5027991010,0cyclictest0-21swapper/307:33:023
5027991010,0cyclictest0-21swapper/307:22:483
502699109,0cyclictest0-21swapper/211:37:132
502699109,0cyclictest0-21swapper/210:44:092
502699109,0cyclictest0-21swapper/210:44:092
5026991010,0cyclictest8923-21sshd12:07:052
5026991010,0cyclictest0-21swapper/211:25:442
5026991010,0cyclictest0-21swapper/210:55:252
5026991010,0cyclictest0-21swapper/209:34:052
502699100,0cyclictest0-21swapper/212:02:032
502599109,0cyclictest20-21ksoftirqd/111:50:171
502599109,0cyclictest17113-21sshd09:45:591
502599109,0cyclictest1453-21nfsd10:14:181
502599108,0cyclictest0-21swapper/107:23:331
5025991010,0cyclictest0-21swapper/111:11:061
5025991010,0cyclictest0-21swapper/107:47:501
5025991010,0cyclictest0-21swapper/107:35:531
502599100,0cyclictest0-21swapper/111:36:211
502599100,0cyclictest0-21swapper/108:25:221
502599100,0cyclictest0-21swapper/107:56:111
502499109,0cyclictest29052-21diskmemload12:18:310
5024991010,0cyclictest0-21swapper/009:38:580
502499100,0cyclictest0-21swapper/008:59:450
502499100,0cyclictest0-21swapper/007:41:560
41100,0ktimersoftd/05903-21sshd12:20:400
41100,0ktimersoftd/029052-21diskmemload12:33:330
41100,0ktimersoftd/029052-21diskmemload10:34:090
41100,0ktimersoftd/00-21swapper/007:20:050
351100,0ktimersoftd/30-21swapper/307:17:033
271100,0ktimersoftd/20-21swapper/211:51:342
191100,0ktimersoftd/10-21swapper/111:49:461
110050100,0irq/25-eth027118-21sshd11:00:250
110050100,0irq/25-eth00-21swapper/008:30:370
995090,0irq/24-0000:00:26031-21bash11:33:583
50279999,0cyclictest1391-21sshd11:27:483
50279999,0cyclictest0-21swapper/310:27:383
50279998,0cyclictest0-21swapper/311:35:213
50269999,0cyclictest110050irq/25-eth011:56:512
50269999,0cyclictest0-21swapper/212:27:182
50269999,0cyclictest0-21swapper/207:37:132
50269998,0cyclictest28341-21cp12:19:442
50269998,0cyclictest1453-21nfsd10:13:432
50269998,0cyclictest0-21swapper/207:25:292
50269998,0cyclictest0-21swapper/207:25:292
50269990,0cyclictest0-21swapper/210:17:162
50269990,0cyclictest0-21swapper/208:24:172
50259999,0cyclictest0-21swapper/111:18:271
50259999,0cyclictest0-21swapper/111:05:541
50259999,0cyclictest0-21swapper/108:37:461
50259999,0cyclictest0-21swapper/108:34:321
50259998,0cyclictest0-21swapper/107:28:061
50259998,0cyclictest0-21swapper/107:28:051
50259996,0cyclictest0-21swapper/110:40:481
50259996,0cyclictest0-21swapper/110:40:471
50259992,0cyclictest20-21ksoftirqd/111:55:121
50259990,0cyclictest19583-21bash10:59:401
50249999,0cyclictest0-21swapper/010:22:010
50249999,0cyclictest0-21swapper/008:42:130
50249999,0cyclictest0-21swapper/008:06:120
50249998,0cyclictest0-21swapper/008:04:060
4190,0ktimersoftd/05166-21cp12:06:390
4190,0ktimersoftd/029052-21diskmemload10:06:010
4190,0ktimersoftd/00-21swapper/010:48:420
11005090,0irq/25-eth029052-21diskmemload10:40:390
11005090,0irq/25-eth029052-21diskmemload10:40:380
11005090,0irq/25-eth015855-21munin-node09:50:300
995080,0irq/24-0000:00:22036-21munin-run08:55:000
50279988,0cyclictest0-21swapper/309:08:523
50279988,0cyclictest0-21swapper/308:22:453
50279988,0cyclictest0-21swapper/308:08:483
50279988,0cyclictest0-21swapper/308:01:223
50279987,0cyclictest36-21ksoftirqd/310:50:183
50279987,0cyclictest0-21swapper/308:39:363
50269988,0cyclictest24575-21sshd11:07:422
50269988,0cyclictest0-21swapper/209:08:522
50269987,0cyclictest0-21swapper/208:44:302
50269984,0cyclictest0-21swapper/208:10:272
50259988,0cyclictest0-21swapper/108:46:231
50259985,0cyclictest0-21swapper/107:44:471
50259983,0cyclictest0-21swapper/109:32:031
50259982,0cyclictest0-21swapper/108:59:191
50249988,0cyclictest0-21swapper/009:05:060
50249988,0cyclictest0-21swapper/008:10:230
50249988,0cyclictest0-21swapper/007:58:260
50249988,0cyclictest0-21swapper/007:26:010
50249988,0cyclictest0-21swapper/007:26:000
50249980,0cyclictest0-21swapper/009:00:290
149980,0migration/09169-21nfsd407:15:290
149980,0migration/018767-21nfsd408:45:230
11005080,0irq/25-eth09787-21sshd12:00:153
11005080,0irq/25-eth09701-21sshd09:45:093
11005080,0irq/25-eth09082-21sshd09:57:183
11005080,0irq/25-eth029052-21diskmemload12:17:073
11005080,0irq/25-eth025797-21uname11:30:242
11005080,0irq/25-eth022679-21sshd09:50:593
11005080,0irq/25-eth018432-21sshd12:01:041
11005080,0irq/25-eth018210-21sshd10:10:033
11005080,0irq/25-eth016817-21cp11:40:011
11005080,0irq/25-eth015622-21sshd11:21:523
11005080,0irq/25-eth01512-21sshd10:51:302
11005080,0irq/25-eth011814-21sshd11:32:241
11005080,0irq/25-eth00-21swapper/309:16:493
995070,0irq/24-0000:00:8045-21latency_hist08:24:590
50279977,0cyclictest0-21swapper/308:52:023
50279977,0cyclictest0-21swapper/308:42:503
50279977,0cyclictest0-21swapper/307:54:313
50269977,0cyclictest0-21swapper/208:49:152
50269977,0cyclictest0-21swapper/208:39:432
50269977,0cyclictest0-21swapper/207:45:572
50269977,0cyclictest0-21swapper/207:40:492
50269977,0cyclictest0-21swapper/207:32:072
50269977,0cyclictest0-21swapper/207:32:072
50269977,0cyclictest0-21swapper/207:10:372
50269976,0cyclictest3609-21irqstats11:20:272
50269970,0cyclictest0-21swapper/208:50:182
50269970,0cyclictest0-21swapper/207:20:262
50259977,0cyclictest0-21swapper/109:04:091
50259976,0cyclictest20-21ksoftirqd/110:00:201
50259975,0cyclictest0-21swapper/108:42:191
50259970,0cyclictest0-21swapper/109:06:101
50249976,0cyclictest0-21swapper/007:36:490
50249970,0cyclictest22203-21ls07:45:150
11005070,0irq/25-eth02905-21bash09:35:192
11005070,0irq/25-eth018911-21sshd09:20:412
11005070,0irq/25-eth00-21swapper/210:02:392
50279964,0cyclictest10647-21grep08:30:213
50279962,0cyclictest0-21swapper/309:00:233
50279962,0cyclictest0-21swapper/307:40:353
50279961,0cyclictest0-21swapper/307:45:233
50269966,0cyclictest0-21swapper/208:15:192
50269962,0cyclictest0-21swapper/209:00:242
50269962,0cyclictest0-21swapper/208:55:392
50269962,0cyclictest0-21swapper/208:30:342
50269962,0cyclictest0-21swapper/208:05:342
50269962,0cyclictest0-21swapper/208:00:202
50269962,0cyclictest0-21swapper/207:55:172
50269962,0cyclictest0-21swapper/207:50:272
50269962,0cyclictest0-21swapper/207:15:012
50259962,0cyclictest0-21swapper/108:50:241
50259962,0cyclictest0-21swapper/108:15:201
50259962,0cyclictest0-21swapper/107:30:251
50259962,0cyclictest0-21swapper/107:30:251
50259961,0cyclictest0-21swapper/108:20:271
50249965,0cyclictest0-21swapper/007:50:250
50249962,0cyclictest0-21swapper/008:25:280
50249962,0cyclictest0-21swapper/008:15:210
50249962,0cyclictest0-21swapper/007:30:250
50249962,0cyclictest0-21swapper/007:30:250
50269951,0cyclictest0-21swapper/208:25:282
11795050,0irq/26-eth1-rx-30864-21kworker/3:107:10:183
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional