You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-03-21 - 05:35

x86 Intel Xeon E3-1220L V2 @2300 MHz, Linux 4.4.39-rt50+ (Profile)

Latency plot of system in rack #7, slot #0
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack7slot0.osadl.org (updated Tue Mar 21, 2023 00:44:58)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
110050620,0irq/25-eth00-21swapper/319:06:253
117950600,0irq/26-eth1-rx-0-21swapper/119:06:251
110050550,0irq/25-eth00-21swapper/219:06:182
2717124622,0sleep00-21swapper/019:09:030
9950260,0irq/24-0000:00:0-21swapper/119:10:011
27510991613,0cyclictest0-21swapper/321:41:113
27507991613,0cyclictest0-21swapper/021:30:350
27510991514,0cyclictest12653-21sshd23:10:583
27510991514,0cyclictest12653-21sshd23:10:583
27510991514,0cyclictest0-21swapper/321:45:283
2751099151,0cyclictest0-21swapper/323:50:183
27508991514,0cyclictest0-21swapper/123:29:171
110050150,0irq/25-eth013283-21sshd21:25:163
27510991413,0cyclictest0-21swapper/322:57:373
27510991413,0cyclictest0-21swapper/322:41:283
27510991411,0cyclictest5093-21sshd00:36:213
2751099141,0cyclictest1807-21sshd23:48:573
27509991412,0cyclictest0-21swapper/200:37:012
27508991413,0cyclictest0-21swapper/123:34:161
27508991413,0cyclictest0-21swapper/123:34:161
27508991411,0cyclictest0-21swapper/122:50:081
27507991412,0cyclictest0-21swapper/000:04:090
27507991411,0cyclictest0-21swapper/023:02:490
27510991313,0cyclictest351ktimersoftd/322:23:353
27510991312,0cyclictest2654-21rm00:26:283
27510991312,0cyclictest26231-21crond21:55:003
27510991312,0cyclictest0-21swapper/322:05:273
27510991311,0cyclictest0-21swapper/321:33:313
27510991310,0cyclictest0-21swapper/321:36:533
27509991312,0cyclictest0-21swapper/223:42:252
27509991312,0cyclictest0-21swapper/222:56:392
2750999130,0cyclictest0-21swapper/223:33:222
2750999130,0cyclictest0-21swapper/223:33:222
2750899132,0cyclictest2885-21munin-run21:19:591
27508991312,0cyclictest5792-21sshd23:53:541
27508991312,0cyclictest0-21swapper/121:11:101
27508991312,0cyclictest0-21swapper/100:10:031
27508991311,0cyclictest0-21swapper/123:14:041
27508991311,0cyclictest0-21swapper/123:14:041
110050130,0irq/25-eth00-21swapper/321:22:143
110050130,0irq/25-eth00-21swapper/222:01:082
2751099129,0cyclictest0-21swapper/323:15:373
27510991211,0cyclictest0-21swapper/319:31:463
27510991210,0cyclictest0-21swapper/323:23:033
2751099120,0cyclictest0-21swapper/322:27:523
2751099120,0cyclictest0-21swapper/322:10:413
2751099120,0cyclictest0-21swapper/300:01:233
27509991211,0cyclictest0-21swapper/221:28:492
27509991211,0cyclictest0-21swapper/200:20:382
27509991210,0cyclictest25056-21sshd22:06:562
27509991210,0cyclictest0-21swapper/223:51:082
27509991210,0cyclictest0-21swapper/222:44:132
27509991210,0cyclictest0-21swapper/222:26:552
27509991210,0cyclictest0-21swapper/200:12:162
2750999120,0cyclictest0-21swapper/223:26:092
2750999120,0cyclictest0-21swapper/223:24:182
2750999120,0cyclictest0-21swapper/222:22:372
2750999120,0cyclictest0-21swapper/221:21:102
2750999120,0cyclictest0-21swapper/219:35:282
27508991211,0cyclictest0-21swapper/121:51:001
27508991210,0cyclictest0-21swapper/120:04:561
2750899120,0cyclictest20324-21sshd22:19:351
2750899120,0cyclictest0-21swapper/122:57:551
2750899120,0cyclictest0-21swapper/122:46:241
2750899120,0cyclictest0-21swapper/122:44:331
2750899120,0cyclictest0-21swapper/122:27:151
2750899120,0cyclictest0-21swapper/121:33:481
2750899120,0cyclictest0-21swapper/100:06:201
27507991211,0cyclictest24238-21munin-node00:20:170
27507991211,0cyclictest23571-21rm22:55:020
27507991211,0cyclictest0-21swapper/022:44:270
27507991211,0cyclictest0-21swapper/000:17:300
27507991210,0cyclictest0-21swapper/023:43:560
2750799120,0cyclictest19119-21diskmemload00:34:490
2750799120,0cyclictest0-21swapper/023:52:400
2750799120,0cyclictest0-21swapper/022:52:460
2750799120,0cyclictest0-21swapper/022:26:130
2750799120,0cyclictest0-21swapper/021:17:060
271120,0ktimersoftd/20-21swapper/221:40:272
110050120,0irq/25-eth00-21swapper/021:48:420
9950110,0irq/24-0000:00:19119-21diskmemload21:57:101
41110,0ktimersoftd/016625-21bash23:15:460
41110,0ktimersoftd/010603-21sshd21:52:560
2751099119,0cyclictest0-21swapper/320:56:263
2751099118,0cyclictest0-21swapper/321:59:183
27510991111,0cyclictest0-21swapper/322:31:153
27510991111,0cyclictest0-21swapper/322:19:173
27510991110,0cyclictest0-21swapper/321:13:183
27510991110,0cyclictest0-21swapper/319:54:593
27509991111,0cyclictest0-21swapper/221:51:152
27509991111,0cyclictest0-21swapper/221:46:572
27509991110,0cyclictest32183-21sshd21:55:332
27509991110,0cyclictest17093-21sshd22:14:432
27509991110,0cyclictest0-21swapper/222:45:092
27509991110,0cyclictest0-21swapper/220:50:002
27509991110,0cyclictest0-21swapper/219:45:132
27509991110,0cyclictest0-21swapper/200:15:592
2750999110,0cyclictest0-21swapper/221:11:582
27508991111,0cyclictest0-21swapper/123:43:201
27508991110,0cyclictest13983-21sshd00:32:561
27508991110,0cyclictest0-21swapper/123:15:551
27508991110,0cyclictest0-21swapper/122:22:571
27508991110,0cyclictest0-21swapper/121:45:471
2750899110,0cyclictest0-21swapper/122:10:591
2750899110,0cyclictest0-21swapper/122:00:321
2750899110,0cyclictest0-21swapper/120:59:331
2750899110,0cyclictest0-21swapper/120:12:151
27507991111,0cyclictest0-21swapper/023:35:480
27507991111,0cyclictest0-21swapper/022:15:460
27507991110,0cyclictest6387-21sshd00:27:030
27507991110,0cyclictest1453-21nfsd23:14:190
27507991110,0cyclictest1453-21nfsd23:14:190
27507991110,0cyclictest0-21swapper/022:45:230
27507991110,0cyclictest0-21swapper/022:20:590
27507991110,0cyclictest0-21swapper/022:10:320
27507991110,0cyclictest0-21swapper/021:58:130
27507991110,0cyclictest0-21swapper/021:43:280
27507991110,0cyclictest0-21swapper/021:25:210
27507991110,0cyclictest0-21swapper/000:05:040
271110,0ktimersoftd/20-21swapper/222:50:232
191110,0ktimersoftd/132709-21sshd21:43:341
110050110,0irq/25-eth07081-21sshd21:12:250
2751099109,0cyclictest0-21swapper/323:29:543
2751099109,0cyclictest0-21swapper/322:36:063
2751099109,0cyclictest0-21swapper/321:02:163
27510991010,0cyclictest0-21swapper/323:42:063
27510991010,0cyclictest0-21swapper/323:33:583
27510991010,0cyclictest0-21swapper/323:33:583
2751099100,0cyclictest0-21swapper/320:48:123
2751099100,0cyclictest0-21swapper/319:40:213
2751099100,0cyclictest0-21swapper/319:14:423
2750999109,0cyclictest9843-21sshd23:45:302
2750999109,0cyclictest1415-21sshd21:15:352
2750999109,0cyclictest0-21swapper/221:36:302
2750999109,0cyclictest0-21swapper/200:01:422
2750999100,0cyclictest0-21swapper/222:32:232
2750899109,0cyclictest23044-21sshd00:39:021
2750899109,0cyclictest19119-21diskmemload23:38:201
2750899109,0cyclictest19119-21diskmemload23:20:331
2750899109,0cyclictest15078-21cp21:29:311
2750899109,0cyclictest0-21swapper/122:30:371
2750899109,0cyclictest0-21swapper/120:41:331
2750899109,0cyclictest0-21swapper/119:47:101
2750799109,0cyclictest19660-21sshd23:07:330
2750799109,0cyclictest0-21swapper/021:38:150
27507991010,0cyclictest0-21swapper/020:20:110
27507991010,0cyclictest0-21swapper/020:20:110
27507991010,0cyclictest0-21swapper/020:07:170
2750799100,0cyclictest0-21swapper/020:58:370
2750799100,0cyclictest0-21swapper/000:36:240
110050100,0irq/25-eth019119-21diskmemload23:58:090
4190,0ktimersoftd/08930-21sshd00:12:570
4190,0ktimersoftd/00-21swapper/023:22:460
4190,0ktimersoftd/00-21swapper/022:31:300
4190,0ktimersoftd/00-21swapper/021:20:510
275109999,0cyclictest0-21swapper/321:19:063
275109999,0cyclictest0-21swapper/321:05:383
275109999,0cyclictest0-21swapper/320:41:273
275109999,0cyclictest0-21swapper/320:38:063
275109999,0cyclictest0-21swapper/320:25:323
275109999,0cyclictest0-21swapper/320:06:373
275109999,0cyclictest0-21swapper/319:23:033
275109998,0cyclictest7905-21sshd22:00:363
275109998,0cyclictest30872-21sshd23:35:113
275109998,0cyclictest26837-21sshd00:05:543
275109998,0cyclictest23349-21sshd23:08:053
275109998,0cyclictest22679-21sshd00:10:153
275109998,0cyclictest1453-21nfsd23:00:153
275109998,0cyclictest0-21swapper/323:58:363
275109990,0cyclictest0-21swapper/320:23:073
275109990,0cyclictest0-21swapper/320:23:073
275109990,0cyclictest0-21swapper/320:12:263
275109990,0cyclictest0-21swapper/300:15:113
275099999,0cyclictest0-21swapper/223:38:212
275099998,0cyclictest30943-21sshd22:16:272
275099998,0cyclictest28766-21sshd22:37:562
275099998,0cyclictest16645-21sshd00:28:412
275099998,0cyclictest0-21swapper/219:25:202
275099990,0cyclictest0-21swapper/223:18:392
275099990,0cyclictest0-21swapper/223:06:132
275089999,0cyclictest0-21swapper/120:30:511
275089999,0cyclictest0-21swapper/100:22:081
275089998,0cyclictest0-21swapper/123:55:461
275089990,0cyclictest0-21swapper/122:37:071
275089990,0cyclictest0-21swapper/121:21:431
275079999,0cyclictest0-21swapper/020:41:040
275079999,0cyclictest0-21swapper/020:02:040
275079999,0cyclictest0-21swapper/019:38:280
275079998,0cyclictest0-21swapper/019:45:190
275079990,0cyclictest0-21swapper/019:21:160
275079990,0cyclictest0-21swapper/019:13:290
11005090,0irq/25-eth026611-21sshd23:25:440
11005090,0irq/25-eth010392-21ntp_states22:35:280
11005090,0irq/25-eth00-21swapper/223:11:052
11005090,0irq/25-eth00-21swapper/223:11:052
11005090,0irq/25-eth00-21swapper/122:09:491
995080,0irq/24-0000:00:32761-21sshd21:35:281
275109988,0cyclictest0-21swapper/320:50:393
275109988,0cyclictest0-21swapper/320:34:443
275109988,0cyclictest0-21swapper/320:00:493
275109988,0cyclictest0-21swapper/319:35:283
275109987,0cyclictest31412-21sshd00:21:063
275109987,0cyclictest0-21swapper/322:46:073
275109980,0cyclictest0-21swapper/319:45:153
275099987,0cyclictest5438-21sshd21:32:122
275099987,0cyclictest31186-21sshd00:06:332
275099987,0cyclictest11970-21sshd23:59:062
275089988,0cyclictest0-21swapper/119:54:161
275089987,0cyclictest9451-21sshd23:45:281
275089987,0cyclictest2306-21sshd23:00:421
275079988,0cyclictest0-21swapper/021:04:470
275079988,0cyclictest0-21swapper/019:44:240
275079987,0cyclictest28194-21sshd23:30:230
275079987,0cyclictest28194-21sshd23:30:230
275079987,0cyclictest0-21swapper/022:03:270
275079987,0cyclictest0-21swapper/020:11:350
275079980,0cyclictest0-21swapper/020:15:230
11795080,0irq/26-eth1-rx-3765-21fw_forwarded_lo20:35:181
11005080,0irq/25-eth08987-21bash23:06:021
11005080,0irq/25-eth06009-21bash23:45:090
11005080,0irq/25-eth019119-21diskmemload00:26:131
11005080,0irq/25-eth00-21swapper/100:00:301
4170,0ktimersoftd/011549-21cpuspeed_turbos22:05:180
275109977,0cyclictest0-21swapper/320:17:183
275109971,0cyclictest0-21swapper/322:53:533
275099977,0cyclictest0-21swapper/220:10:122
275099976,0cyclictest10257-21sshd23:01:512
275089977,0cyclictest0-21swapper/120:28:251
275089977,0cyclictest0-21swapper/119:20:141
275079977,0cyclictest0-21swapper/020:46:190
275079977,0cyclictest0-21swapper/019:53:070
275079977,0cyclictest0-21swapper/019:15:220
275079970,0cyclictest0-21swapper/019:25:170
11005070,0irq/25-eth031102-21sshd00:30:372
11005070,0irq/25-eth022869-21sshd00:15:071
4160,0ktimersoftd/00-21swapper/019:55:130
275109965,0cyclictest26175-21sshd00:30:063
275109962,0cyclictest0-21swapper/319:55:183
275109962,0cyclictest0-21swapper/319:25:193
275109962,0cyclictest0-21swapper/319:15:013
275099965,0cyclictest5131-21atenall_r7power19:30:132
275099965,0cyclictest27890-21expr19:10:142
275099965,0cyclictest10017-21aten_r7power_fr20:50:142
275099962,0cyclictest0-21swapper/220:55:192
275099962,0cyclictest0-21swapper/220:35:212
275099962,0cyclictest0-21swapper/220:30:172
275099962,0cyclictest0-21swapper/220:25:192
275099962,0cyclictest0-21swapper/219:55:172
275099962,0cyclictest0-21swapper/219:40:172
275089966,0cyclictest0-21swapper/120:51:171
275089965,0cyclictest3603-21if_eth019:25:191
275089965,0cyclictest30491-21grep19:15:011
275089962,0cyclictest0-21swapper/121:05:211
275089962,0cyclictest0-21swapper/121:00:181
275089962,0cyclictest0-21swapper/120:45:191
275089962,0cyclictest0-21swapper/120:20:201
275089962,0cyclictest0-21swapper/120:20:191
275089962,0cyclictest0-21swapper/120:15:191
275089962,0cyclictest0-21swapper/119:35:231
275089962,0cyclictest0-21swapper/119:30:161
275089961,0cyclictest0-21swapper/120:07:131
275089961,0cyclictest0-21swapper/119:42:531
275079965,0cyclictest5237-21cpu19:30:130
275079965,0cyclictest17014-21munin-node21:05:210
275079962,0cyclictest0-21swapper/020:50:190
275079962,0cyclictest0-21swapper/020:35:210
275079962,0cyclictest0-21swapper/020:30:180
275079960,0cyclictest0-21swapper/020:25:200
259960,0migration/217772-21munin-node21:05:212
11005060,0irq/25-eth019523-21diskstats20:00:162
275099951,0cyclictest0-21swapper/219:50:202
275099950,0cyclictest0-21swapper/221:00:072
275099950,0cyclictest0-21swapper/220:40:132
275099950,0cyclictest0-21swapper/220:20:222
275099950,0cyclictest0-21swapper/220:20:212
275099950,0cyclictest0-21swapper/220:15:102
275099950,0cyclictest0-21swapper/220:05:052
275099950,0cyclictest0-21swapper/219:20:052
275099950,0cyclictest0-21swapper/219:15:012
275089950,0cyclictest0-21swapper/119:55:081
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional