You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-05-21 - 03:57

x86 Intel Xeon E3-1220L V2 @2300 MHz, Linux 4.4.39-rt50+ (Profile)

Latency plot of system in rack #7, slot #0
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the alll highest latencies:
System rack7slot0.osadl.org (updated Fri May 20, 2022 00:44:58)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
110050630,0irq/25-eth00-21swapper/119:06:101
110050630,0irq/25-eth00-21swapper/119:06:101
117950600,0irq/26-eth1-rx-0-21swapper/319:05:553
117950600,0irq/26-eth1-rx-0-21swapper/319:05:543
110050590,0irq/25-eth00-21swapper/219:08:522
110050590,0irq/25-eth00-21swapper/219:08:512
110050450,0irq/25-eth00-21swapper/019:07:460
110050450,0irq/25-eth00-21swapper/019:07:450
2625399221,0cyclictest32170-21sshd00:22:530
2625699180,0cyclictest29319-21sshd21:28:193
26256991615,0cyclictest17875-21diskmemload22:28:013
191160,0ktimersoftd/123384-21sshd22:15:541
26256991514,0cyclictest0-21swapper/322:52:043
26256991512,0cyclictest15420-21sshd21:33:473
26256991413,0cyclictest0-21swapper/323:32:523
26256991413,0cyclictest0-21swapper/322:39:523
26256991413,0cyclictest0-21swapper/322:20:213
2625399140,0cyclictest0-21swapper/022:35:260
41130,0ktimersoftd/00-21swapper/022:49:430
271130,0ktimersoftd/20-21swapper/219:55:162
26256991312,0cyclictest0-21swapper/323:35:173
26256991312,0cyclictest0-21swapper/322:40:553
26256991312,0cyclictest0-21swapper/300:22:343
2625699130,0cyclictest0-21swapper/323:29:283
26255991312,0cyclictest26641-21sshd21:55:302
26255991312,0cyclictest0-21swapper/223:52:022
26255991312,0cyclictest0-21swapper/223:12:522
26255991312,0cyclictest0-21swapper/222:34:532
26255991312,0cyclictest0-21swapper/222:16:462
26255991312,0cyclictest0-21swapper/221:50:172
26255991312,0cyclictest0-21swapper/221:45:312
26255991311,0cyclictest0-21swapper/222:29:192
26255991311,0cyclictest0-21swapper/200:06:122
2625599130,0cyclictest0-21swapper/200:34:462
2625599130,0cyclictest0-21swapper/200:29:462
26254991312,0cyclictest18860-21sshd22:01:371
26254991312,0cyclictest0-21swapper/123:57:471
26254991311,0cyclictest715-21sshd22:58:321
26254991311,0cyclictest0-21swapper/121:46:581
26253991312,0cyclictest0-21swapper/023:32:570
26253991312,0cyclictest0-21swapper/022:58:400
26253991312,0cyclictest0-21swapper/022:30:190
26253991311,0cyclictest0-21swapper/022:00:140
110050130,0irq/25-eth026074-21sshd22:50:451
110050130,0irq/25-eth00-21swapper/322:03:453
110050130,0irq/25-eth00-21swapper/300:28:013
26256991212,0cyclictest0-21swapper/322:06:393
26256991212,0cyclictest0-21swapper/322:06:383
26256991211,0cyclictest9694-21sshd22:59:303
26256991211,0cyclictest9053-21id23:44:143
26256991211,0cyclictest0-21swapper/300:37:333
26256991211,0cyclictest0-21swapper/300:12:133
2625699120,0cyclictest16658-21bash23:58:593
2625699120,0cyclictest0-21swapper/321:15:133
2625699120,0cyclictest0-21swapper/319:10:123
2625699120,0cyclictest0-21swapper/319:10:113
2625699120,0cyclictest0-21swapper/300:31:593
2625699120,0cyclictest0-21swapper/300:16:043
26255991211,0cyclictest22401-21sshd23:56:012
26255991211,0cyclictest0-21swapper/223:47:502
26255991211,0cyclictest0-21swapper/200:39:472
26255991211,0cyclictest0-21swapper/200:21:322
26255991211,0cyclictest0-21swapper/200:11:112
26255991210,0cyclictest0-21swapper/222:41:372
2625599120,0cyclictest0-21swapper/223:35:252
2625599120,0cyclictest0-21swapper/223:04:372
2625599120,0cyclictest0-21swapper/221:17:312
2625599120,0cyclictest0-21swapper/220:05:162
26254991212,0cyclictest0-21swapper/123:47:401
26254991212,0cyclictest0-21swapper/122:05:201
26254991212,0cyclictest0-21swapper/122:05:201
26254991211,0cyclictest7739-21sshd23:37:081
26254991211,0cyclictest32044-21sshd21:14:541
26254991211,0cyclictest0-21swapper/100:24:011
26254991210,0cyclictest17875-21diskmemload00:37:511
2625499120,0cyclictest8686-21sshd23:44:121
2625499120,0cyclictest0-21swapper/123:33:231
2625499120,0cyclictest0-21swapper/123:08:031
2625499120,0cyclictest0-21swapper/100:01:011
26253991211,0cyclictest28189-21sshd22:13:010
26253991211,0cyclictest0-21swapper/023:43:090
26253991211,0cyclictest0-21swapper/023:37:280
26253991211,0cyclictest0-21swapper/000:10:550
26253991211,0cyclictest0-21swapper/000:09:450
2625399120,0cyclictest18660-21sshd23:55:350
191120,0ktimersoftd/10-21swapper/123:26:591
110050120,0irq/25-eth01694-21sshd21:49:350
41110,0ktimersoftd/06805-21sshd22:41:490
41110,0ktimersoftd/032077-21bash23:22:300
41110,0ktimersoftd/022924-21sshd21:13:560
2625699119,0cyclictest0-21swapper/320:07:063
26256991111,0cyclictest0-21swapper/300:02:503
26256991110,0cyclictest9758-21sshd23:54:463
26256991110,0cyclictest4141-21id21:11:533
26256991110,0cyclictest28608-21sshd21:48:593
26256991110,0cyclictest28027-21sshd22:30:183
26256991110,0cyclictest26655-21sshd21:38:253
26256991110,0cyclictest21082-21bash21:20:273
26256991110,0cyclictest18306-21sshd23:03:513
26256991110,0cyclictest16392-21sshd22:11:413
26256991110,0cyclictest0-21swapper/323:18:263
26256991110,0cyclictest0-21swapper/322:18:093
2625699110,0cyclictest0-21swapper/320:35:213
2625699110,0cyclictest0-21swapper/319:34:293
26255991111,0cyclictest0-21swapper/223:44:212
26255991111,0cyclictest0-21swapper/223:25:252
26255991110,0cyclictest610-21sshd23:05:192
26255991110,0cyclictest22755-21sshd21:10:222
26255991110,0cyclictest15008-21sshd21:23:212
26255991110,0cyclictest1486-21sshd00:19:262
26255991110,0cyclictest0-21swapper/223:30:532
26255991110,0cyclictest0-21swapper/222:59:382
26255991110,0cyclictest0-21swapper/222:37:202
26255991110,0cyclictest0-21swapper/200:02:212
26254991111,0cyclictest17875-21diskmemload21:55:131
26254991110,0cyclictest8922-21sshd21:29:381
26254991110,0cyclictest7796-21sshd23:54:311
26254991110,0cyclictest31567-21sshd22:37:341
26254991110,0cyclictest0-21swapper/122:48:461
26254991110,0cyclictest0-21swapper/121:54:381
26254991110,0cyclictest0-21swapper/119:11:261
26254991110,0cyclictest0-21swapper/119:11:251
2625499110,0cyclictest0-21swapper/120:46:481
2625499110,0cyclictest0-21swapper/120:19:231
2625499110,0cyclictest0-21swapper/119:36:311
26253991111,0cyclictest0-21swapper/023:46:050
26253991111,0cyclictest0-21swapper/021:51:310
26253991111,0cyclictest0-21swapper/000:27:460
26253991110,0cyclictest7260-21bash21:36:170
26253991110,0cyclictest28235-21bash21:24:480
26253991110,0cyclictest2334-21consoletype22:17:100
26253991110,0cyclictest0-21swapper/023:03:400
26253991110,0cyclictest0-21swapper/022:22:390
26253991110,0cyclictest0-21swapper/021:30:230
2625399110,0cyclictest10673-21bash00:31:370
2625399110,0cyclictest0-21swapper/019:34:550
191110,0ktimersoftd/10-21swapper/121:00:101
110050110,0irq/25-eth020731-21bash23:21:163
110050110,0irq/25-eth014827-21bash22:08:070
110050110,0irq/25-eth014827-21bash22:08:070
110050110,0irq/25-eth00-21swapper/322:48:283
110050110,0irq/25-eth00-21swapper/023:07:050
110050110,0irq/25-eth00-21swapper/021:18:340
41100,0ktimersoftd/00-21swapper/022:26:320
41100,0ktimersoftd/00-21swapper/000:37:210
41100,0ktimersoftd/00-21swapper/000:19:500
271100,0ktimersoftd/20-21swapper/219:34:122
2625699109,0cyclictest21788-21sshd23:07:383
2625699109,0cyclictest1149-21sshd23:12:173
2625699109,0cyclictest10174-21bash00:05:273
26256991010,0cyclictest0-21swapper/323:47:443
26256991010,0cyclictest0-21swapper/319:54:333
2625599109,0cyclictest904-21sshd21:42:342
2625599109,0cyclictest4643-21sshd23:16:052
2625599109,0cyclictest26890-21sshd21:31:272
2625599109,0cyclictest23911-21bash22:47:072
2625599109,0cyclictest18321-21sshd22:50:022
2625599109,0cyclictest17794-21sshd22:05:022
2625599109,0cyclictest17794-21sshd22:05:022
2625599109,0cyclictest1479-21sshd23:22:422
2625599109,0cyclictest0-21swapper/221:28:402
2625599108,0cyclictest0-21swapper/219:40:592
2625599100,0cyclictest0-21swapper/220:14:202
2625499109,0cyclictest9588-21sshd00:09:161
2625499109,0cyclictest26478-21sshd00:33:261
2625499109,0cyclictest24272-21sshd23:21:391
2625499109,0cyclictest11731-21sshd23:10:031
2625499101,0cyclictest0-21swapper/121:20:141
26254991010,0cyclictest0-21swapper/120:51:481
26254991010,0cyclictest0-21swapper/120:30:531
26254991010,0cyclictest0-21swapper/120:05:061
26254991010,0cyclictest0-21swapper/119:44:321
26254991010,0cyclictest0-21swapper/100:28:271
2625499100,0cyclictest28631-21bash22:20:021
2625499100,0cyclictest22871-21sshd21:17:171
2625499100,0cyclictest0-21swapper/122:30:421
2625499100,0cyclictest0-21swapper/122:26:151
2625399109,0cyclictest23199-21bash21:41:270
2625399109,0cyclictest22750-21sshd23:28:260
2625399109,0cyclictest0-21swapper/019:46:250
2625399109,0cyclictest0-21swapper/000:02:260
2625399108,0cyclictest0-21swapper/021:25:120
2625399100,0cyclictest0-21swapper/020:24:310
2625399100,0cyclictest0-21swapper/020:16:370
110050100,0irq/25-eth08605-21fw_packets21:50:143
110050100,0irq/25-eth024462-21sshd23:52:470
110050100,0irq/25-eth017658-21sshd21:30:251
110050100,0irq/25-eth017523-21sshd00:13:491
4190,0ktimersoftd/00-21swapper/023:17:170
27190,0ktimersoftd/20-21swapper/219:38:522
262569999,0cyclictest0-21swapper/319:39:203
262569998,0cyclictest0-21swapper/321:40:103
262569998,0cyclictest0-21swapper/319:26:073
262559999,0cyclictest0-21swapper/220:21:252
262559998,0cyclictest7313-21sshd22:10:402
262559998,0cyclictest29468-21bash21:38:442
262559998,0cyclictest24867-21bash22:02:162
262549999,0cyclictest0-21swapper/120:41:481
262549999,0cyclictest0-21swapper/120:27:311
262549999,0cyclictest0-21swapper/120:24:091
262549999,0cyclictest0-21swapper/119:28:301
262549998,0cyclictest6342-21id22:10:331
262549998,0cyclictest24624-21turbostat20:15:001
262549998,0cyclictest22546-21sshd00:18:061
262549998,0cyclictest19590-21id21:41:021
262539999,0cyclictest0-21swapper/019:52:270
262539999,0cyclictest0-21swapper/019:27:070
262539990,0cyclictest0-21swapper/022:50:140
262539990,0cyclictest0-21swapper/021:04:430
262539990,0cyclictest0-21swapper/020:59:360
262539990,0cyclictest0-21swapper/020:05:240
262539990,0cyclictest0-21swapper/020:03:580
262539990,0cyclictest0-21swapper/019:59:050
19190,0ktimersoftd/10-21swapper/119:34:111
19190,0ktimersoftd/10-21swapper/119:21:311
11795090,0irq/26-eth1-rx-0-21swapper/121:37:311
11005090,0irq/25-eth00-21swapper/023:10:120
11005090,0irq/25-eth00-21swapper/021:55:130
262569988,0cyclictest0-21swapper/321:03:473
262569988,0cyclictest0-21swapper/320:44:453
262569988,0cyclictest0-21swapper/320:03:453
262569988,0cyclictest0-21swapper/319:59:273
262569987,0cyclictest28536-21bash21:55:433
262569987,0cyclictest1249-21munin-plugin-st19:25:013
262569980,0cyclictest0-21swapper/320:46:093
262559988,0cyclictest0-21swapper/219:10:402
262559988,0cyclictest0-21swapper/219:10:402
262559987,0cyclictest0-21swapper/221:01:502
262549988,0cyclictest0-21swapper/119:56:161
262549988,0cyclictest0-21swapper/119:48:141
262539988,0cyclictest0-21swapper/020:32:250
262539988,0cyclictest0-21swapper/020:14:390
262539985,0cyclictest0-21swapper/019:20:170
262539980,0cyclictest0-21swapper/019:40:200
11005080,0irq/25-eth025907-21sshd23:15:001
11005080,0irq/25-eth020-21ksoftirqd/122:42:261
11005080,0irq/25-eth015752-21sshd23:00:091
11005080,0irq/25-eth00-21swapper/222:21:042
4170,0ktimersoftd/00-21swapper/020:40:120
262569977,0cyclictest0-21swapper/321:08:203
262569977,0cyclictest0-21swapper/320:19:123
262569977,0cyclictest0-21swapper/320:13:513
262569976,0cyclictest0-21swapper/320:56:433
262569972,0cyclictest0-21swapper/320:25:163
262569970,0cyclictest0-21swapper/319:15:023
262549976,0cyclictest0-21swapper/121:05:161
262549972,0cyclictest0-21swapper/119:17:171
262549970,0cyclictest0-21swapper/119:54:311
262539977,0cyclictest0-21swapper/020:36:220
262539974,0cyclictest0-21swapper/019:15:020
262539970,0cyclictest0-21swapper/019:10:200
262539970,0cyclictest0-21swapper/019:10:190
149970,0migration/018042-21latency_hist21:10:000
11005070,0irq/25-eth00-21swapper/020:27:440
995060,0irq/24-0000:00:0-21swapper/019:39:550
339960,0migration/332071-21cpuspeed_turbos20:30:123
262569962,0cyclictest0-21swapper/320:20:153
262569962,0cyclictest0-21swapper/319:45:183
262569962,0cyclictest0-21swapper/319:40:153
262559965,0cyclictest32680-21cat20:30:162
262559965,0cyclictest13921-21fw_forwarded_lo19:50:152
262559965,0cyclictest0-21swapper/220:18:522
262559962,0cyclictest2378-21gawk19:25:152
262559962,0cyclictest0-21swapper/221:05:152
262559962,0cyclictest0-21swapper/220:50:162
262559962,0cyclictest0-21swapper/219:45:182
262559962,0cyclictest0-21swapper/219:20:182
262559962,0cyclictest0-21swapper/219:15:012
262559960,0cyclictest0-21swapper/220:55:152
262549962,0cyclictest0-21swapper/120:55:151
262549962,0cyclictest0-21swapper/120:35:161
262549962,0cyclictest0-21swapper/120:00:141
262539962,0cyclictest0-21swapper/020:50:130
259960,0migration/219048-21munin-node20:00:172
11795060,0irq/26-eth1-rx-0-21swapper/020:47:270
262569950,0cyclictest0-21swapper/320:50:073
262559951,0cyclictest0-21swapper/220:25:162
262559950,0cyclictest0-21swapper/220:45:372
262559950,0cyclictest0-21swapper/220:35:142
249950,0watchdog/24433-21expr20:40:112
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional