You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-04-02 - 12:58

x86 Intel Xeon E3-1220L V2 @2300 MHz, Linux 4.4.39-rt50+ (Profile)

Latency plot of system in rack #7, slot #0
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the alll highest latencies:
System rack7slot0.osadl.org (updated Sun Apr 02, 2023 00:44:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
110050650,0irq/25-eth00-21swapper/119:05:301
110050640,0irq/25-eth00-21swapper/219:09:272
110050600,0irq/25-eth00-21swapper/319:07:093
2841624220,0sleep00-21swapper/019:08:480
110050200,0irq/25-eth00-21swapper/219:10:002
41190,0ktimersoftd/00-21swapper/000:04:490
351190,0ktimersoftd/314000-21sshd00:04:493
271190,0ktimersoftd/213991-21sshd00:04:482
28772991813,0cyclictest10707-21sshd23:56:550
28775991715,0cyclictest1416-21sshd23:10:233
28772991713,0cyclictest20396-21diskmemload22:18:320
2877299170,0cyclictest0-21swapper/000:39:050
28774991514,0cyclictest0-21swapper/221:16:232
28774991513,0cyclictest25831-21sshd23:48:052
28772991512,0cyclictest16497-21sshd23:47:030
110050150,0irq/25-eth03-21ksoftirqd/021:31:290
28775991413,0cyclictest20396-21diskmemload22:53:253
28775991413,0cyclictest0-21swapper/300:12:313
2877599140,0cyclictest0-21swapper/321:30:423
28774991413,0cyclictest20396-21diskmemload00:38:512
28774991413,0cyclictest0-21swapper/222:58:082
2877499140,0cyclictest0-21swapper/222:38:032
28773991413,0cyclictest0-21swapper/100:14:021
2877399141,0cyclictest1512-21sshd21:38:011
2877399140,0cyclictest0-21swapper/123:44:521
2877299142,0cyclictest27520-21sshd00:33:150
28772991413,0cyclictest15631-21sshd21:23:280
28772991411,0cyclictest30984-21sshd22:07:090
28772991411,0cyclictest0-21swapper/021:49:370
28772991411,0cyclictest0-21swapper/021:28:420
2877299140,0cyclictest31089-21sshd22:14:140
9950130,0irq/24-0000:00:0-21swapper/123:14:401
28775991313,0cyclictest30295-21sshd23:41:313
28775991312,0cyclictest22274-21sshd21:52:123
28775991312,0cyclictest0-21swapper/322:24:083
28775991312,0cyclictest0-21swapper/321:46:593
2877599130,0cyclictest24985-21bash22:37:583
2877599130,0cyclictest0-21swapper/323:48:363
2877599130,0cyclictest0-21swapper/321:11:463
28774991311,0cyclictest0-21swapper/222:34:212
28774991311,0cyclictest0-21swapper/221:35:402
28774991311,0cyclictest0-21swapper/200:26:052
28773991312,0cyclictest19396-21sshd22:33:541
28773991312,0cyclictest117950irq/26-eth1-rx-21:09:191
28773991312,0cyclictest0-21swapper/121:46:371
28773991312,0cyclictest0-21swapper/121:12:561
2877399130,0cyclictest1976-21bash00:03:201
2877399130,0cyclictest0-21swapper/123:37:481
2877399130,0cyclictest0-21swapper/123:21:461
2877399130,0cyclictest0-21swapper/121:41:231
2877299134,0cyclictest10132-21sshd22:01:240
2877299132,0cyclictest20396-21diskmemload21:16:570
2877299131,0cyclictest0-21swapper/022:24:510
2877299131,0cyclictest0-21swapper/021:58:400
2877299131,0cyclictest0-21swapper/021:35:180
2877299130,0cyclictest3883-21sshd21:53:460
2877299130,0cyclictest26090-21needreboot20:10:270
2877299130,0cyclictest25184-21id00:25:200
2877299130,0cyclictest13488-21sshd21:12:400
2877299130,0cyclictest0-21swapper/023:22:040
2877299130,0cyclictest0-21swapper/023:06:130
2877299130,0cyclictest0-21swapper/019:30:130
271130,0ktimersoftd/222102-21bash21:34:442
28775991211,0cyclictest18706-21bash23:19:193
28775991211,0cyclictest15453-21sshd21:37:263
28775991211,0cyclictest0-21swapper/323:27:213
28775991211,0cyclictest0-21swapper/323:27:203
28775991211,0cyclictest0-21swapper/323:01:263
28775991211,0cyclictest0-21swapper/321:26:253
28775991211,0cyclictest0-21swapper/300:35:033
28775991211,0cyclictest0-21swapper/300:24:443
28775991210,0cyclictest0-21swapper/300:31:063
2877599120,0cyclictest0-21swapper/321:56:303
28774991211,0cyclictest15738-21sshd23:33:002
28774991211,0cyclictest0-21swapper/223:25:542
28774991211,0cyclictest0-21swapper/223:25:542
28774991211,0cyclictest0-21swapper/221:50:252
2877499120,0cyclictest3265-21sshd23:07:052
2877499120,0cyclictest16790-21sshd21:27:042
2877499120,0cyclictest0-21swapper/222:20:512
2877499120,0cyclictest0-21swapper/222:12:152
2877499120,0cyclictest0-21swapper/221:55:392
2877399124,0cyclictest0-21swapper/121:03:231
28773991212,0cyclictest0-21swapper/122:20:251
28773991211,0cyclictest6367-21ls21:15:221
28773991211,0cyclictest0-21swapper/121:25:081
28773991211,0cyclictest0-21swapper/100:34:071
28773991211,0cyclictest0-21swapper/100:16:291
28773991210,0cyclictest17996-21id21:20:151
28773991210,0cyclictest0-21swapper/122:53:591
2877399120,0cyclictest9825-21sshd22:01:221
2877399120,0cyclictest20396-21diskmemload00:05:471
2877399120,0cyclictest0-21swapper/123:03:521
2877399120,0cyclictest0-21swapper/122:37:381
2877399120,0cyclictest0-21swapper/100:36:351
2877299125,0cyclictest18762-21bash22:40:430
2877299123,0cyclictest12059-21sshd23:11:300
2877299123,0cyclictest0-21swapper/022:49:100
2877299122,0cyclictest0-21swapper/022:25:250
28772991211,0cyclictest0-21swapper/023:38:060
28772991211,0cyclictest0-21swapper/020:50:010
2877299121,0cyclictest15354-21sshd23:05:000
2877299121,0cyclictest0-21swapper/023:40:200
2877299121,0cyclictest0-21swapper/000:14:590
28772991210,0cyclictest0-21swapper/021:41:350
2877299120,0cyclictest5516-21bash22:32:210
2877299120,0cyclictest20396-21diskmemload23:31:010
2877299120,0cyclictest19581-21sshd23:19:250
2877299120,0cyclictest0-21swapper/020:44:330
2877299120,0cyclictest0-21swapper/000:18:590
110050120,0irq/25-eth031136-21munin-node23:50:421
110050120,0irq/25-eth02829-21bash23:56:021
110050120,0irq/25-eth00-21swapper/323:37:193
28775991111,0cyclictest0-21swapper/322:12:103
28775991111,0cyclictest0-21swapper/322:00:473
28775991111,0cyclictest0-21swapper/300:18:203
28775991110,0cyclictest0-21swapper/323:09:273
28775991110,0cyclictest0-21swapper/322:30:533
28775991110,0cyclictest0-21swapper/322:16:283
28774991111,0cyclictest0-21swapper/223:16:022
28774991110,0cyclictest0-21swapper/223:54:162
28774991110,0cyclictest0-21swapper/222:27:352
28774991110,0cyclictest0-21swapper/221:12:112
2877399114,0cyclictest0-21swapper/119:37:391
28773991110,0cyclictest0-21swapper/123:29:461
28773991110,0cyclictest0-21swapper/123:29:461
28773991110,0cyclictest0-21swapper/122:27:091
28773991110,0cyclictest0-21swapper/122:16:081
2877299119,0cyclictest0-21swapper/021:02:260
2877299119,0cyclictest0-21swapper/019:54:350
2877299119,0cyclictest0-21swapper/019:36:010
2877299115,0cyclictest6672-21bash23:53:010
2877299115,0cyclictest2615-21latency19:20:220
2877299113,0cyclictest14693-21sshd00:08:410
2877299112,0cyclictest0-21swapper/023:26:590
2877299112,0cyclictest0-21swapper/023:26:580
2877299111,0cyclictest0-21swapper/022:56:470
2877299110,0cyclictest0-21swapper/022:50:070
2877299110,0cyclictest0-21swapper/020:36:390
2877299110,0cyclictest0-21swapper/019:26:500
110050110,0irq/25-eth00-21swapper/123:09:331
2877599109,0cyclictest31141-21sshd23:55:333
2877599109,0cyclictest25084-21bash22:27:303
2877599109,0cyclictest22635-21sshd23:54:473
2877599109,0cyclictest0-21swapper/321:40:493
2877599109,0cyclictest0-21swapper/321:18:103
28775991010,0cyclictest0-21swapper/322:06:573
2877499109,0cyclictest28-21ksoftirqd/223:40:312
2877499109,0cyclictest0-21swapper/223:24:582
2877499109,0cyclictest0-21swapper/222:06:062
2877499109,0cyclictest0-21swapper/200:30:032
28774991010,0cyclictest0-21swapper/222:41:462
28774991010,0cyclictest0-21swapper/200:08:052
2877499100,0cyclictest0-21swapper/219:18:572
2877399109,0cyclictest26019-21bash22:06:361
2877399109,0cyclictest0-21swapper/100:21:211
28773991010,0cyclictest0-21swapper/120:14:581
2877399100,0cyclictest0-21swapper/119:32:411
2877399100,0cyclictest0-21swapper/119:20:431
28772991010,0cyclictest0-21swapper/021:08:510
28772991010,0cyclictest0-21swapper/020:01:340
28772991010,0cyclictest0-21swapper/019:48:130
2877299100,0cyclictest0-21swapper/022:35:200
2877299100,0cyclictest0-21swapper/020:07:240
2877299100,0cyclictest0-21swapper/020:07:240
2877299100,0cyclictest0-21swapper/019:16:020
2877299100,0cyclictest0-21swapper/000:24:120
110050100,0irq/25-eth023269-21sshd21:52:191
110050100,0irq/25-eth00-21swapper/323:30:353
287759999,0cyclictest0-21swapper/321:20:363
287759999,0cyclictest0-21swapper/320:25:323
287759999,0cyclictest0-21swapper/300:28:413
287759998,0cyclictest31043-21sshd00:06:433
287759990,0cyclictest0-21swapper/322:45:143
287749999,0cyclictest0-21swapper/219:35:222
287749998,0cyclictest28-21ksoftirqd/223:03:142
287749998,0cyclictest28-21ksoftirqd/221:45:542
287749998,0cyclictest28-21ksoftirqd/221:20:252
287749998,0cyclictest28-21ksoftirqd/200:15:582
287749998,0cyclictest28-21ksoftirqd/200:10:322
287749998,0cyclictest12945-21sshd22:15:382
287749998,0cyclictest11661-21df_abs20:50:152
287749998,0cyclictest0-21swapper/200:23:322
287749990,0cyclictest0-21swapper/223:59:032
287739999,0cyclictest0-21swapper/120:57:431
287739999,0cyclictest0-21swapper/120:34:011
287739999,0cyclictest0-21swapper/119:44:181
287739998,0cyclictest0-21swapper/121:56:081
287739998,0cyclictest0-21swapper/121:33:431
287739995,0cyclictest0-21swapper/122:11:301
287739995,0cyclictest0-21swapper/119:52:071
287739990,0cyclictest2720-21bash22:49:371
287739990,0cyclictest0-21swapper/119:58:001
287739990,0cyclictest0-21swapper/100:28:511
287729999,0cyclictest0-21swapper/019:11:580
287729997,0cyclictest0-21swapper/019:43:200
287729992,0cyclictest0-21swapper/020:45:240
287729992,0cyclictest0-21swapper/020:30:200
287729992,0cyclictest0-21swapper/020:20:220
287729992,0cyclictest0-21swapper/020:15:210
11005090,0irq/25-eth021276-21rm22:40:593
287759985,0cyclictest0-21swapper/322:59:343
287749988,0cyclictest0-21swapper/221:43:202
287749988,0cyclictest0-21swapper/220:35:092
287749987,0cyclictest28-21ksoftirqd/223:35:222
287749987,0cyclictest28-21ksoftirqd/222:54:262
287749987,0cyclictest28-21ksoftirqd/222:45:222
287749987,0cyclictest28-21ksoftirqd/222:00:142
287749986,0cyclictest28-21ksoftirqd/223:10:452
287739988,0cyclictest0-21swapper/120:49:491
287739988,0cyclictest0-21swapper/120:41:551
287739988,0cyclictest0-21swapper/119:25:001
287739985,0cyclictest117950irq/26-eth1-rx-22:56:271
287739985,0cyclictest117950irq/26-eth1-rx-22:40:361
287739985,0cyclictest117950irq/26-eth1-rx-20:50:311
287739985,0cyclictest117950irq/26-eth1-rx-20:27:591
287739985,0cyclictest117950irq/26-eth1-rx-19:10:271
287739985,0cyclictest0-21swapper/120:24:471
287739985,0cyclictest0-21swapper/120:15:391
287739985,0cyclictest0-21swapper/120:00:231
287739985,0cyclictest0-21swapper/119:45:181
287739985,0cyclictest0-21swapper/119:17:371
287739984,0cyclictest0-21swapper/120:36:211
287739984,0cyclictest0-21swapper/120:05:011
287739984,0cyclictest0-21swapper/120:05:011
287739981,0cyclictest0-21swapper/123:15:461
287729983,0cyclictest0-21swapper/020:26:280
287729981,0cyclictest0-21swapper/020:55:190
11005080,0irq/25-eth04051-21bash23:21:083
11005080,0irq/25-eth01969-21sshd23:31:271
11005080,0irq/25-eth00-21swapper/123:45:261
287759977,0cyclictest0-21swapper/319:56:583
287759970,0cyclictest0-21swapper/321:05:233
287759970,0cyclictest0-21swapper/321:01:543
287749977,0cyclictest0-21swapper/220:43:102
287749973,0cyclictest0-21swapper/220:30:202
287749970,0cyclictest0-21swapper/220:07:512
287749970,0cyclictest0-21swapper/220:07:512
287749970,0cyclictest0-21swapper/219:45:222
287729970,0cyclictest0-21swapper/019:59:090
339960,0migration/326211-21nfsd420:10:283
287759962,0cyclictest0-21swapper/320:40:223
287759962,0cyclictest0-21swapper/320:35:243
287759962,0cyclictest0-21swapper/320:15:203
287759962,0cyclictest0-21swapper/319:40:293
287759962,0cyclictest0-21swapper/319:15:213
287759962,0cyclictest0-21swapper/319:10:203
287759960,0cyclictest14576-21grep20:55:203
287759960,0cyclictest0-21swapper/319:21:183
287749965,0cyclictest292552sleep220:17:332
287749965,0cyclictest28-21ksoftirqd/219:55:172
287749965,0cyclictest28-21ksoftirqd/219:25:242
287749962,0cyclictest0-21swapper/221:05:192
287749962,0cyclictest0-21swapper/220:55:192
287749962,0cyclictest0-21swapper/220:45:232
287749962,0cyclictest0-21swapper/220:25:232
287749962,0cyclictest0-21swapper/220:20:222
287749962,0cyclictest0-21swapper/220:10:242
287749962,0cyclictest0-21swapper/220:00:202
287749962,0cyclictest0-21swapper/219:50:202
287749962,0cyclictest0-21swapper/219:40:292
287749962,0cyclictest0-21swapper/219:30:202
287749960,0cyclictest16723-21munin-node21:00:192
287759955,0cyclictest0-21swapper/319:45:143
287759951,0cyclictest0-21swapper/320:20:223
287759951,0cyclictest0-21swapper/319:50:203
287759950,0cyclictest0-21swapper/320:05:203
287759950,0cyclictest0-21swapper/320:05:203
287759950,0cyclictest0-21swapper/319:35:573
287749954,0cyclictest28-21ksoftirqd/219:20:172
11795050,0irq/26-eth1-rx-14973-21kworker/3:220:50:153
11795050,0irq/26-eth1-rx-14973-21kworker/3:220:45:233
11795050,0irq/26-eth1-rx-14973-21kworker/3:220:30:013
11795050,0irq/26-eth1-rx-14973-21kworker/3:220:00:173
11795050,0irq/26-eth1-rx-14973-21kworker/3:219:30:013
11795050,0irq/26-eth1-rx-14973-21kworker/3:219:25:173
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional