You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-12-04 - 02:46

x86 Intel Xeon E3-1220L V2 @2300 MHz, Linux 4.4.39-rt50+ (Profile)

Latency plot of system in rack #7, slot #0
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack7slot0.osadl.org (updated Sun Dec 04, 2022 00:44:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
110050640,0irq/25-eth00-21swapper/219:07:192
117950620,0irq/26-eth1-rx-0-21swapper/119:08:531
110050620,0irq/25-eth00-21swapper/319:07:523
110050450,0irq/25-eth00-21swapper/019:05:110
432599201,0cyclictest6397-21sshd23:00:391
432599180,0cyclictest0-21swapper/121:46:481
271180,0ktimersoftd/226561-21bash00:21:282
4327991615,0cyclictest2030-21sshd21:27:053
432599160,0cyclictest5795-21sshd23:40:391
432599160,0cyclictest29223-21sshd00:29:111
4327991514,0cyclictest0-21swapper/323:30:143
432799150,0cyclictest0-21swapper/322:16:423
4325991513,0cyclictest0-21swapper/123:39:421
4325991511,0cyclictest0-21swapper/122:08:171
432799140,0cyclictest0-21swapper/300:25:193
4326991413,0cyclictest0-21swapper/223:47:132
432699140,0cyclictest0-21swapper/223:20:032
4325991413,0cyclictest0-21swapper/123:59:271
4325991411,0cyclictest0-21swapper/123:06:581
4325991411,0cyclictest0-21swapper/120:35:211
432599140,0cyclictest31291-21sshd22:46:371
4324991413,0cyclictest28354-21diskmemload22:42:340
4324991413,0cyclictest28354-21diskmemload22:42:330
4324991413,0cyclictest0-21swapper/023:33:130
4327991312,0cyclictest11702-21sshd22:54:413
4327991312,0cyclictest0-21swapper/323:43:493
4327991312,0cyclictest0-21swapper/321:16:183
4327991312,0cyclictest0-21swapper/300:07:533
4327991311,0cyclictest0-21swapper/300:38:183
432799130,0cyclictest0-21swapper/300:17:443
432799130,0cyclictest0-21swapper/300:01:423
4326991312,0cyclictest19778-21munin-node00:35:192
4326991312,0cyclictest1453-21nfsd23:56:102
4326991312,0cyclictest0-21swapper/223:19:062
432699130,0cyclictest0-21swapper/200:15:552
4325991312,0cyclictest0-21swapper/123:50:301
4325991312,0cyclictest0-21swapper/121:25:471
4325991312,0cyclictest0-21swapper/121:24:501
4325991310,0cyclictest0-21swapper/122:28:511
432599130,0cyclictest9936-21sshd22:14:261
432599130,0cyclictest0-21swapper/119:42:571
432599130,0cyclictest0-21swapper/100:35:481
4324991312,0cyclictest4817-21sshd23:50:320
4324991312,0cyclictest0-21swapper/021:14:090
4324991310,0cyclictest10823-21sshd00:01:190
432499130,0cyclictest24466-21sshd00:14:000
271130,0ktimersoftd/20-21swapper/223:39:412
432799122,0cyclictest0-21swapper/300:11:463
4327991211,0cyclictest9137-21sshd22:24:223
4327991210,0cyclictest0-21swapper/321:55:473
432799120,0cyclictest10695-21sshd22:44:353
432799120,0cyclictest10695-21sshd22:44:343
432799120,0cyclictest0-21swapper/322:56:333
432799120,0cyclictest0-21swapper/322:37:503
4326991211,0cyclictest0-21swapper/222:43:202
4326991211,0cyclictest0-21swapper/222:43:192
4326991211,0cyclictest0-21swapper/222:39:572
4326991211,0cyclictest0-21swapper/222:29:512
4326991211,0cyclictest0-21swapper/222:14:312
4326991211,0cyclictest0-21swapper/221:27:422
4326991210,0cyclictest30661-21sshd22:33:142
432699120,0cyclictest10685-21sshd21:51:102
432699120,0cyclictest0-21swapper/222:57:092
432699120,0cyclictest0-21swapper/220:26:362
432599129,0cyclictest0-21swapper/120:55:411
4325991212,0cyclictest23029-21sshd23:49:111
4325991212,0cyclictest0-21swapper/122:38:581
4325991211,0cyclictest28354-21diskmemload23:23:191
4325991211,0cyclictest0-21swapper/122:04:001
4325991211,0cyclictest0-21swapper/100:08:241
432599120,0cyclictest0-21swapper/121:31:531
432599120,0cyclictest0-21swapper/120:54:451
432599120,0cyclictest0-21swapper/120:17:491
4324991211,0cyclictest0-21swapper/021:47:160
4324991210,0cyclictest699-21sshd21:56:470
4324991210,0cyclictest0-21swapper/022:14:200
4324991210,0cyclictest0-21swapper/022:02:010
4324991210,0cyclictest0-21swapper/021:51:340
432499120,0cyclictest13938-21sshd22:08:110
432499120,0cyclictest13723-21sshd21:15:050
432499120,0cyclictest0-21swapper/023:00:060
432499120,0cyclictest0-21swapper/022:58:140
432499120,0cyclictest0-21swapper/022:34:530
432499120,0cyclictest0-21swapper/022:24:470
351120,0ktimersoftd/30-21swapper/323:27:473
110050120,0irq/25-eth03-21ksoftirqd/022:28:160
110050120,0irq/25-eth00-21swapper/023:05:050
4327991111,0cyclictest28354-21diskmemload22:00:053
4327991111,0cyclictest0-21swapper/322:10:323
4327991111,0cyclictest0-21swapper/322:05:193
4327991111,0cyclictest0-21swapper/319:10:153
4327991110,0cyclictest17012-21bash23:51:513
4327991110,0cyclictest0-21swapper/323:13:513
4327991110,0cyclictest0-21swapper/321:41:353
4327991110,0cyclictest0-21swapper/319:40:133
432799110,0cyclictest25521-21sshd23:09:253
4326991111,0cyclictest0-21swapper/221:46:522
4326991111,0cyclictest0-21swapper/220:57:372
4326991111,0cyclictest0-21swapper/219:13:392
4326991110,0cyclictest10945-21sshd00:05:062
4326991110,0cyclictest0-21swapper/222:46:422
4326991110,0cyclictest0-21swapper/221:58:512
4326991110,0cyclictest0-21swapper/221:12:152
4326991110,0cyclictest0-21swapper/200:14:032
432699110,0cyclictest6280-21missed_timers20:20:202
432699110,0cyclictest0-21swapper/220:08:002
432699110,0cyclictest0-21swapper/219:30:092
432599119,0cyclictest0-21swapper/119:36:351
432599118,0cyclictest28224-21sshd21:43:051
4325991111,0cyclictest28354-21diskmemload22:20:361
4325991111,0cyclictest28354-21diskmemload00:24:061
4325991111,0cyclictest0-21swapper/122:42:191
4325991111,0cyclictest0-21swapper/122:42:191
4325991111,0cyclictest0-21swapper/120:09:551
4325991110,0cyclictest29518-21sshd00:32:471
4325991110,0cyclictest28354-21diskmemload21:50:101
4325991110,0cyclictest0-21swapper/119:21:481
432599110,0cyclictest14787-21sshd21:11:491
432599110,0cyclictest0-21swapper/123:25:121
432599110,0cyclictest0-21swapper/119:11:131
432499110,0cyclictest0-21swapper/023:10:210
41110,0ktimersoftd/09994-21sshd23:24:300
41110,0ktimersoftd/00-21swapper/023:47:160
41110,0ktimersoftd/00-21swapper/000:24:330
271110,0ktimersoftd/20-21swapper/221:31:532
110050110,0irq/25-eth012478-21sshd21:34:520
432799109,0cyclictest5088-21bash21:50:333
432799109,0cyclictest1274-21sshd22:33:323
432799109,0cyclictest0-21swapper/323:58:553
432799109,0cyclictest0-21swapper/320:29:493
432799108,0cyclictest2473-21sshd23:16:593
4327991010,0cyclictest0-21swapper/321:06:043
4327991010,0cyclictest0-21swapper/320:45:093
4327991010,0cyclictest0-21swapper/319:58:003
4327991010,0cyclictest0-21swapper/319:18:233
432799100,0cyclictest32112-21sshd21:13:383
432699109,0cyclictest15747-21sshd22:08:222
432699109,0cyclictest0-21swapper/223:33:372
432699109,0cyclictest0-21swapper/222:20:402
432699109,0cyclictest0-21swapper/220:40:052
432699109,0cyclictest0-21swapper/220:36:562
4326991010,0cyclictest0-21swapper/222:50:592
4326991010,0cyclictest0-21swapper/219:51:312
4326991010,0cyclictest0-21swapper/219:51:312
4326991010,0cyclictest0-21swapper/219:15:162
432699100,0cyclictest15684-21bash22:15:022
432599109,0cyclictest19319-21cp21:58:451
432599109,0cyclictest0-21swapper/121:01:361
432599106,0cyclictest1941-21sshd22:16:511
432599104,0cyclictest5267-21cut00:15:211
432599103,0cyclictest19304-21sshd23:15:221
432599102,0cyclictest21921-21cp22:59:021
4325991010,0cyclictest0-21swapper/123:10:061
4325991010,0cyclictest0-21swapper/120:28:091
4325991010,0cyclictest0-21swapper/119:47:441
432599100,0cyclictest9832-21bash21:37:521
432599100,0cyclictest22266-21sshd21:15:511
432599100,0cyclictest0-21swapper/120:12:561
432599100,0cyclictest0-21swapper/100:00:241
432499109,0cyclictest14719-21sshd22:38:150
4324991010,0cyclictest0-21swapper/022:50:130
432499100,0cyclictest0-21swapper/021:39:150
432499100,0cyclictest0-21swapper/021:27:490
41100,0ktimersoftd/09672-21sed00:30:290
41100,0ktimersoftd/015113-21sshd23:41:390
351100,0ktimersoftd/30-21swapper/320:38:183
110050100,0irq/25-eth036-21ksoftirqd/321:46:223
110050100,0irq/25-eth036-21ksoftirqd/300:33:553
110050100,0irq/25-eth019514-21bash00:28:040
43279999,0cyclictest0-21swapper/320:52:223
43279999,0cyclictest0-21swapper/320:33:533
43279999,0cyclictest0-21swapper/320:19:293
43279999,0cyclictest0-21swapper/320:01:293
43279999,0cyclictest0-21swapper/319:27:053
43279998,0cyclictest2632-21sshd22:47:013
43279990,0cyclictest0-21swapper/300:23:013
43269999,0cyclictest0-21swapper/221:44:232
43269999,0cyclictest0-21swapper/220:52:372
43269999,0cyclictest0-21swapper/219:45:292
43269998,0cyclictest0-21swapper/221:24:562
43269998,0cyclictest0-21swapper/200:25:032
43269990,0cyclictest28354-21diskmemload23:04:432
43269990,0cyclictest0-21swapper/223:05:082
43269990,0cyclictest0-21swapper/219:20:092
43259999,0cyclictest0-21swapper/120:43:291
43259999,0cyclictest0-21swapper/119:55:581
43259998,0cyclictest0-21swapper/119:52:291
43259998,0cyclictest0-21swapper/119:52:291
43259993,0cyclictest2467-21df22:30:201
43259993,0cyclictest24081-21sshd00:13:561
43259990,0cyclictest3365-21rm23:30:241
43259990,0cyclictest0-21swapper/122:52:181
43259990,0cyclictest0-21swapper/121:07:321
43249998,0cyclictest25295-21runrttasks23:59:270
43249990,0cyclictest0-21swapper/023:27:040
43249990,0cyclictest0-21swapper/022:45:280
43249990,0cyclictest0-21swapper/021:43:240
43249990,0cyclictest0-21swapper/021:22:000
43249990,0cyclictest0-21swapper/019:48:040
4190,0ktimersoftd/09649-21sshd23:37:470
4190,0ktimersoftd/04971-21bash00:19:050
4190,0ktimersoftd/022856-21sshd22:15:380
4190,0ktimersoftd/020695-21nfsd400:35:220
4190,0ktimersoftd/00-21swapper/000:05:300
11005090,0irq/25-eth036-21ksoftirqd/323:49:243
11005090,0irq/25-eth036-21ksoftirqd/323:22:353
11005090,0irq/25-eth036-21ksoftirqd/321:30:173
11005090,0irq/25-eth00-21swapper/200:32:282
43279988,0cyclictest0-21swapper/320:59:133
43279987,0cyclictest0-21swapper/322:29:293
43279980,0cyclictest0-21swapper/320:40:213
43269988,0cyclictest0-21swapper/221:08:322
43269988,0cyclictest0-21swapper/220:33:482
43269988,0cyclictest0-21swapper/219:59:462
43259980,0cyclictest8369-21munin-node19:15:181
43259980,0cyclictest17464-21grep20:45:141
43259980,0cyclictest14095-21grep19:30:001
43259980,0cyclictest0-21swapper/120:20:201
43249988,0cyclictest0-21swapper/020:42:200
43249988,0cyclictest0-21swapper/020:00:170
43249988,0cyclictest0-21swapper/019:15:540
43249980,0cyclictest8783-21ntp_kernel_err20:25:210
43249980,0cyclictest0-21swapper/020:20:150
43249980,0cyclictest0-21swapper/020:15:230
11005080,0irq/25-eth036-21ksoftirqd/323:38:503
11005080,0irq/25-eth030419-21sshd23:43:162
11005080,0irq/25-eth028-21ksoftirqd/223:10:072
11005080,0irq/25-eth028-21ksoftirqd/222:02:132
11005080,0irq/25-eth019702-21sshd23:15:230
11005080,0irq/25-eth018016-21sshd21:15:242
11005080,0irq/25-eth016962-21id23:28:302
11005080,0irq/25-eth010646-21sshd23:51:092
11005080,0irq/25-eth00-21swapper/221:38:182
43279977,0cyclictest0-21swapper/319:22:553
43279973,0cyclictest0-21swapper/321:00:193
43269977,0cyclictest0-21swapper/221:03:322
43269977,0cyclictest0-21swapper/220:13:422
43269977,0cyclictest0-21swapper/220:03:152
43259971,0cyclictest14563-21grep19:30:121
43259970,0cyclictest29054-21munin-node20:00:141
43259970,0cyclictest10624-21tune2fs20:30:141
43249977,0cyclictest0-21swapper/019:53:320
43249977,0cyclictest0-21swapper/019:53:320
43249976,0cyclictest0-21swapper/020:12:290
43249975,0cyclictest0-21swapper/019:37:450
43249973,0cyclictest0-21swapper/020:30:170
43249970,0cyclictest0-21swapper/021:00:150
4170,0ktimersoftd/010096-21diskstats19:20:130
339970,0migration/311331-21sshd21:21:233
159970,0watchdog/019608-21fschecks_time19:40:150
11005070,0irq/25-eth08008-21sshd00:00:592
11005070,0irq/25-eth036-21ksoftirqd/323:00:263
11005070,0irq/25-eth036-21ksoftirqd/321:35:063
11005070,0irq/25-eth00-21swapper/020:49:090
43279962,0cyclictest0-21swapper/320:20:183
43279962,0cyclictest0-21swapper/320:10:163
43279962,0cyclictest0-21swapper/320:05:183
43279962,0cyclictest0-21swapper/319:50:163
43279962,0cyclictest0-21swapper/319:50:163
43279962,0cyclictest0-21swapper/319:45:173
43269966,0cyclictest0-21swapper/220:49:292
43269965,0cyclictest16989-21cat19:35:122
43269962,0cyclictest0-21swapper/220:15:172
43269962,0cyclictest0-21swapper/219:25:162
43249965,0cyclictest4382-21atenall_r7power19:10:120
43249965,0cyclictest31110-21fschecks_count20:05:160
43249965,0cyclictest12652-21fschecks_time19:25:130
43249963,0cyclictest0-21swapper/019:55:220
43249962,0cyclictest0-21swapper/021:05:160
43249962,0cyclictest0-21swapper/020:55:150
43249962,0cyclictest0-21swapper/019:30:170
43249960,0cyclictest0-21swapper/020:35:050
149960,0migration/019437-21cpuspeed_turbos20:50:120
43279950,0cyclictest0-21swapper/319:35:153
43269950,0cyclictest0-21swapper/219:40:012
11795050,0irq/26-eth1-rx-18570-21kworker/3:019:30:393
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional