You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2019-06-18 - 17:38

Intel(R) Xeon(R) CPU E31220 @ 3.10GHz, Linux 4.16.15-rt7 (Profile)

Latency plot of system in rack #b, slot #1
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the highest latencies:
System rackbslot1.osadl.org (updated Fri Jan 25, 2019 12:44:18)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
227892650,0sleep20-21swapper/211:45:062
2171726453,7sleep20-21swapper/207:07:142
57642590,0sleep20-21swapper/209:25:062
2191725633,7sleep10-21swapper/107:09:061
2196925341,7sleep00-21swapper/007:09:440
2160425341,7sleep30-21swapper/307:06:073
269522160,0sleep00-21swapper/010:25:060
22078991514,1cyclictest12747-21snmpd09:48:463
22076991412,1cyclictest12747-21snmpd11:21:231
22078991313,0cyclictest12747-21snmpd10:12:233
22078991313,0cyclictest12747-21snmpd09:53:063
22077991312,0cyclictest12747-21snmpd12:03:272
22076991311,1cyclictest12747-21snmpd09:25:251
22075991210,1cyclictest12747-21snmpd11:38:010
22075991210,1cyclictest12747-21snmpd09:40:590
2207599120,1cyclictest12029-21ntp_states08:10:080
2207899119,1cyclictest0-21swapper/310:48:183
2207899119,1cyclictest0-21swapper/308:27:513
2207899110,1cyclictest0-21swapper/309:38:433
2207899110,1cyclictest0-21swapper/309:30:013
2207899110,10cyclictest0-21swapper/312:35:423
2207799119,1cyclictest0-21swapper/212:31:432
2207799119,1cyclictest0-21swapper/211:55:142
2207799119,1cyclictest0-21swapper/209:33:172
22077991111,0cyclictest12747-21snmpd09:38:372
2207799110,1cyclictest0-21swapper/207:51:442
2207799110,10cyclictest0-21swapper/212:10:152
2207799110,10cyclictest0-21swapper/210:32:052
2207799110,10cyclictest0-21swapper/208:23:422
2207799110,10cyclictest0-21swapper/208:19:502
2207699119,1cyclictest0-21swapper/111:51:431
22076991111,0cyclictest12747-21snmpd08:22:201
2207699110,10cyclictest0-21swapper/112:18:021
2207699110,10cyclictest0-21swapper/109:50:461
2207599110,10cyclictest0-21swapper/007:40:060
2207899108,1cyclictest0-21swapper/312:09:033
2207899108,1cyclictest0-21swapper/311:44:023
2207899108,1cyclictest0-21swapper/310:19:443
2207899108,1cyclictest0-21swapper/309:56:303
2207899108,1cyclictest0-21swapper/308:32:023
2207899108,1cyclictest0-21swapper/307:53:473
2207899108,1cyclictest0-21swapper/307:41:303
2207899108,1cyclictest0-21swapper/307:14:423
2207899100,9cyclictest0-21swapper/311:28:513
2207899100,9cyclictest0-21swapper/311:17:223
2207899100,9cyclictest0-21swapper/311:09:383
2207899100,9cyclictest0-21swapper/309:08:313
2207899100,9cyclictest0-21swapper/307:28:343
2207899100,1cyclictest0-21swapper/312:19:513
2207799109,0cyclictest0-21swapper/208:55:212
2207799109,0cyclictest0-21swapper/207:50:182
2207799108,1cyclictest12747-21snmpd09:52:272
2207799108,1cyclictest0-21swapper/212:36:342
2207799108,1cyclictest0-21swapper/211:37:382
2207799108,1cyclictest0-21swapper/211:06:192
2207799108,1cyclictest0-21swapper/210:23:022
2207799108,1cyclictest0-21swapper/210:08:492
2207799108,1cyclictest0-21swapper/209:04:432
2207799100,9cyclictest0-21swapper/212:22:212
2207799100,9cyclictest0-21swapper/211:11:472
2207799100,9cyclictest0-21swapper/208:28:232
2207799100,9cyclictest0-21swapper/207:24:172
2207799100,1cyclictest0-21swapper/207:40:462
2207699109,0cyclictest0-21swapper/111:50:151
2207699108,1cyclictest0-21swapper/112:38:411
2207599108,1cyclictest0-21swapper/008:25:490
220789998,0cyclictest0-21swapper/308:45:073
220789997,1cyclictest0-21swapper/312:12:363
220789997,1cyclictest0-21swapper/311:47:053
220789997,1cyclictest0-21swapper/311:02:223
220789997,1cyclictest0-21swapper/310:34:553
220789997,1cyclictest0-21swapper/310:03:163
220789997,1cyclictest0-21swapper/308:14:063
220789997,1cyclictest0-21swapper/307:35:123
220789990,8cyclictest0-21swapper/312:01:093
220789990,8cyclictest0-21swapper/308:56:553
220779997,1cyclictest0-21swapper/209:07:562
220779997,1cyclictest0-21swapper/208:32:352
220779997,1cyclictest0-21swapper/208:12:152
220779990,8cyclictest0-21swapper/212:29:372
220779990,1cyclictest0-21swapper/207:19:072
220769999,0cyclictest12747-21snmpd08:58:151
220769997,1cyclictest12747-21snmpd07:28:461
220769997,1cyclictest0-21swapper/111:06:481
220769997,1cyclictest0-21swapper/110:30:481
220769997,1cyclictest0-21swapper/108:35:021
220769997,1cyclictest0-21swapper/107:44:491
220769997,1cyclictest0-21swapper/107:10:451
220759998,0cyclictest0-21swapper/007:30:070
220759991,4cyclictest31030-21systemd-cgroups12:09:590
220789988,0cyclictest12747-21snmpd09:21:293
220789986,1cyclictest12747-21snmpd09:17:553
220789986,1cyclictest0-21swapper/312:22:073
220789986,1cyclictest0-21swapper/311:54:593
220789986,1cyclictest0-21swapper/311:36:263
220789986,1cyclictest0-21swapper/311:33:323
220789986,1cyclictest0-21swapper/310:51:223
220789986,1cyclictest0-21swapper/310:06:193
220789986,1cyclictest0-21swapper/309:31:283
220789986,1cyclictest0-21swapper/308:21:433
220789986,1cyclictest0-21swapper/307:57:583
220789986,1cyclictest0-21swapper/307:19:523
220779986,1cyclictest29543-21runrttasks07:29:272
220779986,1cyclictest12747-21snmpd07:11:582
220779986,1cyclictest0-21swapper/210:03:002
220779980,7cyclictest0-21swapper/209:57:222
220779980,7cyclictest0-21swapper/207:34:272
220769986,1cyclictest12747-21snmpd08:14:091
220769986,1cyclictest0-21swapper/109:22:201
220769986,1cyclictest0-21swapper/107:36:461
220759988,0cyclictest13558-21awk08:15:050
220759987,0cyclictest12747-21snmpd09:07:460
220759986,1cyclictest12747-21snmpd12:39:010
220759986,1cyclictest12747-21snmpd12:27:320
220759986,1cyclictest12747-21snmpd11:57:210
220759985,2cyclictest780-21dbus-daemon09:00:000
7225270,0sleep10-21swapper/111:00:161
5599270,0sleep20-21swapper/210:55:172
22969270,0sleep30-21swapper/308:40:173
220789976,0cyclictest12747-21snmpd08:51:013
220789976,0cyclictest0-21swapper/309:45:013
220789975,1cyclictest12747-21snmpd10:57:583
220789975,1cyclictest0-21swapper/311:57:443
220789975,1cyclictest0-21swapper/311:24:283
220789975,1cyclictest0-21swapper/311:11:153
220789975,1cyclictest0-21swapper/310:30:053
220789975,1cyclictest0-21swapper/309:02:043
220789975,1cyclictest0-21swapper/308:05:043
220779975,1cyclictest12747-21snmpd11:20:292
220779975,1cyclictest12747-21snmpd10:57:282
220779975,1cyclictest12747-21snmpd10:44:362
220779975,1cyclictest12747-21snmpd10:18:212
220779975,1cyclictest12747-21snmpd10:14:442
220779975,1cyclictest12747-21snmpd09:26:562
220779975,1cyclictest12747-21snmpd09:15:502
220779975,1cyclictest12747-21snmpd09:13:322
220779975,1cyclictest0-21swapper/211:57:592
220779975,1cyclictest0-21swapper/207:58:402
220769976,0cyclictest12747-21snmpd08:03:211
220769975,1cyclictest13464-21polkitd07:34:011
220769975,1cyclictest12747-21snmpd11:02:151
220769975,1cyclictest12747-21snmpd10:10:541
220769975,1cyclictest12747-21snmpd10:07:201
220769975,1cyclictest12747-21snmpd07:20:181
220769975,1cyclictest0-21swapper/112:10:251
220769975,1cyclictest0-21swapper/109:17:501
220769975,1cyclictest0-21swapper/109:06:321
220769975,1cyclictest0-21swapper/108:50:511
220769970,6cyclictest0-21swapper/110:04:581
220769970,6cyclictest0-21swapper/108:28:341
220769970,6cyclictest0-21swapper/107:56:551
220769970,1cyclictest0-21swapper/112:29:591
220769970,1cyclictest0-21swapper/110:50:311
220759976,0cyclictest12747-21snmpd12:02:140
220759976,0cyclictest12747-21snmpd11:00:180
220759975,1cyclictest12747-21snmpd10:46:250
220759975,1cyclictest12747-21snmpd10:25:200
220759975,1cyclictest12747-21snmpd08:22:140
220759975,1cyclictest12747-21snmpd07:41:100
220759975,1cyclictest0-21swapper/011:47:390
220759975,1cyclictest0-21swapper/008:45:400
220759970,6cyclictest0-21swapper/012:17:020
220759970,6cyclictest0-21swapper/011:06:280
220759970,6cyclictest0-21swapper/010:52:150
220759970,6cyclictest0-21swapper/010:06:050
220759970,6cyclictest0-21swapper/009:20:130
220759970,6cyclictest0-21swapper/008:36:480
27963260,1sleep20-21swapper/208:55:152
22527260,1sleep00-21swapper/010:11:110
220789965,1cyclictest12747-21snmpd10:20:263
220789965,0cyclictest12747-21snmpd07:45:443
220789964,1cyclictest12747-21snmpd12:29:393
220789964,1cyclictest12747-21snmpd09:11:123
220789964,1cyclictest0-21swapper/310:41:223
220789964,1cyclictest0-21swapper/308:06:313
220779965,1cyclictest12747-21snmpd09:49:292
220779964,1cyclictest31837-21cut10:40:042
220779964,1cyclictest12747-21snmpd08:48:072
220779964,1cyclictest12747-21snmpd08:00:552
220779964,1cyclictest0-21swapper/207:39:092
220779963,2cyclictest12747-21snmpd09:44:312
220769966,0cyclictest12747-21snmpd08:16:081
220769965,0cyclictest12747-21snmpd11:40:311
220769965,0cyclictest12747-21snmpd08:06:031
220769964,1cyclictest2269-21cat09:15:071
220769964,1cyclictest12747-21snmpd11:25:531
220769964,1cyclictest12747-21snmpd10:17:061
220769964,1cyclictest12747-21snmpd09:45:381
220769964,1cyclictest0-21swapper/111:36:311
220769964,1cyclictest0-21swapper/111:11:581
220769964,1cyclictest0-21swapper/107:17:411
220769963,2cyclictest9781-21sendmail09:56:521
220769963,2cyclictest12747-21snmpd10:26:361
220759964,1cyclictest14838-21perf08:20:000
220759964,1cyclictest12747-21snmpd09:27:340
220759964,1cyclictest12747-21snmpd07:48:460
220759964,1cyclictest0-21swapper/011:02:540
220759964,1cyclictest0-21swapper/010:00:260
220759963,2cyclictest13466-21gdbus08:45:000
220759963,2cyclictest13466-21gdbus08:05:010
220759963,2cyclictest13464-21polkitd08:52:590
220759963,2cyclictest12747-21snmpd12:22:240
220759963,2cyclictest12747-21snmpd10:15:240
220759963,2cyclictest12747-21snmpd09:58:500
220759963,2cyclictest12747-21snmpd09:54:470
220759963,2cyclictest12747-21snmpd09:03:310
220759960,1cyclictest0-21swapper/010:30:370
220759960,1cyclictest0-21swapper/009:48:570
220759960,1cyclictest0-21swapper/009:21:400
32282250,1sleep00-21swapper/012:10:290
220789953,1cyclictest12747-21snmpd12:31:183
220789953,1cyclictest12747-21snmpd10:37:013
220789953,1cyclictest12747-21snmpd08:47:163
220789953,1cyclictest12747-21snmpd08:15:243
220789953,1cyclictest12747-21snmpd07:36:223
220789953,1cyclictest12747-21snmpd07:21:213
220779955,0cyclictest12747-21snmpd10:46:432
220779953,1cyclictest9781-21sendmail11:31:302
220779953,1cyclictest780-21dbus-daemon11:15:592
220779953,1cyclictest7668-21crond11:01:002
220779953,1cyclictest21398-21dig08:35:452
220779953,1cyclictest12747-21snmpd12:16:292
220779953,1cyclictest12747-21snmpd12:10:232
220779953,1cyclictest12747-21snmpd11:26:532
220779953,1cyclictest12747-21snmpd08:42:042
220779953,1cyclictest12747-21snmpd08:07:172
220779953,1cyclictest0-21swapper/211:47:182
220779951,3cyclictest0-21swapper/210:25:572
220769954,1cyclictest12747-21snmpd10:20:241
220769953,1cyclictest29940-21kworker/1:010:35:211
220769953,1cyclictest26727-21kworker/1:109:00:451
220769953,1cyclictest25711-21kworker/1:112:05:491
220769953,1cyclictest23647-21kworker/1:008:45:191
220769953,1cyclictest12747-21snmpd12:22:511
220769953,1cyclictest12747-21snmpd12:02:571
220769953,1cyclictest12747-21snmpd11:56:031
220769953,1cyclictest12747-21snmpd11:30:251
220769953,1cyclictest12747-21snmpd10:46:371
220769953,1cyclictest12747-21snmpd10:41:191
220769953,1cyclictest12747-21snmpd09:42:501
220769953,1cyclictest12747-21snmpd09:35:431
220769953,1cyclictest12747-21snmpd09:30:311
220769953,1cyclictest12747-21snmpd08:41:461
220769953,1cyclictest12747-21snmpd08:35:541
220769953,1cyclictest12747-21snmpd07:51:101
220769953,1cyclictest12747-21snmpd07:47:131
220769950,4cyclictest7039-21sshd12:31:391
220759953,1cyclictest6942-21dig12:30:450
220759953,1cyclictest12747-21snmpd11:41:560
220759953,1cyclictest12747-21snmpd11:25:340
220759953,1cyclictest12747-21snmpd11:15:260
220759953,1cyclictest12747-21snmpd11:10:460
220759953,1cyclictest12747-21snmpd10:41:040
220759953,1cyclictest12747-21snmpd09:31:250
220759953,1cyclictest12747-21snmpd09:11:580
220759953,1cyclictest12747-21snmpd08:30:270
220759953,1cyclictest12747-21snmpd07:55:440
220759953,1cyclictest12747-21snmpd07:50:580
220759953,1cyclictest12747-21snmpd07:32:530
220759953,1cyclictest12747-21snmpd07:21:570
220759953,1cyclictest12747-21snmpd07:15:330
220759953,1cyclictest12747-21snmpd07:11:060
220759952,2cyclictest12747-21snmpd11:30:430
220759952,2cyclictest12747-21snmpd10:38:070
220759951,3cyclictest0-21swapper/011:20:230
220759950,4cyclictest25258-21sshd11:50:410
220759950,4cyclictest10121-21sshd09:35:450
220769943,1cyclictest12747-21snmpd11:15:321
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional