You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2021-10-23 - 18:38

Intel(R) Core(TM) i7 CPU E 610 @ 2.53GHz, Linux 5.15.0-rc4-rt7 (Profile)

Latency plot of system in rack #5, slot #4
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot4.osadl.org (updated Tue Jul 06, 2021 12:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2611629990,5sleep20-21swapper/207:07:302
2611928273,5sleep00-21swapper/007:07:310
186372750,1sleep1231ktimersoftd/112:03:521
186372750,1sleep1231ktimersoftd/112:03:521
2620127457,13sleep30-21swapper/307:08:353
225782740,0sleep20-21swapper/211:00:352
2619427257,10sleep10-21swapper/107:08:291
159142710,1sleep00-21swapper/012:01:280
159142710,1sleep00-21swapper/012:01:270
146342700,1sleep20-21swapper/209:30:182
178382690,0sleep20-21swapper/210:58:022
17592520,0sleep30-21swapper/307:25:293
186572310,0sleep018658-21ssh10:58:430
316272260,0sleep331628-21ssh11:29:083
2656799251,16cyclictest8064-21ssh11:35:102
2656799232,19cyclictest0-21swapper/210:10:172
2656699229,12cyclictest0-21swapper/109:56:481
2656699226,14cyclictest30317-21ssh12:33:321
154402220,1sleep10-21swapper/109:51:521
2656699216,2cyclictest28438-21ssh11:48:421
2656699216,2cyclictest13453-21ssh11:16:151
2656699216,13cyclictest9521-21ssh11:57:221
2656699216,13cyclictest0-21swapper/111:05:281
35502200,0sleep20-21swapper/209:44:532
2656799201,3cyclictest15160-21ssh10:35:102
2656699206,8cyclictest7887-21ssh09:46:581
2656699206,8cyclictest0-21swapper/111:43:321
2656699206,8cyclictest0-21swapper/110:55:021
2656699206,13cyclictest0-21swapper/112:17:551
2656699206,13cyclictest0-21swapper/111:02:441
2656699206,13cyclictest0-21swapper/110:42:481
2656699206,13cyclictest0-21swapper/110:31:341
2656699206,13cyclictest0-21swapper/109:34:461
2656699205,9cyclictest21985-21ssh10:39:451
2656699205,9cyclictest13555-21ssh11:38:391
2656699205,14cyclictest0-21swapper/112:05:501
2656699204,14cyclictest0-21swapper/110:54:581
26566992017,2cyclictest0-21swapper/110:14:451
2656699200,1cyclictest0-21swapper/112:26:121
2656699196,8cyclictest0-21swapper/109:24:551
2656699196,12cyclictest0-21swapper/110:07:011
2656699195,8cyclictest0-21swapper/112:37:061
2656699195,13cyclictest0-21swapper/111:22:511
2656699195,13cyclictest0-21swapper/110:48:231
2656699195,13cyclictest0-21swapper/110:24:481
2656699195,13cyclictest0-21swapper/110:24:471
2656699195,13cyclictest0-21swapper/109:15:101
2656699185,8cyclictest0-21swapper/110:00:351
2656699185,12cyclictest0-21swapper/110:27:241
2656699185,12cyclictest0-21swapper/109:28:131
2656699184,9cyclictest0-21swapper/112:24:311
2656699184,13cyclictest0-21swapper/109:42:201
2656899171,11cyclictest0-21swapper/309:57:003
2656699176,10cyclictest0-21swapper/111:51:551
26566991714,2cyclictest0-21swapper/109:13:091
26565991714,2cyclictest198050irq/24-eth012:20:100
26565991714,2cyclictest198050irq/24-eth012:11:000
26565991714,2cyclictest198050irq/24-eth011:38:120
26565991714,2cyclictest198050irq/24-eth011:26:550
26565991714,2cyclictest198050irq/24-eth011:18:370
26565991714,2cyclictest198050irq/24-eth011:10:170
26565991714,2cyclictest198050irq/24-eth011:08:180
26565991714,2cyclictest198050irq/24-eth011:02:390
26565991714,2cyclictest198050irq/24-eth010:47:200
26565991714,2cyclictest198050irq/24-eth010:29:160
26565991714,2cyclictest198050irq/24-eth010:24:550
26565991714,2cyclictest198050irq/24-eth010:24:550
26565991714,2cyclictest198050irq/24-eth010:15:480
26565991714,2cyclictest198050irq/24-eth009:34:060
26565991714,2cyclictest198050irq/24-eth009:25:000
26565991714,2cyclictest198050irq/24-eth009:21:310
26565991714,2cyclictest198050irq/24-eth009:13:580
26565991713,2cyclictest198050irq/24-eth009:52:030
2656899160,14cyclictest0-21swapper/311:40:173
2656899160,14cyclictest0-21swapper/308:51:013
2656799162,10cyclictest3000-21ssh10:48:172
26567991612,2cyclictest0-21swapper/210:50:162
2656799160,14cyclictest0-21swapper/212:35:112
2656699167,7cyclictest0-21swapper/111:11:221
26566991614,1cyclictest0-21swapper/112:10:371
26565991614,1cyclictest198050irq/24-eth012:30:330
26565991614,1cyclictest198050irq/24-eth010:35:090
26565991614,1cyclictest198050irq/24-eth009:55:340
26565991613,2cyclictest198050irq/24-eth012:35:090
26565991613,2cyclictest198050irq/24-eth012:28:150
26565991613,2cyclictest198050irq/24-eth012:17:400
26565991613,2cyclictest198050irq/24-eth012:08:250
26565991613,2cyclictest198050irq/24-eth011:55:140
26565991613,2cyclictest198050irq/24-eth011:52:150
26565991613,2cyclictest198050irq/24-eth011:47:130
26565991613,2cyclictest198050irq/24-eth011:41:150
26565991613,2cyclictest198050irq/24-eth011:31:060
26565991613,2cyclictest198050irq/24-eth010:52:560
26565991613,2cyclictest198050irq/24-eth010:41:440
26565991613,2cyclictest198050irq/24-eth010:32:290
26565991613,2cyclictest198050irq/24-eth010:11:210
26565991613,2cyclictest198050irq/24-eth010:05:470
26565991613,2cyclictest198050irq/24-eth010:00:440
26565991613,2cyclictest198050irq/24-eth009:40:390
26565991613,2cyclictest198050irq/24-eth009:39:450
26565991613,2cyclictest198050irq/24-eth009:15:260
26565991612,2cyclictest198050irq/24-eth011:20:460
279662150,2sleep230-21rcuc/209:18:002
2656899150,13cyclictest0-21swapper/312:06:333
2656799151,2cyclictest21615-21sshd09:55:532
26567991512,2cyclictest0-21swapper/208:50:172
2656799150,8cyclictest0-21swapper/209:35:152
2656799150,13cyclictest29925-21aten_r5power_po08:30:122
2656699155,4cyclictest0-21swapper/111:31:181
26566991512,2cyclictest0-21swapper/111:27:461
26565991512,2cyclictest198050irq/24-eth009:45:210
26568991411,2cyclictest0-21swapper/309:30:173
26568991410,2cyclictest5674-21expr07:35:123
2656899140,3cyclictest10111-21sed08:55:233
2656899140,2cyclictest0-21swapper/309:45:143
2656899140,13cyclictest0-21swapper/312:34:183
2656799142,7cyclictest9866-21ssh12:20:002
26567991411,2cyclictest0-21swapper/207:32:432
2656799141,11cyclictest31617-21ssh11:29:082
2656799140,8cyclictest0-21swapper/209:21:202
2656799140,8cyclictest0-21swapper/208:40:222
2656799140,2cyclictest0-21swapper/208:39:302
2656799140,13cyclictest0-21swapper/208:08:292
2656799140,12cyclictest0-21swapper/211:10:112
2656699145,8cyclictest0-21swapper/109:35:521
2656699145,2cyclictest20145-21ssh10:16:191
26565991412,1cyclictest198050irq/24-eth008:59:070
26565991412,1cyclictest198050irq/24-eth007:21:440
26565991411,2cyclictest198050irq/24-eth009:05:270
2656599140,3cyclictest3699-21cat07:30:160
2656899131,8cyclictest14984-21diskmemload10:39:463
2656899131,6cyclictest0-21swapper/308:05:013
26568991310,2cyclictest0-21swapper/311:55:163
26568991310,2cyclictest0-21swapper/311:50:133
26568991310,2cyclictest0-21swapper/311:15:193
2656899130,1cyclictest0-21swapper/311:00:173
2656899130,11cyclictest0-21swapper/312:10:153
2656899130,11cyclictest0-21swapper/307:58:413
2656799139,3cyclictest14526-21ssh10:34:492
2656799139,2cyclictest0-21swapper/210:02:332
2656799132,6cyclictest0-21swapper/207:20:012
2656799131,9cyclictest0-21swapper/208:15:302
2656799131,5cyclictest3232-21snmpd11:45:012
2656799131,2cyclictest7818-21ssh09:46:552
2656799131,2cyclictest3468-21ssh10:26:522
26567991310,2cyclictest14984-21diskmemload10:05:522
2656799130,2cyclictest30559-21cpuspeed11:50:142
2656799130,2cyclictest0-21swapper/212:14:212
2656799130,2cyclictest0-21swapper/211:05:572
2656799130,11cyclictest0-21swapper/211:31:282
2656699130,12cyclictest4082-21munin-node1
2656599139,2cyclictest0-21swapper/008:21:150
26565991310,2cyclictest26591-21apt-get07:10:160
26565991310,2cyclictest0-21swapper/007:44:450
2656899129,2cyclictest23483-21cstates08:15:203
2656899129,2cyclictest0-21swapper/311:10:213
2656899129,2cyclictest0-21swapper/309:27:013
2656899129,2cyclictest0-21swapper/309:15:163
2656899123,5cyclictest0-21swapper/310:30:573
2656899122,6cyclictest0-21swapper/308:35:003
2656899122,5cyclictest0-21swapper/311:00:003
2656899120,2cyclictest1400-21ssh10:04:493
2656899120,11cyclictest0-21swapper/311:32:313
2656899120,11cyclictest0-21swapper/310:45:143
2656899120,10cyclictest0-21swapper/310:21:433
2656899120,10cyclictest0-21swapper/310:21:433
2656799129,2cyclictest0-21swapper/212:32:002
2656799129,2cyclictest0-21swapper/210:41:462
2656799121,6cyclictest0-21swapper/207:50:012
26567991210,1cyclictest0-21swapper/212:25:032
2656799120,2cyclictest0-21swapper/211:45:242
2656799120,1cyclictest0-21swapper/209:14:002
2656799120,11cyclictest12994-21apt-get12:00:152
2656799120,11cyclictest12994-21apt-get12:00:142
2656799120,11cyclictest11240-21ssh09:28:162
2656799120,11cyclictest0-21swapper/212:21:532
2656799120,11cyclictest0-21swapper/212:07:372
2656799120,11cyclictest0-21swapper/211:55:152
2656799120,11cyclictest0-21swapper/211:22:262
2656799120,11cyclictest0-21swapper/211:15:372
2656799120,11cyclictest0-21swapper/210:23:202
2656799120,11cyclictest0-21swapper/210:23:202
2656799120,11cyclictest0-21swapper/210:18:562
2656799120,11cyclictest0-21swapper/209:50:452
2656799120,11cyclictest0-21swapper/209:00:162
2656799120,11cyclictest0-21swapper/208:20:132
2656699129,2cyclictest0-21swapper/109:00:151
2656699120,11cyclictest17314-21ps08:00:191
2656699120,11cyclictest0-21swapper/107:30:131
26565991210,1cyclictest198050irq/24-eth008:50:070
65472110,2sleep36569-21ssh10:07:383
2656899118,2cyclictest0-21swapper/307:15:393
2656899117,2cyclictest8136-21hddtemp_smartct07:40:193
2656899117,2cyclictest0-21swapper/310:29:313
2656899117,2cyclictest0-21swapper/309:05:153
2656899114,5cyclictest0-21swapper/312:21:513
2656899112,6cyclictest0-21swapper/309:40:013
2656899112,5cyclictest0-21swapper/308:30:003
2656899112,5cyclictest0-21swapper/307:50:003
2656899111,8cyclictest0-21swapper/311:49:163
2656899111,8cyclictest0-21swapper/309:35:063
2656899111,6cyclictest0-21swapper/312:25:513
2656899111,6cyclictest0-21swapper/308:14:113
2656899110,9cyclictest0-21swapper/310:16:163
2656899110,7cyclictest0-21swapper/307:21:043
2656899110,2cyclictest14984-21diskmemload09:51:053
2656899110,1cyclictest0-21swapper/312:00:283
2656899110,1cyclictest0-21swapper/312:00:283
2656899110,1cyclictest0-21swapper/311:35:123
2656899110,10cyclictest0-21swapper/312:15:463
2656899110,10cyclictest0-21swapper/310:50:123
2656899110,10cyclictest0-21swapper/310:41:253
2656899110,10cyclictest0-21swapper/309:20:193
2656799110,9cyclictest27841-21python07:10:292
2656799110,4cyclictest0-21swapper/207:40:002
2656799110,1cyclictest0-21swapper/208:55:212
2656799110,1cyclictest0-21swapper/208:10:192
2656799110,1cyclictest0-21swapper/207:20:292
2656799110,10cyclictest0-21swapper/209:05:162
2656699119,1cyclictest0-21swapper/108:15:261
2656699118,2cyclictest0-21swapper/108:59:071
2656699116,3cyclictest3232-21snmpd07:44:451
2656699110,10cyclictest0-21swapper/107:10:201
2656599119,1cyclictest198050irq/24-eth008:29:420
2656599119,1cyclictest198050irq/24-eth007:27:240
2656599118,2cyclictest0-21swapper/008:40:170
2656599117,2cyclictest198050irq/24-eth008:30:120
2656599111,8cyclictest0-21swapper/007:50:240
2656599111,2cyclictest0-21swapper/007:58:350
2656599110,9cyclictest0-21swapper/008:00:170
2656599110,10cyclictest0-21swapper/007:35:160
2656899107,2cyclictest0-21swapper/307:10:313
2656899106,2cyclictest3232-21snmpd08:48:463
2656899106,2cyclictest3232-21snmpd07:52:453
2656899102,5cyclictest0-21swapper/310:10:533
2656899100,8cyclictest0-21swapper/312:35:423
2656899100,8cyclictest0-21swapper/311:21:033
2656899100,8cyclictest0-21swapper/311:05:093
2656899100,8cyclictest0-21swapper/309:12:063
2656799100,8cyclictest0-21swapper/207:58:412
2656799100,1cyclictest0-21swapper/207:25:302
2656699108,1cyclictest0-21swapper/108:35:161
2656699108,1cyclictest0-21swapper/107:35:151
2656699107,2cyclictest27637-21munin-node1
2656699107,2cyclictest0-21swapper/107:21:441
2656699105,3cyclictest3232-21snmpd08:21:151
2656599107,2cyclictest198050irq/24-eth008:35:060
2656599107,2cyclictest198050irq/24-eth008:15:300
2656599107,2cyclictest198050irq/24-eth008:10:150
2656599107,2cyclictest198050irq/24-eth007:45:350
2656599107,2cyclictest198050irq/24-eth007:15:180
2656599100,3cyclictest4270-21cpuspeed08:45:130
131192100,2sleep1231ktimersoftd/109:05:131
265689994,3cyclictest27438-21cat08:25:003
265689991,6cyclictest0-21swapper/308:42:333
265689991,6cyclictest0-21swapper/307:31:233
265689990,2cyclictest0-21swapper/308:05:203
265679996,2cyclictest7598-21apt-get07:40:152
265679996,2cyclictest3232-21snmpd07:54:302
265679995,2cyclictest6142-21sh08:50:012
265679994,3cyclictest29612-21cron08:30:002
265669996,2cyclictest0-21swapper/108:40:231
265669996,2cyclictest0-21swapper/108:30:121
265669990,2cyclictest0-21swapper/108:50:181
265669990,2cyclictest0-21swapper/108:14:281
265669990,2cyclictest0-21swapper/107:54:221
265659996,2cyclictest198050irq/24-eth009:00:120
265659996,2cyclictest198050irq/24-eth008:05:160
265689983,3cyclictest32440-21sort08:35:133
265689983,3cyclictest11844-21ntp_kernel_pll_09:00:203
265679983,3cyclictest17355-21cut08:00:202
265669986,1cyclictest0-21swapper/107:45:141
265669986,1cyclictest0-21swapper/107:25:321
265669986,1cyclictest0-21swapper/107:15:361
265669980,7cyclictest0-21swapper/107:57:571
265669975,1cyclictest0-21swapper/108:08:591
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional