You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2021-04-15 - 00:13

Intel(R) Core(TM) i7 CPU E 610 @ 2.53GHz, Linux 4.9.47-rt37 (Profile)

Latency plot of system in rack #5, slot #4
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot4.osadl.org (updated Wed Apr 14, 2021 12:44:24)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
27652210292,5sleep10-21swapper/107:07:471
2688429990,4sleep20-21swapper/207:05:122
2751629890,4sleep30-21swapper/307:06:053
2751328174,4sleep00-21swapper/007:06:020
2806399270,26cyclictest0-21swapper/307:50:023
2806399260,25cyclictest0-21swapper/308:36:183
2806399260,25cyclictest0-21swapper/308:11:323
2806399260,25cyclictest0-21swapper/308:08:023
2806399260,20cyclictest0-21swapper/311:38:333
2806399250,24cyclictest0-21swapper/308:33:183
2806399250,19cyclictest0-21swapper/310:22:173
2806399250,19cyclictest0-21swapper/310:05:473
2806399250,19cyclictest0-21swapper/307:35:333
28063992422,1cyclictest0-21swapper/312:25:033
28063992322,1cyclictest0-21swapper/310:10:023
28063992322,1cyclictest0-21swapper/307:45:473
203662230,4sleep12806199cyclictest10:40:081
2806299223,18cyclictest0-21swapper/208:45:572
28062992220,1cyclictest0-21swapper/209:09:022
2806299221,19cyclictest0-21swapper/211:00:072
2806399210,20cyclictest0-21swapper/310:55:033
2806399210,20cyclictest0-21swapper/309:49:483
28062992118,2cyclictest3266-21lldpd07:14:132
2806299210,2cyclictest0-21swapper/210:55:062
28060992118,2cyclictest3257-21snmpd09:41:020
28060992118,2cyclictest3257-21snmpd07:38:020
2806399207,8cyclictest0-21swapper/310:00:123
2806399207,8cyclictest0-21swapper/309:55:113
28063992018,1cyclictest0-21swapper/311:25:483
2806399200,2cyclictest0-21swapper/309:37:033
2806399200,1cyclictest0-21swapper/309:08:473
2806399200,19cyclictest0-21swapper/312:08:473
2806399200,19cyclictest0-21swapper/311:40:043
2806399200,19cyclictest0-21swapper/311:24:323
2806399200,19cyclictest0-21swapper/309:23:173
2806399200,19cyclictest0-21swapper/309:19:473
2806399200,19cyclictest0-21swapper/308:40:483
2806399200,19cyclictest0-21swapper/307:41:473
2806399200,19cyclictest0-21swapper/307:26:023
28062992018,1cyclictest3257-21snmpd11:18:482
28062992018,1cyclictest3257-21snmpd10:08:172
28062992018,1cyclictest3257-21snmpd08:22:032
28062992018,1cyclictest3257-21snmpd08:15:332
28062992018,1cyclictest0-21swapper/207:45:062
2806299200,4cyclictest0-21swapper/212:26:292
2806299200,3cyclictest0-21swapper/209:16:502
2806299200,19cyclictest0-21swapper/210:35:072
28061992017,2cyclictest3266-21lldpd07:44:131
28061992017,2cyclictest0-21swapper/111:57:031
28061992017,2cyclictest0-21swapper/109:00:071
28061992017,2cyclictest0-21swapper/108:07:031
28061992017,2cyclictest0-21swapper/107:51:471
28060992018,1cyclictest3257-21snmpd10:43:320
28060992018,1cyclictest3257-21snmpd08:54:330
28060992018,1cyclictest3257-21snmpd08:35:470
28060992017,2cyclictest3257-21snmpd10:58:030
28060992017,2cyclictest3257-21snmpd08:43:320
28060992017,2cyclictest3257-21snmpd07:40:170
28060992017,2cyclictest3257-21snmpd07:17:320
2806399196,8cyclictest0-21swapper/312:35:143
28063991917,1cyclictest4077-21cat08:45:063
28063991917,1cyclictest27240-21cat09:40:073
28063991916,2cyclictest3257-21snmpd09:34:333
2806399190,2cyclictest0-21swapper/308:52:473
2806399190,1cyclictest0-21swapper/311:10:173
2806399190,1cyclictest0-21swapper/310:53:483
2806399190,1cyclictest0-21swapper/307:14:133
2806399190,18cyclictest0-21swapper/312:32:033
2806399190,18cyclictest0-21swapper/311:48:033
2806399190,18cyclictest0-21swapper/310:37:483
2806399190,18cyclictest0-21swapper/307:17:343
2806399190,13cyclictest0-21swapper/310:32:183
28062991917,1cyclictest3257-21snmpd11:06:322
28062991917,1cyclictest3257-21snmpd10:51:472
28062991917,1cyclictest3257-21snmpd07:50:172
28062991917,1cyclictest3257-21snmpd07:35:172
28062991917,1cyclictest0-21swapper/211:44:142
28062991916,2cyclictest3257-21snmpd07:19:472
2806299191,2cyclictest0-21swapper/210:30:032
2806299191,2cyclictest0-21swapper/207:25:032
2806299190,2cyclictest0-21swapper/212:05:032
28061991917,1cyclictest0-21swapper/112:12:481
28061991917,1cyclictest0-21swapper/108:58:171
28061991917,1cyclictest0-21swapper/108:49:171
28061991917,1cyclictest0-21swapper/107:17:321
28061991916,2cyclictest0-21swapper/109:42:321
28061991916,2cyclictest0-21swapper/108:35:321
28061991915,3cyclictest0-21swapper/110:35:021
28060991917,1cyclictest3257-21snmpd09:46:480
28060991917,1cyclictest3257-21snmpd07:51:480
28060991916,2cyclictest3257-21snmpd11:57:030
28060991916,2cyclictest3257-21snmpd11:43:480
28060991916,2cyclictest3257-21snmpd11:31:170
28060991916,2cyclictest3257-21snmpd08:58:180
28060991916,2cyclictest3257-21snmpd08:49:170
28060991916,2cyclictest3257-21snmpd08:07:020
28063991817,1cyclictest0-21swapper/311:08:473
28063991816,1cyclictest0-21swapper/312:22:333
28063991816,1cyclictest0-21swapper/309:27:323
2806399180,17cyclictest0-21swapper/309:52:323
2806399180,17cyclictest0-21swapper/308:19:023
2806299184,8cyclictest6033-21munin-node2
28062991816,1cyclictest3257-21snmpd12:35:482
28062991815,2cyclictest3266-21lldpd12:14:142
2806299180,2cyclictest0-21swapper/207:20:032
28061991816,1cyclictest0-21swapper/111:43:471
28061991816,1cyclictest0-21swapper/111:31:171
28061991815,2cyclictest0-21swapper/111:46:471
28061991815,2cyclictest0-21swapper/108:16:321
28060991816,1cyclictest10459-21cat09:00:070
28060991815,2cyclictest3257-21snmpd10:23:020
28060991815,2cyclictest25562-21cat12:10:060
28060991814,2cyclictest3257-21snmpd10:35:020
2806399178,8cyclictest0-21swapper/310:45:043
2806399174,8cyclictest0-21swapper/308:20:093
28063991716,1cyclictest0-21swapper/307:57:323
2806399170,1cyclictest0-21swapper/312:18:023
2806399170,1cyclictest0-21swapper/308:25:173
2806299175,8cyclictest13174-21munin-node2
2806299174,8cyclictest873-21munin-node2
2806299174,8cyclictest0-21swapper/208:35:062
2806299174,8cyclictest0-21swapper/207:30:122
2806299173,9cyclictest3856-21munin-node2
28062991715,1cyclictest3257-21snmpd11:36:482
28062991715,1cyclictest3257-21snmpd09:55:472
28062991715,1cyclictest3257-21snmpd09:49:472
28062991715,1cyclictest3257-21snmpd09:00:472
28062991715,1cyclictest0-21swapper/208:25:082
2806299170,4cyclictest0-21swapper/210:40:042
2806299170,12cyclictest0-21swapper/209:22:342
28061991715,2cyclictest0-21swapper/108:22:321
28061991715,1cyclictest0-21swapper/110:23:031
28061991714,2cyclictest14648-21cat09:10:091
28061991714,2cyclictest0-21swapper/112:05:181
28061991714,2cyclictest0-21swapper/111:15:481
28061991714,2cyclictest0-21swapper/108:53:021
28061991714,2cyclictest0-21swapper/108:10:081
28060991714,2cyclictest3257-21snmpd12:05:180
28060991714,2cyclictest3257-21snmpd08:16:320
28060991714,2cyclictest21784-21cat08:10:070
2806299163,8cyclictest7701-21munin-node2
2806299163,8cyclictest3257-21snmpd12:00:072
2806299163,8cyclictest0-21swapper/208:00:502
28062991614,1cyclictest3257-21snmpd11:48:022
28062991614,1cyclictest3257-21snmpd09:52:322
28062991614,1cyclictest3257-21snmpd09:40:172
28062991614,1cyclictest3257-21snmpd07:58:172
2806299160,4cyclictest0-21swapper/211:13:212
28061991613,2cyclictest0-21swapper/110:15:031
28061991613,2cyclictest0-21swapper/107:39:021
28060991614,1cyclictest3257-21snmpd11:46:480
28060991614,1cyclictest3257-21snmpd11:15:470
28060991614,1cyclictest3257-21snmpd08:22:320
2806399150,14cyclictest0-21swapper/311:30:183
2806399150,14cyclictest0-21swapper/311:00:113
2806399150,14cyclictest0-21swapper/308:00:023
2806399150,14cyclictest0-21swapper/307:30:083
2806299153,7cyclictest22391-21munin-node2
28062991513,1cyclictest0-21swapper/210:24:142
28062991512,2cyclictest31-21ksoftirqd/208:10:022
2806299151,13cyclictest31-21ksoftirqd/208:08:512
2806299151,13cyclictest29-21rcuc/211:50:072
2806299151,13cyclictest0-21swapper/210:45:382
2806299151,13cyclictest0-21swapper/210:04:322
2806299150,14cyclictest0-21swapper/207:40:292
2806299150,13cyclictest29132-21munin-node2
28061991513,1cyclictest0-21swapper/108:27:471
28061991512,2cyclictest0-21swapper/111:35:021
28060991513,1cyclictest0-21swapper/009:10:080
28060991512,2cyclictest3257-21snmpd10:15:020
28060991512,2cyclictest3257-21snmpd08:27:470
66682141,3sleep00-21swapper/010:10:010
2806399146,7cyclictest0-21swapper/310:15:063
2806399140,8cyclictest0-21swapper/307:20:093
2806399140,13cyclictest0-21swapper/312:10:243
2806399140,13cyclictest0-21swapper/312:00:053
2806399140,13cyclictest0-21swapper/311:55:023
2806399140,13cyclictest0-21swapper/311:50:043
2806399140,13cyclictest0-21swapper/311:15:023
2806399140,13cyclictest0-21swapper/310:40:123
2806399140,13cyclictest0-21swapper/310:25:043
2806399140,13cyclictest0-21swapper/309:10:153
2806399140,13cyclictest0-21swapper/309:00:093
2806399140,13cyclictest0-21swapper/308:55:593
2806299141,8cyclictest8123-21munin-node2
2806299140,8cyclictest31-21ksoftirqd/210:15:032
2806299140,8cyclictest0-21swapper/209:35:032
2806299140,13cyclictest29-21rcuc/210:10:032
2806299140,13cyclictest0-21swapper/212:15:032
2806299140,13cyclictest0-21swapper/211:55:062
2806299140,13cyclictest0-21swapper/209:25:072
2806299140,13cyclictest0-21swapper/209:10:052
2806299140,13cyclictest0-21swapper/208:50:452
2806299140,13cyclictest0-21swapper/208:40:072
2806299140,13cyclictest0-21swapper/208:32:372
28061991411,2cyclictest0-21swapper/109:28:021
28060991411,2cyclictest3257-21snmpd11:50:180
28061991311,1cyclictest0-21swapper/111:50:171
28061991311,1cyclictest0-21swapper/110:56:171
28061991310,2cyclictest14069-21ls10:25:091
28061991310,2cyclictest0-21swapper/112:22:181
2806099135,6cyclictest3257-21snmpd07:55:030
28060991311,1cyclictest3257-21snmpd11:35:020
28060991311,1cyclictest3257-21snmpd09:28:020
28060991310,2cyclictest3257-21snmpd12:22:170
28060991310,2cyclictest3257-21snmpd10:48:180
2806199129,2cyclictest0-21swapper/107:29:471
2806199125,6cyclictest14618-21copy07:55:021
28061991210,1cyclictest0-21swapper/110:48:171
2806199120,2cyclictest32057-21apt-get11:10:031
2806199120,2cyclictest0-21swapper/110:05:121
2806199120,11cyclictest0-21swapper/109:15:121
2806099129,2cyclictest3257-21snmpd10:14:470
28060991210,1cyclictest0-21swapper/009:50:090
2806099120,1cyclictest5289-21awk07:30:140
2806099120,11cyclictest0-21swapper/010:50:110
2806099120,11cyclictest0-21swapper/010:00:140
2806099120,11cyclictest0-21swapper/007:45:130
2806199119,2cyclictest0-21swapper/109:54:471
2806199119,2cyclictest0-21swapper/107:24:171
2806199119,1cyclictest0-21swapper/110:14:481
2806199118,2cyclictest0-21swapper/109:37:321
2806099119,1cyclictest25718-21ls09:35:120
2806099119,1cyclictest0-21swapper/008:30:020
2806099119,1cyclictest0-21swapper/008:00:130
2806099110,1cyclictest0-21swapper/012:25:030
2806099110,1cyclictest0-21swapper/007:20:040
2806199107,2cyclictest0-21swapper/111:25:071
2806199101,8cyclictest0-21swapper/110:50:101
2806199101,8cyclictest0-21swapper/110:30:041
2806199101,8cyclictest0-21swapper/109:55:131
2806199101,3cyclictest0-21swapper/111:20:121
2806199100,9cyclictest28065-21munin-node1
2806199100,9cyclictest0-21swapper/111:05:171
2806199100,2cyclictest2335-21apt-get10:00:061
2806099108,1cyclictest3257-21snmpd07:29:470
2806099108,1cyclictest0-21swapper/011:25:110
2806099108,1cyclictest0-21swapper/009:55:150
2806099107,2cyclictest3257-21snmpd12:30:170
2806099107,2cyclictest11904-21munin-node0
2806099100,9cyclictest0-21swapper/009:15:060
280619997,1cyclictest0-21swapper/109:30:051
280619996,2cyclictest230050irq/25-eth008:40:051
280619996,2cyclictest10502-21munin-node1
280619996,2cyclictest0-21swapper/112:15:031
280619996,2cyclictest0-21swapper/109:06:181
280619995,3cyclictest0-21swapper/108:30:061
280619991,7cyclictest0-21swapper/108:00:051
280619991,7cyclictest0-21swapper/107:30:091
280619991,3cyclictest0-21swapper/112:30:091
280619990,8cyclictest0-21swapper/109:45:031
280609996,2cyclictest3257-21snmpd09:30:020
280609996,2cyclictest3257-21snmpd07:10:170
280609990,8cyclictest0-21swapper/011:00:150
280609990,3cyclictest13174-21munin-node0
280609990,2cyclictest18204-21munin-node0
21713290,1sleep10-21swapper/112:00:131
280619985,2cyclictest18851-21cat09:20:061
280619984,3cyclictest0-21swapper/112:36:171
280619980,7cyclictest27863-21munin-node1
280609985,2cyclictest3257-21snmpd11:23:470
280609985,2cyclictest230050irq/25-eth011:05:300
280609980,2cyclictest32053-21munin-node0
280619970,6cyclictest0-21swapper/112:25:021
280609974,2cyclictest27044-21basename12:15:030
280609970,6cyclictest3012-21aten_r5power_cu12:35:030
280609970,6cyclictest0-21swapper/010:30:020
280609970,2cyclictest20715-21munin-node0
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional