You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-06-27 - 01:58

x86 Intel Xeon E3-1220 @3100 MHz, Linux 4.19.1-rt3 (Profile)

Latency plot of system in rack #2, slot #0
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the highest latencies:
System rack2slot0.osadl.org (updated Mon Jun 27, 2022 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
325632490,0sleep10-21swapper/122:26:131
152392480,0sleep2311rcuc/222:38:162
34742470,0sleep30-21swapper/323:15:373
259552470,0sleep30-21swapper/322:40:453
164312470,0sleep20-21swapper/221:57:552
90622440,2sleep0134699cyclictest21:31:370
19452430,0sleep20-21swapper/221:13:532
267472410,0sleep2321ktimersoftd/222:57:182
122423523,7sleep20-21swapper/219:09:502
117923413,7sleep10-21swapper/119:09:171
11562319,7sleep30-21swapper/319:08:593
12222308,7sleep00-21swapper/019:09:480
145802160,0sleep30-21swapper/323:50:423
278762140,0sleep20-21swapper/223:21:492
239772140,0sleep30-21swapper/300:33:523
1360991413,0cyclictest26771-21lspci00:01:552
136199130,1cyclictest614-21dbus-daemon21:33:273
135699132,10cyclictest20157-21consoletype23:35:581
163932120,0sleep10-21swapper/122:30:181
1361991210,1cyclictest1-21systemd23:32:573
136099120,1cyclictest27204-21sshd23:05:222
134699120,1cyclictest619-21systemd-logind23:44:270
136199119,1cyclictest0-21swapper/322:21:373
136199114,4cyclictest633-21polkitd22:34:343
136199114,2cyclictest41-21ksoftirqd/321:42:583
136199110,10cyclictest0-21swapper/322:59:173
136199110,10cyclictest0-21swapper/321:49:273
136199110,10cyclictest0-21swapper/321:15:563
136199110,0cyclictest0-21swapper/322:08:373
136099119,1cyclictest15970-21lspci00:07:222
136099110,10cyclictest0-21swapper/223:37:122
136099110,10cyclictest0-21swapper/222:43:522
136099110,10cyclictest0-21swapper/221:17:512
1356991110,0cyclictest0-21swapper/121:52:081
135699110,1cyclictest0-21swapper/122:52:481
135699110,10cyclictest0-21swapper/123:13:481
1346991110,0cyclictest0-21swapper/023:30:270
134699110,10cyclictest20958-21grep23:28:170
134699110,10cyclictest20958-21grep23:28:160
134699110,10cyclictest0-21swapper/023:19:270
134699110,10cyclictest0-21swapper/023:13:570
134699110,10cyclictest0-21swapper/023:00:470
134699110,10cyclictest0-21swapper/022:09:160
134699110,10cyclictest0-21swapper/021:14:370
136199109,0cyclictest0-21swapper/323:46:073
136199109,0cyclictest0-21swapper/323:37:363
136199109,0cyclictest0-21swapper/300:28:073
136199102,2cyclictest41-21ksoftirqd/300:17:373
136199101,3cyclictest41-21ksoftirqd/322:17:223
136199101,1cyclictest131rcu_sched23:23:553
136199100,0cyclictest0-21swapper/321:54:163
136099109,0cyclictest0-21swapper/223:45:422
136099109,0cyclictest0-21swapper/223:19:422
136099109,0cyclictest0-21swapper/222:33:122
136099109,0cyclictest0-21swapper/222:01:022
136099109,0cyclictest0-21swapper/221:45:222
136099109,0cyclictest0-21swapper/219:17:312
136099109,0cyclictest0-21swapper/200:30:322
136099100,0cyclictest0-21swapper/223:32:322
136099100,0cyclictest0-21swapper/222:11:422
136099100,0cyclictest0-21swapper/219:36:312
136099100,0cyclictest0-21swapper/200:39:222
136099100,0cyclictest0-21swapper/200:18:522
135699109,0cyclictest0-21swapper/123:51:571
135699109,0cyclictest0-21swapper/121:37:281
135699109,0cyclictest0-21swapper/100:06:081
135699100,9cyclictest0-21swapper/122:48:081
135699100,0cyclictest0-21swapper/122:20:381
135699100,0cyclictest0-21swapper/100:37:081
134699109,0cyclictest0-21swapper/023:22:470
134699109,0cyclictest0-21swapper/021:38:070
134699109,0cyclictest0-21swapper/000:25:070
134699102,4cyclictest1690-21crond22:34:590
134699100,9cyclictest0-21swapper/023:51:570
134699100,0cyclictest0-21swapper/023:08:260
134699100,0cyclictest0-21swapper/021:41:460
134699100,0cyclictest0-21swapper/021:21:270
13619998,0cyclictest0-21swapper/322:36:073
13619998,0cyclictest0-21swapper/321:38:273
13619998,0cyclictest0-21swapper/300:12:373
13619998,0cyclictest0-21swapper/300:08:473
13619991,3cyclictest614-21dbus-daemon00:03:543
13619991,2cyclictest41-21ksoftirqd/321:14:313
13619991,1cyclictest41-21ksoftirqd/323:56:573
13619991,1cyclictest41-21ksoftirqd/322:00:403
13619991,1cyclictest131rcu_sched22:47:053
13619990,3cyclictest616-21gdbus23:12:523
13619990,0cyclictest0-21swapper/323:06:573
13609998,0cyclictest970-21snmpd20:04:032
13609998,0cyclictest0-21swapper/223:40:022
13609998,0cyclictest0-21swapper/222:09:222
13609991,2cyclictest121rcu_preempt22:21:152
13569998,0cyclictest0-21swapper/123:47:181
13569998,0cyclictest0-21swapper/123:08:171
13569998,0cyclictest0-21swapper/123:02:481
13569998,0cyclictest0-21swapper/122:44:371
13569998,0cyclictest0-21swapper/122:04:081
13569998,0cyclictest0-21swapper/121:56:581
13569998,0cyclictest0-21swapper/100:22:281
13569998,0cyclictest0-21swapper/100:17:381
13569997,1cyclictest30627-21sshd21:20:481
13569990,0cyclictest0-21swapper/123:29:181
13569990,0cyclictest0-21swapper/123:29:171
13569990,0cyclictest0-21swapper/120:25:271
13469998,0cyclictest0-21swapper/023:36:570
13619987,0cyclictest0-21swapper/323:40:273
13619987,0cyclictest0-21swapper/321:59:073
13619981,2cyclictest131rcu_sched23:27:483
13619981,2cyclictest131rcu_sched23:27:483
13619981,2cyclictest131rcu_sched22:51:323
13619981,2cyclictest131rcu_sched21:21:453
13619981,1cyclictest131rcu_sched00:37:433
13619981,1cyclictest131rcu_sched00:20:563
13619980,1cyclictest614-21dbus-daemon23:04:033
13609987,0cyclictest0-21swapper/223:53:122
13609987,0cyclictest0-21swapper/222:27:122
13609987,0cyclictest0-21swapper/220:46:522
13609985,2cyclictest598-21avahi-daemon22:19:372
13609985,2cyclictest598-21avahi-daemon21:51:012
13609985,2cyclictest598-21avahi-daemon21:23:222
13609985,1cyclictest33-21ksoftirqd/221:26:482
13609981,1cyclictest33-21ksoftirqd/221:30:182
13609981,1cyclictest33-21ksoftirqd/200:23:152
13609981,1cyclictest131rcu_sched23:12:462
13609981,1cyclictest131rcu_sched22:45:252
13609981,1cyclictest131rcu_sched00:11:042
13609980,1cyclictest633-21polkitd21:42:542
13609980,1cyclictest121rcu_preempt19:15:002
13569987,0cyclictest0-21swapper/123:41:381
13569987,0cyclictest0-21swapper/123:17:081
13569987,0cyclictest0-21swapper/122:18:171
13569987,0cyclictest0-21swapper/121:43:371
13569987,0cyclictest0-21swapper/121:26:281
13569987,0cyclictest0-21swapper/121:10:281
13569987,0cyclictest0-21swapper/100:29:181
13569985,2cyclictest598-21avahi-daemon23:56:281
13569985,2cyclictest598-21avahi-daemon21:34:301
13469987,0cyclictest0-21swapper/000:39:470
13469987,0cyclictest0-21swapper/000:31:570
13469987,0cyclictest0-21swapper/000:20:170
13469986,1cyclictest0-21swapper/022:58:370
13469985,2cyclictest598-21avahi-daemon22:45:280
13469985,2cyclictest598-21avahi-daemon22:41:200
13469985,2cyclictest598-21avahi-daemon22:23:560
13469985,2cyclictest598-21avahi-daemon21:57:440
13619975,1cyclictest598-21avahi-daemon22:10:033
13619975,1cyclictest41-21ksoftirqd/319:45:013
13619974,2cyclictest970-21snmpd20:14:573
13619974,2cyclictest598-21avahi-daemon22:25:373
13619971,1cyclictest131rcu_sched21:25:423
13619970,3cyclictest614-21dbus-daemon20:29:543
13619970,0cyclictest131rcu_sched20:10:003
13609976,0cyclictest0-21swapper/220:24:322
13609976,0cyclictest0-21swapper/220:10:222
13609976,0cyclictest0-21swapper/219:51:312
13609976,0cyclictest0-21swapper/219:30:322
13609975,1cyclictest598-21avahi-daemon23:55:282
13609975,1cyclictest33-21ksoftirqd/220:54:592
13609975,1cyclictest33-21ksoftirqd/220:29:592
13609974,2cyclictest598-21avahi-daemon22:50:122
13609971,3cyclictest33-21ksoftirqd/200:25:102
13609971,1cyclictest33-21ksoftirqd/221:37:072
13609971,1cyclictest131rcu_sched23:25:452
13609971,1cyclictest131rcu_sched23:25:452
13609971,1cyclictest131rcu_sched23:01:252
13569976,0cyclictest0-21swapper/121:47:181
13569975,1cyclictest633-21polkitd23:21:051
13569975,1cyclictest633-21polkitd00:11:331
13569975,1cyclictest598-21avahi-daemon22:57:271
13569975,1cyclictest598-21avahi-daemon22:36:431
13569975,1cyclictest598-21avahi-daemon22:05:281
13569975,1cyclictest598-21avahi-daemon00:04:191
13569974,2cyclictest598-21avahi-daemon23:31:211
13569974,2cyclictest598-21avahi-daemon00:30:161
13469976,0cyclictest0-21swapper/023:56:370
13469975,1cyclictest633-21polkitd00:15:340
13469975,1cyclictest633-21polkitd00:02:200
13469975,1cyclictest598-21avahi-daemon22:50:340
13469975,1cyclictest598-21avahi-daemon22:12:230
13469975,1cyclictest598-21avahi-daemon22:01:020
13469975,1cyclictest598-21avahi-daemon21:45:080
13469975,1cyclictest598-21avahi-daemon00:12:030
13469975,1cyclictest27834-21sshd22:25:140
13469974,2cyclictest7694-21sshd23:49:150
13469974,2cyclictest598-21avahi-daemon21:52:110
13469974,2cyclictest598-21avahi-daemon21:25:330
13469974,2cyclictest598-21avahi-daemon21:15:080
13469974,2cyclictest598-21avahi-daemon00:07:040
13619964,1cyclictest41-21ksoftirqd/321:00:003
13619964,1cyclictest41-21ksoftirqd/320:20:003
13619963,2cyclictest970-21snmpd21:09:573
13619963,2cyclictest970-21snmpd20:37:133
13619963,1cyclictest970-21snmpd20:52:103
13619960,5cyclictest0-21swapper/320:34:173
13619960,5cyclictest0-21swapper/320:24:073
13619960,5cyclictest0-21swapper/319:33:063
13619960,5cyclictest0-21swapper/319:23:263
13619960,0cyclictest0-21swapper/319:10:453
13609965,0cyclictest0-21swapper/220:41:522
13609963,2cyclictest970-21snmpd21:00:362
13609960,5cyclictest0-21swapper/221:08:422
13609960,5cyclictest0-21swapper/220:57:222
13569964,1cyclictest643-21gdbus20:00:001
13569964,1cyclictest633-21polkitd22:10:481
13569964,1cyclictest614-21dbus-daemon21:10:001
13569964,1cyclictest598-21avahi-daemon21:15:011
13569963,2cyclictest970-21snmpd19:37:541
13469964,1cyclictest614-21dbus-daemon22:15:260
13469964,1cyclictest598-21avahi-daemon22:35:100
13469963,2cyclictest970-21snmpd21:01:510
13469963,2cyclictest970-21snmpd20:50:520
13469963,2cyclictest970-21snmpd20:48:020
13469963,2cyclictest970-21snmpd20:34:350
13469963,2cyclictest970-21snmpd20:21:490
13469963,2cyclictest970-21snmpd20:09:390
13469963,2cyclictest970-21snmpd19:54:120
13469963,2cyclictest14659-21sed19:40:110
13619954,0cyclictest0-21swapper/320:48:173
13619954,0cyclictest0-21swapper/319:46:563
13619953,1cyclictest970-21snmpd20:00:003
13619953,1cyclictest970-21snmpd19:56:313
13619953,1cyclictest970-21snmpd19:52:043
13619953,1cyclictest970-21snmpd19:36:563
13619953,1cyclictest6822-21cat20:40:133
13619952,2cyclictest970-21snmpd19:27:143
13619952,2cyclictest614-21dbus-daemon19:15:013
13619952,2cyclictest16428-21latency_hist21:05:003
13609953,1cyclictest970-21snmpd20:36:022
13609953,1cyclictest970-21snmpd19:45:572
13609953,1cyclictest970-21snmpd19:41:092
13609953,1cyclictest970-21snmpd19:26:202
13609953,1cyclictest1071-21lldpd20:15:032
13609952,2cyclictest3091-21cut20:30:162
13609952,2cyclictest1078-21ntpd20:05:162
13609950,4cyclictest6159-21/usr/sbin/munin19:20:162
13609950,1cyclictest0-21swapper/219:55:112
13569953,1cyclictest970-21snmpd21:01:541
13569953,1cyclictest970-21snmpd20:45:051
13569953,1cyclictest970-21snmpd20:43:251
13569953,1cyclictest970-21snmpd20:32:171
13569953,1cyclictest970-21snmpd20:22:131
13569953,1cyclictest970-21snmpd20:16:391
13569953,1cyclictest970-21snmpd19:40:271
13569953,1cyclictest970-21snmpd19:31:061
13569953,1cyclictest8678-21cat19:25:131
13569953,1cyclictest6476-21grep19:20:141
13569953,1cyclictest598-21avahi-daemon19:54:041
13569953,1cyclictest2401-21ntp_states19:10:161
13569953,1cyclictest16746-21ls19:45:141
13569953,1cyclictest13354-21open_files20:55:171
13569952,2cyclictest970-21snmpd20:11:341
13569950,2cyclictest4852-21/usr/sbin/munin20:35:141
13469954,0cyclictest0-21swapper/020:04:170
13469954,0cyclictest0-21swapper/019:26:560
13469953,1cyclictest970-21snmpd20:25:110
13469953,1cyclictest970-21snmpd20:15:030
13469953,1cyclictest970-21snmpd20:13:310
13469953,1cyclictest970-21snmpd19:56:490
13469953,1cyclictest970-21snmpd19:34:000
13469953,1cyclictest970-21snmpd19:24:210
13469953,1cyclictest970-21snmpd19:19:200
13469953,1cyclictest970-21snmpd19:13:360
13469953,1cyclictest616-21gdbus19:45:010
13469953,1cyclictest598-21avahi-daemon21:05:220
13469953,1cyclictest598-21avahi-daemon19:35:090
13469952,2cyclictest970-21snmpd20:55:590
13469950,4cyclictest0-21swapper/020:39:270
13569942,1cyclictest25090-21load20:05:151
13569942,1cyclictest22427-21cpu20:00:091
13569942,1cyclictest0-21swapper/120:50:101
13569942,1cyclictest0-21swapper/119:15:011
13469944,0cyclictest28287-21kworker/0:020:40:090
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional