You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-06-26 - 13:25

x86 Intel Xeon E3-1220 @3100 MHz, Linux 4.19.1-rt3 (Profile)

Latency plot of system in rack #2, slot #0
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the alll highest latencies:
System rack2slot0.osadl.org (updated Sun Jun 26, 2022 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
84792520,0sleep10-21swapper/123:11:221
55852520,0sleep00-21swapper/022:54:340
214492520,0sleep10-21swapper/122:09:501
291362510,0sleep20-21swapper/223:16:342
276412500,0sleep10-21swapper/100:05:121
23972500,1sleep12398-21unixbench_singl21:40:221
96042490,0sleep20-21swapper/222:14:592
218932460,0sleep10-21swapper/123:31:001
278262450,0sleep20-21swapper/200:05:132
206252390,0sleep20-21swapper/200:27:432
2210823726,6sleep20-21swapper/219:05:072
2362723525,6sleep10-21swapper/119:06:341
2382322621,3sleep00-21swapper/019:09:010
5802220,1sleep233-21ksoftirqd/200:30:412
236582227,10sleep30-21swapper/319:06:573
282242180,0sleep00-21swapper/023:49:090
23998991311,1cyclictest643-21gdbus21:23:230
23998991210,1cyclictest0-21swapper/022:11:430
170362120,0sleep30-21swapper/321:35:593
2401499110,1cyclictest0-21swapper/321:16:473
2401499110,10cyclictest0-21swapper/322:27:563
2401499110,0cyclictest0-21swapper/323:20:073
2401499110,0cyclictest0-21swapper/300:17:073
2401199110,10cyclictest0-21swapper/223:06:172
2401199110,10cyclictest0-21swapper/222:51:462
2401199110,10cyclictest0-21swapper/221:55:072
2401199110,10cyclictest0-21swapper/219:30:162
2401199110,0cyclictest0-21swapper/221:38:362
2400599119,1cyclictest6903-21lspci21:33:391
2400599119,1cyclictest23102-21lspci22:58:571
2400599110,10cyclictest0-21swapper/123:53:441
2400599110,10cyclictest0-21swapper/122:23:351
2400599110,10cyclictest0-21swapper/121:25:041
23998991110,0cyclictest0-21swapper/000:13:330
2399899110,10cyclictest0-21swapper/023:28:230
2399899110,10cyclictest0-21swapper/021:26:020
2399899110,10cyclictest0-21swapper/000:38:530
2401499109,0cyclictest0-21swapper/323:56:273
2401499109,0cyclictest0-21swapper/323:12:263
2401499109,0cyclictest0-21swapper/322:48:563
2401499109,0cyclictest0-21swapper/322:43:273
2401499109,0cyclictest0-21swapper/322:31:173
2401499109,0cyclictest0-21swapper/321:51:473
2401499100,0cyclictest0-21swapper/322:55:173
2401499100,0cyclictest0-21swapper/322:02:363
2401199109,0cyclictest0-21swapper/222:20:572
2401199109,0cyclictest0-21swapper/221:47:462
2401199109,0cyclictest0-21swapper/221:33:072
2401199109,0cyclictest0-21swapper/200:36:472
2401199109,0cyclictest0-21swapper/200:14:272
2401199100,9cyclictest0-21swapper/222:44:072
2401199100,9cyclictest0-21swapper/222:38:362
2401199100,0cyclictest0-21swapper/223:53:062
2401199100,0cyclictest0-21swapper/223:32:572
2401199100,0cyclictest0-21swapper/222:59:372
2401199100,0cyclictest0-21swapper/222:04:362
2401199100,0cyclictest0-21swapper/200:24:372
2400599109,0cyclictest0-21swapper/123:28:541
2400599109,0cyclictest0-21swapper/122:51:141
2400599109,0cyclictest0-21swapper/122:45:451
2400599109,0cyclictest0-21swapper/121:46:551
2400599109,0cyclictest0-21swapper/121:23:451
2400599109,0cyclictest0-21swapper/121:14:041
2399899109,0cyclictest0-21swapper/023:55:030
2399899109,0cyclictest0-21swapper/023:44:520
2399899109,0cyclictest0-21swapper/023:33:530
2399899109,0cyclictest0-21swapper/022:43:020
2399899109,0cyclictest0-21swapper/022:25:220
2399899109,0cyclictest0-21swapper/022:22:030
2399899109,0cyclictest0-21swapper/022:15:120
2399899109,0cyclictest0-21swapper/022:09:220
2399899109,0cyclictest0-21swapper/021:31:430
2399899109,0cyclictest0-21swapper/021:10:520
2399899109,0cyclictest0-21swapper/020:20:230
2399899109,0cyclictest0-21swapper/019:14:330
2399899103,3cyclictest19480-21perf21:45:000
2399899100,9cyclictest0-21swapper/023:23:330
2399899100,9cyclictest0-21swapper/000:25:430
2399899100,0cyclictest0-21swapper/021:53:520
2399899100,0cyclictest0-21swapper/020:45:030
9860290,1sleep29861-21bash23:28:082
240149998,0cyclictest0-21swapper/323:49:573
240149998,0cyclictest0-21swapper/323:16:173
240149998,0cyclictest0-21swapper/322:21:173
240149998,0cyclictest0-21swapper/319:54:573
240149998,0cyclictest0-21swapper/319:54:563
240149998,0cyclictest0-21swapper/319:35:163
240149998,0cyclictest0-21swapper/300:26:163
240149998,0cyclictest0-21swapper/300:06:573
240119997,1cyclictest0-21swapper/223:10:072
240059998,0cyclictest0-21swapper/123:41:441
240059998,0cyclictest0-21swapper/100:38:351
240059997,1cyclictest614-21dbus-daemon23:58:251
240059997,1cyclictest5460-21lspci22:30:091
240059996,2cyclictest598-21avahi-daemon21:57:511
239989998,0cyclictest0-21swapper/023:11:030
239989998,0cyclictest0-21swapper/000:04:230
239989997,1cyclictest15022-21systemd-cgroups23:37:330
239989990,0cyclictest0-21swapper/020:08:330
15022290,0sleep00-21swapper/022:32:240
240149987,0cyclictest0-21swapper/321:33:273
240149987,0cyclictest0-21swapper/300:01:063
240149986,1cyclictest576-17sedispatch21:27:473
240149985,2cyclictest598-21avahi-daemon23:54:523
240149985,2cyclictest598-21avahi-daemon23:35:393
240149985,2cyclictest598-21avahi-daemon23:33:283
240149985,2cyclictest598-21avahi-daemon22:52:133
240149985,2cyclictest598-21avahi-daemon22:11:453
240149985,2cyclictest598-21avahi-daemon21:41:013
240149985,2cyclictest598-21avahi-daemon21:14:123
240149985,2cyclictest598-21avahi-daemon00:35:253
240119987,0cyclictest0-21swapper/223:57:472
240119987,0cyclictest0-21swapper/223:21:372
240119987,0cyclictest0-21swapper/223:03:262
240119987,0cyclictest0-21swapper/221:27:262
240119987,0cyclictest0-21swapper/221:13:462
240119986,1cyclictest598-21avahi-daemon22:32:282
240119984,3cyclictest633-21polkitd22:29:182
240059987,0cyclictest0-21swapper/123:24:051
240059987,0cyclictest0-21swapper/123:03:541
240059987,0cyclictest0-21swapper/122:40:141
240059987,0cyclictest0-21swapper/100:16:141
240059986,1cyclictest598-21avahi-daemon23:36:561
240059986,1cyclictest25-21ksoftirqd/123:19:591
240059985,2cyclictest598-21avahi-daemon21:19:061
240059985,2cyclictest598-21avahi-daemon00:31:581
240059985,2cyclictest598-21avahi-daemon00:21:181
240059982,1cyclictest25-21ksoftirqd/119:40:011
239989987,0cyclictest0-21swapper/023:04:230
239989987,0cyclictest0-21swapper/021:46:330
239989987,0cyclictest0-21swapper/000:24:430
239989986,1cyclictest573-17auditd21:57:320
239989985,2cyclictest598-21avahi-daemon23:16:240
239989985,2cyclictest598-21avahi-daemon23:05:560
239989985,2cyclictest598-21avahi-daemon22:57:120
239989985,2cyclictest598-21avahi-daemon22:35:020
239989985,2cyclictest598-21avahi-daemon22:03:350
239989985,2cyclictest598-21avahi-daemon21:39:170
239989985,2cyclictest598-21avahi-daemon00:30:370
239989985,2cyclictest598-21avahi-daemon00:07:260
240149976,0cyclictest0-21swapper/323:25:563
240149976,0cyclictest0-21swapper/321:45:363
240149976,0cyclictest0-21swapper/320:49:563
240149975,1cyclictest598-21avahi-daemon23:44:063
240149975,1cyclictest598-21avahi-daemon23:00:373
240149975,1cyclictest598-21avahi-daemon22:36:013
240149975,1cyclictest598-21avahi-daemon22:18:293
240149975,1cyclictest598-21avahi-daemon22:05:473
240149975,1cyclictest598-21avahi-daemon21:20:563
240149975,1cyclictest598-21avahi-daemon00:20:453
240149974,2cyclictest598-21avahi-daemon23:05:333
240149974,2cyclictest598-21avahi-daemon21:55:123
240149974,2cyclictest598-21avahi-daemon00:30:083
240119975,1cyclictest614-21dbus-daemon21:42:422
240119975,1cyclictest598-21avahi-daemon23:40:122
240119975,1cyclictest598-21avahi-daemon22:47:112
240119975,1cyclictest598-21avahi-daemon22:06:052
240119975,1cyclictest598-21avahi-daemon00:00:242
240119974,2cyclictest614-21dbus-daemon23:35:332
240119974,2cyclictest614-21dbus-daemon22:19:392
240119974,2cyclictest598-21avahi-daemon21:17:212
240119974,2cyclictest598-21avahi-daemon00:15:492
240119973,1cyclictest0-21swapper/221:20:222
240119972,3cyclictest20742-21sshd21:53:182
240059976,0cyclictest0-21swapper/120:55:151
240059976,0cyclictest0-21swapper/100:25:251
240059975,1cyclictest633-21polkitd21:52:271
240059975,1cyclictest598-21avahi-daemon23:05:071
240059975,1cyclictest598-21avahi-daemon22:37:411
240059975,1cyclictest598-21avahi-daemon22:25:241
240059975,1cyclictest598-21avahi-daemon22:11:041
240059975,1cyclictest598-21avahi-daemon21:36:141
240059975,1cyclictest598-21avahi-daemon00:14:311
240059975,1cyclictest598-21avahi-daemon00:00:081
240059974,2cyclictest614-21dbus-daemon22:15:581
240059974,2cyclictest598-21avahi-daemon23:45:351
240059974,2cyclictest598-21avahi-daemon22:03:311
239989975,1cyclictest598-21avahi-daemon22:45:040
239989975,1cyclictest598-21avahi-daemon21:16:310
239989975,1cyclictest598-21avahi-daemon00:15:200
239989974,2cyclictest598-21avahi-daemon23:54:410
240149964,1cyclictest598-21avahi-daemon00:10:473
240149963,2cyclictest970-21snmpd20:31:453
240149963,2cyclictest970-21snmpd19:10:103
240149963,1cyclictest970-21snmpd20:00:423
240149960,5cyclictest0-21swapper/321:05:373
240149960,5cyclictest0-21swapper/320:35:363
240149960,5cyclictest0-21swapper/319:33:373
240119964,1cyclictest633-21polkitd19:45:002
240119963,2cyclictest970-21snmpd20:54:052
240119963,2cyclictest970-21snmpd20:30:282
240119963,2cyclictest970-21snmpd20:20:122
240119963,2cyclictest14786-21sshd23:45:292
240059964,1cyclictest614-21dbus-daemon20:10:011
240059964,1cyclictest25-21ksoftirqd/121:10:001
240059963,2cyclictest970-21snmpd20:13:271
240059962,2cyclictest26419-21sshd20:30:321
240059960,5cyclictest0-21swapper/119:40:041
240059960,0cyclictest0-21swapper/120:24:341
239989965,0cyclictest0-21swapper/021:01:120
239989964,1cyclictest643-21gdbus19:54:550
239989964,1cyclictest643-21gdbus19:54:550
239989960,5cyclictest0-21swapper/020:26:020
239989960,5cyclictest0-21swapper/020:19:120
239989960,5cyclictest0-21swapper/020:00:520
239989960,5cyclictest0-21swapper/019:46:030
239989960,5cyclictest0-21swapper/019:43:220
240149954,0cyclictest0-21swapper/320:55:163
240149953,1cyclictest970-21snmpd20:42:463
240149953,1cyclictest970-21snmpd20:17:313
240149953,1cyclictest970-21snmpd20:13:183
240149953,1cyclictest970-21snmpd20:08:083
240149953,1cyclictest970-21snmpd19:55:563
240149953,1cyclictest970-21snmpd19:45:453
240149953,1cyclictest970-21snmpd19:41:443
240149953,1cyclictest970-21snmpd19:26:073
240149953,1cyclictest970-21snmpd19:20:173
240149953,1cyclictest23434-21cat20:25:143
240149953,1cyclictest1002-21rpcbind20:51:553
240149952,2cyclictest970-21snmpd19:16:253
240149950,0cyclictest0-21swapper/320:22:263
240119953,1cyclictest970-21snmpd21:01:072
240119953,1cyclictest970-21snmpd20:55:412
240119953,1cyclictest970-21snmpd20:45:032
240119953,1cyclictest970-21snmpd20:11:592
240119953,1cyclictest970-21snmpd19:55:322
240119953,1cyclictest970-21snmpd19:36:032
240119953,1cyclictest970-21snmpd19:20:052
240119953,1cyclictest9175-21cat19:50:142
240119953,1cyclictest9175-21cat19:50:132
240119953,1cyclictest7216-21grep21:05:132
240119953,1cyclictest643-21gdbus20:20:012
240119953,1cyclictest598-21avahi-daemon20:35:022
240119953,1cyclictest26612-21interrupts19:15:012
240119953,1cyclictest12755-21/usr/sbin/munin20:00:152
240119952,2cyclictest970-21snmpd20:40:372
240119952,2cyclictest970-21snmpd20:27:062
240119952,2cyclictest970-21snmpd19:10:172
240119952,2cyclictest15820-21gunzip20:05:192
240119950,1cyclictest30852-21/usr/sbin/munin19:25:202
240059953,1cyclictest970-21snmpd21:00:311
240059953,1cyclictest970-21snmpd19:55:061
240059953,1cyclictest970-21snmpd19:15:491
240059953,1cyclictest614-21dbus-daemon19:50:011
240059953,1cyclictest4421-21sendmail19:20:231
240059953,1cyclictest31802-21needreboot20:45:151
240059953,1cyclictest31721-21ntp_states19:25:161
240059953,1cyclictest30130-21smart_sda20:40:231
240059953,1cyclictest19393-21irqstats20:15:151
240059952,2cyclictest970-21snmpd20:35:511
240059952,2cyclictest970-21snmpd19:33:141
240059952,2cyclictest970-21snmpd19:10:591
240059950,0cyclictest0-21swapper/119:52:141
240059950,0cyclictest0-21swapper/119:52:141
239989954,0cyclictest0-21swapper/021:05:230
239989953,1cyclictest970-21snmpd20:51:210
239989953,1cyclictest970-21snmpd19:57:270
239989953,1cyclictest970-21snmpd19:33:170
239989953,1cyclictest970-21snmpd19:26:230
239989953,1cyclictest970-21snmpd19:20:580
239989953,1cyclictest970-21snmpd19:15:310
239989953,1cyclictest614-21dbus-daemon19:40:010
239989953,1cyclictest25482-21cat20:30:150
239989952,2cyclictest4560-21taskset20:59:280
239989950,0cyclictest0-21swapper/020:41:420
240149942,1cyclictest5002-21cut21:00:103
240119942,1cyclictest970-21snmpd19:45:032
240059942,1cyclictest23597-21missed_timers20:25:141
240059942,1cyclictest12868-21tr20:00:111
240059942,1cyclictest1235-21if_eno220:50:131
239989942,1cyclictest27024-21cpu20:35:110
239989942,1cyclictest17133-21tr20:10:120
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional