You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2022-01-26 - 06:17

Intel(R) Xeon(R) CPU E31220 @ 3.10GHz, Linux 4.19.1-rt3 (Profile)

Latency plot of system in rack #2, slot #0
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the alll highest latencies:
System rack2slot0.osadl.org (updated Wed Jan 26, 2022 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
102052540,0sleep20-21swapper/222:53:232
164712530,0sleep30-21swapper/323:27:373
82982520,0sleep00-21swapper/000:38:550
82982520,0sleep00-21swapper/000:38:550
178223323,6sleep10-21swapper/119:09:301
171723119,7sleep00-21swapper/019:08:410
164623119,7sleep30-21swapper/319:07:473
179822412,7sleep20-21swapper/219:09:432
232222170,1sleep223188-21sshd21:35:182
167982170,0sleep20-21swapper/200:16:302
85092160,0sleep30-21swapper/321:31:213
266862160,0sleep20-21swapper/221:20:002
197742140,0sleep10-21swapper/123:04:041
194199146,2cyclictest598-21avahi-daemon22:18:363
120992140,0sleep10-21swapper/121:56:381
36962130,0sleep20-21swapper/223:32:322
240202130,0sleep00-21swapper/021:59:550
194199129,0cyclictest0-21swapper/321:29:073
194199120,10cyclictest24539-21bash22:56:563
194199120,0cyclictest0-21swapper/300:37:473
194199120,0cyclictest0-21swapper/300:37:463
193199120,11cyclictest12777-21sshd22:13:130
249512110,0sleep30-21swapper/323:45:543
194199119,1cyclictest30970-21systemd-cgroups22:25:473
194199110,10cyclictest0-21swapper/323:53:373
194199110,10cyclictest0-21swapper/323:43:073
194199110,10cyclictest0-21swapper/323:30:073
194199110,0cyclictest0-21swapper/300:23:273
194099119,1cyclictest614-21dbus-daemon22:56:382
1940991110,0cyclictest0-21swapper/221:45:482
194099110,10cyclictest0-21swapper/223:42:082
194099110,10cyclictest0-21swapper/223:24:472
194099110,10cyclictest0-21swapper/221:56:172
194099110,10cyclictest0-21swapper/221:22:572
1937991110,0cyclictest4258-21lspci22:51:461
193799110,10cyclictest0-21swapper/123:54:201
193799110,10cyclictest0-21swapper/123:43:501
193799110,10cyclictest0-21swapper/123:11:291
193799110,10cyclictest0-21swapper/122:35:291
193799110,10cyclictest0-21swapper/122:22:391
193799110,10cyclictest0-21swapper/122:00:401
193799110,10cyclictest0-21swapper/121:51:301
193799110,10cyclictest0-21swapper/121:11:401
193799110,10cyclictest0-21swapper/100:31:401
193199110,1cyclictest19970-21bash23:36:330
193199110,10cyclictest0-21swapper/023:51:230
193199110,10cyclictest0-21swapper/021:40:340
193199110,10cyclictest0-21swapper/021:27:340
193199110,10cyclictest0-21swapper/021:22:230
193199110,10cyclictest0-21swapper/000:13:440
194199109,0cyclictest0-21swapper/322:11:373
194199109,0cyclictest0-21swapper/321:57:563
194199109,0cyclictest0-21swapper/320:59:573
194199109,0cyclictest0-21swapper/319:34:563
194199109,0cyclictest0-21swapper/300:07:173
194199108,1cyclictest966-21runrttasks21:24:373
194199108,1cyclictest23902-21sshd23:05:073
194199108,1cyclictest0-21swapper/322:46:163
194199100,9cyclictest0-21swapper/323:57:573
194199100,0cyclictest0-21swapper/323:15:473
194199100,0cyclictest0-21swapper/322:05:063
194199100,0cyclictest0-21swapper/321:47:273
194099109,0cyclictest0-21swapper/223:12:172
194099109,0cyclictest0-21swapper/222:10:382
194099100,0cyclictest0-21swapper/222:45:172
194099100,0cyclictest0-21swapper/222:04:082
193799109,0cyclictest0-21swapper/122:18:101
193799109,0cyclictest0-21swapper/122:07:501
193799109,0cyclictest0-21swapper/119:25:301
193799109,0cyclictest0-21swapper/100:19:501
1937991010,0cyclictest31724-21lspci22:34:261
193799100,0cyclictest0-21swapper/100:08:001
193199109,0cyclictest0-21swapper/023:00:230
193199109,0cyclictest0-21swapper/000:06:540
193199108,1cyclictest0-21swapper/000:23:040
193199100,9cyclictest0-21swapper/022:08:040
193199100,9cyclictest0-21swapper/021:32:040
193199100,9cyclictest0-21swapper/019:55:040
193199100,9cyclictest0-21swapper/019:39:520
193199100,0cyclictest0-21swapper/023:13:330
193199100,0cyclictest0-21swapper/022:35:030
193199100,0cyclictest0-21swapper/021:53:040
193199100,0cyclictest0-21swapper/021:15:140
19419998,0cyclictest0-21swapper/322:54:273
19419998,0cyclictest0-21swapper/322:42:273
19419998,0cyclictest0-21swapper/322:21:163
19419998,0cyclictest0-21swapper/300:25:573
19419990,0cyclictest0-21swapper/319:22:563
19409998,0cyclictest0-21swapper/223:48:172
19409998,0cyclictest0-21swapper/222:37:382
19409998,0cyclictest0-21swapper/200:31:482
19379998,0cyclictest0-21swapper/123:17:101
19379998,0cyclictest0-21swapper/122:43:501
19379998,0cyclictest0-21swapper/121:36:201
19379998,0cyclictest0-21swapper/121:26:401
19379998,0cyclictest0-21swapper/100:10:301
19379998,0cyclictest0-21swapper/100:01:091
19379994,2cyclictest614-21dbus-daemon22:47:381
19379991,1cyclictest25-21ksoftirqd/122:28:551
19319998,0cyclictest0-21swapper/023:45:140
19319998,0cyclictest0-21swapper/023:30:240
19319998,0cyclictest0-21swapper/023:21:440
19319998,0cyclictest0-21swapper/000:00:030
19319997,2cyclictest14149-21kworker/0:222:23:190
19319996,2cyclictest9991-21kworker/0:223:25:220
19319996,2cyclictest25669-21kworker/0:200:28:020
19319990,0cyclictest0-21swapper/020:15:530
19319990,0cyclictest0-21swapper/020:05:340
19419987,0cyclictest0-21swapper/323:13:173
19419987,0cyclictest0-21swapper/322:30:163
19419987,0cyclictest0-21swapper/319:26:473
19419987,0cyclictest0-21swapper/300:16:373
19419985,2cyclictest598-21avahi-daemon23:39:583
19419985,2cyclictest598-21avahi-daemon23:00:133
19419985,2cyclictest598-21avahi-daemon00:12:173
19409987,0cyclictest0-21swapper/222:15:482
19409987,0cyclictest0-21swapper/221:33:172
19409987,0cyclictest0-21swapper/221:28:482
19409987,0cyclictest0-21swapper/200:10:372
19409987,0cyclictest0-21swapper/200:03:472
19409986,1cyclictest29481-21sshd22:33:482
19409985,2cyclictest598-21avahi-daemon23:58:182
19409985,2cyclictest598-21avahi-daemon23:28:492
19409985,2cyclictest598-21avahi-daemon22:25:252
19409985,2cyclictest598-21avahi-daemon21:40:132
19409985,2cyclictest598-21avahi-daemon00:27:162
19379987,0cyclictest0-21swapper/123:09:001
19379987,0cyclictest0-21swapper/122:58:201
19379987,0cyclictest0-21swapper/121:20:491
19379987,0cyclictest0-21swapper/100:22:201
19379986,1cyclictest25-21ksoftirqd/123:30:291
19379985,2cyclictest598-21avahi-daemon23:36:571
19379985,2cyclictest598-21avahi-daemon21:30:031
19379985,2cyclictest598-21avahi-daemon21:18:381
19379985,2cyclictest598-21avahi-daemon00:39:211
19379985,2cyclictest598-21avahi-daemon00:39:211
19379981,1cyclictest25-21ksoftirqd/122:10:161
19319987,0cyclictest0-21swapper/022:31:140
19319987,0cyclictest0-21swapper/000:16:130
19319986,2cyclictest9991-21kworker/0:223:06:350
19319986,2cyclictest5643-21kworker/0:000:30:550
19319986,2cyclictest27064-21kworker/0:021:45:380
19319986,2cyclictest25889-21kworker/0:322:15:190
19319986,2cyclictest19886-21kworker/0:223:55:110
19319986,2cyclictest19067-21kworker/0:323:15:110
19319986,2cyclictest18941-21kworker/0:221:10:390
19319986,2cyclictest14149-21kworker/0:222:48:350
19319986,2cyclictest14149-21kworker/0:222:29:180
19319986,1cyclictest12463-21kworker/0:322:56:210
19319985,2cyclictest598-21avahi-daemon23:42:300
19319985,2cyclictest598-21avahi-daemon22:44:030
19319985,2cyclictest598-21avahi-daemon22:01:170
19319985,2cyclictest598-21avahi-daemon21:37:230
19419976,0cyclictest0-21swapper/300:00:273
19419975,1cyclictest598-21avahi-daemon22:36:033
19419975,1cyclictest598-21avahi-daemon21:43:183
19419975,1cyclictest573-17auditd21:10:453
19419974,2cyclictest614-21dbus-daemon23:20:063
19419974,2cyclictest614-21dbus-daemon22:01:573
19419974,2cyclictest598-21avahi-daemon21:15:253
19419970,0cyclictest0-21swapper/319:11:473
19409976,0cyclictest0-21swapper/221:00:182
19409975,1cyclictest598-21avahi-daemon23:50:222
19409975,1cyclictest598-21avahi-daemon23:07:362
19409975,1cyclictest598-21avahi-daemon23:02:032
19409975,1cyclictest598-21avahi-daemon22:21:412
19409975,1cyclictest598-21avahi-daemon22:06:092
19409975,1cyclictest598-21avahi-daemon21:54:562
19409975,1cyclictest598-21avahi-daemon21:10:412
19409975,1cyclictest598-21avahi-daemon00:37:272
19409975,1cyclictest598-21avahi-daemon00:37:262
19409975,1cyclictest598-21avahi-daemon00:22:142
19409974,2cyclictest598-21avahi-daemon23:35:182
19409974,2cyclictest598-21avahi-daemon23:15:042
19409974,2cyclictest598-21avahi-daemon22:41:012
19409974,2cyclictest598-21avahi-daemon00:05:072
19379976,0cyclictest0-21swapper/123:24:401
19379975,1cyclictest598-21avahi-daemon23:47:421
19379975,1cyclictest598-21avahi-daemon21:41:001
19379975,1cyclictest598-21avahi-daemon00:26:461
19379971,1cyclictest131rcu_sched23:25:341
19379971,1cyclictest131rcu_sched21:45:531
19379970,1cyclictest131rcu_sched23:55:131
19319976,0cyclictest0-21swapper/020:46:540
19319976,0cyclictest0-21swapper/020:01:530
19319975,1cyclictest598-21avahi-daemon22:50:260
19419965,0cyclictest0-21swapper/320:16:573
19419965,0cyclictest0-21swapper/319:54:573
19419965,0cyclictest0-21swapper/300:30:573
19419964,1cyclictest614-21dbus-daemon20:35:013
19419964,1cyclictest598-21avahi-daemon21:50:113
19419964,1cyclictest598-21avahi-daemon21:35:373
19419964,1cyclictest598-21avahi-daemon20:29:563
19409965,0cyclictest0-21swapper/221:05:272
19409965,0cyclictest0-21swapper/220:40:182
19409965,0cyclictest0-21swapper/219:40:182
19409963,2cyclictest970-21snmpd20:24:042
19409963,2cyclictest970-21snmpd20:09:592
19409963,2cyclictest970-21snmpd19:31:292
19409960,5cyclictest0-21swapper/220:48:572
19409960,5cyclictest0-21swapper/220:38:572
19409960,5cyclictest0-21swapper/220:00:572
19409960,5cyclictest0-21swapper/219:45:072
19409960,5cyclictest0-21swapper/219:15:482
19379964,1cyclictest25-21ksoftirqd/120:00:011
19379964,1cyclictest0-21swapper/120:45:251
19379963,2cyclictest970-21snmpd21:05:301
19379963,2cyclictest970-21snmpd20:33:531
19379963,2cyclictest970-21snmpd20:00:451
19379962,1cyclictest0-21swapper/120:41:541
19319966,0cyclictest30921-21kworker/0:020:20:280
19319963,2cyclictest970-21snmpd20:43:570
19319963,2cyclictest970-21snmpd20:38:110
19319963,2cyclictest970-21snmpd19:48:030
19419953,1cyclictest970-21snmpd20:50:093
19419953,1cyclictest970-21snmpd20:50:083
19419953,1cyclictest970-21snmpd20:37:473
19419953,1cyclictest970-21snmpd20:23:463
19419953,1cyclictest970-21snmpd20:11:373
19419953,1cyclictest970-21snmpd20:07:593
19419953,1cyclictest970-21snmpd20:00:193
19419953,1cyclictest970-21snmpd19:44:163
19419953,1cyclictest970-21snmpd19:35:463
19419953,1cyclictest616-21gdbus19:15:013
19419952,2cyclictest970-21snmpd20:40:153
19419952,2cyclictest614-21dbus-daemon19:49:593
19419952,2cyclictest0-21swapper/319:55:183
19419950,2cyclictest9574-21/usr/sbin/munin20:45:223
19419950,0cyclictest0-21swapper/321:05:463
19419950,0cyclictest0-21swapper/321:00:373
19409953,1cyclictest970-21snmpd20:50:062
19409953,1cyclictest970-21snmpd20:50:062
19409953,1cyclictest970-21snmpd20:31:352
19409953,1cyclictest970-21snmpd20:14:522
19409953,1cyclictest970-21snmpd19:37:392
19409953,1cyclictest970-21snmpd19:25:392
19409953,1cyclictest970-21snmpd19:12:202
19409953,1cyclictest633-21polkitd20:15:012
19409953,1cyclictest4421-21sendmail19:23:072
19409953,1cyclictest21453-21cat19:55:212
19409953,1cyclictest19457-21cat19:50:212
19409953,1cyclictest1123-21df20:25:192
19409950,4cyclictest0-21swapper/220:58:572
19379954,0cyclictest0-21swapper/119:10:291
19379953,1cyclictest970-21snmpd20:35:161
19379953,1cyclictest970-21snmpd20:29:371
19379953,1cyclictest970-21snmpd20:15:531
19379953,1cyclictest970-21snmpd20:14:311
19379953,1cyclictest970-21snmpd19:37:031
19379953,1cyclictest970-21snmpd19:30:421
19379953,1cyclictest970-21snmpd19:22:081
19379953,1cyclictest970-21snmpd19:17:211
19379953,1cyclictest643-21gdbus20:55:001
19379953,1cyclictest616-21gdbus20:25:011
19379953,1cyclictest25533-21cat20:05:221
19379953,1cyclictest25-21ksoftirqd/121:01:001
19379953,1cyclictest17418-21cat19:45:191
19379952,2cyclictest970-21snmpd19:50:551
19379950,4cyclictest0-21swapper/119:42:501
19379950,1cyclictest0-21swapper/120:55:011
19379950,1cyclictest0-21swapper/120:55:001
19319955,0cyclictest3906-21kworker/0:019:20:030
19319955,0cyclictest14876-21kworker/0:220:10:110
19319955,0cyclictest12867-21kworker/0:021:05:240
19319955,0cyclictest12867-21kworker/0:021:01:280
19319955,0cyclictest12867-21kworker/0:020:55:230
19319953,1cyclictest970-21snmpd20:26:070
19319953,1cyclictest970-21snmpd19:10:080
19319953,1cyclictest5024-21tail19:15:200
19319953,1cyclictest3403-21cat20:30:210
19319953,1cyclictest19421-21cat19:50:200
19319953,1cyclictest15350-21cat19:40:190
19319953,1cyclictest11319-21cat19:30:210
19319953,1cyclictest11155-21grep20:50:190
19319953,1cyclictest11155-21grep20:50:180
19319953,1cyclictest1078-21ntpd19:25:240
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional