You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-07-24 - 06:04
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack2slot0.osadl.org (updated Wed Jul 24, 2024 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
319632990,0sleep20-21swapper/223:07:192
50742790,2sleep33095399cyclictest21:30:123
282022580,0sleep10-21swapper/121:47:321
106482580,1sleep210642-21sshd00:36:442
96062570,16sleep375550irq/24-eno1-rx-21:12:213
112192560,0sleep30-21swapper/321:58:123
213922550,1sleep275650irq/25-eno1-tx-22:26:372
269682540,0sleep30-21swapper/323:43:163
230422540,0sleep30-21swapper/322:48:183
137842530,0sleep113786-21sshd23:57:161
125772530,0sleep012572-21lspci22:56:230
259272520,0sleep10-21swapper/100:14:591
110722520,0sleep10-21swapper/122:50:111
98292510,2sleep23094799cyclictest21:58:032
60002510,0sleep2311rcuc/222:37:272
120472500,0sleep20-21swapper/223:41:472
6612480,0sleep10-21swapper/122:18:351
307902480,0sleep0111rcuc/021:35:330
257632480,0sleep20-21swapper/222:24:022
208172480,0sleep30-21swapper/322:54:103
183562480,0sleep00-21swapper/021:58:520
114312480,0sleep30-21swapper/300:26:423
172052470,0sleep20-21swapper/223:18:002
33082460,0sleep23307-21grep23:04:422
152772450,0sleep20-21swapper/222:47:332
189202410,2sleep23094799cyclictest00:00:512
43862400,0sleep00-21swapper/023:25:520
296952360,0sleep00-21swapper/000:21:590
193932330,0sleep019392-21sshd23:03:090
305672308,7sleep00-21swapper/019:06:380
307562276,7sleep10-21swapper/119:09:031
306642275,7sleep30-21swapper/319:07:533
282842250,0sleep20-21swapper/221:41:192
107402220,0sleep00-21swapper/022:07:170
326962210,0sleep30-21swapper/300:19:013
290422219,7sleep20-21swapper/219:05:042
80412190,1sleep18044-21bash22:07:021
249962190,0sleep10-21swapper/122:36:111
322752180,0sleep30-21swapper/322:24:393
271912180,1sleep327189-21lspci22:02:433
38772170,2sleep33095399cyclictest23:59:273
30953991716,0cyclictest9467-21lspci00:23:153
175012170,0sleep30-21swapper/322:26:143
208772160,0sleep20-21swapper/200:14:292
127062160,1sleep0111rcuc/021:33:560
106752160,0sleep20-21swapper/221:15:242
174372150,0sleep017429-21bash22:38:340
79172140,0sleep31-21systemd22:59:033
3095399140,1cyclictest8163-21sshd21:36:303
3094199140,1cyclictest19510-21lspci23:42:321
173812140,1sleep3401ktimersoftd/323:18:013
106222140,0sleep20-21swapper/223:20:202
30953991311,1cyclictest0-21swapper/321:23:503
3094799134,1cyclictest33-21ksoftirqd/221:45:262
3094799130,12cyclictest23033-21sshd23:12:302
3094199130,1cyclictest0-21swapper/121:21:031
3094199130,11cyclictest12905-21grep23:45:021
186902130,0sleep00-21swapper/000:14:140
3095399120,1cyclictest0-21swapper/323:08:003
3095399120,11cyclictest5160-21sshd22:31:103
3095399120,11cyclictest0-21swapper/323:01:103
3094799122,1cyclictest33-21ksoftirqd/223:53:342
3094799121,1cyclictest646-21dbus-daemon21:12:412
30947991210,1cyclictest0-21swapper/221:29:192
3094199120,11cyclictest5373-21sshd22:34:221
3094199120,11cyclictest19470-21sshd22:57:021
3094199120,11cyclictest0-21swapper/123:35:031
3094199120,11cyclictest0-21swapper/123:34:021
30936991210,1cyclictest18583-21sshd23:24:140
3093699120,1cyclictest25468-21sshd21:44:130
3093699120,11cyclictest0-21swapper/021:52:030
260512120,0sleep10-21swapper/122:11:471
3095399119,1cyclictest10842-21lspci21:42:513
3095399112,1cyclictest41-21ksoftirqd/300:10:003
30953991110,0cyclictest0-21swapper/321:50:403
3095399110,10cyclictest0-21swapper/323:21:213
3094799115,3cyclictest33-21ksoftirqd/221:53:362
3094799113,4cyclictest684-21polkitd22:59:452
3094799113,3cyclictest33-21ksoftirqd/200:30:292
3094799111,2cyclictest17639-21lspci21:31:202
3094799110,0cyclictest616-17auditd00:28:592
3094799110,0cyclictest0-21swapper/222:50:292
3094199119,1cyclictest9871-21lspci21:27:321
30941991110,0cyclictest0-21swapper/121:38:121
3094199110,10cyclictest0-21swapper/122:25:321
3093699113,4cyclictest0-21swapper/000:20:010
3093699110,10cyclictest0-21swapper/023:08:240
3093699110,10cyclictest0-21swapper/021:12:040
3095399109,0cyclictest0-21swapper/321:26:203
3095399108,1cyclictest692-21NetworkManager22:40:303
3095399107,2cyclictest41-21ksoftirqd/322:11:553
3095399102,3cyclictest41-21ksoftirqd/300:03:123
3094799108,1cyclictest33-21ksoftirqd/223:31:482
3094799103,1cyclictest33-21ksoftirqd/221:23:282
3094799102,1cyclictest131rcu_sched22:08:122
3094799101,1cyclictest131rcu_sched23:28:262
3094799101,1cyclictest131rcu_sched22:03:202
3094799100,3cyclictest121rcu_preempt00:16:392
3094799100,1cyclictest25513-21bash22:17:522
3094199109,0cyclictest0-21swapper/122:44:421
3094199109,0cyclictest0-21swapper/121:32:221
3094199109,0cyclictest0-21swapper/121:09:221
3094199100,0cyclictest0-21swapper/122:45:121
3094199100,0cyclictest0-21swapper/121:53:521
3093699109,0cyclictest0-21swapper/021:26:430
3093699100,0cyclictest0-21swapper/000:39:540
309539998,0cyclictest0-21swapper/319:45:003
309539997,1cyclictest41-21ksoftirqd/323:50:203
309539997,1cyclictest41-21ksoftirqd/321:46:123
309539991,3cyclictest15-21kswork23:26:103
309539991,1cyclictest41-21ksoftirqd/321:15:553
309539991,1cyclictest131rcu_sched23:12:223
309539990,8cyclictest11041-21kworker/3:223:31:573
309539990,1cyclictest131rcu_sched00:32:443
309479997,1cyclictest33-21ksoftirqd/223:55:302
309479997,1cyclictest33-21ksoftirqd/223:45:222
309479997,1cyclictest33-21ksoftirqd/200:23:442
309479993,1cyclictest33-21ksoftirqd/222:40:442
309479992,1cyclictest33-21ksoftirqd/222:11:392
309479992,1cyclictest33-21ksoftirqd/221:35:112
309479992,1cyclictest131rcu_sched22:33:152
309479991,2cyclictest33-21ksoftirqd/223:35:222
309479990,1cyclictest33-21ksoftirqd/200:05:232
309419998,0cyclictest0-21swapper/123:02:531
309419998,0cyclictest0-21swapper/120:45:221
309419998,0cyclictest0-21swapper/120:10:021
309419997,1cyclictest5125-21tty23:29:031
309419990,0cyclictest0-21swapper/120:05:031
309369998,0cyclictest0-21swapper/022:23:440
309369996,2cyclictest649-21avahi-daemon23:57:410
309369991,4cyclictest9260-21systemd-cgroups19:35:000
309369991,4cyclictest10684-21perf00:30:010
2716290,0sleep00-21swapper/022:30:560
8746280,1sleep1231rcuc/122:00:571
309539986,1cyclictest41-21ksoftirqd/323:46:103
309539980,2cyclictest7320-21kworker/3:221:05:003
309539980,1cyclictest131rcu_sched23:35:583
309539980,1cyclictest131rcu_sched22:15:213
309539980,1cyclictest131rcu_sched22:08:043
309539980,0cyclictest131rcu_sched00:10:133
309419986,1cyclictest25-21ksoftirqd/123:05:051
309419986,1cyclictest1870-21lspci21:14:391
309419986,1cyclictest16558-21lspci00:27:161
309369986,2cyclictest842-21kworker/0:000:01:080
309369986,2cyclictest8083-21kworker/0:322:27:320
309369986,2cyclictest5444-21kworker/0:122:45:220
309369986,2cyclictest5444-21kworker/0:122:40:020
309369986,2cyclictest4007-21kworker/0:000:31:300
309369986,2cyclictest32567-21kworker/0:323:38:400
309369986,2cyclictest3019-21kworker/0:422:17:300
309369986,2cyclictest25464-21kworker/0:123:47:480
309369986,2cyclictest18567-21kworker/0:421:17:580
309369986,2cyclictest16001-21kworker/0:221:23:310
309369986,2cyclictest10700-21kworker/0:323:13:160
309369986,1cyclictest14875-21lspci23:17:470
309369986,1cyclictest1065-21lspci22:00:150
309369985,3cyclictest32567-21kworker/0:323:40:580
309369984,3cyclictest16001-21kworker/0:221:45:330
26278281,1sleep0700-21gdbus23:30:570
309539975,1cyclictest41-21ksoftirqd/322:35:153
309539970,3cyclictest131rcu_sched00:35:133
309479976,0cyclictest0-21swapper/220:17:202
309479975,1cyclictest33-21ksoftirqd/221:10:012
309479975,1cyclictest33-21ksoftirqd/219:45:002
309419976,0cyclictest0-21swapper/120:55:031
309419975,1cyclictest684-21polkitd00:15:061
309419975,1cyclictest31923-21lspci21:41:431
309419975,1cyclictest25-21ksoftirqd/121:58:321
309419975,1cyclictest25-21ksoftirqd/100:05:511
309419975,1cyclictest24345-21lspci23:15:351
309419974,2cyclictest649-21avahi-daemon00:38:271
309419974,2cyclictest649-21avahi-daemon00:23:121
309419973,3cyclictest649-21avahi-daemon00:32:111
309419973,2cyclictest620-13audispd23:20:181
309369975,2cyclictest842-21kworker/0:023:50:350
309369975,2cyclictest5952-21kworker/0:400:05:060
309369975,2cyclictest5444-21kworker/0:122:53:220
309369975,1cyclictest27445-21sshd22:11:560
21459270,2sleep13094199cyclictest22:23:361
18069270,1sleep1619-17sedispatch00:00:471
309539965,0cyclictest0-21swapper/319:14:203
309539964,1cyclictest700-21gdbus19:39:593
309539964,1cyclictest684-21polkitd21:05:013
309539963,2cyclictest17299-21snmpd20:34:553
309539963,2cyclictest17299-21snmpd19:47:413
309539960,5cyclictest0-21swapper/320:58:203
309539960,5cyclictest0-21swapper/319:23:003
309479965,0cyclictest0-21swapper/220:40:202
309479965,0cyclictest0-21swapper/220:03:292
309479965,0cyclictest0-21swapper/219:21:192
309479964,1cyclictest33-21ksoftirqd/219:34:592
309479964,1cyclictest33-21ksoftirqd/219:30:002
309479963,2cyclictest33-21ksoftirqd/220:55:002
309479960,0cyclictest0-21swapper/220:57:002
309419964,1cyclictest684-21polkitd20:00:001
309419964,1cyclictest646-21dbus-daemon23:50:051
309419964,1cyclictest17299-21snmpd19:32:201
309419963,2cyclictest684-21polkitd23:10:121
309419963,2cyclictest646-21dbus-daemon21:15:021
309419960,0cyclictest0-21swapper/119:23:221
309369965,0cyclictest0-21swapper/021:05:230
309369964,1cyclictest700-21gdbus19:50:000
309369962,3cyclictest491-21perf19:15:000
309539954,0cyclictest0-21swapper/319:53:313
309539953,1cyclictest30706-21awk20:25:183
309539953,1cyclictest17299-21snmpd20:49:553
309539953,1cyclictest17299-21snmpd20:16:163
309539953,1cyclictest17299-21snmpd20:00:273
309539953,1cyclictest17299-21snmpd19:25:093
309539953,1cyclictest17299-21snmpd19:17:353
309539953,1cyclictest1530-21cpu20:35:103
309539953,1cyclictest1058-21sendmail20:21:023
309539952,2cyclictest7996-21basename20:50:143
309539952,2cyclictest646-21dbus-daemon19:55:003
309539952,2cyclictest17299-21snmpd20:10:363
309539950,1cyclictest8203-21cut19:30:163
309539950,1cyclictest22392-21grep20:05:173
309479954,0cyclictest0-21swapper/220:45:192
309479953,1cyclictest646-21dbus-daemon19:45:002
309479953,1cyclictest33-21ksoftirqd/220:15:002
309479953,1cyclictest33-21ksoftirqd/220:05:002
309479953,1cyclictest32299-21cat20:30:112
309479953,1cyclictest17299-21snmpd20:37:072
309479953,1cyclictest17299-21snmpd19:56:272
309479953,1cyclictest1053-21lldpd21:00:322
309479953,1cyclictest1017-21runrttasks19:52:192
309479952,2cyclictest700-21gdbus19:15:002
309479952,2cyclictest32138-21awk19:10:172
309479952,2cyclictest17299-21snmpd20:25:402
309479952,2cyclictest10350-21ntpq19:35:192
309419953,1cyclictest649-21avahi-daemon19:50:121
309419953,1cyclictest616-17auditd20:35:171
309419953,1cyclictest5905-21cat19:25:141
309419953,1cyclictest3306-21kworker/1:120:52:291
309419953,1cyclictest32154-21gltestperf20:30:111
309419953,1cyclictest31568-21sadc20:30:001
309419953,1cyclictest2275-21cut19:15:201
309419953,1cyclictest17299-21snmpd21:03:501
309419953,1cyclictest17299-21snmpd20:20:391
309419953,1cyclictest17299-21snmpd20:15:061
309419953,1cyclictest17299-21snmpd19:56:151
309419953,1cyclictest17299-21snmpd19:49:481
309419953,1cyclictest17299-21snmpd19:36:291
309419953,1cyclictest17299-21snmpd19:11:221
309419953,1cyclictest13070-21sshd19:42:191
309419950,0cyclictest0-21swapper/120:43:421
309369955,0cyclictest7239-21kworker/0:119:50:160
309369954,1cyclictest7321-21kworker/0:021:00:220
309369954,1cyclictest3135-21kworker/0:219:25:240
309369954,0cyclictest0-21swapper/020:15:030
309369953,1cyclictest4476-21switchtime19:20:200
309369953,1cyclictest30262-21cat20:25:130
309369953,1cyclictest24178-21cat20:10:130
309369953,1cyclictest17299-21snmpd20:55:080
309369953,1cyclictest17299-21snmpd20:52:130
309369953,1cyclictest17299-21snmpd20:45:250
309369953,1cyclictest17299-21snmpd20:41:400
309369953,1cyclictest17299-21snmpd20:39:460
309369953,1cyclictest17299-21snmpd19:55:490
309369953,1cyclictest17299-21snmpd19:36:020
309369953,1cyclictest11960-21cat19:40:140
309369953,1cyclictest1058-21sendmail20:09:560
309369952,2cyclictest646-21dbus-daemon19:15:010
309369952,2cyclictest19375-21crond20:00:000
309369950,4cyclictest29551-21munin-run20:25:000
309539942,1cyclictest75550irq/24-eno1-rx-20:40:103
309479942,1cyclictest17299-21snmpd20:20:032
309369944,0cyclictest31599-21kworker/0:120:30:050
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional