You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-24 - 11:49
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the highest latencies:
System rack2slot0.osadl.org (updated Wed Apr 24, 2024 00:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
285372800,1sleep3401ktimersoftd/323:48:033
93962550,0sleep00-21swapper/000:34:410
311552550,0sleep10-21swapper/100:15:311
317232540,0sleep331725-21sshd21:12:513
291602540,0sleep00-21swapper/023:13:270
273192540,0sleep30-21swapper/300:18:303
302072530,1sleep2321ktimersoftd/223:39:372
289302530,0sleep20-21swapper/222:18:382
43992520,0sleep10-21swapper/121:21:491
30962510,1sleep23099-21bash23:28:262
241682510,0sleep324158-21sshd21:15:023
214752500,0sleep10-21swapper/122:12:111
58002490,0sleep30-21swapper/322:07:583
324802480,1sleep232482-21bash21:21:272
320022480,0sleep00-21swapper/022:04:330
287732480,0sleep00-21swapper/000:15:190
281142480,0sleep10-21swapper/122:53:141
16652480,0sleep11666-21bash00:30:091
155082480,1sleep00-21swapper/000:24:240
302722470,0sleep30-21swapper/323:04:573
29312470,0sleep10-21swapper/123:11:041
187462470,0sleep018748-21grep23:52:560
1484524735,7sleep30-21swapper/319:05:033
139262470,0sleep00-21swapper/022:54:520
181342440,0sleep21676099cyclictest23:47:062
317202370,0sleep00-21swapper/023:16:310
153022360,0sleep00-21swapper/023:38:140
164672319,7sleep10-21swapper/119:07:491
164882298,7sleep00-21swapper/019:08:040
1531822210,6sleep20-21swapper/219:05:182
299942210,0sleep30-21swapper/300:02:483
271802190,0sleep00-21swapper/000:29:210
264462180,0sleep20-21swapper/222:35:352
110522180,0sleep20-21swapper/223:52:132
318542170,0sleep331851-21lspci00:33:333
167242170,0sleep30-21swapper/323:35:243
222802160,0sleep10-21swapper/121:46:281
176662150,0sleep233-21ksoftirqd/222:49:252
16754991514,0cyclictest29518-21lspci23:01:561
16754991514,0cyclictest12550-21lspci00:13:491
1675499150,13cyclictest0-21swapper/121:56:171
90762140,1sleep09072-21sshd22:39:580
83932140,0sleep2311rcuc/221:19:232
270282140,1sleep227023-21lspci21:44:062
172472140,0sleep30-21swapper/300:28:113
16754991413,0cyclictest14175-21lspci23:43:531
163662140,0sleep00-21swapper/021:40:150
139522140,0sleep00-21swapper/022:23:010
12642140,0sleep10-21swapper/123:54:171
16765991311,1cyclictest649-21avahi-daemon21:30:183
16765991311,1cyclictest2585-21bash21:44:493
1676599130,1cyclictest27160-21bash22:18:293
1676599130,1cyclictest1011-21sshd22:44:093
1676599130,12cyclictest1-21systemd21:38:493
16754991312,0cyclictest32123-21lspci22:24:401
1674799130,1cyclictest1-21systemd21:32:500
1674799130,12cyclictest646-21dbus-daemon21:38:390
164902130,0sleep20-21swapper/223:58:312
141202130,0sleep10-21swapper/121:19:541
1676599120,11cyclictest19392-21lspci23:55:493
1676599120,11cyclictest1-21systemd21:51:293
16760991210,1cyclictest27809-21lspci23:21:572
1676099120,11cyclictest618-13audispd00:15:172
1676099120,11cyclictest16020-21bash22:03:082
1676099120,0cyclictest21000-21sshd21:34:572
16754991211,0cyclictest6182-21lspci23:22:551
16754991211,0cyclictest21710-21lspci23:07:001
16754991210,1cyclictest23869-21lspci22:15:171
1675499120,1cyclictest2636-21grep21:36:061
1675499120,11cyclictest0-21swapper/121:52:361
1674799120,1cyclictest5688-21sshd23:34:300
1674799120,1cyclictest1-21systemd00:06:590
1674799120,1cyclictest10784-21grepconf.sh23:23:200
1674799120,11cyclictest26958-21lspci23:27:390
1674799120,11cyclictest21994-21sshd21:26:200
1674799120,11cyclictest13896-21sshd00:38:500
1676599119,1cyclictest5359-21lspci22:28:013
1676599119,1cyclictest13465-21sshd23:17:493
1676599110,10cyclictest0-21swapper/323:05:293
1676599110,10cyclictest0-21swapper/319:40:183
1676599110,10cyclictest0-21swapper/300:37:393
1676099119,1cyclictest2793-21sshd23:02:272
16760991110,0cyclictest0-21swapper/222:06:482
16760991110,0cyclictest0-21swapper/221:47:172
16760991110,0cyclictest0-21swapper/200:35:082
1676099110,10cyclictest0-21swapper/223:43:582
1676099110,10cyclictest0-21swapper/222:44:482
1676099110,10cyclictest0-21swapper/222:12:572
1676099110,0cyclictest0-21swapper/200:22:582
16754991110,0cyclictest523-40kipmi000:05:171
16754991110,0cyclictest33050irq/27-ahci[00022:48:491
16754991110,0cyclictest33050irq/27-ahci[00021:34:361
16754991110,0cyclictest27589-21lspci22:35:421
16754991110,0cyclictest0-21swapper/123:37:371
16754991110,0cyclictest0-21swapper/123:30:471
16754991110,0cyclictest0-21swapper/122:32:071
16754991110,0cyclictest0-21swapper/100:28:371
1675499110,10cyclictest0-21swapper/121:28:161
1675499110,10cyclictest0-21swapper/119:33:161
1674799119,1cyclictest18282-21bash23:41:190
16747991110,0cyclictest0-21swapper/023:48:500
1674799110,10cyclictest0-21swapper/022:10:190
1674799110,0cyclictest0-21swapper/022:19:000
116762110,0sleep10-21swapper/122:28:341
189722100,0sleep30-21swapper/322:55:163
17062100,1sleep31704-21sshd22:30:293
1676599109,0cyclictest0-21swapper/300:11:293
1676599108,1cyclictest24801-21sshd21:49:393
1676599108,1cyclictest17142-21sshd00:20:593
16765991010,0cyclictest0-21swapper/300:08:593
1676599100,0cyclictest0-21swapper/323:24:393
1676099109,0cyclictest0-21swapper/221:39:282
1676099109,0cyclictest0-21swapper/221:11:072
1676099109,0cyclictest0-21swapper/220:25:172
1676099109,0cyclictest0-21swapper/200:07:082
1676099108,1cyclictest648-21systemd-logind23:10:272
1676099108,1cyclictest5069-21sshd00:26:482
1676099108,1cyclictest14333-21lspci22:34:382
1676099108,1cyclictest0-21swapper/223:34:382
1675499109,0cyclictest33050irq/27-ahci[00022:08:531
1675499109,0cyclictest0-21swapper/123:55:071
1675499109,0cyclictest0-21swapper/123:15:171
1675499109,0cyclictest0-21swapper/121:14:461
1675499109,0cyclictest0-21swapper/100:20:171
1675499108,1cyclictest25-21ksoftirqd/122:02:521
1675499100,2cyclictest121rcu_preempt21:42:231
1674799108,1cyclictest10700-21systemd-cgroups21:16:400
1674799108,1cyclictest0-21swapper/023:56:200
1674799100,0cyclictest0-21swapper/021:21:090
167659998,0cyclictest0-21swapper/322:38:093
167609998,0cyclictest0-21swapper/221:57:372
167609998,0cyclictest0-21swapper/221:05:162
167549998,0cyclictest0-21swapper/120:54:161
167549997,1cyclictest25-21ksoftirqd/100:00:051
167549995,3cyclictest649-21avahi-daemon22:40:401
167549990,1cyclictest131rcu_sched23:28:181
167479991,4cyclictest12630-21bash22:00:000
19442280,1sleep019444-21bash23:01:010
167659987,0cyclictest0-21swapper/322:51:593
167659986,1cyclictest3927-21grep23:40:093
167659986,1cyclictest2548-21lspci23:25:243
167659986,1cyclictest2489-21lspci21:56:093
167659986,1cyclictest20698-21lspci21:23:193
167659985,2cyclictest649-21avahi-daemon23:10:093
167659985,2cyclictest19535-21sshd22:23:303
167609986,1cyclictest6529-21id00:34:222
167609986,1cyclictest23422-21lspci22:26:432
167609983,2cyclictest4980-21sshd23:17:022
167549986,1cyclictest25-21ksoftirqd/123:00:011
167549986,1cyclictest25-21ksoftirqd/100:35:031
167549980,1cyclictest131rcu_sched23:45:131
167479987,0cyclictest8941-21lspci22:42:500
167479987,0cyclictest0-21swapper/021:48:190
167479987,0cyclictest0-21swapper/020:00:200
167479987,0cyclictest0-21swapper/000:12:000
167479986,1cyclictest5356-21lspci22:28:000
167479986,1cyclictest15700-21lspci22:31:480
167479985,2cyclictest649-21avahi-daemon21:14:270
167479980,4cyclictest648-21systemd-logind23:10:010
14286280,1sleep214284-21lspci00:01:152
167659975,1cyclictest649-21avahi-daemon23:50:153
167659975,1cyclictest649-21avahi-daemon22:00:063
167659975,1cyclictest649-21avahi-daemon21:27:163
167659975,1cyclictest21748-21lspci23:30:083
167659975,1cyclictest1046-21lspci22:45:043
167659974,2cyclictest684-21polkitd22:10:563
167609974,2cyclictest649-21avahi-daemon22:50:332
167609974,2cyclictest649-21avahi-daemon21:52:192
167609974,2cyclictest646-21dbus-daemon22:56:012
167609974,1cyclictest684-21polkitd22:21:392
167609973,1cyclictest0-21swapper/200:11:172
167609972,3cyclictest25229-21sshd21:26:372
167609972,1cyclictest0-21swapper/223:08:252
167609970,2cyclictest28694-21smartctl19:40:132
167549976,0cyclictest0-21swapper/120:17:561
167549975,1cyclictest25-21ksoftirqd/121:10:011
167479976,0cyclictest0-21swapper/022:45:500
167479975,1cyclictest7024-21lspci21:50:500
167479975,1cyclictest646-21dbus-daemon22:05:460
167479975,1cyclictest26082-21lspci22:55:510
167479975,1cyclictest1054-21lspci00:00:030
167659963,2cyclictest17299-21snmpd20:33:023
167609963,2cyclictest48-17kauditd20:55:002
167609963,2cyclictest24222-21latency_hist20:50:002
167609963,2cyclictest17299-21snmpd19:39:522
167609963,2cyclictest1058-21sendmail19:55:182
167549965,0cyclictest0-21swapper/121:02:361
167549965,0cyclictest0-21swapper/120:20:461
167549965,0cyclictest0-21swapper/120:01:261
167549965,0cyclictest0-21swapper/119:59:161
167549965,0cyclictest0-21swapper/119:51:151
167549965,0cyclictest0-21swapper/119:45:161
167549964,1cyclictest646-21dbus-daemon19:19:551
167479964,1cyclictest700-21gdbus20:24:590
167479963,2cyclictest17299-21snmpd21:03:510
167479963,2cyclictest17299-21snmpd20:35:590
167479960,5cyclictest0-21swapper/019:25:200
167659953,1cyclictest649-21avahi-daemon20:35:473
167659953,1cyclictest400-21mii-tool19:50:143
167659953,1cyclictest30957-21cut21:05:133
167659953,1cyclictest29033-21cat21:00:183
167659953,1cyclictest24942-21mii-tool20:50:163
167659953,1cyclictest24724-21cat19:30:143
167659953,1cyclictest1816-21crond19:55:003
167659953,1cyclictest17299-21snmpd20:58:363
167659953,1cyclictest17299-21snmpd20:48:423
167659953,1cyclictest17299-21snmpd20:41:323
167659953,1cyclictest17299-21snmpd20:06:463
167659953,1cyclictest17299-21snmpd20:00:043
167659953,1cyclictest17299-21snmpd19:36:003
167659953,1cyclictest17299-21snmpd19:26:293
167659953,1cyclictest17299-21snmpd19:21:573
167659953,1cyclictest17299-21snmpd19:11:503
167659953,1cyclictest12739-21cat20:20:153
167659953,1cyclictest1053-21lldpd20:27:273
167659952,2cyclictest30501-21cstates19:45:123
167659952,2cyclictest17299-21snmpd20:16:373
167659950,1cyclictest19336-21cat19:15:013
167609953,1cyclictest6631-21cat20:05:132
167609953,1cyclictest649-21avahi-daemon21:01:522
167609953,1cyclictest24690-21cat19:30:142
167609953,1cyclictest17299-21snmpd20:42:262
167609953,1cyclictest17299-21snmpd20:39:352
167609953,1cyclictest17299-21snmpd20:16:432
167609953,1cyclictest17299-21snmpd20:14:442
167609953,1cyclictest17299-21snmpd19:51:392
167609953,1cyclictest17299-21snmpd19:47:132
167609953,1cyclictest17299-21snmpd19:25:292
167609953,1cyclictest17299-21snmpd19:10:212
167609953,1cyclictest16811-21cat20:30:102
167609952,2cyclictest17299-21snmpd20:00:072
167609950,4cyclictest0-21swapper/220:58:072
167609950,4cyclictest0-21swapper/220:20:182
167609950,4cyclictest0-21swapper/219:21:472
167609950,2cyclictest19906-21hddtemp_smartct19:15:142
167549953,1cyclictest649-21avahi-daemon20:55:041
167549953,1cyclictest28797-21cat19:40:151
167549953,1cyclictest22795-21grep20:45:131
167549953,1cyclictest18848-21cat20:35:131
167549953,1cyclictest17299-21snmpd20:30:281
167549953,1cyclictest17299-21snmpd20:11:521
167549953,1cyclictest17299-21snmpd20:05:091
167549953,1cyclictest17299-21snmpd19:38:491
167549953,1cyclictest17299-21snmpd19:20:451
167549953,1cyclictest17299-21snmpd19:10:421
167549953,1cyclictest14734-21cat20:25:131
167479953,1cyclictest6592-21cat20:05:130
167479953,1cyclictest30792-21cat19:45:150
167479953,1cyclictest2513-21cat19:55:140
167479953,1cyclictest19979-21cat19:15:140
167479953,1cyclictest17428-21cat19:10:150
167479953,1cyclictest17299-21snmpd20:59:320
167479953,1cyclictest17299-21snmpd20:51:390
167479953,1cyclictest17299-21snmpd20:45:050
167479953,1cyclictest17299-21snmpd20:31:400
167479953,1cyclictest17299-21snmpd20:27:440
167479953,1cyclictest17299-21snmpd20:10:310
167479953,1cyclictest17299-21snmpd19:50:490
167479953,1cyclictest17299-21snmpd19:42:360
167479953,1cyclictest17299-21snmpd19:38:160
167479953,1cyclictest17299-21snmpd19:30:270
167479953,1cyclictest17299-21snmpd19:24:030
167479953,1cyclictest10702-21cat20:15:150
167479953,1cyclictest1053-21lldpd21:05:360
167479952,2cyclictest20395-21sshd20:40:110
167659942,1cyclictest8304-21cpuspeed_turbos20:10:113
167549942,1cyclictest17299-21snmpd20:40:141
167549942,1cyclictest0-21swapper/119:25:071
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional