You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-20 - 05:40
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the highest latencies:
System rack2slot0.osadl.org (updated Sun Jul 20, 2025 00:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
284572950,1sleep328455-21lspci21:12:433
285422920,0sleep20-21swapper/221:38:532
17272890,0sleep215-21kswork00:06:202
173742750,1sleep017373-21grep22:40:150
55812730,1sleep15584-21bash21:39:511
321642580,0sleep30-21swapper/323:10:453
52642550,0sleep1241ktimersoftd/122:39:031
57712540,0sleep20-21swapper/223:34:162
219722540,0sleep00-21swapper/021:51:210
277922520,0sleep00-21swapper/023:26:320
239892520,1sleep323973-21bash21:45:023
147272520,1sleep10-21swapper/122:56:061
234672510,0sleep20-21swapper/223:16:262
138772510,0sleep00-21swapper/000:01:060
63052500,0sleep20-21swapper/221:30:032
281122500,0sleep20-21swapper/200:02:312
271872500,0sleep20-21swapper/200:25:542
201512500,0sleep10-21swapper/121:51:101
18332500,0sleep10-21swapper/121:19:451
158392500,0sleep0111rcuc/022:13:430
158302500,1sleep2321ktimersoftd/222:30:112
305502490,0sleep10-21swapper/123:36:431
261872480,0sleep30-21swapper/300:05:303
134462480,0sleep10-21swapper/122:29:581
144022450,0sleep00-21swapper/000:20:520
132192440,0sleep20-21swapper/223:48:102
232472430,1sleep01-21systemd21:28:300
239882360,0sleep30-21swapper/322:27:413
133362340,0sleep20-21swapper/200:13:542
128292340,0sleep20-21swapper/223:44:492
148202307,7sleep30-21swapper/319:06:193
150922287,7sleep00-21swapper/019:09:390
149462287,7sleep20-21swapper/219:07:492
157272220,1sleep215720-21lspci22:10:202
133342219,7sleep10-21swapper/119:05:121
311642200,0sleep10-21swapper/123:23:421
277692190,1sleep2321ktimersoftd/221:55:142
1523099190,12cyclictest4017-21lspci21:39:420
221702170,1sleep01095-21nfsd23:03:210
15236991715,1cyclictest3160-21kworker/1:221:43:521
133142170,0sleep10-21swapper/100:24:261
262322160,0sleep126229-21sshd00:36:441
247342160,2sleep21524399cyclictest23:52:342
237722160,0sleep00-21swapper/022:07:590
126992160,1sleep012702-21processes23:15:190
236562150,0sleep10-21swapper/100:32:501
207072150,1sleep2311rcuc/223:55:232
15236991514,0cyclictest6783-21kworker/1:323:43:171
15236991514,0cyclictest26897-21kworker/1:123:10:211
15236991514,0cyclictest26168-21kworker/1:121:09:211
15236991514,0cyclictest18816-21kworker/1:023:59:141
1523099150,14cyclictest0-21swapper/021:59:120
86672140,0sleep20-21swapper/223:11:402
1524399140,1cyclictest0-21swapper/221:12:492
15236991414,0cyclictest4085-21kworker/1:023:07:241
15236991414,0cyclictest12234-21kworker/1:400:06:091
15236991412,1cyclictest16434-21kworker/1:422:04:591
1523099147,6cyclictest670-21packagekitd23:14:520
53552130,0sleep10-21swapper/122:32:231
174832130,0sleep00-21swapper/022:50:000
15243991310,2cyclictest646-21dbus-daemon23:28:452
1524399130,1cyclictest646-21dbus-daemon22:38:482
1523699136,2cyclictest25-21ksoftirqd/123:29:201
15236991313,0cyclictest7617-21kworker/1:000:18:071
15236991313,0cyclictest523-40kipmi021:29:451
15236991313,0cyclictest4991-21kworker/1:222:41:041
15236991313,0cyclictest4085-21kworker/1:023:01:511
15236991312,0cyclictest33050irq/27-ahci[00022:18:281
1523099139,3cyclictest0-21swapper/021:42:330
1523099135,7cyclictest649-21avahi-daemon21:46:270
1523099130,1cyclictest17299-21snmpd21:14:110
1523099130,12cyclictest0-21swapper/023:45:130
115572130,0sleep1231rcuc/122:52:331
254072120,0sleep025402-21bash00:36:390
180862120,0sleep30-21swapper/300:04:483
15251991210,1cyclictest2309-21bash21:59:153
15251991210,1cyclictest12244-21lspci23:44:463
1525199120,11cyclictest25728-21sshd22:47:353
1525199120,11cyclictest24916-21sshd00:22:063
1525199120,11cyclictest23452-21sshd23:16:263
1525199120,0cyclictest0-21swapper/322:15:563
15243991210,1cyclictest0-21swapper/221:43:582
1524399120,1cyclictest7954-21sshd23:21:182
1524399120,1cyclictest21522-21sshd22:01:092
1524399120,1cyclictest10911-21sshd00:16:592
1524399120,11cyclictest0-21swapper/222:49:582
1523699120,11cyclictest4806-21sshd22:22:271
1523099125,6cyclictest684-21polkitd21:18:340
1523099125,6cyclictest649-21avahi-daemon00:07:590
1523099125,6cyclictest29032-21sshd23:23:280
1523099124,7cyclictest649-21avahi-daemon22:31:050
1523099124,7cyclictest649-21avahi-daemon21:23:560
1523099124,2cyclictest649-21avahi-daemon22:17:550
1523099124,2cyclictest648-21systemd-logind23:57:450
1525199119,1cyclictest27283-21bash00:33:163
1525199119,1cyclictest16560-21grep00:10:553
15251991110,0cyclictest0-21swapper/322:21:053
1525199110,10cyclictest0-21swapper/323:45:153
15243991110,0cyclictest0-21swapper/221:20:282
1524399110,10cyclictest0-21swapper/223:09:592
1524399110,10cyclictest0-21swapper/222:44:092
1523699118,1cyclictest25-21ksoftirqd/123:50:231
1523699112,1cyclictest25-21ksoftirqd/123:32:081
1523699112,1cyclictest25-21ksoftirqd/121:55:471
15236991111,0cyclictest8055-21kworker/1:223:45:021
15236991111,0cyclictest3160-21kworker/1:221:49:131
15236991111,0cyclictest29289-21kworker/1:122:48:091
15236991111,0cyclictest19738-21kworker/1:000:28:311
15236991111,0cyclictest15935-21kworker/1:200:12:461
1523099115,5cyclictest649-21avahi-daemon22:22:160
1523099114,6cyclictest649-21avahi-daemon22:51:170
1523099114,6cyclictest31049-21sshd23:43:230
1523099114,1cyclictest700-21gdbus22:35:200
1523099114,1cyclictest700-21gdbus22:25:190
1523099114,1cyclictest684-21polkitd22:55:220
1523099114,1cyclictest684-21polkitd00:34:040
1523099114,1cyclictest649-21avahi-daemon00:27:390
1523099114,1cyclictest649-21avahi-daemon00:11:180
1523099114,1cyclictest646-21dbus-daemon23:30:060
1523099114,1cyclictest616-17auditd23:36:150
1523099114,1cyclictest18940-21sshd22:00:510
1523099113,2cyclictest700-21gdbus23:51:080
1523099113,2cyclictest646-21dbus-daemon23:05:150
1523099113,2cyclictest646-21dbus-daemon19:30:000
1523099113,2cyclictest14617-21sshd21:30:480
1525199100,0cyclictest0-21swapper/322:53:263
1525199100,0cyclictest0-21swapper/321:32:353
1524399107,2cyclictest697-21gdbus21:53:292
1524399100,0cyclictest0-21swapper/200:37:202
1523699109,0cyclictest0-21swapper/121:30:071
1523699109,0cyclictest0-21swapper/121:12:471
1523699109,0cyclictest0-21swapper/120:29:571
1523699102,1cyclictest25-21ksoftirqd/121:21:531
1523699101,1cyclictest131rcu_sched23:15:181
1523699101,1cyclictest131rcu_sched00:04:061
1523699100,1cyclictest131rcu_sched22:05:131
1523099109,0cyclictest0-21swapper/021:06:310
1523099103,1cyclictest700-21gdbus00:15:130
1523099103,1cyclictest684-21polkitd19:20:000
1523099102,2cyclictest646-21dbus-daemon20:49:540
1523099100,9cyclictest0-21swapper/019:36:120
152519998,0cyclictest0-21swapper/322:35:553
152519998,0cyclictest0-21swapper/322:03:063
152519998,0cyclictest0-21swapper/321:52:063
152519998,0cyclictest0-21swapper/321:22:253
152519996,2cyclictest1011-21sshd22:13:193
152439998,0cyclictest75650irq/25-eno1-tx-22:08:482
152439998,0cyclictest0-21swapper/223:01:082
152439998,0cyclictest0-21swapper/222:55:182
152439998,0cyclictest0-21swapper/200:31:392
152369998,0cyclictest17992-21kworker/1:020:43:211
152309993,1cyclictest4431-21cat20:00:150
152309993,1cyclictest17299-21snmpd20:56:080
152309993,1cyclictest17299-21snmpd20:29:110
152309993,1cyclictest17299-21snmpd19:40:210
152309992,2cyclictest8827-21cut20:10:170
152309992,1cyclictest3906-21cat20:00:000
152309992,1cyclictest29324-21pmu-power21:00:210
152309992,1cyclictest17299-21snmpd19:30:510
19433280,1sleep319432-21tty23:06:183
152519987,0cyclictest0-21swapper/322:58:453
152519987,0cyclictest0-21swapper/320:39:543
152519986,1cyclictest31925-21lspci21:39:143
152519986,1cyclictest28280-21lspci23:23:223
152519986,1cyclictest22299-21lspci23:32:383
152519986,1cyclictest15196-21lspci00:39:053
152519986,1cyclictest13530-21sshd22:06:553
152519985,2cyclictest649-21avahi-daemon23:25:073
152519985,2cyclictest649-21avahi-daemon23:02:153
152519985,2cyclictest649-21avahi-daemon21:29:353
152519985,2cyclictest6417-21sshd21:10:003
152519985,1cyclictest684-21polkitd21:44:383
152519980,0cyclictest0-21swapper/319:29:443
152439987,0cyclictest0-21swapper/222:21:582
152439987,0cyclictest0-21swapper/222:16:492
152439986,1cyclictest5313-21lspci21:26:372
152439985,2cyclictest649-21avahi-daemon23:36:102
152439985,2cyclictest649-21avahi-daemon22:50:082
152439982,5cyclictest616-13auditd22:27:242
152369988,0cyclictest523-40kipmi019:18:341
152369988,0cyclictest13084-21kworker/1:219:14:481
152369988,0cyclictest11921-21kworker/1:120:22:331
152369987,0cyclictest0-21swapper/120:05:171
152369986,1cyclictest25-21ksoftirqd/122:10:051
152309982,1cyclictest77522chrt20:09:200
152309982,1cyclictest32530-21entropy19:50:120
152309982,1cyclictest30938-21awk19:45:180
152309982,1cyclictest24652-21gltestperf20:50:150
152309982,1cyclictest18250-21cpu20:35:180
152309982,1cyclictest17299-21snmpd19:20:020
152309982,1cyclictest17299-21snmpd19:13:120
152309982,1cyclictest17048-21ntpq20:30:190
152309982,1cyclictest10091-21/usr/sbin/munin20:15:190
152519976,0cyclictest0-21swapper/320:44:543
152519976,0cyclictest0-21swapper/319:11:343
152519975,1cyclictest649-21avahi-daemon21:15:033
152519975,1cyclictest646-21dbus-daemon22:40:353
152519975,1cyclictest41-21ksoftirqd/319:20:003
152519975,1cyclictest17329-21lspci22:30:183
152519974,2cyclictest649-21avahi-daemon00:28:573
152519974,2cyclictest646-21dbus-daemon00:16:433
152519974,2cyclictest12051-21sshd23:38:103
152519974,1cyclictest11083-21sshd23:54:293
152519973,2cyclictest1646-21sshd23:56:443
152519970,6cyclictest0-21swapper/319:35:343
152439976,0cyclictest0-21swapper/221:16:082
152439975,1cyclictest700-21gdbus21:45:172
152439975,1cyclictest24654-21grep20:50:142
152439975,1cyclictest11067-21lspci00:20:312
152369977,0cyclictest27967-21kworker/1:019:49:481
152309972,1cyclictest21940-21chrt20:43:280
152519965,0cyclictest0-21swapper/319:58:353
152519965,0cyclictest0-21swapper/319:20:353
152519964,1cyclictest646-21dbus-daemon20:05:003
152519963,2cyclictest17299-21snmpd20:28:343
152519960,5cyclictest0-21swapper/320:07:243
152439964,1cyclictest23401-21grep19:25:232
152439964,1cyclictest17299-21snmpd20:57:142
152439963,2cyclictest646-21dbus-daemon21:08:112
152439960,5cyclictest0-21swapper/219:53:082
152369966,0cyclictest9841-21kworker/1:020:15:371
152369966,0cyclictest3720-21kworker/1:220:03:101
152369966,0cyclictest32042-21kworker/1:119:56:131
152369966,0cyclictest32042-21kworker/1:119:51:011
152369966,0cyclictest26168-21kworker/1:120:57:031
152369966,0cyclictest24093-21kworker/1:220:51:511
152369966,0cyclictest13084-21kworker/1:219:35:261
152369966,0cyclictest13084-21kworker/1:219:25:311
152369966,0cyclictest13084-21kworker/1:219:21:541
152369966,0cyclictest11921-21kworker/1:120:31:131
152369965,0cyclictest3720-21kworker/1:220:10:261
152369963,2cyclictest17299-21snmpd21:01:191
152369963,2cyclictest17299-21snmpd20:45:231
152369963,2cyclictest17299-21snmpd19:43:031
152369963,2cyclictest17299-21snmpd19:33:481
152369960,5cyclictest0-21swapper/120:38:181
152309961,1cyclictest13438-21sed20:20:210
152519954,0cyclictest0-21swapper/320:22:453
152519953,1cyclictest8466-21hwlatdetect20:10:153
152519953,1cyclictest30876-21ls19:45:183
152519953,1cyclictest24803-21cat20:50:143
152519953,1cyclictest24598-21cat19:30:153
152519953,1cyclictest17299-21snmpd21:00:363
152519953,1cyclictest17299-21snmpd20:55:503
152519953,1cyclictest17299-21snmpd20:30:533
152519953,1cyclictest17299-21snmpd19:51:243
152519952,2cyclictest1053-21lldpd20:15:443
152519950,2cyclictest28832-21/usr/sbin/munin19:40:153
152439953,1cyclictest649-21avahi-daemon19:15:462
152439953,1cyclictest6023-21/usr/sbin/munin20:05:212
152439953,1cyclictest25256-21timerandwakeup19:30:192
152439953,1cyclictest2428-21cat19:55:172
152439953,1cyclictest17299-21snmpd21:03:012
152439953,1cyclictest17299-21snmpd20:41:382
152439953,1cyclictest17299-21snmpd20:35:032
152439953,1cyclictest17299-21snmpd20:25:522
152439953,1cyclictest17299-21snmpd20:15:372
152439953,1cyclictest17299-21snmpd20:11:082
152439953,1cyclictest17299-21snmpd20:01:312
152439953,1cyclictest17299-21snmpd19:45:132
152439953,1cyclictest17299-21snmpd19:42:182
152439953,1cyclictest17299-21snmpd19:20:392
152439953,1cyclictest16301-21grep20:30:152
152439953,1cyclictest0-21swapper/220:45:202
152439950,4cyclictest12251-21grep20:20:142
152439950,0cyclictest0-21swapper/219:36:482
152519942,1cyclictest17299-21snmpd20:45:033
152439942,1cyclictest15834-21basename19:10:162
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional