You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-04 - 20:44
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2.osadl.org (updated Thu Dec 04, 2025 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1353424938,0sleep30-21swapper/306:15:553
1368223925,0sleep00-21swapper/006:17:360
1357523826,0sleep20-21swapper/206:16:222
1346823526,0sleep10-21swapper/106:15:051
329970,0watchdog/30-21swapper/311:42:293
329970,0watchdog/30-21swapper/310:25:013
329970,0watchdog/30-21swapper/307:29:213
329970,0watchdog/30-21swapper/307:09:303
329970,0watchdog/30-21swapper/307:04:303
29925070,0irq/32-eth0-rx-0-21swapper/310:34:273
29925070,0irq/32-eth0-rx-0-21swapper/309:39:273
29925070,0irq/32-eth0-rx-0-21swapper/308:34:283
4160,0ktimersoftd/00-21swapper/010:09:270
35160,0ktimersoftd/30-21swapper/310:44:303
339960,0migration/34526-21sh11:14:513
339960,0migration/327400-21sh10:51:533
339960,0migration/320000-21sh09:11:353
329960,0watchdog/30-21swapper/311:24:173
329960,0watchdog/30-21swapper/311:22:213
329960,0watchdog/30-21swapper/310:42:173
329960,0watchdog/30-21swapper/310:19:453
329960,0watchdog/30-21swapper/310:15:493
329960,0watchdog/30-21swapper/310:09:253
329960,0watchdog/30-21swapper/310:00:053
329960,0watchdog/30-21swapper/309:36:013
329960,0watchdog/30-21swapper/309:27:573
329960,0watchdog/30-21swapper/309:19:093
329960,0watchdog/30-21swapper/309:14:213
329960,0watchdog/30-21swapper/309:14:213
329960,0watchdog/30-21swapper/309:07:293
329960,0watchdog/30-21swapper/309:00:333
329960,0watchdog/30-21swapper/308:57:173
329960,0watchdog/30-21swapper/308:49:093
329960,0watchdog/30-21swapper/308:45:253
329960,0watchdog/30-21swapper/308:40:333
329960,0watchdog/30-21swapper/308:31:053
329960,0watchdog/30-21swapper/307:27:133
329960,0watchdog/30-21swapper/306:54:303
329960,0watchdog/30-21swapper/306:33:063
29925060,0irq/32-eth0-rx-23908-21ssh09:54:213
29925060,0irq/32-eth0-rx-20146-21port_sshpriv09:29:323
29925060,0irq/32-eth0-rx-14519-21diskstats06:19:233
29925060,0irq/32-eth0-rx-10737-21munin-node10:59:293
29925060,0irq/32-eth0-rx-0-21swapper/311:45:513
29925060,0irq/32-eth0-rx-0-21swapper/311:32:513
29925060,0irq/32-eth0-rx-0-21swapper/311:04:273
29925060,0irq/32-eth0-rx-0-21swapper/310:04:523
29925060,0irq/32-eth0-rx-0-21swapper/308:27:343
29925060,0irq/32-eth0-rx-0-21swapper/307:44:233
29925060,0irq/32-eth0-rx-0-21swapper/307:34:203
149960,0migration/08115-21idleruntime-cro10:04:080
149960,0migration/031463-21kthreadcore08:39:290
149960,0migration/028072-21sh09:15:360
149960,0migration/028072-21sh09:15:360
149960,0migration/027182-21missed_timers06:24:300
149960,0migration/021200-21sh09:31:170
149960,0migration/012003-21sh11:01:330
138879960,0cyclictest0-21swapper/110:20:011
138879960,0cyclictest0-21swapper/110:07:151
138879960,0cyclictest0-21swapper/109:38:331
8315050,0irq/26-0000:00:0-21swapper/307:39:113
339950,0migration/34968-21irqrtprio07:19:263
339950,0migration/331990-21kthreadcore10:54:293
339950,0migration/331879-21aten_r1power_cu11:34:213
339950,0migration/319472-21kthreadcore06:44:263
339950,0migration/317341-21munin-node08:09:343
339950,0migration/315418-21kthreadcore06:39:273
329950,0watchdog/30-21swapper/310:29:173
329950,0watchdog/30-21swapper/309:51:053
329950,0watchdog/30-21swapper/309:44:493
329950,0watchdog/30-21swapper/308:19:133
329950,0watchdog/30-21swapper/308:16:253
329950,0watchdog/30-21swapper/308:05:373
329950,0watchdog/30-21swapper/307:59:253
329950,0watchdog/30-21swapper/307:54:333
329950,0watchdog/30-21swapper/307:49:093
329950,0watchdog/30-21swapper/307:00:063
329950,0watchdog/30-21swapper/306:49:143
329950,0watchdog/30-21swapper/306:34:263
329950,0watchdog/30-21swapper/306:24:183
29935050,0irq/33-eth0-tx-4328-21munin-node09:04:382
29935050,0irq/33-eth0-tx-3566-21munin-node07:19:260
29935050,0irq/33-eth0-tx-31077-21ssh11:32:412
29935050,0irq/33-eth0-tx-2826-21munin-node10:39:242
29935050,0irq/33-eth0-tx-25318-21diskmemload10:23:532
29935050,0irq/33-eth0-tx-18644-21ssh10:46:102
29935050,0irq/33-eth0-tx-16584-21tr09:09:262
29935050,0irq/33-eth0-tx-0-21swapper/211:10:282
29935050,0irq/33-eth0-tx-0-21swapper/208:39:222
29935050,0irq/33-eth0-tx-0-21swapper/208:31:572
29935050,0irq/33-eth0-tx-0-21swapper/007:54:200
29935050,0irq/33-eth0-tx-0-21swapper/006:39:300
29935050,0irq/33-eth0-tx-0-21swapper/006:29:310
29925050,0irq/32-eth0-rx-0-21swapper/311:09:203
259950,0migration/231872-21taskset07:35:122
259950,0migration/231659-21fschecks_time09:39:232
259950,0migration/229399-21kthreadcore06:49:302
259950,0migration/227424-21munin-node07:09:312
259950,0migration/226204-21munin-node10:49:312
259950,0migration/224279-21kthreadcore07:54:262
259950,0migration/221829-21kthreadcore11:44:282
259950,0migration/217911-21switchtime08:49:342
249950,0watchdog/20-21swapper/210:33:212
19150,0ktimersoftd/10-21swapper/108:52:031
179950,0migration/18144-21kthreadcore11:19:271
179950,0migration/15188-21fschecks_time09:04:261
179950,0migration/14526-21fschecks_time08:04:321
179950,0migration/130199-21kthreadcore07:34:261
179950,0migration/129359-21latency_hist09:19:091
179950,0migration/120300-21sh11:06:101
179950,0migration/118900-21kthreadcore11:44:271
179950,0migration/114200-21sh09:46:261
179950,0migration/113625-21munin-node06:59:361
149950,0migration/09692-21irqcore10:04:290
149950,0migration/09315-21smart_nvme0n108:44:340
149950,0migration/08528-21kernelversion09:44:290
149950,0migration/08353-21kthreadcore09:24:270
149950,0migration/06414-21taskset09:43:300
149950,0migration/04797-21kthreadcore10:39:280
149950,0migration/031830-21processes08:19:320
149950,0migration/031176-21if_eth011:14:250
149950,0migration/030148-21taskset09:38:010
149950,0migration/029928-21munin-node09:19:230
149950,0migration/029222-21kthreadcore10:34:270
149950,0migration/027748-21pmu-power11:09:300
149950,0migration/02603-21unixbench-2d08:59:350
149950,0migration/02556-21kthreadcore11:34:280
149950,0migration/021245-21kthreadcore09:49:290
149950,0migration/020394-21sh09:12:510
149950,0migration/019836-21latency_hist08:54:090
149950,0migration/019408-21kthreadcore11:24:270
149950,0migration/017684-21taskset08:09:450
149950,0migration/016228-21kthreadcore11:04:260
149950,0migration/013446-21kthreadcore10:24:300
149950,0migration/011034-21kthreadcore06:59:270
138879955,0cyclictest0-21swapper/110:40:401
138879955,0cyclictest0-21swapper/107:09:431
138879954,0cyclictest23871-21snmpd08:55:351
138879954,0cyclictest0-21swapper/106:39:271
138879950,0cyclictest0-21swapper/111:09:501
138879950,0cyclictest0-21swapper/110:49:391
138879950,0cyclictest0-21swapper/109:00:241
138879950,0cyclictest0-21swapper/108:47:571
8315040,0irq/26-0000:00:0-21swapper/209:34:142
29935040,0irq/33-eth0-tx-382-21ssh08:21:402
29935040,0irq/33-eth0-tx-29079-21telnet07:59:220
29935040,0irq/33-eth0-tx-15364-21munin-node09:49:272
29935040,0irq/33-eth0-tx-0-21swapper/211:24:172
29935040,0irq/33-eth0-tx-0-21swapper/211:19:112
29935040,0irq/33-eth0-tx-0-21swapper/211:04:482
29935040,0irq/33-eth0-tx-0-21swapper/211:01:142
29935040,0irq/33-eth0-tx-0-21swapper/210:55:132
29935040,0irq/33-eth0-tx-0-21swapper/210:24:102
29935040,0irq/33-eth0-tx-0-21swapper/210:11:172
29935040,0irq/33-eth0-tx-0-21swapper/210:04:292
29935040,0irq/33-eth0-tx-0-21swapper/210:00:542
29935040,0irq/33-eth0-tx-0-21swapper/209:54:392
29935040,0irq/33-eth0-tx-0-21swapper/209:44:242
29935040,0irq/33-eth0-tx-0-21swapper/209:29:242
29935040,0irq/33-eth0-tx-0-21swapper/209:25:302
29935040,0irq/33-eth0-tx-0-21swapper/209:19:152
29935040,0irq/33-eth0-tx-0-21swapper/209:15:282
29935040,0irq/33-eth0-tx-0-21swapper/209:15:282
29935040,0irq/33-eth0-tx-0-21swapper/208:59:142
29935040,0irq/33-eth0-tx-0-21swapper/208:44:482
29935040,0irq/33-eth0-tx-0-21swapper/208:24:252
29925040,0irq/32-eth0-rx-0-21swapper/307:14:233
259940,0migration/29028-21kthreadcore06:34:282
259940,0migration/25768-21kthreadcore08:04:362
259940,0migration/232511-21fschecks_count11:34:242
259940,0migration/224902-21kthreadcore08:54:282
259940,0migration/222162-21kthreadcore08:34:282
259940,0migration/21935-21kthreadcore07:59:282
259940,0migration/216484-21kthreadcore07:24:292
259940,0migration/215475-21irqcore06:19:272
259940,0migration/210073-21kthreadcore06:59:272
249940,0watchdog/228025-21chrt06:26:302
249940,0watchdog/225395-21aten2.4-expect10:14:212
249940,0watchdog/211780-21iostat_ios06:39:262
249940,0watchdog/2105412sleep207:23:372
249940,0watchdog/20-21swapper/211:14:212
249940,0watchdog/20-21swapper/207:04:222
249940,0watchdog/20-21swapper/206:54:222
179940,0migration/19840-21kthreadcore07:44:261
179940,0migration/16145-21ntp_offset07:39:291
179940,0migration/14901-21kthreadcore06:54:261
179940,0migration/1348-21kthreadcore07:59:281
179940,0migration/124755-21sh08:34:301
179940,0migration/122105-21fschecks_time06:24:251
179940,0migration/121144-21kthreadcore09:49:291
179940,0migration/119917-21kthreadcore08:14:341
179940,0migration/118531-21kthreadcore11:24:271
179940,0migration/114219-21aten_r1power_po06:19:211
179940,0migration/112613-21irqcore10:44:261
169940,0watchdog/16559-21if_err_eth1.309:24:251
169940,0watchdog/119469-21cut09:29:291
169940,0watchdog/118052-21kthreadcore07:49:291
169940,0watchdog/111318-21irqstats10:24:291
169940,0watchdog/10-21swapper/111:34:291
169940,0watchdog/10-21swapper/111:14:211
169940,0watchdog/10-21swapper/109:41:411
169940,0watchdog/10-21swapper/107:04:221
149940,0migration/07081-21fschecks_time11:19:230
149940,0migration/032746-21cpuspeed_turbos07:39:220
149940,0migration/028389-21kthreadcore06:49:300
149940,0migration/024735-21timerwakeupswit10:29:380
149940,0migration/010051-21kthreadcore11:39:280
138889943,0cyclictest9360-21grep11:39:272
138889943,0cyclictest24734-21dump-pmu-power06:49:092
138889943,0cyclictest19507-21awk08:14:332
138889943,0cyclictest14322-21sadc07:49:092
138889940,0cyclictest0-21swapper/210:34:212
138879944,0cyclictest0-21swapper/109:18:411
138879944,0cyclictest0-21swapper/109:18:411
138879944,0cyclictest0-21swapper/107:19:201
138879943,0cyclictest2673-21awk08:24:251
138879943,0cyclictest26655-21awk08:19:261
138879943,0cyclictest26067-21awk06:49:281
138879943,0cyclictest23871-21snmpd10:58:361
138879943,0cyclictest23871-21snmpd10:37:361
138879943,0cyclictest23871-21snmpd08:31:351
138879943,0cyclictest23871-21snmpd08:11:201
138879943,0cyclictest23871-21snmpd07:26:351
138879943,0cyclictest23871-21snmpd07:16:341
138879943,0cyclictest23871-21snmpd06:46:341
138879943,0cyclictest23871-21snmpd06:37:341
138879943,0cyclictest19088-21awk07:29:251
138879943,0cyclictest17975-21awk10:09:251
138879943,0cyclictest13770-21awk09:09:231
138879940,0cyclictest11336-21kthreadcore11:39:281
138869943,0cyclictest4992-21awk08:04:340
138869943,0cyclictest29761-21awk07:14:260
138869943,0cyclictest23871-21snmpd10:19:500
138869943,0cyclictest23871-21snmpd09:05:500
138869943,0cyclictest23871-21snmpd07:09:040
138869943,0cyclictest19851-21awk08:34:260
138869943,0cyclictest17830-21sadc07:29:090
138869943,0cyclictest11320-21awk08:29:250
138869943,0cyclictest10347-21sadc06:39:090
138869942,0cyclictest8139-21kworker/0:010:15:070
138869940,0cyclictest0-21swapper/008:25:520
249930,0watchdog/20-21swapper/207:49:292
249930,0watchdog/20-21swapper/207:39:212
249930,0watchdog/20-21swapper/206:29:302
179930,0migration/131703-21sh09:58:171
179930,0migration/128023-21if_err_eth1.208:39:261
169930,0watchdog/132674-21ssh10:16:531
169930,0watchdog/10-21swapper/111:33:571
169930,0watchdog/10-21swapper/109:59:091
169930,0watchdog/10-21swapper/106:29:301
138889930,0cyclictest0-21swapper/207:16:242
138879932,0cyclictest5805-21awk10:59:251
138879932,0cyclictest23871-21snmpd07:57:051
138879930,0cyclictest19456-21iostat10:29:291
138869933,0cyclictest31464-21kworker/0:107:49:320
138869933,0cyclictest31464-21kworker/0:107:44:260
138869933,0cyclictest26618-21kworker/0:210:54:170
138869933,0cyclictest26618-21kworker/0:210:44:190
138869933,0cyclictest24837-21kworker/0:006:19:140
138869933,0cyclictest2297-21kworker/0:207:10:340
138869933,0cyclictest2297-21kworker/0:206:54:160
138869933,0cyclictest21074-21awk08:54:260
138869933,0cyclictest12036-21kworker/0:008:15:120
138869932,0cyclictest27588-21cut07:34:250
138869932,0cyclictest24443-21awk11:29:250
138869932,0cyclictest24117-21cut09:54:220
138869932,0cyclictest21085-21awk10:49:260
138869930,0cyclictest18054-21munin-node07:29:230
138869930,0cyclictest17902-21iostat_ios11:44:260
138869930,0cyclictest17778-21munin-node06:44:340
249920,0watchdog/20-21swapper/208:09:132
249920,0watchdog/20-21swapper/207:29:212
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional