You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-15 - 00:52
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack1slot2.osadl.org (updated Fri Nov 14, 2025 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
905425041,0sleep20-21swapper/206:18:122
909024940,0sleep10-21swapper/106:18:391
887023828,0sleep30-21swapper/306:16:053
890123524,0sleep00-21swapper/006:16:260
9307991010,0cyclictest4585-21sadc09:30:200
93099976,0cyclictest10176-21fschecks_time06:20:342
329970,0watchdog/3299250irq/32-eth0-rx-10:20:413
329970,0watchdog/30-21swapper/311:30:413
29925070,0irq/32-eth0-rx-0-21swapper/311:42:223
93099960,0cyclictest0-21swapper/211:10:212
93089960,0cyclictest0-21swapper/109:30:391
93089960,0cyclictest0-21swapper/109:03:131
93089960,0cyclictest0-21swapper/108:20:391
339960,0migration/37570-21fschecks_time11:45:353
339960,0migration/328795-21sh11:36:013
339960,0migration/324322-21sh10:37:083
329960,0watchdog/36119-21iostat_ios09:30:413
329960,0watchdog/328267-21kthreadcore10:00:413
329960,0watchdog/322552-21df08:20:333
329960,0watchdog/30-21swapper/311:26:493
329960,0watchdog/30-21swapper/311:05:213
329960,0watchdog/30-21swapper/311:00:413
329960,0watchdog/30-21swapper/310:56:293
329960,0watchdog/30-21swapper/310:52:493
329960,0watchdog/30-21swapper/310:17:533
329960,0watchdog/30-21swapper/310:13:173
329960,0watchdog/30-21swapper/309:56:253
329960,0watchdog/30-21swapper/309:37:573
329960,0watchdog/30-21swapper/309:10:493
329960,0watchdog/30-21swapper/309:06:453
329960,0watchdog/30-21swapper/308:56:213
329960,0watchdog/30-21swapper/308:45:453
329960,0watchdog/30-21swapper/308:35:373
329960,0watchdog/30-21swapper/308:31:573
329960,0watchdog/30-21swapper/308:10:333
329960,0watchdog/30-21swapper/307:53:063
329960,0watchdog/30-21swapper/307:40:423
329960,0watchdog/30-21swapper/307:30:343
329960,0watchdog/30-21swapper/306:37:583
29935060,0irq/33-eth0-tx-0-21swapper/008:20:210
29935060,0irq/33-eth0-tx-0-21swapper/008:15:370
29925060,0irq/32-eth0-rx-486-21cpu10:25:343
29925060,0irq/32-eth0-rx-31958-21python10:40:483
29925060,0irq/32-eth0-rx-0-21swapper/309:20:573
29925060,0irq/32-eth0-rx-0-21swapper/309:15:363
29925060,0irq/32-eth0-rx-0-21swapper/308:40:413
29925060,0irq/32-eth0-rx-0-21swapper/308:29:013
149960,0migration/028231-21sh09:24:390
149960,0migration/026592-21kthreadcore10:20:410
149960,0migration/026444-21munin-node08:00:400
149960,0migration/020223-21sh11:15:230
149960,0migration/014278-21sh11:48:080
149960,0migration/012233-21kthreadcore09:50:410
93099955,0cyclictest0-21swapper/208:24:202
93099950,0cyclictest0-21swapper/209:31:202
93099950,0cyclictest0-21swapper/209:02:202
93089955,0cyclictest0-21swapper/111:46:051
93089955,0cyclictest0-21swapper/109:26:521
93089950,0cyclictest0-21swapper/110:36:521
93089950,0cyclictest0-21swapper/110:33:051
93089950,0cyclictest0-21swapper/109:39:191
93089950,0cyclictest0-21swapper/109:18:401
339950,0migration/310331-21aten_r1power_en07:05:333
329950,0watchdog/329154-21diskstats09:25:333
329950,0watchdog/30-21swapper/311:21:093
329950,0watchdog/30-21swapper/310:31:573
329950,0watchdog/30-21swapper/310:05:293
329950,0watchdog/30-21swapper/309:45:453
329950,0watchdog/30-21swapper/309:40:333
329950,0watchdog/30-21swapper/309:00:293
329950,0watchdog/30-21swapper/308:50:413
329950,0watchdog/30-21swapper/308:15:253
329950,0watchdog/30-21swapper/308:05:533
329950,0watchdog/30-21swapper/308:00:263
329950,0watchdog/30-21swapper/307:55:223
329950,0watchdog/30-21swapper/307:46:263
329950,0watchdog/30-21swapper/307:35:263
329950,0watchdog/30-21swapper/307:27:063
329950,0watchdog/30-21swapper/307:21:503
329950,0watchdog/30-21swapper/307:15:423
329950,0watchdog/30-21swapper/307:11:023
329950,0watchdog/30-21swapper/307:00:303
329950,0watchdog/30-21swapper/306:55:343
329950,0watchdog/30-21swapper/306:51:223
329950,0watchdog/30-21swapper/306:45:223
329950,0watchdog/30-21swapper/306:45:223
329950,0watchdog/30-21swapper/306:40:263
329950,0watchdog/30-21swapper/306:30:543
329950,0watchdog/30-21swapper/306:25:223
329950,0watchdog/30-21swapper/306:20:263
29935050,0irq/33-eth0-tx-31291-21latency_hist10:05:212
29935050,0irq/33-eth0-tx-27251-21munin-node09:05:422
29935050,0irq/33-eth0-tx-1660-21munin-node08:50:442
29935050,0irq/33-eth0-tx-14293-21tail07:30:340
29935050,0irq/33-eth0-tx-0-21swapper/210:22:412
29935050,0irq/33-eth0-tx-0-21swapper/007:24:550
29925050,0irq/32-eth0-rx-0-21swapper/311:15:373
29925050,0irq/32-eth0-rx-0-21swapper/311:10:423
29925050,0irq/32-eth0-rx-0-21swapper/310:45:213
29925050,0irq/32-eth0-rx-0-21swapper/309:50:243
249950,0watchdog/212713-21memory11:45:412
179950,0migration/131148-21taskset08:28:171
179950,0migration/128932-21missed_timers10:00:421
179950,0migration/116893-21aten_r1power_po08:40:351
179950,0migration/11293-21gltestperf10:25:351
169950,0watchdog/128955-21aten2.4-expect11:20:331
169950,0watchdog/113615-21cut11:10:411
169950,0watchdog/111182-21tr10:10:451
149950,0migration/08566-21taskset10:46:110
149950,0migration/07934-21cpuspeed_turbos08:10:340
149950,0migration/07522-21kthreadcore11:25:380
149950,0migration/07270-21latency_hist10:10:210
149950,0migration/06090-21kthreadcore08:50:400
149950,0migration/05905-21kthreadcore11:05:400
149950,0migration/05019-21sh09:47:160
149950,0migration/03579-21munin-node09:10:350
149950,0migration/032737-21sh08:47:310
149950,0migration/031207-21kthreadcore06:55:390
149950,0migration/031194-21kthreadcore11:20:390
149950,0migration/029972-21mdstat06:30:410
149950,0migration/029800-21sh11:39:210
149950,0migration/028589-21irqcore09:05:390
149950,0migration/025459-21latency_hist10:40:210
149950,0migration/024990-21kthreadcore06:50:380
149950,0migration/022026-21fschecks_count07:35:390
149950,0migration/022010-21taskset09:58:170
149950,0migration/02151-21latency07:40:430
149950,0migration/018632-21latency_hist10:55:210
149950,0migration/017907-21kthreadcore09:35:410
149950,0migration/015395-21timerwakeupswit10:30:480
149950,0migration/015220-21kthreadcore11:30:400
149950,0migration/014990-21kthreadcore08:55:390
149950,0migration/01461-21if_eth010:25:370
149950,0migration/014045-21kthreadcore11:10:410
149950,0migration/011520-21irqcore10:50:380
93099944,0cyclictest0-21swapper/210:36:172
93099943,0cyclictest3599-21lldpd08:17:102
93099943,0cyclictest3599-21lldpd08:10:402
93099943,0cyclictest2681-21sadc07:00:212
93099943,0cyclictest26520-21awk08:00:412
93099943,0cyclictest23871-21snmpd08:09:462
93099943,0cyclictest21867-21awk06:50:362
93099943,0cyclictest19826-21awk10:55:392
93099943,0cyclictest19264-21awk07:55:372
93099943,0cyclictest17949-21awk06:25:412
93099943,0cyclictest11358-21awk07:05:372
93089944,0cyclictest0-21swapper/111:19:351
93089944,0cyclictest0-21swapper/110:19:031
93089944,0cyclictest0-21swapper/108:48:241
93089943,0cyclictest8123-21awk07:25:381
93089943,0cyclictest7437-21awk09:50:381
93089943,0cyclictest4438-21awk09:10:381
93089943,0cyclictest23058-21awk11:35:401
93089943,0cyclictest15946-21awk08:15:361
93079943,0cyclictest25201-21awk08:00:200
93079943,0cyclictest20063-21sadc09:20:200
93079943,0cyclictest17018-21awk10:15:410
93079943,0cyclictest1114-21kworker/0:008:40:340
93079940,0cyclictest0-21swapper/010:01:020
8315040,0irq/26-0000:00:0-21swapper/208:55:262
29935040,0irq/33-eth0-tx-7451-21munin-node10:10:532
29935040,0irq/33-eth0-tx-32057-21munin-node08:30:352
29935040,0irq/33-eth0-tx-28790-21ssh11:20:312
29935040,0irq/33-eth0-tx-27230-21ssh11:00:322
29935040,0irq/33-eth0-tx-22899-21ssh10:00:332
29935040,0irq/33-eth0-tx-21904-21diskmemload11:30:322
29935040,0irq/33-eth0-tx-20221-21ssh11:15:232
29935040,0irq/33-eth0-tx-12216-21df_abs09:15:322
29935040,0irq/33-eth0-tx-10288-21telnet10:50:332
29935040,0irq/33-eth0-tx-0-21swapper/211:36:192
29935040,0irq/33-eth0-tx-0-21swapper/211:25:302
29935040,0irq/33-eth0-tx-0-21swapper/211:12:242
29935040,0irq/33-eth0-tx-0-21swapper/210:40:382
29935040,0irq/33-eth0-tx-0-21swapper/210:26:012
29935040,0irq/33-eth0-tx-0-21swapper/209:55:232
29935040,0irq/33-eth0-tx-0-21swapper/209:50:222
29935040,0irq/33-eth0-tx-0-21swapper/209:35:322
29935040,0irq/33-eth0-tx-0-21swapper/209:10:302
29935040,0irq/33-eth0-tx-0-21swapper/208:45:482
29935040,0irq/33-eth0-tx-0-21swapper/208:40:292
29935040,0irq/33-eth0-tx-0-21swapper/208:35:402
29935040,0irq/33-eth0-tx-0-21swapper/208:27:482
29935040,0irq/33-eth0-tx-0-21swapper/007:50:360
29935040,0irq/33-eth0-tx-0-21swapper/007:25:460
29935040,0irq/33-eth0-tx-0-21swapper/007:15:370
29935040,0irq/33-eth0-tx-0-21swapper/006:40:350
29935040,0irq/33-eth0-tx-0-21swapper/006:35:420
29935040,0irq/33-eth0-tx-0-21swapper/006:20:340
259940,0migration/24908-21irqcore07:45:402
259940,0migration/23939-21kthreadcore11:40:392
259940,0migration/23800-21kthreadcore10:45:392
259940,0migration/228616-21latency_hist09:25:212
259940,0migration/227968-21munin-node09:40:472
259940,0migration/225505-21kthreadcore07:35:442
259940,0migration/220368-21kthreadcore07:10:482
259940,0migration/217216-21kthreadcore06:45:442
259940,0migration/217216-21kthreadcore06:45:442
259940,0migration/216376-21kthreadcore07:50:412
259940,0migration/216370-21fschecks_count10:15:382
259940,0migration/215933-21runrttasks10:32:032
249940,0watchdog/2469-21kthreadcore06:35:422
249940,0watchdog/231512-21irqcore09:45:412
249940,0watchdog/229918-21irqcore07:40:422
249940,0watchdog/20-21swapper/209:20:412
179940,0migration/19994-21fschecks_count06:20:341
179940,0migration/18150-21switchtime10:45:471
179940,0migration/17918-21kthreadcore11:25:381
179940,0migration/15839-21munin-node08:30:421
179940,0migration/15754-21kthreadcore08:05:401
179940,0migration/14953-21sh11:41:091
179940,0migration/13082-21kthreadcore06:35:431
179940,0migration/130570-21kthreadcore08:00:441
179940,0migration/126544-21sh09:20:451
179940,0migration/122741-21missed_timers06:25:441
179940,0migration/121449-21latency_hist07:35:211
179940,0migration/118183-21fschecks_time07:10:441
179940,0migration/11777-21kthreadcore10:05:441
179940,0migration/116689-21kthreadcore07:30:401
179940,0migration/115872-21irqcore09:55:381
179940,0migration/112375-21kthreadcore07:05:391
179940,0migration/111118-21taskset11:09:021
179940,0migration/111051-21aten_r1power_vo07:50:351
179940,0migration/110859-21fschecks_time10:50:351
169940,0watchdog/1924-21ssh10:44:091
169940,0watchdog/131510-21irqcore09:45:411
169940,0watchdog/128919-21ssh09:42:051
169940,0watchdog/127581-21df_abs11:00:331
169940,0watchdog/125580-21ssh10:56:131
169940,0watchdog/125183-21awk06:30:381
169940,0watchdog/118325-21kthreadcore11:30:411
169940,0watchdog/110627-21taskset07:49:501
169940,0watchdog/10-21swapper/108:35:291
149940,0migration/03408-21kthreadcore08:30:400
149940,0migration/029397-21kthreadcore11:00:380
149940,0migration/023652-21runrttasks06:28:130
93099932,0cyclictest4055-21awk07:00:382
93099930,0cyclictest787-21iostat_ios07:20:382
93089932,0cyclictest8671-21if_err_eth008:10:371
93089932,0cyclictest7424-21awk06:40:381
93089932,0cyclictest29755-21awk07:40:401
93089932,0cyclictest28378-21awk09:05:381
93089932,0cyclictest2799-21awk08:50:371
93089932,0cyclictest14636-21awk06:45:411
93089932,0cyclictest14636-21awk06:45:411
93089932,0cyclictest11555-21awk08:55:371
93079933,0cyclictest14437-21kworker/0:211:41:530
93079933,0cyclictest1114-21kworker/0:010:35:330
93079933,0cyclictest1114-21kworker/0:010:10:270
93079933,0cyclictest1114-21kworker/0:008:36:120
93079933,0cyclictest1114-21kworker/0:008:25:440
93079933,0cyclictest1114-21kworker/0:007:10:280
93079933,0cyclictest1114-21kworker/0:006:45:540
93079933,0cyclictest1114-21kworker/0:006:45:540
93079932,0cyclictest8412-21cut09:30:430
93079932,0cyclictest27842-21tail09:40:450
93079932,0cyclictest18808-21munin-node09:00:350
93079932,0cyclictest16102-21cat07:05:400
93079930,0cyclictest299350irq/33-eth0-tx-07:00:360
93079930,0cyclictest0-21swapper/008:05:350
93079930,0cyclictest0-21swapper/007:45:210
8315030,0irq/26-0000:00:0-21swapper/207:30:442
27130,0ktimersoftd/20-21swapper/207:20:002
259930,0migration/28477-21irqrtprio07:25:392
249930,0watchdog/20-21swapper/206:40:342
249930,0watchdog/20-21swapper/206:30:382
169930,0watchdog/122357-21kthreadcore06:50:381
169930,0watchdog/11914-21smartctl06:55:461
169930,0watchdog/10-21swapper/110:20:371
169930,0watchdog/10-21swapper/107:15:341
93089921,0cyclictest18101-21sed07:55:211
169920,0watchdog/10-21swapper/107:20:221
169920,0watchdog/10-21swapper/107:00:221
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional