You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-09-18 - 16:40
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2.osadl.org (updated Thu Sep 18, 2025 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2534223929,0sleep30-21swapper/306:19:513
2474723627,0sleep20-21swapper/206:19:132
1877223425,0sleep10-21swapper/106:19:001
1875823127,0sleep00-21swapper/006:18:510
329970,0watchdog/39265-21port_sshpriv09:19:123
329970,0watchdog/30-21swapper/309:00:053
329970,0watchdog/30-21swapper/308:24:013
8359260,0sleep38525-21sort07:19:033
8315060,0irq/26-0000:00:30741-21latency_hist08:38:482
8315060,0irq/26-0000:00:19117-21latency_hist07:48:482
339960,0migration/326205-21sh07:53:483
339960,0migration/322685-21munin-node10:24:063
329960,0watchdog/3299250irq/32-eth0-rx-09:55:123
329960,0watchdog/30-21swapper/311:48:523
329960,0watchdog/30-21swapper/311:47:443
329960,0watchdog/30-21swapper/311:08:523
329960,0watchdog/30-21swapper/311:04:043
329960,0watchdog/30-21swapper/311:01:203
329960,0watchdog/30-21swapper/310:48:483
329960,0watchdog/30-21swapper/310:44:123
329960,0watchdog/30-21swapper/310:19:243
329960,0watchdog/30-21swapper/310:16:203
329960,0watchdog/30-21swapper/309:41:363
329960,0watchdog/30-21swapper/309:29:043
329960,0watchdog/30-21swapper/309:14:123
329960,0watchdog/30-21swapper/308:54:093
329960,0watchdog/30-21swapper/308:36:173
329960,0watchdog/30-21swapper/308:09:053
329960,0watchdog/30-21swapper/307:44:133
329960,0watchdog/30-21swapper/306:59:013
329960,0watchdog/30-21swapper/306:39:173
329960,0watchdog/30-21swapper/306:38:253
29935060,0irq/33-eth0-tx-752-21grep11:09:022
29935060,0irq/33-eth0-tx-22712-21munin-node07:29:040
29935060,0irq/33-eth0-tx-12000-21munin-node07:44:140
29925060,0irq/32-eth0-rx-9718-21diskstats11:14:013
29925060,0irq/32-eth0-rx-4963-21perl10:34:033
29925060,0irq/32-eth0-rx-20902-21ssh09:44:393
29925060,0irq/32-eth0-rx-1754-21python10:29:123
29925060,0irq/32-eth0-rx-15575-21kthreadcore09:24:043
29925060,0irq/32-eth0-rx-0-21swapper/311:39:143
29925060,0irq/32-eth0-rx-0-21swapper/311:24:063
29925060,0irq/32-eth0-rx-0-21swapper/311:19:023
29925060,0irq/32-eth0-rx-0-21swapper/310:42:193
29925060,0irq/32-eth0-rx-0-21swapper/310:04:013
29925060,0irq/32-eth0-rx-0-21swapper/310:00:393
29925060,0irq/32-eth0-rx-0-21swapper/309:49:063
29925060,0irq/32-eth0-rx-0-21swapper/308:45:023
29925060,0irq/32-eth0-rx-0-21swapper/308:31:583
29925060,0irq/32-eth0-rx-0-21swapper/307:29:063
257539960,0cyclictest0-21swapper/211:23:092
257529965,0cyclictest8723-21ssh11:13:041
149960,0migration/07497-21timerandwakeup08:59:110
149960,0migration/05744-21munin-node11:49:050
149960,0migration/028431-21sh10:44:150
339950,0migration/35790-21idleruntime07:39:013
339950,0migration/34434-21kthreadcore07:59:073
339950,0migration/332632-21kthreadcore07:34:073
339950,0migration/328136-21kthreadcore07:54:063
339950,0migration/324769-21munin-node07:04:083
339950,0migration/319088-21kthreadcore07:24:073
329950,0watchdog/329231-21cpuspeed_turbos09:34:043
329950,0watchdog/326508-21cut10:09:083
329950,0watchdog/312186-21df_abs11:34:043
329950,0watchdog/30-21swapper/311:28:483
329950,0watchdog/30-21swapper/310:53:563
329950,0watchdog/30-21swapper/309:08:483
329950,0watchdog/30-21swapper/308:48:533
329950,0watchdog/30-21swapper/308:18:573
329950,0watchdog/30-21swapper/308:14:093
329950,0watchdog/30-21swapper/308:04:053
329950,0watchdog/30-21swapper/307:10:333
329950,0watchdog/30-21swapper/306:53:573
329950,0watchdog/30-21swapper/306:49:093
329950,0watchdog/30-21swapper/306:45:053
329950,0watchdog/30-21swapper/306:28:493
329950,0watchdog/30-21swapper/306:24:013
29935050,0irq/33-eth0-tx-9924-21ssh11:29:392
29935050,0irq/33-eth0-tx-5452-21fschecks_count08:24:020
29935050,0irq/33-eth0-tx-30434-21munin-node10:49:042
29935050,0irq/33-eth0-tx-29927-21munin-node07:34:150
29935050,0irq/33-eth0-tx-28359-21grep10:29:022
29935050,0irq/33-eth0-tx-26157-21kthreadcore11:04:062
29935050,0irq/33-eth0-tx-19297-21cpu10:24:012
29935050,0irq/33-eth0-tx-13393-21munin-node08:29:172
29935050,0irq/33-eth0-tx-1192-21munin-node07:59:160
29935050,0irq/33-eth0-tx-11631-21munin-node09:24:052
29935050,0irq/33-eth0-tx-0-21swapper/211:27:122
29935050,0irq/33-eth0-tx-0-21swapper/210:40:122
29935050,0irq/33-eth0-tx-0-21swapper/210:34:062
29935050,0irq/33-eth0-tx-0-21swapper/209:22:122
29935050,0irq/33-eth0-tx-0-21swapper/208:53:522
29935050,0irq/33-eth0-tx-0-21swapper/208:43:042
29935050,0irq/33-eth0-tx-0-21swapper/008:19:040
29935050,0irq/33-eth0-tx-0-21swapper/007:09:110
29925050,0irq/32-eth0-rx-1347-21grep08:39:073
29925050,0irq/32-eth0-rx-0-21swapper/309:04:023
29925050,0irq/32-eth0-rx-0-21swapper/307:14:003
257539955,0cyclictest0-21swapper/211:47:072
257539955,0cyclictest0-21swapper/209:49:212
257539955,0cyclictest0-21swapper/209:31:132
257539955,0cyclictest0-21swapper/209:07:532
257539955,0cyclictest0-21swapper/206:38:542
257529950,0cyclictest0-21swapper/110:59:591
257529950,0cyclictest0-21swapper/109:36:451
259950,0migration/26932-21munin-node11:49:112
259950,0migration/25451-21kthreadcore07:14:072
259950,0migration/232279-21sh10:10:022
259950,0migration/228708-21munin-node08:14:052
259950,0migration/225171-21kthreadcore06:44:042
259950,0migration/22273-21munin-node06:29:072
259950,0migration/22164-21kthreadcore09:54:062
259950,0migration/220367-21kthreadcore09:09:062
259950,0migration/215252-21sh08:47:482
259950,0migration/211747-21switchtime09:39:132
249950,0watchdog/25168-21perl06:54:012
249950,0watchdog/227335-21iostat_ios07:09:092
179950,0migration/17080-21switchtime06:29:151
179950,0migration/1643-21taskset07:55:481
179950,0migration/13726-21fschecks_time09:19:051
179950,0migration/126171-21irqrtprio08:54:071
179950,0migration/126090-21sh09:12:391
179950,0migration/121300-21proc_pri06:39:131
179950,0migration/115091-21kthreadcore08:29:101
179950,0migration/113313-21munin-node10:39:051
169950,0watchdog/13534-21latency08:19:091
149950,0migration/06710-21munin-node06:29:090
149950,0migration/06428-21sh08:42:210
149950,0migration/04449-21sh10:49:100
149950,0migration/04297-21sh09:36:550
149950,0migration/031498-21kthreadcore10:29:060
149950,0migration/031158-21sh09:53:420
149950,0migration/030010-21kthreadcore06:24:060
149950,0migration/028369-21sh09:14:050
149950,0migration/026553-21kthreadcore11:04:060
149950,0migration/022454-21kthreadcore10:24:060
149950,0migration/022039-21kthreadcore11:19:090
149950,0migration/020977-21munin-node11:39:020
149950,0migration/020779-21taskset08:31:020
149950,0migration/020715-21munin-node09:29:010
149950,0migration/017661-21missed_timers09:24:050
149950,0migration/016426-21taskset11:15:240
149950,0migration/014035-21kthreadcore06:59:060
149950,0migration/012434-21kthreadcore09:59:070
149950,0migration/011209-21timerwakeupswit07:39:110
29935040,0irq/33-eth0-tx-14696-21ssh09:59:282
29935040,0irq/33-eth0-tx-0-21swapper/211:39:102
29935040,0irq/33-eth0-tx-0-21swapper/210:59:052
29935040,0irq/33-eth0-tx-0-21swapper/210:14:392
29935040,0irq/33-eth0-tx-0-21swapper/210:04:162
29935040,0irq/33-eth0-tx-0-21swapper/208:59:152
29935040,0irq/33-eth0-tx-0-21swapper/208:49:002
29935040,0irq/33-eth0-tx-0-21swapper/007:04:050
257539944,0cyclictest0-21swapper/208:25:472
257539944,0cyclictest0-21swapper/206:24:142
257539943,0cyclictest9448-21awk07:19:062
257539943,0cyclictest27943-21awk09:14:042
257539943,0cyclictest23871-21snmpd06:51:222
257539943,0cyclictest19964-21awk07:04:042
257539943,0cyclictest15255-21awk09:44:052
257529944,0cyclictest0-21swapper/111:47:081
257529944,0cyclictest0-21swapper/110:23:531
257529944,0cyclictest0-21swapper/108:28:381
257529944,0cyclictest0-21swapper/107:14:041
257529943,0cyclictest9561-21awk08:04:051
257529943,0cyclictest9400-21awk07:19:061
257529943,0cyclictest9068-21awk06:34:041
257529943,0cyclictest3262-21irqbalance10:50:021
257529943,0cyclictest32091-21awk08:39:061
257529943,0cyclictest27917-21awk09:14:041
257529943,0cyclictest27014-21awk06:24:031
257529943,0cyclictest26460-21awk10:09:081
257529943,0cyclictest23941-21awk09:49:051
257529943,0cyclictest23871-21snmpd08:09:371
257529943,0cyclictest13048-21awk07:44:031
257529943,0cyclictest11044-21irqrtprio09:04:051
257519943,0cyclictest6448-21awk09:39:040
257519943,0cyclictest4266-21awk11:29:040
257519943,0cyclictest3599-21lldpd07:57:220
257519940,0cyclictest0-21swapper/009:13:480
259940,0migration/29761-21kthreadcore06:34:052
259940,0migration/28305-21munin-node07:39:032
259940,0migration/231979-21kthreadcore07:34:072
259940,0migration/225920-21kthreadcore10:44:062
259940,0migration/217799-21kthreadcore07:24:072
259940,0migration/214058-21kthreadcore11:14:062
259940,0migration/212411-21fschecks_time06:59:022
259940,0migration/210323-21latency_hist10:18:482
249940,0watchdog/211729-21ssh11:33:482
249940,0watchdog/20-21swapper/210:54:002
249940,0watchdog/20-21swapper/209:34:042
179940,0migration/19844-21kthreadcore10:34:071
179940,0migration/16883-21irqrtprio09:39:051
179940,0migration/16333-21kthreadcore06:54:041
179940,0migration/15746-21aten_r1power_en11:49:051
179940,0migration/132311-21fschecks_time09:54:021
179940,0migration/12965-21runrttasks10:31:251
179940,0migration/128337-21open_files08:34:131
179940,0migration/126418-21munin-node09:29:081
179940,0migration/118534-21kthreadcore08:49:071
179940,0migration/114265-21kthreadcore11:34:091
179940,0migration/112799-21if_eth006:59:041
179940,0migration/112430-21fschecks_time09:24:001
179940,0migration/110913-21kthreadcore10:54:071
169940,0watchdog/13750-21kthreadcore10:14:041
169940,0watchdog/127825-21kthreadcore08:14:051
169940,0watchdog/122908-21iostat10:44:041
169940,0watchdog/116564-21ssh10:03:521
169940,0watchdog/11632-21ssh11:26:001
169940,0watchdog/115997-21ssh10:03:001
169940,0watchdog/10-21swapper/111:28:521
149940,0migration/0743-21runrttasks10:12:290
149940,0migration/05440-21iostat10:34:050
149940,0migration/029059-21kthreadcore11:24:050
149940,0migration/027681-21kthreadcore08:54:080
149940,0migration/026646-21kthreadcore08:14:040
149940,0migration/018637-21kthreadcore07:24:070
149940,0migration/015331-21sh08:48:010
149940,0migration/010685-21diskmemload09:21:550
149940,0migration/01032-21kthreadcore09:54:060
8315030,0irq/26-0000:00:0-21swapper/208:04:122
8315030,0irq/26-0000:00:0-21swapper/207:53:592
29935030,0irq/33-eth0-tx-8076-21aten2.4-expect06:34:000
29935030,0irq/33-eth0-tx-0-21swapper/007:49:020
29935030,0irq/33-eth0-tx-0-21swapper/006:54:010
257539932,0cyclictest23871-21snmpd07:58:522
257539932,0cyclictest23871-21snmpd07:33:072
257539931,0cyclictest2912-21cut08:19:082
257529933,0cyclictest30990-21awk07:34:051
257529932,0cyclictest30795-21tail06:49:051
257529932,0cyclictest27186-21awk07:09:081
257529932,0cyclictest23568-21awk06:44:021
257529932,0cyclictest2280-21awk07:59:051
257529932,0cyclictest2026-21awk08:59:011
257529932,0cyclictest19123-21awk11:19:071
257529932,0cyclictest14867-21uniq09:44:021
257529932,0cyclictest12855-21cut08:44:101
257529930,0cyclictest19962-21perl07:49:031
257529930,0cyclictest0-21swapper/110:19:061
257519933,0cyclictest30544-21kworker/0:106:48:590
257519933,0cyclictest30544-21kworker/0:106:43:530
257519933,0cyclictest30544-21kworker/0:106:39:050
257519933,0cyclictest25140-21kworker/0:007:18:530
257519933,0cyclictest25140-21kworker/0:007:13:590
257519933,0cyclictest22675-21kworker/0:111:43:580
257519933,0cyclictest22675-21kworker/0:111:34:040
257519933,0cyclictest22675-21kworker/0:111:09:020
257519933,0cyclictest18212-21kworker/0:108:33:490
257519933,0cyclictest18212-21kworker/0:108:08:550
257519933,0cyclictest18212-21kworker/0:108:03:530
257519933,0cyclictest11711-21kworker/0:010:38:540
257519933,0cyclictest11711-21kworker/0:009:43:490
257519932,0cyclictest8093-21awk10:54:050
257519932,0cyclictest8053-21sensors10:14:090
257519932,0cyclictest21051-21cut10:59:070
257519932,0cyclictest17128-21awk08:49:060
257519932,0cyclictest16317-21latency_hist10:03:480
257519932,0cyclictest13846-21kthreadcore10:19:060
257519932,0cyclictest10763-21awk09:04:040
249930,0watchdog/20-21swapper/208:09:572
179930,0migration/15621-21fschecks_time07:39:001
179930,0migration/125796-21runrttasks07:07:421
179930,0migration/115481-21runrttasks07:23:511
169930,0watchdog/10-21swapper/111:39:441
169930,0watchdog/10-21swapper/111:13:481
169930,0watchdog/10-21swapper/111:04:041
257539920,0cyclictest12217-21cpu07:44:002
257529921,0cyclictest22480-21latency_hist07:28:481
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional