You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-08 - 01:04
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2.osadl.org (updated Fri Nov 07, 2025 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
191400,0ktimersoftd/117401-21perf06:20:361
2447123825,0sleep00-21swapper/006:20:030
2416623826,0sleep30-21swapper/306:16:263
2437223425,0sleep20-21swapper/206:18:532
246069988,0cyclictest0-21swapper/008:23:010
329970,0watchdog/326535-21cut11:06:063
329970,0watchdog/30-21swapper/308:51:513
329970,0watchdog/30-21swapper/306:56:153
8315060,0irq/26-0000:00:4669-21munin-run10:55:452
339960,0migration/327929-21sh10:48:183
339960,0migration/321402-21sh11:21:393
329960,0watchdog/322046-21cut09:46:073
329960,0watchdog/30-21swapper/311:47:423
329960,0watchdog/30-21swapper/311:40:463
329960,0watchdog/30-21swapper/311:26:063
329960,0watchdog/30-21swapper/311:11:583
329960,0watchdog/30-21swapper/310:16:423
329960,0watchdog/30-21swapper/309:43:433
329960,0watchdog/30-21swapper/309:31:033
329960,0watchdog/30-21swapper/309:22:273
329960,0watchdog/30-21swapper/309:11:233
329960,0watchdog/30-21swapper/309:05:113
329960,0watchdog/30-21swapper/308:59:473
329960,0watchdog/30-21swapper/308:46:193
329960,0watchdog/30-21swapper/308:45:393
329960,0watchdog/30-21swapper/308:32:273
329960,0watchdog/30-21swapper/307:50:473
329960,0watchdog/30-21swapper/307:44:113
329960,0watchdog/30-21swapper/307:21:073
329960,0watchdog/30-21swapper/307:08:073
329960,0watchdog/30-21swapper/306:48:153
29935060,0irq/33-eth0-tx-0-21swapper/006:41:010
29925060,0irq/32-eth0-rx-7057-21perl11:16:013
29925060,0irq/32-eth0-rx-5854-21perl10:56:033
29925060,0irq/32-eth0-rx-10278-21munin-node10:36:143
29925060,0irq/32-eth0-rx-0-21swapper/311:35:503
29925060,0irq/32-eth0-rx-0-21swapper/311:32:593
29925060,0irq/32-eth0-rx-0-21swapper/310:08:163
29925060,0irq/32-eth0-rx-0-21swapper/309:56:023
29925060,0irq/32-eth0-rx-0-21swapper/309:36:413
29925060,0irq/32-eth0-rx-0-21swapper/309:30:203
29925060,0irq/32-eth0-rx-0-21swapper/309:16:063
29925060,0irq/32-eth0-rx-0-21swapper/308:26:053
246079965,0cyclictest0-21swapper/110:44:171
246069966,0cyclictest23871-21snmpd08:13:180
149960,0migration/03371-21sh08:36:540
149960,0migration/022822-21munin-node11:41:100
8315050,0irq/26-0000:00:26267-21crond10:10:452
339950,0migration/36334-21taskset06:27:023
339950,0migration/323166-21munin-node08:11:043
339950,0migration/320608-21munin-node08:06:093
339950,0migration/320470-21kthreadcore07:46:053
339950,0migration/320226-21kthreadcore07:01:043
339950,0migration/314808-21aten_r1power_po09:05:583
339950,0migration/312633-21kthreadcore10:21:083
339950,0migration/312290-21kthreadcore08:01:053
339950,0migration/310511-21unixbench_multi08:21:123
329950,0watchdog/315654-21irqstats06:36:073
329950,0watchdog/30-21swapper/311:00:503
329950,0watchdog/30-21swapper/310:51:183
329950,0watchdog/30-21swapper/310:40:503
329950,0watchdog/30-21swapper/310:30:583
329950,0watchdog/30-21swapper/310:11:463
329950,0watchdog/30-21swapper/308:35:473
329950,0watchdog/30-21swapper/308:16:273
329950,0watchdog/30-21swapper/307:56:393
329950,0watchdog/30-21swapper/307:31:073
329950,0watchdog/30-21swapper/307:26:073
329950,0watchdog/30-21swapper/307:16:073
329950,0watchdog/30-21swapper/306:51:433
329950,0watchdog/30-21swapper/306:42:033
329950,0watchdog/30-21swapper/306:31:473
329950,0watchdog/30-21swapper/306:22:553
29935050,0irq/33-eth0-tx-3554-21ntpd08:46:082
29935050,0irq/33-eth0-tx-31456-21switchtime11:46:132
29935050,0irq/33-eth0-tx-30239-21munin-node11:10:592
29935050,0irq/33-eth0-tx-28767-21munin-node07:31:000
29935050,0irq/33-eth0-tx-14598-21munin-node08:06:040
29935050,0irq/33-eth0-tx-11000-21munin-node10:21:072
29935050,0irq/33-eth0-tx-10586-21munin-node06:56:030
29935050,0irq/33-eth0-tx-0-21swapper/210:03:282
29935050,0irq/33-eth0-tx-0-21swapper/209:36:052
29935050,0irq/33-eth0-tx-0-21swapper/208:32:462
29935050,0irq/33-eth0-tx-0-21swapper/006:46:000
29925050,0irq/32-eth0-rx-25956-21fschecks_time09:51:013
29925050,0irq/32-eth0-rx-0-21swapper/310:26:053
29925050,0irq/32-eth0-rx-0-21swapper/310:00:553
29925050,0irq/32-eth0-rx-0-21swapper/307:36:033
29925050,0irq/32-eth0-rx-0-21swapper/307:11:013
259950,0migration/26050-21kthreadcore09:56:052
259950,0migration/23854-21cpuspeed_turbos07:36:012
246089955,0cyclictest0-21swapper/211:31:442
246089955,0cyclictest0-21swapper/210:41:262
246089955,0cyclictest0-21swapper/210:13:412
246089955,0cyclictest0-21swapper/209:35:232
246089955,0cyclictest0-21swapper/209:27:112
246089955,0cyclictest0-21swapper/209:22:082
246089955,0cyclictest0-21swapper/208:58:582
246089954,0cyclictest3599-21lldpd11:06:222
246089954,0cyclictest14129-21sadc07:20:462
246079953,0cyclictest22998-21awk08:51:031
246069955,0cyclictest0-21swapper/011:02:570
246069955,0cyclictest0-21swapper/010:29:410
246069950,0cyclictest0-21swapper/011:16:400
249950,0watchdog/20-21swapper/209:05:432
179950,0migration/16844-21fschecks_time11:16:011
179950,0migration/113904-21timerwakeupswit09:21:101
179950,0migration/111715-21kthreadcore10:01:051
179950,0migration/111245-21kthreadcore11:36:051
149950,0migration/09536-21sh10:16:450
149950,0migration/08633-21irqcore08:01:030
149950,0migration/07927-21kthreadcore06:51:050
149950,0migration/07730-21latency_hist11:35:450
149950,0migration/07412-21sh09:38:120
149950,0migration/05374-21kthreadcore10:36:060
149950,0migration/05053-21kthreadcore11:31:050
149950,0migration/03480-21kthreadcore09:56:040
149950,0migration/03370-21sh10:35:350
149950,0migration/032017-21kthreadcore11:11:040
149950,0migration/031944-21kthreadcore10:11:080
149950,0migration/031514-21sh09:34:020
149950,0migration/029774-21taskset08:54:360
149950,0migration/029611-21fschecks_count10:51:000
149950,0migration/029121-21kthreadcore07:51:050
149950,0migration/028155-21kthreadcore06:21:060
149950,0migration/027633-21taskset10:47:230
149950,0migration/026246-21irqcore07:06:050
149950,0migration/022861-21kthreadcore07:46:060
149950,0migration/022145-21munin-node11:06:010
149950,0migration/020038-21kthreadcore10:06:060
149950,0migration/018809-21kthreadcore11:21:050
149950,0migration/017333-21kthreadcore09:06:050
149950,0migration/017009-21irqcore09:26:040
149950,0migration/015414-21kthreadcore08:46:050
149950,0migration/013164-21kthreadcore07:41:040
149950,0migration/012407-21munin-node09:01:050
149950,0migration/012272-21aten_r1power_vo08:25:590
149950,0migration/011454-21munin-node10:21:040
149950,0migration/010702-21kthreadcore09:41:050
149950,0migration/010197-21unixbench_singl07:36:140
8315040,0irq/26-0000:00:28624-21latency_hist08:35:462
8315040,0irq/26-0000:00:0-21swapper/208:25:472
8315040,0irq/26-0000:00:0-21swapper/206:40:472
29935040,0irq/33-eth0-tx-29306-21irqstats10:31:042
29935040,0irq/33-eth0-tx-27917-21grep11:26:042
29935040,0irq/33-eth0-tx-0-21swapper/211:36:082
29935040,0irq/33-eth0-tx-0-21swapper/210:26:242
29935040,0irq/33-eth0-tx-0-21swapper/209:45:542
29935040,0irq/33-eth0-tx-0-21swapper/209:10:542
29935040,0irq/33-eth0-tx-0-21swapper/208:40:502
29935040,0irq/33-eth0-tx-0-21swapper/007:11:000
29935040,0irq/33-eth0-tx-0-21swapper/006:39:190
259940,0migration/25244-21pmu-power09:16:082
259940,0migration/231415-21runrttasks06:21:342
259940,0migration/23095-21runrttasks07:33:072
259940,0migration/218327-21kthreadcore09:06:052
259940,0migration/217471-21kthreadcore06:36:082
259940,0migration/216585-21kthreadcore07:21:042
259940,0migration/212586-21kthreadcore09:41:062
259940,0migration/211788-21munin-node06:56:062
246089943,0cyclictest8504-21awk08:01:022
246089943,0cyclictest3599-21lldpd10:16:002
246089943,0cyclictest3599-21lldpd08:50:542
246089943,0cyclictest3599-21lldpd08:08:222
246089943,0cyclictest3599-21lldpd07:55:522
246089943,0cyclictest3599-21lldpd07:50:522
246089943,0cyclictest3599-21lldpd07:45:512
246089943,0cyclictest3599-21lldpd07:26:212
246089943,0cyclictest3599-21lldpd07:10:202
246089943,0cyclictest3599-21lldpd07:00:502
246089943,0cyclictest3599-21lldpd06:50:502
246089943,0cyclictest3599-21lldpd06:33:192
246089943,0cyclictest3599-21lldpd06:27:482
246089943,0cyclictest3262-21irqbalance11:20:502
246089943,0cyclictest3262-21irqbalance11:15:502
246089943,0cyclictest3262-21irqbalance11:02:502
246089943,0cyclictest3262-21irqbalance10:46:002
246089943,0cyclictest3262-21irqbalance10:35:502
246089943,0cyclictest3262-21irqbalance08:21:512
246089940,0cyclictest25278-21munin-node09:51:002
246079944,0cyclictest0-21swapper/109:13:571
246079943,0cyclictest586-21awk06:26:021
246079943,0cyclictest3599-21lldpd11:24:331
246079943,0cyclictest3599-21lldpd08:30:491
246079943,0cyclictest3599-21lldpd07:06:201
246079943,0cyclictest29549-21awk06:46:021
246079943,0cyclictest23871-21snmpd09:43:411
246079943,0cyclictest23871-21snmpd07:40:551
246079943,0cyclictest23871-21snmpd07:35:551
246079943,0cyclictest23871-21snmpd07:25:551
246079943,0cyclictest23871-21snmpd06:56:401
246079943,0cyclictest23871-21snmpd06:38:251
246079943,0cyclictest1971-21sed10:31:121
246079943,0cyclictest19253-21awk10:06:051
246079943,0cyclictest15537-21awk07:21:031
246079943,0cyclictest13204-21awk08:26:031
246069943,0cyclictest8223-21awk07:16:050
246069943,0cyclictest32017-21munin-node06:26:000
246069943,0cyclictest23871-21snmpd07:25:250
246069943,0cyclictest23871-21snmpd06:32:400
246069943,0cyclictest18967-21awk07:01:030
246069943,0cyclictest1234-21awk07:56:010
246069943,0cyclictest11009-21awk10:01:030
249940,0watchdog/27721-21kthreadcore10:56:062
249940,0watchdog/229699-21iostat_ios06:46:032
249940,0watchdog/229028-21stty08:15:592
249940,0watchdog/221147-21kthreadcore11:41:062
249940,0watchdog/211083-21cpu07:40:592
19140,0ktimersoftd/10-21swapper/110:12:551
179940,0migration/19027-21kthreadcore06:51:061
179940,0migration/1842-21kthreadcore08:16:061
179940,0migration/17396-21kthreadcore10:16:051
179940,0migration/17103-21runrttasks09:20:441
179940,0migration/15460-21kthreadcore09:56:051
179940,0migration/1542-21fschecks_count07:55:591
179940,0migration/132207-21aten_r1power_cu07:11:001
179940,0migration/129606-21fschecks_time08:36:021
179940,0migration/128180-21kthreadcore11:26:051
179940,0migration/126769-21taskset10:28:341
179940,0migration/126346-21sh09:51:031
179940,0migration/124946-21latency_hist11:45:451
179940,0migration/122617-21runrttasks09:26:491
179940,0migration/121295-21taskset08:08:041
179940,0migration/119256-21kthreadcore11:41:061
179940,0migration/118700-21kthreadcore08:46:061
179940,0migration/118017-21kthreadcore09:06:051
179940,0migration/117806-21aten_r1power_cu07:00:581
179940,0migration/112616-21kthreadcore10:21:081
179940,0migration/112044-21taskset08:42:111
169940,0watchdog/1926-21ssh09:36:031
169940,0watchdog/18716-21ps07:16:071
169940,0watchdog/17723-21kthreadcore10:56:061
169940,0watchdog/14691-21ssh11:11:301
169940,0watchdog/14365-21cut10:36:021
169940,0watchdog/129833-21if_err_eth1.207:31:031
169940,0watchdog/125349-21kthreadcore06:41:071
169940,0watchdog/122854-21if_err_eth1.308:11:031
169940,0watchdog/122047-21kthreadcore09:46:071
169940,0watchdog/121470-21grep10:46:021
169940,0watchdog/10-21swapper/111:06:061
169940,0watchdog/10-21swapper/108:56:031
149940,0migration/07911-21kthreadcore10:56:060
149940,0migration/029391-21df_abs08:16:000
149940,0migration/026865-21kthreadcore07:26:050
149940,0migration/024941-21kthreadcore08:31:050
149940,0migration/023156-21aten_r1power_vo09:10:590
149940,0migration/015139-21runrttasks09:24:440
149940,0migration/012189-21sh08:42:360
259930,0migration/2964-21if_eth1.307:11:052
246079932,0cyclictest7527-21awk09:01:011
246079932,0cyclictest3599-21lldpd09:30:531
246079932,0cyclictest10228-21sensors08:21:091
246079930,0cyclictest30214-21iostat_ios10:51:031
246079930,0cyclictest0-21swapper/106:32:401
246069933,0cyclictest29839-21kworker/0:008:55:490
246069933,0cyclictest15671-21kworker/0:111:45:520
246069933,0cyclictest15671-21kworker/0:111:25:470
246069933,0cyclictest15671-21kworker/0:110:40:550
246069933,0cyclictest15671-21kworker/0:109:50:470
246069933,0cyclictest15671-21kworker/0:109:45:510
246069933,0cyclictest15671-21kworker/0:109:15:530
249930,0watchdog/20-21swapper/208:10:592
169930,0watchdog/10-21swapper/111:34:021
169930,0watchdog/10-21swapper/111:01:221
169930,0watchdog/10-21swapper/108:01:071
169930,0watchdog/10-21swapper/107:51:071
169930,0watchdog/10-21swapper/107:46:071
169930,0watchdog/10-21swapper/106:21:151
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional