You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-15 - 01:03
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2.osadl.org (updated Mon Sep 14, 2026 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1885324839,0sleep10-21swapper/107:29:531
1892024738,0sleep20-21swapper/207:30:392
1904723727,0sleep30-21swapper/307:32:123
1874122925,0sleep00-21swapper/007:28:320
191789983,0cyclictest3828-21awk10:12:590
191789983,0cyclictest3524-21snmpd08:07:580
191789983,0cyclictest25600-21awk12:38:010
191789983,0cyclictest1502-21sadc09:12:440
191789980,0cyclictest0-21swapper/008:53:090
191789973,0cyclictest3571-21lldpd08:51:410
191789973,0cyclictest30904-21sadc12:22:440
191789972,0cyclictest6525-21cut11:28:020
191789972,0cyclictest5767-21awk10:32:580
191789972,0cyclictest3524-21snmpd08:05:150
191789972,0cyclictest20028-21awk11:38:010
191789972,0cyclictest19005-21awk12:53:000
191789972,0cyclictest16324-21awk10:58:020
191789971,0cyclictest19522-21sadc09:02:440
191789970,0cyclictest9283-21ssh12:47:500
191789970,0cyclictest8139-21perl12:27:570
191789970,0cyclictest8035-21python09:13:060
191789970,0cyclictest7510-21grep12:08:010
191789970,0cyclictest569-21sh12:42:500
191789970,0cyclictest5689-21kthreadcore08:28:020
191789970,0cyclictest5061-21tr09:53:030
191789970,0cyclictest3616-21cut11:47:440
191789970,0cyclictest32464-21sh09:52:080
191789970,0cyclictest27953-21irqcore08:23:000
191789970,0cyclictest26581-21munin-node07:37:580
191789970,0cyclictest26021-21entropy11:22:580
191789970,0cyclictest24449-21kthreadcore11:58:040
191789970,0cyclictest20940-21kthreadcore11:18:020
191789970,0cyclictest19755-21munin-node09:03:040
191789970,0cyclictest19570-21munin-node08:17:570
191789970,0cyclictest17954-21grep07:53:000
191789970,0cyclictest15511-21taskset08:34:100
191789970,0cyclictest14211-21sh09:39:500
191789970,0cyclictest13719-21kthreadcore09:18:010
191789970,0cyclictest1367-21munin-node07:43:050
191789970,0cyclictest13056-21tune2fs11:52:580
191789970,0cyclictest12426-21kthreadcore11:13:010
191789970,0cyclictest10263-21cat11:32:440
191789970,0cyclictest0-21swapper/010:24:320
191789970,0cyclictest0-21swapper/007:48:100
8305060,0irq/26-0000:00:8290-21crond11:12:452
339960,0migration/323331-21sh11:02:123
339960,0migration/318065-21sh10:18:153
339960,0migration/316786-21sh12:50:353
339960,0migration/313258-21sh10:37:503
29805060,0irq/28-eth0-rx-3715-21yum12:03:123
29805060,0irq/28-eth0-rx-29332-21perl10:27:593
29805060,0irq/28-eth0-rx-25041-21ntp_kernel_pll_11:38:063
191909960,0cyclictest0-21swapper/210:28:012
191789961,0cyclictest27814-21tr11:42:570
191789961,0cyclictest19712-21sed07:32:560
191789961,0cyclictest12650-21cpuspeed_turbos08:12:560
191789960,0cyclictest9634-21tr09:57:570
191789960,0cyclictest6989-21tr10:52:560
191789960,0cyclictest32284-21taskset11:07:450
191789960,0cyclictest31148-21ssh12:22:490
191789960,0cyclictest30783-21cut09:32:560
191789960,0cyclictest30371-21sh10:47:460
191789960,0cyclictest30054-21perl12:02:590
191789960,0cyclictest28751-21cpuspeed_turbos10:27:580
191789960,0cyclictest26480-21ssh10:07:520
191789960,0cyclictest26472-21aten_r1power_po12:57:570
191789960,0cyclictest23747-21latency_hist11:02:450
191789960,0cyclictest23713-21grep08:42:570
191789960,0cyclictest23500-21df_abs07:57:580
191789960,0cyclictest23485-21aten2.4-expect09:27:560
191789960,0cyclictest21906-21stty10:42:550
191789960,0cyclictest17885-21sh10:02:500
191789960,0cyclictest16488-21tr09:22:570
191789960,0cyclictest15984-21latency_hist08:37:440
191789960,0cyclictest15858-21grep12:32:440
191789960,0cyclictest15562-21sh09:42:470
191789960,0cyclictest14097-21sh12:12:480
191789960,0cyclictest13714-21diskstats10:37:560
191789960,0cyclictest11454-21aten2.4-expect10:17:570
8305050,0irq/26-0000:00:26358-21perf07:42:383
339950,0migration/38430-21munin-run07:47:433
339950,0migration/37705-21missed_timers09:13:023
339950,0migration/34287-21kthreadcore12:43:023
339950,0migration/33777-21latency08:03:033
339950,0migration/329639-21kthreadcore09:08:013
339950,0migration/328332-21kthreadcore12:58:023
339950,0migration/326838-21aten2.4-expect08:22:563
339950,0migration/324814-21kthreadcore12:18:023
339950,0migration/32376-21kthreadcore09:53:023
339950,0migration/322485-21kthreadcore08:18:023
339950,0migration/320905-21if_eth1.209:02:593
339950,0migration/319549-21kthreadcore12:53:023
339950,0migration/318175-21kthreadcore12:33:003
339950,0migration/318132-21kthreadcore08:38:023
339950,0migration/316103-21taskset09:58:163
339950,0migration/315593-21taskset08:35:083
339950,0migration/312342-21kthreadcore11:33:013
329950,0watchdog/38061-21if_eth1.309:37:593
329950,0watchdog/36102-21ntp_kernel_pll_11:08:043
329950,0watchdog/330085-21chrt09:29:033
329950,0watchdog/322869-21kthreadcore11:58:043
329950,0watchdog/320975-21cut09:23:033
329950,0watchdog/314324-21rm12:28:483
329950,0watchdog/30-21swapper/312:08:043
329950,0watchdog/30-21swapper/311:27:563
29815050,0irq/29-eth0-tx-9024-21grep12:28:012
29815050,0irq/29-eth0-tx-3524-21snmpd11:25:142
29815050,0irq/29-eth0-tx-27513-21munin-node11:43:072
29815050,0irq/29-eth0-tx-25261-21cut11:03:012
29815050,0irq/29-eth0-tx-19889-21munin-node10:22:552
29815050,0irq/29-eth0-tx-18656-21perl10:02:572
29815050,0irq/29-eth0-tx-16632-21sed10:58:032
29815050,0irq/29-eth0-tx-13160-21ps09:38:052
29815050,0irq/29-eth0-tx-12232-21fschecks_time10:17:592
29815050,0irq/29-eth0-tx-0-21swapper/209:56:412
29815050,0irq/29-eth0-tx-0-21swapper/209:47:562
29815050,0irq/29-eth0-tx-0-21swapper/209:37:152
29805050,0irq/28-eth0-rx-4149-21diskstats11:47:573
29805050,0irq/28-eth0-rx-30923-21kthreadcore11:23:043
29805050,0irq/28-eth0-rx-29179-21kthreadcore11:43:023
29805050,0irq/28-eth0-rx-28071-21kthreadcore12:38:033
29805050,0irq/28-eth0-rx-27557-21if_err_eth1.210:07:593
29805050,0irq/28-eth0-rx-26208-21ssh10:23:053
29805050,0irq/28-eth0-rx-23823-21tr10:43:013
29805050,0irq/28-eth0-rx-2326-21kthreadcore12:23:023
29805050,0irq/28-eth0-rx-19514-21kthreadcore10:03:013
29805050,0irq/28-eth0-rx-17673-21kthreadcore08:58:033
29805050,0irq/28-eth0-rx-12562-21cpuspeed_turbos11:52:573
29805050,0irq/28-eth0-rx-11820-21kthreadcore11:13:013
29805050,0irq/28-eth0-rx-0-21swapper/311:04:513
29805050,0irq/28-eth0-rx-0-21swapper/310:13:093
259950,0migration/230954-21latency_hist12:22:442
259950,0migration/23092-21kthreadcore12:43:022
259950,0migration/229216-21sh10:45:022
259950,0migration/220327-21kthreadcore12:53:022
259950,0migration/219389-21aten_r1power_cu07:32:552
259950,0migration/216802-21sh12:50:372
259950,0migration/211802-21sh11:52:042
259950,0migration/211402-21munin-node10:33:082
249950,0watchdog/216528-21sh11:33:042
249950,0watchdog/20-21swapper/211:58:042
191909955,0cyclictest0-21swapper/212:33:022
191909955,0cyclictest0-21swapper/210:49:462
191909950,0cyclictest0-21swapper/212:15:032
191909950,0cyclictest0-21swapper/211:21:562
191909950,0cyclictest0-21swapper/209:43:422
191859955,0cyclictest0-21swapper/112:35:121
191859955,0cyclictest0-21swapper/109:37:501
191859950,0cyclictest0-21swapper/112:09:021
191859950,0cyclictest0-21swapper/110:24:021
191859950,0cyclictest0-21swapper/109:54:331
191859950,0cyclictest0-21swapper/109:48:431
19150,0ktimersoftd/10-21swapper/112:39:311
19150,0ktimersoftd/10-21swapper/112:16:271
19150,0ktimersoftd/10-21swapper/111:43:581
179950,0migration/19488-21munin-node12:47:571
179950,0migration/16488-21timerandwakeup11:08:081
179950,0migration/132476-21kthreadcore09:33:021
179950,0migration/126888-21munin-node11:58:061
179950,0migration/126070-21kthreadcore10:43:011
179950,0migration/125663-21taskset11:38:141
179950,0migration/117615-21taskset11:34:371
179950,0migration/112626-21cpuspeed_turbos11:52:571
179950,0migration/110272-21kthreadcore11:13:001
169950,0watchdog/111911-21yum08:53:111
8305040,0irq/26-0000:00:0-21swapper/211:12:452
339940,0migration/322285-21kthreadcore07:33:013
339940,0migration/311600-21kthreadcore09:18:003
339940,0migration/310775-21kthreadcore08:08:023
329940,0watchdog/30-21swapper/308:53:113
329940,0watchdog/30-21swapper/308:12:593
29815040,0irq/29-eth0-tx-0-21swapper/212:57:532
29815040,0irq/29-eth0-tx-0-21swapper/212:38:462
29815040,0irq/29-eth0-tx-0-21swapper/212:22:582
29815040,0irq/29-eth0-tx-0-21swapper/212:09:192
29815040,0irq/29-eth0-tx-0-21swapper/212:04:132
29815040,0irq/29-eth0-tx-0-21swapper/211:37:502
29815040,0irq/29-eth0-tx-0-21swapper/211:27:502
29815040,0irq/29-eth0-tx-0-21swapper/210:12:492
29815040,0irq/29-eth0-tx-0-21swapper/210:07:502
29805040,0irq/28-eth0-rx-4216-21ntp_offset10:48:043
29805040,0irq/28-eth0-rx-30497-21ssh09:48:053
29805040,0irq/28-eth0-rx-14767-21tune2fs12:12:573
29805040,0irq/28-eth0-rx-0-21swapper/311:17:563
29805040,0irq/28-eth0-rx-0-21swapper/310:53:043
29805040,0irq/28-eth0-rx-0-21swapper/310:32:493
29805040,0irq/28-eth0-rx-0-21swapper/309:44:043
29805040,0irq/28-eth0-rx-0-21swapper/309:33:343
29805040,0irq/28-eth0-rx-0-21swapper/308:42:583
259940,0migration/23278-21kthreadcore08:48:032
259940,0migration/226362-21runrttasks08:21:112
259940,0migration/221183-21kthreadcore07:53:012
259940,0migration/218298-21kthreadcore10:38:012
259940,0migration/214767-21kthreadcore11:53:022
259940,0migration/214228-21kthreadcore08:58:022
259940,0migration/212375-21latency_hist08:57:442
259940,0migration/211064-21kthreadcore09:18:002
249940,0watchdog/20-21swapper/209:28:032
249940,0watchdog/20-21swapper/207:58:032
191909943,0cyclictest9934-21awk08:32:592
191909943,0cyclictest7759-21awk10:52:592
191909943,0cyclictest3524-21snmpd09:24:592
191909943,0cyclictest3524-21snmpd09:13:142
191909943,0cyclictest3524-21snmpd09:11:292
191909943,0cyclictest3524-21snmpd08:23:132
191909943,0cyclictest3524-21snmpd08:12:582
191909943,0cyclictest17171-21awk08:38:012
191909943,0cyclictest15647-21sadc07:52:432
191909943,0cyclictest10432-21awk09:58:002
191859943,0cyclictest6293-21awk08:07:591
191859943,0cyclictest31510-21awk10:48:001
191859943,0cyclictest27824-21awk08:22:591
191859943,0cyclictest26704-21awk11:23:011
191859943,0cyclictest19518-21kthreadcore10:03:011
191859943,0cyclictest17358-21awk09:23:011
191859943,0cyclictest17033-21awk07:52:581
191859943,0cyclictest14517-21ps10:38:001
179940,0migration/18282-21kernelversion10:53:001
179940,0migration/15477-21kthreadcore11:28:021
179940,0migration/1499-21munin-node09:08:041
179940,0migration/1395-21switchtime07:38:091
179940,0migration/1336-21kthreadcore12:03:031
179940,0migration/130586-21cpuspeed_turbos08:02:571
179940,0migration/12546-21sh08:27:581
179940,0migration/124901-21kthreadcore12:18:021
179940,0migration/119712-21latency_hist10:22:451
179940,0migration/118653-21kthreadcore10:58:051
179940,0migration/116345-21cpuspeed_turbos08:37:571
179940,0migration/11551-21fschecks_time12:42:581
179940,0migration/113845-21kthreadcore07:48:031
179940,0migration/113611-21irqcore08:13:001
179940,0migration/112022-21kthreadcore09:18:001
169940,0watchdog/15919-21iostat10:32:591
169940,0watchdog/131164-21ssh11:05:081
169940,0watchdog/125174-21cut07:58:031
169940,0watchdog/124743-21ssh11:21:081
169940,0watchdog/123337-21ssh09:46:071
169940,0watchdog/123269-21munin-node08:42:551
169940,0watchdog/119414-21awk07:32:551
169940,0watchdog/115069-21kthreadcore09:58:031
169940,0watchdog/111646-21ssh11:51:361
169940,0watchdog/10-21swapper/110:09:311
329930,0watchdog/321904-21ntpq07:53:033
329930,0watchdog/30-21swapper/308:27:513
29805030,0irq/28-eth0-rx-0-21swapper/308:48:373
29805030,0irq/28-eth0-rx-0-21swapper/307:57:573
29805030,0irq/28-eth0-rx-0-21swapper/307:47:563
249930,0watchdog/20-21swapper/207:42:552
191909933,0cyclictest0-21swapper/208:45:222
191909932,0cyclictest5218-21stty08:07:552
191909932,0cyclictest3524-21snmpd08:29:282
191909932,0cyclictest3524-21snmpd08:02:582
191909932,0cyclictest263132sleep209:03:092
191859933,0cyclictest0-21swapper/112:23:041
191859932,0cyclictest31830-21grep12:58:031
191859932,0cyclictest31620-21awk08:48:001
191859932,0cyclictest24582-21awk09:28:011
191859932,0cyclictest20672-21awk08:18:001
191859932,0cyclictest20646-21grep09:02:561
191859932,0cyclictest18873-21awk12:52:591
191859932,0cyclictest13627-21awk08:58:001
191859931,0cyclictest7349-21sed07:43:031
169930,0watchdog/19609-21sh10:13:271
169930,0watchdog/10-21swapper/112:27:521
169930,0watchdog/10-21swapper/110:28:031
169930,0watchdog/10-21swapper/109:15:271
249920,0watchdog/20-21swapper/207:37:472
169920,0watchdog/10-21swapper/108:32:471
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional