You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-20 - 11:17
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot6.osadl.org (updated Sat Dec 20, 2025 00:44:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
280182999686,7cyclictest328868-21latency_hist21:40:011
280182999286,4cyclictest370529-21kworker/u8:1+flush-179:000:03:591
280179998680,4cyclictest288292-21latency_hist19:35:020
280179998274,6cyclictest370741-21latency_hist23:50:010
280179998172,7cyclictest330468-21apt-get21:44:590
280182998070,8cyclictest323826-21latency_hist21:25:001
280179997266,4cyclictest346454-21kworker/u8:2+flush-179:022:44:590
280179997263,7cyclictest266-21systemd-journal00:04:000
280182996961,6cyclictest379748-21kworker/u8:1+flush-179:000:20:001
280182996858,8cyclictest294653-21latency_hist19:55:011
280179996462,1cyclictest0-21swapper/023:02:590
280179996354,7cyclictest337074-21latency_hist22:05:010
280183996259,2cyclictest0-21swapper/223:31:592
280182996257,4cyclictest314044-21kworker/u8:2+flush-179:021:00:251
280182996253,7cyclictest338915-21apt-get22:10:121
280182995954,4cyclictest320383-21apt-get21:15:031
280182995924,21cyclictest295473-21latency19:55:191
280179995850,6cyclictest346613-21latency_hist22:35:020
28017999581,53cyclictest296293-21true20:00:010
231999580,7rtkit-daemon2318-21rtkit-daemon00:25:121
280182995545,8cyclictest343695-21apt-get22:25:131
280182995545,7cyclictest377165-21latency_hist00:10:011
280182995530,23cyclictest299554-21unixbench-2d20:05:261
280179995545,8cyclictest281921-21apt-get19:15:130
28017999552,49cyclictest0-21swapper/022:35:140
280182995329,15cyclictest0-21swapper/119:35:001
28018299530,4cyclictest354568-21turbostat23:04:561
280187995250,1cyclictest0-21swapper/300:12:583
280182995244,6cyclictest280050-21latency_hist19:10:031
280182995225,24cyclictest313861-21unixbench-2d20:50:231
280179995251,1cyclictest0-21swapper/023:31:590
280179995242,8cyclictest306057-21latency_hist20:30:010
28018299514,33cyclictest283288-21munin-run19:20:001
280182995141,9cyclictest386734-21munin-run00:40:001
280182995139,10cyclictest314266-21apt-get20:55:131
280179995141,8cyclictest372583-21apt-get23:55:110
28018299504,43cyclictest332550-21apt-get21:50:141
280182995041,7cyclictest367606-21kworker/u8:0+flush-179:000:00:001
280179995044,5cyclictest294892-21kworker/u8:2+flush-179:020:06:110
4577091490,13ptp4l441ktimers/319:10:003
280182994940,8cyclictest330564-21cp21:45:001
280179994940,7cyclictest364376-21latency_hist23:30:010
280179994940,7cyclictest364376-21latency_hist23:30:000
280182994824,23cyclictest283934-21fschecks_time19:20:141
280179994842,4cyclictest335494-21awk22:00:010
280182994743,3cyclictest289935-21systemd-run19:40:001
280182994739,6cyclictest510-21rs:main1
280182994628,16cyclictest366166-21latency_hist23:35:011
28018299461,27cyclictest346869-21apt-get22:35:141
280179994630,14cyclictest714-21Xorg21:55:000
4577091450,37ptp4l61490irq/134-eno1-TxRx-023:42:593
28018399450,36cyclictest0-21swapper/221:54:592
280182994539,5cyclictest309511-21kworker/u8:0+flush-179:020:45:231
280182994538,6cyclictest266-21systemd-journal23:26:081
280182994538,6cyclictest266-21systemd-journal23:26:081
280179994541,3cyclictest349841-21kworker/u8:3+flush-179:022:49:110
28017999452,30cyclictest322498-21apt-get21:20:130
280179994522,20cyclictest0-21swapper/022:27:000
280182994434,8cyclictest358192-21cat23:10:011
280182994427,16cyclictest291608-21latency_hist19:45:011
280179994440,3cyclictest714-21Xorg23:22:000
280179994421,4cyclictest0-21swapper/023:05:200
280182994339,3cyclictest1-21systemd20:30:011
280182994338,4cyclictest281667-21munin-run19:15:001
280179994337,5cyclictest330838-21kworker/u8:0+flush-179:022:25:000
280179994336,6cyclictest370998-21apt-get23:50:110
280187994235,5cyclictest383542-21turbostat00:30:003
280183994239,2cyclictest0-21swapper/219:28:592
280182994235,6cyclictest370529-21kworker/u8:1+flush-179:023:55:011
280182994235,5cyclictest372359-21kworker/u8:3+events_unbound00:10:241
280182994226,14cyclictest325715-21kworker/u8:2+flush-179:022:00:001
280182994223,18cyclictest337439-21cpuspeed_turbos22:05:131
280179994238,3cyclictest320522-21latency_hist21:15:010
280179994236,4cyclictest314044-21kworker/u8:2+events_unbound21:00:010
28017999420,40cyclictest340251-21turbostat22:19:540
28018799410,5cyclictest302894-21turbostat20:20:003
28018799410,39cyclictest365971-21turbostat23:34:593
280183994132,2cyclictest0-21swapper/223:54:552
280182994135,5cyclictest348476-21apt-get22:40:131
280182994122,16cyclictest360511-21missed_timers23:15:221
280182994119,21cyclictest351334-21unixbench_singl22:45:241
280182994117,22cyclictest380618-21apt-get00:20:131
280179994137,2cyclictest714-21Xorg22:09:000
280179994134,6cyclictest367606-21kworker/u8:0+events_unbound00:20:260
280179994119,4cyclictest0-21swapper/020:15:010
28017999411,38cyclictest714-21Xorg19:22:590
280187994035,4cyclictest320389-21turbostat21:15:013
280187994035,4cyclictest314015-21turbostat20:55:013
280187994035,4cyclictest306070-21turbostat20:30:013
280187994034,5cyclictest294665-21turbostat19:55:003
28018799400,38cyclictest357759-21turbostat23:10:003
280182994021,18cyclictest0-21swapper/120:05:001
28018299401,2cyclictest714-21Xorg22:23:001
280179994035,3cyclictest714-21Xorg21:30:590
28017999403,12cyclictest2304-21lightdm-gtk-gre20:15:590
28017999401,37cyclictest714-21Xorg21:15:590
28017999400,4cyclictest377177-21turbostat00:14:550
280187993934,4cyclictest323839-21turbostat21:25:003
280187993934,4cyclictest309249-21turbostat20:40:003
280187993933,5cyclictest286456-21turbostat19:30:003
28018799390,38cyclictest296256-21turbostat20:00:003
28018799390,38cyclictest291491-21turbostat19:45:003
280183993934,3cyclictest0-21swapper/222:37:002
280182993936,2cyclictest0-21swapper/123:38:591
280182993928,8cyclictest306315-21apt-get20:30:131
280182993922,16cyclictest341681-21unixbench-2d22:15:251
280182993917,19cyclictest0-21swapper/123:25:021
280179993934,4cyclictest286593-21kworker/u8:0+flush-179:019:45:260
280179993931,6cyclictest311885-21kworker/u8:3+flush-179:020:55:010
280179993919,18cyclictest0-21swapper/019:50:170
280179993916,6cyclictest0-21swapper/020:20:220
280179993916,5cyclictest291484-21latency_hist19:45:010
28017999391,2cyclictest714-21Xorg00:38:000
280187993833,4cyclictest330458-21turbostat21:44:593
280187993831,6cyclictest304486-21turbostat20:25:003
280183993835,2cyclictest0-21swapper/222:19:592
28018399380,36cyclictest0-21swapper/223:56:582
28018399380,36cyclictest0-21swapper/219:37:592
280182993835,2cyclictest0-21swapper/120:15:001
280182993830,7cyclictest286297-21unixbench-2d19:25:251
280182993817,18cyclictest0-21swapper/122:00:171
280182993813,23cyclictest351626-21/usr/sbin/munin22:50:221
28018299380,2cyclictest0-21swapper/100:32:591
28017999384,12cyclictest2304-21lightdm-gtk-gre00:06:590
280179993834,3cyclictest309289-21systemd-run20:40:000
280179993832,4cyclictest0-21swapper/022:57:580
280179993817,3cyclictest0-21swapper/023:10:180
28018799370,36cyclictest0-21swapper/321:09:583
280183993734,2cyclictest0-21swapper/223:49:562
280183993734,2cyclictest0-21swapper/220:37:592
280183993734,2cyclictest0-21swapper/220:29:562
28018399370,35cyclictest0-21swapper/221:28:002
28018399370,35cyclictest0-21swapper/219:52:592
28018299379,26cyclictest303254-21grep20:20:151
280182993731,4cyclictest354224-21switchtime22:55:261
280182993719,17cyclictest0-21swapper/121:10:231
28018299370,7cyclictest302905-21perf20:20:001
28018299370,36cyclictest0-21swapper/123:06:591
280179993733,3cyclictest367568-21idleruntime-cro23:40:010
280179993730,5cyclictest297860-21apt-get20:05:000
280179993716,4cyclictest0-21swapper/000:15:150
280187993631,4cyclictest359605-21turbostat23:15:003
28018799360,35cyclictest0-21swapper/322:24:543
28018799360,35cyclictest0-21swapper/321:40:003
28018799360,35cyclictest0-21swapper/319:49:593
28018799360,35cyclictest0-21swapper/319:36:593
28018799360,1cyclictest0-21swapper/322:34:563
28018799360,1cyclictest0-21swapper/319:16:583
28018799360,1cyclictest0-21swapper/300:31:593
280183993633,2cyclictest0-21swapper/220:30:592
28018399360,1cyclictest0-21swapper/221:56:002
28018399360,1cyclictest0-21swapper/200:10:002
280182993632,3cyclictest332325-21systemd-run21:50:001
280182993628,7cyclictest292115-21fschecks_time19:45:171
280182993618,16cyclictest0-21swapper/120:35:191
280182993616,18cyclictest714-21Xorg21:31:591
280179993631,4cyclictest383578-21latency_hist00:30:000
280179993631,3cyclictest714-21Xorg22:13:000
280179993629,4cyclictest0-21swapper/019:28:590
28017999362,11cyclictest2304-21lightdm-gtk-gre23:18:590
28017999362,11cyclictest2304-21lightdm-gtk-gre20:31:590
28017999362,10cyclictest685-21snmpd20:45:120
28017999360,35cyclictest0-21swapper/020:44:550
4577091350,3ptp4l441ktimers/319:30:033
280187993533,1cyclictest0-21swapper/323:19:593
280187993533,1cyclictest0-21swapper/319:14:553
280187993529,5cyclictest301283-21turbostat20:15:003
28018799350,34cyclictest0-21swapper/320:43:003
280183993532,2cyclictest0-21swapper/220:46:592
280183993531,2cyclictest0-21swapper/221:03:592
280183993531,2cyclictest0-21swapper/220:41:592
280182993530,4cyclictest369160-21munin-run23:45:001
280182993529,4cyclictest0-21swapper/122:32:581
280179993531,3cyclictest714-21Xorg23:42:590
28017999352,14cyclictest318238-21ntpq21:05:210
28017999351,31cyclictest0-21swapper/019:35:150
280187993432,1cyclictest0-21swapper/323:01:583
280187993432,1cyclictest0-21swapper/321:04:553
280187993429,4cyclictest341840-21turbostat22:19:593
280183993431,2cyclictest0-21swapper/222:28:582
28018299343,30cyclictest370768-21awk23:50:001
280182993428,5cyclictest309511-21kworker/u8:0+events_unbound20:40:201
28017999344,11cyclictest2304-21lightdm-gtk-gre19:13:590
28017999343,11cyclictest2304-21lightdm-gtk-gre21:29:580
280187993328,4cyclictest364387-21turbostat23:30:003
280187993328,4cyclictest364387-21turbostat23:29:593
280187993328,4cyclictest349818-21turbostat22:44:593
280183993330,2cyclictest0-21swapper/222:54:562
280182993328,4cyclictest318776-21kworker/u8:0+flush-179:021:25:251
280179993329,3cyclictest316708-21/usr/sbin/munin21:00:230
280179993329,3cyclictest0-21swapper/021:50:010
280187993230,1cyclictest0-21swapper/323:36:593
280187993229,2cyclictest0-21swapper/300:18:003
280187993226,5cyclictest345033-21turbostat22:30:013
280183993229,2cyclictest0-21swapper/223:41:592
280183993128,2cyclictest0-21swapper/222:14:562
280179993127,3cyclictest0-21swapper/021:40:010
280187993023,4cyclictest362801-21turbostat23:25:003
280183993027,2cyclictest0-21swapper/223:02:582
280183993027,2cyclictest0-21swapper/222:48:582
280183993027,2cyclictest0-21swapper/220:59:002
28018399300,28cyclictest0-21swapper/223:23:592
28018399300,28cyclictest0-21swapper/223:10:022
4577091290,7ptp4l441ktimers/322:55:013
4577091290,7ptp4l441ktimers/320:05:003
4577091290,6ptp4l441ktimers/321:00:003
280187992926,2cyclictest0-21swapper/322:05:123
280187992924,4cyclictest284876-21turbostat19:25:003
28018799290,28cyclictest0-21swapper/300:05:143
28018799290,27cyclictest0-21swapper/322:10:153
28018799290,27cyclictest0-21swapper/321:25:023
280183992927,1cyclictest0-21swapper/223:40:002
280183992926,2cyclictest0-21swapper/221:49:542
280183992926,2cyclictest0-21swapper/220:05:142
28018399290,27cyclictest0-21swapper/223:15:152
280179992927,1cyclictest0-21swapper/022:50:130
280179992922,5cyclictest383788-21apt-get00:30:140
4577091280,7ptp4l61490irq/134-eno1-TxRx-023:00:003
4577091280,6ptp4l61490irq/134-eno1-TxRx-022:05:003
4577091280,6ptp4l441ktimers/320:35:003
280187992826,1cyclictest0-21swapper/321:50:143
280187992826,1cyclictest0-21swapper/300:35:143
280187992825,2cyclictest0-21swapper/321:45:033
28018799280,26cyclictest0-21swapper/320:05:013
280183992825,2cyclictest0-21swapper/220:18:592
280183992825,2cyclictest0-21swapper/219:30:022
280183992824,2cyclictest0-21swapper/200:02:592
28018399280,2cyclictest0-21swapper/220:24:552
28018399280,26cyclictest0-21swapper/222:55:152
28018399280,26cyclictest0-21swapper/221:15:022
28018399280,26cyclictest0-21swapper/200:25:592
28018399280,26cyclictest0-21swapper/200:20:152
4577091270,6ptp4l61490irq/134-eno1-TxRx-022:39:593
4577091270,6ptp4l61490irq/134-eno1-TxRx-000:25:003
4577091270,6ptp4l61490irq/134-eno1-TxRx-000:05:003
4577091270,6ptp4l441ktimers/323:55:003
4577091270,5ptp4l441ktimers/321:30:013
280187992725,1cyclictest0-21swapper/321:16:593
28018799270,26cyclictest335473-21turbostat22:00:013
280183992724,2cyclictest0-21swapper/222:24:002
280183992724,2cyclictest0-21swapper/221:35:152
280183992724,2cyclictest0-21swapper/200:11:582
28018399270,1cyclictest0-21swapper/221:20:592
4577091260,5ptp4l61490irq/134-eno1-TxRx-020:49:563
28018799260,2cyclictest370747-21turbostat23:50:003
28018799260,1cyclictest0-21swapper/322:50:003
280183992624,1cyclictest0-21swapper/219:19:592
280183992623,2cyclictest0-21swapper/221:34:592
280183992623,2cyclictest0-21swapper/220:14:552
280183992623,2cyclictest0-21swapper/220:00:592
280183992623,2cyclictest0-21swapper/219:13:492
28018399260,24cyclictest0-21swapper/223:08:592
28018399260,24cyclictest0-21swapper/222:06:582
28018399260,24cyclictest0-21swapper/221:10:592
28018399260,24cyclictest0-21swapper/219:20:592
28018399260,24cyclictest0-21swapper/200:16:592
28018399250,2cyclictest0-21swapper/222:32:592
28018399250,2cyclictest0-21swapper/222:04:562
28018399250,2cyclictest0-21swapper/219:55:592
4577091240,3ptp4l441ktimers/323:59:553
280183992422,1cyclictest0-21swapper/200:40:002
280183992420,2cyclictest0-21swapper/221:43:002
280183992320,2cyclictest0-21swapper/222:40:142
280183992017,2cyclictest0-21swapper/219:43:592
280183992017,2cyclictest0-21swapper/200:31:592
28017999181,13cyclictest280050-21latency_hist19:10:020
280183991512,2cyclictest0-21swapper/221:09:592
280183991411,2cyclictest0-21swapper/223:27:002
280183991411,2cyclictest0-21swapper/223:26:592
280183991311,1cyclictest0-21swapper/220:55:012
280183991211,1cyclictest0-21swapper/219:49:592
2801839930,2cyclictest0-21swapper/219:10:022
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional