You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-21 - 11:56
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot6.osadl.org (updated Sun Dec 21, 2025 00:44:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
30742519910090,7cyclictest3123749-21latency_hist21:45:001
3074251999773,9cyclictest3167546-40gzip00:00:001
3074249999081,7cyclictest3125332-21latency_hist21:50:010
3074251998775,10cyclictest3111257-21/usr/sbin/munin21:05:231
3074251997973,4cyclictest3133577-21kworker/u8:1+flush-179:022:30:011
3074249997970,6cyclictest3172314-21latency_hist00:15:010
3074251997771,4cyclictest3090036-21kworker/u8:0+events_unbound20:05:261
3074251997771,3cyclictest3133577-21kworker/u8:1+events_unbound23:50:231
3074249997765,5cyclictest3165917-21kworker/u8:3+events_unbound00:15:000
3074251997669,5cyclictest3144733-21kworker/u8:3+flush-179:022:55:011
3074249997564,8cyclictest3176006-21/usr/sbin/munin00:25:260
3074251997267,4cyclictest3082086-21kworker/u8:2+flush-179:019:50:251
3074251997262,7cyclictest3095074-21/usr/sbin/munin20:15:271
3074251997163,6cyclictest3074168-21kworker/u8:1+flush-179:019:20:251
3074251997160,8cyclictest3079478-21diskstats19:25:191
3074249997161,8cyclictest3133577-21kworker/u8:1+flush-179:000:00:000
3074251997062,5cyclictest3109416-21latency_hist21:00:011
3074251996963,4cyclictest3125857-21kworker/u8:3+flush-179:021:50:171
3074249996959,8cyclictest3133541-21latency_hist22:15:010
307425599650,63cyclictest0-21swapper/200:25:582
3074251996559,4cyclictest3133577-21kworker/u8:1+flush-179:023:45:211
3074251996559,4cyclictest3111254-21kworker/u8:0+events_unbound21:45:131
3074249996553,10cyclictest3141775-21apt-get22:40:120
3074251996460,3cyclictest3133577-21kworker/u8:1+flush-179:023:29:551
3074251996456,6cyclictest3169146-21latency_hist00:05:011
3074249996454,8cyclictest3136994-21apt-get22:25:120
307425599630,61cyclictest0-21swapper/220:52:592
3074251996358,3cyclictest3115750-21kworker/u8:1+flush-179:021:30:181
3074251996352,9cyclictest3172329-21apt-get00:15:031
3074249996251,9cyclictest266-21systemd-journal00:10:000
3074251996053,6cyclictest3170985-21apt-get00:10:111
3074249996051,7cyclictest3141514-21latency_hist22:40:010
3074249996050,7cyclictest3160858-21latency_hist23:40:010
3074251995949,7cyclictest3120577-21kworker/u8:4+flush-179:021:35:211
3074251995948,8cyclictest266-21systemd-journal20:35:001
3074251995852,4cyclictest266-21systemd-journal22:45:011
3074251995851,5cyclictest3172292-21kworker/u8:0+flush-179:000:20:241
3074251995851,5cyclictest3125857-21kworker/u8:3+events_unbound21:55:241
3074251995851,5cyclictest3125857-21kworker/u8:3+events_unbound21:55:241
3074251995752,4cyclictest3125857-21kworker/u8:3+flush-179:022:10:231
3074251995752,3cyclictest3144733-21kworker/u8:3+flush-179:022:55:181
3074251995749,6cyclictest3090036-21kworker/u8:0+flush-179:020:00:011
3074251995749,6cyclictest3090036-21kworker/u8:0+flush-179:020:00:001
3074251995748,7cyclictest3154743-21/usr/sbin/munin23:20:261
3074249995749,6cyclictest3075740-21latency_hist19:15:010
3074251995650,4cyclictest3174215-21kworker/u8:1+flush-179:000:25:151
3074251995650,4cyclictest3090037-21kworker/u8:3+flush-179:020:45:011
3074251995650,4cyclictest3083680-21kworker/u8:0+flush-179:019:41:101
3074249995647,8cyclictest3167776-21apt-get00:00:120
3074249995647,7cyclictest3144687-21latency_hist22:50:010
3074249995621,9cyclictest3102791-21latency_hist20:40:010
3074249995444,7cyclictest3164300-21latency_hist23:50:010
3074255995350,2cyclictest0-21swapper/222:25:592
3074251995347,4cyclictest3133577-21kworker/u8:1+flush-179:023:35:191
3074251995345,6cyclictest3102765-21kworker/u8:1+flush-179:020:40:011
307425199530,52cyclictest1-21systemd23:10:001
3074249995348,4cyclictest3152050-21kworker/u8:0+flush-179:023:28:100
3074249995345,5cyclictest3117382-21latency_hist21:25:000
3074251995241,8cyclictest3133577-21kworker/u8:1+events_unbound23:40:031
3074249995243,8cyclictest3088727-21apt-get19:55:140
3074249995243,8cyclictest3088727-21apt-get19:55:130
3074251995147,3cyclictest3125857-21kworker/u8:3+flush-179:022:04:571
3074251995146,4cyclictest3111254-21kworker/u8:0+flush-179:021:25:201
3074251995146,4cyclictest3111254-21kworker/u8:0+flush-179:021:20:221
3074251995146,3cyclictest2399125-21kworker/u8:3+flush-179:019:30:241
3074251995145,4cyclictest3090037-21kworker/u8:3+flush-179:020:00:271
3074249995142,8cyclictest3174439-21apt-get00:20:110
3074249995142,7cyclictest266-21systemd-journal20:15:000
3074251995046,3cyclictest3176027-21kworker/u8:2+events_unbound00:35:251
4577091490,14ptp4l61490irq/134-eno1-TxRx-019:10:003
3074251994944,4cyclictest3102765-21kworker/u8:1+flush-179:020:50:171
3074251994943,5cyclictest3152050-21kworker/u8:0+flush-179:023:35:001
3074249994941,6cyclictest3133577-21kworker/u8:1+flush-179:023:50:210
3074249994940,8cyclictest3119225-21apt-get21:30:130
3074251994839,7cyclictest266-21systemd-journal22:10:001
3074251994836,11cyclictest3140895-21kworker/u8:2+flush-179:022:49:551
3074251994741,4cyclictest3090037-21kworker/u8:3+flush-179:020:20:191
3074249994743,3cyclictest3152050-21kworker/u8:0+flush-179:023:33:100
3074249994740,5cyclictest3136740-21latency_hist22:25:010
3074249994740,5cyclictest266-21systemd-journal22:53:260
3074251994641,4cyclictest3109465-21kworker/u8:2+flush-179:021:00:171
3074251994639,5cyclictest3083680-21kworker/u8:0+flush-179:019:45:251
3074251994638,6cyclictest3074168-21kworker/u8:1+flush-179:019:20:001
307425199461,2cyclictest714-21Xorg21:15:591
3074251994541,2cyclictest3133577-21kworker/u8:1+flush-179:023:10:211
3074251994540,3cyclictest3112635-21kworker/u8:3+events_unbound21:10:201
3074251994537,5cyclictest3125857-21kworker/u8:3+events_unbound22:35:171
3074251994439,4cyclictest3125857-21kworker/u8:3+flush-179:022:15:221
3074251994429,6cyclictest3082086-21kworker/u8:2+events_unbound19:39:591
307425599430,41cyclictest0-21swapper/200:09:582
3074251994337,5cyclictest3165917-21kworker/u8:3+flush-179:000:05:261
3074257994240,1cyclictest0-21swapper/320:56:593
307425799420,41cyclictest0-21swapper/323:34:553
3074251994236,4cyclictest3102765-21kworker/u8:1+flush-179:020:45:221
3074251994236,4cyclictest3102765-21kworker/u8:1+flush-179:020:45:221
3074249994236,5cyclictest3090037-21kworker/u8:3+flush-179:020:00:260
3074249994137,2cyclictest714-21Xorg21:38:590
307424999411,38cyclictest714-21Xorg21:00:590
307425799400,38cyclictest3122152-21turbostat21:44:553
307425799400,38cyclictest3086892-21turbostat19:50:003
3074251994031,7cyclictest3176027-21kworker/u8:2+events_unbound00:30:221
307424999400,38cyclictest714-21Xorg19:52:000
307425799390,1cyclictest0-21swapper/319:36:003
307425599390,36cyclictest0-21swapper/219:14:552
3074251993936,2cyclictest714-21Xorg20:12:581
3074251993933,4cyclictest3125857-21kworker/u8:3+flush-179:022:20:211
3074251993926,5cyclictest3133577-21kworker/u8:1+events_unbound23:20:001
3074251993926,4cyclictest2399125-21kworker/u8:3+events_unbound19:15:001
3074249993935,2cyclictest714-21Xorg19:20:590
3074255993835,2cyclictest0-21swapper/221:45:002
307425599380,36cyclictest0-21swapper/219:42:592
307425599380,36cyclictest0-21swapper/200:21:592
307425599380,35cyclictest0-21swapper/223:21:002
3074249993834,2cyclictest714-21Xorg19:37:000
3074249993833,4cyclictest0-21swapper/000:34:000
3074249993832,4cyclictest3126956-21systemd-run21:55:000
3074249993831,4cyclictest3099613-21latency_hist20:30:010
307424999381,35cyclictest0-21swapper/020:50:130
3074257993734,2cyclictest0-21swapper/320:21:593
307425599370,35cyclictest0-21swapper/223:02:592
307425599370,35cyclictest0-21swapper/222:48:592
307425599370,35cyclictest0-21swapper/222:34:592
3074251993732,4cyclictest3133577-21kworker/u8:1+flush-179:023:00:241
3074251993731,5cyclictest3125857-21kworker/u8:3+flush-179:022:30:251
3074249993734,2cyclictest3111254-21kworker/u8:0-i91521:41:590
3074249993732,4cyclictest2399125-21kworker/u8:3+flush-179:019:32:110
3074249993731,4cyclictest3178957-21turbostat00:39:540
3074249993727,6cyclictest0-21swapper/021:55:590
3074249993727,6cyclictest0-21swapper/021:55:590
307425799360,35cyclictest0-21swapper/322:37:593
307425799360,35cyclictest0-21swapper/321:38:003
307425799360,35cyclictest0-21swapper/319:54:563
307425799360,35cyclictest0-21swapper/300:37:583
307425799360,1cyclictest0-21swapper/321:58:593
307425799360,1cyclictest0-21swapper/321:58:593
307425799360,1cyclictest0-21swapper/321:21:593
3074255993634,1cyclictest0-21swapper/223:55:002
3074255993632,2cyclictest0-21swapper/219:51:592
307425599360,34cyclictest0-21swapper/220:18:002
307425599360,1cyclictest0-21swapper/222:09:592
3074249993632,3cyclictest3133577-21kworker/u8:1+flush-179:023:10:510
3074249993632,3cyclictest0-21swapper/020:30:590
3074249993627,7cyclictest0-21swapper/023:05:150
3074257993533,1cyclictest0-21swapper/322:22:593
3074257993533,1cyclictest0-21swapper/321:33:593
3074257993533,1cyclictest0-21swapper/321:12:593
307425799350,34cyclictest0-21swapper/323:37:593
3074255993532,2cyclictest0-21swapper/223:39:552
3074255993532,2cyclictest0-21swapper/221:00:592
3074255993532,1cyclictest0-21swapper/200:15:002
3074249993532,2cyclictest714-21Xorg22:08:000
3074249993531,3cyclictest484-21dbus-daemon20:20:000
3074249993526,7cyclictest510-21rs:main0
231999350,9rtkit-daemon2304-21lightdm-gtk-gre19:09:001
3074257993432,1cyclictest0-21swapper/323:11:593
3074257993432,1cyclictest0-21swapper/322:34:563
3074257993432,1cyclictest0-21swapper/321:05:593
3074257993431,2cyclictest0-21swapper/320:18:593
3074257993431,2cyclictest0-21swapper/320:00:583
3074249993431,2cyclictest3077332-21systemctl19:20:000
3074249993430,3cyclictest3139915-21munin-run22:35:000
3074249993427,5cyclictest3152895-21apt-get23:15:140
3074249993426,6cyclictest3093250-21latency_hist20:10:010
3074257993330,2cyclictest0-21swapper/322:04:553
3074255993330,2cyclictest0-21swapper/222:24:592
3074255993330,2cyclictest0-21swapper/200:02:002
3074249993329,3cyclictest3086887-21munin-run19:50:000
3074249993328,4cyclictest3115149-21needreboot21:15:230
3074249993327,4cyclictest3149632-21latency_hist23:05:010
3074249993324,7cyclictest510-21rs:main0
307424999331,31cyclictest3160847-21cron23:40:020
3074257993230,1cyclictest0-21swapper/323:41:583
3074257993230,1cyclictest0-21swapper/322:49:543
3074257993230,1cyclictest0-21swapper/320:47:003
3074257993230,1cyclictest0-21swapper/320:46:593
3074257993230,1cyclictest0-21swapper/320:29:593
307424999321,29cyclictest3104911-21apt-get20:45:150
307424999321,29cyclictest3104911-21apt-get20:45:140
307424999321,29cyclictest3103325-21apt-get20:40:140
307424999320,31cyclictest714-21Xorg19:43:580
4577091310,7ptp4l441ktimers/321:50:003
3074257993129,1cyclictest0-21swapper/323:51:593
3074257993129,1cyclictest0-21swapper/322:44:593
307425799310,29cyclictest3178957-21turbostat00:35:003
3074255993128,2cyclictest0-21swapper/223:58:592
307424999310,29cyclictest714-21Xorg22:15:590
231999310,3rtkit-daemon2318-21rtkit-daemon21:28:220
4577091300,7ptp4l441ktimers/323:50:003
3074257993028,1cyclictest0-21swapper/322:18:593
3074257993028,1cyclictest0-21swapper/322:14:553
3074255993027,2cyclictest0-21swapper/221:30:152
3074255993027,2cyclictest0-21swapper/220:40:132
3074255993026,2cyclictest0-21swapper/220:04:552
3074255993025,2cyclictest0-21swapper/223:40:032
307425599300,2cyclictest0-21swapper/223:05:152
307425599300,26cyclictest0-21swapper/220:10:172
3074249993024,4cyclictest714-21Xorg20:20:590
3074249993021,7cyclictest266-21systemd-journal21:00:010
4577091290,7ptp4l61490irq/134-eno1-TxRx-019:15:003
4577091290,6ptp4l441ktimers/321:20:013
4577091290,5ptp4l441ktimers/300:09:563
3074257992927,1cyclictest0-21swapper/300:14:593
3074257992926,2cyclictest0-21swapper/320:05:143
307425799290,28cyclictest0-21swapper/300:15:033
307425799290,27cyclictest0-21swapper/322:25:143
3074255992927,1cyclictest0-21swapper/200:37:592
3074255992926,2cyclictest0-21swapper/222:15:142
3074255992926,2cyclictest0-21swapper/219:37:002
3074255992924,2cyclictest0-21swapper/223:15:142
307425599290,27cyclictest0-21swapper/223:10:122
307425599290,27cyclictest0-21swapper/222:00:122
3074251992922,5cyclictest3090037-21kworker/u8:3+events_unbound20:29:591
3074249992926,2cyclictest3147873-21basename23:00:000
307424999291,25cyclictest3114191-21apt21:15:000
4577091280,7ptp4l61490irq/134-eno1-TxRx-023:10:003
4577091280,7ptp4l441ktimers/323:19:563
4577091280,6ptp4l61490irq/134-eno1-TxRx-022:10:003
4577091280,6ptp4l441ktimers/323:25:013
4577091280,6ptp4l441ktimers/323:00:013
4577091280,6ptp4l441ktimers/300:25:003
3074257992826,1cyclictest0-21swapper/323:04:553
3074257992826,1cyclictest0-21swapper/321:51:593
3074257992825,2cyclictest0-21swapper/319:40:143
307425799280,2cyclictest0-21swapper/319:15:153
307425799280,26cyclictest0-21swapper/323:55:133
3074255992824,2cyclictest0-21swapper/220:31:002
307425599280,26cyclictest0-21swapper/222:36:592
307425599280,26cyclictest0-21swapper/220:45:142
307425599280,26cyclictest0-21swapper/220:45:132
307425599280,26cyclictest0-21swapper/219:20:002
3074249992822,5cyclictest3078949-21turbostat19:29:550
4577091270,6ptp4l61490irq/134-eno1-TxRx-023:30:003
4577091270,5ptp4l441ktimers/321:30:003
4577091270,5ptp4l441ktimers/320:45:013
4577091270,5ptp4l441ktimers/300:30:003
3074257992725,1cyclictest0-21swapper/322:50:153
307425799270,26cyclictest0-21swapper/320:34:593
3074255992724,2cyclictest351rcuc/219:21:002
3074255992724,2cyclictest0-21swapper/221:50:592
3074255992724,2cyclictest0-21swapper/221:21:002
3074255992724,2cyclictest0-21swapper/200:34:002
3074255992724,2cyclictest0-21swapper/200:17:002
3074255992723,2cyclictest0-21swapper/219:58:592
3074255992723,2cyclictest0-21swapper/219:58:592
307425599270,24cyclictest0-21swapper/221:37:592
4577091260,5ptp4l61490irq/134-eno1-TxRx-000:04:543
4577091260,5ptp4l441ktimers/321:04:563
4577091260,5ptp4l441ktimers/320:15:013
4577091260,5ptp4l441ktimers/319:34:563
4577091260,5ptp4l441ktimers/319:29:553
4577091260,5ptp4l441ktimers/319:24:563
3074257992624,1cyclictest0-21swapper/319:59:593
3074257992624,1cyclictest0-21swapper/319:59:583
3074255992623,2cyclictest0-21swapper/223:49:002
3074255992623,2cyclictest0-21swapper/222:54:542
3074255992623,2cyclictest0-21swapper/222:13:592
307424999260,25cyclictest0-21swapper/021:05:590
3074255992523,1cyclictest0-21swapper/221:19:592
307425599250,2cyclictest0-21swapper/221:12:592
307425599250,2cyclictest0-21swapper/220:56:582
307425599250,2cyclictest0-21swapper/219:31:582
307425599250,2cyclictest0-21swapper/219:25:592
4577091240,4ptp4l441ktimers/320:40:003
3074257992422,1cyclictest0-21swapper/320:53:593
3074255992421,2cyclictest0-21swapper/223:28:592
3074255992421,2cyclictest0-21swapper/222:00:022
3074255992421,2cyclictest0-21swapper/221:47:592
3074255992421,2cyclictest0-21swapper/220:21:592
3074255992118,2cyclictest0-21swapper/221:06:002
3074255992018,1cyclictest0-21swapper/221:59:592
3074255992018,1cyclictest0-21swapper/220:29:582
3074255991917,1cyclictest0-21swapper/222:44:592
3074255991916,2cyclictest0-21swapper/220:05:592
3074255991715,1cyclictest0-21swapper/219:49:592
3074255991613,2cyclictest0-21swapper/221:25:162
3074255991511,2cyclictest351rcuc/222:57:432
3074249991512,2cyclictest3074245-21cyclictest19:10:010
3074255991412,1cyclictest0-21swapper/223:34:592
3074255991311,1cyclictest0-21swapper/220:40:002
30742559930,1cyclictest0-21swapper/219:10:022
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional