You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-04 - 14:34
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot6.osadl.org (updated Thu Jun 04, 2026 00:44:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
84127499125115,7cyclictest915716-21kworker/u8:2+flush-179:000:04:000
8412749911298,11cyclictest855037-21latency_hist20:20:020
8412749911298,11cyclictest855037-21latency_hist20:20:010
8412759910892,12cyclictest1213033-21systemd-journal20:15:011
8412749910697,7cyclictest1213033-21systemd-journal21:35:000
8412759910377,23cyclictest887911-21latency_hist22:25:001
8412759910293,6cyclictest856885-21latency_hist20:25:021
8412749910189,9cyclictest930378-21latency_hist00:35:000
841274999987,10cyclictest1213033-21systemd-journal23:50:000
841275999890,5cyclictest866168-21latency_hist21:15:021
841274999787,7cyclictest876247-21latency_hist21:50:000
841274999383,8cyclictest844958-21latency_hist19:30:010
841275999068,19cyclictest857403-21apt-get20:30:071
841274999078,10cyclictest884971-21apt-get22:15:130
841274998978,9cyclictest1213033-21systemd-journal22:15:010
841274998776,6cyclictest854990-21kworker/u8:2+flush-179:020:20:170
841274998674,10cyclictest928546-21apt-get00:30:000
841275998474,7cyclictest860865-21latency_hist20:50:011
841274998462,20cyclictest1213033-21systemd-journal22:10:000
841274998272,8cyclictest1213033-21systemd-journal00:00:010
841275998175,4cyclictest902532-21kworker/u8:2+flush-179:023:20:251
841274998073,5cyclictest850777-21latency_hist19:55:010
841274997973,4cyclictest820055-21kworker/u8:1+flush-179:019:14:560
841274997870,5cyclictest881321-21kworker/u8:3+flush-179:022:25:140
841274997870,5cyclictest878874-21kworker/u8:0+events_unbound22:00:020
841275997771,4cyclictest866404-21kworker/u8:1+flush-179:021:45:011
841274997771,4cyclictest902532-21kworker/u8:2+events_unbound23:20:230
841274997770,5cyclictest892648-21kworker/u8:0+flush-179:022:45:000
841275997566,7cyclictest862989-21apt-get21:00:081
841274997567,6cyclictest895895-21kworker/u8:3+flush-179:023:35:220
841274997564,8cyclictest917070-21latency_hist23:55:010
841275997467,5cyclictest851066-21latency_hist20:00:021
841275997464,8cyclictest913927-21kworker/u8:1+flush-179:000:10:191
841275997459,13cyclictest843061-21kworker/u8:4+flush-179:019:35:011
841275997449,22cyclictest1213033-21systemd-journal23:59:391
841274997363,7cyclictest895852-21kworker/u8:2+flush-179:023:00:210
841275997264,6cyclictest930714-21gpgv00:35:011
841275997166,4cyclictest854990-21kworker/u8:2+flush-179:021:37:361
841274997164,5cyclictest849168-21kworker/u8:1+flush-179:020:00:120
841274997061,6cyclictest846790-21kworker/u8:0+flush-179:019:40:180
841275996957,10cyclictest879954-21apt-get22:00:111
841274996863,4cyclictest854990-21kworker/u8:2+flush-179:022:20:220
841274996837,1cyclictest0-21swapper/023:30:590
841274996837,1cyclictest0-21swapper/023:30:590
841275996759,6cyclictest880708-21kworker/u8:1+flush-179:022:10:161
841275996758,6cyclictest874404-21latency_hist21:45:001
841275996756,9cyclictest1213033-21systemd-journal19:40:001
841274996760,5cyclictest915716-21kworker/u8:2+flush-179:000:05:240
841274996661,4cyclictest866404-21kworker/u8:1+flush-179:021:40:200
841274996661,3cyclictest854990-21kworker/u8:2+flush-179:021:04:560
84127699650,1cyclictest0-21swapper/200:27:592
841275996556,6cyclictest849193-21latency_hist19:50:011
841274996560,4cyclictest854990-21kworker/u8:2+flush-179:022:35:010
841274996460,3cyclictest892648-21kworker/u8:0+flush-179:022:50:270
841275996358,4cyclictest854990-21kworker/u8:2+events_unbound20:50:191
841275996357,5cyclictest894261-21kworker/u8:1+events_unbound22:45:221
841274996356,5cyclictest854990-21kworker/u8:2+flush-179:022:00:210
841275996256,4cyclictest892648-21kworker/u8:0+flush-179:022:50:231
841274996248,5cyclictest843061-21kworker/u8:4+events_unbound19:30:190
841275996152,6cyclictest891083-21latency_hist22:35:011
841274996151,7cyclictest892684-21latency_hist22:40:010
841276996057,2cyclictest0-21swapper/222:18:592
841275996043,15cyclictest881524-21/usr/sbin/munin22:05:171
841274996057,2cyclictest0-21swapper/019:55:590
841274996055,4cyclictest854990-21kworker/u8:2+events_unbound21:35:260
841274995953,4cyclictest921949-21kworker/u8:0+flush-179:000:20:210
841274995951,5cyclictest923494-21latency_hist00:15:010
841274995850,5cyclictest930893-21apt-get00:35:110
841275995748,6cyclictest884913-21cat22:15:011
841275995623,31cyclictest714-21Xorg23:30:591
841275995623,31cyclictest714-21Xorg23:30:591
841274995650,4cyclictest843061-21kworker/u8:4+events_unbound19:20:110
841279995545,7cyclictest866186-21turbostat21:15:003
841275995549,4cyclictest854990-21kworker/u8:2+events_unbound21:20:201
841275995548,5cyclictest854990-21kworker/u8:2+flush-179:020:20:011
841275995548,5cyclictest854990-21kworker/u8:2+flush-179:020:20:001
841275995546,7cyclictest1213033-21systemd-journal20:30:001
841275995448,4cyclictest925061-21kworker/u8:3+events_unbound00:25:171
841274995447,5cyclictest892648-21kworker/u8:0+flush-179:022:55:210
841274995445,7cyclictest1213033-21systemd-journal20:53:340
841275995350,2cyclictest895895-21kworker/u8:3+events_unbound23:15:231
841274995348,4cyclictest858551-21kworker/u8:0+events_unbound21:00:000
841274995342,7cyclictest905923-21latency_hist23:20:010
841275995246,4cyclictest895895-21kworker/u8:3+flush-179:023:25:191
841275995245,5cyclictest915716-21kworker/u8:2+events_unbound00:35:151
841275995245,5cyclictest854990-21kworker/u8:2+events_unbound21:50:251
841274995249,2cyclictest858551-21kworker/u8:0+flush-179:021:20:110
841274995247,4cyclictest854990-21kworker/u8:2+events_unbound21:10:130
841274995246,4cyclictest866404-21kworker/u8:1+flush-179:021:25:210
841274995240,9cyclictest843100-21latency_hist19:20:010
84127499520,48cyclictest860643-21munin-run20:45:010
841275995144,5cyclictest918716-21kworker/u8:3+flush-179:000:00:141
841276995047,2cyclictest0-21swapper/219:52:592
841275995043,5cyclictest907509-21kworker/u8:0+events_unbound23:50:131
841274995045,4cyclictest849168-21kworker/u8:1+events_unbound20:30:110
841275994841,5cyclictest1213033-21systemd-journal21:30:131
841275994839,7cyclictest852910-21latency_hist20:10:011
841274994842,5cyclictest894261-21kworker/u8:1+flush-179:022:45:210
841274994838,8cyclictest1213033-21systemd-journal20:49:560
841279994745,1cyclictest0-21swapper/300:00:593
841275994741,4cyclictest866404-21kworker/u8:1+flush-179:021:25:251
841275994740,6cyclictest921949-21kworker/u8:0+flush-179:000:15:221
841274994742,4cyclictest854990-21kworker/u8:2+flush-179:021:50:220
4577091470,13ptp4l441ktimers/319:10:013
841275994640,4cyclictest881321-21kworker/u8:3+events_unbound22:35:221
841274994641,4cyclictest852685-21kworker/u8:2+flush-179:020:10:100
841275994540,3cyclictest714-21Xorg23:03:001
841275994539,5cyclictest820055-21kworker/u8:1+flush-179:019:14:321
841275994536,7cyclictest843061-21kworker/u8:4+events_unbound19:20:171
841275994439,4cyclictest907509-21kworker/u8:0+flush-179:023:40:201
841275994437,6cyclictest858551-21kworker/u8:0+flush-179:020:45:011
841275994435,7cyclictest895895-21kworker/u8:3+flush-179:023:35:261
841274994440,3cyclictest909095-21kworker/u8:1+flush-179:023:29:550
841274994439,4cyclictest921949-21kworker/u8:0+flush-179:000:15:220
841274994438,5cyclictest902532-21kworker/u8:2+events_unbound23:10:220
841276994340,2cyclictest0-21swapper/222:04:002
841275994335,7cyclictest850812-21kworker/u8:3+flush-179:020:00:181
841275994237,4cyclictest915716-21kworker/u8:2+flush-179:000:05:251
841275994236,5cyclictest895895-21kworker/u8:3+flush-179:023:05:131
841274994238,2cyclictest852685-21kworker/u8:2+flush-179:020:05:350
841274994236,5cyclictest895895-21kworker/u8:3+flush-179:023:05:200
841279994129,10cyclictest915496-21turbostat23:50:003
84127999410,39cyclictest899301-21turbostat23:00:003
841276994138,2cyclictest0-21swapper/221:54:552
841275994136,4cyclictest921949-21kworker/u8:0+flush-179:000:20:171
841275994135,5cyclictest846790-21kworker/u8:0+events_unbound19:40:201
841274994134,4cyclictest0-21swapper/019:39:590
841274994133,6cyclictest912850-21diskstats23:40:150
84127999400,38cyclictest847331-21turbostat19:40:003
84127699390,36cyclictest0-21swapper/219:44:552
841275993933,4cyclictest902983-21/usr/sbin/munin23:10:161
841275993931,7cyclictest888604-21hwlatdetect22:25:161
84127699380,36cyclictest0-21swapper/220:45:592
84127699380,35cyclictest0-21swapper/222:15:002
84127699380,35cyclictest0-21swapper/200:14:552
841275993835,2cyclictest0-21swapper/122:44:591
841274993835,2cyclictest858551-21kworker/u8:0+flush-179:021:16:360
841279993735,1cyclictest0-21swapper/321:53:583
841276993734,2cyclictest0-21swapper/223:20:592
841276993734,2cyclictest0-21swapper/220:29:552
841276993734,2cyclictest0-21swapper/219:15:002
841276993734,2cyclictest0-21swapper/200:32:592
84127699370,35cyclictest0-21swapper/221:31:002
841275993732,4cyclictest878991-21sendmail_mailqu21:55:221
841275993731,4cyclictest0-21swapper/119:30:001
841275993729,7cyclictest898336-21fschecks_time22:55:141
841274993734,2cyclictest0-21swapper/019:47:580
841279993634,1cyclictest0-21swapper/323:16:593
841279993634,1cyclictest0-21swapper/323:13:593
841279993634,1cyclictest0-21swapper/319:56:593
84127999360,35cyclictest0-21swapper/319:43:003
841276993633,2cyclictest0-21swapper/223:00:592
84127699360,34cyclictest0-21swapper/222:09:582
841275993632,3cyclictest714-21Xorg19:18:591
841275993628,7cyclictest850623-21unixbench-2d19:50:201
841274993629,5cyclictest510-21rs:main0
841279993533,1cyclictest0-21swapper/323:24:583
841279993533,1cyclictest0-21swapper/320:51:593
841276993532,2cyclictest0-21swapper/223:05:592
84127699350,1cyclictest0-21swapper/223:48:592
841275993526,8cyclictest914524-21fschecks_time23:45:151
4577091350,3ptp4l61490irq/134-eno1-TxRx-019:10:133
841279993432,1cyclictest0-21swapper/322:34:563
841279993432,1cyclictest0-21swapper/322:02:593
84127999340,1cyclictest0-21swapper/300:10:133
841276993431,2cyclictest0-21swapper/222:49:592
841276993431,2cyclictest0-21swapper/222:37:592
841275993428,4cyclictest864670-21latency_hist21:10:001
841279993331,1cyclictest0-21swapper/320:34:593
841279993331,1cyclictest0-21swapper/319:24:593
841279993331,1cyclictest0-21swapper/300:06:593
84127999330,26cyclictest0-21swapper/300:21:593
841276993331,1cyclictest0-21swapper/219:49:592
841276993330,2cyclictest0-21swapper/221:12:002
841276993330,2cyclictest0-21swapper/220:10:582
84127699330,31cyclictest0-21swapper/220:39:592
841279993230,1cyclictest0-21swapper/320:47:003
841279993230,1cyclictest0-21swapper/320:14:003
84127999320,1cyclictest0-21swapper/300:35:023
841276993229,2cyclictest0-21swapper/220:33:002
231999320,2rtkit-daemon0-21swapper/120:59:141
841279993129,1cyclictest0-21swapper/321:34:553
841279993123,7cyclictest0-21swapper/300:35:013
841276993129,1cyclictest0-21swapper/200:24:592
841276993128,2cyclictest0-21swapper/223:17:592
841276993128,2cyclictest0-21swapper/221:15:592
841276993128,2cyclictest0-21swapper/220:59:552
84127999300,1cyclictest0-21swapper/322:15:143
841276993027,2cyclictest0-21swapper/221:20:092
841276993027,2cyclictest0-21swapper/219:17:592
84127699300,2cyclictest0-21swapper/222:40:582
84127699300,28cyclictest0-21swapper/221:45:122
841279992927,1cyclictest0-21swapper/322:20:133
841279992927,1cyclictest0-21swapper/322:00:023
841279992927,1cyclictest0-21swapper/320:40:093
841279992926,2cyclictest0-21swapper/320:20:023
841279992921,7cyclictest0-21swapper/323:02:593
84127999290,28cyclictest0-21swapper/320:20:013
84127999290,27cyclictest0-21swapper/319:30:083
84127999290,21cyclictest0-21swapper/321:25:133
841276992927,1cyclictest0-21swapper/221:25:592
841276992926,2cyclictest0-21swapper/223:30:132
841276992926,2cyclictest0-21swapper/223:30:122
841276992926,2cyclictest0-21swapper/223:10:022
841276992926,2cyclictest0-21swapper/222:55:012
841276992926,2cyclictest0-21swapper/200:07:592
84127699290,28cyclictest0-21swapper/219:35:012
84127699290,27cyclictest0-21swapper/221:55:122
84127699290,27cyclictest0-21swapper/220:20:092
84127699290,27cyclictest0-21swapper/220:15:032
84127699290,27cyclictest0-21swapper/220:15:032
841275992925,3cyclictest841364-21latency_hist19:10:021
84127499290,28cyclictest0-21swapper/020:35:580
4577091290,7ptp4l441ktimers/323:35:013
4577091290,7ptp4l441ktimers/323:35:013
841279992825,2cyclictest0-21swapper/321:45:013
84127999280,2cyclictest0-21swapper/320:20:023
84127999280,26cyclictest0-21swapper/323:25:123
841276992825,2cyclictest0-21swapper/223:26:592
841276992825,2cyclictest0-21swapper/222:55:022
841276992825,2cyclictest0-21swapper/222:25:112
841276992825,2cyclictest0-21swapper/219:35:022
841276992825,2cyclictest0-21swapper/200:02:592
84127699280,2cyclictest0-21swapper/221:00:082
84127699280,21cyclictest0-21swapper/223:40:132
84127599280,27cyclictest714-21Xorg20:37:591
4577091280,7ptp4l441ktimers/322:10:003
4577091280,6ptp4l441ktimers/322:15:003
4577091280,6ptp4l441ktimers/321:45:013
4577091280,6ptp4l441ktimers/320:40:003
841279992725,1cyclictest0-21swapper/323:07:593
841279992725,1cyclictest0-21swapper/321:03:003
84127999270,26cyclictest0-21swapper/319:30:013
84127999270,1cyclictest0-21swapper/322:53:593
84127999270,1cyclictest0-21swapper/322:45:123
84127999270,1cyclictest0-21swapper/300:25:023
841275992720,5cyclictest1213033-21systemd-journal21:17:351
841274992723,3cyclictest0-21swapper/021:10:010
4577091270,6ptp4l61490irq/134-eno1-TxRx-020:30:003
4577091270,6ptp4l61490irq/134-eno1-TxRx-019:55:013
4577091270,6ptp4l441ktimers/323:59:563
4577091270,6ptp4l441ktimers/323:40:003
4577091270,6ptp4l441ktimers/322:39:553
841279992624,1cyclictest0-21swapper/323:50:593
841279992624,1cyclictest0-21swapper/319:48:593
841279992623,2cyclictest0-21swapper/321:38:593
841276992623,2cyclictest0-21swapper/221:35:582
84127699260,2cyclictest0-21swapper/222:21:002
84127699260,2cyclictest0-21swapper/220:53:592
84127699260,24cyclictest0-21swapper/222:30:132
84127699260,24cyclictest0-21swapper/221:05:592
84127699260,24cyclictest0-21swapper/219:26:592
4577091260,6ptp4l441ktimers/322:45:003
4577091260,6ptp4l441ktimers/321:00:003
4577091260,6ptp4l441ktimers/319:19:553
4577091260,5ptp4l61490irq/134-eno1-TxRx-020:09:593
4577091260,5ptp4l441ktimers/323:44:553
841279992522,2cyclictest0-21swapper/300:15:143
84127999250,1cyclictest0-21swapper/321:07:593
841276992523,1cyclictest0-21swapper/223:35:592
841276992523,1cyclictest0-21swapper/220:07:592
841276992522,2cyclictest0-21swapper/219:21:592
84127699250,1cyclictest0-21swapper/220:44:592
4577091250,6ptp4l61490irq/134-eno1-TxRx-022:30:003
4577091250,5ptp4l61490irq/134-eno1-TxRx-021:25:013
4577091250,5ptp4l441ktimers/321:19:553
841276992422,1cyclictest0-21swapper/200:38:592
841276992421,2cyclictest0-21swapper/223:51:592
84127699240,1cyclictest0-21swapper/220:04:592
4577091240,4ptp4l441ktimers/320:04:563
841276992320,2cyclictest0-21swapper/223:59:552
841274992117,3cyclictest2318-21rtkit-daemon19:10:020
841276992018,1cyclictest0-21swapper/219:59:592
841276991714,2cyclictest0-21swapper/200:17:592
8412769980,7cyclictest0-21swapper/221:44:592
8412769941,2cyclictest0-21swapper/219:10:022
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional