You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-13 - 07:16
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot6.osadl.org (updated Mon Apr 13, 2026 00:44:54)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
393936599111101,7cyclictest4037534-21latency_hist00:15:011
3939364999381,9cyclictest4032470-40mandb00:00:010
3939365999282,8cyclictest4002126-21/usr/sbin/munin22:25:231
3939365998575,7cyclictest3987970-21kworker/u8:0+flush-179:021:50:231
3939365998377,4cyclictest3945672-21kworker/u8:1+flush-179:019:35:241
3939365998374,7cyclictest3984111-21latency_hist21:30:011
3939364998373,7cyclictest3982523-21latency_hist21:25:010
3939365998273,7cyclictest266-21systemd-journal21:50:001
3939364998254,2cyclictest0-21swapper/022:59:000
3939365998176,4cyclictest4031427-21kworker/u8:1+flush-179:000:00:261
3939364998063,14cyclictest4006660-21latency_hist22:40:000
3939364997761,14cyclictest3959960-21latency_hist20:15:010
3939365997567,6cyclictest3995262-21latency_hist22:05:021
3939364997568,5cyclictest3987970-21kworker/u8:0+flush-179:021:55:260
3939365997466,6cyclictest3954609-21kworker/u8:1+flush-179:019:55:241
4577091720,13ptp4l431rcuc/319:10:013
3939364997264,6cyclictest3918648-21kworker/u8:2+flush-179:019:10:260
3939365997163,6cyclictest3944048-21latency_hist19:25:011
3939365997163,6cyclictest3944048-21latency_hist19:25:011
3939365997162,7cyclictest3948835-21kworker/u8:3+flush-179:019:40:201
3939364997161,8cyclictest4034113-21apt-get00:04:590
3939364997062,5cyclictest3971371-21latency_hist20:50:010
3939365996959,8cyclictest3960473-21apt-get20:15:111
3939365996760,5cyclictest4010102-21latency_hist22:50:011
3939365996760,5cyclictest3965974-21kworker/u8:0+flush-179:020:50:181
3939365996760,5cyclictest3965974-21kworker/u8:0+flush-179:020:50:181
3939365996756,8cyclictest266-21systemd-journal21:00:001
3939364996757,7cyclictest4045762-21latency_hist00:40:020
3939365996457,5cyclictest3973553-21kworker/u8:1+flush-179:021:05:171
3939364996353,7cyclictest4016463-21kworker/u8:0+flush-179:023:15:200
3939365996252,8cyclictest266-21systemd-journal23:15:011
3939364996254,5cyclictest3942451-21latency_hist19:20:010
3939364996158,2cyclictest0-21swapper/021:40:580
3939365996052,6cyclictest3945615-21kworker/u8:0+flush-179:020:20:221
3939364996055,3cyclictest4008561-21kworker/u8:2+flush-179:023:00:210
3939365995953,4cyclictest3948835-21kworker/u8:3+flush-179:019:50:241
3939364995850,6cyclictest4036205-21apt-get00:10:120
3939365995649,5cyclictest3990514-21kworker/u8:2+flush-179:022:15:231
3939365995649,5cyclictest3987970-21kworker/u8:0+flush-179:021:55:151
3939364995649,5cyclictest3954609-21kworker/u8:1+events_unbound20:05:010
3939365995550,4cyclictest4026125-21kworker/u8:1+events_unbound23:40:011
3939365995549,5cyclictest3960021-21kworker/u8:2+flush-179:020:50:001
3939365995348,4cyclictest3979563-21kworker/u8:1+flush-179:021:20:011
3939365995341,10cyclictest3950673-21apt-get19:45:131
3939365995341,10cyclictest3950673-21apt-get19:45:131
3939365995244,6cyclictest4026125-21kworker/u8:1+flush-179:023:40:221
3939365995244,6cyclictest3988894-21latency_hist21:45:011
3939365995243,7cyclictest4044421-21apt-get00:35:121
3939364995245,5cyclictest3987970-21kworker/u8:0+events_unbound21:45:130
3939365995146,4cyclictest3937787-21kworker/u8:3+flush-179:019:25:151
3939365995144,5cyclictest4030848-21latency_hist23:55:011
3939365995141,8cyclictest3959964-21apt-get20:15:001
3939365995046,3cyclictest3937787-21kworker/u8:3+flush-179:019:15:161
3939365995043,6cyclictest4008561-21kworker/u8:2+flush-179:023:00:211
3939365995039,9cyclictest4041247-21apt-get00:25:121
3939365994943,4cyclictest4016463-21kworker/u8:0+events_unbound23:20:241
3939365994942,5cyclictest3979563-21kworker/u8:1+flush-179:021:40:001
3939364994940,7cyclictest3948835-21kworker/u8:3+flush-179:019:50:230
3939365994843,4cyclictest3990459-21kworker/u8:1+events_unbound22:10:141
3939365994840,5cyclictest3918648-21kworker/u8:2+events_unbound19:30:251
3939365994742,4cyclictest4035996-21kworker/u8:2+flush-179:000:20:231
3939365994742,4cyclictest4035996-21kworker/u8:2+flush-179:000:20:221
3939365994736,9cyclictest4035996-21kworker/u8:2+events_unbound00:15:021
3939364994743,2cyclictest4038031-21kworker/u8:0+events_unbound00:15:220
3939364994740,5cyclictest4035996-21kworker/u8:2+flush-179:000:20:190
3939364994740,5cyclictest4035996-21kworker/u8:2+flush-179:000:20:180
393936499471,43cyclictest4027930-21apt-get23:45:120
3939365994640,4cyclictest3954609-21kworker/u8:1+flush-179:020:04:561
3939365994638,6cyclictest3979563-21kworker/u8:1+events_unbound21:30:261
3939364994639,5cyclictest4038031-21kworker/u8:0+flush-179:000:25:220
3939364994638,7cyclictest266-21systemd-journal19:29:590
3939365994540,3cyclictest4006893-21sort22:40:011
3939365994538,5cyclictest3956822-21kworker/u8:3+events_unbound20:05:211
3939365994534,9cyclictest4000844-21diskstats22:20:171
3939365994437,5cyclictest4031427-21kworker/u8:1+flush-179:000:30:181
3939365994436,6cyclictest4016463-21kworker/u8:0+events_unbound23:45:151
3939364994439,3cyclictest266-21systemd-journal00:34:560
3939364994428,14cyclictest510-21rs:main0
393936499440,1cyclictest0-21swapper/021:50:150
3939365994339,3cyclictest4006116-21kworker/u8:4+flush-179:022:50:271
3939365994337,5cyclictest3963291-21kworker/u8:1+flush-179:020:30:001
3939365994337,4cyclictest3996847-21latency_hist22:10:011
3939365994336,4cyclictest3963291-21kworker/u8:1+flush-179:020:30:181
3939364994340,2cyclictest714-21Xorg19:25:000
3939364994340,2cyclictest714-21Xorg19:24:590
3939364994326,15cyclictest4022926-21systemd-run23:30:000
393936499430,40cyclictest3998688-21apt-get22:15:120
393937099420,40cyclictest3956797-21turbostat20:05:003
3939365994234,5cyclictest0-21swapper/119:12:581
3939364994236,5cyclictest3990514-21kworker/u8:2+events_unbound22:30:250
3939364994226,14cyclictest3979490-21latency_hist21:15:010
393936499420,40cyclictest4024892-21cpuspeed_turbos23:35:140
3939365994135,4cyclictest4008561-21kworker/u8:2+flush-179:023:28:191
3939365994134,5cyclictest3965974-21kworker/u8:0+flush-179:021:10:161
393936599411,38cyclictest271rcuc/100:09:591
3939364994136,4cyclictest3948835-21kworker/u8:3+flush-179:019:55:160
3939364994135,5cyclictest4029270-21kworker/u8:3+flush-179:023:50:190
3939364994135,4cyclictest3965974-21kworker/u8:0+flush-179:021:30:170
3939364994119,20cyclictest266-21systemd-journal22:00:170
3939365994036,2cyclictest714-21Xorg23:57:591
3939365994033,5cyclictest4001874-21kworker/u8:0+events_unbound22:40:151
3939365994032,6cyclictest510-21rs:main1
393936499401,37cyclictest714-21Xorg21:17:590
393936499401,37cyclictest714-21Xorg21:08:590
393936499400,39cyclictest3947232-21turbostat19:39:560
393936499400,38cyclictest3952000-21cron19:49:590
393936499400,38cyclictest3952000-21cron19:49:590
393936999390,36cyclictest0-21swapper/221:23:592
393936999390,2cyclictest0-21swapper/223:39:552
3939365993933,4cyclictest3967262-21gltestperf20:35:171
393936599390,2cyclictest714-21Xorg20:40:591
3939364993933,4cyclictest4001874-21kworker/u8:0+flush-179:022:45:210
3939364993921,17cyclictest4011447-21timerandwakeup22:50:250
3939364993919,19cyclictest3972963-21/usr/sbin/munin20:55:010
3939364993919,19cyclictest3972963-21/usr/sbin/munin20:55:010
393936499390,3cyclictest714-21Xorg23:42:590
3939370993832,5cyclictest3942452-21turbostat19:20:003
393937099380,2cyclictest0-21swapper/320:32:593
393937099380,1cyclictest0-21swapper/319:29:593
393937099380,1cyclictest0-21swapper/300:19:583
393936999380,36cyclictest0-21swapper/220:05:002
393936599382,32cyclictest4018315-21sendmail-mta23:15:161
393937099370,35cyclictest0-21swapper/321:00:593
3939364993733,3cyclictest3980982-21kworker/u8:3+flush-179:021:35:260
3939364993732,4cyclictest4001922-21kworker/u8:3+events_unbound22:25:140
3939364993732,4cyclictest151ktimers/022:05:130
3939364993726,9cyclictest3962370-21diskstats20:20:160
393936499371,34cyclictest908065-21snmpd20:15:010
393937099360,35cyclictest0-21swapper/321:23:593
393937099360,35cyclictest0-21swapper/319:38:003
3939369993633,2cyclictest0-21swapper/219:16:592
3939369993632,2cyclictest0-21swapper/221:33:592
393936999360,2cyclictest0-21swapper/220:32:592
393936999360,1cyclictest0-21swapper/219:30:002
3939365993631,4cyclictest3973553-21kworker/u8:1+flush-179:021:00:211
3939365993630,4cyclictest4012576-21missed_timers22:55:191
3939364993633,2cyclictest714-21Xorg23:25:000
3939364993632,3cyclictest714-21Xorg23:32:000
3939364993629,6cyclictest3998443-21dump-pmu-power22:15:000
3939364993616,19cyclictest0-21swapper/019:40:150
3939364993614,3cyclictest0-21swapper/021:25:150
4577091350,7ptp4l61490irq/134-eno1-TxRx-022:59:593
3939370993533,1cyclictest0-21swapper/321:51:003
393937099350,1cyclictest0-21swapper/323:39:563
3939369993532,2cyclictest0-21swapper/223:44:002
3939369993532,2cyclictest0-21swapper/221:51:002
3939369993531,2cyclictest0-21swapper/200:29:002
393936999350,1cyclictest0-21swapper/200:19:592
3939365993529,4cyclictest4011722-21kworker/u8:1+flush-179:023:10:001
3939364993530,4cyclictest3963291-21kworker/u8:1+flush-179:020:25:210
393936499352,30cyclictest4034618-21apt-get00:05:140
3939364993513,3cyclictest0-21swapper/020:41:560
3939370993432,1cyclictest0-21swapper/321:34:003
3939370993431,2cyclictest0-21swapper/300:28:593
3939370993428,5cyclictest4024500-21turbostat23:35:003
3939370993331,1cyclictest0-21swapper/323:43:593
3939369993329,2cyclictest0-21swapper/223:13:002
393936999330,30cyclictest0-21swapper/221:12:592
3939364993328,4cyclictest3945649-21turbostat19:34:550
3939364993327,5cyclictest3974555-21sh21:00:000
3939364993318,13cyclictest714-21Xorg22:22:590
3939370993229,2cyclictest0-21swapper/323:12:593
3939369993229,2cyclictest0-21swapper/220:41:592
3939369993229,2cyclictest0-21swapper/219:13:592
3939369993228,2cyclictest0-21swapper/221:38:592
3939369993228,2cyclictest0-21swapper/219:38:592
393936499321,29cyclictest4015139-21apt-get23:05:130
3939370993128,2cyclictest0-21swapper/320:41:593
393937099310,30cyclictest0-21swapper/321:25:123
393937099310,30cyclictest0-21swapper/321:12:593
393937099310,28cyclictest0-21swapper/319:40:123
3939369993128,2cyclictest0-21swapper/221:15:142
3939369993126,2cyclictest0-21swapper/222:04:002
393936999310,2cyclictest0-21swapper/220:54:552
393936999310,2cyclictest0-21swapper/220:54:552
3939365993125,4cyclictest4004576-21phc2sys-jitter22:30:221
393936599310,30cyclictest714-21Xorg21:24:591
3939364993122,8cyclictest908065-21snmpd23:10:120
393936499310,2cyclictest4006661-21apt-get22:40:020
393936499310,24cyclictest0-21swapper/020:09:590
3939370993028,1cyclictest0-21swapper/321:38:593
3939370993026,2cyclictest4577091ptp4l19:10:163
3939370993026,2cyclictest0-21swapper/322:09:593
393937099300,28cyclictest0-21swapper/320:05:143
393937099300,28cyclictest0-21swapper/319:20:133
393937099300,28cyclictest0-21swapper/319:20:123
3939369993028,1cyclictest0-21swapper/223:35:002
3939369993027,2cyclictest0-21swapper/219:40:132
3939369993026,2cyclictest0-21swapper/223:29:562
393936999300,28cyclictest0-21swapper/222:20:012
393936999300,28cyclictest0-21swapper/221:25:132
393936999300,28cyclictest0-21swapper/221:00:132
393936999300,28cyclictest0-21swapper/220:55:132
393936999300,28cyclictest0-21swapper/220:15:032
393936999300,27cyclictest0-21swapper/222:40:022
3939364993027,2cyclictest714-21Xorg20:34:590
4577091290,6ptp4l441ktimers/322:15:003
4577091290,6ptp4l441ktimers/300:40:003
3939370992927,1cyclictest0-21swapper/323:29:553
3939370992927,1cyclictest0-21swapper/322:04:543
3939370992927,1cyclictest0-21swapper/321:15:143
3939370992926,2cyclictest0-21swapper/323:55:133
3939370992923,4cyclictest3955185-21turbostat20:00:003
393937099290,27cyclictest0-21swapper/321:40:133
393937099290,27cyclictest0-21swapper/320:55:133
393937099290,27cyclictest0-21swapper/300:15:023
393937099290,1cyclictest0-21swapper/320:15:023
3939369992926,2cyclictest0-21swapper/222:55:122
3939369992926,2cyclictest0-21swapper/219:54:592
3939369992926,2cyclictest0-21swapper/219:20:122
3939369992926,2cyclictest0-21swapper/219:20:122
3939369992924,3cyclictest0-21swapper/219:30:592
393936999290,27cyclictest0-21swapper/223:55:132
393936999290,27cyclictest0-21swapper/222:25:122
393936999290,27cyclictest0-21swapper/220:47:592
393936999290,27cyclictest0-21swapper/220:20:132
393936999290,27cyclictest0-21swapper/220:05:132
393936999290,27cyclictest0-21swapper/200:20:592
393936999290,27cyclictest0-21swapper/200:20:592
393936999290,27cyclictest0-21swapper/200:05:022
4577091280,7ptp4l441ktimers/322:19:593
4577091280,6ptp4l61490irq/134-eno1-TxRx-022:54:593
4577091280,5ptp4l441ktimers/320:50:013
3939370992825,2cyclictest0-21swapper/322:40:033
3939370992825,2cyclictest0-21swapper/322:25:133
3939370992825,2cyclictest0-21swapper/319:54:593
393937099280,26cyclictest0-21swapper/320:20:133
393937099280,26cyclictest0-21swapper/300:05:033
3939369992826,1cyclictest0-21swapper/222:09:582
3939369992826,1cyclictest0-21swapper/219:55:132
3939369992825,2cyclictest0-21swapper/200:10:592
3939369992824,2cyclictest0-21swapper/223:00:592
3939369992824,2cyclictest0-21swapper/222:20:022
3939369992824,2cyclictest0-21swapper/220:14:002
393936999280,26cyclictest0-21swapper/220:39:592
393936999280,26cyclictest0-21swapper/200:30:582
393936999280,25cyclictest0-21swapper/223:25:002
393936999280,1cyclictest0-21swapper/221:40:132
3939364992823,4cyclictest3975386-21latency21:00:180
4577091270,6ptp4l61490irq/134-eno1-TxRx-022:34:563
4577091270,5ptp4l61490irq/134-eno1-TxRx-023:04:553
4577091270,5ptp4l61490irq/134-eno1-TxRx-022:50:013
4577091270,5ptp4l441ktimers/323:50:003
4577091270,5ptp4l441ktimers/323:19:573
4577091270,5ptp4l441ktimers/322:39:543
4577091270,5ptp4l441ktimers/321:09:563
4577091270,5ptp4l441ktimers/320:54:563
4577091270,5ptp4l441ktimers/320:54:563
4577091270,5ptp4l441ktimers/320:29:563
4577091270,5ptp4l441ktimers/300:05:003
4577091270,4ptp4l441ktimers/321:55:013
3939370992725,1cyclictest0-21swapper/322:20:013
3939370992723,3cyclictest0-21swapper/320:39:593
393937099270,2cyclictest0-21swapper/300:30:583
3939369992723,2cyclictest0-21swapper/220:27:592
3939369992722,3cyclictest0-21swapper/222:51:592
393936999270,25cyclictest0-21swapper/221:59:592
393936999270,24cyclictest0-21swapper/223:18:582
393936999270,24cyclictest0-21swapper/223:05:142
393936999270,19cyclictest0-21swapper/222:10:132
4577091260,6ptp4l441ktimers/319:50:003
4577091260,6ptp4l441ktimers/319:50:003
4577091260,4ptp4l61490irq/134-eno1-TxRx-000:25:023
4577091260,4ptp4l61490irq/134-eno1-TxRx-000:25:013
4577091260,4ptp4l441ktimers/323:54:563
3939370992624,1cyclictest0-21swapper/319:30:593
3939369992623,2cyclictest0-21swapper/222:36:592
3939370992523,1cyclictest0-21swapper/320:13:593
393937099250,24cyclictest0-21swapper/323:25:003
393937099250,24cyclictest0-21swapper/323:05:143
3939369992522,2cyclictest0-21swapper/200:04:582
393936999240,1cyclictest0-21swapper/221:08:592
3939365992417,5cyclictest3939239-21latency_hist19:10:011
3939369992319,2cyclictest0-21swapper/200:38:592
3939369992218,2cyclictest0-21swapper/219:47:592
3939369992218,2cyclictest0-21swapper/219:47:592
231999220,13rtkit-daemon3203871-21ntpd19:08:430
3939369992118,2cyclictest0-21swapper/223:48:582
3939369992117,2cyclictest0-21swapper/222:34:002
3939369991815,2cyclictest0-21swapper/222:49:002
4577091170,3ptp4l61490irq/134-eno1-TxRx-021:45:013
3939369991512,2cyclictest0-21swapper/221:48:002
393936999107,2cyclictest0-21swapper/223:53:592
39393699940,2cyclictest0-21swapper/219:10:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional