You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-31 - 09:03
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot6.osadl.org (updated Wed Dec 31, 2025 00:44:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1963842999081,7cyclictest1986604-21latency_hist20:20:001
1963842999080,7cyclictest2039642-21/usr/sbin/munin23:05:251
1963842999080,7cyclictest2039642-21/usr/sbin/munin23:05:251
1963841998572,10cyclictest2059434-21gpgv00:05:010
1963842998477,5cyclictest2042028-21kworker/u8:0+flush-179:023:15:001
1963841997563,9cyclictest1972028-21kworker/u8:3+flush-179:020:00:010
1963842997468,4cyclictest2004599-21kworker/u8:0+flush-179:021:30:261
1963842997063,5cyclictest1988241-21kworker/u8:2+flush-179:020:35:001
1963842996864,3cyclictest2047654-21kworker/u8:0+events_unbound23:55:231
1963842996860,5cyclictest2049444-21/usr/sbin/munin23:35:251
1963841996860,5cyclictest1979935-21kworker/u8:1+flush-179:020:15:010
1963841996858,8cyclictest2004600-21/usr/sbin/munin21:15:240
1963841996858,8cyclictest2004600-21/usr/sbin/munin21:15:230
1963841996758,7cyclictest1970672-21apt-get19:30:130
1963842996658,5cyclictest2023727-21kworker/u8:0+flush-179:022:20:211
1963842996657,7cyclictest2010382-21kworker/u8:2+flush-179:021:40:221
1963842996556,7cyclictest1986844-21/usr/sbin/munin20:20:221
1963842996457,5cyclictest2047654-21kworker/u8:0+events_unbound00:04:011
1963849996361,1cyclictest0-21swapper/319:20:593
1963841996355,6cyclictest1988197-21latency_hist20:25:020
1963842996258,3cyclictest2031419-21kworker/u8:1+flush-179:023:40:171
1963842996256,4cyclictest1944780-21kworker/u8:0+flush-179:019:10:231
1963841996258,3cyclictest1979935-21kworker/u8:1+events_unbound20:16:410
1963842996054,4cyclictest2061021-21kworker/u8:2+flush-179:000:25:241
1963841996052,6cyclictest2068947-21latency_hist00:35:010
1963842995956,2cyclictest0-21swapper/121:56:581
1963842995951,5cyclictest1993012-21kworker/u8:0+flush-179:020:55:191
1963842995852,5cyclictest2023727-21kworker/u8:0+flush-179:022:30:231
1963841995851,5cyclictest2047654-21kworker/u8:0+flush-179:023:40:000
1963849995750,5cyclictest1973598-21turbostat19:40:003
1963842995753,3cyclictest1965353-21kworker/u8:2+flush-179:019:20:261
196384199571,52cyclictest2061018-21true00:10:000
1963845995653,2cyclictest0-21swapper/219:25:592
1963842995649,5cyclictest2047654-21kworker/u8:0+flush-179:000:35:171
1963842995550,4cyclictest2010382-21kworker/u8:2+events_unbound22:05:001
1963842995548,5cyclictest1988241-21kworker/u8:2+flush-179:021:10:221
1963841995545,8cyclictest1966990-21latency_hist19:20:010
1963842995346,5cyclictest2064191-21latency_hist00:20:011
1963842995246,5cyclictest1972028-21kworker/u8:3+events_unbound20:45:291
1963842995244,6cyclictest1972028-21kworker/u8:3+flush-179:020:10:231
1963842995243,7cyclictest2031688-21/usr/sbin/munin22:40:251
1963842995243,7cyclictest2007017-21kworker/u8:1+flush-179:021:30:001
1963841995245,4cyclictest2020297-21latency_hist22:05:010
1963842995146,4cyclictest2010382-21kworker/u8:2+flush-179:021:50:011
1963842995146,4cyclictest2010382-21kworker/u8:2+flush-179:021:50:001
1963841995142,6cyclictest2013931-21latency_hist21:45:020
1963842995044,5cyclictest2034878-21apt-get22:50:121
1963842995040,8cyclictest266-21systemd-journal20:45:011
1963849994942,5cyclictest2046047-21turbostat23:25:003
1963842994943,4cyclictest1967479-21kworker/u8:1+flush-179:019:25:221
1963842994942,5cyclictest2031440-21latency_hist22:40:011
1963842994940,7cyclictest266-21systemd-journal21:54:551
1963842994940,6cyclictest2042028-21kworker/u8:0+events_unbound23:15:011
4577091480,13ptp4l61490irq/134-eno1-TxRx-019:10:013
1963842994844,3cyclictest2023727-21kworker/u8:0+flush-179:022:29:561
1963842994842,4cyclictest2044400-21kworker/u8:2+events_unbound23:25:201
1963842994838,8cyclictest1967479-21kworker/u8:1+events_unbound19:30:211
1963841994835,11cyclictest2057474-40logrotate00:00:000
1963842994741,4cyclictest1972028-21kworker/u8:3+events_unbound20:05:231
1963842994740,5cyclictest2029831-21kworker/u8:2+events_unbound23:00:201
1963841994743,3cyclictest2000964-21kworker/u8:1+flush-179:021:10:170
1963842994641,4cyclictest2044400-21kworker/u8:2+flush-179:023:30:151
1963841994641,4cyclictest2061021-21kworker/u8:2+flush-179:000:15:260
1963841994638,6cyclictest2049206-21latency_hist23:35:010
1963841994638,6cyclictest2015513-21latency_hist21:50:000
1963841994638,6cyclictest2015513-21latency_hist21:50:000
196384199460,5cyclictest2044438-21turbostat23:24:550
1963842994541,3cyclictest2061021-21kworker/u8:2+flush-179:000:20:241
1963842994537,6cyclictest1972028-21kworker/u8:3+flush-179:019:50:261
1963842994437,5cyclictest1988241-21kworker/u8:2+flush-179:021:20:011
1963842994437,5cyclictest1988241-21kworker/u8:2+flush-179:021:20:011
1963842994434,8cyclictest510-21rs:main1
1963841994437,5cyclictest1991370-21latency_hist20:35:010
1963841994432,9cyclictest2010382-21kworker/u8:2+flush-179:022:10:260
1963842994337,4cyclictest2007017-21kworker/u8:1+events_unbound22:15:171
1963842994337,4cyclictest1987272-21kworker/u8:0+events_unbound20:25:251
1963842994333,8cyclictest2054222-21/usr/sbin/munin23:50:231
196384199431,40cyclictest2065781-21grep00:25:000
1963842994236,4cyclictest2004599-21kworker/u8:0+flush-179:021:20:211
1963842994235,5cyclictest2007017-21kworker/u8:1+flush-179:021:40:011
1963842994233,7cyclictest2047654-21kworker/u8:0+events_unbound00:10:201
1963841994239,2cyclictest0-21swapper/022:41:580
1963841994234,6cyclictest2040990-21latency_hist23:10:010
1963841994234,6cyclictest2040990-21latency_hist23:10:000
1963841994232,7cyclictest266-21systemd-journal21:05:010
196384999410,40cyclictest0-21swapper/323:44:553
1963842994138,2cyclictest1972028-21kworker/u8:3+flush-179:019:35:161
1963841994135,4cyclictest2029858-21apt22:35:010
1963841994131,8cyclictest266-21systemd-journal20:59:590
196384199411,4cyclictest1981581-21ls20:05:000
196384199411,38cyclictest714-21Xorg21:26:580
1963842994030,8cyclictest2068967-21dump-pmu-power00:35:001
196384299401,37cyclictest714-21Xorg21:08:001
196384299401,2cyclictest714-21Xorg22:14:581
196384299400,38cyclictest2046048-21ls23:25:001
1963841994037,2cyclictest151ktimers/000:39:590
1963841994033,5cyclictest2038357-21diskstats23:00:160
1963841994032,5cyclictest1997734-21latency_hist20:55:010
1963841994032,5cyclictest1997734-21latency_hist20:55:010
196384999390,4cyclictest2046047-21turbostat23:29:563
1963842993935,3cyclictest0-21swapper/123:48:591
1963842993934,4cyclictest1979935-21kworker/u8:1+flush-179:020:00:231
196384299393,30cyclictest1978626-21apt-get19:55:161
1963841993934,3cyclictest1996152-21munin-run20:50:000
1963841993934,3cyclictest1983200-21apt-config20:09:590
1963841993932,5cyclictest1963746-21latency_hist19:10:020
1963841993931,6cyclictest2012338-21latency_hist21:40:020
196384199391,36cyclictest2044423-21sendmail23:20:000
196384199390,37cyclictest714-21Xorg19:42:000
1963849993832,5cyclictest2068962-21turbostat00:35:003
1963842993834,3cyclictest2031419-21kworker/u8:1+flush-179:022:55:211
1963841993835,2cyclictest1970430-21ls19:30:000
1963841993834,3cyclictest2035584-21needreboot22:50:210
1963841993827,9cyclictest2009689-21diskstats21:30:170
196384199380,37cyclictest714-21Xorg19:39:590
196384999370,36cyclictest0-21swapper/323:14:543
196384999370,36cyclictest0-21swapper/321:19:593
196384999370,36cyclictest0-21swapper/321:19:593
196384599370,35cyclictest0-21swapper/220:28:592
196384599370,35cyclictest0-21swapper/219:35:582
1963841993733,3cyclictest1994579-21basename20:45:000
1963841993731,4cyclictest2023727-21kworker/u8:0+flush-179:022:35:130
196384199371,34cyclictest1966998-21apt-get19:20:010
196384199370,1cyclictest0-21swapper/023:49:550
1963849993634,1cyclictest0-21swapper/321:00:583
196384999360,1cyclictest0-21swapper/320:16:593
1963842993630,3cyclictest1975455-21apt-get19:45:151
1963841993633,2cyclictest714-21Xorg21:51:000
1963841993633,2cyclictest0-21swapper/022:25:580
1963841993632,3cyclictest2042589-21grep23:15:000
1963841993628,6cyclictest1992980-21perf20:40:000
1963841993627,8cyclictest2025686-21fschecks_time22:20:150
1963849993533,1cyclictest0-21swapper/320:08:593
1963849993533,1cyclictest0-21swapper/300:07:593
1963849993532,2cyclictest0-21swapper/323:01:593
196384999350,1cyclictest0-21swapper/322:52:003
1963845993533,1cyclictest0-21swapper/200:29:592
196384599350,1cyclictest0-21swapper/222:05:002
196384299353,29cyclictest0-21swapper/100:05:031
1963842993530,3cyclictest484-21dbus-daemon22:10:021
1963842993529,4cyclictest1997772-21systemd-run20:55:001
1963842993529,4cyclictest1997772-21systemd-run20:54:591
1963842993526,7cyclictest1972028-21kworker/u8:3+flush-179:020:35:201
1963841993531,3cyclictest2067389-21systemd-run00:30:000
1963841993531,3cyclictest2047623-21munin-run23:30:000
1963841993528,5cyclictest1999598-21apt-get21:00:130
1963841993527,4cyclictest0-21swapper/023:44:590
1963849993432,1cyclictest0-21swapper/322:20:593
1963845993432,1cyclictest0-21swapper/221:09:592
1963845993431,2cyclictest0-21swapper/200:30:592
1963841993430,3cyclictest2031419-21kworker/u8:1+events_unbound22:57:590
1963841993430,3cyclictest1964483-21hwlatdetect19:10:180
1963841993426,6cyclictest1988433-21/usr/sbin/munin20:25:230
1963849993332,1cyclictest0-21swapper/321:40:003
1963849993331,1cyclictest0-21swapper/321:55:003
1963845993331,1cyclictest0-21swapper/222:09:592
1963845993330,2cyclictest0-21swapper/221:01:582
1963842993328,4cyclictest1972028-21kworker/u8:3+flush-179:019:40:251
1963849993230,1cyclictest0-21swapper/321:23:003
1963849993230,1cyclictest0-21swapper/319:14:003
1963849993230,1cyclictest0-21swapper/300:23:583
1963849993229,2cyclictest0-21swapper/320:23:003
1963845993229,2cyclictest0-21swapper/221:49:002
1963845993229,2cyclictest0-21swapper/221:48:592
1963845993229,2cyclictest0-21swapper/200:08:592
1963841993231,1cyclictest191rcuc/022:16:590
1963841993228,3cyclictest1972028-21kworker/u8:3+events_unbound19:54:000
1963841993222,8cyclictest2020551-21apt-get22:05:120
196384199321,5cyclictest740979-21munin-node23:50:140
1963845993128,2cyclictest0-21swapper/220:52:002
1963845993128,2cyclictest0-21swapper/220:51:592
1963845993128,2cyclictest0-21swapper/219:51:592
1963842993125,5cyclictest1993012-21kworker/u8:0+flush-179:021:00:291
1963841993127,3cyclictest1976787-21sh19:49:590
4577091300,7ptp4l441ktimers/320:40:013
1963849993028,1cyclictest0-21swapper/322:44:593
1963849993028,1cyclictest0-21swapper/319:44:593
1963849993027,2cyclictest0-21swapper/323:55:033
1963849993027,2cyclictest0-21swapper/322:55:143
1963845993028,1cyclictest0-21swapper/221:13:592
1963845993027,2cyclictest0-21swapper/221:39:552
1963845993027,2cyclictest0-21swapper/220:18:592
196384599300,28cyclictest0-21swapper/200:35:142
1963841993027,2cyclictest191rcuc/021:58:590
1963849992927,1cyclictest0-21swapper/322:28:593
1963849992926,2cyclictest0-21swapper/322:05:143
196384999290,28cyclictest0-21swapper/323:45:143
196384999290,27cyclictest0-21swapper/321:40:143
196384999290,27cyclictest0-21swapper/319:30:143
196384999290,26cyclictest0-21swapper/319:50:143
1963845992926,2cyclictest0-21swapper/223:55:012
1963845992926,2cyclictest0-21swapper/222:41:582
1963845992926,2cyclictest0-21swapper/219:20:032
1963845992926,2cyclictest0-21swapper/219:15:112
1963845992926,2cyclictest0-21swapper/200:17:592
196384599290,28cyclictest0-21swapper/223:15:022
196384599290,28cyclictest0-21swapper/222:45:142
196384599290,28cyclictest0-21swapper/200:10:132
196384599290,27cyclictest0-21swapper/222:30:122
196384599290,27cyclictest0-21swapper/221:20:132
196384599290,27cyclictest0-21swapper/220:10:012
196384599290,27cyclictest0-21swapper/219:10:162
4577091280,7ptp4l61490irq/134-eno1-TxRx-021:30:003
4577091280,7ptp4l441ktimers/320:45:013
4577091280,6ptp4l441ktimers/322:00:013
4577091280,6ptp4l441ktimers/321:15:013
4577091280,6ptp4l441ktimers/300:40:003
1963849992826,1cyclictest0-21swapper/322:49:543
1963849992826,1cyclictest0-21swapper/300:00:153
196384999280,27cyclictest0-21swapper/323:15:013
196384999280,27cyclictest0-21swapper/319:19:593
196384999280,26cyclictest0-21swapper/320:10:023
196384999280,26cyclictest0-21swapper/300:25:153
196384999280,26cyclictest0-21swapper/300:15:153
1963845992825,2cyclictest0-21swapper/223:25:132
1963845992825,2cyclictest0-21swapper/220:40:152
1963845992825,2cyclictest0-21swapper/219:45:142
196384599280,2cyclictest0-21swapper/223:55:022
196384599280,27cyclictest0-21swapper/222:50:142
196384599280,26cyclictest0-21swapper/222:35:132
196384599280,26cyclictest0-21swapper/222:25:132
196384599280,26cyclictest0-21swapper/220:00:142
1963842992817,8cyclictest510-21rs:main1
196384199280,2cyclictest714-21Xorg22:46:590
4577091270,6ptp4l441ktimers/323:10:013
4577091270,6ptp4l441ktimers/323:10:003
4577091270,6ptp4l441ktimers/322:39:563
4577091270,6ptp4l441ktimers/322:35:013
4577091270,6ptp4l441ktimers/322:20:003
4577091270,6ptp4l441ktimers/321:49:593
4577091270,6ptp4l441ktimers/321:49:593
4577091270,6ptp4l441ktimers/320:30:003
4577091270,6ptp4l441ktimers/320:05:013
4577091270,6ptp4l441ktimers/300:14:553
4577091270,5ptp4l441ktimers/320:54:563
4577091270,5ptp4l441ktimers/320:54:563
4577091270,5ptp4l441ktimers/319:50:003
1963849992725,1cyclictest0-21swapper/322:10:593
196384999270,26cyclictest0-21swapper/321:30:143
196384999270,26cyclictest0-21swapper/319:25:143
196384999270,1cyclictest0-21swapper/320:30:153
1963845992725,1cyclictest0-21swapper/223:40:142
1963845992725,1cyclictest0-21swapper/221:59:582
1963845992723,2cyclictest0-21swapper/220:20:542
196384599270,26cyclictest0-21swapper/220:10:002
196384599270,25cyclictest0-21swapper/221:34:082
1963841992723,3cyclictest2047654-21kworker/u8:0+events_unbound00:10:380
1963841992720,6cyclictest0-21swapper/021:20:140
4577091260,6ptp4l61490irq/134-eno1-TxRx-023:34:553
4577091260,6ptp4l61490irq/134-eno1-TxRx-019:59:563
4577091260,5ptp4l61490irq/134-eno1-TxRx-023:54:563
4577091260,5ptp4l441ktimers/323:39:563
4577091260,5ptp4l441ktimers/321:09:563
1963845992624,1cyclictest0-21swapper/223:35:142
1963845992623,2cyclictest0-21swapper/223:30:092
1963845992623,2cyclictest0-21swapper/219:41:402
1963845992622,2cyclictest0-21swapper/220:33:042
196384599260,24cyclictest0-21swapper/222:55:592
196384599260,24cyclictest0-21swapper/222:19:002
196384599260,24cyclictest0-21swapper/221:50:382
196384599260,24cyclictest0-21swapper/221:26:442
196384599260,24cyclictest0-21swapper/221:18:532
196384599260,24cyclictest0-21swapper/221:18:522
196384599260,24cyclictest0-21swapper/220:57:372
196384599260,24cyclictest0-21swapper/220:47:202
196384599260,24cyclictest0-21swapper/220:37:072
196384599260,24cyclictest0-21swapper/219:55:312
196384599260,18cyclictest0-21swapper/221:41:082
1963849992523,1cyclictest0-21swapper/320:59:223
1963849992523,1cyclictest0-21swapper/320:48:583
196384999250,24cyclictest0-21swapper/322:02:593
1963845992523,1cyclictest0-21swapper/222:12:582
1963845992522,2cyclictest0-21swapper/200:22:042
1963845992422,1cyclictest0-21swapper/222:23:242
1963845992421,2cyclictest0-21swapper/223:20:122
196384599240,1cyclictest0-21swapper/200:04:592
1963845992321,1cyclictest0-21swapper/223:49:592
1963845992320,2cyclictest0-21swapper/219:30:182
1963842992116,3cyclictest0-21swapper/119:10:021
1963845991815,2cyclictest0-21swapper/223:02:592
1963845991614,1cyclictest0-21swapper/223:12:002
19638459996,2cyclictest0-21swapper/223:09:072
19638459996,2cyclictest0-21swapper/223:09:072
19638459930,2cyclictest0-21swapper/219:10:022
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional