You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-03 - 06:15
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot6.osadl.org (updated Sat Jan 03, 2026 00:44:53)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2016361999689,4cyclictest2022649-21kworker/u8:2+flush-179:019:50:000
2016361999689,4cyclictest2022649-21kworker/u8:2+flush-179:019:50:000
2016361999484,8cyclictest2036979-21latency_hist20:15:000
2016361999484,8cyclictest2036979-21latency_hist20:15:000
2016364999080,7cyclictest2062976-21latency_hist21:35:011
2016364999080,7cyclictest2062976-21latency_hist21:35:011
2016364998983,4cyclictest2098309-21kworker/u8:3+flush-179:000:04:011
2016361998880,5cyclictest2066203-21true21:45:000
2016364997971,5cyclictest2010749-21kworker/u8:1+flush-179:019:10:261
2016364997767,8cyclictest2056843-21/usr/sbin/munin21:15:251
2016364997767,7cyclictest2108054-21/usr/sbin/munin23:55:221
2016361997768,6cyclictest2055018-21latency_hist21:10:010
2016361997768,6cyclictest2055018-21latency_hist21:10:010
2016364997668,5cyclictest2022658-21latency_hist19:30:001
2016361997261,9cyclictest2117985-21diskstats00:25:140
2016364996960,6cyclictest2072539-21latency_hist22:05:001
2016364996757,8cyclictest2050213-21/usr/sbin/munin20:55:251
2016364996660,4cyclictest2109468-21kworker/u8:0+flush-179:000:10:011
2016361996640,18cyclictest2109429-40mandb00:00:000
2016361996555,8cyclictest2074379-21apt-get22:10:090
2016364996455,6cyclictest2075712-21latency_hist22:15:011
2016364996356,4cyclictest2039085-21kworker/u8:2+flush-179:021:00:161
2016364996355,5cyclictest2022649-21kworker/u8:2+events_unbound19:35:241
2016364996156,4cyclictest2039085-21kworker/u8:2+flush-179:020:30:241
2016364996153,6cyclictest2048390-21latency_hist20:50:011
201636499610,60cyclictest714-21Xorg00:24:581
2016361996050,7cyclictest2099861-21latency_hist23:30:010
2016364995954,4cyclictest2082061-21kworker/u8:2+flush-179:022:55:221
2016364995954,2cyclictest2082061-21kworker/u8:2+events_unbound23:05:251
2016364995747,8cyclictest2040690-21apt-get20:25:141
2016361995746,8cyclictest2061625-21/usr/sbin/munin21:30:240
2016361995746,8cyclictest2061625-21/usr/sbin/munin21:30:230
2016364995553,1cyclictest0-21swapper/121:40:001
2016364995546,7cyclictest2082321-21/usr/sbin/munin22:35:231
2016364995546,7cyclictest2053670-21/usr/sbin/munin21:05:261
2016364995546,7cyclictest2053670-21/usr/sbin/munin21:05:251
2016361995548,5cyclictest2039085-21kworker/u8:2+flush-179:020:55:260
201636199550,52cyclictest2077319-21apt22:20:010
2016364995348,4cyclictest2029072-21kworker/u8:0+events_unbound19:55:241
2016361995345,7cyclictest2063231-21apt-get21:35:120
2016361995344,6cyclictest2093492-21latency_hist23:10:020
201636199532,47cyclictest2112909-21apt-get00:10:150
2016364995245,5cyclictest2098309-21kworker/u8:3+flush-179:023:55:001
2016364995245,5cyclictest2022649-21kworker/u8:2+flush-179:019:50:211
2016364995243,7cyclictest2022696-21kworker/u8:3+flush-179:019:45:011
2016361995247,4cyclictest2098309-21kworker/u8:3+flush-179:000:30:260
2016364995143,5cyclictest2098309-21kworker/u8:3+flush-179:023:40:161
2016361995141,8cyclictest266-21systemd-journal23:20:000
2016361995138,11cyclictest266-21systemd-journal19:40:010
2016364995041,7cyclictest2104880-21/usr/sbin/munin23:45:241
2016364995040,7cyclictest2045454-21kworker/u8:0+flush-179:020:40:191
201636199501,45cyclictest0-21swapper/000:05:140
4577091490,14ptp4l61490irq/134-eno1-TxRx-019:10:013
4577091490,14ptp4l61490irq/134-eno1-TxRx-019:10:003
2016364994944,4cyclictest2045454-21kworker/u8:0+flush-179:020:54:591
2016364994943,4cyclictest2010749-21kworker/u8:1+flush-179:019:15:161
2016364994938,8cyclictest2022649-21kworker/u8:2+events_unbound20:05:201
2016361994939,8cyclictest2098276-21latency_hist23:25:000
2016369994846,1cyclictest0-21swapper/320:03:593
2016364994840,6cyclictest2101434-21kworker/u8:1+flush-179:023:40:001
2016364994840,6cyclictest2075692-21kworker/u8:0+flush-179:022:25:161
2016364994839,6cyclictest2010749-21kworker/u8:1+events_unbound19:20:191
2016364994838,7cyclictest2069377-21kworker/u8:2+events_unbound21:55:181
2016361994841,5cyclictest2056609-21latency_hist21:15:000
201636199480,47cyclictest2016815-21expr19:10:160
2016364994744,2cyclictest2098309-21kworker/u8:3+events_unbound23:25:231
2016361994742,3cyclictest2058188-21cron21:20:000
2016361994739,6cyclictest2039085-21kworker/u8:2+flush-179:020:50:230
2016361994733,12cyclictest714-21Xorg22:32:590
201636199470,45cyclictest2069357-21idleruntime-cro21:55:010
2016364994640,4cyclictest2022649-21kworker/u8:2+flush-179:020:05:001
2016364994637,7cyclictest2036965-21kworker/u8:0+events_unbound20:15:231
2016364994637,5cyclictest0-21swapper/100:36:001
2016361994641,4cyclictest2026110-21apt-get19:40:120
2016361994638,6cyclictest266-21systemd-journal23:35:010
2016361994634,10cyclictest2091649-21apt-get23:05:000
2016361994627,17cyclictest2085289-21apt22:45:000
201636199461,43cyclictest2033815-21munin-run20:05:000
2016364994540,3cyclictest2069380-21systemd-run21:55:011
2016361994537,7cyclictest2042036-21perf20:30:000
2016361994536,7cyclictest2010749-21kworker/u8:1+flush-179:019:25:260
2016361994534,9cyclictest2083696-21sendmail-msp22:40:000
2016361994531,12cyclictest2035557-21latency_hist20:10:010
201636199451,40cyclictest0-21swapper/023:40:120
201636199450,1cyclictest0-21swapper/021:28:000
2016364994440,3cyclictest2098309-21kworker/u8:3+flush-179:000:30:301
2016364994439,4cyclictest2063011-21kworker/u8:1+flush-179:021:43:301
2016364994438,4cyclictest2063011-21kworker/u8:1+flush-179:022:20:181
2016364994437,5cyclictest2081181-21kworker/u8:1+events_unbound22:50:221
2016364994437,5cyclictest2069377-21kworker/u8:2+flush-179:022:05:221
2016361994438,4cyclictest2053434-21latency_hist21:05:010
201636199440,2cyclictest714-21Xorg19:51:590
2016364994336,5cyclictest2092977-21kworker/u8:1+events_unbound23:10:181
2016364994333,7cyclictest2055054-21kworker/u8:3+events_unbound21:45:201
2016361994334,7cyclictest2045212-21latency_hist20:40:010
2016364994237,4cyclictest2043583-21kworker/u8:1+flush-179:020:35:191
2016361994234,6cyclictest2117443-21latency_hist00:25:010
201636199420,40cyclictest2090047-21ls22:59:590
201636199420,28cyclictest714-21Xorg22:51:000
2016369994139,1cyclictest0-21swapper/323:18:593
201636999410,40cyclictest0-21swapper/321:58:593
2016364994135,4cyclictest2114243-21latency_hist00:15:001
2016369994038,1cyclictest0-21swapper/320:54:553
201636999400,5cyclictest2106236-21turbostat23:50:003
2016365994037,2cyclictest0-21swapper/200:18:592
2016364994033,5cyclictest2022696-21kworker/u8:3+flush-179:019:30:181
2016364994032,6cyclictest2061388-21latency_hist21:30:001
2016364994031,6cyclictest2115889-21kworker/u8:1+events_unbound00:25:201
2016361994035,4cyclictest2032212-21munin-run20:00:000
201636199401,3cyclictest2080498-21munin-run22:30:000
2016369993933,4cyclictest2119031-21turbostat00:30:013
201636999390,38cyclictest0-21swapper/323:07:003
2016364993935,3cyclictest2081181-21kworker/u8:1+flush-179:022:40:021
2016364993933,5cyclictest2082061-21kworker/u8:2+events_unbound23:20:201
2016364993933,4cyclictest2040436-21latency_hist20:25:011
2016364993933,4cyclictest2022696-21kworker/u8:3+flush-179:019:45:201
2016364993933,4cyclictest2022696-21kworker/u8:3+flush-179:019:45:201
2016364993928,10cyclictest2101457-21idleruntime-cro23:35:001
2016361993933,3cyclictest2059798-21cron21:25:000
2016361993927,10cyclictest2093746-21apt-get23:10:130
2016365993834,2cyclictest0-21swapper/221:34:592
2016365993834,2cyclictest0-21swapper/221:34:582
201636599380,36cyclictest0-21swapper/222:15:592
201636599380,36cyclictest0-21swapper/220:58:592
201636599380,35cyclictest0-21swapper/223:20:002
201636599380,35cyclictest0-21swapper/221:46:592
201636599380,35cyclictest0-21swapper/220:49:592
201636599380,30cyclictest0-21swapper/220:11:592
201636599380,30cyclictest0-21swapper/220:11:592
2016361993833,3cyclictest2024254-21kworker/u8:0+flush-179:019:35:010
2016365993734,2cyclictest0-21swapper/221:05:592
2016365993734,2cyclictest0-21swapper/221:05:582
201636599370,1cyclictest0-21swapper/220:16:592
201636499372,30cyclictest2080750-21apt-get22:30:121
2016361993734,2cyclictest0-21swapper/023:37:590
2016361993731,4cyclictest266-21systemd-journal20:25:000
2016361993720,16cyclictest0-21swapper/022:01:580
2016369993634,1cyclictest0-21swapper/322:28:583
2016369993634,1cyclictest0-21swapper/320:47:593
2016369993633,2cyclictest0-21swapper/323:35:593
2016369993633,2cyclictest0-21swapper/322:20:593
201636999360,35cyclictest0-21swapper/320:14:003
201636999360,35cyclictest0-21swapper/320:14:003
201636999360,1cyclictest0-21swapper/319:47:593
201636999360,1cyclictest0-21swapper/319:47:593
2016365993633,2cyclictest0-21swapper/223:53:592
2016365993633,2cyclictest0-21swapper/220:00:002
2016365993633,2cyclictest0-21swapper/200:07:592
2016364993633,2cyclictest0-21swapper/121:12:001
2016361993622,13cyclictest714-21Xorg20:43:000
2016369993533,1cyclictest0-21swapper/320:43:593
201636999350,34cyclictest0-21swapper/322:45:583
201636999350,1cyclictest0-21swapper/300:24:553
2016365993532,2cyclictest0-21swapper/223:47:592
2016365993532,2cyclictest0-21swapper/222:04:592
201636599350,33cyclictest0-21swapper/220:01:592
201636599350,1cyclictest0-21swapper/223:05:002
2016364993530,4cyclictest2076674-21needreboot22:15:181
2016364993528,5cyclictest2109468-21kworker/u8:0+flush-179:000:15:241
2016364993528,4cyclictest2085533-21apt-get22:45:131
2016364993526,6cyclictest2016258-21latency_hist19:10:011
2016364993526,6cyclictest2016258-21latency_hist19:10:011
2016361993532,2cyclictest0-21swapper/020:32:590
2016361993531,3cyclictest714-21Xorg00:18:590
2016361993523,10cyclictest0-21swapper/022:50:010
201636199351,30cyclictest2018128-21apt-get19:15:150
201636199351,20cyclictest2121892-21wc00:35:230
2016369993432,1cyclictest0-21swapper/323:54:543
2016369993432,1cyclictest0-21swapper/323:12:593
2016369993432,1cyclictest0-21swapper/321:39:553
2016369993432,1cyclictest0-21swapper/321:27:003
2016364993425,8cyclictest2058468-21apt-get21:20:121
201636499340,32cyclictest2095337-21http23:15:141
201636199346,27cyclictest2020368-21missed_timers19:20:220
2016361993431,2cyclictest0-21swapper/022:24:590
2016361993430,3cyclictest2067795-21systemd21:50:000
2016361993429,3cyclictest501-21systemd-logind00:04:000
201636199340,33cyclictest714-21Xorg22:08:590
2016369993331,1cyclictest0-21swapper/323:57:583
2016369993328,4cyclictest2025848-21turbostat19:44:553
2016365993330,2cyclictest0-21swapper/222:25:592
2016365993329,2cyclictest0-21swapper/220:35:592
2016361993329,3cyclictest2106924-21hwlatdetect23:50:160
2016361993227,4cyclictest2071114-21latency_hist22:00:020
2016369993129,1cyclictest0-21swapper/322:44:583
2016369993128,2cyclictest0-21swapper/323:35:003
2016369993128,2cyclictest0-21swapper/300:31:593
201636599310,29cyclictest0-21swapper/219:20:162
201636599310,29cyclictest0-21swapper/200:30:182
2016361993126,3cyclictest2106261-21systemd-run23:50:010
2016361993121,8cyclictest266-21systemd-journal20:20:000
201636199311,29cyclictest0-21swapper/020:45:140
2016369993028,1cyclictest0-21swapper/321:44:593
201636999300,1cyclictest0-21swapper/321:07:593
201636999300,1cyclictest0-21swapper/321:07:593
2016365993027,2cyclictest0-21swapper/223:29:582
2016365993027,2cyclictest0-21swapper/220:43:002
4577091290,7ptp4l441ktimers/321:25:013
4577091290,6ptp4l441ktimers/321:55:013
2016369992927,1cyclictest0-21swapper/300:39:553
2016369992926,2cyclictest0-21swapper/320:30:153
201636999290,28cyclictest0-21swapper/319:10:143
201636999290,27cyclictest0-21swapper/323:25:143
2016365992926,2cyclictest0-21swapper/222:56:002
2016365992926,2cyclictest0-21swapper/221:42:002
201636599290,2cyclictest0-21swapper/221:00:022
201636599290,28cyclictest0-21swapper/221:20:122
201636599290,28cyclictest0-21swapper/200:35:132
201636599290,27cyclictest0-21swapper/222:10:102
201636599290,27cyclictest0-21swapper/220:30:592
201636599290,27cyclictest0-21swapper/219:10:142
201636599290,1cyclictest0-21swapper/223:35:152
4577091280,7ptp4l441ktimers/322:20:013
4577091280,7ptp4l441ktimers/322:04:563
4577091280,6ptp4l441ktimers/321:00:003
4577091280,6ptp4l441ktimers/300:15:003
2016369992826,1cyclictest0-21swapper/322:37:003
2016369992826,1cyclictest0-21swapper/320:22:593
2016369992825,2cyclictest0-21swapper/321:15:143
201636999280,26cyclictest0-21swapper/320:15:163
201636999280,26cyclictest0-21swapper/320:05:143
2016365992826,1cyclictest0-21swapper/200:12:592
2016365992825,2cyclictest0-21swapper/223:07:592
2016365992825,2cyclictest0-21swapper/222:40:142
2016365992825,2cyclictest0-21swapper/220:20:012
2016365992825,2cyclictest0-21swapper/219:35:132
201636599280,2cyclictest0-21swapper/221:55:132
201636599280,26cyclictest0-21swapper/222:20:152
201636599280,21cyclictest0-21swapper/223:20:142
2016364992823,3cyclictest2092021-21latency_hist23:05:011
4577091270,6ptp4l441ktimers/323:05:003
4577091270,6ptp4l441ktimers/321:50:003
4577091270,6ptp4l441ktimers/319:55:003
4577091270,6ptp4l441ktimers/319:34:553
4577091270,6ptp4l441ktimers/319:30:003
2016369992726,1cyclictest0-21swapper/321:30:593
2016369992726,1cyclictest0-21swapper/321:30:583
2016369992725,1cyclictest0-21swapper/323:43:583
2016369992725,1cyclictest0-21swapper/322:35:003
2016369992725,1cyclictest0-21swapper/319:55:133
2016369992724,2cyclictest0-21swapper/300:16:593
201636999270,26cyclictest0-21swapper/322:55:133
2016365992725,1cyclictest0-21swapper/219:45:592
2016365992725,1cyclictest0-21swapper/219:45:592
2016365992724,2cyclictest0-21swapper/223:40:582
2016365992724,2cyclictest0-21swapper/221:36:592
201636599270,25cyclictest0-21swapper/219:53:592
201636599270,1cyclictest0-21swapper/221:26:002
201636599270,19cyclictest0-21swapper/220:26:592
4577091260,6ptp4l61490irq/134-eno1-TxRx-021:14:593
4577091260,5ptp4l61490irq/134-eno1-TxRx-022:15:013
4577091260,5ptp4l441ktimers/321:04:553
4577091260,5ptp4l441ktimers/320:39:563
4577091260,5ptp4l441ktimers/319:24:563
4577091260,5ptp4l441ktimers/300:05:013
201636999260,2cyclictest0-21swapper/319:35:013
201636999260,24cyclictest0-21swapper/319:15:213
201636999260,1cyclictest0-21swapper/322:53:003
2016365992623,2cyclictest0-21swapper/220:50:592
201636599260,24cyclictest0-21swapper/221:54:492
201636599260,24cyclictest0-21swapper/219:41:092
2016364992621,4cyclictest2036212-21latency20:10:201
2016364992621,4cyclictest2036212-21latency20:10:201
4577091250,5ptp4l61490irq/134-eno1-TxRx-000:09:573
4577091250,4ptp4l441ktimers/323:24:543
2016365992523,1cyclictest0-21swapper/220:06:592
2016365992522,2cyclictest0-21swapper/219:30:012
201636199250,22cyclictest2016258-21latency_hist19:10:010
201636199250,22cyclictest2016258-21latency_hist19:10:010
4577091240,4ptp4l61490irq/134-eno1-TxRx-020:30:013
4577091240,4ptp4l441ktimers/322:09:553
2016365992422,1cyclictest0-21swapper/221:12:002
2016365992421,2cyclictest0-21swapper/223:58:592
2016365992421,2cyclictest0-21swapper/200:20:592
2016365992321,1cyclictest0-21swapper/219:17:592
2016365992320,2cyclictest0-21swapper/200:04:592
2016365992219,2cyclictest0-21swapper/200:25:582
2016365992017,2cyclictest0-21swapper/222:40:002
2016365991816,1cyclictest0-21swapper/219:28:002
2016365991815,2cyclictest0-21swapper/222:45:582
2016365991613,2cyclictest0-21swapper/223:31:592
2016365991512,2cyclictest0-21swapper/221:19:592
2016365991411,2cyclictest0-21swapper/223:10:122
20163659996,2cyclictest0-21swapper/222:54:222
20163659996,2cyclictest0-21swapper/222:07:522
20163659985,2cyclictest0-21swapper/222:34:542
20163659953,1cyclictest0-21swapper/219:10:022
20163659953,1cyclictest0-21swapper/219:10:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional