You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-22 - 12:14
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot6.osadl.org (updated Sun Mar 22, 2026 00:44:54)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
201360499154134,15cyclictest2108299-40logrotate00:00:000
201361299111105,4cyclictest2013393-21kworker/u8:1+events_unbound19:50:011
2013612999989,7cyclictest2117918-21latency_hist00:30:001
2013612999486,5cyclictest2031974-21latency_hist20:05:001
2013612998982,5cyclictest266-21systemd-journal23:40:011
2013604998875,10cyclictest2051082-21apt-get21:05:000
2013612998475,6cyclictest2080476-21latency_hist22:35:011
2013612998273,7cyclictest2038577-21/usr/sbin/munin20:25:231
2013612998270,9cyclictest2027431-21kworker/u8:0+events_unbound20:40:161
2013612998268,4cyclictest2100383-21kworker/u8:3+events_unbound00:10:241
2013612998173,5cyclictest2098500-21latency_hist23:30:011
2013612998170,9cyclictest2091877-21latency_hist23:10:011
2013604997971,6cyclictest2100383-21kworker/u8:3+events_unbound00:10:010
2013604997970,7cyclictest2030405-21latency_hist20:00:020
2013604997970,7cyclictest2030405-21latency_hist20:00:010
2013612997866,10cyclictest266-21systemd-journal22:20:001
2013612997667,6cyclictest2018725-21latency_hist19:25:011
2013612997569,4cyclictest2062693-21kworker/u8:2+flush-179:021:50:231
2013612997565,8cyclictest2056347-21latency_hist21:20:001
2013612997465,6cyclictest2105361-21/usr/sbin/munin23:50:231
2013612997366,5cyclictest2013393-21kworker/u8:1+flush-179:019:15:151
2013612997365,6cyclictest266-21systemd-journal23:00:001
2013604997369,3cyclictest2072242-21kworker/u8:0+flush-179:023:00:230
2013604997063,5cyclictest2074109-21latency_hist22:15:010
2013612996962,5cyclictest2013393-21kworker/u8:1+flush-179:019:50:021
2013612996856,8cyclictest2015272-21latency_hist19:15:021
2013612996655,8cyclictest2049493-21latency_hist21:00:001
2013612996457,4cyclictest2072242-21kworker/u8:0+flush-179:023:10:201
2013612996453,9cyclictest2041767-21apt-get20:35:121
2013604996458,4cyclictest2080152-21kworker/u8:2+flush-179:022:35:240
2013604996452,6cyclictest2088454-21latency_hist23:00:010
2013612996357,5cyclictest2018760-21kworker/u8:0+flush-179:019:35:231
2013612996357,4cyclictest2100383-21kworker/u8:3+flush-179:000:10:011
2013612996355,6cyclictest2121114-21latency_hist00:40:011
2013604996354,7cyclictest2076498-21kworker/u8:1+events_unbound22:25:000
2013612996256,5cyclictest2025042-21kworker/u8:2+flush-179:019:55:211
2013612996256,5cyclictest2025042-21kworker/u8:2+flush-179:019:55:211
2013604996256,5cyclictest2027431-21kworker/u8:0+events_unbound20:45:020
2013604996256,4cyclictest2027431-21kworker/u8:0+flush-179:020:20:180
2013604996254,6cyclictest2119512-21latency_hist00:35:020
2013604996152,6cyclictest2080152-21kworker/u8:2+flush-179:022:40:210
2013604996151,8cyclictest2025606-21apt-get19:45:110
2013612996055,3cyclictest2027431-21kworker/u8:0+flush-179:020:30:211
2013612996053,5cyclictest266-21systemd-journal00:00:481
201361299600,57cyclictest1-21systemd23:20:001
2013612995953,4cyclictest2067721-21kworker/u8:1+flush-179:021:55:231
2013612995953,4cyclictest2033814-21apt-get20:10:121
2013604995951,6cyclictest2098996-21kworker/u8:0+flush-179:023:30:250
2013612995853,4cyclictest2013393-21kworker/u8:1+flush-179:019:30:201
2013612995851,5cyclictest2044741-21kworker/u8:3+events_unbound21:15:011
2013612995851,5cyclictest2018760-21kworker/u8:0+flush-179:019:40:231
2013612995849,7cyclictest2072242-21kworker/u8:0+flush-179:022:30:011
2013604995852,5cyclictest1252047-21kworker/u8:2+events_unbound19:10:240
201361299571,53cyclictest1-21systemd22:55:001
201361299571,53cyclictest1-21systemd22:55:001
2013604995744,11cyclictest714-21Xorg00:21:590
2013612995651,4cyclictest2044696-21kworker/u8:2+flush-179:020:45:181
2013612995650,4cyclictest2072242-21kworker/u8:0+flush-179:022:15:011
2013604995646,8cyclictest2113139-21latency_hist00:15:010
2013612995549,4cyclictest2080152-21kworker/u8:2+flush-179:022:40:151
2013612995348,4cyclictest2044741-21kworker/u8:3+flush-179:021:25:221
2013612995348,4cyclictest2044696-21kworker/u8:2+events_unbound20:50:241
2013612995347,5cyclictest2111590-21kworker/u8:0+flush-179:000:15:151
2013604995348,4cyclictest2095289-21kworker/u8:2+flush-179:023:20:240
2013612995245,5cyclictest2080152-21kworker/u8:2+flush-179:022:45:191
2013612995245,5cyclictest2077304-21latency_hist22:25:011
2013612995243,7cyclictest2082084-21latency_hist22:40:011
2013612995143,4cyclictest2055696-21kworker/u8:0+flush-179:021:20:161
2013612995141,8cyclictest2102211-21apt-get23:40:111
2013612995141,8cyclictest2102211-21apt-get23:40:111
2013612995045,4cyclictest2098996-21kworker/u8:0+flush-179:023:30:171
2013612995044,4cyclictest2067721-21kworker/u8:1+events_unbound22:05:011
2013604995042,6cyclictest2044741-21kworker/u8:3+flush-179:021:15:010
2013604995040,7cyclictest2020312-21latency_hist19:30:010
2013612994944,4cyclictest2116367-21kworker/u8:2+events_unbound00:30:251
2013612994942,5cyclictest2108289-21kworker/u8:1+flush-179:000:20:231
2013604994944,4cyclictest2044696-21kworker/u8:2+flush-179:020:55:180
2013604994938,9cyclictest2030641-21apt-get20:00:130
4577091480,13ptp4l441ktimers/319:10:013
2013612994843,4cyclictest2027431-21kworker/u8:0+flush-179:020:05:171
2013612994834,10cyclictest2072275-21apt-get22:10:001
2013604994841,5cyclictest2055696-21kworker/u8:0+flush-179:021:25:180
2013612994741,4cyclictest2030644-21kworker/u8:1+flush-179:020:15:211
2013604994742,4cyclictest2072242-21kworker/u8:0+flush-179:023:10:210
2013604994740,5cyclictest2015787-21apt-get19:15:110
2013612994642,2cyclictest2055696-21kworker/u8:0+events_unbound21:40:021
2013612994642,2cyclictest2055696-21kworker/u8:0+events_unbound21:40:011
2013612994641,4cyclictest2062693-21kworker/u8:2+flush-179:021:44:551
2013612994639,5cyclictest2044741-21kworker/u8:3+events_unbound21:05:001
2013604994635,9cyclictest2051870-21diskstats21:05:160
2013604994635,9cyclictest2040467-21diskstats20:30:140
2013614994541,2cyclictest0-21swapper/200:06:592
2013612994539,5cyclictest2072242-21kworker/u8:0+flush-179:023:00:211
2013604994540,4cyclictest2067721-21kworker/u8:1+flush-179:021:55:240
2013604994535,9cyclictest0-21swapper/022:04:560
2013614994441,2cyclictest0-21swapper/220:53:582
2013604994438,5cyclictest2072242-21kworker/u8:0+flush-179:022:15:180
2013604994437,5cyclictest2100383-21kworker/u8:3+flush-179:023:50:170
2013604994435,5cyclictest2027431-21kworker/u8:0+flush-179:020:45:230
2013615994340,2cyclictest0-21swapper/300:06:593
2013612994337,4cyclictest2044741-21kworker/u8:3+events_unbound21:05:271
201360499435,37cyclictest714-21Xorg21:50:580
2013604994338,4cyclictest2027431-21kworker/u8:0+flush-179:019:50:220
2013615994240,1cyclictest0-21swapper/320:53:583
2013615994236,4cyclictest2046302-21turbostat20:50:003
2013612994225,7cyclictest2055696-21kworker/u8:0+events_unbound21:45:231
2013604994233,7cyclictest1252047-21kworker/u8:2+flush-179:019:20:190
201360499422,39cyclictest211irq_work/022:05:580
201361499410,2cyclictest0-21swapper/222:53:592
201361499410,2cyclictest0-21swapper/222:53:592
2013604994136,4cyclictest2018760-21kworker/u8:0+flush-179:019:40:200
2013604994135,5cyclictest2027431-21kworker/u8:0+flush-179:020:35:220
2013604994135,4cyclictest2076498-21kworker/u8:1+flush-179:022:25:220
201360499411,3cyclictest714-21Xorg23:46:590
201360499410,39cyclictest714-21Xorg19:30:590
2013615994034,5cyclictest2041522-21turbostat20:34:593
2013612994031,7cyclictest2020327-21apt19:30:011
201361299401,37cyclictest714-21Xorg23:23:591
2013604994034,4cyclictest2033560-21latency_hist20:10:010
2013604994033,6cyclictest2055696-21kworker/u8:0+flush-179:021:35:190
2013604994033,6cyclictest2055696-21kworker/u8:0+flush-179:021:35:190
2013604994032,6cyclictest2055696-21kworker/u8:0+flush-179:021:20:250
2013615993934,4cyclictest2067490-21turbostat21:55:003
2013615993933,5cyclictest2074115-21turbostat22:15:003
201361599390,1cyclictest0-21swapper/322:53:593
201361599390,1cyclictest0-21swapper/322:53:593
201361599390,1cyclictest0-21swapper/319:29:553
2013614993935,2cyclictest0-21swapper/223:12:592
201361499390,36cyclictest0-21swapper/222:32:592
201361499390,36cyclictest0-21swapper/221:09:552
201361499390,2cyclictest0-21swapper/219:16:592
201360499390,38cyclictest2117931-21basename00:30:010
2013615993832,5cyclictest2015282-21turbostat19:15:013
2013614993834,3cyclictest0-21swapper/200:15:592
2013614993834,2cyclictest0-21swapper/220:14:592
201361499380,36cyclictest0-21swapper/221:23:592
201361499380,36cyclictest0-21swapper/220:15:582
2013612993833,4cyclictest2030644-21kworker/u8:1+flush-179:020:20:241
2013604993834,3cyclictest538-21gdbus00:04:010
2013604993833,4cyclictest714-21Xorg23:08:590
2013604993832,5cyclictest2072242-21kworker/u8:0+flush-179:022:30:210
2013604993831,5cyclictest2013393-21kworker/u8:1+flush-179:019:35:170
201360499381,36cyclictest1-21systemd21:16:470
2013615993734,2cyclictest0-21swapper/323:12:593
2013615993734,1cyclictest0-21swapper/300:15:583
2013614993735,1cyclictest0-21swapper/221:55:002
2013614993733,2cyclictest0-21swapper/222:41:592
2013604993734,2cyclictest2027431-21kworker/u8:0+events_unbound20:20:020
2013604993731,4cyclictest1-21systemd22:55:000
2013604993731,4cyclictest1-21systemd22:55:000
2013615993633,2cyclictest0-21swapper/322:09:593
2013615993633,2cyclictest0-21swapper/320:14:583
2013615993630,5cyclictest2064326-21turbostat21:45:003
201361599360,35cyclictest0-21swapper/322:32:593
201361599360,35cyclictest0-21swapper/321:23:583
201361599360,35cyclictest0-21swapper/320:15:593
201361599360,1cyclictest0-21swapper/321:09:553
201361599360,1cyclictest0-21swapper/319:16:593
2013614993633,2cyclictest0-21swapper/223:24:552
201361499360,2cyclictest0-21swapper/219:29:542
2013612993633,2cyclictest0-21swapper/123:55:591
2013604993633,2cyclictest714-21Xorg00:19:580
2013604993632,3cyclictest2103566-21idleruntime-cro23:45:000
2013604993632,3cyclictest2103566-21idleruntime-cro23:45:000
2013604993628,6cyclictest510-21rs:main0
2013615993533,1cyclictest0-21swapper/323:24:553
2013615993533,1cyclictest0-21swapper/322:41:593
2013615993530,4cyclictest2082069-21turbostat22:40:013
2013614993533,1cyclictest0-21swapper/222:15:002
2013614993533,1cyclictest0-21swapper/222:10:002
2013614993533,1cyclictest0-21swapper/220:34:592
2013614993532,2cyclictest0-21swapper/222:03:592
2013614993532,2cyclictest0-21swapper/219:31:592
2013614993531,2cyclictest0-21swapper/219:58:592
2013614993531,2cyclictest0-21swapper/219:58:582
2013612993530,3cyclictest2105162-21systemd-run23:50:001
2013604993526,7cyclictest510-21rs:main0
2013615993432,1cyclictest0-21swapper/322:04:003
2013615993432,1cyclictest0-21swapper/319:31:593
2013615993431,2cyclictest0-21swapper/319:58:593
2013615993431,2cyclictest0-21swapper/319:58:583
201361599340,1cyclictest0-21swapper/322:16:593
2013614993431,2cyclictest0-21swapper/222:47:592
2013614993431,2cyclictest0-21swapper/222:25:592
2013614993431,2cyclictest0-21swapper/219:15:002
2013614993430,2cyclictest0-21swapper/220:49:562
2013612993426,6cyclictest2055696-21kworker/u8:0+flush-179:021:30:131
2013604993429,4cyclictest2044696-21kworker/u8:2+flush-179:020:50:200
2013604993427,5cyclictest2027431-21kworker/u8:0+flush-179:020:25:190
2013604993427,4cyclictest2093976-21apt-get23:15:130
2013615993331,1cyclictest0-21swapper/322:25:593
2013615993327,5cyclictest2116341-21turbostat00:25:003
2013615993326,6cyclictest2061134-21turbostat21:35:013
201361599330,32cyclictest0-21swapper/300:02:593
2013614993331,1cyclictest0-21swapper/221:44:592
2013614993330,2cyclictest0-21swapper/221:17:592
2013614993329,2cyclictest0-21swapper/223:29:552
201361499330,2cyclictest0-21swapper/222:16:592
2013604993329,3cyclictest2097603-21hwlatdetect23:25:160
2013604993325,7cyclictest266-21systemd-journal22:49:470
2013615993230,1cyclictest0-21swapper/322:47:593
2013615993229,2cyclictest0-21swapper/321:59:593
2013615993229,2cyclictest0-21swapper/321:18:003
2013615993229,2cyclictest0-21swapper/320:24:593
2013615993226,5cyclictest2065914-21turbostat21:50:003
2013614993230,1cyclictest0-21swapper/222:40:002
2013614993229,2cyclictest0-21swapper/221:59:592
2013614993229,2cyclictest0-21swapper/220:24:592
2013614993228,2cyclictest0-21swapper/221:30:592
2013614993228,2cyclictest0-21swapper/220:36:592
201361499320,30cyclictest0-21swapper/200:02:592
2013615993128,2cyclictest0-21swapper/323:29:543
2013614993128,2cyclictest0-21swapper/200:36:592
2013614993127,2cyclictest0-21swapper/223:59:002
201361499310,2cyclictest0-21swapper/219:40:132
201361499310,23cyclictest0-21swapper/221:45:132
2013604993122,7cyclictest510-21rs:main0
201360499310,31cyclictest0-21swapper/021:30:120
2013615993028,2cyclictest0-21swapper/320:37:003
2013615993028,1cyclictest0-21swapper/300:36:583
2013615993027,2cyclictest0-21swapper/319:40:003
201361599300,28cyclictest0-21swapper/323:05:013
2013614993027,2cyclictest0-21swapper/200:24:592
2013614993027,2cyclictest0-21swapper/200:13:002
4577091290,7ptp4l441ktimers/321:30:003
4577091290,7ptp4l441ktimers/321:05:003
2013615992927,1cyclictest0-21swapper/300:13:003
2013615992926,2cyclictest0-21swapper/321:10:133
2013615992925,2cyclictest0-21swapper/323:15:593
201361599290,2cyclictest0-21swapper/323:30:013
201361599290,27cyclictest0-21swapper/319:35:143
2013614992926,2cyclictest0-21swapper/223:50:132
2013614992926,2cyclictest0-21swapper/223:05:022
2013614992926,2cyclictest0-21swapper/223:02:592
2013614992926,2cyclictest0-21swapper/221:10:122
2013614992926,2cyclictest0-21swapper/219:20:132
2013614992925,2cyclictest0-21swapper/223:15:592
201361499290,27cyclictest0-21swapper/219:35:142
4577091280,6ptp4l441ktimers/323:45:003
4577091280,6ptp4l441ktimers/323:45:003
4577091280,6ptp4l441ktimers/321:00:013
4577091280,6ptp4l441ktimers/319:50:013
2013615992826,1cyclictest0-21swapper/323:59:003
2013615992826,1cyclictest0-21swapper/323:02:593
2013615992825,2cyclictest0-21swapper/323:38:593
2013615992825,2cyclictest0-21swapper/319:20:123
201361599280,27cyclictest0-21swapper/323:50:133
2013614992825,2cyclictest0-21swapper/223:38:592
201361499280,27cyclictest0-21swapper/223:30:012
201361499280,25cyclictest0-21swapper/221:29:592
2013604992822,5cyclictest2055696-21kworker/u8:0+flush-179:021:45:160
4577091270,5ptp4l441ktimers/320:29:563
4577091270,5ptp4l441ktimers/300:29:563
2013615992724,2cyclictest0-21swapper/320:09:593
2013615992723,3cyclictest2062743-21turbostat21:40:003
2013615992723,3cyclictest2062743-21turbostat21:40:003
2013615992723,2cyclictest0-21swapper/319:54:003
2013615992721,4cyclictest2119521-21turbostat00:35:013
2013614992724,2cyclictest0-21swapper/219:48:582
2013614992724,1cyclictest0-21swapper/220:09:592
2013614992723,2cyclictest0-21swapper/219:54:002
4577091260,5ptp4l61490irq/134-eno1-TxRx-020:04:563
4577091260,5ptp4l441ktimers/320:44:563
4577091260,4ptp4l441ktimers/323:49:553
4577091260,4ptp4l441ktimers/323:00:003
2013615992518,6cyclictest2077314-21turbostat22:25:003
2013614992523,1cyclictest0-21swapper/221:40:012
2013614992523,1cyclictest0-21swapper/221:40:002
2013614992522,2cyclictest0-21swapper/223:43:592
2013614992522,2cyclictest0-21swapper/223:43:592
201361499250,2cyclictest0-21swapper/222:20:132
2013604992521,3cyclictest0-21swapper/000:40:000
2013614992421,2cyclictest0-21swapper/200:30:132
2013614992320,2cyclictest0-21swapper/220:29:592
2013614992117,2cyclictest0-21swapper/220:56:582
2013604992117,3cyclictest2013749-21gpgv19:10:030
201361299200,18cyclictest2013784-21apt-key19:10:031
2013614991916,2cyclictest0-21swapper/220:00:142
2013614991815,2cyclictest0-21swapper/223:48:582
2013614991310,2cyclictest0-21swapper/220:44:562
2013614991210,1cyclictest0-21swapper/221:05:012
201361499107,2cyclictest0-21swapper/200:28:002
20136149985,2cyclictest0-21swapper/222:59:562
20136149930,2cyclictest0-21swapper/219:10:022
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional