You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-24 - 05:31
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot6.osadl.org (updated Mon Nov 24, 2025 00:44:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
17730519910797,7cyclictest1878321-21latency_hist00:35:000
17730529910082,8cyclictest1777793-21latency_hist19:25:001
1773052999783,10cyclictest271-21systemd-journal00:09:591
1773052999684,8cyclictest1863680-21latency_hist23:50:001
1773052998777,7cyclictest1843001-21latency_hist22:45:011
1773052998476,6cyclictest1854320-21kworker/u8:1+flush-179:023:30:251
1773052998171,7cyclictest1829751-21latency_hist22:05:011
1773051997967,10cyclictest1868523-21apt-get00:05:010
1773052997766,8cyclictest1825208-21/usr/sbin/munin21:50:231
1773051997771,5cyclictest1787565-21kworker/u8:0+events_unbound20:05:010
1773051997770,5cyclictest1813588-21kworker/u8:3+flush-179:021:25:180
1773051997569,5cyclictest1870393-21kworker/u8:1+flush-179:000:15:280
1773052997465,7cyclictest1855968-21/usr/sbin/munin23:25:241
1773051997369,2cyclictest723-21Xorg20:30:590
1773051997264,6cyclictest1829786-21kworker/u8:3+flush-179:023:15:010
1773051997161,7cyclictest1839549-21latency_hist22:35:010
1773051997062,5cyclictest1790560-21kworker/u8:1+flush-179:020:15:190
1773052996859,7cyclictest1792112-21latency_hist20:10:001
1773051996861,4cyclictest1813588-21kworker/u8:3+flush-179:021:35:220
1773052996659,5cyclictest1813588-21kworker/u8:3+flush-179:021:25:251
1773051996353,7cyclictest1860500-21latency_hist23:40:010
1773051996353,7cyclictest1797147-21latency_hist20:25:010
1773057996255,5cyclictest1879935-21turbostat00:39:593
1773051996156,3cyclictest1803749-21kworker/u8:0+flush-179:021:10:260
1773051996150,8cyclictest1806922-21kworker/u8:3+events_unbound20:55:030
1773051996053,5cyclictest271-21systemd-journal23:04:590
1773052995948,8cyclictest271-21systemd-journal20:50:001
1773051995948,9cyclictest1800575-21apt-get20:35:140
1773052995851,4cyclictest1829786-21kworker/u8:3+flush-179:022:10:231
1773051995750,5cyclictest1829786-21kworker/u8:3+flush-179:023:25:170
1773051995749,5cyclictest1834514-21latency_hist22:20:010
1773051995748,6cyclictest1841113-21kworker/u8:1+events_unbound22:40:230
1773051995748,6cyclictest1787327-21latency_hist19:55:010
1773051995742,8cyclictest1829786-21kworker/u8:3+events_unbound22:20:020
1773052995648,5cyclictest1780966-21latency_hist19:35:021
1773052995612,1cyclictest723-21Xorg23:19:001
1773051995649,5cyclictest1829786-21kworker/u8:3+flush-179:022:10:250
1773051995646,8cyclictest1854402-21apt-get23:20:160
1773052995548,5cyclictest271-21systemd-journal19:26:201
1773051995548,4cyclictest1854320-21kworker/u8:1+flush-179:023:50:250
1773052995346,5cyclictest1805107-21apt-get20:50:041
1773051995347,4cyclictest1777828-21kworker/u8:0+flush-179:019:40:240
1773051995342,8cyclictest1844620-21kworker/u8:2+events_unbound22:55:010
1773052995240,9cyclictest1793712-21apt-get20:15:011
1773051995247,4cyclictest1829786-21kworker/u8:3+events_unbound23:05:240
1773051995246,4cyclictest1788777-21kworker/u8:2+flush-179:020:10:180
1773051995242,7cyclictest1879926-21latency_hist00:40:010
1773052995141,7cyclictest1847775-21latency_hist23:00:011
1773051995142,6cyclictest1803749-21kworker/u8:0+events_unbound20:50:010
1773052995044,4cyclictest0-21swapper/122:23:591
1773052995038,10cyclictest1833176-21apt-get22:15:121
1773051995044,4cyclictest1829786-21kworker/u8:3+flush-179:022:55:250
1773052994940,7cyclictest1875377-21/usr/sbin/munin00:25:251
1773051994940,6cyclictest1854320-21kworker/u8:1+flush-179:023:30:230
1773051994938,9cyclictest1806955-21apt-get20:55:000
1773052994844,3cyclictest0-21swapper/121:09:591
1773052994835,5cyclictest1809066-21apt-get21:00:141
177305399470,39cyclictest0-21swapper/220:28:002
1773052994743,3cyclictest1821763-21kworker/u8:1+flush-179:021:40:261
1773052994738,8cyclictest1781220-21apt-get19:35:121
1773052994712,17cyclictest1803427-21unixbench_multi20:40:251
1773051994742,3cyclictest1775673-21kworker/u8:1+flush-179:019:25:170
1773051994740,5cyclictest1773007-21kworker/u8:2+events_unbound19:15:140
4670791460,12ptp4l441ktimers/319:10:013
1773052994637,7cyclictest0-21swapper/120:00:141
1773052994634,9cyclictest1866856-40logrotate00:00:011
177305199461,43cyclictest1870367-21apt00:10:000
177305799450,43cyclictest0-21swapper/323:44:563
1773052994534,9cyclictest1838504-21diskstats22:30:151
1773051994542,2cyclictest1783730-21kworker/u8:1+events_unbound19:45:240
1773051994538,5cyclictest1826793-21kworker/u8:0+flush-179:022:25:270
1773051994538,5cyclictest1826793-21kworker/u8:0+flush-179:022:25:270
177305799440,43cyclictest0-21swapper/320:27:593
177305399440,41cyclictest0-21swapper/223:44:552
1773052994412,18cyclictest1816936-21latency_hist21:25:011
1773051994438,4cyclictest1790560-21kworker/u8:1+flush-179:020:09:550
1773051994437,4cyclictest1828644-21kworker/u8:2+flush-179:022:05:210
177305799430,42cyclictest0-21swapper/320:51:593
177305399430,41cyclictest0-21swapper/220:51:592
1773052994339,2cyclictest1849362-21cron23:05:001
1773052994330,10cyclictest1868517-21latency_hist00:05:021
1773051994339,2cyclictest723-21Xorg19:13:580
1773051994339,2cyclictest1803749-21kworker/u8:0+events_unbound21:30:230
1773051994337,4cyclictest1775673-21kworker/u8:1+flush-179:019:30:250
1773051994335,6cyclictest1854320-21kworker/u8:1+flush-179:023:45:210
1773051994330,8cyclictest1841142-21latency_hist22:40:010
1773051994330,10cyclictest271-21systemd-journal23:45:000
1773052994232,8cyclictest1851206-21apt-get23:10:121
1773051994237,4cyclictest1870616-21apt-get00:10:170
1773051994237,4cyclictest1803749-21kworker/u8:0+flush-179:021:05:260
1773057994134,5cyclictest1784174-21turbostat19:45:013
1773053994137,2cyclictest0-21swapper/221:31:592
177305299419,31cyclictest0-21swapper/100:22:581
1773052994137,2cyclictest1878575-21apt-get00:35:141
1773052994134,5cyclictest1860063-21wc23:35:231
177305299411,36cyclictest1843257-21apt-get22:45:121
1773052994111,16cyclictest0-21swapper/123:05:261
1773052994012,15cyclictest1815245-21cut21:20:001
1773052994012,14cyclictest491-21systemd-logind22:10:001
1773051994036,2cyclictest723-21Xorg21:56:590
1773051994031,7cyclictest1806922-21kworker/u8:3+flush-179:021:05:000
177305199400,3cyclictest723-21Xorg19:38:590
1773057993936,2cyclictest0-21swapper/321:31:593
177305399390,36cyclictest0-21swapper/223:54:552
177305399390,36cyclictest0-21swapper/222:43:592
177305399390,36cyclictest0-21swapper/222:04:592
177305399390,30cyclictest0-21swapper/219:42:582
1773052993934,3cyclictest1820231-21munin-run21:35:011
177305299390,37cyclictest723-21Xorg21:11:591
1773051993935,3cyclictest1821763-21kworker/u8:1+flush-179:021:45:260
1773057993834,3cyclictest0-21swapper/323:15:593
177305799380,37cyclictest0-21swapper/321:19:593
177305799380,37cyclictest0-21swapper/319:57:583
1773053993835,2cyclictest0-21swapper/223:06:582
1773053993834,2cyclictest0-21swapper/223:15:592
177305399380,36cyclictest0-21swapper/219:57:582
1773051993833,4cyclictest1787337-21turbostat19:59:560
1773051993828,7cyclictest1813588-21kworker/u8:3+events_unbound21:15:170
177305199380,37cyclictest723-21Xorg21:43:590
177305799370,2cyclictest0-21swapper/322:04:593
177305399370,35cyclictest0-21swapper/221:19:592
1773052993734,2cyclictest723-21Xorg22:53:591
1773052993734,2cyclictest281ktimers/120:39:581
1773052993734,2cyclictest0-21swapper/122:28:591
1773052993734,2cyclictest0-21swapper/122:28:591
1773052993731,4cyclictest281ktimers/123:40:141
1773051993733,2cyclictest723-21Xorg21:23:590
1773051993727,8cyclictest1773007-21kworker/u8:2+events_unbound19:20:230
177305199370,34cyclictest1873811-21apt-get00:20:130
1773057993634,1cyclictest0-21swapper/323:06:593
1773057993630,4cyclictest1801927-21turbostat20:40:013
1773052993629,4cyclictest1787586-21apt-get19:55:151
177305299362,31cyclictest1784415-21apt-get19:45:141
1773051993631,4cyclictest1843963-21needreboot22:45:180
1773057993530,4cyclictest1837972-21turbostat22:30:003
1773057993530,4cyclictest1837972-21turbostat22:29:593
177305799350,2cyclictest0-21swapper/320:00:153
1773053993431,2cyclictest0-21swapper/220:04:552
1773052993431,2cyclictest0-21swapper/119:51:001
1773052993430,2cyclictest1841487-21latency_hist22:40:001
1773052993428,4cyclictest1774724-21latency_hist19:15:021
1773052993427,5cyclictest1823624-21/usr/sbin/munin21:45:171
1773052993427,5cyclictest1795819-21apt-get20:20:131
1773052993426,6cyclictest271-21systemd-journal23:20:011
177305299341,29cyclictest0-21swapper/120:25:131
1773051993432,1cyclictest0-21swapper/000:27:590
1773053993329,2cyclictest0-21swapper/221:09:552
1773052993329,3cyclictest1871983-21systemd-run00:15:011
1773052993328,4cyclictest1820207-21turbostat21:39:551
1773052993327,5cyclictest0-21swapper/100:16:001
1773051993327,5cyclictest1826793-21kworker/u8:0+events_unbound22:00:230
1773057993225,5cyclictest1811992-21turbostat21:10:003
1773053993230,1cyclictest0-21swapper/220:40:002
1773053993229,2cyclictest0-21swapper/222:30:002
1773053993229,2cyclictest0-21swapper/222:30:002
177305399320,2cyclictest0-21swapper/219:30:152
177305399320,29cyclictest0-21swapper/221:25:152
1773052993229,2cyclictest723-21Xorg23:53:581
1773052993221,8cyclictest1773048-21cyclictest19:10:011
1773051993227,4cyclictest1790560-21kworker/u8:1+flush-179:020:25:170
1773057993126,3cyclictest1821790-21turbostat21:40:003
1773053993125,3cyclictest0-21swapper/222:15:592
177305399310,29cyclictest0-21swapper/223:20:172
177305399310,29cyclictest0-21swapper/223:00:132
177305399310,29cyclictest0-21swapper/219:20:132
1773052993125,5cyclictest1875398-21kworker/u8:2+flush-179:000:33:551
1773051993125,4cyclictest723-21Xorg23:57:580
1773051993123,6cyclictest1803659-21sed20:45:010
1773057993028,1cyclictest0-21swapper/300:28:003
1773057993027,2cyclictest1801927-21turbostat20:44:553
1773057993027,2cyclictest0-21swapper/322:55:143
1773057993026,2cyclictest0-21swapper/320:05:133
1773057993024,4cyclictest1785749-21turbostat19:50:003
177305799300,29cyclictest0-21swapper/319:20:143
177305799300,21cyclictest0-21swapper/323:20:173
177305799300,1cyclictest0-21swapper/320:15:023
177305799300,1cyclictest0-21swapper/319:30:143
1773053993027,2cyclictest0-21swapper/222:55:142
1773053993027,2cyclictest0-21swapper/220:33:592
1773053993026,2cyclictest0-21swapper/222:22:002
177305399300,2cyclictest0-21swapper/220:15:022
177305399300,29cyclictest0-21swapper/200:25:162
177305399300,28cyclictest0-21swapper/222:10:152
177305399300,28cyclictest0-21swapper/219:10:162
177305399300,27cyclictest0-21swapper/223:10:152
177305399300,27cyclictest0-21swapper/200:36:582
1773052993010,19cyclictest0-21swapper/119:20:011
1773051993027,2cyclictest0-21swapper/021:54:590
1773051993024,4cyclictest1854123-21cron23:19:590
1773057992926,2cyclictest0-21swapper/322:22:003
1773057992926,2cyclictest0-21swapper/321:50:133
177305799290,2cyclictest0-21swapper/323:45:133
177305799290,2cyclictest0-21swapper/323:10:153
177305799290,28cyclictest0-21swapper/323:00:123
177305799290,28cyclictest0-21swapper/319:10:153
177305799290,27cyclictest0-21swapper/300:05:023
177305799290,21cyclictest0-21swapper/322:40:133
177305799290,20cyclictest0-21swapper/321:25:153
1773053992926,2cyclictest0-21swapper/220:05:142
1773053992925,2cyclictest0-21swapper/221:55:592
1773053992925,2cyclictest0-21swapper/200:01:592
177305399290,2cyclictest0-21swapper/223:45:132
177305399290,28cyclictest0-21swapper/200:05:022
177305399290,27cyclictest0-21swapper/221:50:132
177305299290,28cyclictest0-21swapper/120:15:021
1773057992826,1cyclictest0-21swapper/320:33:593
1773057992826,1cyclictest0-21swapper/300:05:003
1773057992825,1cyclictest0-21swapper/321:55:593
177305799280,26cyclictest0-21swapper/322:10:153
1773053992826,1cyclictest0-21swapper/221:40:002
1773053992823,3cyclictest0-21swapper/200:17:592
177305399280,26cyclictest0-21swapper/219:36:002
177305399280,25cyclictest0-21swapper/222:45:592
177305399280,25cyclictest0-21swapper/221:46:592
1773052992822,4cyclictest1808797-21perf21:00:001
1773052992812,3cyclictest1799748-21ntpq20:30:211
4670791270,7ptp4l61690irq/136-eno1-TxRx-000:15:003
4670791270,6ptp4l441ktimers/323:30:013
4670791270,5ptp4l441ktimers/300:20:003
1773057992725,1cyclictest0-21swapper/322:15:593
1773057992720,6cyclictest0-21swapper/300:22:593
1773053992724,2cyclictest0-21swapper/220:44:552
1773053992723,2cyclictest0-21swapper/220:46:592
1773053992723,2cyclictest0-21swapper/219:26:142
177305399270,2cyclictest0-21swapper/221:12:592
1773052992724,2cyclictest723-21Xorg19:42:581
1773052992724,2cyclictest1828162-21systemctl22:00:011
4670791260,7ptp4l441ktimers/322:40:003
4670791260,7ptp4l441ktimers/320:15:003
4670791260,6ptp4l61690irq/136-eno1-TxRx-023:55:013
4670791260,6ptp4l61690irq/136-eno1-TxRx-021:05:003
4670791260,6ptp4l441ktimers/322:10:003
4670791260,5ptp4l61690irq/136-eno1-TxRx-023:34:553
4670791260,5ptp4l441ktimers/319:55:003
1773057992622,3cyclictest0-21swapper/320:46:593
1773057992617,2cyclictest0-21swapper/319:15:013
177305799260,24cyclictest0-21swapper/319:36:003
1773053992623,2cyclictest0-21swapper/222:36:002
1773053992623,2cyclictest0-21swapper/220:59:552
1773053992622,2cyclictest0-21swapper/221:42:592
1773053992619,4cyclictest0-21swapper/219:46:542
1773053992616,2cyclictest0-21swapper/221:20:152
4670791250,6ptp4l441ktimers/322:54:563
4670791250,6ptp4l441ktimers/321:25:003
4670791250,5ptp4l441ktimers/321:45:003
4670791250,5ptp4l441ktimers/300:00:003
1773057992523,1cyclictest0-21swapper/320:59:563
1773057992523,1cyclictest0-21swapper/319:26:143
177305799250,24cyclictest0-21swapper/321:46:593
177305799250,1cyclictest0-21swapper/322:45:593
177305799250,1cyclictest0-21swapper/321:12:593
1773053992521,2cyclictest0-21swapper/219:15:582
1773053992519,4cyclictest0-21swapper/200:22:582
4670791240,6ptp4l61690irq/136-eno1-TxRx-000:34:553
4670791240,5ptp4l441ktimers/320:24:563
4670791240,4ptp4l61690irq/136-eno1-TxRx-022:34:553
1773053992422,1cyclictest0-21swapper/222:54:592
1773053992421,2cyclictest0-21swapper/220:14:552
4670791230,4ptp4l441ktimers/323:40:003
1773053992321,1cyclictest0-21swapper/223:35:002
1773053992320,2cyclictest0-21swapper/223:59:002
1773053992219,2cyclictest0-21swapper/222:34:592
1773053992016,2cyclictest0-21swapper/223:28:582
1773053992016,2cyclictest0-21swapper/219:54:552
1773053991916,2cyclictest0-21swapper/221:02:592
1773053991715,1cyclictest0-21swapper/222:07:592
1773053991714,2cyclictest0-21swapper/200:10:592
1773051991713,3cyclictest1773101-21latency_hist19:10:020
1773053991512,2cyclictest0-21swapper/220:20:592
17730539996,2cyclictest0-21swapper/223:37:502
17730539990,2cyclictest0-21swapper/200:32:182
17730539930,2cyclictest0-21swapper/219:10:022
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional