You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-21 - 07:43
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot6.osadl.org (updated Fri Nov 21, 2025 00:44:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
17098649910391,8cyclictest1778701-21latency_hist22:45:001
1709861999481,10cyclictest1770741-21latency_hist22:20:000
1709861999166,22cyclictest1720899-21latency_hist19:45:010
1709861998878,7cyclictest1756156-21latency_hist21:35:010
1709864998776,8cyclictest271-21systemd-journal21:25:001
1709864998776,8cyclictest271-21systemd-journal21:25:001
1709864998375,5cyclictest1795126-21apt-get23:35:031
1709864997564,8cyclictest1765712-21latency_hist22:05:011
1709861997569,5cyclictest1793567-21kworker/u8:3+flush-179:023:40:240
1709861997468,4cyclictest1800389-21kworker/u8:2+flush-179:023:50:250
1709861997464,7cyclictest1743162-21latency_hist20:55:010
4670791730,18ptp4l61690irq/136-eno1-TxRx-019:07:313
1709861997364,6cyclictest1800412-21latency_hist23:50:020
1709861997364,5cyclictest1805205-21systemd00:03:590
1709864997264,5cyclictest1759587-21/usr/sbin/munin21:45:251
1709864997259,10cyclictest271-21systemd-journal22:15:011
1709864997252,18cyclictest1791924-21kworker/u8:0+flush-179:023:30:241
1709864997162,7cyclictest1733629-21latency_hist20:25:011
1709864997064,4cyclictest1791924-21kworker/u8:0+flush-179:023:45:231
1709861997059,8cyclictest1759350-21latency_hist21:45:010
1709864996959,7cyclictest1709726-21latency_hist19:10:031
1709861996860,6cyclictest1746006-21kworker/u8:2+events_unbound21:15:140
1709861996859,6cyclictest1760933-21latency_hist21:50:020
1709864996762,4cyclictest1736780-21kworker/u8:0+flush-179:020:50:261
1709864996753,12cyclictest271-21systemd-journal21:40:001
1709861996761,4cyclictest1785358-21kworker/u8:2+flush-179:023:25:210
1709861996761,4cyclictest1785358-21kworker/u8:2+flush-179:023:25:200
170986699660,63cyclictest0-21swapper/219:28:582
1709864996657,6cyclictest1745007-21/usr/sbin/munin21:00:241
1709864996655,8cyclictest1749787-21latency_hist21:15:011
1709864996559,4cyclictest1734690-21kworker/u8:1+flush-179:020:30:301
1709864996554,9cyclictest1738666-21apt-get20:40:151
1709864996554,8cyclictest1785865-21diskstats23:05:191
1709861996557,6cyclictest1753019-21kworker/u8:3+flush-179:022:00:170
1709861996153,6cyclictest1736780-21kworker/u8:0+flush-179:020:35:250
1709870996056,2cyclictest0-21swapper/321:46:593
1709864996050,7cyclictest1781871-21latency_hist22:55:021
1709861996055,4cyclictest1778691-21kworker/u8:1+flush-179:023:00:150
1709861996053,5cyclictest1810000-21kworker/u8:1+flush-179:000:25:240
1709864995949,8cyclictest1787145-21/usr/sbin/munin23:10:241
1709864995949,6cyclictest1767588-21kworker/u8:1+flush-179:022:20:271
170986499590,2cyclictest0-21swapper/120:29:591
170986199581,54cyclictest1811680-21munin-run00:25:010
1709864995746,8cyclictest1773920-21latency_hist22:30:001
1709861995753,3cyclictest1709782-21kworker/u8:1+flush-179:019:25:170
1709861995750,5cyclictest1709782-21kworker/u8:1+flush-179:019:20:150
1709861995748,7cyclictest1803617-40mandb00:00:010
1709861995649,5cyclictest1778745-21kworker/u8:3+flush-179:022:50:000
1709861995649,5cyclictest1736780-21kworker/u8:0+flush-179:020:45:180
1709861995550,4cyclictest1767588-21kworker/u8:1+flush-179:022:20:250
1709861995548,5cyclictest1762551-21kworker/u8:2+flush-179:022:40:240
1709861995547,5cyclictest1762518-21latency_hist21:55:000
1709861995546,5cyclictest1778691-21kworker/u8:1+events_unbound23:20:010
1709861995447,6cyclictest1753019-21kworker/u8:3+events_unbound22:10:300
1709861995447,5cyclictest1753019-21kworker/u8:3+events_unbound22:05:020
1709861995447,4cyclictest1722464-21kworker/u8:2+flush-179:019:55:260
1709861995343,8cyclictest1739995-21latency_hist20:45:010
1709861995343,7cyclictest1719315-21latency_hist19:40:000
1709864995244,6cyclictest1716096-21kworker/u8:0+flush-179:019:30:221
1709861995243,7cyclictest1772572-21/usr/sbin/munin22:25:200
1709864995143,5cyclictest1798829-21latency_hist23:45:021
1709864995136,13cyclictest1722464-21kworker/u8:2+flush-179:020:40:001
170986499510,1cyclictest0-21swapper/119:24:591
1709861995146,4cyclictest1720373-21kworker/u8:1+flush-179:019:50:210
1709861995140,9cyclictest1756411-21apt-get21:35:120
1709864994947,1cyclictest0-21swapper/119:28:581
1709864994944,4cyclictest1810000-21kworker/u8:1+flush-179:000:30:261
1709864994930,16cyclictest1777096-21latency_hist22:40:021
1709864994930,16cyclictest1777096-21latency_hist22:40:011
1709866994845,2cyclictest0-21swapper/220:19:592
1709864994838,7cyclictest1712947-21latency_hist19:20:011
1709861994843,4cyclictest1722464-21kworker/u8:2+flush-179:020:10:250
1709861994838,7cyclictest1815053-21latency_hist00:35:010
170986199480,46cyclictest723-21Xorg21:11:590
1709864994636,9cyclictest1710011-21apt-get19:10:131
1709864994627,17cyclictest723-21Xorg19:44:591
1709861994639,5cyclictest1727300-21kworker/u8:1+events_unbound20:15:220
1709861994637,6cyclictest1736780-21kworker/u8:0+flush-179:021:05:220
1709861994630,8cyclictest1796967-21apt-get23:40:000
1709864994540,3cyclictest1751434-21systemd-run21:20:001
1709864994538,5cyclictest1774158-21/usr/sbin/munin22:30:181
1709864994537,6cyclictest1790599-21/usr/sbin/munin23:20:201
170986499451,42cyclictest0-21swapper/122:55:031
170986499450,4cyclictest1724084-21turbostat19:59:551
1709861994539,4cyclictest1778691-21kworker/u8:1+events_unbound23:10:200
1709861994537,5cyclictest1731734-21kworker/u8:0+flush-179:020:20:150
1709864994426,15cyclictest0-21swapper/119:45:121
1709864994425,16cyclictest0-21swapper/121:40:141
1709861994439,4cyclictest1762551-21kworker/u8:2+flush-179:022:34:550
1709861994437,5cyclictest1807774-21kworker/u8:1+flush-179:000:10:260
1709861994437,4cyclictest1800389-21kworker/u8:2+flush-179:000:15:260
1709864994334,6cyclictest1785323-21latency_hist23:05:001
1709864994331,11cyclictest1803652-21cupsd00:00:011
1709870994238,2cyclictest0-21swapper/321:11:593
1709861994230,9cyclictest1800389-21kworker/u8:2+events_unbound00:05:200
1709866994138,2cyclictest0-21swapper/223:42:582
1709866994138,2cyclictest0-21swapper/221:43:582
170986699410,39cyclictest0-21swapper/219:40:592
1709861994136,4cyclictest1793567-21kworker/u8:3+flush-179:023:30:160
1709861994131,8cyclictest1716378-21apt-get19:30:120
1709864994034,4cyclictest723-21Xorg21:52:591
1709861994035,3cyclictest1712985-21systemd-run19:20:000
1709861994034,4cyclictest1783711-21cron23:00:010
1709861994033,5cyclictest1777079-21kworker/u8:0+flush-179:022:50:250
1709861994031,6cyclictest1753019-21kworker/u8:3+events_unbound21:55:210
170986199401,10cyclictest1721154-21apt-get19:45:130
1709870993936,2cyclictest0-21swapper/323:01:593
170986699390,37cyclictest0-21swapper/222:09:002
1709864993926,10cyclictest1769395-21apt-get22:15:151
1709864993922,15cyclictest723-21Xorg23:54:581
170986499390,37cyclictest723-21Xorg20:14:591
1709861993932,5cyclictest1727300-21kworker/u8:1+flush-179:020:05:210
1709870993834,2cyclictest0-21swapper/319:20:593
1709866993834,2cyclictest0-21swapper/221:46:592
170986699380,36cyclictest0-21swapper/223:16:002
170986699380,36cyclictest0-21swapper/200:12:002
1709864993834,2cyclictest723-21Xorg22:06:591
1709864993833,3cyclictest1810066-21systemd-run00:20:011
1709864993832,4cyclictest1731121-21hwlatdetect20:15:181
1709864993821,16cyclictest0-21swapper/100:29:591
1709861993835,2cyclictest1753221-21/usr/sbin/munin21:25:230
1709861993832,4cyclictest1785358-21kworker/u8:2+events_unbound23:15:200
1709861993831,5cyclictest1722464-21kworker/u8:2+flush-179:020:29:550
1709861993829,5cyclictest1750778-21kworker/u8:0+flush-179:021:20:190
1709861993829,5cyclictest1750778-21kworker/u8:0+flush-179:021:20:180
1709861993828,7cyclictest1778691-21kworker/u8:1+events_unbound23:05:280
1709870993734,2cyclictest0-21swapper/323:47:003
170986699370,35cyclictest0-21swapper/223:32:002
170986699370,35cyclictest0-21swapper/220:39:592
170986699370,1cyclictest0-21swapper/220:12:592
170986499373,32cyclictest1816802-21cat00:40:011
1709864993729,5cyclictest1800389-21kworker/u8:2+events_unbound00:05:231
1709864993718,17cyclictest0-21swapper/120:55:221
170986199371,31cyclictest0-21swapper/000:35:150
1709870993634,1cyclictest0-21swapper/323:28:593
1709870993634,1cyclictest0-21swapper/323:28:583
1709870993634,1cyclictest0-21swapper/322:38:003
1709870993634,1cyclictest0-21swapper/322:37:593
1709870993634,1cyclictest0-21swapper/321:28:593
1709870993633,2cyclictest0-21swapper/321:41:003
170987099360,1cyclictest0-21swapper/319:48:593
1709866993633,2cyclictest0-21swapper/221:39:002
170986699360,1cyclictest0-21swapper/200:31:002
1709864993630,4cyclictest1716096-21kworker/u8:0+flush-179:019:40:001
1709864993621,13cyclictest723-21Xorg20:04:591
1709864993619,16cyclictest0-21swapper/100:10:251
1709864993618,16cyclictest0-21swapper/100:04:001
1709861993631,4cyclictest1762551-21kworker/u8:2+flush-179:022:35:260
1709861993631,4cyclictest1762551-21kworker/u8:2+flush-179:022:35:260
1709870993533,1cyclictest0-21swapper/322:24:553
1709870993532,2cyclictest0-21swapper/300:20:593
170987099350,1cyclictest0-21swapper/323:43:593
1709866993532,2cyclictest0-21swapper/222:41:592
1709866993532,2cyclictest0-21swapper/221:59:552
1709864993516,2cyclictest0-21swapper/121:05:161
1709870993431,2cyclictest0-21swapper/320:04:553
1709866993430,2cyclictest0-21swapper/220:51:592
1709864993426,4cyclictest0-21swapper/119:50:131
1709864993415,18cyclictest0-21swapper/120:07:271
1709870993331,1cyclictest0-21swapper/320:49:583
1709866993330,2cyclictest0-21swapper/222:40:002
1709866993330,2cyclictest0-21swapper/222:40:002
1709866993330,2cyclictest0-21swapper/219:40:002
1709866993330,2cyclictest0-21swapper/200:07:592
170986499333,6cyclictest723-21Xorg00:20:591
1709864993329,3cyclictest1762534-21dump-pmu-power21:59:551
1709864993328,3cyclictest1-21systemd23:30:011
1709864993328,3cyclictest1-21systemd23:30:001
1709864993324,7cyclictest1788730-21/usr/sbin/munin23:15:231
1709864993317,15cyclictest0-21swapper/122:45:211
1709861993328,3cyclictest1743535-21tr20:55:200
1709870993230,1cyclictest0-21swapper/323:58:593
1709866993230,1cyclictest0-21swapper/219:34:592
1709866993229,2cyclictest0-21swapper/220:33:592
1709866993229,2cyclictest0-21swapper/200:37:592
170986699320,30cyclictest0-21swapper/220:09:592
1709861993227,3cyclictest0-21swapper/020:00:150
1709870993129,1cyclictest0-21swapper/319:38:593
1709870993128,2cyclictest0-21swapper/320:19:593
1709870993126,4cyclictest1711347-21turbostat19:15:013
170987099310,30cyclictest0-21swapper/323:34:593
1709866993128,2cyclictest0-21swapper/220:23:592
1709864993113,3cyclictest281ktimers/121:25:211
170986499310,28cyclictest0-21swapper/121:30:151
170986199310,28cyclictest0-21swapper/019:10:150
1709870993028,1cyclictest0-21swapper/322:55:593
1709870993028,1cyclictest0-21swapper/319:17:593
1709870993027,2cyclictest0-21swapper/321:32:593
1709870993027,2cyclictest0-21swapper/319:25:143
170987099300,28cyclictest0-21swapper/323:15:133
1709866993027,2cyclictest0-21swapper/223:39:562
1709866993027,2cyclictest0-21swapper/222:15:142
1709866993027,2cyclictest0-21swapper/222:01:582
1709866993027,2cyclictest0-21swapper/221:51:002
1709866993027,2cyclictest0-21swapper/219:45:132
1709866993027,2cyclictest0-21swapper/200:15:592
170986699300,28cyclictest0-21swapper/222:20:152
1709864993026,3cyclictest1741579-21munin-plugin-st20:50:011
1709861993025,4cyclictest1743141-21kworker/u8:1+flush-179:021:04:550
1709861993024,4cyclictest1735560-21cpuspeed_turbos20:30:200
4670791290,7ptp4l441ktimers/320:10:003
1709870992926,2cyclictest0-21swapper/322:25:163
1709870992926,2cyclictest0-21swapper/322:00:143
1709870992926,2cyclictest0-21swapper/319:55:143
1709870992926,2cyclictest0-21swapper/300:25:013
1709870992925,1cyclictest0-21swapper/300:15:153
1709870992924,2cyclictest0-21swapper/323:35:023
170987099290,2cyclictest0-21swapper/321:36:593
170987099290,27cyclictest0-21swapper/322:15:143
170987099290,27cyclictest0-21swapper/321:05:013
170987099290,1cyclictest0-21swapper/323:20:023
1709866992926,2cyclictest0-21swapper/223:00:142
1709866992926,2cyclictest0-21swapper/220:00:592
170986699290,27cyclictest0-21swapper/223:20:032
170986699290,27cyclictest0-21swapper/222:48:002
170986699290,27cyclictest0-21swapper/200:20:142
170986699290,1cyclictest0-21swapper/223:10:132
4670791280,6ptp4l441ktimers/300:35:013
1709870992826,1cyclictest0-21swapper/322:05:043
1709870992826,1cyclictest0-21swapper/300:35:143
1709870992822,4cyclictest1781889-21turbostat22:55:013
170987099280,3cyclictest1769147-21turbostat22:15:003
1709866992825,2cyclictest0-21swapper/223:57:592
1709866992825,2cyclictest0-21swapper/221:11:582
1709866992825,2cyclictest0-21swapper/219:50:132
1709866992825,2cyclictest0-21swapper/200:25:022
1709866992825,2cyclictest0-21swapper/200:04:552
170986699280,27cyclictest0-21swapper/222:55:142
170986699280,26cyclictest0-21swapper/221:03:592
170986699280,25cyclictest0-21swapper/220:44:552
4670791270,7ptp4l61690irq/136-eno1-TxRx-019:35:003
4670791270,6ptp4l441ktimers/320:35:003
170987099270,26cyclictest0-21swapper/322:40:593
170987099270,1cyclictest0-21swapper/321:55:143
1709866992724,2cyclictest0-21swapper/223:27:582
1709866992724,2cyclictest0-21swapper/223:27:582
1709866992724,2cyclictest0-21swapper/222:13:592
1709866992724,2cyclictest0-21swapper/220:56:592
1709866992724,2cyclictest0-21swapper/219:16:002
4670791260,7ptp4l61690irq/136-eno1-TxRx-023:54:593
4670791260,6ptp4l61690irq/136-eno1-TxRx-021:05:003
4670791260,6ptp4l61690irq/136-eno1-TxRx-020:39:553
4670791260,6ptp4l61690irq/136-eno1-TxRx-019:55:013
4670791260,6ptp4l441ktimers/321:20:013
4670791260,6ptp4l441ktimers/320:55:023
4670791260,6ptp4l441ktimers/320:54:563
4670791260,6ptp4l441ktimers/320:44:553
4670791260,5ptp4l61690irq/136-eno1-TxRx-021:55:003
4670791260,5ptp4l61690irq/136-eno1-TxRx-019:44:553
4670791260,5ptp4l441ktimers/323:09:593
4670791260,5ptp4l441ktimers/322:34:553
4670791260,5ptp4l441ktimers/320:14:553
1709870992623,2cyclictest0-21swapper/300:07:003
1709870992623,2cyclictest0-21swapper/300:01:593
170987099260,24cyclictest0-21swapper/322:45:133
170987099260,1cyclictest0-21swapper/321:23:003
170987099260,1cyclictest0-21swapper/321:23:003
1709866992623,2cyclictest0-21swapper/223:54:562
1709866992623,2cyclictest0-21swapper/222:35:002
1709866992623,2cyclictest0-21swapper/219:22:592
170986699260,2cyclictest0-21swapper/221:30:592
4670791250,5ptp4l61690irq/136-eno1-TxRx-020:29:563
4670791250,4ptp4l441ktimers/320:24:553
1709870992522,2cyclictest0-21swapper/300:14:593
1709866992523,1cyclictest0-21swapper/221:29:592
1709866992522,2cyclictest0-21swapper/220:29:552
170986699250,1cyclictest0-21swapper/219:10:272
4670791240,4ptp4l441ktimers/323:14:553
1709866992320,2cyclictest0-21swapper/221:09:002
1709866992219,2cyclictest0-21swapper/223:48:592
1709861992218,3cyclictest1709868-21latency_hist19:10:020
1709866992119,1cyclictest0-21swapper/220:49:592
1709866991816,1cyclictest0-21swapper/223:09:592
1709866991716,1cyclictest0-21swapper/222:54:582
1709866991614,1cyclictest0-21swapper/219:58:592
1709866991412,1cyclictest0-21swapper/222:28:592
170986699129,2cyclictest0-21swapper/221:23:002
170986699129,2cyclictest0-21swapper/221:23:002
17098669985,2cyclictest0-21swapper/221:15:572
17098669930,2cyclictest0-21swapper/219:10:022
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional