You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-30 - 12:42
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot6.osadl.org (updated Sun Nov 30, 2025 00:44:53)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
178892699118102,12cyclictest1882480-40mandb00:00:020
178892699118102,12cyclictest1882480-40mandb00:00:020
1788927999791,4cyclictest1799966-21kworker/u8:0+events_unbound20:05:001
1788926999281,8cyclictest1879020-21latency_hist23:50:010
1788927998979,7cyclictest1869215-21latency_hist23:20:011
1788926998979,7cyclictest1811127-21latency_hist20:20:010
1788927998779,5cyclictest1799966-21kworker/u8:0+flush-179:020:00:001
1788927997868,8cyclictest1817473-21latency_hist20:40:001
1788927997469,4cyclictest1834160-21kworker/u8:1+flush-179:021:50:261
1788926997467,5cyclictest1850370-21apt-get22:20:130
1788927997163,6cyclictest1877436-21latency_hist23:45:001
1788927997145,7cyclictest1862807-21kworker/u8:1+flush-179:023:00:271
1788926997162,7cyclictest1880866-21latency_hist23:55:020
1788927997065,3cyclictest1787181-21kworker/u8:1+flush-179:019:30:261
1788926996858,8cyclictest1825687-21/usr/sbin/munin21:05:270
178892799660,7cyclictest1793621-21dump-pmu-power19:25:011
1788927996459,4cyclictest1854910-21kworker/u8:1+events_unbound22:45:001
1788926996457,5cyclictest1835545-21kworker/u8:3+flush-179:021:59:590
4670791630,5ptp4l61690irq/136-eno1-TxRx-019:09:313
1788927996257,4cyclictest1869458-21kworker/u8:2+events_unbound00:04:011
1788927996151,7cyclictest1861242-21latency_hist22:55:011
1788926996153,6cyclictest1822254-21latency_hist20:55:020
1788927996049,8cyclictest1843562-21kworker/u8:0+flush-179:022:55:271
1788926995950,7cyclictest1804738-21latency_hist20:00:010
178892699590,56cyclictest0-21swapper/000:03:580
1788927995848,7cyclictest1799966-21kworker/u8:0+flush-179:019:50:241
1788926995848,8cyclictest1888056-21/usr/sbin/munin00:15:230
1788927995644,9cyclictest1882480-40mandb00:00:011
1788927995644,9cyclictest1882480-40mandb00:00:011
1788926995649,5cyclictest1872399-21apt-get23:30:000
1788927995546,6cyclictest1787181-21kworker/u8:1+flush-179:019:39:591
1788927995445,6cyclictest1889423-21latency_hist00:20:021
1788926995437,15cyclictest723-21Xorg22:05:590
178892699540,52cyclictest723-21Xorg19:42:590
1788927995347,5cyclictest1820701-21kworker/u8:2+flush-179:020:50:231
1788927995344,7cyclictest1869458-21kworker/u8:2+flush-179:023:20:241
1788926995344,6cyclictest1866007-21kworker/u8:0+events_unbound23:10:000
178892699530,51cyclictest723-21Xorg21:35:000
1788927995241,8cyclictest1871046-21/usr/sbin/munin23:25:261
178892699523,44cyclictest1835767-21apt-get21:35:130
1788926995145,5cyclictest1869458-21kworker/u8:2+flush-179:000:05:270
1788926995127,22cyclictest1819685-21fschecks_time20:45:180
1788927995036,10cyclictest1827054-21apt-get21:10:001
178892699502,44cyclictest1844001-21apt-get22:00:170
1788927994946,2cyclictest1787181-21kworker/u8:1+events_unbound19:25:251
1788927994944,3cyclictest1869458-21kworker/u8:2+events_unbound00:20:201
1788927994944,3cyclictest1869458-21kworker/u8:2+events_unbound00:20:201
1788927994940,7cyclictest1808592-21kworker/u8:1+flush-179:020:15:001
1788926994916,31cyclictest723-21Xorg20:42:580
178892699490,45cyclictest1840307-21apt-get21:50:020
1788926994838,8cyclictest1834160-21kworker/u8:1+flush-179:021:45:010
1788926994837,9cyclictest1832592-21apt-get21:25:120
1788926994837,9cyclictest1816141-21apt-get20:35:130
1788926994829,17cyclictest1803182-21sh19:55:000
178892699481,43cyclictest1856739-21apt-get22:40:140
178892699480,47cyclictest0-21swapper/000:14:550
1788926994738,6cyclictest1869458-21kworker/u8:2+flush-179:023:30:020
1788926994738,6cyclictest1869458-21kworker/u8:2+flush-179:023:30:010
1788926994716,17cyclictest0-21swapper/023:00:010
1788930994641,2cyclictest0-21swapper/200:03:582
178893399450,4cyclictest1803156-21turbostat19:55:003
1788927994540,4cyclictest1814332-21kworker/u8:3+events_unbound20:40:171
1788927994537,6cyclictest1792006-21latency_hist19:20:001
1788933994441,2cyclictest0-21swapper/300:03:583
1788927994439,4cyclictest1834160-21kworker/u8:1+events_unbound22:20:271
1788927994438,5cyclictest1834160-21kworker/u8:1+flush-179:021:55:231
1788927994437,5cyclictest271-21systemd-journal20:07:441
178892799440,43cyclictest723-21Xorg22:32:001
1788926994436,6cyclictest1822490-21kworker/u8:0+events_unbound21:00:010
1788926994435,7cyclictest1867612-21latency_hist23:15:000
178892699442,39cyclictest1858074-21turbostat22:49:550
1788927994337,5cyclictest1852744-21kworker/u8:2+events_unbound22:25:231
1788927994337,4cyclictest1862807-21kworker/u8:1+events_unbound23:50:251
1788927994337,4cyclictest1769267-21kworker/u8:2+flush-179:019:10:221
1788927994337,4cyclictest1769267-21kworker/u8:2+flush-179:019:10:221
1788926994338,2cyclictest723-21Xorg00:21:000
1788926994338,2cyclictest723-21Xorg00:20:590
1788926994336,5cyclictest1806352-21turbostat20:09:550
178892699430,42cyclictest723-21Xorg21:22:590
178893099429,2cyclictest0-21swapper/220:42:592
1788927994235,5cyclictest1862807-21kworker/u8:1+flush-179:023:30:201
1788927994235,5cyclictest1862807-21kworker/u8:1+flush-179:023:30:191
1788927994235,5cyclictest1814332-21kworker/u8:3+flush-179:020:55:161
1788927994234,6cyclictest1830459-21kworker/u8:0+flush-179:021:20:161
1788927994230,9cyclictest1820666-21latency_hist20:50:011
178892799422,37cyclictest1811380-21apt-get20:20:151
1788926994233,7cyclictest271-21systemd-journal20:24:300
178892699420,40cyclictest723-21Xorg22:29:000
178893399413,1cyclictest0-21swapper/321:35:003
178893399410,5cyclictest1798386-21turbostat19:40:003
178893399410,40cyclictest0-21swapper/300:10:003
178893399410,39cyclictest1796795-21turbostat19:35:003
178893099413,29cyclictest351rcuc/221:35:002
1788927994137,3cyclictest1869458-21kworker/u8:2+flush-179:000:30:291
1788927994136,3cyclictest1830459-21kworker/u8:0+flush-179:021:30:011
1788927994131,8cyclictest1862807-21kworker/u8:1+events_unbound23:05:261
178892799410,5cyclictest1894213-21turbostat00:39:551
1788926994134,5cyclictest1888055-21kworker/u8:1+flush-179:000:35:000
1788926994132,7cyclictest271-21systemd-journal21:20:010
1788926994130,8cyclictest1824117-21apt-get21:00:130
178892699412,2cyclictest151ktimers/022:10:580
178892699411,38cyclictest1815894-21latency_hist20:35:000
178893399409,2cyclictest0-21swapper/320:42:593
1788933994036,2cyclictest0-21swapper/322:54:543
1788933994033,5cyclictest1815888-21turbostat20:35:003
178893399400,38cyclictest1887830-21turbostat00:15:003
1788927994033,6cyclictest1827531-21/usr/sbin/munin21:10:261
1788927994029,9cyclictest1838962-21apt-get21:45:131
1788926994036,3cyclictest1892636-21systemd-run00:30:000
1788926994036,2cyclictest0-21swapper/022:34:590
1788926994035,4cyclictest1788866-21kworker/u8:3+events_unbound19:40:000
178892699401,2cyclictest1795197-21munin-run19:30:010
178893399390,4cyclictest1814313-21turbostat20:30:003
178893399390,38cyclictest0-21swapper/322:37:583
178893099390,36cyclictest0-21swapper/222:37:582
178893099390,36cyclictest0-21swapper/200:34:552
178893099390,36cyclictest0-21swapper/200:14:562
178893099390,2cyclictest0-21swapper/223:37:582
1788927993936,2cyclictest723-21Xorg23:48:001
1788927993935,3cyclictest723-21Xorg22:17:001
1788927993933,4cyclictest1843562-21kworker/u8:0+flush-179:022:35:171
1788927993932,5cyclictest1814332-21kworker/u8:3+flush-179:021:15:221
178892799390,2cyclictest723-21Xorg22:06:581
1788926993935,2cyclictest1861255-21ls22:55:000
1788926993922,16cyclictest1840485-21latency_hist21:50:010
1788933993835,2cyclictest0-21swapper/321:54:593
1788933993833,4cyclictest0-21swapper/319:42:593
1788930993836,1cyclictest0-21swapper/221:54:592
1788930993834,3cyclictest0-21swapper/219:42:592
1788930993834,2cyclictest0-21swapper/222:54:552
1788930993834,2cyclictest0-21swapper/220:11:592
1788927993835,2cyclictest0-21swapper/121:02:591
1788927993831,5cyclictest1866007-21kworker/u8:0+events_unbound23:10:151
1788927993823,12cyclictest1847175-21apt-get22:10:131
178892799380,37cyclictest0-21swapper/100:26:591
1788926993833,3cyclictest723-21Xorg23:02:000
1788926993821,15cyclictest0-21swapper/019:30:260
178892699381,23cyclictest2523597-21snmpd23:16:000
178892699380,37cyclictest0-21swapper/023:21:590
178892699380,2cyclictest0-21swapper/020:11:590
1788933993733,3cyclictest0-21swapper/323:21:593
1788933993732,3cyclictest0-21swapper/322:29:003
1788930993734,2cyclictest0-21swapper/219:58:592
1788930993733,2cyclictest0-21swapper/223:21:582
1788930993733,2cyclictest0-21swapper/222:09:562
1788927993731,4cyclictest1834160-21kworker/u8:1+flush-179:021:39:561
1788927993731,4cyclictest1834160-21kworker/u8:1+events_unbound22:00:181
1788927993726,8cyclictest1884397-21apt-get00:05:021
1788926993718,17cyclictest1793395-21timerwakeupswit19:20:250
178892699371,22cyclictest2523597-21snmpd23:44:500
178892699370,33cyclictest1789824-21/usr/sbin/munin19:10:210
178892699370,33cyclictest1789824-21/usr/sbin/munin19:10:210
178892699370,23cyclictest1895841-21munin-run00:40:010
1788933993634,1cyclictest0-21swapper/320:11:593
1788933993634,1cyclictest0-21swapper/319:58:593
1788933993633,2cyclictest0-21swapper/322:09:563
1788933993633,2cyclictest0-21swapper/322:04:553
178893399360,35cyclictest0-21swapper/300:29:593
178893399360,1cyclictest0-21swapper/323:37:593
1788930993633,1cyclictest0-21swapper/220:35:002
1788930993632,2cyclictest0-21swapper/222:29:002
1788930993632,2cyclictest0-21swapper/222:04:542
1788927993630,5cyclictest1808592-21kworker/u8:1+flush-179:020:30:201
1788927993628,5cyclictest1832592-21apt-get21:25:141
1788933993532,2cyclictest0-21swapper/300:21:003
1788933993532,2cyclictest0-21swapper/300:20:593
1788933993531,2cyclictest0-21swapper/319:13:593
1788933993531,2cyclictest0-21swapper/319:13:593
178893399350,1cyclictest0-21swapper/300:34:553
1788930993532,2cyclictest0-21swapper/200:20:592
1788930993532,2cyclictest0-21swapper/200:20:592
1788930993531,2cyclictest0-21swapper/223:56:002
1788930993531,2cyclictest0-21swapper/223:55:592
1788930993531,2cyclictest0-21swapper/219:13:592
1788930993531,2cyclictest0-21swapper/219:13:592
178893099350,1cyclictest0-21swapper/220:24:552
1788927993531,3cyclictest1788866-21kworker/u8:3+flush-179:019:40:261
1788927993529,4cyclictest1812965-21apt-get20:25:141
178893399340,4cyclictest1869221-21turbostat23:20:013
1788930993430,2cyclictest0-21swapper/220:59:562
1788930993430,2cyclictest0-21swapper/220:25:592
1788930993429,2cyclictest0-21swapper/222:35:002
1788927993426,5cyclictest1808592-21kworker/u8:1+flush-179:020:15:251
1788926993430,3cyclictest1875887-21systemd-run23:40:000
178892699341,29cyclictest0-21swapper/019:15:160
178892699340,22cyclictest0-21swapper/019:45:210
1788930993330,2cyclictest0-21swapper/222:40:142
1788930993329,2cyclictest0-21swapper/220:09:552
1788927993326,5cyclictest1882630-21kworker/u8:0+flush-179:000:10:201
1788927993325,6cyclictest1835545-21kworker/u8:3+flush-179:021:40:211
178892699333,29cyclictest1806427-21latency_hist20:05:000
1788926993324,7cyclictest1848743-21/usr/sbin/munin22:15:250
1788926993321,9cyclictest1827836-21diskstats21:10:180
178892699331,25cyclictest0-21swapper/022:35:120
1788933993230,1cyclictest0-21swapper/323:55:593
1788933993230,1cyclictest0-21swapper/323:55:593
1788933993230,1cyclictest0-21swapper/322:40:153
1788933993230,1cyclictest0-21swapper/320:59:563
1788933993229,2cyclictest0-21swapper/322:34:593
1788930993228,2cyclictest0-21swapper/223:45:142
178893099320,29cyclictest0-21swapper/222:20:002
1788926993213,17cyclictest0-21swapper/020:30:010
178893399313,1cyclictest0-21swapper/323:45:153
1788933993129,1cyclictest0-21swapper/320:09:563
178893399310,29cyclictest0-21swapper/300:15:133
1788927993125,4cyclictest1869458-21kworker/u8:2+flush-179:023:40:001
1788933993026,3cyclictest0-21swapper/323:02:003
1788933993026,2cyclictest0-21swapper/320:52:593
178893399300,1cyclictest0-21swapper/322:19:593
1788930993027,2cyclictest0-21swapper/223:16:592
1788930993027,2cyclictest0-21swapper/220:52:592
1788930993027,2cyclictest0-21swapper/219:45:152
1788930993026,2cyclictest0-21swapper/223:01:592
178893099300,27cyclictest0-21swapper/221:20:032
1788927993023,5cyclictest723-21Xorg19:49:591
1788933992924,3cyclictest0-21swapper/319:28:593
178893399290,28cyclictest0-21swapper/321:40:153
178893399290,28cyclictest0-21swapper/321:20:023
178893399290,28cyclictest0-21swapper/319:45:153
178893399290,27cyclictest0-21swapper/323:30:043
178893399290,27cyclictest0-21swapper/323:30:033
1788930992926,2cyclictest0-21swapper/223:30:032
1788930992926,2cyclictest0-21swapper/223:30:032
1788930992926,2cyclictest0-21swapper/221:40:152
1788930992926,2cyclictest0-21swapper/200:05:022
1788930992925,2cyclictest0-21swapper/219:35:162
178893399280,27cyclictest0-21swapper/319:15:163
178893399280,26cyclictest0-21swapper/320:02:593
1788930992825,2cyclictest0-21swapper/221:10:162
1788930992824,2cyclictest0-21swapper/219:28:592
1788930992824,2cyclictest0-21swapper/200:37:592
1788930992823,2cyclictest0-21swapper/222:49:002
178893099280,2cyclictest0-21swapper/219:15:172
178893099280,26cyclictest0-21swapper/200:15:132
178893099280,25cyclictest0-21swapper/223:26:002
178893099280,25cyclictest0-21swapper/223:05:592
178893099280,25cyclictest0-21swapper/221:15:592
178893099280,19cyclictest0-21swapper/222:59:552
1788927992824,2cyclictest723-21Xorg22:49:591
4670791270,6ptp4l61690irq/136-eno1-TxRx-020:50:013
1788933992725,1cyclictest0-21swapper/323:14:593
1788933992724,2cyclictest0-21swapper/322:13:593
1788933992724,2cyclictest0-21swapper/320:20:593
178893399270,26cyclictest0-21swapper/323:51:593
1788930992724,2cyclictest0-21swapper/220:36:002
1788930992724,2cyclictest0-21swapper/200:26:002
1788930992723,2cyclictest0-21swapper/220:45:592
178893099270,25cyclictest0-21swapper/219:22:592
178893099270,24cyclictest0-21swapper/223:51:582
178893099270,24cyclictest0-21swapper/220:02:592
178892699276,18cyclictest1789009-21latency_hist19:10:020
4670791260,6ptp4l441ktimers/322:00:003
4670791260,6ptp4l441ktimers/321:30:013
4670791260,6ptp4l441ktimers/320:20:013
1788933992624,1cyclictest0-21swapper/320:36:003
1788933992623,2cyclictest0-21swapper/322:49:003
178893399260,2cyclictest1827043-21turbostat21:10:003
178893399260,25cyclictest0-21swapper/321:15:593
1788930992624,1cyclictest0-21swapper/223:14:592
1788930992623,2cyclictest0-21swapper/222:13:592
1788930992622,2cyclictest0-21swapper/221:37:592
4670791250,6ptp4l61690irq/136-eno1-TxRx-023:45:003
4670791250,6ptp4l61690irq/136-eno1-TxRx-021:15:013
4670791250,6ptp4l441ktimers/323:00:003
4670791250,6ptp4l441ktimers/322:25:013
4670791250,5ptp4l441ktimers/321:04:593
1788933992523,1cyclictest0-21swapper/323:25:153
1788933992523,1cyclictest0-21swapper/300:37:593
1788933992522,2cyclictest0-21swapper/321:37:583
1788933992521,2cyclictest0-21swapper/319:21:193
178893399250,24cyclictest0-21swapper/323:05:583
1788930992521,2cyclictest0-21swapper/221:29:542
1788930992521,2cyclictest0-21swapper/219:54:552
4670791240,6ptp4l61690irq/136-eno1-TxRx-021:50:003
1788930992320,2cyclictest0-21swapper/220:19:592
1788930992018,1cyclictest0-21swapper/219:10:022
1788930992016,2cyclictest0-21swapper/223:42:002
1788930991915,2cyclictest0-21swapper/221:58:592
1788927991714,2cyclictest0-21swapper/119:10:031
1788930991511,2cyclictest0-21swapper/221:48:592
1788930991411,2cyclictest0-21swapper/222:20:582
178893099117,2cyclictest0-21swapper/219:35:002
17889309990,8cyclictest0-21swapper/221:00:332
17889309986,1cyclictest0-21swapper/221:09:592
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional