You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-25 - 21:09
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot6.osadl.org (updated Tue Nov 25, 2025 12:44:55)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
17931609910190,8cyclictest1867283-21latency_hist11:00:010
17931609910094,5cyclictest1788710-21kworker/u8:0+flush-179:007:20:010
17931609910094,5cyclictest1788710-21kworker/u8:0+flush-179:007:20:000
1793160998371,9cyclictest1864109-21latency_hist10:50:010
1793160998070,8cyclictest1857987-21/usr/sbin/munin10:30:250
1793160997766,8cyclictest1868085-21journal-offline11:00:190
1793162997469,4cyclictest1831787-21apt-get09:10:041
1793162997469,4cyclictest1831787-21apt-get09:10:031
1793160997468,3cyclictest1851106-21kworker/u8:0+events_unbound10:25:130
1793162997262,8cyclictest1839985-21latency_hist09:35:011
1793160997266,4cyclictest1861179-21kworker/u8:3+events_unbound10:45:010
1793162997161,7cyclictest1875253-21latency_hist11:25:011
1793160996962,4cyclictest1867538-21kworker/u8:2+flush-179:011:20:010
1793160996860,5cyclictest1793075-21kworker/u8:3+events_unbound07:40:010
1793160996760,5cyclictest1807417-21kworker/u8:1+flush-179:008:10:250
1793162996663,2cyclictest0-21swapper/108:06:001
1793160996660,4cyclictest1818765-21kworker/u8:2+flush-179:008:30:170
1793160996557,5cyclictest1873701-21kworker/u8:1+events_unbound11:40:010
1793166996458,4cyclictest1868891-21turbostat11:05:003
179316599630,61cyclictest0-21swapper/207:48:002
179316599630,61cyclictest0-21swapper/207:47:592
1793160996355,6cyclictest1809245-21apt-get08:00:140
1793160996251,8cyclictest1854816-21/usr/sbin/munin10:20:230
1793160996154,5cyclictest1892973-21kworker/u8:0+events_unbound12:20:290
1793160996154,5cyclictest1884832-21kworker/u8:2+flush-179:012:30:010
1793160996153,5cyclictest1861179-21kworker/u8:3+flush-179:011:05:200
1793162996050,7cyclictest1793020-21latency_hist07:10:011
1793162995953,4cyclictest1793075-21kworker/u8:3+events_unbound07:50:191
1793160995953,5cyclictest1830425-21kworker/u8:1+events_unbound09:15:250
1793160995953,4cyclictest1836295-21kworker/u8:3+flush-179:009:40:220
1793160995948,8cyclictest1872059-21latency_hist11:15:010
1793162995843,14cyclictest1839973-21kworker/u8:1+flush-179:010:45:001
1793160995852,5cyclictest1793075-21kworker/u8:3+flush-179:007:25:210
1793160995846,9cyclictest1811118-21diskstats08:05:170
1793160995846,5cyclictest1821873-21kworker/u8:3+events_unbound09:00:000
1793162995755,1cyclictest0-21swapper/108:35:591
1793162995751,4cyclictest723-21Xorg11:47:591
1793162995751,4cyclictest723-21Xorg11:47:591
1793162995748,7cyclictest1890061-21/usr/sbin/munin12:10:171
1793166995650,5cyclictest1891427-21turbostat12:15:013
1793166995649,4cyclictest1799431-21turbostat07:29:593
1793162995645,9cyclictest1843719-21diskstats09:45:171
1793160995649,5cyclictest1836295-21kworker/u8:3+flush-179:010:05:120
1793160995648,5cyclictest1801035-21kworker/u8:2+events_unbound07:45:240
1793160995648,5cyclictest1801035-21kworker/u8:2+events_unbound07:45:230
1793162995545,7cyclictest1835465-21/usr/sbin/munin09:20:231
1793162995545,7cyclictest1801000-21latency_hist07:35:011
1793162995543,9cyclictest1889565-21latency_hist12:10:011
1793160995543,9cyclictest271-21systemd-journal11:25:000
1793160995448,4cyclictest1873701-21kworker/u8:1+flush-179:011:30:000
1793160995448,4cyclictest1843219-21kworker/u8:0+flush-179:009:50:240
1793160995445,6cyclictest1881651-21kworker/u8:3+flush-179:012:05:010
1793160995443,8cyclictest1807382-21latency_hist07:55:010
1793162995347,4cyclictest723-21Xorg11:40:591
1793160995346,5cyclictest1821873-21kworker/u8:3+flush-179:008:40:250
1793162995242,7cyclictest1794635-21latency_hist07:15:011
1793160995245,5cyclictest1825441-21kworker/u8:0+flush-179:009:00:200
1793160995245,4cyclictest1895658-21kworker/u8:3+events_unbound12:35:000
1793160995244,6cyclictest1859590-21apt-get10:35:120
1793162995145,4cyclictest1816320-21kworker/u8:0+flush-179:008:25:001
1793162995144,5cyclictest1839973-21kworker/u8:1+flush-179:010:00:001
1793160995144,5cyclictest1801035-21kworker/u8:2+events_unbound07:40:250
1793160995041,7cyclictest1816320-21kworker/u8:0+events_unbound08:25:250
1793160995041,6cyclictest1873701-21kworker/u8:1+flush-179:011:40:200
1793166994947,1cyclictest0-21swapper/308:54:593
1793162994942,6cyclictest1839973-21kworker/u8:1+flush-179:010:50:001
1793160994945,3cyclictest1823574-21apt-get08:45:040
1793160994943,4cyclictest1788710-21kworker/u8:0+flush-179:007:35:000
1793160994941,6cyclictest1839973-21kworker/u8:1+flush-179:009:35:210
4670791480,13ptp4l61690irq/136-eno1-TxRx-007:10:003
1793160994831,9cyclictest1821708-21apt-get08:40:000
1793162994738,6cyclictest1816320-21kworker/u8:0+events_unbound08:35:001
1793160994735,4cyclictest1851106-21kworker/u8:0+events_unbound10:20:010
179316299462,3cyclictest1828857-21apt-get09:00:161
1793160994639,5cyclictest271-21systemd-journal08:24:470
1793160994637,7cyclictest1825646-21/usr/sbin/munin08:50:160
1793160994633,9cyclictest271-21systemd-journal08:15:270
179316599450,42cyclictest0-21swapper/207:34:542
1793162994539,5cyclictest1836295-21kworker/u8:3+flush-179:010:00:251
1793162994538,5cyclictest1826994-21latency_hist08:55:001
1793162994523,5cyclictest1893276-21apt-get12:20:171
179316299450,2cyclictest723-21Xorg09:19:591
179316299450,21cyclictest0-21swapper/111:50:121
1793160994540,3cyclictest1835184-21kworker/u8:2+flush-179:009:35:010
1793160994539,4cyclictest1838379-21kworker/u8:0+flush-179:009:30:000
1793160994539,4cyclictest1825441-21kworker/u8:0+flush-179:009:10:220
1793160994539,4cyclictest1825441-21kworker/u8:0+flush-179:009:10:220
1793162994440,3cyclictest1870517-21kworker/u8:0+flush-179:011:30:211
1793162994439,4cyclictest723-21Xorg11:06:591
1793162994432,9cyclictest1814002-21apt-get08:15:111
1793162994426,2cyclictest723-21Xorg09:53:591
1793160994440,3cyclictest1884832-21kworker/u8:2+flush-179:012:15:170
1793160994433,6cyclictest0-21swapper/007:59:590
179316599430,41cyclictest0-21swapper/210:43:592
1793162994338,4cyclictest1801035-21kworker/u8:2+flush-179:007:45:001
1793162994320,5cyclictest271-21systemd-journal10:18:271
1793166994240,1cyclictest0-21swapper/311:56:583
1793162994234,4cyclictest1821873-21kworker/u8:3+flush-179:008:40:191
1793162994232,7cyclictest1813748-21latency_hist08:15:011
1793160994235,5cyclictest1889575-21apt-get12:10:030
1793160994232,8cyclictest1851639-21apt-get10:10:110
1793162994136,4cyclictest1794889-21apt-get07:15:151
1793162994136,4cyclictest1794889-21apt-get07:15:151
1793162994135,5cyclictest1807417-21kworker/u8:1+flush-179:007:55:251
1793162994134,6cyclictest1867538-21kworker/u8:2+events_unbound11:10:231
179316299410,39cyclictest1851135-21turbostat10:14:551
1793160994136,4cyclictest1873701-21kworker/u8:1+flush-179:011:45:210
1793160994136,4cyclictest1873701-21kworker/u8:1+flush-179:011:45:210
1793160994130,8cyclictest1865695-21latency_hist10:55:010
1793166994037,2cyclictest0-21swapper/311:24:593
179316599400,37cyclictest0-21swapper/209:04:552
1793162994033,4cyclictest1840242-21apt-get09:35:121
1793162994032,6cyclictest1867538-21kworker/u8:2+flush-179:011:15:221
1793160994035,3cyclictest723-21Xorg11:56:580
179316699390,38cyclictest1883211-21turbostat11:50:003
179316699390,38cyclictest1883211-21turbostat11:50:003
1793166993833,4cyclictest1867299-21turbostat11:00:003
1793166993832,4cyclictest1860929-21turbostat10:40:003
1793162993834,2cyclictest723-21Xorg10:23:591
1793162993829,6cyclictest1825408-21latency_hist08:50:001
1793162993827,8cyclictest1886649-21apt-get12:00:161
179316299380,37cyclictest0-21swapper/110:07:591
179316699370,36cyclictest0-21swapper/307:17:003
179316699370,36cyclictest0-21swapper/307:16:593
179316699370,35cyclictest0-21swapper/307:55:593
1793165993734,2cyclictest0-21swapper/210:26:592
1793165993734,2cyclictest0-21swapper/208:26:582
1793165993734,2cyclictest0-21swapper/207:36:592
179316599370,35cyclictest0-21swapper/212:14:002
179316599370,35cyclictest0-21swapper/212:02:582
179316599370,35cyclictest0-21swapper/211:38:592
179316599370,35cyclictest0-21swapper/208:49:592
1793162993734,2cyclictest0-21swapper/111:36:581
1793162993731,5cyclictest1884832-21kworker/u8:2+flush-179:012:15:291
179316299372,31cyclictest1865951-21apt-get10:55:151
1793160993734,2cyclictest1844761-21/usr/sbin/munin09:50:000
1793160993730,5cyclictest1895658-21kworker/u8:3+flush-179:012:35:260
1793160993730,5cyclictest1895658-21kworker/u8:3+flush-179:012:35:250
1793160993725,9cyclictest1889575-21apt-get12:10:010
1793166993634,1cyclictest0-21swapper/312:32:593
179316699360,35cyclictest0-21swapper/311:43:583
1793165993633,2cyclictest0-21swapper/211:56:592
1793165993633,2cyclictest0-21swapper/210:39:552
1793165993633,2cyclictest0-21swapper/210:24:552
179316599360,1cyclictest0-21swapper/210:06:592
179316599360,1cyclictest0-21swapper/208:34:562
1793162993631,3cyclictest1797825-21cron07:24:591
1793162993626,8cyclictest1831787-21apt-get09:10:001
1793162993626,8cyclictest1831787-21apt-get09:10:001
179316299360,34cyclictest1886432-21munin-run12:00:011
1793160993631,4cyclictest1870517-21kworker/u8:0+flush-179:011:30:280
1793160993631,4cyclictest1825441-21kworker/u8:0+events_unbound09:20:160
1793160993630,4cyclictest1846327-21kworker/u8:2+events_unbound09:55:190
1793160993629,6cyclictest1796293-21kworker/u8:1+flush-179:007:20:210
1793160993629,5cyclictest1793075-21kworker/u8:3+flush-179:007:10:200
1793166993533,1cyclictest0-21swapper/311:29:553
179316699350,1cyclictest0-21swapper/311:10:593
1793165993533,1cyclictest0-21swapper/211:45:012
1793165993531,2cyclictest0-21swapper/207:40:592
1793162993532,2cyclictest0-21swapper/110:50:591
1793162993530,3cyclictest1856417-21apt-get10:25:121
1793160993530,4cyclictest1881651-21kworker/u8:3+events_unbound11:50:180
1793160993527,6cyclictest1825441-21kworker/u8:0+events_unbound09:05:240
1793160993527,6cyclictest1825441-21kworker/u8:0+events_unbound09:05:230
1793166993431,2cyclictest0-21swapper/309:49:553
1793166993429,4cyclictest1873676-21turbostat11:20:013
1793165993431,2cyclictest0-21swapper/207:29:552
1793162993431,3cyclictest281ktimers/112:37:591
1793162993431,3cyclictest281ktimers/112:37:591
1793162993431,2cyclictest723-21Xorg11:27:591
1793162993431,2cyclictest723-21Xorg10:31:591
1793162993429,3cyclictest0-21swapper/107:40:011
1793162993420,11cyclictest1827248-21apt-get08:55:131
1793166993331,1cyclictest0-21swapper/307:54:593
1793166993330,2cyclictest0-21swapper/309:56:593
1793166993327,4cyclictest1884792-21turbostat11:55:003
1793162993329,3cyclictest1838437-21systemd-run09:30:011
1793162993327,5cyclictest1860906-21kworker/u8:2+flush-179:010:40:001
1793162993325,6cyclictest271-21systemd-journal08:30:001
179316299331,29cyclictest1804469-21apt07:45:121
179316299331,29cyclictest1804469-21apt07:45:121
179316299330,29cyclictest0-21swapper/112:25:141
1793166993230,1cyclictest0-21swapper/308:59:593
1793165993230,1cyclictest0-21swapper/210:00:002
1793165993229,2cyclictest0-21swapper/212:05:592
1793165993229,2cyclictest0-21swapper/211:24:562
1793162993228,3cyclictest1897783-21munin-run12:35:001
1793162993227,4cyclictest1810269-21switchtime08:00:271
179316299321,29cyclictest0-21swapper/111:00:141
1793166993129,1cyclictest0-21swapper/308:11:593
1793166993128,2cyclictest0-21swapper/312:25:593
179316699310,1cyclictest0-21swapper/309:04:593
1793165993128,2cyclictest0-21swapper/209:24:562
1793166993027,2cyclictest0-21swapper/309:20:583
1793166993025,2cyclictest0-21swapper/310:30:143
179316699300,2cyclictest0-21swapper/308:00:163
179316699300,28cyclictest0-21swapper/308:40:023
1793165993027,2cyclictest0-21swapper/209:40:142
179316599300,28cyclictest0-21swapper/210:45:132
1793162993028,1cyclictest0-21swapper/107:27:591
1793166992927,1cyclictest0-21swapper/311:37:593
1793166992927,1cyclictest0-21swapper/307:21:003
1793166992926,2cyclictest0-21swapper/309:35:143
1793166992922,5cyclictest1812172-21turbostat08:10:013
179316699290,1cyclictest0-21swapper/308:45:143
1793165992926,1cyclictest0-21swapper/209:39:592
1793165992925,2cyclictest0-21swapper/211:11:582
1793165992925,2cyclictest0-21swapper/210:34:562
179316599290,27cyclictest0-21swapper/212:30:142
179316599290,27cyclictest0-21swapper/210:50:132
179316599290,27cyclictest0-21swapper/210:15:162
179316599290,27cyclictest0-21swapper/209:10:022
179316599290,27cyclictest0-21swapper/209:10:022
179316599290,27cyclictest0-21swapper/208:40:032
179316599290,27cyclictest0-21swapper/208:05:142
179316599290,27cyclictest0-21swapper/207:15:172
179316599290,27cyclictest0-21swapper/207:15:162
1793160992923,5cyclictest1839973-21kworker/u8:1+flush-179:010:00:220
4672291280,26phc2sys0-21swapper/308:30:143
4670791280,7ptp4l61690irq/136-eno1-TxRx-011:35:013
1793166992826,1cyclictest0-21swapper/310:18:593
1793166992825,2cyclictest0-21swapper/309:15:143
179316699280,2cyclictest0-21swapper/312:35:143
179316699280,2cyclictest0-21swapper/312:35:133
179316699280,27cyclictest0-21swapper/312:15:143
179316699280,27cyclictest0-21swapper/312:05:133
179316699280,26cyclictest0-21swapper/308:35:143
1793165992825,2cyclictest0-21swapper/209:00:002
179316599280,26cyclictest0-21swapper/210:12:592
179316599280,26cyclictest0-21swapper/209:45:142
179316599280,26cyclictest0-21swapper/209:34:592
179316599280,1cyclictest0-21swapper/208:03:002
4670791270,6ptp4l441ktimers/309:35:003
4670791270,6ptp4l441ktimers/308:30:013
1793166992725,1cyclictest0-21swapper/309:10:023
1793166992725,1cyclictest0-21swapper/309:10:023
1793166992724,2cyclictest0-21swapper/309:25:143
179316699270,26cyclictest0-21swapper/309:50:163
179316699270,26cyclictest0-21swapper/308:15:133
179316699270,1cyclictest0-21swapper/308:20:153
1793165992724,2cyclictest0-21swapper/212:18:592
1793165992724,2cyclictest0-21swapper/209:29:592
179316599270,1cyclictest0-21swapper/208:36:592
4670791260,6ptp4l61690irq/136-eno1-TxRx-010:25:003
4670791260,6ptp4l441ktimers/311:09:543
4670791260,6ptp4l441ktimers/309:44:563
4670791260,5ptp4l61690irq/136-eno1-TxRx-009:10:023
4670791260,5ptp4l61690irq/136-eno1-TxRx-009:10:013
4670791260,5ptp4l441ktimers/310:45:013
4670791260,5ptp4l441ktimers/310:30:013
1793166992624,1cyclictest0-21swapper/310:45:593
179316699260,24cyclictest0-21swapper/307:10:263
1793165992624,1cyclictest0-21swapper/211:09:592
1793165992623,2cyclictest0-21swapper/211:30:122
1793165992622,2cyclictest0-21swapper/211:15:592
179316599260,2cyclictest0-21swapper/207:50:592
179316599260,24cyclictest0-21swapper/209:06:592
179316599260,24cyclictest0-21swapper/209:06:582
179316599260,24cyclictest0-21swapper/207:21:272
179316599260,24cyclictest0-21swapper/207:10:272
179316599260,1cyclictest0-21swapper/212:28:002
4670791250,6ptp4l61690irq/136-eno1-TxRx-010:15:003
4670791250,6ptp4l441ktimers/307:49:593
4670791250,6ptp4l441ktimers/307:49:593
4670791250,5ptp4l441ktimers/307:44:563
1793166992523,1cyclictest0-21swapper/312:04:553
1793166992523,1cyclictest0-21swapper/310:03:593
179316699250,24cyclictest0-21swapper/307:30:593
1793165992523,1cyclictest0-21swapper/207:55:592
179316599250,2cyclictest0-21swapper/211:48:592
179316599250,2cyclictest0-21swapper/211:48:592
1793162992524,1cyclictest0-21swapper/109:41:591
4670791240,5ptp4l61690irq/136-eno1-TxRx-010:54:553
4670791240,4ptp4l441ktimers/310:09:553
4670791240,4ptp4l441ktimers/307:39:553
1793165992422,1cyclictest0-21swapper/208:20:002
1793165992421,2cyclictest0-21swapper/208:22:582
4670791230,4ptp4l61690irq/136-eno1-TxRx-012:24:553
1793165992320,2cyclictest0-21swapper/210:00:142
1793165992320,2cyclictest0-21swapper/209:51:592
1793165992220,1cyclictest0-21swapper/212:38:592
1793165992220,1cyclictest0-21swapper/212:38:582
1793165992219,2cyclictest0-21swapper/208:50:592
1793165992018,1cyclictest0-21swapper/211:54:592
1793165992017,2cyclictest0-21swapper/211:28:592
1793165992017,2cyclictest0-21swapper/210:59:562
1793165991917,1cyclictest0-21swapper/208:11:582
1793165991614,1cyclictest0-21swapper/211:03:002
1793165991310,2cyclictest0-21swapper/209:16:592
179316099136,5cyclictest1793224-21cat07:10:020
1793165991210,1cyclictest0-21swapper/212:22:582
17931659930,2cyclictest0-21swapper/207:10:022
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional