You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-04 - 20:42
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot6.osadl.org (updated Thu Dec 04, 2025 12:44:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
18418319910377,23cyclictest1934223-21/usr/sbin/munin11:55:230
1841831999180,8cyclictest1848384-21latency_hist07:30:010
1841831998879,7cyclictest1922776-21dpkg11:17:470
1841831998777,8cyclictest1911784-21/usr/sbin/munin10:45:230
1841831998269,9cyclictest1943802-21latency_hist12:25:010
1841833998068,9cyclictest271-21systemd-journal07:50:001
1841833997868,8cyclictest1948828-21apt-get12:40:001
1841833997754,20cyclictest271-21systemd-journal09:40:001
1841833997667,6cyclictest1846799-21latency_hist07:25:011
1841831997465,7cyclictest1907020-21apt-get10:30:110
1841831997361,9cyclictest1867511-21latency_hist08:30:010
1841833997260,10cyclictest1885565-21apt-get09:25:011
1841833997165,4cyclictest1841782-21kworker/u8:3+flush-179:008:20:011
1841840997040,1cyclictest0-21swapper/307:17:593
1841833997063,5cyclictest1917924-21kworker/u8:2+flush-179:011:17:371
1841833996856,10cyclictest1908377-21awk10:35:011
1841831996855,10cyclictest271-21systemd-journal10:10:250
1841831996758,7cyclictest1885552-21latency_hist09:25:000
1841831996658,5cyclictest1931804-21kworker/u8:2+flush-179:012:05:010
1841831996657,7cyclictest1864595-21apt-get08:20:130
1841831996655,8cyclictest1877832-21/usr/sbin/munin09:00:230
1841833996454,8cyclictest1895608-21apt-get09:55:131
1841833996357,4cyclictest1935830-21kworker/u8:1+events_unbound12:20:241
1841831996356,4cyclictest1894249-21kworker/u8:2+events_unbound09:55:140
184183199631,60cyclictest1-21systemd08:05:010
1841831996153,6cyclictest1868849-21kworker/u8:0+events_unbound09:00:010
1841837996058,1cyclictest0-21swapper/208:39:592
1841831996053,5cyclictest1891903-21kworker/u8:1+flush-179:009:45:010
1841831996053,5cyclictest1891903-21kworker/u8:1+flush-179:009:45:000
1841833995949,7cyclictest1849971-21latency_hist07:35:011
1841831995948,9cyclictest1947489-21apt-get12:35:130
1841831995948,8cyclictest1864328-21latency_hist08:20:010
1841833995851,5cyclictest1894249-21kworker/u8:2+events_unbound10:05:251
1841833995849,7cyclictest1895352-21latency_hist09:55:011
1841833995848,7cyclictest1857957-21latency_hist08:00:021
1841831995854,3cyclictest1891903-21kworker/u8:1+flush-179:010:25:260
1841833995755,1cyclictest0-21swapper/110:27:591
1841833995746,10cyclictest1841782-21kworker/u8:3+flush-179:008:20:161
1841837995653,2cyclictest0-21swapper/210:02:582
1841831995648,6cyclictest1859579-21kworker/u8:0+events_unbound08:10:000
1841831995648,6cyclictest1843342-21latency_hist07:15:020
1841833995548,5cyclictest1879419-21kworker/u8:1+flush-179:009:25:181
1841831995550,4cyclictest1935830-21kworker/u8:1+flush-179:012:10:260
1841831995546,7cyclictest271-21systemd-journal11:05:000
1841831995538,15cyclictest1929237-21perf11:40:000
1841831995533,20cyclictest1932525-21latency_hist11:50:000
1841831995533,20cyclictest1932525-21latency_hist11:49:590
1841833995447,6cyclictest1924486-21kworker/u8:0+flush-179:012:00:001
1841833995447,5cyclictest1931051-21/usr/sbin/munin11:45:241
1841833995447,5cyclictest1931051-21/usr/sbin/munin11:45:231
1841833995446,6cyclictest1891903-21kworker/u8:1+events_unbound10:15:011
1841831995444,7cyclictest1874400-21latency_hist08:50:010
1841831995248,3cyclictest723-21Xorg07:30:590
1841833995148,2cyclictest0-21swapper/108:49:591
1841831995128,5cyclictest1851840-21/usr/sbin/munin07:40:190
1841833995044,5cyclictest1891903-21kworker/u8:1+flush-179:010:15:171
1841833995044,4cyclictest1868849-21kworker/u8:0+flush-179:008:35:231
1841831995045,4cyclictest1886147-21kworker/u8:0+flush-179:009:45:170
1841831995043,5cyclictest1894249-21kworker/u8:2+flush-179:010:20:000
1841833994944,3cyclictest1926275-21kworker/u8:1+flush-179:011:30:261
1841833994942,5cyclictest1841782-21kworker/u8:3+flush-179:007:15:181
1841833994940,6cyclictest1924451-21latency_hist11:25:011
1841831994945,3cyclictest1910197-21kworker/u8:1+flush-179:010:56:250
1841831994943,4cyclictest1871488-21kworker/u8:3+flush-179:008:40:190
1841831994942,5cyclictest1856362-21latency_hist07:55:000
1841831994941,5cyclictest1869348-21kworker/u8:2+events_unbound08:35:000
1841833994844,3cyclictest1859579-21kworker/u8:0+flush-179:008:10:271
1841833994744,2cyclictest1894249-21kworker/u8:2+events_unbound10:20:221
1841833994741,4cyclictest1839476-21kworker/u8:0+events_unbound07:35:141
1841831994741,4cyclictest1845677-21kworker/u8:1+events_unbound07:20:230
1841831994740,5cyclictest1879419-21kworker/u8:1+events_unbound09:10:010
4670791460,13ptp4l61690irq/136-eno1-TxRx-007:09:563
1841833994641,4cyclictest1874446-21kworker/u8:2+flush-179:008:55:251
1841833994639,5cyclictest1887434-21kworker/u8:3+flush-179:009:30:181
1841833994638,6cyclictest1875987-21latency_hist08:55:021
1841831994638,6cyclictest1938487-21kworker/u8:0+flush-179:012:35:010
1841833994540,4cyclictest1845677-21kworker/u8:1+flush-179:007:25:241
1841833994540,4cyclictest1839476-21kworker/u8:0+flush-179:007:50:221
1841833994538,5cyclictest1910197-21kworker/u8:1+flush-179:011:00:171
184183199451,42cyclictest1895371-21idleruntime-cro09:55:000
1841837994441,2cyclictest0-21swapper/211:50:592
1841833994439,4cyclictest1939067-21kworker/u8:3+flush-179:012:15:251
1841833994438,4cyclictest1908341-21kworker/u8:3+flush-179:010:35:191
1841833994434,8cyclictest1843360-21awk07:15:011
1841833994340,2cyclictest1862709-21kworker/u8:1+events_unbound08:25:191
1841833994338,4cyclictest1935830-21kworker/u8:1+events_unbound12:10:251
184183399431,40cyclictest1939040-21perf12:10:011
184183399430,42cyclictest0-21swapper/109:10:001
184184099420,40cyclictest1940625-21turbostat12:15:013
1841833994238,3cyclictest1922711-21kworker/u8:3+flush-179:011:25:261
1841833994236,5cyclictest1938487-21kworker/u8:0+events_unbound12:30:231
1841833994236,4cyclictest1926275-21kworker/u8:1+flush-179:011:45:011
184183399420,40cyclictest723-21Xorg11:36:591
184184099410,39cyclictest1927635-21turbostat11:35:003
1841833994138,2cyclictest1894249-21kworker/u8:2+events_unbound10:00:231
1841831994136,4cyclictest1841782-21kworker/u8:3+flush-179:007:15:160
1841831994136,4cyclictest1841782-21kworker/u8:3+events_unbound07:35:240
1841831994130,9cyclictest1899063-21apt-get10:05:130
184183199410,37cyclictest475-21dbus-daemon07:50:010
1841840994037,2cyclictest0-21swapper/309:38:593
184184099400,38cyclictest1905182-21turbostat10:25:003
184183799400,38cyclictest0-21swapper/210:56:592
1841833994034,4cyclictest1916319-21ls11:00:001
1841831994035,4cyclictest723-21Xorg10:44:580
1841831994031,6cyclictest1933987-21latency_hist11:55:010
184183199400,4cyclictest1926050-21turbostat11:34:550
184184099390,4cyclictest1859574-21turbostat08:05:013
184184099390,38cyclictest1856367-21turbostat07:54:593
184183399393,31cyclictest1913388-21apt-get10:50:141
1841833993932,5cyclictest1905209-21kworker/u8:0+flush-179:010:40:161
1841833993931,6cyclictest1893765-21latency_hist09:50:001
1841833993926,10cyclictest1868028-21apt-get08:30:141
184183399390,37cyclictest723-21Xorg09:13:001
184183399390,2cyclictest723-21Xorg11:05:591
184183199397,30cyclictest0-21swapper/011:10:140
1841831993932,5cyclictest1905209-21kworker/u8:0+flush-179:010:55:010
184183199390,2cyclictest0-21swapper/009:36:590
1841840993833,4cyclictest1862746-21turbostat08:14:593
1841833993833,3cyclictest1856083-21kworker/u8:2+flush-179:008:00:191
184183399380,1cyclictest0-21swapper/108:42:591
1841831993835,2cyclictest723-21Xorg12:17:590
184183199380,2cyclictest0-21swapper/011:43:590
1841840993735,1cyclictest0-21swapper/311:29:553
1841840993735,1cyclictest0-21swapper/309:19:563
1841840993735,1cyclictest0-21swapper/309:19:553
1841840993734,2cyclictest0-21swapper/309:48:583
1841840993731,5cyclictest0-21swapper/309:34:563
184183799370,35cyclictest0-21swapper/209:56:592
184183799370,35cyclictest0-21swapper/208:01:592
184183799370,35cyclictest0-21swapper/207:19:002
1841833993730,5cyclictest1917924-21kworker/u8:2+flush-179:011:15:001
1841831993733,3cyclictest1945684-21systemd-run12:30:000
1841840993634,1cyclictest0-21swapper/310:18:593
1841840993634,1cyclictest0-21swapper/309:20:583
1841840993634,1cyclictest0-21swapper/308:07:593
184184099360,35cyclictest0-21swapper/311:38:593
184184099360,1cyclictest0-21swapper/311:03:593
1841837993633,2cyclictest0-21swapper/211:32:592
1841831993632,2cyclictest723-21Xorg09:33:580
1841831993632,2cyclictest723-21Xorg08:36:590
1841831993631,4cyclictest1874446-21kworker/u8:2+flush-179:008:50:250
1841831993630,4cyclictest723-21Xorg09:19:590
1841831993630,4cyclictest723-21Xorg09:19:580
1841840993533,1cyclictest0-21swapper/312:24:553
1841840993533,1cyclictest0-21swapper/311:19:553
184183799350,1cyclictest0-21swapper/211:28:002
184183799350,1cyclictest0-21swapper/211:09:552
1841833993529,5cyclictest1839476-21kworker/u8:0+flush-179:007:40:261
1841833993528,4cyclictest1913145-21perf10:50:001
202199340,4rtkit-daemon636646-21munin-node11:28:240
202199340,4rtkit-daemon2005-21lightdm-gtk-gre10:38:450
1841840993432,1cyclictest0-21swapper/312:00:593
1841840993432,1cyclictest0-21swapper/307:40:593
1841840993431,2cyclictest0-21swapper/311:12:593
184184099340,2cyclictest0-21swapper/307:15:023
1841837993432,1cyclictest0-21swapper/211:37:592
1841837993431,2cyclictest0-21swapper/211:57:002
1841837993431,2cyclictest0-21swapper/209:46:002
1841840993331,1cyclictest0-21swapper/308:28:593
1841833993325,7cyclictest1943812-21apt-get12:25:021
1841831993327,5cyclictest1879419-21kworker/u8:1+events_unbound09:25:270
1841831993327,4cyclictest0-21swapper/010:00:120
184183199330,5cyclictest723-21Xorg08:14:590
1841840993230,1cyclictest0-21swapper/308:50:583
184184099320,1cyclictest0-21swapper/307:58:003
1841837993229,2cyclictest0-21swapper/212:20:592
1841833993229,2cyclictest723-21Xorg11:53:591
1841833993227,4cyclictest1937320-21latency_hist12:05:011
1841833993227,4cyclictest1841782-21kworker/u8:3+flush-179:008:05:171
1841831993228,3cyclictest1919500-21munin-run11:10:000
1841831993222,8cyclictest1935849-21apt-get12:00:120
1841840993125,5cyclictest1932404-21turbostat11:50:003
1841840993125,5cyclictest1932404-21turbostat11:50:003
1841840993124,5cyclictest0-21swapper/309:25:033
184183799310,29cyclictest0-21swapper/210:20:152
1841831993126,4cyclictest1882480-21latency_hist09:15:020
1841840993027,2cyclictest0-21swapper/312:39:003
1841840993027,2cyclictest0-21swapper/308:15:143
1841840993025,4cyclictest1896967-21turbostat10:00:003
1841840993025,4cyclictest0-21swapper/309:50:143
1841837993027,2cyclictest0-21swapper/211:17:402
1841837993027,2cyclictest0-21swapper/209:00:142
1841837993027,2cyclictest0-21swapper/207:47:002
184183799300,2cyclictest0-21swapper/212:15:132
1841833993024,5cyclictest1891267-21grep09:40:171
1841833993024,5cyclictest1891267-21grep09:40:161
1841833993023,5cyclictest1882592-21kworker/u8:2+flush-179:009:20:001
1841833993023,5cyclictest1882592-21kworker/u8:2+flush-179:009:20:001
1841831993024,4cyclictest1858133-21latency_hist08:00:020
1841831993023,5cyclictest1891903-21kworker/u8:1+flush-179:010:25:000
4670791290,7ptp4l441ktimers/307:40:003
184184099290,27cyclictest0-21swapper/311:55:133
184184099290,21cyclictest0-21swapper/310:45:133
1841837992926,2cyclictest0-21swapper/212:00:142
1841837992926,2cyclictest0-21swapper/211:20:162
1841837992926,2cyclictest0-21swapper/210:34:552
1841837992926,2cyclictest0-21swapper/210:25:152
1841837992926,2cyclictest0-21swapper/209:43:592
1841837992926,2cyclictest0-21swapper/209:43:592
1841837992926,2cyclictest0-21swapper/209:20:132
1841837992926,2cyclictest0-21swapper/207:50:142
184183799290,28cyclictest0-21swapper/212:25:032
184183799290,27cyclictest0-21swapper/208:30:022
184183799290,27cyclictest0-21swapper/207:25:142
184183399290,27cyclictest723-21Xorg09:02:581
1841831992926,2cyclictest0-21swapper/011:24:590
1841840992826,1cyclictest0-21swapper/310:31:593
1841840992826,1cyclictest0-21swapper/310:29:553
1841840992825,2cyclictest0-21swapper/312:30:153
184184099280,27cyclictest0-21swapper/310:00:123
184184099280,27cyclictest0-21swapper/309:40:133
184184099280,27cyclictest0-21swapper/309:40:123
184184099280,27cyclictest0-21swapper/308:55:163
184184099280,26cyclictest0-21swapper/312:05:013
184184099280,26cyclictest0-21swapper/308:40:033
184184099280,26cyclictest0-21swapper/307:30:123
1841837992825,2cyclictest0-21swapper/210:35:152
1841837992825,2cyclictest0-21swapper/209:10:592
1841837992825,2cyclictest0-21swapper/207:57:592
1841837992825,2cyclictest0-21swapper/207:35:132
184183799280,2cyclictest0-21swapper/208:40:022
184183799280,20cyclictest0-21swapper/207:32:002
184183799280,1cyclictest0-21swapper/211:14:592
4670791270,7ptp4l441ktimers/308:50:013
1841840992725,1cyclictest0-21swapper/308:20:143
1841837992724,2cyclictest0-21swapper/208:51:582
184183799270,25cyclictest0-21swapper/210:43:592
184183799270,1cyclictest0-21swapper/212:10:142
4670791260,7ptp4l441ktimers/311:25:003
4670791260,6ptp4l441ktimers/310:45:003
4670791260,6ptp4l441ktimers/310:40:003
4670791260,6ptp4l441ktimers/309:05:003
4670791260,6ptp4l441ktimers/307:20:013
1841840992624,1cyclictest0-21swapper/310:54:553
1841840992624,1cyclictest0-21swapper/309:05:143
1841840992624,1cyclictest0-21swapper/308:40:003
1841840992623,2cyclictest0-21swapper/307:45:123
184183799260,2cyclictest0-21swapper/212:06:592
184183799260,24cyclictest0-21swapper/210:14:592
184183799260,24cyclictest0-21swapper/207:21:582
4670791250,6ptp4l61690irq/136-eno1-TxRx-011:10:003
4670791250,6ptp4l441ktimers/310:15:003
4670791250,6ptp4l441ktimers/309:14:563
4670791250,5ptp4l441ktimers/312:19:563
4670791250,5ptp4l441ktimers/311:45:013
1841840992523,1cyclictest0-21swapper/312:25:003
184184099250,24cyclictest0-21swapper/310:56:593
1841837992522,2cyclictest0-21swapper/209:25:592
1841837992522,2cyclictest0-21swapper/207:10:242
184183799250,2cyclictest0-21swapper/208:29:562
4670791240,5ptp4l61690irq/136-eno1-TxRx-007:29:563
4670791240,4ptp4l441ktimers/311:54:553
1841840992422,1cyclictest0-21swapper/310:05:143
1841840992422,1cyclictest0-21swapper/308:32:583
1841837992422,1cyclictest0-21swapper/209:40:002
1841837992421,2cyclictest0-21swapper/210:46:592
1841831992420,3cyclictest1841841-21latency_hist07:10:020
1841837992321,1cyclictest0-21swapper/208:15:592
1841837992320,2cyclictest0-21swapper/210:15:582
1841837992320,2cyclictest0-21swapper/208:10:132
202199220,13rtkit-daemon485-21gpm07:05:261
1841837992220,1cyclictest0-21swapper/210:54:592
1841837992220,1cyclictest0-21swapper/209:31:582
1841837992219,2cyclictest0-21swapper/208:58:592
1841837992119,1cyclictest0-21swapper/209:08:592
1841837992018,1cyclictest0-21swapper/208:22:592
1841837991917,1cyclictest0-21swapper/211:40:592
1841837991917,1cyclictest0-21swapper/207:40:152
1841837991916,2cyclictest0-21swapper/208:50:012
1841837991715,1cyclictest0-21swapper/212:32:582
1841837991714,2cyclictest0-21swapper/208:05:592
1841837991613,2cyclictest0-21swapper/209:15:022
1841837991613,2cyclictest0-21swapper/209:15:012
1841837991513,1cyclictest0-21swapper/211:45:142
1841837991513,1cyclictest0-21swapper/211:45:142
1841837991412,1cyclictest0-21swapper/211:01:592
1841837991210,1cyclictest0-21swapper/209:52:592
184183799118,2cyclictest0-21swapper/210:06:002
18418379985,2cyclictest0-21swapper/212:36:352
18418379930,2cyclictest0-21swapper/207:10:022
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional