You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-19 - 19:47

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot4s.osadl.org (updated Fri Jun 19, 2026 12:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1728208994538,5cyclictest335557-21systemd-journal11:30:100
1728223994239,2cyclictest75-21kswapd011:30:173
1728215994238,3cyclictest1982939-21kworker/u8:5+flush-259:011:30:221
1728223994135,4cyclictest2046207-21kworker/u8:2+flush-259:012:25:153
1728208993935,3cyclictest1962247-21kworker/u8:2+flush-259:011:05:000
1728208993630,4cyclictest1816244-21latency_hist08:35:010
1728220993531,3cyclictest335557-21systemd-journal08:10:192
1728215993531,3cyclictest1733549-21kworker/u8:4+flush-259:007:40:211
1728215993430,3cyclictest1878457-21kworker/u8:2+flush-259:009:55:211
1728220993329,3cyclictest1878457-21kworker/u8:2+flush-259:010:20:012
1728215993330,2cyclictest1899180-21kworker/u8:3+events_unbound10:20:211
1728220993229,2cyclictest1936455-21kworker/u8:0+flush-259:011:29:592
1728220993229,2cyclictest1852312-21kworker/u8:0+flush-259:009:35:152
1728215993229,2cyclictest2046207-21kworker/u8:2+events_unbound12:25:141
1728215993228,3cyclictest1982939-21kworker/u8:5+flush-259:011:45:001
1728215993227,4cyclictest1842215-21kworker/u8:3+flush-259:009:10:001
1728215993227,3cyclictest2046241-21apt-get12:15:121
1728223993124,5cyclictest1837248-21diskstats08:55:143
1728215993129,1cyclictest1899180-21kworker/u8:3+flush-259:010:30:231
1728208993127,3cyclictest1899180-21kworker/u8:3+flush-259:010:45:000
1728208993124,5cyclictest1982047-21ntpq11:10:210
1728208993124,5cyclictest1982047-21ntpq11:10:200
1728220993025,3cyclictest1952151-21kworker/u8:1+flush-259:011:10:162
1728220993025,3cyclictest1952151-21kworker/u8:1+flush-259:011:10:152
1728215993027,2cyclictest1775157-21kworker/u8:1+flush-259:008:25:201
1728215993026,3cyclictest2014705-21kworker/u8:2+flush-259:011:50:301
1728220992923,4cyclictest1888833-21latency_hist09:45:002
1728220992922,5cyclictest1825685-21ntpq08:40:212
1728215992924,4cyclictest2014705-21kworker/u8:2+events_unbound12:03:331
1728215992924,3cyclictest1744297-21kworker/u8:0+events_unbound08:00:011
1728208992923,4cyclictest1-21systemd10:05:300
1728223992822,4cyclictest2025528-21kworker/u8:3+flush-259:012:05:183
1728223992822,4cyclictest1926186-21latency_hist10:20:023
1728220992823,4cyclictest1936455-21kworker/u8:0+flush-259:011:35:162
1728215992825,2cyclictest1820887-21kworker/u8:4+events_unbound09:20:211
1728215992821,5cyclictest2055417-21ntpq12:20:201
1728215992820,5cyclictest2040873-21latency_hist12:10:001
1728208992824,3cyclictest1993599-21apt-get11:25:110
1728208992822,4cyclictest335557-21systemd-journal09:02:210
1728208992822,4cyclictest1946732-21latency_hist10:40:010
1728220992720,5cyclictest1999202-21dpkg11:30:162
1728215992724,2cyclictest1733549-21kworker/u8:4+events_unbound08:20:231
1728215992723,3cyclictest2046207-21kworker/u8:2+events_unbound12:39:261
1728215992723,3cyclictest1842215-21kworker/u8:3+flush-259:009:15:251
1728215992723,3cyclictest1733549-21kworker/u8:4+events_unbound08:00:271
1728215992723,2cyclictest1775157-21kworker/u8:1+flush-259:008:15:151
1728215992721,4cyclictest1962219-21latency_hist10:55:011
1728220992619,5cyclictest335557-21systemd-journal09:30:302
1728215992622,3cyclictest2000032-21kworker/u8:3+events_unbound11:39:211
1728215992622,3cyclictest1727696-21kworker/u8:3+events_unbound07:33:141
1728208992620,4cyclictest1904420-21apt-key10:00:010
1728223992521,2cyclictest1878457-21kworker/u8:2+events_unbound10:10:003
1728223992518,4cyclictest2056347-21latency_hist12:25:013
1728220992521,3cyclictest1744297-21kworker/u8:0+flush-259:008:05:162
1728215992521,3cyclictest1727696-21kworker/u8:3+events_unbound07:25:131
1728208992520,4cyclictest509-21rs:main0
1728223992419,4cyclictest1899180-21kworker/u8:3+flush-259:010:30:173
1728223992418,4cyclictest1732652-21ntpq07:10:233
1728220992417,5cyclictest1988153-21latency_hist11:20:012
1728215992420,3cyclictest1744297-21kworker/u8:0+flush-259:008:10:191
1728208992420,3cyclictest1952151-21kworker/u8:1+flush-259:010:49:590
1728208992420,2cyclictest1-21systemd07:30:000
1728223992315,5cyclictest1852289-21latency_hist09:10:013
1728220992320,2cyclictest1962247-21kworker/u8:2+events_unbound10:55:232
1728220992316,5cyclictest335557-21systemd-journal09:20:012
1728215992320,2cyclictest1826584-21kworker/u8:0+events_unbound08:55:231
1728215992320,2cyclictest1744297-21kworker/u8:0+events_unbound08:05:221
1728215992320,2cyclictest1727696-21kworker/u8:3+flush-259:007:45:211
1728215992319,3cyclictest1952151-21kworker/u8:1+events_unbound11:12:331
1728215992319,3cyclictest1952151-21kworker/u8:1+events_unbound11:12:321
1728215992319,3cyclictest1936455-21kworker/u8:0+flush-259:012:30:111
1728215992319,3cyclictest1888861-21kworker/u8:1+events_unbound09:50:131
1728208992319,3cyclictest335557-21systemd-journal07:12:130
1728208992318,3cyclictest1878457-21kworker/u8:2+flush-259:010:30:000
1728220992218,3cyclictest1733549-21kworker/u8:4+flush-259:007:30:002
1728220992217,3cyclictest1764784-21kworker/u8:2+flush-259:007:55:002
1728215992218,3cyclictest1888861-21kworker/u8:1+events_unbound10:15:151
1728215992218,3cyclictest1888861-21kworker/u8:1+events_unbound09:45:141
1728215992218,3cyclictest1878457-21kworker/u8:2+flush-259:009:35:221
1728215992218,2cyclictest1842215-21kworker/u8:3+flush-259:009:00:161
1728208992217,3cyclictest1920611-21latency_hist10:15:020
1728208992216,4cyclictest1868601-21diskstats09:25:140
1728208992216,4cyclictest1738687-21latency_hist07:20:010
172820899221,19cyclictest1-21systemd07:31:130
1728220992118,2cyclictest1952151-21kworker/u8:1+events_unbound12:15:232
1728220992117,3cyclictest1936455-21kworker/u8:0+events_unbound12:20:222
1728220992116,3cyclictest1826807-21/usr/sbin/munin08:45:142
1728220992114,5cyclictest1961362-21ntpq10:50:202
1728215992117,3cyclictest1733549-21kworker/u8:4+events_unbound07:18:271
1728215992116,4cyclictest2014738-21apt-get11:45:111
172822399206,7cyclictest1816483-21/usr/sbin/munin08:35:243
1728223992014,4cyclictest2030624-21cp12:00:013
172822399201,17cyclictest1957067-21cron10:50:003
1728220992016,3cyclictest1775157-21kworker/u8:1+flush-259:008:15:202
1728220992016,3cyclictest1711803-21kworker/u8:2+flush-259:007:15:252
1728220992014,4cyclictest2035701-21latency_hist12:05:012
1728215992017,2cyclictest1733549-21kworker/u8:4+flush-259:007:20:211
1728215992017,2cyclictest1705946-21kworker/u8:0+events_unbound07:10:021
1728215992014,4cyclictest335557-21systemd-journal07:53:321
1728208992016,3cyclictest1852312-21kworker/u8:0+events_unbound09:30:220
1728208992016,3cyclictest1744297-21kworker/u8:0+flush-259:008:05:220
172822399199,8cyclictest0-21swapper/311:09:593
1728223991915,2cyclictest0-21swapper/309:20:003
1728223991913,4cyclictest1831966-21/usr/sbin/munin08:50:203
172822099195,7cyclictest2061511-21awk12:30:012
1728220991915,3cyclictest1775157-21kworker/u8:1+flush-259:008:35:162
1728220991915,3cyclictest1733549-21kworker/u8:4+events_unbound07:20:182
1728220991915,3cyclictest1727696-21kworker/u8:3+events_unbound07:30:232
1728215991916,2cyclictest1982939-21kworker/u8:5+flush-259:011:20:211
1728215991916,2cyclictest1878457-21kworker/u8:2+flush-259:010:40:251
1728215991915,3cyclictest1962247-21kworker/u8:2+events_unbound10:57:321
1728208991916,2cyclictest1733549-21kworker/u8:4+flush-259:007:50:170
1728208991915,3cyclictest1733549-21kworker/u8:4+flush-259:008:00:210
1728208991914,4cyclictest2024007-21ntpq11:50:200
172822399184,7cyclictest832-21runrttasks07:32:133
172822399181,5cyclictest2030074-21latency_hist12:00:003
1728223991811,5cyclictest1952151-21kworker/u8:1+flush-259:012:10:153
172822399180,2cyclictest1993334-21latency_hist11:25:013
1728220991814,3cyclictest1936455-21kworker/u8:0+events_unbound10:45:142
1728220991811,5cyclictest2065801-21ldconfig12:30:202
1728215991815,2cyclictest1936455-21kworker/u8:0+events_unbound11:55:211
1728215991815,2cyclictest1826584-21kworker/u8:0+events_unbound08:52:201
1728215991814,3cyclictest1952151-21kworker/u8:1+events_unbound11:09:321
1728215991814,3cyclictest1936455-21kworker/u8:0+events_unbound10:39:111
1728215991811,5cyclictest335557-21systemd-journal10:15:011
1728208991814,3cyclictest1899180-21kworker/u8:3+flush-259:011:05:150
172822399172,7cyclictest832-21runrttasks09:51:083
1728223991715,1cyclictest1733549-21kworker/u8:4+events_unbound07:40:173
172822399171,2cyclictest1846293-21ntpq09:00:233
172822399171,2cyclictest1805081-21ntpq08:20:233
1728223991711,5cyclictest0-21swapper/308:34:593
172822399171,14cyclictest335557-21systemd-journal09:45:003
172822399171,14cyclictest1893558-21grep09:45:263
172822399171,14cyclictest1826571-21idleruntime-cro08:45:003
172822399171,14cyclictest1769725-21sleep07:50:003
1728223991710,4cyclictest509-21rs:main3
172822399170,3cyclictest1759137-21taskset07:36:533
172822399170,2cyclictest1869059-21irqcore09:25:163
1728220991714,2cyclictest1733549-21kworker/u8:4+flush-259:008:20:182
1728220991713,3cyclictest1899180-21kworker/u8:3+flush-259:010:25:242
1728220991713,3cyclictest1820887-21kworker/u8:4+flush-259:009:20:182
172822099170,8cyclictest1785255-21awk08:05:002
1728215991714,2cyclictest1952151-21kworker/u8:1+flush-259:012:15:011
1728215991714,2cyclictest1775157-21kworker/u8:1+events_unbound08:43:211
1728215991713,3cyclictest1878457-21kworker/u8:2+events_unbound10:04:301
1728215991712,3cyclictest335557-21systemd-journal10:28:331
1728208991714,2cyclictest60350irq/132-enp0s29f1:rx-008:20:200
1728208991712,4cyclictest335557-21systemd-journal10:25:000
1728208991712,3cyclictest1744297-21kworker/u8:0+flush-259:007:55:150
172822399162,13cyclictest1857441-21idleruntime-cro09:15:003
172822399162,12cyclictest2066692-21idleruntime-cro12:35:013
1728223991613,2cyclictest1867216-21grep09:20:213
172822399161,2cyclictest335557-21systemd-journal07:20:013
1728223991612,3cyclictest1800744-21idleruntime-cro08:19:593
172822399161,13cyclictest335557-21systemd-journal10:05:013
172822399161,13cyclictest1988150-21apt11:19:593
172822399161,13cyclictest1904303-21sh10:00:003
172822399160,4cyclictest2014713-21/usr/sbin/munin11:45:253
172822399160,14cyclictest335557-21systemd-journal08:09:593
172822399160,13cyclictest1983055-21cat11:15:013
172822399160,13cyclictest1983055-21cat11:15:013
1728220991613,2cyclictest1727696-21kworker/u8:3+flush-259:007:35:172
1728220991612,3cyclictest1888861-21kworker/u8:1+events_unbound09:50:232
1728220991612,3cyclictest1878457-21kworker/u8:2+flush-259:010:35:202
1728220991611,3cyclictest1899376-21/usr/sbin/munin09:55:142
1728215991613,2cyclictest1868175-21kworker/u8:1+events_unbound09:25:211
1728215991612,3cyclictest1878457-21kworker/u8:2+flush-259:009:40:161
1728215991612,3cyclictest1727696-21kworker/u8:3+events_unbound07:35:031
1728215991611,4cyclictest1878457-21kworker/u8:2+events_unbound10:08:501
1728215991611,3cyclictest1983157-21/usr/sbin/munin11:15:151
1728208991614,1cyclictest60350irq/132-enp0s29f1:rx-011:45:160
1728208991614,1cyclictest60350irq/132-enp0s29f1:rx-009:48:580
1728208991613,2cyclictest60350irq/132-enp0s29f1:rx-010:55:240
1728208991613,2cyclictest60350irq/132-enp0s29f1:rx-010:30:190
1728208991610,4cyclictest335557-21systemd-journal12:37:310
172822399153,11cyclictest1967365-21sh11:00:003
172822399152,11cyclictest0-21swapper/309:30:183
172822399152,10cyclictest1744137-21apt-get07:25:093
172822399151,4cyclictest1794956-21cat08:10:263
172822399151,2cyclictest20043422sleep311:35:113
172822399151,2cyclictest1968033-21ps11:00:163
172822399151,2cyclictest1935572-21ntpq10:25:223
172822399151,2cyclictest1882734-21ntpq09:35:203
172822399151,2cyclictest1830761-21needreboot08:45:193
172822399151,2cyclictest1773990-21ntpq07:50:203
172822399151,12cyclictest0-21swapper/311:55:013
172822399150,6cyclictest1775381-21/usr/sbin/munin07:55:143
172822399150,5cyclictest1946974-21/usr/sbin/munin10:40:243
172822399150,5cyclictest1946758-21systemd-run10:39:593
172822399150,5cyclictest1738929-21/usr/sbin/munin07:20:113
172822399150,4cyclictest2071864-21cron12:39:593
172822399150,3cyclictest2046651-21grep12:15:173
172822399150,2cyclictest1961789-21grep10:50:233
172822399150,2cyclictest1920836-21apt-key10:15:003
172822399150,2cyclictest1784478-21ps08:00:233
172822399150,13cyclictest2014467-21basename11:45:003
1728220991513,1cyclictest0-21swapper/208:00:012
1728220991512,2cyclictest1852312-21kworker/u8:0+events_unbound09:10:222
1728220991511,3cyclictest1952151-21kworker/u8:1+flush-259:012:10:162
1728220991511,3cyclictest1899180-21kworker/u8:3+events_unbound10:30:192
1728220991511,3cyclictest1842215-21kworker/u8:3+flush-259:009:05:262
1728215991513,2cyclictest1962247-21kworker/u8:2+flush-259:011:00:231
1728215991512,2cyclictest1852312-21kworker/u8:0+events_unbound09:10:141
1728215991510,4cyclictest1852312-21kworker/u8:0+events_unbound09:34:161
1728208991513,1cyclictest60350irq/132-enp0s29f1:rx-012:20:190
1728208991513,1cyclictest60350irq/132-enp0s29f1:rx-011:35:140
1728208991513,1cyclictest60350irq/132-enp0s29f1:rx-010:00:230
1728208991513,1cyclictest60350irq/132-enp0s29f1:rx-009:44:140
1728208991513,1cyclictest60350irq/132-enp0s29f1:rx-009:36:410
1728208991513,1cyclictest60350irq/132-enp0s29f1:rx-009:15:260
1728208991513,1cyclictest60350irq/132-enp0s29f1:rx-007:36:500
1728208991512,2cyclictest61150irq/133-enp0s29f1:tx-007:45:190
1728208991512,2cyclictest60350irq/132-enp0s29f1:rx-012:34:490
1728208991512,2cyclictest60350irq/132-enp0s29f1:rx-012:05:180
1728208991512,2cyclictest60350irq/132-enp0s29f1:rx-009:54:070
1728208991512,2cyclictest60350irq/132-enp0s29f1:rx-009:20:220
1728208991512,2cyclictest60350irq/132-enp0s29f1:rx-008:50:110
1728208991512,2cyclictest60350irq/132-enp0s29f1:rx-008:35:120
1728208991512,2cyclictest60350irq/132-enp0s29f1:rx-008:25:390
1728208991510,4cyclictest1795828-21/usr/sbin/munin08:15:140
172822399141,4cyclictest1993597-21apt11:25:103
172822399141,11cyclictest1928730-21grep10:20:183
172822399140,2cyclictest0-21swapper/307:10:013
172822099149,3cyclictest1868175-21kworker/u8:1+events_unbound09:25:172
172822099148,4cyclictest335557-21systemd-journal11:50:002
1728220991410,3cyclictest2025528-21kworker/u8:3+events_unbound12:05:242
1728220991410,3cyclictest1982939-21kworker/u8:5+flush-259:011:40:232
1728220991410,3cyclictest1826584-21kworker/u8:0+flush-259:008:50:212
1728220991410,2cyclictest0-21swapper/207:50:002
1728215991410,3cyclictest1810755-21kworker/u8:2+events_unbound08:31:161
1728215991410,3cyclictest1775157-21kworker/u8:1+events_unbound08:37:461
1728208991412,2cyclictest60350irq/132-enp0s29f1:rx-010:50:210
1728208991412,1cyclictest60350irq/132-enp0s29f1:rx-010:15:150
1728208991412,1cyclictest60350irq/132-enp0s29f1:rx-009:05:170
1728208991412,1cyclictest60350irq/132-enp0s29f1:rx-008:40:160
1728208991412,1cyclictest1857451-21turbostat09:15:000
1728208991411,3cyclictest60350irq/132-enp0s29f1:rx-011:40:110
1728208991411,2cyclictest60350irq/132-enp0s29f1:rx-012:25:120
1728208991411,2cyclictest60350irq/132-enp0s29f1:rx-008:10:220
172822099139,3cyclictest1878457-21kworker/u8:2+events_unbound10:44:122
172822099139,3cyclictest1826584-21kworker/u8:0+flush-259:008:55:152
172822099138,4cyclictest1915121-21gpgv10:10:002
1728220991310,2cyclictest2020981-21kthreadcore11:50:172
1728220991310,2cyclictest1919597-21needreboot10:10:202
1728220991310,1cyclictest0-21swapper/212:00:002
172821599139,3cyclictest1810755-21kworker/u8:2+events_unbound08:45:051
172820899139,3cyclictest1761962-21tr07:40:170
1728208991311,2cyclictest60350irq/132-enp0s29f1:rx-008:55:160
1728208991311,1cyclictest60350irq/132-enp0s29f1:rx-012:10:190
1728208991311,1cyclictest60350irq/132-enp0s29f1:rx-012:00:260
172822099129,2cyclictest1820887-21kworker/u8:4+events_unbound09:00:232
172822099128,3cyclictest1888861-21kworker/u8:1+flush-259:009:45:162
1728220991210,1cyclictest0-21swapper/210:24:592
172820899129,2cyclictest60350irq/132-enp0s29f1:rx-007:20:110
1728208991210,1cyclictest60350irq/132-enp0s29f1:rx-011:15:480
172822099117,3cyclictest1775157-21kworker/u8:1+flush-259:008:25:152
172821599117,3cyclictest1936455-21kworker/u8:0+events_unbound10:49:231
172820899118,2cyclictest0-21swapper/008:45:140
172820899115,4cyclictest335557-21systemd-journal11:24:590
17282209995,3cyclictest1711803-21kworker/u8:2+flush-259:007:10:182
17282209994,3cyclictest1971991-21sed11:00:232
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional