You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-19 - 03:32

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot4s.osadl.org (updated Fri Dec 19, 2025 00:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1858851993933,4cyclictest252-21systemd-journal19:50:010
1858864993327,4cyclictest2025272-21latency_hist21:50:013
1858852993327,4cyclictest1978377-21latency_hist21:05:011
185885199330,31cyclictest1-21systemd00:15:000
1858851993227,3cyclictest2092886-21apt-get22:55:140
1858851993127,3cyclictest1864037-21kworker/u8:2+events_unbound19:40:150
1858852993025,4cyclictest1915699-21kworker/u8:2+flush-259:020:10:001
1858852993024,4cyclictest252-21systemd-journal23:20:001
1858851993026,3cyclictest1973412-21kworker/u8:0+flush-259:021:08:090
1858852992924,4cyclictest1915699-21kworker/u8:2+flush-259:020:10:181
1858852992921,5cyclictest252-21systemd-journal23:05:001
1858851992925,3cyclictest2087498-21kworker/u8:2+flush-259:023:00:170
1858864992717,9cyclictest1951950-21idleruntime-cro20:40:003
1858852992721,4cyclictest2019627-21apt-get21:45:001
1858851992723,2cyclictest2009324-21kworker/u8:2+events_unbound22:05:180
1858864992616,8cyclictest509-21rs:main3
1858859992623,2cyclictest1884706-21kworker/u8:0+flush-259:019:55:012
1858852992621,3cyclictest1957608-21kworker/u8:5+flush-259:021:30:171
1858852992620,4cyclictest252-21systemd-journal22:55:001
1858851992622,3cyclictest1900205-21kworker/u8:4+flush-259:020:20:270
1858851992621,4cyclictest1904500-21ntpq19:50:240
1858864992522,2cyclictest2051213-21kworker/u8:1+flush-259:022:15:163
1858864992520,3cyclictest2140644-21apt-get23:40:133
1858852992518,5cyclictest2160124-21ntpq23:55:221
185886499245,10cyclictest1929744-21cut20:15:203
1858859992421,2cyclictest2051213-21kworker/u8:1+events_unbound22:30:182
1858859992421,2cyclictest1957608-21kworker/u8:5+events_unbound20:50:252
1858859992421,2cyclictest1957608-21kworker/u8:5+events_unbound20:50:252
1858859992418,5cyclictest252-21systemd-journal21:35:002
1858852992418,4cyclictest1926031-21kworker/u8:3+flush-259:020:15:181
1858852992416,5cyclictest2196225-21ntpq00:30:231
1858851992419,3cyclictest1874330-21latency_hist19:25:010
1858851992415,6cyclictest252-21systemd-journal21:15:000
1858864992315,7cyclictest2155829-21munin-run23:55:003
185886499230,13cyclictest2081657-21sed22:40:253
185886499230,13cyclictest1919702-21grep20:05:193
185886499230,11cyclictest2091702-21ntpq22:50:223
1858859992320,2cyclictest2051213-21kworker/u8:1+flush-259:022:15:182
1858859992320,2cyclictest2009324-21kworker/u8:2+flush-259:021:35:262
1858859992319,3cyclictest2102993-21kworker/u8:1+flush-259:023:10:152
1858859992319,3cyclictest2030657-21kworker/u8:1+events_unbound21:55:162
1858859992319,3cyclictest1957607-21kworker/u8:4+events_unbound20:55:252
1858859992319,3cyclictest1957607-21kworker/u8:4+events_unbound20:55:252
1858859992319,3cyclictest1926031-21kworker/u8:3+events_unbound20:45:182
1858859992318,3cyclictest2129424-21/usr/sbin/munin23:30:142
1858859992317,5cyclictest2040703-21latency_hist22:05:012
1858851992320,2cyclictest509-21rs:main0
1858851992319,3cyclictest2019258-21kworker/u8:3+flush-259:021:50:180
1858851992317,4cyclictest1961922-21ntpq20:45:210
1858859992219,2cyclictest2166190-21turbostat.cron00:05:002
1858859992215,5cyclictest2170411-21ntpq00:05:262
1858852992217,3cyclictest2082582-21apt-get22:45:121
1858852992214,5cyclictest2013609-21ntpq21:35:231
1858851992218,3cyclictest1957608-21kworker/u8:5+events_unbound21:25:190
1858864992115,4cyclictest2072061-21latency_hist22:35:003
185886499210,10cyclictest2000258-21/usr/sbin/munin21:25:183
1858859992117,3cyclictest1805136-21kworker/u8:3+flush-259:019:27:292
1858851992118,2cyclictest507-21in:imuxsock21:40:000
1858851992116,3cyclictest252-21systemd-journal22:25:000
1858864992017,2cyclictest1931901-21irqcore20:20:203
1858864992016,3cyclictest2051213-21kworker/u8:1+flush-259:022:45:173
1858864992014,4cyclictest1894997-21latency_hist19:45:013
185886499200,9cyclictest1873615-21/usr/sbin/munin19:20:223
1858859992017,2cyclictest1864037-21kworker/u8:2+flush-259:019:35:222
1858859992017,2cyclictest1864037-21kworker/u8:2+events_unbound19:40:142
1858859992016,3cyclictest2166855-21kworker/u8:2+flush-259:000:15:252
1858859992016,3cyclictest1890069-21kworker/u8:1+flush-259:019:45:172
1858859992016,3cyclictest1805136-21kworker/u8:3+events_unbound19:30:162
1858859992015,4cyclictest252-21systemd-journal00:40:002
1858852992016,3cyclictest1864316-21apt-get19:15:101
1858852992016,3cyclictest1864037-21kworker/u8:2+flush-259:019:25:011
1858852992014,4cyclictest2134945-21cat23:35:011
1858852992014,4cyclictest2103186-21cat23:05:001
1858852992013,5cyclictest2149839-21ntpq23:45:221
1858851992017,1cyclictest0-21swapper/021:45:000
1858851992015,4cyclictest2056177-21apt-get22:20:000
1858851992014,4cyclictest1930330-21ntpq20:15:230
1858851992013,5cyclictest1945950-21ntpq20:30:250
1858864991915,3cyclictest2009324-21kworker/u8:2+flush-259:022:05:183
185886499190,11cyclictest2150495-21taskset23:48:073
185886499190,11cyclictest2030401-21chrt21:54:543
185886499190,10cyclictest2196875-21taskset00:32:183
185886499190,10cyclictest2097523-21taskset22:56:493
1858859991916,3cyclictest2019258-21kworker/u8:3+events_unbound22:20:242
1858859991916,2cyclictest2114277-21kworker/u8:4+flush-259:023:35:212
1858859991916,2cyclictest2102993-21kworker/u8:1+flush-259:023:05:282
1858859991916,2cyclictest2051213-21kworker/u8:1+flush-259:022:45:162
1858859991916,2cyclictest1983791-21kworker/u8:1+events_unbound21:20:152
1858859991913,4cyclictest1936626-21/usr/sbin/munin20:25:192
1858852991915,3cyclictest1926031-21kworker/u8:3+flush-259:020:32:301
1858852991915,3cyclictest1884931-21apt-get19:35:131
1858852991913,4cyclictest252-21systemd-journal22:45:001
1858851991915,3cyclictest2150927-21apt-get23:50:130
1858851991915,3cyclictest2114277-21kworker/u8:4+events_unbound23:55:230
1858851991915,3cyclictest1988986-21kworker/u8:0+flush-259:021:17:360
1858851991915,3cyclictest1805136-21kworker/u8:3+events_unbound19:35:250
1858851991914,4cyclictest2134882-21cat23:35:020
1858851991914,3cyclictest2056865-21apt-get22:20:120
1858864991814,3cyclictest1805136-21kworker/u8:3+flush-259:019:30:243
1858864991813,3cyclictest252-21systemd-journal22:15:003
185886499180,9cyclictest832-21runrttasks23:32:133
185886499180,9cyclictest2061608-21chrt22:23:203
185886499180,9cyclictest2040641-21chrt22:04:233
185886499180,9cyclictest1983340-21taskset21:07:323
185886499180,9cyclictest1978152-21runrttasks21:02:263
185886499180,9cyclictest1972858-21taskset20:56:143
185886499180,9cyclictest1972858-21taskset20:56:143
185886499180,9cyclictest1884895-21taskset19:35:073
185886499180,8cyclictest2123903-21taskset23:24:043
185886499180,10cyclictest2202229-21chrt00:39:073
1858859991816,1cyclictest1921542-21irqcore20:10:182
1858859991815,2cyclictest2114277-21kworker/u8:4+flush-259:023:50:152
1858859991815,2cyclictest2114277-21kworker/u8:4+flush-259:000:32:072
1858859991815,2cyclictest2108771-21kworker/u8:0+flush-259:023:25:002
1858859991815,2cyclictest2087498-21kworker/u8:2+events_unbound22:55:222
1858859991815,2cyclictest1926028-21kworker/u8:1+events_unbound20:35:242
1858859991815,2cyclictest1926028-21kworker/u8:1+events_unbound20:15:262
1858859991815,2cyclictest0-21swapper/222:45:002
1858852991813,4cyclictest2166855-21kworker/u8:2+flush-259:000:20:271
1858852991813,4cyclictest2019258-21kworker/u8:3+flush-259:021:50:171
1858852991813,4cyclictest1941558-21kworker/u8:2+flush-259:021:05:271
1858852991813,3cyclictest252-21systemd-journal20:42:061
1858852991811,5cyclictest252-21systemd-journal19:35:011
1858852991811,5cyclictest1910519-21latency_hist20:00:001
1858851991816,1cyclictest0-21swapper/000:25:000
1858851991815,2cyclictest1973412-21kworker/u8:0+flush-259:021:00:170
1858851991815,2cyclictest1884706-21kworker/u8:0+flush-259:020:25:250
1858851991814,3cyclictest1957608-21kworker/u8:5+flush-259:020:50:190
1858851991814,3cyclictest1957608-21kworker/u8:5+flush-259:020:50:190
1858851991814,3cyclictest1805136-21kworker/u8:3+flush-259:019:30:240
1858851991814,3cyclictest0-21swapper/021:00:000
1858851991814,3cyclictest0-21swapper/021:00:000
1858851991813,3cyclictest252-21systemd-journal19:29:060
185886499176,9cyclictest2176482-21sh00:15:013
1858864991712,3cyclictest1957608-21kworker/u8:5-flush-259:020:55:003
1858864991712,3cyclictest1957608-21kworker/u8:5-flush-259:020:55:003
1858864991712,3cyclictest1900205-21kworker/u8:4+flush-259:019:55:183
185886499170,9cyclictest2128982-21taskset23:27:493
185886499170,9cyclictest2118865-21munin-run23:20:013
185886499170,9cyclictest2035284-21taskset21:57:033
185886499170,8cyclictest832-21runrttasks20:27:133
185886499170,16cyclictest1999017-21idleruntime-cro21:25:003
185886499170,14cyclictest2156347-21/usr/sbin/munin23:55:153
185886499170,13cyclictest2191660-21taskset00:26:583
185886499170,13cyclictest2186431-21taskset00:20:513
185886499170,10cyclictest2139647-21taskset23:36:273
185886499170,10cyclictest2014439-21taskset21:39:323
185886499170,10cyclictest1988678-21taskset21:14:213
1858859991714,2cyclictest2166855-21kworker/u8:2+flush-259:000:25:212
1858859991714,2cyclictest2114277-21kworker/u8:4+events_unbound23:55:222
1858859991714,2cyclictest2108771-21kworker/u8:0+flush-259:023:15:202
1858859991714,2cyclictest2102993-21kworker/u8:1+events_unbound23:40:242
1858859991714,2cyclictest1941558-21kworker/u8:2+events_unbound21:05:182
1858859991714,2cyclictest1900205-21kworker/u8:4+flush-259:020:20:202
1858859991713,3cyclictest1926031-21kworker/u8:3+events_unbound20:40:142
1858852991714,2cyclictest2111126-21kthreadcore23:10:211
1858852991713,3cyclictest2134351-21apt-get23:35:001
1858852991713,3cyclictest1993018-21ntp_kernel_pll_21:15:221
1858852991711,3cyclictest1895235-21/usr/sbin/munin19:45:151
1858851991712,4cyclictest252-21systemd-journal23:42:560
185886499162,12cyclictest2019639-21basename21:45:003
1858864991614,1cyclictest0-21swapper/300:05:003
185886499160,8cyclictest2108786-21/usr/sbin/munin23:10:273
185886499160,8cyclictest2076831-21chrt22:36:013
185886499160,8cyclictest2008923-21runrttasks21:30:283
185886499160,8cyclictest1993589-21taskset21:16:353
185886499160,7cyclictest1915318-21taskset20:00:493
185886499160,10cyclictest0-21swapper/300:18:353
1858859991614,1cyclictest2025957-21irqcore21:50:172
1858859991613,2cyclictest2166855-21kworker/u8:2+events_unbound00:10:202
1858859991612,3cyclictest1884706-21kworker/u8:0+flush-259:020:00:252
185885299169,4cyclictest1999251-21/usr/sbin/munin21:25:241
1858852991612,3cyclictest1988986-21kworker/u8:0+flush-259:022:07:371
1858852991612,3cyclictest1941558-21kworker/u8:2+flush-259:021:10:281
1858851991613,2cyclictest60350irq/132-enp0s29f1:rx-023:20:190
1858851991613,2cyclictest60350irq/132-enp0s29f1:rx-020:13:010
1858851991613,2cyclictest60350irq/132-enp0s29f1:rx-019:10:170
1858851991613,2cyclictest2124041-21kworker/u8:2+flush-259:023:30:150
1858851991612,3cyclictest2114277-21kworker/u8:4+flush-259:023:15:240
1858851991612,3cyclictest2087482-21turbostat22:50:000
1858851991612,3cyclictest1884706-21kworker/u8:0+events_unbound20:00:190
1858851991612,2cyclictest1920857-21turbostat20:10:000
185886499150,8cyclictest21036992sleep323:05:143
185886499150,8cyclictest1905158-21runrttasks19:52:523
185886499150,8cyclictest1863911-21chrt19:13:303
185886499150,7cyclictest832-21runrttasks20:41:483
185886499150,13cyclictest2101899-21sed23:00:213
185886499150,13cyclictest1958913-21grep20:45:183
185886499150,13cyclictest1897201-21cut19:45:173
185886499150,12cyclictest1946787-21latency_hist20:35:003
1858859991512,3cyclictest2114277-21kworker/u8:4+flush-259:023:45:152
1858859991512,2cyclictest0-21swapper/219:25:012
1858859991511,3cyclictest1941558-21kworker/u8:2+flush-259:021:12:372
1858859991511,3cyclictest1941558-21kworker/u8:2+flush-259:020:30:242
1858859991511,2cyclictest0-21swapper/219:15:002
185885299158,4cyclictest2056839-21/usr/sbin/munin22:20:151
1858852991512,1cyclictest0-21swapper/121:00:001
1858852991512,1cyclictest0-21swapper/121:00:001
1858852991511,3cyclictest252-21systemd-journal21:24:551
185885299151,13cyclictest2072066-21latency_hist22:35:001
1858852991510,4cyclictest2114277-21kworker/u8:4+flush-259:000:25:241
1858851991513,1cyclictest60350irq/132-enp0s29f1:rx-023:45:250
1858851991513,1cyclictest60350irq/132-enp0s29f1:rx-000:32:180
1858851991513,1cyclictest60350irq/132-enp0s29f1:rx-000:25:240
1858851991513,1cyclictest60350irq/132-enp0s29f1:rx-000:15:140
1858851991513,1cyclictest60350irq/132-enp0s29f1:rx-000:05:230
1858851991512,2cyclictest60350irq/132-enp0s29f1:rx-019:15:110
1858851991512,2cyclictest2114277-21kworker/u8:4+flush-259:000:35:230
1858851991512,2cyclictest2087498-21kworker/u8:2+flush-259:023:05:210
1858851991512,2cyclictest2072081-21kworker/u8:0+flush-259:022:50:250
1858851991511,3cyclictest2030695-21apt-get21:55:120
1858851991510,3cyclictest2087498-21kworker/u8:2+flush-259:023:10:200
185886499140,12cyclictest2064872-21cut22:25:193
185886499140,12cyclictest1921385-21grep20:10:163
185886499140,12cyclictest1874599-21apt19:25:133
185886499140,12cyclictest1865486-21cut19:15:173
185886499140,12cyclictest1858980-21ls19:10:003
1858859991411,2cyclictest1900205-21kworker/u8:4+events_unbound19:57:202
1858859991411,2cyclictest1864037-21kworker/u8:2+flush-259:019:15:102
1858859991410,3cyclictest2019258-21kworker/u8:3+events_unbound22:25:142
1858859991410,3cyclictest1988986-21kworker/u8:0+flush-259:021:15:262
1858859991410,2cyclictest0-21swapper/223:05:012
185885299149,4cyclictest2009324-21kworker/u8:2+flush-259:022:00:161
185885299148,4cyclictest252-21systemd-journal20:35:071
1858852991411,2cyclictest2141065-21irqcore23:40:171
185885299141,10cyclictest252-21systemd-journal19:55:011
1858852991410,3cyclictest2129176-21basename23:30:011
185885199149,3cyclictest509-21rs:main0
185885199147,5cyclictest252-21systemd-journal22:32:060
1858851991413,1cyclictest60350irq/132-enp0s29f1:rx-020:41:370
1858851991412,1cyclictest60350irq/132-enp0s29f1:rx-023:25:130
1858851991412,1cyclictest60350irq/132-enp0s29f1:rx-022:00:120
1858851991410,3cyclictest2077441-21apt-get22:40:130
185885999136,5cyclictest252-21systemd-journal21:45:012
1858859991311,1cyclictest2072081-21kworker/u8:0+events_unbound22:50:142
1858859991310,2cyclictest2124330-21cpuspeed_turbos23:25:142
185885299139,3cyclictest1941558-21kworker/u8:2+flush-259:020:46:371
185885299138,3cyclictest1988986-21kworker/u8:0+flush-259:021:55:161
185885299137,5cyclictest1879508-21latency_hist19:30:011
185885299137,4cyclictest252-21systemd-journal22:16:071
185885299133,2cyclictest18926742kthreadcore19:40:181
1858851991311,1cyclictest60350irq/132-enp0s29f1:rx-021:20:320
1858851991311,1cyclictest60350irq/132-enp0s29f1:rx-020:35:130
1858851991311,1cyclictest60350irq/132-enp0s29f1:rx-000:03:000
1858851991310,2cyclictest2009324-21kworker/u8:2+events_unbound22:10:210
185885999128,3cyclictest2051213-21kworker/u8:1+events_unbound22:35:172
185885999128,3cyclictest1973412-21kworker/u8:0+flush-259:021:00:152
185885999128,3cyclictest1957608-21kworker/u8:5+flush-259:021:28:052
185885999125,5cyclictest1920004-21ntpq20:05:232
185885299129,2cyclictest2171980-21irqcore00:10:181
185885299128,3cyclictest1957608-21kworker/u8:5+flush-259:020:50:281
185885299128,3cyclictest1957608-21kworker/u8:5+flush-259:020:50:271
185885299128,3cyclictest1805136-21kworker/u8:3+flush-259:019:10:261
185885299126,4cyclictest2020302-21/usr/sbin/munin21:45:161
1858851991211,1cyclictest60350irq/132-enp0s29f1:rx-021:30:180
185885299119,1cyclictest0-21swapper/120:20:281
185885299117,3cyclictest2197399-21apt-get00:35:121
185885299117,2cyclictest0-21swapper/122:25:131
185885299116,3cyclictest252-21systemd-journal00:04:551
185885199119,1cyclictest0-21swapper/019:55:180
185885299107,1cyclictest0-21swapper/100:05:301
18588529996,2cyclictest1911198-21irqcore20:00:171
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional