You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-08 - 10:48

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot4s.osadl.org (updated Fri May 08, 2026 00:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1871573994237,3cyclictest2157893-21latency_hist23:45:003
1871565994036,3cyclictest1887300-21kworker/u8:2+events_unbound19:32:071
1871559994036,3cyclictest2204568-21apt-get00:30:100
1871565993631,4cyclictest1949292-21apt-get20:25:111
1871565993428,4cyclictest1964633-21gpgv20:40:001
1871565993324,6cyclictest2132132-21latency_hist23:20:001
1871559993328,4cyclictest1887065-21latency_hist19:25:010
1871565993125,4cyclictest2189106-21apt-get00:15:111
1871559993125,4cyclictest2137293-21latency_hist23:25:000
1871565993025,4cyclictest2147836-21apt-get23:35:111
1871565992923,4cyclictest509-21rs:main1
1871565992923,4cyclictest2100829-21latency_hist22:50:001
1871559992926,2cyclictest2058825-21kworker/u8:2+flush-259:022:20:110
1871559992923,4cyclictest2042517-21ntpq21:50:220
1871573992825,2cyclictest2116959-21apt-get23:05:123
1871565992821,5cyclictest1917144-21ntpq19:50:211
1871559992825,2cyclictest2152961-21kworker/u8:4+flush-259:023:50:150
1871573992723,3cyclictest1970493-21/usr/sbin/munin20:45:143
1871559992724,2cyclictest1996773-21kworker/u8:3+events_unbound21:22:520
1871559992724,2cyclictest1959634-21kworker/u8:4+flush-259:021:13:030
1871573992623,2cyclictest2017857-21kworker/u8:2+flush-259:021:35:243
1871569992621,3cyclictest252-21systemd-journal22:50:002
1871569992621,3cyclictest1959634-21kworker/u8:4+flush-259:021:20:002
1871569992620,4cyclictest252-21systemd-journal19:11:062
1871565992621,4cyclictest1975404-21kworker/u8:0+flush-259:021:00:001
1871573992522,2cyclictest2085005-21kworker/u8:3+flush-259:022:45:163
1871573992522,2cyclictest1985758-21kworker/u8:1+flush-259:021:10:013
1871573992522,2cyclictest1923212-21kworker/u8:0+flush-259:020:10:013
1871565992521,2cyclictest1876481-21fwupd19:13:001
1871559992520,4cyclictest2059070-21apt-get22:10:130
1871573992420,3cyclictest2033065-21kworker/u8:0+flush-259:021:50:003
1871569992420,3cyclictest1902598-21kworker/u8:2+flush-259:020:55:212
1871569992419,4cyclictest2052824-21ntpq22:00:212
1871565992421,2cyclictest1959634-21kworker/u8:4+events_unbound21:54:111
1871559992418,4cyclictest2116694-21latency_hist23:05:000
1871573992318,4cyclictest2017897-21apt-get21:30:133
1871565992316,5cyclictest2163033-21latency_hist23:50:001
1871559992320,2cyclictest1985758-21kworker/u8:1+flush-259:021:00:200
1871559992319,3cyclictest1897152-21kworker/u8:0+events_unbound19:45:150
1871559992318,4cyclictest1959672-21apt-get20:35:110
1871573992219,2cyclictest1902598-21kworker/u8:2+flush-259:020:55:113
1871573992218,3cyclictest2152961-21kworker/u8:4+flush-259:023:50:143
1871573992218,3cyclictest2152961-21kworker/u8:4+flush-259:000:05:013
1871569992219,2cyclictest1985758-21kworker/u8:1+flush-259:021:05:172
1871569992218,3cyclictest1902598-21kworker/u8:2+events_unbound20:00:212
1871569992217,4cyclictest2100943-21gpgv22:50:002
1871569992217,3cyclictest252-21systemd-journal20:20:012
1871569992216,4cyclictest252-21systemd-journal20:11:072
1871569992216,4cyclictest252-21systemd-journal00:02:102
1871565992219,2cyclictest1996253-21apt-key21:10:001
1871565992218,3cyclictest2079403-21apt-get22:30:011
1871565992215,5cyclictest2110662-21ntpq22:55:201
1871559992219,2cyclictest1902598-21kworker/u8:2+flush-259:020:40:150
1871559992219,2cyclictest1856388-21kworker/u8:4+flush-259:019:25:170
1871559992218,3cyclictest2168462-21apt-get23:55:140
1871559992218,3cyclictest2152961-21kworker/u8:4+events_unbound00:20:190
1871559992217,3cyclictest2017416-21gpgv21:30:000
1871573992118,2cyclictest2094825-21kworker/u8:0+flush-259:023:20:213
1871573992118,2cyclictest2094825-21kworker/u8:0+flush-259:022:55:243
1871573992117,3cyclictest2094825-21kworker/u8:0+flush-259:023:18:203
1871569992118,2cyclictest2074266-21kworker/u8:1+flush-259:022:35:242
1871569992116,3cyclictest252-21systemd-journal21:46:102
1871569992115,4cyclictest509-21rs:main2
1871565992118,2cyclictest1856388-21kworker/u8:4+events_unbound19:49:561
1871565992114,5cyclictest2214651-21latency_hist00:40:021
1871565992114,5cyclictest2115805-21ntpq23:00:211
1871565992113,5cyclictest509-21rs:main1
1871559992117,3cyclictest2017857-21kworker/u8:2+flush-259:021:40:230
1871559992114,3cyclictest2017857-21kworker/u8:2+events_unbound21:30:230
1871573992017,2cyclictest2188864-21kworker/u8:2+events_unbound00:25:003
1871573992017,2cyclictest1912845-21kworker/u8:1+flush-259:019:55:203
1871559992016,3cyclictest2058825-21kworker/u8:2+events_unbound22:40:220
1871573991916,2cyclictest2142448-21kworker/u8:1+flush-259:023:45:213
1871573991916,2cyclictest2142448-21kworker/u8:1+flush-259:000:10:153
1871573991916,2cyclictest2127012-21kworker/u8:2+flush-259:023:30:253
1871573991916,2cyclictest2127012-21kworker/u8:2+flush-259:023:30:253
1871569991915,3cyclictest2094825-21kworker/u8:0+flush-259:023:15:242
1871569991915,3cyclictest1959634-21kworker/u8:4+flush-259:021:15:002
1871565991917,2cyclictest1959634-21kworker/u8:4+events_unbound21:44:561
1871565991917,2cyclictest1856388-21kworker/u8:4+events_unbound19:40:021
1871565991915,3cyclictest2085257-21apt-get22:35:131
1871565991913,4cyclictest509-21rs:main1
1871565991912,5cyclictest2006885-21latency_hist21:20:011
1871559991912,5cyclictest2047607-21ntpq21:55:200
1871573991815,2cyclictest2127012-21kworker/u8:2+flush-259:023:35:163
1871573991814,3cyclictest2116706-21kworker/u8:1+events_unbound23:10:263
1871569991815,2cyclictest2058825-21kworker/u8:2+events_unbound22:15:252
1871569991815,2cyclictest1975404-21kworker/u8:0+flush-259:020:55:002
1871569991815,2cyclictest1856388-21kworker/u8:4+events_unbound19:40:012
1871569991814,3cyclictest2017857-21kworker/u8:2+events_unbound21:35:222
187156599186,3cyclictest32-21ksoftirqd/121:30:001
1871565991816,2cyclictest2094825-21kworker/u8:0+events_unbound23:24:571
1871565991815,2cyclictest1975404-21kworker/u8:0+events_unbound20:50:081
1871565991813,3cyclictest2199126-21cron00:25:001
1871565991812,4cyclictest252-21systemd-journal22:11:111
1871559991814,3cyclictest2152961-21kworker/u8:4+events_unbound00:13:100
1871559991814,3cyclictest2142448-21kworker/u8:1+flush-259:023:40:170
1871559991814,3cyclictest2074266-21kworker/u8:1+flush-259:022:28:100
1871559991814,2cyclictest0-21swapper/023:15:000
1871573991714,2cyclictest2204990-21irqcore00:30:163
1871573991714,2cyclictest2044649-21kthreadcore21:55:173
1871573991714,2cyclictest1866428-21kworker/u8:0+flush-259:019:15:003
1871573991713,3cyclictest1985758-21kworker/u8:1+events_unbound21:00:133
1871573991713,3cyclictest1902598-21kworker/u8:2+events_unbound20:25:003
1871573991710,5cyclictest252-21systemd-journal19:53:063
1871569991713,3cyclictest2127012-21kworker/u8:2+flush-259:023:35:242
1871569991713,3cyclictest1959634-21kworker/u8:4+events_unbound21:25:012
1871569991713,3cyclictest1928605-21/usr/sbin/munin20:05:132
1871569991712,4cyclictest1953332-21kworker/u8:3+flush-259:020:45:172
1871569991710,5cyclictest252-21systemd-journal20:00:002
1871565991715,2cyclictest1902598-21kworker/u8:2+events_unbound20:31:311
1871565991714,2cyclictest2162661-21kworker/u8:0+events_unbound00:04:571
1871565991714,2cyclictest2064193-21kworker/u8:0+events_unbound22:34:581
1871565991714,2cyclictest1996773-21kworker/u8:3+events_unbound22:20:031
1871565991711,4cyclictest252-21systemd-journal20:09:061
1871559991714,2cyclictest61150irq/133-enp0s29f1:tx-022:05:200
1871559991714,2cyclictest2132359-21kworker/u8:3+events_unbound23:35:140
1871559991713,3cyclictest2058825-21kworker/u8:2+events_unbound22:50:240
1871559991712,3cyclictest252-21systemd-journal23:33:110
1871559991712,3cyclictest252-21systemd-journal23:33:100
1871573991614,1cyclictest0-21swapper/300:00:003
1871573991613,2cyclictest1856388-21kworker/u8:4+flush-259:019:40:213
1871573991613,1cyclictest0-21swapper/319:34:593
1871573991612,3cyclictest2142448-21kworker/u8:1+events_unbound00:05:213
1871573991612,3cyclictest2074266-21kworker/u8:1+flush-259:022:25:213
1871573991612,3cyclictest2069392-21apt-get22:20:113
1871573991612,3cyclictest1996773-21kworker/u8:3+flush-259:022:20:013
1871573991612,3cyclictest1953332-21kworker/u8:3+flush-259:020:50:183
1871569991613,2cyclictest2142448-21kworker/u8:1+flush-259:000:10:222
1871569991613,2cyclictest0-21swapper/222:30:012
1871569991612,3cyclictest1943874-21kworker/u8:1+flush-259:020:20:232
1871569991612,3cyclictest1902598-21kworker/u8:2+flush-259:020:30:112
1871569991611,3cyclictest2063968-21latency_hist22:15:012
187156599169,5cyclictest2095687-21latency_hist22:45:001
1871565991614,1cyclictest1902598-21kworker/u8:2+events_unbound20:24:581
1871565991613,2cyclictest2188864-21kworker/u8:2+events_unbound00:27:401
1871565991613,2cyclictest2038222-21kworker/u8:1+events_unbound22:05:281
1871565991613,2cyclictest1996773-21kworker/u8:3+events_unbound21:23:261
1871565991613,2cyclictest1987020-21kthreadcore21:00:161
1871565991613,2cyclictest1959634-21kworker/u8:4+events_unbound20:45:551
1871565991613,2cyclictest1856388-21kworker/u8:4+events_unbound19:21:581
187155999169,5cyclictest2027022-21ntpq21:35:210
1871559991614,2cyclictest2094825-21kworker/u8:0+events_unbound23:00:000
1871559991614,1cyclictest0-21swapper/023:50:000
1871559991613,2cyclictest60350irq/132-enp0s29f1:rx-020:10:270
1871559991613,2cyclictest60350irq/132-enp0s29f1:rx-019:40:230
1871559991613,2cyclictest2199162-21kworker/u8:0+flush-259:000:25:240
1871573991512,2cyclictest2199162-21kworker/u8:0+flush-259:000:30:003
1871573991512,2cyclictest1902598-21kworker/u8:2+flush-259:020:30:153
1871573991512,2cyclictest1902598-21kworker/u8:2+events_unbound20:40:133
1871573991511,3cyclictest1953332-21kworker/u8:3+flush-259:020:40:003
1871569991512,2cyclictest2152961-21kworker/u8:4+flush-259:023:55:252
1871569991512,2cyclictest2064193-21kworker/u8:0+events_unbound22:30:202
1871565991513,2cyclictest1959634-21kworker/u8:4+events_unbound21:33:071
1871565991512,2cyclictest2152961-21kworker/u8:4+events_unbound23:52:121
1871565991512,2cyclictest2152961-21kworker/u8:4+events_unbound00:10:341
1871565991512,2cyclictest2038222-21kworker/u8:1+events_unbound21:56:091
1871565991512,2cyclictest1923212-21kworker/u8:0+events_unbound20:01:191
1871565991512,2cyclictest1866428-21kworker/u8:0+events_unbound19:18:531
1871565991512,1cyclictest0-21swapper/120:15:241
1871565991511,3cyclictest2121858-21sh23:10:011
1871565991510,3cyclictest509-21rs:main1
1871559991514,1cyclictest60350irq/132-enp0s29f1:rx-021:09:100
1871559991513,1cyclictest60350irq/132-enp0s29f1:rx-020:30:120
1871559991513,1cyclictest60350irq/132-enp0s29f1:rx-000:35:190
1871559991513,1cyclictest503-21tpm2-abrmd21:20:000
1871559991512,2cyclictest2096213-21fschecks_time22:45:140
1871559991512,2cyclictest1902598-21kworker/u8:2+events_unbound20:45:080
1871559991511,3cyclictest2074266-21kworker/u8:1+events_unbound22:36:110
1871559991511,3cyclictest1902598-21kworker/u8:2+events_unbound19:50:130
1871559991510,4cyclictest2152961-21kworker/u8:4+events_unbound00:07:110
1871573991411,2cyclictest1923212-21kworker/u8:0+flush-259:020:00:113
1871573991411,2cyclictest1856388-21kworker/u8:4+flush-259:019:35:163
1871573991410,3cyclictest2058825-21kworker/u8:2+events_unbound22:40:183
1871569991412,1cyclictest0-21swapper/223:50:142
1871569991412,1cyclictest0-21swapper/200:20:002
1871569991411,2cyclictest2126987-21sh23:15:002
1871569991410,3cyclictest2094825-21kworker/u8:0+events_unbound23:00:222
187156599146,3cyclictest0-21swapper/122:04:171
187156599145,3cyclictest32-21ksoftirqd/122:50:011
187156599143,2cyclictest32-21ksoftirqd/123:25:181
187156599142,3cyclictest32-21ksoftirqd/123:30:191
187156599142,3cyclictest32-21ksoftirqd/123:30:191
1871565991411,2cyclictest2017857-21kworker/u8:2+events_unbound21:45:431
1871565991411,2cyclictest1923212-21kworker/u8:0+events_unbound20:13:531
1871565991411,2cyclictest1897152-21kworker/u8:0+events_unbound19:38:041
187155999149,4cyclictest252-21systemd-journal20:17:000
1871559991413,1cyclictest60350irq/132-enp0s29f1:rx-022:19:230
1871559991413,1cyclictest60350irq/132-enp0s29f1:rx-020:05:170
1871559991413,1cyclictest60350irq/132-enp0s29f1:rx-019:19:280
1871559991413,1cyclictest60350irq/132-enp0s29f1:rx-000:19:120
1871559991412,1cyclictest60350irq/132-enp0s29f1:rx-020:50:150
1871559991410,3cyclictest1996773-21kworker/u8:3+events_unbound22:02:120
1871559991410,3cyclictest1959634-21kworker/u8:4+events_unbound21:45:220
1871559991410,3cyclictest1856388-21kworker/u8:4+flush-259:019:35:200
187155999140,2cyclictest1-21systemd19:13:000
187157399138,3cyclictest2080073-21/usr/sbin/munin22:30:253
187156999139,3cyclictest2188864-21kworker/u8:2+events_unbound00:35:232
187156999138,4cyclictest2090640-21base6422:40:022
187156999138,4cyclictest0-21swapper/200:23:312
187156999138,3cyclictest252-21systemd-journal23:27:102
187156999138,3cyclictest1913099-21/usr/sbin/munin19:50:142
1871569991310,2cyclictest500-21systemd-logind19:50:012
187156599132,4cyclictest32-21ksoftirqd/100:05:201
187156599131,3cyclictest32-21ksoftirqd/123:55:241
1871565991311,2cyclictest1959634-21kworker/u8:4+events_unbound21:35:011
1871565991310,1cyclictest191rcu_preempt22:15:241
187156599130,4cyclictest2205582-21kthreadcore00:30:161
1871559991310,2cyclictest61150irq/133-enp0s29f1:tx-020:25:130
1871559991310,2cyclictest60350irq/132-enp0s29f1:rx-022:30:280
1871559991310,2cyclictest1985722-21idleruntime-cro21:00:000
187157399129,2cyclictest2085005-21kworker/u8:3+events_unbound22:38:113
187157399129,2cyclictest2038222-21kworker/u8:1+events_unbound22:10:223
187157399129,2cyclictest2032128-21ntp_kernel_err21:40:223
187157399129,2cyclictest1996773-21kworker/u8:3+events_unbound22:03:013
187157399129,2cyclictest1866428-21kworker/u8:0+events_unbound19:15:163
187157399128,3cyclictest2137559-21apt-get23:25:123
187157399128,3cyclictest1996773-21kworker/u8:3+flush-259:022:05:163
187157399128,2cyclictest17650irq/131-nvme0q421:10:003
187157399126,4cyclictest252-21systemd-journal21:55:003
187156999129,1cyclictest0-21swapper/219:30:172
187156999128,3cyclictest2199162-21kworker/u8:0+events_unbound00:25:212
187156999120,10cyclictest2183711-21cron00:10:002
187156599129,2cyclictest1902598-21kworker/u8:2+events_unbound19:55:101
187155999129,2cyclictest2094825-21kworker/u8:0+events_unbound23:15:190
1871559991211,1cyclictest60350irq/132-enp0s29f1:rx-020:23:310
1871559991210,2cyclictest60350irq/132-enp0s29f1:rx-000:00:220
1871559991210,1cyclictest60350irq/132-enp0s29f1:rx-023:25:250
187157399119,1cyclictest1912873-21munin-plugin-st19:50:013
187157399118,2cyclictest2058825-21kworker/u8:2+flush-259:022:54:593
187157399118,2cyclictest1959634-21kworker/u8:4+events_unbound21:20:163
187157399118,2cyclictest1942963-21ntp_kernel_pll_20:15:223
187157399118,2cyclictest0-21swapper/321:25:253
187157399117,3cyclictest2058825-21kworker/u8:2+flush-259:023:03:153
187156999118,2cyclictest0-21swapper/223:35:012
187156999118,2cyclictest0-21swapper/223:35:002
187156999117,3cyclictest2058825-21kworker/u8:2+flush-259:022:20:212
187155999119,1cyclictest60350irq/132-enp0s29f1:rx-020:00:370
187155999119,1cyclictest60350irq/132-enp0s29f1:rx-019:55:240
187157399107,2cyclictest2002071-21cpuspeed_turbos21:15:123
187156999107,2cyclictest1887511-21fschecks_count19:25:152
18715739995,3cyclictest2210120-21cat00:35:183
18715699997,1cyclictest0-21swapper/221:50:212
18715699995,3cyclictest1996773-21kworker/u8:3+flush-259:022:05:162
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional