You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-30 - 02:32

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot4s.osadl.org (updated Wed Apr 29, 2026 12:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2038073994442,1cyclictest2266531-21fwupd10:46:273
2038067994137,3cyclictest2126404-21kworker/u8:3+flush-259:008:50:151
2038070993836,2cyclictest2246187-21kworker/u8:4+events_unbound11:30:022
2038070993834,3cyclictest2234736-21kworker/u8:0+events_unbound10:35:022
2038067993834,3cyclictest2147020-21kworker/u8:2+events_unbound09:25:231
2038067993834,3cyclictest2085036-21kworker/u8:2+events_unbound08:25:211
2038067993731,2cyclictest2183683-21kworker/u8:1+events_unbound10:20:211
2038063993730,5cyclictest2167602-21latency_hist09:15:010
2038070993633,2cyclictest2157328-21kworker/u8:3+events_unbound09:54:062
2038070993633,2cyclictest2053935-21kworker/u8:3+events_unbound07:40:252
2038070993530,4cyclictest2048872-21cat07:20:012
2038067993532,2cyclictest2105400-21kworker/u8:1+flush-259:008:30:241
2038067993529,2cyclictest2147020-21kworker/u8:2+events_unbound08:55:121
2038070993431,2cyclictest2167642-21kworker/u8:1+events_unbound09:18:032
2038070993431,2cyclictest2126404-21kworker/u8:3+events_unbound08:40:232
2038070993430,3cyclictest2085036-21kworker/u8:2+events_unbound08:27:062
2038070993430,3cyclictest2079699-21kworker/u8:0+events_unbound07:56:072
2038067993427,3cyclictest2313963-21kworker/u8:3+events_unbound11:50:221
2038067993330,2cyclictest2131773-21kworker/u8:0+events_unbound09:10:211
2038067993329,3cyclictest2157328-21kworker/u8:3+flush-259:009:20:051
2038067993327,2cyclictest2053935-21kworker/u8:3+flush-259:007:50:001
2038070993229,2cyclictest2234736-21kworker/u8:0+events_unbound10:43:082
2038070993128,2cyclictest2183683-21kworker/u8:1+events_unbound11:28:062
2038067993127,3cyclictest2246187-21kworker/u8:4+flush-259:011:00:061
2038067993127,2cyclictest2053935-21kworker/u8:3+events_unbound07:30:131
2038067993125,4cyclictest2198073-21ntpq09:40:221
2038073993024,4cyclictest2235630-21cp10:20:013
2038070993028,2cyclictest2246187-21kworker/u8:4+events_unbound10:51:062
2038070993027,2cyclictest2267077-21kworker/u8:2+events_unbound10:57:062
2038070993026,3cyclictest2246187-21kworker/u8:4+events_unbound10:46:062
2038073992925,3cyclictest2147020-21kworker/u8:2+flush-259:009:30:003
2038070992927,2cyclictest2157328-21kworker/u8:3+events_unbound10:15:242
2038070992926,2cyclictest2147020-21kworker/u8:2+events_unbound08:56:062
2038067992926,2cyclictest2131773-21kworker/u8:0+flush-259:009:00:061
2038067992926,2cyclictest2126404-21kworker/u8:3+events_unbound08:45:181
2038067992925,3cyclictest2085036-21kworker/u8:2+flush-259:008:17:051
2038063992924,4cyclictest2152433-21apt-get09:00:110
2038070992826,2cyclictest2105400-21kworker/u8:1+events_unbound08:33:052
2038070992825,2cyclictest2183683-21kworker/u8:1+events_unbound11:12:072
2038070992825,2cyclictest2022701-21kworker/u8:1+events_unbound07:37:072
2038070992822,4cyclictest2178306-21latency_hist09:25:002
2038067992825,2cyclictest2350292-21kworker/u8:2+flush-259:012:35:231
2038067992825,2cyclictest2246187-21kworker/u8:4+flush-259:012:10:161
2038067992825,2cyclictest2198971-21kworker/u8:0+flush-259:010:00:161
2038067992825,2cyclictest2053935-21kworker/u8:3+flush-259:007:25:251
2038067992823,2cyclictest2361028-21kworker/u8:3+events_unbound12:20:151
2038070992724,2cyclictest2183683-21kworker/u8:1+events_unbound11:07:072
2038067992724,2cyclictest2323940-21kworker/u8:1+flush-259:012:25:161
2038067992724,2cyclictest2032917-21kworker/u8:0+events_unbound07:20:211
2038073992622,3cyclictest2022701-21kworker/u8:1+flush-259:007:20:013
2038073992620,4cyclictest2183877-21diskstats09:30:153
2038070992623,2cyclictest2375411-21kworker/u8:0+events_unbound12:33:062
2038067992623,2cyclictest2219719-21kworker/u8:4+events_unbound10:08:121
2038067992623,2cyclictest2022701-21kworker/u8:1+events_unbound07:15:221
2038067992620,2cyclictest2234736-21kworker/u8:0+events_unbound10:45:001
2038063992624,1cyclictest2240660-21kworker/0:0+pm10:46:290
2038073992520,3cyclictest2345310-21latency_hist12:05:013
2038073992519,4cyclictest252-21systemd-journal10:10:003
2038070992522,2cyclictest2198971-21kworker/u8:0+events_unbound10:03:102
2038070992522,2cyclictest2022701-21kworker/u8:1+events_unbound07:27:402
2038067992523,2cyclictest2183683-21kworker/u8:1+flush-259:011:28:251
2038067992521,3cyclictest2246187-21kworker/u8:4+events_unbound10:46:281
2038063992522,2cyclictest2064481-21apt-get07:35:100
2038070992422,2cyclictest2095118-21kworker/u8:3+events_unbound08:16:052
2038070992421,2cyclictest2339700-21kworker/u8:0+events_unbound12:02:072
2038067992421,2cyclictest2323940-21kworker/u8:1+events_unbound12:15:131
2038067992421,2cyclictest2313963-21kworker/u8:3+events_unbound11:45:231
2038067992421,2cyclictest2298483-21kworker/u8:0+events_unbound11:30:221
2038067992421,2cyclictest2183683-21kworker/u8:1+flush-259:009:50:151
2038073992317,4cyclictest252-21systemd-journal07:27:053
2038073992317,4cyclictest2298246-21latency_hist11:20:003
2038070992321,2cyclictest2313963-21kworker/u8:3+events_unbound11:49:072
2038070992321,2cyclictest2313963-21kworker/u8:3+events_unbound11:39:062
2038070992319,3cyclictest2183683-21kworker/u8:1+events_unbound10:13:082
2038067992320,2cyclictest2183683-21kworker/u8:1+flush-259:011:05:211
2038067992320,2cyclictest2079699-21kworker/u8:0+events_unbound08:00:221
2038067992320,2cyclictest2022701-21kworker/u8:1+events_unbound07:35:181
2038063992317,4cyclictest2260880-21ntpq10:40:210
2038073992219,2cyclictest2033106-21kworker/u8:4+flush-259:007:10:213
2038070992219,2cyclictest2361028-21kworker/u8:3+events_unbound12:20:202
2038070992219,2cyclictest2079699-21kworker/u8:0+events_unbound08:01:062
2038070992216,2cyclictest2085036-21kworker/u8:2+events_unbound08:13:062
2038067992220,2cyclictest2323940-21kworker/u8:1+events_unbound11:55:161
2038067992219,2cyclictest2246187-21kworker/u8:4+flush-259:010:50:271
2038067992219,2cyclictest2183683-21kworker/u8:1+events_unbound10:10:221
2038067992219,2cyclictest2085036-21kworker/u8:2+events_unbound08:10:221
2038067992216,4cyclictest2293499-21diskstats11:15:151
2038063992216,5cyclictest2172329-21ntpq09:15:210
2038063992216,5cyclictest2110522-21latency_hist08:20:010
203806399220,21cyclictest1-21systemd12:26:060
2038073992115,4cyclictest2100482-21/usr/sbin/munin08:10:153
2038073992114,5cyclictest2089047-21ntpq07:55:213
2038067992118,2cyclictest2157328-21kworker/u8:3+flush-259:010:15:211
2038067992118,2cyclictest2079699-21kworker/u8:0+flush-259:008:05:211
2038067992118,2cyclictest2079699-21kworker/u8:0+events_unbound07:55:211
2038067992118,2cyclictest2022701-21kworker/u8:1+flush-259:007:50:051
2038073992016,3cyclictest2235112-21gpgv10:20:013
2038070992018,2cyclictest2323940-21kworker/u8:1+events_unbound12:16:062
2038070992018,2cyclictest2298483-21kworker/u8:0+events_unbound11:21:062
2038070992017,2cyclictest2313963-21kworker/u8:3+events_unbound11:51:062
2038070992017,2cyclictest2183683-21kworker/u8:1+events_unbound10:07:062
2038070992017,2cyclictest2131773-21kworker/u8:0+events_unbound08:52:062
2038070992015,4cyclictest2037996-21latency_hist07:10:012
2038067992017,2cyclictest2350292-21kworker/u8:2+flush-259:012:30:211
2038067992017,2cyclictest2313963-21kworker/u8:3+flush-259:011:40:051
2038067992017,2cyclictest2234736-21kworker/u8:0+events_unbound10:35:121
2038067992017,2cyclictest2147020-21kworker/u8:2+events_unbound09:05:121
2038067992017,2cyclictest2053935-21kworker/u8:3+events_unbound07:40:161
2038067992016,3cyclictest2147020-21kworker/u8:2+events_unbound09:35:051
2038067992015,4cyclictest2214398-21latency_hist10:00:021
2038067992013,1cyclictest2350473-21munin-run12:10:001
2038063992015,3cyclictest2329552-21/usr/sbin/munin11:50:190
2038073991916,2cyclictest2168717-21irqcore09:15:153
2038073991912,6cyclictest2115166-21ntpq08:20:183
2038070991917,2cyclictest2085036-21kworker/u8:2+events_unbound08:23:072
2038070991916,2cyclictest2313963-21kworker/u8:3+events_unbound11:41:042
2038070991916,2cyclictest2246187-21kworker/u8:4+events_unbound11:58:062
2038070991916,2cyclictest2022701-21kworker/u8:1+events_unbound07:12:072
2038070991913,2cyclictest2234736-21kworker/u8:0+events_unbound10:33:162
2038067991916,2cyclictest2157328-21kworker/u8:3+flush-259:009:30:201
2038063991915,3cyclictest2183683-21kworker/u8:1+flush-259:010:30:170
2038073991815,2cyclictest2323940-21kworker/u8:1+flush-259:012:10:243
2038070991816,2cyclictest2105400-21kworker/u8:1+events_unbound08:35:032
2038070991815,2cyclictest2323940-21kworker/u8:1+events_unbound12:25:302
2038070991815,2cyclictest2246187-21kworker/u8:4+events_unbound12:07:062
2038070991815,2cyclictest2183683-21kworker/u8:1+events_unbound09:43:062
2038070991812,3cyclictest2147020-21kworker/u8:2+events_unbound09:29:202
2038067991816,1cyclictest2105400-21kworker/u8:1+events_unbound08:20:131
2038067991815,2cyclictest2246186-21kworker/u8:2+flush-259:010:30:001
2038067991815,2cyclictest2126404-21kworker/u8:3+flush-259:008:40:181
2038063991814,3cyclictest2323940-21kworker/u8:1+flush-259:012:13:360
2038063991814,3cyclictest2157328-21kworker/u8:3+flush-259:010:15:000
2038063991814,3cyclictest2147020-21kworker/u8:2+flush-259:009:35:000
2038073991711,4cyclictest509-21rs:main3
2038073991711,3cyclictest2313963-21kworker/u8:3+flush-259:011:45:153
2038073991710,5cyclictest2360801-21latency_hist12:20:013
203807099176,2cyclictest0-21swapper/209:45:142
2038070991715,2cyclictest2053935-21kworker/u8:3+events_unbound07:46:072
2038070991714,2cyclictest2131773-21kworker/u8:0+events_unbound09:07:052
2038070991714,2cyclictest2126404-21kworker/u8:3+events_unbound08:47:142
2038070991714,2cyclictest2022701-21kworker/u8:1+events_unbound07:16:072
2038070991713,1cyclictest191rcu_preempt10:25:242
2038070991712,4cyclictest2240628-21gpgv10:24:592
2038070991712,2cyclictest2079699-21kworker/u8:0+events_unbound07:50:592
203807099170,15cyclictest1-21systemd12:13:042
2038067991714,2cyclictest2323940-21kworker/u8:1+events_unbound12:00:181
2038067991714,2cyclictest2246187-21kworker/u8:4+events_unbound10:55:131
2038067991712,2cyclictest2246187-21kworker/u8:4+events_unbound11:35:281
2038067991711,2cyclictest2157328-21kworker/u8:3+events_unbound09:15:021
2038063991714,2cyclictest60350irq/132-enp0s29f1:rx-009:55:220
2038063991714,2cyclictest2147020-21kworker/u8:2+flush-259:009:35:210
2038063991713,2cyclictest60350irq/132-enp0s29f1:rx-008:20:010
2038063991712,4cyclictest252-21systemd-journal09:00:010
203807399169,6cyclictest2157305-21latency_hist09:05:003
2038073991613,2cyclictest2204095-21idleruntime-cro09:50:013
2038073991613,2cyclictest2183683-21kworker/u8:1+flush-259:010:25:123
2038073991613,2cyclictest2167602-21latency_hist09:15:003
2038073991613,2cyclictest2116759-21irqcore08:25:163
2038073991612,3cyclictest2126404-21kworker/u8:3+events_unbound08:50:243
2038073991612,3cyclictest2065496-21kthreadcore07:35:143
2038073991612,3cyclictest0-21swapper/311:55:163
2038073991612,2cyclictest0-21swapper/311:40:013
2038073991610,4cyclictest2345550-21/usr/sbin/munin12:05:203
2038073991610,4cyclictest2313963-21kworker/u8:3+flush-259:011:40:153
203807099169,1cyclictest191rcu_preempt09:35:222
2038070991614,2cyclictest2246187-21kworker/u8:4+events_unbound11:16:072
2038070991613,2cyclictest2323940-21kworker/u8:1+events_unbound12:36:302
2038070991612,2cyclictest2022701-21kworker/u8:1+events_unbound07:32:242
2038067991613,2cyclictest2246187-21kworker/u8:4+events_unbound11:10:121
2038063991614,1cyclictest60350irq/132-enp0s29f1:rx-010:15:190
2038063991614,1cyclictest60350irq/132-enp0s29f1:rx-009:40:160
2038063991614,1cyclictest60350irq/132-enp0s29f1:rx-007:15:150
2038063991614,1cyclictest60350irq/132-enp0s29f1:rx-007:10:170
2038063991613,2cyclictest60350irq/132-enp0s29f1:rx-008:50:160
2038063991613,2cyclictest60350irq/132-enp0s29f1:rx-008:30:160
2038063991613,2cyclictest2345315-21turbostat12:05:010
2038063991610,5cyclictest2208336-21ntpq09:50:220
203807399158,5cyclictest252-21systemd-journal07:35:003
2038073991512,2cyclictest2147693-21irqcore08:55:163
2038073991510,4cyclictest252-21systemd-journal08:44:553
2038073991510,3cyclictest2308797-21/usr/sbin/munin11:30:133
2038073991510,3cyclictest2261712-21sh10:45:003
203807099155,5cyclictest0-21swapper/211:00:222
203807099155,1cyclictest0-21swapper/209:30:142
2038070991512,2cyclictest2157328-21kworker/u8:3+events_unbound09:59:572
2038070991512,2cyclictest2131773-21kworker/u8:0+events_unbound09:10:412
2038070991512,2cyclictest2079699-21kworker/u8:0+events_unbound08:05:102
2038067991512,2cyclictest2183683-21kworker/u8:1+events_unbound10:30:191
2038067991511,3cyclictest2183683-21kworker/u8:1+events_unbound09:47:061
203806399158,5cyclictest2318932-21latency_hist11:40:000
2038063991514,1cyclictest60350irq/132-enp0s29f1:rx-011:07:120
2038063991513,2cyclictest60350irq/132-enp0s29f1:rx-011:10:420
2038063991513,1cyclictest60350irq/132-enp0s29f1:rx-011:25:140
2038063991513,1cyclictest60350irq/132-enp0s29f1:rx-008:10:240
2038063991513,1cyclictest60350irq/132-enp0s29f1:rx-007:50:110
2038063991512,2cyclictest60350irq/132-enp0s29f1:rx-011:55:170
2038063991512,2cyclictest60350irq/132-enp0s29f1:rx-009:05:200
2038063991512,2cyclictest60350irq/132-enp0s29f1:rx-008:40:190
2038063991512,2cyclictest2277184-21idleruntime-cro10:59:590
2038063991510,3cyclictest2157328-21kworker/u8:3+flush-259:009:30:010
203807399148,5cyclictest2121462-21/usr/sbin/munin08:30:143
203807399148,4cyclictest509-21rs:main3
203807399146,5cyclictest252-21systemd-journal08:04:593
2038073991410,2cyclictest0-21swapper/308:20:003
203807099147,1cyclictest191rcu_preempt09:00:252
2038067991410,3cyclictest2298483-21kworker/u8:0+events_unbound11:22:081
203806399148,4cyclictest252-21systemd-journal10:10:010
2038063991413,1cyclictest60350irq/132-enp0s29f1:rx-010:35:180
2038063991413,1cyclictest60350irq/132-enp0s29f1:rx-007:28:560
2038063991412,1cyclictest60350irq/132-enp0s29f1:rx-012:20:200
2038063991412,1cyclictest60350irq/132-enp0s29f1:rx-011:48:310
2038063991412,1cyclictest60350irq/132-enp0s29f1:rx-010:25:120
2038063991412,1cyclictest60350irq/132-enp0s29f1:rx-009:45:240
2038063991412,1cyclictest60350irq/132-enp0s29f1:rx-008:25:150
2038063991412,1cyclictest60350irq/132-enp0s29f1:rx-007:40:120
2038063991412,1cyclictest60350irq/132-enp0s29f1:rx-007:30:280
2038063991411,2cyclictest61150irq/133-enp0s29f1:tx-008:00:160
2038063991411,2cyclictest60350irq/132-enp0s29f1:rx-012:35:180
2038063991411,2cyclictest60350irq/132-enp0s29f1:rx-011:30:220
2038063991411,2cyclictest60350irq/132-enp0s29f1:rx-010:50:180
2038063991411,2cyclictest2371485-21cpuspeed_turbos12:30:130
203807399138,3cyclictest2350292-21kworker/u8:2+flush-259:012:30:253
203807399137,4cyclictest0-21swapper/312:25:013
203807399136,5cyclictest490-21dbus-daemon11:55:013
203807399136,5cyclictest2209244-21awk09:55:003
203807399135,6cyclictest2213537-21ntpq09:55:223
203806399137,5cyclictest2240789-21cat10:25:010
2038063991311,1cyclictest60350irq/132-enp0s29f1:rx-012:15:110
2038063991311,1cyclictest60350irq/132-enp0s29f1:rx-012:07:350
2038063991311,1cyclictest60350irq/132-enp0s29f1:rx-009:20:230
2038063991311,1cyclictest60350irq/132-enp0s29f1:rx-007:55:190
2038063991310,2cyclictest60350irq/132-enp0s29f1:rx-008:35:160
203807399129,2cyclictest500-21systemd-logind08:09:593
203807399129,2cyclictest2073579-21cut07:40:203
203807399128,2cyclictest2288796-21kthreadcore11:10:173
203807399126,4cyclictest2080102-21diskstats07:50:133
203807399121,4cyclictest22730301irqcore10:55:173
203806799129,2cyclictest2033106-21kworker/u8:4+flush-259:007:10:391
203806399129,2cyclictest60350irq/132-enp0s29f1:rx-008:45:120
2038063991210,1cyclictest60350irq/132-enp0s29f1:rx-011:20:130
2038063991210,1cyclictest60350irq/132-enp0s29f1:rx-011:00:220
2038063991210,1cyclictest60350irq/132-enp0s29f1:rx-008:05:170
203807399118,2cyclictest2303734-21cpuspeed_turbos11:25:123
203807399111,8cyclictest2229851-21awk10:15:013
203807399111,8cyclictest2193774-21latency_hist09:40:003
203807399107,2cyclictest2376677-21cpuspeed_turbos12:35:133
203807399107,2cyclictest2140861-21needreboot08:45:193
203807399106,2cyclictest2255071-21cut10:35:163
203807399106,2cyclictest0-21swapper/307:20:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional