OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #f, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails

Click here to display the system's profile data.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rackfslot7s.osadl.org (updated Wed Dec 03, 2025 00:44:18)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
858535	99	3028	15,3007	cyclictest	871968	-21	kworker/1:2+events	20:15:18	1
858533	99	2814	13,2796	cyclictest	932065	-21	kworker/0:1+events	23:05:01	0
858542	99	2698	13,2680	cyclictest	888950	-21	kworker/3:1+events	20:45:14	3
858539	99	2628	12,2612	cyclictest	900345	-21	kworker/2:0+events	21:25:13	2
858542	99	2497	12,2482	cyclictest	962513	-21	kworker/3:1+events	00:37:57	3
858535	99	2494	12,2478	cyclictest	898580	-21	kworker/1:2+events	21:15:18	1
858535	99	2475	8,2464	cyclictest	914756	-21	kworker/1:1+events	22:20:14	1
858535	99	2474	11,2459	cyclictest	871968	-21	kworker/1:2+events	19:57:38	1
858539	99	2458	12,2443	cyclictest	905089	-21	kworker/2:1+events	21:36:21	2
858533	99	2448	9,2437	cyclictest	893842	-21	kworker/0:0+events	21:10:01	0
858542	99	2441	12,2425	cyclictest	914701	-21	kworker/3:2+events	22:15:19	3
858539	99	2441	12,2425	cyclictest	957505	-21	kworker/2:3+events	00:37:34	2
858542	99	2438	12,2423	cyclictest	881149	-21	kworker/3:2+events	20:40:20	3
858542	99	2438	12,2423	cyclictest	881149	-21	kworker/3:2+events	20:40:19	3
858533	99	2438	12,2423	cyclictest	911309	-21	kworker/0:0+events	22:10:21	0
858533	99	2408	12,2392	cyclictest	943325	-21	kworker/0:1+events	23:39:58	0
858535	99	2403	10,2390	cyclictest	871968	-21	kworker/1:2+events	20:23:40	1
858535	99	2399	12,2384	cyclictest	959203	-21	kworker/1:1+events	00:25:20	1
858535	99	2394	11,2380	cyclictest	930363	-21	kworker/1:1+events	22:55:47	1
858535	99	2392	9,2379	cyclictest	919180	-21	kworker/1:0+events	22:19:53	1
858533	99	2390	10,2377	cyclictest	949420	-21	kworker/0:1+events	00:02:07	0
858535	99	2388	12,2373	cyclictest	898580	-21	kworker/1:2+events	21:12:18	1
858535	99	2387	11,2373	cyclictest	903330	-21	kworker/1:1+events	21:31:42	1
858535	99	2384	11,2370	cyclictest	940146	-21	kworker/1:0+events	23:24:50	1
858533	99	2365	12,2350	cyclictest	880931	-21	kworker/0:1+events	20:27:28	0
858542	99	2361	12,2346	cyclictest	866619	-21	kworker/3:0+events	19:48:06	3
858535	99	2353	12,2338	cyclictest	898580	-21	kworker/1:2+events	21:26:22	1
858542	99	2340	11,2326	cyclictest	938330	-21	kworker/3:0+events	23:25:13	3
858542	99	2339	12,2324	cyclictest	960852	-21	kworker/3:2+events	00:34:09	3
858542	99	2338	12,2323	cyclictest	901732	-21	kworker/3:2+events	21:24:53	3
858535	99	2331	10,2318	cyclictest	896713	-21	kworker/1:0+events	21:09:39	1
858539	99	2328	10,2315	cyclictest	951081	-21	kworker/2:2+events	23:57:53	2
858533	99	2325	12,2309	cyclictest	908080	-21	kworker/0:0+events	21:42:05	0
858535	99	2299	12,2284	cyclictest	945330	-21	kworker/1:0+events	23:44:54	1
858542	99	2298	10,2285	cyclictest	938330	-21	kworker/3:0+events	23:19:51	3
858535	99	2266	12,2251	cyclictest	909890	-21	kworker/1:1+events	21:51:31	1
858533	99	2265	11,2250	cyclictest	954365	-21	kworker/0:0+events	00:08:13	0
858542	99	2253	5,2245	cyclictest	954248	-21	kworker/3:0+events	00:12:25	3
858535	99	2244	11,2230	cyclictest	928699	-21	kworker/1:2+events	22:47:45	1
858533	99	2244	9,2232	cyclictest	888918	-21	kworker/0:2+events	21:32:05	0
858533	99	2241	12,2225	cyclictest	920895	-21	kworker/0:1+events	22:39:20	0
858533	99	2240	10,2227	cyclictest	890551	-21	kworker/0:1+events	20:57:46	0
858533	99	2238	10,2225	cyclictest	935793	-21	kworker/0:2+events	23:13:47	0
858535	99	2230	11,2215	cyclictest	885139	-21	kworker/1:2+events	20:31:59	1
858535	99	2225	11,2211	cyclictest	932166	-21	kworker/1:0+events	23:02:22	1
858533	99	2225	11,2210	cyclictest	914671	-21	kworker/0:2+events	22:09:35	0
858539	99	2221	11,2207	cyclictest	878134	-21	kworker/2:2+events	20:51:40	2
858533	99	2221	11,2208	cyclictest	957615	-21	kworker/0:2+events	00:20:02	0
858533	99	2221	11,2207	cyclictest	909709	-21	kworker/0:1+events	21:48:30	0
858533	99	2220	11,2206	cyclictest	933686	-21	kworker/0:0+events	23:09:33	0
858533	99	2216	10,2203	cyclictest	928790	-21	kworker/0:2+events	22:54:16	0
858539	99	2215	10,2201	cyclictest	873063	-21	kworker/2:2+events	19:56:32	2
858533	99	2214	11,2200	cyclictest	869626	-21	kworker/0:2+events	19:45:29	0
858533	99	2214	10,2201	cyclictest	880931	-21	kworker/0:1+events	20:33:52	0
858539	99	2209	10,2195	cyclictest	878134	-21	kworker/2:2+events	20:27:04	2
858533	99	2201	10,2188	cyclictest	863185	-21	kworker/0:2+events	19:23:38	0
858533	99	2199	11,2186	cyclictest	888918	-21	kworker/0:2+events	21:35:11	0
858533	99	2199	10,2186	cyclictest	893842	-21	kworker/0:0+events	21:05:48	0
858535	99	2195	11,2181	cyclictest	959203	-21	kworker/1:1+events	00:30:46	1
858542	99	2193	9,2181	cyclictest	918115	-21	kworker/3:1+events	22:20:37	3
858533	99	2190	10,2176	cyclictest	893842	-21	kworker/0:0+events	21:04:19	0
858542	99	2188	9,2175	cyclictest	866619	-21	kworker/3:0+events	19:35:39	3
858535	99	2188	10,2175	cyclictest	930363	-21	kworker/1:1+events	22:53:52	1
858535	99	2187	10,2171	cyclictest	871968	-21	kworker/1:2+events	20:06:13	1
858535	99	2186	10,2173	cyclictest	912769	-21	kworker/1:0+events	22:03:30	1
858535	99	2185	10,2171	cyclictest	906527	-21	kworker/1:2+events	21:49:16	1
858535	99	2184	9,2172	cyclictest	922499	-21	kworker/1:0+events	22:31:42	1
858535	99	2184	11,2170	cyclictest	940146	-21	kworker/1:0+events	23:29:47	1
858535	99	2183	9,2172	cyclictest	888888	-21	kworker/1:0+events	20:50:08	1
858533	99	2183	11,2169	cyclictest	864784	-21	kworker/0:0+events	19:36:45	0
858535	99	2182	7,2171	cyclictest	941840	-21	kworker/1:2+events	23:57:30	1
858535	99	2182	11,2167	cyclictest	954183	-21	kworker/1:0+events	00:13:59	1
858535	99	2182	10,2169	cyclictest	924292	-21	kworker/1:1+events	22:38:57	1
858533	99	2182	11,2168	cyclictest	920895	-21	kworker/0:1+events	22:24:03	0
858535	99	2180	11,2165	cyclictest	865052	-21	kworker/1:1+events	19:39:47	1
858535	99	2180	10,2167	cyclictest	954183	-21	kworker/1:0+events	00:23:47	1
858533	99	2180	7,2169	cyclictest	898608	-21	kworker/0:1+events	21:16:48	0
858533	99	2178	9,2166	cyclictest	938370	-21	kworker/0:0+events	23:19:05	0
858535	99	2177	10,2164	cyclictest	887210	-21	kworker/1:1+events	20:46:44	1
858535	99	2175	11,2161	cyclictest	871968	-21	kworker/1:2+events	19:53:07	1
858535	99	2173	11,2158	cyclictest	896713	-21	kworker/1:0+events	21:20:16	1
858535	99	2171	11,2157	cyclictest	941840	-21	kworker/1:2+events	00:00:12	1
858542	99	2170	8,2159	cyclictest	931941	-21	kworker/3:2+events	23:04:13	3
858535	99	2166	10,2153	cyclictest	906527	-21	kworker/1:2+events	21:40:32	1
858533	99	2166	10,2153	cyclictest	909709	-21	kworker/0:1+events	21:50:48	0
858542	99	2165	11,2151	cyclictest	903362	-21	kworker/3:1+events	21:37:29	3
858533	99	2164	11,2150	cyclictest	960883	-21	kworker/0:0+events	00:30:23	0
858539	99	2163	10,2150	cyclictest	936888	-21	kworker/2:2+events	23:15:18	2
858539	99	2159	11,2145	cyclictest	920803	-21	kworker/2:1+events	22:25:11	2
858535	99	2158	10,2144	cyclictest	930363	-21	kworker/1:1+events	23:11:53	1
858539	99	2155	11,2141	cyclictest	865084	-21	kworker/2:1+events	19:33:48	2
858539	99	2153	11,2139	cyclictest	940236	-21	kworker/2:2+events	23:43:00	2
858539	99	2153	11,2139	cyclictest	933763	-21	kworker/2:0+events	23:06:09	2
858539	99	2153	10,2140	cyclictest	957505	-21	kworker/2:3+events	00:23:03	2
858539	99	2152	11,2137	cyclictest	928608	-21	kworker/2:1+events	22:51:11	2
858539	99	2152	10,2139	cyclictest	863362	-21	kworker/2:0+events	19:24:43	2
858539	99	2150	10,2137	cyclictest	902296	-21	kworker/2:2+events	21:43:14	2
858539	99	2149	10,2136	cyclictest	911280	-21	kworker/2:2+events	22:07:42	2
858542	99	2144	10,2131	cyclictest	949817	-21	kworker/3:2+events	00:04:46	3
858539	99	2142	11,2128	cyclictest	878134	-21	kworker/2:2+events	20:49:46	2
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2024-07-29 - 23:00