OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #f, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails

Click here to display the system's profile data.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rackfslot7s.osadl.org (updated Wed Dec 31, 2025 00:44:16)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
363565	99	5885	0,5881	cyclictest	359538	-21	kworker/3:1+events	19:12:03	3
363555	99	5857	24,5830	cyclictest	351956	-21	kworker/0:0+events	19:11:40	0
363559	99	5773	18,5751	cyclictest	361804	-21	kworker/1:1+events	19:12:24	1
363565	99	3219	12,3201	cyclictest	370449	-21	kworker/3:0+events	19:35:01	3
363561	99	3059	15,3040	cyclictest	368336	-21	kworker/2:1+events	19:55:00	2
363559	99	2765	10,2750	cyclictest	420535	-21	kworker/1:0+events	22:05:16	1
363559	99	2764	13,2746	cyclictest	450728	-21	kworker/1:0+events	00:25:19	1
363565	99	2690	12,2674	cyclictest	449796	-21	kworker/3:0+events	23:45:21	3
363565	99	2497	14,2480	cyclictest	460676	-21	kworker/3:2+events	00:20:01	3
363559	99	2492	12,2477	cyclictest	425286	-21	kworker/1:1+events	22:35:15	1
363559	99	2471	12,2456	cyclictest	388384	-21	kworker/1:2+events	20:30:22	1
363561	99	2440	9,2427	cyclictest	397649	-21	kworker/2:2+events	21:00:17	2
363559	99	2377	11,2361	cyclictest	468773	-21	kworker/1:2+events	00:35:13	1
363559	99	2332	12,2317	cyclictest	442459	-21	kworker/1:0+events	23:15:14	1
363559	99	2308	11,2294	cyclictest	386762	-21	kworker/1:0+events	20:22:02	1
363559	99	2304	8,2293	cyclictest	396101	-21	kworker/1:0+events	20:51:07	1
363565	99	2247	9,2233	cyclictest	407807	-21	kworker/3:2+events	21:39:32	3
363565	99	2245	11,2231	cyclictest	442640	-21	kworker/3:2+events	23:22:52	3
363561	99	2243	11,2228	cyclictest	440851	-21	kworker/2:2+events	23:10:16	2
363559	99	2227	10,2213	cyclictest	417281	-21	kworker/1:0+events	21:56:09	1
363565	99	2226	11,2211	cyclictest	373708	-21	kworker/3:1+events	19:44:28	3
363555	99	2226	11,2211	cyclictest	434475	-21	kworker/0:2+events	22:51:29	0
363555	99	2201	11,2187	cyclictest	384974	-21	kworker/0:2+events	20:45:20	0
363565	99	2168	10,2155	cyclictest	370449	-21	kworker/3:0+events	19:38:00	3
363565	99	2162	11,2147	cyclictest	379975	-21	kworker/3:0+events	20:05:12	3
363561	99	2159	9,2146	cyclictest	437805	-21	kworker/2:1+events	23:02:11	2
363559	99	2136	9,2124	cyclictest	442459	-21	kworker/1:0+events	23:13:20	1
363561	99	2125	9,2110	cyclictest	452205	-21	kworker/2:0+events	00:13:32	2
363559	99	2123	10,2109	cyclictest	385125	-21	kworker/1:2+events	20:18:13	1
363559	99	2116	9,2104	cyclictest	415661	-21	kworker/1:1+events	21:53:30	1
363559	99	2115	10,2102	cyclictest	414060	-21	kworker/1:2+events	21:46:25	1
363559	99	2114	10,2101	cyclictest	436235	-21	kworker/1:0+events	22:54:13	1
363559	99	2114	10,2101	cyclictest	378090	-21	kworker/1:1+events	19:59:31	1
363559	99	2113	6,2103	cyclictest	404795	-21	kworker/1:0+events	21:22:29	1
363559	99	2112	9,2100	cyclictest	414060	-21	kworker/1:2+events	21:43:23	1
363559	99	2110	10,2097	cyclictest	407569	-21	kworker/1:1+events	21:30:21	1
363559	99	2108	8,2097	cyclictest	425286	-21	kworker/1:1+events	22:44:15	1
363559	99	2108	6,2099	cyclictest	402887	-21	kworker/1:2+events	21:19:25	1
363559	99	2108	6,2099	cyclictest	372021	-21	kworker/1:2+events	19:47:55	1
363559	99	2108	10,2094	cyclictest	439613	-21	kworker/1:2+events	23:06:01	1
363559	99	2107	10,2094	cyclictest	423563	-21	kworker/1:2+events	22:14:48	1
363559	99	2106	6,2097	cyclictest	425286	-21	kworker/1:1+events	22:27:16	1
363559	99	2106	10,2093	cyclictest	420535	-21	kworker/1:0+events	22:15:11	1
363559	99	2106	10,2093	cyclictest	361804	-21	kworker/1:1+events	19:33:53	1
363559	99	2105	8,2094	cyclictest	388384	-21	kworker/1:2+events	20:48:49	1
363559	99	2104	10,2090	cyclictest	388384	-21	kworker/1:2+events	20:38:26	1
363559	99	2103	8,2092	cyclictest	409201	-21	kworker/1:2+events	21:29:36	1
363559	99	2103	10,2090	cyclictest	407569	-21	kworker/1:1+events	21:35:21	1
363559	99	2103	10,2089	cyclictest	450728	-21	kworker/1:0+events	00:18:28	1
363559	99	2103	10,2089	cyclictest	372021	-21	kworker/1:2+events	19:42:34	1
363559	99	2102	8,2090	cyclictest	378090	-21	kworker/1:1+events	19:52:23	1
363565	99	2101	11,2087	cyclictest	457030	-21	kworker/3:1+events	00:08:13	3
363559	99	2101	8,2090	cyclictest	453956	-21	kworker/1:1+events	23:52:15	1
363559	99	2100	8,2088	cyclictest	460350	-21	kworker/1:1+events	00:09:01	1
363559	99	2100	10,2087	cyclictest	379943	-21	kworker/1:2+events	20:05:58	1
363559	99	2099	10,2086	cyclictest	423563	-21	kworker/1:2+events	22:24:37	1
363559	99	2099	10,2086	cyclictest	399658	-21	kworker/1:0+events	21:04:30	1
363559	99	2096	5,2088	cyclictest	386762	-21	kworker/1:0+events	20:29:35	1
363559	99	2094	8,2083	cyclictest	447551	-21	kworker/1:2+events	23:48:00	1
363559	99	2092	10,2079	cyclictest	436235	-21	kworker/1:0+events	23:03:20	1
363565	99	2081	8,2070	cyclictest	442640	-21	kworker/3:2+events	23:36:33	3
363555	99	2072	5,2064	cyclictest	384974	-21	kworker/0:2+events	20:41:54	0
363561	99	2068	10,2055	cyclictest	423502	-21	kworker/2:1+events	22:29:34	2
363559	99	2067	7,2057	cyclictest	455590	-21	kworker/1:2+events	00:01:24	1
363559	99	2052	9,2040	cyclictest	466788	-21	kworker/1:1+events	00:30:12	1
363565	99	2051	9,2039	cyclictest	421865	-21	kworker/3:1+events	22:07:32	3
363559	99	2050	9,2038	cyclictest	434599	-21	kworker/1:2+events	22:57:37	1
363559	99	2050	10,2037	cyclictest	368460	-21	kworker/1:2+events	19:27:21	1
363565	99	2049	10,2036	cyclictest	376459	-21	kworker/3:2+events	19:58:01	3
363561	99	2047	10,2034	cyclictest	402555	-21	kworker/2:1+events	21:12:28	2
363559	99	2047	8,2035	cyclictest	447551	-21	kworker/1:2+events	23:40:24	1
363559	99	2047	7,2037	cyclictest	383197	-21	kworker/1:1+events	20:10:57	1
363559	99	2047	6,2038	cyclictest	361804	-21	kworker/1:1+events	19:24:17	1
363565	99	2046	10,2033	cyclictest	376459	-21	kworker/3:2+events	19:47:08	3
363561	99	2045	10,2032	cyclictest	428445	-21	kworker/2:2+events	22:56:28	2
363559	99	2045	7,2035	cyclictest	379943	-21	kworker/1:2+events	20:00:14	1
363559	99	2045	6,2037	cyclictest	447551	-21	kworker/1:2+events	23:33:30	1
363559	99	2045	10,2032	cyclictest	397767	-21	kworker/1:1+events	20:57:38	1
363559	99	2044	3,2038	cyclictest	465179	-21	kworker/1:2+events	00:22:19	1
363561	99	2043	8,2032	cyclictest	452205	-21	kworker/2:0+events	00:07:52	2
363559	99	2043	9,2031	cyclictest	425286	-21	kworker/1:1+events	22:31:05	1
363559	99	2043	9,2031	cyclictest	402887	-21	kworker/1:2+events	21:13:14	1
363559	99	2043	9,2031	cyclictest	361804	-21	kworker/1:1+events	19:16:34	1
363559	99	2043	3,2036	cyclictest	445832	-21	kworker/1:1+events	23:21:42	1
363565	99	2042	10,2029	cyclictest	460676	-21	kworker/3:2+events	00:12:00	3
363559	99	2042	2,2036	cyclictest	445832	-21	kworker/1:1+events	23:27:47	1
363565	99	2041	4,2034	cyclictest	453777	-21	kworker/3:2+events	00:02:09	3
363559	99	2041	9,2029	cyclictest	401284	-21	kworker/1:1+events	21:05:13	1
363559	99	2041	5,2033	cyclictest	394433	-21	kworker/1:1+events	20:42:16	1
363565	99	2040	9,2028	cyclictest	436082	-21	kworker/3:2+events	23:04:29	3
363561	99	2040	7,2031	cyclictest	378030	-21	kworker/2:2+events	20:20:09	2
363559	99	2040	9,2028	cyclictest	418884	-21	kworker/1:2+events	22:03:46	1
363555	99	2040	11,2026	cyclictest	404516	-21	kworker/0:0+events	21:12:52	0
363561	99	2039	10,2026	cyclictest	404543	-21	kworker/2:0+events	21:17:54	2
363565	99	2038	3,2032	cyclictest	401015	-21	kworker/3:0+events	21:18:16	3
363565	99	2038	10,2024	cyclictest	453777	-21	kworker/3:2+events	23:56:26	3
363561	99	2038	5,2030	cyclictest	428445	-21	kworker/2:2+events	22:35:37	2
363565	99	2036	10,2023	cyclictest	463643	-21	kworker/3:1+events	00:26:05	3
363561	99	2036	4,2029	cyclictest	378030	-21	kworker/2:2+events	20:09:01	2
363561	99	2036	10,2023	cyclictest	392895	-21	kworker/2:0+events	20:40:20	2
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2024-07-29 - 23:00