OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #f, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails

Click here to display the system's profile data.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rackfslot7s.osadl.org (updated Fri Jan 23, 2026 00:44:17)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
27891	99	2900	12,2884	cyclictest	49249	-21	kworker/0:2+events	20:35:19	0
27899	99	2768	13,2752	cyclictest	27301	-21	kworker/2:1+events	19:15:02	2
27894	99	2745	13,2728	cyclictest	105076	-21	kworker/1:1+events	23:25:13	1
27901	99	2711	11,2696	cyclictest	123532	-21	kworker/3:2+events	00:11:34	3
27894	99	2681	13,2664	cyclictest	45827	-21	kworker/1:2+events	20:10:21	1
27894	99	2681	13,2664	cyclictest	45827	-21	kworker/1:2+events	20:10:20	1
27899	99	2565	13,2549	cyclictest	108198	-21	kworker/2:0+events	23:20:13	2
27891	99	2549	12,2532	cyclictest	34400	-21	kworker/0:2+events	19:35:13	0
27891	99	2486	10,2473	cyclictest	104921	-21	kworker/0:2+events	23:15:16	0
27894	99	2482	11,2467	cyclictest	57074	-21	kworker/1:1+events	20:40:19	1
27901	99	2452	12,2436	cyclictest	37811	-21	kworker/3:1+events	20:00:14	3
27899	99	2451	12,2434	cyclictest	92474	-21	kworker/2:1+events	22:30:15	2
27894	99	2449	12,2433	cyclictest	117595	-21	kworker/1:2+events	23:50:19	1
27891	99	2438	12,2423	cyclictest	98771	-21	kworker/0:1+events	22:51:40	0
27901	99	2436	12,2421	cyclictest	66684	-21	kworker/3:1+events	21:07:46	3
27899	99	2435	12,2419	cyclictest	86163	-21	kworker/2:0+events	22:12:51	2
27899	99	2433	11,2418	cyclictest	73035	-21	kworker/2:2+events	21:28:26	2
27901	99	2432	10,2418	cyclictest	109620	-21	kworker/3:1+events	23:33:55	3
27899	99	2431	12,2415	cyclictest	42896	-21	kworker/2:1+events	19:57:10	2
27894	99	2430	11,2415	cyclictest	76253	-21	kworker/1:0+events	21:43:22	1
27894	99	2427	9,2415	cyclictest	66802	-21	kworker/1:2+events	21:10:47	1
27899	99	2425	7,2415	cyclictest	113032	-21	kworker/2:2+events	23:48:47	2
27901	99	2424	11,2409	cyclictest	44322	-21	kworker/3:0+events	20:07:23	3
27894	99	2424	11,2410	cyclictest	89095	-21	kworker/1:0+events	22:32:31	1
27901	99	2422	12,2407	cyclictest	36210	-21	kworker/3:2+events	19:38:58	3
27894	99	2422	12,2407	cyclictest	119232	-21	kworker/1:0+events	00:00:27	1
27894	99	2419	11,2404	cyclictest	89095	-21	kworker/1:0+events	22:19:14	1
27894	99	2411	6,2402	cyclictest	81161	-21	kworker/1:0+events	21:56:51	1
27894	99	2411	11,2396	cyclictest	57074	-21	kworker/1:1+events	20:36:30	1
27891	99	2411	10,2398	cyclictest	84500	-21	kworker/0:0+events	22:07:56	0
27899	99	2408	11,2393	cyclictest	80811	-21	kworker/2:1+events	21:59:32	2
27899	99	2404	12,2388	cyclictest	41122	-21	kworker/2:0+events	19:50:01	2
27894	99	2397	12,2381	cyclictest	124434	-21	kworker/1:2+events	00:20:17	1
27891	99	2388	12,2374	cyclictest	120949	-21	kworker/0:0+events	00:10:01	0
27891	99	2388	12,2373	cyclictest	92177	-21	kworker/0:0+events	22:26:54	0
27891	99	2384	11,2370	cyclictest	104921	-21	kworker/0:2+events	23:25:59	0
27891	99	2382	12,2367	cyclictest	111432	-21	kworker/0:0+events	23:30:30	0
27894	99	2356	8,2345	cyclictest	79477	-21	kworker/1:1+events	21:52:47	1
27901	99	2344	11,2330	cyclictest	68347	-21	kworker/3:2+events	21:33:01	3
27901	99	2340	12,2324	cyclictest	68347	-21	kworker/3:2+events	21:29:12	3
27901	99	2328	12,2312	cyclictest	68347	-21	kworker/3:2+events	21:23:29	3
27899	99	2319	10,2306	cyclictest	100361	-21	kworker/2:0+events	23:02:37	2
27899	99	2282	9,2270	cyclictest	76402	-21	kworker/2:1+events	21:40:42	2
27901	99	2271	11,2256	cyclictest	63721	-21	kworker/3:0+events	21:19:18	3
27891	99	2266	10,2253	cyclictest	65348	-21	kworker/0:2+events	21:32:39	0
27891	99	2265	12,2250	cyclictest	60480	-21	kworker/0:0+events	20:49:25	0
27899	99	2247	11,2233	cyclictest	71434	-21	kworker/2:1+events	21:31:06	2
27901	99	2245	11,2230	cyclictest	120890	-21	kworker/3:1+events	00:01:35	3
27901	99	2243	12,2228	cyclictest	89331	-21	kworker/3:2+events	22:28:02	3
27901	99	2241	10,2228	cyclictest	104983	-21	kworker/3:2+events	23:16:47	3
27899	99	2240	9,2227	cyclictest	100361	-21	kworker/2:0+events	23:10:18	2
27901	99	2239	8,2228	cyclictest	44322	-21	kworker/3:0+events	19:58:41	3
27901	99	2239	11,2225	cyclictest	82639	-21	kworker/3:1+events	22:02:34	3
27901	99	2238	9,2226	cyclictest	29101	-21	kworker/3:0+events	19:33:42	3
27901	99	2238	10,2224	cyclictest	132622	-21	kworker/3:2+events	00:36:34	3
27899	99	2238	10,2225	cyclictest	108198	-21	kworker/2:0+events	23:26:45	2
27901	99	2237	10,2223	cyclictest	104983	-21	kworker/3:2+events	23:11:27	3
27901	99	2236	9,2223	cyclictest	63721	-21	kworker/3:0+events	21:13:07	3
27891	99	2236	8,2224	cyclictest	65348	-21	kworker/0:2+events	22:17:23	0
27891	99	2236	10,2223	cyclictest	104921	-21	kworker/0:2+events	23:09:08	0
27901	99	2235	11,2221	cyclictest	29101	-21	kworker/3:0+events	19:23:28	3
27901	99	2235	10,2222	cyclictest	33035	-21	kworker/3:1+events	19:27:17	3
27901	99	2233	6,2223	cyclictest	81016	-21	kworker/3:0+events	21:54:13	3
27901	99	2233	10,2220	cyclictest	129379	-21	kworker/3:0+events	00:34:43	3
27899	99	2233	10,2219	cyclictest	84347	-21	kworker/2:2+events	22:02:58	2
27894	99	2233	10,2220	cyclictest	115186	-21	kworker/1:1+events	23:57:06	1
27901	99	2232	10,2219	cyclictest	120890	-21	kworker/3:1+events	00:05:26	3
27899	99	2232	10,2219	cyclictest	119126	-21	kworker/2:1+events	23:51:51	2
27901	99	2231	10,2217	cyclictest	119202	-21	kworker/3:2+events	23:55:37	3
27894	99	2231	10,2218	cyclictest	54312	-21	kworker/1:0+events	20:50:33	1
27899	99	2230	9,2218	cyclictest	90752	-21	kworker/2:2+events	22:24:14	2
27894	99	2230	11,2215	cyclictest	76253	-21	kworker/1:0+events	21:39:10	1
27901	99	2229	11,2215	cyclictest	104983	-21	kworker/3:2+events	23:36:57	3
27899	99	2229	11,2214	cyclictest	50788	-21	kworker/2:1+events	20:23:10	2
27894	99	2229	9,2216	cyclictest	77878	-21	kworker/1:2+events	22:00:38	1
27901	99	2228	11,2214	cyclictest	25825	-21	kworker/3:2+events	19:19:37	3
27899	99	2227	9,2215	cyclictest	73035	-21	kworker/2:2+events	21:37:14	2
27899	99	2227	6,2217	cyclictest	119126	-21	kworker/2:1+events	00:09:13	2
27899	99	2227	11,2213	cyclictest	37599	-21	kworker/2:0+events	19:39:22	2
27899	99	2227	11,2212	cyclictest	55770	-21	kworker/2:0+events	20:37:37	2
27901	99	2226	8,2215	cyclictest	49554	-21	kworker/3:1+events	20:25:51	3
27901	99	2226	8,2215	cyclictest	49554	-21	kworker/3:1+events	20:21:38	3
27899	99	2226	10,2213	cyclictest	33125	-21	kworker/2:1+events	19:32:34	2
27899	99	2225	10,2212	cyclictest	119126	-21	kworker/2:1+events	00:19:54	2
27899	99	2224	7,2213	cyclictest	119126	-21	kworker/2:1+events	23:56:44	2
27899	99	2224	10,2210	cyclictest	50788	-21	kworker/2:1+events	20:28:28	2
27894	99	2224	9,2212	cyclictest	122567	-21	kworker/1:1+events	00:08:30	1
27894	99	2224	8,2212	cyclictest	113148	-21	kworker/1:3+events	23:38:30	1
27894	99	2224	10,2212	cyclictest	45827	-21	kworker/1:2+events	20:20:08	1
27894	99	2224	10,2211	cyclictest	65138	-21	kworker/1:1+events	21:08:32	1
27899	99	2223	10,2210	cyclictest	61998	-21	kworker/2:2+events	21:11:11	2
27894	99	2222	11,2208	cyclictest	65138	-21	kworker/1:1+events	21:03:11	1
27894	99	2222	11,2207	cyclictest	95645	-21	kworker/1:0+events	23:01:53	1
27899	99	2221	11,2208	cyclictest	113032	-21	kworker/2:2+events	23:35:04	2
27894	99	2221	12,2205	cyclictest	90783	-21	kworker/1:1+events	22:27:38	1
27894	99	2221	10,2208	cyclictest	84258	-21	kworker/1:1+events	22:10:33	1
27891	99	2221	12,2206	cyclictest	79272	-21	kworker/0:0+events	21:55:22	0
27899	99	2220	8,2209	cyclictest	127541	-21	kworker/2:0+events	00:32:22	2
27894	99	2220	9,2208	cyclictest	31222	-21	kworker/1:2+events	19:41:58	1
27894	99	2219	10,2205	cyclictest	31222	-21	kworker/1:2+events	19:26:09	1
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2024-07-29 - 23:00