OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #f, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails

Click here to display the system's profile data.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rackfslot7s.osadl.org (updated Tue Mar 10, 2026 00:44:16)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
915751	99	3059	9,3048	cyclictest	1018283	-21	kworker/1:1+events	00:30:18	1
915753	99	2897	12,2880	cyclictest	1003812	-21	kworker/2:0+events	23:45:17	2
915753	99	2889	14,2871	cyclictest	1018312	-21	kworker/2:0+events	00:35:16	2
915753	99	2822	14,2805	cyclictest	1003812	-21	kworker/2:0+events	23:55:16	2
915753	99	2790	13,2774	cyclictest	1011741	-21	kworker/2:1+events	00:20:00	2
915757	99	2746	12,2730	cyclictest	985132	-21	kworker/3:2+events	22:50:17	3
915753	99	2746	13,2729	cyclictest	948069	-21	kworker/2:0+events	20:50:19	2
915757	99	2703	11,2688	cyclictest	970616	-21	kworker/3:1+events	22:20:12	3
915753	99	2675	13,2658	cyclictest	927460	-21	kworker/2:3+events	19:50:21	2
915751	99	2660	12,2644	cyclictest	920553	-21	kworker/1:1+events	19:30:16	1
915747	99	2566	7,2555	cyclictest	951275	-21	kworker/0:1+events	21:00:18	0
915757	99	2552	9,2540	cyclictest	957651	-21	kworker/3:2+events	21:20:11	3
915757	99	2498	11,2484	cyclictest	920760	-21	kworker/3:0+events	19:25:18	3
915747	99	2488	11,2473	cyclictest	1018251	-21	kworker/0:1+events	00:36:03	0
915757	99	2470	10,2456	cyclictest	990929	-21	kworker/3:0+events	23:06:51	3
915747	99	2451	12,2436	cyclictest	988001	-21	kworker/0:0+events	23:03:29	0
915757	99	2448	12,2432	cyclictest	930720	-21	kworker/3:2+events	19:58:43	3
915753	99	2442	12,2427	cyclictest	998887	-21	kworker/2:1+events	23:39:33	2
915757	99	2436	12,2420	cyclictest	970616	-21	kworker/3:1+events	22:07:35	3
915753	99	2425	12,2409	cyclictest	966909	-21	kworker/2:2+events	21:50:07	2
915753	99	2423	11,2410	cyclictest	942150	-21	kworker/2:0+events	20:35:05	2
915751	99	2423	12,2408	cyclictest	954598	-21	kworker/1:2+events	21:18:17	1
915751	99	2408	11,2394	cyclictest	970287	-21	kworker/1:1+events	22:05:20	1
915753	99	2403	12,2387	cyclictest	919590	-21	kworker/2:0+events	19:22:41	2
915757	99	2391	12,2375	cyclictest	973477	-21	kworker/3:0+events	22:16:00	3
915757	99	2386	12,2371	cyclictest	1002247	-21	kworker/3:0+events	23:48:44	3
915757	99	2386	12,2370	cyclictest	963791	-21	kworker/3:2+events	21:47:52	3
915757	99	2383	12,2368	cyclictest	932347	-21	kworker/3:0+events	20:00:14	3
915751	99	2380	10,2366	cyclictest	971940	-21	kworker/1:0+events	22:12:36	1
915753	99	2377	11,2362	cyclictest	970374	-21	kworker/2:1+events	22:07:13	2
915757	99	2376	11,2362	cyclictest	954355	-21	kworker/3:0+events	21:17:07	3
915753	99	2369	12,2355	cyclictest	933849	-21	kworker/2:1+events	20:30:05	2
915757	99	2365	12,2349	cyclictest	1008586	-21	kworker/3:0+events	00:23:48	3
915751	99	2365	12,2350	cyclictest	998796	-21	kworker/1:2+events	23:38:03	1
915753	99	2364	12,2349	cyclictest	966909	-21	kworker/2:2+events	21:56:58	2
915751	99	2363	12,2349	cyclictest	965265	-21	kworker/1:2+events	21:45:11	1
915751	99	2363	12,2348	cyclictest	943133	-21	kworker/1:2+events	20:41:11	1
915751	99	2363	11,2348	cyclictest	965265	-21	kworker/1:2+events	21:55:51	1
915753	99	2362	12,2347	cyclictest	998887	-21	kworker/2:1+events	23:28:31	2
915751	99	2362	12,2347	cyclictest	960820	-21	kworker/1:2+events	21:33:37	1
915757	99	2361	12,2345	cyclictest	1019614	-21	kworker/3:0+events	00:32:12	3
915751	99	2361	12,2345	cyclictest	1003749	-21	kworker/1:1+events	00:00:15	1
915747	99	2361	12,2345	cyclictest	979821	-21	kworker/0:2+events	22:27:26	0
915753	99	2360	11,2345	cyclictest	992616	-21	kworker/2:2+events	23:11:42	2
915757	99	2359	11,2345	cyclictest	960884	-21	kworker/3:1+events	21:39:22	3
915751	99	2359	11,2345	cyclictest	1015010	-21	kworker/1:2+events	00:28:48	1
915751	99	2358	12,2343	cyclictest	943133	-21	kworker/1:2+events	20:39:17	1
915757	99	2356	12,2341	cyclictest	955955	-21	kworker/3:1+events	21:14:23	3
915757	99	2354	12,2338	cyclictest	1008586	-21	kworker/3:0+events	00:07:50	3
915757	99	2351	12,2337	cyclictest	1002247	-21	kworker/3:0+events	23:50:39	3
915757	99	2351	12,2336	cyclictest	943296	-21	kworker/3:1+events	20:36:35	3
915757	99	2350	12,2335	cyclictest	997438	-21	kworker/3:0+events	23:31:37	3
915753	99	2350	12,2334	cyclictest	973719	-21	kworker/2:2+events	22:15:14	2
915751	99	2349	12,2334	cyclictest	978549	-21	kworker/1:1+events	22:29:44	1
915747	99	2349	12,2334	cyclictest	973658	-21	kworker/0:1+events	22:48:45	0
915757	99	2347	12,2332	cyclictest	963791	-21	kworker/3:2+events	22:04:58	3
915757	99	2347	12,2332	cyclictest	1008586	-21	kworker/3:0+events	00:19:14	3
915751	99	2345	12,2329	cyclictest	932112	-21	kworker/1:1+events	19:57:12	1
915751	99	2342	12,2327	cyclictest	920553	-21	kworker/1:1+events	19:23:03	1
915753	99	2341	12,2325	cyclictest	972059	-21	kworker/2:0+events	22:33:57	2
915747	99	2341	10,2327	cyclictest	955893	-21	kworker/0:0+events	21:13:17	0
915753	99	2340	12,2325	cyclictest	1008930	-21	kworker/2:3+events	00:02:54	2
915753	99	2340	11,2327	cyclictest	933849	-21	kworker/2:1+events	21:01:28	2
915753	99	2340	11,2326	cyclictest	933849	-21	kworker/2:1+events	20:42:18	2
915747	99	2340	10,2327	cyclictest	992586	-21	kworker/0:1+events	23:09:06	0
915753	99	2339	12,2323	cyclictest	927238	-21	kworker/2:0+events	19:46:53	2
915751	99	2339	12,2323	cyclictest	984569	-21	kworker/1:2+events	22:53:17	1
915753	99	2338	12,2322	cyclictest	955926	-21	kworker/2:2+events	21:17:54	2
915751	99	2338	10,2325	cyclictest	940294	-21	kworker/1:1+events	20:27:03	1
915753	99	2337	11,2322	cyclictest	987738	-21	kworker/2:2+events	22:58:11	2
915751	99	2336	11,2321	cyclictest	987883	-21	kworker/1:0+events	23:26:12	1
915747	99	2336	12,2321	cyclictest	942706	-21	kworker/0:0+events	20:37:45	0
915747	99	2336	10,2323	cyclictest	1003780	-21	kworker/0:2+events	23:50:16	0
915747	99	2335	12,2320	cyclictest	954206	-21	kworker/0:2+events	21:15:57	0
915753	99	2332	11,2317	cyclictest	994207	-21	kworker/2:1+events	23:17:49	2
915751	99	2331	12,2317	cyclictest	976928	-21	kworker/1:2+events	22:21:43	1
915751	99	2331	10,2318	cyclictest	935363	-21	kworker/1:2+events	20:08:59	1
915751	99	2329	12,2315	cyclictest	989514	-21	kworker/1:1+events	23:00:06	1
915747	99	2329	12,2314	cyclictest	984688	-21	kworker/0:0+events	22:44:57	0
915753	99	2327	12,2312	cyclictest	933849	-21	kworker/2:1+events	20:21:39	2
915751	99	2327	12,2311	cyclictest	1003749	-21	kworker/1:1+events	00:21:56	1
915751	99	2326	12,2311	cyclictest	994115	-21	kworker/1:2+events	23:14:43	1
915751	99	2326	12,2311	cyclictest	984569	-21	kworker/1:2+events	22:41:56	1
915751	99	2326	12,2311	cyclictest	962460	-21	kworker/1:1+events	21:40:31	1
915751	99	2326	12,2311	cyclictest	1003749	-21	kworker/1:1+events	23:59:52	1
915751	99	2326	12,2310	cyclictest	918928	-21	kworker/1:0+events	19:26:27	1
915751	99	2325	12,2311	cyclictest	987883	-21	kworker/1:0+events	22:55:07	1
915751	99	2325	12,2310	cyclictest	989514	-21	kworker/1:1+events	23:09:50	1
915747	99	2325	12,2310	cyclictest	962187	-21	kworker/0:2+events	21:32:50	0
915753	99	2324	11,2310	cyclictest	984539	-21	kworker/2:2+events	22:43:24	2
915753	99	2316	12,2301	cyclictest	962442	-21	kworker/2:0+events	21:36:17	2
915747	99	2299	11,2284	cyclictest	930793	-21	kworker/0:0+events	20:17:30	0
915753	99	2296	10,2283	cyclictest	962442	-21	kworker/2:0+events	21:40:54	2
915751	99	2276	11,2262	cyclictest	949579	-21	kworker/1:1+events	21:00:41	1
915757	99	2268	11,2254	cyclictest	1008586	-21	kworker/3:0+events	00:13:33	3
915753	99	2267	10,2253	cyclictest	964102	-21	kworker/2:1+events	21:49:21	2
915757	99	2266	10,2253	cyclictest	968981	-21	kworker/3:0+events	21:55:29	3
915757	99	2265	11,2250	cyclictest	938395	-21	kworker/3:2+events	20:19:00	3
915753	99	2265	11,2250	cyclictest	954236	-21	kworker/2:0+events	21:12:31	2
915753	99	2262	10,2250	cyclictest	960853	-21	kworker/2:2+events	21:34:46	2
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2024-07-29 - 23:00