OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #f, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails

Click here to display the system's profile data.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rackfslot7s.osadl.org (updated Sat Feb 28, 2026 00:44:17)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
371397	99	3401	15,3379	cyclictest	377682	-21	kworker/3:1+events	19:30:16	3
371390	99	2889	14,2869	cyclictest	393782	-21	kworker/1:2+events	20:20:13	1
371390	99	2820	10,2807	cyclictest	412927	-21	kworker/1:1+events	21:25:13	1
371397	99	2808	11,2791	cyclictest	475603	-21	kworker/3:1+events	00:35:11	3
371394	99	2795	10,2781	cyclictest	438624	-21	kworker/2:2+events	22:40:17	2
371388	99	2758	13,2741	cyclictest	374847	-21	kworker/0:0+events	20:00:17	0
371397	99	2736	13,2719	cyclictest	394025	-21	kworker/3:1+events	20:45:17	3
371397	99	2727	13,2710	cyclictest	394025	-21	kworker/3:1+events	20:25:14	3
371397	99	2721	13,2704	cyclictest	435595	-21	kworker/3:1+events	22:33:07	3
371394	99	2710	12,2695	cyclictest	422792	-21	kworker/2:1+events	21:55:01	2
371390	99	2710	5,2701	cyclictest	435275	-21	kworker/1:0+events	22:30:10	1
371394	99	2708	13,2691	cyclictest	392445	-21	kworker/2:2+events	20:15:13	2
371394	99	2698	13,2681	cyclictest	385243	-21	kworker/2:1+events	20:05:17	2
371388	99	2670	8,2658	cyclictest	411556	-21	kworker/0:2+events	21:17:37	0
371394	99	2666	13,2649	cyclictest	408408	-21	kworker/2:0+events	21:20:15	2
371390	99	2595	12,2579	cyclictest	455917	-21	kworker/1:0+events	23:51:13	1
371394	99	2573	12,2557	cyclictest	385243	-21	kworker/2:1+events	19:57:15	2
371390	99	2571	12,2555	cyclictest	465891	-21	kworker/1:2+events	00:06:56	1
371397	99	2563	13,2546	cyclictest	432089	-21	kworker/3:2+events	22:35:46	3
371394	99	2562	12,2548	cyclictest	412955	-21	kworker/2:2+events	21:45:04	2
371397	99	2541	12,2526	cyclictest	449385	-21	kworker/3:2+events	23:56:36	3
371394	99	2535	12,2520	cyclictest	436964	-21	kworker/2:1+events	22:47:33	2
371394	99	2519	12,2503	cyclictest	475425	-21	kworker/2:0+events	00:35:35	2
371394	99	2518	9,2505	cyclictest	467496	-21	kworker/2:0+events	00:12:19	2
371388	99	2501	10,2488	cyclictest	372563	-21	kworker/0:1+events	19:15:01	0
371388	99	2498	11,2485	cyclictest	465646	-21	kworker/0:1+events	00:15:00	0
371388	99	2498	11,2485	cyclictest	465646	-21	kworker/0:1+events	00:15:00	0
371394	99	2495	12,2479	cyclictest	432151	-21	kworker/2:0+events	22:38:01	2
371390	99	2495	11,2480	cyclictest	362955	-21	kworker/1:2+events	19:20:43	1
371388	99	2493	12,2477	cyclictest	396988	-21	kworker/0:2+events	20:33:13	0
371388	99	2490	12,2474	cyclictest	400542	-21	kworker/0:0+events	20:47:11	0
371394	99	2489	12,2473	cyclictest	389206	-21	kworker/2:0+events	20:14:03	2
371388	99	2488	11,2473	cyclictest	432328	-21	kworker/0:0+events	22:32:44	0
371394	99	2487	12,2472	cyclictest	470578	-21	kworker/2:1+events	00:24:30	2
371394	99	2484	10,2470	cyclictest	472206	-21	kworker/2:2+events	00:27:11	2
371394	99	2482	10,2468	cyclictest	412955	-21	kworker/2:2+events	21:27:52	2
371397	99	2481	12,2467	cyclictest	462384	-21	kworker/3:0+events	00:05:04	3
371394	99	2481	12,2466	cyclictest	462478	-21	kworker/2:0+events	23:53:33	2
371394	99	2481	12,2466	cyclictest	398681	-21	kworker/2:0+events	20:40:44	2
371394	99	2481	12,2465	cyclictest	394175	-21	kworker/2:1+events	20:27:56	2
371388	99	2480	12,2464	cyclictest	432328	-21	kworker/0:0+events	22:24:09	0
371394	99	2479	12,2464	cyclictest	408408	-21	kworker/2:0+events	21:13:26	2
371394	99	2479	11,2464	cyclictest	412955	-21	kworker/2:2+events	21:33:36	2
371388	99	2479	9,2466	cyclictest	405007	-21	kworker/0:2+events	20:58:10	0
371394	99	2477	12,2462	cyclictest	457634	-21	kworker/2:2+events	00:04:17	2
371394	99	2477	12,2461	cyclictest	412955	-21	kworker/2:2+events	21:40:55	2
371394	99	2476	12,2460	cyclictest	432151	-21	kworker/2:0+events	22:26:45	2
371394	99	2476	11,2462	cyclictest	424426	-21	kworker/2:0+events	21:56:30	2
371394	99	2476	10,2463	cyclictest	436964	-21	kworker/2:1+events	22:54:02	2
371397	99	2474	11,2460	cyclictest	379585	-21	kworker/3:0+events	19:35:13	3
371394	99	2473	12,2458	cyclictest	381248	-21	kworker/2:0+events	19:50:25	2
371394	99	2473	12,2458	cyclictest	381248	-21	kworker/2:0+events	19:50:25	2
371394	99	2473	12,2457	cyclictest	451144	-21	kworker/2:2+events	23:27:04	2
371394	99	2472	12,2456	cyclictest	417489	-21	kworker/2:0+events	21:35:09	2
371388	99	2468	12,2453	cyclictest	374847	-21	kworker/0:0+events	19:25:14	0
371388	99	2467	10,2453	cyclictest	472450	-21	kworker/0:2+events	00:29:09	0
371394	99	2464	12,2449	cyclictest	385243	-21	kworker/2:1+events	20:02:58	2
371388	99	2425	9,2412	cyclictest	449568	-21	kworker/0:0+events	23:14:46	0
371388	99	2425	11,2410	cyclictest	392235	-21	kworker/0:0+events	20:26:23	0
371397	99	2420	9,2408	cyclictest	443395	-21	kworker/3:1+events	22:57:30	3
371388	99	2415	12,2400	cyclictest	440067	-21	kworker/0:0+events	22:45:16	0
371397	99	2410	11,2396	cyclictest	424394	-21	kworker/3:2+events	21:55:44	3
371397	99	2408	12,2393	cyclictest	386191	-21	kworker/3:0+events	20:14:27	3
371397	99	2408	12,2392	cyclictest	467282	-21	kworker/3:2+events	00:10:25	3
371397	99	2406	12,2391	cyclictest	473994	-21	kworker/3:0+events	00:32:57	3
371397	99	2406	11,2392	cyclictest	449385	-21	kworker/3:2+events	23:31:34	3
371397	99	2405	9,2393	cyclictest	412742	-21	kworker/3:0+events	21:17:14	3
371388	99	2404	10,2391	cyclictest	411556	-21	kworker/0:2+events	21:22:56	0
371397	99	2403	12,2387	cyclictest	429102	-21	kworker/3:0+events	22:13:16	3
371397	99	2403	11,2389	cyclictest	419511	-21	kworker/3:0+events	21:42:50	3
371397	99	2402	12,2386	cyclictest	449385	-21	kworker/3:2+events	23:11:42	3
371397	99	2402	12,2386	cyclictest	394025	-21	kworker/3:1+events	20:38:52	3
371397	99	2402	10,2388	cyclictest	424394	-21	kworker/3:2+events	22:02:17	3
371397	99	2402	10,2388	cyclictest	424394	-21	kworker/3:2+events	22:02:17	3
371390	99	2402	12,2388	cyclictest	444759	-21	kworker/1:1+events	23:00:11	1
371397	99	2401	12,2385	cyclictest	414460	-21	kworker/3:2+events	21:27:06	3
371388	99	2400	12,2386	cyclictest	411556	-21	kworker/0:2+events	21:40:11	0
371397	99	2399	12,2383	cyclictest	404921	-21	kworker/3:0+events	20:53:14	3
371397	99	2398	12,2383	cyclictest	386191	-21	kworker/3:0+events	20:07:36	3
371397	99	2398	11,2383	cyclictest	446683	-21	kworker/3:0+events	23:15:56	3
371388	99	2397	9,2385	cyclictest	374847	-21	kworker/0:0+events	19:48:06	0
371397	99	2396	12,2381	cyclictest	439946	-21	kworker/3:1+events	22:44:51	3
371388	99	2396	12,2381	cyclictest	411556	-21	kworker/0:2+events	21:51:54	0
371388	99	2396	10,2383	cyclictest	411556	-21	kworker/0:2+events	21:59:58	0
371388	99	2394	12,2378	cyclictest	374847	-21	kworker/0:0+events	19:41:22	0
371388	99	2394	10,2381	cyclictest	396988	-21	kworker/0:2+events	20:42:17	0
371388	99	2393	12,2377	cyclictest	393871	-21	kworker/0:1+events	20:24:27	0
371388	99	2393	11,2379	cyclictest	390605	-21	kworker/0:1+events	20:06:28	0
371388	99	2392	11,2377	cyclictest	444940	-21	kworker/0:1+events	23:00:34	0
371388	99	2391	12,2376	cyclictest	406609	-21	kworker/0:0+events	21:07:00	0
371388	99	2390	12,2375	cyclictest	415940	-21	kworker/0:0+events	21:30:58	0
371388	99	2389	11,2375	cyclictest	409871	-21	kworker/0:1+events	21:11:34	0
371388	99	2388	12,2373	cyclictest	403292	-21	kworker/0:1+events	20:52:50	0
371388	99	2388	12,2372	cyclictest	468953	-21	kworker/0:2+events	00:14:15	0
371390	99	2386	12,2371	cyclictest	411325	-21	kworker/1:2+events	21:18:00	1
371388	99	2386	12,2371	cyclictest	459484	-21	kworker/0:2+events	23:59:40	0
371390	99	2374	12,2359	cyclictest	398860	-21	kworker/1:1+events	20:50:59	1
371394	99	2367	10,2353	cyclictest	398681	-21	kworker/2:0+events	20:34:21	2
371394	99	2366	12,2351	cyclictest	456032	-21	kworker/2:0+events	23:39:55	2
371397	99	2364	10,2350	cyclictest	377682	-21	kworker/3:1+events	19:26:23	3
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2024-07-29 - 23:00