OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #f, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails

Click here to display the system's profile data.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rackfslot7s.osadl.org (updated Thu Aug 06, 2026 00:44:17)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
297427	99	4080	17,4058	cyclictest	357108	-21	kworker/1:3+events	22:20:01	1
297424	99	3920	19,3896	cyclictest	395647	-21	kworker/0:2+events	00:15:00	0
297432	99	3560	17,3539	cyclictest	321378	-21	kworker/2:1+events	20:45:19	2
297427	99	3220	14,3202	cyclictest	360165	-21	kworker/1:0+events	22:45:13	1
297432	99	3198	15,3179	cyclictest	331252	-21	kworker/2:0+events	21:00:10	2
297427	99	3044	15,3025	cyclictest	325020	-21	kworker/1:1+events	20:35:27	1
297432	99	3018	11,3002	cyclictest	298539	-21	kworker/2:0+events	19:55:19	2
297424	99	3015	4,3007	cyclictest	397654	-21	kworker/0:1+events	00:25:14	0
297427	99	3000	15,2981	cyclictest	339634	-21	kworker/1:0+events	21:40:23	1
297424	99	2955	14,2939	cyclictest	302940	-21	kworker/0:0+events	19:30:01	0
297424	99	2932	14,2915	cyclictest	298762	-21	kworker/0:2+events	19:40:23	0
297432	99	2918	14,2901	cyclictest	369733	-21	kworker/2:2+events	23:00:21	2
297427	99	2909	10,2895	cyclictest	307262	-21	kworker/1:1+events	19:40:01	1
297424	99	2804	14,2784	cyclictest	353618	-21	kworker/0:1+events	22:05:30	0
297438	99	2802	13,2785	cyclictest	384392	-21	kworker/3:2+events	23:48:24	3
297438	99	2790	13,2773	cyclictest	387622	-21	kworker/3:0+events	23:53:43	3
297438	99	2789	12,2773	cyclictest	342368	-21	kworker/3:2+events	21:48:24	3
297438	99	2787	12,2771	cyclictest	381190	-21	kworker/3:1+events	23:36:57	3
297424	99	2780	12,2764	cyclictest	384453	-21	kworker/0:1+events	23:55:12	0
297438	99	2764	13,2747	cyclictest	321468	-21	kworker/3:2+events	20:25:59	3
297432	99	2764	13,2747	cyclictest	356978	-21	kworker/2:2+events	22:19:14	2
297432	99	2763	13,2746	cyclictest	348646	-21	kworker/2:2+events	21:56:44	2
297424	99	2761	13,2744	cyclictest	369710	-21	kworker/0:0+events	22:52:01	0
297424	99	2758	11,2743	cyclictest	318181	-21	kworker/0:1+events	20:14:14	0
297438	99	2752	12,2736	cyclictest	325106	-21	kworker/3:0+events	20:38:53	3
297432	99	2752	13,2735	cyclictest	351874	-21	kworker/2:1+events	22:03:14	2
297427	99	2749	14,2731	cyclictest	310512	-21	kworker/1:1+events	20:25:15	1
297432	99	2741	13,2724	cyclictest	347020	-21	kworker/2:1+events	21:42:17	2
297424	99	2741	11,2726	cyclictest	312232	-21	kworker/0:0+events	19:59:28	0
297424	99	2740	13,2723	cyclictest	368104	-21	kworker/0:2+events	22:48:33	0
297427	99	2738	13,2721	cyclictest	308904	-21	kworker/1:2+events	19:44:39	1
297427	99	2736	12,2721	cyclictest	392784	-21	kworker/1:0+events	00:27:34	1
297427	99	2735	13,2718	cyclictest	392784	-21	kworker/1:0+events	00:16:08	1
297438	99	2733	13,2716	cyclictest	369974	-21	kworker/3:0+events	23:24:42	3
297438	99	2732	12,2716	cyclictest	310661	-21	kworker/3:3+events	19:54:35	3
297424	99	2730	7,2719	cyclictest	331142	-21	kworker/0:0+events	20:53:41	0
297424	99	2730	13,2714	cyclictest	307141	-21	kworker/0:1+events	20:00:14	0
297432	99	2729	13,2712	cyclictest	360479	-21	kworker/2:1+events	22:32:42	2
297424	99	2729	13,2712	cyclictest	355280	-21	kworker/0:0+events	22:18:51	0
297432	99	2728	13,2711	cyclictest	317439	-21	kworker/2:2+events	20:16:08	2
297432	99	2728	12,2712	cyclictest	364879	-21	kworker/2:2+events	22:39:32	2
297427	99	2722	12,2706	cyclictest	378096	-21	kworker/1:1+events	23:26:14	1
297424	99	2722	13,2705	cyclictest	379635	-21	kworker/0:3+events	23:31:57	0
297432	99	2721	11,2706	cyclictest	348646	-21	kworker/2:2+events	21:54:27	2
297432	99	2718	13,2701	cyclictest	393858	-21	kworker/2:2+events	00:10:47	2
297424	99	2718	13,2701	cyclictest	358360	-21	kworker/0:2+events	22:38:48	0
297424	99	2718	10,2705	cyclictest	395647	-21	kworker/0:2+events	00:18:26	0
297432	99	2717	13,2700	cyclictest	400598	-21	kworker/2:1+events	00:31:46	2
297424	99	2717	12,2701	cyclictest	318181	-21	kworker/0:1+events	20:22:38	0
297427	99	2716	13,2699	cyclictest	383678	-21	kworker/1:2+events	23:36:13	1
297432	99	2715	13,2698	cyclictest	372953	-21	kworker/2:1+events	23:12:04	2
297432	99	2714	13,2697	cyclictest	345375	-21	kworker/2:0+events	21:46:30	2
297432	99	2714	13,2697	cyclictest	310518	-21	kworker/2:2+events	19:49:59	2
297432	99	2713	13,2696	cyclictest	378043	-21	kworker/2:1+events	23:29:41	2
297427	99	2713	13,2696	cyclictest	387564	-21	kworker/1:1+events	23:57:51	1
297432	99	2712	13,2695	cyclictest	360479	-21	kworker/2:1+events	22:26:56	2
297432	99	2712	13,2695	cyclictest	337725	-21	kworker/2:1+events	21:20:36	2
297432	99	2711	13,2694	cyclictest	369733	-21	kworker/2:2+events	22:57:18	2
297432	99	2711	10,2697	cyclictest	331252	-21	kworker/2:0+events	20:59:26	2
297424	99	2711	12,2695	cyclictest	319869	-21	kworker/0:0+events	20:18:01	0
297432	99	2710	13,2695	cyclictest	332848	-21	kworker/2:2+events	21:05:30	2
297432	99	2708	13,2693	cyclictest	390584	-21	kworker/2:0+events	00:00:08	2
297432	99	2708	13,2693	cyclictest	390584	-21	kworker/2:0+events	00:00:07	2
297427	99	2708	13,2692	cyclictest	351797	-21	kworker/1:0+events	22:09:42	1
297432	99	2706	13,2689	cyclictest	398982	-21	kworker/2:0+events	00:24:50	2
297432	99	2706	13,2689	cyclictest	392253	-21	kworker/2:1+events	00:07:22	2
297432	99	2706	13,2689	cyclictest	381342	-21	kworker/2:2+events	23:44:59	2
297427	99	2703	13,2686	cyclictest	345663	-21	kworker/1:2+events	21:40:01	1
297424	99	2703	13,2686	cyclictest	358360	-21	kworker/0:2+events	22:27:18	0
297432	99	2701	11,2686	cyclictest	386990	-21	kworker/2:0+events	23:47:16	2
297427	99	2701	13,2685	cyclictest	360165	-21	kworker/1:0+events	22:40:43	1
297427	99	2701	11,2686	cyclictest	318239	-21	kworker/1:2+events	20:43:49	1
297424	99	2700	13,2683	cyclictest	365236	-21	kworker/0:0+events	22:41:29	0
297432	99	2698	13,2682	cyclictest	351874	-21	kworker/2:1+events	22:06:15	2
297427	99	2694	9,2681	cyclictest	323120	-21	kworker/1:0+events	20:30:33	1
297427	99	2693	11,2679	cyclictest	387564	-21	kworker/1:1+events	00:02:28	1
297427	99	2693	11,2679	cyclictest	387564	-21	kworker/1:1+events	00:02:28	1
297427	99	2689	13,2672	cyclictest	399014	-21	kworker/1:1+events	00:30:59	1
297427	99	2687	13,2670	cyclictest	326663	-21	kworker/1:0+events	21:13:06	1
297438	99	2683	13,2666	cyclictest	387622	-21	kworker/3:0+events	00:08:56	3
297438	99	2683	13,2666	cyclictest	348826	-21	kworker/3:0+events	21:50:15	3
297432	99	2683	13,2666	cyclictest	321378	-21	kworker/2:1+events	20:29:46	2
297427	99	2683	13,2668	cyclictest	332889	-21	kworker/1:1+events	21:20:11	1
297438	99	2679	12,2663	cyclictest	329537	-21	kworker/3:0+events	20:47:14	3
297427	99	2679	13,2662	cyclictest	374649	-21	kworker/1:2+events	23:23:33	1
297427	99	2679	12,2663	cyclictest	318239	-21	kworker/1:2+events	20:24:31	1
297432	99	2678	11,2663	cyclictest	388997	-21	kworker/2:1+events	23:51:04	2
297438	99	2676	10,2662	cyclictest	344608	-21	kworker/3:0+events	21:39:15	3
297427	99	2676	13,2659	cyclictest	360165	-21	kworker/1:0+events	22:33:04	1
297427	99	2674	12,2660	cyclictest	374649	-21	kworker/1:2+events	23:15:06	1
297427	99	2673	13,2656	cyclictest	331169	-21	kworker/1:2+events	20:52:12	1
297427	99	2673	12,2657	cyclictest	332889	-21	kworker/1:1+events	21:03:13	1
297427	99	2672	13,2655	cyclictest	355040	-21	kworker/1:1+events	22:12:46	1
297427	99	2672	13,2655	cyclictest	310512	-21	kworker/1:1+events	20:05:55	1
297427	99	2672	13,2655	cyclictest	300795	-21	kworker/1:0+events	19:57:34	1
297427	99	2672	12,2656	cyclictest	348563	-21	kworker/1:1+events	21:54:03	1
297432	99	2666	12,2650	cyclictest	307232	-21	kworker/2:2+events	19:38:28	2
297432	99	2666	11,2651	cyclictest	343787	-21	kworker/2:2+events	21:38:53	2
297438	99	2665	12,2649	cyclictest	369974	-21	kworker/3:0+events	23:10:35	3
297427	99	2664	11,2649	cyclictest	297054	-21	kworker/1:2+events	19:19:43	1
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2024-07-29 - 23:00