OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #f, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails

Click here to display the system's profile data.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rackfslot7s.osadl.org (updated Thu Jul 02, 2026 00:44:18)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
526449	99	5176	25,5145	cyclictest	581674	-21	kworker/0:3+events	22:24:33	0
526460	99	3100	15,3081	cyclictest	562323	-21	kworker/3:2+events	21:20:01	3
526460	99	3086	12,3069	cyclictest	522728	-21	kworker/3:2+events	19:25:01	3
526460	99	3041	14,3020	cyclictest	536405	-21	kworker/3:2+events	19:45:18	3
526460	99	2996	5,2987	cyclictest	575244	-21	kworker/3:2+events	21:50:14	3
526460	99	2918	10,2904	cyclictest	559058	-21	kworker/3:2+events	20:50:15	3
526457	99	2849	13,2832	cyclictest	563831	-21	kworker/2:2+events	21:20:45	2
526457	99	2843	13,2826	cyclictest	546204	-21	kworker/2:2+events	20:26:43	2
526453	99	2840	13,2823	cyclictest	599244	-21	kworker/1:0+events	23:15:12	1
526453	99	2834	13,2817	cyclictest	546265	-21	kworker/1:2+events	20:12:43	1
526457	99	2820	6,2810	cyclictest	555915	-21	kworker/2:1+events	21:01:20	2
526453	99	2811	0,2807	cyclictest	589861	-21	kworker/1:2+events	22:48:14	1
526457	99	2810	13,2793	cyclictest	530740	-21	kworker/2:0+events	19:26:33	2
526453	99	2810	13,2794	cyclictest	527835	-21	kworker/1:2+events	19:25:45	1
526457	99	2807	13,2790	cyclictest	536195	-21	kworker/2:1+events	19:38:46	2
526453	99	2807	13,2790	cyclictest	623333	-21	kworker/1:2+events	00:27:49	1
526453	99	2807	13,2790	cyclictest	582869	-21	kworker/1:1+events	22:02:46	1
526453	99	2806	13,2789	cyclictest	584480	-21	kworker/1:0+events	22:10:50	1
526453	99	2806	13,2788	cyclictest	557365	-21	kworker/1:0+events	20:42:37	1
526457	99	2805	13,2788	cyclictest	620251	-21	kworker/2:2+events	00:05:32	2
526457	99	2805	13,2788	cyclictest	602647	-21	kworker/2:2+events	23:04:59	2
526453	99	2805	13,2789	cyclictest	534681	-21	kworker/1:0+events	20:04:44	1
526453	99	2805	13,2788	cyclictest	594381	-21	kworker/1:0+events	22:36:17	1
526453	99	2805	13,2788	cyclictest	560613	-21	kworker/1:2+events	20:53:21	1
526457	99	2804	13,2787	cyclictest	582993	-21	kworker/2:0+events	22:03:55	2
526457	99	2803	13,2786	cyclictest	565753	-21	kworker/2:0+events	21:12:46	2
526457	99	2802	13,2786	cyclictest	620251	-21	kworker/2:2+events	00:12:46	2
526457	99	2802	13,2786	cyclictest	587809	-21	kworker/2:1+events	22:22:38	2
526457	99	2802	13,2785	cyclictest	593214	-21	kworker/2:2+events	22:38:12	2
526457	99	2801	13,2784	cyclictest	605760	-21	kworker/2:0+events	23:14:26	2
526457	99	2801	13,2784	cyclictest	582993	-21	kworker/2:0+events	22:12:46	2
526457	99	2801	13,2784	cyclictest	546204	-21	kworker/2:2+events	20:19:29	2
526457	99	2800	13,2783	cyclictest	617039	-21	kworker/2:2+events	23:52:18	2
526457	99	2799	13,2782	cyclictest	575276	-21	kworker/2:1+events	21:40:54	2
526457	99	2797	13,2780	cyclictest	563831	-21	kworker/2:2+events	21:38:58	2
526453	99	2795	13,2779	cyclictest	560613	-21	kworker/1:2+events	21:09:18	1
526457	99	2794	13,2777	cyclictest	600907	-21	kworker/2:1+events	23:19:25	2
526457	99	2790	13,2773	cyclictest	568604	-21	kworker/2:1+events	21:31:20	2
526457	99	2788	13,2771	cyclictest	596057	-21	kworker/2:0+events	22:43:10	2
526453	99	2786	13,2771	cyclictest	587897	-21	kworker/1:0+events	22:22:15	1
526457	99	2776	13,2759	cyclictest	591586	-21	kworker/2:1+events	22:30:36	2
526457	99	2772	13,2755	cyclictest	582993	-21	kworker/2:0+events	22:07:46	2
526457	99	2767	10,2753	cyclictest	530740	-21	kworker/2:0+events	20:03:34	2
526449	99	2765	13,2748	cyclictest	541713	-21	kworker/0:0+events	20:05:51	0
526460	99	2757	13,2741	cyclictest	602498	-21	kworker/3:2+events	23:09:31	3
526460	99	2757	13,2740	cyclictest	631500	-21	kworker/3:1+events	00:38:39	3
526457	99	2757	13,2740	cyclictest	546204	-21	kworker/2:2+events	20:43:47	2
526453	99	2757	12,2741	cyclictest	549502	-21	kworker/1:0+events	20:23:43	1
526453	99	2756	13,2739	cyclictest	604376	-21	kworker/1:2+events	23:11:23	1
526460	99	2752	13,2735	cyclictest	602498	-21	kworker/3:2+events	23:12:33	3
526453	99	2751	13,2736	cyclictest	566579	-21	kworker/1:1+events	21:15:02	1
526460	99	2734	12,2718	cyclictest	552768	-21	kworker/3:2+events	20:35:51	3
526460	99	2726	13,2709	cyclictest	571891	-21	kworker/3:1+events	21:27:57	3
526460	99	2722	12,2706	cyclictest	571891	-21	kworker/3:1+events	21:39:43	3
526453	99	2721	10,2708	cyclictest	570381	-21	kworker/1:1+events	21:24:33	1
526460	99	2719	13,2701	cyclictest	538301	-21	kworker/3:0+events	19:50:17	3
526460	99	2719	13,2701	cyclictest	538301	-21	kworker/3:0+events	19:50:17	3
526460	99	2716	12,2700	cyclictest	586351	-21	kworker/3:0+events	22:21:28	3
526460	99	2712	7,2701	cyclictest	586351	-21	kworker/3:0+events	22:46:16	3
526460	99	2712	13,2696	cyclictest	562323	-21	kworker/3:2+events	21:11:13	3
526460	99	2711	13,2694	cyclictest	541231	-21	kworker/3:1+events	20:15:25	3
526460	99	2711	11,2696	cyclictest	547834	-21	kworker/3:2+events	20:21:02	3
526460	99	2710	13,2693	cyclictest	617010	-21	kworker/3:0+events	23:49:19	3
526460	99	2710	12,2694	cyclictest	575244	-21	kworker/3:2+events	21:42:46	3
526457	99	2710	11,2695	cyclictest	573612	-21	kworker/2:0+events	21:45:21	2
526460	99	2709	13,2692	cyclictest	534617	-21	kworker/3:0+events	19:36:31	3
526460	99	2709	11,2694	cyclictest	541231	-21	kworker/3:1+events	20:59:48	3
526460	99	2707	13,2690	cyclictest	618752	-21	kworker/3:2+events	23:58:46	3
526460	99	2705	13,2690	cyclictest	612413	-21	kworker/3:1+events	23:40:06	3
526460	99	2705	13,2690	cyclictest	612413	-21	kworker/3:1+events	23:40:06	3
526460	99	2704	13,2687	cyclictest	581366	-21	kworker/3:1+events	21:57:47	3
526460	99	2704	13,2687	cyclictest	541231	-21	kworker/3:1+events	20:25:12	3
526460	99	2702	13,2685	cyclictest	541231	-21	kworker/3:1+events	20:34:43	3
526449	99	2699	13,2682	cyclictest	565453	-21	kworker/0:0+events	21:20:22	0
526453	99	2696	13,2679	cyclictest	615580	-21	kworker/1:2+events	23:52:42	1
526449	99	2693	11,2679	cyclictest	562450	-21	kworker/0:1+events	21:07:00	0
526453	99	2692	12,2676	cyclictest	621645	-21	kworker/1:1+events	00:07:05	1
526453	99	2691	13,2674	cyclictest	534681	-21	kworker/1:0+events	19:35:20	1
526453	99	2690	13,2672	cyclictest	612203	-21	kworker/1:0+events	23:32:51	1
526449	99	2688	13,2671	cyclictest	581674	-21	kworker/0:3+events	22:29:52	0
526460	99	2681	13,2664	cyclictest	586351	-21	kworker/3:0+events	22:17:17	3
526453	99	2681	13,2664	cyclictest	547867	-21	kworker/1:1+events	20:17:17	1
526457	99	2674	13,2656	cyclictest	568604	-21	kworker/2:1+events	21:27:34	2
526457	99	2666	10,2651	cyclictest	610891	-21	kworker/2:2+events	23:35:52	2
526457	99	2656	13,2639	cyclictest	618662	-21	kworker/2:0+events	23:58:23	2
526449	99	2655	13,2639	cyclictest	549472	-21	kworker/0:0+events	20:35:28	0
526449	99	2654	12,2640	cyclictest	578320	-21	kworker/0:1+events	21:55:10	0
526449	99	2652	13,2635	cyclictest	567120	-21	kworker/0:2+events	21:16:57	0
526449	99	2651	13,2634	cyclictest	581674	-21	kworker/0:3+events	22:05:51	0
526449	99	2651	13,2634	cyclictest	549472	-21	kworker/0:0+events	20:30:29	0
526449	99	2651	13,2634	cyclictest	537819	-21	kworker/0:1+events	19:42:37	0
526457	99	2650	12,2634	cyclictest	557660	-21	kworker/2:0+events	20:46:26	2
526453	99	2648	13,2631	cyclictest	551171	-21	kworker/1:1+events	20:38:27	1
526457	99	2647	8,2635	cyclictest	624833	-21	kworker/2:0+events	00:29:20	2
526457	99	2646	13,2630	cyclictest	522654	-21	kworker/2:1+events	19:20:28	2
526457	99	2646	12,2630	cyclictest	629861	-21	kworker/2:1+events	00:33:58	2
526453	99	2645	12,2630	cyclictest	574974	-21	kworker/1:1+events	21:40:31	1
526449	99	2644	13,2627	cyclictest	546233	-21	kworker/0:2+events	20:28:36	0
526449	99	2642	12,2626	cyclictest	527808	-21	kworker/0:1+events	19:30:44	0
526460	99	2640	10,2627	cyclictest	573526	-21	kworker/3:0+events	21:34:23	3
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2024-07-29 - 23:00