OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack8slot6s.osadl.org (updated Sun Aug 23, 2026 12:43:39)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
3037	99	1051	1050,1	cyclictest	14448	-21	kworker/3:2+events	10:47:52	3
3028	99	1049	1048,1	cyclictest	13840	-21	kworker/1:0+events	09:43:34	1
3028	99	1049	1046,3	cyclictest	19696	-21	kworker/1:2+events	09:23:50	1
3023	99	1049	1047,2	cyclictest	30026	-21	turbostat	10:34:55	0
3037	99	1048	1047,1	cyclictest	14448	-21	kworker/3:2+events	10:52:45	3
3032	99	1047	1045,2	cyclictest	20444	-21	kworker/2:1+events	08:27:10	2
3032	99	1042	1040,2	cyclictest	29892	-21	kworker/2:2+events	08:17:55	2
3023	99	1040	1036,4	cyclictest	2604	-21	turbostat	11:49:56	0
3023	99	1039	1037,2	cyclictest	26335	-21	kworker/0:1+events	11:42:22	0
3037	99	1038	1036,2	cyclictest	20895	-21	kworker/3:1+events	12:06:28	3
3037	99	1035	1033,2	cyclictest	20895	-21	kworker/3:1+events	12:11:36	3
3023	99	1031	1029,2	cyclictest	16559	-21	kworker/0:0+events	10:17:00	0
3037	99	1029	1026,3	cyclictest	2876	-21	kworker/3:1+events	07:15:20	3
3028	99	1029	1026,3	cyclictest	30604	-21	kworker/1:0+events	09:28:31	1
3023	99	1029	1028,1	cyclictest	14327	-21	kworker/0:2+events	11:57:53	0
3037	99	1028	1026,2	cyclictest	14349	-21	kworker/3:1+events	08:53:39	3
3037	99	1026	1024,2	cyclictest	21459	-21	kworker/3:1+events	08:05:44	3
3037	99	1025	1024,1	cyclictest	21459	-21	kworker/3:1+events	08:13:06	3
3037	99	1025	1023,2	cyclictest	20790	-21	turbostat	11:34:55	3
3037	99	1025	1023,2	cyclictest	13187	-21	kworker/3:0+events	07:53:26	3
3037	99	1023	1022,1	cyclictest	14448	-21	kworker/3:2+events	11:02:43	3
3037	99	1023	1021,2	cyclictest	20757	-21	kworker/3:0+events	10:24:52	3
3028	99	1023	1022,1	cyclictest	6271	-21	kworker/1:1+events	10:08:40	1
3037	99	1022	1020,2	cyclictest	32168	-21	kworker/3:1+events	10:55:21	3
3037	99	1021	1018,3	cyclictest	18514	-21	kworker/3:1+events	09:07:37	3
3037	99	1020	1019,1	cyclictest	28077	-21	kworker/3:0+events	09:11:03	3
3037	99	1020	1018,2	cyclictest	19406	-21	kworker/3:2+events	07:48:41	3
3037	99	1020	1018,2	cyclictest	18514	-21	kworker/3:1+events	09:03:41	3
3028	99	1020	1018,2	cyclictest	8956	-21	kworker/1:1+events	11:17:44	1
3037	99	1019	1017,2	cyclictest	32168	-21	kworker/3:1+events	10:35:41	3
3037	99	1019	1017,2	cyclictest	27360	-21	kworker/3:0+events	12:20:44	3
3037	99	1019	1017,2	cyclictest	21459	-21	kworker/3:1+events	08:02:40	3
3037	99	1019	1017,2	cyclictest	10708	-21	kworker/3:1+events	07:31:29	3
3037	99	1019	1016,3	cyclictest	27360	-21	kworker/3:0+events	12:18:59	3
3032	99	1018	1015,2	cyclictest	16228	-21	turbostat	10:19:55	2
3037	99	1017	1015,2	cyclictest	4693	-21	kworker/3:0+events	10:43:27	3
3037	99	1017	1015,2	cyclictest	28077	-21	kworker/3:0+events	08:44:46	3
3028	99	1017	1015,2	cyclictest	15941	-21	kworker/1:2+events	11:23:40	1
3037	99	1016	1014,2	cyclictest	17648	-21	kworker/3:0+events	07:14:11	3
3037	99	1015	1013,2	cyclictest	28077	-21	kworker/3:0+events	08:58:39	3
3037	99	1014	1012,2	cyclictest	17648	-21	kworker/3:0+events	07:25:22	3
3037	99	1013	1011,2	cyclictest	6944	-21	turbostat	12:29:55	3
3023	99	1011	1008,3	cyclictest	20754	-21	turbostat	09:14:56	0
3037	99	1010	1008,2	cyclictest	8665	-21	kworker/3:2+events	07:21:27	3
3037	99	1009	1007,2	cyclictest	20757	-21	kworker/3:0+events	10:29:09	3
3037	99	1008	1006,2	cyclictest	15248	-21	kworker/3:2+events	07:42:35	3
3023	99	1008	1006,2	cyclictest	26335	-21	kworker/0:1+events	11:39:24	0
3023	99	1008	1006,2	cyclictest	19273	-21	kworker/0:1+events	12:04:23	0
3037	99	1006	1004,2	cyclictest	19406	-21	kworker/3:2+events	07:58:51	3
3023	99	1006	1005,1	cyclictest	26335	-21	kworker/0:1+events	11:50:43	0
3037	99	1005	1003,2	cyclictest	28077	-21	kworker/3:0+events	08:48:41	3
3037	99	1004	1001,3	cyclictest	6176	-21	kworker/3:2+events	08:36:57	3
3037	99	1003	998,3	cyclictest	15248	-21	kworker/3:2+events	07:39:25	3
3023	99	1003	1000,3	cyclictest	16602	-21	kworker/0:2+events	09:15:21	0
3037	99	1002	1000,2	cyclictest	5371	-21	kworker/3:2+events	12:37:32	3
3037	99	1001	998,2	cyclictest	27381	-21	kworker/3:2+events	10:31:05	3
3028	99	1000	997,3	cyclictest	1273	-21	kworker/1:0+events	10:12:42	1
3023	99	1000	998,2	cyclictest	2320	-21	turbostat	12:24:56	0
3028	99	999	997,2	cyclictest	30166	-21	kworker/1:0+events	11:05:46	1
3028	99	993	992,1	cyclictest	19146	-21	kworker/1:1+events	09:47:34	1
3028	99	992	992,0	cyclictest	15941	-21	kworker/1:2+events	11:26:05	1
3028	99	992	991,1	cyclictest	1273	-21	kworker/1:0+events	10:01:05	1
3028	99	990	989,1	cyclictest	15941	-21	kworker/1:2+events	11:32:01	1
3028	99	988	986,2	cyclictest	19696	-21	kworker/1:2+events	09:32:34	1
3028	99	985	984,1	cyclictest	15941	-21	kworker/1:2+events	12:34:36	1
3037	99	984	981,3	cyclictest	11755	-21	turbostat	11:59:56	3
3028	99	983	981,2	cyclictest	30166	-21	kworker/1:0+events	11:14:10	1
3028	99	982	981,1	cyclictest	13840	-21	kworker/1:0+events	09:37:22	1
3028	99	981	980,1	cyclictest	19146	-21	kworker/1:1+events	09:56:27	1
3028	99	980	978,2	cyclictest	19146	-21	kworker/1:1+events	09:52:53	1
3028	99	978	977,1	cyclictest	4160	-21	kworker/1:1+events	08:32:42	1
3028	99	973	971,2	cyclictest	25176	-21	turbostat	11:04:56	1
3023	99	963	960,3	cyclictest	1737	-21	turbostat	08:29:55	0
3037	99	951	949,2	cyclictest	1995	-21	turbostat	11:14:55	3
3037	99	887	884,3	cyclictest	25423	-21	turbostat	09:19:55	3
3032	99	887	886,1	cyclictest	20444	-21	kworker/2:1+events	08:20:43	2
3032	99	873	871,2	cyclictest	30137	-21	turbostat	12:19:56	2
3037	99	856	854,2	cyclictest	11648	-21	turbostat	10:14:55	3
3037	99	692	689,3	cyclictest	16228	-21	turbostat	10:19:55	3
3037	99	670	669,1	cyclictest	16332	-21	turbostat	12:04:56	3
3032	99	638	637,1	cyclictest	27644	-21	turbostat	08:14:56	2
3023	99	611	610,1	cyclictest	11648	-21	turbostat	10:14:55	0
3028	99	610	606,4	cyclictest	25423	-21	turbostat	09:19:55	1
3023	99	582	579,3	cyclictest	20790	-21	turbostat	11:34:55	0
3037	99	555	553,2	cyclictest	4180	-21	turbostat	08:34:55	3
3037	99	554	552,2	cyclictest	11518	-21	turbostat	12:34:55	3
3028	99	537	533,4	cyclictest	6944	-21	turbostat	12:29:55	1
3023	99	520	518,2	cyclictest	16111	-21	turbostat	12:39:55	0
3028	99	373	371,2	cyclictest	1737	-21	turbostat	08:29:55	1
3028	99	106	106,0	cyclictest	0	-21	swapper/1	08:21:27	1
3028	99	21	21,0	cyclictest	15941	-21	kworker/1:2+events_freezable	10:31:32	1
3028	99	19	18,1	cyclictest	762	-21	ntpd	11:35:46	1
3028	99	18	18,0	cyclictest	677	50	irq/123-lan	12:07:16	1
3028	99	18	18,0	cyclictest	441	-21	hwrng	12:20:02	1
3028	99	18	18,0	cyclictest	441	-21	hwrng	08:03:00	1
3028	99	18	17,1	cyclictest	677	50	irq/123-lan	12:16:06	1
3028	99	18	17,1	cyclictest	677	50	irq/123-lan	11:55:11	1
3028	99	18	17,1	cyclictest	677	50	irq/123-lan	11:50:17	1
3028	99	18	17,1	cyclictest	677	50	irq/123-lan	11:46:47	1
3028	99	18	17,1	cyclictest	677	50	irq/123-lan	08:51:12	1
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17