OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rackcslot7s.osadl.org (updated Tue Dec 23, 2025 12:44:54)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
2008854	2	110	0,6	sleep0	0	-21	swapper/0	07:55:12	0
2008854	2	110	0,6	sleep0	0	-21	swapper/0	07:55:11	0
2104727	2	106	0,8	sleep3	41	1	rcuc/3	12:27:35	3
1973	99	93	0,27	rtkit-daemon	4497	-21	PK-Backend	11:27:16	1
1973	99	93	0,24	rtkit-daemon	4497	-21	PK-Backend	09:44:25	1
1973	99	84	0,10	rtkit-daemon	2072531	-21	ntp_states	10:55:27	1
1973	99	71	0,61	rtkit-daemon	0	-21	swapper/0	07:07:45	0
1991783	2	70	38,26	sleep2	0	-21	swapper/2	07:06:29	2
2022486	2	65	0,6	sleep2	33	1	rcuc/2	08:35:02	2
1973	99	65	0,5	rtkit-daemon	1994399	-21	munin-node	08:06:24	3
2041126	2	64	0,5	sleep2	0	-21	swapper/2	09:25:28	2
1990578	2	62	42,14	sleep3	0	-21	swapper/3	07:05:09	3
1973	99	62	0,9	rtkit-daemon	1972	-21	rtkit-daemon	08:01:14	2
1973	99	61	0,10	rtkit-daemon	1972	-21	rtkit-daemon	10:39:35	2
1990571	2	60	41,14	sleep1	0	-21	swapper/1	07:05:04	1
1973	99	60	0,9	rtkit-daemon	1972	-21	rtkit-daemon	10:51:49	2
2104120	2	59	0,6	sleep1	4497	-21	PK-Backend	12:25:28	1
2010441	2	58	0,15	sleep3	42	1	ksoftirqd/3	08:00:01	3
2010441	2	58	0,15	sleep3	42	1	ksoftirqd/3	08:00:01	3
1973	99	58	0,8	rtkit-daemon	1972	-21	rtkit-daemon	10:32:28	2
1992275	99	57	0,52	cyclictest	1972	-21	rtkit-daemon	08:12:28	2
1973	99	56	0,9	rtkit-daemon	1972	-21	rtkit-daemon	08:15:10	2
1973	99	55	0,9	rtkit-daemon	1972	-21	rtkit-daemon	08:40:18	2
1973	99	54	0,9	rtkit-daemon	1972	-21	rtkit-daemon	08:08:17	2
1973	99	53	0,8	rtkit-daemon	1972	-21	rtkit-daemon	08:37:49	2
2039140	2	52	0,7	sleep1	4497	-21	PK-Backend	09:20:19	1
1992275	99	51	17,29	cyclictest	2042695	-21	cat	09:30:24	2
1992275	99	51	17,29	cyclictest	2042695	-21	cat	09:30:24	2
1992275	99	51	14,32	cyclictest	2086203	-21	cat	11:35:25	2
1992275	99	50	18,28	cyclictest	2005860	-21	uname	07:45:22	2
1992275	99	50	16,30	cyclictest	2002859	-21	wc	07:35:29	2
1992275	99	50	14,31	cyclictest	2079509	-21	processes	11:15:25	2
1992275	99	49	7,12	cyclictest	1074	-21	lldpd	11:21:10	2
1992275	99	49	17,27	cyclictest	2074981	-21	latency_hist	11:05:02	2
1992275	99	49	13,31	cyclictest	2077105	-21	expr	11:10:13	2
1992275	99	49	12,32	cyclictest	2031991	-21	cat	09:00:23	2
1992275	99	48	18,26	cyclictest	1137	-21	runrttasks	10:04:17	2
1992275	99	48	18,25	cyclictest	2090538	-21	cron	11:50:00	2
1992275	99	48	16,27	cyclictest	2068318	-21	sh	10:45:10	2
1992275	99	48	15,28	cyclictest	1	-21	systemd	12:15:00	2
1992275	99	48	14,31	cyclictest	2048713	-21	sh	09:50:01	2
1973	99	48	0,39	rtkit-daemon	1074	-21	lldpd	08:25:51	2
1973	99	48	0,24	rtkit-daemon	0	-21	swapper/0	07:45:08	0
1992275	99	47	16,26	cyclictest	2107878	-21	cut	12:35:21	2
1992275	99	47	12,31	cyclictest	2097472	-21	cron	12:10:01	2
2045926	2	46	0,6	sleep3	0	-21	swapper/3	09:40:18	3
1992275	99	46	19,23	cyclictest	2046973	-21	latency_hist	09:45:01	2
1992275	99	46	16,26	cyclictest	2104555	-21	apt-key	12:27:35	2
1992275	99	46	15,26	cyclictest	2076100	-21	gzip	11:05:26	2
1992275	99	46	15,26	cyclictest	2049084	-21	cpuspeed_turbos	09:50:13	2
1992263	99	46	28,6	cyclictest	13	-21	ksoftirqd/0	08:30:14	0
1992263	99	46	24,11	cyclictest	13	-21	ksoftirqd/0	07:35:14	0
1973	99	46	0,31	rtkit-daemon	2087722	-21	df	11:40:16	1
2100993	2	45	0,5	sleep3	41	1	rcuc/3	12:17:04	3
1992275	99	45	17,24	cyclictest	4497	-21	PK-Backend	07:51:52	2
1992275	99	45	14,27	cyclictest	2050696	-21	acpi	09:55:12	2
1992275	99	45	14,27	cyclictest	2050696	-21	acpi	09:55:11	2
1992275	99	45	1,39	cyclictest	2005108	-21	awk	07:44:59	2
1992275	99	45	12,28	cyclictest	2087555	-21	cstates	11:40:15	2
1992275	99	45	10,30	cyclictest	2082297	-21	tr	11:25:13	2
1992263	99	45	19,14	cyclictest	13	-21	ksoftirqd/0	07:15:06	0
1992275	99	44	9,10	cyclictest	2058575	-21	uname	10:15:23	2
1992275	99	44	18,22	cyclictest	1137	-21	runrttasks	08:57:45	2
1992275	99	44	15,25	cyclictest	1998590	-21	cstates	07:25:14	2
1992275	99	44	14,26	cyclictest	2038797	-21	cstates	09:20:13	2
1992275	99	44	14,26	cyclictest	1997148	-21	kernelversion	07:20:22	2
1992275	99	44	10,29	cyclictest	2084260	-21	df	11:30:16	2
1992263	99	44	27,5	cyclictest	13	-21	ksoftirqd/0	09:00:01	0
1992263	99	44	27,13	cyclictest	13	-21	ksoftirqd/0	09:05:14	0
1992263	99	44	25,7	cyclictest	13	-21	ksoftirqd/0	07:50:23	0
1973	99	44	0,31	rtkit-daemon	0	-21	swapper/0	10:05:27	0
1992279	99	43	24,16	cyclictest	1054	-21	snmpd	10:28:48	3
1992275	99	43	16,23	cyclictest	2092713	-21	cpuspeed_turbos	11:55:12	2
1992275	99	43	13,26	cyclictest	2055627	-21	latency_hist	10:10:01	2
1992275	99	43	12,26	cyclictest	2057654	-21	latency_hist	10:15:03	2
1992275	99	43	10,29	cyclictest	2067386	-21	proc_pri	10:40:26	2
1992275	99	43	10,29	cyclictest	2062918	-21	date	10:30:01	2
1992263	99	43	26,6	cyclictest	13	-21	ksoftirqd/0	11:40:20	0
1992263	99	43	25,6	cyclictest	13	-21	ksoftirqd/0	12:27:48	0
1992263	99	43	25,6	cyclictest	13	-21	ksoftirqd/0	12:20:11	0
1992263	99	43	25,6	cyclictest	13	-21	ksoftirqd/0	12:20:11	0
1992263	99	43	25,6	cyclictest	13	-21	ksoftirqd/0	11:35:48	0
1992263	99	43	25,6	cyclictest	13	-21	ksoftirqd/0	10:40:09	0
1992279	99	42	15,22	cyclictest	42	-21	ksoftirqd/3	12:35:29	3
1992275	99	42	16,22	cyclictest	1137	-21	runrttasks	12:31:06	2
1992275	99	42	16,22	cyclictest	1137	-21	runrttasks	09:18:50	2
1992270	99	42	19,19	cyclictest	1	-21	systemd	12:10:00	1
1992270	99	42	1,36	cyclictest	2025601	-21	sshd	08:41:17	1
1992263	99	42	24,6	cyclictest	13	-21	ksoftirqd/0	12:30:15	0
1992263	99	42	24,6	cyclictest	13	-21	ksoftirqd/0	12:15:15	0
1992263	99	42	24,6	cyclictest	13	-21	ksoftirqd/0	09:55:02	0
1992263	99	42	24,6	cyclictest	13	-21	ksoftirqd/0	09:05:01	0
1992263	99	42	24,6	cyclictest	13	-21	ksoftirqd/0	08:55:02	0
1992263	99	42	24,6	cyclictest	13	-21	ksoftirqd/0	08:45:25	0
1992263	99	42	24,6	cyclictest	13	-21	ksoftirqd/0	08:35:11	0
1992263	99	42	24,6	cyclictest	13	-21	ksoftirqd/0	08:25:30	0
1992263	99	42	24,6	cyclictest	13	-21	ksoftirqd/0	07:40:14	0
1992263	99	42	24,6	cyclictest	13	-21	ksoftirqd/0	07:10:18	0
1992263	99	42	23,6	cyclictest	13	-21	ksoftirqd/0	10:00:25	0
1992263	99	42	23,6	cyclictest	13	-21	ksoftirqd/0	09:30:00	0
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17