OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #7, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system

Characteristics of the 376 highest latencies:

System rack7slot2.osadl.org (updated Sun Sep 13, 2026 00:44:10)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
1936506	99	115	107,5	cyclictest	2116690	-21	kworker/u4:1+events_unbound	23:58:12	0
1936507	99	102	89,6	cyclictest	1945412	-21	kworker/u4:3+events_unbound	19:42:43	1
1936506	99	100	63,34	cyclictest	53	-21	kswapd0	00:25:12	0
1936506	99	100	4,27	cyclictest	1936484	-21	munin-run	19:10:00	0
1936506	99	100	4,27	cyclictest	1936484	-21	munin-run	19:10:00	0
1936506	99	91	48,40	cyclictest	2029946	-21	latency_hist	22:20:02	0
1936506	99	90	57,30	cyclictest	53	-21	kswapd0	22:00:13	0
1936507	99	86	78,6	cyclictest	1955726	-21	kworker/u4:12+events_unbound	20:27:51	1
1936507	99	86	78,6	cyclictest	1955726	-21	kworker/u4:12+events_unbound	20:27:51	1
1936506	99	81	74,5	cyclictest	2067842	-21	kworker/u4:15+events_unbound	23:03:09	0
1936506	99	78	73,3	cyclictest	2018978	-21	kworker/u4:15+events_unbound	22:08:06	0
1936507	99	68	60,5	cyclictest	0	-21	swapper/1	21:40:13	1
1936507	99	65	54,7	cyclictest	2101534	-21	ssh	23:40:19	1
1936507	99	64	50,10	cyclictest	37	-21	kcompactd0	22:12:00	1
1936506	99	64	47,5	cyclictest	2041489	-21	kworker/u4:2+events_unbound	22:33:08	0
1936506	99	63	43,18	cyclictest	2153981	-21	munin-run	00:40:01	0
1936506	99	62	55,5	cyclictest	53	-21	kswapd0	23:10:00	0
1936506	99	62	52,7	cyclictest	2134679	-21	kworker/u4:1+events_unbound	00:18:16	0
1936507	99	61	52,7	cyclictest	2121183	-21	kworker/u4:9+events_unbound	00:15:00	1
1936506	99	61	43,6	cyclictest	1970115	-21	kworker/u4:3+events_unbound	21:17:59	0
1936506	99	61	40,17	cyclictest	0	-21	swapper/0	00:05:06	0
1936506	99	60	52,5	cyclictest	2032476	-21	kworker/u4:1+events_unbound	22:23:07	0
1936507	99	59	53,5	cyclictest	1922503	-21	kworker/u4:9+events_unbound	19:35:02	1
1936507	99	59	50,7	cyclictest	400	-21	systemd-logind	21:17:58	1
1936506	99	59	42,13	cyclictest	2101732	-21	if_eth0	23:40:20	0
1936507	99	58	50,6	cyclictest	53	-21	kswapd0	22:45:00	1
1936506	99	58	38,16	cyclictest	2105944	-21	grep	23:45:20	0
1936506	99	57	46,9	cyclictest	1939014	-21	systemd	19:17:38	0
1936507	99	56	48,6	cyclictest	2150224	-21	/usr/sbin/munin	00:35:20	1
1936507	99	56	42,6	cyclictest	2010659	-21	kworker/u4:3+events_unbound	22:02:29	1
1936506	99	56	48,6	cyclictest	2146017	-21	/usr/sbin/munin	00:30:25	0
1936506	99	56	47,6	cyclictest	1936502	-21	cyclictest	23:51:10	0
1936506	99	56	47,5	cyclictest	0	-21	swapper/0	21:40:29	0
1936506	99	56	44,8	cyclictest	1936502	-21	cyclictest	23:15:21	0
1936506	99	56	42,8	cyclictest	0	-21	swapper/0	21:35:29	0
1936507	99	55	47,6	cyclictest	2017633	-21	/usr/sbin/munin	22:05:23	1
1936507	99	55	44,8	cyclictest	2115329	-21	ntp_states	23:55:36	1
1936506	99	55	46,7	cyclictest	317	-21	systemd-journal	22:13:05	0
1936507	99	53	44,6	cyclictest	0	-21	swapper/1	21:45:38	1
1936507	99	53	39,7	cyclictest	0	-21	swapper/1	23:20:18	1
1936506	99	53	47,4	cyclictest	2032480	-21	kworker/u4:9+events_unbound	23:30:02	0
1936506	99	53	47,4	cyclictest	2032480	-21	kworker/u4:9+events_unbound	23:30:01	0
1936506	99	53	44,7	cyclictest	2075604	-21	/usr/sbin/munin	23:10:40	0
1936507	99	52	43,7	cyclictest	317	-21	systemd-journal	23:48:11	1
1936506	99	52	5,42	cyclictest	0	-21	swapper/0	00:00:09	0
1936507	99	51	45,5	cyclictest	1948041	-21	kworker/u4:1+events_unbound	20:20:07	1
1936507	99	51	44,5	cyclictest	1967247	-21	diskmemload	00:08:13	1
1936507	99	51	41,8	cyclictest	2030280	-21	/usr/sbin/munin	22:20:15	1
1936506	99	51	46,4	cyclictest	2130175	-21	kworker/u4:11+events_unbound	00:13:15	0
1936506	99	51	42,7	cyclictest	2052801	-21	sh	22:45:21	0
1936507	99	50	42,6	cyclictest	1983215	-21	kworker/u4:12+events_unbound	21:29:46	1
1936506	99	50	42,5	cyclictest	2004114	-21	fschecks_count	21:50:25	0
1936506	99	50	2,11	cyclictest	1945450	-21	apt-get	19:45:11	0
1936507	99	49	43,4	cyclictest	0	-21	swapper/1	23:10:17	1
1936507	99	49	42,5	cyclictest	1952902	-21	switchtime	20:10:37	1
1936507	99	49	41,5	cyclictest	1964395	-21	switchtime	20:55:41	1
1936507	99	49	40,6	cyclictest	0	-21	swapper/1	21:30:41	1
1936507	99	49	40,6	cyclictest	0	-21	swapper/1	21:30:41	1
1936507	99	49	39,7	cyclictest	1951876	-21	kworker/u4:6-rpciod	21:10:17	1
1936507	99	49	39,7	cyclictest	0	-21	swapper/1	23:00:12	1
1936506	99	49	42,5	cyclictest	2061664	-21	ssh	22:55:24	0
1936506	99	49	41,6	cyclictest	1948042	-21	kworker/u4:2+events_unbound	20:32:52	0
1936506	99	49	41,6	cyclictest	1922503	-21	kworker/u4:9+events_unbound	19:32:40	0
1936506	99	49	32,15	cyclictest	1977220	-21	kernelversion	21:20:24	0
1936507	99	48	42,5	cyclictest	2085206	-21	kworker/u4:6+events_unbound	23:35:45	1
1936507	99	48	41,5	cyclictest	2041489	-21	kworker/u4:2+events_unbound	23:29:45	1
1936507	99	48	40,6	cyclictest	2059483	-21	kworker/u4:13+events_unbound	22:53:08	1
1936507	99	48	38,6	cyclictest	1937728	-21	kworker/u4:8-xprtiod	19:25:37	1
1936507	99	48	37,6	cyclictest	0	-21	swapper/1	00:25:06	1
1936507	99	48	34,6	cyclictest	2032480	-21	kworker/u4:9+events_unbound	22:37:45	1
1936506	99	48	42,4	cyclictest	2044387	-21	ntp_offset	22:35:32	0
1936506	99	48	36,10	cyclictest	2038853	-21	latency_hist	22:30:01	0
1936506	99	48	35,9	cyclictest	0	-21	swapper/0	21:55:40	0
1936506	99	48	35,10	cyclictest	2052149	-21	munin-run	22:45:00	0
1936507	99	47	40,5	cyclictest	2143651	-21	kworker/u4:16+events_unbound	00:32:23	1
1936506	99	47	40,5	cyclictest	1925263	-21	kworker/u4:0+events_unbound	20:15:00	0
1936506	99	47	39,6	cyclictest	1937727	-21	kworker/u4:7+events_unbound	19:22:39	0
1936506	99	47	38,7	cyclictest	2136646	-21	aten2__	00:20:16	0
1936506	99	47	38,5	cyclictest	2058379	-21	ssh	22:50:58	0
1936507	99	46	39,5	cyclictest	2062613	-21	unixbench_singl	22:55:44	1
1936507	99	46	39,5	cyclictest	2032478	-21	kworker/u4:3+events_unbound	23:16:45	1
1936507	99	46	39,5	cyclictest	2006560	-21	ssh	21:53:02	1
1936507	99	46	38,6	cyclictest	1976799	-21	aten_r7power_vo	21:20:18	1
1936507	99	46	37,7	cyclictest	2119092	-21	aten_r7power_en	00:00:25	1
1936507	99	46	37,6	cyclictest	1936502	-21	cyclictest	23:07:45	1
1936507	99	46	36,8	cyclictest	1951405	-21	ntp_kernel_pll_	20:05:28	1
1936506	99	46	40,4	cyclictest	1949020	-21	cat	19:55:40	0
1936506	99	46	39,5	cyclictest	2097196	-21	grep	23:35:20	0
1936506	99	46	38,6	cyclictest	1981482	-21	grep	21:25:18	0
1936506	99	46	38,6	cyclictest	0	-21	swapper/0	20:00:24	0
1936506	99	46	36,7	cyclictest	1953455	-21	aten_r7power_po	20:15:18	0
1936507	99	45	39,4	cyclictest	27	1	ktimers/1	23:51:57	1
1936507	99	45	38,5	cyclictest	1959998	-21	fschecks_count	20:40:16	1
1936507	99	45	37,5	cyclictest	2009305	-21	timerwakeupswit	21:55:40	1
1936507	99	45	36,7	cyclictest	2096443	-21	munin-run	23:35:00	1
1936507	99	45	36,7	cyclictest	2096443	-21	munin-run	23:35:00	1
1936507	99	45	36,6	cyclictest	1991275	-21	threads	21:35:40	1
1936507	99	45	13,7	cyclictest	27	1	ktimers/1	22:30:31	1
1936506	99	45	37,6	cyclictest	1951876	-21	kworker/u4:6+rpciod	20:42:29	0
1936506	99	45	37,6	cyclictest	1951102	-21	fschecks_time	20:05:20	0
1936506	99	45	37,5	cyclictest	2085840	-21	kworker/u4:8-rpciod	23:25:20	0
1936506	99	45	36,7	cyclictest	1963966	-21	gltestperf	20:55:21	0
1936506	99	45	36,5	cyclictest	1985635	-21	/usr/sbin/munin	21:30:15	0
1936506	99	45	36,5	cyclictest	1985635	-21	/usr/sbin/munin	21:30:15	0
1936506	99	45	30,11	cyclictest	1937785	-21	grep	19:15:01	0
1936507	99	44	35,7	cyclictest	1938034	-21	aten_r7power_po	19:15:17	1
1936506	99	44	37,5	cyclictest	1945462	-21	apt-get	19:45:01	0
1936506	99	44	36,6	cyclictest	1955721	-21	kworker/u4:10+events_unbound	21:02:57	0
1936506	99	44	36,5	cyclictest	2076824	-21	kworker/u4:0-rpciod	23:20:25	0
1936506	99	44	35,6	cyclictest	1947291	-21	fschecks_time	19:50:22	0
1936506	99	44	35,6	cyclictest	0	-21	swapper/0	20:25:28	0
1936506	99	44	35,6	cyclictest	0	-21	swapper/0	20:25:28	0
1936507	99	43	6,27	cyclictest	2137389	-21	ssh	00:20:31	1
1936507	99	43	37,4	cyclictest	1950579	-21	unixbench_singl	20:00:38	1
1936507	99	43	35,6	cyclictest	2053256	-21	ntpq	22:45:32	1
1936507	99	43	35,6	cyclictest	0	-21	swapper/1	00:15:23	1
1936506	99	43	8,21	cyclictest	1999931	-21	ssh	21:45:27	0
1936506	99	43	34,7	cyclictest	1954738	-21	aten_r7power_po	20:20:18	0
1936506	99	43	33,7	cyclictest	0	-21	swapper/0	21:10:26	0
1936507	99	42	35,5	cyclictest	2026263	-21	gltestperf	22:15:21	1
1936507	99	42	34,6	cyclictest	1933998	-21	kworker/u4:5+events_unbound	20:17:50	1
1936507	99	42	34,5	cyclictest	1939263	-21	aten2__	19:20:19	1
1936507	99	42	33,6	cyclictest	1957457	-21	fschecks_count	20:30:20	1
1936507	99	42	33,6	cyclictest	0	-21	swapper/1	19:10:37	1
1936506	99	42	32,6	cyclictest	1933998	-21	kworker/u4:5-xprtiod	20:50:28	0
1936507	99	41	35,4	cyclictest	1963428	-21	latency_hist	20:55:01	1
1936507	99	41	34,5	cyclictest	2035156	-21	gltestperf	22:25:19	1
1936507	99	41	34,5	cyclictest	1966421	-21	fschecks_count	21:05:20	1
1936507	99	41	31,7	cyclictest	0	-21	swapper/1	20:45:22	1
1936506	99	41	34,5	cyclictest	1944182	-21	awk	19:40:00	0
1936507	99	40	36,3	cyclictest	27	1	ktimers/1	19:32:32	1
1936507	99	40	34,4	cyclictest	1948042	-21	kworker/u4:2+events_unbound	20:35:46	1
1936507	99	40	34,4	cyclictest	0	-21	swapper/1	19:47:43	1
1936507	99	40	29,9	cyclictest	1949374	-21	sendmail	20:00:00	1
1936507	99	40	29,8	cyclictest	0	-21	swapper/1	21:00:23	1
1936506	99	40	32,5	cyclictest	0	-21	swapper/0	20:45:36	0
1936507	99	39	33,4	cyclictest	1947186	-21	df_abs	19:50:20	1
1936506	99	39	32,5	cyclictest	1966427	-21	cut	21:05:20	0
1936506	99	38	29,7	cyclictest	1958952	-21	kernelversion	20:35:23	0
1936506	99	37	28,7	cyclictest	1940936	-21	head	19:25:22	0
1936507	99	34	7,4	cyclictest	27	1	ktimers/1	19:10:01	1
1936507	99	34	7,4	cyclictest	27	1	ktimers/1	19:10:01	1
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17