OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #7, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system

Characteristics of all highest latencies:

System rack7slot2.osadl.org (updated Fri Mar 27, 2026 12:44:10)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
1968111	99	88	78,6	cyclictest	2087437	-21	kworker/u4:14+events_unbound	11:00:03	0
1968112	99	81	37,28	cyclictest	2036557	-21	grep	09:50:39	1
1968112	99	81	37,28	cyclictest	2036557	-21	grep	09:50:39	1
1968111	99	79	60,16	cyclictest	53	-21	kswapd0	12:10:06	0
1968111	99	73	64,6	cyclictest	2069241	-21	kworker/u4:6+events_unbound	10:29:59	0
1968112	99	71	38,30	cyclictest	2003535	-21	apt	09:14:59	1
1968112	99	71	30,35	cyclictest	2140773	-21	kworker/u4:15-rpciod	11:50:19	1
1968112	99	70	35,31	cyclictest	0	-21	swapper/1	09:55:16	1
1968112	99	70	35,31	cyclictest	0	-21	swapper/1	09:55:16	1
1968111	99	70	63,5	cyclictest	1977193	-21	kworker/u4:0+events_unbound	08:44:48	0
1968112	99	69	37,28	cyclictest	2142900	-21	latency_hist	11:50:01	1
1968111	99	69	61,6	cyclictest	2055530	-21	kworker/u4:18+events_unbound	10:14:58	0
1968112	99	68	34,30	cyclictest	0	-21	swapper/1	09:25:38	1
1968112	99	67	36,28	cyclictest	1976824	-21	proc_pri	07:40:37	1
1968111	99	67	56,8	cyclictest	2031195	-21	ssh	09:45:20	0
1968112	99	66	33,31	cyclictest	2152723	-21	ssh	12:00:26	1
1968112	99	66	31,33	cyclictest	2089049	-21	ssh	10:50:02	1
1968112	99	65	34,28	cyclictest	2133979	-21	latency_hist	11:40:01	1
1968112	99	65	32,30	cyclictest	2124920	-21	latency_hist	11:30:00	1
1968111	99	65	51,10	cyclictest	1977202	-21	systemd	07:44:42	0
1968111	99	65	29,33	cyclictest	256	50	irq/23-42890000.ethernet	07:10:06	0
1968112	99	64	29,31	cyclictest	0	-21	swapper/1	11:30:20	1
1968112	99	64	28,32	cyclictest	0	-21	swapper/1	11:05:38	1
1968112	99	64	28,32	cyclictest	0	-21	swapper/1	11:05:38	1
1968111	99	64	56,5	cyclictest	2069241	-21	kworker/u4:6+events_unbound	10:40:01	0
1968111	99	64	23,37	cyclictest	1066	-21	snmpd	08:55:07	0
1968111	99	64	23,37	cyclictest	1066	-21	snmpd	08:55:07	0
1968112	99	63	6,9	cyclictest	53	-21	kswapd0	10:25:11	1
1968112	99	63	33,27	cyclictest	2075947	-21	/usr/sbin/munin	10:35:16	1
1968112	99	63	33,27	cyclictest	2066613	-21	awk	10:25:01	1
1968111	99	63	47,11	cyclictest	37	-21	kcompactd0	10:34:57	0
1968112	99	62	28,31	cyclictest	0	-21	swapper/1	10:10:30	1
1968112	99	62	25,34	cyclictest	0	-21	swapper/1	09:40:39	1
1968111	99	62	49,9	cyclictest	2140772	-21	kworker/u4:14-rpciod	12:00:39	0
1968112	99	61	33,25	cyclictest	0	-21	swapper/1	10:00:17	1
1968112	99	61	2,16	cyclictest	1995932	-21	ntpq	08:55:31	1
1968112	99	61	2,16	cyclictest	1995932	-21	ntpq	08:55:31	1
1968111	99	61	6,44	cyclictest	1971610	-21	sshd	07:20:33	0
1968111	99	61	53,6	cyclictest	1968040	-21	kworker/u4:10+events_unbound	07:19:42	0
1968111	99	61	42,16	cyclictest	0	-21	swapper/0	11:15:12	0
1968112	99	60	34,24	cyclictest	2183268	-21	ssh	12:35:18	1
1968112	99	60	33,24	cyclictest	0	-21	swapper/1	10:40:30	1
1968112	99	60	31,26	cyclictest	0	-21	swapper/1	11:40:27	1
1968112	99	60	26,31	cyclictest	0	-21	swapper/1	12:20:28	1
1968112	99	60	26,31	cyclictest	0	-21	swapper/1	11:20:39	1
1968112	99	60	25,31	cyclictest	2003897	-21	aten_r7power_cu	09:15:15	1
1968112	99	59	30,27	cyclictest	0	-21	swapper/1	11:57:35	1
1968112	99	59	30,26	cyclictest	2099306	-21	pluginstate	11:00:42	1
1968112	99	59	28,28	cyclictest	1990353	-21	sed	08:35:16	1
1968112	99	59	27,30	cyclictest	2156677	-21	aten_r7power_vo	12:05:17	1
1968112	99	59	26,29	cyclictest	0	-21	swapper/1	11:10:17	1
1968112	99	59	24,31	cyclictest	1968110	-21	cyclictest	07:10:02	1
1968112	99	59	15,41	cyclictest	0	-21	swapper/1	09:35:23	1
1968111	99	59	52,5	cyclictest	2123237	-21	kworker/u4:5+events_unbound	11:35:10	0
1968112	99	58	31,24	cyclictest	2154380	-21	kworker/u4:9-rpciod	12:30:21	1
1968112	99	58	27,29	cyclictest	2053072	-21	munin-run	10:10:00	1
1968112	99	58	27,28	cyclictest	0	-21	swapper/1	11:15:19	1
1968112	99	58	26,30	cyclictest	0	-21	swapper/1	12:25:25	1
1968112	99	58	26,29	cyclictest	0	-21	swapper/1	09:20:29	1
1968112	99	58	25,30	cyclictest	1988688	-21	kworker/u4:10+rpciod	08:30:16	1
1968112	99	58	23,33	cyclictest	1993486	-21	mailstats	08:45:34	1
1968112	99	58	1,13	cyclictest	1994608	-21	ntpq	08:50:29	1
1968112	99	57	25,29	cyclictest	2169434	-21	ssh	12:20:00	1
1968112	99	57	24,30	cyclictest	0	-21	swapper/1	10:50:21	1
1968112	99	57	12,41	cyclictest	1987313	-21	unixbench_multi	08:20:37	1
1968112	99	57	1,14	cyclictest	2161773	-21	ntpq	12:10:34	1
1968111	99	57	48,7	cyclictest	2169521	-21	latency_hist	12:20:01	0
1968111	99	57	48,6	cyclictest	2172369	-21	kworker/u4:16+events_unbound	12:35:00	0
1968111	99	57	47,7	cyclictest	1968110	-21	cyclictest	12:38:17	0
1968112	99	56	3,18	cyclictest	2031622	-21	sh	09:45:30	1
1968112	99	56	2,9	cyclictest	2018337	-21	ps	09:30:39	1
1968112	99	56	2,9	cyclictest	2018337	-21	ps	09:30:39	1
1968112	99	56	29,24	cyclictest	0	-21	swapper/1	07:10:02	1
1968112	99	56	25,28	cyclictest	0	-21	swapper/1	10:56:33	1
1968112	99	56	2,17	cyclictest	1970317	-21	ps	07:15:34	1
1968111	99	56	48,5	cyclictest	2042433	-21	kworker/u4:11+events_unbound	09:59:57	0
1968111	99	56	48,5	cyclictest	2042433	-21	kworker/u4:11+events_unbound	09:59:57	0
1968111	99	56	47,6	cyclictest	1981007	-21	kworker/u4:6+events_unbound	08:00:00	0
1968111	99	56	45,9	cyclictest	0	-21	swapper/0	11:45:12	0
1968112	99	55	3,8	cyclictest	1975413	-21	ntpq	07:35:28	1
1968112	99	55	31,21	cyclictest	0	-21	swapper/1	07:45:18	1
1968112	99	55	26,27	cyclictest	0	-21	swapper/1	08:25:37	1
1968112	99	55	1,13	cyclictest	1998573	-21	ntpq	09:05:33	1
1968111	99	55	43,8	cyclictest	1981499	-21	fschecks_count	08:00:17	0
1968112	99	54	24,27	cyclictest	0	-21	swapper/1	10:15:26	1
1968112	99	54	2,13	cyclictest	2071648	-21	ssh	10:30:19	1
1968112	99	54	1,13	cyclictest	1985698	-21	ntpq	08:15:33	1
1968111	99	54	47,5	cyclictest	15	1	ktimers/0	09:21:47	0
1968111	99	54	46,6	cyclictest	2098037	-21	latency_hist	11:00:00	0
1968112	99	53	17,31	cyclictest	1982343	-21	munin-run	08:05:01	1
1968112	99	53	16,34	cyclictest	1971385	-21	grep	07:20:24	1
1968112	99	53	1,14	cyclictest	1980150	-21	df_inode	07:55:17	1
1968111	99	53	46,5	cyclictest	2116356	-21	/usr/sbin/munin	11:20:15	0
1968111	99	53	40,9	cyclictest	2004918	-21	ls	09:15:37	0
1968111	99	53	27,16	cyclictest	0	-21	swapper/0	12:25:28	0
1968112	99	52	27,22	cyclictest	0	-21	swapper/1	08:45:00	1
1968112	99	52	22,27	cyclictest	0	-21	swapper/1	07:50:17	1
1968112	99	52	14,4	cyclictest	1984374	-21	ntp_kernel_pll_	08:10:29	1
1968112	99	52	1,13	cyclictest	1972838	-21	ntpq	07:25:24	1
1968111	99	52	44,5	cyclictest	2018529	-21	timerandwakeup	09:30:41	0
1968111	99	52	44,5	cyclictest	2018529	-21	timerandwakeup	09:30:41	0
1968111	99	52	27,20	cyclictest	0	-21	swapper/0	08:30:00	0
1968112	99	51	27,21	cyclictest	0	-21	swapper/1	07:30:15	1
1968111	99	51	30,6	cyclictest	2046367	-21	kworker/u4:12-rpciod	10:05:38	0
1968112	99	50	12,12	cyclictest	1996794	-21	grep	09:00:16	1
1968112	99	50	1,10	cyclictest	1983091	-21	ntpq	08:05:27	1
1968111	99	50	38,5	cyclictest	0	-21	swapper/0	10:50:01	0
1968111	99	49	42,5	cyclictest	2156978	-21	gltestperf	12:05:21	0
1968111	99	49	41,5	cyclictest	2048629	-21	sed	10:05:01	0
1968111	99	49	41,5	cyclictest	15	1	ktimers/0	11:51:58	0
1968111	99	49	38,8	cyclictest	1989101	-21	df_abs	08:30:17	0
1968111	99	49	34,11	cyclictest	53	-21	kswapd0	10:40:13	0
1968111	99	48	4,20	cyclictest	2134829	-21	ssh	11:40:24	0
1968111	99	48	39,5	cyclictest	2151706	-21	modprobe	12:00:00	0
1968111	99	48	38,7	cyclictest	398	-21	systemd-logind	11:30:08	0
1968111	99	48	38,7	cyclictest	1972373	-21	aten_r7power_po	07:25:16	0
1968111	99	48	29,5	cyclictest	2121202	-21	gltestperf	11:25:20	0
1968111	99	47	4,33	cyclictest	2030465	-21	ssh	09:44:55	0
1968111	99	47	40,5	cyclictest	2063041	-21	/usr/sbin/munin	10:20:23	0
1968111	99	47	39,6	cyclictest	1974623	-21	kworker/u4:3+events_unbound	07:49:43	0
1968111	99	47	39,5	cyclictest	1993835	-21	munin-plugin-st	08:50:01	0
1968111	99	47	37,6	cyclictest	1972085	-21	kworker/u4:6-rpciod	07:39:59	0
1968111	99	46	39,5	cyclictest	2107372	-21	aten_r7power_po	11:10:15	0
1968111	99	46	38,6	cyclictest	2170583	-21	diskmemload	12:20:31	0
1968111	99	46	37,7	cyclictest	2089617	-21	fschecks_count	10:50:20	0
1968111	99	46	36,6	cyclictest	2019601	-21	kworker/u4:7+events_unbound	09:50:13	0
1968111	99	46	36,6	cyclictest	2019601	-21	kworker/u4:7+events_unbound	09:50:13	0
1968111	99	46	32,8	cyclictest	0	-21	swapper/0	10:17:12	0
1968111	99	45	35,8	cyclictest	2013343	-21	gltestperf	09:25:20	0
1968111	99	45	34,8	cyclictest	2022966	-21	xargs	09:35:38	0
1968111	99	45	27,16	cyclictest	1982496	-21	acpi	08:05:11	0
1968111	99	44	35,7	cyclictest	1973653	-21	aten_r7power_po	07:30:16	0
1968111	99	44	35,6	cyclictest	2102937	-21	/usr/sbin/munin	11:05:16	0
1968111	99	44	35,6	cyclictest	2102937	-21	/usr/sbin/munin	11:05:16	0
1968111	99	43	36,5	cyclictest	1997355	-21	grep	09:00:35	0
1968111	99	43	36,5	cyclictest	1984753	-21	unixbench_multi	08:10:38	0
1968111	99	43	34,7	cyclictest	1994003	-21	aten2__	08:50:14	0
1968111	99	43	34,6	cyclictest	1998213	-21	fschecks_time	09:05:17	0
1968111	99	42	34,5	cyclictest	1986671	-21	fschecks_time	08:20:18	0
1968111	99	42	33,6	cyclictest	1987418	-21	kworker/u4:6+events_unbound	08:40:01	0
1968111	99	41	33,6	cyclictest	1978718	-21	aten_r7power_cu	07:50:16	0
1968111	99	41	32,6	cyclictest	0	-21	swapper/0	08:15:35	0
1968111	99	41	31,7	cyclictest	0	-21	swapper/0	09:10:33	0
1968111	99	39	30,6	cyclictest	0	-21	swapper/0	07:10:02	0
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17