OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #7, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system

Characteristics of the 792 highest latencies:

System rack7slot2.osadl.org (updated Thu Sep 24, 2026 00:44:10)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
3770077	99	103	88,5	cyclictest	3857804	-21	kworker/u4:7+events_unbound	22:40:58	1
3770076	99	102	95,5	cyclictest	3937825	-21	kworker/u4:10+events_unbound	23:46:58	0
3770077	99	101	94,5	cyclictest	3857801	-21	kworker/u4:4+events_unbound	22:15:52	1
3770076	99	100	92,6	cyclictest	3972764	-21	kworker/u4:7+events_unbound	00:23:25	0
3770076	99	100	92,6	cyclictest	3795667	-21	kworker/u4:1+events_unbound	21:10:42	0
3770077	99	97	88,6	cyclictest	3857804	-21	kworker/u4:7+events_unbound	22:20:53	1
3770077	99	93	85,6	cyclictest	3768804	-21	kworker/u4:6+events_unbound	19:20:22	1
3770077	99	90	83,5	cyclictest	3866981	-21	kworker/u4:2+events_unbound	22:30:55	1
3770077	99	90	83,5	cyclictest	3866981	-21	kworker/u4:2+events_unbound	22:30:55	1
3770077	99	86	78,6	cyclictest	3915951	-21	kworker/u4:13+events_unbound	23:36:05	1
3770077	99	86	72,5	cyclictest	3888919	-21	kworker/u4:5+events_unbound	23:01:02	1
3770077	99	86	68,7	cyclictest	3955941	-21	kworker/u4:1+events_unbound	00:08:20	1
3770077	99	82	74,6	cyclictest	3955943	-21	kworker/u4:4+events_unbound	00:03:11	1
3770076	99	82	12,12	cyclictest	3770056	-21	munin-run	19:10:00	0
3770076	99	81	73,6	cyclictest	3771304	-21	kworker/u4:13+events_unbound	19:40:27	0
3770076	99	80	58,16	cyclictest	53	-21	kswapd0	22:40:10	0
3770076	99	79	71,6	cyclictest	3812562	-21	kworker/u4:9+events_unbound	21:40:48	0
3770077	99	74	63,5	cyclictest	3793103	-21	kworker/u4:15+events_unbound	20:45:37	1
3770077	99	74	63,5	cyclictest	3793103	-21	kworker/u4:15+events_unbound	20:45:37	1
3770077	99	72	64,6	cyclictest	3982415	-21	kworker/u4:4+events_unbound	00:38:36	1
3770076	99	70	62,6	cyclictest	3897937	-21	kworker/u4:10+events_unbound	23:15:03	0
3770077	99	68	61,5	cyclictest	3955941	-21	kworker/u4:1+events_unbound	00:33:33	1
3770077	99	68	56,4	cyclictest	3835545	-21	kworker/u4:12+events_unbound	21:50:48	1
3770076	99	68	49,7	cyclictest	3791832	-21	kworker/u4:3+events_unbound	21:01:46	0
3770077	99	66	60,5	cyclictest	3893445	-21	kworker/u4:3+events_unbound	23:21:05	1
3770077	99	64	56,6	cyclictest	3906310	-21	kworker/u4:4+events_unbound	23:16:04	1
3770077	99	64	48,9	cyclictest	3770010	-21	kworker/u4:9+events_unbound	20:15:00	1
3770077	99	64	47,7	cyclictest	3781615	-21	kworker/u4:18+events_unbound	20:25:00	1
3770077	99	63	54,7	cyclictest	3793103	-21	kworker/u4:15+events_unbound	21:05:03	1
3770077	99	62	49,7	cyclictest	3821715	-21	kworker/u4:4+events_unbound	21:37:46	1
3770076	99	62	46,9	cyclictest	3791849	-21	kworker/u4:3	20:35:00	0
3770077	99	60	53,4	cyclictest	3773880	-21	kworker/u4:3+events_unbound	20:00:29	1
3770077	99	60	52,6	cyclictest	3834866	-21	kworker/u4:11+events_unbound	22:25:04	1
3770076	99	60	42,16	cyclictest	53	-21	kswapd0	23:35:08	0
3770077	99	57	46,5	cyclictest	3775151	-21	kworker/u4:7+events_unbound	19:40:15	1
3770077	99	57	39,7	cyclictest	3812560	-21	kworker/u4:6+events_unbound	21:40:03	1
3770077	99	56	50,4	cyclictest	3795667	-21	kworker/u4:1+events_unbound	21:15:02	1
3770077	99	56	41,6	cyclictest	3897278	-21	kworker/u4:9+events_unbound	23:10:02	1
3770077	99	56	40,6	cyclictest	3897278	-21	kworker/u4:9+events_unbound	23:09:46	1
3770076	99	56	37,17	cyclictest	3837498	-21	aten2__	21:50:19	0
3770076	99	56	25,16	cyclictest	3780342	-21	kworker/u4:2+events_unbound	21:30:58	0
3770077	99	55	47,6	cyclictest	3781615	-21	kworker/u4:18+events_unbound	20:15:02	1
3770077	99	55	47,5	cyclictest	0	-21	swapper/1	22:45:35	1
3770077	99	55	44,6	cyclictest	3770010	-21	kworker/u4:9+events_unbound	19:35:00	1
3770076	99	55	48,5	cyclictest	3830937	-21	kworker/u4:0+events_unbound	22:10:02	0
3770076	99	55	35,15	cyclictest	53	-21	kswapd0	22:25:09	0
3770077	99	54	48,4	cyclictest	3770010	-21	kworker/u4:9+events_unbound	21:11:36	1
3770077	99	54	47,5	cyclictest	3888919	-21	kworker/u4:5+events_unbound	23:30:01	1
3770077	99	54	47,5	cyclictest	3888919	-21	kworker/u4:5+events_unbound	23:30:01	1
3770077	99	54	47,5	cyclictest	3857804	-21	kworker/u4:7+events_unbound	22:40:01	1
3770077	99	54	43,5	cyclictest	3775151	-21	kworker/u4:7+events_unbound	19:50:03	1
3770076	99	54	23,17	cyclictest	15	1	ktimers/0	22:35:36	0
3770077	99	53	46,5	cyclictest	3896839	-21	timerwakeupswit	22:55:45	1
3770077	99	53	44,4	cyclictest	3888919	-21	kworker/u4:5+events_unbound	00:28:30	1
3770077	99	52	45,5	cyclictest	3967806	-21	timerandwakeup	00:15:39	1
3770077	99	52	45,5	cyclictest	3812562	-21	kworker/u4:9+events_unbound	21:25:45	1
3770077	99	52	45,5	cyclictest	3791833	-21	kworker/u4:12+events_unbound	21:00:41	1
3770076	99	52	44,6	cyclictest	3787528	-21	sshd	20:15:31	0
3770076	99	52	30,19	cyclictest	0	-21	swapper/0	21:45:40	0
3770077	99	51	44,5	cyclictest	3892723	-21	systemd	22:50:59	1
3770077	99	51	41,6	cyclictest	3927231	-21	netstat	23:30:24	1
3770077	99	51	34,5	cyclictest	3770011	-21	kworker/u4:10+events_unbound	20:35:00	1
3770076	99	51	45,3	cyclictest	0	-21	swapper/0	20:30:00	0
3770076	99	51	43,5	cyclictest	3877544	-21	munin-plugin-st	22:35:00	0
3770076	99	51	43,5	cyclictest	3877544	-21	munin-plugin-st	22:35:00	0
3770076	99	51	43,5	cyclictest	3806915	-21	sshd	21:15:42	0
3770077	99	50	43,5	cyclictest	3837126	-21	idleruntime-cro	21:50:00	1
3770077	99	50	40,8	cyclictest	3949455	-21	/usr/sbin/munin	23:55:22	1
3770077	99	50	39,8	cyclictest	3962447	-21	df_abs	00:10:14	1
3770077	99	50	39,8	cyclictest	3962447	-21	df_abs	00:10:14	1
3770076	99	50	6,21	cyclictest	3926303	-21	modprobe	23:30:00	0
3770076	99	50	6,21	cyclictest	3926303	-21	modprobe	23:30:00	0
3770076	99	50	44,4	cyclictest	3937825	-21	kworker/u4:10+events_unbound	00:00:04	0
3770076	99	50	43,5	cyclictest	3944705	-21	/usr/sbin/munin	23:50:20	0
3770076	99	50	43,5	cyclictest	3936458	-21	/usr/sbin/munin	23:40:40	0
3770076	99	50	43,5	cyclictest	3933261	-21	kworker/u4:6+events_unbound	23:55:02	0
3770076	99	50	43,5	cyclictest	3928777	-21	kworker/u4:3+events_unbound	00:31:46	0
3770076	99	50	43,5	cyclictest	3785835	-21	df_abs	20:10:16	0
3770076	99	50	42,6	cyclictest	3796205	-21	fschecks_time	20:50:21	0
3770076	99	50	41,6	cyclictest	3926901	-21	sort	23:30:20	0
3770076	99	50	38,10	cyclictest	3963126	-21	open_inodes	00:10:33	0
3770076	99	50	38,10	cyclictest	3963126	-21	open_inodes	00:10:33	0
3770077	99	49	42,5	cyclictest	3834866	-21	kworker/u4:11+events_unbound	21:57:46	1
3770077	99	49	42,5	cyclictest	3791832	-21	kworker/u4:3+events_unbound	20:35:35	1
3770077	99	49	36,6	cyclictest	3830937	-21	kworker/u4:0+events_unbound	22:10:18	1
3770076	99	49	42,5	cyclictest	507	-21	ntpd	21:55:48	0
3770076	99	49	42,5	cyclictest	3970879	-21	sendmail-msp	00:20:00	0
3770076	99	49	42,5	cyclictest	3958068	-21	df	00:05:19	0
3770076	99	49	42,5	cyclictest	3897278	-21	kworker/u4:9+events_unbound	22:56:15	0
3770076	99	49	42,5	cyclictest	3770010	-21	kworker/u4:9+events_unbound	21:10:01	0
3770076	99	49	41,6	cyclictest	3915951	-21	kworker/u4:13+events_unbound	23:22:44	0
3770076	99	49	41,6	cyclictest	3777692	-21	kworker/u4:17+events_unbound	20:00:03	0
3770077	99	48	39,6	cyclictest	3972107	-21	proc_pri	00:20:37	1
3770076	99	48	41,5	cyclictest	3910045	-21	switchtime	23:10:42	0
3770076	99	48	41,5	cyclictest	3855274	-21	latency_hist	22:10:01	0
3770076	99	48	41,5	cyclictest	3788686	-21	/usr/sbin/munin	20:20:20	0
3770076	99	48	41,5	cyclictest	3780342	-21	kworker/u4:2+events_unbound	20:45:02	0
3770076	99	48	41,5	cyclictest	3780342	-21	kworker/u4:2+events_unbound	20:45:02	0
3770076	99	48	38,7	cyclictest	3847914	-21	sd-accept	22:00:49	0
3770077	99	47	41,4	cyclictest	3820636	-21	sshd	21:30:45	1
3770077	99	47	40,5	cyclictest	3935830	-21	fschecks_count	23:40:22	1
3770077	99	47	40,5	cyclictest	3770073	-21	cyclictest	19:35:25	1
3770077	99	47	36,8	cyclictest	3771317	-21	cron	19:15:00	1
3770076	99	47	38,7	cyclictest	3886711	-21	acpi	22:45:11	0
3770076	99	47	38,7	cyclictest	378	-21	dbus-daemon	20:35:35	0
3770076	99	47	36,9	cyclictest	3782873	-21	vmstat	19:55:40	0
3770076	99	47	29,5	cyclictest	3895479	-21	latency_hist	22:55:02	0
3770077	99	46	39,5	cyclictest	3939965	-21	apt-get	23:45:14	1
3770077	99	46	38,4	cyclictest	0	-21	swapper/1	20:55:38	1
3770077	99	46	36,8	cyclictest	3790590	-21	kworker/u4:18	20:30:00	1
3770077	99	46	36,8	cyclictest	3770186	-21	latency_hist	19:10:02	1
3770077	99	46	35,7	cyclictest	0	-21	swapper/1	22:05:34	1
3770076	99	46	39,5	cyclictest	3777692	-21	kworker/u4:17+events_unbound	20:09:45	0
3770076	99	46	39,5	cyclictest	3770010	-21	kworker/u4:9+events_unbound	19:45:03	0
3770076	99	46	38,6	cyclictest	3976017	-21	fschecks_time	00:25:25	0
3770076	99	46	3,39	cyclictest	3824516	-21	ssh	21:35:26	0
3770077	99	45	39,4	cyclictest	3847148	-21	nfs4_client	22:00:26	1
3770077	99	45	38,5	cyclictest	3784564	-21	sed	20:05:21	1
3770077	99	45	36,6	cyclictest	3774456	-21	gltestperf	19:25:19	1
3770077	99	45	35,7	cyclictest	3772751	-21	latency_hist	19:20:00	1
3770076	99	45	39,4	cyclictest	3904634	-21	aten_r7power_vo	23:05:17	0
3770076	99	45	38,5	cyclictest	3797324	-21	df_abs	20:55:17	0
3770076	99	45	38,5	cyclictest	3770010	-21	kworker/u4:9+events_unbound	19:25:02	0
3770076	99	45	37,6	cyclictest	3860438	-21	gltestperf	22:15:23	0
3770076	99	45	35,8	cyclictest	3865199	-21	ssh	22:20:26	0
3770076	99	45	22,21	cyclictest	3773095	-21	fschecks_time	19:20:19	0
3770077	99	44	37,4	cyclictest	3790562	-21	kworker/u4:0-rpciod	20:40:19	1
3770077	99	44	33,9	cyclictest	3796599	-21	sendmail_mailqu	20:50:37	1
3770076	99	44	37,5	cyclictest	3770445	-21	sed	19:10:19	0
3770076	99	44	36,6	cyclictest	3776421	-21	kworker/u4:15+events_unbound	19:39:18	0
3770076	99	44	34,8	cyclictest	0	-21	swapper/0	19:15:39	0
3770076	99	44	33,8	cyclictest	3810465	-21	aten_r7power_vo	21:20:19	0
3770076	99	44	17,15	cyclictest	3986739	-21	rm	00:37:07	0
3770077	99	43	35,5	cyclictest	3810343	-21	aten2__	21:20:16	1
3770076	99	43	15,16	cyclictest	15	1	ktimers/0	23:00:28	0
3770076	99	43	15,15	cyclictest	15	1	ktimers/0	21:29:53	0
3770077	99	41	32,6	cyclictest	400	-21	systemd-logind	19:59:44	1
3770077	99	41	31,8	cyclictest	0	-21	swapper/1	23:50:34	1
3770076	99	41	34,5	cyclictest	3776291	-21	grep	19:30:41	0
3770077	99	40	33,5	cyclictest	3779525	-21	fschecks_count	19:45:21	1
3770076	99	40	1,14	cyclictest	3794435	-21	idleruntime-cro	20:45:02	0
3770076	99	39	30,7	cyclictest	3780664	-21	aten_r7power_vo	19:50:16	0
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17