OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #7, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system

Characteristics of the highest latencies:

System rack7slot2.osadl.org (updated Tue Jan 27, 2026 12:44:09)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
3649339	99	114	100,10	cyclictest	3764682	-21	kworker/u4:14+events_unbound	11:15:13	0
3649340	99	111	103,6	cyclictest	3743467	-21	kworker/u4:3+events_unbound	10:45:07	1
3649340	99	101	96,3	cyclictest	3739070	-21	kworker/u4:6+events_unbound	10:30:04	1
3649340	99	101	96,3	cyclictest	3739070	-21	kworker/u4:6+events_unbound	10:30:04	1
3649340	99	100	82,7	cyclictest	3730379	-21	kworker/u4:5+events_unbound	10:25:01	1
3649340	99	96	89,6	cyclictest	3668508	-21	kworker/u4:4+events_unbound	09:54:52	1
3649339	99	95	83,9	cyclictest	54	-21	kswapd0	10:00:08	0
3649340	99	91	82,6	cyclictest	54	-21	kswapd0	11:05:09	1
3649340	99	90	83,5	cyclictest	3824552	-21	kworker/u4:5+events_unbound	12:05:24	1
3649340	99	88	81,5	cyclictest	3677325	-21	kworker/u4:6+events_unbound	09:24:43	1
3649340	99	85	78,5	cyclictest	3700057	-21	kworker/u4:13+events_unbound	09:39:50	1
3649340	99	85	76,7	cyclictest	3777703	-21	kworker/u4:3+events_unbound	11:10:12	1
3649339	99	85	72,11	cyclictest	54	-21	kswapd0	12:25:11	0
3649339	99	84	34,21	cyclictest	1068	-21	snmpd	12:05:37	0
3649339	99	82	65,13	cyclictest	54	-21	kswapd0	10:50:13	0
3649340	99	80	66,6	cyclictest	3659487	-21	kworker/u4:15+events_unbound	08:54:38	1
3649340	99	79	69,5	cyclictest	3777704	-21	kworker/u4:4+events_unbound	11:25:15	1
3649340	99	78	69,7	cyclictest	3811987	-21	kworker/u4:9+events_unbound	11:55:01	1
3649339	99	78	39,36	cyclictest	54	-21	kswapd0	11:05:07	0
3649339	99	77	66,8	cyclictest	0	-21	swapper/0	11:30:11	0
3649340	99	75	67,6	cyclictest	3668507	-21	kworker/u4:2+events_unbound	08:49:35	1
3649339	99	75	32,13	cyclictest	3814818	-21	ntpq	11:45:27	0
3649340	99	74	61,5	cyclictest	3682245	-21	kworker/u4:0+events_unbound	09:17:00	1
3649340	99	72	66,4	cyclictest	3658230	-21	kworker/u4:10+events_unbound	08:15:00	1
3649339	99	72	61,8	cyclictest	3768994	-21	kworker/u4:0+events_unbound	11:00:09	0
3649340	99	71	63,6	cyclictest	3764682	-21	kworker/u4:14+events_unbound	10:55:09	1
3649340	99	71	60,7	cyclictest	0	-21	swapper/1	11:45:12	1
3649340	99	69	62,5	cyclictest	3656943	-21	kworker/u4:2+events_unbound	07:49:28	1
3649339	99	67	23,16	cyclictest	3840794	-21	ntpq	12:15:27	0
3649339	99	67	20,25	cyclictest	3797270	-21	fschecks_time	11:25:24	0
3649340	99	66	55,5	cyclictest	3667128	-21	kworker/u4:6+events_unbound	08:40:01	1
3649339	99	65	60,3	cyclictest	3811987	-21	kworker/u4:9+events_unbound	11:55:21	0
3649339	99	65	24,36	cyclictest	0	-21	swapper/0	10:10:07	0
3649340	99	64	56,6	cyclictest	3739070	-21	kworker/u4:6+events_unbound	10:50:07	1
3649339	99	64	4,52	cyclictest	3667183	-21	apt-get	08:20:06	0
3649339	99	64	27,10	cyclictest	3673760	-21	apt-get	08:45:12	0
3649339	99	64	17,26	cyclictest	3657108	-21	latency_hist	07:40:00	0
3649340	99	63	55,6	cyclictest	3768994	-21	kworker/u4:0+events_unbound	11:00:04	1
3649339	99	63	23,14	cyclictest	3810437	-21	ntpq	11:40:24	0
3649339	99	63	17,26	cyclictest	3828058	-21	/usr/sbin/munin	12:00:34	0
3649339	99	63	17,26	cyclictest	3828058	-21	/usr/sbin/munin	12:00:33	0
3649340	99	62	47,5	cyclictest	3664532	-21	kworker/u4:5+events_unbound	08:07:44	1
3649339	99	62	19,11	cyclictest	3659527	-21	munin-run	07:50:01	0
3649340	99	61	54,4	cyclictest	0	-21	swapper/1	12:30:14	1
3649340	99	61	42,6	cyclictest	3734710	-21	kworker/u4:1+events_unbound	10:15:59	1
3649339	99	61	32,25	cyclictest	3750504	-21	ntp_states	10:30:34	0
3649339	99	61	32,25	cyclictest	3750504	-21	ntp_states	10:30:34	0
3649339	99	61	19,21	cyclictest	3836954	-21	grep	12:10:38	0
3649339	99	61	13,25	cyclictest	3737019	-21	df_inode	10:15:16	0
3649340	99	60	46,6	cyclictest	3669765	-21	kworker/u4:12+events_unbound	08:29:34	1
3649340	99	60	43,6	cyclictest	3700057	-21	kworker/u4:13+events_unbound	09:45:01	1
3649339	99	60	51,7	cyclictest	3668508	-21	kworker/u4:4+events_unbound	09:55:19	0
3649339	99	60	43,12	cyclictest	3698659	-21	ps	09:30:36	0
3649339	99	60	35,20	cyclictest	3715205	-21	apt-get	09:50:16	0
3649339	99	60	33,21	cyclictest	3784509	-21	ssh	11:10:22	0
3649339	99	60	22,13	cyclictest	3694057	-21	ntpq	09:25:26	0
3649339	99	60	21,18	cyclictest	3660776	-21	modprobe	07:55:00	0
3649339	99	60	16,24	cyclictest	3685002	-21	grep	09:15:21	0
3649339	99	59	41,12	cyclictest	3754682	-21	ntpq	10:35:28	0
3649339	99	59	23,12	cyclictest	3858057	-21	ntp_states	12:35:34	0
3649339	99	59	19,14	cyclictest	3703533	-21	ssh	09:35:50	0
3649339	99	59	18,17	cyclictest	3679914	-21	apt	09:10:00	0
3649339	99	58	7,47	cyclictest	0	-21	swapper/0	08:15:11	0
3649339	99	58	36,16	cyclictest	3711668	-21	ssh	09:45:33	0
3649339	99	58	23,13	cyclictest	3763240	-21	ntpq	10:45:25	0
3649339	99	58	21,11	cyclictest	3655214	-21	ntpq	07:30:25	0
3649339	99	58	21,10	cyclictest	3683269	-21	ssh	09:13:25	0
3649339	99	58	17,15	cyclictest	3662850	-21	ntpq	08:00:29	0
3649339	99	58	15,21	cyclictest	3742090	-21	ssh	10:20:35	0
3649340	99	57	50,5	cyclictest	3686672	-21	kworker/u4:5+events_unbound	09:28:01	1
3649340	99	57	49,6	cyclictest	3648075	-21	kworker/u4:1+events_unbound	07:34:26	1
3649340	99	57	45,5	cyclictest	3790751	-21	kworker/u4:11+events_unbound	11:38:59	1
3649340	99	57	40,15	cyclictest	0	-21	swapper/1	12:25:35	1
3649339	99	57	27,23	cyclictest	3852993	-21	ssh	12:30:15	0
3649339	99	57	21,15	cyclictest	3818396	-21	/usr/sbin/munin	11:50:26	0
3649339	99	57	19,15	cyclictest	3747526	-21	ssh	10:27:01	0
3649339	99	57	19,14	cyclictest	3793176	-21	ssh	11:20:23	0
3649339	99	57	18,18	cyclictest	3662056	-21	sendmail	08:00:01	0
3649339	99	57	16,23	cyclictest	3668820	-21	aten_r7power_vo	08:25:16	0
3649339	99	57	15,21	cyclictest	3806154	-21	ssh	11:35:28	0
3649339	99	57	15,21	cyclictest	1068	-21	snmpd	12:25:02	0
3649339	99	56	30,9	cyclictest	3676736	-21	/usr/sbin/munin	08:55:20	0
3649339	99	56	19,14	cyclictest	3673039	-21	ntpq	08:40:27	0
3649339	99	56	18,14	cyclictest	3665322	-21	ntpq	08:10:27	0
3649340	99	55	48,5	cyclictest	3654434	-21	kworker/u4:15+events_unbound	07:29:25	1
3649339	99	55	39,12	cyclictest	3675626	-21	ntpq	08:50:27	0
3649339	99	55	22,21	cyclictest	255	50	irq/23-42890000.ethernet	07:15:11	0
3649339	99	55	21,9	cyclictest	3671758	-21	ntpq	08:35:26	0
3649339	99	55	19,11	cyclictest	3650027	-21	ntpq	07:10:24	0
3649340	99	54	47,5	cyclictest	3739071	-21	kworker/u4:8+events_unbound	10:40:01	1
3649339	99	54	37,12	cyclictest	3670537	-21	ntpq	08:30:26	0
3649339	99	54	36,14	cyclictest	3771079	-21	apt-get	10:55:15	0
3649339	99	54	21,18	cyclictest	3678306	-21	mailstats	09:00:37	0
3649339	99	54	20,10	cyclictest	3758990	-21	ntpq	10:40:28	0
3649339	99	54	18,10	cyclictest	3690820	-21	ssh	09:21:28	0
3649339	99	54	15,14	cyclictest	1068	-21	snmpd	09:43:54	0
3649340	99	53	45,6	cyclictest	3790751	-21	kworker/u4:11+events_unbound	11:18:01	1
3649340	99	53	45,6	cyclictest	3668508	-21	kworker/u4:4+events_unbound	09:14:59	1
3649339	99	53	40,11	cyclictest	3652752	-21	ntpq	07:20:27	0
3649339	99	53	36,13	cyclictest	3654017	-21	ntpq	07:25:32	0
3649340	99	52	44,6	cyclictest	3836344	-21	/usr/sbin/munin	12:10:22	1
3649340	99	52	44,6	cyclictest	3648077	-21	kworker/u4:3+events_unbound	07:54:27	1
3649340	99	52	41,5	cyclictest	3859403	-21	kworker/u4:0+events_unbound	12:39:00	1
3649340	99	52	34,7	cyclictest	3668506	-21	kworker/u4:1+events_unbound	08:22:59	1
3649340	99	52	34,6	cyclictest	3713374	-21	kworker/u4:2+events_unbound	10:04:20	1
3649339	99	52	35,11	cyclictest	3729112	-21	ssh	10:05:35	0
3649339	99	52	20,10	cyclictest	3657684	-21	memory	07:40:23	0
3649339	99	52	18,10	cyclictest	3664116	-21	ntpq	08:05:31	0
3649340	99	51	44,5	cyclictest	3824552	-21	kworker/u4:5+events_unbound	12:24:59	1
3649340	99	51	43,6	cyclictest	3650544	-21	kworker/u4:6+events_unbound	07:14:22	1
3649340	99	51	36,11	cyclictest	3732172	-21	apt-get	10:10:06	1
3649340	99	50	44,4	cyclictest	3650544	-21	kworker/u4:6+events_unbound	07:15:15	1
3649340	99	50	42,6	cyclictest	3673567	-21	systemd	08:44:34	1
3649340	99	50	31,6	cyclictest	3773324	-21	kworker/u4:1+events_unbound	11:22:53	1
3649339	99	50	18,10	cyclictest	3649319	-21	munin-run	07:10:00	0
3649340	99	49	42,5	cyclictest	3827418	-21	cut	12:00:18	1
3649340	99	49	42,5	cyclictest	3827418	-21	cut	12:00:18	1
3649340	99	49	39,8	cyclictest	3747197	-21	ssh	10:26:28	1
3649340	99	49	31,6	cyclictest	3837705	-21	kworker/u4:11+events_unbound	12:16:13	1
3649340	99	48	39,7	cyclictest	3810035	-21	diskstats	11:40:16	1
3649340	99	48	39,7	cyclictest	3714974	-21	latency_hist	09:50:01	1
3649340	99	47	40,5	cyclictest	3720393	-21	ntpq	09:55:31	1
3649340	99	47	40,5	cyclictest	3648074	-21	kworker/u4:0+events_unbound	07:24:23	1
3649340	99	47	39,6	cyclictest	3678619	-21	systemd	09:04:38	1
3649340	99	47	38,7	cyclictest	3698203	-21	gltestperf	09:30:20	1
3649340	99	47	31,5	cyclictest	3777705	-21	kworker/u4:7+events_unbound	11:31:12	1
3649340	99	46	39,5	cyclictest	3807557	-21	kworker/u4:4+events_unbound	11:57:00	1
3649340	99	46	38,3	cyclictest	0	-21	swapper/1	08:00:33	1
3649340	99	46	36,7	cyclictest	0	-21	swapper/1	10:05:12	1
3649340	99	45	38,5	cyclictest	3677328	-21	kworker/u4:10+events_unbound	09:06:44	1
3649340	99	45	36,7	cyclictest	3662004	-21	systemd	07:59:28	1
3649340	99	43	32,7	cyclictest	3664532	-21	kworker/u4:5-rpciod	08:20:01	1
3649340	99	43	15,15	cyclictest	0	-21	swapper/1	08:55:12	1
3649340	99	42	28,5	cyclictest	3743467	-21	kworker/u4:3+events_unbound	10:42:21	1
3649340	99	41	32,7	cyclictest	3656232	-21	fschecks_time	07:35:23	1
3649340	99	40	31,6	cyclictest	0	-21	swapper/1	08:30:34	1
3649340	99	40	28,9	cyclictest	0	-21	swapper/1	07:40:17	1
3649340	99	37	29,5	cyclictest	0	-21	swapper/1	07:10:00	1
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17