OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #7, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system

Characteristics of the 792 highest latencies:

System rack7slot2.osadl.org (updated Sun Jan 18, 2026 12:44:14)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
3870746	99	113	96,9	cyclictest	0	-21	swapper/0	09:35:25	0
3870747	99	105	96,6	cyclictest	4072026	-21	kworker/u4:0+events_unbound	12:35:38	1
3870747	99	94	85,7	cyclictest	3909377	-21	kworker/u4:4+events_unbound	09:25:00	1
3870747	99	94	85,7	cyclictest	3909377	-21	kworker/u4:4+events_unbound	09:24:59	1
3870747	99	93	84,6	cyclictest	3986765	-21	kworker/u4:4+events_unbound	10:55:19	1
3870747	99	91	71,8	cyclictest	4025100	-21	kworker/u4:19+events_unbound	11:40:00	1
3870747	99	85	76,6	cyclictest	4038047	-21	kworker/u4:2+events_unbound	12:20:34	1
3870747	99	83	73,7	cyclictest	3995421	-21	kworker/u4:1+events_unbound	11:30:24	1
3870747	99	80	73,5	cyclictest	3956895	-21	kworker/u4:4+events_unbound	10:20:10	1
3870747	99	80	73,5	cyclictest	3956895	-21	kworker/u4:4+events_unbound	10:20:10	1
3870747	99	78	63,5	cyclictest	3922613	-21	kworker/u4:0+events_unbound	09:40:02	1
3870746	99	77	68,6	cyclictest	3980107	-21	apt-get	10:45:09	0
3870746	99	76	56,13	cyclictest	255	50	irq/23-42890000.ethernet	07:50:22	0
3870747	99	75	64,9	cyclictest	4042389	-21	kworker/u4:9+events_unbound	12:25:34	1
3870746	99	72	34,29	cyclictest	255	50	irq/23-42890000.ethernet	07:45:13	0
3870746	99	67	44,19	cyclictest	3932982	-21	apt-get	09:50:09	0
3870747	99	66	58,6	cyclictest	3883893	-21	kworker/u4:4+events_unbound	08:30:01	1
3870746	99	66	45,8	cyclictest	0	-21	swapper/0	08:45:09	0
3870746	99	66	15,26	cyclictest	4002125	-21	fschecks_time	11:10:21	0
3870747	99	64	45,7	cyclictest	4029409	-21	kworker/u4:1+events_unbound	11:45:01	1
3870746	99	64	50,11	cyclictest	54	-21	kswapd0	10:05:09	0
3870747	99	63	56,5	cyclictest	3895625	-21	kworker/u4:4+events_unbound	08:59:51	1
3870747	99	63	55,6	cyclictest	3974321	-21	kworker/u4:3+events_unbound	12:00:31	1
3870747	99	63	55,6	cyclictest	3974321	-21	kworker/u4:3+events_unbound	12:00:31	1
3870747	99	63	54,6	cyclictest	3952563	-21	kworker/u4:0+events_unbound	10:45:00	1
3870747	99	63	54,6	cyclictest	3952563	-21	kworker/u4:0+events_unbound	10:45:00	1
3870747	99	62	54,5	cyclictest	3997813	-21	ssh	11:05:19	1
3870747	99	62	4,7	cyclictest	17	1	rcu_preempt	10:30:27	1
3870746	99	62	14,27	cyclictest	4010000	-21	awk	11:20:00	0
3870747	99	61	55,5	cyclictest	3986765	-21	kworker/u4:4+events_unbound	11:00:19	1
3870747	99	61	55,4	cyclictest	3883895	-21	kworker/u4:6+events_unbound	08:19:40	1
3870747	99	60	55,3	cyclictest	0	-21	swapper/1	09:10:22	1
3870747	99	60	43,5	cyclictest	3917550	-21	kworker/u4:2+events_unbound	09:50:13	1
3870747	99	60	36,8	cyclictest	4025100	-21	kworker/u4:19+events_unbound	11:51:58	1
3870746	99	60	43,15	cyclictest	3970566	-21	sh	10:33:27	0
3870747	99	59	51,6	cyclictest	3995421	-21	kworker/u4:1+events_unbound	11:20:22	1
3870746	99	59	19,14	cyclictest	4045015	-21	ntpq	12:00:25	0
3870746	99	59	19,14	cyclictest	4045015	-21	ntpq	12:00:25	0
3870746	99	59	13,24	cyclictest	4028008	-21	ntpq	11:40:27	0
3870747	99	58	51,5	cyclictest	3948237	-21	kworker/u4:3+events_unbound	10:25:11	1
3870746	99	58	48,7	cyclictest	3913792	-21	kworker/u4:1+events_unbound	09:30:00	0
3870746	99	58	48,7	cyclictest	3913792	-21	kworker/u4:1+events_unbound	09:30:00	0
3870746	99	58	21,9	cyclictest	3991828	-21	ssh	10:58:15	0
3870747	99	57	50,5	cyclictest	3999753	-21	kworker/u4:0+events_unbound	11:25:23	1
3870747	99	57	50,5	cyclictest	3999753	-21	kworker/u4:0+events_unbound	11:25:23	1
3870746	99	57	42,12	cyclictest	3959896	-21	sh	10:20:34	0
3870746	99	57	42,12	cyclictest	3959896	-21	sh	10:20:34	0
3870746	99	55	48,5	cyclictest	4057385	-21	/usr/sbin/munin	12:15:20	0
3870746	99	55	47,5	cyclictest	0	-21	swapper/0	09:30:31	0
3870746	99	55	45,7	cyclictest	3891478	-21	systemd	08:39:46	0
3870746	99	55	35,17	cyclictest	3878280	-21	cron	07:40:00	0
3870746	99	55	33,19	cyclictest	3965005	-21	ssh	10:26:21	0
3870746	99	55	17,16	cyclictest	3972546	-21	ntpq	10:35:28	0
3870746	99	54	28,20	cyclictest	3998779	-21	ssh	11:05:40	0
3870746	99	54	18,10	cyclictest	3896488	-21	ntpq	09:00:31	0
3870746	99	54	13,15	cyclictest	3931170	-21	ssh	09:47:06	0
3870747	99	53	47,4	cyclictest	3974323	-21	kworker/u4:12+events_unbound	10:40:01	1
3870747	99	53	46,5	cyclictest	3911951	-21	fschecks_count	09:25:20	1
3870747	99	53	46,5	cyclictest	3911951	-21	fschecks_count	09:25:20	1
3870747	99	53	46,5	cyclictest	3870674	-21	kworker/u4:1+events_unbound	07:15:01	1
3870746	99	53	5,36	cyclictest	0	-21	swapper/0	08:55:10	0
3870746	99	53	5,36	cyclictest	0	-21	swapper/0	08:55:10	0
3870746	99	53	36,15	cyclictest	3951649	-21	sed	10:10:36	0
3870746	99	53	17,12	cyclictest	4066381	-21	ntpq	12:25:28	0
3870746	99	53	17,10	cyclictest	3872799	-21	ntpq	07:15:32	0
3870746	99	53	14,15	cyclictest	3888512	-21	ntpq	08:25:27	0
3870746	99	53	11,19	cyclictest	4022736	-21	munin-run	11:35:00	0
3870746	99	52	9,20	cyclictest	3903205	-21	ssh	09:15:19	0
3870746	99	52	31,18	cyclictest	3924617	-21	apt-get	09:40:13	0
3870746	99	52	2,11	cyclictest	0	-21	swapper/0	08:55:17	0
3870746	99	52	13,14	cyclictest	4017326	-21	ssh	11:28:06	0
3870746	99	52	13,14	cyclictest	4017326	-21	ssh	11:28:06	0
3870746	99	52	11,19	cyclictest	1055	-21	sshd	09:55:06	0
3870747	99	51	44,5	cyclictest	3927036	-21	kworker/u4:9+events_unbound	09:45:03	1
3870747	99	51	43,6	cyclictest	4038047	-21	kworker/u4:2+events_unbound	12:19:00	1
3870747	99	51	43,6	cyclictest	3955047	-21	fschecks_count	10:15:20	1
3870747	99	51	41,7	cyclictest	0	-21	swapper/1	09:15:24	1
3870746	99	51	7,22	cyclictest	4053483	-21	/usr/sbin/munin	12:10:23	0
3870746	99	51	31,14	cyclictest	4031535	-21	apt-get	11:45:11	0
3870746	99	51	20,20	cyclictest	3985637	-21	mailstats	10:50:35	0
3870746	99	51	13,10	cyclictest	4050432	-21	ssh	12:06:31	0
3870747	99	50	43,6	cyclictest	4025100	-21	kworker/u4:19+events_unbound	11:45:03	1
3870747	99	50	43,5	cyclictest	3961280	-21	kworker/u4:9+events_unbound	10:45:14	1
3870746	99	50	41,5	cyclictest	4026972	-21	modprobe	11:39:59	0
3870746	99	50	14,14	cyclictest	3957342	-21	ssh	10:17:52	0
3870746	99	50	14,12	cyclictest	3977753	-21	ssh	10:41:11	0
3870746	99	50	14,12	cyclictest	3977753	-21	ssh	10:41:11	0
3870746	99	50	14,10	cyclictest	4073154	-21	ssh	12:33:52	0
3870746	99	50	14,10	cyclictest	3892231	-21	ntpq	08:40:23	0
3870746	99	50	13,13	cyclictest	3908779	-21	ssh	09:21:00	0
3870746	99	50	13,13	cyclictest	3908779	-21	ssh	09:21:00	0
3870747	99	49	42,5	cyclictest	4053069	-21	df_abs	12:10:17	1
3870747	99	49	40,7	cyclictest	3999756	-21	kworker/u4:7+events_unbound	11:12:00	1
3870747	99	49	39,7	cyclictest	4070198	-21	/usr/sbin/munin	12:30:17	1
3870746	99	49	44,3	cyclictest	3885153	-21	kworker/u4:12+events_unbound	09:09:51	0
3870746	99	49	29,17	cyclictest	0	-21	swapper/0	09:10:29	0
3870746	99	49	17,10	cyclictest	4074989	-21	ntpq	12:35:33	0
3870746	99	49	15,10	cyclictest	4042803	-21	ssh	11:57:59	0
3870747	99	48	42,4	cyclictest	3937823	-21	df_abs	09:55:17	1
3870747	99	48	41,5	cyclictest	3916716	-21	ld	09:30:30	1
3870747	99	48	41,5	cyclictest	3875419	-21	sendmail_mailqu	07:25:34	1
3870747	99	48	39,7	cyclictest	4033732	-21	kworker/u4:7+events_unbound	11:55:33	1
3870747	99	48	38,7	cyclictest	0	-21	swapper/1	10:10:33	1
3870747	99	48	31,13	cyclictest	4048937	-21	fschecks_count	12:05:21	1
3870746	99	48	39,7	cyclictest	4061512	-21	aten_r7power_en	12:20:15	0
3870746	99	48	32,11	cyclictest	3885967	-21	ntpq	08:15:29	0
3870746	99	48	12,10	cyclictest	4037568	-21	ssh	11:51:24	0
3870746	99	48	12,10	cyclictest	3944097	-21	ssh	10:02:09	0
3870746	99	48	11,10	cyclictest	3874032	-21	ntpq	07:20:29	0
3870746	99	48	11,10	cyclictest	3874032	-21	ntpq	07:20:28	0
3870747	99	47	39,6	cyclictest	3887861	-21	latency_hist	08:25:01	1
3870747	99	47	38,7	cyclictest	314	-21	systemd-journal	10:03:43	1
3870746	99	47	29,14	cyclictest	3878489	-21	apt-get	07:40:10	0
3870747	99	46	40,4	cyclictest	3898293	-21	munin-run	09:10:01	1
3870747	99	46	40,4	cyclictest	3892872	-21	cat	08:45:01	1
3870747	99	46	38,5	cyclictest	3884290	-21	df_abs	08:10:16	1
3870746	99	46	9,11	cyclictest	1068	-21	snmpd	08:20:38	0
3870746	99	46	38,5	cyclictest	4010377	-21	/usr/sbin/munin	11:20:18	0
3870746	99	46	38,5	cyclictest	3871949	-21	kworker/u4:5-rpciod	07:30:38	0
3870746	99	46	30,10	cyclictest	3883389	-21	ntpq	08:05:25	0
3870747	99	45	36,6	cyclictest	0	-21	swapper/1	11:15:21	1
3870747	99	45	35,8	cyclictest	3988683	-21	basename	10:55:00	1
3870746	99	45	38,5	cyclictest	3872011	-21	latency_hist	07:15:01	0
3870746	99	45	36,6	cyclictest	0	-21	swapper/0	11:00:30	0
3870746	99	45	16,26	cyclictest	3881475	-21	sed	08:00:01	0
3870746	99	45	13,9	cyclictest	3875243	-21	ntpq	07:25:23	0
3870747	99	44	37,5	cyclictest	54	-21	kswapd0	10:05:10	1
3870747	99	44	36,6	cyclictest	3896145	-21	df_abs	09:00:23	1
3870747	99	44	28,6	cyclictest	3922615	-21	kworker/u4:6+events_unbound	09:44:00	1
3870747	99	43	34,7	cyclictest	3890698	-21	fschecks_count	08:35:20	1
3870746	99	43	36,5	cyclictest	3889478	-21	fschecks_time	08:30:21	0
3870746	99	43	23,16	cyclictest	3882140	-21	ntpq	08:00:26	0
3870746	99	43	23,16	cyclictest	3882140	-21	ntpq	08:00:26	0
3870747	99	42	35,5	cyclictest	3874498	-21	kworker/u4:2+events_unbound	07:59:39	1
3870747	99	41	34,5	cyclictest	0	-21	swapper/1	08:45:19	1
3870747	99	41	33,5	cyclictest	0	-21	swapper/1	07:20:37	1
3870747	99	41	33,5	cyclictest	0	-21	swapper/1	07:20:36	1
3870747	99	41	32,7	cyclictest	3879955	-21	latency_hist	07:50:02	1
3870747	99	41	31,7	cyclictest	0	-21	swapper/1	07:15:17	1
3870747	99	41	12,17	cyclictest	3870722	-21	munin-run	07:10:00	1
3870746	99	41	5,16	cyclictest	0	-21	swapper/0	08:10:21	0
3870747	99	40	34,4	cyclictest	3883028	-21	df_abs	08:05:19	1
3870747	99	40	32,6	cyclictest	3878302	-21	sh	07:40:00	1
3870747	99	40	32,5	cyclictest	0	-21	swapper/1	08:55:10	1
3870747	99	40	32,5	cyclictest	0	-21	swapper/1	08:55:09	1
3870747	99	39	33,4	cyclictest	3881605	-21	grep	08:00:16	1
3870747	99	39	33,4	cyclictest	3881605	-21	grep	08:00:16	1
3870747	99	39	20,6	cyclictest	3647114	-21	kworker/u4:11+events_unbound	07:42:25	1
3870747	99	38	4,26	cyclictest	3880101	-21	apt-get	07:50:14	1
3870747	99	36	28,5	cyclictest	3871947	-21	kworker/u4:3-rpciod	07:30:35	1
3870747	99	36	25,8	cyclictest	0	-21	swapper/1	08:30:27	1
3870746	99	33	25,6	cyclictest	0	-21	swapper/0	07:10:01	0
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17