OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #e, slot #6

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rackeslot6.osadl.org (updated Mon Jun 16, 2025 12:44:56)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
4109654	99	121	107,11	cyclictest	4170084	-21	apt-get	10:15:14	0
4109654	99	115	105,8	cyclictest	22864	-21	latency_hist	12:40:01	0
4109656	99	114	101,10	cyclictest	4144953	-21	dpkg	08:56:04	1
4109654	99	113	101,9	cyclictest	502	-21	rs:main		0
4109656	99	112	101,9	cyclictest	4177785	-21	apt-get	10:40:00	1
4109654	99	112	98,10	cyclictest	4174865	-21	apt-get	10:30:15	0
4109654	99	101	90,8	cyclictest	4132120	-21	latency_hist	08:20:00	0
4109654	99	100	86,10	cyclictest	3500	-21	apt-get	11:40:16	0
4109654	99	99	86,10	cyclictest	16750	-21	apt-get	12:20:13	0
4109656	99	97	85,8	cyclictest	4109506	-21	kworker/u8:1+events_unbound	07:15:01	1
4109656	99	93	83,7	cyclictest	4160290	-21	apt-get	09:45:11	1
4109656	99	93	80,10	cyclictest	4158703	-21	apt-get	09:40:13	1
4109656	99	90	82,6	cyclictest	8268	-21	apt-get	11:54:59	1
4109656	99	88	77,9	cyclictest	4171694	-21	apt-get	10:20:15	1
4109656	99	86	74,8	cyclictest	4138740	-21	kworker/u8:2+events_unbound	09:05:01	1
4109656	99	86	74,8	cyclictest	4138740	-21	kworker/u8:2+events_unbound	09:05:01	1
4109654	99	86	76,7	cyclictest	4177800	-21	latency_hist	10:40:00	0
4109656	99	85	75,7	cyclictest	3105085	-21	systemd-journal	10:15:00	1
4109656	99	85	75,7	cyclictest	3105085	-21	systemd-journal	10:15:00	1
4109654	99	85	75,7	cyclictest	4109576	-21	kworker/u8:3+events_unbound	07:15:03	0
4109656	99	84	74,7	cyclictest	3105085	-21	systemd-journal	11:40:00	1
4109656	99	84	69,12	cyclictest	4166916	-21	apt-get	10:05:13	1
4109654	99	84	74,6	cyclictest	4153704	-21	kworker/u8:2+flush-179:0	09:40:15	0
4109654	99	84	73,8	cyclictest	4133958	-21	/usr/sbin/munin	08:25:24	0
4109656	99	82	77,4	cyclictest	4109576	-21	kworker/u8:3+events_unbound	07:40:14	1
4109656	99	82	74,5	cyclictest	4135563	-21	apt-get	08:30:13	1
4109656	99	82	71,8	cyclictest	3255707	-21	kworker/u8:0+events_unbound	07:20:17	1
4109656	99	81	72,6	cyclictest	4171450	-21	kworker/u8:1+flush-179:0	10:25:19	1
4109656	99	81	63,15	cyclictest	3105085	-21	systemd-journal	08:21:34	1
4109654	99	81	69,9	cyclictest	4186267	-21	apt-get	11:05:13	0
4109656	99	80	67,10	cyclictest	4164800	-21	latency_hist	10:00:01	1
4109654	99	80	67,10	cyclictest	4127607	-21	apt-get	08:05:13	0
4109656	99	79	69,8	cyclictest	6546	-21	store	11:50:02	1
4109656	99	79	69,7	cyclictest	4129194	-21	apt-get	08:10:14	1
4109654	99	78	67,8	cyclictest	10141	-21	latency_hist	12:00:01	0
4109654	99	78	1,73	cyclictest	3276	-21	true	11:40:00	0
4109654	99	77	68,6	cyclictest	4116214	-21	kworker/u8:1+flush-179:0	07:30:15	0
4109656	99	76	67,7	cyclictest	4153669	-21	latency_hist	09:25:01	1
4109656	99	76	62,11	cyclictest	3105085	-21	systemd-journal	11:58:35	1
4109654	99	76	69,5	cyclictest	8392	-21	kworker/u8:2+events_unbound	12:00:18	0
4109656	99	75	67,6	cyclictest	8392	-21	kworker/u8:2+flush-179:0	12:10:25	1
4109656	99	75	66,7	cyclictest	4143263	-21	latency_hist	08:55:01	1
4109656	99	74	66,6	cyclictest	4122605	-21	kworker/u8:2+events_unbound	08:00:15	1
4109654	99	74	66,5	cyclictest	4148663	-21	kworker/u8:3+flush-179:0	09:20:01	0
4109654	99	72	67,3	cyclictest	4176191	-21	kworker/u8:4+flush-179:0	10:55:23	0
4109656	99	71	63,6	cyclictest	3105085	-21	systemd-journal	07:59:34	1
4109656	99	71	61,8	cyclictest	4166659	-21	latency_hist	10:05:01	1
4109654	99	71	45,23	cyclictest	4168243	-21	latency_hist	10:10:01	0
4109654	99	70	57,10	cyclictest	4181486	-21	apt-get	10:50:15	0
4109660	99	69	41,26	cyclictest	0	-21	swapper/3	09:11:59	3
4109656	99	69	61,6	cyclictest	4187629	-21	kworker/u8:1+flush-179:0	11:35:00	1
4109654	99	69	61,5	cyclictest	4138509	-21	latency_hist	08:40:01	0
4109654	99	69	61,5	cyclictest	4138509	-21	latency_hist	08:40:01	0
4109654	99	69	56,10	cyclictest	4148884	-21	apt-get	09:10:15	0
4109656	99	68	59,6	cyclictest	4179904	-21	apt-get	10:45:14	1
4109654	99	68	61,4	cyclictest	4193999	-21	kworker/u8:3+events_unbound	11:45:20	0
4109654	99	68	58,8	cyclictest	4187645	-21	kworker/u8:2+flush-179:0	11:15:19	0
4109654	99	67	62,4	cyclictest	4133699	-21	kworker/u8:0+flush-179:0	08:40:18	0
4109654	99	67	44,21	cyclictest	4165141	-21	store	10:00:01	0
4109654	99	66	59,5	cyclictest	4120976	-21	kworker/u8:1+flush-179:0	07:55:25	0
4109656	99	65	56,7	cyclictest	4122605	-21	kworker/u8:2+flush-179:0	08:05:18	1
4109654	99	65	54,9	cyclictest	4191048	-21	apt-get	11:20:16	0
4109654	99	65	54,9	cyclictest	4191048	-21	apt-get	11:20:15	0
4109654	99	64	55,7	cyclictest	3105085	-21	systemd-journal	09:39:59	0
4109656	99	63	57,5	cyclictest	4138740	-21	kworker/u8:2+flush-179:0	09:15:00	1
4109656	99	63	55,5	cyclictest	4187645	-21	kworker/u8:2+flush-179:0	11:25:25	1
4109654	99	63	58,4	cyclictest	4148663	-21	kworker/u8:3+flush-179:0	09:20:27	0
4109654	99	63	55,5	cyclictest	4148663	-21	kworker/u8:3+events_unbound	09:30:21	0
4109654	99	63	55,5	cyclictest	4133699	-21	kworker/u8:0+flush-179:0	08:30:15	0
4109654	99	63	51,8	cyclictest	4176191	-21	kworker/u8:4+flush-179:0	11:10:25	0
4109659	99	61	57,2	cyclictest	0	-21	swapper/2	08:55:59	2
4109656	99	61	56,4	cyclictest	4176191	-21	kworker/u8:4+flush-179:0	11:05:27	1
4109656	99	61	52,6	cyclictest	4109576	-21	kworker/u8:3+events_unbound	07:35:14	1
4109656	99	61	52,6	cyclictest	18338	-21	apt-get	12:25:14	1
4109654	99	61	57,3	cyclictest	4109576	-21	kworker/u8:3+events_unbound	07:45:25	0
4109654	99	61	53,5	cyclictest	4169351	-21	kworker/u8:2+flush-179:0	10:50:01	0
4109654	99	61	52,6	cyclictest	3255707	-21	kworker/u8:0+flush-179:0	07:25:22	0
4109656	99	60	53,5	cyclictest	4153704	-21	kworker/u8:2+flush-179:0	09:50:20	1
4109656	99	60	52,5	cyclictest	502	-21	rs:main		1
4109654	99	60	55,4	cyclictest	4168231	-21	kworker/u8:1+flush-179:0	10:10:24	0
4109654	99	60	55,4	cyclictest	4168231	-21	kworker/u8:1+flush-179:0	10:10:23	0
4109654	99	60	53,5	cyclictest	4109576	-21	kworker/u8:3+flush-179:0	07:38:29	0
4109654	99	60	52,6	cyclictest	4147588	-21	diskstats	09:05:20	0
4109654	99	60	51,6	cyclictest	4187645	-21	kworker/u8:2+events_unbound	11:25:17	0
4109656	99	59	50,6	cyclictest	4109576	-21	kworker/u8:3+events_unbound	07:45:22	1
4109656	99	59	40,17	cyclictest	4155290	-21	kworker/u8:4+flush-179:0	10:19:59	1
4109654	99	59	53,5	cyclictest	4128954	-21	kworker/u8:3+flush-179:0	08:20:19	0
4109654	99	59	51,5	cyclictest	4187629	-21	kworker/u8:1+events_unbound	11:30:29	0
4109656	99	58	48,8	cyclictest	3105085	-21	systemd-journal	09:30:01	1
4109654	99	58	48,7	cyclictest	8261	-21	latency_hist	11:55:01	0
4109656	99	57	47,7	cyclictest	4147301	-21	apt-get	09:05:16	1
4109654	99	57	48,7	cyclictest	502	-21	rs:main		0
4109654	99	57	46,9	cyclictest	4146796	-21	apt-get	09:05:01	0
4109654	99	57	46,9	cyclictest	4146796	-21	apt-get	09:05:01	0
4109656	99	56	48,6	cyclictest	8392	-21	kworker/u8:2+events_unbound	12:05:01	1
4109656	99	56	47,6	cyclictest	4187645	-21	kworker/u8:2+flush-179:0	11:15:16	1
4109656	99	56	46,7	cyclictest	4135307	-21	latency_hist	08:30:01	1
4109654	99	56	50,4	cyclictest	4153704	-21	kworker/u8:2+flush-179:0	09:55:27	0
4109656	99	55	48,5	cyclictest	4133699	-21	kworker/u8:0+flush-179:0	08:45:23	1
4109654	99	55	47,6	cyclictest	4120976	-21	kworker/u8:1+flush-179:0	08:05:00	0
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2020-10-04 - 12:21