OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Sat Dec 20, 2025 00:43:31)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
3603335	2	53	20,29	sleep7	0	-21	swapper/7	19:05:56	7
3603475	2	48	37,8	sleep5	0	-21	swapper/5	19:07:55	5
3603896	99	47	0,45	cyclictest	3626909	-21	latency_hist	19:19:30	7
3603896	99	42	0,40	cyclictest	3613442	-21	latency_hist	19:14:30	7
3603343	2	41	31,7	sleep6	0	-21	swapper/6	19:06:03	6
3603359	2	39	29,7	sleep4	0	-21	swapper/4	19:06:17	4
3603896	99	37	31,5	cyclictest	4023537	-21	latency_hist	22:44:29	7
3603887	99	34	32,2	cyclictest	3985239	-21	kworker/u16:0+events_unbound	23:09:58	4
3603887	99	34	31,2	cyclictest	3985239	-21	kworker/u16:0+events_unbound	00:04:31	4
3603887	99	34	31,2	cyclictest	3743323	-21	kworker/u16:2+flush-8:0	20:39:31	4
3603887	99	31	29,2	cyclictest	4100665	-21	kworker/u16:4+flush-8:0	23:54:53	4
3603887	99	31	28,2	cyclictest	3530709	-21	kworker/u16:1+flush-8:0	19:24:40	4
3603896	99	30	24,5	cyclictest	3810705	-21	latency_hist	20:54:29	7
3603896	99	29	28,0	cyclictest	0	-21	swapper/7	20:59:47	7
3603887	99	29	27,2	cyclictest	4023536	-21	kworker/u16:1+events_unbound	23:31:39	4
3603887	99	29	27,2	cyclictest	3665911	-21	kworker/u16:0+events_unbound	19:58:40	4
3603887	99	29	27,2	cyclictest	3665911	-21	kworker/u16:0+events_unbound	19:58:40	4
3603887	99	29	26,2	cyclictest	3937940	-21	kworker/u16:4+flush-8:0	22:34:29	4
3603887	99	29	26,2	cyclictest	3782015	-21	kworker/u16:0+flush-8:0	20:49:39	4
3603887	99	29	25,3	cyclictest	3530709	-21	kworker/u16:1+events_unbound	20:29:39	4
3603887	99	28	26,2	cyclictest	4023536	-21	kworker/u16:1+flush-8:0	22:53:39	4
3603887	99	28	26,2	cyclictest	3974207	-21	kworker/u16:2+events_unbound	22:36:40	4
3603887	99	28	26,2	cyclictest	3849261	-21	kworker/u16:2+events_unbound	21:14:26	4
3603887	99	27	25,2	cyclictest	4023536	-21	kworker/u16:1+events_unbound	22:47:39	4
3603887	99	27	25,2	cyclictest	3937940	-21	kworker/u16:4+flush-8:0	22:00:59	4
3603896	99	26	20,5	cyclictest	4004194	-21	latency_hist	22:34:29	7
3603896	99	26	20,5	cyclictest	3878249	-21	latency_hist	21:29:29	7
3603887	99	25	23,2	cyclictest	3985239	-21	kworker/u16:0+events_unbound	23:00:31	4
3603887	99	25	23,2	cyclictest	3782015	-21	kworker/u16:0+events_unbound	21:34:30	4
3603887	99	25	23,2	cyclictest	3782015	-21	kworker/u16:0+events_unbound	21:15:50	4
3603887	99	25	22,2	cyclictest	3530709	-21	kworker/u16:1+events_unbound	19:59:39	4
3603896	99	24	0,22	cyclictest	650	-21	systemd-journal	00:19:24	7
3603887	99	24	22,2	cyclictest	3849261	-21	kworker/u16:2+events_unbound	21:25:40	4
3603896	99	23	22,0	cyclictest	0	-21	swapper/7	23:09:43	7
3603896	99	23	18,4	cyclictest	4168654	-21	latency_hist	23:59:29	7
3603887	99	23	21,2	cyclictest	4169047	-21	kworker/u16:3+events_unbound	00:01:39	4
3603887	99	23	21,2	cyclictest	4100665	-21	kworker/u16:4+events_unbound	23:51:39	4
3603887	99	23	21,2	cyclictest	3985239	-21	kworker/u16:0+events_unbound	23:18:47	4
3603887	99	23	21,2	cyclictest	3985239	-21	kworker/u16:0+events_unbound	23:06:40	4
3603887	99	23	21,2	cyclictest	3985239	-21	kworker/u16:0+events_unbound	22:43:39	4
3603887	99	23	21,2	cyclictest	3772115	-21	kworker/u16:3+events_unbound	21:01:40	4
3603891	99	22	20,2	cyclictest	4100665	-21	kworker/u16:4+flush-8:0	23:49:54	6
3603887	99	22	20,2	cyclictest	3772115	-21	kworker/u16:3+events_unbound	21:05:40	4
3603887	99	21	19,2	cyclictest	3530709	-21	kworker/u16:1+events_unbound	19:11:40	4
3603887	99	20	18,2	cyclictest	4169047	-21	kworker/u16:3+events_unbound	00:30:03	4
3603887	99	20	18,2	cyclictest	4100665	-21	kworker/u16:4+events_unbound	23:44:39	4
3603887	99	20	18,2	cyclictest	4100665	-21	kworker/u16:4+events_unbound	23:23:40	4
3603887	99	20	18,2	cyclictest	3917047	-21	kworker/u16:1+events_unbound	21:52:42	4
3603887	99	20	18,2	cyclictest	3782015	-21	kworker/u16:0+events_unbound	21:20:00	4
3603887	99	20	18,2	cyclictest	13946	-21	kworker/u16:1+events_unbound	00:28:39	4
3603887	99	19	16,2	cyclictest	3772115	-21	kworker/u16:3+flush-8:0	20:54:34	4
3603896	99	18	12,5	cyclictest	3752613	-21	latency_hist	20:24:30	7
3603896	99	18	0,17	cyclictest	0	-21	swapper/7	23:19:52	7
3603896	99	18	0,16	cyclictest	0	-21	swapper/7	23:49:52	7
3603896	99	18	0,16	cyclictest	0	-21	swapper/7	21:14:55	7
3603887	99	18	16,2	cyclictest	3906927	-21	kworker/u16:3+events_unbound	22:11:43	4
3603887	99	18	16,2	cyclictest	3849261	-21	kworker/u16:2+flush-8:0	21:44:45	4
3603887	99	18	16,2	cyclictest	3626949	-21	kworker/u16:2+events_unbound	19:35:44	4
3603887	99	18	0,2	cyclictest	4813	-21	diskstats	00:14:42	4
3603896	99	17	0,15	cyclictest	3897605	-21	latency_hist	21:39:29	7
3603896	99	17	0,15	cyclictest	3733303	-21	latency_hist	20:14:30	7
3603887	99	17	0,1	cyclictest	0	-21	swapper/4	23:40:06	4
3603887	99	17	0,1	cyclictest	0	-21	swapper/4	23:36:00	4
3603887	99	17	0,1	cyclictest	0	-21	swapper/4	22:59:12	4
3603887	99	17	0,1	cyclictest	0	-21	swapper/4	22:19:16	4
3603887	99	17	0,1	cyclictest	0	-21	swapper/4	22:06:59	4
3603887	99	17	0,1	cyclictest	0	-21	swapper/4	21:56:45	4
3603887	99	17	0,1	cyclictest	0	-21	swapper/4	21:44:27	4
3603887	99	17	0,1	cyclictest	0	-21	swapper/4	21:34:13	4
3603887	99	17	0,1	cyclictest	0	-21	swapper/4	20:36:52	4
3603887	99	17	0,1	cyclictest	0	-21	swapper/4	20:23:42	4
3603887	99	17	0,1	cyclictest	0	-21	swapper/4	20:18:39	4
3603887	99	17	0,1	cyclictest	0	-21	swapper/4	20:11:33	4
3603887	99	17	0,1	cyclictest	0	-21	swapper/4	19:51:04	4
3603887	99	17	0,1	cyclictest	0	-21	swapper/4	19:44:56	4
3603887	99	17	0,1	cyclictest	0	-21	swapper/4	19:42:53	4
3603887	99	17	0,1	cyclictest	0	-21	swapper/4	19:30:35	4
3603887	99	17	0,1	cyclictest	0	-21	swapper/4	19:20:21	4
3603887	99	17	0,1	cyclictest	0	-21	swapper/4	00:35:11	4
3603887	99	17	0,1	cyclictest	0	-21	swapper/4	00:19:49	4
3603887	99	17	0,1	cyclictest	0	-21	swapper/4	00:11:38	4
3603896	99	16	12,3	cyclictest	1267	-21	snmpd	19:29:23	7
3603887	99	16	0,1	cyclictest	0	-21	swapper/4	23:24:44	4
3603887	99	16	0,1	cyclictest	0	-21	swapper/4	22:27:28	4
3603887	99	16	0,1	cyclictest	0	-21	swapper/4	22:21:19	4
3603887	99	16	0,1	cyclictest	0	-21	swapper/4	22:21:19	4
3603887	99	16	0,1	cyclictest	0	-21	swapper/4	20:45:04	4
3603887	99	16	0,1	cyclictest	0	-21	swapper/4	20:24:39	4
3603887	99	16	0,1	cyclictest	0	-21	swapper/4	20:06:26	4
3603887	99	16	0,1	cyclictest	0	-21	swapper/4	19:15:14	4
3603896	99	15	0,14	cyclictest	4120466	-21	turbostat	23:34:29	7
3603896	99	15	0,13	cyclictest	1267	-21	snmpd	00:28:48	7
3603890	99	15	9,5	cyclictest	3769379	-21	grep	20:29:51	5
3603896	99	14	8,5	cyclictest	3636553	-21	latency_hist	19:24:30	7
3603896	99	14	7,6	cyclictest	3801012	-21	latency_hist	20:49:29	7
3603896	99	14	5,8	cyclictest	5944	-21	gmain	19:36:10	7
3603896	99	14	10,3	cyclictest	1267	-21	snmpd	21:54:57	7
3603896	99	14	10,3	cyclictest	1267	-21	snmpd	00:07:27	7
3603896	99	14	0,1	cyclictest	0	-21	swapper/7	20:54:55	7
3603896	99	14	0,12	cyclictest	3742975	-21	latency_hist	20:19:30	7
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17