OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Sun Aug 30, 2026 12:43:33)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
1522415	2	54	19,30	sleep7	0	-21	swapper/7	07:06:38	7
1522496	2	42	31,8	sleep4	0	-21	swapper/4	07:07:47	4
1522365	2	41	31,7	sleep5	0	-21	swapper/5	07:05:55	5
1522922	99	37	35,2	cyclictest	1908593	-21	kworker/u16:5+events_unbound	10:14:47	4
1522922	99	36	34,2	cyclictest	1764648	-21	kworker/u16:2+events_unbound	09:47:59	4
1522922	99	36	34,2	cyclictest	1483734	-21	kworker/u16:0+flush-8:0	07:59:44	4
1522922	99	36	33,2	cyclictest	1871356	-21	kworker/u16:0+events_unbound	10:11:00	4
1522922	99	35	33,2	cyclictest	2229227	-21	kworker/u16:1+events_unbound	12:34:45	4
1522922	99	35	33,2	cyclictest	2139818	-21	kworker/u16:3+events_unbound	12:08:47	4
1522922	99	35	33,2	cyclictest	2091542	-21	kworker/u16:1+events_unbound	11:53:07	4
1522922	99	35	33,2	cyclictest	2091542	-21	kworker/u16:1+events_unbound	11:47:35	4
1522922	99	35	33,2	cyclictest	2007123	-21	kworker/u16:4+flush-8:0	11:00:22	4
1522922	99	35	33,2	cyclictest	1956082	-21	kworker/u16:2+events_unbound	10:27:35	4
1522922	99	35	33,2	cyclictest	1883737	-21	kworker/u16:1+events_unbound	10:08:59	4
1522922	99	34	32,2	cyclictest	2228690	-21	kworker/u16:0+events_unbound	12:22:55	4
1522922	99	34	32,2	cyclictest	2139818	-21	kworker/u16:3+flush-8:0	11:44:44	4
1522922	99	34	32,2	cyclictest	2043031	-21	kworker/u16:2+events_unbound	11:26:19	4
1522922	99	34	32,2	cyclictest	1832605	-21	kworker/u16:3+events_unbound	10:04:00	4
1522922	99	34	32,2	cyclictest	1764648	-21	kworker/u16:2+events_unbound	09:54:00	4
1522922	99	34	31,2	cyclictest	2179985	-21	kworker/u16:4+events_unbound	12:24:47	4
1522922	99	33	30,2	cyclictest	1760696	-21	kworker/u16:1+events_unbound	09:10:06	4
1522935	99	32	26,5	cyclictest	648	-21	systemd-journal	09:24:44	7
1522922	99	32	30,2	cyclictest	2179985	-21	kworker/u16:4+events_unbound	12:19:38	4
1522922	99	32	30,2	cyclictest	2139818	-21	kworker/u16:3+events_unbound	12:12:59	4
1522922	99	32	30,2	cyclictest	2091542	-21	kworker/u16:1+events_unbound	11:57:54	4
1522922	99	32	30,2	cyclictest	2091542	-21	kworker/u16:1+events_unbound	11:57:54	4
1522922	99	32	30,2	cyclictest	2091542	-21	kworker/u16:1+events_unbound	11:24:35	4
1522922	99	32	30,2	cyclictest	1764648	-21	kworker/u16:2+events_unbound	09:42:23	4
1522922	99	32	30,2	cyclictest	1764648	-21	kworker/u16:2+events_unbound	09:29:23	4
1522922	99	32	30,2	cyclictest	1764648	-21	kworker/u16:2+events_unbound	09:22:19	4
1522498	2	32	22,7	sleep6	0	-21	swapper/6	07:07:49	6
1522922	99	31	29,2	cyclictest	1764648	-21	kworker/u16:2+flush-8:0	09:34:03	4
1522922	99	31	29,2	cyclictest	1764648	-21	kworker/u16:2+events_unbound	09:55:28	4
1522922	99	31	29,2	cyclictest	1764648	-21	kworker/u16:2+events_unbound	09:55:28	4
1522935	99	30	25,4	cyclictest	2266259	-21	latency_hist	12:34:44	7
1522935	99	30	25,4	cyclictest	1809641	-21	latency_hist	09:29:45	7
1522935	99	30	24,5	cyclictest	1555845	-21	latency_hist	07:24:45	7
1522922	99	30	28,2	cyclictest	2254335	-21	kworker/u16:5+events_unbound	12:31:27	4
1522922	99	30	28,2	cyclictest	2007123	-21	kworker/u16:4+events_unbound	11:08:59	4
1522935	99	29	24,4	cyclictest	2081285	-21	latency_hist	11:19:44	7
1522935	99	29	24,4	cyclictest	2081285	-21	latency_hist	11:19:44	7
1522935	99	29	0,27	cyclictest	2229214	-21	latency_hist	12:19:44	7
1522922	99	29	27,2	cyclictest	2116885	-21	kworker/u16:0+events_unbound	12:04:19	4
1522922	99	29	27,2	cyclictest	2116885	-21	kworker/u16:0+events_unbound	11:35:51	4
1522922	99	29	27,2	cyclictest	2068951	-21	kworker/u16:0+events_unbound	11:16:50	4
1522922	99	29	27,2	cyclictest	2068951	-21	kworker/u16:0+events_unbound	11:16:50	4
1522922	99	29	27,2	cyclictest	1702107	-21	kworker/u16:0+events_unbound	09:16:31	4
1522922	99	28	26,2	cyclictest	1883737	-21	kworker/u16:1+events_unbound	10:49:28	4
1522922	99	28	26,2	cyclictest	1883737	-21	kworker/u16:1+events_unbound	10:43:44	4
1522922	99	28	26,2	cyclictest	1871356	-21	kworker/u16:0+events_unbound	10:37:19	4
1522922	99	28	25,2	cyclictest	1575651	-21	kworker/u16:3+events_unbound	08:14:55	4
1522935	99	27	22,4	cyclictest	1896067	-21	latency_hist	10:04:44	7
1522922	99	27	25,2	cyclictest	2104194	-21	kworker/u16:3+events_unbound	11:34:00	4
1522922	99	27	25,2	cyclictest	1653237	-21	kworker/u16:4+events_unbound	08:20:55	4
1956775	2	26	0,1	chrt	1956773	-21	ssh	10:27:57	7
1522935	99	26	9,16	cyclictest	2152058	-21	ssh	11:45:07	7
1522922	99	26	24,2	cyclictest	2007123	-21	kworker/u16:4+events_unbound	11:10:58	4
1522922	99	26	24,2	cyclictest	1883737	-21	kworker/u16:1+events_unbound	10:54:00	4
1522922	99	26	24,2	cyclictest	1653237	-21	kworker/u16:4+events_unbound	09:08:54	4
1522922	99	25	23,2	cyclictest	1883737	-21	kworker/u16:1+flush-8:0	10:20:38	4
1522922	99	25	23,2	cyclictest	1883737	-21	kworker/u16:1+events_unbound	10:31:47	4
1522922	99	25	23,2	cyclictest	1653237	-21	kworker/u16:4+events_unbound	08:32:56	4
1522922	99	25	23,2	cyclictest	1653237	-21	kworker/u16:4+events_unbound	08:32:56	4
1522922	99	25	23,2	cyclictest	1624356	-21	kworker/u16:1+events_unbound	08:47:55	4
1522922	99	24	22,2	cyclictest	2007123	-21	kworker/u16:4+events_unbound	10:56:07	4
1522935	99	22	20,1	cyclictest	2192229	-21	ssh	12:04:44	7
1522922	99	22	20,2	cyclictest	1832605	-21	kworker/u16:3+flush-8:0	09:37:58	4
1522922	99	22	20,2	cyclictest	1653237	-21	kworker/u16:4+flush-8:0	08:40:07	4
1522922	99	22	20,2	cyclictest	1653237	-21	kworker/u16:4+events_unbound	08:25:36	4
1522922	99	22	20,2	cyclictest	1532523	-21	kworker/u16:2+flush-8:0	07:19:45	4
1522922	99	22	20,2	cyclictest	1483734	-21	kworker/u16:0+events_unbound	07:51:54	4
1522922	99	21	19,2	cyclictest	1575651	-21	kworker/u16:3+events_unbound	08:05:44	4
1522922	99	21	19,2	cyclictest	1483734	-21	kworker/u16:0+events_unbound	07:46:55	4
1522922	99	21	19,2	cyclictest	1483734	-21	kworker/u16:0+events_unbound	07:46:55	4
2167974	2	20	0,1	chrt	2167964	-21	ssh	11:54:44	7
2167974	2	20	0,1	chrt	2167964	-21	ssh	11:54:44	7
1522922	99	20	18,2	cyclictest	1653237	-21	kworker/u16:4+events_unbound	08:55:54	4
1522922	99	20	18,2	cyclictest	1575651	-21	kworker/u16:3+events_unbound	08:35:54	4
1522922	99	20	18,2	cyclictest	1493718	-21	kworker/u16:4+events_unbound	07:10:55	4
1522935	99	19	0,17	cyclictest	5701	-21	gnome-shell	09:30:54	7
1522922	99	19	17,2	cyclictest	1532523	-21	kworker/u16:2+events_unbound	08:10:08	4
1522935	99	18	0,16	cyclictest	0	-21	swapper/7	11:25:08	7
1522935	99	17	0,15	cyclictest	1277	-21	snmpd	07:13:16	7
1522922	99	17	15,2	cyclictest	1532523	-21	kworker/u16:2+events_unbound	07:34:07	4
1522922	99	17	14,2	cyclictest	1702107	-21	kworker/u16:0+flush-8:0	09:00:01	4
1522935	99	16	13,2	cyclictest	0	-21	swapper/7	09:44:45	7
1522935	99	16	11,4	cyclictest	648	-21	systemd-journal	08:14:40	7
1522931	99	16	0,1	cyclictest	0	-21	swapper/6	12:00:43	6
1522931	99	16	0,15	cyclictest	1603564	-21	ntpq	07:45:10	6
1522931	99	16	0,15	cyclictest	1603564	-21	ntpq	07:45:10	6
1522926	99	16	15,1	cyclictest	1832605	-21	kworker/u16:3+events_unbound	10:02:46	5
1522922	99	16	14,2	cyclictest	1483734	-21	kworker/u16:0+flush-8:0	07:35:03	4
1522922	99	16	14,2	cyclictest	1483734	-21	kworker/u16:0+events_unbound	07:27:55	4
1522935	99	15	10,4	cyclictest	648	-21	systemd-journal	10:59:39	7
1522935	99	15	0,2	cyclictest	2250683	-21	ssh	12:25:09	7
1522926	99	15	0,14	cyclictest	0	-21	swapper/5	09:54:47	5
1522926	99	15	0,14	cyclictest	0	-21	swapper/5	09:54:47	5
1522922	99	15	13,2	cyclictest	1532523	-21	kworker/u16:2+events_unbound	07:40:55	4
1522935	99	14	7,6	cyclictest	1970120	-21	turbostat	10:34:44	7
1522935	99	14	6,7	cyclictest	5907	-21	gmain	08:35:10	7
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17