OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Fri Jun 12, 2026 12:43:34)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
652481	2	96	0,1	sleep7	652483	-21	tr	08:21:25	7
1132608	2	56	0,0	sleep7	1132609	-21	ssh	11:50:50	7
501810	99	47	45,1	cyclictest	620634	-21	kworker/u16:2+flush-8:0	12:06:06	5
6280	89	45	0,37	rtkit-daemon	0	-21	swapper/4	07:02:42	4
501816	99	44	38,5	cyclictest	649343	-21	latency_hist	08:21:06	7
501336	2	43	5,32	sleep7	0	-21	swapper/7	07:03:37	7
501466	2	42	32,7	sleep5	0	-21	swapper/5	07:05:28	5
501816	99	38	37,0	cyclictest	0	-21	swapper/7	08:36:06	7
501810	99	37	35,1	cyclictest	879924	-21	kworker/u16:1+flush-8:0	10:16:06	5
501802	99	37	35,2	cyclictest	856249	-21	kworker/u16:3+events_unbound	10:46:34	4
501802	99	37	34,2	cyclictest	581875	-21	kworker/u16:1+flush-8:0	09:16:29	4
501810	99	35	33,2	cyclictest	879924	-21	kworker/u16:1+events_unbound	10:21:29	5
501810	99	34	28,5	cyclictest	716605	-21	kworker/u16:4+flush-8:0	09:06:32	5
501816	99	33	27,5	cyclictest	591823	-21	latency_hist	07:51:07	7
501802	99	33	31,2	cyclictest	620634	-21	kworker/u16:2+flush-8:0	11:51:36	4
501816	99	32	30,1	cyclictest	735623	-21	diskmemload	12:06:24	7
501816	99	32	26,5	cyclictest	795752	-21	latency_hist	09:31:06	7
501802	99	32	30,2	cyclictest	831350	-21	kworker/u16:4+flush-8:0	09:45:10	4
501802	99	31	30,1	cyclictest	856249	-21	kworker/u16:3+events_unbound	11:28:45	4
501802	99	31	28,2	cyclictest	831336	-21	kworker/u16:0+flush-8:0	09:56:17	4
827899	2	29	0,1	chrt	827901	-21	ssh	09:41:30	7
501816	99	29	24,4	cyclictest	988644	-21	latency_hist	10:51:06	7
501802	99	29	26,2	cyclictest	1228632	-21	kworker/u16:0+flush-8:0	12:31:17	4
976484	2	28	0,1	chrt	976485	-21	ssh	10:45:50	7
501816	99	28	22,5	cyclictest	735622	-21	latency_hist	09:06:06	7
501812	99	28	24,3	cyclictest	620634	-21	kworker/u16:2+flush-8:0	11:16:06	6
501810	99	28	26,2	cyclictest	620634	-21	kworker/u16:2+flush-8:0	09:36:29	5
501802	99	28	25,3	cyclictest	581875	-21	kworker/u16:1+events_unbound	09:11:32	4
501438	2	28	18,7	sleep6	0	-21	swapper/6	07:05:05	6
772248	2	27	0,1	chrt	0	-21	swapper/7	09:21:18	7
501816	99	27	22,4	cyclictest	639771	-21	latency_hist	08:16:06	7
501816	99	27	22,4	cyclictest	639771	-21	latency_hist	08:16:06	7
501816	99	27	22,4	cyclictest	1024659	-21	latency_hist	11:06:06	7
501816	99	26	25,0	cyclictest	0	-21	swapper/7	10:01:22	7
501816	99	26	24,1	cyclictest	881588	-21	ssh	10:06:19	7
501810	99	26	24,2	cyclictest	856249	-21	kworker/u16:3+flush-8:0	12:16:22	5
501810	99	26	24,2	cyclictest	856249	-21	kworker/u16:3+flush-8:0	12:16:22	5
501802	99	25	23,2	cyclictest	581875	-21	kworker/u16:1+flush-8:0	09:41:01	4
501816	99	24	20,2	cyclictest	925657	-21	ssh	10:21:31	7
501810	99	24	22,2	cyclictest	831336	-21	kworker/u16:0+events_unbound	09:56:34	5
501810	99	24	22,2	cyclictest	620634	-21	kworker/u16:2+flush-8:0	11:21:29	5
501810	99	24	22,2	cyclictest	620634	-21	kworker/u16:2+events_unbound	11:11:31	5
501810	99	24	21,3	cyclictest	1168889	-21	kworker/u16:4+flush-8:0	12:35:36	5
501802	99	24	22,2	cyclictest	856249	-21	kworker/u16:3+events_unbound	11:17:44	4
501802	99	24	21,2	cyclictest	667327	-21	kworker/u16:3+events_unbound	08:41:20	4
501802	99	23	21,2	cyclictest	581875	-21	kworker/u16:1+flush-8:0	08:11:31	4
501802	99	23	21,2	cyclictest	581875	-21	kworker/u16:1+flush-8:0	08:11:31	4
501816	99	22	21,0	cyclictest	0	-21	swapper/7	11:35:53	7
501810	99	22	20,2	cyclictest	879924	-21	kworker/u16:1+events_unbound	10:16:34	5
501802	99	22	20,2	cyclictest	581875	-21	kworker/u16:1+events_unbound	08:26:21	4
501816	99	21	20,0	cyclictest	0	-21	swapper/7	10:35:08	7
501816	99	21	20,0	cyclictest	0	-21	swapper/7	10:16:23	7
501816	99	21	0,20	cyclictest	1079	-21	lldpd	10:12:01	7
501810	99	21	19,2	cyclictest	3965255	-21	kworker/u16:3+flush-8:0	07:26:33	5
501810	99	21	18,2	cyclictest	1010641	-21	kworker/u16:1+events_unbound	11:26:24	5
501802	99	21	19,2	cyclictest	620634	-21	kworker/u16:2+flush-8:0	10:51:36	4
501802	99	21	19,2	cyclictest	620634	-21	kworker/u16:2+flush-8:0	10:01:34	4
501816	99	20	19,0	cyclictest	0	-21	swapper/7	11:11:25	7
501816	99	20	18,1	cyclictest	684711	-21	cut	08:36:27	7
501810	99	20	18,2	cyclictest	620634	-21	kworker/u16:2+events_unbound	08:56:21	5
501810	99	20	18,2	cyclictest	620634	-21	kworker/u16:2+events_unbound	08:56:21	5
501810	99	20	17,3	cyclictest	3965255	-21	kworker/u16:3+flush-8:0	07:56:37	5
501802	99	20	17,3	cyclictest	1168889	-21	kworker/u16:4+flush-8:0	12:31:06	4
501802	99	20	17,2	cyclictest	856249	-21	kworker/u16:3+events_unbound	11:36:20	4
501816	99	19	18,0	cyclictest	0	-21	swapper/7	11:26:20	7
501816	99	19	16,2	cyclictest	124	-21	kswapd0	12:11:19	7
501816	99	19	16,2	cyclictest	124	-21	kswapd0	12:11:19	7
501810	99	19	16,2	cyclictest	620634	-21	kworker/u16:2+flush-8:0	09:46:23	5
501802	99	19	16,3	cyclictest	473330	-21	kworker/u16:0+flush-8:0	07:56:29	4
501802	99	19	16,2	cyclictest	856249	-21	kworker/u16:3+flush-8:0	12:11:23	4
501802	99	19	16,2	cyclictest	856249	-21	kworker/u16:3+flush-8:0	12:11:23	4
501816	99	18	16,1	cyclictest	1205110	-21	latency_hist	12:21:06	7
501816	99	18	16,1	cyclictest	1205110	-21	latency_hist	12:21:06	7
501812	99	18	3,14	cyclictest	736423	-21	cat	09:06:18	6
501812	99	18	13,4	cyclictest	1134032	-21	cat	11:51:20	6
501812	99	18	11,6	cyclictest	1220293	-21	cut	12:26:22	6
501810	99	18	9,8	cyclictest	958166	-21	cut	10:36:25	5
501810	99	18	16,2	cyclictest	856249	-21	kworker/u16:3+flush-8:0	12:06:21	5
501810	99	18	16,2	cyclictest	856249	-21	kworker/u16:3+flush-8:0	10:46:29	5
501810	99	18	16,2	cyclictest	581875	-21	kworker/u16:1+events_unbound	09:11:30	5
501802	99	18	16,2	cyclictest	620634	-21	kworker/u16:2+flush-8:0	09:26:23	4
501802	99	18	15,3	cyclictest	581875	-21	kworker/u16:1+events_unbound	09:06:31	4
501816	99	17	15,1	cyclictest	735623	-21	diskmemload	09:11:20	7
501816	99	17	11,5	cyclictest	668560	-21	latency_hist	08:31:07	7
501816	99	17	0,1	cyclictest	0	-21	swapper/7	10:36:31	7
501816	99	16	16,0	cyclictest	0	-21	swapper/7	12:25:53	7
501816	99	16	13,2	cyclictest	928874	-21	ssh	10:26:06	7
501816	99	16	13,2	cyclictest	0	-21	swapper/7	11:41:06	7
501816	99	16	13,2	cyclictest	0	-21	swapper/7	09:06:07	7
501816	99	16	0,14	cyclictest	0	-21	swapper/7	12:01:29	7
501812	99	16	3,6	cyclictest	1147606	-21	cut	11:56:24	6
501810	99	16	14,2	cyclictest	620634	-21	kworker/u16:2+flush-8:0	11:31:39	5
501810	99	16	14,2	cyclictest	581875	-21	kworker/u16:1+events_unbound	08:31:34	5
501810	99	16	14,2	cyclictest	1168889	-21	kworker/u16:4+flush-8:0	12:16:01	5
501810	99	16	14,2	cyclictest	1168889	-21	kworker/u16:4+flush-8:0	12:16:01	5
501810	99	16	13,2	cyclictest	856249	-21	kworker/u16:3+flush-8:0	11:36:20	5
501802	99	16	13,2	cyclictest	856249	-21	kworker/u16:3+flush-8:0	11:01:11	4
501802	99	16	0,1	cyclictest	0	-21	swapper/4	10:06:08	4
501816	99	15	14,0	cyclictest	0	-21	swapper/7	08:51:24	7
501816	99	15	14,0	cyclictest	0	-21	swapper/7	07:21:40	7
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17