OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #7, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system

Characteristics of all highest latencies:

System rack7slot2.osadl.org (updated Sat May 16, 2026 00:44:09)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
2049232	99	110	102,6	cyclictest	2049163	-21	kworker/u4:5+events_unbound	19:46:01	1
2049230	99	109	96,10	cyclictest	2053040	-21	kworker/u4:18+events_unbound	19:41:01	0
2049230	99	108	100,6	cyclictest	2049162	-21	kworker/u4:4+events_unbound	19:35:59	0
2049230	99	100	93,5	cyclictest	2140061	-21	kworker/u4:9+events_unbound	22:31:43	0
2049232	99	98	83,6	cyclictest	2227743	-21	kworker/u4:8+events_unbound	00:17:45	1
2049232	99	97	90,5	cyclictest	2214426	-21	kworker/u4:13+events_unbound	00:02:33	1
2049230	99	97	90,5	cyclictest	2062041	-21	kworker/u4:7+events_unbound	20:02:02	0
2049232	99	96	88,6	cyclictest	2069826	-21	kworker/u4:15+events_unbound	20:36:15	1
2049230	99	95	88,5	cyclictest	2069822	-21	kworker/u4:8+events_unbound	20:51:16	0
2049232	99	94	87,5	cyclictest	2262233	-21	kworker/u4:9+events_unbound	00:38:38	1
2049230	99	94	86,6	cyclictest	2050457	-21	kworker/u4:9+events_unbound	20:11:04	0
2049232	99	93	86,5	cyclictest	2050457	-21	kworker/u4:9+events_unbound	21:01:24	1
2049232	99	89	82,5	cyclictest	2075020	-21	kworker/u4:1+events_unbound	20:56:23	1
2049232	99	89	82,5	cyclictest	2075020	-21	kworker/u4:1+events_unbound	20:56:23	1
2049232	99	84	76,6	cyclictest	2029727	-21	kworker/u4:10+events_unbound	19:15:56	1
2049230	99	84	76,6	cyclictest	2069826	-21	kworker/u4:15+events_unbound	20:31:13	0
2049230	99	83	76,5	cyclictest	2049163	-21	kworker/u4:5+events_unbound	19:25:58	0
2049230	99	81	74,5	cyclictest	2051727	-21	kworker/u4:13+events_unbound	20:06:04	0
2049232	99	76	69,5	cyclictest	2049162	-21	kworker/u4:4+events_unbound	19:10:55	1
2049230	99	72	67,3	cyclictest	19	1	rcuc/0	21:51:34	0
2049232	99	71	64,5	cyclictest	2114103	-21	kworker/u4:11+events_unbound	21:51:34	1
2049230	99	62	48,6	cyclictest	2046637	-21	kworker/u4:1+events_unbound	19:12:42	0
2049230	99	62	43,17	cyclictest	2194343	-21	munin-run	23:20:01	0
2049232	99	61	48,6	cyclictest	2053040	-21	kworker/u4:18+events_unbound	19:39:43	1
2049230	99	61	52,7	cyclictest	53	-21	kswapd0	21:35:08	0
2049230	99	60	53,5	cyclictest	2078859	-21	kworker/u4:5+events_unbound	21:02:44	0
2049230	99	60	47,10	cyclictest	2242626	-21	ssh	00:15:08	0
2049230	99	60	42,16	cyclictest	53	-21	kswapd0	23:00:13	0
2049230	99	60	42,16	cyclictest	53	-21	kswapd0	23:00:12	0
2049232	99	57	45,8	cyclictest	2143879	-21	kworker/u4:10+events_unbound	22:21:41	1
2049230	99	57	50,5	cyclictest	2134251	-21	diskmemload	22:10:32	0
2049232	99	56	49,5	cyclictest	2060685	-21	kworker/u4:2+events_unbound	19:58:45	1
2049232	99	56	49,5	cyclictest	2051727	-21	kworker/u4:13+events_unbound	20:26:02	1
2049232	99	56	49,5	cyclictest	2050457	-21	kworker/u4:9+events_unbound	19:44:44	1
2049232	99	56	49,5	cyclictest	2050457	-21	kworker/u4:9+events_unbound	19:28:44	1
2049232	99	56	48,6	cyclictest	2050457	-21	kworker/u4:9+events_unbound	20:46:00	1
2049232	99	56	42,9	cyclictest	37	-21	kcompactd0	22:30:26	1
2049230	99	56	51,3	cyclictest	2143879	-21	kworker/u4:10+events_unbound	22:36:45	0
2049232	99	55	45,6	cyclictest	2129432	-21	grep	22:05:19	1
2049232	99	55	44,9	cyclictest	2066423	-21	tune2fs	20:15:23	1
2049230	99	55	37,7	cyclictest	256	50	irq/23-42890000.ethernet	19:10:00	0
2049230	99	55	37,16	cyclictest	2102492	-21	rm	21:35:00	0
2049232	99	54	47,5	cyclictest	2218891	-21	kworker/u4:0+events_unbound	23:52:59	1
2049232	99	54	40,6	cyclictest	2069820	-21	kworker/u4:6+events_unbound	21:07:01	1
2049230	99	54	47,5	cyclictest	2217542	-21	processes	23:45:36	0
2049230	99	54	47,5	cyclictest	2139758	-21	systemd	22:16:39	0
2049230	99	54	46,6	cyclictest	2125017	-21	fschecks_count	22:00:17	0
2049230	99	54	42,9	cyclictest	2222702	-21	kworker/u4:2+xprtiod	23:55:01	0
2049232	99	53	46,5	cyclictest	2049162	-21	kworker/u4:4+events_unbound	20:14:01	1
2049230	99	53	40,10	cyclictest	0	-21	swapper/0	23:30:01	0
2049232	99	52	45,5	cyclictest	2077562	-21	kworker/u4:3+events_unbound	21:35:59	1
2049232	99	52	45,4	cyclictest	0	-21	swapper/1	23:05:27	1
2049232	99	52	44,6	cyclictest	2192726	-21	kworker/u4:4+events_unbound	23:20:02	1
2049230	99	52	46,4	cyclictest	2077562	-21	kworker/u4:3+events_unbound	21:20:04	0
2049230	99	52	44,6	cyclictest	2131224	-21	kworker/u4:8+events_unbound	22:08:59	0
2049230	99	52	42,7	cyclictest	0	-21	swapper/0	00:25:12	0
2049230	99	52	40,9	cyclictest	2089152	-21	sendmail-mta	21:20:01	0
2049232	99	51	44,5	cyclictest	2166007	-21	kworker/u4:15+events_unbound	22:51:00	1
2049232	99	51	44,5	cyclictest	2166007	-21	kworker/u4:15+events_unbound	22:50:59	1
2049232	99	51	44,5	cyclictest	2078861	-21	kworker/u4:10+events_unbound	21:11:58	1
2049232	99	51	43,6	cyclictest	2214420	-21	kworker/u4:5+events_unbound	00:14:59	1
2049232	99	51	43,4	cyclictest	0	-21	swapper/1	23:25:16	1
2049232	99	51	42,6	cyclictest	2255924	-21	sort	00:30:18	1
2049232	99	51	42,5	cyclictest	2214426	-21	kworker/u4:13-rpciod	23:45:29	1
2049232	99	51	41,8	cyclictest	2099631	-21	unixbench_singl	21:30:41	1
2049230	99	51	44,5	cyclictest	2197201	-21	kworker/u4:11+events_unbound	23:32:00	0
2049230	99	51	44,5	cyclictest	2197201	-21	kworker/u4:11+events_unbound	23:32:00	0
2049230	99	51	44,5	cyclictest	2056848	-21	kworker/u4:10+events_unbound	19:50:04	0
2049230	99	51	44,5	cyclictest	0	-21	swapper/0	00:20:14	0
2049230	99	51	43,6	cyclictest	2255906	-21	fschecks_count	00:30:17	0
2049230	99	51	43,6	cyclictest	2060685	-21	kworker/u4:2+events_unbound	19:59:45	0
2049230	99	51	42,6	cyclictest	2100655	-21	kworker/u4:0-rpciod	21:40:29	0
2049232	99	50	38,5	cyclictest	2049165	-21	kworker/u4:8+events_unbound	19:31:01	1
2049230	99	50	43,5	cyclictest	2153321	-21	kworker/u4:1+events_unbound	22:44:59	0
2049230	99	50	43,5	cyclictest	2131221	-21	kworker/u4:4+events_unbound	22:45:03	0
2049230	99	50	31,17	cyclictest	0	-21	swapper/0	23:45:00	0
2049232	99	49	42,5	cyclictest	2091629	-21	kworker/u4:4+events_unbound	21:24:59	1
2049232	99	49	42,5	cyclictest	2072368	-21	kworker/u4:20+events_unbound	20:44:00	1
2049232	99	49	42,5	cyclictest	2051727	-21	kworker/u4:13+events_unbound	20:22:32	1
2049232	99	49	41,6	cyclictest	2203741	-21	fschecks_count	23:30:21	1
2049232	99	49	41,6	cyclictest	2203741	-21	fschecks_count	23:30:20	1
2049232	99	49	41,5	cyclictest	2134688	-21	awk	22:10:41	1
2049232	99	49	41,5	cyclictest	0	-21	swapper/1	23:35:27	1
2049232	99	49	40,6	cyclictest	2125007	-21	fschecks_count	22:00:17	1
2049230	99	49	41,5	cyclictest	2176746	-21	munin-run	23:00:00	0
2049232	99	48	42,4	cyclictest	2172748	-21	aten_r7power_en	22:55:16	1
2049232	99	48	41,5	cyclictest	2166007	-21	kworker/u4:15+events_unbound	22:47:59	1
2049232	99	48	40,6	cyclictest	2216305	-21	apt	23:45:00	1
2049232	99	48	40,6	cyclictest	2183788	-21	kworker/u4:7+events_unbound	23:16:01	1
2049230	99	48	41,5	cyclictest	2240279	-21	kworker/u4:6+events_unbound	00:14:41	0
2049230	99	48	41,5	cyclictest	2059037	-21	grep	19:45:35	0
2049230	99	48	40,6	cyclictest	2051807	-21	sendmail-msp	19:20:02	0
2049230	99	48	39,7	cyclictest	2120504	-21	aten_r7power_vo	21:55:17	0
2049230	99	48	38,7	cyclictest	2234509	-21	gltestperf	00:05:22	0
2049230	99	48	38,7	cyclictest	2211887	-21	modprobe	23:40:00	0
2049230	99	48	37,9	cyclictest	2079095	-21	aten_r7power_cu	21:05:15	0
2049232	99	47	40,5	cyclictest	2077562	-21	kworker/u4:3+events_unbound	21:29:31	1
2049232	99	47	37,7	cyclictest	2112013	-21	gltestperf	21:45:19	1
2049230	99	47	39,6	cyclictest	2072812	-21	aten_r7power_vo	20:40:21	0
2049230	99	47	39,6	cyclictest	2051728	-21	kworker/u4:14+events_unbound	19:33:59	0
2049230	99	47	24,20	cyclictest	0	-21	swapper/0	00:00:38	0
2049232	99	46	39,5	cyclictest	2131222	-21	kworker/u4:6+events_unbound	22:26:00	1
2049232	99	46	37,7	cyclictest	2247872	-21	ntp_states	00:20:37	1
2049232	99	46	36,7	cyclictest	2155240	-21	ssh	22:35:15	1
2049232	99	46	36,7	cyclictest	2075406	-21	df	20:50:17	1
2049232	99	46	35,9	cyclictest	2225837	-21	fschecks_time	23:55:28	1
2049232	99	46	35,9	cyclictest	2120734	-21	fschecks_time	21:55:20	1
2049230	99	46	39,5	cyclictest	2069820	-21	kworker/u4:6+events_unbound	20:27:59	0
2049230	99	46	39,5	cyclictest	2050457	-21	kworker/u4:9+events_unbound	20:55:03	0
2049230	99	46	39,5	cyclictest	2050457	-21	kworker/u4:9+events_unbound	20:55:03	0
2049230	99	46	39,5	cyclictest	2035359	-21	kworker/u4:6+events_unbound	19:24:00	0
2049230	99	46	37,7	cyclictest	2146919	-21	fschecks_time	22:25:19	0
2049230	99	46	37,7	cyclictest	2112019	-21	gltestperf	21:45:19	0
2049230	99	46	36,7	cyclictest	2179315	-21	kworker/u4:4-rpciod	23:05:20	0
2049230	99	46	34,9	cyclictest	2168934	-21	/usr/sbin/munin	22:50:25	0
2049230	99	46	34,9	cyclictest	2168934	-21	/usr/sbin/munin	22:50:24	0
2049230	99	46	28,13	cyclictest	53	-21	kswapd0	23:55:09	0
2049232	99	45	37,6	cyclictest	2131221	-21	kworker/u4:4+events_unbound	22:42:00	1
2049232	99	45	37,5	cyclictest	2052663	-21	proc_pri	19:20:37	1
2049230	99	45	40,4	cyclictest	2143879	-21	kworker/u4:10+events_unbound	22:22:00	0
2049230	99	45	38,5	cyclictest	2253474	-21	kworker/u4:5+events_unbound	00:35:02	0
2049230	99	45	38,5	cyclictest	2051727	-21	kworker/u4:13+events_unbound	20:23:05	0
2049230	99	45	37,5	cyclictest	2066424	-21	fschecks_count	20:15:23	0
2049232	99	44	38,4	cyclictest	0	-21	swapper/1	00:05:21	1
2049232	99	44	37,5	cyclictest	2084909	-21	/usr/sbin/munin	21:15:16	1
2049232	99	44	37,5	cyclictest	2056848	-21	kworker/u4:10+events_unbound	19:50:58	1
2049232	99	44	36,5	cyclictest	2064274	-21	sendmail_mailqu	20:05:36	1
2049232	99	44	35,6	cyclictest	0	-21	swapper/1	22:15:42	1
2049232	99	44	31,6	cyclictest	2049162	-21	kworker/u4:4+events_unbound	20:00:44	1
2049230	99	44	38,4	cyclictest	2190024	-21	latency_hist	23:15:03	0
2049230	99	44	36,5	cyclictest	2094749	-21	ntp_states	21:25:34	0
2049230	99	44	33,9	cyclictest	2194969	-21	fschecks_count	23:20:19	0
2049232	99	43	34,7	cyclictest	2107250	-21	aten2__	21:40:15	1
2049232	99	43	34,7	cyclictest	2071088	-21	systemd	20:31:13	1
2049230	99	43	36,5	cyclictest	2084790	-21	latency_hist	21:15:02	0
2049230	99	43	36,5	cyclictest	2075054	-21	idleruntime-cro	20:50:00	0
2049230	99	43	36,4	cyclictest	2072439	-21	sendmail-mta	20:40:02	0
2049232	99	42	34,6	cyclictest	2187097	-21	unixbench_singl	23:10:39	1
2049232	99	42	27,12	cyclictest	53	-21	kswapd0	23:00:12	1
2049232	99	42	27,12	cyclictest	53	-21	kswapd0	23:00:11	1
2049232	99	41	31,5	cyclictest	0	-21	swapper/1	00:30:00	1
2049232	99	29	22,5	cyclictest	0	-21	swapper/1	19:10:02	1
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17