OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system

Characteristics of the highest latencies:

System rack3slot5s.osadl.org (updated Tue Aug 11, 2026 00:43:29)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
17533	2	95	0	sleep0	1217	50	irq/30-eth0_g0_	23:15:37	0
12	1	81	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	21:25:53	1
12	1	57	0	ksoftirqd/1	21414	99	cyclictest	22:57:09	0
5364	2	55	0	sleep1	1217	50	irq/30-eth0_g0_	22:26:48	1
21415	99	55	32	cyclictest	25790	-21	grep	23:47:10	1
12	1	55	0	ksoftirqd/1	21414	99	cyclictest	00:00:00	0
21414	99	54	17	cyclictest	24823	-21	chrt	21:32:08	0
21415	99	53	32	cyclictest	16639	-21	grep	23:12:03	1
21415	99	53	31	cyclictest	10579	-21	ethtool	22:47:04	1
12	1	53	0	ksoftirqd/1	21414	99	cyclictest	23:40:02	0
12	1	53	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	19:02:06	0
12	1	53	0	ksoftirqd/1	1216	50	irq/29-eth0_g0_	23:43:51	0
12	1	53	0	ksoftirqd/1	1216	50	irq/29-eth0_g0_	23:43:51	0
21415	99	52	32	cyclictest	369	-21	/usr/sbin/munin	00:17:08	1
21415	99	52	30	cyclictest	29973	-21	cat	00:06:49	1
21415	99	51	32	cyclictest	6119	-21	unixbench_multi	22:27:09	1
21415	99	51	32	cyclictest	15170	-21	grep	20:52:08	1
12	1	51	0	ksoftirqd/1	21414	99	cyclictest	22:52:11	0
21415	99	50	37	cyclictest	3641	-21	df_abs	20:07:00	1
21415	99	50	37	cyclictest	31640	-21	unixbench_multi	19:47:10	1
21415	99	50	32	cyclictest	1964	-21	runrttasks	21:44:25	1
21415	99	50	31	cyclictest	24843	-21	latency_hist	19:21:50	1
21415	99	50	31	cyclictest	0	-21	swapper/1	20:02:05	1
21415	99	50	26	cyclictest	14682	-21	unixbench_multi	23:02:10	1
21415	99	50	24	cyclictest	0	-21	swapper/1	21:27:06	1
21414	99	50	6	cyclictest	0	-21	swapper/0	20:17:03	0
1559	2	50	0	sleep0	1217	50	irq/30-eth0_g0_	22:10:33	0
1217	50	50	0	irq/30-eth0_g0_	21414	99	cyclictest	00:36:49	0
1216	50	50	0	irq/29-eth0_g0_	21415	99	cyclictest	22:42:11	1
1216	50	50	0	irq/29-eth0_g0_	21415	99	cyclictest	22:42:11	1
1216	50	50	0	irq/29-eth0_g0_	21414	99	cyclictest	19:42:00	0
21415	99	49	37	cyclictest	6599	-21	unixbench_multi	20:17:10	1
21415	99	49	31	cyclictest	29831	-21	mailstats	21:52:07	1
21415	99	49	31	cyclictest	0	-21	swapper/1	20:01:49	1
21415	99	49	27	cyclictest	1964	-21	runrttasks	23:29:16	1
21415	99	49	27	cyclictest	1964	-21	runrttasks	22:09:38	1
21415	99	49	25	cyclictest	7450	-21	chrt	22:32:33	1
21415	99	49	22	cyclictest	32377	-21	grep	22:02:10	1
21414	99	49	2	cyclictest	1216	50	irq/29-eth0_g0_	23:32:07	0
21414	99	49	1	cyclictest	9070	-21	tail	20:27:09	0
1216	50	49	0	irq/29-eth0_g0_	21415	99	cyclictest	23:22:01	1
12	1	49	0	ksoftirqd/1	21414	99	cyclictest	22:42:10	0
12	1	49	0	ksoftirqd/1	21414	99	cyclictest	22:42:10	0
21415	99	48	35	cyclictest	13979	-21	unixbench_multi	20:47:10	1
21415	99	48	31	cyclictest	17472	-21	proc_pri	21:02:08	1
21415	99	48	26	cyclictest	1964	-21	runrttasks	23:44:33	1
21415	99	48	26	cyclictest	1964	-21	runrttasks	23:44:33	1
21415	99	48	26	cyclictest	1964	-21	runrttasks	23:10:58	1
21415	99	48	26	cyclictest	1964	-21	runrttasks	22:14:49	1
21415	99	48	26	cyclictest	1964	-21	runrttasks	19:07:40	1
21415	99	48	23	cyclictest	3541	-21	sendmail_mailqu	22:17:09	1
21414	99	48	2	cyclictest	24931	-21	df_abs	19:22:00	0
21414	99	48	1	cyclictest	4297	-21	df_abs	22:22:00	0
1217	50	48	0	irq/30-eth0_g0_	21414	99	cyclictest	21:29:35	0
1216	50	48	0	irq/29-eth0_g0_	21415	99	cyclictest	21:57:10	1
12	1	48	0	ksoftirqd/1	21414	99	cyclictest	22:32:07	0
12	1	48	0	ksoftirqd/1	21414	99	cyclictest	19:21:10	0
12	1	48	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	21:08:24	1
21415	99	47	35	cyclictest	28457	-21	ntp_states	21:47:06	1
21415	99	47	35	cyclictest	18141	-21	diskmemload	00:10:36	1
21415	99	47	33	cyclictest	31557	-21	cat	00:12:05	1
21415	99	47	33	cyclictest	2709	-21	if_eth0	00:27:02	1
21415	99	47	33	cyclictest	26988	-21	tail	23:52:08	1
21415	99	47	33	cyclictest	26363	-21	latency	19:27:05	1
21415	99	47	31	cyclictest	13199	-21	ntpdc	22:57:05	1
21415	99	47	27	cyclictest	1964	-21	runrttasks	22:40:57	1
21415	99	47	26	cyclictest	1964	-21	runrttasks	23:37:50	1
21415	99	47	26	cyclictest	1964	-21	runrttasks	21:35:14	1
21415	99	47	26	cyclictest	1964	-21	runrttasks	21:14:05	1
21415	99	47	25	cyclictest	1964	-21	runrttasks	22:53:58	1
21415	99	47	25	cyclictest	1964	-21	runrttasks	19:12:43	1
21415	99	47	23	cyclictest	0	-21	swapper/1	21:17:03	1
12	1	47	0	ksoftirqd/1	21414	99	cyclictest	20:51:05	0
21415	99	46	33	cyclictest	5352	-21	grep	20:12:11	1
21415	99	46	33	cyclictest	29022	-21	processes	19:37:08	1
21415	99	46	33	cyclictest	1962	-21	grep	00:22:09	1
21415	99	46	33	cyclictest	18272	-21	sendmail_mailqu	23:17:10	1
21415	99	46	27	cyclictest	1964	-21	runrttasks	00:00:00	1
21415	99	46	25	cyclictest	1964	-21	runrttasks	20:39:37	1
21415	99	46	25	cyclictest	1964	-21	runrttasks	20:39:37	1
21415	99	46	25	cyclictest	1964	-21	runrttasks	20:34:24	1
21415	99	46	23	cyclictest	21709	-21	memory	23:32:03	1
21415	99	46	22	cyclictest	27755	-21	seq	19:32:05	1
21415	99	46	21	cyclictest	0	-21	swapper/1	20:22:09	1
21415	99	46	1	cyclictest	32460	-21	chrt	19:52:02	1
21414	99	46	8	cyclictest	0	-21	swapper/0	19:48:20	0
21414	99	46	7	cyclictest	0	-21	swapper/0	20:01:16	0
21414	99	46	3	cyclictest	0	-21	swapper/0	23:17:05	0
21414	99	46	1	cyclictest	472	-21	chrt	19:52:10	0
21414	99	46	1	cyclictest	0	-21	swapper/0	21:41:50	0
1216	50	46	0	irq/29-eth0_g0_	21415	99	cyclictest	20:46:48	1
1216	50	46	0	irq/29-eth0_g0_	21414	99	cyclictest	20:56:59	0
12	1	46	0	ksoftirqd/1	21414	99	cyclictest	23:09:10	0
12	1	46	0	ksoftirqd/1	21414	99	cyclictest	21:48:29	0
21415	99	45	31	cyclictest	26266	-21	unixbench_multi	21:37:10	1
21415	99	45	31	cyclictest	26266	-21	unixbench_multi	21:37:10	1
21415	99	45	26	cyclictest	29847	-21	grep	19:42:00	1
21415	99	45	24	cyclictest	16358	-21	timerandwakeup	20:57:08	1
21415	99	45	19	cyclictest	0	-21	swapper/1	20:29:33	1
21414	99	45	6	cyclictest	0	-21	swapper/0	20:43:11	0
1217	50	45	0	irq/30-eth0_g0_	21414	99	cyclictest	21:22:00	0
1216	50	45	0	irq/29-eth0_g0_	32425	-21	unixbench_singl	22:02:11	0
12	1	45	0	ksoftirqd/1	21414	99	cyclictest	21:03:50	0
12	1	45	0	ksoftirqd/1	21414	99	cyclictest	20:38:13	0
12	1	45	0	ksoftirqd/1	21414	99	cyclictest	20:38:13	0
12	1	45	0	ksoftirqd/1	21414	99	cyclictest	20:33:17	0
21415	99	44	23	cyclictest	4690	-21	runrttasks	00:34:50	1
21414	99	44	5	cyclictest	0	-21	swapper/0	00:13:28	0
1216	50	44	0	irq/29-eth0_g0_	3182	-21	unixbench_singl	00:27:10	0
12	1	44	0	ksoftirqd/1	21414	99	cyclictest	22:16:54	0
12	1	44	0	ksoftirqd/1	21414	99	cyclictest	20:23:41	0
21415	99	43	32	cyclictest	25262	-21	chrt	19:22:06	1
12	1	43	0	ksoftirqd/1	21414	99	cyclictest	20:12:04	0
12	1	43	0	ksoftirqd/1	21414	99	cyclictest	19:32:01	0
12	1	42	0	ksoftirqd/1	21414	99	cyclictest	23:02:48	0
12	1	42	0	ksoftirqd/1	21414	99	cyclictest	21:57:08	0
12	1	42	0	ksoftirqd/1	21414	99	cyclictest	21:38:21	0
12	1	42	0	ksoftirqd/1	21414	99	cyclictest	21:38:21	0
12	1	42	0	ksoftirqd/1	21414	99	cyclictest	21:09:50	0
12	1	42	0	ksoftirqd/1	21414	99	cyclictest	20:52:07	0
12	1	42	0	ksoftirqd/1	21414	99	cyclictest	20:02:10	0
21414	99	41	2	cyclictest	20558	-21	awk	23:27:05	0
21414	99	41	1	cyclictest	29686	-21	munin-run	19:41:49	0
12	1	41	0	ksoftirqd/1	21414	99	cyclictest	21:17:03	0
12	1	41	0	ksoftirqd/1	21414	99	cyclictest	21:12:05	0
12	1	41	0	ksoftirqd/1	21414	99	cyclictest	19:07:02	0
1216	50	40	0	irq/29-eth0_g0_	13	-21	kworker/0:1	00:25:28	0
1216	50	40	0	irq/29-eth0_g0_	13	-21	kworker/0:1	00:20:10	0
12	1	40	0	ksoftirqd/1	21414	99	cyclictest	19:12:10	0
2539	2	39	0	sleep0	1217	50	irq/30-eth0_g0_	22:12:30	0
21414	99	39	1	cyclictest	0	-21	swapper/0	00:07:04	0
1216	50	39	0	irq/29-eth0_g0_	13	-21	kworker/0:1	23:52:29	0
1216	50	39	0	irq/29-eth0_g0_	13	-21	kworker/0:1	22:29:32	0
1216	50	39	0	irq/29-eth0_g0_	13	-21	kworker/0:1	21:54:49	0
1216	50	39	0	irq/29-eth0_g0_	13	-21	kworker/0:1	00:03:48	0
12	1	39	0	ksoftirqd/1	21414	99	cyclictest	22:47:13	0
12	1	39	0	ksoftirqd/1	21414	99	cyclictest	20:07:09	0
12	1	39	0	ksoftirqd/1	21414	99	cyclictest	19:27:08	0
12	1	39	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	23:48:56	0
1216	50	38	0	irq/29-eth0_g0_	21017	-21	runrttasks	19:02:21	1
12	1	38	0	ksoftirqd/1	21414	99	cyclictest	23:22:44	0
8660	2	37	0	sleep0	1217	50	irq/30-eth0_g0_	22:37:19	0
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17