OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system

Characteristics of the 792 highest latencies:

System rack3slot5s.osadl.org (updated Wed Nov 19, 2025 00:43:30)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
12	1	107	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	23:41:29	1
12	1	104	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	21:15:37	1
12	1	103	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	20:40:12	1
25521	2	69	0	sleep1	1217	50	irq/30-eth0_g0_	00:00:08	1
12	1	68	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	00:00:11	0
29318	2	61	0	sleep0	1217	50	irq/30-eth0_g0_	22:03:37	0
16790	2	55	0	sleep0	1216	50	irq/29-eth0_g0_	21:10:26	0
12	1	54	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	23:27:36	1
12	1	52	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	22:52:05	1
12	1	52	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	22:52:05	1
12	1	51	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	21:45:25	0
21261	2	49	0	sleep0	1217	50	irq/30-eth0_g0_	23:41:57	0
10698	2	49	0	sleep0	1216	50	irq/29-eth0_g0_	20:45:53	0
12	1	48	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	00:05:14	1
28635	2	47	0	sleep0	1217	50	irq/30-eth0_g0_	00:12:16	0
12	1	47	0	ksoftirqd/1	1216	50	irq/29-eth0_g0_	23:23:32	1
12	1	47	0	ksoftirqd/1	1216	50	irq/29-eth0_g0_	19:46:05	1
11046	2	47	0	sleep1	1217	50	irq/30-eth0_g0_	20:49:53	1
12	1	46	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	23:19:31	1
12	1	46	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	23:02:01	1
12	1	46	0	ksoftirqd/1	1216	50	irq/29-eth0_g0_	20:52:07	1
12	1	46	0	ksoftirqd/1	1216	50	irq/29-eth0_g0_	20:20:48	1
1217	50	45	0	irq/30-eth0_g0_	1219	-21	kworker/1:2	00:09:59	1
1216	50	45	0	irq/29-eth0_g0_	3	1	ksoftirqd/0	00:18:14	0
12	1	45	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	23:01:39	0
12	1	45	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	22:58:27	1
12	1	45	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	22:46:51	0
12	1	45	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	22:46:51	0
938	2	44	0	chrt	1217	50	irq/30-eth0_g0_	00:30:34	0
1217	50	44	0	irq/30-eth0_g0_	1219	-21	kworker/1:2	22:26:58	1
1216	50	44	0	irq/29-eth0_g0_	13	-21	kworker/0:1	00:25:10	0
1216	50	44	0	irq/29-eth0_g0_	13	-21	kworker/0:1	00:25:10	0
12	1	44	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	21:03:10	0
12	1	44	0	ksoftirqd/1	1216	50	irq/29-eth0_g0_	22:18:44	1
1216	50	43	0	irq/29-eth0_g0_	3	1	ksoftirqd/0	22:30:27	0
1216	50	43	0	irq/29-eth0_g0_	1894	-21	runrttasks	21:21:12	0
12	1	43	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	22:30:01	1
12	1	43	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	00:25:22	1
12	1	43	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	00:25:22	1
12	1	43	0	ksoftirqd/1	1216	50	irq/29-eth0_g0_	23:39:26	1
12	1	43	0	ksoftirqd/1	1216	50	irq/29-eth0_g0_	22:58:48	0
12	1	43	0	ksoftirqd/1	1216	50	irq/29-eth0_g0_	22:00:11	1
12	1	43	0	ksoftirqd/1	1216	50	irq/29-eth0_g0_	21:37:51	0
12	1	43	0	ksoftirqd/1	1216	50	irq/29-eth0_g0_	20:15:18	1
26929	2	42	0	sleep0	1216	50	irq/29-eth0_g0_	21:54:21	0
1217	50	42	0	irq/30-eth0_g0_	32702	-21	df_abs	00:30:06	1
1217	50	42	0	irq/30-eth0_g0_	1219	-21	kworker/1:2	23:52:26	1
1217	50	42	0	irq/30-eth0_g0_	1219	-21	kworker/1:2	23:34:30	1
1217	50	42	0	irq/30-eth0_g0_	1219	-21	kworker/1:2	23:11:47	1
1217	50	42	0	irq/30-eth0_g0_	1219	-21	kworker/1:2	22:45:16	1
1217	50	42	0	irq/30-eth0_g0_	1219	-21	kworker/1:2	22:45:16	1
1216	50	42	0	irq/29-eth0_g0_	3	1	ksoftirqd/0	23:48:26	0
12	1	42	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	22:36:36	1
12	1	42	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	22:09:28	0
12	1	42	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	21:50:15	1
12	1	42	0	ksoftirqd/1	1216	50	irq/29-eth0_g0_	21:30:18	1
1217	50	41	0	irq/30-eth0_g0_	1219	-21	kworker/1:2	22:08:52	1
1216	50	41	0	irq/29-eth0_g0_	3	1	ksoftirqd/0	23:34:59	0
1216	50	41	0	irq/29-eth0_g0_	1894	-21	runrttasks	19:06:23	0
1216	50	41	0	irq/29-eth0_g0_	1894	-21	runrttasks	00:20:09	0
1216	50	41	0	irq/29-eth0_g0_	13	-21	kworker/0:1	22:40:17	0
1216	50	41	0	irq/29-eth0_g0_	12026	-21	/usr/sbin/munin	23:05:08	0
12	1	41	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	21:40:36	0
12	1	41	0	ksoftirqd/1	1216	50	irq/29-eth0_g0_	23:31:09	0
12	1	41	0	ksoftirqd/1	1216	50	irq/29-eth0_g0_	21:45:10	1
12	1	41	0	ksoftirqd/1	1216	50	irq/29-eth0_g0_	00:36:01	1
1216	50	40	0	irq/29-eth0_g0_	22109	-21	latency_hist	21:34:53	0
1216	50	40	0	irq/29-eth0_g0_	1894	-21	runrttasks	22:28:18	0
1216	50	40	0	irq/29-eth0_g0_	13	-21	kworker/0:1	23:16:47	0
1216	50	40	0	irq/29-eth0_g0_	13	-21	kworker/0:1	21:28:15	0
1216	50	40	0	irq/29-eth0_g0_	12	1	ksoftirqd/1	22:52:03	0
1216	50	40	0	irq/29-eth0_g0_	12	1	ksoftirqd/1	22:52:03	0
12	1	40	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	23:53:03	0
12	1	40	0	ksoftirqd/1	1216	50	irq/29-eth0_g0_	21:19:56	1
18958	99	39	1	cyclictest	31955	-21	grep	20:00:19	1
1217	50	39	0	irq/30-eth0_g0_	18956	-21	cyclictest	21:12:24	1
1217	50	39	0	irq/30-eth0_g0_	12	1	ksoftirqd/1	00:19:55	1
1216	50	39	0	irq/29-eth0_g0_	13	-21	kworker/0:1	22:13:34	0
1216	50	39	0	irq/29-eth0_g0_	13	-21	kworker/0:1	00:05:16	0
12	1	39	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	23:21:44	0
12	1	39	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	22:36:07	0
12	1	39	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	22:21:32	0
12	1	39	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	00:39:27	0
12	1	39	0	ksoftirqd/1	1216	50	irq/29-eth0_g0_	21:04:55	1
18958	99	38	1	cyclictest	22143	-21	unixbench_multi	19:20:17	1
1217	50	38	0	irq/30-eth0_g0_	12	1	ksoftirqd/1	23:08:17	1
1217	50	38	0	irq/30-eth0_g0_	12	1	ksoftirqd/1	22:41:19	1
1217	50	38	0	irq/30-eth0_g0_	12	1	ksoftirqd/1	22:13:11	1
1216	50	38	0	irq/29-eth0_g0_	32407	-21	open_inodes	22:15:13	0
1216	50	38	0	irq/29-eth0_g0_	3	1	ksoftirqd/0	20:20:10	0
1216	50	38	0	irq/29-eth0_g0_	13	-21	kworker/0:1	23:56:00	0
1216	50	38	0	irq/29-eth0_g0_	13	-21	kworker/0:1	23:26:19	0
1216	50	38	0	irq/29-eth0_g0_	13	-21	kworker/0:1	23:12:44	0
12	1	38	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	21:07:03	0
12	1	38	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	19:24:11	0
18958	99	37	8	cyclictest	1217	50	irq/30-eth0_g0_	21:39:23	1
18958	99	37	2	cyclictest	26535	-21	grep	19:40:06	1
18958	99	37	2	cyclictest	13908	-21	cat	21:00:10	1
1217	50	37	0	irq/30-eth0_g0_	460	-21	ntp_offset	20:05:11	1
1217	50	37	0	irq/30-eth0_g0_	1219	-21	kworker/1:2	23:55:08	1
1217	50	37	0	irq/30-eth0_g0_	1219	-21	kworker/1:2	23:47:34	1
1217	50	37	0	irq/30-eth0_g0_	1219	-21	kworker/1:2	21:27:35	1
1217	50	37	0	irq/30-eth0_g0_	12	1	ksoftirqd/1	22:20:58	1
1216	50	37	0	irq/29-eth0_g0_	5544	-21	sendmail_mailqu	20:25:14	0
1216	50	37	0	irq/29-eth0_g0_	28299	-21	unixbench_singl	19:45:17	0
1216	50	37	0	irq/29-eth0_g0_	2635	-21	df	20:15:09	0
1216	50	37	0	irq/29-eth0_g0_	23033	-21	ls	19:25:10	0
1216	50	37	0	irq/29-eth0_g0_	1894	-21	runrttasks	19:59:18	0
1216	50	37	0	irq/29-eth0_g0_	1894	-21	runrttasks	19:59:18	0
1216	50	37	0	irq/29-eth0_g0_	13	-21	kworker/0:1	21:57:07	0
1216	50	37	0	irq/29-eth0_g0_	13	-21	kworker/0:1	21:15:06	0
12	1	37	0	ksoftirqd/1	28219	-21	taskset	21:59:53	1
18958	99	36	1	cyclictest	29273	-21	sed	00:15:11	1
18958	99	36	1	cyclictest	20744	-21	ps	19:15:15	1
18958	99	36	1	cyclictest	19206	-21	chrt	19:10:05	1
18958	99	36	1	cyclictest	1587	-21	cat	20:10:12	1
1217	50	36	0	irq/30-eth0_g0_	6873	-21	unixbench_multi	20:30:16	1
1216	50	36	0	irq/29-eth0_g0_	3	1	ksoftirqd/0	20:04:53	0
1216	50	36	0	irq/29-eth0_g0_	19368	-21	cat	19:10:08	0
1216	50	36	0	irq/29-eth0_g0_	1894	-21	runrttasks	19:18:36	0
1216	50	36	0	irq/29-eth0_g0_	18591	-21	runrttasks	19:05:46	1
1216	50	36	0	irq/29-eth0_g0_	0	-21	swapper/0	20:40:16	0
12	1	36	0	ksoftirqd/1	23926	-21	zcat	21:40:14	1
12	1	36	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	20:59:02	0
12	1	36	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	19:52:07	0
18958	99	35	2	cyclictest	25738	-21	awk	19:35:16	1
18958	99	35	1	cyclictest	5700	-21	unixbench_singl	20:25:16	1
18958	99	35	1	cyclictest	13044	-21	grep	20:55:16	1
1216	50	35	0	irq/29-eth0_g0_	25818	-21	unixbench_multi	19:35:17	0
1216	50	35	0	irq/29-eth0_g0_	1894	-21	runrttasks	20:10:21	0
1216	50	35	0	irq/29-eth0_g0_	1894	-21	runrttasks	20:08:37	0
12	1	35	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	19:40:05	0
18958	99	34	1	cyclictest	7589	-21	df_abs	20:35:07	1
18958	99	34	1	cyclictest	24215	-21	cat	19:30:12	1
18958	99	34	1	cyclictest	22733	-21	egrep	19:25:06	1
1217	50	34	0	irq/30-eth0_g0_	30535	-21	proc_pri	19:55:13	1
1217	50	34	0	irq/30-eth0_g0_	30535	-21	proc_pri	19:55:13	1
1216	50	34	0	irq/29-eth0_g0_	6528	-21	missed_timers	20:30:10	0
1216	50	34	0	irq/29-eth0_g0_	24090	-21	sed	19:30:09	0
1216	50	34	0	irq/29-eth0_g0_	12271	-21	munin-run	20:54:53	0
12	1	34	0	ksoftirqd/1	1216	50	irq/29-eth0_g0_	19:50:16	1
1216	50	33	0	irq/29-eth0_g0_	1894	-21	runrttasks	20:36:31	0
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17