OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

x86 Intel Atom E3825 @1333 MHz, Linux 4.9.47-rt37 (Profile)

Latency plot of shadow in rack #9, slot #4

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack9slot4s.osadl.org (updated Tue Jul 05, 2022 00:59:00)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
29279	99	1242	1212,24	cyclictest	29423	-21	apt-get	19:25:12	1
29278	99	1128	1080,36	cyclictest	572	-21	timerandwakeup	19:30:31	0
28892	2	693	329,353	sleep1	0	-21	swapper/1	19:21:33	1
29217	2	564	517,32	sleep0	0	-21	swapper/0	19:24:57	0
29278	99	170	148,4	cyclictest	31151	-21	pidof	19:30:00	0
29279	99	68	63,3	cyclictest	0	-21	swapper/1	19:30:30	1
29279	99	39	13,16	cyclictest	22	-21	ksoftirqd/1	23:35:20	1
29279	99	38	22,5	cyclictest	22	-21	ksoftirqd/1	00:19:47	1
29279	99	38	1,18	cyclictest	16442	-21	apt-get	20:15:19	1
29279	99	37	22,5	cyclictest	22	-21	ksoftirqd/1	23:51:30	1
29279	99	37	22,5	cyclictest	22	-21	ksoftirqd/1	22:35:33	1
29279	99	37	22,5	cyclictest	22	-21	ksoftirqd/1	21:04:21	1
29279	99	37	21,6	cyclictest	22	-21	ksoftirqd/1	23:06:24	1
29279	99	37	21,6	cyclictest	22	-21	ksoftirqd/1	00:35:08	1
29279	99	37	20,6	cyclictest	22	-21	ksoftirqd/1	19:53:13	1
13425	2	37	1,32	sleep1	2374	-21	NetworkManager	22:55:30	1
8435	2	36	1,31	sleep1	8438	-21	apt-get	00:10:11	1
29279	99	36	21,5	cyclictest	22	-21	ksoftirqd/1	22:19:16	1
29279	99	36	21,5	cyclictest	22	-21	ksoftirqd/1	22:07:50	1
29279	99	36	20,6	cyclictest	22	-21	ksoftirqd/1	23:12:55	1
29279	99	36	20,6	cyclictest	22	-21	ksoftirqd/1	19:47:51	1
29279	99	36	20,5	cyclictest	22	-21	ksoftirqd/1	22:32:05	1
29279	99	36	19,6	cyclictest	22	-21	ksoftirqd/1	23:16:10	1
29279	99	36	15,6	cyclictest	22	-21	ksoftirqd/1	23:25:11	1
29278	99	36	25,8	cyclictest	19164	-21	users	00:35:31	0
928	2	35	0,4	sleep0	4	1	ktimersoftd/0	22:25:01	0
29279	99	35	8,7	cyclictest	21554	-21	needreboot	00:45:00	1
29279	99	35	2,19	cyclictest	4477	-21	apt-get	00:00:13	1
29279	99	35	20,6	cyclictest	22	-21	ksoftirqd/1	21:35:25	1
29279	99	35	20,5	cyclictest	22	-21	ksoftirqd/1	23:49:51	1
29279	99	35	20,5	cyclictest	22	-21	ksoftirqd/1	22:28:19	1
29279	99	35	20,5	cyclictest	22	-21	ksoftirqd/1	19:58:21	1
29279	99	35	20,5	cyclictest	22	-21	ksoftirqd/1	00:33:14	1
29279	99	35	13,8	cyclictest	22	-21	ksoftirqd/1	20:57:14	1
29279	99	35	12,7	cyclictest	22	-21	ksoftirqd/1	20:52:33	1
29279	99	35	1,21	cyclictest	19118	-21	ntp_states	21:45:27	1
29279	99	35	12,12	cyclictest	17140	-21	needreboot	21:40:24	1
29279	99	35	1,20	cyclictest	25737	-21	aten_r4power_vo	23:30:17	1
4724	2	34	1,29	sleep0	4728	-21	pidof	19:40:59	0
29279	99	34	9,13	cyclictest	22	-21	ksoftirqd/1	21:54:00	1
29279	99	34	19,5	cyclictest	22	-21	ksoftirqd/1	22:21:45	1
29279	99	34	19,5	cyclictest	22	-21	ksoftirqd/1	22:14:50	1
29279	99	34	19,5	cyclictest	22	-21	ksoftirqd/1	20:45:03	1
29279	99	34	13,6	cyclictest	22	-21	ksoftirqd/1	22:51:59	1
29279	99	34	12,8	cyclictest	22	-21	ksoftirqd/1	21:21:30	1
29279	99	34	12,7	cyclictest	22	-21	ksoftirqd/1	21:58:38	1
29279	99	34	11,7	cyclictest	5119	-21	munin-run	19:45:00	1
29279	99	34	11,7	cyclictest	22	-21	ksoftirqd/1	20:35:37	1
29279	99	34	11,7	cyclictest	22	-21	ksoftirqd/1	20:33:35	1
29279	99	33	9,8	cyclictest	22	-21	ksoftirqd/1	23:57:56	1
29279	99	33	9,8	cyclictest	22	-21	ksoftirqd/1	22:04:52	1
29279	99	33	2,19	cyclictest	0	-21	swapper/1	20:25:22	1
29279	99	33	18,5	cyclictest	22	-21	ksoftirqd/1	21:15:55	1
29279	99	33	10,8	cyclictest	22	-21	ksoftirqd/1	20:23:21	1
29278	99	33	6,5	cyclictest	8	1	rcu_preempt	23:35:20	0
29278	99	33	4,4	cyclictest	8	1	rcu_preempt	23:25:12	0
15120	2	33	1,4	sleep1	15135	-21	ntp_states	23:00:26	1
13249	2	33	1,4	sleep1	13368	-21	sed	00:20:30	1
9354	2	32	1,28	sleep0	9414	-21	if_eth0	21:20:19	0
29279	99	32	2,9	cyclictest	11351	-21	uname	21:25:22	1
29279	99	32	1,7	cyclictest	3274	-21	latency_hist	19:40:01	1
29278	99	32	5,4	cyclictest	8	1	rcu_preempt	22:51:59	0
29278	99	32	12,13	cyclictest	26350	-21	ls	23:30:27	0
27677	2	32	1,5	sleep1	0	-21	swapper/1	20:43:54	1
19753	2	32	1,27	sleep0	2297	-21	irqbalance	21:47:27	0
14186	2	32	1,4	sleep1	26557	-21	snmpd	20:06:18	1
9985	2	31	1,4	sleep1	0	-21	swapper/1	22:46:13	1
8233	2	31	1,5	sleep1	0	-21	swapper/1	00:09:38	1
29279	99	31	9,7	cyclictest	22	-21	ksoftirqd/1	00:45:48	1
29279	99	31	8,7	cyclictest	22	-21	ksoftirqd/1	00:53:47	1
29278	99	31	4,21	cyclictest	9860	-21	grep	22:45:40	0
29278	99	31	1,5	cyclictest	8	1	rcu_preempt	22:09:28	0
29278	99	31	1,4	cyclictest	8	1	rcu_preempt	23:08:04	0
29278	99	31	1,4	cyclictest	8	1	rcu_preempt	20:57:15	0
29278	99	31	1,4	cyclictest	8	1	rcu_preempt	20:52:32	0
23433	2	31	1,5	sleep1	0	-21	swapper/1	23:23:24	1
8868	2	30	1,5	sleep0	0	-21	swapper/0	19:54:57	0
8518	2	30	1,5	sleep0	0	-21	swapper/0	21:18:30	0
29279	99	30	12,5	cyclictest	14653	-21	http_loadtime	00:25:27	1
29278	99	30	20,6	cyclictest	30843	-21	unixbench_singl	22:15:34	0
29278	99	30	1,4	cyclictest	8	1	rcu_preempt	23:13:13	0
29278	99	30	1,4	cyclictest	8	1	rcu_preempt	00:00:22	0
16127	2	30	1,5	sleep1	0	-21	swapper/1	20:11:55	1
9862	2	29	1,5	sleep0	0	-21	swapper/0	00:12:01	0
723	2	29	1,5	sleep0	0	-21	swapper/0	23:50:04	0
4035	2	29	1,5	sleep0	0	-21	swapper/0	23:55:38	0
29279	99	29	8,6	cyclictest	22	-21	ksoftirqd/1	23:44:11	1
29279	99	29	7,12	cyclictest	22	-21	ksoftirqd/1	21:33:44	1
29279	99	29	5,8	cyclictest	22	-21	ksoftirqd/1	20:05:00	1
29279	99	29	4,22	cyclictest	0	-21	swapper/1	21:05:29	1
29279	99	29	4,19	cyclictest	7057	-21	diskstats	22:40:18	1
29279	99	29	2,17	cyclictest	0	-21	swapper/1	21:10:09	1
29278	99	29	5,4	cyclictest	8	1	rcu_preempt	20:15:22	0
29278	99	29	21,5	cyclictest	10847	-21	aten_r4power_vo	21:25:13	0
29278	99	29	19,6	cyclictest	25666	-21	cron	22:05:00	0
29278	99	29	1,4	cyclictest	8	1	rcu_preempt	21:37:06	0
29278	99	29	1,4	cyclictest	8	1	rcu_preempt	20:35:37	0
29278	99	29	1,4	cyclictest	8	1	rcu_preempt	20:33:36	0
2890	2	29	1,5	sleep0	0	-21	swapper/0	19:36:52	0
15959	2	29	1,5	sleep0	0	-21	swapper/0	23:04:27	0
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17