OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #d, slot #3

x86 Intel Atom E3845 @1910 MHz, Linux 4.19.37-rt19 (Profile)

Latency plot of system in rack #d, slot #3

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Up99 -i200 -h400 --smi -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance

Characteristics of the 100, Linux 4.9.20-rt16, x86_64 highest latencies:

System rackdslot3.osadl.org (updated Mon Mar 07, 2022 12:46:05)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
1515	2	181	0,8	sleep3	32304	99	cyclictest	11:10:15	3
18754	2	159	0,8	sleep0	32301	99	cyclictest	08:50:19	0
1282	91	101	80,9	phc2sys	0	-21	swapper/3	07:08:49	3
1282	91	101	80,9	phc2sys	0	-21	swapper/3	07:08:49	3
17081	2	77	0,7	sleep1	32302	99	cyclictest	10:10:11	1
30944	2	69	44,10	sleep2	0	-21	swapper/2	07:05:29	2
30944	2	69	44,10	sleep2	0	-21	swapper/2	07:05:29	2
30651	2	68	0,3	sleep1	0	-21	swapper/1	09:05:19	1
32301	99	57	0,3	cyclictest	0	-21	swapper/0	10:35:19	0
32301	99	56	1,3	cyclictest	0	-21	swapper/0	07:15:17	0
32302	99	54	47,4	cyclictest	25	-21	ksoftirqd/1	07:40:17	1
32302	99	53	46,4	cyclictest	25	-21	ksoftirqd/1	09:10:17	1
2073	2	53	0,2	sleep3	0	-21	swapper/3	09:10:18	3
32302	99	52	46,3	cyclictest	25	-21	ksoftirqd/1	09:30:17	1
32302	99	52	44,3	cyclictest	25	-21	ksoftirqd/1	08:55:16	1
32302	99	51	44,4	cyclictest	25	-21	ksoftirqd/1	07:30:18	1
32302	99	50	44,4	cyclictest	25	-21	ksoftirqd/1	08:15:18	1
32302	99	50	42,4	cyclictest	25	-21	ksoftirqd/1	08:30:16	1
32302	99	50	42,4	cyclictest	25	-21	ksoftirqd/1	08:30:16	1
32304	99	49	3,38	cyclictest	5805	-21	apt-get	10:35:18	3
32303	99	49	43,4	cyclictest	33	-21	ksoftirqd/2	12:30:18	2
32304	99	48	2,39	cyclictest	0	-21	swapper/3	08:55:16	3
32304	99	47	2,38	cyclictest	30181	-21	apt-get	08:25:17	3
32302	99	47	41,3	cyclictest	25	-21	ksoftirqd/1	08:20:21	1
6295	2	46	2,4	sleep2	33	-21	ksoftirqd/2	10:35:22	2
32304	99	46	2,40	cyclictest	16063	-21	grep	08:45:38	3
32304	99	46	2,36	cyclictest	2020	-21	apt-get	12:30:19	3
32303	99	46	2,13	cyclictest	1849	-21	chrt	11:50:17	2
32304	99	45	1,36	cyclictest	14133	-21	apt-get	12:05:18	3
32304	99	45	0,4	cyclictest	20334	-21	latency_hist	11:35:01	3
32303	99	45	3,11	cyclictest	5381	-21	sed	07:15:17	2
32302	99	45	39,3	cyclictest	25	-21	ksoftirqd/1	09:35:17	1
32302	99	45	31,3	cyclictest	25	-21	ksoftirqd/1	07:25:18	1
32301	99	45	0,3	cyclictest	12	1	rcu_preempt	07:40:18	0
31700	2	45	15,19	sleep1	0	-21	swapper/1	07:05:57	1
31700	2	45	15,19	sleep1	0	-21	swapper/1	07:05:57	1
32304	99	44	3,33	cyclictest	30177	-21	apt-get	11:05:18	3
32304	99	44	3,33	cyclictest	30177	-21	apt-get	11:05:18	3
32304	99	44	1,36	cyclictest	21835	-21	apt-get	07:35:17	3
32303	99	44	4,9	cyclictest	0	-21	swapper/2	11:30:31	2
32303	99	44	2,13	cyclictest	1883	-21	sort	10:30:18	2
32302	99	44	38,4	cyclictest	25	-21	ksoftirqd/1	08:50:20	1
32302	99	44	36,4	cyclictest	25	-21	ksoftirqd/1	07:35:16	1
566	98	43	0,3	rtkit-daemon	565	-21	rtkit-daemon	11:56:06	1
566	98	43	0,3	rtkit-daemon	565	-21	rtkit-daemon	11:13:58	1
32304	99	43	1,36	cyclictest	26166	-21	apt-get	09:00:18	3
32303	99	43	24,16	cyclictest	33	-21	ksoftirqd/2	08:37:41	2
32302	99	43	37,3	cyclictest	25	-21	ksoftirqd/1	12:00:21	1
32302	99	43	37,3	cyclictest	25	-21	ksoftirqd/1	08:05:17	1
31780	2	43	13,19	sleep0	0	-21	swapper/0	07:06:57	0
31780	2	43	13,19	sleep0	0	-21	swapper/0	07:06:57	0
32304	99	42	3,33	cyclictest	25879	-21	apt-get	07:40:18	3
32304	99	42	2,32	cyclictest	30355	-21	apt-get	12:25:18	3
32304	99	42	2,32	cyclictest	29897	-21	apt-get	07:45:17	3
32302	99	42	35,4	cyclictest	25	-21	ksoftirqd/1	10:00:19	1
32304	99	41	3,33	cyclictest	18187	-21	apt-get	08:50:17	3
32304	99	41	2,32	cyclictest	9904	-21	apt-get	09:20:17	3
32304	99	41	2,32	cyclictest	17634	-21	apt-get	08:10:17	3
32304	99	41	15,23	cyclictest	10922	-21	iostat	12:00:29	3
32304	99	41	1,34	cyclictest	9576	-21	acpi	08:00:17	3
32304	99	41	1,34	cyclictest	5577	-21	apt-get	07:55:17	3
32304	99	41	1,34	cyclictest	22485	-21	apt-get	11:35:21	3
32304	99	41	1,33	cyclictest	1428	-21	apt-get	07:50:16	3
32302	99	41	35,4	cyclictest	25	-21	ksoftirqd/1	10:25:17	1
32302	99	41	35,4	cyclictest	25	-21	ksoftirqd/1	07:50:18	1
32302	99	41	35,3	cyclictest	25	-21	ksoftirqd/1	11:35:12	1
32302	99	41	35,3	cyclictest	25	-21	ksoftirqd/1	11:05:16	1
32302	99	41	35,3	cyclictest	25	-21	ksoftirqd/1	11:05:16	1
32302	99	41	35,3	cyclictest	25	-21	ksoftirqd/1	09:20:21	1
32304	99	40	3,31	cyclictest	30326	-21	apt-get	10:25:21	3
32304	99	40	3,31	cyclictest	17894	-21	apt-get	10:10:17	3
32304	99	40	2,33	cyclictest	26139	-21	apt-get	09:40:18	3
32304	99	40	2,33	cyclictest	21870	-21	apt-get	08:15:16	3
32304	99	40	2,32	cyclictest	13692	-21	apt-get	08:05:18	3
32304	99	40	2,31	cyclictest	22016	-21	apt-get	12:15:17	3
32304	99	40	1,34	cyclictest	9446	-21	apt-get	07:20:17	3
32304	99	40	1,34	cyclictest	0	-21	swapper/3	09:35:17	3
32304	99	40	1,34	cyclictest	0	-21	swapper/3	09:05:16	3
32304	99	40	1,33	cyclictest	18266	-21	apt-get	10:50:19	3
32304	99	40	1,32	cyclictest	5854	-21	apt-get	08:35:17	3
32304	99	40	0,4	cyclictest	23312	-21	awk	10:15:32	3
32303	99	40	2,11	cyclictest	17694	-21	snmpd	08:45:18	2
32302	99	40	34,4	cyclictest	25	-21	ksoftirqd/1	12:20:19	1
32302	99	40	34,4	cyclictest	25	-21	ksoftirqd/1	09:55:19	1
32302	99	40	34,4	cyclictest	25	-21	ksoftirqd/1	09:50:20	1
32302	99	40	34,4	cyclictest	25	-21	ksoftirqd/1	09:50:19	1
32302	99	40	34,3	cyclictest	25	-21	ksoftirqd/1	09:45:16	1
32302	99	40	33,4	cyclictest	25	-21	ksoftirqd/1	10:40:19	1
32302	99	40	33,4	cyclictest	25	-21	ksoftirqd/1	10:20:00	1
32302	99	40	33,4	cyclictest	25	-21	ksoftirqd/1	09:40:37	1
32301	99	40	33,4	cyclictest	9	-21	ksoftirqd/0	09:50:00	0
32304	99	39	3,31	cyclictest	13980	-21	apt-get	09:25:16	3
32304	99	39	2,31	cyclictest	9890	-21	apt-get	08:40:17	3
32304	99	39	2,31	cyclictest	2067	-21	apt-get	09:50:19	3
32304	99	39	2,31	cyclictest	2067	-21	apt-get	09:50:19	3
32304	99	39	2,31	cyclictest	18359	-21	apt-get	12:10:20	3
32304	99	39	1,33	cyclictest	0	-21	swapper/3	10:40:18	3
32304	99	39	1,32	cyclictest	30182	-21	apt-get	11:45:17	3
32304	99	39	12,24	cyclictest	10363	-21	cat	10:00:22	3
32304	99	39	0,4	cyclictest	6027	-21	/usr/sbin/munin	09:55:32	3
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-06-15 - 15:03