OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #c, slot #4

Intel(R) Celeron(R) CPU N3350 @ 1.10GHz, Linux 5.9.1-rt18 (Profile)

Latency plot of system in rack #c, slot #4

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the highest latencies:

System rackcslot4.osadl.org (updated Fri Oct 15, 2021 12:43:47)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
4143	99	29	3,18	cyclictest	13656	-21	mandb	07:40:00	1
4143	99	29	1,22	cyclictest	28023	-21	apt-get	12:10:16	1
4143	99	28	2,21	cyclictest	29572	-21	apt-get	08:35:15	1
4142	99	28	12,9	cyclictest	2932	-21	apt-get	08:55:15	0
4143	99	27	2,17	cyclictest	19	-21	ksoftirqd/1	07:40:02	1
4143	99	27	16,7	cyclictest	11931	-21	apt-get	09:25:13	1
4143	99	27	16,6	cyclictest	1814	-21	apt-get	10:40:19	1
4143	99	27	1,21	cyclictest	17967	-21	apt-get	11:35:17	1
4143	99	26	2,5	cyclictest	19421	-21	apt-get	11:40:17	1
4143	99	26	2,19	cyclictest	9003	-21	apt-get	11:05:12	1
4143	99	26	16,6	cyclictest	17679	-21	apt-get	09:45:18	1
4143	99	26	14,7	cyclictest	16536	-21	apt-get	11:30:20	1
4143	99	26	14,6	cyclictest	9042	-21	apt-get	09:15:21	1
4143	99	25	5,9	cyclictest	19541	-21	apt-get	08:00:16	1
4143	99	25	2,7	cyclictest	11863	-21	apt-get	11:15:16	1
4143	99	25	2,5	cyclictest	25161	-21	apt-get	12:00:14	1
4143	99	25	0,21	cyclictest	29823	-21	apt-esm-hook	12:15:19	1
4142	99	25	9,9	cyclictest	18090	-21	apt-get	07:55:14	0
4142	99	25	10,10	cyclictest	23829	-21	apt-get	08:15:14	0
4143	99	24	2,4	cyclictest	26593	-21	apt-get	12:05:15	1
4143	99	24	1,3	cyclictest	18999	-1	kworker/1:0H	10:25:00	1
4143	99	24	1,19	cyclictest	29529	-21	apt-get	10:25:15	1
4143	99	24	1,14	cyclictest	10450	-21	apt-esm-json-ho	11:10:15	1
4143	99	24	0,1	cyclictest	10802	-21	apt-get	07:30:21	1
4143	99	24	0,17	cyclictest	7478	-21	sort	11:00:00	1
4143	99	24	0,16	cyclictest	16148	-21	grep	09:40:00	1
4142	99	24	2,14	cyclictest	1492	-21	apt-get	08:50:17	0
4143	99	23	9,10	cyclictest	3178	-21	apt-get	12:35:15	1
4143	99	23	2,15	cyclictest	20849	-21	apt-get	11:45:14	1
4143	99	23	2,14	cyclictest	4687	-21	apt-get	10:50:15	1
4143	99	23	2,14	cyclictest	2932	-21	apt-get	08:55:15	1
4143	99	23	2,13	cyclictest	16659	-21	apt-get	07:50:19	1
4143	99	23	1,3	cyclictest	7572	-21	apt-get	11:00:15	1
4143	99	23	1,19	cyclictest	25197	-40	apt-check	10:10:08	1
4143	99	23	10,9	cyclictest	1745	-21	apt-get	12:30:13	1
4142	99	23	3,13	cyclictest	10445	-21	apt-esm-hook	11:10:15	0
4142	99	23	2,14	cyclictest	9014	-21	apt-esm-hook	11:05:13	0
4142	99	23	2,14	cyclictest	31269	-21	apt-get	12:20:13	0
4142	99	23	2,14	cyclictest	22400	-21	apt-get	08:10:18	0
4142	99	23	2,14	cyclictest	13798	-21	apt-get	07:40:20	0
4142	99	23	2,14	cyclictest	11942	-21	apt-esm-hook	09:25:15	0
4143	99	22	8,10	cyclictest	26639	-21	apt-get	10:15:11	1
4143	99	22	7,8	cyclictest	6012	-21	latency_hist	07:15:00	1
4143	99	22	2,3	cyclictest	0	-21	swapper/1	10:30:14	1
4143	99	22	2,14	cyclictest	25296	-21	apt-esm-hook	08:20:15	1
4143	99	22	2,14	cyclictest	19119	-21	apt-esm-hook	09:50:13	1
4143	99	22	2,14	cyclictest	15238	-21	apt-esm-hook	07:45:13	1
4143	99	22	2,12	cyclictest	22002	-21	apt-esm-hook	10:00:15	1
4143	99	22	0,9	cyclictest	6024	-1	kworker/1:3H	11:30:01	1
4143	99	22	0,4	cyclictest	2786	-21	munin-run	08:55:00	1
4142	99	22	8,10	cyclictest	6105	-21	apt-get	07:15:17	0
4142	99	22	2,14	cyclictest	17967	-21	apt-get	11:35:16	0
4142	99	22	2,14	cyclictest	11863	-21	apt-get	11:15:17	0
4142	99	22	2,13	cyclictest	29540	-21	apt-esm-hook	10:25:15	0
4142	99	22	2,13	cyclictest	25224	-21	apt-esm-hook	10:10:20	0
4142	99	22	2,13	cyclictest	21991	-21	apt-get	10:00:14	0
4142	99	22	2,13	cyclictest	20860	-21	apt-esm-hook	11:45:15	0
4142	99	22	2,13	cyclictest	20538	-21	apt-get	09:55:21	0
4142	99	22	2,13	cyclictest	13328	-21	apt-esm-hook	11:20:13	0
4142	99	22	2,13	cyclictest	12324	-21	apt-get	07:35:16	0
4142	99	22	1,16	cyclictest	32388	-21	apt-get	10:35:14	0
4143	99	21	5,10	cyclictest	4391	-21	apt-get	09:00:14	1
4143	99	21	2,14	cyclictest	13317	-21	apt-get	11:20:11	1
4143	99	21	2,13	cyclictest	9003	-21	apt-esm-hook	07:25:22	1
4143	99	21	2,13	cyclictest	7562	-21	apt-get	07:20:17	1
4143	99	21	2,13	cyclictest	32466	-21	apt-esm-hook	08:45:14	1
4143	99	21	2,13	cyclictest	18090	-21	apt-get	07:55:14	1
4143	99	21	2,13	cyclictest	16259	-21	apt-esm-hook	09:40:18	1
4143	99	21	2,13	cyclictest	10498	-21	apt-get	09:20:14	1
4143	99	21	11,5	cyclictest	23421	-21	apt-get	10:05:16	1
4143	99	21	1,14	cyclictest	32710	-21	apt-esm-hook	12:25:14	1
4143	99	21	1,14	cyclictest	13362	-21	apt-get	09:30:14	1
4143	99	21	10,4	cyclictest	19	-21	ksoftirqd/1	08:20:00	1
4142	99	21	4,10	cyclictest	32454	-21	apt-get	08:45:15	0
4142	99	21	2,13	cyclictest	7573	-21	apt-esm-hook	07:20:17	0
4142	99	21	2,13	cyclictest	5821	-21	apt-get	09:05:13	0
4142	99	21	2,13	cyclictest	3260	-21	apt-esm-hook	10:45:12	0
4142	99	21	2,13	cyclictest	31025	-21	apt-get	08:40:15	0
4142	99	21	2,13	cyclictest	22281	-21	apt-get	11:50:13	0
4142	99	21	2,13	cyclictest	20970	-21	apt-get	08:05:17	0
4142	99	21	2,13	cyclictest	1801	-40	apt-check	10:40:11	0
4142	99	21	2,13	cyclictest	17690	-21	apt-esm-hook	09:45:18	0
4142	99	21	2,13	cyclictest	1756	-21	apt-esm-hook	12:30:14	0
4142	99	21	2,13	cyclictest	16522	-40	apt-check	11:30:12	0
4142	99	21	2,13	cyclictest	10509	-21	apt-esm-hook	09:20:13	0
4142	99	21	1,4	cyclictest	0	-21	swapper/0	08:00:16	0
4142	99	21	1,14	cyclictest	26604	-21	apt-esm-hook	12:05:15	0
4142	99	21	1,14	cyclictest	19108	-21	apt-get	09:50:14	0
4142	99	21	1,14	cyclictest	15227	-21	apt-get	07:45:14	0
4143	99	20	2,13	cyclictest	23719	-21	apt-esm-hook	11:55:13	1
4143	99	20	2,12	cyclictest	6105	-21	apt-get	07:15:17	1
4143	99	20	2,12	cyclictest	5832	-21	apt-esm-hook	09:05:13	1
4143	99	20	2,12	cyclictest	20981	-21	apt-esm-hook	08:05:17	1
4143	99	20	1,8	cyclictest	3249	-21	apt-get	10:45:12	1
4143	99	20	15,3	cyclictest	1599	-21	snmpd	08:14:31	1
4143	99	20	13,5	cyclictest	32290	-21	cron	08:45:01	1
4143	99	20	0,5	cyclictest	22459	-21	df_inode	11:50:14	1
4143	99	20	0,4	cyclictest	32548	-21	munin-run	12:25:01	1
4142	99	20	2,13	cyclictest	4699	-21	apt-esm-hook	10:50:14	0
4142	99	20	2,13	cyclictest	29572	-21	apt-get	08:35:15	0
4142	99	20	2,12	cyclictest	8990	-21	apt-get	07:25:21	0
4142	99	20	2,12	cyclictest	7583	-21	apt-esm-hook	11:00:15	0
4142	99	20	2,12	cyclictest	28142	-21	apt-get	08:30:16	0
4142	99	20	2,12	cyclictest	28098	-21	apt-get	10:20:16	0
4142	99	20	2,12	cyclictest	26713	-21	apt-get	08:25:14	0
4142	99	20	2,12	cyclictest	26713	-21	apt-get	08:25:14	0
4142	99	20	2,12	cyclictest	25172	-21	apt-esm-hook	12:00:14	0
4142	99	20	1,14	cyclictest	29808	-21	apt-get	12:15:19	0
4142	99	20	1,13	cyclictest	32699	-21	apt-get	12:25:14	0
4142	99	20	0,4	cyclictest	30972	-21	apt-get	10:30:14	0
4143	99	19	2,12	cyclictest	32399	-21	apt-esm-hook	10:35:14	1
4143	99	19	2,12	cyclictest	26724	-21	apt-esm-hook	08:25:15	1
4143	99	19	2,12	cyclictest	26724	-21	apt-esm-hook	08:25:14	1
4143	99	19	1,14	cyclictest	8187	-21	snmp_o5powerctl	09:10:33	1
4143	99	19	0,6	cyclictest	29457	-21	cat	08:35:00	1
4142	99	19	3,9	cyclictest	23708	-21	apt-get	11:55:13	0
4142	99	19	2,12	cyclictest	9055	-21	apt-esm-hook	09:15:21	0
4142	99	19	2,11	cyclictest	7264	-21	apt-esm-hook	09:10:19	0
4142	99	19	1,14	cyclictest	14302	-21	snmp_o5powerctl	09:30:26	0
4142	99	19	1,13	cyclictest	25284	-21	apt-get	08:20:15	0
4142	99	19	1,13	cyclictest	19432	-21	apt-esm-hook	11:40:17	0
4142	99	19	1,12	cyclictest	3189	-21	apt-esm-hook	12:35:16	0
4142	99	19	1,12	cyclictest	26650	-21	apt-esm-hook	10:15:13	0
4142	99	19	1,12	cyclictest	16670	-21	apt-esm-hook	07:50:18	0
4142	99	19	1,12	cyclictest	14802	-21	apt-esm-hook	09:35:15	0
4142	99	19	1,12	cyclictest	14759	-21	apt-esm-hook	11:25:14	0
4142	99	19	1,12	cyclictest	10802	-21	apt-get	07:30:15	0
4143	99	18	1,13	cyclictest	20538	-21	apt-get	09:55:20	1
4142	99	18	2,11	cyclictest	28034	-21	apt-esm-hook	12:10:17	0
4142	99	18	1,13	cyclictest	5331	-21	snmp_o5powerctl	09:00:36	0
4142	99	18	1,12	cyclictest	4686	-21	apt-esm-hook	07:10:19	0
4142	99	18	1,12	cyclictest	23432	-21	apt-esm-hook	10:05:18	0
4142	99	17	3,8	cyclictest	16248	-21	apt-get	09:40:17	0
4142	99	17	2,11	cyclictest	6129	-21	apt-esm-hook	10:55:14	0
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17