OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #4, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the highest latencies:

System rack4slot7s.osadl.org (updated Sat Sep 30, 2023 00:43:28)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
25900	2	482	401,58	sleep0	0	-21	swapper/0	19:05:08	0
27010	2	463	397,52	sleep1	0	-21	swapper/1	19:08:18	1
25848	2	69	0,1	chrt	25847	-21	wc	20:29:37	0
20046	2	59	0,1	sleep1	0	-21	swapper/1	20:14:36	1
20584	2	55	0,1	sleep1	0	-21	swapper/1	20:15:02	1
23509	2	48	0,1	sleep0	0	-21	swapper/0	23:10:03	0
2894	2	23	1,19	sleep1	2914	-21	awk	19:29:37	1
27307	99	22	1,16	cyclictest	26159	-21	apt-get	20:29:57	0
27308	99	21	13,6	cyclictest	347	50	irq/90-enp1s0-r	23:45:50	1
27308	99	21	12,7	cyclictest	348	50	irq/91-enp1s0-r	21:26:34	1
27308	99	21	1,18	cyclictest	2132	-21	awk	20:50:07	1
27307	99	21	4,14	cyclictest	12578	-21	/usr/sbin/munin	19:54:55	0
27307	99	21	13,6	cyclictest	1	-21	systemd	23:00:06	0
27307	99	21	12,6	cyclictest	4425	-21	ntpq	23:45:08	0
27307	99	21	1,15	cyclictest	24610	-21	apt-get	23:14:56	0
27308	99	20	2,16	cyclictest	3067	-21	/usr/sbin/munin	19:30:05	1
27308	99	20	2,14	cyclictest	15050	-21	apt-get	22:49:55	1
27308	99	20	13,5	cyclictest	32052	-21	/usr/sbin/munin	20:45:01	1
27308	99	20	13,5	cyclictest	31187	-21	ls	22:05:10	1
27308	99	20	12,6	cyclictest	5471	-21	grep	19:35:02	1
27308	99	20	12,6	cyclictest	23199	-21	apt-get	00:34:54	1
27308	99	20	12,6	cyclictest	22829	-21	snmpd	21:45:15	1
27308	99	20	12,6	cyclictest	18099	-21	seq	21:30:20	1
27308	99	20	12,6	cyclictest	1560	-21	sed	23:39:37	1
27308	99	20	11,7	cyclictest	22829	-21	snmpd	22:47:35	1
27308	99	20	11,7	cyclictest	22829	-21	snmpd	22:35:13	1
27308	99	20	11,7	cyclictest	22829	-21	snmpd	22:13:31	1
27308	99	20	11,7	cyclictest	22829	-21	snmpd	21:52:18	1
27308	99	20	11,7	cyclictest	22829	-21	snmpd	21:00:25	1
27308	99	20	11,7	cyclictest	22829	-21	snmpd	19:47:35	1
27308	99	20	0,18	cyclictest	13301	-21	ls	21:19:58	1
27307	99	20	4,14	cyclictest	882	-21	/usr/sbin/munin	22:10:14	0
27307	99	20	4,14	cyclictest	27447	-21	aten2_r4power_p	19:09:59	0
27307	99	20	3,11	cyclictest	31741	-21	taskset	19:19:59	0
27307	99	20	2,14	cyclictest	7424	-21	apt-get	23:54:57	0
27307	99	20	12,6	cyclictest	1507	-21	seq	22:14:58	0
27307	99	20	11,7	cyclictest	9330	-21	cat	19:45:03	0
27307	99	20	11,7	cyclictest	22829	-21	snmpd	20:58:49	0
27307	99	20	11,7	cyclictest	22829	-21	snmpd	00:03:31	0
27308	99	19	7,3	cyclictest	26575	-21	aten2_r4power_e	23:19:58	1
27308	99	19	5,3	cyclictest	349	50	irq/92-enp1s0-t	20:09:14	1
27308	99	19	3,14	cyclictest	17786	-21	ntp_states	22:55:07	1
27308	99	19	14,3	cyclictest	6524	-21	grep	22:25:13	1
27308	99	19	12,5	cyclictest	4651	-21	seq	20:56:29	1
27308	99	19	11,6	cyclictest	8199	-21	ntpq	23:55:09	1
27308	99	19	11,6	cyclictest	347	50	irq/90-enp1s0-r	00:05:54	1
27308	99	19	11,6	cyclictest	3304	-21	seq	23:43:20	1
27308	99	19	11,6	cyclictest	29524	-21	seq	20:37:02	1
27308	99	19	11,6	cyclictest	28721	-21	seq	19:10:56	1
27308	99	19	11,6	cyclictest	27892	-21	seq	21:58:16	1
27308	99	19	11,6	cyclictest	27565	-21	seq	20:31:23	1
27308	99	19	11,6	cyclictest	2301	-21	cat	22:15:08	1
27308	99	19	11,6	cyclictest	22829	-21	snmpd	20:25:06	1
27308	99	19	11,6	cyclictest	22829	-21	snmpd	19:41:17	1
27308	99	19	11,6	cyclictest	15943	-21	seq	20:01:34	1
27308	99	19	11,6	cyclictest	10228	-21	ls	21:10:10	1
27308	99	19	1,14	cyclictest	13354	-21	runrttasks	00:09:56	1
27308	99	19	1,12	cyclictest	7039	-21	apt-get	22:29:42	1
27308	99	19	10,7	cyclictest	22829	-21	snmpd	21:38:29	1
27308	99	19	10,7	cyclictest	22795	-21	cut	23:09:54	1
27307	99	19	6,11	cyclictest	12259	-21	ntp_states	00:05:09	0
27307	99	19	5,9	cyclictest	17759	-21	ntp_states	22:55:06	0
27307	99	19	3,12	cyclictest	22829	-21	snmpd	21:19:56	0
27307	99	19	2,15	cyclictest	11102	-21	fschecks_count	19:50:04	0
27307	99	19	13,4	cyclictest	5870	-21	sort	23:50:01	0
27307	99	19	11,6	cyclictest	22829	-21	snmpd	22:38:07	0
27307	99	19	11,6	cyclictest	22829	-21	snmpd	22:24:28	0
27307	99	19	11,6	cyclictest	22829	-21	snmpd	21:36:23	0
27307	99	19	11,6	cyclictest	22829	-21	snmpd	21:06:55	0
27307	99	19	11,6	cyclictest	22829	-21	snmpd	20:47:43	0
27307	99	19	11,6	cyclictest	22829	-21	snmpd	19:17:19	0
27307	99	19	11,6	cyclictest	22829	-21	snmpd	00:32:02	0
27307	99	19	11,6	cyclictest	22829	-21	snmpd	00:22:43	0
27307	99	19	10,7	cyclictest	22829	-21	snmpd	22:44:34	0
27307	99	19	10,7	cyclictest	22829	-21	snmpd	19:59:41	0
27307	99	19	10,7	cyclictest	19893	-21	taskset	20:13:11	0
27307	99	19	10,7	cyclictest	15456	-21	cat	21:25:04	0
27307	99	19	10,7	cyclictest	12208	-21	kworker/u4:2	00:14:47	0
27308	99	18	9,7	cyclictest	22829	-21	snmpd	21:14:41	1
27308	99	18	9,6	cyclictest	25159	-21	if_err_enp1s0	23:15:05	1
27308	99	18	4,9	cyclictest	19190	-21	apt-get	00:24:56	1
27308	99	18	2,14	cyclictest	20812	-21	cut	21:39:57	1
27308	99	18	11,5	cyclictest	9862	-21	expr	23:59:58	1
27308	99	18	11,5	cyclictest	8537	-21	seq	21:07:09	1
27308	99	18	11,5	cyclictest	4639	-21	awk	22:20:14	1
27308	99	18	11,5	cyclictest	30837	-21	cat	23:30:01	1
27308	99	18	1,15	cyclictest	29786	-21	/usr/sbin/munin	19:15:11	1
27308	99	18	11,5	cyclictest	29001	-21	ntp_kernel_err	22:00:08	1
27308	99	18	11,5	cyclictest	12197	-21	tr	22:40:08	1
27308	99	18	1,14	cyclictest	30580	-21	taskset	20:40:08	1
27308	99	18	1,13	cyclictest	21091	-21	apt-get	00:29:57	1
27308	99	18	10,6	cyclictest	6189	-21	ls	23:50:05	1
27308	99	18	10,6	cyclictest	29162	-21	ls	23:25:05	1
27308	99	18	10,6	cyclictest	22829	-21	snmpd	19:51:38	1
27308	99	18	10,6	cyclictest	22110	-21	seq	20:19:51	1
27308	99	18	10,6	cyclictest	21797	-21	sensors_temp	23:05:11	1
27308	99	18	10,6	cyclictest	19578	-21	ls	23:00:05	1
27308	99	18	10,6	cyclictest	18684	-21	seq	00:21:16	1
27308	99	18	10,6	cyclictest	17055	-21	seq	00:19:25	1
27308	99	18	10,6	cyclictest	13269	-21	load	19:55:04	1
27307	99	18	9,7	cyclictest	29028	-21	ntpq	22:00:08	0
27307	99	18	9,7	cyclictest	22829	-21	snmpd	19:37:59	0
27307	99	18	9,7	cyclictest	21578	-21	awk	23:05:08	0
27307	99	18	9,6	cyclictest	20616	-21	seq	21:39:55	0
27307	99	18	4,12	cyclictest	23557	-21	fschecks_count	00:34:59	0
27307	99	18	4,12	cyclictest	19162	-21	/usr/sbin/munin	00:25:04	0
27307	99	18	3,8	cyclictest	1345	-21	apt-get	20:49:58	0
27307	99	18	13,3	cyclictest	31573	-21	grep	23:30:11	0
27307	99	18	12,4	cyclictest	31119	-21	timerwakeupswit	20:40:15	0
27307	99	18	12,4	cyclictest	28838	-21	ntpq	20:35:08	0
27307	99	18	11,5	cyclictest	9552	-21	sort	21:10:00	0
27307	99	18	11,5	cyclictest	22829	-21	snmpd	20:21:06	0
27307	99	18	1,15	cyclictest	1878	-21	cat	19:25:05	0
27307	99	18	10,6	cyclictest	22829	-21	snmpd	23:44:27	0
27307	99	18	10,6	cyclictest	22829	-21	snmpd	22:05:33	0
27307	99	18	10,6	cyclictest	22829	-21	snmpd	19:31:41	0
27307	99	18	10,6	cyclictest	22829	-21	snmpd	00:13:52	0
27308	99	17	10,5	cyclictest	22829	-21	snmpd	19:21:34	1
27307	99	17	9,6	cyclictest	5645	-21	seq	22:25:00	0
27307	99	17	7,7	cyclictest	15043	-21	cat	22:49:53	0
27307	99	17	4,9	cyclictest	6914	-21	apt-get	19:39:56	0
27307	99	17	4,9	cyclictest	16423	-21	apt-get	20:04:55	0
27307	99	17	4,8	cyclictest	8773	-21	diskmemload	21:14:56	0
27307	99	17	3,9	cyclictest	5165	-21	apt-get	20:59:57	0
27307	99	17	3,8	cyclictest	26311	-21	apt-get	21:54:56	0
27307	99	17	2,11	cyclictest	8773	-21	diskmemload	23:34:58	0
27307	99	17	2,11	cyclictest	26536	-21	apt-get	23:19:57	0
27307	99	17	12,3	cyclictest	25670	-21	grep	21:50:11	0
27307	99	17	12,3	cyclictest	22829	-21	snmpd	22:45:43	0
27307	99	17	11,4	cyclictest	20959	-21	grep	20:15:09	0
27307	99	17	10,5	cyclictest	23030	-21	cat	21:45:02	0
27307	99	16	4,7	cyclictest	8773	-21	diskmemload	22:29:54	0
27307	99	16	2,11	cyclictest	28438	-21	apt-get	23:24:57	0
27307	99	16	11,3	cyclictest	22829	-21	snmpd	21:32:38	0
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17