OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Sat Jul 27, 2024 00:43:30)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
1512845	2	56	22,29	sleep7	0	-21	swapper/7	18:58:25	7
1513012	2	55	45,7	sleep5	0	-21	swapper/5	19:00:48	5
1513020	2	47	36,8	sleep4	0	-21	swapper/4	19:00:55	4
1513433	99	38	33,4	cyclictest	630	-21	systemd-journal	22:22:30	4
1513441	99	36	31,4	cyclictest	630	-21	systemd-journal	23:37:30	6
1512919	2	35	23,8	sleep6	0	-21	swapper/6	18:59:29	6
1513433	99	29	27,2	cyclictest	2083059	-21	kworker/u16:2+events_unbound	00:02:30	4
1513438	99	27	24,2	cyclictest	1848483	-21	kworker/u16:3+events_unbound	22:17:47	5
1513438	99	26	22,3	cyclictest	2131909	-21	latency_hist	00:17:31	5
1513433	99	24	21,2	cyclictest	1986204	-21	kworker/u16:3+flush-8:0	23:12:50	4
1513433	99	24	21,2	cyclictest	1502294	-21	kworker/u16:4+flush-8:0	19:12:54	4
1513433	99	24	21,2	cyclictest	1502294	-21	kworker/u16:4+flush-8:0	19:12:54	4
1513433	99	23	21,2	cyclictest	2054078	-21	kworker/u16:1+flush-8:0	23:42:44	4
1513433	99	23	21,2	cyclictest	1566189	-21	kworker/u16:3+flush-8:0	19:32:31	4
1513438	99	22	18,3	cyclictest	2004855	-21	latency_hist	23:12:30	5
1513438	99	21	17,3	cyclictest	1653759	-21	latency_hist	20:12:31	5
1513433	99	21	19,2	cyclictest	1776838	-21	kworker/u16:0+events_unbound	21:37:43	4
1513433	99	21	19,2	cyclictest	1740648	-21	kworker/u16:1+events_unbound	20:57:45	4
1513433	99	21	19,2	cyclictest	1673218	-21	kworker/u16:2+flush-8:0	21:27:49	4
1513433	99	21	19,2	cyclictest	1653535	-21	kworker/u16:1+flush-8:0	20:27:26	4
1513441	99	20	13,7	cyclictest	0	-21	swapper/6	19:10:25	6
1513441	99	20	13,7	cyclictest	0	-21	swapper/6	19:10:25	6
1513438	99	20	18,2	cyclictest	2083381	-21	kworker/u16:3+events_unbound	23:52:59	5
1513433	99	20	18,2	cyclictest	1776838	-21	kworker/u16:0+flush-8:0	21:43:07	4
1513441	99	19	17,2	cyclictest	1888195	-21	kworker/u16:4+flush-8:0	22:38:01	6
1513438	99	19	17,2	cyclictest	1986204	-21	kworker/u16:3+events_unbound	23:42:43	5
1513433	99	19	17,2	cyclictest	2141610	-21	kworker/u16:3+events_unbound	00:22:30	4
1513433	99	19	17,2	cyclictest	2131835	-21	kworker/u16:2+flush-8:0	00:27:40	4
1513433	99	19	17,2	cyclictest	2102825	-21	kworker/u16:0+events_unbound	00:05:39	4
1513433	99	19	17,2	cyclictest	1986204	-21	kworker/u16:3+flush-8:0	23:27:31	4
1513433	99	19	17,2	cyclictest	1986204	-21	kworker/u16:3+flush-8:0	23:27:31	4
1513433	99	19	17,2	cyclictest	1907684	-21	kworker/u16:2+events_unbound	22:47:25	4
1513433	99	19	17,2	cyclictest	1653535	-21	kworker/u16:1+flush-8:0	20:27:54	4
1513433	99	19	17,2	cyclictest	1653535	-21	kworker/u16:1+flush-8:0	20:17:52	4
1513433	99	19	17,2	cyclictest	1633174	-21	kworker/u16:4+events_unbound	21:35:49	4
1513433	99	19	16,3	cyclictest	1566188	-21	kworker/u16:0+flush-8:0	19:37:31	4
1513433	99	19	15,3	cyclictest	1907684	-21	kworker/u16:2+flush-8:0	23:02:46	4
1513441	99	18	0,16	cyclictest	1523393	-21	latency_hist	19:07:31	6
1513438	99	18	16,2	cyclictest	2054078	-21	kworker/u16:1+flush-8:0	00:27:46	5
1513433	99	18	16,2	cyclictest	2131835	-21	kworker/u16:2+events_unbound	00:27:05	4
1513433	99	18	16,2	cyclictest	1986204	-21	kworker/u16:3+events_unbound	23:25:02	4
1513433	99	18	16,2	cyclictest	1986204	-21	kworker/u16:3+events_unbound	23:25:02	4
1513433	99	18	16,2	cyclictest	1776838	-21	kworker/u16:0+flush-8:0	22:02:42	4
1513433	99	18	16,2	cyclictest	1776838	-21	kworker/u16:0+events_unbound	21:55:01	4
1513433	99	18	16,2	cyclictest	1483693	-21	kworker/u16:1+flush-8:0	19:47:41	4
1513433	99	18	16,2	cyclictest	1483693	-21	kworker/u16:1+events_unbound	19:24:57	4
1513446	99	17	4,5	cyclictest	0	-21	swapper/7	19:04:24	7
1513446	99	17	16,0	cyclictest	0	-21	swapper/7	22:42:54	7
1513446	99	17	0,1	cyclictest	0	-21	swapper/7	22:57:54	7
1513433	99	17	15,2	cyclictest	2083381	-21	kworker/u16:3+events_unbound	23:52:43	4
1513433	99	17	15,2	cyclictest	2054078	-21	kworker/u16:1+flush-8:0	00:12:50	4
1513433	99	17	15,2	cyclictest	1995451	-21	kworker/u16:4+events_unbound	23:32:41	4
1513433	99	17	15,2	cyclictest	1907684	-21	kworker/u16:2+events_unbound	22:30:21	4
1513433	99	17	15,2	cyclictest	1888195	-21	kworker/u16:4+events_unbound	22:41:41	4
1513433	99	17	15,2	cyclictest	1776838	-21	kworker/u16:0+flush-8:0	21:47:40	4
1513433	99	17	15,2	cyclictest	1633174	-21	kworker/u16:4+events_unbound	20:02:07	4
1513433	99	17	14,2	cyclictest	1576190	-21	kworker/u16:2+flush-8:0	19:52:56	4
1513433	99	17	14,2	cyclictest	1135	-21	snmpd	23:48:51	4
1513446	99	16	3,6	cyclictest	0	-21	swapper/7	00:19:10	7
1513446	99	16	3,5	cyclictest	0	-21	swapper/7	20:13:54	7
1513446	99	16	2,6	cyclictest	0	-21	swapper/7	21:29:56	7
1513446	99	16	0,1	cyclictest	0	-21	swapper/7	19:42:57	7
1513441	99	16	13,3	cyclictest	1673218	-21	kworker/u16:2+flush-8:0	20:37:30	6
1513441	99	16	13,2	cyclictest	2013860	-21	awk	23:12:59	6
1513441	99	16	0,15	cyclictest	1784896	-21	grep	21:18:04	6
1513441	99	16	0,15	cyclictest	1784896	-21	grep	21:18:04	6
1513438	99	16	15,1	cyclictest	1624254	-21	kworker/u16:0+flush-8:0	20:02:31	5
1513438	99	16	14,2	cyclictest	1464948	-21	kworker/u16:2+events_unbound	19:02:50	5
1513438	99	16	13,2	cyclictest	1740648	-21	kworker/u16:1+flush-8:0	21:02:31	5
1513438	99	16	13,2	cyclictest	1566189	-21	kworker/u16:3+flush-8:0	19:53:01	5
1513438	99	16	0,15	cyclictest	0	-21	swapper/5	20:22:57	5
1513433	99	16	14,2	cyclictest	1740648	-21	kworker/u16:1+events_unbound	21:06:44	4
1513433	99	16	14,2	cyclictest	1633174	-21	kworker/u16:4+events_unbound	20:12:40	4
1513433	99	16	14,2	cyclictest	1556099	-21	kworker/u16:2+events_unbound	19:22:25	4
1513446	99	15	3,5	cyclictest	0	-21	swapper/7	20:57:31	7
1513441	99	15	7,8	cyclictest	0	-21	swapper/6	23:05:35	6
1513441	99	15	13,2	cyclictest	1888195	-21	kworker/u16:4+events_unbound	22:22:40	6
1513441	99	15	0,14	cyclictest	1684409	-21	irqcore	20:27:46	6
1513438	99	15	13,2	cyclictest	1907684	-21	kworker/u16:2+events_unbound	23:32:30	5
1513438	99	15	13,2	cyclictest	1907684	-21	kworker/u16:2+events_unbound	23:32:30	5
1513438	99	15	13,2	cyclictest	1673218	-21	kworker/u16:2+flush-8:0	21:22:46	5
1513438	99	15	10,4	cyclictest	2024406	-21	latency_hist	23:22:30	5
1513438	99	15	0,1	cyclictest	0	-21	swapper/5	20:17:48	5
1513433	99	15	13,2	cyclictest	2054078	-21	kworker/u16:1+events_unbound	23:40:13	4
1513433	99	15	13,2	cyclictest	2054078	-21	kworker/u16:1+events_unbound	00:12:30	4
1513433	99	15	13,2	cyclictest	1907684	-21	kworker/u16:2+flush-8:0	23:07:51	4
1513433	99	15	13,2	cyclictest	1691585	-21	kworker/u16:0+flush-8:0	20:32:51	4
1513433	99	15	13,2	cyclictest	1691585	-21	kworker/u16:0+events_unbound	20:57:00	4
1513433	99	15	13,2	cyclictest	1633174	-21	kworker/u16:4+flush-8:0	21:23:23	4
1513433	99	15	13,2	cyclictest	1566188	-21	kworker/u16:0+events_unbound	19:42:05	4
1513433	99	15	13,2	cyclictest	1483693	-21	kworker/u16:1+flush-8:0	19:07:51	4
1513433	99	15	13,2	cyclictest	1483693	-21	kworker/u16:1+flush-8:0	19:07:51	4
1513446	99	14	6,7	cyclictest	0	-21	swapper/7	21:19:52	7
1513446	99	14	6,7	cyclictest	0	-21	swapper/7	21:19:52	7
1513446	99	14	6,7	cyclictest	0	-21	swapper/7	20:51:58	7
1513446	99	14	13,0	cyclictest	0	-21	swapper/7	20:22:30	7
1513446	99	14	12,1	cyclictest	0	-21	swapper/7	21:07:55	7
1513446	99	14	12,1	cyclictest	0	-21	swapper/7	19:27:31	7
1513446	99	14	11,2	cyclictest	0	-21	swapper/7	19:15:52	7
1513446	99	14	11,2	cyclictest	0	-21	swapper/7	19:15:52	7
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17