OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #3, slot #4

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance

Characteristics of the 100 highest latencies:

System rack3slot4.osadl.org (updated Mon Mar 09, 2026 00:46:29)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
1774041	99	166	83,63	cyclictest	1881220	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:10:06	0
1774045	99	140	126,7	cyclictest	1881220	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:01:52	1
1774060	99	132	125,3	cyclictest	0	-21	swapper/4	22:11:52	4
1774041	99	128	62,47	cyclictest	1920171	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:55:37	0
1774041	99	122	69,21	cyclictest	1821119	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	21:15:46	0
1774045	99	116	102,7	cyclictest	1779737	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	19:26:17	1
1774045	99	110	97,7	cyclictest	2005966	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:37:05	1
1774045	99	108	94,7	cyclictest	1823255	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:16:48	1
1774041	99	108	55,41	cyclictest	0	-21	swapper/0	21:37:35	0
1774041	99	108	55,41	cyclictest	0	-21	swapper/0	21:37:35	0
1774050	99	107	93,7	cyclictest	1920171	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	23:27:06	2
1774050	99	106	92,7	cyclictest	1888960	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	22:21:53	2
1774050	99	106	92,7	cyclictest	1888960	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	22:21:52	2
1774041	99	106	69,26	cyclictest	691	50	irq/529-eth0-Tx	00:25:28	0
1774054	99	105	91,7	cyclictest	1810317	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	20:38:34	3
1774041	99	105	33,25	cyclictest	0	-21	swapper/0	22:15:30	0
1774041	99	104	49,47	cyclictest	1779737	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	19:47:13	0
1774041	99	104	49,47	cyclictest	1779737	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	19:47:12	0
1774054	99	102	89,7	cyclictest	1810317	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	20:55:32	3
1774054	99	102	89,7	cyclictest	1810317	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	20:55:32	3
1774041	99	102	25,61	cyclictest	692	50	irq/530-eth0-Tx	22:25:24	0
1774041	99	102	25,61	cyclictest	692	50	irq/530-eth0-Tx	22:25:23	0
1774050	99	99	85,7	cyclictest	1777685	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	19:16:16	2
1774045	99	98	86,6	cyclictest	1767726	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	19:11:15	1
1774045	99	98	84,7	cyclictest	1792149	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	19:55:54	1
1774041	99	98	73,18	cyclictest	1767726	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	19:21:17	0
1774041	99	98	73,18	cyclictest	1767726	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	19:21:17	0
1774054	99	97	83,7	cyclictest	1790053	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	20:25:35	3
1774041	99	97	64,26	cyclictest	1821119	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	21:21:47	0
1774041	99	97	61,29	cyclictest	1998590	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	23:30:23	0
1774041	99	96	69,19	cyclictest	1961485	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:36:39	0
2093540	2	95	0,4	sleep1	0	-21	swapper/1	00:38:25	1
1774045	99	95	81,7	cyclictest	1865716	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	21:58:24	1
1774041	99	95	68,20	cyclictest	1920171	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	23:20:29	0
1774041	99	95	54,24	cyclictest	1798127	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	20:14:06	0
1774045	99	94	64,8	cyclictest	1901845	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:14:54	1
1774041	99	94	68,19	cyclictest	1873432	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	21:53:33	0
1774041	99	94	20,37	cyclictest	0	-21	swapper/0	22:05:15	0
1774050	99	92	79,7	cyclictest	2076094	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:31:03	2
1774041	99	92	60,22	cyclictest	689	50	irq/527-eth0-Tx	22:45:20	0
1774041	99	92	26,51	cyclictest	15	-21	rcuc/0	21:40:40	0
1774060	99	91	76,9	cyclictest	2016336	-21	ssh	23:39:42	4
1774045	99	91	78,7	cyclictest	1873432	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	21:54:05	1
1774045	99	91	77,7	cyclictest	2017650	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	23:43:54	1
1774045	99	91	62,7	cyclictest	2038558	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:04:58	1
1774045	99	90	77,7	cyclictest	1865716	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:19:14	1
1774045	99	90	62,7	cyclictest	1779737	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	19:31:56	1
1774060	99	89	74,9	cyclictest	1842086	-21	ssh	21:25:43	4
1774045	99	89	77,6	cyclictest	1881220	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:05:06	1
1774045	99	89	61,9	cyclictest	1842732	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:30:13	1
1774045	99	89	61,9	cyclictest	1842732	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:30:13	1
1774041	99	89	36,45	cyclictest	2038558	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:20:14	0
1774041	99	89	22,53	cyclictest	0	-21	swapper/0	22:03:35	0
1774045	99	88	75,7	cyclictest	1865716	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	21:45:14	1
1774041	99	88	29,36	cyclictest	1968464	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:05:15	0
1774041	99	88	27,35	cyclictest	690	50	irq/528-eth0-Tx	21:55:31	0
1774045	99	87	74,7	cyclictest	2060014	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	00:23:33	1
1774045	99	87	74,6	cyclictest	1842732	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:41:50	1
1774041	99	87	65,19	cyclictest	2069554	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	00:35:35	0
1774041	99	87	2,54	cyclictest	1979353	-21	/usr/sbin/munin	23:10:42	0
1774054	99	86	72,7	cyclictest	1821119	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	21:15:33	3
1774054	99	86	60,8	cyclictest	1873432	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	21:53:45	3
1774045	99	86	75,5	cyclictest	1818907	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	21:06:47	1
1774045	99	86	74,6	cyclictest	2069554	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	00:33:54	1
1774041	99	86	66,16	cyclictest	210	-21	kswapd0	00:05:26	0
1774041	99	86	61,18	cyclictest	1941006	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:51:58	0
1774041	99	86	57,21	cyclictest	2038558	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:00:23	0
1774041	99	86	50,25	cyclictest	1842732	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:47:49	0
1774041	99	86	46,23	cyclictest	0	-21	swapper/0	20:00:22	0
1774041	99	86	17,54	cyclictest	0	-21	swapper/0	23:50:18	0
1774041	99	86	1,16	cyclictest	1931693	-21	apt-key	22:35:00	0
1774064	99	85	79,3	cyclictest	0	-21	swapper/5	22:10:06	5
1774060	99	85	71,8	cyclictest	1888212	-21	ssh	22:00:44	4
1774041	99	85	30,43	cyclictest	0	-21	swapper/0	23:00:52	0
1774041	99	85	30,43	cyclictest	0	-21	swapper/0	23:00:51	0
1774054	99	84	70,7	cyclictest	1930494	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:40:16	3
1774050	99	84	54,8	cyclictest	1908712	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:33:39	2
1774045	99	84	72,6	cyclictest	1790053	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	19:51:35	1
1774041	99	84	57,20	cyclictest	1810317	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	20:40:30	0
1774041	99	84	45,35	cyclictest	14	-21	rcu_preempt	21:14:39	0
1774041	99	84	41,27	cyclictest	1938297	-21	sendmail-msp	22:40:01	0
1774054	99	83	70,7	cyclictest	1920171	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:26:53	3
1774054	99	83	70,7	cyclictest	1920171	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:26:53	3
1774050	99	83	56,7	cyclictest	1873432	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	21:52:50	2
1774045	99	83	71,6	cyclictest	1968464	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:03:21	1
1774045	99	83	71,6	cyclictest	1968464	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:03:20	1
1774045	99	83	70,7	cyclictest	1784497	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	19:45:34	1
1774045	99	83	70,7	cyclictest	1784497	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	19:45:33	1
1774041	99	83	2,58	cyclictest	1913638	-21	ntpq	22:20:31	0
1774041	99	83	2,58	cyclictest	1913638	-21	ntpq	22:20:31	0
1773624	2	83	65,9	sleep1	0	-21	swapper/1	19:06:37	1
1773624	2	83	65,9	sleep1	0	-21	swapper/1	19:06:36	1
1774060	99	82	71,6	cyclictest	1936533	-21	ssh	22:38:14	4
1774050	99	82	50,8	cyclictest	1908712	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:25:36	2
1774050	99	82	50,8	cyclictest	1908712	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:25:36	2
1774045	99	82	70,6	cyclictest	1810317	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	20:40:43	1
1774045	99	82	69,6	cyclictest	1842732	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:26:29	1
1774045	99	82	68,7	cyclictest	1831369	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	21:35:34	1
1774045	99	82	68,7	cyclictest	1831369	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	21:35:34	1
1774041	99	82	61,18	cyclictest	1961485	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:17:03	0
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17