OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #3, slot #4

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance

Characteristics of the 100 highest latencies:

System rack3slot4.osadl.org (updated Fri May 01, 2026 12:46:47)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
1498477	99	136	59,39	cyclictest	1710496	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	11:49:07	0
1498477	99	131	2,98	cyclictest	1517004	-21	aten2_r3power_e	07:55:14	0
1498477	99	128	113,8	cyclictest	1623493	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	10:35:04	0
1498477	99	124	72,41	cyclictest	689	50	irq/527-eth0-Tx	07:20:00	0
1498477	99	122	1,95	cyclictest	0	-21	swapper/0	07:25:18	0
1498477	99	122	1,95	cyclictest	0	-21	swapper/0	07:25:18	0
1498477	99	116	55,52	cyclictest	1609217	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	10:05:20	0
1498477	99	116	55,52	cyclictest	1609217	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	10:05:19	0
1498489	99	112	101,5	cyclictest	1631352	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	10:30:03	2
1498477	99	112	77,4	cyclictest	0	-21	swapper/0	08:55:40	0
1498477	99	112	35,29	cyclictest	1631352	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	10:20:10	0
1498477	99	109	6,54	cyclictest	13	-21	ksoftirqd/0	10:45:29	0
1498477	99	109	59,42	cyclictest	1652674	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	10:50:31	0
1498477	99	107	76,23	cyclictest	1615692	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	10:14:34	0
1498477	99	106	62,36	cyclictest	1557919	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	09:28:56	0
1498477	99	105	74,24	cyclictest	1678045	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	10:56:35	0
1498477	99	105	67,17	cyclictest	1530554	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	08:54:46	0
1498482	99	104	92,6	cyclictest	1724350	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	11:35:54	1
1498477	99	102	73,22	cyclictest	1546800	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	09:20:04	0
1498489	99	101	89,6	cyclictest	1754354	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	12:04:34	2
1498489	99	101	89,6	cyclictest	1754354	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	12:04:34	2
1498489	99	101	88,7	cyclictest	1678045	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	11:00:25	2
1498489	99	101	88,7	cyclictest	1678045	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	11:00:25	2
1498489	99	101	88,6	cyclictest	1508269	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	07:59:31	2
1498477	99	101	75,17	cyclictest	1762964	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	12:05:16	0
1498503	99	100	94,3	cyclictest	0	-21	swapper/5	12:15:25	5
1498477	99	100	62,17	cyclictest	1541205	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	09:17:41	0
1498489	99	99	86,7	cyclictest	1623493	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	10:19:34	2
1498477	99	99	67,24	cyclictest	1724350	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	12:02:35	0
1498477	99	99	67,24	cyclictest	1724350	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	12:02:35	0
1498477	99	99	65,14	cyclictest	1678045	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	11:12:31	0
1498477	99	99	65,14	cyclictest	1678045	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	11:12:31	0
1498477	99	99	60,30	cyclictest	1541205	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	09:10:34	0
1498489	99	98	84,7	cyclictest	1652674	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	11:14:36	2
1498489	99	98	84,7	cyclictest	1652674	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	11:14:35	2
1498477	99	98	78,17	cyclictest	1623493	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	10:29:34	0
1498477	99	98	75,6	cyclictest	1631352	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	10:33:06	0
1498477	99	98	70,24	cyclictest	1724350	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	12:18:38	0
1498477	99	98	1,59	cyclictest	1770745	-21	/usr/sbin/munin	12:10:18	0
1498494	99	97	83,7	cyclictest	1678045	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	11:10:11	3
1498494	99	97	83,7	cyclictest	1678045	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	11:10:10	3
1498477	99	97	1,72	cyclictest	0	-21	swapper/0	09:00:23	0
1498477	99	96	60,15	cyclictest	1565278	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	09:31:47	0
1498477	99	96	60,13	cyclictest	690	50	irq/528-eth0-Tx	12:25:05	0
1498477	99	96	51,19	cyclictest	0	-21	swapper/0	11:25:17	0
1498477	99	96	42,17	cyclictest	13	-21	ksoftirqd/0	11:30:36	0
1498489	99	95	71,7	cyclictest	1537057	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	09:41:34	2
1498489	99	95	71,7	cyclictest	1537057	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	09:41:33	2
1498477	99	95	66,8	cyclictest	1742239	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	11:55:35	0
1498477	99	95	66,8	cyclictest	1742239	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	11:55:35	0
1498477	99	95	48,18	cyclictest	0	-21	swapper/0	11:20:01	0
1498477	99	95	48,18	cyclictest	0	-21	swapper/0	11:20:00	0
1498494	99	94	80,7	cyclictest	1609217	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	10:05:36	3
1498494	99	94	80,7	cyclictest	1609217	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	10:05:35	3
1498477	99	94	62,16	cyclictest	1710496	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	11:24:38	0
1498477	99	94	62,16	cyclictest	1710496	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	11:24:38	0
1498494	99	93	78,8	cyclictest	1724350	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	12:17:09	3
1498477	99	93	61,16	cyclictest	1565278	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	09:39:05	0
1498477	99	93	61,16	cyclictest	1565278	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	09:39:04	0
1498477	99	93	61,15	cyclictest	1710496	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	11:38:56	0
1498477	99	93	42,14	cyclictest	0	-21	swapper/0	11:08:55	0
1498477	99	93	42,14	cyclictest	0	-21	swapper/0	11:08:54	0
1498489	99	92	79,7	cyclictest	1600834	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	09:59:03	2
1498489	99	92	79,6	cyclictest	1710496	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	11:25:15	2
1498477	99	92	61,15	cyclictest	1600834	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	09:57:44	0
1498489	99	91	78,7	cyclictest	1652674	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	10:43:44	2
1498489	99	91	78,7	cyclictest	1652674	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	10:43:43	2
1498489	99	91	61,7	cyclictest	1785016	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	12:34:42	2
1498489	99	90	77,7	cyclictest	1704026	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	11:20:24	2
1498489	99	90	77,7	cyclictest	1704026	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	11:20:24	2
1498489	99	90	76,7	cyclictest	1583016	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	09:47:37	2
1498477	99	90	59,16	cyclictest	1573819	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	09:53:55	0
1498477	99	90	56,7	cyclictest	1600834	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	10:00:05	0
1498498	99	89	83,3	cyclictest	0	-21	swapper/4	10:14:59	4
1498489	99	89	60,7	cyclictest	1499452	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	07:30:27	2
1498482	99	89	76,6	cyclictest	1678045	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	11:28:23	1
1498498	99	88	82,2	cyclictest	0	-21	swapper/4	09:20:33	4
1498494	99	88	74,7	cyclictest	1724350	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	12:25:36	3
1498494	99	88	60,8	cyclictest	1710496	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	11:52:08	3
1498494	99	88	60,8	cyclictest	1710496	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	11:52:08	3
1498489	99	88	76,6	cyclictest	1710496	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	11:37:34	2
1498489	99	88	76,6	cyclictest	1512567	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	07:54:31	2
1498489	99	88	75,7	cyclictest	1591731	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	10:26:40	2
1498489	99	88	74,7	cyclictest	1710496	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	11:45:41	2
1498482	99	88	76,6	cyclictest	1591731	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	09:59:38	1
1498477	99	88	57,8	cyclictest	1583016	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	09:43:39	0
1498477	99	88	57,8	cyclictest	1583016	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	09:43:38	0
1498477	99	88	57,7	cyclictest	1631352	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	10:40:30	0
1498477	99	88	57,7	cyclictest	1631352	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	10:40:29	0
1498494	99	87	74,6	cyclictest	1754354	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	12:08:25	3
1498489	99	87	75,6	cyclictest	1644968	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	10:48:36	2
1601548	2	86	0,5	sleep5	0	-21	swapper/5	09:55:09	5
1498494	99	86	72,6	cyclictest	1541205	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	09:22:39	3
1498494	99	86	55,7	cyclictest	1631352	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	10:48:57	3
1498489	99	86	74,6	cyclictest	1678045	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	11:42:45	2
1498489	99	86	73,7	cyclictest	1695039	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	11:17:54	2
1498489	99	86	73,7	cyclictest	1695039	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	11:17:54	2
1498489	99	86	73,7	cyclictest	1631352	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	10:20:05	2
1498482	99	86	74,6	cyclictest	1591731	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	10:25:50	1
1498482	99	86	62,10	cyclictest	1798642	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	12:36:43	1
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17