OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #3, slot #4

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance

Characteristics of the 100 highest latencies:

System rack3slot4.osadl.org (updated Thu Jan 01, 2026 12:46:50)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
1233799	99	155	15,106	cyclictest	1241494	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	07:40:53	0
1233799	99	155	15,106	cyclictest	1241494	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	07:40:52	0
1233799	99	132	24,95	cyclictest	0	-21	swapper/0	12:32:43	0
1233807	99	127	113,7	cyclictest	1241494	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	07:45:28	1
1233799	99	124	98,22	cyclictest	210	-21	kswapd0	11:55:07	0
1233799	99	122	1,80	cyclictest	0	-21	swapper/0	09:20:18	0
1233799	99	121	1,84	cyclictest	0	-21	swapper/0	08:00:29	0
1233799	99	116	86,23	cyclictest	1360044	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	10:51:02	0
1233817	99	115	104,7	cyclictest	210	-21	kswapd0	10:30:00	3
1233799	99	115	2,78	cyclictest	1345733	-21	ps	10:05:40	0
1233799	99	113	1,85	cyclictest	0	-21	swapper/0	08:20:26	0
1233799	99	112	2,79	cyclictest	1477557	-21	/usr/sbin/munin	12:00:45	0
1233799	99	112	2,79	cyclictest	0	-21	swapper/0	12:10:47	0
1233799	99	111	83,21	cyclictest	1353043	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	10:15:56	0
1233799	99	111	1,83	cyclictest	0	-21	swapper/0	11:10:43	0
1233799	99	111	1,83	cyclictest	0	-21	swapper/0	11:10:43	0
1233822	99	110	101,5	cyclictest	1362563	-21	ssh	10:20:28	4
1233799	99	107	2,78	cyclictest	1244351	-21	/usr/sbin/munin	07:35:19	0
1233799	99	107	2,62	cyclictest	1305188	-21	ntp_states	09:30:41	0
1233813	99	106	93,6	cyclictest	1462782	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	12:01:11	2
1233807	99	106	94,6	cyclictest	1360044	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	10:41:00	1
1233799	99	106	59,40	cyclictest	1273928	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	08:58:47	0
1233799	99	105	76,5	cyclictest	210	-21	kswapd0	11:15:13	0
1233799	99	105	76,5	cyclictest	210	-21	kswapd0	11:15:13	0
1233799	99	104	72,24	cyclictest	1456981	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	11:53:34	0
1233807	99	103	88,8	cyclictest	1241494	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	07:40:27	1
1233807	99	103	88,8	cyclictest	1241494	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	07:40:26	1
1233799	99	102	23,37	cyclictest	0	-21	swapper/0	09:50:36	0
1233799	99	101	71,23	cyclictest	1406285	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	11:03:16	0
1233799	99	101	71,23	cyclictest	1406285	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	11:03:15	0
1233799	99	101	53,40	cyclictest	1324190	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	09:47:18	0
1233799	99	101	2,73	cyclictest	1333498	-21	/usr/sbin/munin	09:55:22	0
1233799	99	101	1,67	cyclictest	0	-21	swapper/0	12:15:29	0
1233828	99	99	93,3	cyclictest	0	-21	swapper/5	11:55:07	5
1233807	99	98	84,8	cyclictest	1257769	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	08:15:34	1
1233807	99	98	84,7	cyclictest	1257769	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	08:45:42	1
1233799	99	98	56,34	cyclictest	1368319	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	10:39:56	0
1233828	99	97	92,2	cyclictest	0	-21	swapper/5	10:20:29	5
1233799	99	97	63,27	cyclictest	1233395	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	07:27:07	0
1233799	99	97	63,27	cyclictest	1233395	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	07:27:07	0
1483127	2	95	0,5	sleep4	0	-21	swapper/4	12:05:02	4
1407314	2	95	0,18	sleep1	23	-21	ksoftirqd/1	10:59:43	1
1233813	99	95	82,7	cyclictest	1509872	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	12:36:14	2
1233799	99	95	69,23	cyclictest	1284177	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	09:44:28	0
1233807	99	94	79,8	cyclictest	1307415	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	09:40:34	1
1233799	99	94	61,30	cyclictest	1233395	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	07:14:42	0
1233799	99	94	61,30	cyclictest	1233395	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	07:14:42	0
1233828	99	93	86,3	cyclictest	0	-21	swapper/5	09:35:51	5
1233828	99	93	86,3	cyclictest	0	-21	swapper/5	09:35:51	5
1233817	99	93	79,7	cyclictest	1239459	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	07:25:15	3
1233817	99	93	79,7	cyclictest	1239459	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	07:25:14	3
1233813	99	93	78,8	cyclictest	1462782	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	11:51:10	2
1233799	99	93	64,22	cyclictest	1353043	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	11:34:00	0
1233799	99	92	61,24	cyclictest	1353043	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	11:25:22	0
1233799	99	91	58,25	cyclictest	1360044	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	10:50:02	0
1286885	2	90	0,3	sleep3	1282239	-21	diskmemload	09:15:16	3
1233828	99	90	84,3	cyclictest	0	-21	swapper/5	12:15:14	5
1498454	2	89	0,7	sleep1	22	-21	rcuc/1	12:17:55	1
1233799	99	89	60,6	cyclictest	1337133	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	10:04:02	0
1233822	99	88	76,6	cyclictest	1342780	-21	sh	10:03:32	4
1233813	99	88	76,6	cyclictest	1353043	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	10:25:58	2
1233807	99	88	73,8	cyclictest	1509872	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	12:31:05	1
1233799	99	88	68,17	cyclictest	1412875	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	11:23:45	0
1233799	99	88	68,17	cyclictest	1412875	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	11:23:45	0
1233799	99	88	62,6	cyclictest	1257768	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	08:11:43	0
1233799	99	88	1,75	cyclictest	0	-21	swapper/0	12:05:19	0
1233817	99	87	78,6	cyclictest	210	-21	kswapd0	11:45:01	3
1233799	99	87	61,23	cyclictest	1360044	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	10:21:42	0
1233799	99	87	43,23	cyclictest	1239459	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	07:24:08	0
1233799	99	86	63,6	cyclictest	1251821	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	07:55:48	0
1233799	99	86	63,6	cyclictest	1251821	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	07:55:48	0
1233799	99	86	49,6	cyclictest	1241494	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	07:32:00	0
1233799	99	86	47,7	cyclictest	1412875	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	11:05:14	0
1233799	99	86	1,74	cyclictest	0	-21	swapper/0	08:25:30	0
1233828	99	85	79,3	cyclictest	0	-21	swapper/5	07:35:21	5
1233799	99	85	59,23	cyclictest	1353043	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	10:57:36	0
1233799	99	85	54,7	cyclictest	1307415	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	09:36:09	0
1233799	99	85	54,7	cyclictest	1307415	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	09:36:08	0
1309728	2	84	0,5	sleep3	0	-21	swapper/3	09:35:09	3
1309728	2	84	0,5	sleep3	0	-21	swapper/3	09:35:09	3
1233817	99	84	69,8	cyclictest	1412875	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	11:10:33	3
1233817	99	84	69,8	cyclictest	1412875	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	11:10:32	3
1233813	99	84	72,6	cyclictest	1412875	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	11:31:44	2
1233807	99	84	75,6	cyclictest	210	-21	kswapd0	10:50:27	1
1233813	99	83	71,6	cyclictest	1412875	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	11:56:10	2
1233799	99	83	54,6	cyclictest	1456981	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	11:42:33	0
1233799	99	83	1,68	cyclictest	0	-21	swapper/0	07:45:37	0
1233828	99	82	77,2	cyclictest	0	-21	swapper/5	08:30:37	5
1233817	99	82	69,6	cyclictest	1440426	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	11:36:09	3
1233807	99	82	68,7	cyclictest	1494849	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	12:25:17	1
1233799	99	82	53,6	cyclictest	1494849	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	12:29:58	0
1233828	99	81	71,7	cyclictest	0	-21	swapper/5	11:30:26	5
1233813	99	81	73,5	cyclictest	210	-21	kswapd0	12:10:28	2
1233799	99	81	2,8	cyclictest	1281046	-21	wc	09:05:22	0
1233828	99	80	75,2	cyclictest	0	-21	swapper/5	11:15:13	5
1233828	99	80	75,2	cyclictest	0	-21	swapper/5	11:15:13	5
1233822	99	80	67,7	cyclictest	1460977	-21	ssh	11:45:26	4
1233807	99	80	65,8	cyclictest	1284177	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	09:35:32	1
1233807	99	80	65,8	cyclictest	1284177	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	09:35:31	1
1233813	99	79	65,7	cyclictest	1353043	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	11:01:35	2
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17