OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #3, slot #4

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance

Characteristics of the 100 highest latencies:

System rack3slot4.osadl.org (updated Mon Feb 23, 2026 00:46:34)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
1369484	99	132	46,64	cyclictest	1456496	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	21:55:37	0
1369484	99	126	25,77	cyclictest	0	-21	swapper/0	23:10:17	0
1369484	99	126	22,51	cyclictest	1604107	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:45:14	0
1369500	99	124	112,7	cyclictest	1556248	-21	ssh	22:57:26	4
1369495	99	124	110,7	cyclictest	1543832	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	23:02:25	3
1369484	99	119	64,30	cyclictest	1467357	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:19:52	0
1369495	99	118	104,7	cyclictest	1448075	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	21:47:18	3
1369484	99	117	69,40	cyclictest	1654976	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	00:25:57	0
1369484	99	115	81,15	cyclictest	1654976	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	00:15:50	0
1369495	99	114	100,7	cyclictest	1448075	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	21:35:34	3
1369484	99	114	61,46	cyclictest	1475321	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:45:43	0
1369484	99	113	60,46	cyclictest	1475321	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:35:33	0
1369484	99	111	36,50	cyclictest	1399636	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	20:25:33	0
1369484	99	109	28,15	cyclictest	1599539	-21	ntpq	23:30:39	0
1369484	99	108	59,24	cyclictest	1411575	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	21:09:08	0
1369491	99	107	91,8	cyclictest	1543832	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:57:26	2
1369484	99	107	84,15	cyclictest	1475321	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:25:02	0
1369484	99	107	54,46	cyclictest	1429342	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	21:26:31	0
1369484	99	107	42,54	cyclictest	0	-21	swapper/0	22:25:29	0
1369484	99	107	42,54	cyclictest	0	-21	swapper/0	22:25:29	0
1369495	99	105	90,7	cyclictest	1634055	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:13:37	3
1369484	99	105	60,29	cyclictest	1448075	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	21:41:32	0
1369507	99	104	99,1	cyclictest	0	-21	swapper/5	22:57:26	5
1369484	99	104	46,51	cyclictest	1619400	-21	kworker/u12:4+events_unbound@ flush_memcg_stats_dwork	23:55:10	0
1369484	99	104	46,51	cyclictest	1619400	-21	kworker/u12:4+events_unbound@ flush_memcg_stats_dwork	23:55:09	0
1369484	99	103	84,16	cyclictest	1475321	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:12:22	0
1369484	99	103	47,39	cyclictest	1475321	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:00:36	0
1369484	99	102	49,31	cyclictest	1420713	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:20:49	0
1369484	99	102	49,31	cyclictest	1420713	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:20:49	0
1369495	99	101	88,6	cyclictest	1420713	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:27:47	3
1369484	99	101	70,23	cyclictest	1531286	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:45:01	0
1369484	99	101	34,56	cyclictest	1394043	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	20:15:12	0
1369484	99	101	34,56	cyclictest	1394043	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	20:15:12	0
1369484	99	100	69,23	cyclictest	1579055	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:15:38	0
1369484	99	100	69,23	cyclictest	1579055	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:15:38	0
1369484	99	100	61,32	cyclictest	1401675	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	20:35:07	0
1369484	99	100	46,42	cyclictest	1403452	-1	kworker/0:1H-events_highpri	21:34:51	0
1369484	99	100	14,63	cyclictest	13	-21	ksoftirqd/0	23:05:30	0
1369484	99	100	12,78	cyclictest	13	-21	ksoftirqd/0	00:00:34	0
1369484	99	99	64,26	cyclictest	1406693	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	20:45:12	0
1369484	99	99	30,8	cyclictest	1677798	-21	/usr/sbin/munin	00:30:44	0
1369484	99	97	65,25	cyclictest	1569281	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:21:38	0
1369484	99	97	65,25	cyclictest	1569281	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:21:37	0
1369484	99	97	64,25	cyclictest	1429342	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	21:38:45	0
1369484	99	97	49,24	cyclictest	1412477	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	21:00:30	0
1369484	99	97	49,24	cyclictest	1412477	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	21:00:29	0
1369484	99	97	23,18	cyclictest	690	50	irq/528-eth0-Tx	23:25:31	0
1369484	99	97	23,18	cyclictest	690	50	irq/528-eth0-Tx	23:25:31	0
1369495	99	96	84,6	cyclictest	1516664	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	22:49:53	3
1369495	99	96	83,6	cyclictest	1543832	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:54:34	3
1369484	99	96	64,24	cyclictest	1641899	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	00:10:13	0
1369484	99	96	32,6	cyclictest	13	-21	ksoftirqd/0	23:50:17	0
1369484	99	96	10,37	cyclictest	13	-21	ksoftirqd/0	21:50:38	0
1369484	99	96	10,37	cyclictest	13	-21	ksoftirqd/0	21:50:38	0
1369484	99	93	69,16	cyclictest	1369091	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	19:10:34	0
1369484	99	93	59,26	cyclictest	1420713	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:15:42	0
1369484	99	93	40,45	cyclictest	1543832	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	00:05:11	0
1369495	99	92	79,6	cyclictest	1420713	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:22:18	3
1369495	99	92	79,6	cyclictest	1420713	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:22:18	3
1369484	99	92	35,4	cyclictest	691	50	irq/529-eth0-Tx	23:00:31	0
1369484	99	91	7,39	cyclictest	13	-21	ksoftirqd/0	23:40:37	0
1369484	99	91	43,37	cyclictest	0	-21	swapper/0	19:40:25	0
1369484	99	90	58,17	cyclictest	1550300	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:53:24	0
1369484	99	90	54,26	cyclictest	1373236	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	19:29:34	0
1369495	99	89	76,7	cyclictest	1379393	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	19:52:12	3
1369484	99	89	60,26	cyclictest	1654976	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	00:20:13	0
1369484	99	89	33,7	cyclictest	13	-21	ksoftirqd/0	21:45:30	0
1369484	99	89	10,41	cyclictest	13	-21	ksoftirqd/0	00:35:54	0
1369484	99	89	10,41	cyclictest	13	-21	ksoftirqd/0	00:35:54	0
1369495	99	88	74,7	cyclictest	1569281	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:10:46	3
1369484	99	88	56,24	cyclictest	1391846	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	20:09:31	0
1369484	99	88	56,24	cyclictest	1391846	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	20:09:31	0
1369484	99	88	54,31	cyclictest	1543832	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:58:43	0
1369484	99	88	54,24	cyclictest	690	50	irq/528-eth0-Tx	20:25:01	0
1369500	99	87	69,11	cyclictest	1563554	-21	ssh	23:03:18	4
1369495	99	87	59,9	cyclictest	1456496	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	22:20:16	3
1369495	99	86	71,8	cyclictest	1475321	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:26:57	3
1369495	99	86	71,8	cyclictest	1475321	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:26:57	3
1369500	99	85	75,6	cyclictest	1567307	-21	ssh	23:05:45	4
1369495	99	85	71,7	cyclictest	1569281	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:22:54	3
1369495	99	85	71,7	cyclictest	1569281	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:22:54	3
1369495	99	85	71,7	cyclictest	1475321	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:16:18	3
1369488	99	85	42,18	cyclictest	1641899	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	00:27:34	1
1369484	99	85	64,17	cyclictest	1387478	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	19:57:12	0
1369484	99	85	45,36	cyclictest	1484294	-1	kworker/0:0H-events_highpri	23:35:34	0
1369495	99	84	70,7	cyclictest	1590775	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	23:30:29	3
1369495	99	84	69,8	cyclictest	1475321	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:41:34	3
1369491	99	84	61,8	cyclictest	1634055	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:26:28	2
1369484	99	84	22,21	cyclictest	0	-21	swapper/0	22:05:43	0
1369495	99	83	70,6	cyclictest	1475321	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:32:06	3
1369495	99	83	70,6	cyclictest	1475321	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:32:06	3
1369495	99	83	69,7	cyclictest	1579055	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:15:33	3
1369495	99	83	69,7	cyclictest	1579055	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:15:33	3
1369488	99	83	69,7	cyclictest	1475321	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:52:25	1
1369484	99	83	18,27	cyclictest	0	-21	swapper/0	22:30:28	0
1369484	99	83	18,27	cyclictest	0	-21	swapper/0	22:30:28	0
1369495	99	82	69,7	cyclictest	1475321	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:35:34	3
1369495	99	82	67,7	cyclictest	1641899	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	00:35:46	3
1369495	99	82	67,7	cyclictest	1641899	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	00:35:45	3
1369495	99	81	68,6	cyclictest	1654976	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	00:20:18	3
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17