OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #3, slot #4

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance

Characteristics of the 100 highest latencies:

System rack3slot4.osadl.org (updated Wed Aug 27, 2025 12:46:41)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
1487658	2	745	737,4	sleep5	0	-21	swapper/5	07:08:12	5
1487975	99	137	123,7	cyclictest	1723952	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	12:29:45	3
1487975	99	136	122,7	cyclictest	1528941	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	09:34:21	3
1487975	99	136	122,7	cyclictest	1528941	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	09:34:20	3
1487961	99	129	91,19	cyclictest	1554290	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	09:59:22	0
1487975	99	125	111,7	cyclictest	1538788	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	09:24:18	3
1487975	99	124	111,6	cyclictest	1723952	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	12:19:42	3
1487975	99	122	109,6	cyclictest	1554290	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	09:49:21	3
1487975	99	119	105,7	cyclictest	1528941	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	09:14:16	3
1487961	99	114	89,18	cyclictest	1467272	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	07:44:04	0
1487970	99	109	95,7	cyclictest	1749863	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	12:39:47	2
1487961	99	107	81,19	cyclictest	1588175	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	10:29:29	0
1487984	99	104	97,3	cyclictest	0	-21	swapper/5	07:39:04	5
1487984	99	104	97,3	cyclictest	0	-21	swapper/5	07:39:03	5
1487984	99	103	96,3	cyclictest	0	-21	swapper/5	09:19:17	5
1678227	2	102	0,6	chrt	0	-21	swapper/0	11:27:34	0
1678227	2	102	0,6	chrt	0	-21	swapper/0	11:27:33	0
1487975	99	102	89,6	cyclictest	1668590	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	11:39:38	3
1487961	99	102	1,89	cyclictest	1495260	-21	timerandwakeup	07:25:42	0
1487984	99	101	94,3	cyclictest	0	-21	swapper/5	10:49:33	5
1487961	99	101	78,16	cyclictest	1621707	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	11:49:42	0
1487975	99	100	86,7	cyclictest	1621707	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	10:54:30	3
1487984	99	99	92,4	cyclictest	0	-21	swapper/5	08:49:13	5
1487984	99	99	92,4	cyclictest	0	-21	swapper/5	08:49:13	5
1487961	99	99	79,16	cyclictest	1528941	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	09:09:16	0
1487961	99	99	36,15	cyclictest	1588175	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	10:02:57	0
1487961	99	98	70,20	cyclictest	1554290	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	10:24:26	0
1487970	99	97	82,7	cyclictest	1682356	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	11:34:38	2
1487970	99	97	82,7	cyclictest	1682356	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	11:34:38	2
1487961	99	97	74,16	cyclictest	1554290	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	09:49:19	0
1487984	99	96	89,3	cyclictest	0	-21	swapper/5	10:09:24	5
1487975	99	95	80,7	cyclictest	1521598	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	08:34:10	3
1487961	99	95	32,13	cyclictest	643	50	irq/528-eth0-Tx	11:39:00	0
1487961	99	95	26,32	cyclictest	1647429	-21	kworker/0:1+pm@ pm_runtime_work	11:15:35	0
1487961	99	95	26,32	cyclictest	1647429	-21	kworker/0:1+pm@ pm_runtime_work	11:15:35	0
1696165	2	94	0,5	sleep0	1535687	-21	diskmemload	11:42:36	0
1487970	99	94	80,7	cyclictest	1696900	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	11:49:40	2
1487961	99	94	2,61	cyclictest	1660171	-21	mailstats	11:10:39	0
1487961	99	94	2,61	cyclictest	1660171	-21	mailstats	11:10:38	0
1487984	99	93	86,3	cyclictest	0	-21	swapper/5	11:54:40	5
1487970	99	92	78,7	cyclictest	1528941	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	08:54:13	2
1487961	99	92	2,41	cyclictest	1758543	-21	cat	12:30:39	0
1487961	99	92	1,54	cyclictest	1711083	-21	ssh	11:54:45	0
1487961	99	91	1,73	cyclictest	0	-21	swapper/0	12:29:42	0
1487961	99	90	65,16	cyclictest	1467272	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	07:39:06	0
1487961	99	90	65,16	cyclictest	1467272	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	07:39:06	0
1487961	99	90	61,13	cyclictest	642	50	irq/527-eth0-Tx	08:14:07	0
1487961	99	90	28,20	cyclictest	1731970	-21	rm	12:10:26	0
1487984	99	89	82,3	cyclictest	0	-21	swapper/5	11:19:37	5
1487984	99	89	82,3	cyclictest	0	-21	swapper/5	11:19:37	5
1487970	99	89	77,6	cyclictest	1495779	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	07:34:03	2
1487970	99	88	75,6	cyclictest	1716617	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	11:59:40	2
1487961	99	88	11,40	cyclictest	1628757	-21	sed	10:40:24	0
1487961	99	87	67,13	cyclictest	1554290	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	09:39:20	0
1487984	99	86	80,2	cyclictest	0	-21	swapper/5	11:05:27	5
1487961	99	86	48,20	cyclictest	643	50	irq/528-eth0-Tx	10:50:30	0
1487961	99	86	2,53	cyclictest	13	-21	ksoftirqd/0	12:02:40	0
1487970	99	85	70,8	cyclictest	1668590	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	12:04:42	2
1487961	99	85	31,12	cyclictest	645	50	irq/530-eth0-Tx	10:59:01	0
1487961	99	85	2,39	cyclictest	1548562	-21	ssh	09:23:09	0
1487961	99	84	27,14	cyclictest	645	50	irq/530-eth0-Tx	09:16:29	0
1487961	99	83	2,6	cyclictest	1738114	-21	/usr/sbin/munin	12:15:20	0
1487961	99	82	2,69	cyclictest	645	50	irq/530-eth0-Tx	11:56:36	0
1487975	99	81	68,6	cyclictest	1528941	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	09:04:16	3
1487970	99	81	68,6	cyclictest	1555473	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	09:30:58	2
1487970	99	81	68,6	cyclictest	1555473	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	09:30:57	2
1487961	99	81	1,57	cyclictest	0	-21	swapper/0	08:44:49	0
1585137	2	80	0,5	chrt	0	-21	swapper/1	09:59:00	1
1487984	99	80	74,3	cyclictest	0	-21	swapper/5	09:54:21	5
1487970	99	80	65,7	cyclictest	1467272	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	07:10:58	2
1487970	99	80	65,7	cyclictest	1467272	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	07:10:58	2
1487961	99	80	1,63	cyclictest	642	50	irq/527-eth0-Tx	10:05:41	0
1487984	99	78	72,3	cyclictest	0	-21	swapper/5	10:34:28	5
1487984	99	78	72,3	cyclictest	0	-21	swapper/5	10:34:28	5
1487975	99	78	68,4	cyclictest	1528941	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	09:29:19	3
1487961	99	77	1,24	cyclictest	0	-21	swapper/0	11:08:59	0
1487970	99	76	62,7	cyclictest	1528941	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	08:59:15	2
1487961	99	76	21,27	cyclictest	13	-21	ksoftirqd/0	12:20:18	0
1487961	99	76	10,19	cyclictest	644	50	irq/529-eth0-Tx	09:42:08	0
1606912	2	75	0,3	sleep1	23	-21	ksoftirqd/1	10:20:01	1
1487975	99	75	61,7	cyclictest	1723952	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	12:14:42	3
1487961	99	75	31,14	cyclictest	643	50	irq/528-eth0-Tx	09:50:37	0
1487984	99	74	69,2	cyclictest	0	-21	swapper/5	09:49:19	5
1487984	99	74	68,3	cyclictest	0	-21	swapper/5	08:39:12	5
1487975	99	74	65,4	cyclictest	1637174	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	11:14:38	3
1487975	99	74	65,4	cyclictest	1637174	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	11:14:37	3
1487961	99	74	19,40	cyclictest	13	-21	ksoftirqd/0	09:31:25	0
1487961	99	74	19,40	cyclictest	13	-21	ksoftirqd/0	09:31:25	0
1487961	99	73	2,9	cyclictest	642	50	irq/527-eth0-Tx	11:00:32	0
1487961	99	73	27,17	cyclictest	13	-21	ksoftirqd/0	08:30:17	0
1754694	2	72	0,5	sleep1	0	-21	swapper/1	12:28:00	1
1487975	99	72	41,9	cyclictest	1621707	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	11:00:13	3
1487966	99	72	59,7	cyclictest	1554290	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	10:14:25	1
1487975	99	71	57,7	cyclictest	1467272	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	07:12:58	3
1487975	99	71	57,7	cyclictest	1467272	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	07:12:57	3
1487961	99	71	33,23	cyclictest	642	50	irq/527-eth0-Tx	10:15:31	0
1487961	99	71	2,52	cyclictest	1539348	-21	sed	09:15:00	0
1487961	99	71	1,53	cyclictest	13	-21	ksoftirqd/0	08:00:27	0
1487961	99	71	1,53	cyclictest	13	-21	ksoftirqd/0	08:00:27	0
1487979	99	70	65,2	cyclictest	0	-21	swapper/4	11:04:33	4
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17