OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #3, slot #4

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance

Characteristics of the 100 highest latencies:

System rack3slot4.osadl.org (updated Wed Jan 07, 2026 00:46:50)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
1631414	2	1256	1229,17	sleep5	0	-21	swapper/5	19:05:20	5
1632856	99	195	59,93	cyclictest	1767391	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:30:35	0
1632856	99	195	59,93	cyclictest	1767391	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:30:35	0
1632856	99	130	93,30	cyclictest	1864510	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:22:37	0
1632865	99	124	111,6	cyclictest	1700468	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	21:27:01	2
1632865	99	124	111,6	cyclictest	1700468	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	21:27:01	2
1898439	2	121	0,4	sleep2	0	-21	swapper/2	00:10:18	2
1898439	2	121	0,4	sleep2	0	-21	swapper/2	00:10:17	2
1632856	99	119	83,28	cyclictest	1733453	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:05:22	0
1632856	99	117	42,52	cyclictest	1715830	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	21:52:12	0
1632860	99	113	99,7	cyclictest	1752524	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:57:18	1
1632856	99	112	82,23	cyclictest	1767391	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	23:12:20	0
1632856	99	112	1,50	cyclictest	0	-21	swapper/0	00:35:19	0
1632856	99	111	82,24	cyclictest	210	-21	kswapd0	23:50:23	0
1632856	99	111	24,81	cyclictest	1832805	-21	ntpq	23:15:37	0
1632856	99	109	78,24	cyclictest	1752524	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:47:16	0
1632856	99	109	78,24	cyclictest	1752524	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:47:16	0
1632856	99	109	27,7	cyclictest	1640673	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	19:47:15	0
1632856	99	108	27,44	cyclictest	13	-21	ksoftirqd/0	00:05:26	0
1632856	99	108	27,44	cyclictest	13	-21	ksoftirqd/0	00:05:25	0
1632856	99	107	65,34	cyclictest	1671648	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	21:15:34	0
1632856	99	107	1,50	cyclictest	1681949	-21	ssh	21:10:18	0
1632860	99	106	90,8	cyclictest	1752524	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:50:12	1
1632860	99	106	90,8	cyclictest	1634531	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	19:20:36	1
1632860	99	106	90,8	cyclictest	1634531	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	19:20:35	1
1632856	99	106	10,15	cyclictest	1904314	-21	/usr/sbin/munin	00:15:23	0
1632856	99	106	10,15	cyclictest	1904314	-21	/usr/sbin/munin	00:15:22	0
1632869	99	105	91,7	cyclictest	1634531	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	19:50:34	3
1632869	99	105	91,7	cyclictest	1634531	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	19:50:33	3
1632856	99	105	77,24	cyclictest	210	-21	kswapd0	00:10:26	0
1632856	99	105	77,24	cyclictest	210	-21	kswapd0	00:10:25	0
1632869	99	104	92,8	cyclictest	1759996	-21	bingrep	22:15:26	3
1632856	99	103	76,23	cyclictest	210	-21	kswapd0	21:55:27	0
1632865	99	101	92,5	cyclictest	210	-21	kswapd0	22:00:28	2
1632856	99	101	39,32	cyclictest	0	-21	swapper/0	20:55:19	0
1632856	99	101	27,7	cyclictest	14	-21	rcu_preempt	22:00:13	0
1632869	99	100	89,5	cyclictest	1681149	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:32:04	3
1632869	99	100	89,5	cyclictest	1681149	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:32:04	3
1632865	99	100	86,7	cyclictest	1823034	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:47:28	2
1632856	99	100	46,24	cyclictest	1823034	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:35:35	0
1632856	99	100	29,6	cyclictest	1752524	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:40:54	0
1632856	99	100	29,6	cyclictest	1752524	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:40:54	0
1632856	99	100	2,27	cyclictest	0	-21	swapper/0	22:55:24	0
1632869	99	99	87,6	cyclictest	1706698	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	21:38:34	3
1632856	99	98	67,23	cyclictest	1715830	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	21:47:07	0
1632856	99	98	41,14	cyclictest	1917759	-21	gcc	00:25:38	0
1632856	99	98	30,14	cyclictest	689	50	irq/527-eth0-Tx	23:30:12	0
1632856	99	98	30,14	cyclictest	689	50	irq/527-eth0-Tx	23:30:12	0
1632869	99	97	88,6	cyclictest	210	-21	kswapd0	23:35:16	3
1632856	99	97	42,27	cyclictest	15	-21	rcuc/0	23:45:34	0
1632856	99	97	32,14	cyclictest	689	50	irq/527-eth0-Tx	22:14:19	0
1632856	99	97	30,44	cyclictest	0	-21	swapper/0	22:50:25	0
1632878	99	96	90,3	cyclictest	0	-21	swapper/5	22:45:26	5
1632878	99	96	90,3	cyclictest	0	-21	swapper/5	22:45:26	5
1632856	99	96	31,7	cyclictest	13	-21	ksoftirqd/0	22:15:35	0
1632856	99	96	2,50	cyclictest	0	-21	swapper/0	00:30:24	0
1632878	99	95	89,3	cyclictest	0	-21	swapper/5	21:22:00	5
1632869	99	95	82,7	cyclictest	1840356	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	23:22:23	3
1632856	99	95	4,28	cyclictest	0	-21	swapper/0	23:40:24	0
1632856	99	95	35,14	cyclictest	689	50	irq/527-eth0-Tx	20:50:18	0
1632856	99	95	31,15	cyclictest	691	50	irq/529-eth0-Tx	21:30:37	0
1632856	99	95	31,15	cyclictest	691	50	irq/529-eth0-Tx	21:30:37	0
1632869	99	94	67,8	cyclictest	1752524	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:39:21	3
1632869	99	94	67,8	cyclictest	1752524	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:39:20	3
1632860	99	94	80,7	cyclictest	1864510	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:22:40	1
1632856	99	94	62,24	cyclictest	1767391	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	23:27:22	0
1632856	99	93	60,25	cyclictest	1653465	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	20:31:55	0
1632856	99	93	3,63	cyclictest	692	50	irq/530-eth0-Tx	21:20:21	0
1632865	99	92	77,8	cyclictest	1767391	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:50:32	2
1632860	99	92	79,6	cyclictest	1752524	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:26:34	1
1632856	99	92	59,24	cyclictest	1767391	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	23:02:28	0
1632856	99	92	27,30	cyclictest	692	50	irq/530-eth0-Tx	22:20:34	0
1632874	99	91	86,2	cyclictest	0	-21	swapper/4	22:50:12	4
1632874	99	91	81,5	cyclictest	1899096	-21	ssh	00:10:26	4
1632874	99	91	81,5	cyclictest	1899096	-21	ssh	00:10:25	4
1632869	99	91	79,6	cyclictest	1864510	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:26:35	3
1632865	99	91	49,20	cyclictest	1752524	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:14:11	2
1632856	99	91	2,28	cyclictest	1716428	-21	/usr/sbin/munin	21:40:28	0
1632856	99	90	30,34	cyclictest	1681149	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:35:30	0
1632856	99	90	25,4	cyclictest	692	50	irq/530-eth0-Tx	23:20:38	0
1632856	99	89	57,23	cyclictest	1651270	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	19:56:46	0
1632856	99	89	5,18	cyclictest	692	50	irq/530-eth0-Tx	23:05:45	0
1632869	99	88	76,6	cyclictest	1653465	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	20:48:34	3
1632874	99	87	74,8	cyclictest	1880634	-21	ssh	23:55:21	4
1632869	99	87	76,7	cyclictest	210	-21	kswapd0	22:40:24	3
1632869	99	87	76,7	cyclictest	210	-21	kswapd0	22:40:24	3
1632869	99	87	75,6	cyclictest	1767391	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	23:09:34	3
1632856	99	86	55,24	cyclictest	1767391	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:35:15	0
1632856	99	86	55,24	cyclictest	1767391	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:35:15	0
1632856	99	86	46,37	cyclictest	1745060	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:28:20	0
1632878	99	85	79,3	cyclictest	0	-21	swapper/5	21:05:01	5
1632874	99	85	79,3	cyclictest	0	-21	swapper/4	23:02:19	4
1632869	99	85	74,5	cyclictest	1752524	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:23:10	3
1632869	99	85	73,6	cyclictest	1653465	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	20:14:33	3
1632869	99	84	72,6	cyclictest	1816695	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	23:04:24	3
1632869	99	84	72,6	cyclictest	1715830	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	21:47:35	3
1632869	99	84	71,6	cyclictest	1881385	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	00:01:06	3
1632860	99	84	70,7	cyclictest	1873550	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	23:51:10	1
1632856	99	84	44,15	cyclictest	1823034	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:55:19	0
1632874	99	83	78,2	cyclictest	0	-21	swapper/4	23:40:36	4
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17