OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #3, slot #4

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance

Characteristics of the 100 highest latencies:

System rack3slot4.osadl.org (updated Sun May 03, 2026 12:46:40)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
2134878	2	2207	2190,8	sleep2	2108047	-21	kworker/2:0+events@ dbs_work_handler	07:05:45	2
2135069	2	288	272,8	sleep0	2134910	-21	kworker/0:0+events@ dbs_work_handler	07:08:21	0
2135359	99	150	111,30	cyclictest	2301206	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	10:55:19	0
2135359	99	150	111,30	cyclictest	2301206	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	10:55:18	0
2135359	99	140	102,29	cyclictest	2389727	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	12:35:44	0
2135359	99	140	102,29	cyclictest	2389727	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	12:35:44	0
2135359	99	133	25,24	cyclictest	0	-21	swapper/0	09:15:14	0
2135374	99	120	107,7	cyclictest	2389727	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	12:25:39	3
2135374	99	120	107,7	cyclictest	2389727	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	12:25:38	3
2135359	99	118	2,52	cyclictest	2259580	-21	/usr/sbin/munin	10:10:42	0
2135359	99	117	60,50	cyclictest	2375310	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	11:45:44	0
2135359	99	117	60,50	cyclictest	2375310	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	11:45:44	0
2135359	99	113	41,48	cyclictest	2315956	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	11:20:27	0
2135359	99	113	1,47	cyclictest	0	-21	swapper/0	11:35:30	0
2135359	99	112	2,50	cyclictest	0	-21	swapper/0	11:40:12	0
2135359	99	111	60,31	cyclictest	13	-21	ksoftirqd/0	10:50:19	0
2135359	99	111	60,31	cyclictest	13	-21	ksoftirqd/0	10:50:18	0
2135359	99	110	66,32	cyclictest	2183797	-21	diskmemload	09:35:20	0
2135359	99	110	54,49	cyclictest	2147173	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	08:17:31	0
2135359	99	110	54,49	cyclictest	2147173	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	08:17:31	0
2135359	99	110	1,57	cyclictest	0	-21	swapper/0	12:30:21	0
2135359	99	110	1,57	cyclictest	0	-21	swapper/0	12:30:21	0
2135359	99	109	75,26	cyclictest	2147173	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	08:04:43	0
2135359	99	109	23,27	cyclictest	13	-21	ksoftirqd/0	11:05:32	0
2135359	99	109	21,25	cyclictest	0	-21	swapper/0	10:23:30	0
2135359	99	108	66,38	cyclictest	210	-21	kswapd0	11:10:26	0
2135359	99	108	2,89	cyclictest	0	-21	swapper/0	12:00:01	0
2135359	99	107	72,26	cyclictest	2348537	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	12:05:36	0
2135359	99	106	63,32	cyclictest	0	-21	swapper/0	08:55:38	0
2135359	99	106	22,27	cyclictest	0	-21	swapper/0	09:10:15	0
2135359	99	105	5,5	cyclictest	691	50	irq/529-eth0-Tx	09:25:00	0
2135359	99	105	2,50	cyclictest	0	-21	swapper/0	10:35:08	0
2135359	99	105	1,46	cyclictest	0	-21	swapper/0	09:25:16	0
2135359	99	104	2,62	cyclictest	0	-21	swapper/0	10:40:22	0
2135359	99	104	15,40	cyclictest	0	-21	swapper/0	12:20:31	0
2135379	99	103	84,14	cyclictest	2257964	-21	diskmemload-4	10:10:09	4
2135359	99	103	66,30	cyclictest	2163472	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	08:30:03	0
2135359	99	103	2,50	cyclictest	0	-21	swapper/0	10:30:19	0
2135359	99	103	2,48	cyclictest	2411521	-21	runrttasks	12:11:34	0
2135359	99	103	21,25	cyclictest	0	-21	swapper/0	11:50:44	0
2135359	99	102	3,86	cyclictest	690	50	irq/528-eth0-Tx	10:25:17	0
2135359	99	102	29,54	cyclictest	2197538	-21	kworker/0:0+pm@ pm_runtime_work	09:45:23	0
2135359	99	102	19,47	cyclictest	13	-21	ksoftirqd/0	10:45:35	0
2135359	99	102	1,49	cyclictest	0	-21	swapper/0	12:25:16	0
2135359	99	102	1,49	cyclictest	0	-21	swapper/0	12:25:16	0
2135374	99	101	87,7	cyclictest	2242770	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	10:25:13	3
2135369	99	101	88,7	cyclictest	2325495	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	11:15:22	2
2135359	99	101	69,24	cyclictest	2179657	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	09:02:16	0
2135359	99	101	69,24	cyclictest	2179657	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	09:02:15	0
2135359	99	101	1,87	cyclictest	0	-21	swapper/0	10:00:34	0
2135359	99	101	16,69	cyclictest	0	-21	swapper/0	10:15:48	0
2135359	99	100	19,16	cyclictest	2116185	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	07:11:51	0
2135359	99	100	15,6	cyclictest	2177638	-21	kworker/u12:2+rpciod@ rpc_async_schedule	09:30:29	0
2135359	99	99	69,22	cyclictest	2256008	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	10:09:34	0
2135359	99	99	44,38	cyclictest	2358806	-21	ssh	11:30:06	0
2135359	99	99	2,49	cyclictest	0	-21	swapper/0	09:50:44	0
2135359	99	99	2,49	cyclictest	0	-21	swapper/0	09:50:43	0
2135359	99	99	2,46	cyclictest	2415221	-21	apt-get	12:15:11	0
2135374	99	98	85,6	cyclictest	2266902	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	10:39:34	3
2135374	99	98	84,7	cyclictest	2375310	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	12:15:39	3
2135359	99	98	2,51	cyclictest	2353759	-21	ssh	11:25:36	0
2135364	99	97	82,8	cyclictest	2294207	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	10:40:18	1
2135359	99	97	8,38	cyclictest	0	-21	swapper/0	11:00:38	0
2135359	99	97	8,38	cyclictest	0	-21	swapper/0	11:00:37	0
2135359	99	97	17,5	cyclictest	2239648	-21	df_abs	09:55:20	0
2135359	99	97	17,5	cyclictest	2239648	-21	df_abs	09:55:19	0
2135359	99	96	66,23	cyclictest	2171463	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	08:44:48	0
2135374	99	95	82,6	cyclictest	2163472	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	08:24:47	3
2135374	99	95	81,7	cyclictest	2375310	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	11:44:09	3
2135359	99	95	62,25	cyclictest	2147247	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	07:39:39	0
2135359	99	95	1,45	cyclictest	0	-21	swapper/0	07:30:36	0
2135359	99	94	38,25	cyclictest	2159162	-21	sed	08:05:43	0
2135385	99	93	87,3	cyclictest	0	-21	swapper/5	10:55:19	5
2135385	99	93	87,3	cyclictest	0	-21	swapper/5	10:55:18	5
2135374	99	93	79,7	cyclictest	2342040	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	11:17:04	3
2135359	99	93	31,14	cyclictest	13	-21	ksoftirqd/0	07:15:35	0
2135359	99	92	2,50	cyclictest	0	-21	swapper/0	12:00:34	0
2135359	99	92	20,27	cyclictest	2150911	-21	timerwakeupswit	07:45:43	0
2135359	99	92	20,27	cyclictest	2150911	-21	timerwakeupswit	07:45:43	0
2135359	99	91	61,27	cyclictest	2218422	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	09:41:35	0
2135369	99	90	78,6	cyclictest	2232679	-21	kworker/u12:4+events_unbound@ flush_memcg_stats_dwork	09:55:06	2
2135369	99	90	78,6	cyclictest	2232679	-21	kworker/u12:4+events_unbound@ flush_memcg_stats_dwork	09:55:05	2
2135359	99	90	22,17	cyclictest	13301	-21	runrttasks	11:20:02	0
2135374	99	89	75,7	cyclictest	2188008	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	09:20:39	3
2135374	99	89	63,8	cyclictest	2232679	-21	kworker/u12:4+events_unbound@ flush_memcg_stats_dwork	10:01:02	3
2135359	99	89	2,55	cyclictest	2143028	-21	chrt	07:26:06	0
2135374	99	88	76,7	cyclictest	210	-21	kswapd0	10:45:16	3
2135374	99	88	74,7	cyclictest	2325495	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	11:05:20	3
2135374	99	88	74,7	cyclictest	2315956	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	11:01:32	3
2135374	99	88	74,7	cyclictest	2315956	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	11:01:31	3
2135374	99	88	74,7	cyclictest	2218422	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	09:45:00	3
2135359	99	88	5,53	cyclictest	2165254	-21	unixbench_multi	08:20:43	0
2257628	2	87	0,3	sleep5	0	-21	swapper/5	10:10:02	5
2135379	99	87	75,7	cyclictest	2288716	-21	ssh	10:34:34	4
2135374	99	87	73,7	cyclictest	2348537	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	11:32:37	3
2135359	99	87	2,45	cyclictest	2151687	-21	/usr/sbin/munin	07:50:42	0
2135385	99	86	78,4	cyclictest	0	-21	swapper/5	12:35:44	5
2135385	99	86	78,4	cyclictest	0	-21	swapper/5	12:35:44	5
2135374	99	86	72,7	cyclictest	2389727	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	11:59:35	3
2135374	99	86	72,7	cyclictest	2179657	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	09:05:22	3
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17