OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #3, slot #4

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance

Characteristics of the 100 highest latencies:

System rack3slot4.osadl.org (updated Mon Jun 01, 2026 12:46:31)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
735647	2	2609	2585,9	sleep0	733485	-21	kworker/0:1+events@ dbs_work_handler	07:08:24	0
735647	2	2609	2585,9	sleep0	733485	-21	kworker/0:1+events@ dbs_work_handler	07:08:24	0
735929	99	137	19,82	cyclictest	745806	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	07:58:29	0
735929	99	137	19,82	cyclictest	745806	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	07:58:28	0
735929	99	134	30,64	cyclictest	782232	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	09:20:59	0
735929	99	132	45,68	cyclictest	919303	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	11:19:46	0
735929	99	131	32,62	cyclictest	857535	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	10:41:15	0
735929	99	123	19,68	cyclictest	737540	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	07:14:45	0
735929	99	120	32,55	cyclictest	737540	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	07:22:28	0
735929	99	117	0,53	cyclictest	0	-21	swapper/0	11:55:32	0
735929	99	115	32,55	cyclictest	776755	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	08:54:19	0
735929	99	115	32,55	cyclictest	776755	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	08:54:18	0
735929	99	114	27,65	cyclictest	776755	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	09:06:21	0
735936	99	111	97,7	cyclictest	819597	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	09:40:46	1
735929	99	110	24,64	cyclictest	0	-21	swapper/0	10:10:30	0
735929	99	110	24,55	cyclictest	761883	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	08:13:31	0
735929	99	108	82,18	cyclictest	819597	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	09:45:48	0
735929	99	107	32,50	cyclictest	774561	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	08:46:35	0
735929	99	105	24,5	cyclictest	985437	-21	aten2_r3power_e	11:45:17	0
735929	99	104	1,7	cyclictest	855436	-21	df_abs	10:05:18	0
735929	99	101	68,12	cyclictest	690	50	irq/528-eth0-Tx	12:10:21	0
735929	99	101	49,27	cyclictest	1013409	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	12:20:28	0
735929	99	101	49,27	cyclictest	1013409	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	12:20:27	0
735929	99	101	31,40	cyclictest	689	50	irq/527-eth0-Tx	09:32:12	0
735929	99	101	20,5	cyclictest	691	50	irq/529-eth0-Tx	11:00:31	0
735929	99	101	19,77	cyclictest	0	-21	swapper/0	12:05:30	0
735929	99	101	12,10	cyclictest	0	-21	swapper/0	12:35:45	0
735929	99	99	1,8	cyclictest	13	-21	ksoftirqd/0	11:10:02	0
735929	99	99	0,27	cyclictest	0	-21	swapper/0	11:40:36	0
735929	99	99	0,27	cyclictest	0	-21	swapper/0	11:40:36	0
735929	99	98	40,18	cyclictest	780458	-21	latency_hist	09:00:01	0
735929	99	98	21,4	cyclictest	0	-21	swapper/0	11:23:32	0
735936	99	97	66,7	cyclictest	847997	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	11:07:51	1
735936	99	96	82,7	cyclictest	755948	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	08:00:44	1
735936	99	96	82,7	cyclictest	737540	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	07:23:28	1
735929	99	96	45,5	cyclictest	0	-21	swapper/0	10:15:45	0
735929	99	96	20,3	cyclictest	0	-21	swapper/0	09:40:19	0
735929	99	96	0,81	cyclictest	0	-21	swapper/0	08:43:27	0
735929	99	96	0,4	cyclictest	973345	-21	ntpq	11:35:28	0
735929	99	96	0,4	cyclictest	973345	-21	ntpq	11:35:28	0
735929	99	96	0,47	cyclictest	0	-21	swapper/0	10:48:10	0
889157	2	95	0,8	sleep0	14	-21	rcu_preempt	10:30:44	0
735929	99	95	0,29	cyclictest	914588	-21	cut	10:50:25	0
735936	99	94	81,6	cyclictest	919303	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	10:56:35	1
735936	99	94	81,6	cyclictest	737540	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	07:17:35	1
735929	99	93	22,5	cyclictest	690	50	irq/528-eth0-Tx	11:50:45	0
735929	99	92	40,37	cyclictest	847786	-21	ssh	09:59:46	0
735929	99	92	0,77	cyclictest	0	-21	swapper/0	07:48:03	0
735929	99	92	0,75	cyclictest	0	-21	swapper/0	08:18:35	0
735955	99	91	85,3	cyclictest	0	-21	swapper/5	08:15:30	5
735929	99	91	0,49	cyclictest	0	-21	swapper/0	07:15:11	0
735929	99	91	0,28	cyclictest	0	-21	swapper/0	09:29:51	0
735949	99	90	84,3	cyclictest	0	-21	swapper/4	09:45:48	4
735929	99	90	0,78	cyclictest	0	-21	swapper/0	07:44:17	0
735929	99	90	0,78	cyclictest	0	-21	swapper/0	07:44:17	0
735929	99	90	0,47	cyclictest	921487	-21	grep	10:55:31	0
735945	99	89	79,7	cyclictest	210	-21	kswapd0	10:40:27	3
735936	99	89	75,7	cyclictest	919303	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	11:11:30	1
735929	99	89	66,15	cyclictest	962084	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	11:29:14	0
735929	99	89	64,18	cyclictest	755948	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	08:05:23	0
735929	99	89	64,18	cyclictest	755948	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	08:05:23	0
735929	99	89	64,17	cyclictest	833497	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	09:50:47	0
735929	99	89	17,6	cyclictest	691	50	irq/529-eth0-Tx	12:31:25	0
735929	99	89	13,39	cyclictest	737540	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	07:25:40	0
735929	99	89	0,76	cyclictest	0	-21	swapper/0	09:01:15	0
735929	99	89	0,52	cyclictest	0	-21	swapper/0	08:01:52	0
884445	2	88	0,6	sleep4	884440	-21	ssh	10:27:10	4
735940	99	88	74,7	cyclictest	819597	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	09:40:06	2
735929	99	88	0,53	cyclictest	0	-21	swapper/0	07:34:13	0
735929	99	88	0,51	cyclictest	1024274	-21	aten2_r3power_f	12:15:16	0
735936	99	87	74,6	cyclictest	782232	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	09:32:07	1
735929	99	87	68,15	cyclictest	210	-21	kswapd0	11:10:16	0
735929	99	87	0,4	cyclictest	0	-21	swapper/0	08:33:03	0
735936	99	86	72,7	cyclictest	929127	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	12:06:02	1
735936	99	86	72,7	cyclictest	929127	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	11:02:05	1
735936	99	86	72,7	cyclictest	776755	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	09:08:05	1
735929	99	86	55,13	cyclictest	895426	-21	ssh	10:35:37	0
735929	99	86	1,26	cyclictest	804094	-21	kworker/u12:2+rpciod@ rpc_async_schedule	10:25:00	0
735929	99	86	12,24	cyclictest	13	-21	ksoftirqd/0	09:10:20	0
735929	99	86	0,74	cyclictest	0	-21	swapper/0	08:21:31	0
735929	99	86	0,74	cyclictest	0	-21	swapper/0	08:21:30	0
735929	99	86	0,4	cyclictest	0	-21	swapper/0	07:37:40	0
735940	99	85	73,6	cyclictest	847997	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	10:10:50	2
735936	99	85	73,6	cyclictest	766439	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	08:30:26	1
735929	99	85	24,6	cyclictest	0	-21	swapper/0	12:05:03	0
735929	99	85	0,76	cyclictest	0	-21	swapper/0	08:36:03	0
735929	99	85	0,72	cyclictest	0	-21	swapper/0	10:00:59	0
735940	99	84	72,6	cyclictest	929127	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	12:16:13	2
735936	99	84	70,7	cyclictest	747852	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	07:51:02	1
735929	99	84	59,16	cyclictest	776755	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	09:36:41	0
910731	2	83	0,4	sleep3	0	-21	swapper/3	10:47:22	3
735936	99	83	71,6	cyclictest	804094	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	10:20:09	1
735936	99	83	71,6	cyclictest	776755	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	09:29:54	1
735945	99	82	69,7	cyclictest	997194	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	11:56:10	3
735945	99	82	67,8	cyclictest	776755	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	08:55:00	3
735945	99	82	67,8	cyclictest	776755	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	08:55:00	3
735936	99	82	69,6	cyclictest	919303	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	11:22:37	1
735936	99	82	69,6	cyclictest	804094	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	10:09:26	1
735936	99	82	68,7	cyclictest	857535	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	10:17:12	1
735936	99	82	68,7	cyclictest	1013409	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	12:20:06	1
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17