OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #3, slot #4

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance

Characteristics of the 100 highest latencies:

System rack3slot4.osadl.org (updated Tue May 12, 2026 12:46:36)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
2729962	2	3550	3529,7	sleep1	2728715	-21	kworker/1:3+events@ dbs_work_handler	07:05:36	1
2730996	99	127	113,7	cyclictest	2873927	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	12:10:43	3
2730980	99	127	2,109	cyclictest	2756572	-21	timerandwakeup	08:10:41	0
2730980	99	126	55,56	cyclictest	2762845	-21	unixbench_singl	08:25:43	0
2730980	99	124	90,26	cyclictest	2873927	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	12:35:50	0
2730980	99	124	72,39	cyclictest	2744977	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	08:00:33	0
2730980	99	123	1,106	cyclictest	2747031	-21	gpgv	07:50:01	0
2730980	99	122	67,48	cyclictest	2791636	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	09:26:11	0
2730980	99	121	1,104	cyclictest	0	-21	swapper/0	07:10:33	0
2730980	99	120	75,36	cyclictest	689	50	irq/527-eth0-Tx	09:55:14	0
2730980	99	120	71,37	cyclictest	689	50	irq/527-eth0-Tx	11:19:23	0
2730980	99	120	61,38	cyclictest	2992797	-21	/usr/sbin/munin	12:00:46	0
2921548	2	119	0,5	sleep5	0	-21	swapper/5	11:04:59	5
2730980	99	119	75,31	cyclictest	689	50	irq/527-eth0-Tx	09:15:34	0
2730980	99	119	1,72	cyclictest	0	-21	swapper/0	08:45:17	0
2730980	99	118	84,27	cyclictest	2978455	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	12:22:35	0
2730980	99	118	84,27	cyclictest	2978455	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	12:22:34	0
2730980	99	118	72,33	cyclictest	690	50	irq/528-eth0-Tx	07:30:25	0
2730980	99	118	2,73	cyclictest	0	-21	swapper/0	09:35:30	0
2730980	99	118	2,73	cyclictest	0	-21	swapper/0	09:35:30	0
2730980	99	118	1,98	cyclictest	0	-21	swapper/0	09:22:08	0
2730980	99	117	47,48	cyclictest	2771255	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	09:03:35	0
2730980	99	117	2,99	cyclictest	2887695	-21	seq	10:37:18	0
2730980	99	117	2,99	cyclictest	2887695	-21	seq	10:37:18	0
2730980	99	116	65,26	cyclictest	2913067	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	11:27:17	0
2730980	99	116	65,26	cyclictest	2913067	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	11:27:16	0
2730980	99	115	45,46	cyclictest	2829506	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	10:09:11	0
2730991	99	114	99,8	cyclictest	2913067	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	11:10:32	2
2730980	99	114	69,32	cyclictest	690	50	irq/528-eth0-Tx	12:05:35	0
2730980	99	114	55,45	cyclictest	689	50	irq/527-eth0-Tx	07:20:31	0
2730980	99	113	2,96	cyclictest	0	-21	swapper/0	07:55:43	0
2730980	99	113	2,105	cyclictest	13301	-21	runrttasks	08:15:09	0
2731000	99	112	101,6	cyclictest	2865874	-21	ssh	10:20:18	4
2730980	99	112	69,31	cyclictest	690	50	irq/528-eth0-Tx	11:00:20	0
2730980	99	112	68,32	cyclictest	690	50	irq/528-eth0-Tx	10:50:40	0
2730980	99	111	63,37	cyclictest	0	-21	swapper/0	08:05:21	0
2730980	99	110	59,38	cyclictest	0	-21	swapper/0	07:30:01	0
2730980	99	110	58,38	cyclictest	0	-21	swapper/0	11:05:25	0
2730980	99	110	2,97	cyclictest	0	-21	swapper/0	10:02:00	0
2730980	99	110	1,90	cyclictest	15	-21	rcuc/0	08:40:01	0
2730996	99	109	96,7	cyclictest	2863063	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	10:21:34	3
2730980	99	109	62,35	cyclictest	690	50	irq/528-eth0-Tx	10:20:22	0
2730980	99	109	50,43	cyclictest	0	-21	swapper/0	10:28:47	0
2730980	99	108	60,35	cyclictest	690	50	irq/528-eth0-Tx	09:32:43	0
2730980	99	108	60,33	cyclictest	0	-21	swapper/0	09:40:20	0
2730980	99	108	60,33	cyclictest	0	-21	swapper/0	09:40:20	0
2730980	99	108	55,34	cyclictest	0	-21	swapper/0	07:35:29	0
2730980	99	108	55,34	cyclictest	0	-21	swapper/0	07:35:29	0
2730980	99	108	54,36	cyclictest	2779320	-21	diskmemload	12:30:34	0
2730980	99	108	49,27	cyclictest	2929234	-21	ssh	11:10:30	0
2730996	99	107	95,6	cyclictest	2779173	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	09:20:05	3
2730980	99	107	63,35	cyclictest	0	-21	swapper/0	07:15:27	0
2730980	99	105	62,32	cyclictest	689	50	irq/527-eth0-Tx	11:40:34	0
2730980	99	105	60,34	cyclictest	0	-21	swapper/0	10:17:07	0
2730980	99	105	57,37	cyclictest	689	50	irq/527-eth0-Tx	12:19:35	0
2730980	99	105	2,4	cyclictest	0	-21	swapper/0	11:45:37	0
2730980	99	105	2,3	cyclictest	0	-21	swapper/0	07:50:43	0
2730980	99	104	64,30	cyclictest	691	50	irq/529-eth0-Tx	10:30:32	0
2730980	99	104	64,30	cyclictest	691	50	irq/529-eth0-Tx	10:30:32	0
2730980	99	104	50,25	cyclictest	2898004	-21	ntpq	10:45:36	0
2730980	99	104	50,25	cyclictest	2898004	-21	ntpq	10:45:36	0
2730980	99	104	48,26	cyclictest	0	-21	swapper/0	09:50:36	0
2730980	99	104	2,48	cyclictest	13	-21	ksoftirqd/0	12:10:30	0
2730980	99	103	59,33	cyclictest	689	50	irq/527-eth0-Tx	11:50:42	0
2730996	99	102	88,7	cyclictest	2829506	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	10:05:14	3
2730980	99	102	63,30	cyclictest	0	-21	swapper/0	07:40:41	0
2730980	99	102	61,31	cyclictest	691	50	irq/529-eth0-Tx	09:10:00	0
2730980	99	102	61,31	cyclictest	691	50	irq/529-eth0-Tx	09:10:00	0
2730980	99	102	61,13	cyclictest	0	-21	swapper/0	08:20:27	0
2730980	99	102	59,14	cyclictest	689	50	irq/527-eth0-Tx	10:55:35	0
2730980	99	102	58,34	cyclictest	690	50	irq/528-eth0-Tx	09:45:32	0
2730980	99	102	58,34	cyclictest	690	50	irq/528-eth0-Tx	09:45:32	0
2730980	99	102	1,71	cyclictest	0	-21	swapper/0	11:39:17	0
2730980	99	101	66,25	cyclictest	0	-21	swapper/0	11:25:01	0
2730980	99	101	66,25	cyclictest	0	-21	swapper/0	11:25:00	0
2730980	99	101	53,24	cyclictest	2854104	-21	missed_timers	10:10:26	0
2730980	99	101	52,39	cyclictest	0	-21	swapper/0	10:40:33	0
2730980	99	101	36,12	cyclictest	0	-21	swapper/0	08:50:30	0
2730980	99	101	2,7	cyclictest	2954331	-21	ssh	11:30:24	0
2730996	99	100	84,9	cyclictest	2779173	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	09:35:15	3
2730996	99	100	84,9	cyclictest	2779173	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	09:35:15	3
2730991	99	100	86,7	cyclictest	2873927	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	12:32:36	2
2730991	99	100	85,8	cyclictest	2873927	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	11:00:30	2
2730980	99	100	51,23	cyclictest	691	50	irq/529-eth0-Tx	12:25:33	0
2730980	99	100	51,23	cyclictest	691	50	irq/529-eth0-Tx	12:25:33	0
2730980	99	100	49,39	cyclictest	0	-21	swapper/0	08:59:52	0
2730980	99	100	40,27	cyclictest	0	-21	swapper/0	09:10:36	0
2731000	99	99	93,2	cyclictest	0	-21	swapper/4	11:55:41	4
2730980	99	99	44,26	cyclictest	0	-21	swapper/0	08:30:35	0
2730980	99	98	1,4	cyclictest	2985851	-21	ssh	11:55:33	0
2730996	99	97	85,6	cyclictest	2738135	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	07:29:45	3
2730991	99	97	56,9	cyclictest	2771255	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	09:25:41	2
2730996	99	95	81,7	cyclictest	2853952	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	10:15:18	3
2730991	99	95	83,6	cyclictest	2873927	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	10:57:34	2
2730996	99	94	82,6	cyclictest	2922776	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	11:26:35	3
2730996	99	94	82,6	cyclictest	2922776	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	11:26:34	3
2731000	99	93	88,2	cyclictest	0	-21	swapper/4	12:00:43	4
2730996	99	93	81,6	cyclictest	2734876	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	07:19:43	3
2730991	99	93	80,6	cyclictest	2922776	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	11:25:34	2
2730991	99	93	80,6	cyclictest	2922776	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	11:25:33	2
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17