OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #3, slot #4

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance

Characteristics of the 100 highest latencies:

System rack3slot4.osadl.org (updated Sun Apr 12, 2026 00:46:32)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
544353	2	3262	3245,7	sleep1	542421	-21	kworker/1:1+events@ dbs_work_handler	19:05:41	1
543186	2	2959	2941,6	sleep2	525541	-21	kworker/2:2+events@ dbs_work_handler	19:05:18	2
544444	2	1990	1966,15	sleep4	0	-21	swapper/4	19:06:45	4
544869	99	133	123,5	cyclictest	838788	-21	ssh	00:18:22	4
544851	99	127	74,34	cyclictest	567364	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	20:09:27	0
544851	99	122	72,39	cyclictest	689	50	irq/527-eth0-Tx	00:25:29	0
544875	99	121	115,3	cyclictest	0	-21	swapper/5	00:18:21	5
544851	99	121	62,35	cyclictest	13300	-21	snmpd	23:23:42	0
544851	99	120	28,74	cyclictest	567364	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:01:20	0
544851	99	119	97,15	cyclictest	572959	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	20:17:41	0
544851	99	117	77,31	cyclictest	690	50	irq/528-eth0-Tx	21:50:34	0
544851	99	117	71,36	cyclictest	689	50	irq/527-eth0-Tx	20:55:42	0
544851	99	117	71,36	cyclictest	689	50	irq/527-eth0-Tx	20:55:42	0
544851	99	116	2,5	cyclictest	683080	-21	sendmail-msp	22:20:01	0
544851	99	115	58,35	cyclictest	766063	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:33:42	0
544851	99	114	70,22	cyclictest	800372	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:11:42	0
544851	99	114	70,22	cyclictest	800372	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:11:42	0
544856	99	113	99,7	cyclictest	591001	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	21:07:54	1
544851	99	113	76,27	cyclictest	691	50	irq/529-eth0-Tx	22:30:31	0
544851	99	113	2,4	cyclictest	0	-21	swapper/0	22:50:01	0
544851	99	113	2,4	cyclictest	0	-21	swapper/0	22:50:00	0
544851	99	112	3,95	cyclictest	0	-21	swapper/0	23:50:25	0
544851	99	111	69,32	cyclictest	691	50	irq/529-eth0-Tx	23:55:26	0
544851	99	110	61,41	cyclictest	558685	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	20:02:19	0
544851	99	110	60,33	cyclictest	15	-21	rcuc/0	19:40:23	0
544851	99	110	1,98	cyclictest	0	-21	swapper/0	19:10:31	0
544851	99	109	63,22	cyclictest	743961	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	23:16:02	0
544851	99	109	1,80	cyclictest	0	-21	swapper/0	21:30:42	0
544865	99	108	95,6	cyclictest	668288	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:18:03	3
544860	99	108	91,7	cyclictest	591001	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	21:12:52	2
544851	99	108	60,23	cyclictest	792771	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	00:02:08	0
544851	99	108	2,82	cyclictest	0	-21	swapper/0	20:20:17	0
544851	99	107	70,24	cyclictest	689	50	irq/527-eth0-Tx	00:10:01	0
544851	99	107	70,24	cyclictest	689	50	irq/527-eth0-Tx	00:10:00	0
544851	99	107	2,91	cyclictest	0	-21	swapper/0	00:20:17	0
544856	99	106	91,8	cyclictest	600756	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	21:32:58	1
544851	99	106	75,23	cyclictest	591001	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	21:22:58	0
544851	99	106	75,23	cyclictest	591001	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	21:22:57	0
544851	99	106	58,31	cyclictest	0	-21	swapper/0	20:25:41	0
544851	99	106	58,31	cyclictest	0	-21	swapper/0	20:25:41	0
544851	99	106	48,33	cyclictest	593297	-21	diskmemload	21:15:18	0
544851	99	106	2,75	cyclictest	0	-21	swapper/0	21:05:30	0
544851	99	105	2,80	cyclictest	0	-21	swapper/0	21:40:37	0
544865	99	104	89,8	cyclictest	564969	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	20:22:46	3
544851	99	104	67,28	cyclictest	0	-21	swapper/0	23:14:10	0
544851	99	104	56,37	cyclictest	689	50	irq/527-eth0-Tx	00:35:19	0
544851	99	104	1,87	cyclictest	0	-21	swapper/0	19:50:24	0
544860	99	103	87,8	cyclictest	633193	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	22:04:33	2
544851	99	103	74,15	cyclictest	552810	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	19:30:37	0
544851	99	103	74,15	cyclictest	552810	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	19:30:37	0
544851	99	103	65,30	cyclictest	633193	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:01:11	0
544851	99	103	65,15	cyclictest	580918	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	20:47:28	0
544851	99	103	63,30	cyclictest	689	50	irq/527-eth0-Tx	22:44:43	0
544851	99	103	43,34	cyclictest	0	-21	swapper/0	22:58:50	0
544851	99	103	36,50	cyclictest	600756	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	21:25:45	0
544851	99	103	36,50	cyclictest	600756	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	21:25:44	0
544851	99	103	35,46	cyclictest	13	-21	ksoftirqd/0	23:05:18	0
544851	99	103	35,46	cyclictest	13	-21	ksoftirqd/0	23:05:17	0
544851	99	102	72,22	cyclictest	610595	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:06:01	0
544851	99	102	63,25	cyclictest	0	-21	swapper/0	23:35:10	0
544851	99	102	51,35	cyclictest	210	-21	kswapd0	23:40:29	0
544851	99	102	51,35	cyclictest	210	-21	kswapd0	23:40:28	0
544851	99	102	50,30	cyclictest	0	-21	swapper/0	20:12:49	0
544865	99	101	89,6	cyclictest	844144	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:38:24	3
544865	99	101	87,7	cyclictest	623084	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	21:35:46	3
544856	99	101	87,7	cyclictest	600756	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	21:29:35	1
544856	99	101	87,7	cyclictest	600756	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	21:29:35	1
544851	99	101	67,26	cyclictest	552810	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	19:56:22	0
544851	99	101	59,24	cyclictest	558685	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	19:48:17	0
544851	99	101	56,30	cyclictest	0	-21	swapper/0	19:25:35	0
544851	99	101	2,86	cyclictest	0	-21	swapper/0	19:20:29	0
544875	99	100	94,3	cyclictest	0	-21	swapper/5	20:07:41	5
544865	99	100	87,6	cyclictest	552810	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	19:52:38	3
544851	99	100	43,32	cyclictest	0	-21	swapper/0	22:25:12	0
544851	99	100	2,93	cyclictest	48	-21	rcuop/4	00:34:46	0
544851	99	100	2,5	cyclictest	673240	-21	seq	22:11:42	0
544851	99	100	2,4	cyclictest	15	-21	rcuc/0	21:45:22	0
544851	99	99	59,29	cyclictest	0	-21	swapper/0	23:48:33	0
544851	99	99	59,29	cyclictest	0	-21	swapper/0	23:48:32	0
544851	99	99	56,25	cyclictest	0	-21	swapper/0	19:35:19	0
544851	99	99	2,83	cyclictest	0	-21	swapper/0	20:45:01	0
544851	99	99	2,83	cyclictest	0	-21	swapper/0	20:45:00	0
544851	99	98	68,20	cyclictest	689	50	irq/527-eth0-Tx	20:50:30	0
544851	99	98	67,23	cyclictest	623084	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	21:36:12	0
544851	99	98	49,31	cyclictest	0	-21	swapper/0	21:14:22	0
544865	99	97	85,6	cyclictest	758976	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	23:48:17	3
544865	99	97	85,6	cyclictest	758976	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	23:48:16	3
544865	99	97	83,7	cyclictest	564969	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	20:10:13	3
544856	99	97	66,7	cyclictest	792771	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	00:04:09	1
544856	99	97	62,11	cyclictest	633193	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	22:10:31	1
544851	99	97	78,14	cyclictest	142	50	irq/98-imx_mu_c	19:15:21	0
544865	99	96	83,7	cyclictest	567364	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:02:52	3
544865	99	96	82,8	cyclictest	610595	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	21:27:58	3
544865	99	96	82,8	cyclictest	610595	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	21:27:58	3
544851	99	96	53,29	cyclictest	0	-21	swapper/0	20:30:25	0
544851	99	96	42,39	cyclictest	0	-21	swapper/0	22:35:22	0
544869	99	95	80,10	cyclictest	596748	-21	diskmemload-4	23:15:18	4
544851	99	95	52,31	cyclictest	690	50	irq/528-eth0-Tx	22:00:12	0
544851	99	95	50,32	cyclictest	0	-21	swapper/0	21:55:35	0
544851	99	95	44,30	cyclictest	0	-21	swapper/0	20:35:44	0
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17