OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #3, slot #4

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance

Characteristics of the 100 highest latencies:

System rack3slot4.osadl.org (updated Sat May 23, 2026 12:46:38)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
4065903	2	3164	3139,15	sleep4	0	-21	swapper/4	07:05:29	4
4065765	2	703	0,13	sleep5	0	-21	swapper/5	07:05:23	5
4067014	99	116	110,3	cyclictest	0	-21	swapper/5	10:31:10	5
4067014	99	116	110,3	cyclictest	0	-21	swapper/5	10:31:09	5
4067014	99	113	108,2	cyclictest	0	-21	swapper/5	10:41:12	5
4066992	99	108	29,7	cyclictest	13	-21	ksoftirqd/0	07:10:01	0
4067014	99	106	100,3	cyclictest	0	-21	swapper/5	11:36:24	5
4067014	99	106	100,3	cyclictest	0	-21	swapper/5	11:36:24	5
4066999	99	105	93,6	cyclictest	4192440	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	10:21:11	2
4067014	99	103	97,3	cyclictest	0	-21	swapper/5	10:59:34	5
4067014	99	102	95,3	cyclictest	0	-21	swapper/5	12:11:28	5
4067014	99	102	95,3	cyclictest	0	-21	swapper/5	09:46:04	5
4066996	99	100	87,6	cyclictest	4091645	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	08:10:46	1
137876	2	100	0,6	sleep2	0	-21	swapper/2	11:57:11	2
4177581	2	96	0,13	sleep3	37	-21	ksoftirqd/3	09:59:07	3
4066999	99	96	75,8	cyclictest	159177	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	12:15:33	2
4067014	99	94	89,3	cyclictest	0	-21	swapper/5	07:45:15	5
4067014	99	94	89,3	cyclictest	0	-21	swapper/5	07:45:14	5
4067014	99	94	88,3	cyclictest	0	-21	swapper/5	08:14:34	5
4067014	99	94	87,3	cyclictest	0	-21	swapper/5	10:16:10	5
4066999	99	93	64,17	cyclictest	4141147	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	09:55:38	2
4066999	99	92	77,8	cyclictest	97715	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	11:47:35	2
4066992	99	92	43,4	cyclictest	689	50	irq/527-eth0-Tx	11:05:43	0
4066999	99	90	69,8	cyclictest	4120753	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	09:17:13	2
4066992	99	90	66,20	cyclictest	210	-21	kswapd0	11:25:12	0
4067014	99	89	82,3	cyclictest	0	-21	swapper/5	11:43:36	5
4067014	99	89	82,3	cyclictest	0	-21	swapper/5	11:43:35	5
4067004	99	89	75,8	cyclictest	125525	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	12:00:06	3
4067004	99	89	42,17	cyclictest	97715	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	11:42:00	3
4067004	99	89	42,17	cyclictest	97715	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	11:42:00	3
4066999	99	88	72,9	cyclictest	125525	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	11:50:35	2
4066992	99	88	67,14	cyclictest	167218	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	12:26:54	0
4066992	99	88	67,14	cyclictest	167218	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	12:26:54	0
4067014	99	87	80,3	cyclictest	0	-21	swapper/5	08:25:51	5
4067014	99	87	79,4	cyclictest	0	-21	swapper/5	12:28:37	5
4067014	99	87	79,4	cyclictest	0	-21	swapper/5	12:28:37	5
4066996	99	87	75,6	cyclictest	4141147	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	10:25:11	1
4066996	99	87	75,6	cyclictest	4141147	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	10:25:11	1
4067014	99	86	71,10	cyclictest	4085007	-21	chrt	07:54:46	5
4067004	99	86	75,5	cyclictest	4134004	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	10:06:08	3
4067004	99	86	74,6	cyclictest	4109807	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	08:55:57	3
4066999	99	86	75,5	cyclictest	111044	-21	kworker/u12:4+events_unbound@ flush_memcg_stats_dwork	11:41:24	2
4066999	99	86	75,5	cyclictest	111044	-21	kworker/u12:4+events_unbound@ flush_memcg_stats_dwork	11:41:24	2
4066999	99	86	72,7	cyclictest	87442	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	11:24:43	2
4067014	99	85	78,3	cyclictest	0	-21	swapper/5	10:50:33	5
4067014	99	85	77,4	cyclictest	0	-21	swapper/5	09:53:32	5
4067014	99	85	77,4	cyclictest	0	-21	swapper/5	09:53:32	5
4066999	99	85	70,8	cyclictest	4134004	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	09:50:34	2
4066999	99	85	70,8	cyclictest	4134004	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	09:50:34	2
4066992	99	85	5,6	cyclictest	689	50	irq/527-eth0-Tx	10:05:20	0
4067014	99	84	78,3	cyclictest	0	-21	swapper/5	11:25:14	5
4067014	99	84	78,3	cyclictest	0	-21	swapper/5	11:10:10	5
4067014	99	84	78,3	cyclictest	0	-21	swapper/5	07:20:17	5
4066999	99	84	71,6	cyclictest	4192440	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	10:18:35	2
4066999	99	84	56,16	cyclictest	20271	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	11:10:21	2
4066992	99	84	69,8	cyclictest	20271	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	11:15:34	0
4066992	99	84	68,4	cyclictest	0	-21	swapper/0	10:55:19	0
4067014	99	83	77,3	cyclictest	0	-21	swapper/5	09:16:56	5
4067014	99	83	76,3	cyclictest	0	-21	swapper/5	09:09:11	5
4067009	99	83	70,8	cyclictest	52342	-21	ssh	10:51:13	4
4066999	99	83	69,7	cyclictest	4120753	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	09:25:34	2
4066996	99	83	69,7	cyclictest	4134004	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	09:36:02	1
106522	2	83	0,6	sleep2	30	-21	ksoftirqd/2	11:33:26	2
106522	2	83	0,6	sleep2	30	-21	ksoftirqd/2	11:33:25	2
4067004	99	82	67,8	cyclictest	4141147	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	10:00:43	3
4066999	99	82	67,8	cyclictest	4076706	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	07:35:14	2
4066996	99	82	70,6	cyclictest	111044	-21	kworker/u12:4+events_unbound@ flush_memcg_stats_dwork	11:37:10	1
4066996	99	82	70,6	cyclictest	111044	-21	kworker/u12:4+events_unbound@ flush_memcg_stats_dwork	11:37:09	1
4067014	99	81	74,3	cyclictest	0	-21	swapper/5	11:53:51	5
4067009	99	81	75,3	cyclictest	0	-21	swapper/4	07:50:31	4
4066999	99	81	67,7	cyclictest	159177	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	12:25:06	2
4066999	99	81	67,7	cyclictest	159177	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	12:25:05	2
4066999	99	81	66,8	cyclictest	4062451	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	07:10:14	2
4066999	99	81	66,8	cyclictest	4062451	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	07:10:13	2
4066999	99	81	37,14	cyclictest	13752	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	10:34:04	2
4066999	99	81	37,14	cyclictest	13752	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	10:34:04	2
4066992	99	81	7,25	cyclictest	0	-21	swapper/0	10:50:31	0
4067014	99	80	74,3	cyclictest	0	-21	swapper/5	11:00:23	5
4067014	99	80	73,4	cyclictest	0	-21	swapper/5	12:31:15	5
4067009	99	80	75,2	cyclictest	0	-21	swapper/4	09:00:58	4
4066999	99	80	66,7	cyclictest	4113115	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	09:11:04	2
4066999	99	80	65,8	cyclictest	4192440	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	10:25:29	2
4066999	99	80	65,8	cyclictest	4192440	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	10:25:29	2
4066992	99	80	31,3	cyclictest	692	50	irq/530-eth0-Tx	10:35:15	0
4066992	99	80	28,4	cyclictest	690	50	irq/528-eth0-Tx	10:40:39	0
4066992	99	80	11,5	cyclictest	161442	-21	uname	12:15:24	0
4067014	99	79	72,3	cyclictest	0	-21	swapper/5	11:15:50	5
4067014	99	79	72,3	cyclictest	0	-21	swapper/5	11:05:23	5
4067004	99	79	58,8	cyclictest	4113898	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	09:15:38	3
4066999	99	79	66,7	cyclictest	125525	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	12:01:26	2
4066992	99	79	35,3	cyclictest	689	50	irq/527-eth0-Tx	12:25:00	0
4066992	99	79	23,45	cyclictest	125525	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	12:11:28	0
4067014	99	78	72,3	cyclictest	0	-21	swapper/5	08:05:34	5
4067004	99	78	66,6	cyclictest	4082778	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	08:50:55	3
4066999	99	78	65,6	cyclictest	167218	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	12:20:20	2
4066999	99	78	64,7	cyclictest	4141147	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	09:30:38	2
4066999	99	78	63,8	cyclictest	49079	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	11:25:38	2
4066992	99	78	68,7	cyclictest	4134004	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	09:26:00	0
4066992	99	78	37,4	cyclictest	691	50	irq/529-eth0-Tx	11:50:37	0
4066992	99	78	14,28	cyclictest	691	50	irq/529-eth0-Tx	12:35:25	0
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17