OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #3, slot #4

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance

Characteristics of the 100 highest latencies:

System rack3slot4.osadl.org (updated Fri May 01, 2026 00:46:36)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
302774	2	3527	3507,6	sleep0	301014	-21	kworker/0:2+events@ dbs_work_handler	19:05:40	0
303254	99	170	135,27	cyclictest	351537	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	21:27:40	0
303254	99	154	88,55	cyclictest	351537	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	21:20:08	0
303254	99	154	62,42	cyclictest	543011	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	23:53:23	0
303254	99	152	95,50	cyclictest	296764	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	19:44:35	0
303263	99	143	130,6	cyclictest	543011	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	00:33:05	2
303254	99	142	76,58	cyclictest	597260	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	00:28:13	0
303254	99	142	108,27	cyclictest	430634	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:32:52	0
303254	99	134	96,18	cyclictest	334661	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	20:27:33	0
303254	99	134	96,18	cyclictest	334661	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	20:27:33	0
303254	99	128	74,35	cyclictest	439827	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	22:43:24	0
303254	99	128	67,38	cyclictest	351537	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	21:47:42	0
303254	99	128	67,38	cyclictest	351537	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	21:47:41	0
303254	99	127	74,22	cyclictest	430634	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:18:25	0
303254	99	127	74,22	cyclictest	430634	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:18:25	0
303254	99	125	64,41	cyclictest	325383	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	20:45:29	0
303254	99	125	64,41	cyclictest	325383	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	20:45:29	0
303254	99	125	59,41	cyclictest	523309	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:06:26	0
303254	99	125	59,41	cyclictest	523309	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:06:26	0
303254	99	124	64,36	cyclictest	584595	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:17:51	0
460260	2	123	0,5	sleep1	0	-21	swapper/1	22:36:50	1
303254	99	123	63,36	cyclictest	439827	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:12:32	0
303254	99	122	65,34	cyclictest	415971	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:18:02	0
303254	99	121	65,33	cyclictest	404923	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	21:56:56	0
303254	99	121	61,37	cyclictest	351537	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	21:11:51	0
303254	99	116	92,16	cyclictest	439827	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	22:47:33	0
303278	99	113	106,3	cyclictest	0	-21	swapper/5	21:27:40	5
303254	99	112	54,36	cyclictest	318596	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	19:48:55	0
303254	99	111	88,15	cyclictest	563775	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	00:25:00	0
303254	99	110	87,16	cyclictest	388089	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	21:52:44	0
303254	99	110	51,35	cyclictest	339449	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	20:38:30	0
303272	99	109	97,7	cyclictest	556106	-21	ssh	23:53:01	4
303268	99	109	95,7	cyclictest	456319	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:37:33	3
303254	99	109	65,36	cyclictest	430634	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:51:48	0
303254	99	108	86,15	cyclictest	352221	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	21:32:42	0
303254	99	108	66,35	cyclictest	472637	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	23:01:35	0
303268	99	105	91,7	cyclictest	430634	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:22:31	3
303268	99	105	91,7	cyclictest	371262	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	21:57:45	3
303254	99	104	83,14	cyclictest	308901	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	19:27:27	0
303278	99	102	97,2	cyclictest	0	-21	swapper/5	20:42:37	5
303254	99	102	80,15	cyclictest	296764	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	19:11:35	0
303278	99	101	95,3	cyclictest	0	-21	swapper/5	21:12:39	5
303254	99	101	78,15	cyclictest	506664	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	23:27:55	0
303254	99	101	78,15	cyclictest	506664	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	23:27:55	0
303254	99	101	77,16	cyclictest	371262	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	21:35:54	0
303254	99	101	59,39	cyclictest	523309	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:01:57	0
303254	99	101	59,39	cyclictest	523309	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:01:57	0
303254	99	101	45,34	cyclictest	325383	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	20:18:36	0
303254	99	101	38,40	cyclictest	296764	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	19:35:25	0
303278	99	100	95,2	cyclictest	0	-21	swapper/5	22:07:47	5
303254	99	100	78,15	cyclictest	404923	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:23:34	0
303254	99	99	76,15	cyclictest	439827	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	22:27:22	0
303254	99	99	76,15	cyclictest	439827	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	22:27:21	0
303278	99	98	92,3	cyclictest	0	-21	swapper/5	23:32:58	5
303254	99	98	76,15	cyclictest	543011	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	23:46:23	0
303254	99	98	65,23	cyclictest	690	50	irq/528-eth0-Tx	00:35:33	0
303254	99	98	55,26	cyclictest	15	-21	rcuc/0	21:00:43	0
303268	99	97	83,7	cyclictest	404923	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:09:39	3
303254	99	97	75,15	cyclictest	415971	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:06:06	0
303254	99	97	74,15	cyclictest	371262	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:03:23	0
303254	99	97	19,60	cyclictest	691	50	irq/529-eth0-Tx	21:15:26	0
426392	2	96	0,5	sleep5	0	-21	swapper/5	22:10:18	5
303254	99	96	70,19	cyclictest	334661	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	20:30:37	0
303268	99	95	79,8	cyclictest	472637	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:55:34	3
303254	99	95	62,24	cyclictest	690	50	irq/528-eth0-Tx	00:10:11	0
303254	99	95	62,24	cyclictest	690	50	irq/528-eth0-Tx	00:10:11	0
303263	99	94	81,6	cyclictest	584595	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:17:35	2
303254	99	94	7,54	cyclictest	347478	-21	kworker/0:2+pm@ pm_runtime_work	21:10:01	0
303254	99	94	72,15	cyclictest	430634	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:08:35	0
303254	99	94	53,31	cyclictest	689	50	irq/527-eth0-Tx	23:45:01	0
303254	99	94	2,4	cyclictest	532740	-21	ssh	23:35:01	0
303278	99	93	90,1	cyclictest	0	-21	swapper/5	21:05:34	5
303268	99	93	79,7	cyclictest	347373	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	21:06:35	3
303268	99	93	63,7	cyclictest	506664	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	23:38:06	3
303254	99	93	70,15	cyclictest	325383	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	20:55:40	0
303254	99	93	65,24	cyclictest	513923	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:24:51	0
303254	99	93	65,24	cyclictest	513923	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:24:51	0
303254	99	93	61,22	cyclictest	691	50	irq/529-eth0-Tx	22:55:31	0
303268	99	92	78,8	cyclictest	553744	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	23:53:34	3
303268	99	92	78,7	cyclictest	597260	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	00:31:34	3
303268	99	91	78,7	cyclictest	371262	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:00:19	3
303268	99	91	77,7	cyclictest	334661	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	20:35:22	3
303254	99	91	65,19	cyclictest	523309	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:55:23	0
303254	99	91	65,19	cyclictest	523309	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:55:23	0
303278	99	90	84,2	cyclictest	0	-21	swapper/5	00:19:40	5
303268	99	90	77,6	cyclictest	325383	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	20:09:25	3
303268	99	90	76,7	cyclictest	576941	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	00:15:43	3
303268	99	90	76,6	cyclictest	523309	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:05:35	3
303268	99	90	76,6	cyclictest	523309	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:05:34	3
303263	99	90	59,7	cyclictest	472637	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	23:32:26	2
303254	99	90	69,14	cyclictest	329461	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	20:14:45	0
303254	99	90	50,24	cyclictest	0	-21	swapper/0	22:37:44	0
303278	99	89	84,2	cyclictest	0	-21	swapper/5	22:56:14	5
303278	99	89	84,2	cyclictest	0	-21	swapper/5	00:04:35	5
303278	99	89	84,2	cyclictest	0	-21	swapper/5	00:04:35	5
303268	99	89	76,6	cyclictest	563775	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	00:25:32	3
303268	99	89	74,7	cyclictest	334661	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	20:30:42	3
303263	99	89	76,6	cyclictest	523309	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:41:20	2
303259	99	89	78,5	cyclictest	109909	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	19:17:25	1
303254	99	89	71,15	cyclictest	415971	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:13:54	0
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17