OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #3, slot #4

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance

Characteristics of the 100 highest latencies:

System rack3slot4.osadl.org (updated Wed May 20, 2026 12:46:26)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
963532	2	2992	2983,4	sleep5	0	-21	swapper/5	07:05:49	5
964008	99	153	117,28	cyclictest	1166402	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	11:08:33	0
964008	99	146	70,51	cyclictest	1073102	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	10:21:16	0
964008	99	145	108,29	cyclictest	1127733	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	10:43:27	0
964008	99	145	108,29	cyclictest	1127733	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	10:43:27	0
964008	99	140	107,26	cyclictest	1270939	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	12:38:54	0
964008	99	135	80,48	cyclictest	1192458	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	11:29:25	0
964008	99	134	63,48	cyclictest	968354	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	07:24:26	0
964008	99	132	64,44	cyclictest	1073102	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	10:02:02	0
964008	99	131	78,46	cyclictest	971672	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	07:31:34	0
964008	99	131	78,46	cyclictest	971672	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	07:31:33	0
964008	99	131	57,49	cyclictest	1192458	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	11:32:57	0
964008	99	130	75,48	cyclictest	1127733	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	10:46:49	0
964008	99	130	57,49	cyclictest	1010883	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	09:21:06	0
964008	99	129	86,26	cyclictest	1183918	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	11:35:04	0
964008	99	129	56,48	cyclictest	1176632	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	11:18:09	0
964008	99	128	2,110	cyclictest	0	-21	swapper/0	09:05:41	0
964008	99	127	55,48	cyclictest	1222546	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	12:24:29	0
964008	99	127	55,48	cyclictest	1222546	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	12:24:29	0
964008	99	126	59,44	cyclictest	1073102	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	10:11:14	0
964008	99	124	91,26	cyclictest	1048058	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	09:58:19	0
964008	99	124	91,26	cyclictest	1048058	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	09:58:19	0
964008	99	124	53,51	cyclictest	1004179	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	08:48:21	0
964008	99	124	50,48	cyclictest	1232567	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	12:34:12	0
964008	99	121	87,26	cyclictest	1073102	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	10:30:50	0
964008	99	121	51,47	cyclictest	1004179	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	08:52:53	0
964008	99	120	67,46	cyclictest	1119239	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	10:38:15	0
964008	99	120	67,46	cyclictest	1119239	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	10:38:15	0
964008	99	120	64,49	cyclictest	1054724	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	09:47:16	0
964008	99	119	49,47	cyclictest	959734	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	07:12:07	0
964008	99	118	84,27	cyclictest	1222546	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	12:13:49	0
964012	99	116	103,7	cyclictest	1111505	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	10:33:24	1
964008	99	116	1,68	cyclictest	0	-21	swapper/0	11:58:04	0
964008	99	116	1,102	cyclictest	0	-21	swapper/0	08:55:29	0
964030	99	114	106,4	cyclictest	0	-21	swapper/5	11:08:33	5
964008	99	114	78,28	cyclictest	1062954	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	09:53:35	0
964008	99	114	78,28	cyclictest	1062954	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	09:53:35	0
964008	99	114	78,23	cyclictest	1036835	-21	latency_hist	09:30:01	0
964008	99	114	73,34	cyclictest	990399	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	08:26:35	0
964008	99	114	65,36	cyclictest	1183918	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	11:23:00	0
964008	99	114	62,43	cyclictest	980395	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	08:11:54	0
964008	99	114	3,98	cyclictest	0	-21	swapper/0	11:10:22	0
964008	99	112	77,27	cyclictest	1145935	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	10:55:35	0
964008	99	112	44,47	cyclictest	978003	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	07:48:58	0
964008	99	111	77,26	cyclictest	1192458	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	12:06:41	0
964008	99	111	67,35	cyclictest	0	-21	swapper/0	08:00:31	0
964008	99	110	70,31	cyclictest	0	-21	swapper/0	08:05:13	0
964008	99	110	1,99	cyclictest	0	-21	swapper/0	07:40:35	0
964008	99	109	75,26	cyclictest	1082433	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	10:08:14	0
964008	99	108	75,26	cyclictest	998312	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	09:40:27	0
964008	99	108	2,83	cyclictest	13	-21	ksoftirqd/0	11:50:14	0
964008	99	108	2,83	cyclictest	13	-21	ksoftirqd/0	11:50:14	0
964008	99	108	1,84	cyclictest	0	-21	swapper/0	09:35:45	0
964008	99	108	1,69	cyclictest	0	-21	swapper/0	10:50:21	0
964008	99	107	66,29	cyclictest	0	-21	swapper/0	09:15:38	0
964008	99	106	2,90	cyclictest	0	-21	swapper/0	10:25:27	0
964012	99	105	90,7	cyclictest	1073102	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	10:23:24	1
964008	99	105	48,50	cyclictest	1222546	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	12:00:30	0
964008	99	105	44,26	cyclictest	0	-21	swapper/0	09:10:29	0
964030	99	104	97,3	cyclictest	0	-21	swapper/5	10:43:27	5
964030	99	104	97,3	cyclictest	0	-21	swapper/5	10:43:27	5
964008	99	104	56,34	cyclictest	1010883	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	09:30:18	0
964030	99	103	96,3	cyclictest	0	-21	swapper/5	12:38:54	5
964008	99	103	49,39	cyclictest	1097503	-21	ssh	10:15:31	0
964008	99	102	68,27	cyclictest	1270939	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	12:29:58	0
964008	99	102	68,26	cyclictest	961826	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	07:17:35	0
964008	99	102	68,26	cyclictest	961826	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	07:17:34	0
964008	99	102	62,31	cyclictest	689	50	irq/527-eth0-Tx	07:50:23	0
964021	99	101	87,8	cyclictest	1222546	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	12:24:12	3
964021	99	101	87,8	cyclictest	1222546	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	12:24:11	3
964008	99	101	49,34	cyclictest	0	-21	swapper/0	11:00:19	0
964008	99	101	45,38	cyclictest	0	-21	swapper/0	08:15:19	0
964008	99	101	43,51	cyclictest	998312	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	08:35:12	0
964008	99	100	66,27	cyclictest	1166402	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	11:40:49	0
964017	99	99	85,7	cyclictest	1192458	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	11:30:45	2
964008	99	99	66,26	cyclictest	965910	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	07:28:00	0
964008	99	99	64,27	cyclictest	1210706	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	11:45:29	0
964008	99	99	64,27	cyclictest	1210706	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	11:45:28	0
964012	99	98	84,7	cyclictest	1192458	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	11:30:17	1
964008	99	98	38,26	cyclictest	0	-21	swapper/0	09:00:28	0
964026	99	97	91,3	cyclictest	0	-21	swapper/4	12:33:52	4
964008	99	97	62,28	cyclictest	1192458	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	12:18:44	0
964008	99	97	31,14	cyclictest	0	-21	swapper/0	07:35:20	0
964012	99	96	82,7	cyclictest	1232567	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	12:23:12	1
964012	99	96	82,7	cyclictest	1232567	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	12:23:11	1
964008	99	95	44,15	cyclictest	990399	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	08:30:21	0
964012	99	94	80,7	cyclictest	961826	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	07:15:34	1
964012	99	94	80,7	cyclictest	961826	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	07:15:33	1
964008	99	94	61,26	cyclictest	998312	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	08:43:20	0
964008	99	94	60,27	cyclictest	980395	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	07:56:17	0
964021	99	90	76,7	cyclictest	1082433	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	10:08:22	3
964012	99	89	75,7	cyclictest	1119239	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	11:00:40	1
964012	99	88	74,7	cyclictest	1082433	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	10:07:36	1
964012	99	88	74,7	cyclictest	1073102	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	10:35:15	1
964012	99	88	74,7	cyclictest	1073102	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	10:35:15	1
964030	99	87	80,3	cyclictest	0	-21	swapper/5	09:58:19	5
964030	99	87	80,3	cyclictest	0	-21	swapper/5	09:58:19	5
964012	99	87	73,7	cyclictest	1232567	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	12:40:02	1
964012	99	87	73,7	cyclictest	1232567	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	12:06:24	1
964012	99	86	74,6	cyclictest	1010883	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	09:43:15	1
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17