OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #3, slot #4

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance

Characteristics of the 100 highest latencies:

System rack3slot4.osadl.org (updated Sat May 02, 2026 12:46:36)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
3932215	2	1745	1726,10	sleep4	0	-21	swapper/4	07:08:44	4
3932474	99	138	112,19	cyclictest	4130676	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	11:25:50	0
3932474	99	138	112,19	cyclictest	4130676	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	11:25:49	0
3932474	99	136	65,50	cyclictest	4077893	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	10:29:37	0
3932474	99	129	75,46	cyclictest	4188675	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	11:56:25	0
3932483	99	125	111,7	cyclictest	3992234	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	09:40:33	2
3932483	99	125	111,7	cyclictest	3992234	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	09:40:32	2
3932474	99	123	51,64	cyclictest	3938134	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	07:47:35	0
3932474	99	123	51,64	cyclictest	3938134	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	07:47:34	0
3932474	99	123	49,49	cyclictest	4188675	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	12:12:08	0
3932474	99	122	95,20	cyclictest	4166903	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	12:05:55	0
3932483	99	121	106,7	cyclictest	4055542	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	10:25:40	2
3932474	99	120	94,19	cyclictest	3976931	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	09:05:28	0
3932474	99	120	63,49	cyclictest	3976931	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	09:15:04	0
3932474	99	120	56,42	cyclictest	3992234	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	10:10:00	0
3932474	99	118	87,24	cyclictest	4130676	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	11:20:49	0
3932474	99	117	49,43	cyclictest	3982325	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	09:15:01	0
3932474	99	116	89,19	cyclictest	4140101	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	11:54:55	0
3932474	99	116	55,37	cyclictest	4040464	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	10:11:30	0
3932474	99	115	63,44	cyclictest	3976931	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	09:25:01	0
3932474	99	114	87,20	cyclictest	3976931	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	09:00:26	0
3932474	99	114	56,50	cyclictest	4188675	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	12:30:25	0
3932474	99	114	2,77	cyclictest	0	-21	swapper/0	12:20:01	0
3932474	99	114	2,77	cyclictest	0	-21	swapper/0	12:20:00	0
3932492	99	112	106,3	cyclictest	0	-21	swapper/4	11:25:50	4
3932492	99	112	106,3	cyclictest	0	-21	swapper/4	11:25:49	4
3932474	99	111	61,42	cyclictest	4558	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	12:26:55	0
3932474	99	111	47,56	cyclictest	4092203	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	10:46:59	0
3932474	99	111	1,76	cyclictest	0	-21	swapper/0	08:15:19	0
3932474	99	110	80,22	cyclictest	4109664	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	11:03:16	0
3932474	99	110	73,29	cyclictest	3992234	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	09:31:05	0
3932474	99	109	28,39	cyclictest	15	-21	rcuc/0	11:35:22	0
3932474	99	106	51,33	cyclictest	4010467	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	09:39:33	0
3932474	99	106	51,33	cyclictest	4010467	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	09:39:33	0
3932474	99	104	75,21	cyclictest	4109664	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	11:12:52	0
3932474	99	104	68,28	cyclictest	3956608	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	08:21:05	0
3932483	99	103	88,8	cyclictest	4002446	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	09:30:32	2
3932474	99	103	76,20	cyclictest	4055542	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	10:20:39	0
3932474	99	103	76,20	cyclictest	4055542	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	10:20:38	0
3932474	99	103	68,28	cyclictest	3992234	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	09:45:24	0
3932474	99	103	59,31	cyclictest	4068252	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	10:42:39	0
3932474	99	102	60,8	cyclictest	3937795	-21	switchtime	07:20:43	0
3932474	99	102	31,12	cyclictest	0	-21	swapper/0	11:43:12	0
3932474	99	101	65,28	cyclictest	3992234	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	09:50:28	0
3932483	99	100	86,7	cyclictest	3964754	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	08:55:26	2
3932483	99	100	86,7	cyclictest	3964754	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	08:55:25	2
3932483	99	100	83,9	cyclictest	4166903	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	11:42:34	2
3932474	99	100	2,69	cyclictest	689	50	irq/527-eth0-Tx	12:20:34	0
3932474	99	100	2,69	cyclictest	689	50	irq/527-eth0-Tx	12:20:34	0
3932474	99	100	2,67	cyclictest	4158933	2	sleep0	11:31:45	0
3932483	99	99	85,7	cyclictest	4109664	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	11:03:34	2
3932483	99	99	84,8	cyclictest	3992234	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	10:00:36	2
3932479	99	99	86,7	cyclictest	4157010	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	11:35:52	1
3932492	99	98	89,5	cyclictest	8423	-21	ssh	12:05:55	4
3932474	99	98	63,27	cyclictest	3992234	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	09:44:01	0
3932474	99	98	63,27	cyclictest	3992234	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	09:44:00	0
3932483	99	97	65,7	cyclictest	4025928	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	09:48:30	2
3932474	99	97	64,26	cyclictest	3972866	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	08:48:21	0
3932474	99	97	54,22	cyclictest	690	50	irq/528-eth0-Tx	10:17:13	0
3932474	99	97	51,23	cyclictest	0	-21	swapper/0	08:40:33	0
3932492	99	96	81,9	cyclictest	38981	-21	ssh	12:30:37	4
3932474	99	96	71,21	cyclictest	4173899	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	11:46:03	0
3932474	99	96	45,18	cyclictest	4068252	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	11:05:04	0
3932474	99	96	1,58	cyclictest	0	-21	swapper/0	08:30:33	0
3932474	99	95	61,27	cyclictest	3929379	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	07:11:34	0
3932474	99	95	61,27	cyclictest	3929379	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	07:11:34	0
3932474	99	95	55,30	cyclictest	4166903	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	12:04:57	0
3932474	99	95	49,24	cyclictest	0	-21	swapper/0	11:17:51	0
3932492	99	94	88,3	cyclictest	0	-21	swapper/4	09:05:27	4
3932474	99	94	62,28	cyclictest	3938134	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	07:50:34	0
3932474	99	94	59,26	cyclictest	4068252	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	10:37:36	0
3932474	99	94	59,26	cyclictest	4068252	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	10:37:36	0
3932492	99	93	81,6	cyclictest	3984513	-21	diskmemload-4	11:20:49	4
3932483	99	93	64,9	cyclictest	37245	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	12:38:59	2
3932479	99	93	78,8	cyclictest	3964754	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	08:40:23	1
3932483	99	92	62,7	cyclictest	4077893	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	10:52:46	2
3932474	99	92	55,29	cyclictest	3938134	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	07:30:35	0
3932474	99	92	55,29	cyclictest	3938134	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	07:30:35	0
3932483	99	91	77,7	cyclictest	4558	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	12:22:14	2
3932483	99	91	77,7	cyclictest	4558	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	12:22:13	2
3932483	99	91	75,7	cyclictest	4077893	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	10:50:01	2
3932479	99	91	76,8	cyclictest	4040464	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	10:15:33	1
3932474	99	91	62,21	cyclictest	29002	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	12:35:19	0
3932483	99	90	66,8	cyclictest	3976931	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	09:23:36	2
3932474	99	90	63,20	cyclictest	3982325	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	10:00:54	0
3932474	99	90	63,20	cyclictest	3934371	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	07:35:02	0
3932474	99	89	61,20	cyclictest	3954497	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	08:05:34	0
3932474	99	89	53,19	cyclictest	3992234	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	09:59:22	0
3932492	99	88	83,2	cyclictest	0	-21	swapper/4	11:54:55	4
3932483	99	88	73,7	cyclictest	3929379	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	07:10:29	2
3932483	99	88	73,7	cyclictest	3929379	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	07:10:29	2
3932483	99	88	71,8	cyclictest	3982325	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	09:50:24	2
3932474	99	88	56,25	cyclictest	3934371	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	07:25:34	0
3932483	99	87	73,7	cyclictest	4068252	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	10:30:41	2
3932483	99	87	73,7	cyclictest	4068252	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	10:30:41	2
3932483	99	87	73,7	cyclictest	3938134	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	07:40:02	2
3932474	99	87	58,21	cyclictest	3938134	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	08:10:34	0
3932474	99	87	58,21	cyclictest	3938134	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	08:10:34	0
3932474	99	87	52,27	cyclictest	4068252	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	10:50:32	0
3932492	99	86	81,2	cyclictest	0	-21	swapper/4	09:00:27	4
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17