OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #3, slot #4

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance

Characteristics of the 100 highest latencies:

System rack3slot4.osadl.org (updated Mon Apr 20, 2026 12:46:18)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
222157	2	734	704,12	sleep0	220593	-21	kworker/0:0+events@ dbs_work_handler	07:05:39	0
222157	2	734	704,12	sleep0	220593	-21	kworker/0:0+events@ dbs_work_handler	07:05:39	0
222989	99	128	114,7	cyclictest	271208	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	09:10:25	2
222981	99	126	71,48	cyclictest	255099	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	08:49:35	0
222981	99	121	93,20	cyclictest	289914	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	09:35:43	0
222981	99	114	77,30	cyclictest	371655	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	11:45:16	0
222981	99	114	77,30	cyclictest	371655	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	11:45:16	0
222981	99	113	25,46	cyclictest	273827	-21	kworker/u12:1+rpciod@ rpc_async_schedule	09:15:21	0
222981	99	112	56,32	cyclictest	216523	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	07:11:21	0
222981	99	109	81,20	cyclictest	239029	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	08:06:56	0
222985	99	106	92,7	cyclictest	261399	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	08:55:15	1
222989	99	105	91,7	cyclictest	273827	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	09:15:26	2
222981	99	104	69,21	cyclictest	251735	2	sleep0	08:20:17	0
222981	99	103	2,86	cyclictest	232346	-21	wc	07:30:40	0
222985	99	102	87,8	cyclictest	261399	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	08:50:11	1
222985	99	102	87,8	cyclictest	261399	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	08:50:11	1
222981	99	102	74,20	cyclictest	294475	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	09:45:55	0
222981	99	102	58,36	cyclictest	315578	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	10:30:58	0
222989	99	101	87,7	cyclictest	251799	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	08:25:04	2
222981	99	101	75,19	cyclictest	277776	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	09:25:57	0
222981	99	101	74,20	cyclictest	218556	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	07:25:35	0
222981	99	101	56,22	cyclictest	273827	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	09:13:36	0
222989	99	100	88,6	cyclictest	432330	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	12:36:28	2
222999	99	99	89,5	cyclictest	428055	-21	ssh	12:26:26	4
222985	99	98	84,7	cyclictest	255099	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	09:00:18	1
222994	99	97	84,6	cyclictest	236976	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	07:54:47	3
222989	99	97	85,6	cyclictest	300998	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	10:11:06	2
222989	99	97	83,7	cyclictest	421944	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	12:30:35	2
222981	99	97	44,16	cyclictest	370893	-21	ntpq	11:20:28	0
222981	99	96	74,15	cyclictest	432330	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	12:31:28	0
222981	99	96	45,43	cyclictest	315578	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	10:40:39	0
222989	99	95	83,6	cyclictest	315578	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	11:11:11	2
222981	99	95	1,82	cyclictest	0	-21	swapper/0	12:25:36	0
222989	99	93	79,7	cyclictest	255099	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	09:50:04	2
222981	99	92	70,15	cyclictest	371655	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	11:44:34	0
222981	99	92	56,25	cyclictest	413674	-21	uname	12:10:24	0
222981	99	92	56,25	cyclictest	413674	-21	uname	12:10:24	0
222981	99	92	48,37	cyclictest	354804	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	11:10:40	0
222989	99	91	77,7	cyclictest	218556	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	07:24:36	2
222985	99	91	78,7	cyclictest	315578	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	11:20:36	1
222989	99	90	77,7	cyclictest	224819	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	07:35:34	2
222981	99	90	68,15	cyclictest	287885	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	09:30:41	0
222981	99	90	68,15	cyclictest	287885	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	09:30:41	0
222981	99	90	48,21	cyclictest	0	-21	swapper/0	08:00:33	0
222999	99	89	82,3	cyclictest	0	-21	swapper/4	09:25:37	4
222999	99	89	79,5	cyclictest	333005	-21	ssh	10:36:02	4
222989	99	89	77,6	cyclictest	255099	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	08:40:04	2
222989	99	89	75,7	cyclictest	255099	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	08:30:06	2
222985	99	88	75,7	cyclictest	224819	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	07:15:34	1
222981	99	88	62,19	cyclictest	300998	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	10:50:23	0
222981	99	88	61,20	cyclictest	403646	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	12:20:12	0
222981	99	88	58,22	cyclictest	0	-21	swapper/0	08:15:29	0
222981	99	88	54,24	cyclictest	0	-21	swapper/0	10:55:29	0
222999	99	87	81,3	cyclictest	0	-21	swapper/4	10:15:35	4
222989	99	87	73,7	cyclictest	371655	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	11:40:36	2
222981	99	87	63,15	cyclictest	294475	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	10:15:35	0
222981	99	87	26,37	cyclictest	403646	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	12:00:25	0
222999	99	86	73,7	cyclictest	367591	-21	ssh	11:16:12	4
222989	99	86	72,7	cyclictest	236976	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	07:50:34	2
222981	99	86	63,15	cyclictest	257266	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	08:38:34	0
222981	99	86	44,23	cyclictest	0	-21	swapper/0	07:50:42	0
222981	99	86	37,35	cyclictest	366669	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	11:34:58	0
222981	99	86	37,35	cyclictest	366669	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	11:34:58	0
222989	99	85	71,7	cyclictest	318768	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	10:25:25	2
222989	99	85	71,7	cyclictest	318768	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	10:25:24	2
222989	99	85	71,7	cyclictest	287885	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	09:47:08	2
222981	99	85	59,19	cyclictest	287885	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	09:40:23	0
222981	99	85	42,23	cyclictest	0	-21	swapper/0	09:10:01	0
222981	99	85	42,23	cyclictest	0	-21	swapper/0	09:10:01	0
222999	99	84	77,3	cyclictest	0	-21	swapper/4	07:10:36	4
222989	99	84	70,7	cyclictest	224819	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	07:34:38	2
222989	99	84	70,7	cyclictest	224819	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	07:29:44	2
222989	99	84	69,7	cyclictest	277776	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	09:22:04	2
222981	99	84	65,15	cyclictest	210	-21	kswapd0	12:15:26	0
222981	99	84	59,17	cyclictest	366669	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	11:16:08	0
222989	99	83	68,8	cyclictest	315578	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	10:20:28	2
222989	99	83	68,8	cyclictest	241146	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	07:55:33	2
222981	99	83	49,15	cyclictest	438561	-21	gpgv	12:40:00	0
222981	99	83	45,24	cyclictest	0	-21	swapper/0	07:50:01	0
222999	99	82	71,6	cyclictest	373397	-21	ssh	11:24:00	4
222994	99	82	67,8	cyclictest	388447	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	11:50:44	3
222989	99	82	72,4	cyclictest	239029	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	08:20:03	2
222989	99	82	69,7	cyclictest	388447	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	11:50:35	2
222989	99	82	68,7	cyclictest	403646	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	12:13:34	2
222989	99	82	68,7	cyclictest	403646	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	12:13:34	2
222981	99	82	60,14	cyclictest	300998	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	10:36:03	0
222981	99	82	57,12	cyclictest	0	-21	swapper/0	08:27:16	0
222981	99	82	53,15	cyclictest	0	-21	swapper/0	09:00:17	0
222999	99	81	75,3	cyclictest	0	-21	swapper/4	12:21:25	4
222989	99	81	69,6	cyclictest	300998	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	10:46:10	2
222989	99	81	68,6	cyclictest	371655	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	12:04:43	2
222989	99	81	67,7	cyclictest	388447	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	12:15:38	2
222999	99	80	74,3	cyclictest	0	-21	swapper/4	09:10:40	4
222989	99	80	66,7	cyclictest	300998	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	10:35:38	2
222989	99	80	55,7	cyclictest	371655	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	11:38:56	2
222985	99	80	66,7	cyclictest	315578	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	11:00:34	1
222999	99	79	67,6	cyclictest	274953	-21	diskmemload-4	12:15:32	4
222994	99	79	65,7	cyclictest	371655	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	12:05:23	3
222989	99	79	65,7	cyclictest	224819	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	07:45:28	2
222981	99	79	46,23	cyclictest	691	50	irq/529-eth0-Tx	09:55:13	0
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17