OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #3, slot #4

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance

Characteristics of the 100 highest latencies:

System rack3slot4.osadl.org (updated Tue Apr 28, 2026 12:46:37)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
2732919	2	3570	3550,10	sleep4	0	-21	swapper/4	07:06:47	4
2732142	2	3479	3451,11	sleep3	2727191	-21	kworker/3:0+events@ dbs_work_handler	07:05:27	3
2732775	2	1180	1162,10	sleep5	0	-21	swapper/5	07:05:41	5
2733322	99	144	37,72	cyclictest	2949728	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	11:44:27	0
2934081	2	129	0,4	sleep5	0	-21	swapper/5	11:11:10	5
2733322	99	115	28,51	cyclictest	2747790	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	08:12:54	0
2733332	99	110	97,6	cyclictest	2775513	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	09:04:15	2
2733322	99	105	75,20	cyclictest	689	50	irq/527-eth0-Tx	09:40:35	0
2733322	99	104	50,36	cyclictest	0	-21	swapper/0	11:45:45	0
2733322	99	102	36,45	cyclictest	0	-21	swapper/0	11:25:01	0
2733322	99	102	36,45	cyclictest	0	-21	swapper/0	11:25:01	0
2733322	99	102	2,66	cyclictest	2826549	-1	kworker/u13:2+xprtiod@ xs_stream_data_receive_workfn	09:52:17	0
2733336	99	101	87,7	cyclictest	2968170	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	12:04:48	3
2733336	99	101	87,7	cyclictest	2968170	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	12:04:48	3
2768230	2	100	0,4	sleep4	0	-21	swapper/4	08:35:20	4
2733322	99	100	71,16	cyclictest	0	-21	swapper/0	08:05:36	0
2733322	99	100	71,16	cyclictest	0	-21	swapper/0	08:05:36	0
2733322	99	100	2,5	cyclictest	2900457	2	sleep0	10:45:07	0
2733322	99	100	2,5	cyclictest	2900457	2	sleep0	10:45:07	0
2733322	99	99	1,5	cyclictest	2825688	-21	ssh	09:45:31	0
2733336	99	97	82,7	cyclictest	2801823	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	09:26:35	3
2733336	99	97	82,7	cyclictest	2801823	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	09:26:34	3
2733322	99	97	76,17	cyclictest	210	-21	kswapd0	11:55:17	0
2733326	99	96	57,16	cyclictest	2749932	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	08:12:52	1
2733322	99	96	65,22	cyclictest	0	-21	swapper/0	07:45:20	0
2733332	99	95	84,5	cyclictest	2745295	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	07:49:04	2
2733322	99	94	59,21	cyclictest	0	-21	swapper/0	10:40:37	0
2733322	99	94	59,21	cyclictest	0	-21	swapper/0	10:40:37	0
2733322	99	93	4,6	cyclictest	2839072	-21	unixbench_singl	09:55:43	0
2733322	99	92	1,80	cyclictest	2935848	-21	kworker/u12:2+rpciod@ rpc_async_schedule	11:35:08	0
2733348	99	91	84,3	cyclictest	0	-21	swapper/5	12:20:44	5
2733322	99	91	69,15	cyclictest	2775513	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	09:28:34	0
2733322	99	91	69,15	cyclictest	2775513	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	09:28:34	0
2733322	99	91	61,21	cyclictest	0	-21	swapper/0	12:34:27	0
2733322	99	91	59,23	cyclictest	689	50	irq/527-eth0-Tx	07:35:22	0
2733322	99	91	56,23	cyclictest	689	50	irq/527-eth0-Tx	12:20:29	0
2733322	99	90	49,22	cyclictest	48	-21	rcuop/4	10:20:59	0
2733322	99	90	47,27	cyclictest	0	-21	swapper/0	07:55:38	0
2733322	99	90	2,4	cyclictest	2781856	-21	diskmemload	09:30:32	0
2733326	99	89	76,6	cyclictest	2977786	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	11:49:46	1
2733322	99	89	62,20	cyclictest	2949728	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	12:00:21	0
2733322	99	89	62,20	cyclictest	2949728	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	12:00:21	0
2733322	99	89	53,25	cyclictest	689	50	irq/527-eth0-Tx	10:15:01	0
2733322	99	89	24,41	cyclictest	2775513	-21	kworker/u12:1+rpciod@ rpc_async_schedule	09:10:12	0
2733322	99	89	2,3	cyclictest	0	-21	swapper/0	10:28:49	0
2733322	99	88	50,28	cyclictest	690	50	irq/528-eth0-Tx	11:00:12	0
2733322	99	88	2,5	cyclictest	2738347	-21	ntpq	07:20:31	0
2766373	2	87	0,5	sleep4	0	-21	swapper/4	08:30:22	4
2733341	99	87	78,5	cyclictest	0	-21	swapper/4	09:46:19	4
2733336	99	87	73,7	cyclictest	2968170	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	12:34:55	3
2733322	99	87	56,22	cyclictest	0	-21	swapper/0	08:25:22	0
2733322	99	87	51,21	cyclictest	68	-21	kcompactd0	11:16:01	0
2733322	99	86	54,23	cyclictest	689	50	irq/527-eth0-Tx	12:06:19	0
2733322	99	86	54,23	cyclictest	689	50	irq/527-eth0-Tx	12:06:19	0
2733322	99	86	2,4	cyclictest	0	-21	swapper/0	08:00:34	0
2733322	99	86	2,3	cyclictest	0	-21	swapper/0	09:05:37	0
2733322	99	86	1,6	cyclictest	2889038	-21	/usr/sbin/munin	10:35:29	0
2733322	99	85	64,14	cyclictest	3005339	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	12:13:21	0
2733322	99	85	59,19	cyclictest	2781548	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	09:21:26	0
2733322	99	85	59,19	cyclictest	2781548	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	09:21:25	0
2733322	99	85	55,21	cyclictest	0	-21	swapper/0	11:05:33	0
2733322	99	85	51,23	cyclictest	0	-21	swapper/0	07:50:39	0
2733322	99	85	51,23	cyclictest	0	-21	swapper/0	07:50:38	0
2733322	99	85	49,22	cyclictest	0	-21	swapper/0	08:15:32	0
2733322	99	85	47,29	cyclictest	689	50	irq/527-eth0-Tx	08:45:29	0
3024948	2	84	0,4	sleep3	0	-21	swapper/3	12:24:37	3
2733326	99	84	62,8	cyclictest	3005339	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	12:11:34	1
2733326	99	84	58,13	cyclictest	2781548	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	10:03:47	1
2733326	99	84	58,13	cyclictest	2781548	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	10:03:46	1
2733322	99	84	54,20	cyclictest	0	-21	swapper/0	10:30:23	0
2733322	99	84	50,23	cyclictest	0	-21	swapper/0	07:40:23	0
2733322	99	84	2,4	cyclictest	0	-21	swapper/0	11:31:55	0
2733322	99	83	6,45	cyclictest	689	50	irq/527-eth0-Tx	09:37:17	0
2733322	99	83	6,45	cyclictest	689	50	irq/527-eth0-Tx	09:37:16	0
2733322	99	83	54,21	cyclictest	0	-21	swapper/0	10:05:30	0
2733332	99	82	67,7	cyclictest	2767343	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	08:40:14	2
2733322	99	82	1,65	cyclictest	2767923	-21	apt-get	08:35:12	0
2733341	99	81	65,10	cyclictest	2900456	-21	ssh	10:45:07	4
2733341	99	81	65,10	cyclictest	2900456	-21	ssh	10:45:07	4
2733336	99	81	60,7	cyclictest	2801823	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	09:41:59	3
2733322	99	81	50,20	cyclictest	689	50	irq/527-eth0-Tx	11:28:05	0
2733322	99	81	50,20	cyclictest	689	50	irq/527-eth0-Tx	11:28:04	0
2733322	99	81	42,22	cyclictest	0	-21	swapper/0	07:30:19	0
2733322	99	81	41,24	cyclictest	0	-21	swapper/0	10:15:22	0
2733322	99	81	2,4	cyclictest	692	50	irq/530-eth0-Tx	10:00:41	0
2733322	99	81	2,4	cyclictest	692	50	irq/530-eth0-Tx	10:00:40	0
2733322	99	81	23,35	cyclictest	2882192	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	10:55:11	0
2733332	99	80	67,6	cyclictest	2949728	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	11:44:39	2
2733326	99	80	49,19	cyclictest	2745295	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	07:49:08	1
2733322	99	80	53,20	cyclictest	2968170	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	11:50:23	0
2733322	99	80	47,23	cyclictest	0	-21	swapper/0	09:17:46	0
2733322	99	80	45,20	cyclictest	0	-21	swapper/0	08:50:16	0
2733326	99	79	68,8	cyclictest	210	-21	kswapd0	10:15:25	1
2733326	99	79	66,6	cyclictest	2968170	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	12:24:34	1
2733322	99	79	5,51	cyclictest	2932134	-21	cut	11:10:18	0
2733322	99	79	42,22	cyclictest	0	-21	swapper/0	07:15:10	0
2733322	99	79	41,24	cyclictest	0	-21	swapper/0	12:25:20	0
2733332	99	78	67,5	cyclictest	2775513	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	09:43:04	2
2733332	99	78	63,8	cyclictest	2949728	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	11:29:25	2
2733332	99	78	63,8	cyclictest	2949728	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	11:29:25	2
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17