OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #3, slot #4

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance

Characteristics of the 100 highest latencies:

System rack3slot4.osadl.org (updated Fri Jun 05, 2026 00:46:41)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
814021	99	137	108,9	cyclictest	922848	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:07:46	1
814033	99	112	79,9	cyclictest	933296	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	22:05:35	3
814016	99	111	2,66	cyclictest	862407	-21	kworker/u12:3+rpciod@ rpc_async_schedule	21:35:40	0
814016	99	111	2,64	cyclictest	889866	-21	kworker/u12:1+rpciod@ rpc_async_schedule	22:05:36	0
814016	99	110	26,46	cyclictest	858316	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:16:35	0
814016	99	108	16,38	cyclictest	0	-21	swapper/0	00:19:36	0
814016	99	108	16,38	cyclictest	0	-21	swapper/0	00:19:35	0
814021	99	107	93,8	cyclictest	889866	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:57:51	1
814021	99	107	93,7	cyclictest	880677	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	21:47:43	1
814016	99	107	1,75	cyclictest	0	-21	swapper/0	21:55:36	0
814016	99	107	1,75	cyclictest	0	-21	swapper/0	21:55:35	0
814016	99	106	2,69	cyclictest	837250	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	20:30:41	0
814016	99	106	2,69	cyclictest	837250	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	20:30:40	0
814016	99	104	2,16	cyclictest	875128	-21	/usr/sbin/munin	21:20:31	0
814016	99	104	2,16	cyclictest	875128	-21	/usr/sbin/munin	21:20:31	0
1052571	2	104	0,5	sleep1	889866	-21	kworker/u12:1+rpciod@ rpc_async_schedule	23:35:39	1
814016	99	103	7,72	cyclictest	821710	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	19:39:29	0
814021	99	102	88,7	cyclictest	907963	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	21:57:45	1
814021	99	102	88,7	cyclictest	907963	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	21:57:45	1
814016	99	102	2,82	cyclictest	0	-21	swapper/0	23:15:23	0
814016	99	99	2,79	cyclictest	0	-21	swapper/0	23:30:29	0
814016	99	98	79,6	cyclictest	689	50	irq/527-eth0-Tx	23:50:20	0
814016	99	98	2,78	cyclictest	13300	-21	snmpd	23:45:37	0
814016	99	98	2,77	cyclictest	1019575	-21	taskset	23:10:33	0
814016	99	98	1,78	cyclictest	0	-21	swapper/0	22:39:51	0
814021	99	97	84,6	cyclictest	889866	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:52:51	1
814021	99	97	84,6	cyclictest	889866	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:52:51	1
814021	99	97	83,7	cyclictest	889866	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:23:35	1
814021	99	97	83,7	cyclictest	889866	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:23:35	1
814021	99	97	83,7	cyclictest	1019667	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:17:52	1
814016	99	97	2,88	cyclictest	689	50	irq/527-eth0-Tx	00:25:45	0
814016	99	97	2,77	cyclictest	0	-21	swapper/0	23:22:06	0
814016	99	97	1,74	cyclictest	0	-21	swapper/0	23:44:59	0
814016	99	97	1,74	cyclictest	0	-21	swapper/0	23:44:58	0
814016	99	96	43,42	cyclictest	907071	-21	aten2_r3power_c	21:45:18	0
814021	99	95	77,11	cyclictest	1096110	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	00:23:00	1
814016	99	95	41,33	cyclictest	13	-21	ksoftirqd/0	21:30:44	0
814016	99	95	24,44	cyclictest	0	-21	swapper/0	22:59:56	0
814016	99	95	1,17	cyclictest	835313	-21	ntpq	20:00:36	0
814016	99	95	0,83	cyclictest	0	-21	swapper/0	19:21:45	0
814016	99	95	0,79	cyclictest	0	-21	swapper/0	20:20:31	0
814016	99	95	0,79	cyclictest	0	-21	swapper/0	20:20:30	0
814016	99	94	1,48	cyclictest	962396	-21	kworker/u12:2+rpciod@ rpc_async_schedule	22:40:11	0
814016	99	94	0,79	cyclictest	0	-21	swapper/0	20:45:27	0
814016	99	94	0,79	cyclictest	0	-21	swapper/0	20:45:27	0
814016	99	93	2,85	cyclictest	992991	-21	sed	22:50:25	0
814016	99	93	2,85	cyclictest	992991	-21	sed	22:50:25	0
814016	99	93	2,78	cyclictest	1039404	-21	ssh	23:25:39	0
814016	99	93	2,78	cyclictest	1039404	-21	ssh	23:25:39	0
814016	99	93	1,76	cyclictest	0	-21	swapper/0	19:30:39	0
814016	99	93	1,18	cyclictest	851435	-21	ntpq	20:40:29	0
814016	99	93	1,15	cyclictest	832485	-21	apt-get	19:55:15	0
814016	99	93	1,14	cyclictest	854739	-21	apt-get	20:50:15	0
814016	99	93	0,79	cyclictest	0	-21	swapper/0	23:01:54	0
814021	99	92	78,7	cyclictest	962396	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:49:35	1
814021	99	92	76,8	cyclictest	1081391	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:57:57	1
814016	99	92	1,73	cyclictest	859171	-21	sed	21:00:18	0
814016	99	92	0,61	cyclictest	0	-21	swapper/0	19:45:17	0
814026	99	91	78,7	cyclictest	1019667	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:21:34	2
814016	99	91	63,20	cyclictest	949019	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:17:46	0
814016	99	91	60,22	cyclictest	0	-21	swapper/0	19:16:24	0
814016	99	91	0,77	cyclictest	0	-21	swapper/0	19:40:02	0
814016	99	90	2,69	cyclictest	862537	-21	diskmemload	00:30:37	0
814016	99	90	1,78	cyclictest	0	-21	swapper/0	22:31:55	0
814016	99	90	0,85	cyclictest	0	-21	swapper/0	19:27:02	0
814033	99	89	74,7	cyclictest	933296	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	22:37:48	3
814033	99	89	52,9	cyclictest	848064	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	21:30:28	3
814021	99	89	71,9	cyclictest	812543	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	19:20:42	1
814021	99	89	56,17	cyclictest	1067469	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	00:26:41	1
814016	99	89	34,39	cyclictest	690	50	irq/528-eth0-Tx	21:43:54	0
814016	99	89	1,75	cyclictest	0	-21	swapper/0	20:57:03	0
814016	99	89	1,73	cyclictest	0	-21	swapper/0	00:10:21	0
814016	99	89	10,63	cyclictest	1087370	-21	rm	00:02:10	0
814016	99	89	0,55	cyclictest	0	-21	swapper/0	20:35:39	0
814021	99	88	74,6	cyclictest	1067469	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:51:02	1
814016	99	88	52,23	cyclictest	0	-21	swapper/0	20:07:37	0
814016	99	88	52,23	cyclictest	0	-21	swapper/0	20:07:36	0
814016	99	88	2,70	cyclictest	0	-21	swapper/0	22:10:31	0
814016	99	88	2,53	cyclictest	85544	-1	kworker/u13:2+xprtiod@ xs_stream_data_receive_workfn	21:25:16	0
814016	99	88	2,53	cyclictest	85544	-1	kworker/u13:2+xprtiod@ xs_stream_data_receive_workfn	21:25:16	0
814016	99	88	1,68	cyclictest	0	-21	swapper/0	23:35:17	0
814016	99	88	1,17	cyclictest	861302	-21	irqstats	21:05:23	0
814016	99	88	1,17	cyclictest	830584	-21	/usr/sbin/munin	19:50:17	0
814016	99	88	1,17	cyclictest	830584	-21	/usr/sbin/munin	19:50:16	0
814026	99	87	72,8	cyclictest	889866	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:55:14	2
814026	99	87	72,8	cyclictest	889866	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:55:14	2
814016	99	87	3,63	cyclictest	1090725	-21	apt-get	00:05:15	0
814016	99	87	1,73	cyclictest	0	-21	swapper/0	22:29:41	0
814016	99	87	0,77	cyclictest	0	-21	swapper/0	21:12:39	0
814016	99	87	0,74	cyclictest	0	-21	swapper/0	19:14:51	0
814016	99	87	0,70	cyclictest	0	-21	swapper/0	20:25:03	0
814021	99	86	71,8	cyclictest	1074229	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	00:32:59	1
814021	99	86	70,8	cyclictest	862407	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	21:22:14	1
814021	99	86	70,8	cyclictest	862407	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	21:22:14	1
814016	99	86	2,71	cyclictest	0	-21	swapper/0	00:23:42	0
814016	99	86	1,65	cyclictest	0	-21	swapper/0	22:20:32	0
814016	99	86	1,65	cyclictest	0	-21	swapper/0	22:20:32	0
814016	99	86	1,18	cyclictest	840856	-21	diskstats	20:15:19	0
814021	99	85	54,7	cyclictest	889866	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:44:10	1
814016	99	85	2,68	cyclictest	0	-21	swapper/0	00:35:39	0
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17