OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #3, slot #4

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance

Characteristics of the 100 highest latencies:

System rack3slot4.osadl.org (updated Thu Oct 01, 2026 12:46:33)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
3716570	2	1432	1423,4	sleep5	0	-21	swapper/5	07:08:48	5
3716828	99	162	45,41	cyclictest	3779180	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	09:23:45	0
3716828	99	127	89,29	cyclictest	3961101	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	11:56:08	0
3716828	99	125	1,98	cyclictest	3765113	-21	diskmemload	10:15:29	0
3716828	99	125	1,8	cyclictest	3747066	-21	cut	08:25:01	0
3716828	99	123	4,44	cyclictest	3753575	-21	kworker/u12:2-events_unbound	08:45:11	0
3716828	99	122	2,90	cyclictest	0	-21	swapper/0	12:20:20	0
3716828	99	120	35,55	cyclictest	13	-21	ksoftirqd/0	12:10:10	0
3716828	99	112	72,20	cyclictest	3729278	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	08:50:33	0
3716828	99	104	64,19	cyclictest	3729278	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	09:55:21	0
3716828	99	104	2,70	cyclictest	3997138	-21	latency_hist	12:20:01	0
3716828	99	101	31,13	cyclictest	676	50	irq/530-eth0-Tx	11:25:23	0
3716828	99	100	1,82	cyclictest	3862772	-21	aten2_r3power_p	10:30:17	0
3716828	99	100	1,82	cyclictest	3862772	-21	aten2_r3power_p	10:30:17	0
3716828	99	98	31,13	cyclictest	4005407	-21	users	12:25:49	0
3716828	99	98	2,82	cyclictest	0	-21	swapper/0	10:20:24	0
3716828	99	98	27,48	cyclictest	3900565	-21	sshd	11:00:28	0
3959177	2	97	0,4	sleep0	0	-21	swapper/0	11:48:31	0
3716828	99	97	8,61	cyclictest	13	-21	ksoftirqd/0	11:30:28	0
3716828	99	97	8,61	cyclictest	13	-21	ksoftirqd/0	11:30:27	0
3716828	99	97	19,23	cyclictest	0	-21	swapper/0	11:05:37	0
3716842	99	96	82,7	cyclictest	3928186	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	12:11:10	3
3716828	99	95	49,27	cyclictest	3870451	-21	unixbench_multi	10:35:45	0
3716828	99	95	49,27	cyclictest	3870451	-21	unixbench_multi	10:35:45	0
3716828	99	95	1,84	cyclictest	0	-21	swapper/0	07:24:41	0
3716828	99	94	24,16	cyclictest	13	-21	ksoftirqd/0	09:35:14	0
3716828	99	93	3,59	cyclictest	13	-21	ksoftirqd/0	09:15:23	0
3716828	99	93	33,12	cyclictest	0	-21	swapper/0	11:43:56	0
3716828	99	93	33,12	cyclictest	0	-21	swapper/0	11:43:55	0
3716828	99	93	2,78	cyclictest	3765113	-21	diskmemload	11:15:49	0
3716831	99	92	76,8	cyclictest	3999519	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	12:31:14	1
3716828	99	92	29,15	cyclictest	13	-21	ksoftirqd/0	10:10:01	0
3716828	99	92	29,14	cyclictest	3966793	-21	ssh	11:55:00	0
3716828	99	91	29,13	cyclictest	0	-21	swapper/0	10:25:23	0
3716828	99	90	2,46	cyclictest	3765042	-21	kworker/u12:2+rpciod@ rpc_async_schedule	09:15:01	0
3716837	99	89	75,7	cyclictest	3729278	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	09:10:37	2
3716828	99	89	1,48	cyclictest	13	-21	ksoftirqd/0	12:00:18	0
3716851	99	88	80,3	cyclictest	0	-21	swapper/5	10:40:59	5
3716828	99	88	2,5	cyclictest	3838847	-21	ld	10:10:33	0
3716842	99	87	72,8	cyclictest	3729278	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	08:55:34	3
3716842	99	87	72,8	cyclictest	3729278	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	08:55:34	3
3716828	99	87	37,15	cyclictest	13	-21	ksoftirqd/0	10:55:20	0
3716828	99	87	32,40	cyclictest	673	50	irq/527-eth0-Tx	10:45:34	0
3716828	99	87	31,13	cyclictest	675	50	irq/529-eth0-Tx	09:40:47	0
3940592	2	86	0,5	sleep1	0	-21	swapper/1	11:33:24	1
3940592	2	86	0,5	sleep1	0	-21	swapper/1	11:33:23	1
3716851	99	86	79,3	cyclictest	0	-21	swapper/5	09:50:52	5
3716831	99	86	73,6	cyclictest	3845066	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	10:25:57	1
3716828	99	86	37,12	cyclictest	0	-21	swapper/0	09:25:32	0
3716828	99	84	30,14	cyclictest	15	-21	rcuc/0	12:35:01	0
3716828	99	83	24,15	cyclictest	13	-21	ksoftirqd/0	10:50:31	0
3716828	99	83	18,35	cyclictest	3928100	-21	ssh	11:23:02	0
3716828	99	82	63,12	cyclictest	3999519	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	12:36:13	0
3716828	99	82	45,21	cyclictest	3717132	-21	/usr/sbin/munin	07:10:17	0
3716828	99	82	45,21	cyclictest	3717132	-21	/usr/sbin/munin	07:10:16	0
3716828	99	82	34,13	cyclictest	676	50	irq/530-eth0-Tx	09:50:23	0
3716828	99	82	19,34	cyclictest	0	-21	swapper/0	09:45:02	0
3716828	99	82	1,40	cyclictest	3912266	-21	/usr/sbin/munin	11:10:17	0
3716828	99	82	12,36	cyclictest	13	-21	ksoftirqd/0	10:40:35	0
3716828	99	82	10,49	cyclictest	3895566	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	11:35:15	0
3716828	99	82	10,49	cyclictest	3895566	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	11:35:14	0
3716846	99	81	68,7	cyclictest	3969290	-21	ssh	11:56:08	4
3716831	99	81	67,7	cyclictest	3729278	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	09:40:52	1
3716828	99	81	2,39	cyclictest	13	-21	ksoftirqd/0	09:30:52	0
3716842	99	79	64,7	cyclictest	3779180	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	09:25:52	3
3716842	99	78	61,8	cyclictest	3729278	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	08:05:10	3
3716842	99	78	61,8	cyclictest	3729278	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	08:05:10	3
3716828	99	78	47,17	cyclictest	0	-21	swapper/0	08:30:29	0
3716831	99	77	62,7	cyclictest	3895566	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	11:46:09	1
3716828	99	77	14,24	cyclictest	675	50	irq/529-eth0-Tx	10:00:34	0
3716828	99	76	30,28	cyclictest	3732982	-21	date	07:50:01	0
3716828	99	76	2,51	cyclictest	3738371	-21	find	08:00:43	0
3716828	99	75	2,42	cyclictest	15	-21	rcuc/0	08:15:43	0
3716846	99	74	62,7	cyclictest	3936037	-21	ssh	11:30:01	4
3716828	99	74	2,43	cyclictest	3980445	-21	ld	12:05:36	0
3716851	99	73	68,2	cyclictest	0	-21	swapper/5	11:06:02	5
3716828	99	73	43,16	cyclictest	0	-21	swapper/0	07:30:26	0
3716828	99	73	2,35	cyclictest	3726663	-1	kworker/0:2H+mmc_complete@ mmc_blk_mq_complete_work	07:45:02	0
3716851	99	71	64,3	cyclictest	0	-21	swapper/5	12:37:46	5
3775696	2	70	0,7	chrt	3775665	-21	idleruntime-cro	09:20:00	2
3716851	99	70	63,3	cyclictest	0	-21	swapper/5	10:45:29	5
3716837	99	70	56,7	cyclictest	3757452	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	09:00:36	2
3716828	99	70	5,41	cyclictest	13	-21	ksoftirqd/0	07:35:38	0
3716828	99	70	27,24	cyclictest	3764162	-21	ntpq	09:05:29	0
3716828	99	70	1,45	cyclictest	0	-21	swapper/0	09:00:38	0
3716851	99	69	62,3	cyclictest	0	-21	swapper/5	07:53:01	5
3716851	99	69	62,3	cyclictest	0	-21	swapper/5	07:53:00	5
3716828	99	68	32,14	cyclictest	673	50	irq/527-eth0-Tx	07:25:40	0
3716828	99	68	32,14	cyclictest	673	50	irq/527-eth0-Tx	07:25:39	0
3716851	99	67	61,3	cyclictest	0	-21	swapper/5	08:15:30	5
3716846	99	67	61,3	cyclictest	0	-21	swapper/4	10:10:33	4
3716846	99	67	60,3	cyclictest	0	-21	swapper/4	11:00:24	4
3716831	99	67	54,6	cyclictest	3928186	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	12:16:12	1
3716828	99	67	47,16	cyclictest	3738902	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	08:10:27	0
3716828	99	67	32,13	cyclictest	673	50	irq/527-eth0-Tx	08:05:31	0
3716828	99	67	32,13	cyclictest	673	50	irq/527-eth0-Tx	08:05:30	0
3716851	99	66	60,3	cyclictest	0	-21	swapper/5	07:20:31	5
3716846	99	66	60,3	cyclictest	0	-21	swapper/4	07:55:01	4
3716846	99	66	60,3	cyclictest	0	-21	swapper/4	07:55:00	4
3716828	99	66	30,11	cyclictest	0	-21	swapper/0	08:40:14	0
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17