OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #3, slot #4

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance

Characteristics of the 100 highest latencies:

System rack3slot4.osadl.org (updated Mon Jun 08, 2026 00:46:30)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
4173419	2	160	0,4	sleep3	0	-21	swapper/3	00:05:28	3
3906732	99	122	67,45	cyclictest	690	50	irq/528-eth0-Tx	00:00:20	0
3906732	99	122	32,7	cyclictest	3928763	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	20:03:53	0
3906732	99	121	1,110	cyclictest	0	-21	swapper/0	22:45:01	0
3906732	99	121	1,110	cyclictest	0	-21	swapper/0	22:45:01	0
3906732	99	119	78,24	cyclictest	4154301	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	00:18:44	0
3906732	99	119	78,24	cyclictest	4154301	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	00:18:44	0
3906732	99	118	71,29	cyclictest	3996577	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	21:45:19	0
3906736	99	117	102,7	cyclictest	4135545	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	23:42:18	1
3906732	99	115	7,24	cyclictest	690	50	irq/528-eth0-Tx	21:00:39	0
3906732	99	115	67,38	cyclictest	689	50	irq/527-eth0-Tx	21:15:44	0
3906732	99	114	29,26	cyclictest	690	50	irq/528-eth0-Tx	21:30:38	0
3906732	99	113	3,104	cyclictest	3943014	-21	sendmail-msp	20:40:00	0
3906732	99	111	85,17	cyclictest	4021286	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:05:24	0
3906732	99	111	85,17	cyclictest	4021286	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:05:24	0
3906732	99	111	2,103	cyclictest	4136871	-21	cat	23:35:42	0
4019210	2	107	0,2	sleep4	3958499	-21	diskmemload-4	22:01:38	4
3906732	99	104	70,26	cyclictest	4126575	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:50:29	0
3906732	99	104	58,4	cyclictest	690	50	irq/528-eth0-Tx	22:15:19	0
3906732	99	104	57,5	cyclictest	689	50	irq/527-eth0-Tx	23:18:39	0
3906732	99	103	2,57	cyclictest	0	-21	swapper/0	20:47:00	0
3906732	99	103	2,48	cyclictest	13	-21	ksoftirqd/0	19:15:14	0
3906732	99	103	1,83	cyclictest	0	-21	swapper/0	00:25:01	0
3906732	99	101	71,23	cyclictest	4082827	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	23:00:32	0
3906732	99	101	69,25	cyclictest	3961952	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	21:21:34	0
3906732	99	100	55,11	cyclictest	689	50	irq/527-eth0-Tx	23:07:17	0
3906743	99	99	85,7	cyclictest	4126575	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:39:50	3
3906732	99	99	55,4	cyclictest	0	-21	swapper/0	19:35:10	0
3906732	99	98	44,18	cyclictest	13	-21	ksoftirqd/0	21:25:17	0
3906732	99	96	71,17	cyclictest	4075276	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	22:57:18	0
3906732	99	90	70,17	cyclictest	4063575	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:45:34	0
3906732	99	90	70,17	cyclictest	4063575	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:45:34	0
3906732	99	90	69,17	cyclictest	3938691	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	20:31:59	0
3906732	99	90	60,21	cyclictest	3938691	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:14:35	0
3906736	99	89	74,7	cyclictest	20215	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	00:38:49	1
3906732	99	87	66,18	cyclictest	9020	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:33:50	0
3906732	99	87	62,17	cyclictest	3928763	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	20:10:06	0
3906732	99	87	62,17	cyclictest	3928763	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	20:10:06	0
3906743	99	85	71,7	cyclictest	3930861	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	20:11:59	3
3906743	99	85	71,7	cyclictest	3930861	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	20:11:58	3
3906743	99	85	70,7	cyclictest	4126575	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:31:23	3
3906738	99	85	70,7	cyclictest	4055433	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	22:39:24	2
3906743	99	83	69,7	cyclictest	4162879	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	00:11:30	3
3906738	99	83	58,9	cyclictest	4145192	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	23:45:01	2
3906732	99	83	37,5	cyclictest	3994157	-21	sh	21:41:49	0
3906747	99	82	71,6	cyclictest	3958499	-21	diskmemload-4	23:52:19	4
3906738	99	82	69,6	cyclictest	4145192	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	23:58:03	2
3906732	99	82	49,7	cyclictest	4075276	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:46:09	0
3906738	99	81	67,6	cyclictest	3938691	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:17:35	2
3906732	99	81	59,7	cyclictest	3938691	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	20:27:30	0
3906747	99	80	74,3	cyclictest	0	-21	swapper/4	21:52:09	4
3906738	99	80	66,7	cyclictest	3961952	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	21:20:36	2
4095344	2	79	0,5	sleep3	0	-21	swapper/3	23:02:58	3
3906736	99	79	65,7	cyclictest	3955474	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	21:21:24	1
3906736	99	79	51,10	cyclictest	3989420	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:48:39	1
3906732	99	79	48,14	cyclictest	4145192	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	23:55:12	0
4020920	2	78	0,5	sleep5	0	-21	swapper/5	22:03:28	5
3906736	99	78	63,7	cyclictest	3928763	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	20:05:34	1
3906736	99	78	47,7	cyclictest	4184855	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	00:20:23	1
3906747	99	77	68,5	cyclictest	4174613	-21	ssh	00:06:02	4
3906747	99	77	65,7	cyclictest	4069876	-21	ssh	22:42:16	4
3906747	99	77	65,7	cyclictest	4069876	-21	ssh	22:42:16	4
3906743	99	77	64,7	cyclictest	4154301	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	23:54:13	3
3906732	99	77	54,7	cyclictest	3928763	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	20:17:03	0
4129910	2	76	0,4	sleep1	4129883	-21	bingrep	23:30:26	1
3906743	99	76	62,7	cyclictest	3914674	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	19:45:33	3
3906736	99	76	62,7	cyclictest	3977539	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:02:09	1
3906736	99	76	60,8	cyclictest	4028538	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:23:34	1
4122874	2	75	0,5	sleep5	0	-21	swapper/5	23:25:15	5
4122874	2	75	0,5	sleep5	0	-21	swapper/5	23:25:15	5
3906747	99	75	69,3	cyclictest	0	-21	swapper/4	22:30:21	4
3906747	99	75	69,3	cyclictest	0	-21	swapper/4	19:21:06	4
3906747	99	75	62,7	cyclictest	4151458	-21	ssh	23:47:58	4
3906743	99	75	62,6	cyclictest	4101057	-21	kworker/u12:4+events_unbound@ flush_memcg_stats_dwork	23:24:42	3
3906738	99	75	60,7	cyclictest	4028538	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:27:14	2
3906736	99	75	44,9	cyclictest	4046284	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:25:35	1
3906732	99	75	52,7	cyclictest	3936005	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	20:20:35	0
3906732	99	75	52,7	cyclictest	3902981	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	19:20:35	0
3906732	99	75	19,5	cyclictest	47	-21	rcuog/4	23:30:44	0
3906747	99	74	69,2	cyclictest	0	-21	swapper/4	19:45:13	4
3906732	99	74	43,7	cyclictest	4184855	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	00:27:06	0
3906732	99	74	41,28	cyclictest	15	-21	rcuc/0	00:10:39	0
3906747	99	73	68,2	cyclictest	0	-21	swapper/4	19:50:42	4
3906738	99	73	59,7	cyclictest	4032347	-21	kworker/u12:4+events_unbound@ flush_memcg_stats_dwork	22:15:02	2
3906736	99	73	58,7	cyclictest	4154301	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	00:03:17	1
3906743	99	72	59,6	cyclictest	4046284	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:35:58	3
3906743	99	72	59,6	cyclictest	3977539	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	21:41:40	3
3906738	99	72	61,7	cyclictest	4097892	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:16:30	2
3906738	99	72	58,7	cyclictest	4154301	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	00:23:47	2
3906738	99	72	58,7	cyclictest	3938691	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:27:56	2
3906736	99	72	57,8	cyclictest	4145192	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	23:47:39	1
3906732	99	72	21,7	cyclictest	13	-21	ksoftirqd/0	22:50:39	0
3906747	99	71	61,6	cyclictest	3960919	-21	ssh	21:15:29	4
3906736	99	71	57,7	cyclictest	4075276	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:35:21	1
3906736	99	71	44,9	cyclictest	4126575	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:53:29	1
3906732	99	71	35,5	cyclictest	15	-21	rcuc/0	22:35:40	0
3906747	99	70	64,3	cyclictest	0	-21	swapper/4	22:49:13	4
3906747	99	70	64,3	cyclictest	0	-21	swapper/4	22:49:13	4
3906743	99	70	60,7	cyclictest	4126575	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:28:15	3
3906743	99	70	60,7	cyclictest	4126575	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:28:15	3
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17