OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #3, slot #4

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance

Characteristics of the 100 highest latencies:

System rack3slot4.osadl.org (updated Sat Jan 03, 2026 00:47:12)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
572859	2	1467	1448,9	sleep1	570155	-21	kworker/1:0+events@ dbs_work_handler	19:05:20	1
574361	99	132	103,24	cyclictest	210	-21	kswapd0	21:15:27	0
574361	99	125	54,48	cyclictest	823275	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	00:11:00	0
574361	99	120	87,26	cyclictest	598289	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	20:16:43	0
574373	99	119	106,6	cyclictest	812699	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:58:17	3
784256	2	117	0,6	sleep1	0	-21	swapper/1	23:32:02	1
784256	2	117	0,6	sleep1	0	-21	swapper/1	23:32:02	1
574361	99	117	81,28	cyclictest	677941	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	22:01:57	0
574361	99	117	81,28	cyclictest	677941	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	22:01:57	0
574361	99	116	1,82	cyclictest	789294	-1	kworker/u13:3+xprtiod@ xs_stream_data_receive_workfn	23:42:49	0
574361	99	116	1,82	cyclictest	789294	-1	kworker/u13:3+xprtiod@ xs_stream_data_receive_workfn	23:42:49	0
574361	99	115	34,65	cyclictest	580025	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	19:45:04	0
574361	99	113	48,50	cyclictest	13	-21	ksoftirqd/0	00:28:00	0
574361	99	113	2,6	cyclictest	626727	-21	awk	21:15:00	0
574361	99	111	38,25	cyclictest	647185	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:15:32	0
574373	99	109	95,7	cyclictest	632220	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	21:21:53	3
574361	99	109	79,25	cyclictest	210	-21	kswapd0	22:40:00	0
574361	99	109	79,25	cyclictest	210	-21	kswapd0	22:39:59	0
574361	99	108	81,23	cyclictest	210	-21	kswapd0	22:45:16	0
574361	99	108	81,23	cyclictest	210	-21	kswapd0	22:45:15	0
574361	99	106	25,22	cyclictest	0	-21	swapper/0	23:15:46	0
574361	99	105	29,63	cyclictest	0	-21	swapper/0	23:25:34	0
574361	99	105	29,63	cyclictest	0	-21	swapper/0	23:25:34	0
574373	99	104	90,7	cyclictest	712213	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:50:14	3
574369	99	104	93,7	cyclictest	210	-21	kswapd0	00:35:26	2
574361	99	104	44,46	cyclictest	13	-21	ksoftirqd/0	22:25:27	0
574361	99	103	70,26	cyclictest	592877	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	20:01:41	0
574361	99	103	30,15	cyclictest	645035	-21	ssh	21:30:23	0
574361	99	103	2,89	cyclictest	0	-21	swapper/0	21:35:22	0
574361	99	102	73,24	cyclictest	210	-21	kswapd0	23:10:25	0
574361	99	102	68,27	cyclictest	823275	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	00:18:34	0
795173	2	101	0,4	sleep1	0	-21	swapper/1	23:41:26	1
795173	2	101	0,4	sleep1	0	-21	swapper/1	23:41:26	1
574373	99	101	87,7	cyclictest	705925	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:27:04	3
574369	99	100	85,8	cyclictest	746239	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	23:25:08	2
574369	99	100	85,8	cyclictest	746239	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	23:25:08	2
574369	99	99	85,7	cyclictest	831808	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:33:52	2
574361	99	99	2,64	cyclictest	582573	-21	latency_hist	19:30:01	0
574361	99	99	10,53	cyclictest	692	50	irq/530-eth0-Tx	23:20:38	0
574369	99	98	81,8	cyclictest	752175	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:20:01	2
574364	99	98	88,6	cyclictest	210	-21	kswapd0	23:20:00	1
574361	99	98	52,34	cyclictest	0	-21	swapper/0	19:15:33	0
574361	99	98	27,17	cyclictest	625051	-1	kworker/0:1H+kblockd@ blk_mq_run_work_fn	22:15:02	0
574361	99	98	27,17	cyclictest	625051	-1	kworker/0:1H+kblockd@ blk_mq_run_work_fn	22:15:01	0
574369	99	97	82,7	cyclictest	712213	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:47:08	2
574369	99	97	82,7	cyclictest	712213	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:47:07	2
574361	99	97	52,30	cyclictest	647185	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:47:37	0
574364	99	96	83,6	cyclictest	602400	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	20:21:42	1
574361	99	96	50,14	cyclictest	731404	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:57:47	0
574361	99	96	50,14	cyclictest	731404	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:57:46	0
574361	99	96	22,37	cyclictest	690	50	irq/528-eth0-Tx	22:40:29	0
574361	99	96	22,37	cyclictest	690	50	irq/528-eth0-Tx	22:40:29	0
574373	99	95	74,6	cyclictest	647185	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:08:31	3
573643	2	95	78,7	sleep2	0	-21	swapper/2	19:05:39	2
574383	99	93	86,4	cyclictest	0	-21	swapper/5	21:20:08	5
574361	99	93	44,12	cyclictest	0	-21	swapper/0	23:50:27	0
574361	99	93	43,20	cyclictest	690	50	irq/528-eth0-Tx	00:00:13	0
574383	99	92	87,2	cyclictest	0	-21	swapper/5	23:10:24	5
574383	99	92	87,2	cyclictest	0	-21	swapper/5	22:55:15	5
574383	99	92	87,2	cyclictest	0	-21	swapper/5	22:55:14	5
574379	99	92	86,2	cyclictest	0	-21	swapper/4	21:15:26	4
574361	99	92	20,40	cyclictest	622722	-21	diskmemload	23:45:45	0
574373	99	91	76,8	cyclictest	572494	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	19:20:35	3
574373	99	91	76,8	cyclictest	572494	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	19:20:35	3
574364	99	91	61,7	cyclictest	647185	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:45:18	1
574373	99	90	59,8	cyclictest	752175	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:07:39	3
574364	99	90	81,5	cyclictest	210	-21	kswapd0	21:25:02	1
574361	99	90	5,40	cyclictest	691	50	irq/529-eth0-Tx	23:35:37	0
574361	99	90	5,40	cyclictest	691	50	irq/529-eth0-Tx	23:35:37	0
574361	99	90	2,60	cyclictest	687673	-21	ssh	22:07:20	0
574361	99	89	57,24	cyclictest	604451	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	20:26:14	0
574361	99	89	57,24	cyclictest	604451	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	20:26:13	0
574361	99	89	51,29	cyclictest	602400	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	20:20:35	0
574361	99	89	26,14	cyclictest	692	50	irq/530-eth0-Tx	00:38:27	0
574361	99	88	2,5	cyclictest	668431	-21	collect2	21:50:30	0
574361	99	88	2,5	cyclictest	668431	-21	collect2	21:50:30	0
574361	99	87	2,47	cyclictest	815280	-21	idleruntime-cro	00:00:01	0
574383	99	86	81,2	cyclictest	0	-21	swapper/5	22:30:25	5
574383	99	86	81,2	cyclictest	0	-21	swapper/5	22:30:24	5
574373	99	86	77,6	cyclictest	210	-21	kswapd0	21:27:09	3
574364	99	86	72,7	cyclictest	638407	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	21:36:56	1
574361	99	86	56,23	cyclictest	793806	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	00:08:11	0
574361	99	86	45,11	cyclictest	0	-21	swapper/0	21:25:58	0
574361	99	86	1,41	cyclictest	0	-21	swapper/0	22:52:54	0
574373	99	85	75,6	cyclictest	210	-21	kswapd0	21:35:26	3
574369	99	85	70,8	cyclictest	580025	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	19:46:01	2
574364	99	85	71,7	cyclictest	698840	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:20:28	1
574364	99	85	71,7	cyclictest	698840	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:20:27	1
574361	99	85	43,20	cyclictest	0	-21	swapper/0	19:50:14	0
574373	99	84	70,7	cyclictest	775878	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:02:33	3
574369	99	84	35,18	cyclictest	752175	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:46:01	2
574364	99	84	74,6	cyclictest	210	-21	kswapd0	21:55:24	1
574364	99	84	74,6	cyclictest	210	-21	kswapd0	21:55:24	1
574364	99	84	74,6	cyclictest	210	-21	kswapd0	21:45:01	1
574364	99	84	71,6	cyclictest	592877	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	20:26:43	1
574364	99	84	71,6	cyclictest	592877	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	20:26:43	1
574364	99	84	70,7	cyclictest	604451	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	20:41:45	1
574361	99	84	8,4	cyclictest	692	50	irq/530-eth0-Tx	23:30:37	0
574361	99	84	8,4	cyclictest	692	50	irq/530-eth0-Tx	23:30:36	0
574373	99	83	69,7	cyclictest	684648	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:10:15	3
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17