OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #3, slot #4

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance

Characteristics of the 100 highest latencies:

System rack3slot4.osadl.org (updated Thu Nov 06, 2025 00:46:30)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
2932578	2	2832	2803,17	sleep5	0	-21	swapper/5	19:05:22	5
2932578	2	2832	2803,17	sleep5	0	-21	swapper/5	19:05:22	5
2933792	2	1187	1168,9	sleep1	2931646	-21	kworker/1:1+events@ dbs_work_handler	19:09:08	1
2933792	2	1187	1168,9	sleep1	2931646	-21	kworker/1:1+events@ dbs_work_handler	19:09:07	1
2934023	99	137	122,8	cyclictest	3145620	-21	kworker/u12:4+events_unbound@ flush_memcg_stats_dwork	23:47:09	2
2934028	99	121	108,6	cyclictest	3217197	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:38:55	3
2934023	99	119	106,6	cyclictest	3109503	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	23:16:43	2
2934033	99	118	109,5	cyclictest	2988455	-21	ssh	21:16:21	4
2934028	99	117	103,7	cyclictest	2965864	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	20:46:17	3
2934023	99	117	104,6	cyclictest	2995698	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:16:35	2
2934033	99	111	105,3	cyclictest	0	-21	swapper/4	21:01:19	4
2934015	99	109	2,71	cyclictest	645	50	irq/530-eth0-Tx	22:25:31	0
2934015	99	106	78,20	cyclictest	3210821	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	00:33:53	0
2934015	99	106	78,20	cyclictest	2995698	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:46:39	0
2934015	99	106	26,41	cyclictest	3109503	-21	kworker/u12:2+events_unbound@ flush_to_ldisc	23:30:16	0
2934033	99	105	98,3	cyclictest	0	-21	swapper/4	21:46:27	4
2934015	99	104	67,30	cyclictest	2981795	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:01:31	0
2934033	99	103	97,3	cyclictest	0	-21	swapper/4	20:31:11	4
2934033	99	103	97,3	cyclictest	0	-21	swapper/4	20:31:10	4
2934023	99	102	90,6	cyclictest	3062489	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:56:41	2
2934023	99	101	86,8	cyclictest	3217197	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:28:46	2
2934028	99	100	86,7	cyclictest	3062489	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:26:35	3
2934028	99	100	77,8	cyclictest	3187597	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	00:13:37	3
2934028	99	100	77,8	cyclictest	3187597	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	00:13:36	3
2934015	99	100	17,3	cyclictest	0	-21	swapper/0	22:40:13	0
2934028	99	98	84,7	cyclictest	3006523	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:36:25	3
2934015	99	98	66,22	cyclictest	643	50	irq/528-eth0-Tx	23:50:37	0
2934023	99	96	82,7	cyclictest	3006523	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:36:27	2
2934023	99	95	82,7	cyclictest	2965864	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	20:51:14	2
2934015	99	94	71,16	cyclictest	3145620	-21	kworker/u12:4+events_unbound@ flush_memcg_stats_dwork	23:58:24	0
2934015	99	93	71,14	cyclictest	2967887	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	20:36:11	0
2934015	99	93	2,63	cyclictest	2982497	-21	aten2_r3power_c	21:10:16	0
2934015	99	93	19,42	cyclictest	642	50	irq/527-eth0-Tx	00:15:47	0
2934028	99	92	78,7	cyclictest	2946627	-21	kworker/u12:4+events_unbound@ flush_memcg_stats_dwork	19:46:05	3
2934015	99	91	72,16	cyclictest	2995698	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:36:38	0
2934015	99	91	39,36	cyclictest	3062489	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:00:56	0
2934033	99	90	84,3	cyclictest	0	-21	swapper/4	19:31:05	4
2934015	99	90	63,12	cyclictest	0	-21	swapper/0	00:10:20	0
2934015	99	90	63,12	cyclictest	0	-21	swapper/0	00:10:20	0
2934015	99	90	59,22	cyclictest	642	50	irq/527-eth0-Tx	00:25:30	0
2934015	99	90	11,5	cyclictest	2999956	-21	/usr/sbin/munin	21:25:41	0
2934015	99	89	47,24	cyclictest	0	-21	swapper/0	19:25:44	0
2934015	99	89	40,25	cyclictest	13	-21	ksoftirqd/0	23:00:00	0
3071253	2	88	0,5	sleep1	0	-21	swapper/1	22:25:01	1
2934015	99	88	1,62	cyclictest	3139881	-21	timerandwakeup	23:20:37	0
2934015	99	88	1,62	cyclictest	3139881	-21	timerandwakeup	23:20:36	0
2934028	99	87	73,7	cyclictest	3062489	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:36:36	3
2934015	99	87	52,24	cyclictest	0	-21	swapper/0	20:40:35	0
2934015	99	87	52,24	cyclictest	0	-21	swapper/0	20:40:35	0
2934015	99	87	47,13	cyclictest	643	50	irq/528-eth0-Tx	00:00:09	0
2934033	99	86	74,7	cyclictest	3210875	-21	ssh	00:20:37	4
2934023	99	86	73,7	cyclictest	3109503	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	23:01:41	2
2934015	99	86	39,15	cyclictest	13	-21	ksoftirqd/0	00:05:25	0
2934015	99	85	35,13	cyclictest	642	50	irq/527-eth0-Tx	22:50:31	0
2934015	99	84	55,20	cyclictest	0	-21	swapper/0	23:40:21	0
2934015	99	84	55,20	cyclictest	0	-21	swapper/0	23:40:20	0
2934015	99	84	20,50	cyclictest	0	-21	swapper/0	00:21:22	0
2934033	99	83	70,7	cyclictest	2985277	-21	diskmemload-4	23:27:58	4
2934033	99	83	70,7	cyclictest	2985277	-21	diskmemload-4	23:27:57	4
2934020	99	83	70,6	cyclictest	3109503	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	23:20:56	1
2934020	99	83	70,6	cyclictest	3109503	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	23:20:56	1
2934015	99	83	13,49	cyclictest	13	-21	ksoftirqd/0	22:15:44	0
2934015	99	83	11,41	cyclictest	13	-21	ksoftirqd/0	21:51:23	0
2934015	99	83	11,41	cyclictest	13	-21	ksoftirqd/0	21:51:23	0
3051932	2	82	0,4	sleep1	0	-21	swapper/1	22:08:51	1
2934015	99	82	48,23	cyclictest	0	-21	swapper/0	23:25:25	0
2934015	99	82	48,23	cyclictest	0	-21	swapper/0	23:25:24	0
2934015	99	81	41,21	cyclictest	0	-21	swapper/0	23:05:04	0
2934015	99	81	15,38	cyclictest	642	50	irq/527-eth0-Tx	21:30:39	0
2934023	99	80	66,7	cyclictest	3145620	-21	kworker/u12:4+events_unbound@ flush_memcg_stats_dwork	23:36:46	2
2934015	99	80	49,22	cyclictest	0	-21	swapper/0	20:06:51	0
2934015	99	80	34,36	cyclictest	643	50	irq/528-eth0-Tx	23:45:27	0
2934015	99	80	16,35	cyclictest	15	-21	rcuc/0	21:35:30	0
2934033	99	79	70,5	cyclictest	0	-21	swapper/4	19:40:00	4
2934023	99	79	65,7	cyclictest	2965864	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:06:20	2
2934015	99	79	31,22	cyclictest	0	-21	swapper/0	21:45:40	0
2934015	99	79	20,26	cyclictest	0	-21	swapper/0	21:40:17	0
2934015	99	79	1,51	cyclictest	13	-21	ksoftirqd/0	21:58:33	0
3220650	2	78	0,5	chrt	0	-21	swapper/1	00:30:20	1
2934015	99	78	43,24	cyclictest	0	-21	swapper/0	00:36:06	0
2934015	99	78	23,28	cyclictest	2993774	-21	ld	21:20:34	0
2934015	99	77	41,24	cyclictest	0	-21	swapper/0	19:30:35	0
2934015	99	77	40,25	cyclictest	0	-21	swapper/0	23:15:01	0
2934015	99	77	40,25	cyclictest	0	-21	swapper/0	23:15:00	0
2934015	99	77	38,15	cyclictest	13	-21	ksoftirqd/0	23:15:19	0
2934015	99	77	34,26	cyclictest	644	50	irq/529-eth0-Tx	22:24:43	0
2934033	99	76	67,5	cyclictest	3054700	-21	ssh	22:10:39	4
2934033	99	76	67,5	cyclictest	3054700	-21	ssh	22:10:39	4
2934023	99	76	63,7	cyclictest	2963822	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	20:31:15	2
2934023	99	76	63,7	cyclictest	2963822	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	20:31:15	2
2934015	99	76	57,15	cyclictest	2943560	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	19:41:05	0
2934015	99	76	2,55	cyclictest	13	-21	ksoftirqd/0	22:10:18	0
2934015	99	76	2,55	cyclictest	13	-21	ksoftirqd/0	22:10:18	0
2934033	99	75	68,4	cyclictest	0	-21	swapper/4	21:10:37	4
2934033	99	75	63,7	cyclictest	3072506	-21	ssh	22:25:29	4
2934028	99	75	65,4	cyclictest	3145620	-21	kworker/u12:4+events_unbound@ flush_memcg_stats_dwork	23:31:45	3
2934015	99	75	13,36	cyclictest	0	-21	swapper/0	21:17:02	0
2934033	99	74	68,3	cyclictest	0	-21	swapper/4	00:05:22	4
2934020	99	74	64,4	cyclictest	2981795	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	21:21:22	1
2934015	99	74	2,63	cyclictest	643	50	irq/528-eth0-Tx	22:10:01	0
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17