OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #3, slot #4

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance

Characteristics of the 100 highest latencies:

System rack3slot4.osadl.org (updated Thu Apr 16, 2026 00:46:43)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
1836684	2	3449	3422,17	sleep5	0	-21	swapper/5	19:05:27	5
2028992	2	148	0,4	sleep0	0	-21	swapper/0	23:00:01	0
1837876	99	145	130,8	cyclictest	1999927	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:40:26	2
1837867	99	144	72,47	cyclictest	2029276	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	23:18:13	0
1837876	99	130	100,10	cyclictest	2014752	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:05:32	2
1837876	99	130	100,10	cyclictest	2014752	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:05:31	2
1837876	99	128	113,8	cyclictest	1876630	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	21:05:14	2
1837871	99	116	101,7	cyclictest	1942103	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	22:30:25	1
1837871	99	116	101,7	cyclictest	1942103	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	22:30:24	1
1837867	99	114	32,45	cyclictest	1950105	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:36:48	0
1837880	99	108	95,6	cyclictest	1864000	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	20:35:08	3
1837871	99	106	92,7	cyclictest	1992732	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:45:28	1
1837867	99	105	42,23	cyclictest	2109246	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:01:35	0
1837867	99	105	42,23	cyclictest	2109246	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:01:34	0
1837867	99	105	19,28	cyclictest	690	50	irq/528-eth0-Tx	23:55:21	0
1837867	99	105	19,28	cyclictest	690	50	irq/528-eth0-Tx	23:55:21	0
1837867	99	103	73,16	cyclictest	1992732	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:10:11	0
1837867	99	103	73,16	cyclictest	1992732	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:10:10	0
1837867	99	103	40,4	cyclictest	691	50	irq/529-eth0-Tx	23:30:15	0
1837871	99	100	83,9	cyclictest	2014752	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	22:55:36	1
1837867	99	99	50,27	cyclictest	690	50	irq/528-eth0-Tx	21:40:43	0
1837880	99	98	83,8	cyclictest	1974436	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:20:23	3
1837867	99	98	63,32	cyclictest	2119035	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	00:31:22	0
1837867	99	97	46,29	cyclictest	689	50	irq/527-eth0-Tx	21:20:13	0
1837885	99	95	81,8	cyclictest	2075725	-21	ssh	23:35:20	4
1837876	99	95	80,8	cyclictest	2059930	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	23:30:34	2
1837880	99	94	67,9	cyclictest	2029276	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	23:16:26	3
1837876	99	94	80,7	cyclictest	1914186	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:40:19	2
1837867	99	94	61,25	cyclictest	1876630	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	20:51:37	0
1837867	99	94	53,3	cyclictest	689	50	irq/527-eth0-Tx	22:45:50	0
1837867	99	94	47,3	cyclictest	0	-21	swapper/0	23:40:35	0
1837867	99	94	47,26	cyclictest	690	50	irq/528-eth0-Tx	21:10:11	0
1837871	99	93	83,4	cyclictest	2014752	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:20:34	1
1837889	99	92	84,4	cyclictest	0	-21	swapper/5	22:50:38	5
1837876	99	92	78,7	cyclictest	2052895	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:55:05	2
1837876	99	92	78,7	cyclictest	2052895	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:55:04	2
1837867	99	92	60,24	cyclictest	2101499	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	00:15:15	0
1837867	99	92	60,24	cyclictest	2101499	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	00:15:15	0
1837867	99	90	43,14	cyclictest	1942103	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	22:02:31	0
1837871	99	89	73,8	cyclictest	2052895	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:39:41	1
1837871	99	88	75,6	cyclictest	2140764	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	00:25:34	1
1837867	99	88	59,26	cyclictest	1942103	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	22:23:09	0
1910668	2	87	0,5	sleep3	36	-21	rcuc/3	21:30:01	3
1910668	2	87	0,5	sleep3	36	-21	rcuc/3	21:30:01	3
1837876	99	87	72,7	cyclictest	1876630	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	21:35:18	2
1837889	99	86	80,3	cyclictest	0	-21	swapper/5	22:35:26	5
1837889	99	85	80,2	cyclictest	0	-21	swapper/5	00:13:28	5
1837889	99	85	80,2	cyclictest	0	-21	swapper/5	00:13:27	5
1837880	99	85	69,8	cyclictest	1876630	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:05:22	3
1837871	99	85	71,7	cyclictest	1876630	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	21:50:19	1
1837876	99	84	70,7	cyclictest	2094357	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	23:54:33	2
1837871	99	84	70,7	cyclictest	2052895	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:45:39	1
1837871	99	84	70,7	cyclictest	2052895	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:40:39	1
1837871	99	84	70,7	cyclictest	1859745	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	20:25:09	1
1837867	99	84	52,24	cyclictest	1883783	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:17:25	0
1837876	99	83	59,7	cyclictest	1876630	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	21:15:56	2
1837871	99	83	60,9	cyclictest	2109246	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:02:51	1
1837871	99	83	60,9	cyclictest	2109246	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:02:50	1
1837867	99	83	26,26	cyclictest	692	50	irq/530-eth0-Tx	23:00:48	0
1837871	99	82	69,6	cyclictest	2109246	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:08:25	1
1837871	99	82	69,6	cyclictest	2109246	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:08:24	1
1837871	99	82	66,8	cyclictest	1869826	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	20:40:12	1
1837871	99	82	66,8	cyclictest	1869826	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	20:40:12	1
1837885	99	81	69,7	cyclictest	1889755	-21	diskmemload-4	21:50:08	4
1837885	99	81	67,9	cyclictest	2017158	-21	ssh	22:50:33	4
1837880	99	81	65,7	cyclictest	1851689	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	20:03:34	3
1837871	99	81	68,6	cyclictest	1942103	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	22:36:36	1
1837871	99	81	50,7	cyclictest	1876630	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	21:40:15	1
1837867	99	81	50,15	cyclictest	1844925	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	19:31:33	0
1837876	99	80	70,4	cyclictest	1853722	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	19:50:01	2
1837871	99	80	66,7	cyclictest	2109246	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:13:58	1
1837871	99	80	66,7	cyclictest	2109246	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:13:57	1
1837871	99	80	64,8	cyclictest	1876630	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	20:50:13	1
1837871	99	80	63,9	cyclictest	1974436	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:25:32	1
1837871	99	80	63,9	cyclictest	1974436	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:25:31	1
1837871	99	80	46,7	cyclictest	1888825	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	21:30:23	1
1837871	99	80	46,7	cyclictest	1888825	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	21:30:23	1
1837867	99	80	30,6	cyclictest	692	50	irq/530-eth0-Tx	00:28:18	0
1837885	99	79	68,6	cyclictest	1889755	-21	diskmemload-4	00:02:09	4
1837885	99	79	68,6	cyclictest	1889755	-21	diskmemload-4	00:02:08	4
1837871	99	79	63,7	cyclictest	2029276	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	23:05:18	1
1837871	99	79	63,7	cyclictest	2029276	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	23:05:17	1
1837867	99	79	59,6	cyclictest	1942103	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	21:59:36	0
1837885	99	78	71,3	cyclictest	0	-21	swapper/4	23:45:45	4
1837885	99	78	69,5	cyclictest	1905749	-21	ssh	21:25:39	4
1837885	99	78	69,5	cyclictest	1905749	-21	ssh	21:25:39	4
1837876	99	78	65,6	cyclictest	1859745	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	20:05:05	2
1837871	99	78	65,7	cyclictest	2126719	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	00:15:31	1
1837871	99	78	65,7	cyclictest	2126719	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	00:15:30	1
1837871	99	78	64,7	cyclictest	2094357	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	23:54:10	1
1837867	99	78	47,15	cyclictest	1992732	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:06:11	0
1837867	99	78	47,15	cyclictest	1992732	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:06:10	0
1837867	99	78	4,37	cyclictest	2120647	-21	cron	00:10:00	0
1837867	99	78	4,37	cyclictest	2120647	-21	cron	00:10:00	0
1837880	99	77	62,8	cyclictest	2140764	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	00:30:37	3
1837880	99	77	41,8	cyclictest	1942103	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	22:04:23	3
1837876	99	77	64,6	cyclictest	1859745	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	20:15:08	2
1837876	99	77	64,6	cyclictest	1859745	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	20:15:08	2
1837876	99	77	63,6	cyclictest	1869826	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	20:41:35	2
1837876	99	77	63,6	cyclictest	1869826	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	20:41:35	2
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17