OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Sat Nov 22, 2025 00:43:32)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
2086964	2	73	63,7	sleep5	0	-21	swapper/5	03:21:13	5
2087106	2	57	47,7	sleep4	0	-21	swapper/4	03:23:13	4
2087492	99	56	51,4	cyclictest	2606320	-21	latency_hist	07:38:33	7
2087492	99	52	0,50	cyclictest	2097451	-21	latency_hist	03:28:34	7
2087492	99	51	46,4	cyclictest	2616417	-21	latency_hist	07:43:33	7
2087089	2	50	16,30	sleep7	0	-21	swapper/7	03:23:00	7
2458677	2	49	0,0	sleep7	2458678	-21	kthreadcore	06:23:51	7
2086877	2	49	39,7	sleep6	0	-21	swapper/6	03:20:02	6
2087479	99	40	37,2	cyclictest	2181760	-21	kworker/u16:3+events_unbound	04:53:48	4
2087480	99	39	37,2	cyclictest	2394231	-21	kworker/u16:2+flush-8:0	06:19:00	5
2087488	99	37	35,2	cyclictest	2333693	-21	kworker/u16:4+flush-8:0	05:49:06	6
2087480	99	37	35,2	cyclictest	2576135	-21	kworker/u16:1+events_unbound	07:56:30	5
2087480	99	37	35,2	cyclictest	2476656	-21	kworker/u16:3+events_unbound	06:37:07	5
2087480	99	35	32,2	cyclictest	2555675	-21	kworker/u16:0+events_unbound	07:13:36	5
2087480	99	35	31,3	cyclictest	2697424	-21	kworker/u16:2+flush-8:0	08:26:11	5
2087492	99	34	32,1	cyclictest	0	-21	swapper/7	08:13:58	7
2087480	99	34	32,2	cyclictest	2262980	-21	kworker/u16:4+flush-8:0	04:54:01	5
2087479	99	34	31,2	cyclictest	2576135	-21	kworker/u16:1+flush-8:0	07:25:43	4
2087479	99	34	31,2	cyclictest	2181760	-21	kworker/u16:3+flush-8:0	04:23:47	4
2087488	99	33	31,2	cyclictest	2333693	-21	kworker/u16:4+events_unbound	06:33:58	6
2087480	99	33	31,2	cyclictest	2333693	-21	kworker/u16:4+events_unbound	05:29:12	5
2087480	99	33	31,2	cyclictest	2262979	-21	kworker/u16:1+flush-8:0	05:20:43	5
2087480	99	33	31,2	cyclictest	2202093	-21	kworker/u16:0+flush-8:0	04:39:43	5
2087492	99	32	0,31	cyclictest	589235	-21	systemd-journal	03:47:06	7
2087480	99	32	30,2	cyclictest	2394231	-21	kworker/u16:2+events_unbound	06:28:43	5
2087480	99	32	29,2	cyclictest	2636673	-21	kworker/u16:4+events_unbound	08:08:35	5
2087480	99	31	29,2	cyclictest	2717227	-21	kworker/u16:3+flush-8:0	08:45:51	5
2087480	99	31	29,2	cyclictest	2545544	-21	kworker/u16:2+events_unbound	07:05:47	5
2087480	99	31	29,2	cyclictest	2171694	-21	kworker/u16:2+flush-8:0	04:14:34	5
2087480	99	31	29,2	cyclictest	2076962	-21	kworker/u16:1+flush-8:0	03:36:06	5
2087480	99	31	28,2	cyclictest	2565786	-21	kworker/u16:2+flush-8:0	07:33:58	5
2087492	99	30	26,2	cyclictest	2264534	-21	kthreadcore	04:48:53	7
2087488	99	30	28,2	cyclictest	2394231	-21	kworker/u16:2+flush-8:0	06:28:57	6
2087480	99	30	28,2	cyclictest	2648296	-21	kworker/u16:0+events_unbound	08:39:51	5
2087480	99	30	28,2	cyclictest	2626299	-21	kworker/u16:2+events_unbound	08:07:20	5
2087480	99	30	28,2	cyclictest	2616239	-21	kworker/u16:0+events_unbound	07:42:20	5
2087480	99	30	28,2	cyclictest	2576135	-21	kworker/u16:1+flush-8:0	07:32:59	5
2087480	99	30	28,2	cyclictest	2576135	-21	kworker/u16:1+events_unbound	07:50:42	5
2087480	99	30	28,2	cyclictest	2576135	-21	kworker/u16:1+events_unbound	07:46:20	5
2087480	99	30	28,2	cyclictest	2332738	-21	kworker/u16:2+events_unbound	05:27:05	5
2087480	99	30	28,2	cyclictest	2111324	-21	kworker/u16:3+flush-8:0	03:42:36	5
2087488	99	29	27,2	cyclictest	2576135	-21	kworker/u16:1+events_unbound	07:48:27	6
2087488	99	29	26,2	cyclictest	2333693	-21	kworker/u16:4+events_unbound	05:33:36	6
2087480	99	29	27,2	cyclictest	2717227	-21	kworker/u16:3+events_unbound	08:36:12	5
2087480	99	29	27,2	cyclictest	2636673	-21	kworker/u16:4+flush-8:0	08:00:15	5
2087480	99	29	27,2	cyclictest	2333693	-21	kworker/u16:4+events_unbound	05:46:03	5
2087480	99	29	27,2	cyclictest	2296131	-21	kworker/u16:0+flush-8:0	05:15:31	5
2087479	99	29	26,3	cyclictest	2394231	-21	kworker/u16:2+flush-8:0	06:48:57	4
2087492	99	28	23,4	cyclictest	2293234	-21	latency_hist	05:03:33	7
2087488	99	28	26,2	cyclictest	2648296	-21	kworker/u16:0+flush-8:0	08:23:47	6
2087488	99	28	26,2	cyclictest	2545544	-21	kworker/u16:2+events_unbound	07:08:47	6
2087488	99	28	26,2	cyclictest	2474805	-21	kworker/u16:0+flush-8:0	06:43:48	6
2087480	99	28	26,2	cyclictest	2494943	-21	kworker/u16:1+events_unbound	06:40:03	5
2087480	99	28	26,2	cyclictest	2476656	-21	kworker/u16:3+events_unbound	07:09:43	5
2087480	99	28	26,2	cyclictest	2303517	-21	kworker/u16:2+events_unbound	05:10:23	5
2087480	99	28	26,2	cyclictest	2066803	-21	kworker/u16:2+events_unbound	03:26:35	5
2087479	99	28	25,3	cyclictest	2414462	-21	kworker/u16:1+flush-8:0	06:25:42	4
2087492	99	27	26,0	cyclictest	0	-21	swapper/7	05:08:48	7
2087492	99	27	26,0	cyclictest	0	-21	swapper/7	04:53:46	7
2087480	99	27	25,2	cyclictest	2576135	-21	kworker/u16:1+events_unbound	07:27:40	5
2087480	99	27	25,2	cyclictest	2394231	-21	kworker/u16:2+events_unbound	06:02:04	5
2087480	99	27	25,2	cyclictest	2262979	-21	kworker/u16:1+events_unbound	05:39:24	5
2087480	99	27	25,2	cyclictest	2131625	-21	kworker/u16:0+flush-8:0	03:49:35	5
2087480	99	27	25,2	cyclictest	1270234	-21	kworker/u16:0+flush-8:0	03:29:46	5
2087492	99	26	25,0	cyclictest	0	-21	swapper/7	05:26:24	7
2087488	99	26	24,2	cyclictest	2353695	-21	kworker/u16:0+events_unbound	05:53:50	6
2087480	99	26	24,2	cyclictest	2687002	-21	kworker/u16:6+flush-8:0	08:18:23	5
2087480	99	26	24,2	cyclictest	2648296	-21	kworker/u16:0+events_unbound	08:32:21	5
2087480	99	26	24,2	cyclictest	2476656	-21	kworker/u16:3+flush-8:0	07:02:47	5
2087480	99	26	24,2	cyclictest	2476656	-21	kworker/u16:3+events_unbound	06:56:20	5
2087480	99	26	24,2	cyclictest	2394231	-21	kworker/u16:2+flush-8:0	05:56:43	5
2087480	99	26	24,2	cyclictest	2353695	-21	kworker/u16:0+events_unbound	06:17:40	5
2087480	99	26	24,2	cyclictest	2353695	-21	kworker/u16:0+events_unbound	06:04:31	5
2087480	99	26	24,2	cyclictest	2332738	-21	kworker/u16:2+events_unbound	05:37:31	5
2087480	99	26	24,2	cyclictest	2202093	-21	kworker/u16:0+flush-8:0	04:29:34	5
2087480	99	26	24,2	cyclictest	2181760	-21	kworker/u16:3+flush-8:0	04:22:06	5
2087480	99	26	24,2	cyclictest	2181760	-21	kworker/u16:3+events_unbound	04:59:20	5
2087480	99	26	23,2	cyclictest	2141759	-21	kworker/u16:4+flush-8:0	03:53:43	5
2087479	99	26	24,2	cyclictest	2303517	-21	kworker/u16:2+flush-8:0	05:08:57	4
2087492	99	25	24,0	cyclictest	0	-21	swapper/7	07:32:55	7
2087488	99	25	23,2	cyclictest	2353695	-21	kworker/u16:0+flush-8:0	05:43:49	6
2087480	99	25	23,2	cyclictest	2474805	-21	kworker/u16:0+flush-8:0	06:47:56	5
2087480	99	25	23,2	cyclictest	2202093	-21	kworker/u16:0+flush-8:0	04:35:44	5
2087480	99	25	23,2	cyclictest	2181760	-21	kworker/u16:3+flush-8:0	04:12:43	5
2087480	99	25	23,2	cyclictest	2181760	-21	kworker/u16:3+events_unbound	04:46:19	5
2087480	99	25	23,2	cyclictest	2171694	-21	kworker/u16:2+flush-8:0	04:05:28	5
2087492	99	24	23,0	cyclictest	0	-21	swapper/7	06:52:59	7
2087492	99	24	21,2	cyclictest	589235	-21	systemd-journal	03:38:33	7
2087488	99	24	22,2	cyclictest	2636673	-21	kworker/u16:4+flush-8:0	07:55:10	6
2087488	99	24	22,2	cyclictest	2191965	-21	kworker/u16:1+flush-8:0	04:12:11	6
2087480	99	24	22,2	cyclictest	2476656	-21	kworker/u16:3+events_unbound	07:20:54	5
2087480	99	24	22,2	cyclictest	2262979	-21	kworker/u16:1+events_unbound	04:50:25	5
2087480	99	24	22,2	cyclictest	2181760	-21	kworker/u16:3+flush-8:0	04:24:12	5
2087479	99	24	21,2	cyclictest	2131625	-21	kworker/u16:0+flush-8:0	03:55:17	4
2087492	99	23	22,0	cyclictest	0	-21	swapper/7	04:23:03	7
2087488	99	23	21,2	cyclictest	2676963	-21	kworker/u16:1+flush-8:0	08:46:15	6
2087488	99	23	21,2	cyclictest	2545544	-21	kworker/u16:2+flush-8:0	07:07:03	6
2087488	99	23	21,2	cyclictest	2494943	-21	kworker/u16:1+flush-8:0	06:53:58	6
2087488	99	23	21,2	cyclictest	2474805	-21	kworker/u16:0+events_unbound	06:38:47	6
2087488	99	23	21,2	cyclictest	2333693	-21	kworker/u16:4+events_unbound	05:24:31	6
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17