OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Thu Mar 05, 2026 00:43:34)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
3255225	99	66	63,2	cyclictest	3786937	-21	kworker/u16:0+events_unbound	23:34:56	5
3255228	99	57	54,2	cyclictest	3786937	-21	kworker/u16:0+flush-8:0	23:19:41	6
3255225	99	56	54,2	cyclictest	3619305	-21	kworker/u16:5+events_unbound	22:00:02	5
3255228	99	54	52,2	cyclictest	3882738	-21	kworker/u16:2+events_unbound	23:54:58	6
3255228	99	52	50,2	cyclictest	3756768	-21	kworker/u16:1+events_unbound	22:54:53	6
3255228	99	52	50,2	cyclictest	3481188	-21	kworker/u16:0+events_unbound	21:09:42	6
3254865	2	52	42,7	sleep6	0	-21	swapper/6	19:03:52	6
3255225	99	51	49,2	cyclictest	3812845	-21	kworker/u16:3+events_unbound	00:09:57	5
3255225	99	51	48,2	cyclictest	3481188	-21	kworker/u16:0+events_unbound	20:59:54	5
3255228	99	50	48,2	cyclictest	3288155	-21	kworker/u16:2+events_unbound	20:34:53	6
3255225	99	50	48,2	cyclictest	3583541	-21	kworker/u16:2+events_unbound	21:54:57	5
3255225	99	50	47,2	cyclictest	3942630	-21	kworker/u16:1+events_unbound	00:19:30	5
3255225	99	49	47,2	cyclictest	3812845	-21	kworker/u16:3+events_unbound	23:59:53	5
3255228	99	48	46,2	cyclictest	3450803	-21	kworker/u16:3+events_unbound	20:59:44	6
3255225	99	48	46,2	cyclictest	3786937	-21	kworker/u16:0+flush-8:0	23:15:02	5
3255225	99	48	46,2	cyclictest	3481188	-21	kworker/u16:0+events_unbound	21:39:44	5
3254837	2	48	38,7	sleep5	0	-21	swapper/5	19:03:27	5
3255230	99	47	0,45	cyclictest	3621338	-21	latency_hist	21:59:28	7
3255228	99	47	45,2	cyclictest	3632728	-21	kworker/u16:0+events_unbound	22:39:42	6
3255228	99	47	45,2	cyclictest	3596896	-21	kworker/u16:0+flush-8:0	21:49:53	6
3255228	99	47	45,2	cyclictest	3596896	-21	kworker/u16:0+flush-8:0	21:49:53	6
3255228	99	47	44,2	cyclictest	3288155	-21	kworker/u16:2+events_unbound	20:54:43	6
3255225	99	47	45,2	cyclictest	3756768	-21	kworker/u16:1+events_unbound	23:09:42	5
3255225	99	47	45,2	cyclictest	3756768	-21	kworker/u16:1+events_unbound	23:09:42	5
3255228	99	46	44,2	cyclictest	3681643	-21	kworker/u16:2+events_unbound	22:29:58	6
3255228	99	46	44,2	cyclictest	3681643	-21	kworker/u16:2+events_unbound	22:29:58	6
3255228	99	46	44,2	cyclictest	3288155	-21	kworker/u16:2+flush-8:0	21:04:57	6
3255225	99	46	45,1	cyclictest	3681643	-21	kworker/u16:2+flush-8:0	22:44:28	5
3255218	99	46	44,2	cyclictest	3632728	-21	kworker/u16:0+flush-8:0	22:49:43	4
3255228	99	45	43,2	cyclictest	3756768	-21	kworker/u16:1+events_unbound	23:04:53	6
3255228	99	45	43,2	cyclictest	3288155	-21	kworker/u16:2+flush-8:0	21:29:57	6
3255228	99	45	43,2	cyclictest	3288155	-21	kworker/u16:2+flush-8:0	21:25:00	6
3255228	99	45	42,2	cyclictest	3523856	-21	kworker/u16:1+flush-8:0	22:24:59	6
3255218	99	45	43,2	cyclictest	3812845	-21	kworker/u16:3+events_unbound	23:29:59	4
3255228	99	44	42,2	cyclictest	3481188	-21	kworker/u16:0+events_unbound	21:39:54	6
3255225	99	44	41,2	cyclictest	3646683	-21	kworker/u16:2+flush-8:0	22:09:59	5
6265	89	43	0,36	rtkit-daemon	0	-21	swapper/4	19:00:25	4
3255230	99	43	37,5	cyclictest	3431923	-21	latency_hist	20:34:28	7
3255230	99	43	37,5	cyclictest	3431923	-21	latency_hist	20:34:28	7
3255228	99	43	41,2	cyclictest	3632728	-21	kworker/u16:0+events_unbound	22:54:25	6
3255228	99	43	41,2	cyclictest	3583541	-21	kworker/u16:2+events_unbound	21:54:53	6
3255228	99	43	41,2	cyclictest	3450803	-21	kworker/u16:3+events_unbound	21:16:21	6
3255225	99	43	41,2	cyclictest	3897411	-21	kworker/u16:4+flush-8:0	00:27:57	5
3255225	99	43	41,2	cyclictest	3897411	-21	kworker/u16:4+flush-8:0	00:09:28	5
3255218	99	43	41,2	cyclictest	3942630	-21	kworker/u16:1+flush-8:0	00:25:01	4
3255218	99	43	41,2	cyclictest	3812845	-21	kworker/u16:3+flush-8:0	23:22:46	4
3255218	99	43	41,2	cyclictest	3812845	-21	kworker/u16:3+flush-8:0	00:04:28	4
3255218	99	43	41,2	cyclictest	3812845	-21	kworker/u16:3+events_unbound	23:40:01	4
3255218	99	43	41,2	cyclictest	3812845	-21	kworker/u16:3+events_unbound	23:34:54	4
3255218	99	43	41,2	cyclictest	3696615	-21	kworker/u16:3+events_unbound	22:32:34	4
3255218	99	43	41,2	cyclictest	3696615	-21	kworker/u16:3+events_unbound	22:32:34	4
3255218	99	43	41,2	cyclictest	3481188	-21	kworker/u16:0+events_unbound	21:34:55	4
3255230	99	42	37,4	cyclictest	3489582	-21	latency_hist	21:04:28	7
3255228	99	42	40,2	cyclictest	3978622	-21	kworker/u16:3+events_unbound	00:29:49	6
3255228	99	42	40,2	cyclictest	3978622	-21	kworker/u16:3+events_unbound	00:29:49	6
3255228	99	42	40,2	cyclictest	3523856	-21	kworker/u16:1+events_unbound	22:04:59	6
3255218	99	42	40,2	cyclictest	3942630	-21	kworker/u16:1+events_unbound	00:18:41	4
3255218	99	42	40,2	cyclictest	3756768	-21	kworker/u16:1+events_unbound	23:53:01	4
3255218	99	42	40,2	cyclictest	3756768	-21	kworker/u16:1+events_unbound	23:53:01	4
3255218	99	42	40,2	cyclictest	3756768	-21	kworker/u16:1+events_unbound	22:59:54	4
3255218	99	42	40,2	cyclictest	3450803	-21	kworker/u16:3+events_unbound	21:19:21	4
3255230	99	41	36,4	cyclictest	3944912	-21	latency_hist	00:14:28	7
3255228	99	41	39,2	cyclictest	3882738	-21	kworker/u16:2+events_unbound	00:03:17	6
3255228	99	41	39,2	cyclictest	3523856	-21	kworker/u16:1+events_unbound	22:23:46	6
3255228	99	41	39,2	cyclictest	3523856	-21	kworker/u16:1+events_unbound	22:14:41	6
3255228	99	41	39,2	cyclictest	3481188	-21	kworker/u16:0+events_unbound	21:35:01	6
3255228	99	41	38,2	cyclictest	3374661	-21	kworker/u16:1+flush-8:0	20:29:28	6
3255225	99	41	39,2	cyclictest	3706558	-21	kworker/u16:5+events_unbound	22:44:41	5
3255225	99	41	38,2	cyclictest	3619305	-21	kworker/u16:5+flush-8:0	22:19:28	5
3255218	99	41	39,2	cyclictest	3942630	-21	kworker/u16:1+events_unbound	00:13:01	4
3255218	99	41	39,2	cyclictest	3706558	-21	kworker/u16:5+events_unbound	23:06:50	4
3255218	99	41	39,2	cyclictest	3619305	-21	kworker/u16:5+events_unbound	22:09:26	4
3255218	99	41	39,2	cyclictest	3583541	-21	kworker/u16:2+events_unbound	21:53:03	4
3255218	99	41	39,2	cyclictest	3583541	-21	kworker/u16:2+events_unbound	21:53:03	4
3255218	99	41	39,2	cyclictest	3523856	-21	kworker/u16:1+events_unbound	21:22:50	4
3255228	99	40	38,2	cyclictest	3942630	-21	kworker/u16:1+events_unbound	00:21:45	6
3255228	99	40	38,2	cyclictest	3942630	-21	kworker/u16:1+events_unbound	00:11:21	6
3255228	99	40	38,2	cyclictest	3882738	-21	kworker/u16:2+events_unbound	00:17:21	6
3255228	99	40	38,2	cyclictest	3786937	-21	kworker/u16:0+flush-8:0	23:45:01	6
3255228	99	40	38,2	cyclictest	3756768	-21	kworker/u16:1+events_unbound	23:40:34	6
3255228	99	40	38,2	cyclictest	3681643	-21	kworker/u16:2+events_unbound	22:37:58	6
3255228	99	40	38,2	cyclictest	3596896	-21	kworker/u16:0+events_unbound	21:49:13	6
3255228	99	40	38,2	cyclictest	3288155	-21	kworker/u16:2+events_unbound	21:21:14	6
3255225	99	40	37,2	cyclictest	3739089	-21	kworker/u16:3+events_unbound	22:59:30	5
3255218	99	40	38,2	cyclictest	3632728	-21	kworker/u16:0+flush-8:0	22:40:06	4
3255218	99	40	38,2	cyclictest	3632728	-21	kworker/u16:0+events_unbound	22:36:30	4
3245763	2	40	3,31	sleep7	0	-21	swapper/7	18:59:36	7
3255228	99	39	37,2	cyclictest	3786937	-21	kworker/u16:0+events_unbound	23:10:38	6
3255228	99	39	37,2	cyclictest	3786937	-21	kworker/u16:0+events_unbound	23:10:38	6
3255218	99	39	37,2	cyclictest	3632728	-21	kworker/u16:0+events_unbound	22:16:09	4
3255218	99	39	37,2	cyclictest	3523856	-21	kworker/u16:1+events_unbound	22:26:50	4
3956037	2	38	0,0	sleep7	3956038	-21	sh	00:17:50	7
3255228	99	38	36,2	cyclictest	3756768	-21	kworker/u16:1+events_unbound	23:37:29	6
3255225	99	38	36,2	cyclictest	3583541	-21	kworker/u16:2+flush-8:0	21:44:44	5
3255225	99	38	36,2	cyclictest	3523856	-21	kworker/u16:1+flush-8:0	21:17:14	5
3255218	99	38	36,2	cyclictest	3930934	-21	kworker/u16:0+events_unbound	00:06:22	4
3255218	99	38	36,2	cyclictest	3646683	-21	kworker/u16:2+events_unbound	22:13:33	4
3255218	99	38	36,2	cyclictest	3632728	-21	kworker/u16:0+events_unbound	22:20:29	4
3255218	99	38	36,2	cyclictest	3583541	-21	kworker/u16:2+flush-8:0	21:54:55	4
3255218	99	38	36,2	cyclictest	3523856	-21	kworker/u16:1+events_unbound	21:29:03	4
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17