OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Mon Jun 29, 2026 00:43:33)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
3831819	2	65	55,7	sleep5	0	-21	swapper/5	19:05:36	5
3832077	2	60	48,8	sleep4	0	-21	swapper/4	19:09:12	4
3832372	99	47	41,5	cyclictest	3953121	-21	latency_hist	20:09:14	7
3832361	99	44	42,2	cyclictest	4001526	-21	kworker/u16:4+events_unbound	22:44:15	4
3822765	2	44	7,30	sleep7	0	-21	swapper/7	19:04:18	7
3832372	99	41	36,4	cyclictest	4031071	-21	latency_hist	20:49:14	7
3832361	99	39	38,1	cyclictest	33454	-21	kworker/u16:0+events_unbound	22:10:44	4
3832361	99	39	37,2	cyclictest	4092501	-21	kworker/u16:3+events_unbound	21:16:32	4
3832361	99	39	36,2	cyclictest	33454	-21	kworker/u16:0+events_unbound	22:33:46	4
3832372	99	38	33,4	cyclictest	353709	-21	latency_hist	00:24:13	7
3832362	99	37	34,2	cyclictest	33454	-21	kworker/u16:0+flush-8:0	22:34:12	5
3832361	99	37	36,1	cyclictest	255953	-21	kworker/u16:3+events_unbound	00:18:52	4
3832361	99	37	35,2	cyclictest	354016	-21	kworker/u16:0+events_unbound	00:37:12	4
3832361	99	37	35,2	cyclictest	354016	-21	kworker/u16:0+events_unbound	00:37:12	4
3832361	99	37	35,2	cyclictest	155635	-21	kworker/u16:0+events_unbound	23:03:08	4
3832361	99	36	34,2	cyclictest	4001526	-21	kworker/u16:4+flush-8:0	21:09:13	4
3832361	99	36	34,2	cyclictest	4001526	-21	kworker/u16:4+events_unbound	21:43:15	4
3832361	99	36	34,2	cyclictest	155635	-21	kworker/u16:0+events_unbound	23:56:08	4
3832361	99	36	34,2	cyclictest	155635	-21	kworker/u16:0+events_unbound	23:56:08	4
3832361	99	36	34,2	cyclictest	155635	-21	kworker/u16:0+events_unbound	23:04:15	4
3832367	99	35	31,3	cyclictest	170646	-21	kworker/u16:5+flush-8:0	00:39:12	6
3832367	99	35	31,3	cyclictest	170646	-21	kworker/u16:5+flush-8:0	00:39:12	6
3832361	99	35	33,2	cyclictest	95528	-21	kworker/u16:2+flush-8:0	22:59:13	4
3832361	99	35	33,2	cyclictest	46643	-21	kworker/u16:1+events_unbound	22:20:40	4
3832361	99	35	33,2	cyclictest	170646	-21	kworker/u16:5+events_unbound	23:49:18	4
3832361	99	35	33,2	cyclictest	155635	-21	kworker/u16:0+events_unbound	23:39:10	4
3832361	99	35	33,2	cyclictest	121335	-21	kworker/u16:1+events_unbound	23:14:03	4
3832361	99	34	32,2	cyclictest	46643	-21	kworker/u16:1+events_unbound	22:26:16	4
3832361	99	34	32,2	cyclictest	4180220	-21	kworker/u16:2+events_unbound	22:03:44	4
3832361	99	34	32,2	cyclictest	4092501	-21	kworker/u16:3+events_unbound	21:29:16	4
3832361	99	34	32,2	cyclictest	233157	-21	kworker/u16:2+events_unbound	00:05:15	4
3832361	99	34	32,2	cyclictest	230306	-21	kworker/u16:1+events_unbound	23:33:12	4
3832361	99	34	32,2	cyclictest	155635	-21	kworker/u16:0+events_unbound	00:03:12	4
3832361	99	34	31,2	cyclictest	4001526	-21	kworker/u16:4+flush-8:0	21:44:13	4
3822764	2	34	24,7	sleep6	0	-21	swapper/6	19:04:17	6
3832361	99	33	31,2	cyclictest	95528	-21	kworker/u16:2+events_unbound	22:49:43	4
3832361	99	33	31,2	cyclictest	4001526	-21	kworker/u16:4+events_unbound	22:07:19	4
3832361	99	33	31,2	cyclictest	4001526	-21	kworker/u16:4+events_unbound	21:52:24	4
3832361	99	33	31,2	cyclictest	4001526	-21	kworker/u16:4+events_unbound	21:52:24	4
3832361	99	33	31,2	cyclictest	4001526	-21	kworker/u16:4+events_unbound	21:37:40	4
3832361	99	33	31,2	cyclictest	317142	-21	kworker/u16:1+events_unbound	00:11:40	4
3832361	99	33	31,2	cyclictest	170646	-21	kworker/u16:5+events_unbound	23:27:36	4
3832361	99	32	30,2	cyclictest	95528	-21	kworker/u16:2+events_unbound	22:40:15	4
3832361	99	32	30,2	cyclictest	4141203	-21	kworker/u16:0+events_unbound	21:58:19	4
3832361	99	32	30,2	cyclictest	4060778	-21	kworker/u16:0+events_unbound	21:13:03	4
3832361	99	32	30,2	cyclictest	4060778	-21	kworker/u16:0+events_unbound	21:13:03	4
3832361	99	32	30,2	cyclictest	3992492	-21	kworker/u16:2+events_unbound	21:26:32	4
3832361	99	32	30,2	cyclictest	170646	-21	kworker/u16:5+events_unbound	00:20:24	4
3832361	99	32	29,2	cyclictest	95528	-21	kworker/u16:2+events_unbound	22:38:06	4
3832372	99	31	30,0	cyclictest	0	-21	swapper/7	21:04:43	7
3832372	99	31	25,5	cyclictest	4011623	-21	latency_hist	20:39:13	7
3832361	99	31	30,1	cyclictest	170646	-21	kworker/u16:5+events_unbound	23:48:39	4
3832361	99	31	29,2	cyclictest	4094798	-21	kworker/u16:1+events_unbound	21:23:24	4
3832361	99	31	29,2	cyclictest	255953	-21	kworker/u16:3+events_unbound	00:33:22	4
3832367	99	30	26,3	cyclictest	155635	-21	kworker/u16:0+flush-8:0	23:14:13	6
3832362	99	30	28,2	cyclictest	155635	-21	kworker/u16:0+flush-8:0	23:09:42	5
3832361	99	30	28,2	cyclictest	33454	-21	kworker/u16:0+events_unbound	22:18:00	4
3832361	99	30	28,2	cyclictest	255953	-21	kworker/u16:3+events_unbound	00:28:27	4
3832361	99	30	28,2	cyclictest	155635	-21	kworker/u16:0+events_unbound	23:21:11	4
3832361	99	30	27,2	cyclictest	3792311	-21	kworker/u16:3+events_unbound	19:19:15	4
3832361	99	30	27,2	cyclictest	155635	-21	kworker/u16:0+events_unbound	23:39:22	4
3832361	99	29	27,2	cyclictest	3893600	-21	kworker/u16:1+events_unbound	19:54:08	4
3832361	99	29	27,2	cyclictest	155635	-21	kworker/u16:0+events_unbound	23:17:15	4
3832372	99	28	27,0	cyclictest	0	-21	swapper/7	22:14:27	7
3832367	99	28	25,2	cyclictest	170646	-21	kworker/u16:5+flush-8:0	23:29:18	6
3832362	99	28	26,2	cyclictest	170646	-21	kworker/u16:5+events_unbound	00:19:27	5
3832367	99	27	25,2	cyclictest	4001526	-21	kworker/u16:4+events_unbound	21:49:28	6
3832367	99	27	25,2	cyclictest	4001526	-21	kworker/u16:4+events_unbound	21:49:28	6
3832372	99	26	24,1	cyclictest	0	-21	swapper/7	23:19:31	7
3832362	99	26	24,2	cyclictest	3992492	-21	kworker/u16:2+flush-8:0	21:24:29	5
3832362	99	25	23,2	cyclictest	3778371	-21	kworker/u16:0+events_unbound	20:17:25	5
3832362	99	25	22,2	cyclictest	4153417	-21	kworker/u16:1+events_unbound	21:49:24	5
3832362	99	25	22,2	cyclictest	4153417	-21	kworker/u16:1+events_unbound	21:49:24	5
3832361	99	25	23,2	cyclictest	3792311	-21	kworker/u16:3+flush-8:0	19:29:08	4
3832361	99	25	23,2	cyclictest	3778371	-21	kworker/u16:0+events_unbound	20:24:43	4
3832362	99	24	22,2	cyclictest	3778371	-21	kworker/u16:0+events_unbound	19:54:29	5
3832372	99	23	0,21	cyclictest	911	-21	dbus-daemon	22:54:13	7
3832367	99	23	20,3	cyclictest	3893600	-21	kworker/u16:1+flush-8:0	19:44:29	6
3832372	99	22	20,1	cyclictest	0	-21	swapper/7	22:59:28	7
3832362	99	22	20,2	cyclictest	4001526	-21	kworker/u16:4+flush-8:0	21:14:43	5
3832361	99	22	20,2	cyclictest	3934003	-21	kworker/u16:3+flush-8:0	20:09:09	4
3832372	99	21	20,0	cyclictest	0	-21	swapper/7	21:44:43	7
3832372	99	21	0,19	cyclictest	650	-21	systemd-journal	21:14:08	7
3832372	99	21	0,19	cyclictest	650	-21	systemd-journal	21:14:08	7
3832367	99	21	19,2	cyclictest	3934003	-21	kworker/u16:3+flush-8:0	19:59:39	6
3832361	99	21	19,2	cyclictest	3778371	-21	kworker/u16:0+events_unbound	19:18:24	4
3832361	99	21	18,2	cyclictest	3778371	-21	kworker/u16:0+events_unbound	20:54:23	4
3832372	99	20	18,1	cyclictest	0	-21	swapper/7	23:39:42	7
3832367	99	20	18,2	cyclictest	3832572	-21	kworker/u16:1+flush-8:0	19:09:26	6
3832362	99	20	18,2	cyclictest	3778371	-21	kworker/u16:0+events_unbound	20:00:52	5
3832362	99	20	18,2	cyclictest	354016	-21	kworker/u16:0+events_unbound	00:24:42	5
3832362	99	20	18,2	cyclictest	155635	-21	kworker/u16:0+events_unbound	23:39:08	5
3832361	99	20	18,2	cyclictest	4001526	-21	kworker/u16:4+events_unbound	20:32:28	4
3832372	99	19	18,0	cyclictest	0	-21	swapper/7	22:24:30	7
3832372	99	19	17,1	cyclictest	0	-21	swapper/7	00:14:32	7
3832367	99	19	18,1	cyclictest	3778371	-21	kworker/u16:0+events_unbound	20:54:39	6
3832362	99	19	18,1	cyclictest	33454	-21	kworker/u16:0+events_unbound	22:23:11	5
3832362	99	19	17,2	cyclictest	4153417	-21	kworker/u16:1+flush-8:0	21:54:48	5
3832362	99	19	17,2	cyclictest	3934003	-21	kworker/u16:3+events_unbound	20:37:05	5
3832362	99	19	17,2	cyclictest	3792311	-21	kworker/u16:3+flush-8:0	19:19:29	5
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17