OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Fri Nov 21, 2025 00:43:33)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
908815	99	53	50,2	cyclictest	1558838	-21	kworker/u16:3+flush-8:0	08:43:36	6
908819	99	52	47,4	cyclictest	1033988	-21	latency_hist	04:23:37	7
908819	99	52	47,4	cyclictest	1033988	-21	latency_hist	04:23:37	7
908815	99	51	49,1	cyclictest	1165059	-21	kworker/u16:0+flush-8:0	05:48:36	6
908819	99	50	45,4	cyclictest	1529284	-21	latency_hist	08:28:37	7
908819	99	50	43,6	cyclictest	589235	-21	systemd-journal	05:48:37	7
908295	2	46	35,8	sleep4	0	-21	swapper/4	03:21:00	4
908813	99	42	40,2	cyclictest	202842	-21	kworker/u16:2+events_unbound	04:03:37	5
908376	2	41	31,7	sleep6	0	-21	swapper/6	03:22:07	6
1402614	2	41	0,0	sleep7	1402615	-21	cat	07:23:57	7
908813	99	40	37,2	cyclictest	1296618	-21	kworker/u16:4+events_unbound	06:58:51	5
908809	99	40	38,2	cyclictest	1498845	-21	kworker/u16:1+events_unbound	08:14:07	4
898601	2	40	3,31	sleep7	0	-21	swapper/7	03:18:39	7
908809	99	38	35,2	cyclictest	1165059	-21	kworker/u16:0+flush-8:0	05:28:53	4
908819	99	37	35,1	cyclictest	0	-21	swapper/7	06:18:55	7
908813	99	37	34,2	cyclictest	213150	-21	kworker/u16:3+flush-8:0	03:43:38	5
908809	99	37	34,2	cyclictest	1417864	-21	kworker/u16:4+events_unbound	07:33:54	4
908815	99	36	34,2	cyclictest	1387563	-21	kworker/u16:2+flush-8:0	07:34:06	6
908813	99	36	34,2	cyclictest	1458906	-21	kworker/u16:4+events_unbound	08:03:28	5
908809	99	36	32,3	cyclictest	1064317	-21	kworker/u16:5+flush-8:0	04:58:57	4
908809	99	35	31,3	cyclictest	1347598	-21	kworker/u16:3+flush-8:0	07:54:09	4
908813	99	34	32,2	cyclictest	1347598	-21	kworker/u16:3+events_unbound	07:39:36	5
908819	99	33	30,2	cyclictest	1466005	-21	cut	07:53:58	7
908809	99	33	30,2	cyclictest	1347598	-21	kworker/u16:3+events_unbound	07:38:51	4
908813	99	32	30,2	cyclictest	1347598	-21	kworker/u16:3+events_unbound	07:24:05	5
908334	2	32	22,7	sleep5	0	-21	swapper/5	03:21:34	5
908819	99	31	29,1	cyclictest	0	-21	swapper/7	06:33:52	7
908819	99	31	29,1	cyclictest	0	-21	swapper/7	06:33:52	7
908813	99	31	29,2	cyclictest	973362	-21	kworker/u16:1+flush-8:0	04:51:08	5
908813	99	31	29,2	cyclictest	1519187	-21	kworker/u16:5+events_unbound	08:49:51	5
908813	99	31	29,2	cyclictest	1519187	-21	kworker/u16:5+events_unbound	08:49:51	5
908813	99	31	29,2	cyclictest	1165059	-21	kworker/u16:0+events_unbound	05:45:24	5
908809	99	31	28,2	cyclictest	1387563	-21	kworker/u16:2+flush-8:0	08:03:52	4
908809	99	31	28,2	cyclictest	1387563	-21	kworker/u16:2+flush-8:0	08:03:52	4
908819	99	30	0,29	cyclictest	1064281	-21	latency_hist	04:38:37	7
908813	99	30	28,2	cyclictest	973362	-21	kworker/u16:1+events_unbound	05:54:51	5
908813	99	30	28,2	cyclictest	973362	-21	kworker/u16:1+events_unbound	05:43:36	5
908813	99	30	28,2	cyclictest	942953	-21	kworker/u16:0+flush-8:0	04:09:21	5
908813	99	30	28,2	cyclictest	213150	-21	kworker/u16:3+flush-8:0	03:25:17	5
908813	99	30	28,2	cyclictest	1508549	-21	kworker/u16:3+flush-8:0	08:20:07	5
908813	99	30	28,2	cyclictest	1448818	-21	kworker/u16:0+events_unbound	08:10:40	5
908813	99	30	28,2	cyclictest	1215937	-21	kworker/u16:2+events_unbound	06:05:00	5
908813	99	30	28,2	cyclictest	1175604	-21	kworker/u16:3+events_unbound	05:51:04	5
908813	99	30	28,2	cyclictest	1175604	-21	kworker/u16:3+events_unbound	05:51:04	5
908813	99	30	28,2	cyclictest	1165059	-21	kworker/u16:0+events_unbound	06:55:51	5
908813	99	30	28,2	cyclictest	1064317	-21	kworker/u16:5+events_unbound	04:57:04	5
908819	99	29	28,0	cyclictest	0	-21	swapper/7	07:13:58	7
908819	99	29	28,0	cyclictest	0	-21	swapper/7	04:13:52	7
908819	99	29	0,27	cyclictest	589235	-21	systemd-journal	07:53:14	7
908813	99	29	27,2	cyclictest	973362	-21	kworker/u16:1+events_unbound	04:14:41	5
908813	99	29	27,2	cyclictest	942953	-21	kworker/u16:0+events_unbound	04:06:13	5
908813	99	29	27,2	cyclictest	202842	-21	kworker/u16:2+events_unbound	03:41:32	5
908813	99	29	27,2	cyclictest	1387563	-21	kworker/u16:2+events_unbound	07:30:52	5
908813	99	29	27,2	cyclictest	1296618	-21	kworker/u16:4+flush-8:0	06:36:52	5
908813	99	29	27,2	cyclictest	1296618	-21	kworker/u16:4+flush-8:0	06:36:52	5
908813	99	29	27,2	cyclictest	1175246	-21	kworker/u16:2+events_unbound	05:32:24	5
908813	99	29	27,2	cyclictest	1165059	-21	kworker/u16:0+flush-8:0	06:02:48	5
908813	99	29	27,2	cyclictest	1064317	-21	kworker/u16:5+events_unbound	05:00:25	5
908813	99	29	27,2	cyclictest	1064241	-21	kworker/u16:2+events_unbound	05:21:59	5
908813	99	29	27,2	cyclictest	1064241	-21	kworker/u16:2+events_unbound	05:17:25	5
908813	99	29	27,2	cyclictest	1053951	-21	kworker/u16:3+events_unbound	04:33:01	5
908813	99	29	27,2	cyclictest	1043824	-21	kworker/u16:0+events_unbound	04:27:57	5
914221	2	28	0,1	sleep6	0	-21	swapper/6	03:23:58	6
908819	99	28	27,0	cyclictest	0	-21	swapper/7	08:08:52	7
908815	99	28	24,3	cyclictest	1064241	-21	kworker/u16:2+flush-8:0	04:43:37	6
908813	99	28	26,2	cyclictest	973362	-21	kworker/u16:1+flush-8:0	04:34:40	5
908813	99	28	26,2	cyclictest	213150	-21	kworker/u16:3+events_unbound	03:30:55	5
908813	99	28	26,2	cyclictest	1498845	-21	kworker/u16:1+events_unbound	08:16:16	5
908813	99	28	26,2	cyclictest	1417793	-21	kworker/u16:1+events_unbound	07:37:11	5
908813	99	28	26,2	cyclictest	1387563	-21	kworker/u16:2+events_unbound	07:21:43	5
908813	99	28	26,2	cyclictest	1387563	-21	kworker/u16:2+events_unbound	07:21:43	5
908813	99	28	26,2	cyclictest	1347598	-21	kworker/u16:3+events_unbound	07:58:35	5
908813	99	28	26,2	cyclictest	1296618	-21	kworker/u16:4+flush-8:0	07:07:27	5
908813	99	28	26,2	cyclictest	1296618	-21	kworker/u16:4+events_unbound	07:14:44	5
908813	99	28	26,2	cyclictest	1296618	-21	kworker/u16:4+events_unbound	06:40:40	5
908813	99	28	26,2	cyclictest	1266240	-21	kworker/u16:3+events_unbound	06:19:44	5
908813	99	28	26,2	cyclictest	1064241	-21	kworker/u16:2+events_unbound	04:39:36	5
908813	99	28	26,2	cyclictest	1023848	-21	kworker/u16:2+flush-8:0	04:18:38	5
908813	99	28	26,2	cyclictest	1023848	-21	kworker/u16:2+flush-8:0	04:18:38	5
908813	99	28	25,2	cyclictest	1165059	-21	kworker/u16:0+events_unbound	06:08:41	5
908813	99	28	25,2	cyclictest	1114946	-21	kworker/u16:0+flush-8:0	05:08:48	5
908819	99	27	26,0	cyclictest	0	-21	swapper/7	07:59:05	7
908819	99	27	26,0	cyclictest	0	-21	swapper/7	05:38:59	7
908815	99	27	25,2	cyclictest	1134880	-21	kworker/u16:3+events_unbound	05:19:01	6
908813	99	27	25,2	cyclictest	953562	-21	kworker/u16:4+events_unbound	03:52:44	5
908813	99	27	25,2	cyclictest	1539597	-21	kworker/u16:2+events_unbound	08:44:23	5
908813	99	27	25,2	cyclictest	1498845	-21	kworker/u16:1+flush-8:0	08:33:47	5
908813	99	27	25,2	cyclictest	1448818	-21	kworker/u16:0+events_unbound	08:32:31	5
908813	99	27	25,2	cyclictest	1367535	-21	kworker/u16:0+events_unbound	07:12:55	5
908819	99	26	25,0	cyclictest	0	-21	swapper/7	08:18:49	7
908819	99	26	24,1	cyclictest	0	-21	swapper/7	03:53:53	7
908813	99	26	24,2	cyclictest	942953	-21	kworker/u16:0+events_unbound	03:54:47	5
908813	99	26	24,2	cyclictest	202842	-21	kworker/u16:2+flush-8:0	03:36:28	5
908813	99	26	24,2	cyclictest	202842	-21	kworker/u16:2+flush-8:0	03:36:28	5
908813	99	26	24,2	cyclictest	1448818	-21	kworker/u16:0+flush-8:0	07:53:19	5
908813	99	26	24,2	cyclictest	1448818	-21	kworker/u16:0+events_unbound	08:23:44	5
908813	99	26	24,2	cyclictest	1266240	-21	kworker/u16:3+events_unbound	06:30:17	5
908813	99	26	24,2	cyclictest	1175604	-21	kworker/u16:3+events_unbound	05:34:29	5
908813	99	26	24,2	cyclictest	1165059	-21	kworker/u16:0+flush-8:0	06:49:07	5
908813	99	25	23,2	cyclictest	1448818	-21	kworker/u16:0+events_unbound	08:05:35	5
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17