OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Sat Dec 27, 2025 12:43:32)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
1802997	2	46	36,7	sleep6	0	-21	swapper/6	07:06:49	6
1803005	2	44	32,9	sleep5	0	-21	swapper/5	07:06:56	5
1803111	2	40	6,30	sleep7	0	-21	swapper/7	07:08:27	7
1803031	2	39	27,8	sleep4	0	-21	swapper/4	07:07:19	4
1803459	99	36	29,6	cyclictest	2185978	-21	latency_hist	10:23:59	7
1803450	99	35	33,2	cyclictest	1784108	-21	kworker/u16:3+flush-8:0	07:09:14	4
1803450	99	33	31,2	cyclictest	2118398	-21	kworker/u16:0+events_unbound	10:24:15	4
1803459	99	31	25,5	cyclictest	2079144	-21	latency_hist	09:29:00	7
2224667	2	30	0,0	chrt	2224665	-21	sshd-session	10:42:18	7
1803458	99	30	28,2	cyclictest	2233617	-21	kworker/u16:1+events_unbound	10:44:25	6
1803458	99	30	28,2	cyclictest	1924387	-21	kworker/u16:0+events_unbound	08:11:06	6
1803458	99	29	27,2	cyclictest	2118398	-21	kworker/u16:0+events_unbound	09:53:17	6
1803458	99	29	27,2	cyclictest	2118398	-21	kworker/u16:0+events_unbound	09:53:17	6
1803458	99	28	26,2	cyclictest	1835976	-21	kworker/u16:4+events_unbound	07:30:23	6
1803458	99	28	26,2	cyclictest	1535055	-21	kworker/u16:2+events_unbound	07:39:54	6
1803450	99	28	25,2	cyclictest	1903966	-21	kworker/u16:3+events_unbound	09:54:19	4
1803458	99	27	25,2	cyclictest	2430545	-21	kworker/u16:1+flush-8:0	12:37:25	6
1803458	99	27	25,2	cyclictest	2301648	-21	kworker/u16:3+events_unbound	11:21:54	6
1803458	99	27	25,2	cyclictest	2156860	-21	kworker/u16:3+flush-8:0	11:09:00	6
1803458	99	27	25,2	cyclictest	2156860	-21	kworker/u16:3+events_unbound	10:59:43	6
1803458	99	26	24,2	cyclictest	2312017	-21	kworker/u16:2+events_unbound	11:26:43	6
1803458	99	26	24,2	cyclictest	2312017	-21	kworker/u16:2+events_unbound	11:26:43	6
1803458	99	26	24,2	cyclictest	1942808	-21	kworker/u16:1+events_unbound	09:32:18	6
1803458	99	26	24,2	cyclictest	1835976	-21	kworker/u16:4+flush-8:0	07:25:58	6
1803458	99	26	24,2	cyclictest	1803213	-21	kworker/u16:1+events_unbound	07:13:54	6
1803450	99	26	24,2	cyclictest	2030747	-21	kworker/u16:0+events_unbound	09:04:15	4
1803458	99	25	23,2	cyclictest	2361000	-21	kworker/u16:5+events_unbound	12:18:17	6
1803458	99	25	23,2	cyclictest	2361000	-21	kworker/u16:5+events_unbound	12:02:45	6
1803458	99	25	23,2	cyclictest	2312017	-21	kworker/u16:2+flush-8:0	11:39:30	6
1803458	99	25	23,2	cyclictest	2254388	-21	kworker/u16:2+events_unbound	11:15:06	6
1803458	99	25	23,2	cyclictest	2156860	-21	kworker/u16:3+events_unbound	11:13:22	6
1803458	99	25	23,2	cyclictest	2020674	-21	kworker/u16:2+events_unbound	09:38:57	6
1803458	99	25	23,2	cyclictest	1942808	-21	kworker/u16:1+events_unbound	10:09:26	6
1803458	99	25	23,2	cyclictest	1942808	-21	kworker/u16:1+events_unbound	08:59:46	6
1803458	99	25	23,2	cyclictest	1903966	-21	kworker/u16:3+events_unbound	08:41:37	6
1803458	99	25	23,2	cyclictest	1784108	-21	kworker/u16:3+events_unbound	07:23:03	6
1803450	99	25	23,2	cyclictest	2254388	-21	kworker/u16:2+events_unbound	11:04:27	4
1803450	99	25	23,2	cyclictest	1535055	-21	kworker/u16:2+events_unbound	08:14:12	4
1803450	99	25	23,2	cyclictest	1535055	-21	kworker/u16:2+events_unbound	08:14:12	4
1803458	99	24	22,2	cyclictest	2361000	-21	kworker/u16:5+events_unbound	12:11:34	6
1803458	99	24	22,2	cyclictest	2361000	-21	kworker/u16:5+events_unbound	12:07:13	6
1803458	99	24	22,2	cyclictest	2312017	-21	kworker/u16:2+events_unbound	11:52:06	6
1803458	99	24	22,2	cyclictest	2200556	-21	kworker/u16:2+events_unbound	10:33:13	6
1803458	99	24	22,2	cyclictest	2118398	-21	kworker/u16:0+events_unbound	10:20:17	6
1803458	99	24	22,2	cyclictest	1942808	-21	kworker/u16:1+events_unbound	09:41:23	6
1803458	99	24	22,2	cyclictest	1942808	-21	kworker/u16:1+events_unbound	09:13:58	6
1803458	99	24	22,2	cyclictest	1924387	-21	kworker/u16:0+events_unbound	08:23:54	6
1803458	99	24	22,2	cyclictest	1903966	-21	kworker/u16:3+events_unbound	08:16:34	6
1803458	99	24	22,2	cyclictest	1903966	-21	kworker/u16:3+events_unbound	08:16:34	6
1803458	99	24	22,2	cyclictest	1784108	-21	kworker/u16:3+events_unbound	07:17:23	6
1803458	99	23	21,2	cyclictest	2312017	-21	kworker/u16:2+events_unbound	12:33:57	6
1803458	99	23	21,2	cyclictest	2284099	-21	kworker/u16:1+events_unbound	11:48:25	6
1803458	99	23	21,2	cyclictest	2273455	-21	kworker/u16:0+events_unbound	11:33:22	6
1803458	99	23	21,2	cyclictest	2216224	-21	kworker/u16:4+events_unbound	10:53:18	6
1803458	99	23	21,2	cyclictest	2020674	-21	kworker/u16:2+events_unbound	09:17:58	6
1803458	99	23	21,2	cyclictest	1942808	-21	kworker/u16:1+events_unbound	09:28:35	6
1803458	99	23	21,2	cyclictest	1942808	-21	kworker/u16:1+events_unbound	08:35:22	6
1803450	99	23	21,2	cyclictest	2430545	-21	kworker/u16:1+flush-8:0	12:33:54	4
1803450	99	23	21,2	cyclictest	2020674	-21	kworker/u16:2+flush-8:0	09:38:54	4
1803450	99	23	21,2	cyclictest	1942808	-21	kworker/u16:1+events_unbound	09:09:22	4
1803450	99	23	21,2	cyclictest	1924387	-21	kworker/u16:0+events_unbound	08:29:26	4
1803450	99	23	21,2	cyclictest	1835976	-21	kworker/u16:4+flush-8:0	07:44:30	4
1803458	99	22	20,2	cyclictest	2312017	-21	kworker/u16:2+events_unbound	11:58:25	6
1803458	99	22	20,2	cyclictest	2020674	-21	kworker/u16:2+flush-8:0	09:06:13	6
1803458	99	22	20,2	cyclictest	2010863	-21	kworker/u16:0+events_unbound	08:52:18	6
1803458	99	22	20,2	cyclictest	1942808	-21	kworker/u16:1+events_unbound	08:32:46	6
1803458	99	22	20,2	cyclictest	1903966	-21	kworker/u16:3+events_unbound	08:26:07	6
1803458	99	22	20,2	cyclictest	1903966	-21	kworker/u16:3+events_unbound	07:57:06	6
1803458	99	22	20,2	cyclictest	1535055	-21	kworker/u16:2+events_unbound	07:52:22	6
1803450	99	22	20,2	cyclictest	2284099	-21	kworker/u16:1+flush-8:0	11:24:25	4
1803450	99	22	20,2	cyclictest	2284099	-21	kworker/u16:1+flush-8:0	11:24:25	4
1803450	99	22	20,2	cyclictest	2273455	-21	kworker/u16:0+flush-8:0	11:33:54	4
1803450	99	22	20,2	cyclictest	2118398	-21	kworker/u16:0+events_unbound	10:49:18	4
1803450	99	22	20,2	cyclictest	1903966	-21	kworker/u16:3+flush-8:0	09:59:13	4
1803459	99	21	0,19	cyclictest	2370611	-21	latency_hist	11:58:59	7
1803458	99	21	19,2	cyclictest	2312017	-21	kworker/u16:2+events_unbound	12:26:49	6
1803458	99	21	19,2	cyclictest	2146830	-21	kworker/u16:2+events_unbound	10:00:58	6
1803458	99	21	19,2	cyclictest	1942808	-21	kworker/u16:1+events_unbound	10:17:34	6
1803458	99	21	19,2	cyclictest	1942808	-21	kworker/u16:1+events_unbound	08:46:07	6
1803458	99	21	19,2	cyclictest	1903966	-21	kworker/u16:3+events_unbound	09:57:54	6
1803458	99	21	19,2	cyclictest	1903966	-21	kworker/u16:3+events_unbound	09:46:18	6
1803458	99	21	19,2	cyclictest	1903966	-21	kworker/u16:3+events_unbound	08:03:25	6
1803458	99	21	19,2	cyclictest	1875529	-21	kworker/u16:1+events_unbound	08:06:34	6
1803458	99	21	19,2	cyclictest	1535055	-21	kworker/u16:2+events_unbound	07:36:58	6
1803450	99	21	19,2	cyclictest	2284099	-21	kworker/u16:1+flush-8:0	11:34:25	4
1803450	99	21	19,2	cyclictest	2233617	-21	kworker/u16:1+events_unbound	10:54:25	4
1803450	99	21	19,2	cyclictest	2118398	-21	kworker/u16:0+flush-8:0	10:34:22	4
1803450	99	21	19,2	cyclictest	2020674	-21	kworker/u16:2+events_unbound	09:29:22	4
1803450	99	21	19,2	cyclictest	1942808	-21	kworker/u16:1+flush-8:0	08:28:55	4
1803450	99	21	19,2	cyclictest	1903966	-21	kworker/u16:3+events_unbound	09:44:19	4
1803458	99	20	18,2	cyclictest	2233617	-21	kworker/u16:1+events_unbound	10:58:49	6
1803458	99	20	18,2	cyclictest	1903966	-21	kworker/u16:3+flush-8:0	08:54:42	6
1803458	99	20	18,2	cyclictest	1835976	-21	kworker/u16:4+events_unbound	07:45:59	6
1803450	99	20	18,2	cyclictest	2312017	-21	kworker/u16:2+events_unbound	11:44:34	4
1803450	99	20	18,2	cyclictest	2273455	-21	kworker/u16:0+flush-8:0	11:19:13	4
1803450	99	20	18,2	cyclictest	1962945	-21	kworker/u16:4+events_unbound	08:39:28	4
1803450	99	20	18,2	cyclictest	1924387	-21	kworker/u16:0+events_unbound	08:34:27	4
1803450	99	20	18,2	cyclictest	1803213	-21	kworker/u16:1+events_unbound	07:24:14	4
1803458	99	19	17,2	cyclictest	2284099	-21	kworker/u16:1+events_unbound	11:36:18	6
1803458	99	19	17,2	cyclictest	2118398	-21	kworker/u16:0+events_unbound	10:36:30	6
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17