OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Wed Jul 15, 2026 00:43:29)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
2136671	2	46	35,8	sleep4	0	-21	swapper/4	19:07:11	4
2136563	2	40	3,31	sleep7	0	-21	swapper/7	19:05:38	7
2136683	2	38	27,8	sleep5	0	-21	swapper/5	19:07:20	5
2127627	2	36	24,8	sleep6	0	-21	swapper/6	19:05:04	6
2137152	99	30	26,3	cyclictest	2557849	-21	cat	22:24:55	5
2137157	99	29	26,2	cyclictest	2133482	-21	kworker/u16:0+flush-8:0	20:04:55	6
2137149	99	27	23,3	cyclictest	2208951	-21	latency_hist	19:44:55	4
2137149	99	25	23,2	cyclictest	2701716	-21	kworker/u16:0+flush-8:0	23:54:55	4
2137149	99	25	22,2	cyclictest	2851009	-21	kworker/u16:0+flush-8:0	00:34:54	4
2137157	99	24	22,2	cyclictest	2423734	-21	kworker/u16:3+flush-8:0	22:35:11	6
2137157	99	24	22,2	cyclictest	2365015	-21	kworker/u16:2+events_unbound	21:35:46	6
2137157	99	24	22,2	cyclictest	2316401	-21	kworker/u16:3+flush-8:0	20:40:21	6
2137157	99	23	21,2	cyclictest	2316401	-21	kworker/u16:3+flush-8:0	20:50:20	6
2137157	99	23	21,2	cyclictest	2150951	-21	kworker/u16:1+flush-8:0	19:20:22	6
2137162	99	22	0,21	cyclictest	2887298	-21	turbostat	00:39:54	7
2137162	99	22	0,20	cyclictest	2408110	-21	ssh	21:20:21	7
2137157	99	22	20,2	cyclictest	2814442	-21	kworker/u16:4+flush-8:0	00:15:15	6
2137157	99	22	20,2	cyclictest	2814442	-21	kworker/u16:4+events_unbound	00:20:05	6
2137157	99	22	20,2	cyclictest	2568559	-21	kworker/u16:1+events_unbound	22:30:00	6
2137157	99	22	20,2	cyclictest	2496920	-21	kworker/u16:4+events_unbound	22:05:05	6
2137157	99	22	20,2	cyclictest	2383563	-21	kworker/u16:0+flush-8:0	21:15:38	6
2137157	99	22	20,2	cyclictest	2292689	-21	kworker/u16:4+events_unbound	20:39:13	6
2137157	99	22	20,2	cyclictest	2292689	-21	kworker/u16:4+events_unbound	20:34:01	6
2137152	99	22	17,4	cyclictest	2717602	-21	cat	23:30:11	5
2137157	99	21	19,2	cyclictest	2814441	-21	kworker/u16:2+flush-8:0	00:30:09	6
2137157	99	21	19,2	cyclictest	2355276	-21	kworker/u16:0+flush-8:0	21:00:16	6
2137157	99	21	19,2	cyclictest	2227533	-21	kworker/u16:3+flush-8:0	20:09:33	6
2137149	99	21	16,4	cyclictest	2354899	-21	latency_hist	20:59:55	4
2137157	99	20	18,2	cyclictest	2640804	-21	kworker/u16:1+flush-8:0	23:15:26	6
2137157	99	20	18,2	cyclictest	2159402	-21	kworker/u16:2+events_unbound	19:34:09	6
2137149	99	20	16,3	cyclictest	2533476	-21	cat	22:14:55	4
2137162	99	19	0,17	cyclictest	0	-21	swapper/7	23:10:21	7
2137157	99	19	17,2	cyclictest	2640804	-21	kworker/u16:1+flush-8:0	23:00:12	6
2137157	99	19	17,2	cyclictest	2344393	-21	kworker/u16:1+events_unbound	21:10:10	6
2137157	99	19	17,2	cyclictest	2159402	-21	kworker/u16:2+flush-8:0	19:27:37	6
2137152	99	19	15,3	cyclictest	2436943	-21	cat	21:35:12	5
2137162	99	18	17,0	cyclictest	0	-21	swapper/7	19:29:56	7
2137162	99	18	0,17	cyclictest	0	-21	swapper/7	19:35:20	7
2137162	99	18	0,16	cyclictest	1302	-21	snmpd	21:14:36	7
2137162	99	18	0,16	cyclictest	0	-21	swapper/7	23:40:19	7
2137157	99	18	16,2	cyclictest	2533585	-21	kworker/u16:2+events_unbound	23:05:34	6
2137157	99	18	16,2	cyclictest	2383563	-21	kworker/u16:0+events_unbound	22:20:00	6
2137157	99	18	16,2	cyclictest	2247617	-21	kworker/u16:2+flush-8:0	20:28:45	6
2137157	99	18	16,2	cyclictest	2247617	-21	kworker/u16:2+flush-8:0	20:13:13	6
2137157	99	18	16,2	cyclictest	2150951	-21	kworker/u16:1+flush-8:0	19:16:01	6
2137152	99	18	14,3	cyclictest	2367625	-21	kthreadcore	21:05:13	5
2137162	99	17	13,3	cyclictest	1302	-21	snmpd	19:59:55	7
2137157	99	17	15,2	cyclictest	2814441	-21	kworker/u16:2+events_unbound	00:10:23	6
2137157	99	17	15,2	cyclictest	2423734	-21	kworker/u16:3+flush-8:0	22:00:00	6
2137157	99	17	15,2	cyclictest	2386990	-21	kworker/u16:5+flush-8:0	21:25:48	6
2137157	99	17	15,2	cyclictest	2316401	-21	kworker/u16:3+events_unbound	20:59:31	6
2137162	99	16	13,2	cyclictest	1302	-21	snmpd	22:13:22	7
2137162	99	16	0,14	cyclictest	648	-21	systemd-journal	22:19:55	7
2137162	99	16	0,14	cyclictest	2688787	-21	ssh	23:15:19	7
2137162	99	16	0,14	cyclictest	2505749	-21	ssh	22:00:19	7
2137162	99	16	0,14	cyclictest	0	-21	swapper/7	22:25:19	7
2137157	99	16	14,2	cyclictest	2701716	-21	kworker/u16:0+flush-8:0	00:08:28	6
2137157	99	16	14,2	cyclictest	2496920	-21	kworker/u16:4+events_unbound	22:10:06	6
2137157	99	16	14,2	cyclictest	2292689	-21	kworker/u16:4+events_unbound	21:20:43	6
2137157	99	16	14,2	cyclictest	2159402	-21	kworker/u16:2+events_unbound	19:37:53	6
2137157	99	16	14,2	cyclictest	2133482	-21	kworker/u16:0+flush-8:0	20:45:59	6
2137157	99	16	14,2	cyclictest	2133482	-21	kworker/u16:0+flush-8:0	20:45:59	6
2137157	99	16	12,3	cyclictest	2286815	-21	latency_hist	20:24:55	6
2137149	99	16	13,2	cyclictest	2448165	-21	kworker/u16:1+flush-8:0	21:45:13	4
2137162	99	15	13,1	cyclictest	0	-21	swapper/7	00:14:54	7
2137162	99	15	0,14	cyclictest	0	-21	swapper/7	19:20:21	7
2137162	99	15	0,13	cyclictest	2354901	-21	turbostat	20:59:55	7
2137157	99	15	13,2	cyclictest	2787194	-21	kworker/u16:1+events_unbound	00:01:39	6
2137157	99	15	13,2	cyclictest	2640804	-21	kworker/u16:1+events_unbound	23:29:13	6
2137157	99	15	13,2	cyclictest	2533585	-21	kworker/u16:2+events_unbound	22:57:18	6
2137157	99	15	13,2	cyclictest	2383563	-21	kworker/u16:0+events_unbound	22:44:26	6
2137157	99	15	13,2	cyclictest	2292689	-21	kworker/u16:4+flush-8:0	21:05:36	6
2137157	99	15	13,2	cyclictest	2247617	-21	kworker/u16:2+flush-8:0	20:15:57	6
2137157	99	15	13,2	cyclictest	2133482	-21	kworker/u16:0+events_unbound	19:47:49	6
2137157	99	15	0,2	cyclictest	0	-21	swapper/6	19:40:11	6
2137152	99	15	14,1	cyclictest	0	-21	swapper/5	22:05:45	5
2137152	99	15	12,2	cyclictest	2640804	-21	kworker/u16:1+flush-8:0	23:05:17	5
2137152	99	15	0,1	cyclictest	0	-21	swapper/5	19:20:22	5
2137152	99	15	0,14	cyclictest	0	-21	swapper/5	22:55:06	5
2137149	99	15	12,2	cyclictest	2640804	-21	kworker/u16:1+flush-8:0	23:05:12	4
2137149	99	15	12,2	cyclictest	2423734	-21	kworker/u16:3+flush-8:0	22:55:06	4
2137162	99	14	8,5	cyclictest	648	-21	systemd-journal	00:24:54	7
2137162	99	14	5,7	cyclictest	6233	-21	gvfs-afc-volume	20:25:21	7
2137162	99	14	10,3	cyclictest	1302	-21	snmpd	20:07:22	7
2137162	99	14	0,1	cyclictest	0	-21	swapper/7	20:45:18	7
2137162	99	14	0,1	cyclictest	0	-21	swapper/7	20:45:18	7
2137162	99	14	0,13	cyclictest	0	-21	swapper/7	21:45:21	7
2137157	99	14	13,1	cyclictest	0	-21	swapper/6	21:48:31	6
2137157	99	14	12,2	cyclictest	2640804	-21	kworker/u16:1+events_unbound	23:42:09	6
2137157	99	14	12,2	cyclictest	2423734	-21	kworker/u16:3+flush-8:0	23:31:01	6
2137157	99	14	12,2	cyclictest	2423734	-21	kworker/u16:3+events_unbound	23:55:06	6
2137157	99	14	12,2	cyclictest	2423734	-21	kworker/u16:3+events_unbound	22:27:37	6
2137157	99	14	12,2	cyclictest	2383563	-21	kworker/u16:0+events_unbound	21:42:17	6
2137157	99	14	12,2	cyclictest	2365015	-21	kworker/u16:2+events_unbound	21:30:06	6
2137157	99	14	12,2	cyclictest	2227533	-21	kworker/u16:3+events_unbound	19:57:12	6
2137157	99	14	12,2	cyclictest	2133482	-21	kworker/u16:0+events_unbound	19:12:19	6
2137157	99	14	0,14	cyclictest	0	-21	swapper/6	21:57:59	6
2137152	99	14	13,1	cyclictest	0	-21	swapper/5	21:45:15	5
2137152	99	14	10,3	cyclictest	2508838	-21	latency_hist	22:04:54	5
2137152	99	14	0,1	cyclictest	0	-21	swapper/5	00:01:57	5
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17