OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Sat Jul 11, 2026 12:43:33)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
2522975	2	98	0,1	sleep4	0	-21	swapper/4	10:54:59	4
2589200	2	64	0,1	sleep7	2589201	-21	grep	11:20:03	7
2024349	2	60	49,7	sleep5	0	-21	swapper/5	07:06:48	5
2024447	2	56	21,30	sleep7	0	-21	swapper/7	07:08:10	7
2024852	99	42	39,2	cyclictest	2409988	-21	kworker/u16:2+flush-8:0	10:24:41	6
2024848	99	41	39,2	cyclictest	2574009	-21	kworker/u16:3+events_unbound	12:35:14	5
2024857	99	40	0,38	cyclictest	2286617	-21	latency_hist	09:19:41	7
2024848	99	40	38,2	cyclictest	2299063	-21	kworker/u16:0+events_unbound	10:08:47	5
2024852	99	39	36,2	cyclictest	1995185	-21	kworker/u16:0+flush-8:0	07:14:41	6
2024848	99	39	37,2	cyclictest	2640810	-21	kworker/u16:0+events_unbound	11:47:23	5
2024848	99	38	36,2	cyclictest	2299063	-21	kworker/u16:0+events_unbound	10:15:47	5
2024848	99	38	36,2	cyclictest	2299063	-21	kworker/u16:0+events_unbound	10:00:15	5
2024848	99	38	36,2	cyclictest	2100492	-21	kworker/u16:3+events_unbound	09:50:47	5
2024848	99	37	35,2	cyclictest	2444799	-21	kworker/u16:3+events_unbound	10:22:03	5
2024848	99	36	34,2	cyclictest	2640810	-21	kworker/u16:0+events_unbound	11:57:11	5
2024848	99	36	34,2	cyclictest	2483239	-21	kworker/u16:0+events_unbound	11:02:42	5
2024848	99	36	34,2	cyclictest	2409988	-21	kworker/u16:2+events_unbound	10:37:35	5
2024848	99	36	34,2	cyclictest	2271071	-21	kworker/u16:1+flush-8:0	10:40:07	5
2024857	99	35	29,5	cyclictest	2619794	-21	latency_hist	11:34:41	7
2024857	99	35	29,5	cyclictest	2619794	-21	latency_hist	11:34:41	7
2024848	99	35	33,2	cyclictest	2574009	-21	kworker/u16:3+events_unbound	12:14:23	5
2024848	99	35	33,2	cyclictest	2574009	-21	kworker/u16:3+events_unbound	12:14:23	5
2024848	99	35	33,2	cyclictest	2444799	-21	kworker/u16:3+events_unbound	10:54:23	5
2024848	99	35	33,2	cyclictest	2409988	-21	kworker/u16:2+events_unbound	10:31:55	5
2024848	99	35	33,2	cyclictest	2271071	-21	kworker/u16:1+events_unbound	09:56:32	5
2024848	99	35	33,2	cyclictest	2100492	-21	kworker/u16:3+events_unbound	09:30:19	5
2024848	99	34	32,2	cyclictest	2574009	-21	kworker/u16:3+events_unbound	11:40:19	5
2024848	99	34	32,2	cyclictest	2271071	-21	kworker/u16:1+events_unbound	10:45:55	5
2024848	99	34	32,2	cyclictest	2271071	-21	kworker/u16:1+events_unbound	09:48:59	5
2024839	99	34	31,2	cyclictest	2508958	-21	kworker/u16:2+flush-8:0	11:04:56	4
2024471	2	34	23,8	sleep4	0	-21	swapper/4	07:08:31	4
2024848	99	33	31,2	cyclictest	2694280	-21	kworker/u16:5+events_unbound	12:28:14	5
2024848	99	33	31,2	cyclictest	2508958	-21	kworker/u16:2+events_unbound	11:17:15	5
2024848	99	33	31,2	cyclictest	2483239	-21	kworker/u16:0+events_unbound	11:28:27	5
2024848	99	33	31,2	cyclictest	2193597	-21	kworker/u16:0+events_unbound	08:43:56	5
2024848	99	32	30,2	cyclictest	2694280	-21	kworker/u16:5+flush-8:0	12:15:38	5
2024848	99	32	30,2	cyclictest	2694280	-21	kworker/u16:5+events_unbound	12:09:20	5
2024848	99	32	30,2	cyclictest	2299063	-21	kworker/u16:0+events_unbound	09:43:39	5
2024848	99	32	30,2	cyclictest	2271071	-21	kworker/u16:1+events_unbound	09:13:43	5
2024848	99	31	29,2	cyclictest	2533557	-21	kworker/u16:4+flush-8:0	11:05:22	5
2024848	99	30	28,2	cyclictest	2204854	-21	kworker/u16:2+events_unbound	08:45:20	5
2024852	99	29	27,2	cyclictest	2100492	-21	kworker/u16:3+flush-8:0	09:34:57	6
2024848	99	29	27,2	cyclictest	2640810	-21	kworker/u16:0+events_unbound	12:02:00	5
2024848	99	29	27,2	cyclictest	2574009	-21	kworker/u16:3+flush-8:0	12:29:47	5
2024848	99	29	27,2	cyclictest	2574009	-21	kworker/u16:3+events_unbound	11:35:02	5
2024848	99	29	27,2	cyclictest	2574009	-21	kworker/u16:3+events_unbound	11:35:02	5
2024848	99	29	27,2	cyclictest	2005562	-21	kworker/u16:4+events_unbound	07:12:52	5
2024848	99	29	26,2	cyclictest	2125817	-21	kworker/u16:1+events_unbound	08:34:43	5
2024473	2	29	19,7	sleep6	0	-21	swapper/6	07:08:33	6
2024852	99	28	25,2	cyclictest	2299063	-21	kworker/u16:0+flush-8:0	10:19:41	6
2024848	99	28	26,2	cyclictest	2656851	-21	kworker/u16:4+events_unbound	12:23:58	5
2024848	99	28	26,2	cyclictest	2574009	-21	kworker/u16:3+events_unbound	11:51:04	5
2024848	99	28	26,2	cyclictest	2508958	-21	kworker/u16:2+events_unbound	10:57:09	5
2024848	99	28	26,2	cyclictest	2100492	-21	kworker/u16:3+events_unbound	09:15:36	5
2024848	99	27	25,2	cyclictest	2582914	-21	kworker/u16:5+events_unbound	11:31:49	5
2024848	99	27	25,2	cyclictest	2582914	-21	kworker/u16:5+events_unbound	11:31:49	5
2024848	99	27	25,2	cyclictest	2299063	-21	kworker/u16:0+events_unbound	09:38:57	5
2024848	99	27	25,2	cyclictest	2271071	-21	kworker/u16:1+events_unbound	10:12:32	5
2024848	99	26	24,2	cyclictest	2574009	-21	kworker/u16:3+events_unbound	11:20:36	5
2024848	99	26	24,2	cyclictest	2409988	-21	kworker/u16:2+events_unbound	10:25:49	5
2024848	99	26	24,2	cyclictest	2299063	-21	kworker/u16:0+flush-8:0	09:25:31	5
2024848	99	26	24,2	cyclictest	2299063	-21	kworker/u16:0+flush-8:0	09:25:31	5
2024848	99	26	24,2	cyclictest	2100492	-21	kworker/u16:3+events_unbound	09:22:40	5
2024848	99	26	23,2	cyclictest	2100492	-21	kworker/u16:3+events_unbound	07:54:43	5
2024848	99	26	23,2	cyclictest	2100492	-21	kworker/u16:3+events_unbound	07:54:43	5
2024848	99	25	23,2	cyclictest	2483239	-21	kworker/u16:0+flush-8:0	11:10:26	5
2024848	99	25	23,2	cyclictest	2024614	-21	kworker/u16:1+events_unbound	07:40:43	5
2024857	99	24	19,4	cyclictest	648	-21	systemd-journal	12:19:41	7
2024848	99	24	22,2	cyclictest	2034491	-21	kworker/u16:3+events_unbound	07:27:28	5
2024852	99	23	21,2	cyclictest	2299063	-21	kworker/u16:0+flush-8:0	10:01:18	6
2024848	99	23	21,2	cyclictest	2100492	-21	kworker/u16:3+events_unbound	08:31:59	5
2024848	99	23	21,2	cyclictest	2034491	-21	kworker/u16:3+events_unbound	07:23:55	5
2024848	99	22	20,2	cyclictest	2024614	-21	kworker/u16:1+events_unbound	07:48:52	5
2024848	99	21	19,2	cyclictest	2100492	-21	kworker/u16:3+events_unbound	09:06:58	5
2024848	99	21	18,2	cyclictest	2100492	-21	kworker/u16:3+flush-8:0	08:14:47	5
2024839	99	21	20,1	cyclictest	2483239	-21	kworker/u16:0+events_unbound	11:27:17	4
2024839	99	21	19,2	cyclictest	2767855	-21	kworker/u16:1+flush-8:0	12:39:40	4
2024848	99	20	18,2	cyclictest	2193597	-21	kworker/u16:0+events_unbound	08:50:27	5
2024848	99	20	18,2	cyclictest	2100492	-21	kworker/u16:3+flush-8:0	08:27:42	5
2024848	99	20	18,2	cyclictest	2100492	-21	kworker/u16:3+events_unbound	08:22:52	5
2024848	99	20	18,2	cyclictest	1995185	-21	kworker/u16:0+events_unbound	07:35:58	5
2024848	99	19	17,2	cyclictest	2125817	-21	kworker/u16:1+events_unbound	08:01:53	5
2024848	99	19	17,2	cyclictest	2024614	-21	kworker/u16:1+events_unbound	07:15:58	5
2024857	99	18	17,0	cyclictest	0	-21	swapper/7	10:13:27	7
2024852	99	18	16,2	cyclictest	2100492	-21	kworker/u16:3+flush-8:0	08:40:06	6
2024852	99	18	15,3	cyclictest	2299063	-21	kworker/u16:0+flush-8:0	09:29:56	6
2024848	99	18	16,2	cyclictest	2100492	-21	kworker/u16:3+events_unbound	08:59:38	5
2024848	99	18	16,2	cyclictest	2100492	-21	kworker/u16:3+events_unbound	08:11:53	5
2024848	99	18	16,2	cyclictest	1995185	-21	kworker/u16:0+events_unbound	07:52:34	5
2024857	99	17	0,1	cyclictest	0	-21	swapper/7	07:45:08	7
2024857	99	17	0,16	cyclictest	0	-21	swapper/7	07:20:05	7
2024852	99	17	15,2	cyclictest	2100492	-21	kworker/u16:3+flush-8:0	08:14:57	6
2024839	99	17	14,2	cyclictest	2024614	-21	kworker/u16:1+flush-8:0	07:35:07	4
2024857	99	16	13,2	cyclictest	1274	-21	snmpd	10:58:10	7
2024857	99	16	13,2	cyclictest	0	-21	swapper/7	09:54:42	7
2024857	99	16	0,14	cyclictest	0	-21	swapper/7	07:55:07	7
2024857	99	16	0,14	cyclictest	0	-21	swapper/7	07:55:07	7
2024852	99	16	0,15	cyclictest	0	-21	swapper/6	07:40:01	6
2024839	99	16	0,15	cyclictest	2261462	-21	timerwakeupswit	09:05:09	4
2024857	99	15	12,2	cyclictest	0	-21	swapper/7	12:35:09	7
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17