OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Wed Aug 27, 2025 12:43:32)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
3713391	2	58	47,8	sleep4	0	-21	swapper/4	15:26:43	4
3713283	2	54	20,30	sleep7	0	-21	swapper/7	15:25:09	7
4130192	2	48	0,0	sleep4	0	-21	swapper/4	18:54:51	4
3713893	99	47	45,2	cyclictest	74865	-21	kworker/u16:0+flush-8:0	20:34:38	5
3713893	99	46	44,2	cyclictest	3874325	-21	kworker/u16:0+flush-8:0	17:04:45	5
3713337	2	46	36,7	sleep5	0	-21	swapper/5	15:25:55	5
3713893	99	45	43,2	cyclictest	4155035	-21	kworker/u16:2+flush-8:0	19:14:46	5
3713893	99	44	42,2	cyclictest	4102016	-21	kworker/u16:2+events_unbound	18:54:55	5
3713897	99	43	41,2	cyclictest	4190002	-21	kworker/u16:4+events_unbound	20:24:37	6
3713897	99	43	41,2	cyclictest	4136442	-21	kworker/u16:4+flush-8:0	19:19:31	6
3713897	99	43	41,2	cyclictest	4136442	-21	kworker/u16:4+flush-8:0	19:19:31	6
3713893	99	43	40,2	cyclictest	3683996	-21	kworker/u16:1+events_unbound	15:49:35	5
3713897	99	42	40,2	cyclictest	4190002	-21	kworker/u16:4+flush-8:0	20:39:44	6
3713897	99	42	40,2	cyclictest	4022589	-21	kworker/u16:4+events_unbound	18:24:38	6
3713309	2	42	32,7	sleep6	0	-21	swapper/6	15:25:32	6
3713898	99	40	35,4	cyclictest	3963239	-21	latency_hist	17:34:20	7
3713897	99	40	38,2	cyclictest	4136442	-21	kworker/u16:4+flush-8:0	19:04:54	6
3713897	99	40	37,2	cyclictest	4102016	-21	kworker/u16:2+flush-8:0	18:49:34	6
3713893	99	40	38,2	cyclictest	74865	-21	kworker/u16:0+flush-8:0	20:29:45	5
3713893	99	40	38,2	cyclictest	74865	-21	kworker/u16:0+events_unbound	20:14:45	5
3713893	99	40	38,2	cyclictest	3992134	-21	kworker/u16:0+events_unbound	18:04:20	5
3713893	99	40	37,2	cyclictest	4002592	-21	kworker/u16:1+events_unbound	17:54:22	5
3713897	99	39	37,2	cyclictest	3004625	-21	kworker/u16:7+events_unbound	15:34:38	6
3713897	99	39	37,2	cyclictest	3004625	-21	kworker/u16:7+events_unbound	15:34:38	6
3713893	99	39	37,2	cyclictest	4155035	-21	kworker/u16:2+flush-8:0	20:04:42	5
3713893	99	39	37,2	cyclictest	4155035	-21	kworker/u16:2+flush-8:0	20:04:42	5
3713893	99	39	37,2	cyclictest	4002592	-21	kworker/u16:1+events_unbound	18:09:47	5
3713893	99	39	37,2	cyclictest	3972928	-21	kworker/u16:6+flush-8:0	17:39:48	5
3713893	99	39	37,2	cyclictest	3844894	-21	kworker/u16:4+flush-8:0	16:39:54	5
3713898	99	38	32,5	cyclictest	3825574	-21	latency_hist	16:24:20	7
3713897	99	38	36,2	cyclictest	4155035	-21	kworker/u16:2+events_unbound	19:15:07	6
3713897	99	38	36,2	cyclictest	4136442	-21	kworker/u16:4+flush-8:0	19:09:49	6
3713893	99	38	36,2	cyclictest	4189788	-21	kworker/u16:1+events_unbound	19:34:49	5
3713897	99	37	35,2	cyclictest	3844894	-21	kworker/u16:4+events_unbound	16:38:15	6
3713897	99	37	34,2	cyclictest	4155035	-21	kworker/u16:2+events_unbound	19:54:22	6
3713893	99	37	35,2	cyclictest	3884273	-21	kworker/u16:1+events_unbound	17:37:51	5
3713897	99	36	34,2	cyclictest	4190002	-21	kworker/u16:4+events_unbound	20:24:14	6
3713897	99	36	34,2	cyclictest	4155035	-21	kworker/u16:2+events_unbound	20:09:38	6
3713897	99	36	34,2	cyclictest	4022589	-21	kworker/u16:4+events_unbound	18:14:48	6
3713897	99	36	34,2	cyclictest	3844894	-21	kworker/u16:4+flush-8:0	17:14:48	6
3713897	99	36	34,2	cyclictest	3770600	-21	kworker/u16:0+events_unbound	16:07:50	6
3713897	99	36	34,2	cyclictest	25396	-21	kworker/u16:5+flush-8:0	19:44:57	6
3713893	99	36	34,2	cyclictest	4190002	-21	kworker/u16:4+flush-8:0	20:49:43	5
3713893	99	36	34,2	cyclictest	4190002	-21	kworker/u16:4+flush-8:0	20:49:43	5
3713893	99	36	34,2	cyclictest	113094	-21	kworker/u16:5+events_unbound	20:29:19	5
3713893	99	36	32,3	cyclictest	3786224	-21	kworker/u16:1+flush-8:0	16:09:46	5
3713893	99	35	33,2	cyclictest	4022589	-21	kworker/u16:4+events_unbound	18:24:53	5
3713893	99	35	32,2	cyclictest	15964	-21	kworker/u16:0+events_unbound	19:55:30	5
3713897	99	34	32,2	cyclictest	74865	-21	kworker/u16:0+events_unbound	20:14:47	6
3713897	99	34	32,2	cyclictest	4171558	-21	kworker/u16:3+flush-8:0	19:24:48	6
3713897	99	34	32,2	cyclictest	4155035	-21	kworker/u16:2+events_unbound	20:06:19	6
3713897	99	34	32,2	cyclictest	4155035	-21	kworker/u16:2+events_unbound	20:06:19	6
3713897	99	34	32,2	cyclictest	4081718	-21	kworker/u16:0+events_unbound	18:38:54	6
3713897	99	34	32,2	cyclictest	4081718	-21	kworker/u16:0+events_unbound	18:38:54	6
3713897	99	34	32,2	cyclictest	3874325	-21	kworker/u16:0+events_unbound	17:11:50	6
3713897	99	34	32,2	cyclictest	3553516	-21	kworker/u16:6+events_unbound	15:54:46	6
3713897	99	34	31,2	cyclictest	3844894	-21	kworker/u16:4+events_unbound	16:39:37	6
3713893	99	34	32,2	cyclictest	4155035	-21	kworker/u16:2+flush-8:0	20:09:45	5
3713893	99	34	32,2	cyclictest	4155035	-21	kworker/u16:2+events_unbound	20:43:19	5
3713893	99	34	32,2	cyclictest	4092191	-21	kworker/u16:1+flush-8:0	19:02:15	5
3713893	99	34	32,2	cyclictest	3835342	-21	kworker/u16:2+events_unbound	16:48:52	5
3713893	99	34	32,2	cyclictest	3004625	-21	kworker/u16:7+events_unbound	15:41:52	5
3713893	99	34	31,2	cyclictest	4092191	-21	kworker/u16:1+events_unbound	18:39:39	5
3713898	99	33	27,5	cyclictest	124082	-21	latency_hist	20:34:20	7
3713897	99	33	31,2	cyclictest	3835342	-21	kworker/u16:2+flush-8:0	16:29:35	6
3713897	99	33	31,2	cyclictest	3004625	-21	kworker/u16:7+events_unbound	16:02:59	6
3713897	99	33	30,2	cyclictest	4022113	-21	kworker/u16:3+events_unbound	18:04:32	6
3713893	99	33	31,2	cyclictest	4190002	-21	kworker/u16:4+events_unbound	20:23:19	5
3713893	99	33	31,2	cyclictest	4023021	-21	kworker/u16:6+events_unbound	18:16:31	5
3713893	99	33	31,2	cyclictest	4022589	-21	kworker/u16:4+events_unbound	18:30:22	5
3713893	99	33	30,2	cyclictest	3553516	-21	kworker/u16:6+events_unbound	15:29:30	5
3713897	99	32	30,2	cyclictest	59779	-21	kworker/u16:1+events_unbound	20:01:30	6
3713897	99	32	30,2	cyclictest	4022589	-21	kworker/u16:4+events_unbound	18:10:43	6
3713897	99	32	30,2	cyclictest	3835342	-21	kworker/u16:2+events_unbound	16:48:15	6
3713897	99	32	30,2	cyclictest	3786224	-21	kworker/u16:1+flush-8:0	16:14:50	6
3713897	99	32	30,2	cyclictest	3004625	-21	kworker/u16:7+events_unbound	15:39:53	6
3713893	99	32	30,2	cyclictest	74865	-21	kworker/u16:0+events_unbound	20:45:30	5
3713893	99	32	30,2	cyclictest	4189788	-21	kworker/u16:1+flush-8:0	19:39:46	5
3713893	99	32	30,2	cyclictest	3884273	-21	kworker/u16:1+events_unbound	17:17:20	5
3713893	99	32	30,2	cyclictest	3874325	-21	kworker/u16:0+flush-8:0	16:54:36	5
3713893	99	32	30,2	cyclictest	3844894	-21	kworker/u16:4+events_unbound	17:12:49	5
3713893	99	32	30,2	cyclictest	15964	-21	kworker/u16:0+events_unbound	19:53:19	5
3713893	99	32	30,2	cyclictest	15964	-21	kworker/u16:0+events_unbound	19:44:49	5
3713897	99	31	30,1	cyclictest	3553516	-21	kworker/u16:6+flush-8:0	16:09:40	6
3713897	99	31	29,2	cyclictest	74865	-21	kworker/u16:0+flush-8:0	20:31:54	6
3713897	99	31	29,2	cyclictest	4022113	-21	kworker/u16:3+flush-8:0	18:29:36	6
3713897	99	31	29,2	cyclictest	3884273	-21	kworker/u16:1+events_unbound	17:37:42	6
3713897	99	31	29,2	cyclictest	3884273	-21	kworker/u16:1+events_unbound	17:19:53	6
3713897	99	31	29,2	cyclictest	3884273	-21	kworker/u16:1+events_unbound	16:51:54	6
3713897	99	31	29,2	cyclictest	3826062	-21	kworker/u16:0+events_unbound	16:26:59	6
3713897	99	31	29,2	cyclictest	3786224	-21	kworker/u16:1+events_unbound	16:22:19	6
3713897	99	31	29,2	cyclictest	153558	-21	kworker/u16:3+flush-8:0	20:50:14	6
3713897	99	31	29,2	cyclictest	153558	-21	kworker/u16:3+flush-8:0	20:50:14	6
3713893	99	31	29,2	cyclictest	59779	-21	kworker/u16:1+events_unbound	20:00:51	5
3713893	99	31	29,2	cyclictest	4171558	-21	kworker/u16:3+events_unbound	19:27:22	5
3713893	99	31	29,2	cyclictest	4136442	-21	kworker/u16:4+events_unbound	19:06:43	5
3713893	99	31	29,2	cyclictest	4022589	-21	kworker/u16:4+flush-8:0	18:34:49	5
3713893	99	31	29,2	cyclictest	4022589	-21	kworker/u16:4+flush-8:0	18:34:49	5
3713893	99	31	29,2	cyclictest	3844894	-21	kworker/u16:4+events_unbound	16:51:50	5
3713893	99	31	29,2	cyclictest	3553516	-21	kworker/u16:6+flush-8:0	16:37:15	5
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17