OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Wed Dec 10, 2025 12:43:32)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
441015	2	60	49,7	sleep5	0	-21	swapper/5	15:22:02	5
440721	2	59	48,7	sleep6	0	-21	swapper/6	15:18:02	6
440791	2	57	22,30	sleep7	0	-21	swapper/7	15:18:59	7
441356	99	55	50,4	cyclictest	515643	-21	latency_hist	15:57:19	7
441356	99	55	48,6	cyclictest	707266	-21	latency_hist	17:32:19	7
441344	99	55	53,2	cyclictest	606395	-21	kworker/u16:5+flush-8:0	16:52:19	6
441344	99	55	53,2	cyclictest	606395	-21	kworker/u16:5+flush-8:0	16:52:19	6
440779	2	52	40,8	sleep4	0	-21	swapper/4	15:18:48	4
441344	99	51	49,2	cyclictest	1010509	-21	kworker/u16:3+events_unbound	20:17:46	6
441356	99	48	43,4	cyclictest	757741	-21	latency_hist	17:57:20	7
441343	99	48	46,2	cyclictest	3185582	-21	kworker/u16:3+flush-8:0	15:22:35	5
441356	99	43	37,5	cyclictest	589235	-21	systemd-journal	20:47:38	7
441343	99	43	41,2	cyclictest	889465	-21	kworker/u16:0+events_unbound	19:06:34	5
441343	99	43	41,2	cyclictest	505560	-21	kworker/u16:4+events_unbound	16:41:20	5
441343	99	43	41,2	cyclictest	3185582	-21	kworker/u16:3+events_unbound	16:03:23	5
441343	99	43	41,2	cyclictest	3185582	-21	kworker/u16:3+events_unbound	16:03:23	5
441356	99	42	37,4	cyclictest	589235	-21	systemd-journal	20:26:38	7
441343	99	42	39,2	cyclictest	354755	-21	kworker/u16:0+flush-8:0	15:47:50	5
441343	99	41	39,2	cyclictest	968917	-21	kworker/u16:2+events_unbound	20:03:34	5
441343	99	41	39,2	cyclictest	505560	-21	kworker/u16:4+flush-8:0	16:12:45	5
441343	99	41	39,2	cyclictest	1039541	-21	kworker/u16:5+events_unbound	20:13:34	5
441343	99	41	39,2	cyclictest	1009282	-21	kworker/u16:1+events_unbound	20:28:39	5
441343	99	40	38,2	cyclictest	968917	-21	kworker/u16:2+flush-8:0	19:45:34	5
441343	99	40	38,2	cyclictest	818702	-21	kworker/u16:3+events_unbound	19:24:33	5
441343	99	40	38,2	cyclictest	787076	-21	kworker/u16:5+events_unbound	19:54:33	5
441343	99	40	38,2	cyclictest	1009282	-21	kworker/u16:1+events_unbound	20:41:39	5
441344	99	39	36,2	cyclictest	3185582	-21	kworker/u16:3+events_unbound	15:22:38	6
441343	99	39	37,2	cyclictest	827573	-21	kworker/u16:4+events_unbound	18:40:33	5
441344	99	38	36,2	cyclictest	827573	-21	kworker/u16:4+events_unbound	20:45:19	6
441343	99	38	36,2	cyclictest	818702	-21	kworker/u16:3+events_unbound	19:13:19	5
441343	99	38	36,2	cyclictest	606395	-21	kworker/u16:5+flush-8:0	17:02:14	5
441343	99	38	36,2	cyclictest	354755	-21	kworker/u16:0+events_unbound	16:20:23	5
441343	99	37	35,2	cyclictest	818702	-21	kworker/u16:3+events_unbound	18:50:33	5
441343	99	37	34,2	cyclictest	827573	-21	kworker/u16:4+flush-8:0	20:46:09	5
441344	99	36	34,2	cyclictest	606395	-21	kworker/u16:5+events_unbound	17:29:19	6
441343	99	36	34,2	cyclictest	827573	-21	kworker/u16:4+events_unbound	18:56:34	5
441343	99	36	33,2	cyclictest	606395	-21	kworker/u16:5+flush-8:0	16:47:36	5
441343	99	36	33,2	cyclictest	606395	-21	kworker/u16:5+flush-8:0	16:47:36	5
441344	99	35	33,2	cyclictest	666691	-21	kworker/u16:0+events_unbound	17:14:31	6
441343	99	35	33,2	cyclictest	666691	-21	kworker/u16:0+flush-8:0	17:37:14	5
441343	99	35	33,2	cyclictest	1009282	-21	kworker/u16:1+flush-8:0	20:01:53	5
441356	99	34	28,5	cyclictest	868897	-21	latency_hist	18:52:19	7
441344	99	34	32,2	cyclictest	726327	-21	kworker/u16:1+events_unbound	17:41:33	6
441344	99	34	32,2	cyclictest	666691	-21	kworker/u16:0+events_unbound	17:10:31	6
441356	99	33	0,31	cyclictest	1080786	-21	latency_hist	20:37:19	7
441344	99	33	32,1	cyclictest	768206	-21	kworker/u16:2+flush-8:0	18:28:42	6
441344	99	33	31,2	cyclictest	726327	-21	kworker/u16:1+events_unbound	18:09:32	6
441344	99	33	31,2	cyclictest	606395	-21	kworker/u16:5+events_unbound	16:56:27	6
441343	99	33	31,2	cyclictest	768206	-21	kworker/u16:2+events_unbound	18:30:29	5
441343	99	33	31,2	cyclictest	768206	-21	kworker/u16:2+events_unbound	18:20:33	5
441343	99	33	31,2	cyclictest	585275	-21	kworker/u16:3+events_unbound	18:04:33	5
441343	99	33	31,2	cyclictest	576112	-21	kworker/u16:0+events_unbound	16:27:55	5
441356	99	32	30,1	cyclictest	0	-21	swapper/7	20:17:38	7
441344	99	32	30,2	cyclictest	827573	-21	kworker/u16:4+events_unbound	19:45:18	6
441344	99	32	30,2	cyclictest	606395	-21	kworker/u16:5+flush-8:0	16:43:30	6
441344	99	32	29,2	cyclictest	787076	-21	kworker/u16:5+flush-8:0	18:37:29	6
441343	99	32	30,2	cyclictest	354755	-21	kworker/u16:0+events_unbound	15:57:08	5
441356	99	31	26,4	cyclictest	576161	-21	latency_hist	16:27:20	7
441344	99	31	29,2	cyclictest	827573	-21	kworker/u16:4+events_unbound	19:09:22	6
441344	99	31	29,2	cyclictest	666691	-21	kworker/u16:0+events_unbound	17:34:51	6
441344	99	31	29,2	cyclictest	646193	-21	kworker/u16:2+flush-8:0	17:19:58	6
441344	99	31	29,2	cyclictest	1009282	-21	kworker/u16:1+events_unbound	20:30:26	6
441344	99	31	29,2	cyclictest	1009282	-21	kworker/u16:1+events_unbound	20:22:22	6
441343	99	31	29,2	cyclictest	768206	-21	kworker/u16:2+events_unbound	18:34:08	5
441343	99	31	29,2	cyclictest	726551	-21	kworker/u16:4+events_unbound	17:43:33	5
441343	99	31	29,2	cyclictest	566022	-21	kworker/u16:1+events_unbound	16:25:30	5
441343	99	31	29,2	cyclictest	354755	-21	kworker/u16:0+events_unbound	15:36:13	5
441344	99	30	28,2	cyclictest	818702	-21	kworker/u16:3+flush-8:0	19:12:58	6
441344	99	30	28,2	cyclictest	818702	-21	kworker/u16:3+events_unbound	18:50:15	6
441344	99	30	28,2	cyclictest	808260	-21	kworker/u16:0+events_unbound	18:25:26	6
441344	99	30	28,2	cyclictest	585275	-21	kworker/u16:3+events_unbound	16:36:03	6
441344	99	30	28,2	cyclictest	536184	-21	kworker/u16:2+events_unbound	16:13:31	6
441344	99	30	28,2	cyclictest	354755	-21	kworker/u16:0+events_unbound	16:00:59	6
441344	99	30	28,2	cyclictest	3185582	-21	kworker/u16:3+events_unbound	15:54:04	6
441344	99	30	28,2	cyclictest	3185582	-21	kworker/u16:3+events_unbound	15:44:47	6
441344	99	30	27,2	cyclictest	787076	-21	kworker/u16:5+events_unbound	19:17:23	6
441343	99	30	28,2	cyclictest	475320	-21	kworker/u16:1+events_unbound	15:42:44	5
441343	99	30	28,2	cyclictest	354755	-21	kworker/u16:0+events_unbound	15:32:19	5
441343	99	30	28,2	cyclictest	1010509	-21	kworker/u16:3+flush-8:0	20:20:29	5
441344	99	29	27,2	cyclictest	818702	-21	kworker/u16:3+flush-8:0	18:42:58	6
441344	99	29	27,2	cyclictest	787076	-21	kworker/u16:5+events_unbound	19:28:33	6
441344	99	29	27,2	cyclictest	505560	-21	kworker/u16:4+events_unbound	16:42:15	6
441344	99	29	27,2	cyclictest	505560	-21	kworker/u16:4+events_unbound	16:20:06	6
441344	99	29	27,2	cyclictest	475320	-21	kworker/u16:1+events_unbound	15:39:58	6
441344	99	29	27,2	cyclictest	3185582	-21	kworker/u16:3+events_unbound	16:03:31	6
441344	99	29	27,2	cyclictest	3185582	-21	kworker/u16:3+events_unbound	16:03:31	6
441344	99	29	27,2	cyclictest	1039541	-21	kworker/u16:5+events_unbound	20:14:50	6
441343	99	29	27,2	cyclictest	946095	-21	kworker/u16:1+events_unbound	19:29:59	5
441343	99	29	27,2	cyclictest	808260	-21	kworker/u16:0+events_unbound	18:25:30	5
441356	99	28	22,5	cyclictest	990129	-21	latency_hist	19:52:19	7
441344	99	28	26,2	cyclictest	889465	-21	kworker/u16:0+flush-8:0	19:02:46	6
441344	99	28	26,2	cyclictest	827573	-21	kworker/u16:4+events_unbound	19:25:14	6
441344	99	28	26,2	cyclictest	768206	-21	kworker/u16:2+events_unbound	18:56:46	6
441344	99	28	26,2	cyclictest	726327	-21	kworker/u16:1+flush-8:0	18:05:42	6
441344	99	28	26,2	cyclictest	726327	-21	kworker/u16:1+events_unbound	18:18:51	6
441344	99	28	26,2	cyclictest	606395	-21	kworker/u16:5+flush-8:0	17:55:58	6
441344	99	28	26,2	cyclictest	606395	-21	kworker/u16:5+events_unbound	17:26:03	6
441344	99	28	26,2	cyclictest	566022	-21	kworker/u16:1+flush-8:0	16:23:47	6
441344	99	28	25,2	cyclictest	818702	-21	kworker/u16:3+flush-8:0	18:57:28	6
441343	99	28	26,2	cyclictest	827573	-21	kworker/u16:4+events_unbound	20:48:29	5
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17