OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Mon May 04, 2026 12:43:34)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
724878	2	98	0,1	sleep7	724879	-21	expr	11:33:55	7
138642	99	65	62,2	cyclictest	602232	-21	kworker/u16:1+events_unbound	10:43:57	4
138642	99	65	62,2	cyclictest	588395	-21	kworker/u16:4+flush-8:0	10:38:57	4
138642	99	58	55,2	cyclictest	4067447	-21	kworker/u16:4+events_unbound	07:38:57	4
138642	99	58	55,2	cyclictest	4067447	-21	kworker/u16:4+events_unbound	07:38:57	4
138335	2	58	48,7	sleep4	0	-21	swapper/4	07:08:35	4
138648	99	54	51,2	cyclictest	352279	-21	kworker/u16:1+events_unbound	08:59:10	6
138642	99	52	50,2	cyclictest	772646	-21	kworker/u16:3+flush-8:0	12:03:57	4
138648	99	51	49,2	cyclictest	380990	-21	kworker/u16:2+events_unbound	10:24:09	6
138648	99	51	48,2	cyclictest	362378	-21	kworker/u16:3+events_unbound	09:24:09	6
138648	99	50	48,2	cyclictest	286315	-21	kworker/u16:0+events_unbound	09:04:11	6
138648	99	49	47,2	cyclictest	380990	-21	kworker/u16:2+events_unbound	10:04:13	6
138648	99	49	46,2	cyclictest	772646	-21	kworker/u16:3+events_unbound	12:08:56	6
138648	99	48	46,2	cyclictest	90620	-21	kworker/u16:1+flush-8:0	07:33:57	6
138642	99	48	45,2	cyclictest	698776	-21	kworker/u16:4+flush-8:0	11:38:57	4
138642	99	48	45,2	cyclictest	602232	-21	kworker/u16:1+flush-8:0	10:48:57	4
138648	99	47	45,2	cyclictest	830419	-21	kworker/u16:1+events_unbound	12:28:58	6
138648	99	47	45,2	cyclictest	830419	-21	kworker/u16:1+events_unbound	12:28:58	6
138642	99	47	45,2	cyclictest	698776	-21	kworker/u16:4+events_unbound	11:48:58	4
138642	99	47	45,2	cyclictest	698776	-21	kworker/u16:4+events_unbound	11:48:58	4
138648	99	46	44,2	cyclictest	190396	-21	kworker/u16:2+flush-8:0	08:03:57	6
138648	99	45	43,2	cyclictest	684444	-21	kworker/u16:2+events_unbound	11:34:17	6
138648	99	45	43,2	cyclictest	380990	-21	kworker/u16:2+events_unbound	09:33:56	6
138648	99	44	41,2	cyclictest	380990	-21	kworker/u16:2+flush-8:0	10:19:12	6
138642	99	44	42,2	cyclictest	380990	-21	kworker/u16:2+events_unbound	09:53:56	4
138652	99	43	37,5	cyclictest	372247	-21	latency_hist	09:08:42	7
138648	99	42	40,2	cyclictest	698776	-21	kworker/u16:4+events_unbound	11:43:10	6
138648	99	42	40,2	cyclictest	468622	-21	kworker/u16:1+events_unbound	10:00:01	6
138648	99	41	39,2	cyclictest	772646	-21	kworker/u16:3+flush-8:0	12:14:15	6
138642	99	41	39,2	cyclictest	698776	-21	kworker/u16:4+events_unbound	11:44:09	4
138648	99	40	38,2	cyclictest	602232	-21	kworker/u16:1+events_unbound	11:13:59	6
138648	99	40	38,2	cyclictest	588395	-21	kworker/u16:4+events_unbound	10:36:26	6
138648	99	40	38,2	cyclictest	521056	-21	kworker/u16:0+events_unbound	12:20:57	6
138648	99	40	38,2	cyclictest	4067447	-21	kworker/u16:4+flush-8:0	07:39:11	6
138648	99	40	38,2	cyclictest	4067447	-21	kworker/u16:4+flush-8:0	07:39:11	6
138642	99	40	37,2	cyclictest	362378	-21	kworker/u16:3+flush-8:0	09:58:57	4
138101	2	40	3,31	sleep7	0	-21	swapper/7	07:05:18	7
138648	99	39	36,2	cyclictest	380990	-21	kworker/u16:2+flush-8:0	09:53:58	6
138648	99	38	36,2	cyclictest	772645	-21	kworker/u16:2+events_unbound	11:55:54	6
138648	99	38	36,2	cyclictest	698776	-21	kworker/u16:4+events_unbound	11:26:53	6
138648	99	38	36,2	cyclictest	684444	-21	kworker/u16:2+events_unbound	11:30:53	6
138648	99	38	36,2	cyclictest	602232	-21	kworker/u16:1+events_unbound	10:58:13	6
138648	99	38	36,2	cyclictest	352279	-21	kworker/u16:1+events_unbound	09:13:26	6
138648	99	38	36,2	cyclictest	257752	-21	kworker/u16:3+flush-8:0	08:19:07	6
138648	99	37	35,2	cyclictest	711969	-21	kworker/u16:1+events_unbound	11:52:08	6
138648	99	37	35,2	cyclictest	711969	-21	kworker/u16:1+events_unbound	11:52:08	6
138648	99	37	35,2	cyclictest	362378	-21	kworker/u16:3+flush-8:0	09:44:14	6
138648	99	37	35,2	cyclictest	362378	-21	kworker/u16:3+events_unbound	10:08:56	6
138648	99	37	34,2	cyclictest	602232	-21	kworker/u16:1+flush-8:0	10:48:55	6
138642	99	37	34,2	cyclictest	380990	-21	kworker/u16:2+flush-8:0	09:53:41	4
138648	99	36	34,2	cyclictest	772646	-21	kworker/u16:3+events_unbound	12:04:26	6
138648	99	36	34,2	cyclictest	521056	-21	kworker/u16:0+events_unbound	10:30:38	6
138648	99	36	34,2	cyclictest	380990	-21	kworker/u16:2+flush-8:0	09:29:10	6
138648	99	36	33,2	cyclictest	602232	-21	kworker/u16:1+events_unbound	11:03:44	6
138648	99	36	33,2	cyclictest	588395	-21	kworker/u16:4+flush-8:0	10:39:01	6
138648	99	36	33,2	cyclictest	362378	-21	kworker/u16:3+flush-8:0	09:13:57	6
138642	99	36	34,2	cyclictest	772645	-21	kworker/u16:2+flush-8:0	12:13:57	4
138648	99	35	33,2	cyclictest	830419	-21	kworker/u16:1+events_unbound	12:24:52	6
138648	99	35	33,2	cyclictest	626550	-21	kworker/u16:2+events_unbound	11:00:18	6
138648	99	35	33,2	cyclictest	521056	-21	kworker/u16:0+events_unbound	12:00:53	6
138648	99	35	33,2	cyclictest	521056	-21	kworker/u16:0+events_unbound	11:11:33	6
138648	99	35	33,2	cyclictest	521056	-21	kworker/u16:0+events_unbound	11:11:33	6
138648	99	35	33,2	cyclictest	468622	-21	kworker/u16:1+events_unbound	09:51:58	6
138648	99	35	33,2	cyclictest	380990	-21	kworker/u16:2+events_unbound	09:41:13	6
138648	99	35	33,2	cyclictest	190396	-21	kworker/u16:2+flush-8:0	07:54:05	6
138648	99	35	32,2	cyclictest	90620	-21	kworker/u16:1+events_unbound	07:48:55	6
138648	99	34	32,2	cyclictest	772646	-21	kworker/u16:3+events_unbound	12:37:01	6
138648	99	34	32,2	cyclictest	602232	-21	kworker/u16:1+events_unbound	10:48:29	6
138648	99	34	32,2	cyclictest	521056	-21	kworker/u16:0+events_unbound	11:46:45	6
138648	99	34	32,2	cyclictest	286315	-21	kworker/u16:0+flush-8:0	08:34:09	6
138642	99	34	32,2	cyclictest	772645	-21	kworker/u16:2+events_unbound	11:59:12	4
138642	99	34	32,2	cyclictest	362378	-21	kworker/u16:3+events_unbound	09:23:56	4
138642	99	34	31,2	cyclictest	286315	-21	kworker/u16:0+events_unbound	08:49:05	4
138279	2	34	24,7	sleep5	0	-21	swapper/5	07:07:48	5
138648	99	33	30,2	cyclictest	319258	-21	kworker/u16:2+flush-8:0	08:43:57	6
138642	99	33	31,2	cyclictest	602232	-21	kworker/u16:1+events_unbound	10:59:14	4
138642	99	33	31,2	cyclictest	4067447	-21	kworker/u16:4+flush-8:0	08:28:42	4
138642	99	33	30,3	cyclictest	362378	-21	kworker/u16:3+flush-8:0	09:33:59	4
138648	99	32	29,2	cyclictest	684444	-21	kworker/u16:2+events_unbound	11:18:43	6
138648	99	32	29,2	cyclictest	257752	-21	kworker/u16:3+events_unbound	08:08:55	6
138642	99	32	30,2	cyclictest	521056	-21	kworker/u16:0+flush-8:0	11:09:13	4
138642	99	32	30,2	cyclictest	521056	-21	kworker/u16:0+flush-8:0	11:09:13	4
138642	99	32	29,2	cyclictest	190396	-21	kworker/u16:2+flush-8:0	07:54:13	4
138185	2	32	22,7	sleep6	0	-21	swapper/6	07:06:29	6
138648	99	31	29,2	cyclictest	4067447	-21	kworker/u16:4+flush-8:0	08:24:12	6
138648	99	31	29,2	cyclictest	137422	-21	kworker/u16:0+flush-8:0	07:14:13	6
138648	99	31	28,2	cyclictest	90620	-21	kworker/u16:1+flush-8:0	07:58:59	6
138642	99	31	29,2	cyclictest	286315	-21	kworker/u16:0+flush-8:0	08:34:06	4
138648	99	30	28,2	cyclictest	286315	-21	kworker/u16:0+events_unbound	09:18:45	6
138648	99	30	28,2	cyclictest	137422	-21	kworker/u16:0+events_unbound	07:18:55	6
138642	99	30	28,2	cyclictest	772646	-21	kworker/u16:3+events_unbound	11:56:53	4
138652	99	29	24,4	cyclictest	493164	-21	latency_hist	09:58:42	7
138648	99	29	27,2	cyclictest	521056	-21	kworker/u16:0+events_unbound	10:14:11	6
138648	99	29	27,2	cyclictest	4067447	-21	kworker/u16:4+events_unbound	08:32:26	6
138642	99	28	26,2	cyclictest	830419	-21	kworker/u16:1+events_unbound	12:28:21	4
138642	99	28	26,2	cyclictest	711969	-21	kworker/u16:1+events_unbound	11:38:21	4
138642	99	28	26,2	cyclictest	362378	-21	kworker/u16:3+events_unbound	09:14:11	4
588495	2	27	0,1	chrt	588494	-21	ssh	10:36:00	7
237643	2	27	0,1	sleep5	0	-21	swapper/5	07:54:11	5
138648	99	27	25,2	cyclictest	90620	-21	kworker/u16:1+events_unbound	07:48:06	6
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17