OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Thu Mar 26, 2026 12:43:34)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
212921	2	104	0,0	sleep4	0	-21	swapper/4	09:23:38	4
591166	2	97	0,1	sleep7	591167	-21	tr	12:03:22	7
571577	2	97	0,0	sleep5	0	-21	swapper/5	11:53:26	5
443932	2	78	0,1	sleep7	443933	-21	ssh	11:03:03	7
4115091	99	55	53,2	cyclictest	395620	-21	kworker/u16:5+events_unbound	10:53:32	5
4115091	99	55	53,2	cyclictest	3415691	-21	kworker/u16:5+flush-8:0	07:18:18	5
4115091	99	53	51,2	cyclictest	335520	-21	kworker/u16:3+events_unbound	10:23:32	5
4115099	99	52	0,50	cyclictest	635476	-21	latency_hist	12:23:02	7
4115091	99	51	49,2	cyclictest	585448	-21	kworker/u16:0+flush-8:0	12:08:32	5
4115091	99	51	49,2	cyclictest	383629	-21	kworker/u16:1+flush-8:0	10:43:32	5
4115099	99	50	49,0	cyclictest	0	-21	swapper/7	07:30:50	7
4115099	99	50	49,0	cyclictest	0	-21	swapper/7	07:30:50	7
4115091	99	50	49,1	cyclictest	395620	-21	kworker/u16:5+events_unbound	11:06:31	5
4115091	99	50	48,2	cyclictest	177026	-21	kworker/u16:4+flush-8:0	09:13:34	5
4115099	99	49	44,4	cyclictest	515695	-21	latency_hist	11:33:02	7
4115091	99	49	47,2	cyclictest	383629	-21	kworker/u16:1+flush-8:0	10:58:35	5
4115091	99	49	47,2	cyclictest	238384	-21	kworker/u16:2+flush-8:0	09:43:17	5
4115091	99	48	46,2	cyclictest	4166519	-21	kworker/u16:3+events_unbound	09:08:31	5
4114782	2	48	38,7	sleep5	0	-21	swapper/5	07:02:37	5
4115091	99	47	45,2	cyclictest	441574	-21	kworker/u16:0+events_unbound	11:27:53	5
4115091	99	47	45,2	cyclictest	395620	-21	kworker/u16:5+events_unbound	12:28:36	5
4115091	99	47	45,2	cyclictest	261960	-21	kworker/u16:0+events_unbound	09:52:25	5
4115091	99	46	44,2	cyclictest	261960	-21	kworker/u16:0+events_unbound	10:53:00	5
4115091	99	46	44,2	cyclictest	261960	-21	kworker/u16:0+events_unbound	10:28:29	5
4115091	99	46	44,2	cyclictest	238384	-21	kworker/u16:2+events_unbound	10:40:36	5
4115096	99	45	42,2	cyclictest	3405491	-21	kworker/u16:4+events_unbound	07:08:06	6
4115091	99	45	43,2	cyclictest	585448	-21	kworker/u16:0+events_unbound	12:05:52	5
4115091	99	45	43,2	cyclictest	395620	-21	kworker/u16:5+events_unbound	11:44:53	5
4115091	99	45	43,2	cyclictest	177026	-21	kworker/u16:4+events_unbound	09:41:09	5
4115091	99	44	42,2	cyclictest	275476	-21	kworker/u16:1+events_unbound	10:07:05	5
4115091	99	43	41,2	cyclictest	146014	-21	kworker/u16:1+events_unbound	09:25:29	5
4115091	99	43	40,2	cyclictest	146014	-21	kworker/u16:1+events_unbound	09:28:13	5
4115089	99	43	41,2	cyclictest	238384	-21	kworker/u16:2+events_unbound	10:38:19	4
4115091	99	42	40,2	cyclictest	467802	-21	kworker/u16:2+events_unbound	11:15:01	5
4115091	99	42	40,2	cyclictest	261960	-21	kworker/u16:0+flush-8:0	10:13:34	5
4115091	99	42	40,2	cyclictest	261960	-21	kworker/u16:0+flush-8:0	10:13:34	5
4115091	99	41	39,2	cyclictest	68851	-21	kworker/u16:2+flush-8:0	08:28:27	5
4115091	99	41	39,2	cyclictest	395620	-21	kworker/u16:5+flush-8:0	11:33:41	5
4115091	99	41	39,2	cyclictest	395620	-21	kworker/u16:5+flush-8:0	11:33:41	5
4115091	99	41	39,2	cyclictest	395620	-21	kworker/u16:5+events_unbound	11:48:15	5
4115091	99	41	39,2	cyclictest	395620	-21	kworker/u16:5+events_unbound	11:40:45	5
4115091	99	41	39,2	cyclictest	238384	-21	kworker/u16:2+events_unbound	10:01:01	5
4115091	99	41	39,2	cyclictest	238384	-21	kworker/u16:2+events_unbound	09:36:45	5
4115091	99	40	38,2	cyclictest	585448	-21	kworker/u16:0+events_unbound	12:18:34	5
4115091	99	40	38,2	cyclictest	467802	-21	kworker/u16:2+events_unbound	11:20:16	5
4115091	99	40	38,2	cyclictest	395620	-21	kworker/u16:5+events_unbound	12:26:00	5
4115091	99	40	38,2	cyclictest	395620	-21	kworker/u16:5+events_unbound	12:02:25	5
4115091	99	40	37,2	cyclictest	4177880	-21	kworker/u16:4+events_unbound	07:48:16	5
4115099	99	39	34,4	cyclictest	440837	-21	switchtime	10:58:41	7
4115096	99	39	35,3	cyclictest	383629	-21	kworker/u16:1+flush-8:0	10:43:03	6
4115091	99	39	37,2	cyclictest	395620	-21	kworker/u16:5+flush-8:0	12:13:20	5
4115091	99	39	37,2	cyclictest	395620	-21	kworker/u16:5+events_unbound	11:30:49	5
4115091	99	39	37,2	cyclictest	383629	-21	kworker/u16:1+events_unbound	11:10:41	5
4115091	99	39	37,2	cyclictest	335520	-21	kworker/u16:3+events_unbound	10:20:53	5
4115091	99	39	37,2	cyclictest	261960	-21	kworker/u16:0+flush-8:0	09:54:09	5
4115091	99	39	37,2	cyclictest	238384	-21	kworker/u16:2+events_unbound	10:10:40	5
4115091	99	39	36,2	cyclictest	335520	-21	kworker/u16:3+events_unbound	10:33:06	5
4115089	99	39	36,2	cyclictest	395620	-21	kworker/u16:5+flush-8:0	11:48:18	4
4114811	2	39	29,7	sleep4	0	-21	swapper/4	07:03:00	4
4115091	99	38	36,2	cyclictest	68851	-21	kworker/u16:2+flush-8:0	08:23:32	5
4115091	99	38	35,2	cyclictest	68851	-21	kworker/u16:2+events_unbound	08:18:17	5
4114569	2	38	3,31	sleep7	0	-21	swapper/7	06:59:34	7
4115096	99	37	34,2	cyclictest	585448	-21	kworker/u16:0+flush-8:0	12:17:57	6
4115091	99	37	35,2	cyclictest	146014	-21	kworker/u16:1+events_unbound	09:04:34	5
4115091	99	37	35,2	cyclictest	11766	-21	kworker/u16:0+events_unbound	08:48:33	5
4115089	99	37	35,2	cyclictest	467802	-21	kworker/u16:2+flush-8:0	11:43:31	4
4115099	99	36	35,0	cyclictest	0	-21	swapper/7	09:32:55	7
4115089	99	36	34,2	cyclictest	4177879	-21	kworker/u16:1+events_unbound	07:33:30	4
4115089	99	36	34,2	cyclictest	146014	-21	kworker/u16:1+events_unbound	08:58:27	4
4115091	99	35	33,2	cyclictest	3405491	-21	kworker/u16:4+events_unbound	07:13:34	5
4115089	99	35	33,2	cyclictest	4177880	-21	kworker/u16:4+events_unbound	07:39:53	4
4115089	99	35	33,2	cyclictest	177026	-21	kworker/u16:4+events_unbound	09:13:44	4
4115099	99	34	33,0	cyclictest	0	-21	swapper/7	08:07:54	7
4115099	99	34	0,33	cyclictest	1033	-21	rs:main		7
4115091	99	34	32,2	cyclictest	4166519	-21	kworker/u16:3+flush-8:0	07:58:26	5
4115089	99	34	32,2	cyclictest	383629	-21	kworker/u16:1+events_unbound	11:02:39	4
4114709	2	34	24,7	sleep6	0	-21	swapper/6	07:01:34	6
4115099	99	33	30,2	cyclictest	108248	-21	df	08:38:15	7
4115096	99	33	31,2	cyclictest	395620	-21	kworker/u16:5+flush-8:0	11:48:30	6
4115091	99	33	31,2	cyclictest	177026	-21	kworker/u16:4+events_unbound	09:21:01	5
4115089	99	33	31,2	cyclictest	238384	-21	kworker/u16:2+events_unbound	10:37:46	4
4115089	99	33	31,2	cyclictest	238384	-21	kworker/u16:2+events_unbound	10:15:43	4
4115089	99	33	31,2	cyclictest	238384	-21	kworker/u16:2+events_unbound	10:15:43	4
4115089	99	33	31,2	cyclictest	146014	-21	kworker/u16:1+events_unbound	09:03:46	4
4115089	99	33	30,2	cyclictest	261960	-21	kworker/u16:0+events_unbound	10:43:10	4
4115099	99	32	29,2	cyclictest	393267	-21	ssh	10:38:51	7
4115099	99	32	27,4	cyclictest	611573	-21	latency_hist	12:13:02	7
4115089	99	32	30,2	cyclictest	261960	-21	kworker/u16:0+events_unbound	10:49:04	4
268896	2	32	0,1	chrt	268897	-21	kthreadcore	09:48:25	7
4115099	99	31	28,2	cyclictest	525416	-21	ssh	11:33:47	7
4115099	99	31	28,2	cyclictest	525416	-21	ssh	11:33:47	7
4115091	99	31	29,2	cyclictest	146014	-21	kworker/u16:1+flush-8:0	08:58:26	5
4115089	99	31	29,2	cyclictest	68851	-21	kworker/u16:2+events_unbound	08:19:53	4
4115089	99	31	29,2	cyclictest	3405491	-21	kworker/u16:4+events_unbound	07:15:49	4
4115089	99	31	29,2	cyclictest	3405491	-21	kworker/u16:4+events_unbound	07:05:53	4
4115089	99	31	29,2	cyclictest	275476	-21	kworker/u16:1+events_unbound	10:03:43	4
4115089	99	31	29,2	cyclictest	146014	-21	kworker/u16:1+flush-8:0	09:18:36	4
4115099	99	30	29,0	cyclictest	0	-21	swapper/7	08:28:35	7
4115099	99	30	29,0	cyclictest	0	-21	swapper/7	07:11:41	7
4115099	99	30	25,4	cyclictest	563610	-21	latency_hist	11:53:02	7
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17