OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #b, slot #1

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance

Characteristics of the 100 highest latencies:

System rackbslot1.osadl.org (updated Sun Nov 30, 2025 12:44:32)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
4058501	99	1211	1206,3	cyclictest	20062	-21	apache_accesses	09:40:05	0
4058294	2	215	186,19	sleep3	0	-21	swapper/3	07:09:59	3
4058294	2	215	186,19	sleep3	0	-21	swapper/3	07:09:58	3
4058182	2	208	184,14	sleep1	0	-21	swapper/1	07:08:32	1
4058182	2	208	184,14	sleep1	0	-21	swapper/1	07:08:31	1
4058117	2	208	184,14	sleep2	0	-21	swapper/2	07:07:45	2
4058117	2	208	184,14	sleep2	0	-21	swapper/2	07:07:45	2
4058247	2	203	180,13	sleep0	0	-21	swapper/0	07:09:22	0
4058247	2	203	180,13	sleep0	0	-21	swapper/0	07:09:22	0
4058501	99	73	68,4	cyclictest	247515	-21	ThreadPoolForeg	10:05:24	0
4058504	99	71	54,14	cyclictest	4157520	-21	snmp_ryswitch.c	08:45:01	2
4058504	99	67	62,3	cyclictest	35185	-21	latency_hist	09:55:02	2
4058507	99	61	57,3	cyclictest	56308	-21	kworker/u8:3+flush-179:0	10:40:30	3
4058507	99	61	5,55	cyclictest	4031426	-21	kworker/u8:4+efi_rts_wq	07:46:58	3
4058507	99	61	48,12	cyclictest	4130057	-21	kworker/u8:0+flush-179:0	08:26:49	3
200864	2	61	0,1	chrt	200855	-21	snmp_areswitch.	12:35:04	0
4058502	99	60	56,2	cyclictest	4130057	-21	kworker/u8:0+flush-179:0	09:45:00	1
4058502	99	60	55,4	cyclictest	4031426	-21	kworker/u8:4+flush-179:0	07:20:36	1
4058502	99	60	55,4	cyclictest	4031426	-21	kworker/u8:4+flush-179:0	07:20:36	1
4058502	99	60	45,2	cyclictest	4157533	-21	ThreadPoolForeg	08:45:02	1
4058507	99	59	56,2	cyclictest	148094	-21	kworker/u8:0+flush-179:0	12:10:08	3
4058501	99	59	54,4	cyclictest	247561	-21	ThreadPoolForeg	10:00:31	0
4058502	99	58	54,3	cyclictest	4074652	-21	kworker/u8:1+events_unbound	08:00:01	1
4058507	99	56	52,3	cyclictest	184692	-21	kworker/u8:1+events_unbound	12:35:18	3
4058507	99	56	38,3	cyclictest	4100353	-21	kworker/u8:3+flush-179:0	08:00:15	3
4058502	99	56	51,4	cyclictest	29705	-21	kworker/u8:1+flush-179:0	10:00:31	1
4058504	99	55	51,3	cyclictest	76405	-21	systemd-journal	07:30:28	2
195765	2	55	0,1	chrt	195618	-21	/usr/share/muni	12:30:03	0
4058507	99	54	50,3	cyclictest	4156760	-21	kworker/u8:4+events_unbound	09:20:29	3
4058507	99	54	50,3	cyclictest	4047794	-21	kworker/u8:3+events_unbound	07:30:33	3
4058504	99	54	44,9	cyclictest	4172951	-21	/usr/share/muni	09:00:15	2
4058502	99	54	51,2	cyclictest	127996	-21	kworker/u8:0+events_unbound	11:30:16	1
4058501	99	54	4,50	cyclictest	4031426	-21	kworker/u8:4+efi_rts_wq	07:46:59	0
4058507	99	53	50,2	cyclictest	39256	-21	kworker/u8:2+flush-179:0	10:00:21	3
4058507	99	53	49,3	cyclictest	148094	-21	kworker/u8:0+events_unbound	12:17:02	3
4058507	99	53	47,4	cyclictest	87006	-21	kworker/u8:0+flush-179:0	11:12:31	3
4058502	99	53	42,10	cyclictest	4130057	-21	kworker/u8:0+flush-179:0	08:50:33	1
4058501	99	53	47,5	cyclictest	4193122	-21	latency_hist	09:20:01	0
4058502	99	52	48,3	cyclictest	102627	-21	kworker/u8:1+flush-179:0	11:20:01	1
4058502	99	52	47,3	cyclictest	127996	-21	kworker/u8:0+flush-179:0	11:25:20	1
4058502	99	52	46,4	cyclictest	3125725	-21	ThreadPoolForeg	11:39:46	1
4058507	99	51	47,3	cyclictest	4156760	-21	kworker/u8:4+events_unbound	09:50:12	3
4058507	99	51	47,3	cyclictest	4100353	-21	kworker/u8:3+flush-179:0	07:55:00	3
4058507	99	51	47,3	cyclictest	4094932	-21	kworker/u8:2+flush-179:0	08:17:02	3
4058507	99	51	47,3	cyclictest	14137	-21	kworker/u8:3+events_unbound	09:35:21	3
4058504	99	51	39,11	cyclictest	3125724	-21	ThreadPoolForeg	10:20:15	2
4058504	99	51	39,11	cyclictest	3125724	-21	ThreadPoolForeg	10:20:15	2
4058502	99	51	48,2	cyclictest	14137	-21	kworker/u8:3+flush-179:0	09:35:27	1
4058502	99	51	47,3	cyclictest	4192457	-21	kworker/u8:1+flush-179:0	09:20:04	1
4058502	99	51	47,3	cyclictest	4074652	-21	kworker/u8:1+events_unbound	08:05:14	1
4058502	99	51	47,3	cyclictest	4031426	-21	kworker/u8:4+flush-179:0	07:50:14	1
4058502	99	51	47,3	cyclictest	39256	-21	kworker/u8:2+events_unbound	10:35:08	1
4058502	99	51	47,2	cyclictest	4130057	-21	kworker/u8:0+flush-179:0	10:15:28	1
4058502	99	51	46,4	cyclictest	247560	-21	ThreadPoolForeg	07:45:00	1
4058507	99	50	46,3	cyclictest	87006	-21	kworker/u8:0+flush-179:0	11:05:24	3
4058507	99	50	46,3	cyclictest	102627	-21	kworker/u8:1+events_unbound	11:44:04	3
4058507	99	49	45,3	cyclictest	133021	-21	kworker/u8:2+flush-179:0	11:50:22	3
4058507	99	49	45,3	cyclictest	127996	-21	kworker/u8:0+flush-179:0	11:35:21	3
4058502	99	49	45,3	cyclictest	158878	-21	kworker/u8:4+events_unbound	12:00:32	1
4058502	99	49	45,3	cyclictest	102627	-21	kworker/u8:1+events_unbound	11:45:30	1
4058502	99	49	44,4	cyclictest	29705	-21	kworker/u8:1+flush-179:0	09:50:33	1
4058501	99	49	43,4	cyclictest	4085202	-21	/usr/share/muni	07:35:15	0
4058507	99	48	46,2	cyclictest	102627	-21	kworker/u8:1+events_unbound	11:55:09	3
4058507	99	48	45,2	cyclictest	4130057	-21	kworker/u8:0+events_unbound	08:21:33	3
4058507	99	48	44,3	cyclictest	39256	-21	kworker/u8:2+events_unbound	10:10:25	3
4058507	99	48	44,3	cyclictest	186640	-21	apt-get	12:20:12	3
4058507	99	48	43,4	cyclictest	4130057	-21	kworker/u8:0+flush-179:0	08:50:16	3
4058504	99	48	44,3	cyclictest	4156760	-21	kworker/u8:4+flush-179:0	09:20:14	2
4058504	99	48	1,28	cyclictest	3125727	-21	ThreadPoolForeg	09:37:38	2
4058502	99	48	44,3	cyclictest	4074652	-21	kworker/u8:1+flush-179:0	07:25:18	1
4058502	99	48	44,3	cyclictest	133021	-21	kworker/u8:2+flush-179:0	12:15:14	1
4058502	99	48	43,4	cyclictest	133021	-21	kworker/u8:2+flush-179:0	11:50:27	1
4058501	99	48	41,5	cyclictest	247560	-21	ThreadPoolForeg	09:05:05	0
4058507	99	47	44,2	cyclictest	4100353	-21	kworker/u8:3+flush-179:0	08:45:22	3
4058507	99	47	43,3	cyclictest	4074652	-21	kworker/u8:1+events_unbound	08:10:01	3
4058507	99	47	43,3	cyclictest	153405	-21	munin-limits	11:45:36	3
4058504	99	47	40,5	cyclictest	102930	-21	/usr/share/muni	11:00:10	2
4058504	99	47	36,10	cyclictest	4155	-21	kworker/2:0+pm	09:30:38	2
4058502	99	47	44,2	cyclictest	4156760	-21	kworker/u8:4+flush-179:0	09:15:33	1
4058502	99	47	43,3	cyclictest	4100353	-21	kworker/u8:3+flush-179:0	08:10:32	1
4058502	99	47	31,15	cyclictest	3954558	-1	kworker/1:2H+kblockd	07:35:49	1
4058501	99	47	44,2	cyclictest	4130057	-21	kworker/u8:0+flush-179:0	09:30:15	0
4058501	99	47	41,4	cyclictest	56308	-21	kworker/u8:3+flush-179:0	10:23:52	0
4058501	99	47	41,4	cyclictest	56308	-21	kworker/u8:3+flush-179:0	10:23:52	0
4058501	99	47	40,4	cyclictest	4084685	-31	munin-graph	07:32:19	0
4058507	99	46	43,2	cyclictest	4063311	-21	kworker/u8:0+events_unbound	07:35:25	3
4058507	99	46	42,3	cyclictest	4074652	-21	kworker/u8:1+flush-179:0	07:55:20	3
4058507	99	46	41,4	cyclictest	4162096	-21	kworker/u8:5+flush-179:0	08:57:05	3
4058507	99	46	41,3	cyclictest	4063311	-21	kworker/u8:0+flush-179:0	07:20:13	3
4058507	99	46	41,3	cyclictest	4063311	-21	kworker/u8:0+flush-179:0	07:20:13	3
4058507	99	46	40,4	cyclictest	25071	-21	/usr/share/muni	09:45:25	3
4058507	99	46	40,4	cyclictest	25071	-21	/usr/share/muni	09:45:25	3
4058502	99	46	41,3	cyclictest	4111011	-21	/usr/share/muni	08:00:16	1
4058501	99	46	42,3	cyclictest	184344	-21	munin-limits	12:15:34	0
4178190	2	45	0,3	sleep2	4178174	-21	snmp_wlanswitch	09:05:03	2
4058507	99	45	42,2	cyclictest	56308	-21	kworker/u8:3+events_unbound	11:20:35	3
4058507	99	45	41,3	cyclictest	4074652	-21	kworker/u8:1+flush-179:0	07:25:34	3
4058507	99	45	41,3	cyclictest	39256	-21	kworker/u8:2+events_unbound	10:50:15	3
4058504	99	45	36,8	cyclictest	4152016	-21	kworker/2:2+pm	09:16:03	2
4058504	99	45	35,9	cyclictest	4042929	-21	kworker/2:0+pm	08:14:06	2
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17