OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Fri Aug 14, 2026 12:43:34)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
2223842	2	56	46,7	sleep4	0	-21	swapper/4	07:07:56	4
2223536	2	49	25,7	sleep5	0	-21	swapper/5	07:03:40	5
2223796	2	46	12,29	sleep7	0	-21	swapper/7	07:07:19	7
2224153	99	37	32,4	cyclictest	2791014	-21	latency_hist	11:22:57	7
2224147	99	35	32,2	cyclictest	2778148	-21	kworker/u16:0+flush-8:0	11:22:57	4
2224150	99	34	32,2	cyclictest	2803199	-21	kworker/u16:4+events_unbound	11:53:31	5
2224150	99	34	32,2	cyclictest	2803199	-21	kworker/u16:4+events_unbound	11:53:31	5
2224150	99	33	32,1	cyclictest	2581590	-21	kworker/u16:2+events_unbound	10:13:00	5
2224150	99	33	30,2	cyclictest	2581590	-21	kworker/u16:2+flush-8:0	09:58:03	5
2224152	99	30	28,2	cyclictest	2168740	-21	kworker/u16:1+events_unbound	07:16:52	6
2224150	99	30	28,2	cyclictest	2224335	-21	kworker/u16:2+events_unbound	08:03:28	5
2223766	2	30	20,7	sleep6	0	-21	swapper/6	07:06:54	6
2224153	99	29	23,5	cyclictest	2344745	-21	latency_hist	08:07:58	7
2224150	99	28	26,2	cyclictest	2522283	-21	kworker/u16:1+events_unbound	10:21:07	5
2874809	2	27	0,1	chrt	2874808	-21	ssh	11:55:02	7
2874809	2	27	0,1	chrt	2874808	-21	ssh	11:55:02	7
2224150	99	27	25,2	cyclictest	2471920	-21	kworker/u16:0+events_unbound	09:16:51	5
2224150	99	27	25,2	cyclictest	2471920	-21	kworker/u16:0+events_unbound	09:16:51	5
2224150	99	26	24,2	cyclictest	2657074	-21	kworker/u16:3+flush-8:0	10:41:07	5
2224150	99	25	23,2	cyclictest	2168740	-21	kworker/u16:1+events_unbound	07:12:53	5
2224150	99	24	22,2	cyclictest	2740621	-21	kworker/u16:1+events_unbound	11:07:57	5
2224150	99	24	22,2	cyclictest	2471920	-21	kworker/u16:0+flush-8:0	09:12:57	5
2224150	99	24	21,2	cyclictest	2433615	-21	kworker/u16:1+flush-8:0	09:18:04	5
2224150	99	23	21,2	cyclictest	2947592	-21	kworker/u16:1+events_unbound	12:27:28	5
2224150	99	23	21,2	cyclictest	2888769	-21	kworker/u16:5+events_unbound	12:19:07	5
2224150	99	23	21,2	cyclictest	2556468	-21	kworker/u16:0+events_unbound	09:48:23	5
2224150	99	23	21,2	cyclictest	2325333	-21	kworker/u16:5+flush-8:0	08:13:14	5
2224147	99	23	21,2	cyclictest	2534874	-21	kworker/u16:4+events_unbound	09:58:11	4
2224153	99	22	0,20	cyclictest	2522243	-21	latency_hist	09:32:58	7
2224150	99	22	20,2	cyclictest	2742124	-21	kworker/u16:4+events_unbound	11:15:23	5
2224150	99	22	20,2	cyclictest	2657074	-21	kworker/u16:3+events_unbound	11:01:07	5
2224150	99	22	20,2	cyclictest	2471920	-21	kworker/u16:0+flush-8:0	09:37:07	5
2224150	99	22	20,2	cyclictest	2412532	-21	kworker/u16:0+events_unbound	08:43:18	5
2224153	99	21	20,0	cyclictest	0	-21	swapper/7	07:23:15	7
2224150	99	21	19,2	cyclictest	2888769	-21	kworker/u16:5+events_unbound	12:13:21	5
2224150	99	21	19,2	cyclictest	2803199	-21	kworker/u16:4+events_unbound	12:28:11	5
2224150	99	21	19,2	cyclictest	2778148	-21	kworker/u16:0+flush-8:0	11:48:31	5
2224150	99	21	19,2	cyclictest	2679042	-21	kworker/u16:1+flush-8:0	10:53:08	5
2224150	99	21	19,2	cyclictest	2679042	-21	kworker/u16:1+flush-8:0	10:35:08	5
2224150	99	21	19,2	cyclictest	2581590	-21	kworker/u16:2+events_unbound	10:25:33	5
2224150	99	21	18,2	cyclictest	2534874	-21	kworker/u16:4+events_unbound	09:43:08	5
2704335	2	20	0,1	chrt	2704336	-21	ssh	10:45:46	7
2224153	99	20	19,0	cyclictest	0	-21	swapper/7	07:38:12	7
2224153	99	20	18,1	cyclictest	0	-21	swapper/7	10:28:11	7
2224153	99	20	1,17	cyclictest	2677952	-21	ssh	10:33:21	7
2224150	99	20	18,2	cyclictest	2824979	-21	kworker/u16:1+flush-8:0	12:03:07	5
2224150	99	20	18,2	cyclictest	2803199	-21	kworker/u16:4+events_unbound	11:30:39	5
2224150	99	20	18,2	cyclictest	2740621	-21	kworker/u16:1+flush-8:0	11:10:07	5
2224150	99	20	18,2	cyclictest	2642260	-21	kworker/u16:0+flush-8:0	10:49:08	5
2224150	99	20	18,2	cyclictest	2581590	-21	kworker/u16:2+events_unbound	10:10:08	5
2224150	99	20	18,2	cyclictest	2433615	-21	kworker/u16:1+events_unbound	09:23:36	5
2224150	99	20	18,2	cyclictest	2168740	-21	kworker/u16:1+events_unbound	07:13:01	5
2224153	99	19	18,0	cyclictest	0	-21	swapper/7	09:18:12	7
2224153	99	19	15,3	cyclictest	1302	-21	snmpd	07:21:29	7
2224150	99	19	17,2	cyclictest	2838295	-21	kworker/u16:2+flush-8:0	11:43:08	5
2224150	99	19	17,2	cyclictest	2824979	-21	kworker/u16:1+events_unbound	11:40:39	5
2224150	99	19	17,2	cyclictest	2803199	-21	kworker/u16:4+events_unbound	12:12:07	5
2224150	99	19	17,2	cyclictest	2740621	-21	kworker/u16:1+events_unbound	11:25:07	5
2224153	99	18	0,16	cyclictest	2803250	-21	latency_hist	11:27:57	7
2224153	99	18	0,16	cyclictest	2778785	-21	latency_hist	11:17:57	7
2224153	99	18	0,16	cyclictest	1302	-21	snmpd	08:58:28	7
2224150	99	18	16,2	cyclictest	2803199	-21	kworker/u16:4+events_unbound	12:35:32	5
2224150	99	18	16,2	cyclictest	2581590	-21	kworker/u16:2+events_unbound	09:55:39	5
2224150	99	18	16,2	cyclictest	2581590	-21	kworker/u16:2+events_unbound	09:55:39	5
2224153	99	17	15,1	cyclictest	0	-21	swapper/7	08:08:12	7
2224153	99	17	0,1	cyclictest	0	-21	swapper/7	12:03:22	7
2224153	99	17	0,1	cyclictest	0	-21	swapper/7	10:58:24	7
2224153	99	17	0,1	cyclictest	0	-21	swapper/7	07:48:21	7
2224153	99	17	0,1	cyclictest	0	-21	swapper/7	07:48:21	7
2224153	99	17	0,15	cyclictest	2580464	-21	sh	09:53:26	7
2224153	99	17	0,15	cyclictest	2580464	-21	sh	09:53:26	7
2224152	99	17	14,2	cyclictest	2778148	-21	kworker/u16:0+flush-8:0	11:47:58	6
2224152	99	17	0,0	cyclictest	0	-21	swapper/6	09:37:52	6
2224150	99	17	15,2	cyclictest	2657074	-21	kworker/u16:3+events_unbound	11:20:47	5
2224150	99	17	15,2	cyclictest	2364056	-21	kworker/u16:1+flush-8:0	08:18:22	5
2224147	99	17	16,1	cyclictest	2803199	-21	kworker/u16:4+flush-8:0	12:07:57	4
2224147	99	17	13,3	cyclictest	2352931	-21	kthreadcore	08:08:19	4
2224153	99	16	10,5	cyclictest	2305955	-21	latency_hist	07:47:58	7
2224153	99	16	0,14	cyclictest	0	-21	swapper/7	10:18:22	7
2224150	99	16	14,2	cyclictest	2803199	-21	kworker/u16:4+events_unbound	11:59:39	5
2224150	99	16	14,2	cyclictest	2803199	-21	kworker/u16:4+events_unbound	11:59:39	5
2224150	99	16	14,2	cyclictest	2401949	-21	kworker/u16:3+events_unbound	08:52:59	5
2224153	99	15	11,3	cyclictest	2247291	-21	fwupd	07:16:53	7
2224153	99	15	0,13	cyclictest	0	-21	swapper/7	08:23:21	7
2224152	99	15	0,15	cyclictest	0	-21	swapper/6	10:31:10	6
2224152	99	15	0,14	cyclictest	2463324	-21	grep	09:08:15	6
2224150	99	15	13,2	cyclictest	2657074	-21	kworker/u16:3+events_unbound	10:45:31	5
2224150	99	15	13,2	cyclictest	2657074	-21	kworker/u16:3+events_unbound	10:32:47	5
2224150	99	15	13,2	cyclictest	2534874	-21	kworker/u16:4+flush-8:0	10:05:47	5
2224150	99	15	13,2	cyclictest	2497559	-21	kworker/u16:3+flush-8:0	09:38:55	5
2224150	99	15	13,2	cyclictest	2471920	-21	kworker/u16:0+events_unbound	09:32:08	5
2224150	99	15	13,2	cyclictest	2247355	-21	kworker/u16:5+flush-8:0	07:28:40	5
2224147	99	15	0,1	cyclictest	0	-21	swapper/4	09:31:43	4
2224147	99	15	0,15	cyclictest	0	-21	swapper/4	10:37:18	4
2224153	99	14	12,1	cyclictest	0	-21	swapper/7	08:13:23	7
2224153	99	14	11,2	cyclictest	0	-21	swapper/7	11:58:26	7
2224153	99	14	11,2	cyclictest	0	-21	swapper/7	11:58:26	7
2224153	99	14	11,2	cyclictest	0	-21	swapper/7	08:27:58	7
2224153	99	14	0,13	cyclictest	2473348	-21	turbostat	09:12:57	7
2224152	99	14	8,2	cyclictest	0	-21	swapper/6	08:33:25	6
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17