OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #b, slot #1

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance

Characteristics of the 100 highest latencies:

System rackbslot1.osadl.org (updated Thu Dec 11, 2025 12:44:38)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
3609384	2	216	186,19	sleep3	0	-21	swapper/3	07:07:30	3
3609381	2	192	159,24	sleep0	0	-21	swapper/0	07:07:27	0
3609363	2	192	167,14	sleep2	0	-21	swapper/2	07:07:13	2
3609352	2	185	161,14	sleep1	0	-21	swapper/1	07:07:04	1
3609793	99	73	57,14	cyclictest	3816775	-21	kworker/u8:1+flush-179:0	10:31:50	2
3609790	99	68	65,2	cyclictest	3895786	-21	ThreadPoolForeg	11:45:01	0
3609794	99	63	57,5	cyclictest	247560	-21	ThreadPoolForeg	09:10:34	3
3609793	99	63	59,3	cyclictest	3651421	-21	kworker/u8:2+flush-179:0	08:30:01	2
3609793	99	62	56,4	cyclictest	3724489	-21	kworker/u8:3+flush-179:0	09:10:10	2
3609793	99	62	56,4	cyclictest	3677366	-21	kworker/u8:0+events_unbound	08:20:29	2
3609794	99	61	55,5	cyclictest	3125725	-21	ThreadPoolForeg	10:40:01	3
3609793	99	61	56,3	cyclictest	3708438	-21	kworker/u8:2+flush-179:0	10:05:20	2
3609793	99	60	55,4	cyclictest	3942601	-21	kworker/u8:0+flush-179:0	12:40:00	2
3609793	99	59	54,4	cyclictest	3873935	-21	kworker/u8:1+flush-179:0	11:25:40	2
3609793	99	59	54,4	cyclictest	3833924	-21	kworker/u8:5+events_unbound	10:55:01	2
3724071	2	58	0,1	sleep2	0	-21	swapper/2	08:55:39	2
3609793	99	58	53,3	cyclictest	3567397	-21	kworker/u8:0+events_unbound	07:50:19	2
3609793	99	58	52,4	cyclictest	3709113	-21	chrome	08:45:01	2
3609793	99	58	51,5	cyclictest	3760346	-21	kworker/u8:1+flush-179:0	10:15:31	2
3609790	99	58	52,4	cyclictest	3708061	-21	munin-limits	08:40:35	0
3609794	99	57	53,3	cyclictest	3873935	-21	kworker/u8:1+flush-179:0	11:35:42	3
3609793	99	57	52,4	cyclictest	3724489	-21	kworker/u8:3+events_unbound	09:30:11	2
3609793	99	56	51,4	cyclictest	3775599	-21	kworker/u8:0+events_unbound	09:55:17	2
3609790	99	56	52,3	cyclictest	3889926	-21	kworker/u8:3+flush-179:0	12:05:07	0
3609793	99	55	51,3	cyclictest	3775599	-21	kworker/u8:0+flush-179:0	10:35:02	2
3609793	99	55	51,3	cyclictest	3651421	-21	kworker/u8:2+flush-179:0	08:00:00	2
3609793	99	55	51,3	cyclictest	3651421	-21	kworker/u8:2+flush-179:0	08:00:00	2
3609793	99	55	49,4	cyclictest	3775599	-21	kworker/u8:0+flush-179:0	10:40:08	2
3609791	99	54	50,3	cyclictest	3942601	-21	kworker/u8:0+flush-179:0	12:30:15	1
3609791	99	54	46,6	cyclictest	3817490	-21	/usr/share/muni	10:30:02	1
3609790	99	54	49,4	cyclictest	3879702	-21	kworker/u8:0+flush-179:0	11:57:06	0
3609793	99	53	49,3	cyclictest	3708438	-21	kworker/u8:2+flush-179:0	09:25:12	2
3609793	99	52	45,5	cyclictest	3667615	-21	/usr/share/muni	08:05:04	2
3609793	99	52	38,13	cyclictest	3677366	-21	kworker/u8:0+flush-179:0	09:00:16	2
3609793	99	51	47,3	cyclictest	3654405	-21	kworker/u8:4+flush-179:0	08:40:30	2
3609791	99	51	43,5	cyclictest	3921485	-21	/usr/share/muni	12:10:01	1
3609794	99	50	44,5	cyclictest	76405	-21	systemd-journal	12:17:02	3
3609794	99	50	4,45	cyclictest	3646767	-21	chrome	07:45:02	3
3609793	99	50	47,2	cyclictest	3833924	-21	kworker/u8:5+flush-179:0	10:50:01	2
3609793	99	50	46,3	cyclictest	3889926	-21	kworker/u8:3+flush-179:0	11:45:17	2
3609793	99	50	46,3	cyclictest	3760346	-21	kworker/u8:1+events_unbound	10:14:05	2
3609793	99	50	45,4	cyclictest	3801721	-21	kworker/u8:3+flush-179:0	10:20:34	2
3609793	99	50	44,5	cyclictest	3831714	-21	kworker/u8:3+flush-179:0	11:13:22	2
3609790	99	50	41,7	cyclictest	247515	-21	ThreadPoolForeg	08:20:18	0
3609793	99	49	46,2	cyclictest	3801721	-21	kworker/u8:3+events_unbound	10:30:01	2
3609793	99	49	46,2	cyclictest	3599040	-21	kworker/u8:1+flush-179:0	07:10:18	2
3609793	99	49	45,3	cyclictest	3724489	-21	kworker/u8:3+flush-179:0	09:15:33	2
3609793	99	49	45,3	cyclictest	3724489	-21	kworker/u8:3+flush-179:0	09:08:01	2
3609793	99	49	45,3	cyclictest	3630833	-21	kworker/u8:2+flush-179:0	07:30:19	2
3609793	99	49	45,3	cyclictest	3599040	-21	kworker/u8:1+flush-179:0	07:20:31	2
3609793	99	49	36,12	cyclictest	3703324	-21	kworker/2:1+pm	08:36:14	2
3609791	99	49	44,4	cyclictest	3724489	-21	kworker/u8:3+flush-179:0	09:45:30	1
3609791	99	49	42,5	cyclictest	3766335	-21	/usr/share/muni	09:40:14	1
3609793	99	48	45,2	cyclictest	3848239	-21	kworker/u8:1+flush-179:0	11:00:17	2
3609793	99	48	44,3	cyclictest	3724489	-21	kworker/u8:3+flush-179:0	09:35:31	2
3609793	99	48	43,3	cyclictest	3831714	-21	kworker/u8:3+flush-179:0	10:55:15	2
3609793	99	48	42,4	cyclictest	3646463	-21	munin-update	07:45:35	2
3609793	99	48	40,3	cyclictest	3833924	-21	kworker/u8:5+events_unbound	12:00:23	2
3609793	99	48	40,3	cyclictest	3833924	-21	kworker/u8:5+events_unbound	12:00:23	2
3609793	99	48	35,2	cyclictest	3697935	-21	kworker/2:2+pm	08:51:55	2
3609791	99	48	45,2	cyclictest	3651421	-21	kworker/u8:2+flush-179:0	08:15:00	1
3609791	99	48	42,4	cyclictest	3719244	-21	latency_hist	08:55:01	1
3609791	99	48	42,4	cyclictest	3599040	-21	kworker/u8:1+flush-179:0	07:20:25	1
3609793	99	47	43,3	cyclictest	3879702	-21	kworker/u8:0+flush-179:0	11:50:15	2
3609791	99	47	41,4	cyclictest	3884183	-31	munin-html	11:30:36	1
3609790	99	47	42,4	cyclictest	3942601	-21	kworker/u8:0+flush-179:0	12:34:26	0
3609794	99	46	34,10	cyclictest	3874053	-31	munin-html	11:20:37	3
3609793	99	46	42,3	cyclictest	3677366	-21	kworker/u8:0+flush-179:0	08:15:00	2
3609793	99	46	42,3	cyclictest	3614472	-21	kworker/u8:3+flush-179:0	07:39:28	2
3609793	99	46	42,3	cyclictest	3599040	-21	kworker/u8:1+flush-179:0	08:30:22	2
3609791	99	46	43,2	cyclictest	3879702	-21	kworker/u8:0+flush-179:0	11:50:17	1
3609791	99	46	41,4	cyclictest	3719254	-21	munin-update	08:55:05	1
3609791	99	46	39,5	cyclictest	3916276	-21	/usr/share/muni	12:05:02	1
3609791	99	46	38,6	cyclictest	3667143	-21	apt-key	08:05:01	1
3609790	99	46	38,5	cyclictest	3125724	-21	ThreadPoolForeg	10:25:40	0
3609790	99	46	34,11	cyclictest	3879702	-21	kworker/u8:0+flush-179:0	11:39:18	0
3609794	99	45	36,8	cyclictest	3609683	-21	kworker/3:0+pm	07:22:58	3
3609793	99	45	41,3	cyclictest	3889926	-21	kworker/u8:3+events_unbound	12:06:55	2
3609793	99	45	41,3	cyclictest	3775599	-21	kworker/u8:0+events_unbound	09:46:31	2
3609793	99	45	30,14	cyclictest	3614472	-21	kworker/u8:3+flush-179:0	07:41:20	2
3609793	99	45	30,14	cyclictest	3614472	-21	kworker/u8:3+flush-179:0	07:41:20	2
3609791	99	45	39,4	cyclictest	3831714	-21	kworker/u8:3+flush-179:0	11:20:34	1
3609790	99	45	34,10	cyclictest	3838800	-21	kworker/0:2+pm	10:58:19	0
3609794	99	44	32,11	cyclictest	3523218	-1	kworker/3:1H+kblockd	07:51:03	3
3609791	99	44	41,2	cyclictest	3602666	-21	kworker/u8:4+flush-179:0	07:16:54	1
3609790	99	44	37,5	cyclictest	3822835	-21	cp	10:35:01	0
3609790	99	44	34,8	cyclictest	3391386	-1	kworker/0:0H+kblockd	08:30:17	0
3609790	99	44	33,10	cyclictest	3391386	-1	kworker/0:0H+kblockd	12:00:30	0
3609790	99	44	33,10	cyclictest	3391386	-1	kworker/0:0H+kblockd	12:00:30	0
3609794	99	43	39,3	cyclictest	3833924	-21	kworker/u8:5+flush-179:0	11:28:44	3
3609794	99	43	39,3	cyclictest	3703687	-21	latency_hist	08:40:01	3
3609794	99	43	39,3	cyclictest	3125627	-21	ChromeDriver	00:55:19	3
3609794	99	43	38,4	cyclictest	3125451	-21	mqtt2munin	10:45:31	3
3609794	99	43	37,4	cyclictest	3936931	-21	/usr/share/muni	12:25:10	3
3609794	99	43	36,6	cyclictest	3677366	-21	kworker/u8:0+flush-179:0	08:11:06	3
3609793	99	43	39,3	cyclictest	3889926	-21	kworker/u8:3+flush-179:0	12:16:58	2
3609793	99	43	39,3	cyclictest	3833924	-21	kworker/u8:5+events_unbound	11:30:12	2
3609793	99	43	39,3	cyclictest	3831714	-21	kworker/u8:3+flush-179:0	11:05:18	2
3609793	99	43	37,5	cyclictest	3760346	-21	kworker/u8:1+flush-179:0	09:50:14	2
3609791	99	43	29,13	cyclictest	3125624	-21	chromedriver	10:15:26	1
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17