OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #b, slot #1

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance

Characteristics of the 100 highest latencies:

System rackbslot1.osadl.org (updated Mon Dec 22, 2025 12:44:33)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
3212047	2	203	175,18	sleep3	0	-21	swapper/3	07:07:36	3
3211853	2	189	161,18	sleep2	0	-21	swapper/2	07:05:42	2
3212204	2	177	153,14	sleep1	0	-21	swapper/1	07:09:35	1
3212099	2	173	150,14	sleep0	0	-21	swapper/0	07:08:14	0
3212436	99	71	67,3	cyclictest	3492083	-21	kworker/u8:3+events_unbound	11:50:11	3
3212427	99	67	52,14	cyclictest	3249778	-13	chrome	07:45:02	0
3212427	99	62	58,3	cyclictest	3150626	-21	kworker/u8:1+flush-179:0	07:31:49	0
3212427	99	61	58,2	cyclictest	3227821	-21	kworker/u8:3+flush-179:0	09:05:02	0
3212428	99	59	56,2	cyclictest	3227821	-21	kworker/u8:3+events_unbound	08:25:33	1
3212428	99	59	53,4	cyclictest	3497727	-21	chrome	11:45:02	1
3212427	99	59	55,2	cyclictest	3150626	-21	kworker/u8:1+flush-179:0	07:10:07	0
3212436	99	58	54,3	cyclictest	3249294	-21	kworker/u8:2+events_unbound	07:55:31	3
3212436	99	58	53,2	cyclictest	3414088	-21	kworker/u8:4+flush-179:0	11:05:29	3
3212436	99	58	53,2	cyclictest	3414088	-21	kworker/u8:4+flush-179:0	11:05:29	3
3212436	99	58	48,3	cyclictest	3486831	-21	kworker/u8:0+flush-179:0	12:10:29	3
3212431	99	58	52,4	cyclictest	3347259	-21	kworker/u8:2+flush-179:0	10:05:14	2
3212436	99	57	54,2	cyclictest	3227821	-21	kworker/u8:3+flush-179:0	09:17:02	3
3212436	99	57	54,2	cyclictest	3227821	-21	kworker/u8:3+events_unbound	07:40:29	3
3212436	99	57	53,3	cyclictest	3227821	-21	kworker/u8:3+flush-179:0	08:50:10	3
3212436	99	56	53,2	cyclictest	3347259	-21	kworker/u8:2+events_unbound	09:45:17	3
3212436	99	56	52,3	cyclictest	3360273	-21	kworker/u8:1+events_unbound	10:08:18	3
3212436	99	56	52,3	cyclictest	3347259	-21	kworker/u8:2+events_unbound	10:20:01	3
3212436	99	56	43,10	cyclictest	3227821	-21	kworker/u8:3+flush-179:0	07:30:26	3
3212428	99	55	49,4	cyclictest	3326885	-21	/usr/share/muni	09:00:20	1
3212427	99	55	48,6	cyclictest	3476557	-21	latency_hist	11:25:01	0
3212431	99	54	50,3	cyclictest	3228675	-21	munin-update	07:25:34	2
3212431	99	54	39,14	cyclictest	3249821	-21	chrome	07:45:02	2
3212436	99	53	50,2	cyclictest	3472106	-21	kworker/u8:3+flush-179:0	11:30:29	3
3212431	99	53	46,5	cyclictest	3316671	-21	/usr/share/muni	08:50:14	2
3212427	99	53	50,2	cyclictest	3347259	-21	kworker/u8:2+flush-179:0	09:35:15	0
3212427	99	53	49,3	cyclictest	3360273	-21	kworker/u8:1+flush-179:0	09:40:10	0
3212427	99	53	48,3	cyclictest	3434683	-21	kworker/u8:2+flush-179:0	11:10:34	0
3212427	99	53	3,50	cyclictest	3434683	-21	kworker/u8:2+efi_rts_wq	11:29:40	0
3212436	99	52	49,2	cyclictest	3227821	-21	kworker/u8:3+flush-179:0	09:30:23	3
3212428	99	52	49,3	cyclictest	3383301	-21	kworker/u8:3+events_unbound	09:55:33	1
3212427	99	52	47,4	cyclictest	3403254	-31	munin-html	10:10:36	0
3212436	99	51	47,3	cyclictest	3486932	-21	kworker/u8:1+events_unbound	11:44:44	3
3212436	99	51	47,3	cyclictest	3275000	-21	kworker/u8:4+flush-179:0	08:17:03	3
3212436	99	51	45,2	cyclictest	3492083	-21	kworker/u8:3+events_unbound	11:45:34	3
3212431	99	51	46,4	cyclictest	3434996	-21	latency_hist	10:45:01	2
3212431	99	51	45,4	cyclictest	3311418	-21	/usr/share/muni	08:45:21	2
3212427	99	51	47,3	cyclictest	3414088	-21	kworker/u8:4+events_unbound	11:00:35	0
3212427	99	51	45,4	cyclictest	3383461	-21	latency_hist	09:55:01	0
3212436	99	50	47,2	cyclictest	3321251	-21	kworker/u8:0+flush-179:0	09:06:56	3
3212436	99	50	47,2	cyclictest	3275000	-21	kworker/u8:4+flush-179:0	08:20:18	3
3212431	99	50	46,3	cyclictest	3486831	-21	kworker/u8:0+flush-179:0	11:50:37	2
3212431	99	50	46,2	cyclictest	3274513	-31	munin-html	08:05:35	2
3212431	99	50	44,5	cyclictest	3517598	-21	kworker/u8:4+flush-179:0	12:20:14	2
3212427	99	50	43,5	cyclictest	3357985	-21	/usr/share/muni	09:30:10	0
3212436	99	49	46,2	cyclictest	3486831	-21	kworker/u8:0+events_unbound	11:35:13	3
3212436	99	49	44,2	cyclictest	3227821	-21	kworker/u8:3+events_unbound	08:49:12	3
3212436	99	49	41,7	cyclictest	3227821	-21	kworker/u8:3+flush-179:0	09:20:14	3
3212428	99	49	45,3	cyclictest	3513232	-21	/usr/share/muni	12:00:14	1
3212428	99	49	43,4	cyclictest	3451200	-21	/usr/share/muni	11:00:11	1
3212427	99	49	46,2	cyclictest	3201719	-21	kworker/u8:0+flush-179:0	07:20:30	0
3212427	99	49	45,3	cyclictest	3517598	-21	kworker/u8:4+events_unbound	12:10:10	0
3212427	99	49	45,3	cyclictest	3492083	-21	kworker/u8:3+flush-179:0	11:45:12	0
3212427	99	49	45,3	cyclictest	3383301	-21	kworker/u8:3+flush-179:0	10:05:24	0
3212427	99	49	44,4	cyclictest	3227821	-21	kworker/u8:3+flush-179:0	08:20:31	0
3212436	99	48	46,2	cyclictest	3414088	-21	kworker/u8:4+events_unbound	11:15:33	3
3212436	99	48	45,2	cyclictest	3414088	-21	kworker/u8:4+events_unbound	10:25:14	3
3212436	99	48	45,2	cyclictest	3414088	-21	kworker/u8:4+events_unbound	10:25:14	3
3212436	99	48	45,2	cyclictest	3360273	-21	kworker/u8:1+flush-179:0	10:10:33	3
3212427	99	48	42,4	cyclictest	3357509	-21	kworker/u8:0+flush-179:0	09:55:10	0
3212436	99	47	42,3	cyclictest	3360273	-21	kworker/u8:1+events_unbound	10:17:03	3
3212431	99	47	43,3	cyclictest	3227821	-21	kworker/u8:3+flush-179:0	09:10:05	2
3212431	99	47	41,5	cyclictest	247560	-21	ThreadPoolForeg	10:20:39	2
3212431	99	47	40,6	cyclictest	247516	-21	Chrome_ChildIOT	10:25:19	2
3212431	99	47	40,6	cyclictest	247516	-21	Chrome_ChildIOT	10:25:19	2
3212431	99	47	40,5	cyclictest	76405	-21	systemd-journal	09:05:00	2
3212428	99	47	45,2	cyclictest	3243973	-21	kworker/u8:0+flush-179:0	07:39:16	1
3212427	99	47	41,5	cyclictest	3227821	-21	kworker/u8:3+flush-179:0	08:30:10	0
3212436	99	46	43,2	cyclictest	3472106	-21	kworker/u8:3+flush-179:0	11:25:24	3
3212436	99	46	43,2	cyclictest	3383301	-21	kworker/u8:3+events_unbound	10:04:00	3
3212436	99	46	43,2	cyclictest	3383301	-21	kworker/u8:3+events_unbound	10:04:00	3
3212436	99	46	43,2	cyclictest	3360273	-21	kworker/u8:1+flush-179:0	09:50:09	3
3212436	99	46	43,2	cyclictest	3360273	-21	kworker/u8:1+events_unbound	09:40:31	3
3212436	99	46	43,2	cyclictest	3227821	-21	kworker/u8:3+flush-179:0	08:25:21	3
3212436	99	46	41,4	cyclictest	3230587	-21	ntpq	07:25:15	3
3212436	99	46	39,2	cyclictest	3347259	-21	kworker/u8:2+flush-179:0	09:25:22	3
3212436	99	46	36,9	cyclictest	3310917	-21	kworker/u8:1+flush-179:0	09:10:14	3
3212431	99	46	39,5	cyclictest	3388832	-21	/usr/share/muni	10:00:15	2
3212431	99	46	39,5	cyclictest	3388832	-21	/usr/share/muni	10:00:15	2
3212428	99	46	44,2	cyclictest	3531026	-21	kworker/u8:2+events_unbound	12:33:11	1
3212428	99	46	42,3	cyclictest	3227821	-21	kworker/u8:3+flush-179:0	08:20:29	1
3212428	99	46	41,4	cyclictest	3461388	-21	/usr/share/muni	11:10:07	1
3212427	99	46	43,2	cyclictest	3150626	-21	kworker/u8:1+flush-179:0	07:40:10	0
3212427	99	46	42,3	cyclictest	3445544	-21	kworker/u8:0+flush-179:0	10:55:35	0
3212427	99	46	42,3	cyclictest	3285048	-21	kworker/u8:2+flush-179:0	08:35:30	0
3212427	99	46	41,4	cyclictest	3520653	-21	kworker/u8:1+flush-179:0	12:33:43	0
3212427	99	46	40,4	cyclictest	3227821	-21	kworker/u8:3+flush-179:0	08:00:17	0
3212436	99	45	40,3	cyclictest	3215340	-21	kworker/u8:4+flush-179:0	07:20:21	3
3212431	99	45	32,12	cyclictest	3522672	-21	kworker/2:2+pm	12:32:10	2
3212427	99	45	0,44	cyclictest	3125724	-21	ThreadPoolForeg	11:50:10	0
3212436	99	44	42,2	cyclictest	3227821	-21	kworker/u8:3+events_unbound	08:11:59	3
3212436	99	44	41,2	cyclictest	3517598	-21	kworker/u8:4+events_unbound	12:29:17	3
3212436	99	44	41,2	cyclictest	3486831	-21	kworker/u8:0+events_unbound	12:01:32	3
3212436	99	44	41,2	cyclictest	3445544	-21	kworker/u8:0+events_unbound	11:00:29	3
3212436	99	44	41,2	cyclictest	3347259	-21	kworker/u8:2+events_unbound	09:35:52	3
3212436	99	44	40,3	cyclictest	3517598	-21	kworker/u8:4+flush-179:0	12:15:24	3
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17