OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Thu Dec 11, 2025 00:43:32)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
3134249	99	70	64,5	cyclictest	3632171	-21	latency_hist	07:27:17	7
3134249	99	63	58,4	cyclictest	3501056	-21	latency_hist	06:22:17	7
3133804	2	61	50,7	sleep4	0	-21	swapper/4	03:20:46	4
3134249	99	54	49,4	cyclictest	3682551	-21	latency_hist	07:52:17	7
3133653	2	54	19,30	sleep7	0	-21	swapper/7	03:18:41	7
3134249	99	50	47,2	cyclictest	3792427	-21	ntpq	08:42:42	7
3134236	99	48	45,2	cyclictest	3134463	-21	kworker/u16:3+events_unbound	03:42:44	4
3134243	99	46	44,1	cyclictest	3134463	-21	kworker/u16:3+events_unbound	04:07:33	5
3134236	99	46	43,2	cyclictest	3623314	-21	kworker/u16:0+events_unbound	08:37:32	4
3133736	2	46	36,7	sleep5	0	-21	swapper/5	03:19:51	5
3133844	2	45	35,7	sleep6	0	-21	swapper/6	03:21:20	6
3134249	99	44	39,4	cyclictest	3410304	-21	latency_hist	05:37:17	7
3134236	99	43	41,2	cyclictest	3309248	-21	kworker/u16:1+flush-8:0	05:07:46	4
3134236	99	43	40,3	cyclictest	3208487	-21	kworker/u16:1+flush-8:0	04:12:49	4
3664380	2	42	0,1	sleep7	3664378	-21	cut	07:42:34	7
3134245	99	42	39,2	cyclictest	3732378	-21	kworker/u16:5+flush-8:0	08:37:17	6
3134236	99	42	39,2	cyclictest	3561740	-21	kworker/u16:1+flush-8:0	07:12:43	4
3134249	99	40	39,0	cyclictest	0	-21	swapper/7	04:22:33	7
3134249	99	40	39,0	cyclictest	0	-21	swapper/7	04:20:40	7
3134236	99	40	37,2	cyclictest	3460463	-21	kworker/u16:4+flush-8:0	06:42:33	4
3134236	99	39	37,2	cyclictest	3623314	-21	kworker/u16:0+flush-8:0	07:42:41	4
3134236	99	39	37,2	cyclictest	3623314	-21	kworker/u16:0+events_unbound	07:52:45	4
3134236	99	39	37,2	cyclictest	3349474	-21	kworker/u16:0+flush-8:0	05:27:49	4
3134249	99	38	33,3	cyclictest	3389109	-21	ntpq	05:22:43	7
3134249	99	38	33,3	cyclictest	3389109	-21	ntpq	05:22:43	7
3134249	99	38	32,5	cyclictest	589235	-21	systemd-journal	05:02:18	7
3134249	99	38	32,5	cyclictest	3601955	-21	latency_hist	07:12:17	7
3134249	99	37	35,1	cyclictest	0	-21	swapper/7	04:02:44	7
3134236	99	37	35,2	cyclictest	3732378	-21	kworker/u16:5+events_unbound	08:36:14	4
3134236	99	37	35,2	cyclictest	3379773	-21	kworker/u16:1+events_unbound	05:35:42	4
3134249	99	36	33,2	cyclictest	3379051	-21	ntp_offset	05:17:43	7
3134249	99	36	33,2	cyclictest	3379051	-21	ntp_offset	05:17:43	7
3134249	99	36	30,5	cyclictest	3309441	-21	latency_hist	04:47:18	7
3134236	99	35	33,2	cyclictest	3440278	-21	kworker/u16:3+flush-8:0	06:02:34	4
3134236	99	35	33,2	cyclictest	3349474	-21	kworker/u16:0+events_unbound	05:45:45	4
3134236	99	35	33,2	cyclictest	3208716	-21	kworker/u16:2+flush-8:0	04:02:45	4
3134236	99	35	33,2	cyclictest	3134463	-21	kworker/u16:3+flush-8:0	04:00:52	4
3134236	99	35	33,2	cyclictest	3134463	-21	kworker/u16:3+events_unbound	03:51:28	4
3134249	99	34	33,0	cyclictest	0	-21	swapper/7	05:14:43	7
3134249	99	34	29,4	cyclictest	3531310	-21	latency_hist	06:37:18	7
3134236	99	34	32,2	cyclictest	3481284	-21	kworker/u16:1+events_unbound	06:35:51	4
3134236	99	34	32,2	cyclictest	3349474	-21	kworker/u16:0+events_unbound	06:08:28	4
3134236	99	34	32,2	cyclictest	3349474	-21	kworker/u16:0+events_unbound	06:08:28	4
3134249	99	33	32,0	cyclictest	0	-21	swapper/7	03:53:25	7
3134249	99	33	32,0	cyclictest	0	-21	swapper/7	03:53:25	7
3134249	99	33	28,4	cyclictest	3480892	-21	latency_hist	06:12:17	7
3134249	99	33	28,4	cyclictest	3480892	-21	latency_hist	06:12:17	7
3134236	99	33	31,2	cyclictest	3623314	-21	kworker/u16:0+events_unbound	07:27:21	4
3134236	99	33	31,2	cyclictest	3541231	-21	kworker/u16:0+events_unbound	06:49:44	4
3134236	99	33	31,2	cyclictest	3168170	-21	kworker/u16:1+events_unbound	03:40:53	4
3134249	99	32	31,0	cyclictest	0	-21	swapper/7	04:14:08	7
3134249	99	32	31,0	cyclictest	0	-21	swapper/7	03:50:39	7
3134249	99	32	0,30	cyclictest	3420405	-21	latency_hist	05:42:18	7
3134236	99	32	30,2	cyclictest	3581803	-21	kworker/u16:5+flush-8:0	07:05:57	4
3134236	99	32	30,2	cyclictest	3208716	-21	kworker/u16:2+events_unbound	04:28:28	4
3134249	99	31	30,0	cyclictest	0	-21	swapper/7	07:19:41	7
3134249	99	31	30,0	cyclictest	0	-21	swapper/7	04:48:36	7
3134249	99	31	29,1	cyclictest	3632568	-21	/usr/sbin/munin	07:27:32	7
3134236	99	31	29,2	cyclictest	3623314	-21	kworker/u16:0+events_unbound	08:28:21	4
3134236	99	31	29,2	cyclictest	3561740	-21	kworker/u16:1+flush-8:0	07:57:41	4
3134236	99	31	29,2	cyclictest	3500379	-21	kworker/u16:5+events_unbound	06:22:36	4
3134236	99	31	29,2	cyclictest	3349474	-21	kworker/u16:0+flush-8:0	06:17:45	4
3134236	99	31	29,2	cyclictest	3134463	-21	kworker/u16:3+events_unbound	04:18:57	4
3134236	99	31	28,2	cyclictest	3561740	-21	kworker/u16:1+flush-8:0	06:52:33	4
3134236	99	31	28,2	cyclictest	3561740	-21	kworker/u16:1+flush-8:0	06:52:33	4
3134236	99	31	28,2	cyclictest	3561740	-21	kworker/u16:1+events_unbound	08:22:26	4
3134236	99	31	28,2	cyclictest	3561740	-21	kworker/u16:1+events_unbound	08:22:26	4
3134236	99	31	28,2	cyclictest	3077662	-21	kworker/u16:4+events_unbound	03:32:21	4
3134249	99	30	30,0	cyclictest	0	-21	swapper/7	08:14:17	7
3134243	99	30	27,2	cyclictest	3561740	-21	kworker/u16:1+flush-8:0	07:07:48	5
3134243	99	30	27,2	cyclictest	3561740	-21	kworker/u16:1+flush-8:0	07:02:17	5
3134236	99	30	28,2	cyclictest	3623314	-21	kworker/u16:0+events_unbound	08:48:52	4
3134236	99	30	28,2	cyclictest	3349474	-21	kworker/u16:0+events_unbound	05:54:00	4
3134236	99	30	28,2	cyclictest	3349474	-21	kworker/u16:0+events_unbound	05:22:11	4
3134236	99	30	28,2	cyclictest	3349474	-21	kworker/u16:0+events_unbound	05:22:11	4
3134236	99	30	27,2	cyclictest	3561740	-21	kworker/u16:1+flush-8:0	06:57:35	4
3134236	99	30	27,2	cyclictest	3349474	-21	kworker/u16:0+events_unbound	06:12:26	4
3329092	2	29	0,1	chrt	3329093	-21	timerwakeupswit	04:52:47	7
3134249	99	29	28,0	cyclictest	0	-21	swapper/7	07:57:44	7
3134249	99	29	28,0	cyclictest	0	-21	swapper/7	06:04:42	7
3134249	99	29	28,0	cyclictest	0	-21	swapper/7	04:34:45	7
3134245	99	29	26,2	cyclictest	3329333	-21	kworker/u16:3+flush-8:0	04:57:45	6
3134236	99	29	27,2	cyclictest	3732378	-21	kworker/u16:5+events_unbound	08:15:29	4
3134236	99	29	27,2	cyclictest	3631787	-21	kworker/u16:4+events_unbound	07:26:16	4
3134236	99	29	27,2	cyclictest	3623314	-21	kworker/u16:0+flush-8:0	07:47:33	4
3134236	99	29	27,2	cyclictest	3561740	-21	kworker/u16:1+events_unbound	07:38:27	4
3134236	99	29	27,2	cyclictest	3561740	-21	kworker/u16:1+events_unbound	07:38:27	4
3134236	99	29	27,2	cyclictest	3500379	-21	kworker/u16:5+events_unbound	06:31:53	4
3134236	99	29	27,2	cyclictest	3338705	-21	kworker/u16:4+flush-8:0	05:02:33	4
3134236	99	29	27,2	cyclictest	3269533	-21	kworker/u16:4+events_unbound	04:43:21	4
3134236	99	29	26,2	cyclictest	3682363	-21	kworker/u16:2+flush-8:0	08:07:28	4
3134236	99	29	26,2	cyclictest	3600920	-21	kworker/u16:2+flush-8:0	07:17:33	4
3134249	99	28	27,0	cyclictest	0	-21	swapper/7	03:46:09	7
3134236	99	28	26,2	cyclictest	3623314	-21	kworker/u16:0+events_unbound	07:35:49	4
3134236	99	28	26,2	cyclictest	3600920	-21	kworker/u16:2+events_unbound	07:09:16	4
3134236	99	28	26,2	cyclictest	3561740	-21	kworker/u16:1+events_unbound	08:18:47	4
3134236	99	28	26,2	cyclictest	3460463	-21	kworker/u16:4+flush-8:0	06:37:55	4
3134236	99	28	26,2	cyclictest	3360184	-21	kworker/u16:2+events_unbound	05:15:36	4
3134236	99	28	26,2	cyclictest	3338705	-21	kworker/u16:4+flush-8:0	04:57:57	4
3134236	99	28	26,2	cyclictest	3309248	-21	kworker/u16:1+events_unbound	04:54:33	4
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17