OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Thu Jan 01, 2026 12:43:32)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
3341234	2	55	44,8	sleep4	0	-21	swapper/4	07:08:08	4
3341029	2	53	19,30	sleep7	0	-21	swapper/7	07:05:15	7
3341180	2	49	38,7	sleep6	0	-21	swapper/6	07:07:22	6
3332021	2	49	37,8	sleep5	0	-21	swapper/5	07:03:44	5
3341561	99	47	41,5	cyclictest	3801006	-21	latency_hist	11:03:39	7
3341548	99	44	42,2	cyclictest	3312678	-21	kworker/u16:3+flush-8:0	07:33:40	4
3341548	99	42	39,2	cyclictest	3539507	-21	kworker/u16:1+events_unbound	09:03:54	4
3341548	99	42	39,2	cyclictest	3312678	-21	kworker/u16:3+flush-8:0	09:38:55	4
3341561	99	39	38,0	cyclictest	0	-21	swapper/7	12:18:55	7
3341548	99	38	36,2	cyclictest	3287418	-21	kworker/u16:1+events_unbound	07:13:54	4
3341548	99	38	35,2	cyclictest	3481218	-21	kworker/u16:1+events_unbound	08:18:53	4
3341561	99	37	36,0	cyclictest	0	-21	swapper/7	12:26:22	7
3341548	99	37	35,2	cyclictest	2634675	-21	kworker/u16:0+events_unbound	07:43:54	4
3341548	99	37	34,2	cyclictest	3781296	-21	kworker/u16:2+events_unbound	11:14:07	4
3341548	99	35	33,2	cyclictest	3781296	-21	kworker/u16:2+events_unbound	10:59:10	4
3341548	99	35	33,2	cyclictest	3704100	-21	kworker/u16:0+events_unbound	11:38:58	4
3341548	99	35	32,2	cyclictest	3791688	-21	kworker/u16:1+flush-8:0	11:04:06	4
3341548	99	35	32,2	cyclictest	3665631	-21	kworker/u16:1+flush-8:0	09:59:11	4
3341548	99	35	32,2	cyclictest	3539507	-21	kworker/u16:1+flush-8:0	09:23:55	4
3341561	99	34	33,0	cyclictest	0	-21	swapper/7	09:06:24	7
3341548	99	34	32,2	cyclictest	3685958	-21	kworker/u16:3+events_unbound	10:09:10	4
3341561	99	33	32,0	cyclictest	0	-21	swapper/7	11:29:08	7
3341548	99	33	31,2	cyclictest	3675147	-21	kworker/u16:4+events_unbound	10:54:11	4
3341548	99	33	31,2	cyclictest	3539507	-21	kworker/u16:1+events_unbound	09:29:16	4
3688581	2	32	0,0	chrt	3688583	-21	kthreadcore	10:03:58	7
3341561	99	31	30,0	cyclictest	0	-21	swapper/7	08:27:51	7
3341548	99	31	29,2	cyclictest	3704100	-21	kworker/u16:0+events_unbound	10:44:11	4
3341548	99	31	28,2	cyclictest	3927373	-21	kworker/u16:1+events_unbound	12:28:54	4
3341548	99	31	28,2	cyclictest	3312678	-21	kworker/u16:3+flush-8:0	07:33:59	4
3341561	99	30	29,0	cyclictest	0	-21	swapper/7	12:12:31	7
3341561	99	30	29,0	cyclictest	0	-21	swapper/7	11:44:23	7
3341561	99	30	29,0	cyclictest	0	-21	swapper/7	11:36:22	7
3341548	99	30	28,2	cyclictest	3539507	-21	kworker/u16:1+flush-8:0	08:59:23	4
3341548	99	30	27,2	cyclictest	3781296	-21	kworker/u16:2+events_unbound	11:28:51	4
3341548	99	30	27,2	cyclictest	3685958	-21	kworker/u16:3+flush-8:0	10:33:50	4
3341548	99	30	27,2	cyclictest	3539507	-21	kworker/u16:1+events_unbound	08:53:54	4
3341561	99	29	28,0	cyclictest	0	-21	swapper/7	11:08:23	7
3341561	99	29	28,0	cyclictest	0	-21	swapper/7	10:18:27	7
3341561	99	29	28,0	cyclictest	0	-21	swapper/7	09:46:30	7
3341557	99	29	27,2	cyclictest	3916018	-21	kworker/u16:5+events_unbound	12:15:41	6
3341557	99	29	27,2	cyclictest	3727642	-21	kworker/u16:1+flush-8:0	10:48:55	6
3341557	99	29	27,2	cyclictest	3665631	-21	kworker/u16:1+events_unbound	09:59:28	6
3341557	99	29	27,2	cyclictest	3287418	-21	kworker/u16:1+flush-8:0	07:19:35	6
3341557	99	29	27,2	cyclictest	2634675	-21	kworker/u16:0+events_unbound	08:02:03	6
3341548	99	29	27,2	cyclictest	3312678	-21	kworker/u16:3+events_unbound	09:08:59	4
3341561	99	28	27,0	cyclictest	0	-21	swapper/7	11:52:22	7
3341561	99	28	27,0	cyclictest	0	-21	swapper/7	10:01:06	7
3341561	99	28	27,0	cyclictest	0	-21	swapper/7	09:37:23	7
3341561	99	28	25,2	cyclictest	3363731	-21	ntpq	07:14:07	7
3341557	99	28	26,2	cyclictest	3312678	-21	kworker/u16:3+events_unbound	07:56:36	6
3341548	99	28	26,2	cyclictest	3453127	-21	kworker/u16:4+flush-8:0	09:14:10	4
3341548	99	28	26,2	cyclictest	3312678	-21	kworker/u16:3+flush-8:0	08:24:11	4
3341548	99	28	25,2	cyclictest	3927373	-21	kworker/u16:1+events_unbound	12:23:53	4
3341561	99	27	26,0	cyclictest	0	-21	swapper/7	10:32:51	7
3341561	99	27	26,0	cyclictest	0	-21	swapper/7	10:32:51	7
3341561	99	27	26,0	cyclictest	0	-21	swapper/7	09:00:27	7
3341557	99	27	25,2	cyclictest	3916018	-21	kworker/u16:5+events_unbound	12:10:34	6
3341557	99	27	25,2	cyclictest	3908683	-21	kworker/u16:4+events_unbound	12:08:27	6
3341557	99	27	25,2	cyclictest	3704100	-21	kworker/u16:0+events_unbound	10:47:11	6
3341557	99	27	25,2	cyclictest	3675147	-21	kworker/u16:4+flush-8:0	10:23:55	6
3341557	99	27	25,2	cyclictest	3453127	-21	kworker/u16:4+flush-8:0	09:07:42	6
3341557	99	27	25,2	cyclictest	3312678	-21	kworker/u16:3+events_unbound	08:36:58	6
3341557	99	27	25,2	cyclictest	3287418	-21	kworker/u16:1+events_unbound	07:11:16	6
3341548	99	27	25,2	cyclictest	3312678	-21	kworker/u16:3+events_unbound	09:33:58	4
3341561	99	26	25,0	cyclictest	0	-21	swapper/7	12:36:43	7
3341561	99	26	25,0	cyclictest	0	-21	swapper/7	11:57:54	7
3341561	99	26	25,0	cyclictest	0	-21	swapper/7	11:27:07	7
3341561	99	26	25,0	cyclictest	0	-21	swapper/7	10:35:27	7
3341557	99	26	24,2	cyclictest	3927373	-21	kworker/u16:1+events_unbound	12:29:18	6
3341557	99	26	24,2	cyclictest	3781296	-21	kworker/u16:2+events_unbound	11:54:21	6
3341557	99	26	24,2	cyclictest	3704100	-21	kworker/u16:0+events_unbound	10:28:55	6
3341557	99	26	24,2	cyclictest	3704100	-21	kworker/u16:0+events_unbound	10:28:55	6
3341557	99	26	24,2	cyclictest	3675147	-21	kworker/u16:4+events_unbound	10:37:58	6
3341557	99	26	24,2	cyclictest	3539507	-21	kworker/u16:1+events_unbound	09:24:00	6
3341557	99	26	24,2	cyclictest	3453127	-21	kworker/u16:4+events_unbound	08:44:23	6
3341557	99	26	24,2	cyclictest	3393714	-21	kworker/u16:2+events_unbound	07:41:24	6
3341557	99	26	24,2	cyclictest	3312678	-21	kworker/u16:3+events_unbound	07:30:39	6
3341557	99	26	24,2	cyclictest	3287418	-21	kworker/u16:1+events_unbound	07:18:08	6
3341557	99	26	24,2	cyclictest	2634675	-21	kworker/u16:0+events_unbound	08:05:42	6
3341557	99	26	24,2	cyclictest	2634675	-21	kworker/u16:0+events_unbound	07:52:03	6
3341548	99	26	24,2	cyclictest	3927373	-21	kworker/u16:1+events_unbound	12:18:52	4
3341548	99	26	24,2	cyclictest	3453127	-21	kworker/u16:4+events_unbound	09:19:05	4
3341548	99	26	24,2	cyclictest	3312678	-21	kworker/u16:3+events_unbound	07:54:08	4
3341561	99	25	24,0	cyclictest	0	-21	swapper/7	11:15:39	7
3341561	99	25	24,0	cyclictest	0	-21	swapper/7	10:25:02	7
3341561	99	25	24,0	cyclictest	0	-21	swapper/7	09:39:22	7
3341561	99	25	24,0	cyclictest	0	-21	swapper/7	09:31:18	7
3341561	99	25	24,0	cyclictest	0	-21	swapper/7	07:36:23	7
3341561	99	25	24,0	cyclictest	0	-21	swapper/7	07:31:34	7
3341561	99	25	0,23	cyclictest	3594610	-21	grep	09:14:00	7
3341557	99	25	23,2	cyclictest	3916018	-21	kworker/u16:5+events_unbound	12:00:59	6
3341557	99	25	23,2	cyclictest	3781296	-21	kworker/u16:2+events_unbound	11:13:18	6
3341557	99	25	23,2	cyclictest	3675147	-21	kworker/u16:4+flush-8:0	10:53:56	6
3341557	99	25	23,2	cyclictest	3617087	-21	kworker/u16:2+events_unbound	09:53:11	6
3341557	99	25	23,2	cyclictest	3539507	-21	kworker/u16:1+events_unbound	09:11:43	6
3341557	99	25	23,2	cyclictest	3453127	-21	kworker/u16:4+events_unbound	09:23:19	6
3341557	99	25	23,2	cyclictest	3393714	-21	kworker/u16:2+events_unbound	07:36:31	6
3341548	99	25	23,2	cyclictest	3908683	-21	kworker/u16:4+flush-8:0	11:59:10	4
3341548	99	25	23,2	cyclictest	3704100	-21	kworker/u16:0+events_unbound	11:54:03	4
3341548	99	25	23,2	cyclictest	3617087	-21	kworker/u16:2+events_unbound	09:54:52	4
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17