OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Wed Aug 12, 2026 12:43:33)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
81292	2	154	0,1	sleep7	81291	-21	ssh	10:38:33	7
224688	2	98	0,0	sleep5	0	-21	swapper/5	11:38:26	5
275448	2	63	0,1	sleep4	0	-21	swapper/4	11:58:29	4
275448	2	63	0,1	sleep4	0	-21	swapper/4	11:58:29	4
3807295	99	55	53,2	cyclictest	48041	-21	kworker/u16:0+events_unbound	10:33:32	5
3806935	2	55	45,7	sleep5	0	-21	swapper/5	07:07:09	5
3807295	99	50	48,2	cyclictest	4093506	-21	kworker/u16:4+events_unbound	09:33:38	5
3807295	99	49	47,2	cyclictest	4044651	-21	kworker/u16:0+events_unbound	09:38:20	5
3807303	99	48	42,5	cyclictest	3879018	-21	latency_hist	07:43:05	7
3807295	99	48	46,2	cyclictest	4183213	-21	kworker/u16:2+events_unbound	10:53:20	5
3807295	99	47	45,2	cyclictest	4044651	-21	kworker/u16:0+events_unbound	10:08:31	5
3807295	99	47	45,2	cyclictest	121295	-21	kworker/u16:4+flush-8:0	11:22:42	5
3807299	99	46	43,2	cyclictest	192040	-21	kworker/u16:1+events_unbound	11:48:33	6
3807295	99	46	43,3	cyclictest	4183213	-21	kworker/u16:2+flush-8:0	10:48:39	5
3806990	2	46	36,7	sleep4	0	-21	swapper/4	07:07:56	4
3807299	99	45	43,2	cyclictest	192040	-21	kworker/u16:1+events_unbound	12:28:37	6
3807295	99	45	43,2	cyclictest	192040	-21	kworker/u16:1+events_unbound	12:18:39	5
3807295	99	45	43,2	cyclictest	11169	-21	kworker/u16:3+flush-8:0	10:23:40	5
4161332	2	44	0,0	sleep5	0	-21	swapper/5	09:53:27	5
4161332	2	44	0,0	sleep5	0	-21	swapper/5	09:53:27	5
3807295	99	44	42,2	cyclictest	4093506	-21	kworker/u16:4+events_unbound	09:58:39	5
3807295	99	44	42,2	cyclictest	192040	-21	kworker/u16:1+events_unbound	12:33:30	5
3807295	99	43	41,2	cyclictest	192040	-21	kworker/u16:1+events_unbound	12:28:27	5
3807299	99	41	38,2	cyclictest	3859977	-21	kworker/u16:0+events_unbound	07:33:19	6
3806698	2	41	6,30	sleep7	0	-21	swapper/7	07:03:50	7
3807295	99	40	38,2	cyclictest	48041	-21	kworker/u16:0+flush-8:0	10:38:35	5
3807295	99	40	38,2	cyclictest	192040	-21	kworker/u16:1+flush-8:0	11:33:38	5
3807299	99	39	37,2	cyclictest	82147	-21	kworker/u16:1+events_unbound	11:03:34	6
3807299	99	39	37,2	cyclictest	3778349	-21	kworker/u16:3+flush-8:0	07:23:30	6
3807303	99	38	31,5	cyclictest	977	-21	in:imjournal	12:28:05	7
3807299	99	38	36,2	cyclictest	3937050	-21	kworker/u16:0+events_unbound	08:38:34	6
3807295	99	38	36,2	cyclictest	4060302	-21	kworker/u16:3+events_unbound	09:23:07	5
3807295	99	38	36,2	cyclictest	232061	-21	kworker/u16:2+events_unbound	12:13:36	5
3807295	99	38	35,2	cyclictest	3917590	-21	kworker/u16:3+events_unbound	08:38:20	5
3807299	99	37	35,2	cyclictest	4044651	-21	kworker/u16:0+events_unbound	09:28:35	6
3807299	99	37	35,2	cyclictest	3937050	-21	kworker/u16:0+flush-8:0	08:43:29	6
3807299	99	37	34,2	cyclictest	232061	-21	kworker/u16:2+flush-8:0	11:43:21	6
3807295	99	37	35,2	cyclictest	82147	-21	kworker/u16:1+flush-8:0	11:08:34	5
3807303	99	36	31,4	cyclictest	59882	-21	latency_hist	10:33:05	7
3807299	99	36	34,2	cyclictest	4183213	-21	kworker/u16:2+events_unbound	10:48:37	6
3807299	99	36	34,2	cyclictest	3937050	-21	kworker/u16:0+flush-8:0	08:18:39	6
3807299	99	36	34,2	cyclictest	3937050	-21	kworker/u16:0+events_unbound	08:33:29	6
3807299	99	36	34,2	cyclictest	192040	-21	kworker/u16:1+flush-8:0	12:23:25	6
3807295	99	36	34,2	cyclictest	4093506	-21	kworker/u16:4+events_unbound	10:03:07	5
3807295	99	36	34,2	cyclictest	3937050	-21	kworker/u16:0+events_unbound	08:43:35	5
3807295	99	36	34,2	cyclictest	3937050	-21	kworker/u16:0+events_unbound	08:28:20	5
3807295	99	36	34,2	cyclictest	192040	-21	kworker/u16:1+events_unbound	12:23:36	5
3807295	99	36	34,2	cyclictest	167265	-21	kworker/u16:5+events_unbound	11:15:55	5
3807295	99	36	34,2	cyclictest	158480	-21	kworker/u16:3+events_unbound	11:53:34	5
3807299	99	35	33,2	cyclictest	4183213	-21	kworker/u16:2+events_unbound	10:16:32	6
3807299	99	35	33,2	cyclictest	3937050	-21	kworker/u16:0+flush-8:0	08:48:34	6
3807299	99	35	33,2	cyclictest	281417	-21	kworker/u16:0+events_unbound	12:08:25	6
3807299	99	35	33,2	cyclictest	167265	-21	kworker/u16:5+events_unbound	11:16:42	6
3807295	99	35	33,2	cyclictest	4118132	-21	kworker/u16:1+flush-8:0	09:48:51	5
3807295	99	35	33,2	cyclictest	3888934	-21	kworker/u16:1+flush-8:0	08:08:35	5
3807295	99	35	33,2	cyclictest	3778349	-21	kworker/u16:3+events_unbound	07:33:23	5
3807295	99	35	33,2	cyclictest	158480	-21	kworker/u16:3+events_unbound	11:58:19	5
3807295	99	35	33,2	cyclictest	158480	-21	kworker/u16:3+events_unbound	11:58:19	5
3807293	99	35	33,2	cyclictest	48041	-21	kworker/u16:0+flush-8:0	10:43:31	4
4065146	2	34	0,0	chrt	0	-21	swapper/6	09:13:29	6
4065146	2	34	0,0	chrt	0	-21	swapper/6	09:13:29	6
3807299	99	34	32,2	cyclictest	4130412	-21	kworker/u16:2+events_unbound	09:47:23	6
3807299	99	34	32,2	cyclictest	4118132	-21	kworker/u16:1+events_unbound	10:02:28	6
3807299	99	34	32,2	cyclictest	3859977	-21	kworker/u16:0+flush-8:0	07:53:34	6
3807299	99	34	32,2	cyclictest	232061	-21	kworker/u16:2+events_unbound	12:19:47	6
3807299	99	34	31,2	cyclictest	3859977	-21	kworker/u16:0+events_unbound	07:58:22	6
3807295	99	34	32,2	cyclictest	3917590	-21	kworker/u16:3+events_unbound	08:33:19	5
3807295	99	34	32,2	cyclictest	3859977	-21	kworker/u16:0+events_unbound	07:38:36	5
3807293	99	34	32,2	cyclictest	4183213	-21	kworker/u16:2+flush-8:0	11:08:47	4
3807293	99	34	31,2	cyclictest	11169	-21	kworker/u16:3+flush-8:0	10:23:05	4
3807299	99	33	31,2	cyclictest	4118132	-21	kworker/u16:1+events_unbound	10:08:43	6
3807299	99	33	31,2	cyclictest	4044651	-21	kworker/u16:0+events_unbound	09:33:32	6
3807299	99	33	31,2	cyclictest	158479	-21	kworker/u16:0+events_unbound	11:25:31	6
3807295	99	33	31,2	cyclictest	281417	-21	kworker/u16:0+events_unbound	12:08:19	5
3807295	99	33	31,2	cyclictest	281417	-21	kworker/u16:0+events_unbound	12:07:03	5
3807299	99	32	30,2	cyclictest	4093506	-21	kworker/u16:4+flush-8:0	09:48:34	6
3807299	99	32	30,2	cyclictest	4060302	-21	kworker/u16:3+events_unbound	09:41:04	6
3807299	99	32	30,2	cyclictest	121295	-21	kworker/u16:4+events_unbound	11:21:07	6
3807295	99	32	30,2	cyclictest	82147	-21	kworker/u16:1+events_unbound	10:45:12	5
3807295	99	32	30,2	cyclictest	4183213	-21	kworker/u16:2+events_unbound	11:03:32	5
3807295	99	32	30,2	cyclictest	3937050	-21	kworker/u16:0+events_unbound	08:23:23	5
3807295	99	32	30,2	cyclictest	232061	-21	kworker/u16:2+flush-8:0	11:49:30	5
3806936	2	32	21,8	sleep6	0	-21	swapper/6	07:07:10	6
3807299	99	31	29,2	cyclictest	4183213	-21	kworker/u16:2+events_unbound	11:10:23	6
3807299	99	31	29,2	cyclictest	4183213	-21	kworker/u16:2+events_unbound	10:43:22	6
3807299	99	31	29,2	cyclictest	4118132	-21	kworker/u16:1+events_unbound	10:21:39	6
3807295	99	31	29,2	cyclictest	4183213	-21	kworker/u16:2+events_unbound	10:59:42	5
3807295	99	31	29,2	cyclictest	4183213	-21	kworker/u16:2+events_unbound	10:18:31	5
3807295	99	31	29,2	cyclictest	4183213	-21	kworker/u16:2+events_unbound	10:18:00	5
3807295	99	31	29,2	cyclictest	4093506	-21	kworker/u16:4+events_unbound	09:31:59	5
3807295	99	31	29,2	cyclictest	4044651	-21	kworker/u16:0+flush-8:0	09:43:44	5
3807295	99	31	29,2	cyclictest	158479	-21	kworker/u16:0+flush-8:0	11:26:59	5
3807295	99	31	28,2	cyclictest	4060302	-21	kworker/u16:3+flush-8:0	09:15:03	5
3807295	99	31	28,2	cyclictest	4060302	-21	kworker/u16:3+flush-8:0	09:15:03	5
3807295	99	31	28,2	cyclictest	3888934	-21	kworker/u16:1+flush-8:0	07:53:19	5
3807293	99	31	29,2	cyclictest	48041	-21	kworker/u16:0+flush-8:0	10:38:00	4
3807299	99	30	28,2	cyclictest	48041	-21	kworker/u16:0+events_unbound	10:33:56	6
3807299	99	30	28,2	cyclictest	3806030	-21	kworker/u16:2+events_unbound	09:25:04	6
3807299	99	30	28,2	cyclictest	232061	-21	kworker/u16:2+events_unbound	12:07:50	6
3807295	99	30	28,2	cyclictest	4183213	-21	kworker/u16:2+events_unbound	10:28:51	5
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17