OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Fri Jul 10, 2026 12:43:34)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
1284667	2	100	0,1	sleep5	0	-21	swapper/5	12:00:03	5
1021253	2	98	0,1	sleep7	1021254	-21	sh	10:13:38	7
928480	2	97	0,1	sleep7	928483	-21	cut	09:35:07	7
768870	2	97	0,1	sleep7	768871	-21	kthreadcore	08:20:07	7
614029	2	83	0,6	chrt	0	-21	swapper/6	07:04:57	6
622985	2	54	19,30	sleep7	0	-21	swapper/7	07:08:04	7
623012	2	46	36,7	sleep5	0	-21	swapper/5	07:08:26	5
623394	99	43	41,2	cyclictest	1302695	-21	kworker/u16:0+events_unbound	12:20:00	6
623394	99	42	39,2	cyclictest	1083121	-21	kworker/u16:2+events_unbound	10:40:00	6
623394	99	40	37,2	cyclictest	785625	-21	kworker/u16:4+events_unbound	08:40:16	6
623389	99	40	38,2	cyclictest	752714	-21	kworker/u16:2+events_unbound	09:33:08	5
623394	99	39	37,2	cyclictest	922506	-21	kworker/u16:1+events_unbound	09:54:03	6
623394	99	39	37,2	cyclictest	895343	-21	kworker/u16:0+events_unbound	11:01:28	6
623389	99	39	37,2	cyclictest	960017	-21	kworker/u16:2+events_unbound	09:57:53	5
623394	99	38	36,2	cyclictest	969092	-21	kworker/u16:3+events_unbound	10:16:25	6
623394	99	38	36,2	cyclictest	895343	-21	kworker/u16:0+events_unbound	10:24:44	6
623394	99	38	35,2	cyclictest	922506	-21	kworker/u16:1+events_unbound	09:44:59	6
623394	99	38	35,2	cyclictest	922506	-21	kworker/u16:1+events_unbound	09:44:59	6
623389	99	38	36,2	cyclictest	922506	-21	kworker/u16:1+events_unbound	09:48:40	5
623389	99	38	36,2	cyclictest	922506	-21	kworker/u16:1+events_unbound	09:48:40	5
623389	99	38	36,2	cyclictest	895343	-21	kworker/u16:0+events_unbound	10:31:32	5
623394	99	37	35,2	cyclictest	972340	-21	kworker/u16:5+events_unbound	10:00:28	6
623394	99	37	35,2	cyclictest	895343	-21	kworker/u16:0+events_unbound	11:52:48	6
623394	99	37	35,2	cyclictest	895343	-21	kworker/u16:0+events_unbound	11:52:48	6
623394	99	37	35,2	cyclictest	895343	-21	kworker/u16:0+events_unbound	11:36:39	6
623394	99	37	35,2	cyclictest	895343	-21	kworker/u16:0+events_unbound	11:09:00	6
623394	99	37	35,2	cyclictest	785625	-21	kworker/u16:4+events_unbound	09:25:48	6
623394	99	37	35,2	cyclictest	752714	-21	kworker/u16:2+events_unbound	09:17:28	6
623394	99	37	35,2	cyclictest	752714	-21	kworker/u16:2+events_unbound	09:10:16	6
623394	99	37	35,2	cyclictest	1231668	-21	kworker/u16:1+events_unbound	12:02:52	6
623394	99	37	35,2	cyclictest	1182387	-21	kworker/u16:4+events_unbound	11:34:12	6
623394	99	37	35,2	cyclictest	1009581	-21	kworker/u16:1+events_unbound	10:28:23	6
623394	99	37	35,2	cyclictest	1009581	-21	kworker/u16:1+events_unbound	10:28:23	6
623394	99	37	34,2	cyclictest	1231669	-21	kworker/u16:2+events_unbound	11:55:19	6
623389	99	37	35,2	cyclictest	972340	-21	kworker/u16:5+events_unbound	10:07:04	5
623389	99	37	35,2	cyclictest	969092	-21	kworker/u16:3+events_unbound	10:04:36	5
623389	99	37	35,2	cyclictest	785625	-21	kworker/u16:4+flush-8:0	09:50:00	5
623389	99	37	35,2	cyclictest	1302695	-21	kworker/u16:0+events_unbound	12:22:03	5
623389	99	37	35,2	cyclictest	1105810	-21	kworker/u16:3+events_unbound	11:10:51	5
623394	99	36	34,2	cyclictest	972340	-21	kworker/u16:5+events_unbound	10:09:00	6
623394	99	36	34,2	cyclictest	895343	-21	kworker/u16:0+events_unbound	11:17:24	6
623394	99	36	34,2	cyclictest	895343	-21	kworker/u16:0+events_unbound	11:11:48	6
623394	99	36	34,2	cyclictest	895343	-21	kworker/u16:0+events_unbound	10:53:51	6
623394	99	36	34,2	cyclictest	785625	-21	kworker/u16:4+events_unbound	09:22:13	6
623394	99	36	34,2	cyclictest	1302695	-21	kworker/u16:0+events_unbound	12:07:16	6
623394	99	36	34,2	cyclictest	1231668	-21	kworker/u16:1+events_unbound	12:13:43	6
623394	99	36	34,2	cyclictest	1231668	-21	kworker/u16:1+events_unbound	11:49:19	6
623394	99	36	34,2	cyclictest	1009581	-21	kworker/u16:1+events_unbound	10:48:28	6
623394	99	36	34,2	cyclictest	1009581	-21	kworker/u16:1+events_unbound	10:32:27	6
623389	99	36	34,2	cyclictest	895343	-21	kworker/u16:0+events_unbound	11:17:59	5
623389	99	36	34,2	cyclictest	1105810	-21	kworker/u16:3+events_unbound	11:08:52	5
623076	2	36	26,7	sleep4	0	-21	swapper/4	07:09:22	4
623394	99	35	33,2	cyclictest	895343	-21	kworker/u16:0+events_unbound	11:29:44	6
623394	99	35	33,2	cyclictest	1277605	-21	kworker/u16:3+events_unbound	12:19:11	6
623394	99	35	33,2	cyclictest	1105810	-21	kworker/u16:3+events_unbound	11:21:36	6
623394	99	35	33,2	cyclictest	1083121	-21	kworker/u16:2+events_unbound	10:55:19	6
623389	99	35	33,2	cyclictest	895343	-21	kworker/u16:0+events_unbound	09:36:16	5
623389	99	35	33,2	cyclictest	1330442	-21	kworker/u16:4+events_unbound	12:28:42	5
623389	99	35	33,2	cyclictest	1105810	-21	kworker/u16:3+events_unbound	11:28:52	5
623394	99	34	32,2	cyclictest	960017	-21	kworker/u16:2+events_unbound	09:56:43	6
623394	99	34	32,2	cyclictest	895343	-21	kworker/u16:0+events_unbound	09:40:16	6
623394	99	34	32,2	cyclictest	895343	-21	kworker/u16:0+events_unbound	09:37:28	6
623394	99	34	32,2	cyclictest	1231669	-21	kworker/u16:2+events_unbound	11:43:43	6
623389	99	34	32,2	cyclictest	1277605	-21	kworker/u16:3+events_unbound	12:16:27	5
623389	99	34	32,2	cyclictest	1182387	-21	kworker/u16:4+flush-8:0	11:35:00	5
623389	99	34	32,2	cyclictest	1083121	-21	kworker/u16:2+events_unbound	10:46:19	5
623389	99	34	32,2	cyclictest	1009581	-21	kworker/u16:1+events_unbound	10:21:15	5
623394	99	33	31,2	cyclictest	895343	-21	kworker/u16:0+events_unbound	10:11:52	6
623394	99	33	31,2	cyclictest	656306	-21	kworker/u16:0+events_unbound	07:25:08	6
623394	99	33	31,2	cyclictest	1330442	-21	kworker/u16:4+events_unbound	12:33:27	6
623394	99	33	31,2	cyclictest	1330442	-21	kworker/u16:4+events_unbound	12:33:27	6
623394	99	33	31,2	cyclictest	1083121	-21	kworker/u16:2+events_unbound	10:39:39	6
623389	99	33	31,2	cyclictest	895343	-21	kworker/u16:0+events_unbound	11:50:59	5
623389	99	33	31,2	cyclictest	895343	-21	kworker/u16:0+events_unbound	11:50:59	5
623389	99	33	31,2	cyclictest	785625	-21	kworker/u16:4+events_unbound	09:39:51	5
623389	99	33	31,2	cyclictest	1105810	-21	kworker/u16:3+events_unbound	11:40:43	5
623389	99	32	30,2	cyclictest	898037	-21	kworker/u16:3+events_unbound	09:26:39	5
997848	2	31	0,1	chrt	997850	-21	grep	10:04:56	7
623394	99	31	29,2	cyclictest	922506	-21	kworker/u16:1+events_unbound	09:34:13	6
623394	99	31	29,2	cyclictest	695526	-21	kworker/u16:0+flush-8:0	07:59:40	6
623394	99	31	28,2	cyclictest	1367439	-21	kworker/u16:2+flush-8:0	12:35:36	6
623389	99	31	29,2	cyclictest	895343	-21	kworker/u16:0+events_unbound	11:20:31	5
623389	99	31	29,2	cyclictest	895343	-21	kworker/u16:0+events_unbound	10:40:41	5
623389	99	31	29,2	cyclictest	752714	-21	kworker/u16:2+flush-8:0	09:15:04	5
623389	99	31	29,2	cyclictest	698691	-21	kworker/u16:2+events_unbound	07:54:48	5
623389	99	31	29,2	cyclictest	1302695	-21	kworker/u16:0+events_unbound	12:31:53	5
623389	99	31	29,2	cyclictest	1302695	-21	kworker/u16:0+events_unbound	12:31:53	5
623389	99	31	29,2	cyclictest	1231668	-21	kworker/u16:1+events_unbound	12:38:38	5
623394	99	30	28,2	cyclictest	613441	-21	kworker/u16:4+events_unbound	07:20:56	6
623394	99	30	27,2	cyclictest	752714	-21	kworker/u16:2+events_unbound	09:07:12	6
623394	99	30	27,2	cyclictest	752714	-21	kworker/u16:2+events_unbound	09:07:12	6
623394	99	30	27,2	cyclictest	695526	-21	kworker/u16:0+flush-8:0	08:04:56	6
623389	99	30	28,2	cyclictest	969092	-21	kworker/u16:3+events_unbound	10:29:23	5
623389	99	30	28,2	cyclictest	969092	-21	kworker/u16:3+events_unbound	10:29:23	5
623389	99	30	28,2	cyclictest	764568	-21	kworker/u16:3+events_unbound	08:20:57	5
623389	99	30	28,2	cyclictest	763692	-21	kworker/u16:1+flush-8:0	08:25:11	5
623389	99	30	28,2	cyclictest	1231668	-21	kworker/u16:1+events_unbound	12:13:08	5
623389	99	30	27,2	cyclictest	895343	-21	kworker/u16:0+events_unbound	11:30:33	5
623398	99	29	28,0	cyclictest	0	-21	swapper/7	09:26:20	7
623398	99	29	27,1	cyclictest	1330818	-21	ssh	12:19:52	7
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17