OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Sat Jan 24, 2026 00:43:33)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
767709	2	66	55,8	sleep5	0	-21	swapper/5	19:05:48	5
766309	2	63	0,2	chrt	0	-21	swapper/4	19:02:33	4
768102	99	48	43,4	cyclictest	650	-21	systemd-journal	21:52:07	7
767767	2	45	11,30	sleep7	0	-21	swapper/7	19:06:38	7
768102	99	41	37,3	cyclictest	916156	-21	ls	20:17:35	7
768091	99	41	39,2	cyclictest	1220010	-21	kworker/u16:1+events_unbound	22:47:48	5
768096	99	39	37,2	cyclictest	1449506	-21	kworker/u16:3+events_unbound	00:09:13	6
768096	99	39	37,2	cyclictest	1391506	-21	kworker/u16:4+flush-8:0	23:48:29	6
768091	99	39	37,2	cyclictest	800860	-21	kworker/u16:3+events_unbound	19:37:33	5
768091	99	39	37,2	cyclictest	1415743	-21	kworker/u16:0+events_unbound	00:36:21	5
768091	99	39	37,2	cyclictest	1391506	-21	kworker/u16:4+events_unbound	00:03:37	5
768091	99	39	37,2	cyclictest	1270878	-21	kworker/u16:2+events_unbound	23:11:50	5
1124205	2	39	0,0	sleep7	1124206	-21	sh	21:55:44	7
768096	99	38	36,2	cyclictest	1415743	-21	kworker/u16:0+events_unbound	00:14:44	6
768096	99	38	36,2	cyclictest	1233055	-21	kworker/u16:0+events_unbound	22:42:58	6
768096	99	38	36,2	cyclictest	1220010	-21	kworker/u16:1+events_unbound	22:50:36	6
768096	99	38	36,2	cyclictest	1123853	-21	kworker/u16:0+events_unbound	21:55:08	6
768096	99	38	36,2	cyclictest	1028175	-21	kworker/u16:4+events_unbound	21:39:53	6
768091	99	38	36,2	cyclictest	800859	-21	kworker/u16:2+events_unbound	21:15:33	5
768091	99	38	36,2	cyclictest	1220010	-21	kworker/u16:1+events_unbound	23:04:38	5
768091	99	38	36,2	cyclictest	1088467	-21	kworker/u16:2+events_unbound	22:06:36	5
768102	99	37	32,4	cyclictest	1137193	-21	latency_hist	22:02:07	7
768102	99	37	32,4	cyclictest	1137193	-21	latency_hist	22:02:07	7
768096	99	37	35,2	cyclictest	1319018	-21	kworker/u16:3+events_unbound	23:31:56	6
768096	99	37	35,2	cyclictest	1233055	-21	kworker/u16:0+events_unbound	23:09:33	6
768096	99	37	35,2	cyclictest	1220010	-21	kworker/u16:1+events_unbound	23:36:04	6
768096	99	37	35,2	cyclictest	1003922	-21	kworker/u16:0+events_unbound	21:14:25	6
768096	99	37	34,2	cyclictest	887568	-21	kworker/u16:3+flush-8:0	20:37:07	6
768091	99	37	35,2	cyclictest	1233055	-21	kworker/u16:0+flush-8:0	22:43:05	5
768091	99	37	35,2	cyclictest	1170034	-21	kworker/u16:5+events_unbound	22:36:45	5
768091	99	37	35,2	cyclictest	1042182	-21	kworker/u16:1+events_unbound	21:50:48	5
768096	99	36	34,2	cyclictest	1415743	-21	kworker/u16:0+events_unbound	00:33:25	6
768096	99	36	34,2	cyclictest	1391506	-21	kworker/u16:4+events_unbound	00:04:40	6
768096	99	36	34,2	cyclictest	1270878	-21	kworker/u16:2+events_unbound	23:47:08	6
768096	99	36	34,2	cyclictest	1270878	-21	kworker/u16:2+events_unbound	23:01:05	6
768096	99	36	34,2	cyclictest	1233055	-21	kworker/u16:0+events_unbound	22:56:24	6
768096	99	36	34,2	cyclictest	1171266	-21	kworker/u16:6+events_unbound	22:36:57	6
768096	99	36	34,2	cyclictest	1042182	-21	kworker/u16:1+events_unbound	21:51:09	6
768096	99	36	34,2	cyclictest	1028175	-21	kworker/u16:4+events_unbound	21:22:49	6
768091	99	36	34,2	cyclictest	1415743	-21	kworker/u16:0+events_unbound	00:31:32	5
768091	99	36	34,2	cyclictest	1415743	-21	kworker/u16:0+events_unbound	00:24:14	5
768091	99	36	34,2	cyclictest	1319018	-21	kworker/u16:3+events_unbound	23:39:25	5
768091	99	36	34,2	cyclictest	1270878	-21	kworker/u16:2+events_unbound	00:10:10	5
768091	99	36	34,2	cyclictest	1220010	-21	kworker/u16:1+events_unbound	23:30:57	5
768091	99	36	34,2	cyclictest	1220010	-21	kworker/u16:1+events_unbound	23:24:22	5
768091	99	36	34,2	cyclictest	1220010	-21	kworker/u16:1+events_unbound	22:41:01	5
768091	99	36	34,2	cyclictest	1170034	-21	kworker/u16:5+events_unbound	22:56:28	5
768091	99	36	34,2	cyclictest	1088467	-21	kworker/u16:2+events_unbound	22:01:25	5
768091	99	36	34,2	cyclictest	1088467	-21	kworker/u16:2+events_unbound	22:01:25	5
768091	99	36	34,2	cyclictest	1028175	-21	kworker/u16:4+events_unbound	21:20:34	5
768091	99	36	34,2	cyclictest	1028175	-21	kworker/u16:4+events_unbound	21:20:34	5
768102	99	35	0,33	cyclictest	1222412	-21	latency_hist	22:37:07	7
768096	99	35	33,2	cyclictest	1496238	-21	kworker/u16:1+events_unbound	00:30:12	6
768096	99	35	33,2	cyclictest	1220010	-21	kworker/u16:1+events_unbound	23:06:29	6
768096	99	35	33,2	cyclictest	1149419	-21	kworker/u16:1+events_unbound	22:26:45	6
768096	99	35	33,2	cyclictest	1149419	-21	kworker/u16:1+events_unbound	22:14:38	6
768091	99	35	33,2	cyclictest	1391506	-21	kworker/u16:4+events_unbound	23:49:13	5
768091	99	35	33,2	cyclictest	1270878	-21	kworker/u16:2+events_unbound	23:00:53	5
768091	99	35	33,2	cyclictest	1233055	-21	kworker/u16:0+events_unbound	23:13:08	5
768091	99	35	33,2	cyclictest	1042182	-21	kworker/u16:1+events_unbound	21:46:36	5
768091	99	35	33,2	cyclictest	1028175	-21	kworker/u16:4+events_unbound	22:11:17	5
768091	99	35	33,2	cyclictest	1028175	-21	kworker/u16:4+events_unbound	21:31:58	5
768096	99	34	32,2	cyclictest	1220010	-21	kworker/u16:1+events_unbound	23:24:56	6
768096	99	34	32,2	cyclictest	1220010	-21	kworker/u16:1+events_unbound	23:15:13	6
768096	99	34	32,2	cyclictest	1123853	-21	kworker/u16:0+events_unbound	22:04:48	6
768096	99	34	32,2	cyclictest	1027747	-21	kworker/u16:3+events_unbound	21:20:50	6
768096	99	34	32,2	cyclictest	1027747	-21	kworker/u16:3+events_unbound	21:20:50	6
768096	99	34	31,2	cyclictest	1391506	-21	kworker/u16:4+flush-8:0	00:22:33	6
768091	99	34	32,2	cyclictest	1270878	-21	kworker/u16:2+events_unbound	23:53:28	5
768091	99	34	32,2	cyclictest	1170034	-21	kworker/u16:5+events_unbound	22:18:21	5
768091	99	34	32,2	cyclictest	1042182	-21	kworker/u16:1+events_unbound	21:26:45	5
768091	99	34	32,2	cyclictest	1028175	-21	kworker/u16:4+events_unbound	21:53:40	5
768091	99	34	32,2	cyclictest	1028175	-21	kworker/u16:4+events_unbound	21:41:37	5
768091	99	34	32,2	cyclictest	1028175	-21	kworker/u16:4+events_unbound	21:35:41	5
768096	99	33	31,2	cyclictest	1170034	-21	kworker/u16:5+events_unbound	22:19:45	6
768096	99	33	31,2	cyclictest	1003922	-21	kworker/u16:0+events_unbound	21:11:46	6
768091	99	33	31,2	cyclictest	917043	-21	kworker/u16:1+flush-8:0	20:22:37	5
768091	99	33	31,2	cyclictest	1391506	-21	kworker/u16:4+events_unbound	00:14:26	5
768091	99	33	31,2	cyclictest	1270878	-21	kworker/u16:2+events_unbound	23:59:00	5
768091	99	33	31,2	cyclictest	1003922	-21	kworker/u16:0+events_unbound	21:08:17	5
768096	99	32	30,2	cyclictest	1391506	-21	kworker/u16:4+events_unbound	00:21:48	6
768096	99	32	30,2	cyclictest	1220010	-21	kworker/u16:1+events_unbound	23:40:20	6
768096	99	32	30,2	cyclictest	1042182	-21	kworker/u16:1+flush-8:0	21:32:53	6
768096	99	32	30,2	cyclictest	1028175	-21	kworker/u16:4+events_unbound	22:11:01	6
768096	99	32	30,2	cyclictest	1027747	-21	kworker/u16:3+events_unbound	21:27:56	6
768091	99	32	30,2	cyclictest	1270878	-21	kworker/u16:2+events_unbound	23:19:04	5
768091	99	32	30,2	cyclictest	1270878	-21	kworker/u16:2+events_unbound	23:19:04	5
768091	99	32	30,2	cyclictest	1220010	-21	kworker/u16:1+events_unbound	23:45:50	5
768087	99	32	29,2	cyclictest	1391506	-21	kworker/u16:4+events_unbound	23:52:21	4
768096	99	31	29,2	cyclictest	800859	-21	kworker/u16:2+events_unbound	19:37:35	6
768091	99	31	29,2	cyclictest	1220010	-21	kworker/u16:1+events_unbound	23:33:33	5
768091	99	31	29,2	cyclictest	1171266	-21	kworker/u16:6+events_unbound	22:14:54	5
1052000	2	31	0,1	chrt	5436	-21	crond	21:27:07	7
768096	99	30	28,2	cyclictest	800859	-21	kworker/u16:2+events_unbound	20:13:46	6
768096	99	30	28,2	cyclictest	1170034	-21	kworker/u16:5+events_unbound	22:30:36	6
768096	99	30	28,2	cyclictest	1088467	-21	kworker/u16:2+events_unbound	21:59:16	6
768096	99	30	28,2	cyclictest	1088467	-21	kworker/u16:2+events_unbound	21:59:16	6
768096	99	30	27,2	cyclictest	1270878	-21	kworker/u16:2+events_unbound	23:52:11	6
768091	99	30	28,2	cyclictest	974626	-21	kworker/u16:4+events_unbound	21:03:34	5
768091	99	30	28,2	cyclictest	1415743	-21	kworker/u16:0+events_unbound	00:18:01	5
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17