OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Mon Jul 06, 2026 00:43:36)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
628417	2	49	38,8	sleep4	0	-21	swapper/4	19:06:03	4
628891	99	43	41,2	cyclictest	1281483	-21	kworker/u16:1+flush-8:0	00:24:08	6
628391	2	43	8,30	sleep7	0	-21	swapper/7	19:05:42	7
628896	99	42	0,40	cyclictest	1305784	-21	latency_hist	00:08:44	7
628888	99	41	39,2	cyclictest	1110509	-21	kworker/u16:2+flush-8:0	23:03:45	5
628888	99	39	35,4	cyclictest	610438	-21	kworker/u16:3+flush-8:0	19:18:45	5
619289	2	39	27,9	sleep5	0	-21	swapper/5	19:03:47	5
628891	99	38	36,2	cyclictest	987682	-21	kworker/u16:1+events_unbound	22:14:12	6
628891	99	38	36,2	cyclictest	779039	-21	kworker/u16:1+flush-8:0	20:39:12	6
628891	99	37	35,2	cyclictest	1245236	-21	kworker/u16:0+events_unbound	23:54:20	6
628891	99	35	33,2	cyclictest	1110509	-21	kworker/u16:2+events_unbound	23:48:44	6
628891	99	35	33,2	cyclictest	1110424	-21	kworker/u16:0+events_unbound	22:53:43	6
628587	2	35	24,8	sleep6	0	-21	swapper/6	19:08:29	6
628896	99	34	28,5	cyclictest	650	-21	systemd-journal	00:13:44	7
628891	99	34	33,1	cyclictest	915401	-21	kworker/u16:4+events_unbound	21:44:12	6
628891	99	34	32,2	cyclictest	927886	-21	kworker/u16:0+flush-8:0	21:51:24	6
628891	99	34	32,2	cyclictest	927886	-21	kworker/u16:0+flush-8:0	21:42:52	6
628896	99	33	31,1	cyclictest	0	-21	swapper/7	00:34:00	7
628896	99	33	29,3	cyclictest	124	-21	kswapd0	22:48:59	7
628891	99	33	31,2	cyclictest	903279	-21	kworker/u16:1+flush-8:0	21:26:08	6
628891	99	33	31,2	cyclictest	1354057	-21	kworker/u16:0+events_unbound	00:32:52	6
628891	99	33	31,2	cyclictest	1074312	-21	kworker/u16:1+events_unbound	22:59:52	6
628891	99	32	30,2	cyclictest	964373	-21	kworker/u16:2+events_unbound	22:22:56	6
628891	99	32	30,2	cyclictest	610438	-21	kworker/u16:3+flush-8:0	19:37:56	6
628891	99	31	29,2	cyclictest	739759	-21	kworker/u16:3+flush-8:0	20:23:41	6
628891	99	31	29,2	cyclictest	1281483	-21	kworker/u16:1+events_unbound	00:36:19	6
628891	99	31	29,2	cyclictest	1281483	-21	kworker/u16:1+events_unbound	00:13:27	6
628891	99	30	28,2	cyclictest	964373	-21	kworker/u16:2+events_unbound	22:02:44	6
628891	99	30	28,2	cyclictest	964373	-21	kworker/u16:2+events_unbound	22:02:44	6
628891	99	30	28,2	cyclictest	818051	-21	kworker/u16:0+events_unbound	20:44:11	6
628891	99	30	28,2	cyclictest	1281483	-21	kworker/u16:1+events_unbound	00:14:55	6
628891	99	30	28,2	cyclictest	1110509	-21	kworker/u16:2+events_unbound	00:04:51	6
628891	99	30	28,2	cyclictest	1110424	-21	kworker/u16:0+events_unbound	23:11:36	6
628891	99	30	27,2	cyclictest	1110509	-21	kworker/u16:2+flush-8:0	23:13:55	6
628896	99	29	28,0	cyclictest	0	-21	swapper/7	20:59:00	7
628891	99	29	27,2	cyclictest	987682	-21	kworker/u16:1+events_unbound	22:10:24	6
628891	99	29	27,2	cyclictest	987682	-21	kworker/u16:1+events_unbound	22:07:20	6
628891	99	29	27,2	cyclictest	1218265	-21	kworker/u16:3+events_unbound	23:32:23	6
628891	99	29	27,2	cyclictest	1171639	-21	kworker/u16:1+flush-8:0	23:19:31	6
628891	99	29	27,2	cyclictest	1171639	-21	kworker/u16:1+flush-8:0	23:19:31	6
628891	99	29	27,2	cyclictest	1110509	-21	kworker/u16:2+flush-8:0	00:19:43	6
628891	99	29	27,2	cyclictest	1037373	-21	kworker/u16:3+events_unbound	22:35:56	6
628891	99	28	26,2	cyclictest	779039	-21	kworker/u16:1+events_unbound	21:11:01	6
628891	99	28	26,2	cyclictest	1291212	-21	kworker/u16:4+events_unbound	00:02:52	6
628891	99	28	26,2	cyclictest	1110509	-21	kworker/u16:2+events_unbound	23:37:12	6
628891	99	28	26,2	cyclictest	1110424	-21	kworker/u16:0+events_unbound	23:27:55	6
628891	99	28	26,2	cyclictest	1086381	-21	kworker/u16:4+events_unbound	22:38:48	6
628891	99	28	26,2	cyclictest	1086381	-21	kworker/u16:4+events_unbound	22:38:48	6
628888	99	28	26,2	cyclictest	1037373	-21	kworker/u16:3+flush-8:0	22:44:04	5
628888	99	28	26,2	cyclictest	1010440	-21	kworker/u16:0+events_unbound	22:24:12	5
628896	99	27	0,25	cyclictest	1269253	-21	cat	23:53:45	7
628891	99	27	25,2	cyclictest	964373	-21	kworker/u16:2+flush-8:0	22:31:40	6
628891	99	27	25,2	cyclictest	903279	-21	kworker/u16:1+events_unbound	21:32:25	6
628891	99	27	25,2	cyclictest	888428	-21	kworker/u16:0+events_unbound	21:16:59	6
628891	99	27	25,2	cyclictest	888428	-21	kworker/u16:0+events_unbound	21:16:59	6
628891	99	27	25,2	cyclictest	1245236	-21	kworker/u16:0+events_unbound	23:51:45	6
628891	99	27	25,2	cyclictest	1074312	-21	kworker/u16:1+events_unbound	22:47:08	6
628888	99	27	25,2	cyclictest	1110509	-21	kworker/u16:2+events_unbound	00:09:07	5
628896	99	26	21,4	cyclictest	878594	-21	latency_hist	21:13:45	7
628891	99	26	24,2	cyclictest	927886	-21	kworker/u16:0+flush-8:0	21:56:00	6
628891	99	25	23,2	cyclictest	964373	-21	kworker/u16:2+events_unbound	22:26:47	6
628891	99	25	23,2	cyclictest	610438	-21	kworker/u16:3+flush-8:0	19:44:56	6
628891	99	25	23,2	cyclictest	1134993	-21	kworker/u16:3+events_unbound	23:06:09	6
628891	99	25	22,2	cyclictest	702330	-21	kworker/u16:0+flush-8:0	20:13:45	6
628891	99	24	22,2	cyclictest	845879	-21	kworker/u16:3+events_unbound	21:22:31	6
628891	99	24	22,2	cyclictest	1218265	-21	kworker/u16:3+events_unbound	23:42:24	6
628891	99	24	22,2	cyclictest	1110509	-21	kworker/u16:2+events_unbound	22:55:11	6
628888	99	24	23,1	cyclictest	1086381	-21	kworker/u16:4+events_unbound	22:43:23	5
628888	99	24	23,1	cyclictest	1086381	-21	kworker/u16:4+events_unbound	22:43:23	5
628891	99	23	21,2	cyclictest	903279	-21	kworker/u16:1+flush-8:0	21:33:55	6
628888	99	23	21,2	cyclictest	987682	-21	kworker/u16:1+flush-8:0	22:03:39	5
628888	99	23	21,2	cyclictest	987682	-21	kworker/u16:1+flush-8:0	22:03:39	5
628888	99	23	21,2	cyclictest	888428	-21	kworker/u16:0+flush-8:0	21:28:40	5
628888	99	23	21,2	cyclictest	779039	-21	kworker/u16:1+events_unbound	20:44:07	5
628888	99	23	21,2	cyclictest	691002	-21	kworker/u16:1+events_unbound	19:59:02	5
628888	99	23	20,2	cyclictest	856765	-21	kworker/u16:0+events_unbound	21:08:58	5
628888	99	23	20,2	cyclictest	1110424	-21	kworker/u16:0+events_unbound	23:05:34	5
628896	99	22	17,4	cyclictest	1037332	-21	latency_hist	22:18:45	7
628888	99	22	20,2	cyclictest	1110509	-21	kworker/u16:2+flush-8:0	23:39:15	5
628891	99	21	19,2	cyclictest	555238	-21	kworker/u16:2+flush-8:0	19:43:40	6
628896	99	20	0,18	cyclictest	1205064	-21	ssh	23:24:08	7
628891	99	20	18,2	cyclictest	691002	-21	kworker/u16:1+flush-8:0	20:03:40	6
628888	99	20	18,2	cyclictest	1281483	-21	kworker/u16:1+flush-8:0	00:23:51	5
628888	99	20	17,2	cyclictest	1110424	-21	kworker/u16:0+events_unbound	23:33:58	5
628896	99	19	0,17	cyclictest	0	-21	swapper/7	19:14:09	7
628888	99	19	17,2	cyclictest	787496	-21	kworker/u16:4+flush-8:0	20:29:12	5
628888	99	19	17,2	cyclictest	702330	-21	kworker/u16:0+flush-8:0	20:14:13	5
628888	99	19	17,2	cyclictest	555238	-21	kworker/u16:2+flush-8:0	19:29:11	5
628888	99	19	17,2	cyclictest	1245236	-21	kworker/u16:0+events_unbound	23:58:57	5
628888	99	19	15,3	cyclictest	555238	-21	kworker/u16:2+flush-8:0	19:39:00	5
628896	99	18	15,2	cyclictest	866385	-21	diskmemload	23:34:23	7
628896	99	18	13,4	cyclictest	1025105	-21	latency_hist	22:13:45	7
628896	99	18	12,5	cyclictest	1134906	-21	latency_hist	22:58:45	7
628896	99	18	0,17	cyclictest	0	-21	swapper/7	22:24:11	7
628891	99	18	16,2	cyclictest	798214	-21	kworker/u16:2+flush-8:0	21:03:37	6
628888	99	18	16,2	cyclictest	691002	-21	kworker/u16:1+flush-8:0	19:44:07	5
628888	99	18	16,2	cyclictest	1110509	-21	kworker/u16:2+events_unbound	00:04:04	5
628888	99	18	15,2	cyclictest	927886	-21	kworker/u16:0+flush-8:0	21:48:45	5
628884	99	18	16,2	cyclictest	915401	-21	kworker/u16:4+flush-8:0	21:44:11	4
628896	99	17	13,3	cyclictest	1278	-21	snmpd	19:46:13	7
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17