OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Fri Jul 03, 2026 00:43:34)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
1500984	2	90	0,0	sleep6	0	-21	swapper/6	00:24:20	6
1257002	2	71	0,1	sleep4	0	-21	swapper/4	22:44:16	4
786337	2	53	42,8	sleep4	0	-21	swapper/4	19:05:00	4
786322	2	51	17,29	sleep7	0	-21	swapper/7	19:04:47	7
786922	99	49	46,2	cyclictest	1036285	-21	perf	21:13:56	7
786917	99	41	39,2	cyclictest	975889	-21	kworker/u16:3+flush-8:0	21:49:24	6
786922	99	40	35,4	cyclictest	1388501	-21	latency_hist	23:38:57	7
786912	99	40	37,2	cyclictest	1109576	-21	kworker/u16:0+events_unbound	21:44:25	4
786922	99	39	36,2	cyclictest	1240062	-21	smartctl	22:34:28	7
786912	99	39	37,2	cyclictest	975889	-21	kworker/u16:3+events_unbound	21:09:08	4
786912	99	39	37,2	cyclictest	831270	-21	kworker/u16:1+events_unbound	19:44:11	4
6414	89	39	0,32	rtkit-daemon	0	-21	swapper/5	19:04:01	5
786922	99	37	36,0	cyclictest	0	-21	swapper/7	23:33:12	7
786912	99	37	35,2	cyclictest	995338	-21	kworker/u16:4+flush-8:0	20:59:27	4
786912	99	37	35,2	cyclictest	1145190	-21	kworker/u16:3+events_unbound	22:23:36	4
786922	99	36	0,34	cyclictest	1516118	-21	tr	00:29:19	7
786499	2	36	25,8	sleep6	0	-21	swapper/6	19:07:19	6
786912	99	35	33,2	cyclictest	1145632	-21	kworker/u16:5+events_unbound	23:21:54	4
786912	99	35	33,2	cyclictest	1145632	-21	kworker/u16:5+events_unbound	23:21:54	4
786922	99	34	31,2	cyclictest	891487	-21	grep	19:59:16	7
786915	99	34	31,2	cyclictest	995338	-21	kworker/u16:4+flush-8:0	20:54:28	5
786912	99	34	32,2	cyclictest	1014113	-21	kworker/u16:0+flush-8:0	21:04:22	4
786912	99	33	31,2	cyclictest	1131070	-21	kworker/u16:2+events_unbound	00:34:32	4
786922	99	32	29,2	cyclictest	1491636	-21	kthreadcore	00:19:19	7
786922	99	32	0,30	cyclictest	1148654	-21	kthreadcore	21:59:16	7
786912	99	32	30,2	cyclictest	965753	-21	kworker/u16:2+events_unbound	21:32:43	4
786912	99	32	30,2	cyclictest	1145632	-21	kworker/u16:5+events_unbound	23:16:39	4
786922	99	31	30,0	cyclictest	0	-21	swapper/7	22:54:07	7
786922	99	31	30,0	cyclictest	0	-21	swapper/7	21:19:12	7
786922	99	31	28,2	cyclictest	1315706	-21	munin-run	23:08:56	7
786922	99	31	26,3	cyclictest	1007633	-21	cut	20:59:16	7
786917	99	31	29,2	cyclictest	757939	-21	kworker/u16:2+events_unbound	19:14:00	6
786912	99	31	29,2	cyclictest	1267258	-21	kworker/u16:1+events_unbound	00:26:15	4
786912	99	31	29,2	cyclictest	1131070	-21	kworker/u16:2+events_unbound	00:07:51	4
786912	99	31	28,2	cyclictest	1145632	-21	kworker/u16:5+flush-8:0	23:44:12	4
786922	99	30	29,0	cyclictest	0	-21	swapper/7	23:03:08	7
786922	99	30	29,0	cyclictest	0	-21	swapper/7	20:04:13	7
786922	99	30	28,1	cyclictest	1447327	-21	ssh	00:00:20	7
786922	99	30	28,1	cyclictest	1447327	-21	ssh	00:00:20	7
786912	99	30	28,2	cyclictest	975889	-21	kworker/u16:3+events_unbound	21:15:35	4
786912	99	30	28,2	cyclictest	975889	-21	kworker/u16:3+events_unbound	20:57:44	4
786912	99	30	28,2	cyclictest	965753	-21	kworker/u16:2+events_unbound	21:42:43	4
786912	99	30	28,2	cyclictest	1437550	-21	kworker/u16:3+events_unbound	00:33:45	4
786912	99	30	28,2	cyclictest	1145632	-21	kworker/u16:5+events_unbound	22:56:35	4
786912	99	30	28,2	cyclictest	1145632	-21	kworker/u16:5+events_unbound	22:15:19	4
786912	99	30	28,2	cyclictest	1145190	-21	kworker/u16:3+events_unbound	22:42:27	4
786912	99	30	28,2	cyclictest	1131070	-21	kworker/u16:2+events_unbound	22:01:15	4
786922	99	29	28,0	cyclictest	0	-21	swapper/7	22:19:00	7
786917	99	29	27,2	cyclictest	775381	-21	kworker/u16:4+flush-8:0	19:53:52	6
786917	99	29	27,2	cyclictest	775381	-21	kworker/u16:4+flush-8:0	19:28:53	6
786917	99	29	27,2	cyclictest	1267258	-21	kworker/u16:1+flush-8:0	00:18:52	6
786912	99	29	27,2	cyclictest	975889	-21	kworker/u16:3+events_unbound	21:26:14	4
786912	99	29	27,2	cyclictest	897771	-21	kworker/u16:1+events_unbound	20:52:33	4
786912	99	29	27,2	cyclictest	775381	-21	kworker/u16:4+events_unbound	20:18:37	4
786912	99	29	27,2	cyclictest	1145190	-21	kworker/u16:3+flush-8:0	22:49:07	4
786912	99	29	27,2	cyclictest	1131070	-21	kworker/u16:2+events_unbound	23:38:13	4
786912	99	29	27,2	cyclictest	1131070	-21	kworker/u16:2+events_unbound	23:05:03	4
786912	99	29	27,2	cyclictest	1131070	-21	kworker/u16:2+events_unbound	21:52:35	4
786912	99	29	27,2	cyclictest	1058197	-21	kworker/u16:0+events_unbound	21:20:42	4
786922	99	28	27,0	cyclictest	0	-21	swapper/7	19:44:13	7
786917	99	28	26,2	cyclictest	1437550	-21	kworker/u16:3+flush-8:0	00:38:51	6
786915	99	28	26,2	cyclictest	965753	-21	kworker/u16:2+flush-8:0	21:19:20	5
786912	99	28	26,2	cyclictest	965753	-21	kworker/u16:2+events_unbound	21:35:49	4
786912	99	28	26,2	cyclictest	775381	-21	kworker/u16:4+events_unbound	19:33:01	4
786912	99	28	26,2	cyclictest	1470518	-21	kworker/u16:5+events_unbound	00:10:18	4
786912	99	28	26,2	cyclictest	1267258	-21	kworker/u16:1+events_unbound	23:01:09	4
786912	99	28	26,2	cyclictest	1145190	-21	kworker/u16:3+events_unbound	22:12:19	4
786912	99	28	26,2	cyclictest	1131070	-21	kworker/u16:2+events_unbound	23:57:18	4
786912	99	28	26,2	cyclictest	1131070	-21	kworker/u16:2+events_unbound	23:10:02	4
786922	99	27	24,2	cyclictest	1251991	-21	ntpq	22:39:25	7
786917	99	27	25,2	cyclictest	1145190	-21	kworker/u16:3+flush-8:0	22:08:57	6
786912	99	27	25,2	cyclictest	795568	-21	kworker/u16:3+events_unbound	19:14:25	4
786912	99	27	25,2	cyclictest	775381	-21	kworker/u16:4+events_unbound	20:28:17	4
786912	99	27	25,2	cyclictest	1267258	-21	kworker/u16:1+events_unbound	00:02:33	4
786912	99	27	25,2	cyclictest	1267258	-21	kworker/u16:1+events_unbound	00:02:33	4
786912	99	27	25,2	cyclictest	1145190	-21	kworker/u16:3+events_unbound	22:35:29	4
786912	99	27	25,2	cyclictest	1131070	-21	kworker/u16:2+flush-8:0	23:24:23	4
786912	99	27	23,2	cyclictest	1109576	-21	kworker/u16:0+events_unbound	21:57:30	4
786912	99	27	23,2	cyclictest	1109576	-21	kworker/u16:0+events_unbound	21:57:30	4
786922	99	26	25,0	cyclictest	0	-21	swapper/7	23:24:24	7
786922	99	26	23,2	cyclictest	1095538	-21	ssh	21:36:08	7
786912	99	26	24,2	cyclictest	937107	-21	kworker/u16:0+events_unbound	20:37:52	4
786912	99	26	24,2	cyclictest	937107	-21	kworker/u16:0+events_unbound	20:31:43	4
786912	99	26	24,2	cyclictest	937107	-21	kworker/u16:0+events_unbound	20:31:43	4
786912	99	26	24,2	cyclictest	775381	-21	kworker/u16:4+flush-8:0	20:09:13	4
786912	99	26	24,2	cyclictest	1437550	-21	kworker/u16:3+events_unbound	00:21:26	4
786912	99	26	24,2	cyclictest	1437550	-21	kworker/u16:3+events_unbound	00:15:09	4
786912	99	26	24,2	cyclictest	1267258	-21	kworker/u16:1+events_unbound	23:49:41	4
786912	99	26	24,2	cyclictest	1267258	-21	kworker/u16:1+events_unbound	23:43:22	4
786912	99	26	24,2	cyclictest	1145632	-21	kworker/u16:5+events_unbound	23:31:22	4
786912	99	26	24,2	cyclictest	1145190	-21	kworker/u16:3+events_unbound	22:30:51	4
786912	99	26	24,2	cyclictest	1131070	-21	kworker/u16:2+events_unbound	22:25:27	4
786922	99	25	24,0	cyclictest	0	-21	swapper/7	21:49:07	7
786922	99	25	21,2	cyclictest	1389261	-21	cstates	23:39:08	7
786922	99	25	20,3	cyclictest	1134749	-21	sed	21:54:14	7
786922	99	25	20,3	cyclictest	1134749	-21	sed	21:54:14	7
786922	99	25	0,23	cyclictest	1278	-21	snmpd	23:23:04	7
786922	99	25	0,23	cyclictest	1278	-21	snmpd	23:23:04	7
786912	99	25	23,2	cyclictest	937107	-21	kworker/u16:0+events_unbound	20:44:44	4
786912	99	25	23,2	cyclictest	775381	-21	kworker/u16:4+events_unbound	19:24:21	4
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17