OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #7, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system

Characteristics of the 792 highest latencies:

System rack7slot2.osadl.org (updated Fri Dec 26, 2025 12:44:10)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
1198328	99	134	123,8	cyclictest	54	-21	kswapd0	09:40:09	0
1198329	99	128	121,5	cyclictest	54	-21	kswapd0	10:10:12	1
1198329	99	107	95,7	cyclictest	54	-21	kswapd0	10:30:15	1
1198328	99	100	93,5	cyclictest	1219844	-21	kworker/u4:18+events_unbound	08:49:59	0
1198328	99	98	85,11	cyclictest	54	-21	kswapd0	07:35:10	0
1198328	99	97	89,6	cyclictest	1222386	-21	kworker/u4:9+events_unbound	08:44:59	0
1198328	99	97	87,8	cyclictest	54	-21	kswapd0	11:40:36	0
1198329	99	85	67,7	cyclictest	1203338	-21	kworker/u4:1+events_unbound	07:29:42	1
1198328	99	85	78,5	cyclictest	54	-21	kswapd0	10:00:00	0
1198329	99	82	67,6	cyclictest	1222386	-21	kworker/u4:9+events_unbound	09:00:00	1
1198328	99	82	73,6	cyclictest	54	-21	kswapd0	10:35:16	0
1198329	99	81	68,5	cyclictest	1343273	-21	kworker/u4:0+events_unbound	11:30:42	1
1198329	99	80	74,5	cyclictest	1250404	-21	kworker/u4:0+events_unbound	09:35:35	1
1198329	99	79	73,4	cyclictest	1200790	-21	kworker/u4:14+events_unbound	07:35:11	1
1198328	99	79	69,8	cyclictest	54	-21	kswapd0	10:00:12	0
1198329	99	78	64,5	cyclictest	1213532	-21	kworker/u4:0+events_unbound	08:09:53	1
1198329	99	77	63,7	cyclictest	1269550	-21	kworker/u4:6+events_unbound	10:05:33	1
1198329	99	77	63,7	cyclictest	1269550	-21	kworker/u4:6+events_unbound	10:05:33	1
1198328	99	77	69,6	cyclictest	1221106	-21	kworker/u4:5+events_unbound	08:55:00	0
1198329	99	75	67,6	cyclictest	1212273	-21	kworker/u4:16+events_unbound	08:14:54	1
1198328	99	74	66,6	cyclictest	1198252	-21	kworker/u4:1+events_unbound	07:14:40	0
1198329	99	73	66,5	cyclictest	1217332	-21	kworker/u4:6+events_unbound	09:40:10	1
1198329	99	73	58,6	cyclictest	1310486	-21	kworker/u4:12+events_unbound	11:11:10	1
1198329	99	72	66,3	cyclictest	1301828	-21	kworker/u4:2+events_unbound	10:45:07	1
1198329	99	72	65,5	cyclictest	1397151	-21	kworker/u4:5+events_unbound	12:15:46	1
1198329	99	72	65,5	cyclictest	1352431	-21	kworker/u4:7+events_unbound	11:35:39	1
1198328	99	71	63,6	cyclictest	1210995	-21	kworker/u4:15+events_unbound	07:59:47	0
1198328	99	71	63,6	cyclictest	1205941	-21	kworker/u4:4+events_unbound	07:44:45	0
1198328	99	70	63,5	cyclictest	54	-21	kswapd0	12:10:15	0
1198328	99	70	63,5	cyclictest	54	-21	kswapd0	12:10:15	0
1198329	99	69	54,6	cyclictest	1292413	-21	kworker/u4:7+events_unbound	11:05:44	1
1198328	99	69	61,6	cyclictest	54	-21	kswapd0	12:15:31	0
1198328	99	69	61,6	cyclictest	1200790	-21	kworker/u4:14+events_unbound	07:34:42	0
1198329	99	68	61,5	cyclictest	1217332	-21	kworker/u4:6+events_unbound	08:34:57	1
1198329	99	68	53,6	cyclictest	1205941	-21	kworker/u4:4+events_unbound	08:19:55	1
1198328	99	68	61,5	cyclictest	1221103	-21	kworker/u4:0+events_unbound	08:39:58	0
1198329	99	67	60,5	cyclictest	54	-21	kswapd0	12:20:11	1
1198328	99	67	60,5	cyclictest	1397152	-21	kworker/u4:6+events_unbound	12:25:38	0
1198328	99	67	57,7	cyclictest	1379034	-21	kworker/u4:5+events_unbound	12:00:33	0
1198328	99	66	45,18	cyclictest	54	-21	kswapd0	09:35:34	0
1198329	99	65	46,7	cyclictest	1347865	-21	kworker/u4:1+events_unbound	11:29:02	1
1198328	99	64	45,11	cyclictest	1226174	-21	kworker/u4:15	09:00:00	0
1198329	99	63	55,6	cyclictest	54	-21	kswapd0	11:50:36	1
1198329	99	63	51,5	cyclictest	1205940	-21	kworker/u4:0+events_unbound	07:49:45	1
1198328	99	63	58,4	cyclictest	1255239	-21	kworker/u4:5+events_unbound	11:05:14	0
1198328	99	63	53,8	cyclictest	54	-21	kswapd0	10:10:00	0
1198328	99	63	53,8	cyclictest	54	-21	kswapd0	10:10:00	0
1198328	99	63	44,6	cyclictest	1297108	-21	kworker/u4:9+events_unbound	10:23:42	0
1198329	99	62	56,4	cyclictest	54	-21	kswapd0	11:15:34	1
1198328	99	62	54,6	cyclictest	1217334	-21	kworker/u4:13+events_unbound	08:24:56	0
1198329	99	61	53,6	cyclictest	1235877	-21	kworker/u4:10+events_unbound	09:30:06	1
1198328	99	60	46,5	cyclictest	1217332	-21	kworker/u4:6+events_unbound	09:14:00	0
1198329	99	59	34,17	cyclictest	0	-21	swapper/1	09:25:33	1
1198328	99	59	51,6	cyclictest	1210995	-21	kworker/u4:15+events_unbound	08:29:56	0
1198328	99	59	41,5	cyclictest	1349675	-21	munin-run	11:20:01	0
1198329	99	58	50,6	cyclictest	1324598	-21	kworker/u4:9+events_unbound	11:03:43	1
1198329	99	58	46,5	cyclictest	1235873	-21	kworker/u4:0+events_unbound	09:20:00	1
1198328	99	58	52,4	cyclictest	1324597	-21	kworker/u4:8+events_unbound	10:55:20	0
1198328	99	58	52,4	cyclictest	1324597	-21	kworker/u4:8+events_unbound	10:55:20	0
1198328	99	58	51,5	cyclictest	54	-21	kswapd0	12:35:29	0
1198328	99	58	50,6	cyclictest	1404630	-21	switchtime	12:20:36	0
1198329	99	57	51,4	cyclictest	54	-21	kswapd0	12:40:01	1
1198329	99	57	49,6	cyclictest	54	-21	kswapd0	11:20:33	1
1198329	99	57	49,6	cyclictest	1383541	-21	kworker/u4:0+events_unbound	12:07:43	1
1198329	99	57	43,7	cyclictest	1301827	-21	kworker/u4:0+events_unbound	10:29:34	1
1198328	99	57	50,5	cyclictest	54	-21	kswapd0	12:05:32	0
1198328	99	57	49,6	cyclictest	1374509	-21	kworker/u4:0+events_unbound	11:51:43	0
1198329	99	56	50,4	cyclictest	1276273	-21	ssh	10:00:16	1
1198329	99	56	48,6	cyclictest	1209739	-21	kworker/u4:11+events_unbound	08:04:50	1
1198329	99	56	46,8	cyclictest	1224932	-21	apt	08:55:00	1
1198328	99	56	47,7	cyclictest	54	-21	kswapd0	09:25:33	0
1198329	99	55	39,5	cyclictest	1255239	-21	kworker/u4:5+events_unbound	09:50:00	1
1198328	99	55	48,5	cyclictest	1290672	-21	gltestperf	10:15:22	0
1198328	99	55	48,5	cyclictest	1200791	-21	kworker/u4:15+events_unbound	07:19:41	0
1198328	99	55	46,7	cyclictest	1247956	-21	systemd	09:30:05	0
1198328	99	55	38,6	cyclictest	1352431	-21	kworker/u4:7+events_unbound	11:34:42	0
1198328	99	55	33,20	cyclictest	0	-21	swapper/0	11:45:34	0
1198329	99	54	47,5	cyclictest	1317754	-21	apt-key	10:45:02	1
1198329	99	54	47,5	cyclictest	1233071	-21	sshd	09:14:28	1
1198329	99	54	46,6	cyclictest	54	-21	kswapd0	12:10:34	1
1198329	99	54	46,6	cyclictest	54	-21	kswapd0	12:10:34	1
1198328	99	54	47,5	cyclictest	54	-21	kswapd0	11:35:38	0
1198328	99	54	47,5	cyclictest	1235873	-21	kworker/u4:0+events_unbound	09:20:33	0
1198328	99	54	47,5	cyclictest	0	-21	swapper/0	09:45:14	0
1198329	99	53	47,5	cyclictest	1383542	-21	kworker/u4:1+events_unbound	11:59:42	1
1198329	99	53	36,5	cyclictest	1310486	-21	kworker/u4:12+events_unbound	10:39:00	1
1198329	99	53	30,9	cyclictest	1315905	-21	kworker/u4:1+events_unbound	10:55:22	1
1198329	99	53	30,9	cyclictest	1315905	-21	kworker/u4:1+events_unbound	10:55:22	1
1198328	99	53	44,7	cyclictest	1255239	-21	kworker/u4:5+events_unbound	10:43:59	0
1198329	99	52	45,5	cyclictest	1205942	-21	kworker/u4:6+events_unbound	07:54:45	1
1198329	99	52	44,6	cyclictest	1227455	-21	kworker/u4:1	09:05:00	1
1198329	99	52	44,6	cyclictest	1227455	-21	kworker/u4:1	09:05:00	1
1198329	99	52	44,6	cyclictest	1217346	-21	systemd	08:24:54	1
1198329	99	52	43,7	cyclictest	54	-21	kswapd0	09:25:00	1
1198328	99	52	46,4	cyclictest	1304037	-21	sh	10:30:15	0
1198328	99	52	45,5	cyclictest	1235875	-21	kworker/u4:8+events_unbound	09:19:43	0
1198328	99	52	44,6	cyclictest	54	-21	kswapd0	10:25:12	0
1198328	99	52	44,6	cyclictest	1354566	-21	/usr/sbin/munin	11:25:17	0
1198328	99	52	44,5	cyclictest	1317958	-21	aten2__	10:45:20	0
1198329	99	51	43,6	cyclictest	1347865	-21	kworker/u4:1+events_unbound	11:45:45	1
1198329	99	51	43,6	cyclictest	1218601	-21	systemd	08:29:55	1
1198328	99	51	44,5	cyclictest	1310486	-21	kworker/u4:12+events_unbound	10:52:00	0
1198328	99	51	44,5	cyclictest	1310486	-21	kworker/u4:12+events_unbound	10:52:00	0
1198329	99	50	41,7	cyclictest	1271988	-21	fschecks_time	09:55:20	1
1198328	99	50	43,5	cyclictest	1381673	-21	basename	11:55:20	0
1198328	99	50	42,6	cyclictest	1228694	-21	kworker/u4:4+events_unbound	09:07:43	0
1198328	99	50	41,6	cyclictest	0	-21	swapper/0	09:50:34	0
1198328	99	50	32,8	cyclictest	1336123	-21	latency_hist	11:05:01	0
1198329	99	49	42,5	cyclictest	1299092	-21	sh	10:25:02	1
1198329	99	49	40,6	cyclictest	1323546	-21	switchtime	10:50:37	1
1198329	99	49	40,6	cyclictest	1323546	-21	switchtime	10:50:37	1
1198328	99	49	42,5	cyclictest	1285473	-21	sshd	10:10:12	0
1198329	99	48	41,5	cyclictest	1287690	-21	kworker/u4:3+events_unbound	10:17:00	1
1198329	99	48	24,6	cyclictest	1198257	-21	kworker/u4:8+events_unbound	07:10:12	1
1198328	99	48	41,5	cyclictest	0	-21	swapper/0	11:20:30	0
1198328	99	48	39,7	cyclictest	1227444	-21	systemd	09:05:00	0
1198328	99	48	39,7	cyclictest	1227444	-21	systemd	09:05:00	0
1198329	99	47	41,4	cyclictest	1414617	-21	kworker/u4:7+events_unbound	12:32:59	1
1198328	99	47	40,5	cyclictest	1202122	-21	munin-plugin-st	07:25:01	0
1198328	99	47	39,6	cyclictest	1207227	-21	kworker/u4:9+events_unbound	07:45:44	0
1198329	99	46	34,8	cyclictest	1222775	-21	df	08:45:15	1
1198328	99	46	40,4	cyclictest	0	-21	swapper/0	11:10:28	0
1198328	99	46	39,5	cyclictest	1209739	-21	kworker/u4:11+events_unbound	08:03:59	0
1198329	99	45	38,5	cyclictest	1369075	-21	/usr/sbin/munin	11:40:40	1
1198329	99	45	37,6	cyclictest	1267941	-21	mailstats	09:50:36	1
1198328	99	45	38,5	cyclictest	1410096	-21	kworker/u4:3-rpciod	12:30:33	0
1198328	99	45	27,15	cyclictest	1203338	-21	kworker/u4:1-xprtiod	08:20:00	0
1198328	99	45	25,17	cyclictest	0	-21	swapper/0	08:35:00	0
1198329	99	44	37,5	cyclictest	1227874	-21	fschecks_count	09:05:19	1
1198329	99	44	37,5	cyclictest	1200792	-21	kworker/u4:17+events_unbound	07:21:00	1
1198329	99	44	36,5	cyclictest	1383542	-21	kworker/u4:1-rpciod	12:00:28	1
1198329	99	44	35,7	cyclictest	1408979	-21	processes	12:25:36	1
1198329	99	44	31,6	cyclictest	1203338	-21	kworker/u4:1+events_unbound	08:41:58	1
1198329	99	43	35,5	cyclictest	1203338	-21	kworker/u4:1-rpciod	07:35:01	1
1198329	99	42	33,6	cyclictest	0	-21	swapper/1	07:15:33	1
1198329	99	42	29,10	cyclictest	1210621	-21	ps	07:55:34	1
1198329	99	42	21,11	cyclictest	17	1	rcu_preempt	07:40:15	1
1198328	99	42	35,5	cyclictest	1208744	-21	aten_r7power_en	07:50:16	0
1198328	99	41	31,7	cyclictest	1203366	-21	kworker/u4:4	07:30:00	0
1198328	99	40	33,5	cyclictest	1212535	-21	aten_r7power_en	08:05:16	0
1198328	99	38	26,5	cyclictest	1205941	-21	kworker/u4:4+events_unbound	08:10:44	0
1198329	99	37	30,5	cyclictest	1220415	-21	gltestperf	08:35:21	1
1198329	99	33	25,5	cyclictest	0	-21	swapper/1	07:10:01	1
1198328	99	33	26,5	cyclictest	0	-21	swapper/0	07:10:00	0
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17