OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #7, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system

Characteristics of the 792 highest latencies:

System rack7slot2.osadl.org (updated Fri Jan 16, 2026 12:44:08)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
1107505	99	126	101,19	cyclictest	54	-21	kswapd0	10:25:14	0
1107505	99	116	92,15	cyclictest	54	-21	kswapd0	11:20:07	0
1107505	99	87	77,7	cyclictest	1276380	-21	kworker/u4:15+events_unbound	12:10:19	0
1107505	99	83	74,6	cyclictest	1222136	-21	kworker/u4:4+events_unbound	10:50:02	0
1107507	99	73	66,4	cyclictest	0	-21	swapper/1	12:30:34	1
1107507	99	72	64,6	cyclictest	1254015	-21	unixbench_singl	11:25:41	1
1107505	99	69	62,5	cyclictest	877220	-21	kworker/u4:7+events_unbound	07:30:02	0
1107505	99	67	59,6	cyclictest	1121357	-21	kworker/u4:2+events_unbound	08:14:32	0
1107505	99	64	53,8	cyclictest	1160938	-21	sh	09:35:34	0
1107507	99	63	53,7	cyclictest	1226456	-21	kworker/u4:7+events_unbound	11:00:04	1
1107507	99	62	54,5	cyclictest	1113773	-21	kworker/u4:2+events_unbound	07:40:00	1
1107507	99	62	41,16	cyclictest	1170312	-21	ssh	09:46:03	1
1107507	99	61	51,6	cyclictest	0	-21	swapper/1	10:25:37	1
1107507	99	60	55,3	cyclictest	1144882	-21	kworker/u4:1+events_unbound	09:39:48	1
1107505	99	60	50,8	cyclictest	1153745	-21	kworker/u4:4+events_unbound	09:29:46	0
1107507	99	59	50,7	cyclictest	1297222	-21	kworker/u4:11+events_unbound	12:20:21	1
1107505	99	58	36,8	cyclictest	1177372	-21	latency_hist	09:55:03	0
1107507	99	57	45,8	cyclictest	0	-21	swapper/1	11:10:11	1
1107507	99	56	37,10	cyclictest	0	-21	swapper/1	10:35:00	1
1107505	99	56	37,6	cyclictest	1131585	-21	munin-plugin-st	08:44:59	0
1107505	99	56	22,24	cyclictest	255	50	irq/23-42890000.ethernet	10:10:12	0
1107505	99	55	45,7	cyclictest	1263453	-21	kworker/u4:1+events_unbound	12:15:20	0
1107505	99	55	35,17	cyclictest	1126420	-21	kworker/u4:9+rpciod	08:25:18	0
1107507	99	54	46,6	cyclictest	54	-21	kswapd0	09:55:12	1
1107505	99	54	47,5	cyclictest	1213499	-21	kworker/u4:2+events_unbound	10:39:58	0
1107505	99	54	11,19	cyclictest	15	1	ktimers/0	10:45:31	0
1107507	99	53	48,3	cyclictest	1259151	-21	kworker/u4:10+events_unbound	11:40:12	1
1107507	99	53	46,5	cyclictest	1263453	-21	kworker/u4:1+events_unbound	11:49:59	1
1107507	99	53	44,6	cyclictest	1129018	-21	dpkg	08:35:01	1
1107507	99	53	43,8	cyclictest	1150980	-21	latency_hist	09:25:01	1
1107507	99	53	40,9	cyclictest	1312892	-21	mailstats	12:35:37	1
1107505	99	53	42,8	cyclictest	1254069	-21	/usr/sbin/munin	11:25:43	0
1107505	99	53	41,7	cyclictest	1245068	-21	kworker/u4:10	11:15:00	0
1107507	99	52	41,8	cyclictest	0	-21	swapper/1	10:50:01	1
1107505	99	52	5,44	cyclictest	1178154	-21	sh	09:55:30	0
1107505	99	52	44,6	cyclictest	1229257	-21	proc_pri	10:55:34	0
1107505	99	52	42,6	cyclictest	0	-21	swapper/0	12:00:36	0
1107507	99	51	44,5	cyclictest	1135359	-21	sh	09:00:00	1
1107507	99	51	42,5	cyclictest	0	-21	swapper/1	10:55:24	1
1107507	99	51	41,8	cyclictest	1125651	-21	fschecks_count	08:20:18	1
1107507	99	51	25,20	cyclictest	0	-21	swapper/1	11:55:22	1
1107505	99	51	44,5	cyclictest	1240927	-21	sed	11:10:01	0
1107505	99	51	44,5	cyclictest	1240927	-21	sed	11:10:01	0
1107505	99	51	42,5	cyclictest	1165538	-21	/usr/sbin/munin	09:40:36	0
1107505	99	51	41,6	cyclictest	1139006	-21	ld	09:10:29	0
1107507	99	50	43,5	cyclictest	1286732	-21	sshd	12:05:16	1
1107507	99	50	43,5	cyclictest	1195542	-21	sendmail_mailqu	10:15:35	1
1107507	99	50	43,5	cyclictest	1182303	-21	gltestperf	10:00:26	1
1107507	99	50	42,6	cyclictest	1107502	-21	cyclictest	10:41:17	1
1107507	99	50	41,7	cyclictest	1121369	-21	systemd	08:04:30	1
1107505	99	50	43,5	cyclictest	1245542	-21	/usr/sbin/munin	11:15:17	0
1107505	99	50	43,5	cyclictest	1194892	-21	/usr/sbin/munin	10:15:18	0
1107505	99	50	42,6	cyclictest	1142967	-21	/usr/sbin/munin	09:15:20	0
1107505	99	50	41,7	cyclictest	1308248	-21	gltestperf	12:30:26	0
1107507	99	49	43,4	cyclictest	1304004	-21	sshd	12:25:22	1
1107507	99	49	42,5	cyclictest	1295064	-21	aten_r7power_cu	12:15:14	1
1107507	99	49	42,5	cyclictest	1245784	-21	fschecks_time	11:15:20	1
1107507	99	49	40,6	cyclictest	1286419	-21	latency_hist	12:05:00	1
1107507	99	49	37,9	cyclictest	1124755	-21	ps	08:15:39	1
1107507	99	49	33,12	cyclictest	0	-21	swapper/1	11:05:23	1
1107507	99	49	33,12	cyclictest	0	-21	swapper/1	11:05:23	1
1107505	99	49	41,5	cyclictest	1182659	-21	ntp_states	10:00:38	0
1107507	99	48	41,5	cyclictest	1146708	-21	latency_hist	09:20:02	1
1107507	99	48	40,5	cyclictest	1262353	-21	diskmemload	11:35:37	1
1107507	99	48	39,6	cyclictest	1156764	-21	ssh	09:30:37	1
1107507	99	48	36,9	cyclictest	1109988	-21	systemd	07:19:21	1
1107505	99	48	42,4	cyclictest	1138008	-21	latency_hist	09:10:02	0
1107505	99	48	41,5	cyclictest	1269838	-21	aten_r7power_vo	11:45:18	0
1107505	99	48	41,5	cyclictest	1266479	-21	xargs	11:40:36	0
1107505	99	48	41,5	cyclictest	1258174	-21	sed	11:30:40	0
1107505	99	48	39,5	cyclictest	1208344	-21	/usr/sbin/munin	10:30:35	0
1107505	99	48	34,12	cyclictest	1110026	-21	munin-run	07:20:01	0
1107507	99	47	40,5	cyclictest	1131940	-21	sed	08:45:16	1
1107507	99	47	40,5	cyclictest	1128963	-21	apt-get	08:35:06	1
1107507	99	47	38,7	cyclictest	1260921	-21	diskmemload	11:35:01	1
1107507	99	47	38,6	cyclictest	1190780	-21	tune2fs	10:10:24	1
1107507	99	47	37,7	cyclictest	0	-21	swapper/1	11:50:14	1
1107505	99	47	41,4	cyclictest	1299839	-21	kernelversion	12:20:21	0
1107505	99	47	40,5	cyclictest	1307449	-21	apt-config	12:30:02	0
1107505	99	47	40,5	cyclictest	1274353	-21	tail	11:50:19	0
1107505	99	47	37,8	cyclictest	1278170	-21	aten_r7power_cu	11:55:14	0
1107505	99	47	37,7	cyclictest	1116808	-21	fschecks_time	07:45:18	0
1107505	99	47	26,19	cyclictest	0	-21	swapper/0	10:20:09	0
1107507	99	46	36,8	cyclictest	1173589	-21	gltestperf	09:50:18	1
1107507	99	46	36,8	cyclictest	1137893	-21	diskmemload	09:10:34	1
1107507	99	46	36,7	cyclictest	1164530	-21	aten_r7power_po	09:40:15	1
1107507	99	46	36,7	cyclictest	1108871	-21	latency_hist	07:15:01	1
1107507	99	46	29,14	cyclictest	0	-21	swapper/1	11:20:07	1
1107505	99	46	37,6	cyclictest	1315928	-21	sed	12:40:02	0
1107505	99	46	36,7	cyclictest	1213499	-21	kworker/u4:2+rpciod	10:40:15	0
1107505	99	46	27,5	cyclictest	0	-21	swapper/0	09:20:11	0
1107507	99	45	5,30	cyclictest	1186704	-21	ssh	10:05:25	1
1107507	99	45	39,4	cyclictest	1199831	-21	sendmail_mailqu	10:20:34	1
1107507	99	45	38,5	cyclictest	1151339	-21	aten2__	09:25:18	1
1107507	99	45	37,6	cyclictest	1133439	-21	gltestperf	08:50:21	1
1107507	99	45	37,6	cyclictest	1116347	-21	apt	07:45:01	1
1107507	99	45	37,5	cyclictest	1116310	-21	kworker/u4:12-rpciod	07:45:34	1
1107507	99	45	35,7	cyclictest	1122460	-21	timerwakeupswit	08:05:36	1
1107507	99	45	34,9	cyclictest	1123106	-21	fschecks_count	08:10:18	1
1107507	99	45	33,9	cyclictest	0	-21	swapper/1	10:55:01	1
1107507	99	45	33,7	cyclictest	0	-21	swapper/1	12:10:11	1
1107505	99	45	38,5	cyclictest	1169490	-21	ntpq	09:45:28	0
1107505	99	45	38,5	cyclictest	1130336	-21	awk	08:40:01	0
1107505	99	45	37,5	cyclictest	1233402	-21	gcc	11:00:32	0
1107505	99	45	37,5	cyclictest	1119338	-21	fschecks_time	07:55:19	0
1107505	99	45	26,16	cyclictest	0	-21	swapper/0	09:30:34	0
1107505	99	44	37,5	cyclictest	1261740	-21	kernelversion	11:35:20	0
1107505	99	44	29,13	cyclictest	54	-21	kswapd0	12:05:19	0
1107507	99	43	36,5	cyclictest	1127714	-21	munin-plugin-st	08:30:00	1
1107507	99	43	33,6	cyclictest	1110932	-21	sendmail_mailqu	07:20:34	1
1107507	99	43	28,12	cyclictest	0	-21	swapper/1	09:00:15	1
1107507	99	43	2,32	cyclictest	1212085	-21	sh	10:35:30	1
1107505	99	43	36,5	cyclictest	1135361	-21	munin-plugin-st	08:59:59	0
1107505	99	43	36,5	cyclictest	1133326	-21	fschecks_time	08:50:21	0
1107505	99	43	35,5	cyclictest	1187239	-21	unixbench_multi	10:05:37	0
1107505	99	43	34,7	cyclictest	1127965	-21	aten_r7power_po	08:30:17	0
1107507	99	42	28,11	cyclictest	0	-21	swapper/1	07:30:40	1
1107505	99	42	35,5	cyclictest	1114412	-21	head	07:35:18	0
1107505	99	42	35,5	cyclictest	1113773	-21	kworker/u4:2+rpciod	07:40:18	0
1107505	99	42	34,6	cyclictest	1108608	-21	unixbench_multi	07:10:35	0
1107507	99	41	34,5	cyclictest	1111824	-21	gltestperf	07:25:16	1
1107507	99	41	28,10	cyclictest	0	-21	swapper/1	09:05:19	1
1107507	99	40	31,6	cyclictest	0	-21	swapper/1	08:40:20	1
1107507	99	40	25,13	cyclictest	0	-21	swapper/1	07:50:21	1
1107505	99	40	31,7	cyclictest	1117841	-21	aten_r7power_en	07:50:14	0
1107505	99	40	30,7	cyclictest	1111287	-21	kworker/u4:0	07:24:59	0
1107505	99	40	30,7	cyclictest	0	-21	swapper/0	08:45:20	0
1107505	99	40	30,7	cyclictest	0	-21	swapper/0	08:20:17	0
1107505	99	39	32,5	cyclictest	1120599	-21	fschecks_time	08:00:19	0
1107505	99	39	30,7	cyclictest	1107590	-21	latency_hist	07:10:01	0
1107505	99	39	29,8	cyclictest	1136524	-21	unixbench_multi	09:00:36	0
1107505	99	39	25,11	cyclictest	1124237	-21	cpu	08:15:22	0
1107507	99	38	32,4	cyclictest	1109980	-21	kworker/u4:8-rpciod	08:00:01	1
1107505	99	38	28,7	cyclictest	0	-21	swapper/0	08:10:00	0
1107505	99	36	29,5	cyclictest	0	-21	swapper/0	07:35:00	0
1107507	99	34	26,6	cyclictest	879984	-21	kworker/u4:4+events_unbound	07:10:01	1
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17