OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #7, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system

Characteristics of the highest latencies:

System rack7slot2.osadl.org (updated Sat Oct 25, 2025 12:44:09)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
590258	99	128	121,5	cyclictest	713702	-21	kworker/u4:9+events_unbound	11:25:55	0
590258	99	126	117,7	cyclictest	759152	-21	kworker/u4:12+events_unbound	12:31:08	0
590258	99	124	117,5	cyclictest	218092	-21	kworker/u4:3+events_unbound	07:20:09	0
590258	99	122	114,6	cyclictest	605790	-21	kworker/u4:13+events_unbound	08:40:42	0
590258	99	121	113,6	cyclictest	618614	-21	kworker/u4:5+events_unbound	09:00:46	0
590258	99	121	106,6	cyclictest	665364	-21	kworker/u4:2+events_unbound	10:35:44	0
590258	99	110	103,5	cyclictest	751025	-21	kworker/u4:14+events_unbound	12:36:06	0
590258	99	109	101,6	cyclictest	672034	-21	kworker/u4:7+events_unbound	10:55:51	0
590258	99	101	82,7	cyclictest	595441	-21	kworker/u4:15+events_unbound	07:55:09	0
590258	99	100	92,6	cyclictest	613484	-21	kworker/u4:20+events_unbound	08:50:44	0
590258	99	98	84,6	cyclictest	601932	-21	kworker/u4:8+events_unbound	08:45:23	0
590258	99	97	91,5	cyclictest	742949	-21	kworker/u4:1+events_unbound	12:16:04	0
590258	99	97	91,4	cyclictest	658121	-21	kworker/u4:0+events_unbound	10:25:59	0
590258	99	95	76,11	cyclictest	0	-21	swapper/0	09:35:08	0
590260	99	92	84,5	cyclictest	586525	-21	kworker/u4:2+events_unbound	07:40:23	1
590260	99	92	79,6	cyclictest	0	-21	swapper/1	11:50:33	1
590260	99	91	84,5	cyclictest	755382	-21	kworker/u4:3+events_unbound	12:21:05	1
590260	99	91	83,5	cyclictest	612207	-21	kworker/u4:18+events_unbound	08:40:21	1
590258	99	90	83,5	cyclictest	599379	-21	kworker/u4:4+events_unbound	08:15:12	0
590260	99	88	77,8	cyclictest	621193	-21	kworker/u4:14+events_unbound	09:25:26	1
590258	99	85	78,5	cyclictest	742950	-21	kworker/u4:2+events_unbound	12:26:06	0
590260	99	84	73,8	cyclictest	623605	-21	kworker/u4:2+events_unbound	09:15:48	1
590260	99	83	68,11	cyclictest	547015	-21	kworker/u4:5+events_unbound	07:50:27	1
590258	99	83	75,6	cyclictest	218092	-21	kworker/u4:3+events_unbound	08:55:45	0
590260	99	82	77,3	cyclictest	706180	-21	kworker/u4:5+events_unbound	11:36:15	1
590258	99	79	71,6	cyclictest	600656	-21	kworker/u4:0+events_unbound	07:50:06	0
590258	99	78	71,5	cyclictest	581548	-21	kworker/u4:9+events_unbound	09:20:50	0
590258	99	78	71,5	cyclictest	581548	-21	kworker/u4:9+events_unbound	09:20:49	0
590258	99	78	70,6	cyclictest	661783	-21	kworker/u4:13+events_unbound	10:10:56	0
590258	99	78	4,27	cyclictest	594113	-21	apt-get	07:25:06	0
590260	99	77	64,10	cyclictest	581548	-21	kworker/u4:9+events_unbound	09:20:26	1
590260	99	77	64,10	cyclictest	581548	-21	kworker/u4:9+events_unbound	09:20:25	1
590260	99	75	68,5	cyclictest	607067	-21	kworker/u4:16+events_unbound	08:20:33	1
590258	99	75	68,5	cyclictest	604514	-21	kworker/u4:10+events_unbound	08:05:09	0
590260	99	74	66,6	cyclictest	591488	-21	kworker/u4:8+events_unbound	07:25:09	1
590260	99	74	64,8	cyclictest	612208	-21	kworker/u4:19+events_unbound	09:55:31	1
590260	99	74	33,13	cyclictest	616251	-21	apt-get	08:50:15	1
590260	99	72	65,5	cyclictest	637191	-21	kworker/u4:1+events_unbound	09:35:15	1
590260	99	71	30,14	cyclictest	669741	-21	ssh	10:19:01	1
590260	99	70	30,15	cyclictest	716574	-21	ssh	11:25:59	1
590260	99	70	1,58	cyclictest	712414	-21	ssh	11:20:33	1
590260	99	69	35,14	cyclictest	695424	-21	ssh	10:56:29	1
590260	99	69	34,12	cyclictest	720314	-21	ssh	11:31:19	1
590260	99	69	29,11	cyclictest	739617	-21	ssh	11:57:55	1
590260	99	69	29,11	cyclictest	739617	-21	ssh	11:57:54	1
590258	99	69	61,6	cyclictest	702644	-21	kworker/u4:9+events_unbound	11:15:54	0
590260	99	68	30,11	cyclictest	678542	-21	ssh	10:31:31	1
590260	99	68	28,14	cyclictest	743334	-21	ssh	12:03:05	1
590260	99	68	27,17	cyclictest	688444	-21	ssh	10:46:16	1
590260	99	67	28,14	cyclictest	702320	-21	ssh	11:06:17	1
590260	99	67	28,13	cyclictest	707783	-21	ssh	11:15:03	1
590260	99	67	27,15	cyclictest	700176	-21	ssh	11:04:26	1
590260	99	66	57,5	cyclictest	591487	-21	kworker/u4:6+events_unbound	07:25:00	1
590260	99	66	31,12	cyclictest	728162	-21	ssh	11:42:37	1
590260	99	66	30,11	cyclictest	672011	-21	ssh	10:21:17	1
590260	99	66	28,15	cyclictest	692698	-21	ssh	10:53:24	1
590260	99	66	28,15	cyclictest	647186	-21	ssh	09:46:07	1
590260	99	66	28,14	cyclictest	644036	-21	ssh	09:41:36	1
590260	99	66	26,16	cyclictest	658618	-21	ssh	10:03:16	1
590260	99	66	26,16	cyclictest	658618	-21	ssh	10:03:15	1
590260	99	65	29,13	cyclictest	686053	-21	ssh	10:43:48	1
590260	99	65	29,10	cyclictest	599574	-21	awk	07:45:14	1
590260	99	65	27,14	cyclictest	681475	-21	ssh	10:36:01	1
590260	99	64	28,11	cyclictest	752494	-21	ssh	12:14:37	1
590260	99	64	27,12	cyclictest	730703	-21	ssh	11:45:51	1
590260	99	64	27,11	cyclictest	772514	-21	ssh	12:37:17	1
590260	99	64	25,16	cyclictest	674628	-21	ntpq	10:25:34	1
590260	99	63	53,8	cyclictest	613261	-21	timerandwakeup	08:35:39	1
590260	99	63	29,11	cyclictest	745488	-21	ntpq	12:05:39	1
590260	99	63	27,14	cyclictest	652391	-21	ssh	09:54:51	1
590260	99	63	26,13	cyclictest	597105	-21	awk	07:35:22	1
590260	99	63	25,15	cyclictest	707012	-21	ssh	11:13:50	1
590260	99	63	25,14	cyclictest	662684	-21	ssh	10:09:10	1
590260	99	63	24,13	cyclictest	765215	-21	ssh	12:29:19	1
590258	99	63	58,3	cyclictest	637191	-21	kworker/u4:1+events_unbound	09:40:29	0
590260	99	62	25,14	cyclictest	639775	-21	grep	09:35:45	1
590260	99	62	24,13	cyclictest	623467	-21	ssh	09:12:11	1
590260	99	61	25,13	cyclictest	665427	-21	ssh	10:12:26	1
590260	99	61	25,11	cyclictest	617875	-21	perl	08:55:20	1
590260	99	61	23,15	cyclictest	754773	-21	ssh	12:16:21	1
590258	99	61	56,4	cyclictest	658124	-21	kworker/u4:8+events_unbound	10:30:44	0
590258	99	61	54,5	cyclictest	644152	-21	kworker/u4:7+events_unbound	10:00:03	0
590258	99	61	54,5	cyclictest	644152	-21	kworker/u4:7+events_unbound	10:00:02	0
590260	99	60	28,6	cyclictest	606526	-21	ntpq	08:10:23	1
590258	99	60	54,5	cyclictest	218092	-21	kworker/u4:3+events_unbound	08:25:03	0
590260	99	59	27,10	cyclictest	611792	-21	ntpq	08:30:42	1
590260	99	59	26,11	cyclictest	607773	-21	memory	08:15:24	1
590260	99	58	23,13	cyclictest	605315	-21	ntpq	08:05:31	1
590260	99	58	23,12	cyclictest	767286	-21	users	12:30:42	1
590258	99	57	1,13	cyclictest	598025	-21	apt-get	07:40:06	0
590260	99	56	42,11	cyclictest	596190	-21	ntpq	07:30:26	1
590258	99	56	48,6	cyclictest	621193	-21	kworker/u4:14+events_unbound	09:17:01	0
590258	99	55	50,4	cyclictest	15	1	ktimers/0	11:01:57	0
590258	99	55	50,3	cyclictest	594077	-21	kworker/u4:14+events_unbound	08:10:11	0
590258	99	55	49,5	cyclictest	633745	-21	kworker/u4:15+events_unbound	09:47:36	0
590258	99	55	46,7	cyclictest	398	-21	systemd-logind	11:56:15	0
590258	99	55	46,7	cyclictest	398	-21	systemd-logind	11:56:14	0
590258	99	55	33,19	cyclictest	623605	-21	kworker/u4:2-rpciod	09:25:19	0
590260	99	54	38,10	cyclictest	619468	-21	ntpq	09:00:33	1
590260	99	54	23,10	cyclictest	610491	-21	ntp_states	08:25:33	1
590258	99	54	47,6	cyclictest	591488	-21	kworker/u4:8+events_unbound	07:17:00	0
590258	99	54	30,15	cyclictest	1056	-21	lldpd	10:20:09	0
590260	99	53	23,9	cyclictest	1060	-21	snmpd	07:57:52	1
590260	99	53	21,9	cyclictest	620689	-21	ntpq	09:05:28	1
590258	99	52	46,5	cyclictest	668885	-21	kworker/u4:4+events_unbound	10:51:06	0
590258	99	52	45,5	cyclictest	658123	-21	kworker/u4:6+events_unbound	10:15:04	0
590258	99	52	45,5	cyclictest	601931	-21	kworker/u4:6+events_unbound	09:10:04	0
590258	99	52	36,5	cyclictest	742949	-21	kworker/u4:1+events_unbound	12:14:37	0
590260	99	51	40,8	cyclictest	615647	-21	ntpq	08:45:56	1
590260	99	51	37,11	cyclictest	591695	-21	apt-get	07:15:14	1
590258	99	51	44,5	cyclictest	706180	-21	kworker/u4:5+events_unbound	11:35:02	0
590258	99	51	43,6	cyclictest	595621	-21	systemd	07:30:11	0
590260	99	50	37,10	cyclictest	590980	-21	ntpq	07:10:29	1
590260	99	50	36,8	cyclictest	604532	-21	modprobe	08:05:00	1
590258	99	50	43,5	cyclictest	663123	-21	munin-run	10:10:00	0
590258	99	50	42,6	cyclictest	742949	-21	kworker/u4:1+events_unbound	12:21:46	0
590258	99	50	41,7	cyclictest	742394	-21	systemd	12:01:01	0
590258	99	50	36,5	cyclictest	713703	-21	kworker/u4:14+events_unbound	11:45:19	0
590258	99	50	28,20	cyclictest	642121	-21	latency_hist	09:40:02	0
590258	99	49	41,6	cyclictest	699101	-21	kworker/u4:7+events_unbound	11:06:07	0
590258	99	48	41,5	cyclictest	744615	-21	apt-get	12:05:17	0
590258	99	48	41,5	cyclictest	581548	-21	kworker/u4:9+events_unbound	07:45:02	0
590258	99	47	40,5	cyclictest	598037	-21	sendmail	07:40:00	0
590258	99	47	28,5	cyclictest	0	-21	swapper/0	09:55:28	0
590258	99	47	27,18	cyclictest	620403	-21	fschecks_time	09:05:20	0
590258	99	46	39,5	cyclictest	650005	-21	gcc	09:50:30	0
590258	99	46	38,6	cyclictest	684121	-21	/usr/sbin/munin	10:40:20	0
590258	99	46	23,5	cyclictest	17	1	rcu_preempt	08:30:42	0
590260	99	45	37,6	cyclictest	587741	-21	kworker/u4:10+events_unbound	07:10:01	1
590258	99	45	35,8	cyclictest	704724	-21	gltestperf	11:10:22	0
590258	99	44	37,5	cyclictest	736995	-21	latency_hist	11:55:01	0
590258	99	44	36,6	cyclictest	590524	-21	aten_r7power_po	07:10:18	0
590258	99	43	32,8	cyclictest	721756	-21	cron	11:35:00	0
590258	99	43	26,15	cyclictest	621227	-21	diskmemload	11:20:30	0
590258	99	42	35,5	cyclictest	726050	-21	sed	11:40:18	0
590258	99	42	35,5	cyclictest	668885	-21	kworker/u4:4-rpciod	10:45:21	0
590258	99	40	33,5	cyclictest	612206	-21	kworker/u4:17+events_unbound	08:35:02	0
590258	99	40	31,6	cyclictest	0	-21	swapper/0	08:20:25	0
590258	99	36	16,6	cyclictest	17	1	rcu_preempt	08:00:16	0
590258	99	30	22,6	cyclictest	0	-21	swapper/0	07:10:02	0
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17