OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #7, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system

Characteristics of the 376 highest latencies:

System rack7slot2.osadl.org (updated Wed Jul 08, 2026 00:44:10)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
4128138	99	107	91,7	cyclictest	4147441	-21	kworker/u4:9+events_unbound	20:31:46	1
4128135	99	107	94,7	cyclictest	86846	-21	kworker/u4:16+events_unbound	23:27:10	0
4128135	99	106	94,10	cyclictest	25375	-21	kworker/u4:9+events_unbound	22:42:03	0
4128135	99	102	95,5	cyclictest	38176	-21	kworker/u4:11+events_unbound	22:37:04	0
4128135	99	101	93,6	cyclictest	7398	-21	kworker/u4:7+events_unbound	21:56:58	0
4128135	99	101	93,6	cyclictest	7398	-21	kworker/u4:7+events_unbound	21:56:58	0
4128135	99	97	90,5	cyclictest	95743	-21	kworker/u4:6+events_unbound	23:52:15	0
4128138	99	94	79,6	cyclictest	2952	-21	kworker/u4:6+events_unbound	21:51:58	1
4128138	99	93	86,5	cyclictest	4193085	-21	kworker/u4:3+events_unbound	22:07:00	1
4128135	99	93	87,4	cyclictest	4139672	-21	kworker/u4:12+events_unbound	20:01:43	0
4128135	99	93	85,6	cyclictest	38176	-21	kworker/u4:11+events_unbound	22:32:02	0
4128138	99	92	84,6	cyclictest	4147428	-21	kworker/u4:0+events_unbound	20:21:46	1
4128138	99	92	84,6	cyclictest	4125631	-21	kworker/u4:0+events_unbound	19:16:36	1
4128138	99	91	75,6	cyclictest	4193084	-21	kworker/u4:2+events_unbound	22:01:59	1
4128135	99	91	84,5	cyclictest	95743	-21	kworker/u4:6+events_unbound	23:47:14	0
4128138	99	90	82,6	cyclictest	100237	-21	kworker/u4:3+events_unbound	23:42:13	1
4128135	99	90	83,5	cyclictest	68908	-21	kworker/u4:7+events_unbound	23:17:08	0
4128135	99	87	74,7	cyclictest	4128072	-21	kworker/u4:9+events_unbound	19:26:35	0
4128138	99	86	78,6	cyclictest	4174890	-21	kworker/u4:1+events_unbound	21:46:58	1
4128138	99	84	76,6	cyclictest	130796	-21	kworker/u4:2+events_unbound	00:17:18	1
4128138	99	83	68,5	cyclictest	4133210	-21	kworker/u4:15+events_unbound	19:36:40	1
4128135	99	82	75,5	cyclictest	68908	-21	kworker/u4:7+events_unbound	23:07:08	0
4128135	99	78	71,5	cyclictest	4128072	-21	kworker/u4:9+events_unbound	19:46:41	0
4128138	99	77	69,6	cyclictest	4139672	-21	kworker/u4:12+events_unbound	20:36:47	1
4128135	99	77	70,5	cyclictest	4138314	-21	kworker/u4:1+events_unbound	20:56:51	0
4128135	99	75	67,6	cyclictest	4139672	-21	kworker/u4:12+events_unbound	20:06:44	0
4128135	99	74	67,5	cyclictest	4133210	-21	kworker/u4:15+events_unbound	19:31:39	0
4128135	99	74	67,5	cyclictest	144273	-21	kworker/u4:3+events_unbound	00:32:21	0
4128135	99	74	67,5	cyclictest	144273	-21	kworker/u4:3+events_unbound	00:32:21	0
4128135	99	72	65,5	cyclictest	4147440	-21	kworker/u4:8+events_unbound	20:26:44	0
4128138	99	71	63,6	cyclictest	4147428	-21	kworker/u4:0+events_unbound	20:51:51	1
4128135	99	71	62,6	cyclictest	4149981	-21	kworker/u4:5+events_unbound	21:06:51	0
4128135	99	68	60,6	cyclictest	4156358	-21	kworker/u4:6+events_unbound	21:40:04	0
4128135	99	68	60,6	cyclictest	4139661	-21	kworker/u4:3+events_unbound	19:51:41	0
4128135	99	68	60,6	cyclictest	4128072	-21	kworker/u4:9+events_unbound	19:41:39	0
4128138	99	66	59,5	cyclictest	4129366	-21	kworker/u4:11+events_unbound	20:11:44	1
4128138	99	65	58,5	cyclictest	4139672	-21	kworker/u4:12+events_unbound	20:16:45	1
4128138	99	65	58,5	cyclictest	4129365	-21	kworker/u4:10+events_unbound	19:56:42	1
4128135	99	65	58,5	cyclictest	4128062	-21	kworker/u4:3+events_unbound	19:21:37	0
4128135	99	62	55,5	cyclictest	71996	-21	fschecks_count	23:10:25	0
4128135	99	61	54,5	cyclictest	135285	-21	kworker/u4:4+events_unbound	00:25:02	0
4128135	99	60	50,8	cyclictest	109146	-21	kworker/u4:8+events_unbound	00:02:16	0
4128135	99	58	45,12	cyclictest	82387	-21	kworker/u4:15+events_unbound	23:23:46	0
4128138	99	57	49,6	cyclictest	104685	-21	sd-accept	23:47:12	1
4128135	99	57	49,6	cyclictest	53	-21	kswapd0	23:05:01	0
4128135	99	57	47,8	cyclictest	20888	-21	kworker/u4:1+events_unbound	22:25:13	0
4128138	99	56	50,5	cyclictest	109143	-21	kworker/u4:0+events_unbound	23:53:45	1
4128135	99	56	49,5	cyclictest	95743	-21	kworker/u4:6+events_unbound	00:20:18	0
4128135	99	56	49,5	cyclictest	95743	-21	kworker/u4:6+events_unbound	00:20:18	0
4128135	99	56	49,5	cyclictest	91303	-21	kworker/u4:3+events_unbound	23:35:45	0
4128135	99	56	49,5	cyclictest	51645	-21	kworker/u4:1+events_unbound	22:50:04	0
4128135	99	56	45,5	cyclictest	4156358	-21	kworker/u4:6+events_unbound	21:25:03	0
4128138	99	54	47,5	cyclictest	60600	-21	kworker/u4:0+events_unbound	23:13:46	1
4128138	99	54	47,5	cyclictest	27	1	ktimers/1	23:31:50	1
4128138	99	54	46,5	cyclictest	4153027	-21	fschecks_count	20:45:20	1
4128135	99	54	37,14	cyclictest	111442	-21	entropy	23:55:19	0
4128135	99	54	34,18	cyclictest	23435	-21	fschecks_count	22:15:21	0
4128138	99	53	46,5	cyclictest	4160019	-21	/usr/sbin/munin	21:10:29	1
4128135	99	53	46,5	cyclictest	59192	-21	/usr/sbin/munin	22:55:33	0
4128135	99	53	46,5	cyclictest	15431	-21	grep	22:05:38	0
4128135	99	53	32,19	cyclictest	4191051	-21	tail	21:45:20	0
4128138	99	52	45,5	cyclictest	4177570	-21	gltestperf	21:30:21	1
4128138	99	52	44,6	cyclictest	4135748	-21	kworker/u4:2+events_unbound	21:01:52	1
4128138	99	52	43,7	cyclictest	65098	-21	kworker/u4:5+events_unbound	23:29:45	1
4128135	99	52	47,3	cyclictest	4128061	-21	kworker/u4:2+events_unbound	19:11:34	0
4128135	99	52	47,3	cyclictest	4128061	-21	kworker/u4:2+events_unbound	19:11:33	0
4128135	99	52	43,6	cyclictest	146769	-21	fschecks_count	00:35:19	0
4128138	99	51	44,5	cyclictest	23171	-21	aten_r7power_cu	22:15:17	1
4128138	99	51	44,5	cyclictest	111902	-21	ld	23:55:28	1
4128138	99	51	42,6	cyclictest	139770	-21	kworker/u4:8-rpciod	00:27:32	1
4128135	99	51	45,4	cyclictest	4193082	-21	kworker/u4:0+events_unbound	22:12:01	0
4128135	99	51	45,4	cyclictest	4129366	-21	kworker/u4:11+events_unbound	19:15:03	0
4128135	99	51	38,9	cyclictest	31645	-21	modprobe	22:25:01	0
4128135	99	51	38,10	cyclictest	4177543	-21	fschecks_time	21:30:20	0
4128138	99	50	44,5	cyclictest	40753	-21	aten2__	22:35:15	1
4128138	99	50	43,5	cyclictest	4155471	-21	sed	20:55:31	1
4128138	99	50	43,5	cyclictest	4143173	-21	sendmail_mailqu	20:05:43	1
4128138	99	50	43,5	cyclictest	0	-21	swapper/1	22:45:30	1
4128138	99	50	42,5	cyclictest	0	-21	swapper/1	23:40:01	1
4128138	99	50	41,6	cyclictest	0	-21	swapper/1	00:05:14	1
4128138	99	50	40,7	cyclictest	0	-21	swapper/1	22:20:27	1
4128138	99	50	39,8	cyclictest	4164046	-21	uniq	21:15:21	1
4128138	99	50	22,5	cyclictest	26	1	rcuc/1	21:20:14	1
4128135	99	50	40,7	cyclictest	0	-21	swapper/0	00:15:29	0
4128135	99	50	39,9	cyclictest	9905	-21	/usr/sbin/munin	22:00:19	0
4128138	99	49	42,5	cyclictest	6376	-21	ntp_offset	21:55:48	1
4128138	99	49	42,5	cyclictest	6376	-21	ntp_offset	21:55:48	1
4128138	99	49	42,5	cyclictest	4129366	-21	kworker/u4:11+events_unbound	19:40:03	1
4128138	99	49	42,5	cyclictest	36202	-21	latency_hist	22:30:02	1
4128138	99	49	41,6	cyclictest	4129366	-21	kworker/u4:11+events_unbound	20:25:03	1
4128138	99	49	38,9	cyclictest	4181746	-21	aten_r7power_po	21:35:16	1
4128135	99	49	41,5	cyclictest	4159617	-21	fschecks_count	21:10:20	0
4128138	99	48	41,5	cyclictest	54221	-21	grep	22:50:18	1
4128138	99	48	41,5	cyclictest	4175288	-21	sshd	21:27:40	1
4128138	99	48	41,5	cyclictest	4151785	-21	fschecks_time	20:40:19	1
4128138	99	48	41,5	cyclictest	4142278	-21	idleruntime-cro	20:05:00	1
4128138	99	48	41,5	cyclictest	4128071	-21	kworker/u4:8+events_unbound	19:10:02	1
4128138	99	48	41,5	cyclictest	4128071	-21	kworker/u4:8+events_unbound	19:10:02	1
4128138	99	48	41,4	cyclictest	0	-21	swapper/1	23:15:37	1
4128138	99	48	40,6	cyclictest	0	-21	swapper/1	00:30:15	1
4128138	99	48	40,6	cyclictest	0	-21	swapper/1	00:30:14	1
4128138	99	48	40,5	cyclictest	4159012	-21	date	21:10:01	1
4128138	99	48	39,7	cyclictest	67332	-21	aten_r7power_po	23:05:17	1
4128138	99	48	39,5	cyclictest	0	-21	swapper/1	23:20:26	1
4128135	99	48	42,4	cyclictest	0	-21	swapper/0	23:40:35	0
4128135	99	48	39,7	cyclictest	89538	-21	/usr/sbin/munin	23:30:19	0
4128135	99	48	37,5	cyclictest	25375	-21	kworker/u4:9+events_unbound	22:48:20	0
4128138	99	47	7,22	cyclictest	4158956	-21	diskmemload	22:40:27	1
4128138	99	47	41,4	cyclictest	4133235	-21	latency_hist	19:30:01	1
4128138	99	47	40,5	cyclictest	4129365	-21	kworker/u4:10+events_unbound	19:45:02	1
4128138	99	47	40,5	cyclictest	22858	-21	latency_hist	22:15:02	1
4128138	99	47	38,6	cyclictest	0	-21	swapper/1	00:35:15	1
4128138	99	46	39,5	cyclictest	62930	-21	grep	23:00:17	1
4128138	99	46	39,5	cyclictest	124355	-21	acpi	00:10:12	1
4128138	99	46	37,7	cyclictest	115749	-21	aten_r7power_vo	00:00:17	1
4128135	99	46	39,5	cyclictest	4139672	-21	kworker/u4:12+events_unbound	20:20:03	0
4128135	99	46	36,8	cyclictest	4182008	-21	fschecks_time	21:35:20	0
4128135	99	46	34,7	cyclictest	122523	-21	kworker/u4:15+events_unbound	00:10:06	0
4128135	99	46	28,9	cyclictest	4149451	-21	ntpq	20:30:26	0
4128135	99	46	27,4	cyclictest	0	-21	swapper/0	21:53:40	0
4128135	99	46	17,23	cyclictest	17	1	rcu_preempt	20:10:36	0
4128138	99	45	38,4	cyclictest	0	-21	swapper/1	00:20:25	1
4128138	99	45	38,4	cyclictest	0	-21	swapper/1	00:20:24	1
4128138	99	45	31,11	cyclictest	4138852	-21	grep	19:50:24	1
4128135	99	45	39,4	cyclictest	4147428	-21	kworker/u4:0+events_unbound	20:35:03	0
4128135	99	45	38,5	cyclictest	4164154	-21	tune2fs	21:15:20	0
4128138	99	44	36,6	cyclictest	0	-21	swapper/1	22:57:05	1
4128138	99	44	33,8	cyclictest	4186229	-21	aten_r7power_po	21:40:17	1
4128138	99	44	16,14	cyclictest	27	1	ktimers/1	22:30:23	1
4128135	99	44	36,6	cyclictest	4135702	-21	unixbench_singl	19:35:39	0
4128135	99	44	34,8	cyclictest	4169348	-21	xargs	21:20:41	0
4128135	99	44	34,8	cyclictest	120429	-21	gltestperf	00:05:21	0
4128138	99	43	36,4	cyclictest	0	-21	swapper/1	19:35:00	1
4128135	99	43	36,5	cyclictest	4146302	-21	latency_hist	20:20:02	0
4128135	99	43	34,7	cyclictest	4153813	-21	systemd	20:46:48	0
4128135	99	43	34,7	cyclictest	4151795	-21	fschecks_time	20:40:19	0
4128135	99	41	32,7	cyclictest	4139944	-21	aten_r7power_en	19:55:18	0
4128135	99	40	32,6	cyclictest	4143509	-21	kworker/u4:3-nfsiod	21:00:29	0
4128135	99	40	10,23	cyclictest	14	-21	ksoftirqd/0	20:50:18	0
4128138	99	39	32,4	cyclictest	0	-21	swapper/1	19:25:01	1
4128135	99	38	6,26	cyclictest	4128133	-21	cyclictest	19:10:01	0
4128138	99	36	27,5	cyclictest	0	-21	swapper/1	19:10:01	1
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17