OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #7, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system

Characteristics of all highest latencies:

System rack7slot2.osadl.org (updated Mon Jul 20, 2026 00:44:10)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
4159229	99	98	90,6	cyclictest	61745	-21	kworker/u4:16+events_unbound	22:52:25	1
4159228	99	95	88,5	cyclictest	4190019	-21	kworker/u4:4+events_unbound	21:12:08	0
4159229	99	94	86,6	cyclictest	4155422	-21	kworker/u4:0+events_unbound	19:11:50	1
4159229	99	93	80,5	cyclictest	21172	-21	kworker/u4:5+events_unbound	22:37:22	1
4159229	99	93	79,5	cyclictest	4178441	-21	kworker/u4:3+events_unbound	20:37:04	1
4159228	99	93	86,5	cyclictest	4163036	-21	kworker/u4:13+events_unbound	19:26:54	0
4159229	99	91	83,6	cyclictest	4187356	-21	kworker/u4:3+events_unbound	20:57:06	1
4159228	99	91	84,5	cyclictest	52574	-21	kworker/u4:13+events_unbound	22:27:20	0
4159228	99	88	81,5	cyclictest	4182262	-21	kworker/u4:13+events_unbound	20:52:06	0
4159228	99	88	80,6	cyclictest	4172027	-21	kworker/u4:2+events_unbound	20:22:02	0
4159229	99	87	80,5	cyclictest	3907237	-21	kworker/u4:9+events_unbound	19:46:57	1
4159228	99	86	78,6	cyclictest	4175875	-21	kworker/u4:15+events_unbound	20:27:02	0
4159229	99	85	70,6	cyclictest	30319	-21	kworker/u4:2+events_unbound	21:57:17	1
4159229	99	85	70,6	cyclictest	30319	-21	kworker/u4:2+events_unbound	21:57:17	1
4159228	99	84	76,6	cyclictest	4160457	-21	kworker/u4:6+events_unbound	20:07:00	0
4159229	99	83	76,5	cyclictest	4182261	-21	kworker/u4:10+events_unbound	21:07:08	1
4159229	99	83	75,6	cyclictest	4178453	-21	kworker/u4:7+events_unbound	21:02:08	1
4159228	99	83	76,5	cyclictest	4178441	-21	kworker/u4:3+events_unbound	20:32:03	0
4159228	99	83	76,5	cyclictest	4168228	-21	kworker/u4:17+events_unbound	20:01:59	0
4159228	99	78	70,6	cyclictest	4163037	-21	kworker/u4:14+events_unbound	19:31:55	0
4159228	99	78	70,6	cyclictest	4146755	-21	kworker/u4:2+events_unbound	19:36:56	0
4159228	99	77	69,6	cyclictest	4161740	-21	kworker/u4:12+events_unbound	19:21:51	0
4159228	99	77	69,6	cyclictest	4161740	-21	kworker/u4:12+events_unbound	19:21:51	0
4159229	99	76	69,5	cyclictest	4178453	-21	kworker/u4:7+events_unbound	20:47:04	1
4159229	99	74	67,5	cyclictest	4161738	-21	kworker/u4:8+events_unbound	19:51:58	1
4159229	99	72	64,6	cyclictest	4182261	-21	kworker/u4:10+events_unbound	20:42:02	1
4159228	99	72	68,3	cyclictest	132898	-21	kworker/u4:7+events_unbound	23:42:33	0
4159229	99	70	62,5	cyclictest	27	1	ktimers/1	23:17:01	1
4159229	99	64	57,5	cyclictest	4155422	-21	kworker/u4:0+events_unbound	20:12:01	1
4159229	99	64	55,7	cyclictest	53	-21	kswapd0	21:50:19	1
4159228	99	64	49,9	cyclictest	152764	-21	modprobe	00:05:00	0
4159228	99	63	51,9	cyclictest	169538	-21	systemd	00:23:38	0
4159228	99	62	38,15	cyclictest	53	-21	kswapd0	23:10:10	0
4159229	99	61	54,5	cyclictest	146360	-21	kworker/u4:1+events_unbound	23:58:16	1
4159228	99	59	49,8	cyclictest	70930	-21	systemd	22:32:20	0
4159228	99	58	47,8	cyclictest	0	-21	swapper/0	23:45:27	0
4159229	99	56	51,4	cyclictest	61745	-21	kworker/u4:16+events_unbound	22:22:19	1
4159229	99	56	47,6	cyclictest	4190958	-21	diskmemload	21:10:23	1
4159228	99	56	46,8	cyclictest	53	-21	kswapd0	23:10:01	0
4159229	99	55	48,5	cyclictest	4190020	-21	kworker/u4:12+events_unbound	21:20:00	1
4159228	99	55	48,5	cyclictest	43899	-21	sh	22:02:16	0
4159228	99	54	46,6	cyclictest	3907237	-21	kworker/u4:9+events_unbound	19:10:03	0
4159228	99	53	46,5	cyclictest	4172027	-21	kworker/u4:2+events_unbound	21:15:04	0
4159228	99	53	46,5	cyclictest	32928	-21	gltestperf	21:50:20	0
4159228	99	53	45,5	cyclictest	63829	-21	cat	22:25:01	0
4159228	99	53	45,5	cyclictest	154085	-21	/usr/sbin/munin	00:05:44	0
4159228	99	53	44,6	cyclictest	0	-21	swapper/0	23:50:15	0
4159229	99	52	46,4	cyclictest	91392	-21	fschecks_time	22:55:20	1
4159229	99	52	46,4	cyclictest	115602	-21	kworker/u4:9+events_unbound	23:40:00	1
4159229	99	52	45,5	cyclictest	149239	-21	/usr/sbin/munin	00:00:25	1
4159229	99	52	43,6	cyclictest	4190019	-21	kworker/u4:4-rpciod	21:30:00	1
4159229	99	52	39,10	cyclictest	37	-21	kcompactd0	22:15:01	1
4159229	99	52	36,13	cyclictest	0	-21	swapper/1	22:30:28	1
4159228	99	52	45,5	cyclictest	3907237	-21	kworker/u4:9+events_unbound	19:50:03	0
4159228	99	52	44,5	cyclictest	0	-21	swapper/0	23:30:12	0
4159228	99	52	43,6	cyclictest	15526	-21	mailstats	21:30:43	0
4159228	99	52	41,6	cyclictest	256	50	irq/23-42890000.ethernet	00:25:35	0
4159228	99	51	44,5	cyclictest	120232	-21	sshd	23:27:30	0
4159229	99	50	39,8	cyclictest	4185717	-21	ps	20:50:36	1
4159228	99	50	43,5	cyclictest	59265	-21	ssh	22:20:01	0
4159228	99	50	43,5	cyclictest	4182718	-21	grep	20:40:18	0
4159228	99	50	43,5	cyclictest	4155433	-21	kworker/u4:1+events_unbound	19:15:03	0
4159228	99	50	43,5	cyclictest	39386	-21	sshd	21:57:16	0
4159228	99	50	43,5	cyclictest	39386	-21	sshd	21:57:16	0
4159228	99	50	42,6	cyclictest	21171	-21	kworker/u4:0+events_unbound	21:40:45	0
4159228	99	50	40,8	cyclictest	163020	-21	timerandwakeup	00:15:39	0
4159228	99	50	39,8	cyclictest	54720	-21	latency_hist	22:15:02	0
4159228	99	50	27,20	cyclictest	32207	-21	latency_hist	21:50:01	0
4159229	99	49	42,5	cyclictest	44083	-21	kworker/u4:1+events_unbound	22:25:44	1
4159229	99	49	42,5	cyclictest	4176982	-21	timerwakeupswit	20:15:38	1
4159229	99	49	41,6	cyclictest	4166971	-21	latency_hist	19:40:01	1
4159228	99	49	6,29	cyclictest	4190053	-21	diskmemload	22:10:00	0
4159228	99	49	41,5	cyclictest	10137	-21	aten_r7power_vo	21:25:16	0
4159228	99	49	30,14	cyclictest	22808	-21	sshd	21:39:29	0
4159228	99	49	18,27	cyclictest	256	50	irq/23-42890000.ethernet	22:50:34	0
4159229	99	48	41,5	cyclictest	4178441	-21	kworker/u4:3+events_unbound	20:35:01	1
4159229	99	48	41,5	cyclictest	4163037	-21	kworker/u4:14+events_unbound	19:30:03	1
4159229	99	48	41,5	cyclictest	166471	-21	aten_r7power_po	00:20:16	1
4159228	99	48	41,5	cyclictest	4175875	-21	kworker/u4:15+events_unbound	20:35:02	0
4159228	99	48	40,5	cyclictest	173328	-21	kworker/u4:5-rpciod	00:35:37	0
4159228	99	48	28,16	cyclictest	0	-21	swapper/0	23:55:38	0
4159229	99	47	40,5	cyclictest	4174636	-21	idleruntime-cro	20:10:00	1
4159229	99	47	40,5	cyclictest	102098	-21	ssh	23:06:52	1
4159229	99	47	36,8	cyclictest	83929	-21	ssh	22:46:34	1
4159228	99	47	40,5	cyclictest	73305	-21	cut	22:35:20	0
4159228	99	47	40,5	cyclictest	108934	-21	aten_r7power_cu	23:15:15	0
4159229	99	46	40,4	cyclictest	28411	-21	cut	21:45:20	1
4159229	99	46	40,4	cyclictest	23624	-21	basename	21:40:15	1
4159229	99	46	40,4	cyclictest	135377	-21	acpi	23:45:12	1
4159229	99	46	40,4	cyclictest	104600	-21	aten_r7power_cu	23:10:22	1
4159229	99	46	38,5	cyclictest	14624	-21	aten_r7power_po	21:30:19	1
4159229	99	46	36,7	cyclictest	0	-21	swapper/1	23:00:18	1
4159229	99	46	34,9	cyclictest	4161796	-21	sendmail	19:20:00	1
4159229	99	46	24,11	cyclictest	134313	-21	ssh	23:43:47	1
4159228	99	46	40,4	cyclictest	4184072	-21	fschecks_time	20:45:17	0
4159228	99	46	40,4	cyclictest	175416	-21	df_abs	00:30:17	0
4159228	99	46	39,5	cyclictest	95920	-21	fschecks_time	23:00:21	0
4159228	99	46	37,6	cyclictest	5824	-21	fschecks_count	21:20:22	0
4159228	99	46	35,9	cyclictest	4188931	-21	acpi	21:05:12	0
4159228	99	46	32,12	cyclictest	115602	-21	kworker/u4:9-rpciod	23:35:25	0
4159229	99	45	39,4	cyclictest	4173191	-21	unixbench-2d	20:00:38	1
4159229	99	45	38,5	cyclictest	55138	-21	aten_r7power_en	22:15:18	1
4159229	99	45	38,5	cyclictest	173329	-21	kworker/u4:7-rpciod	00:30:37	1
4159229	99	45	38,5	cyclictest	168172	-21	kworker/u4:3+events_unbound	00:25:45	1
4159229	99	45	38,5	cyclictest	162179	-21	fschecks_count	00:15:20	1
4159229	99	45	37,5	cyclictest	4178442	-21	kworker/u4:4-rpciod	20:25:24	1
4159229	99	45	36,7	cyclictest	19303	-21	fschecks_count	21:35:22	1
4159229	99	45	35,7	cyclictest	0	-21	swapper/1	22:05:29	1
4159229	99	45	33,9	cyclictest	0	-21	swapper/1	22:40:26	1
4159229	99	45	33,9	cyclictest	0	-21	swapper/1	22:40:26	1
4159229	99	45	33,10	cyclictest	6187	-21	sshd	21:20:30	1
4159228	99	45	39,4	cyclictest	0	-21	swapper/0	00:10:37	0
4159228	99	45	37,5	cyclictest	0	-21	swapper/0	21:00:43	0
4159228	99	45	34,7	cyclictest	4168239	-21	modprobe	19:45:00	0
4159228	99	45	18,23	cyclictest	15	1	ktimers/0	22:48:55	0
4159228	99	45	15,21	cyclictest	256	50	irq/23-42890000.ethernet	19:09:59	0
4159229	99	44	37,5	cyclictest	41948	-21	gltestperf	22:00:20	1
4159229	99	44	37,5	cyclictest	113907	-21	kernelversion	23:20:22	1
4159229	99	44	35,5	cyclictest	0	-21	swapper/1	23:50:10	1
4159229	99	44	34,7	cyclictest	117736	-21	dpkg	23:25:13	1
4159228	99	44	34,8	cyclictest	0	-21	swapper/0	20:15:00	0
4159229	99	43	37,4	cyclictest	4168259	-21	munin-plugin-st	19:45:00	1
4159229	99	43	36,4	cyclictest	157862	-21	fschecks_count	00:10:26	1
4159229	99	43	34,6	cyclictest	0	-21	swapper/1	23:30:30	1
4159229	99	43	31,10	cyclictest	4171812	-21	timerandwakeup	19:55:39	1
4159229	99	42	34,6	cyclictest	0	-21	swapper/1	00:35:45	1
4159228	99	42	35,5	cyclictest	88500	-21	kworker/u4:10-rpciod	23:20:27	0
4159228	99	42	32,8	cyclictest	91925	-21	open_inodes	22:55:37	0
4159228	99	42	29,9	cyclictest	4187098	-21	sort	20:55:39	0
4159229	99	41	35,4	cyclictest	4177567	-21	df_abs	20:20:16	1
4159229	99	41	35,4	cyclictest	153969	-21	/usr/sbin/munin	00:05:38	1
4159229	99	41	34,5	cyclictest	4160455	-21	kworker/u4:4+events_unbound	19:20:04	1
4159229	99	41	34,5	cyclictest	4160455	-21	kworker/u4:4+events_unbound	19:20:03	1
4159228	99	41	33,6	cyclictest	4168445	-21	aten2__	19:45:12	0
4159228	99	41	33,5	cyclictest	0	-21	swapper/0	22:40:14	0
4159228	99	41	33,5	cyclictest	0	-21	swapper/0	22:40:14	0
4159228	99	40	30,7	cyclictest	0	-21	swapper/0	20:15:38	0
4159228	99	39	20,17	cyclictest	15	1	ktimers/0	19:55:30	0
4159229	99	38	31,5	cyclictest	4155434	-21	kworker/u4:3+events_unbound	19:25:02	1
4159229	99	32	18,4	cyclictest	17	1	rcu_preempt	19:10:01	1
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17