OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #7, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system

Characteristics of the highest latencies:

System rack7slot2.osadl.org (updated Fri May 08, 2026 00:44:08)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
3867811	99	104	89,6	cyclictest	3867738	-21	kworker/u4:3+events_unbound	19:17:27	1
3867811	99	95	89,4	cyclictest	3869032	-21	kworker/u4:5+events_unbound	19:37:30	1
3867811	99	95	82,6	cyclictest	3871610	-21	kworker/u4:6+events_unbound	19:22:28	1
3867811	99	94	87,5	cyclictest	3876723	-21	kworker/u4:10+events_unbound	19:47:32	1
3867811	99	90	84,4	cyclictest	3874151	-21	kworker/u4:2+events_unbound	19:32:30	1
3867811	99	89	82,5	cyclictest	3941639	-21	kworker/u4:11+events_unbound	21:58:02	1
3867811	99	86	79,5	cyclictest	3885734	-21	kworker/u4:17+events_unbound	21:02:51	1
3867811	99	85	75,7	cyclictest	3937086	-21	kworker/u4:1+events_unbound	22:48:18	1
3867811	99	83	76,5	cyclictest	4003274	-21	kworker/u4:2+events_unbound	23:08:22	1
3867811	99	83	75,6	cyclictest	3871610	-21	kworker/u4:6+events_unbound	19:42:31	1
3867810	99	73	58,10	cyclictest	53	-21	kswapd0	00:15:14	0
3867810	99	73	3,6	cyclictest	3904160	-21	ntpq	21:15:27	0
3867811	99	72	66,5	cyclictest	4060571	-21	kworker/u4:15+events_unbound	00:28:34	1
3867810	99	72	61,8	cyclictest	3894785	-21	kworker/u4:19+events_unbound	22:18:10	0
3867810	99	72	61,8	cyclictest	3894785	-21	kworker/u4:19+events_unbound	22:18:10	0
3867810	99	72	27,42	cyclictest	4031610	-21	sendmail	23:40:00	0
3867810	99	71	58,11	cyclictest	53	-21	kswapd0	23:30:11	0
3867810	99	71	37,31	cyclictest	4062335	-21	latency_hist	00:15:02	0
3867810	99	70	21,46	cyclictest	256	50	irq/23-42890000.ethernet	23:05:43	0
3867810	99	69	33,30	cyclictest	3894823	-21	munin-plugin-st	20:55:00	0
3867810	99	69	24,42	cyclictest	3883546	-21	fschecks_count	20:10:18	0
3867810	99	69	1,15	cyclictest	3992357	-21	apt-get	22:55:19	0
3867811	99	67	60,5	cyclictest	3876724	-21	kworker/u4:12+events_unbound	20:50:41	1
3867810	99	67	34,29	cyclictest	3925566	-21	latency_hist	21:40:01	0
3867810	99	67	31,31	cyclictest	3927076	-21	unixbench_multi	21:40:43	0
3867810	99	67	28,33	cyclictest	3913370	-21	processes	21:25:40	0
3867810	99	66	51,9	cyclictest	3917196	-21	ls	21:30:20	0
3867810	99	66	4,50	cyclictest	4050157	-21	ntpq	00:00:27	0
3867810	99	66	35,29	cyclictest	4011156	-21	unixbench-2d	23:15:44	0
3867810	99	66	3,19	cyclictest	3997523	-21	ntpq	23:00:28	0
3867810	99	65	2,20	cyclictest	4041455	-21	ssh	23:50:31	0
3867810	99	65	20,42	cyclictest	0	-21	swapper/0	23:45:25	0
3867810	99	64	4,13	cyclictest	3935655	-21	ps	21:50:37	0
3867810	99	64	3,57	cyclictest	3908650	-21	ntpq	21:20:29	0
3867810	99	64	31,30	cyclictest	3889956	-21	aten_r7power_po	20:35:15	0
3867810	99	64	31,30	cyclictest	3889956	-21	aten_r7power_po	20:35:15	0
3867810	99	64	30,31	cyclictest	3878462	-21	fschecks_count	19:50:18	0
3867810	99	64	29,32	cyclictest	3887201	-21	latency_hist	20:25:01	0
3867810	99	63	3,7	cyclictest	4019613	-21	ssh	23:25:37	0
3867810	99	63	3,52	cyclictest	4080791	-21	ssh	00:35:27	0
3867810	99	63	34,26	cyclictest	3944116	-21	gltestperf	22:00:20	0
3867810	99	63	31,29	cyclictest	3948283	-21	aten_r7power_po	22:05:16	0
3867810	99	63	28,32	cyclictest	3934484	-21	latency_hist	21:50:01	0
3867810	99	63	2,10	cyclictest	4006463	-21	ssh	23:10:38	0
3867810	99	63	16,43	cyclictest	3872736	-21	sed	19:25:37	0
3867810	99	62	7,50	cyclictest	3966693	-21	ntpq	22:25:33	0
3867810	99	62	32,26	cyclictest	0	-21	swapper/0	23:20:20	0
3867810	99	62	31,27	cyclictest	0	-21	swapper/0	22:20:37	0
3867810	99	62	30,28	cyclictest	0	-21	swapper/0	22:30:37	0
3867810	99	62	27,32	cyclictest	4076719	-21	sendmail_mailqu	00:30:38	0
3867810	99	62	25,35	cyclictest	3899902	-21	ps	21:10:36	0
3867810	99	61	31,28	cyclictest	4054458	-21	ntp_kernel_err	00:05:23	0
3867810	99	61	30,28	cyclictest	3898681	-21	idleruntime-cro	21:10:00	0
3867810	99	61	30,27	cyclictest	0	-21	swapper/0	20:00:01	0
3867810	99	61	2,9	cyclictest	3939896	-21	ssh	21:55:28	0
3867810	99	61	28,30	cyclictest	3937086	-21	kworker/u4:1-rpciod	22:50:37	0
3867810	99	61	28,29	cyclictest	0	-21	swapper/0	22:10:16	0
3867810	99	61	2,7	cyclictest	3896282	-21	apt-get	21:00:11	0
3867810	99	61	2,18	cyclictest	4045710	-21	ntpq	23:55:28	0
3867810	99	61	1,58	cyclictest	0	-21	swapper/0	22:40:11	0
3867810	99	60	7,47	cyclictest	4021152	-21	kworker/u4:13-rpciod	23:40:17	0
3867810	99	60	25,32	cyclictest	0	-21	swapper/0	22:35:28	0
3867810	99	60	1,15	cyclictest	4067793	-21	ssh	00:20:33	0
3867811	99	59	48,7	cyclictest	0	-21	swapper/1	22:30:32	1
3867811	99	59	46,5	cyclictest	3885733	-21	kworker/u4:14+events_unbound	20:34:17	1
3867811	99	59	45,6	cyclictest	4051743	-21	kworker/u4:8+events_unbound	00:39:59	1
3867810	99	59	3,10	cyclictest	3895563	-21	ntpq	20:55:27	0
3867810	99	59	23,32	cyclictest	0	-21	swapper/0	20:45:36	0
3867810	99	59	2,14	cyclictest	3887879	-21	ntpq	20:25:23	0
3867810	99	59	16,39	cyclictest	3873659	-21	ntpq	19:30:26	0
3867811	99	58	52,5	cyclictest	3994987	-21	kworker/u4:5+events_unbound	22:58:20	1
3867811	99	58	52,4	cyclictest	3894785	-21	kworker/u4:19+events_unbound	21:50:02	1
3867810	99	58	3,6	cyclictest	3874906	-21	ntpq	19:35:24	0
3867810	99	58	2,8	cyclictest	4071995	-21	ntpq	00:25:25	0
3867810	99	58	2,6	cyclictest	3881376	-21	ntpq	20:00:28	0
3867810	99	58	2,6	cyclictest	3881376	-21	ntpq	20:00:28	0
3867810	99	58	21,33	cyclictest	0	-21	swapper/0	19:40:15	0
3867811	99	57	45,8	cyclictest	0	-21	swapper/1	23:40:29	1
3867810	99	57	5,12	cyclictest	3880535	-21	kworker/u4:15-rpciod	20:10:01	0
3867810	99	57	3,7	cyclictest	3869142	-21	ls	19:15:01	0
3867810	99	57	2,5	cyclictest	3877503	-21	ntpq	19:45:27	0
3867810	99	57	23,31	cyclictest	3891389	-21	sshd	20:40:17	0
3867810	99	57	2,16	cyclictest	3984449	-21	ntpq	22:45:37	0
3867811	99	56	44,10	cyclictest	4060575	-21	kworker/u4:19+events_unbound	00:33:36	1
3867810	99	56	2,7	cyclictest	3888601	-21	apt-get	20:30:11	0
3867810	99	56	2,14	cyclictest	3871143	-21	ntpq	19:20:29	0
3867810	99	56	2,10	cyclictest	3884665	-21	apt-get	20:15:19	0
3867810	99	55	2,15	cyclictest	3869803	-21	ntpq	19:15:29	0
3867811	99	54	47,5	cyclictest	3921724	-21	/usr/sbin/munin	21:35:20	1
3867811	99	54	46,6	cyclictest	4007729	-21	kworker/u4:7+events_unbound	23:12:41	1
3867811	99	54	46,6	cyclictest	3885734	-21	kworker/u4:17+events_unbound	21:07:01	1
3867811	99	54	40,6	cyclictest	3887132	-21	kworker/u4:1+events_unbound	20:45:46	1
3867811	99	53	46,5	cyclictest	3653137	-21	kworker/u4:11+events_unbound	19:14:01	1
3867811	99	53	40,9	cyclictest	4067349	-21	fschecks_count	00:20:23	1
3867811	99	53	37,7	cyclictest	3890943	-21	kworker/u4:6+events_unbound	20:57:13	1
3867811	99	52	45,5	cyclictest	4050792	-21	/usr/sbin/munin	00:00:43	1
3867811	99	52	44,6	cyclictest	3885733	-21	kworker/u4:14+events_unbound	20:28:17	1
3867811	99	52	44,5	cyclictest	3926568	-21	ntpq	21:40:31	1
3867811	99	52	44,5	cyclictest	0	-21	swapper/1	23:30:14	1
3867811	99	51	44,6	cyclictest	3884372	-21	kworker/u4:2+events_unbound	21:32:59	1
3867811	99	51	36,7	cyclictest	3875423	-21	kworker/u4:7+events_unbound	20:16:01	1
3867811	99	50	44,5	cyclictest	4003275	-21	kworker/u4:4+events_unbound	23:22:59	1
3867811	99	50	42,5	cyclictest	3884426	-21	idleruntime-cro	20:15:00	1
3867811	99	50	41,7	cyclictest	3953103	-21	gltestperf	22:10:22	1
3867811	99	50	36,6	cyclictest	3871610	-21	kworker/u4:6+events_unbound	19:50:03	1
3867810	99	50	12,33	cyclictest	0	-21	swapper/0	19:10:01	0
3867811	99	49	43,5	cyclictest	3954629	-21	kworker/u4:13+events_unbound	22:15:53	1
3867811	99	49	43,5	cyclictest	3954629	-21	kworker/u4:13+events_unbound	22:15:53	1
3867811	99	49	42,5	cyclictest	3872918	-21	munin-run	19:30:01	1
3867811	99	49	41,6	cyclictest	4020057	-21	/usr/sbin/munin	23:25:42	1
3867811	99	49	41,6	cyclictest	3876725	-21	kworker/u4:13+events_unbound	20:03:16	1
3867811	99	49	41,6	cyclictest	3876725	-21	kworker/u4:13+events_unbound	20:03:16	1
3867811	99	48	41,5	cyclictest	3884380	-21	kworker/u4:11+events_unbound	20:21:11	1
3867811	99	48	40,5	cyclictest	3943718	-21	cut	22:00:15	1
3867811	99	47	39,6	cyclictest	3867808	-21	cyclictest	21:10:25	1
3867811	99	47	39,5	cyclictest	4009983	-21	aten2__	23:15:21	1
3867811	99	47	38,6	cyclictest	3962595	-21	proc_pri	22:20:51	1
3867811	99	47	38,6	cyclictest	3916558	-21	ssh	21:30:01	1
3867811	99	46	5,32	cyclictest	4045899	-21	ssh	23:55:36	1
3867811	99	46	39,5	cyclictest	4054748	-21	proc_pri	00:05:36	1
3867811	99	46	36,7	cyclictest	3989114	-21	timerwakeupswit	22:50:41	1
3867811	99	46	36,6	cyclictest	0	-21	swapper/1	22:05:26	1
3867811	99	45	39,4	cyclictest	3975257	-21	interrupts	22:35:22	1
3867811	99	45	37,6	cyclictest	4037373	-21	timerandwakeup	23:45:39	1
3867811	99	45	37,6	cyclictest	3889672	-21	kworker/u4:0+events_unbound	20:40:01	1
3867811	99	45	37,6	cyclictest	3889672	-21	kworker/u4:0+events_unbound	20:40:01	1
3867811	99	45	34,9	cyclictest	3904194	-21	ntp_kernel_pll_	21:15:28	1
3867811	99	44	37,5	cyclictest	3877994	-21	kworker/u4:0+events_unbound	20:06:27	1
3867811	99	44	36,6	cyclictest	3997850	-21	switchtime	23:00:37	1
3867811	99	44	36,6	cyclictest	3983300	-21	latency_hist	22:45:01	1
3867811	99	44	36,5	cyclictest	3908054	-21	grep	21:20:17	1
3867811	99	44	35,6	cyclictest	0	-21	swapper/1	19:55:31	1
3867811	99	43	36,5	cyclictest	3888399	-21	kworker/u4:4+events_unbound	20:41:00	1
3867811	99	43	35,6	cyclictest	4027673	-21	aten_r7power_vo	23:35:16	1
3867811	99	43	34,7	cyclictest	0	-21	swapper/1	23:50:32	1
3867811	99	43	34,6	cyclictest	0	-21	swapper/1	21:50:02	1
3867811	99	42	31,8	cyclictest	0	-21	swapper/1	00:15:36	1
3867811	99	41	32,6	cyclictest	0	-21	swapper/1	00:10:28	1
3867811	99	39	30,6	cyclictest	0	-21	swapper/1	22:25:25	1
3867811	99	38	29,6	cyclictest	0	-21	swapper/1	19:10:01	1
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17