OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #7, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system

Characteristics of the 376 highest latencies:

System rack7slot2.osadl.org (updated Thu Sep 10, 2026 12:44:09)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
2835414	99	98	87,7	cyclictest	2843025	-21	kworker/u4:9+events_unbound	09:15:23	0
2835414	99	92	82,7	cyclictest	53	-21	kswapd0	12:25:13	0
2835414	99	92	61,5	cyclictest	2898404	-21	fschecks_count	09:45:19	0
2835414	99	90	80,7	cyclictest	2994276	-21	kworker/u4:5+events_unbound	11:40:54	0
2835414	99	89	80,7	cyclictest	2914088	-21	kworker/u4:14+events_unbound	10:05:33	0
2835414	99	86	76,7	cyclictest	2909499	-21	kworker/u4:4+events_unbound	10:10:34	0
2835414	99	82	75,5	cyclictest	2945007	-21	kworker/u4:3+events_unbound	10:45:40	0
2835415	99	76	45,29	cyclictest	2871542	-21	sshd	09:15:21	1
2835414	99	76	68,6	cyclictest	2953452	-21	kworker/u4:17+events_unbound	10:55:44	0
2835414	99	76	65,8	cyclictest	2923122	-21	kworker/u4:15+events_unbound	10:25:35	0
2835415	99	75	64,7	cyclictest	2855818	-21	kworker/u4:5+events_unbound	08:30:13	1
2835414	99	75	67,6	cyclictest	3016301	-21	kworker/u4:6+events_unbound	12:05:56	0
2835414	99	75	65,7	cyclictest	2923121	-21	kworker/u4:12+events_unbound	10:15:35	0
2835414	99	75	2,31	cyclictest	2992679	-21	ssh	11:30:39	0
2835414	99	74	66,6	cyclictest	2835333	-21	kworker/u4:2+events_unbound	07:24:51	0
2835415	99	71	36,32	cyclictest	2840537	-21	grep	07:30:02	1
2835415	99	70	63,5	cyclictest	2824050	-21	kworker/u4:13+events_unbound	07:19:47	1
2835414	99	70	62,6	cyclictest	2980634	-21	kworker/u4:11+events_unbound	11:25:50	0
2835414	99	70	62,6	cyclictest	2868920	-21	kworker/u4:14+events_unbound	09:20:24	0
2835414	99	70	55,12	cyclictest	2909503	-21	kworker/u4:9+events_unbound	10:00:32	0
2835414	99	70	55,12	cyclictest	2909503	-21	kworker/u4:9+events_unbound	10:00:32	0
2835414	99	69	58,7	cyclictest	53	-21	kswapd0	10:20:13	0
2835415	99	68	63,3	cyclictest	2843021	-21	kworker/u4:4+events_unbound	08:00:01	1
2835415	99	68	59,6	cyclictest	2835339	-21	kworker/u4:5+events_unbound	07:15:02	1
2835415	99	68	37,28	cyclictest	2858807	-21	aten_r7power_vo	08:40:17	1
2835415	99	68	34,30	cyclictest	0	-21	swapper/1	12:25:28	1
2835414	99	68	58,7	cyclictest	2922469	-21	kworker/u4:7+events_unbound	10:30:36	0
2835415	99	67	33,31	cyclictest	2987344	-21	ssh	11:25:14	1
2835415	99	67	33,31	cyclictest	2845674	-21	latency_hist	07:50:01	1
2835415	99	66	35,28	cyclictest	0	-21	swapper/1	08:50:30	1
2835415	99	66	34,30	cyclictest	2901726	-21	sh	09:49:11	1
2835415	99	66	34,28	cyclictest	0	-21	swapper/1	07:35:23	1
2835415	99	66	33,31	cyclictest	2992850	-21	switchtime	11:30:42	1
2835415	99	65	33,29	cyclictest	3037415	-21	/usr/sbin/munin	12:20:38	1
2835415	99	65	28,34	cyclictest	0	-21	swapper/1	12:10:38	1
2835414	99	65	56,7	cyclictest	3011754	-21	kworker/u4:8+events_unbound	11:55:55	0
2835414	99	65	5,50	cyclictest	3010287	-21	ssh	11:50:27	0
2835415	99	64	33,28	cyclictest	0	-21	swapper/1	10:55:31	1
2835415	99	64	32,29	cyclictest	0	-21	swapper/1	11:40:10	1
2835415	99	64	31,31	cyclictest	2973951	-21	aten2__	11:10:18	1
2835415	99	64	31,30	cyclictest	0	-21	swapper/1	10:30:45	1
2835414	99	64	57,5	cyclictest	2945012	-21	kworker/u4:11+events_unbound	11:05:47	0
2835415	99	63	58,3	cyclictest	3016301	-21	kworker/u4:6+events_unbound	12:00:54	1
2835415	99	63	40,21	cyclictest	2890068	-21	cat	09:35:35	1
2835415	99	63	32,28	cyclictest	0	-21	swapper/1	09:25:30	1
2835415	99	63	31,29	cyclictest	0	-21	swapper/1	09:20:28	1
2835415	99	63	31,28	cyclictest	0	-21	swapper/1	10:05:24	1
2835415	99	63	30,31	cyclictest	0	-21	swapper/1	09:55:21	1
2835415	99	62	32,26	cyclictest	0	-21	swapper/1	10:45:29	1
2835415	99	62	29,30	cyclictest	0	-21	swapper/1	12:05:22	1
2835415	99	62	29,30	cyclictest	0	-21	swapper/1	09:00:31	1
2835415	99	62	2,17	cyclictest	2929520	-21	ssh	10:20:26	1
2835415	99	62	2,16	cyclictest	2838693	-21	ntpq	07:20:25	1
2835415	99	62	16,43	cyclictest	2987086	-21	sed	11:25:02	1
2835415	99	62	1,14	cyclictest	2894933	-21	ssh	09:40:43	1
2835414	99	62	40,14	cyclictest	3009298	-21	modprobe	11:50:00	0
2835415	99	61	32,26	cyclictest	0	-21	swapper/1	11:35:31	1
2835415	99	61	31,27	cyclictest	2835340	-21	kworker/u4:6-rpciod	08:10:39	1
2835415	99	61	28,31	cyclictest	2867797	-21	unixbench_singl	09:10:42	1
2835415	99	61	25,33	cyclictest	0	-21	swapper/1	12:15:30	1
2835415	99	61	14,44	cyclictest	3007201	-21	kworker/u4:14-rpciod	11:50:28	1
2835415	99	60	54,4	cyclictest	27	1	ktimers/1	10:15:35	1
2835415	99	60	41,7	cyclictest	2985185	-21	kworker/u4:0-rpciod	12:40:01	1
2835415	99	60	28,29	cyclictest	0	-21	swapper/1	09:50:41	1
2835415	99	60	27,30	cyclictest	0	-21	swapper/1	09:30:26	1
2835415	99	59	28,28	cyclictest	0	-21	swapper/1	10:00:08	1
2835415	99	59	28,28	cyclictest	0	-21	swapper/1	10:00:08	1
2835415	99	59	26,30	cyclictest	378	-21	dbus-daemon	11:05:46	1
2835415	99	58	31,24	cyclictest	0	-21	swapper/1	10:35:42	1
2835415	99	58	2,8	cyclictest	2978338	-21	apt-get	11:15:12	1
2835415	99	58	27,28	cyclictest	0	-21	swapper/1	10:30:01	1
2835415	99	58	24,31	cyclictest	2853643	-21	cpu	08:20:15	1
2835415	99	58	24,31	cyclictest	0	-21	swapper/1	10:40:35	1
2835415	99	58	2,15	cyclictest	2848583	-21	diskstats	08:00:19	1
2835414	99	58	43,9	cyclictest	2857175	-21	kworker/u4:7-rpciod	09:05:25	0
2835414	99	58	42,10	cyclictest	256	50	irq/23-42890000.ethernet	12:10:30	0
2835415	99	57	45,9	cyclictest	2840480	-21	kworker/u4:1+events_unbound	08:45:17	1
2835415	99	57	3,11	cyclictest	3014053	-21	apt-get	11:55:13	1
2835414	99	57	45,9	cyclictest	0	-21	swapper/0	09:43:22	0
2835415	99	56	4,9	cyclictest	3005925	-21	ssh	11:45:31	1
2835415	99	56	3,9	cyclictest	2964973	-21	apt-get	11:00:11	1
2835415	99	56	28,25	cyclictest	0	-21	swapper/1	10:50:41	1
2835415	99	56	24,29	cyclictest	0	-21	swapper/1	08:05:27	1
2835415	99	56	2,12	cyclictest	3045339	-21	sh	12:30:13	1
2835414	99	56	8,46	cyclictest	0	-21	swapper/0	11:00:13	0
2835414	99	56	38,15	cyclictest	2943686	-21	processes	10:35:43	0
2835415	99	55	24,28	cyclictest	2921440	-21	proc_pri	10:10:36	1
2835415	99	55	1,14	cyclictest	2846338	-21	ntpq	07:50:24	1
2835414	99	55	48,5	cyclictest	53	-21	kswapd0	12:35:10	0
2835414	99	55	48,5	cyclictest	2889786	-21	ssh	09:35:25	0
2835414	99	55	48,5	cyclictest	2843025	-21	kworker/u4:9+events_unbound	07:39:54	0
2835414	99	55	34,11	cyclictest	2859760	-21	modprobe	08:45:00	0
2835415	99	54	3,7	cyclictest	1073	-21	snmpd	08:25:30	1
2835415	99	54	34,17	cyclictest	2858448	-21	kworker/u4:16-rpciod	08:55:16	1
2835415	99	54	30,22	cyclictest	2866216	-21	latency_hist	09:10:01	1
2835415	99	54	2,16	cyclictest	2858073	-21	ps	08:35:36	1
2835415	99	54	2,16	cyclictest	2858073	-21	ps	08:35:36	1
2835414	99	54	40,9	cyclictest	3038060	-21	ssh	12:21:08	0
2835415	99	53	2,12	cyclictest	2852746	-21	ntpq	08:15:26	1
2835415	99	53	2,12	cyclictest	2852746	-21	ntpq	08:15:26	1
2835415	99	51	7,7	cyclictest	1073	-21	snmpd	07:43:23	1
2835415	99	51	7,7	cyclictest	1073	-21	snmpd	07:43:23	1
2835414	99	51	40,8	cyclictest	2903781	-21	grep	09:50:39	0
2835414	99	50	33,13	cyclictest	2884868	-21	fschecks_count	09:30:18	0
2835415	99	49	13,33	cyclictest	0	-21	swapper/1	07:30:20	1
2835414	99	49	41,6	cyclictest	2996256	-21	aten2__	11:35:14	0
2835414	99	49	41,6	cyclictest	2843025	-21	kworker/u4:9+events_unbound	08:15:00	0
2835414	99	49	2,37	cyclictest	3019911	-21	users	12:00:43	0
2835414	99	48	39,7	cyclictest	2840480	-21	kworker/u4:1+events_unbound	08:25:13	0
2835414	99	48	38,7	cyclictest	2866858	-21	sh	09:10:21	0
2835414	99	48	38,7	cyclictest	2858332	-21	/usr/sbin/munin	08:35:38	0
2835414	99	48	38,7	cyclictest	2858332	-21	/usr/sbin/munin	08:35:38	0
2835414	99	48	37,7	cyclictest	2953450	-21	kworker/u4:13-rpciod	10:50:36	0
2835414	99	48	35,11	cyclictest	2948280	-21	timerandwakeup	10:40:37	0
2835414	99	48	3,32	cyclictest	3032865	-21	ssh	12:15:35	0
2835414	99	48	29,17	cyclictest	3049713	-21	latency_hist	12:35:03	0
2835414	99	47	38,7	cyclictest	2860267	-21	fschecks_time	08:45:18	0
2835414	99	47	38,6	cyclictest	2862222	-21	unixbench_singl	08:50:40	0
2835414	99	47	37,8	cyclictest	2846226	-21	gltestperf	07:50:19	0
2835414	99	47	35,10	cyclictest	0	-21	swapper/0	08:30:12	0
2835414	99	47	25,19	cyclictest	0	-21	swapper/0	09:00:37	0
2835414	99	46	36,7	cyclictest	2847806	-21	sendmail_mailqu	07:55:35	0
2835414	99	46	35,8	cyclictest	2880828	-21	grep	09:25:30	0
2835414	99	46	26,18	cyclictest	0	-21	swapper/0	09:00:00	0
2835415	99	45	1,11	cyclictest	2835416	-21	latency_hist	07:10:01	1
2835414	99	45	37,6	cyclictest	2843647	-21	gltestperf	07:40:19	0
2835414	99	45	37,6	cyclictest	2843647	-21	gltestperf	07:40:19	0
2835414	99	45	31,10	cyclictest	2841678	-21	tail	07:30:38	0
2835414	99	45	28,11	cyclictest	0	-21	swapper/0	09:55:38	0
2835414	99	44	6,23	cyclictest	2837469	-21	ntp_states	07:15:32	0
2835414	99	44	37,5	cyclictest	2844837	-21	fschecks_time	07:45:20	0
2835414	99	44	35,7	cyclictest	2983949	-21	timerandwakeup	11:20:39	0
2835414	99	44	34,7	cyclictest	0	-21	swapper/0	11:10:37	0
2835414	99	44	33,7	cyclictest	2836388	-21	switchtime	07:10:36	0
2835414	99	44	27,15	cyclictest	2978415	-21	aten_r7power_cu	11:15:16	0
2835414	99	43	36,5	cyclictest	2853699	-21	/usr/sbin/munin	08:20:17	0
2835414	99	43	34,6	cyclictest	2839417	-21	aten2__	07:25:13	0
2835414	99	43	30,10	cyclictest	0	-21	swapper/0	08:20:00	0
2835414	99	43	30,10	cyclictest	0	-21	swapper/0	08:20:00	0
2835414	99	41	34,5	cyclictest	2849950	-21	fschecks_time	08:05:19	0
2835414	99	41	33,6	cyclictest	0	-21	swapper/0	08:05:01	0
2835414	99	41	31,6	cyclictest	0	-21	swapper/0	07:10:01	0
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17