OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #7, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system

Characteristics of the 792 highest latencies:

System rack7slot2.osadl.org (updated Fri May 01, 2026 12:44:08)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
3545810	99	121	112,6	cyclictest	3623005	-21	kworker/u4:4+events_unbound	10:20:01	1
3545810	99	107	91,8	cyclictest	3541994	-21	kworker/u4:5+events_unbound	07:34:25	1
3545810	99	99	82,6	cyclictest	3701162	-21	kworker/u4:11+events_unbound	11:35:16	1
3545809	99	99	91,6	cyclictest	3662098	-21	kworker/u4:1+events_unbound	10:55:08	0
3545810	99	96	86,8	cyclictest	3718884	-21	kworker/u4:3+events_unbound	11:55:19	1
3545810	99	92	82,7	cyclictest	3710043	-21	kworker/u4:4+events_unbound	12:00:23	1
3545809	99	91	84,5	cyclictest	3583486	-21	kworker/u4:5+events_unbound	09:19:47	0
3545809	99	86	79,5	cyclictest	3653287	-21	kworker/u4:17+events_unbound	11:05:11	0
3545809	99	86	68,6	cyclictest	3596346	-21	kworker/u4:10+events_unbound	09:39:52	0
3545810	99	85	78,5	cyclictest	3583487	-21	kworker/u4:6+events_unbound	09:24:48	1
3545809	99	82	70,8	cyclictest	3696737	-21	kworker/u4:4+events_unbound	11:30:15	0
3545810	99	80	73,5	cyclictest	3618545	-21	kworker/u4:1+events_unbound	10:30:03	1
3545810	99	80	71,7	cyclictest	3675343	-21	kworker/u4:9+events_unbound	11:45:19	1
3545810	99	78	71,5	cyclictest	3648883	-21	kworker/u4:11+events_unbound	10:35:03	1
3545810	99	78	70,6	cyclictest	3592520	-21	kworker/u4:9+events_unbound	09:44:53	1
3545810	99	78	70,6	cyclictest	3534419	-21	kworker/u4:12+events_unbound	08:35:00	1
3545809	99	78	71,5	cyclictest	3618545	-21	kworker/u4:1+events_unbound	10:09:57	0
3545809	99	77	69,5	cyclictest	3731635	-21	kworker/u4:4+events_unbound	12:15:25	0
3545810	99	75	69,4	cyclictest	3592520	-21	kworker/u4:9+events_unbound	09:29:50	1
3545809	99	75	68,5	cyclictest	3549612	-21	kworker/u4:14+events_unbound	08:40:01	0
3545809	99	75	66,7	cyclictest	3591865	-21	kworker/u4:8+events_unbound	09:54:55	0
3545810	99	73	65,6	cyclictest	3541996	-21	kworker/u4:7+events_unbound	07:54:32	1
3545809	99	72	58,6	cyclictest	3701160	-21	kworker/u4:8+events_unbound	11:40:18	0
3545809	99	70	62,6	cyclictest	3631849	-21	kworker/u4:11+events_unbound	10:14:58	0
3545809	99	69	62,5	cyclictest	3548300	-21	kworker/u4:8+events_unbound	08:55:00	0
3545810	99	67	61,4	cyclictest	3618545	-21	kworker/u4:1+events_unbound	10:30:00	1
3545810	99	67	59,5	cyclictest	3596348	-21	kworker/u4:11+events_unbound	09:59:56	1
3545810	99	66	59,5	cyclictest	3653287	-21	kworker/u4:17+events_unbound	11:00:09	1
3545810	99	66	59,5	cyclictest	3653287	-21	kworker/u4:17+events_unbound	11:00:09	1
3545810	99	66	58,6	cyclictest	3540784	-21	kworker/u4:0+events_unbound	08:24:37	1
3545810	99	66	50,6	cyclictest	3550886	-21	kworker/u4:16+events_unbound	07:50:01	1
3545809	99	66	60,4	cyclictest	3540784	-21	kworker/u4:0+events_unbound	08:04:33	0
3545810	99	65	46,6	cyclictest	3731633	-21	kworker/u4:2+events_unbound	12:20:26	1
3545809	99	64	57,5	cyclictest	3653287	-21	kworker/u4:17+events_unbound	10:45:06	0
3545809	99	63	55,6	cyclictest	53	-21	kswapd0	11:25:14	0
3545810	99	62	55,5	cyclictest	3674706	-21	kworker/u4:8+events_unbound	11:25:15	1
3545810	99	62	53,6	cyclictest	3583487	-21	kworker/u4:6+events_unbound	09:40:01	1
3545810	99	60	48,8	cyclictest	3568775	-21	cron	08:40:00	1
3545810	99	59	52,5	cyclictest	53	-21	kswapd0	10:50:01	1
3545810	99	59	51,6	cyclictest	3540784	-21	kworker/u4:0+events_unbound	08:49:41	1
3545810	99	59	50,5	cyclictest	3731636	-21	kworker/u4:6+events_unbound	12:09:59	1
3545809	99	59	52,5	cyclictest	3534419	-21	kworker/u4:12+events_unbound	07:59:31	0
3545809	99	57	50,5	cyclictest	3592520	-21	kworker/u4:9+events_unbound	09:35:00	0
3545810	99	56	44,9	cyclictest	3574871	-21	ps	09:00:47	1
3545810	99	56	44,10	cyclictest	0	-21	swapper/1	11:15:15	1
3545809	99	56	44,10	cyclictest	3564956	-21	systemd	08:24:36	0
3545809	99	56	35,12	cyclictest	3627426	-21	kworker/u4:8+events_unbound	10:16:00	0
3545810	99	55	47,5	cyclictest	3592520	-21	kworker/u4:9+events_unbound	09:30:14	1
3545809	99	55	48,5	cyclictest	3583487	-21	kworker/u4:6+events_unbound	09:55:03	0
3545810	99	54	46,6	cyclictest	3675343	-21	kworker/u4:9+events_unbound	11:30:32	1
3545810	99	54	43,8	cyclictest	3563264	-21	ps	08:15:36	1
3545810	99	54	40,7	cyclictest	3614124	-21	kworker/u4:3+events_unbound	10:05:21	1
3545809	99	54	33,19	cyclictest	3555159	-21	fschecks_count	07:45:20	0
3545810	99	53	43,8	cyclictest	3609693	-21	ssh	09:46:42	1
3545809	99	53	46,5	cyclictest	1053	-21	sshd	09:24:47	0
3545809	99	53	44,5	cyclictest	0	-21	swapper/0	12:25:27	0
3545809	99	53	44,5	cyclictest	0	-21	swapper/0	09:10:40	0
3545809	99	53	29,20	cyclictest	3545806	-21	cyclictest	11:55:00	0
3545810	99	52	44,5	cyclictest	3684125	-21	kworker/u4:15+events_unbound	11:14:00	1
3545810	99	52	38,9	cyclictest	37	-21	kcompactd0	10:40:34	1
3545809	99	52	31,14	cyclictest	3720742	-21	ssh	11:55:09	0
3545810	99	51	44,5	cyclictest	3550557	-21	sendmail_mailqu	07:25:37	1
3545810	99	51	43,6	cyclictest	378	-21	dbus-daemon	10:55:06	1
3545810	99	51	43,5	cyclictest	3627424	-21	kworker/u4:6+events_unbound	10:25:00	1
3545810	99	51	41,8	cyclictest	3541996	-21	kworker/u4:7+events_unbound	09:05:12	1
3545810	99	51	40,8	cyclictest	3573869	-21	systemd	08:59:42	1
3545809	99	51	45,4	cyclictest	3646597	-21	sshd	10:30:01	0
3545809	99	51	44,5	cyclictest	3534419	-21	kworker/u4:12+events_unbound	07:39:29	0
3545809	99	51	30,18	cyclictest	0	-21	swapper/0	11:15:39	0
3545810	99	50	41,6	cyclictest	3633546	-21	modprobe	10:14:59	1
3545810	99	50	40,8	cyclictest	3621290	-21	gltestperf	10:00:26	1
3545810	99	50	40,7	cyclictest	3566353	-21	latency_hist	08:30:02	1
3545810	99	50	39,9	cyclictest	3613159	-21	/usr/sbin/munin	09:50:37	1
3545809	99	50	42,6	cyclictest	3753951	-21	kworker/u4:11+events_unbound	12:32:41	0
3545809	99	50	42,6	cyclictest	3718885	-21	kworker/u4:5+events_unbound	12:04:01	0
3545809	99	50	40,8	cyclictest	3755899	-21	/usr/sbin/munin	12:35:14	0
3545809	99	50	40,8	cyclictest	3561359	-21	aten_r7power_en	08:10:15	0
3545809	99	50	40,7	cyclictest	3712403	-21	/usr/sbin/munin	11:45:20	0
3545809	99	50	40,7	cyclictest	3666505	-21	kworker/u4:7+events_unbound	11:15:01	0
3545810	99	49	42,5	cyclictest	3578186	-21	who	09:10:41	1
3545810	99	49	41,6	cyclictest	3760048	-21	latency_hist	12:40:01	1
3545810	99	49	41,5	cyclictest	3752244	-21	/usr/sbin/munin	12:30:26	1
3545809	99	49	40,6	cyclictest	3548809	-21	fschecks_count	07:20:17	0
3545809	99	49	39,7	cyclictest	3551461	-21	gltestperf	07:30:20	0
3545809	99	49	30,17	cyclictest	3707387	-21	munin-run	11:40:01	0
3545810	99	48	41,5	cyclictest	3547029	-21	kworker/u4:4+events_unbound	07:24:26	1
3545809	99	48	42,4	cyclictest	3648883	-21	kworker/u4:11+events_unbound	10:33:00	0
3545809	99	48	41,5	cyclictest	3744035	-21	grep	12:20:45	0
3545809	99	48	40,5	cyclictest	3673080	-21	aten_r7power_po	11:00:17	0
3545809	99	48	40,5	cyclictest	3673080	-21	aten_r7power_po	11:00:16	0
3545809	99	48	40,5	cyclictest	3651269	-21	/usr/sbin/munin	10:35:16	0
3545809	99	48	39,6	cyclictest	3608056	-21	fschecks_count	09:45:19	0
3545809	99	48	39,6	cyclictest	0	-21	swapper/0	10:40:20	0
3545809	99	48	38,7	cyclictest	3546898	-21	unixbench-2d	07:10:36	0
3545809	99	48	36,9	cyclictest	3730818	-21	unixbench_multi	12:05:42	0
3545809	99	48	28,17	cyclictest	0	-21	swapper/0	12:10:40	0
3545810	99	47	41,4	cyclictest	3748296	-21	timerandwakeup	12:25:39	1
3545810	99	47	40,5	cyclictest	3554778	-21	grep	07:45:02	1
3545810	99	47	39,6	cyclictest	3570161	-21	latency_hist	08:45:02	1
3545810	99	47	38,6	cyclictest	0	-21	swapper/1	11:05:30	1
3545809	99	47	37,7	cyclictest	3569297	-21	fschecks_time	08:40:21	0
3545809	99	47	27,17	cyclictest	0	-21	swapper/0	10:20:24	0
3545810	99	46	40,4	cyclictest	3571320	-21	kworker/u4:2+events_unbound	08:50:01	1
3545810	99	46	38,6	cyclictest	3534419	-21	kworker/u4:12+events_unbound	08:00:22	1
3545810	99	46	37,7	cyclictest	3708119	-21	fschecks_time	11:40:20	1
3545809	99	46	39,4	cyclictest	0	-21	swapper/0	10:50:44	0
3545809	99	46	37,7	cyclictest	3560415	-21	gltestperf	08:05:22	0
3545809	99	46	35,9	cyclictest	3556401	-21	grep	07:50:16	0
3545809	99	46	35,8	cyclictest	3548390	-21	sendmail	07:20:00	0
3545809	99	46	33,9	cyclictest	3603406	-21	aten_r7power_cu	09:40:15	0
3545809	99	46	23,21	cyclictest	3562809	-21	fschecks_count	08:15:18	0
3545810	99	45	38,5	cyclictest	3691587	-21	unixbench_singl	11:20:39	1
3545810	99	45	36,7	cyclictest	3664359	-21	aten_r7power_cu	10:50:25	1
3545809	99	45	39,4	cyclictest	3691319	-21	switchtime	11:20:38	0
3545809	99	45	38,5	cyclictest	3594370	-21	sed	09:30:01	0
3545809	99	45	38,5	cyclictest	3550236	-21	gltestperf	07:25:19	0
3545809	99	45	35,7	cyclictest	3574872	-21	grep	09:00:47	0
3545810	99	44	38,4	cyclictest	3731635	-21	kworker/u4:4+events_unbound	12:15:02	1
3545809	99	44	37,5	cyclictest	3576619	-21	ls	09:10:01	0
3545809	99	44	35,6	cyclictest	3565545	-21	gltestperf	08:25:18	0
3545809	99	44	34,8	cyclictest	53	-21	kswapd0	10:00:09	0
3545810	99	43	36,5	cyclictest	3583488	-21	kworker/u4:7+events_unbound	09:18:21	1
3545810	99	43	36,5	cyclictest	3546218	-21	fschecks_count	07:10:19	1
3545810	99	43	35,6	cyclictest	3548301	-21	kworker/u4:9+events_unbound	07:19:24	1
3545810	99	43	35,6	cyclictest	0	-21	swapper/1	12:15:25	1
3545810	99	43	32,9	cyclictest	3558565	-21	cron	08:00:00	1
3545810	99	42	32,8	cyclictest	3561509	-21	grep	08:10:18	1
3545810	99	42	31,9	cyclictest	3720537	-21	latency_hist	11:55:02	1
3545809	99	42	20,18	cyclictest	3567529	-21	idleruntime-cro	08:34:59	0
3545809	99	41	32,6	cyclictest	3553678	-21	grep	07:40:13	0
3545809	99	41	30,7	cyclictest	3571349	-21	modprobe	08:49:59	0
3545810	99	40	33,4	cyclictest	0	-21	swapper/1	08:05:29	1
3545809	99	40	15,22	cyclictest	3573033	-21	fschecks_count	08:55:19	0
3545810	99	39	33,4	cyclictest	3545920	-21	latency_hist	07:10:02	1
3545810	99	39	31,5	cyclictest	0	-21	swapper/1	07:35:18	1
3545809	99	33	26,4	cyclictest	0	-21	swapper/0	07:10:02	0
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17