OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #7, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system

Characteristics of the 792 highest latencies:

System rack7slot2.osadl.org (updated Mon May 25, 2026 12:44:08)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
3631003	99	99	87,8	cyclictest	3708357	-21	kworker/u4:14+events_unbound	10:15:31	1
3631003	99	98	82,6	cyclictest	3630931	-21	kworker/u4:5+events_unbound	08:45:15	1
3631002	99	98	94,3	cyclictest	19	1	rcuc/0	11:50:52	0
3631002	99	98	90,6	cyclictest	3822823	-21	kworker/u4:5+events_unbound	12:36:01	0
3631003	99	95	81,10	cyclictest	3654204	-21	kworker/u4:1+events_unbound	08:40:14	1
3631003	99	92	84,6	cyclictest	3770025	-21	kworker/u4:6+events_unbound	11:20:47	1
3631002	99	91	83,6	cyclictest	3638702	-21	kworker/u4:11+events_unbound	07:49:56	0
3631003	99	87	78,6	cyclictest	3730609	-21	kworker/u4:6+events_unbound	10:30:31	1
3631003	99	86	77,7	cyclictest	3800516	-21	kworker/u4:5+events_unbound	11:50:12	1
3631002	99	86	78,6	cyclictest	3641272	-21	kworker/u4:14+events_unbound	07:54:56	0
3631002	99	84	72,4	cyclictest	3630932	-21	kworker/u4:6+events_unbound	09:00:21	0
3631003	99	83	76,5	cyclictest	3628493	-21	kworker/u4:0+events_unbound	07:19:45	1
3631002	99	83	76,5	cyclictest	3643808	-21	kworker/u4:4+events_unbound	08:04:58	0
3631003	99	80	60,7	cyclictest	3691228	-21	kworker/u4:6+events_unbound	10:00:28	1
3631002	99	79	67,9	cyclictest	3708357	-21	kworker/u4:14+events_unbound	10:25:30	0
3631003	99	77	68,6	cyclictest	3730608	-21	kworker/u4:4+events_unbound	11:00:41	1
3631002	99	77	34,40	cyclictest	53	-21	kswapd0	12:25:12	0
3631003	99	74	67,5	cyclictest	3630930	-21	kworker/u4:3+events_unbound	08:20:01	1
3631003	99	73	60,5	cyclictest	3831683	-21	kworker/u4:10+events_unbound	12:31:01	1
3631002	99	69	62,5	cyclictest	3630930	-21	kworker/u4:3+events_unbound	07:10:03	0
3631003	99	68	62,4	cyclictest	3765575	-21	kworker/u4:7+events_unbound	11:40:50	1
3631003	99	67	62,3	cyclictest	3770024	-21	kworker/u4:2+events_unbound	11:15:46	1
3631003	99	67	59,6	cyclictest	3686840	-21	kworker/u4:0+events_unbound	09:55:01	1
3631003	99	67	59,6	cyclictest	3645110	-21	kworker/u4:0+events_unbound	08:35:13	1
3631003	99	66	59,5	cyclictest	3630930	-21	kworker/u4:3+events_unbound	08:10:00	1
3631003	99	65	58,5	cyclictest	3646416	-21	apt-get	08:10:13	1
3631002	99	65	6,42	cyclictest	3698448	-21	ssh	09:50:30	0
3631002	99	65	57,5	cyclictest	3778981	-21	kworker/u4:10+events_unbound	11:45:51	0
3631003	99	64	56,5	cyclictest	3673289	-21	kworker/u4:11+events_unbound	09:25:25	1
3631003	99	62	55,5	cyclictest	3646382	-21	kworker/u4:9+events_unbound	09:05:01	1
3631003	99	61	46,7	cyclictest	3654204	-21	kworker/u4:1+events_unbound	09:45:27	1
3631002	99	59	51,6	cyclictest	3721670	-21	kworker/u4:2+events_unbound	10:20:31	0
3631003	99	58	52,4	cyclictest	3632227	-21	kworker/u4:11+events_unbound	07:14:41	1
3631003	99	58	49,6	cyclictest	3844379	-21	kworker/u4:3+events_unbound	12:40:01	1
3631002	99	58	45,8	cyclictest	3743583	-21	ssh	10:41:58	0
3631003	99	57	48,6	cyclictest	3700066	-21	kworker/u4:10+events_unbound	09:55:27	1
3631002	99	57	48,7	cyclictest	3706836	-21	/usr/sbin/munin	10:00:20	0
3631003	99	56	40,5	cyclictest	3739506	-21	kworker/u4:5+events_unbound	10:40:34	1
3631002	99	56	47,6	cyclictest	3715639	-21	gltestperf	10:10:20	0
3631003	99	55	46,7	cyclictest	53	-21	kswapd0	10:40:00	1
3631003	99	55	46,7	cyclictest	3645110	-21	kworker/u4:0+events_unbound	08:55:17	1
3631002	99	55	48,5	cyclictest	3695671	-21	kworker/u4:3+events_unbound	10:08:00	0
3631002	99	55	47,6	cyclictest	3704515	-21	kworker/u4:12+events_unbound	10:00:00	0
3631002	99	55	42,6	cyclictest	3630932	-21	kworker/u4:6+events_unbound	08:45:02	0
3631002	99	55	36,16	cyclictest	3758637	-21	apt-get	11:00:14	0
3631002	99	55	3,40	cyclictest	3746344	-21	irqstats	10:45:21	0
3631002	99	55	32,20	cyclictest	53	-21	kswapd0	12:00:12	0
3631003	99	54	46,6	cyclictest	3752198	-21	kworker/u4:7+events_unbound	10:55:11	1
3631003	99	54	45,6	cyclictest	53	-21	kswapd0	10:20:35	1
3631003	99	54	42,9	cyclictest	3700066	-21	kworker/u4:10-rpciod	10:10:40	1
3631002	99	54	46,6	cyclictest	3765573	-21	kworker/u4:4+events_unbound	11:13:59	0
3631002	99	54	45,6	cyclictest	3830455	-21	/usr/sbin/munin	12:20:47	0
3631002	99	54	44,7	cyclictest	3676324	-21	/usr/sbin/munin	09:25:24	0
3631003	99	53	42,8	cyclictest	3632226	-21	kworker/u4:10+events_unbound	08:50:17	1
3631002	99	53	45,6	cyclictest	3765573	-21	kworker/u4:4+events_unbound	11:25:16	0
3631002	99	53	32,17	cyclictest	3656766	-21	kworker/u4:3-events_unbound	09:20:23	0
3631003	99	52	45,5	cyclictest	3834780	-21	sed	12:25:44	1
3631003	99	52	45,5	cyclictest	3632226	-21	kworker/u4:10+events_unbound	09:05:22	1
3631003	99	52	44,6	cyclictest	3813897	-21	kworker/u4:11+events_unbound	12:11:58	1
3631003	99	52	44,6	cyclictest	3632222	-21	kworker/u4:8+events_unbound	07:25:01	1
3631002	99	52	41,8	cyclictest	3817104	-21	/usr/sbin/munin	12:05:39	0
3631003	99	51	44,5	cyclictest	3812556	-21	/usr/sbin/munin	12:00:45	1
3631003	99	51	44,5	cyclictest	3765575	-21	kworker/u4:7+events_unbound	11:10:44	1
3631003	99	51	35,7	cyclictest	3654204	-21	kworker/u4:1+events_unbound	09:35:59	1
3631002	99	51	43,6	cyclictest	3778981	-21	kworker/u4:10+events_unbound	11:43:52	0
3631002	99	51	43,6	cyclictest	3656770	-21	kworker/u4:4+events_unbound	08:50:04	0
3631003	99	50	40,8	cyclictest	3746901	-21	timerandwakeup	10:45:38	1
3631003	99	50	38,10	cyclictest	3666598	-21	latency_hist	09:15:06	1
3631002	99	50	5,16	cyclictest	3733523	-21	ntpq	10:30:34	0
3631002	99	50	43,5	cyclictest	3681288	-21	timerwakeupswit	09:30:38	0
3631002	99	50	38,9	cyclictest	3820404	-21	aten_r7power_en	12:10:16	0
3631002	99	50	26,21	cyclictest	3737374	-21	ssh	10:35:20	0
3631003	99	49	41,6	cyclictest	3673289	-21	kworker/u4:11+events_unbound	09:23:46	1
3631002	99	49	42,5	cyclictest	3773112	-21	cat	11:15:37	0
3631002	99	49	42,5	cyclictest	3764204	-21	uname	11:05:35	0
3631002	99	49	41,6	cyclictest	3791611	-21	kworker/u4:16+events_unbound	11:38:45	0
3631002	99	49	38,6	cyclictest	3659395	-21	kworker/u4:8-xprtiod	09:40:40	0
3631002	99	49	29,17	cyclictest	0	-21	swapper/0	12:30:21	0
3631003	99	48	8,26	cyclictest	3807116	-21	ssh	11:55:13	1
3631003	99	48	41,5	cyclictest	3711013	-21	/usr/sbin/munin	10:05:17	1
3631003	99	48	41,5	cyclictest	3681228	-21	switchtime	09:30:37	1
3631003	99	48	41,5	cyclictest	3642385	-21	timerwakeupswit	07:50:38	1
3631003	99	48	40,6	cyclictest	3691228	-21	kworker/u4:6+events_unbound	09:42:59	1
3631003	99	48	38,7	cyclictest	3790362	-21	sh	11:35:29	1
3631002	99	48	37,9	cyclictest	3693983	-21	netstat	09:45:27	0
3631002	99	48	35,10	cyclictest	3756651	-21	ssh	10:56:53	0
3631002	99	48	32,5	cyclictest	3778325	-21	kworker/u4:8+events_unbound	11:24:14	0
3631003	99	47	39,6	cyclictest	3778325	-21	kworker/u4:8+events_unbound	11:25:35	1
3631003	99	47	38,7	cyclictest	3816974	-21	switchtime	12:05:38	1
3631003	99	47	37,7	cyclictest	3635113	-21	aten_r7power_po	07:25:17	1
3631003	99	47	35,7	cyclictest	0	-21	swapper/1	10:50:28	1
3631002	99	47	40,5	cyclictest	3639998	-21	kworker/u4:12+events_unbound	08:31:46	0
3631002	99	47	38,6	cyclictest	3811270	-21	latency_hist	12:00:02	0
3631002	99	47	37,7	cyclictest	3824844	-21	aten_r7power_vo	12:15:16	0
3631003	99	46	37,6	cyclictest	3804313	-21	kworker/u4:6+events_unbound	12:15:26	1
3631003	99	46	35,8	cyclictest	3667996	-21	unixbench_multi	09:15:41	1
3631003	99	46	25,7	cyclictest	3647738	-21	kworker/u4:13+events_unbound	08:20:30	1
3631002	99	46	40,5	cyclictest	3747760	-21	kworker/u4:6+events_unbound	10:51:45	0
3631002	99	46	40,4	cyclictest	3663502	-21	timerwakeupswit	09:10:40	0
3631002	99	46	39,5	cyclictest	3712812	-21	kworker/u4:18+rpciod	10:15:20	0
3631002	99	46	35,7	cyclictest	2	-21	kthreadd	11:31:40	0
3631002	99	46	35,5	cyclictest	3632226	-21	kworker/u4:10+events_unbound	09:05:45	0
3631002	99	46	26,17	cyclictest	3645110	-21	kworker/u4:0+rpciod	08:55:41	0
3631003	99	45	31,9	cyclictest	0	-21	swapper/1	10:27:47	1
3631002	99	45	39,5	cyclictest	3630932	-21	kworker/u4:6+events_unbound	08:09:01	0
3631002	99	45	38,5	cyclictest	3642931	-21	df_abs	07:55:14	0
3631002	99	45	38,5	cyclictest	3630930	-21	kworker/u4:3+events_unbound	08:11:01	0
3631002	99	45	30,5	cyclictest	3641272	-21	kworker/u4:14+events_unbound	08:35:07	0
3631002	99	45	24,6	cyclictest	0	-21	swapper/0	09:35:39	0
3631003	99	44	35,6	cyclictest	3763894	-21	ssh	11:05:23	1
3631002	99	44	35,7	cyclictest	3669376	-21	ssh	09:17:40	0
3631002	99	44	35,7	cyclictest	3632738	-21	fschecks_time	07:15:19	0
3631002	99	44	31,5	cyclictest	3628494	-21	kworker/u4:2+events_unbound	07:24:49	0
3631002	99	44	30,12	cyclictest	3655513	-21	kworker/u4:1	08:45:00	0
3631003	99	43	35,5	cyclictest	3785668	-21	gltestperf	11:30:19	1
3631003	99	43	33,7	cyclictest	3830333	-21	grep	12:20:45	1
3631002	99	43	33,7	cyclictest	3636617	-21	fschecks_time	07:30:19	0
3631003	99	42	36,4	cyclictest	3640007	-21	systemd	07:44:50	1
3631003	99	42	34,5	cyclictest	0	-21	swapper/1	11:45:35	1
3631003	99	42	30,9	cyclictest	0	-21	swapper/1	07:45:22	1
3631002	99	42	35,5	cyclictest	3637764	-21	aten_r7power_vo	07:35:21	0
3631002	99	42	34,5	cyclictest	3647738	-21	kworker/u4:13-rpciod	08:20:35	0
3631002	99	42	32,7	cyclictest	3650769	-21	tune2fs	08:25:20	0
3631002	99	41	34,5	cyclictest	3636071	-21	kworker/u4:1+events_unbound	07:42:00	0
3631002	99	41	33,6	cyclictest	3635326	-21	fschecks_time	07:25:20	0
3631003	99	40	33,5	cyclictest	3643856	-21	sendmail	08:00:00	1
3631003	99	40	29,9	cyclictest	3652885	-21	idleruntime-cro	08:35:00	1
3631003	99	39	31,6	cyclictest	3636071	-21	kworker/u4:1+events_unbound	07:35:02	1
3631002	99	39	31,5	cyclictest	3646381	-21	kworker/u4:2-nfsiod	08:15:27	0
3631003	99	38	30,6	cyclictest	0	-21	swapper/1	08:00:21	1
3631003	99	37	31,5	cyclictest	3630931	-21	kworker/u4:5+events_unbound	08:30:00	1
3631003	99	37	31,4	cyclictest	3636071	-21	kworker/u4:1-rpciod	07:35:01	1
3631002	99	32	24,6	cyclictest	3628493	-21	kworker/u4:0-rpciod	07:10:01	0
3631003	99	30	21,7	cyclictest	0	-21	swapper/1	07:10:01	1
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17