OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #7, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system

Characteristics of the highest latencies:

System rack7slot2.osadl.org (updated Fri May 22, 2026 00:44:09)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
3141399	99	127	118,7	cyclictest	3340676	-21	kworker/u4:3+events_unbound	00:17:58	0
3141399	99	123	97,7	cyclictest	3282891	-21	kworker/u4:6+events_unbound	23:22:49	0
3141399	99	115	87,17	cyclictest	3165994	-21	find	20:45:12	0
3141399	99	113	106,5	cyclictest	3179608	-21	kworker/u4:9+events_unbound	21:42:30	0
3141399	99	112	104,6	cyclictest	3304784	-21	kworker/u4:9+events_unbound	23:47:53	0
3141399	99	110	102,6	cyclictest	3300942	-21	kworker/u4:0+events_unbound	23:32:51	0
3141399	99	104	96,6	cyclictest	3270001	-21	kworker/u4:7+events_unbound	23:02:46	0
3141399	99	104	86,7	cyclictest	3229483	-21	kworker/u4:7+events_unbound	22:12:38	0
3141399	99	103	89,6	cyclictest	3287399	-21	kworker/u4:1+events_unbound	23:17:50	0
3141399	99	102	93,7	cyclictest	3206504	-21	kworker/u4:0+events_unbound	21:57:35	0
3141399	99	98	88,4	cyclictest	3179608	-21	kworker/u4:9+events_unbound	21:17:26	0
3141399	99	94	87,5	cyclictest	3304784	-21	kworker/u4:9+events_unbound	23:37:52	0
3141399	99	90	82,6	cyclictest	3155494	-21	kworker/u4:9+events_unbound	20:02:15	0
3141399	99	87	79,6	cyclictest	3156852	-21	kworker/u4:1+events_unbound	20:27:17	0
3141399	99	86	72,7	cyclictest	3142627	-21	kworker/u4:10+events_unbound	19:40:01	0
3141399	99	84	77,5	cyclictest	3215675	-21	kworker/u4:9+events_unbound	22:22:42	0
3141399	99	81	70,8	cyclictest	53	-21	kswapd0	23:30:00	0
3141399	99	77	70,5	cyclictest	3349661	-21	kworker/u4:11+events_unbound	00:27:59	0
3141399	99	77	70,5	cyclictest	3141334	-21	kworker/u4:7+events_unbound	19:32:12	0
3141400	99	76	67,7	cyclictest	3165835	-21	apt-get	20:45:07	1
3141399	99	76	69,5	cyclictest	3141334	-21	kworker/u4:7+events_unbound	19:47:15	0
3141400	99	74	55,13	cyclictest	3235656	-21	modprobe	22:20:00	1
3141399	99	74	61,11	cyclictest	3165806	-21	kworker/u4:14+events_unbound	21:12:25	0
3141399	99	73	64,7	cyclictest	3156854	-21	kworker/u4:6+events_unbound	22:30:12	0
3141400	99	72	54,15	cyclictest	3311814	-21	sh	23:45:11	1
3141399	99	70	64,4	cyclictest	3340679	-21	kworker/u4:6+events_unbound	00:40:02	0
3141399	99	69	62,5	cyclictest	3140128	-21	kworker/u4:0+events_unbound	19:52:15	0
3141400	99	67	52,9	cyclictest	3141398	-21	cyclictest	00:23:35	1
3141399	99	67	59,6	cyclictest	3327267	-21	kworker/u4:13+events_unbound	00:08:19	0
3141400	99	66	58,6	cyclictest	3196207	-21	ld	21:35:38	1
3141399	99	66	59,5	cyclictest	3327267	-21	kworker/u4:13+events_unbound	00:10:14	0
3141400	99	64	57,5	cyclictest	3327267	-21	kworker/u4:13+events_unbound	00:10:20	1
3141400	99	64	54,6	cyclictest	0	-21	swapper/1	19:45:11	1
3141399	99	64	57,5	cyclictest	53	-21	kswapd0	21:52:40	0
3141400	99	63	56,5	cyclictest	3229482	-21	kworker/u4:4+events_unbound	22:20:14	1
3141399	99	63	54,7	cyclictest	3220289	-21	kworker/u4:12+events_unbound	22:10:00	0
3141399	99	63	54,7	cyclictest	3220289	-21	kworker/u4:12+events_unbound	22:10:00	0
3141399	99	63	53,8	cyclictest	3156854	-21	kworker/u4:6+events_unbound	22:35:35	0
3141400	99	62	55,5	cyclictest	53	-21	kswapd0	00:30:12	1
3141400	99	62	55,5	cyclictest	53	-21	kswapd0	00:30:12	1
3141400	99	61	56,3	cyclictest	3296434	-21	kworker/u4:5+events_unbound	00:02:56	1
3141400	99	61	56,3	cyclictest	3296434	-21	kworker/u4:5+events_unbound	00:02:56	1
3141399	99	61	18,31	cyclictest	3141395	-21	awk	19:10:00	0
3141400	99	59	49,7	cyclictest	53	-21	kswapd0	23:45:00	1
3141400	99	59	43,13	cyclictest	53	-21	kswapd0	22:30:19	1
3141399	99	59	50,7	cyclictest	53	-21	kswapd0	21:45:36	0
3141399	99	59	50,7	cyclictest	3256431	-21	kworker/u4:4+events_unbound	23:05:16	0
3141400	99	58	46,9	cyclictest	378	-21	dbus-daemon	00:17:57	1
3141399	99	58	48,8	cyclictest	3269999	-21	kworker/u4:5+events_unbound	23:13:59	0
3141400	99	57	47,7	cyclictest	0	-21	swapper/1	23:15:11	1
3141400	99	57	43,9	cyclictest	3250439	-21	ssh	22:35:36	1
3141399	99	57	50,5	cyclictest	3156851	-21	kworker/u4:0+events_unbound	20:32:18	0
3141399	99	57	45,10	cyclictest	3260959	-21	kworker/u4:10+events_unbound	22:56:00	0
3141399	99	57	43,5	cyclictest	3304785	-21	kworker/u4:10+events_unbound	23:42:59	0
3141399	99	57	38,7	cyclictest	3327264	-21	kworker/u4:6+events_unbound	00:04:59	0
3141399	99	57	38,7	cyclictest	3327264	-21	kworker/u4:6+events_unbound	00:04:59	0
3141400	99	56	42,10	cyclictest	3329640	-21	cut	00:05:15	1
3141400	99	55	47,6	cyclictest	3309963	-21	kworker/u4:3+events_unbound	23:58:09	1
3141400	99	55	43,8	cyclictest	3172088	-21	timerwakeupswit	21:05:38	1
3141399	99	55	44,9	cyclictest	3179609	-21	kworker/u4:10+events_unbound	21:32:00	0
3141399	99	55	38,15	cyclictest	3255292	-21	unixbench-2d	22:40:40	0
3141399	99	55	36,7	cyclictest	3143906	-21	kworker/u4:11+events_unbound	20:19:01	0
3141400	99	54	41,7	cyclictest	3256429	-21	kworker/u4:0-rpciod	23:10:18	1
3141400	99	54	20,22	cyclictest	0	-21	swapper/1	23:30:29	1
3141399	99	54	47,5	cyclictest	3340680	-21	kworker/u4:7+events_unbound	00:30:04	0
3141399	99	54	47,5	cyclictest	3340680	-21	kworker/u4:7+events_unbound	00:30:04	0
3141399	99	54	42,10	cyclictest	3182939	-21	timerwakeupswit	21:20:40	0
3141400	99	53	43,7	cyclictest	3254343	-21	/usr/sbin/munin	22:40:19	1
3141400	99	53	41,8	cyclictest	0	-21	swapper/1	22:00:35	1
3141400	99	53	41,7	cyclictest	3222948	-21	uname	22:05:20	1
3141400	99	53	41,7	cyclictest	3222948	-21	uname	22:05:20	1
3141400	99	53	15,11	cyclictest	0	-21	swapper/1	22:55:26	1
3141400	99	52	44,6	cyclictest	3141398	-21	cyclictest	00:40:00	1
3141400	99	52	38,11	cyclictest	0	-21	swapper/1	23:50:12	1
3141399	99	52	45,5	cyclictest	3256431	-21	kworker/u4:4+events_unbound	22:49:46	0
3141399	99	52	45,5	cyclictest	3165806	-21	kworker/u4:14+events_unbound	21:02:23	0
3141399	99	52	42,8	cyclictest	3154199	-21	systemd	19:57:14	0
3141399	99	52	40,7	cyclictest	3215675	-21	kworker/u4:9+events_unbound	22:25:35	0
3141400	99	51	46,3	cyclictest	0	-21	swapper/1	20:15:23	1
3141400	99	51	45,4	cyclictest	3291912	-21	kworker/u4:3+events_unbound	23:25:37	1
3141400	99	51	40,8	cyclictest	3170729	-21	switchtime	21:00:36	1
3141400	99	51	39,7	cyclictest	0	-21	swapper/1	21:20:13	1
3141400	99	51	38,11	cyclictest	3161999	-21	cron	20:30:00	1
3141399	99	51	32,17	cyclictest	3264084	-21	threads	22:50:38	0
3141399	99	51	31,6	cyclictest	3156854	-21	kworker/u4:6+events_unbound	22:05:01	0
3141400	99	50	39,7	cyclictest	3303510	-21	/usr/sbin/munin	23:35:21	1
3141400	99	50	38,8	cyclictest	0	-21	swapper/1	21:55:39	1
3141400	99	50	36,9	cyclictest	0	-21	swapper/1	21:40:25	1
3141400	99	50	30,12	cyclictest	3227732	-21	ssh	22:10:31	1
3141399	99	50	44,4	cyclictest	3179608	-21	kworker/u4:9+events_unbound	21:28:00	0
3141399	99	50	41,7	cyclictest	3340676	-21	kworker/u4:3+events_unbound	00:21:45	0
3141399	99	50	34,8	cyclictest	256	50	irq/23-42890000.ethernet	20:05:17	0
3141400	99	49	7,35	cyclictest	3155777	-21	apt-get	20:05:15	1
3141400	99	49	41,6	cyclictest	3143952	-21	idleruntime-cro	19:20:00	1
3141400	99	49	39,7	cyclictest	3272998	-21	/usr/sbin/munin	23:00:38	1
3141399	99	49	42,5	cyclictest	3232250	-21	ntpq	22:15:31	0
3141399	99	49	27,20	cyclictest	3171464	-21	fschecks_count	21:05:20	0
3141400	99	48	41,5	cyclictest	53	-21	kswapd0	22:45:12	1
3141400	99	48	25,6	cyclictest	0	-21	swapper/1	22:25:40	1
3141400	99	48	14,18	cyclictest	0	-21	swapper/1	21:45:27	1
3141399	99	48	41,5	cyclictest	3140129	-21	kworker/u4:2+events_unbound	19:42:13	0
3141399	99	48	41,5	cyclictest	3138894	-21	kworker/u4:5+events_unbound	19:22:11	0
3141399	99	48	41,4	cyclictest	3145405	-21	aten2__	19:25:15	0
3141399	99	48	29,7	cyclictest	3304784	-21	kworker/u4:9+events_unbound	23:51:08	0
3141400	99	47	6,30	cyclictest	3349117	-21	sh	00:26:40	1
3141400	99	47	21,6	cyclictest	0	-21	swapper/1	19:25:00	1
3141399	99	47	29,16	cyclictest	3163766	-21	fschecks_time	20:35:17	0
3141400	99	46	5,15	cyclictest	3190366	-21	ssh	21:29:53	1
3141400	99	46	38,6	cyclictest	3159958	-21	fschecks_time	20:20:18	1
3141400	99	46	38,6	cyclictest	3147770	-21	apt	19:35:00	1
3141400	99	46	37,6	cyclictest	3178492	-21	unixbench_singl	21:15:41	1
3141400	99	46	34,10	cyclictest	3176938	-21	latency_hist	21:15:01	1
3141400	99	45	4,16	cyclictest	3192592	-21	ssh	21:31:25	1
3141400	99	45	38,5	cyclictest	3150342	-21	idleruntime-cro	19:45:00	1
3141400	99	45	37,5	cyclictest	3141398	-21	cyclictest	23:05:36	1
3141400	99	45	10,11	cyclictest	0	-21	swapper/1	23:25:00	1
3141399	99	45	38,5	cyclictest	3169767	-21	awk	21:00:00	0
3141399	99	45	37,6	cyclictest	3147737	-21	kworker/u4:3+events_unbound	20:22:17	0
3141400	99	44	37,5	cyclictest	3154246	-21	awk	20:00:00	1
3141400	99	44	33,8	cyclictest	0	-21	swapper/1	22:50:28	1
3141400	99	44	33,7	cyclictest	0	-21	swapper/1	21:50:29	1
3141400	99	44	28,13	cyclictest	3169752	-21	run-parts	21:00:00	1
3141399	99	44	37,5	cyclictest	3291915	-21	kworker/u4:8+events_unbound	23:57:58	0
3141399	99	44	37,4	cyclictest	3196340	-21	mailstats	21:35:39	0
3141400	99	43	4,15	cyclictest	3142628	-21	sshd	19:12:10	1
3141400	99	43	34,7	cyclictest	3152146	-21	fschecks_count	19:50:16	1
3141400	99	43	15,17	cyclictest	0	-21	swapper/1	20:00:25	1
3141400	99	43	13,6	cyclictest	26	1	rcuc/1	20:35:10	1
3141399	99	43	36,5	cyclictest	3143309	-21	kernelversion	19:15:21	0
3141399	99	43	17,6	cyclictest	17	1	rcu_preempt	20:45:31	0
3141400	99	42	35,5	cyclictest	3167775	-21	cut	20:50:20	1
3141400	99	42	32,7	cyclictest	3163398	-21	latency_hist	20:35:01	1
3141399	99	42	34,6	cyclictest	3142536	-21	unixbench_multi	19:10:38	0
3141399	99	41	34,5	cyclictest	3156857	-21	kworker/u4:10+events_unbound	20:15:00	0
3141400	99	40	31,6	cyclictest	0	-21	swapper/1	19:35:18	1
3141400	99	40	12,6	cyclictest	27	1	ktimers/1	19:25:20	1
3141399	99	40	34,4	cyclictest	3167655	-21	fschecks_time	20:50:18	0
3141400	99	38	31,5	cyclictest	0	-21	swapper/1	20:10:21	1
3141400	99	38	28,7	cyclictest	3165805	-21	kworker/u4:7+rpciod	20:45:21	1
3141400	99	37	27,7	cyclictest	3141484	-21	latency_hist	19:10:01	1
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17