OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #7, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system

Characteristics of the 376 highest latencies:

System rack7slot2.osadl.org (updated Wed Jul 22, 2026 12:44:09)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
3466559	99	122	77,40	cyclictest	53	-21	kswapd0	11:45:13	0
3466560	99	105	94,8	cyclictest	3545958	-21	kworker/u4:2+events_unbound	10:54:07	1
3466560	99	86	45,7	cyclictest	3608162	-21	kthreadd	11:12:05	1
3466560	99	83	75,6	cyclictest	3471721	-21	kworker/u4:14+events_unbound	07:33:34	1
3466560	99	79	70,6	cyclictest	3488401	-21	kworker/u4:12+events_unbound	09:23:54	1
3466560	99	77	43,8	cyclictest	3619063	-21	latency_hist	11:25:02	1
3466560	99	75	40,5	cyclictest	3619715	-21	fschecks_time	11:25:25	1
3466560	99	74	43,5	cyclictest	3471001	-21	grep	07:25:20	1
3466559	99	73	3,62	cyclictest	3478132	-21	apt-get	07:55:11	0
3466560	99	71	63,6	cyclictest	3683854	-21	kworker/u4:0+events_unbound	12:39:25	1
3466560	99	69	61,6	cyclictest	3483288	-21	kworker/u4:1+events_unbound	08:43:48	1
3466559	99	69	25,40	cyclictest	3556595	-21	apt-get	10:15:13	0
3466559	99	66	54,7	cyclictest	0	-21	swapper/0	11:25:09	0
3466560	99	65	57,6	cyclictest	3572681	-21	kworker/u4:8+events_unbound	10:34:04	1
3466560	99	63	46,11	cyclictest	0	-21	swapper/1	08:50:34	1
3466559	99	63	43,18	cyclictest	3541316	-21	kworker/u4:4+events_unbound	10:09:59	0
3466559	99	63	1,54	cyclictest	3467821	-21	apt-get	07:15:12	0
3466559	99	61	11,16	cyclictest	3588101	-21	apt-get	10:50:12	0
3466560	99	60	7,38	cyclictest	3534944	-21	ssh	09:50:26	1
3466560	99	59	51,5	cyclictest	3454034	-21	kworker/u4:13+events_unbound	07:38:36	1
3466560	99	58	50,6	cyclictest	3470451	-21	kworker/u4:12+events_unbound	07:23:34	1
3466560	99	58	38,6	cyclictest	3559018	-21	kworker/u4:0+events_unbound	10:19:03	1
3466559	99	58	49,6	cyclictest	3525253	-21	aten_r7power_en	09:40:14	0
3466560	99	57	52,3	cyclictest	3476823	-21	kworker/u4:7+events_unbound	08:18:42	1
3466559	99	57	2,41	cyclictest	37	-21	kcompactd0	10:55:07	0
3466559	99	57	14,16	cyclictest	3565835	-21	apt-get	10:25:12	0
3466560	99	56	48,6	cyclictest	3553632	-21	/usr/sbin/munin	10:10:42	1
3466560	99	55	46,7	cyclictest	3513594	-21	kworker/u4:7+events_unbound	09:48:59	1
3466560	99	55	45,7	cyclictest	3603650	-21	kworker/u4:3+events_unbound	11:45:16	1
3466559	99	55	28,21	cyclictest	3482047	-21	modprobe	08:10:00	0
3466560	99	54	48,4	cyclictest	3627902	-21	idleruntime-cro	11:35:00	1
3466560	99	53	46,5	cyclictest	3668053	-21	sendmail-msp	12:20:01	1
3466559	99	53	46,5	cyclictest	3664383	-21	/usr/sbin/munin	12:15:22	0
3466560	99	52	6,41	cyclictest	3651310	-21	ssh	12:00:35	1
3466560	99	52	44,6	cyclictest	3561688	-21	fschecks_count	10:20:19	1
3466559	99	52	45,5	cyclictest	3583441	-21	latency_hist	10:45:00	0
3466559	99	52	45,5	cyclictest	3506602	-21	idleruntime-cro	09:20:01	0
3466559	99	52	43,7	cyclictest	3585038	-21	/usr/sbin/munin	10:45:45	0
3466559	99	52	33,17	cyclictest	3575233	-21	gltestperf	10:35:20	0
3466560	99	51	45,4	cyclictest	3593738	-21	switchtime	10:55:45	1
3466560	99	51	44,5	cyclictest	3650257	-21	latency_hist	12:00:02	1
3466560	99	51	44,5	cyclictest	3650257	-21	latency_hist	12:00:01	1
3466559	99	51	43,6	cyclictest	3655556	-21	/usr/sbin/munin	12:05:25	0
3466559	99	51	29,20	cyclictest	3491435	-21	fschecks_count	08:45:19	0
3466559	99	51	18,12	cyclictest	0	-21	swapper/0	12:00:16	0
3466559	99	51	17,21	cyclictest	0	-21	swapper/0	09:20:23	0
3466560	99	50	41,6	cyclictest	3579383	-21	/usr/sbin/munin	10:40:18	1
3466560	99	50	36,12	cyclictest	3570043	-21	latency_hist	10:30:01	1
3466559	99	50	4,38	cyclictest	3602346	-21	ssh	11:05:34	0
3466559	99	50	2,36	cyclictest	3538835	-21	ssh	09:55:16	0
3466559	99	50	11,3	cyclictest	15	1	ktimers/0	12:25:39	0
3466560	99	49	42,5	cyclictest	3502806	-21	gltestperf	09:15:23	1
3466560	99	49	41,5	cyclictest	0	-21	swapper/1	11:15:26	1
3466560	99	49	37,9	cyclictest	0	-21	swapper/1	11:05:28	1
3466559	99	49	42,5	cyclictest	3629286	-21	diskmemload	11:35:37	0
3466559	99	49	39,7	cyclictest	3482614	-21	gltestperf	08:10:21	0
3466559	99	49	31,16	cyclictest	3636816	-21	http	11:45:00	0
3466559	99	49	18,28	cyclictest	3570277	-21	apt-get	10:30:12	0
3466560	99	48	41,5	cyclictest	3659094	-21	cut	12:09:59	1
3466560	99	48	41,5	cyclictest	3584078	-21	fschecks_time	10:45:20	1
3466560	99	48	40,5	cyclictest	3515838	-21	wc	09:30:01	1
3466559	99	48	37,9	cyclictest	3535457	-21	timerwakeupswit	09:50:39	0
3466559	99	48	14,7	cyclictest	3477023	-21	apt-get	07:50:13	0
3466560	99	47	36,8	cyclictest	0	-21	swapper/1	09:40:31	1
3466559	99	47	40,5	cyclictest	3490093	-21	df_abs	08:40:17	0
3466559	99	47	39,6	cyclictest	3497468	-21	diskmemload	10:13:58	0
3466559	99	47	39,6	cyclictest	0	-21	swapper/0	11:15:35	0
3466559	99	47	27,17	cyclictest	15	1	ktimers/0	09:10:33	0
3466559	99	47	27,17	cyclictest	0	-21	swapper/0	11:25:01	0
3466560	99	46	5,30	cyclictest	3678114	-21	ssh	12:30:36	1
3466560	99	46	40,5	cyclictest	3541322	-21	kworker/u4:13+events_unbound	10:00:01	1
3466560	99	46	38,6	cyclictest	3551392	-21	sshd	10:09:02	1
3466560	99	46	38,6	cyclictest	3484566	-21	kworker/u4:0+events_unbound	09:00:00	1
3466559	99	46	38,6	cyclictest	3511567	-21	aten_r7power_cu	09:25:19	0
3466559	99	46	28,15	cyclictest	3601179	-21	latency_hist	11:05:01	0
3466560	99	45	39,4	cyclictest	3516353	-21	fschecks_count	09:30:20	1
3466560	99	45	38,5	cyclictest	3488441	-21	ls	08:35:01	1
3466560	99	45	37,6	cyclictest	3478092	-21	kworker/u4:9+events_unbound	08:28:45	1
3466560	99	45	37,5	cyclictest	3483288	-21	kworker/u4:1-rpciod	09:35:19	1
3466559	99	45	7,13	cyclictest	3565129	-21	ssh	10:24:24	0
3466559	99	45	39,4	cyclictest	3668396	-21	aten_r7power_cu	12:20:15	0
3466559	99	45	39,4	cyclictest	3534060	-21	latency_hist	09:50:02	0
3466559	99	45	35,7	cyclictest	3623897	-21	aten_r7power_vo	11:30:17	0
3466559	99	45	34,9	cyclictest	3496094	-21	kworker/u4:4	09:05:01	0
3466559	99	45	29,14	cyclictest	3483288	-21	kworker/u4:1-xprtiod	09:35:32	0
3466559	99	45	26,16	cyclictest	0	-21	swapper/0	12:39:59	0
3466559	99	45	23,4	cyclictest	15	1	ktimers/0	12:00:00	0
3466559	99	45	23,4	cyclictest	15	1	ktimers/0	11:59:59	0
3466559	99	45	18,8	cyclictest	0	-21	swapper/0	08:50:27	0
3466559	99	45	17,16	cyclictest	0	-21	swapper/0	08:15:38	0
3466559	99	45	11,27	cyclictest	256	50	irq/23-42890000.ethernet	09:05:11	0
3466560	99	44	37,5	cyclictest	3544359	-21	sendmail_mailqu	10:00:41	1
3466560	99	44	36,6	cyclictest	3657320	-21	kworker/u4:11-rpciod	12:10:44	1
3466560	99	44	36,5	cyclictest	3621773	-21	kworker/u4:5-rpciod	11:50:33	1
3466560	99	44	36,5	cyclictest	3621773	-21	kworker/u4:5-rpciod	11:50:32	1
3466560	99	44	36,5	cyclictest	0	-21	swapper/1	08:40:00	1
3466559	99	44	6,14	cyclictest	3476396	-21	ntpq	07:45:33	0
3466559	99	44	38,4	cyclictest	3677620	-21	grep	12:30:21	0
3466559	99	44	37,5	cyclictest	3494065	-21	gltestperf	08:55:22	0
3466559	99	44	37,5	cyclictest	3474672	-21	df_abs	07:40:18	0
3466559	99	44	36,6	cyclictest	3486301	-21	fschecks_count	08:25:22	0
3466559	99	44	35,7	cyclictest	3489725	-21	idleruntime-cro	08:39:59	0
3466559	99	44	0,25	cyclictest	3473713	-21	netstat	07:35:22	0
3466559	99	44	0,24	cyclictest	1073	-21	snmpd	10:02:53	0
3466560	99	43	37,4	cyclictest	0	-21	swapper/1	12:25:12	1
3466560	99	43	36,5	cyclictest	3476182	-21	gltestperf	07:45:21	1
3466560	99	43	36,5	cyclictest	1069	-21	lldpd	10:36:10	1
3466560	99	43	35,4	cyclictest	3661808	-21	kworker/u4:5-rpciod	12:20:32	1
3466560	99	43	34,7	cyclictest	3600134	-21	sshd	11:03:25	1
3466560	99	43	22,8	cyclictest	3452736	-21	kworker/u4:11+rpciod	07:10:18	1
3466559	99	43	33,7	cyclictest	3516146	-21	/usr/sbin/munin	09:30:18	0
3466559	99	43	23,17	cyclictest	15	1	ktimers/0	11:10:13	0
3466559	99	43	18,2	cyclictest	15	1	ktimers/0	11:50:19	0
3466559	99	43	18,2	cyclictest	15	1	ktimers/0	11:50:19	0
3466560	99	42	35,5	cyclictest	3468389	-21	fschecks_time	07:15:31	1
3466560	99	42	34,6	cyclictest	3475454	-21	unixbench_multi	07:40:39	1
3466560	99	42	33,7	cyclictest	3621773	-21	kworker/u4:5-nfsiod	11:40:34	1
3466560	99	42	33,7	cyclictest	3477105	-21	aten_r7power_en	07:50:17	1
3466560	99	42	30,7	cyclictest	0	-21	swapper/1	09:10:09	1
3466560	99	42	25,5	cyclictest	3621772	-21	kworker/u4:4+events_unbound	11:39:13	1
3466559	99	42	36,4	cyclictest	0	-21	swapper/0	07:33:33	0
3466559	99	42	35,5	cyclictest	3487350	-21	aten2__	08:30:16	0
3466559	99	42	34,6	cyclictest	3484817	-21	aten2__	08:20:16	0
3466560	99	41	34,5	cyclictest	3478096	-21	kworker/u4:11+events_unbound	07:58:39	1
3466560	99	41	32,6	cyclictest	3479870	-21	sed	08:00:18	1
3466560	99	41	22,8	cyclictest	3467786	-21	kworker/u4:8+rpciod	08:10:19	1
3466559	99	41	34,4	cyclictest	3469776	-21	cut	07:20:22	0
3466559	99	41	33,5	cyclictest	3467778	-21	kworker/u4:2-rpciod	07:25:40	0
3466559	99	41	32,6	cyclictest	3466971	-21	fschecks_count	07:10:18	0
3466560	99	40	32,6	cyclictest	3481874	-21	timerwakeupswit	08:05:37	1
3466560	99	40	32,6	cyclictest	3478096	-21	kworker/u4:11-rpciod	08:20:19	1
3466560	99	40	32,5	cyclictest	0	-21	swapper/1	09:00:13	1
3466559	99	40	6,16	cyclictest	15	1	ktimers/0	12:10:43	0
3466560	99	39	33,4	cyclictest	0	-21	swapper/1	09:05:25	1
3466560	99	38	29,6	cyclictest	0	-21	swapper/1	07:10:01	1
3466560	99	38	27,9	cyclictest	0	-21	swapper/1	08:45:21	1
3466559	99	38	31,4	cyclictest	0	-21	swapper/0	08:00:20	0
3466559	99	33	27,4	cyclictest	3452736	-21	kworker/u4:11+rpciod	07:10:02	0
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17