OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #7, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system

Characteristics of the 376 highest latencies:

System rack7slot2.osadl.org (updated Sun Jul 05, 2026 12:44:11)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
650460	99	127	119,6	cyclictest	420003	-21	kworker/u4:5+events_unbound	07:14:59	0
650460	99	103	91,5	cyclictest	673582	-21	kworker/u4:3+events_unbound	09:05:22	0
650460	99	99	92,5	cyclictest	654264	-21	kworker/u4:7+events_unbound	07:35:03	0
650460	99	93	86,5	cyclictest	673584	-21	kworker/u4:7+events_unbound	08:55:20	0
650461	99	90	82,6	cyclictest	755016	-21	kworker/u4:7+events_unbound	10:40:42	1
650460	99	90	70,17	cyclictest	816507	-21	kworker/u4:2+events_unbound	12:10:59	0
650460	99	83	71,10	cyclictest	53	-21	kswapd0	10:30:33	0
650460	99	82	74,5	cyclictest	664484	-21	kworker/u4:7+events_unbound	08:30:00	0
650460	99	79	71,6	cyclictest	420003	-21	kworker/u4:5+events_unbound	07:17:01	0
650460	99	79	71,6	cyclictest	420003	-21	kworker/u4:5+events_unbound	07:17:01	0
650461	99	72	62,7	cyclictest	838842	-21	kworker/u4:3+events_unbound	12:16:00	1
650461	99	72	62,7	cyclictest	838842	-21	kworker/u4:3+events_unbound	12:16:00	1
650461	99	67	60,5	cyclictest	714551	-21	kworker/u4:6+events_unbound	10:00:01	1
650461	99	65	56,6	cyclictest	654266	-21	kworker/u4:9+events_unbound	07:50:05	1
650460	99	65	54,9	cyclictest	53	-21	kswapd0	10:00:33	0
650460	99	64	56,6	cyclictest	788047	-21	apt-get	11:10:12	0
650460	99	64	56,6	cyclictest	788047	-21	apt-get	11:10:12	0
650461	99	63	54,6	cyclictest	664483	-21	kworker/u4:6+events_unbound	08:35:01	1
650461	99	62	46,12	cyclictest	654266	-21	kworker/u4:9+events_unbound	08:25:14	1
650460	99	61	54,5	cyclictest	659376	-21	kworker/u4:12+events_unbound	08:20:04	0
650461	99	60	37,11	cyclictest	728733	-21	kworker/u4:3-rpciod	10:05:14	1
650460	99	60	53,5	cyclictest	803712	-21	kworker/u4:13+events_unbound	11:35:04	0
650460	99	60	53,5	cyclictest	660645	-21	kworker/u4:15+events_unbound	08:30:02	0
650461	99	59	36,18	cyclictest	819788	-21	ssh	11:45:33	1
650460	99	59	35,17	cyclictest	53	-21	kswapd0	11:30:10	0
650461	99	58	47,6	cyclictest	0	-21	swapper/1	11:55:33	1
650461	99	58	42,10	cyclictest	859312	-21	ssh	12:30:32	1
650461	99	58	34,19	cyclictest	700566	-21	ssh	09:30:44	1
650461	99	58	20,12	cyclictest	776742	-21	ssh	10:56:34	1
650460	99	58	50,6	cyclictest	656802	-21	kworker/u4:13+events_unbound	08:00:02	0
650460	99	58	43,12	cyclictest	828074	-21	rm	11:55:21	0
650461	99	57	47,8	cyclictest	704729	-21	ntpq	09:35:27	1
650461	99	57	45,9	cyclictest	669312	-21	ps	08:20:34	1
650461	99	57	20,10	cyclictest	676988	-21	ntp_states	08:50:31	1
650461	99	57	20,10	cyclictest	676988	-21	ntp_states	08:50:31	1
650461	99	57	18,13	cyclictest	726146	-21	ssh	10:00:07	1
650461	99	57	17,15	cyclictest	697347	-21	ssh	09:27:03	1
650460	99	57	50,5	cyclictest	673582	-21	kworker/u4:3+events_unbound	08:50:04	0
650460	99	57	50,5	cyclictest	673582	-21	kworker/u4:3+events_unbound	08:50:04	0
650460	99	57	49,6	cyclictest	664485	-21	kworker/u4:11+events_unbound	08:40:17	0
650460	99	57	48,7	cyclictest	845861	-21	/usr/sbin/munin	12:15:23	0
650460	99	57	48,7	cyclictest	845861	-21	/usr/sbin/munin	12:15:23	0
650461	99	56	38,8	cyclictest	837367	-21	ntpq	12:05:33	1
650461	99	56	34,17	cyclictest	793303	-21	ssh	11:15:33	1
650461	99	56	21,11	cyclictest	781120	-21	ssh	11:01:29	1
650461	99	56	19,12	cyclictest	772113	-21	ssh	10:51:12	1
650461	99	56	18,13	cyclictest	710020	-21	ssh	09:40:59	1
650460	99	56	50,4	cyclictest	768513	-21	kworker/u4:4+events_unbound	11:00:04	0
650461	99	55	47,6	cyclictest	785790	-21	kworker/u4:0+events_unbound	11:40:01	1
650461	99	55	23,5	cyclictest	713566	-21	ntpq	09:45:23	1
650461	99	55	22,6	cyclictest	749058	-21	ntpq	10:25:30	1
650461	99	55	22,10	cyclictest	849633	-21	ssh	12:20:06	1
650461	99	55	21,6	cyclictest	757158	-21	apt-get	10:35:12	1
650461	99	55	20,10	cyclictest	798773	-21	ssh	11:21:30	1
650461	99	55	19,10	cyclictest	1073	-21	snmpd	11:53:44	1
650461	99	55	18,13	cyclictest	857939	-21	ssh	12:29:38	1
650461	99	54	39,10	cyclictest	766859	-21	ssh	10:45:34	1
650461	99	54	35,16	cyclictest	681584	-21	apt-get	09:10:13	1
650461	99	54	34,15	cyclictest	739739	-21	ssh	10:15:17	1
650461	99	54	34,13	cyclictest	863534	-21	ssh	12:35:24	1
650461	99	54	21,9	cyclictest	788711	-21	ntpq	11:10:25	1
650461	99	54	21,9	cyclictest	788711	-21	ntpq	11:10:25	1
650461	99	54	19,9	cyclictest	804399	-21	ssh	11:28:25	1
650461	99	54	19,11	cyclictest	805728	-21	/usr/sbin/munin	11:30:43	1
650461	99	54	18,14	cyclictest	841743	-21	ntpq	12:10:34	1
650461	99	54	18,12	cyclictest	691361	-21	ntpq	09:20:27	1
650461	99	54	18,10	cyclictest	665349	-21	ntpq	08:05:34	1
650461	99	54	17,13	cyclictest	815317	-21	ssh	11:40:30	1
650461	99	54	17,12	cyclictest	832080	-21	/usr/sbin/munin	12:00:37	1
650461	99	54	17,12	cyclictest	747351	-21	ssh	10:23:59	1
650460	99	54	46,6	cyclictest	785792	-21	kworker/u4:3+events_unbound	11:16:00	0
650461	99	53	33,17	cyclictest	666589	-21	ntpq	08:10:24	1
650461	99	53	18,12	cyclictest	687460	-21	ssh	09:15:43	1
650461	99	53	18,10	cyclictest	785386	-21	ssh	11:06:17	1
650461	99	52	44,6	cyclictest	675287	-21	df_abs	08:45:17	1
650461	99	52	19,10	cyclictest	735614	-21	ssh	10:10:22	1
650461	99	52	19,10	cyclictest	1073	-21	snmpd	09:08:09	1
650461	99	52	19,10	cyclictest	1073	-21	snmpd	07:12:46	1
650461	99	52	18,12	cyclictest	754657	-21	ssh	10:31:34	1
650461	99	52	18,12	cyclictest	718693	-21	users	09:50:41	1
650461	99	52	18,12	cyclictest	718693	-21	users	09:50:41	1
650461	99	52	17,13	cyclictest	664024	-21	ntpq	08:00:30	1
650460	99	52	45,5	cyclictest	658080	-21	kworker/u4:10+events_unbound	09:21:00	0
650460	99	52	38,10	cyclictest	681287	-21	diskmemload	10:38:18	0
650460	99	52	2,42	cyclictest	801302	-21	ssh	11:25:14	0
650461	99	51	18,12	cyclictest	662095	-21	/usr/sbin/munin	07:55:20	1
650460	99	51	45,5	cyclictest	825436	-21	kworker/u4:16+events_unbound	11:52:00	0
650461	99	50	42,5	cyclictest	651688	-21	kworker/u4:11-nfsiod	07:25:00	1
650461	99	50	18,9	cyclictest	679579	-21	ntpq	09:00:37	1
650461	99	50	18,10	cyclictest	678217	-21	ntpq	08:55:28	1
650460	99	50	39,4	cyclictest	688616	-21	kworker/u4:9+events_unbound	09:38:25	0
650461	99	49	41,5	cyclictest	674199	-21	gltestperf	08:40:20	1
650460	99	49	43,4	cyclictest	0	-21	swapper/0	10:55:26	0
650460	99	49	42,5	cyclictest	768514	-21	kworker/u4:6+events_unbound	10:48:01	0
650460	99	49	41,6	cyclictest	730865	-21	ssh	10:05:17	0
650460	99	49	41,6	cyclictest	654266	-21	kworker/u4:9+events_unbound	07:27:59	0
650460	99	49	39,7	cyclictest	740149	-21	ntp_kernel_pll_	10:15:25	0
650461	99	48	39,6	cyclictest	658650	-21	fschecks_time	07:40:22	1
650461	99	48	35,9	cyclictest	652426	-21	ntpq	07:15:22	1
650461	99	48	35,9	cyclictest	652426	-21	ntpq	07:15:22	1
650461	99	48	35,9	cyclictest	27	1	ktimers/1	08:15:26	1
650461	99	48	31,14	cyclictest	673082	-21	ntpq	08:35:30	1
650461	99	48	31,11	cyclictest	660129	-21	ntpq	07:45:25	1
650461	99	48	27,15	cyclictest	658150	-21	modprobe	07:40:00	1
650460	99	48	40,6	cyclictest	863859	-21	/usr/sbin/munin	12:35:37	0
650460	99	48	40,6	cyclictest	817156	-21	kworker/u4:6+events_unbound	11:45:01	0
650460	99	48	39,6	cyclictest	762929	-21	unixbench_singl	10:40:42	0
650461	99	47	35,9	cyclictest	655053	-21	ntpq	07:25:27	1
650461	99	47	31,9	cyclictest	656483	-21	awk	07:30:36	1
650460	99	47	41,4	cyclictest	819456	-21	gltestperf	11:45:21	0
650460	99	47	40,5	cyclictest	784163	-21	gltestperf	11:05:23	0
650460	99	47	40,5	cyclictest	749252	-21	sshd	10:25:39	0
650460	99	47	40,5	cyclictest	654266	-21	kworker/u4:9+events_unbound	08:12:03	0
650460	99	47	40,5	cyclictest	654264	-21	kworker/u4:7+events_unbound	07:23:46	0
650460	99	47	39,6	cyclictest	829887	-21	kworker/u4:18+events_unbound	12:00:59	0
650460	99	47	38,7	cyclictest	663262	-21	latency_hist	08:00:01	0
650460	99	47	38,6	cyclictest	722414	-21	/usr/sbin/munin	09:55:24	0
650460	99	47	36,9	cyclictest	744906	-21	sendmail_mailst	10:20:38	0
650460	99	46	9,4	cyclictest	19	1	rcuc/0	09:27:37	0
650460	99	46	39,5	cyclictest	718005	-21	ld	09:50:24	0
650460	99	46	39,5	cyclictest	718005	-21	ld	09:50:24	0
650460	99	46	39,5	cyclictest	665840	-21	kworker/u4:14+events_unbound	08:36:01	0
650460	99	46	39,5	cyclictest	420003	-21	kworker/u4:5+events_unbound	07:30:59	0
650460	99	46	37,7	cyclictest	794729	-21	kworker/u4:7-rpciod	11:20:20	0
650460	99	46	37,6	cyclictest	850344	-21	ssh	12:20:24	0
650460	99	45	39,4	cyclictest	673581	-21	kworker/u4:1+events_unbound	09:04:01	0
650460	99	45	38,5	cyclictest	736338	-21	grep	10:10:40	0
650460	99	45	38,5	cyclictest	708563	-21	aten_r7power_en	09:40:15	0
650460	99	45	38,5	cyclictest	659376	-21	kworker/u4:12+events_unbound	07:44:00	0
650460	99	45	36,7	cyclictest	675182	-21	aten_r7power_po	08:45:16	0
650460	99	45	35,8	cyclictest	715317	-21	sshd	09:47:18	0
650460	99	45	33,5	cyclictest	658077	-21	kworker/u4:1+events_unbound	08:15:41	0
650460	99	45	24,7	cyclictest	0	-21	swapper/0	09:10:11	0
650460	99	45	11,32	cyclictest	15	1	ktimers/0	09:15:34	0
650460	99	44	38,4	cyclictest	859834	-21	users	12:30:41	0
650460	99	44	37,5	cyclictest	0	-21	swapper/0	09:30:19	0
650460	99	44	36,6	cyclictest	664483	-21	kworker/u4:6+events_unbound	08:09:01	0
650460	99	44	32,10	cyclictest	836824	-21	fschecks_count	12:05:19	0
650460	99	43	37,4	cyclictest	771377	-21	ssh	10:50:38	0
650460	99	43	35,5	cyclictest	0	-21	swapper/0	12:26:02	0
650460	99	42	33,7	cyclictest	659702	-21	aten_r7power_vo	07:45:17	0
650460	99	41	34,5	cyclictest	658080	-21	kworker/u4:10+events_unbound	07:52:59	0
650461	99	39	28,8	cyclictest	581061	-21	kworker/u4:6-rpciod	07:10:01	1
650460	99	35	23,11	cyclictest	650457	-21	cyclictest	07:10:01	0
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17