OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #7, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system

Characteristics of all highest latencies:

System rack7slot2.osadl.org (updated Sun Aug 30, 2026 12:44:10)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
752393	99	124	108,12	cyclictest	53	-21	kswapd0	12:35:13	1
752393	99	110	103,5	cyclictest	942112	-21	kworker/u4:2+events_unbound	12:25:10	1
752391	99	108	99,6	cyclictest	879176	-21	kworker/u4:7+events_unbound	11:00:00	0
752391	99	103	94,7	cyclictest	830786	-21	kworker/u4:1+events_unbound	10:04:56	0
752391	99	94	85,7	cyclictest	826261	-21	kworker/u4:9+events_unbound	10:24:56	0
752393	99	92	83,7	cyclictest	821751	-21	kworker/u4:7+events_unbound	10:09:55	1
752391	99	92	84,6	cyclictest	757465	-21	kworker/u4:6+events_unbound	08:24:33	0
752393	99	91	72,7	cyclictest	789791	-21	kworker/u4:20+events_unbound	09:19:45	1
752391	99	89	81,6	cyclictest	756195	-21	kworker/u4:5+events_unbound	07:39:26	0
752393	99	86	79,5	cyclictest	772880	-21	kworker/u4:1+events_unbound	09:44:51	1
752391	99	83	77,4	cyclictest	906391	-21	kworker/u4:5+events_unbound	11:30:03	0
752391	99	82	74,6	cyclictest	760006	-21	kworker/u4:12+events_unbound	07:49:27	0
752391	99	81	73,6	cyclictest	753612	-21	kworker/u4:10+events_unbound	07:14:20	0
752393	99	80	73,5	cyclictest	817235	-21	kworker/u4:4+events_unbound	09:54:53	1
752391	99	78	26,36	cyclictest	752392	-21	idleruntime-cro	07:10:01	0
752393	99	77	69,6	cyclictest	888221	-21	kworker/u4:12+events_unbound	11:10:02	1
752393	99	75	68,5	cyclictest	835286	-21	kworker/u4:13+events_unbound	10:29:56	1
752393	99	74	58,6	cyclictest	707440	-21	kworker/u4:0+events_unbound	07:19:21	1
752393	99	74	58,6	cyclictest	707440	-21	kworker/u4:0+events_unbound	07:19:21	1
752393	99	73	59,5	cyclictest	769036	-21	kworker/u4:16+events_unbound	09:29:49	1
752393	99	73	54,8	cyclictest	749883	-21	kworker/u4:1+events_unbound	07:45:01	1
752391	99	72	60,10	cyclictest	53	-21	kswapd0	09:45:07	0
752393	99	71	64,5	cyclictest	757465	-21	kworker/u4:6+events_unbound	08:04:30	1
752391	99	68	60,5	cyclictest	0	-21	swapper/0	09:30:38	0
752391	99	68	59,6	cyclictest	851509	-21	ld	10:25:36	0
752391	99	66	53,11	cyclictest	53	-21	kswapd0	11:40:13	0
752391	99	66	41,22	cyclictest	788460	-21	cut	09:15:21	0
752393	99	64	56,6	cyclictest	882091	-21	fschecks_time	11:00:21	1
752393	99	64	56,6	cyclictest	882091	-21	fschecks_time	11:00:21	1
752391	99	64	49,8	cyclictest	0	-21	swapper/0	10:40:37	0
752393	99	63	56,5	cyclictest	734123	-21	kworker/u4:7+events_unbound	07:54:28	1
752391	99	63	55,6	cyclictest	777985	-21	kworker/u4:7+events_unbound	08:49:39	0
752393	99	60	53,4	cyclictest	757466	-21	kworker/u4:9+events_unbound	07:40:01	1
752393	99	60	51,7	cyclictest	897286	-21	kworker/u4:11+events_unbound	11:35:04	1
752391	99	60	45,6	cyclictest	955013	-21	kworker/u4:11+events_unbound	12:31:45	0
752391	99	59	51,6	cyclictest	760006	-21	kworker/u4:12+events_unbound	07:50:04	0
752391	99	59	48,7	cyclictest	53	-21	kswapd0	09:45:01	0
752391	99	59	47,10	cyclictest	789791	-21	kworker/u4:20+events_unbound	09:21:46	0
752391	99	58	53,3	cyclictest	760005	-21	kworker/u4:4+events_unbound	08:10:03	0
752391	99	58	51,5	cyclictest	751117	-21	kworker/u4:4+events_unbound	07:25:02	0
752391	99	58	44,7	cyclictest	256	50	irq/23-42890000.ethernet	10:05:19	0
752391	99	58	2,44	cyclictest	838034	-21	ntpq	10:10:28	0
752393	99	57	50,5	cyclictest	760006	-21	kworker/u4:12+events_unbound	08:29:34	1
752393	99	57	50,5	cyclictest	753282	-21	sendmail_mailqu	07:10:37	1
752393	99	57	47,7	cyclictest	857261	-21	kworker/u4:18+events_unbound	10:35:00	1
752391	99	57	50,5	cyclictest	950467	-21	kworker/u4:6+events_unbound	12:17:01	0
752391	99	56	49,5	cyclictest	769036	-21	kworker/u4:16+events_unbound	09:00:02	0
752391	99	56	49,5	cyclictest	753612	-21	kworker/u4:10+events_unbound	07:16:45	0
752391	99	56	49,5	cyclictest	753612	-21	kworker/u4:10+events_unbound	07:16:45	0
752391	99	56	48,6	cyclictest	752389	-21	cyclictest	11:45:08	0
752391	99	56	46,7	cyclictest	891620	-21	/usr/sbin/munin	11:10:38	0
752391	99	56	44,4	cyclictest	780526	-21	kworker/u4:14+events_unbound	09:05:03	0
752391	99	56	44,4	cyclictest	780526	-21	kworker/u4:14+events_unbound	09:05:03	0
752391	99	56	43,7	cyclictest	959600	-21	ssh	12:26:40	0
752393	99	55	45,5	cyclictest	0	-21	swapper/1	10:45:25	1
752391	99	55	48,5	cyclictest	753612	-21	kworker/u4:10+events_unbound	07:20:02	0
752391	99	55	45,7	cyclictest	256	50	irq/23-42890000.ethernet	12:20:11	0
752391	99	55	45,7	cyclictest	256	50	irq/23-42890000.ethernet	12:20:11	0
752391	99	55	43,7	cyclictest	942116	-21	kworker/u4:7-rpciod	12:10:20	0
752393	99	54	39,7	cyclictest	888221	-21	kworker/u4:12+events_unbound	11:19:46	1
752391	99	54	32,19	cyclictest	797999	-21	proc_pri	09:25:39	0
752393	99	53	44,7	cyclictest	944113	-21	latency_hist	12:10:02	1
752393	99	53	37,8	cyclictest	897286	-21	kworker/u4:11+events_unbound	11:40:36	1
752393	99	52	45,5	cyclictest	846380	-21	aten_r7power_en	10:20:23	1
752391	99	52	45,5	cyclictest	774150	-21	kworker/u4:2+events_unbound	09:15:02	0
752391	99	52	44,6	cyclictest	966467	-21	munin-node	12:35:11	0
752391	99	52	41,8	cyclictest	913397	-21	fschecks_count	11:35:18	0
752391	99	52	41,7	cyclictest	774173	-21	modprobe	08:35:00	0
752391	99	52	41,7	cyclictest	760047	-21	apt-get	07:40:05	0
752391	99	52	38,11	cyclictest	758758	-21	kworker/u4:1	07:35:00	0
752391	99	52	28,17	cyclictest	826261	-21	kworker/u4:9+events_unbound	09:57:47	0
752393	99	51	45,4	cyclictest	815085	-21	ssh	09:45:14	1
752393	99	51	45,4	cyclictest	0	-21	swapper/1	10:00:31	1
752393	99	51	44,5	cyclictest	879176	-21	kworker/u4:7+events_unbound	11:05:03	1
752393	99	51	44,5	cyclictest	1060	-21	sshd	10:39:57	1
752393	99	51	43,6	cyclictest	755179	-21	aten_r7power_cu	07:20:15	1
752393	99	51	37,5	cyclictest	760006	-21	kworker/u4:12+events_unbound	09:11:46	1
752391	99	51	43,5	cyclictest	940288	-21	gltestperf	12:05:20	0
752391	99	51	34,15	cyclictest	887247	-21	awk	11:05:38	0
752391	99	51	31,17	cyclictest	0	-21	swapper/0	10:20:00	0
752393	99	50	42,5	cyclictest	911833	-21	ssh	11:33:38	1
752391	99	50	42,5	cyclictest	819816	-21	fschecks_count	09:50:19	0
752391	99	50	28,19	cyclictest	0	-21	swapper/0	12:00:35	0
752393	99	49	41,6	cyclictest	756195	-21	kworker/u4:5+events_unbound	07:29:23	1
752393	99	49	41,6	cyclictest	741075	-21	kworker/u4:14+events_unbound	08:20:00	1
752393	99	49	41,5	cyclictest	777809	-21	timerwakeupswit	08:45:36	1
752393	99	49	41,5	cyclictest	0	-21	swapper/1	09:30:16	1
752393	99	49	20,18	cyclictest	910964	-21	kworker/u4:0+events_unbound	11:50:46	1
752391	99	49	8,33	cyclictest	855110	-21	ssh	10:30:15	0
752391	99	49	42,5	cyclictest	789791	-21	kworker/u4:20+events_unbound	09:35:15	0
752391	99	49	40,6	cyclictest	780564	-21	sh	09:00:00	0
752391	99	49	39,8	cyclictest	895744	-21	/usr/sbin/munin	11:15:26	0
752391	99	49	36,7	cyclictest	873840	-21	diskmemload	10:50:38	0
752391	99	49	36,7	cyclictest	873840	-21	diskmemload	10:50:37	0
752393	99	48	41,5	cyclictest	783131	-21	diskmemload	10:14:54	1
752393	99	48	40,5	cyclictest	792621	-21	aten_r7power_cu	09:20:19	1
752393	99	48	38,7	cyclictest	953016	-21	latency_hist	12:20:02	1
752391	99	48	43,3	cyclictest	0	-21	swapper/0	11:50:11	0
752391	99	48	31,9	cyclictest	256	50	irq/23-42890000.ethernet	10:35:30	0
752391	99	48	30,13	cyclictest	935086	-21	cron	12:00:00	0
752393	99	47	42,4	cyclictest	760005	-21	kworker/u4:4+events_unbound	08:10:01	1
752393	99	47	41,4	cyclictest	899977	-21	fschecks_time	11:20:19	1
752393	99	47	40,5	cyclictest	963235	-21	timerwakeupswit	12:30:41	1
752393	99	47	39,6	cyclictest	944977	-21	/usr/sbin/munin	12:10:26	1
752393	99	47	39,6	cyclictest	753612	-21	kworker/u4:10+events_unbound	07:34:25	1
752393	99	47	39,5	cyclictest	922701	-21	sed	11:45:23	1
752393	99	47	37,8	cyclictest	783141	-21	munin-run	09:10:00	1
752393	99	47	37,8	cyclictest	783141	-21	munin-run	09:10:00	1
752391	99	47	24,18	cyclictest	0	-21	swapper/0	08:05:00	0
752393	99	46	34,8	cyclictest	778472	-21	fschecks_count	08:50:19	1
752393	99	46	26,6	cyclictest	826261	-21	kworker/u4:9+events_unbound	09:58:48	1
752391	99	46	36,8	cyclictest	869526	-21	switchtime	10:45:38	0
752391	99	46	33,10	cyclictest	905127	-21	mailstats	11:25:39	0
752391	99	46	23,21	cyclictest	766721	-21	aten2__	08:05:15	0
752393	99	45	38,5	cyclictest	762501	-21	timerwakeupswit	07:45:38	1
752393	99	45	37,6	cyclictest	935121	-21	sh	12:00:00	1
752393	99	45	37,6	cyclictest	861789	-21	kworker/u4:5+events_unbound	10:41:45	1
752391	99	45	37,6	cyclictest	882711	-21	tr	11:00:39	0
752391	99	45	37,6	cyclictest	882711	-21	tr	11:00:39	0
752391	99	45	35,7	cyclictest	0	-21	swapper/0	08:35:39	0
752393	99	44	37,5	cyclictest	874199	-21	sed	10:50:42	1
752393	99	44	37,5	cyclictest	874199	-21	sed	10:50:41	1
752393	99	44	37,5	cyclictest	789791	-21	kworker/u4:20+events_unbound	09:35:05	1
752393	99	44	34,7	cyclictest	904553	-21	gltestperf	11:25:21	1
752391	99	44	34,6	cyclictest	734123	-21	kworker/u4:7-rpciod	08:15:39	0
752391	99	44	33,7	cyclictest	879177	-21	kworker/u4:8-rpciod	11:20:38	0
752391	99	44	31,8	cyclictest	0	-21	swapper/0	08:55:00	0
752391	99	44	26,11	cyclictest	1073	-21	snmpd	07:58:48	0
752393	99	43	36,5	cyclictest	936691	-21	unixbench_multi	12:00:42	1
752393	99	43	35,5	cyclictest	757465	-21	kworker/u4:6+rpciod	08:10:16	1
752393	99	43	34,7	cyclictest	0	-21	swapper/1	10:15:31	1
752393	99	43	22,12	cyclictest	769036	-21	kworker/u4:16+events_unbound	08:31:14	1
752391	99	43	36,5	cyclictest	769036	-21	kworker/u4:16+events_unbound	08:28:18	0
752391	99	43	33,7	cyclictest	776450	-21	switchtime	08:40:39	0
752393	99	42	35,5	cyclictest	877522	-21	aten_r7power_po	10:55:23	1
752393	99	42	35,5	cyclictest	0	-21	swapper/1	08:20:15	1
752393	99	42	32,8	cyclictest	776762	-21	idleruntime-cro	08:45:00	1
752393	99	41	35,4	cyclictest	781862	-21	idleruntime-cro	09:05:00	1
752393	99	41	29,9	cyclictest	0	-21	swapper/1	12:20:24	1
752393	99	41	29,9	cyclictest	0	-21	swapper/1	12:20:24	1
752393	99	40	31,7	cyclictest	765234	-21	sh	08:00:00	1
752393	99	38	30,6	cyclictest	752452	-21	latency_hist	07:10:02	1
752393	99	38	30,6	cyclictest	0	-21	swapper/1	08:35:38	1
752393	99	37	25,9	cyclictest	0	-21	swapper/1	08:55:39	1
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17