OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #7, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system

Characteristics of the 376 highest latencies:

System rack7slot2.osadl.org (updated Fri Aug 28, 2026 12:44:09)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
2211529	99	102	51,37	cyclictest	53	-21	kswapd0	11:30:01	0
2211530	99	93	85,6	cyclictest	2418713	-21	kworker/u4:4+events_unbound	12:34:31	1
2211529	99	92	38,33	cyclictest	2278904	-21	wget	09:50:17	0
2211530	99	89	81,6	cyclictest	2234638	-21	kworker/u4:8+events_unbound	08:48:58	1
2211530	99	88	78,7	cyclictest	1970659	-21	kworker/u4:9+events_unbound	08:58:59	1
2211530	99	85	78,5	cyclictest	2258327	-21	kworker/u4:6+events_unbound	09:29:02	1
2211530	99	84	76,6	cyclictest	2356942	-21	kworker/u4:8+events_unbound	11:29:22	1
2211530	99	84	76,6	cyclictest	2226896	-21	kworker/u4:16+events_unbound	08:13:57	1
2211530	99	83	76,5	cyclictest	2214032	-21	kworker/u4:6+events_unbound	07:43:54	1
2211529	99	83	73,8	cyclictest	2392335	-21	kworker/u4:2+events_unbound	11:59:28	0
2211529	99	83	73,8	cyclictest	2392335	-21	kworker/u4:2+events_unbound	11:59:27	0
2211530	99	82	76,5	cyclictest	2241049	-21	kworker/u4:3+events_unbound	09:04:00	1
2211529	99	81	69,9	cyclictest	53	-21	kswapd0	11:50:08	0
2211529	99	79	31,25	cyclictest	0	-21	swapper/0	10:30:29	0
2211529	99	78	2,51	cyclictest	2242655	-21	apt-get	09:10:12	0
2211529	99	78	2,48	cyclictest	2374178	-21	ssh	11:36:28	0
2211529	99	78	2,48	cyclictest	2374178	-21	ssh	11:36:28	0
2211530	99	77	70,6	cyclictest	2207814	-21	kworker/u4:4+events_unbound	08:18:56	1
2211529	99	77	67,7	cyclictest	2311577	-21	grep	10:25:44	0
2211529	99	77	2,50	cyclictest	2351206	-21	/usr/sbin/munin	11:10:38	0
2211530	99	76	69,5	cyclictest	2215331	-21	kworker/u4:15+events_unbound	07:38:53	1
2211530	99	76	69,5	cyclictest	2202539	-21	kworker/u4:2+events_unbound	07:23:51	1
2211530	99	76	68,6	cyclictest	2343421	-21	kworker/u4:0+events_unbound	11:04:15	1
2211530	99	76	66,8	cyclictest	2319003	-21	latency_hist	10:35:01	1
2211529	99	76	2,51	cyclictest	2421757	-21	ssh	12:30:24	0
2211529	99	75	13,37	cyclictest	0	-21	swapper/0	09:00:37	0
2211530	99	74	66,6	cyclictest	2214035	-21	kworker/u4:12+events_unbound	08:03:56	1
2211530	99	74	66,6	cyclictest	2214035	-21	kworker/u4:12+events_unbound	08:03:56	1
2211529	99	74	32,37	cyclictest	0	-21	swapper/0	09:15:34	0
2211530	99	73	65,6	cyclictest	2197344	-21	kworker/u4:13+events_unbound	07:48:54	1
2211529	99	73	26,28	cyclictest	0	-21	swapper/0	12:20:17	0
2211529	99	73	25,25	cyclictest	0	-21	swapper/0	11:40:31	0
2211529	99	73	24,28	cyclictest	0	-21	swapper/0	11:20:25	0
2211529	99	73	21,28	cyclictest	256	50	irq/23-42890000.ethernet	09:20:34	0
2211529	99	73	11,44	cyclictest	256	50	irq/23-42890000.ethernet	12:00:17	0
2211529	99	72	40,29	cyclictest	2292782	-21	gltestperf	10:05:20	0
2211529	99	72	28,28	cyclictest	2226892	-21	kworker/u4:14-rpciod	08:10:37	0
2211529	99	72	24,40	cyclictest	0	-21	swapper/0	11:30:37	0
2211529	99	72	22,27	cyclictest	0	-21	swapper/0	09:00:00	0
2211529	99	72	1,48	cyclictest	2219278	-21	sendmail	07:40:02	0
2211530	99	71	7,16	cyclictest	17	1	rcu_preempt	11:55:27	1
2211530	99	71	7,16	cyclictest	17	1	rcu_preempt	11:55:27	1
2211530	99	71	63,6	cyclictest	2226896	-21	kworker/u4:16+events_unbound	08:28:58	1
2211530	99	71	63,6	cyclictest	2207815	-21	kworker/u4:5+events_unbound	07:18:51	1
2211530	99	71	55,9	cyclictest	0	-21	swapper/1	09:45:37	1
2211529	99	71	26,38	cyclictest	0	-21	swapper/0	07:20:20	0
2211530	99	70	63,5	cyclictest	2214032	-21	kworker/u4:6+events_unbound	07:53:55	1
2211529	99	70	36,30	cyclictest	2224394	-21	grep	08:00:00	0
2211529	99	70	20,30	cyclictest	256	50	irq/23-42890000.ethernet	08:29:05	0
2211529	99	70	16,31	cyclictest	0	-21	swapper/0	10:35:36	0
2211529	99	70	1,46	cyclictest	2269440	-21	cron	09:40:00	0
2211529	99	69	1,60	cyclictest	2236134	-21	apt-get	08:45:12	0
2211529	99	69	1,45	cyclictest	2257684	-21	sh	09:25:51	0
2211529	99	69	1,44	cyclictest	2242360	-21	diskmemload	11:05:24	0
2211529	99	68	25,24	cyclictest	0	-21	swapper/0	12:25:25	0
2211529	99	67	7,53	cyclictest	2215593	-21	munin-node	07:25:11	0
2211529	99	67	57,7	cyclictest	2383325	-21	kworker/u4:5+events_unbound	11:49:25	0
2211529	99	67	23,39	cyclictest	0	-21	swapper/0	10:55:26	0
2211529	99	67	1,44	cyclictest	0	-21	swapper/0	08:20:17	0
2211529	99	67	1,43	cyclictest	2297649	-21	ld	10:10:33	0
2211529	99	66	36,27	cyclictest	2272056	-21	kworker/u4:3-rpciod	09:59:39	0
2211529	99	66	28,34	cyclictest	0	-21	swapper/0	10:40:40	0
2211529	99	66	26,36	cyclictest	0	-21	swapper/0	12:35:29	0
2211529	99	66	1,46	cyclictest	2328982	-21	grep	10:45:36	0
2211529	99	66	1,45	cyclictest	2221417	-21	grep	07:45:35	0
2211529	99	66	12,39	cyclictest	2212224	-21	ntpq	07:10:22	0
2211529	99	65	5,40	cyclictest	2387809	-21	kworker/u4:0-rpciod	12:05:28	0
2211529	99	65	22,25	cyclictest	2211527	-21	cyclictest	09:05:21	0
2211529	99	65	1,38	cyclictest	2288647	-21	ntpq	10:00:34	0
2211530	99	64	56,6	cyclictest	1970659	-21	kworker/u4:9+events_unbound	08:38:58	1
2211530	99	64	55,7	cyclictest	53	-21	kswapd0	11:20:01	1
2211529	99	64	3,54	cyclictest	2228367	-21	apt-get	08:15:11	0
2211529	99	64	2,55	cyclictest	2212950	-21	apt-get	07:15:11	0
2211529	99	64	16,29	cyclictest	2235469	-21	ntpq	08:40:29	0
2211529	99	64	1,41	cyclictest	2354502	-21	ssh	11:15:07	0
2211529	99	63	3,56	cyclictest	2225104	-21	ntpq	08:00:25	0
2211529	99	63	3,56	cyclictest	2225104	-21	ntpq	08:00:24	0
2211529	99	63	15,30	cyclictest	2303425	-21	kworker/u4:15+xprtiod	10:22:45	0
2211529	99	63	1,40	cyclictest	0	-21	swapper/0	09:45:27	0
2211530	99	62	55,5	cyclictest	2399883	-21	/usr/sbin/munin	12:05:36	1
2211530	99	62	55,5	cyclictest	2214035	-21	kworker/u4:12+events_unbound	07:33:53	1
2211529	99	62	30,28	cyclictest	0	-21	swapper/0	08:05:15	0
2211529	99	62	26,32	cyclictest	2233236	-21	unixbench-2d	08:30:37	0
2211529	99	62	2,54	cyclictest	2219448	-21	apt-get	07:40:10	0
2211529	99	62	2,52	cyclictest	2332460	-21	apt-get	10:50:13	0
2211529	99	62	1,40	cyclictest	2238481	-21	192.168.115.60-manager	08:52:29	0
2211529	99	61	3,53	cyclictest	2408676	-21	ssh	12:15:36	0
2211529	99	61	3,53	cyclictest	2234208	-21	ntpq	08:35:32	0
2211529	99	61	2,56	cyclictest	2261381	-21	ntpq	09:30:22	0
2211529	99	59	24,29	cyclictest	2302522	-21	unixbench-2d	10:15:42	0
2211529	99	59	19,22	cyclictest	256	50	irq/23-42890000.ethernet	11:00:21	0
2211529	99	59	1,42	cyclictest	2270671	-21	ntpq	09:40:37	0
2211529	99	58	2,50	cyclictest	2221988	-21	apt-get	07:50:12	0
2211530	99	57	51,5	cyclictest	2226896	-21	kworker/u4:16+events_unbound	08:35:01	1
2211530	99	57	50,5	cyclictest	2267486	-21	kworker/u4:6+events_unbound	09:42:48	1
2211529	99	57	1,51	cyclictest	2403902	-21	ssh	12:10:21	0
2211530	99	56	46,7	cyclictest	0	-21	swapper/1	09:50:11	1
2211529	99	56	25,27	cyclictest	0	-21	swapper/0	07:30:35	0
2211529	99	55	2,47	cyclictest	0	-21	swapper/0	07:10:01	0
2211530	99	54	42,5	cyclictest	2387809	-21	kworker/u4:0+events_unbound	12:02:26	1
2211530	99	53	44,6	cyclictest	0	-21	swapper/1	10:15:29	1
2211530	99	53	42,8	cyclictest	0	-21	swapper/1	09:35:08	1
2211530	99	52	44,5	cyclictest	2377171	-21	fschecks_time	11:40:18	1
2211530	99	50	43,5	cyclictest	2346057	-21	fschecks_count	11:05:20	1
2211530	99	50	42,5	cyclictest	2407849	-21	aten_r7power_cu	12:15:17	1
2211530	99	50	32,7	cyclictest	2267487	-21	kworker/u4:8+events_unbound	10:21:33	1
2211530	99	49	39,7	cyclictest	2370469	-21	kworker/u4:12-rpciod	11:35:40	1
2211530	99	49	39,7	cyclictest	2370469	-21	kworker/u4:12-rpciod	11:35:39	1
2211530	99	48	41,5	cyclictest	2359474	-21	gltestperf	11:20:19	1
2211530	99	48	40,6	cyclictest	2247329	-21	aten_r7power_vo	09:15:16	1
2211530	99	48	39,7	cyclictest	2338046	-21	timerwakeupswit	10:55:38	1
2211530	99	47	41,4	cyclictest	0	-21	swapper/1	10:40:17	1
2211530	99	47	40,5	cyclictest	2310002	-21	apt-get	10:25:09	1
2211530	99	47	39,5	cyclictest	2283798	-21	/usr/sbin/munin	09:55:21	1
2211530	99	47	38,7	cyclictest	2288918	-21	timerandwakeup	10:00:39	1
2211530	99	47	38,7	cyclictest	2261462	-21	ntp_kernel_pll_	09:30:26	1
2211530	99	46	40,4	cyclictest	2230462	-21	proc_pri	08:20:48	1
2211530	99	46	39,5	cyclictest	2368883	-21	sendmail_mailqu	11:30:37	1
2211530	99	46	37,7	cyclictest	400	-21	systemd-logind	10:09:04	1
2211530	99	46	37,7	cyclictest	2296798	-21	aten2__	10:10:12	1
2211530	99	46	34,9	cyclictest	0	-21	swapper/1	11:51:14	1
2211530	99	45	38,5	cyclictest	2317106	-21	kworker/u4:0+events_unbound	10:40:02	1
2211530	99	45	34,6	cyclictest	2241048	-21	kworker/u4:0+events_unbound	09:23:26	1
2211530	99	45	20,12	cyclictest	17	1	rcu_preempt	11:45:25	1
2211530	99	44	37,4	cyclictest	2336658	-21	cat	10:55:01	1
2211530	99	44	36,5	cyclictest	2217956	-21	kworker/u4:17-nfsiod	08:05:28	1
2211530	99	44	35,7	cyclictest	2328604	-21	kernelversion	10:45:21	1
2211530	99	44	35,6	cyclictest	2238478	-21	kworker/u4:14-rpciod	09:10:12	1
2211530	99	44	35,6	cyclictest	0	-21	swapper/1	12:14:28	1
2211530	99	42	35,5	cyclictest	2416984	-21	fschecks_count	12:25:19	1
2211530	99	42	35,5	cyclictest	2412667	-21	gltestperf	12:20:20	1
2211530	99	42	35,5	cyclictest	2235942	-21	sshd	08:43:57	1
2211530	99	42	34,6	cyclictest	2351125	-21	/usr/sbin/munin	11:10:38	1
2211530	99	42	33,6	cyclictest	0	-21	swapper/1	12:35:34	1
2211530	99	42	32,7	cyclictest	2242053	-21	switchtime	09:05:35	1
2211530	99	41	34,5	cyclictest	0	-21	swapper/1	08:00:01	1
2211530	99	40	28,10	cyclictest	0	-21	swapper/1	07:25:43	1
2211530	99	39	31,6	cyclictest	2212523	-21	threads	07:10:35	1
2211530	99	39	28,8	cyclictest	0	-21	swapper/1	08:50:39	1
2211530	99	37	9,18	cyclictest	2211506	-21	munin-run	07:10:00	1
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17