OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #7, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system

Characteristics of the 376 highest latencies:

System rack7slot2.osadl.org (updated Tue Mar 31, 2026 00:44:10)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
2636236	99	96	89,5	cyclictest	2642661	-21	kworker/u4:0+events_unbound	19:47:21	0
2636236	99	95	88,5	cyclictest	2667214	-21	kworker/u4:14+events_unbound	21:07:29	0
2636236	99	90	83,5	cyclictest	2645229	-21	kworker/u4:4+events_unbound	19:42:21	0
2636236	99	88	60,16	cyclictest	0	-21	swapper/0	19:10:01	0
2636238	99	78	4,25	cyclictest	2843744	-21	ssh	00:30:28	1
2636238	99	78	40,34	cyclictest	2661928	-21	timerwakeupswit	20:45:36	1
2636238	99	76	68,6	cyclictest	2637455	-21	kworker/u4:9+events_unbound	19:20:00	1
2636238	99	76	36,7	cyclictest	0	-21	swapper/1	00:20:01	1
2636236	99	76	70,4	cyclictest	2797223	-21	kworker/u4:3+events_unbound	00:17:54	0
2636238	99	70	62,6	cyclictest	2683136	-21	kworker/u4:7+events_unbound	21:27:29	1
2636238	99	70	62,6	cyclictest	2683136	-21	kworker/u4:7+events_unbound	21:27:28	1
2636238	99	70	32,9	cyclictest	0	-21	swapper/1	20:25:25	1
2636238	99	68	61,5	cyclictest	2647856	-21	kworker/u4:6+events_unbound	20:07:21	1
2636236	99	68	61,5	cyclictest	2677171	-21	gcc	21:20:28	0
2636236	99	67	46,12	cyclictest	53	-21	kswapd0	00:05:13	0
2636236	99	67	10,13	cyclictest	2689785	-21	ssh	21:35:08	0
2636236	99	65	46,14	cyclictest	0	-21	swapper/0	21:40:08	0
2636238	99	64	34,4	cyclictest	2631229	-21	kworker/u4:11-rpciod	19:10:51	1
2636238	99	64	34,4	cyclictest	2631229	-21	kworker/u4:11-rpciod	19:10:51	1
2636238	99	62	52,6	cyclictest	2693867	-21	apt-get	21:40:09	1
2636238	99	62	44,13	cyclictest	0	-21	swapper/1	23:10:39	1
2636236	99	62	55,5	cyclictest	2650431	-21	kworker/u4:7+events_unbound	20:07:22	0
2636238	99	60	56,2	cyclictest	2801752	-21	kworker/u4:7+events_unbound	23:42:49	1
2636238	99	60	35,18	cyclictest	2703713	-21	sh	21:50:29	1
2636238	99	59	53,4	cyclictest	2659541	-21	kworker/u4:4+events_unbound	21:05:01	1
2636238	99	59	51,6	cyclictest	2687656	-21	kworker/u4:4+events_unbound	21:32:32	1
2636238	99	59	43,6	cyclictest	2771077	-21	kworker/u4:4+events_unbound	23:07:42	1
2636236	99	59	53,4	cyclictest	2659589	-21	basename	20:40:00	0
2636236	99	59	46,8	cyclictest	2791078	-21	ssh	23:30:20	0
2636236	99	59	44,11	cyclictest	2636233	-21	cyclictest	23:28:37	0
2636236	99	58	45,10	cyclictest	53	-21	kswapd0	21:55:09	0
2636238	99	57	49,6	cyclictest	2650433	-21	kworker/u4:11+events_unbound	20:32:24	1
2636236	99	57	50,5	cyclictest	2810598	-21	kworker/u4:12+events_unbound	00:02:51	0
2636238	99	56	7,41	cyclictest	2730061	-21	ssh	22:20:28	1
2636238	99	56	49,5	cyclictest	2748808	-21	kworker/u4:14+events_unbound	22:42:38	1
2636236	99	56	49,5	cyclictest	2796371	-21	/usr/sbin/munin	23:35:41	0
2636236	99	56	49,5	cyclictest	2651789	-21	kworker/u4:14+events_unbound	20:10:01	0
2636238	99	55	46,7	cyclictest	2756235	-21	ntp_kernel_err	22:50:24	1
2636238	99	55	40,4	cyclictest	2780004	-21	kworker/u4:14+events_unbound	23:26:01	1
2636238	99	55	39,10	cyclictest	2663398	-21	modprobe	20:55:00	1
2636236	99	55	46,6	cyclictest	2746616	-21	latency_hist	22:40:02	0
2636236	99	55	44,8	cyclictest	2825693	-21	aten_r7power_cu	00:10:16	0
2636236	99	55	36,17	cyclictest	2725227	-21	fschecks_count	22:15:20	0
2636238	99	54	47,4	cyclictest	2707781	-21	/usr/sbin/munin	21:55:25	1
2636238	99	54	46,6	cyclictest	2760887	-21	/usr/sbin/munin	22:55:37	1
2636236	99	54	3,43	cyclictest	2835064	-21	ssh	00:20:31	0
2636236	99	54	30,20	cyclictest	2636233	-21	cyclictest	23:57:47	0
2636238	99	53	45,6	cyclictest	2814998	-21	kworker/u4:15+events_unbound	23:57:53	1
2636238	99	53	45,6	cyclictest	2717487	-21	/usr/sbin/munin	22:05:43	1
2636236	99	53	36,14	cyclictest	2760273	-21	ssh	22:55:19	0
2636238	99	52	42,7	cyclictest	37	-21	kcompactd0	00:00:13	1
2636236	99	52	46,5	cyclictest	2650430	-21	kworker/u4:4+events_unbound	20:20:03	0
2636236	99	52	44,5	cyclictest	0	-21	swapper/0	00:30:31	0
2636238	99	51	44,5	cyclictest	2623738	-21	kworker/u4:1+events_unbound	19:30:02	1
2636238	99	51	44,5	cyclictest	2623738	-21	kworker/u4:1+events_unbound	19:30:01	1
2636238	99	51	41,6	cyclictest	2725157	-21	sed	22:15:17	1
2636238	99	51	35,6	cyclictest	2836394	-21	kworker/u4:0+events_unbound	00:27:57	1
2636236	99	51	44,5	cyclictest	2650427	-21	kworker/u4:0+events_unbound	20:45:04	0
2636236	99	51	44,5	cyclictest	2637455	-21	kworker/u4:9+events_unbound	19:35:02	0
2636236	99	51	4,17	cyclictest	2736099	-21	kworker/u4:1+rpciod	22:45:37	0
2636236	99	51	40,5	cyclictest	2768742	-21	ssh	23:05:11	0
2636236	99	51	37,9	cyclictest	2774058	-21	ssh	23:10:37	0
2636236	99	51	30,18	cyclictest	0	-21	swapper/0	23:00:16	0
2636238	99	50	43,5	cyclictest	2834531	-21	df_abs	00:20:17	1
2636236	99	50	44,5	cyclictest	2667223	-21	kworker/u4:15+events_unbound	21:14:00	0
2636236	99	50	43,5	cyclictest	2717080	-21	/usr/sbin/munin	22:05:39	0
2636238	99	49	43,4	cyclictest	2737781	-21	latency_hist	22:30:01	1
2636238	99	49	42,4	cyclictest	0	-21	swapper/1	23:15:23	1
2636238	99	49	39,7	cyclictest	2667809	-21	df_abs	21:10:18	1
2636236	99	49	43,4	cyclictest	2659541	-21	kworker/u4:4+events_unbound	20:54:43	0
2636236	99	49	42,5	cyclictest	2848631	-21	/usr/sbin/munin	00:35:41	0
2636236	99	49	42,5	cyclictest	2663376	-21	kworker/u4:9+events_unbound	20:55:02	0
2636236	99	49	42,5	cyclictest	2650434	-21	kworker/u4:12+events_unbound	20:30:02	0
2636236	99	49	39,8	cyclictest	2682241	-21	uptime	21:25:40	0
2636236	99	49	39,8	cyclictest	2682241	-21	uptime	21:25:39	0
2636238	99	48	44,3	cyclictest	2651789	-21	kworker/u4:14+events_unbound	20:12:24	1
2636236	99	48	41,5	cyclictest	2729479	-21	/usr/sbin/munin	22:20:16	0
2636236	99	48	41,5	cyclictest	2622467	-21	kworker/u4:3+events_unbound	19:26:33	0
2636236	99	48	41,5	cyclictest	2622467	-21	kworker/u4:3+events_unbound	19:26:32	0
2636236	99	48	41,5	cyclictest	0	-21	swapper/0	22:00:26	0
2636236	99	48	41,5	cyclictest	0	-21	swapper/0	20:05:01	0
2636236	99	48	41,4	cyclictest	0	-21	swapper/0	00:25:12	0
2636236	99	48	25,11	cyclictest	256	50	irq/23-42890000.ethernet	19:20:09	0
2636238	99	47	40,5	cyclictest	2799209	-21	idleruntime-cro	23:40:01	1
2636238	99	47	39,6	cyclictest	2755332	-21	latency_hist	22:50:00	1
2636236	99	47	42,4	cyclictest	2748808	-21	kworker/u4:14+events_unbound	22:43:00	0
2636236	99	47	41,5	cyclictest	2678614	-21	kworker/u4:5+events_unbound	21:32:00	0
2636236	99	47	41,4	cyclictest	2698803	-21	df_abs	21:45:18	0
2636236	99	47	4,11	cyclictest	19	1	rcuc/0	22:10:38	0
2636236	99	47	40,5	cyclictest	2638782	-21	idleruntime-cro	19:19:59	0
2636236	99	47	40,5	cyclictest	2637455	-21	kworker/u4:9+events_unbound	19:34:01	0
2636236	99	47	38,6	cyclictest	2807986	-21	idleruntime-cro	23:50:01	0
2636236	99	47	25,20	cyclictest	0	-21	swapper/0	22:50:41	0
2636238	99	46	40,4	cyclictest	2650545	-21	latency_hist	20:05:01	1
2636238	99	46	37,6	cyclictest	2694029	-21	latency_hist	21:40:02	1
2636238	99	46	36,8	cyclictest	2804441	-21	gltestperf	23:45:27	1
2636238	99	46	32,11	cyclictest	0	-21	swapper/1	22:00:13	1
2636236	99	46	39,6	cyclictest	2797223	-21	kworker/u4:3+events_unbound	23:44:11	0
2636236	99	46	39,5	cyclictest	2637455	-21	kworker/u4:9+events_unbound	19:50:59	0
2636236	99	46	38,6	cyclictest	0	-21	swapper/0	23:20:09	0
2636238	99	45	38,5	cyclictest	2739320	-21	unixbench_multi	22:30:39	1
2636238	99	45	38,5	cyclictest	2654363	-21	modprobe	20:20:00	1
2636238	99	45	37,6	cyclictest	2721115	-21	sh	22:10:25	1
2636238	99	45	37,6	cyclictest	2647858	-21	systemd	19:52:20	1
2636238	99	45	36,7	cyclictest	2742427	-21	acpi	22:35:13	1
2636238	99	45	36,7	cyclictest	0	-21	swapper/1	23:50:36	1
2636236	99	45	35,7	cyclictest	2636415	-21	aten2__	19:10:15	0
2636236	99	45	35,7	cyclictest	2636415	-21	aten2__	19:10:14	0
2636236	99	45	27,16	cyclictest	2734116	-21	fschecks_time	22:25:20	0
2636238	99	44	37,5	cyclictest	2698714	-21	aten_r7power_po	21:45:16	1
2636238	99	44	36,6	cyclictest	2663685	-21	aten_r7power_po	20:55:16	1
2636238	99	44	35,6	cyclictest	0	-21	swapper/1	23:20:19	1
2636238	99	44	33,8	cyclictest	0	-21	swapper/1	23:00:17	1
2636238	99	44	32,8	cyclictest	2667277	-21	ssh	21:10:01	1
2636236	99	44	37,5	cyclictest	2672071	-21	aten2__	21:15:17	0
2636236	99	44	37,5	cyclictest	2653604	-21	fschecks_time	20:15:20	0
2636236	99	44	37,5	cyclictest	2623742	-21	kworker/u4:8+events_unbound	19:59:00	0
2636236	99	44	37,5	cyclictest	0	-21	swapper/0	21:52:31	0
2636236	99	44	35,6	cyclictest	2731028	-21	kworker/u4:11-rpciod	22:35:01	0
2636236	99	44	34,7	cyclictest	2636233	-21	cyclictest	23:15:16	0
2636238	99	43	37,4	cyclictest	2660057	-21	fschecks_time	20:40:20	1
2636238	99	43	36,4	cyclictest	0	-21	swapper/1	21:20:24	1
2636238	99	43	34,7	cyclictest	2847576	-21	aten_r7power_po	00:35:17	1
2636238	99	43	33,7	cyclictest	0	-21	swapper/1	00:10:31	1
2636238	99	43	27,6	cyclictest	2674074	-21	kworker/u4:1+events_unbound	21:18:45	1
2636238	99	43	25,7	cyclictest	17	1	rcu_preempt	20:35:46	1
2636238	99	42	34,5	cyclictest	0	-21	swapper/1	23:30:26	1
2636236	99	42	34,6	cyclictest	2650427	-21	kworker/u4:0-xprtiod	20:29:59	0
2636236	99	42	34,5	cyclictest	0	-21	swapper/0	23:50:24	0
2636238	99	41	35,4	cyclictest	0	-21	swapper/1	19:55:21	1
2636238	99	41	32,7	cyclictest	0	-21	swapper/1	00:05:35	1
2636238	99	41	32,6	cyclictest	2634960	-21	kworker/u4:6+rpciod	19:30:37	1
2636236	99	41	34,5	cyclictest	2660832	-21	kworker/u4:6+events_unbound	21:00:04	0
2636236	99	41	34,5	cyclictest	2659850	-21	aten_r7power_po	20:40:16	0
2636238	99	39	31,6	cyclictest	2643906	-21	unixbench_singl	19:35:38	1
2636238	99	39	30,7	cyclictest	2644215	-21	aten_r7power_cu	19:40:15	1
2636238	99	39	30,7	cyclictest	2639839	-21	mailstats	19:20:40	1
2636238	99	37	29,6	cyclictest	2654912	-21	fschecks_time	20:20:19	1
2636238	99	35	26,6	cyclictest	0	-21	swapper/1	19:10:01	1
2636238	99	35	25,7	cyclictest	0	-21	swapper/1	19:45:11	1
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17