OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #7, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system

Characteristics of the 376 highest latencies:

System rack7slot2.osadl.org (updated Thu Dec 11, 2025 12:44:06)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
2667043	99	141	129,6	cyclictest	2766258	-21	kworker/u4:5+events_unbound	11:10:50	1
2667043	99	130	121,6	cyclictest	2734826	-21	kworker/u4:12+events_unbound	10:20:40	1
2667043	99	128	120,6	cyclictest	2804410	-21	kworker/u4:5+events_unbound	11:45:55	1
2667043	99	114	107,5	cyclictest	2666970	-21	kworker/u4:5+events_unbound	07:24:56	1
2667043	99	114	106,6	cyclictest	2696919	-21	kworker/u4:5+events_unbound	09:05:54	1
2667043	99	112	103,7	cyclictest	2790115	-21	kworker/u4:4+events_unbound	11:30:51	1
2667043	99	112	102,8	cyclictest	2685132	-21	kworker/u4:17+events_unbound	08:30:19	1
2667043	99	106	98,6	cyclictest	2793781	-21	kworker/u4:8+events_unbound	12:26:22	1
2667043	99	104	97,5	cyclictest	2821858	-21	kworker/u4:7+events_unbound	12:10:59	1
2667043	99	103	89,7	cyclictest	2727700	-21	kworker/u4:13+events_unbound	10:01:01	1
2667043	99	101	94,5	cyclictest	2677317	-21	kworker/u4:14+events_unbound	08:15:16	1
2667043	99	100	92,6	cyclictest	2748924	-21	kworker/u4:7+events_unbound	10:26:05	1
2667043	99	99	91,6	cyclictest	2672162	-21	kworker/u4:2+events_unbound	08:25:46	1
2667043	99	97	90,5	cyclictest	2691768	-21	kworker/u4:9+events_unbound	08:50:53	1
2667041	99	96	87,7	cyclictest	2714211	-21	kworker/u4:8+events_unbound	09:35:31	0
2667043	99	94	86,6	cyclictest	2793780	-21	kworker/u4:7+events_unbound	11:35:52	1
2667041	99	93	24,56	cyclictest	0	-21	swapper/0	12:35:32	0
2667043	99	92	85,5	cyclictest	2666970	-21	kworker/u4:5+events_unbound	07:19:52	1
2667043	99	92	84,6	cyclictest	2755661	-21	kworker/u4:6+events_unbound	11:15:50	1
2667043	99	91	83,6	cyclictest	2677317	-21	kworker/u4:14+events_unbound	09:25:58	1
2667041	99	91	84,4	cyclictest	2714207	-21	kworker/u4:1+events_unbound	09:50:33	0
2667041	99	91	31,45	cyclictest	2760965	-21	acpi	10:40:12	0
2667043	99	90	82,6	cyclictest	2685132	-21	kworker/u4:17+events_unbound	08:35:50	1
2667041	99	89	79,7	cyclictest	2714207	-21	kworker/u4:1+events_unbound	10:16:04	0
2667043	99	88	80,6	cyclictest	2677317	-21	kworker/u4:14+events_unbound	09:00:54	1
2667043	99	88	80,6	cyclictest	2289481	-21	kworker/u4:10+events_unbound	07:40:08	1
2667041	99	88	79,6	cyclictest	2790115	-21	kworker/u4:4+events_unbound	11:26:11	0
2667041	99	87	76,8	cyclictest	2773350	-21	kworker/u4:2+events_unbound	11:00:47	0
2667043	99	84	76,6	cyclictest	2672162	-21	kworker/u4:2+events_unbound	08:00:41	1
2667041	99	83	65,16	cyclictest	2710752	-21	kworker/u4:0+events_unbound	09:30:30	0
2667043	99	82	74,6	cyclictest	2677318	-21	kworker/u4:16+events_unbound	08:40:50	1
2667041	99	82	74,5	cyclictest	2677317	-21	kworker/u4:14+events_unbound	09:15:29	0
2667043	99	81	72,6	cyclictest	2677318	-21	kworker/u4:16+events_unbound	08:55:25	1
2667041	99	81	70,8	cyclictest	2717288	-21	kworker/u4:10+events_unbound	09:40:33	0
2667043	99	78	71,5	cyclictest	2677317	-21	kworker/u4:14+events_unbound	07:55:11	1
2667043	99	78	70,6	cyclictest	2738518	-21	kworker/u4:0+events_unbound	10:11:00	1
2667041	99	78	26,28	cyclictest	0	-21	swapper/0	12:10:24	0
2667041	99	77	69,6	cyclictest	2793781	-21	kworker/u4:8+events_unbound	11:40:54	0
2667041	99	76	68,5	cyclictest	2691767	-21	kworker/u4:6-rpciod	09:25:08	0
2667041	99	75	29,27	cyclictest	0	-21	swapper/0	12:05:43	0
2667041	99	75	28,30	cyclictest	0	-21	swapper/0	12:20:17	0
2667041	99	75	25,27	cyclictest	0	-21	swapper/0	12:30:32	0
2667041	99	75	2,48	cyclictest	314	-21	systemd-journal	09:45:33	0
2667041	99	75	21,34	cyclictest	2835230	-21	wget	12:25:19	0
2667041	99	74	28,40	cyclictest	0	-21	swapper/0	07:15:21	0
2667041	99	74	2,43	cyclictest	2697114	-21	apt-get	09:05:12	0
2667043	99	72	65,5	cyclictest	2762953	-21	kworker/u4:14+events_unbound	10:46:08	1
2667043	99	72	64,5	cyclictest	2289481	-21	kworker/u4:10+events_unbound	07:35:27	1
2667041	99	72	40,27	cyclictest	53	-21	kswapd0	10:00:07	0
2667041	99	72	36,34	cyclictest	2677021	-21	cat	07:45:37	0
2667041	99	72	27,39	cyclictest	0	-21	swapper/0	08:00:38	0
2667041	99	71	32,19	cyclictest	53	-21	kswapd0	12:00:13	0
2667041	99	71	25,27	cyclictest	0	-21	swapper/0	11:45:36	0
2667041	99	70	1,45	cyclictest	2813842	-21	ssh	11:55:33	0
2667043	99	68	61,5	cyclictest	2766256	-21	kworker/u4:3+events_unbound	10:51:09	1
2667041	99	68	38,27	cyclictest	2802136	-21	idleruntime-cro	11:40:00	0
2667041	99	66	54,10	cyclictest	2672162	-21	kworker/u4:2+events_unbound	08:10:14	0
2667041	99	66	39,21	cyclictest	2730447	-21	/usr/sbin/munin	09:55:41	0
2667041	99	66	34,30	cyclictest	2747227	-21	head	10:20:20	0
2667041	99	66	1,62	cyclictest	2698474	-21	ssh	09:10:15	0
2667043	99	65	54,8	cyclictest	2825451	-21	kworker/u4:2+events_unbound	12:21:01	1
2667041	99	65	24,36	cyclictest	2781818	-21	munin-plugin-st	11:10:00	0
2667041	99	65	20,42	cyclictest	0	-21	swapper/0	10:55:37	0
2667041	99	65	1,55	cyclictest	2754443	-21	ssh	10:30:25	0
2667041	99	64	30,28	cyclictest	2679699	-21	timerwakeupswit	07:55:39	0
2667041	99	64	29,32	cyclictest	2669820	-21	aten2__	07:20:15	0
2667041	99	64	2,58	cyclictest	2688000	-21	apt-get	08:30:20	0
2667041	99	64	19,40	cyclictest	2678133	-21	ntpq	07:50:30	0
2667041	99	63	28,29	cyclictest	2783335	-21	ssh	11:10:48	0
2667041	99	63	2,57	cyclictest	2785517	-21	ntpq	11:15:26	0
2667041	99	63	2,44	cyclictest	2789024	-21	ntpq	11:20:31	0
2667041	99	63	22,39	cyclictest	257	50	irq/23-42890000.ethernet	07:40:30	0
2667041	99	63	1,13	cyclictest	2671625	-21	ntpq	07:25:24	0
2667043	99	62	49,9	cyclictest	37	-21	kcompactd0	10:07:51	1
2667041	99	62	4,54	cyclictest	2691333	-21	ntpq	08:40:37	0
2667041	99	62	3,55	cyclictest	2696503	-21	ntpq	09:00:36	0
2667041	99	62	3,55	cyclictest	2672367	-21	apt-get	07:30:12	0
2667041	99	62	2,7	cyclictest	2764576	-21	apt-get	10:45:13	0
2667041	99	62	2,56	cyclictest	2750283	-21	apt-get	10:25:14	0
2667041	99	62	2,55	cyclictest	2682039	-21	ntpq	08:05:29	0
2667041	99	62	1,57	cyclictest	2827171	-21	apt-get	12:15:12	0
2667041	99	62	14,27	cyclictest	0	-21	swapper/0	10:35:40	0
2667041	99	61	7,47	cyclictest	2810062	-21	ssh	11:50:26	0
2667041	99	61	3,54	cyclictest	2739989	-21	apt-get	10:10:12	0
2667041	99	61	2,54	cyclictest	2736684	-21	ssh	10:05:18	0
2667041	99	61	1,9	cyclictest	2795534	-21	apt-get	11:30:13	0
2667041	99	61	1,7	cyclictest	2769596	-21	ssh	10:51:41	0
2667043	99	60	7,40	cyclictest	2803227	-21	ssh	11:40:36	1
2667041	99	60	12,45	cyclictest	1060	-21	snmpd	08:26:33	0
2667043	99	59	45,12	cyclictest	2778535	-21	aten2__	11:05:16	1
2667041	99	59	3,52	cyclictest	2673647	-21	apt-get	07:35:13	0
2667043	99	58	50,6	cyclictest	2700608	-21	kworker/u4:4+events_unbound	09:15:01	1
2667043	99	58	49,7	cyclictest	2723136	-21	ntpq	09:45:33	1
2667041	99	58	4,50	cyclictest	2695139	-21	ntpq	08:55:33	0
2667041	99	58	36,19	cyclictest	2689351	-21	aten_r7power_en	08:35:16	0
2667041	99	58	2,49	cyclictest	2691965	-21	apt-get	08:45:13	0
2667041	99	58	2,38	cyclictest	2706408	-21	ssh	09:20:33	0
2667043	99	57	49,6	cyclictest	2661990	-21	kworker/u4:9+events_unbound	07:10:03	1
2667041	99	57	1,53	cyclictest	2684032	-21	apt-get	08:15:13	0
2667041	99	56	25,29	cyclictest	0	-21	swapper/0	07:10:40	0
2667041	99	56	2,50	cyclictest	2685354	-21	apt-get	08:20:10	0
2667043	99	55	47,6	cyclictest	2685132	-21	kworker/u4:17+events_unbound	09:20:03	1
2667041	99	54	1,12	cyclictest	2693857	-21	ntpq	08:50:30	0
2667043	99	52	45,5	cyclictest	2792879	-21	switchtime	11:25:39	1
2667043	99	52	43,7	cyclictest	2744301	-21	sendmail_mailst	10:15:39	1
2667043	99	51	44,5	cyclictest	2691769	-21	kworker/u4:10+events_unbound	09:17:36	1
2667043	99	50	43,5	cyclictest	2817527	-21	sendmail_mailqu	12:00:43	1
2667043	99	50	42,5	cyclictest	2848085	-21	latency_hist	12:40:01	1
2667043	99	49	42,5	cyclictest	2692710	-21	sendmail_mailqu	08:45:38	1
2667043	99	49	40,7	cyclictest	2841057	-21	systemd	12:31:02	1
2667043	99	48	41,5	cyclictest	2827311	-21	/usr/sbin/munin	12:15:16	1
2667041	99	48	1,41	cyclictest	2667132	-21	latency_hist	07:10:01	0
2667043	99	47	40,5	cyclictest	2813673	-21	/usr/sbin/munin	11:55:24	1
2667043	99	47	40,5	cyclictest	2757613	-21	cut	10:35:18	1
2667043	99	47	39,6	cyclictest	2674728	-21	kworker/u4:3+events_unbound	08:05:14	1
2667043	99	47	37,7	cyclictest	2685553	-21	df_abs	08:20:16	1
2667043	99	46	39,5	cyclictest	2757186	-21	awk	10:35:00	1
2667043	99	46	39,5	cyclictest	2715496	-21	aten2__	09:35:17	1
2667043	99	45	37,6	cyclictest	2809599	-21	aten_r7power_vo	11:50:18	1
2667043	99	44	36,6	cyclictest	2775833	-21	sendmail_mailtr	11:00:38	1
2667043	99	44	36,6	cyclictest	2683835	-21	kworker/u4:0+events_unbound	08:14:59	1
2667043	99	44	36,6	cyclictest	0	-21	swapper/1	07:45:31	1
2667043	99	44	36,5	cyclictest	2712997	-21	cat	09:30:22	1
2667043	99	44	35,6	cyclictest	0	-21	swapper/1	12:05:38	1
2667043	99	44	32,9	cyclictest	2732600	-21	latency_hist	10:00:00	1
2667043	99	44	31,9	cyclictest	37	-21	kcompactd0	09:52:12	1
2667043	99	43	37,4	cyclictest	2719185	-21	df_abs	09:40:17	1
2667043	99	43	35,6	cyclictest	2766258	-21	kworker/u4:5-xprtiod	10:55:31	1
2667043	99	42	34,6	cyclictest	2734826	-21	kworker/u4:12-rpciod	10:40:16	1
2667043	99	41	35,4	cyclictest	0	-21	swapper/1	11:21:10	1
2667043	99	41	32,6	cyclictest	0	-21	swapper/1	07:55:00	1
2667043	99	40	33,5	cyclictest	2666970	-21	kworker/u4:5+events_unbound	07:25:02	1
2667043	99	38	27,6	cyclictest	17	1	rcu_preempt	07:30:35	1
2667043	99	38	11,16	cyclictest	2667020	-21	munin-run	07:10:00	1
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17