OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #3, slot #6

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the highest latencies:

System rack3slot6.osadl.org (updated Sat Jul 12, 2025 00:45:04)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
24874	99	110	92,10	cyclictest	14	-21	ksoftirqd/0	19:10:13	0
24882	99	107	60,26	cyclictest	26	-21	ksoftirqd/1	22:35:26	1
24874	99	98	6,50	cyclictest	6608	-21	apt-get	00:20:13	0
24405	2	97	20,31	sleep1	0	-21	swapper/1	19:07:01	1
24874	99	96	67,18	cyclictest	14	-21	ksoftirqd/0	23:45:25	0
24874	99	93	62,18	cyclictest	14	-21	ksoftirqd/0	20:10:14	0
24874	99	91	60,20	cyclictest	14	-21	ksoftirqd/0	21:30:15	0
24874	99	90	58,18	cyclictest	14	-21	ksoftirqd/0	20:40:15	0
24874	99	89	57,21	cyclictest	14	-21	ksoftirqd/0	21:20:18	0
24874	99	89	57,20	cyclictest	14	-21	ksoftirqd/0	21:25:17	0
24874	99	87	53,19	cyclictest	14	-21	ksoftirqd/0	20:45:14	0
24874	99	86	56,20	cyclictest	14	-21	ksoftirqd/0	19:45:17	0
24874	99	86	56,20	cyclictest	14	-21	ksoftirqd/0	19:45:17	0
23125	2	85	32,40	sleep0	0	-21	swapper/0	19:05:13	0
24874	99	84	31,42	cyclictest	14	-21	ksoftirqd/0	23:00:13	0
24874	99	84	31,42	cyclictest	14	-21	ksoftirqd/0	23:00:13	0
24874	99	84	13,27	cyclictest	0	-21	swapper/0	22:00:16	0
24874	99	83	7,62	cyclictest	4785	-21	apt-get	22:45:15	0
24874	99	83	7,62	cyclictest	4785	-21	apt-get	22:45:14	0
24874	99	82	9,59	cyclictest	22847	-21	apt-get	22:05:13	0
24874	99	82	7,62	cyclictest	1762	-21	apt-get	19:35:14	0
24874	99	82	6,61	cyclictest	24383	-21	apt-get	23:40:12	0
24874	99	82	17,52	cyclictest	0	-21	swapper/0	22:50:17	0
24874	99	82	12,57	cyclictest	0	-21	swapper/0	20:50:15	0
24874	99	82	12,56	cyclictest	0	-21	swapper/0	21:00:15	0
24874	99	82	12,55	cyclictest	0	-21	swapper/0	20:05:16	0
24874	99	81	6,60	cyclictest	15595	-21	apt-get	23:15:14	0
24874	99	80	7,59	cyclictest	28142	-21	apt-get	22:20:15	0
24874	99	80	7,58	cyclictest	28967	-21	apt-get	19:20:12	0
24874	99	80	7,58	cyclictest	17610	-21	apt-get	21:50:17	0
24874	99	80	49,18	cyclictest	14	-21	ksoftirqd/0	19:40:16	0
24874	99	80	23,52	cyclictest	261	-21	systemd-journal	23:51:32	0
24874	99	80	12,56	cyclictest	0	-21	swapper/0	22:40:16	0
24874	99	80	12,55	cyclictest	0	-21	swapper/0	19:30:13	0
24874	99	80	12,54	cyclictest	0	-21	swapper/0	21:15:16	0
24874	99	79	6,60	cyclictest	4848	-21	apt-get	00:15:14	0
24874	99	79	21,50	cyclictest	261	-21	systemd-journal	22:26:17	0
24874	99	79	12,57	cyclictest	0	-21	swapper/0	20:35:14	0
24882	99	78	5,15	cyclictest	8348	-21	apt-get	00:25:15	1
24874	99	77	8,56	cyclictest	12062	-21	apt-get	23:05:11	0
24874	99	77	7,56	cyclictest	30431	-21	apt-get	20:55:13	0
24874	99	77	6,57	cyclictest	19351	-21	apt-get	21:55:19	0
24874	99	77	14,52	cyclictest	0	-21	swapper/0	23:20:12	0
24874	99	77	13,50	cyclictest	0	-21	swapper/0	23:55:16	0
24874	99	77	13,50	cyclictest	0	-21	swapper/0	23:55:15	0
24874	99	77	13,50	cyclictest	0	-21	swapper/0	22:10:16	0
24874	99	76	7,57	cyclictest	27210	-21	apt-get	19:15:16	0
24874	99	76	6,58	cyclictest	26382	-21	apt-get	22:15:16	0
24874	99	76	6,56	cyclictest	30711	-21	apt-get	19:25:15	0
24874	99	76	6,56	cyclictest	18185	-21	apt-get	20:20:17	0
24874	99	76	6,56	cyclictest	14133	-21	apt-get	21:40:14	0
24874	99	76	11,51	cyclictest	0	-21	swapper/0	19:50:15	0
24874	99	75	8,55	cyclictest	8568	-21	apt-get	22:55:13	0
24874	99	75	6,57	cyclictest	22580	-21	apt-get	23:35:18	0
24874	99	75	30,37	cyclictest	420	-21	dbus-daemon	00:05:00	0
24874	99	75	21,48	cyclictest	17438	-21	ls	21:50:00	0
24874	99	74	9,52	cyclictest	0	-21	swapper/0	21:10:14	0
24874	99	74	7,53	cyclictest	8348	-21	apt-get	00:25:16	0
24874	99	74	21,45	cyclictest	420	-21	dbus-daemon	20:05:01	0
24874	99	74	14,54	cyclictest	1110	-21	apt-key	22:35:05	0
24882	99	73	6,16	cyclictest	0	-21	swapper/1	21:45:12	1
24874	99	73	8,53	cyclictest	21661	-21	apt-get	20:30:14	0
24874	99	73	8,53	cyclictest	21661	-21	apt-get	20:30:14	0
24874	99	73	7,51	cyclictest	19091	-21	apt-get	23:25:13	0
24874	99	73	5,56	cyclictest	1804	-21	apt	21:05:12	0
24874	99	73	3,15	cyclictest	261	-21	systemd-journal	00:06:35	0
24874	99	73	11,48	cyclictest	0	-21	swapper/0	21:35:17	0
24882	99	72	7,50	cyclictest	0	-21	swapper/1	21:35:15	1
24874	99	72	8,51	cyclictest	16418	-21	apt-get	20:15:12	0
24874	99	72	7,10	cyclictest	0	-21	swapper/0	00:30:19	0
24874	99	72	6,53	cyclictest	9394	-21	apt-get	19:55:18	0
24874	99	71	7,50	cyclictest	13863	-21	apt-get	23:10:17	0
24874	99	71	15,48	cyclictest	0	-21	swapper/0	20:30:01	0
24874	99	70	21,42	cyclictest	427	-21	in:imuxsock	22:33:33	0
24874	99	69	7,48	cyclictest	11824	-21	apt-get	00:35:14	0
24874	99	68	6,49	cyclictest	3108	-21	apt-get	00:10:14	0
24882	99	67	6,43	cyclictest	0	-21	swapper/1	21:05:23	1
24882	99	67	10,41	cyclictest	0	-21	swapper/1	20:25:14	1
24874	99	67	6,18	cyclictest	20833	-21	apt-get	23:30:14	0
24882	99	66	10,43	cyclictest	0	-21	swapper/1	21:55:17	1
24882	99	65	7,44	cyclictest	0	-21	swapper/1	00:05:13	1
24882	99	65	6,12	cyclictest	0	-21	swapper/1	19:25:13	1
24882	99	64	20,27	cyclictest	26	-21	ksoftirqd/1	23:30:12	1
24882	99	63	6,41	cyclictest	0	-21	swapper/1	19:15:15	1
24882	99	62	6,42	cyclictest	0	-21	swapper/1	20:20:17	1
24882	99	61	21,29	cyclictest	26	-21	ksoftirqd/1	22:30:12	1
24882	99	59	7,38	cyclictest	0	-21	swapper/1	23:50:14	1
1647	2	59	0,18	sleep1	26	-21	ksoftirqd/1	19:35:01	1
24882	99	57	9,42	cyclictest	0	-21	swapper/1	19:35:36	1
24882	99	57	6,38	cyclictest	0	-21	swapper/1	00:35:15	1
24882	99	57	31,19	cyclictest	261	-21	systemd-journal	23:00:00	1
24882	99	57	21,25	cyclictest	26	-21	ksoftirqd/1	19:55:18	1
24882	99	57	19,26	cyclictest	26	-21	ksoftirqd/1	00:10:14	1
24882	99	56	6,35	cyclictest	0	-21	swapper/1	23:15:14	1
24882	99	56	32,18	cyclictest	30214	-21	expr	23:55:20	1
24882	99	56	32,18	cyclictest	30214	-21	expr	23:55:20	1
24882	99	56	22,21	cyclictest	0	-21	swapper/1	22:05:15	1
24882	99	56	17,26	cyclictest	26	-21	ksoftirqd/1	20:35:13	1
24882	99	55	6,37	cyclictest	0	-21	swapper/1	22:45:12	1
24882	99	55	6,37	cyclictest	0	-21	swapper/1	22:45:12	1
24882	99	55	4,22	cyclictest	0	-21	swapper/1	20:55:25	1
24882	99	55	23,20	cyclictest	0	-21	swapper/1	23:10:17	1
24882	99	55	17,12	cyclictest	25	1	rcuc/1	22:13:40	1
24882	99	54	2,46	cyclictest	22637	-21	runrttasks	22:04:41	1
24882	99	54	21,20	cyclictest	0	-21	swapper/1	21:15:13	1
24882	99	53	7,13	cyclictest	26	-21	ksoftirqd/1	20:50:24	1
24882	99	53	6,9	cyclictest	0	-21	swapper/1	23:35:36	1
24882	99	53	36,11	cyclictest	0	-21	swapper/1	00:30:29	1
24882	99	53	2,12	cyclictest	7734	-21	sort	22:50:39	1
24882	99	53	20,20	cyclictest	0	-21	swapper/1	23:05:11	1
24882	99	53	17,12	cyclictest	26	-21	ksoftirqd/1	00:21:12	1
24882	99	53	16,15	cyclictest	22032	-21	ls	20:30:22	1
24882	99	53	16,15	cyclictest	22032	-21	ls	20:30:22	1
24882	99	53	15,21	cyclictest	26	-21	ksoftirqd/1	19:21:47	1
24882	99	53	13,20	cyclictest	26	-21	ksoftirqd/1	23:25:34	1
24882	99	52	6,40	cyclictest	0	-21	swapper/1	23:40:23	1
24882	99	52	5,42	cyclictest	0	-21	swapper/1	20:15:17	1
24882	99	52	3,18	cyclictest	26428	-21	apt-get	23:45:21	1
24882	99	52	13,20	cyclictest	26	-21	ksoftirqd/1	23:22:35	1
24882	99	51	8,15	cyclictest	29914	-21	telnet	22:25:15	1
24882	99	51	5,41	cyclictest	0	-21	swapper/1	20:40:31	1
24882	99	51	5,40	cyclictest	0	-21	swapper/1	00:15:13	1
24882	99	51	37,10	cyclictest	24867	-21	cyclictest	21:50:16	1
24882	99	51	35,10	cyclictest	0	-21	swapper/1	22:40:36	1
24882	99	51	34,11	cyclictest	0	-21	swapper/1	23:00:28	1
24882	99	51	34,11	cyclictest	0	-21	swapper/1	23:00:28	1
24882	99	51	26,19	cyclictest	514	-21	snmpd	19:45:23	1
24882	99	51	26,19	cyclictest	514	-21	snmpd	19:45:23	1
24882	99	51	15,22	cyclictest	26	-21	ksoftirqd/1	20:10:55	1
24882	99	50	5,10	cyclictest	0	-21	swapper/1	19:50:07	1
24882	99	50	3,11	cyclictest	31483	-21	apt-get	00:00:08	1
24882	99	50	14,24	cyclictest	26	-21	ksoftirqd/1	19:41:01	1
24882	99	50	11,11	cyclictest	26	-21	ksoftirqd/1	21:10:25	1
24882	99	49	7,13	cyclictest	26	-21	ksoftirqd/1	21:30:34	1
24882	99	49	5,39	cyclictest	0	-21	swapper/1	21:20:23	1
24882	99	49	13,12	cyclictest	26	-21	ksoftirqd/1	20:03:48	1
24882	99	49	11,12	cyclictest	26	-21	ksoftirqd/1	21:00:19	1
24882	99	49	10,13	cyclictest	26	-21	ksoftirqd/1	20:05:30	1
24882	99	48	5,38	cyclictest	0	-21	swapper/1	22:20:31	1
24882	99	48	5,38	cyclictest	0	-21	swapper/1	21:40:14	1
24882	99	48	12,19	cyclictest	26	-21	ksoftirqd/1	20:45:43	1
24882	99	47	17,25	cyclictest	25227	-21	cat	19:10:18	1
24882	99	46	9,32	cyclictest	0	-21	swapper/1	22:15:24	1
24882	99	45	3,13	cyclictest	26	-21	ksoftirqd/1	21:25:17	1
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	3
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
none	16:34:23	2
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17