OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #3, slot #6

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the highest latencies:

System rack3slot6.osadl.org (updated Sun May 03, 2026 00:44:58)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
7290	2	666	0,4	sleep0	19	1	irq_work/0	19:05:19	0
29781	2	98	0,6	chrt	29779	-21	grep	20:05:36	0
8166	2	94	16,32	sleep1	0	-21	swapper/1	19:06:46	1
8663	99	88	74,10	cyclictest	26	-21	ksoftirqd/1	00:10:12	1
8663	99	86	27,52	cyclictest	457	-21	dbus-daemon	22:00:00	1
8663	99	85	8,62	cyclictest	29782	-21	apt-get	23:10:14	1
8663	99	85	20,52	cyclictest	0	-21	swapper/1	23:45:12	1
8663	99	85	15,58	cyclictest	0	-21	swapper/1	22:00:13	1
8663	99	85	11,59	cyclictest	0	-21	swapper/1	23:55:13	1
8663	99	84	22,54	cyclictest	8651	-21	cyclictest	19:55:13	1
8663	99	84	18,54	cyclictest	0	-21	swapper/1	20:25:09	1
8663	99	84	15,59	cyclictest	0	-21	swapper/1	22:25:12	1
8663	99	84	14,57	cyclictest	0	-21	swapper/1	20:30:13	1
8663	99	83	7,59	cyclictest	1107	-21	apt-get	21:50:12	1
8663	99	83	68,9	cyclictest	26	-21	ksoftirqd/1	20:20:11	1
8663	99	83	40,32	cyclictest	26	-21	ksoftirqd/1	22:35:14	1
8663	99	83	15,55	cyclictest	0	-21	swapper/1	22:05:23	1
8663	99	83	14,59	cyclictest	0	-21	swapper/1	23:20:12	1
8663	99	83	14,58	cyclictest	0	-21	swapper/1	00:15:13	1
8663	99	82	17,52	cyclictest	0	-21	swapper/1	23:05:23	1
8663	99	82	17,50	cyclictest	0	-21	swapper/1	19:40:24	1
8663	99	82	16,57	cyclictest	0	-21	swapper/1	19:50:13	1
8663	99	82	13,56	cyclictest	0	-21	swapper/1	22:50:11	1
8663	99	82	12,56	cyclictest	0	-21	swapper/1	20:35:12	1
8663	99	82	12,54	cyclictest	0	-21	swapper/1	23:50:15	1
8663	99	81	8,59	cyclictest	27941	-21	apt-get	21:35:14	1
8663	99	81	7,60	cyclictest	13922	-21	apt-get	20:55:11	1
8663	99	81	15,52	cyclictest	0	-21	swapper/1	21:00:15	1
8663	99	81	11,61	cyclictest	0	-21	swapper/1	19:35:13	1
8663	99	80	17,54	cyclictest	0	-21	swapper/1	00:00:14	1
8663	99	80	15,56	cyclictest	0	-21	swapper/1	23:25:12	1
8663	99	80	15,56	cyclictest	0	-21	swapper/1	21:05:15	1
8663	99	80	15,51	cyclictest	0	-21	swapper/1	23:15:14	1
8663	99	80	14,57	cyclictest	0	-21	swapper/1	21:45:13	1
8663	99	80	13,57	cyclictest	0	-21	swapper/1	22:30:15	1
8663	99	80	12,58	cyclictest	0	-21	swapper/1	00:05:24	1
8663	99	80	12,57	cyclictest	0	-21	swapper/1	00:35:14	1
8663	99	80	12,55	cyclictest	0	-21	swapper/1	00:20:14	1
8663	99	79	7,59	cyclictest	30601	-21	apt-get	20:10:12	1
8663	99	79	7,59	cyclictest	26200	-21	apt-get	21:30:11	1
8663	99	79	6,59	cyclictest	12757	-21	apt-get	19:20:12	1
8663	99	79	20,52	cyclictest	243	-21	systemd-journal	20:46:28	1
8663	99	79	16,53	cyclictest	0	-21	swapper/1	23:30:23	1
8663	99	79	16,53	cyclictest	0	-21	swapper/1	23:30:23	1
8663	99	79	14,53	cyclictest	0	-21	swapper/1	19:10:14	1
8663	99	79	13,54	cyclictest	0	-21	swapper/1	20:50:14	1
8663	99	79	13,53	cyclictest	0	-21	swapper/1	00:30:14	1
8663	99	79	12,59	cyclictest	0	-21	swapper/1	00:25:13	1
8663	99	78	7,59	cyclictest	21830	-21	apt-get	19:45:14	1
8663	99	78	6,58	cyclictest	20952	-21	apt-get	21:15:13	1
8663	99	78	15,52	cyclictest	0	-21	swapper/1	22:20:14	1
8663	99	78	13,54	cyclictest	0	-21	swapper/1	20:15:12	1
8663	99	78	12,54	cyclictest	0	-21	swapper/1	22:45:13	1
8663	99	77	7,56	cyclictest	8434	-21	apt-get	22:10:14	1
8663	99	77	7,56	cyclictest	14494	-21	apt-get	19:25:11	1
8663	99	77	7,55	cyclictest	8190	-21	apt-get	23:40:12	1
8663	99	77	21,51	cyclictest	243	-21	systemd-journal	21:25:00	1
8663	99	77	15,48	cyclictest	0	-21	swapper/1	19:15:12	1
8663	99	77	13,52	cyclictest	0	-21	swapper/1	23:35:13	1
8663	99	76	20,49	cyclictest	25383	-21	sshd	22:56:06	1
8663	99	75	20,45	cyclictest	457	-21	dbus-daemon	21:30:00	1
8663	99	75	12,55	cyclictest	0	-21	swapper/1	22:40:12	1
8663	99	75	10,52	cyclictest	25947	-21	apt-get	23:00:10	1
8663	99	74	13,50	cyclictest	0	-21	swapper/1	20:05:13	1
8663	99	73	10,53	cyclictest	0	-21	swapper/1	22:15:16	1
8659	99	73	6,35	cyclictest	0	-21	swapper/0	19:35:29	0
8663	99	72	13,53	cyclictest	17245	-21	cat	19:30:36	1
8659	99	72	7,33	cyclictest	0	-21	swapper/0	22:45:19	0
8663	99	71	16,48	cyclictest	30109	-21	aten2.4_r3power	21:40:26	1
8659	99	71	7,52	cyclictest	0	-21	swapper/0	21:40:23	0
8663	99	70	7,49	cyclictest	19213	-21	apt-get	21:10:11	1
8663	99	69	6,50	cyclictest	8708	-21	apt-get	20:40:13	1
8663	99	69	19,44	cyclictest	28651	-21	basename	20:05:00	1
8659	99	68	5,33	cyclictest	0	-21	swapper/0	22:10:21	0
8659	99	67	6,47	cyclictest	0	-21	swapper/0	00:35:12	0
8659	99	67	5,33	cyclictest	0	-21	swapper/0	22:15:21	0
8659	99	66	7,38	cyclictest	15	1	rcu_preempt	23:30:43	0
8659	99	66	7,38	cyclictest	15	1	rcu_preempt	23:30:43	0
8659	99	64	36,18	cyclictest	11679	-21	apt-get	23:50:13	0
8659	99	64	30,27	cyclictest	0	-21	swapper/0	22:50:12	0
8659	99	63	6,16	cyclictest	19048	-21	apt-get	00:10:12	0
8659	99	63	5,31	cyclictest	0	-21	swapper/0	20:00:21	0
8659	99	63	34,19	cyclictest	6641	-21	apt-get	22:05:25	0
8659	99	62	7,14	cyclictest	15	1	rcu_preempt	00:20:37	0
8659	99	62	38,13	cyclictest	4604	-21	apt-get	22:00:13	0
8659	99	62	12,43	cyclictest	0	-21	swapper/0	19:20:12	0
8659	99	61	28,27	cyclictest	0	-21	swapper/0	21:50:14	0
8659	99	61	23,32	cyclictest	0	-21	swapper/0	22:30:15	0
8659	99	61	14,41	cyclictest	15217	-21	apt-get	00:00:12	0
8659	99	60	30,24	cyclictest	0	-21	swapper/0	20:20:12	0
8659	99	60	18,36	cyclictest	8000	-21	cron	23:40:00	0
8659	99	59	8,37	cyclictest	0	-21	swapper/0	21:25:11	0
8659	99	59	35,10	cyclictest	20786	-21	apt-get	00:15:13	0
8659	99	59	30,24	cyclictest	0	-21	swapper/0	19:45:13	0
8659	99	58	9,44	cyclictest	0	-21	swapper/0	20:45:32	0
8659	99	58	7,38	cyclictest	15	1	rcu_preempt	20:35:33	0
8659	99	58	5,13	cyclictest	15	1	rcu_preempt	21:30:30	0
8659	99	58	35,10	cyclictest	2596	-21	apt-get	23:25:12	0
8659	99	58	34,10	cyclictest	856	-21	apt-get	23:20:12	0
8659	99	58	31,17	cyclictest	9935	-21	apt-get	23:45:14	0
8659	99	58	31,17	cyclictest	27988	-21	apt-get	23:05:22	0
8659	99	58	28,20	cyclictest	25310	-21	apt-get	19:55:14	0
8659	99	58	25,25	cyclictest	0	-21	swapper/0	23:10:13	0
8659	99	57	34,13	cyclictest	13416	-21	apt-get	23:55:13	0
8659	99	57	33,11	cyclictest	23572	-21	apt-get	19:50:13	0
8659	99	57	33,11	cyclictest	13677	-21	apt-get	22:25:11	0
8659	99	57	32,13	cyclictest	24287	-21	apt-get	00:25:13	0
8659	99	57	28,22	cyclictest	17158	-21	apt-get	22:35:14	0
8659	99	57	21,30	cyclictest	19036	-21	latency_hist	21:10:00	0
8659	99	56	32,12	cyclictest	12186	-21	apt-get	20:50:14	0
8659	99	56	3,14	cyclictest	15	1	rcu_preempt	20:55:14	0
8659	99	56	30,17	cyclictest	20089	-21	apt-get	19:40:24	0
8659	99	56	28,22	cyclictest	17490	-21	seq	19:31:45	0
8659	99	56	16,35	cyclictest	0	-21	swapper/0	21:15:13	0
8659	99	56	16,34	cyclictest	12086	-21	/usr/sbin/munin	19:15:42	0
12564	2	56	0,5	sleep0	17	1	rcuc/0	22:20:27	0
8659	99	55	33,14	cyclictest	5153	-21	apt-get	20:30:13	0
8659	99	55	30,11	cyclictest	17258	-21	apt-get	00:05:24	0
8659	99	55	28,15	cyclictest	8740	-21	apt-get	19:10:14	0
8659	99	55	19,31	cyclictest	32174	-21	pluginstate	20:15:01	0
8659	99	55	15,35	cyclictest	14560	-21	tail	19:25:15	0
8659	99	55	14,35	cyclictest	32420	-21	aten2.4_r3power	20:15:15	0
8659	99	54	5,43	cyclictest	0	-21	swapper/0	21:15:01	0
8659	99	54	31,11	cyclictest	3407	-21	apt-get	20:25:09	0
8659	99	54	30,11	cyclictest	31790	-21	apt-get	21:45:13	0
8659	99	54	28,13	cyclictest	2843	-21	apt-get	21:55:14	0
8659	99	54	26,21	cyclictest	0	-21	swapper/0	23:15:12	0
8659	99	54	19,22	cyclictest	14	-21	ksoftirqd/0	23:03:25	0
8659	99	54	15,33	cyclictest	28037	-21	aten2.4-expect	21:35:18	0
8659	99	53	9,39	cyclictest	0	-21	swapper/0	22:55:22	0
8659	99	53	5,43	cyclictest	0	-21	swapper/0	20:40:27	0
8659	99	53	20,26	cyclictest	0	-21	swapper/0	23:40:14	0
8659	99	53	11,37	cyclictest	23450	-21	ntp_states	21:20:29	0
8659	99	52	31,11	cyclictest	15684	-21	apt-get	21:00:15	0
8659	99	52	25,21	cyclictest	19345	-21	cat	22:40:21	0
8659	99	51	26,13	cyclictest	26026	-21	apt-get	00:30:14	0
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17