OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #3, slot #6

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the highest latencies:

System rack3slot6.osadl.org (updated Tue Jul 21, 2026 12:44:59)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
1134	2	91	10,69	sleep0	1212	-21	irqstats	07:05:22	0
2177	2	82	28,43	sleep1	0	-21	swapper/1	07:08:37	1
2490	99	77	15,55	cyclictest	0	-21	swapper/0	12:15:14	0
2495	99	75	53,13	cyclictest	5701	-21	apt-get	11:50:16	1
2495	99	75	34,36	cyclictest	26	-21	ksoftirqd/1	09:50:43	1
2495	99	73	33,20	cyclictest	26	-21	ksoftirqd/1	08:23:06	1
11526	2	73	0,5	sleep1	0	-21	swapper/1	10:35:19	1
11526	2	73	0,5	sleep1	0	-21	swapper/1	10:35:19	1
2490	99	72	55,11	cyclictest	14	-21	ksoftirqd/0	12:20:13	0
2490	99	71	15,50	cyclictest	0	-21	swapper/0	10:35:11	0
2490	99	71	15,50	cyclictest	0	-21	swapper/0	10:35:11	0
2490	99	70	7,53	cyclictest	3850	-21	apt-get	10:15:12	0
2490	99	70	42,19	cyclictest	22023	-21	apt-get	11:05:13	0
2490	99	70	15,15	cyclictest	0	-21	swapper/0	09:40:13	0
2490	99	69	52,9	cyclictest	14	-21	ksoftirqd/0	12:25:24	0
21906	2	69	0,11	sleep1	21907	-21	cut	12:35:17	1
2495	99	68	7,55	cyclictest	0	-21	swapper/1	11:55:16	1
2495	99	68	24,40	cyclictest	26	-21	ksoftirqd/1	08:00:14	1
2490	99	68	44,9	cyclictest	14	-21	ksoftirqd/0	07:40:31	0
2490	99	68	44,9	cyclictest	14	-21	ksoftirqd/0	07:40:31	0
2490	99	67	11,49	cyclictest	0	-21	swapper/0	09:25:13	0
2495	99	66	25,36	cyclictest	0	-21	swapper/1	09:40:12	1
2490	99	66	7,21	cyclictest	29813	-21	apt-get	08:25:14	0
2490	99	66	39,22	cyclictest	14	-21	ksoftirqd/0	07:15:17	0
2490	99	66	37,22	cyclictest	562	-21	runrttasks	12:30:17	0
2490	99	66	32,27	cyclictest	0	-21	swapper/0	12:05:13	0
2490	99	65	42,11	cyclictest	14	-21	ksoftirqd/0	08:50:15	0
2490	99	65	37,20	cyclictest	13189	-21	apt-get	09:10:11	0
2490	99	65	37,14	cyclictest	370	-21	apt-get	11:35:14	0
2490	99	65	32,21	cyclictest	2106	-21	apt-get	10:10:14	0
2490	99	64	37,15	cyclictest	2576	-21	apt-get	07:10:12	0
2490	99	64	34,21	cyclictest	562	-21	runrttasks	10:00:13	0
2490	99	64	29,23	cyclictest	0	-21	swapper/0	09:15:14	0
2490	99	63	41,11	cyclictest	6596	-21	apt-get	07:20:14	0
2490	99	63	40,17	cyclictest	14	-21	ksoftirqd/0	09:00:30	0
2495	99	62	16,41	cyclictest	0	-21	swapper/1	12:15:34	1
2490	99	62	34,15	cyclictest	0	-21	swapper/0	10:20:10	0
2490	99	62	33,23	cyclictest	0	-21	swapper/0	10:45:22	0
2490	99	62	30,21	cyclictest	20743	-21	apt-get	08:00:13	0
2490	99	62	29,25	cyclictest	0	-21	swapper/0	08:05:14	0
2495	99	61	37,9	cyclictest	26	-21	ksoftirqd/1	09:58:44	1
2495	99	61	37,18	cyclictest	26	-21	ksoftirqd/1	08:05:40	1
2490	99	61	34,22	cyclictest	0	-21	swapper/0	11:30:24	0
2490	99	61	32,20	cyclictest	0	-21	swapper/0	11:25:15	0
2490	99	61	31,16	cyclictest	0	-21	swapper/0	09:20:12	0
2490	99	61	29,25	cyclictest	0	-21	swapper/0	07:25:16	0
2490	99	61	28,24	cyclictest	0	-21	swapper/0	11:10:13	0
2490	99	61	27,25	cyclictest	12738	-21	taskset	12:10:16	0
2490	99	61	23,32	cyclictest	9015	-21	cron	12:00:01	0
2490	99	61	22,14	cyclictest	25682	-21	aten2.4-expect	11:15:18	0
2495	99	60	27,21	cyclictest	26	-21	ksoftirqd/1	07:22:36	1
2490	99	60	36,12	cyclictest	16781	-21	apt-get	10:50:13	0
2490	99	60	3,19	cyclictest	9454	-21	seq	10:30:15	0
2490	99	60	30,16	cyclictest	4422	-21	apt-get	08:45:13	0
2490	99	60	28,26	cyclictest	0	-21	swapper/0	12:35:13	0
2490	99	60	28,23	cyclictest	0	-21	swapper/0	11:20:12	0
2490	99	60	17,38	cyclictest	29640	-21	sh	08:25:00	0
2495	99	59	46,8	cyclictest	0	-21	swapper/1	12:26:31	1
2495	99	59	36,9	cyclictest	26	-21	ksoftirqd/1	10:44:29	1
2495	99	59	31,9	cyclictest	26	-21	ksoftirqd/1	11:05:13	1
2495	99	59	13,18	cyclictest	9525	-21	sed	09:00:01	1
2490	99	59	4,31	cyclictest	36	-21	kcompactd0	07:35:14	0
2490	99	59	33,19	cyclictest	243	-21	systemd-journal	08:40:15	0
2490	99	59	31,22	cyclictest	0	-21	swapper/0	09:50:23	0
2490	99	59	30,19	cyclictest	14	-21	ksoftirqd/0	09:30:14	0
2490	99	59	28,26	cyclictest	0	-21	swapper/0	09:55:16	0
2490	99	59	24,31	cyclictest	243	-21	systemd-journal	09:08:55	0
2490	99	59	11,42	cyclictest	562	-21	runrttasks	10:08:03	0
2495	99	58	36,16	cyclictest	26	-21	ksoftirqd/1	08:51:06	1
2495	99	58	34,18	cyclictest	26	-21	ksoftirqd/1	09:05:20	1
2495	99	58	27,14	cyclictest	26	-21	ksoftirqd/1	10:02:40	1
2495	99	58	13,40	cyclictest	0	-21	swapper/1	07:50:21	1
2495	99	58	10,13	cyclictest	3689	-21	head	07:10:36	1
2490	99	58	32,15	cyclictest	18983	-21	apt-get	07:55:15	0
2490	99	58	30,15	cyclictest	0	-21	swapper/0	11:00:14	0
2490	99	58	30,14	cyclictest	12995	-21	apt-get	10:40:13	0
2490	99	58	25,22	cyclictest	0	-21	swapper/0	11:45:15	0
2495	99	57	35,8	cyclictest	26	-21	ksoftirqd/1	10:17:27	1
2495	99	57	17,21	cyclictest	0	-21	swapper/1	10:10:31	1
2495	99	57	15,37	cyclictest	26	-21	ksoftirqd/1	08:40:25	1
2495	99	57	10,11	cyclictest	0	-21	swapper/1	10:05:20	1
2490	99	57	34,13	cyclictest	10078	-21	apt-get	07:30:15	0
2490	99	57	30,16	cyclictest	0	-21	swapper/0	09:35:12	0
2490	99	57	28,15	cyclictest	0	-21	swapper/0	07:45:13	0
2490	99	57	20,31	cyclictest	8065	-21	sshd	08:55:18	0
2495	99	56	6,12	cyclictest	26	-21	ksoftirqd/1	08:25:55	1
2495	99	56	32,18	cyclictest	26	-21	ksoftirqd/1	07:30:32	1
2495	99	56	26,17	cyclictest	26	-21	ksoftirqd/1	10:59:24	1
2495	99	56	18,21	cyclictest	26	-21	ksoftirqd/1	11:41:10	1
2495	99	56	18,20	cyclictest	26	-21	ksoftirqd/1	11:46:36	1
2495	99	56	10,33	cyclictest	0	-21	swapper/1	09:18:49	1
2490	99	56	40,10	cyclictest	14	-21	ksoftirqd/0	08:40:01	0
2490	99	56	31,15	cyclictest	0	-21	swapper/0	11:50:14	0
2490	99	56	21,30	cyclictest	457	-21	dbus-daemon	10:30:00	0
2495	99	55	19,15	cyclictest	26	-21	ksoftirqd/1	07:39:25	1
2495	99	55	11,40	cyclictest	0	-21	swapper/1	07:15:05	1
2495	99	55	11,32	cyclictest	0	-21	swapper/1	07:57:02	1
2495	99	55	10,33	cyclictest	0	-21	swapper/1	09:47:02	1
2495	99	55	10,33	cyclictest	0	-21	swapper/1	08:46:08	1
2495	99	55	10,33	cyclictest	0	-21	swapper/1	08:15:51	1
2495	99	55	10,32	cyclictest	0	-21	swapper/1	12:04:03	1
2490	99	55	30,15	cyclictest	0	-21	swapper/0	08:10:13	0
2490	99	55	27,16	cyclictest	0	-21	swapper/0	08:30:13	0
2490	99	55	22,28	cyclictest	18674	-21	runrttasks	07:53:39	0
2490	99	55	2,19	cyclictest	27145	-21	users	08:15:38	0
2490	99	55	21,29	cyclictest	20096	-21	idleruntime-cro	11:00:00	0
2495	99	54	9,40	cyclictest	0	-21	swapper/1	07:25:28	1
2495	99	54	8,13	cyclictest	26	-21	ksoftirqd/1	11:20:27	1
2495	99	54	3,46	cyclictest	14450	-21	ntpq	07:40:35	1
2495	99	54	3,46	cyclictest	14450	-21	ntpq	07:40:35	1
2495	99	54	25,16	cyclictest	26	-21	ksoftirqd/1	11:10:43	1
2495	99	54	19,14	cyclictest	26	-21	ksoftirqd/1	11:00:41	1
2495	99	54	17,19	cyclictest	26	-21	ksoftirqd/1	11:28:52	1
2495	99	54	17,19	cyclictest	26	-21	ksoftirqd/1	09:26:40	1
2490	99	54	27,14	cyclictest	9201	-21	apt-get	12:00:15	0
2495	99	53	9,14	cyclictest	26	-21	ksoftirqd/1	09:10:03	1
2495	99	53	9,10	cyclictest	0	-21	swapper/1	08:10:37	1
2495	99	53	30,17	cyclictest	26	-21	ksoftirqd/1	09:35:30	1
2495	99	53	17,14	cyclictest	26	-21	ksoftirqd/1	12:34:16	1
2495	99	53	16,21	cyclictest	26	-21	ksoftirqd/1	10:33:07	1
2495	99	53	16,14	cyclictest	26	-21	ksoftirqd/1	09:03:43	1
2495	99	53	13,35	cyclictest	26	-21	ksoftirqd/1	10:25:20	1
2495	99	53	11,37	cyclictest	0	-21	swapper/1	11:39:00	1
2495	99	52	6,41	cyclictest	0	-21	swapper/1	12:05:15	1
2495	99	52	4,9	cyclictest	0	-21	swapper/1	12:11:06	1
2495	99	52	4,43	cyclictest	0	-21	swapper/1	10:23:08	1
2495	99	52	36,11	cyclictest	0	-21	swapper/1	12:20:32	1
2495	99	52	31,16	cyclictest	26	-21	ksoftirqd/1	08:34:09	1
2495	99	52	27,19	cyclictest	26	-21	ksoftirqd/1	07:46:39	1
2495	99	52	16,15	cyclictest	26	-21	ksoftirqd/1	10:49:48	1
2490	99	52	3,44	cyclictest	26462	-21	timerwakeupswit	09:45:36	0
2490	99	52	3,43	cyclictest	2220	-21	apt-get	11:40:15	0
2495	99	51	21,26	cyclictest	0	-21	swapper/1	10:52:04	1
2495	99	50	12,34	cyclictest	0	-21	swapper/1	11:30:16	1
2495	99	49	7,13	cyclictest	26	-21	ksoftirqd/1	09:30:17	1
2495	99	49	7,13	cyclictest	26	-21	ksoftirqd/1	09:20:27	1
2495	99	49	6,13	cyclictest	26	-21	ksoftirqd/1	11:15:27	1
2495	99	48	6,13	cyclictest	26	-21	ksoftirqd/1	08:35:30	1
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17