OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #3, slot #6

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the highest latencies:

System rack3slot6.osadl.org (updated Tue Sep 15, 2026 00:45:02)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
2094	2	146	0,4	sleep1	25	1	rcuc/1	20:25:02	1
7318	99	94	15,64	cyclictest	0	-21	swapper/0	21:35:23	0
7318	99	94	15,64	cyclictest	0	-21	swapper/0	21:35:23	0
7322	99	92	23,46	cyclictest	4697	-21	grep	23:30:37	1
7318	99	92	17,59	cyclictest	0	-21	swapper/0	20:40:14	0
7322	99	90	2,66	cyclictest	23759	-21	runrttasks	21:25:09	1
7318	99	89	20,60	cyclictest	11534	-21	aten2.4_r3power	19:20:16	0
7318	99	89	16,62	cyclictest	0	-21	swapper/0	22:50:12	0
7322	99	88	5,54	cyclictest	0	-21	swapper/1	00:10:01	1
7318	99	88	6,17	cyclictest	25509	-21	apt-get	21:30:14	0
7318	99	88	14,62	cyclictest	0	-21	swapper/0	00:05:15	0
7322	99	87	5,54	cyclictest	25081	-21	kworker/u8:0	21:30:38	1
7318	99	86	17,18	cyclictest	32746	-21	aten2.4-expect	23:20:21	0
7318	99	86	15,58	cyclictest	0	-21	swapper/0	23:30:12	0
7318	99	86	13,57	cyclictest	0	-21	swapper/0	19:35:11	0
7318	99	86	12,60	cyclictest	0	-21	swapper/0	20:45:12	0
7322	99	85	2,58	cyclictest	1757	-21	latency_hist	23:25:01	1
7322	99	85	17,45	cyclictest	503	-21	snmpd	21:06:17	1
7318	99	85	21,58	cyclictest	243	-21	systemd-journal	20:51:15	0
7318	99	85	15,54	cyclictest	0	-21	swapper/0	22:40:12	0
7322	99	84	9,46	cyclictest	0	-21	swapper/1	22:55:32	1
7318	99	84	6,21	cyclictest	23767	-21	apt-get	21:25:13	0
7318	99	84	3,24	cyclictest	21836	-21	munin-plugin-st	21:20:00	0
7322	99	82	7,46	cyclictest	0	-21	swapper/1	19:35:20	1
7322	99	82	7,45	cyclictest	0	-21	swapper/1	23:35:20	1
7322	99	82	6,47	cyclictest	0	-21	swapper/1	21:35:34	1
7322	99	82	6,47	cyclictest	0	-21	swapper/1	21:35:34	1
7322	99	82	5,70	cyclictest	0	-21	swapper/1	20:30:22	1
7318	99	82	8,31	cyclictest	31164	-21	cat	23:15:19	0
7318	99	82	6,23	cyclictest	14507	-21	apt-get	00:00:16	0
7318	99	82	15,51	cyclictest	0	-21	swapper/0	19:55:14	0
7322	99	81	9,43	cyclictest	0	-21	swapper/1	20:00:26	1
7322	99	81	9,43	cyclictest	0	-21	swapper/1	20:00:26	1
7322	99	81	6,46	cyclictest	0	-21	swapper/1	23:55:36	1
7322	99	81	6,46	cyclictest	0	-21	swapper/1	19:15:30	1
7322	99	81	2,56	cyclictest	562	-21	runrttasks	23:52:05	1
7318	99	81	7,23	cyclictest	9657	-21	apt-get	19:15:14	0
7318	99	81	16,55	cyclictest	0	-21	swapper/0	20:15:12	0
7318	99	81	11,57	cyclictest	0	-21	swapper/0	20:55:22	0
7322	99	80	5,46	cyclictest	0	-21	swapper/1	00:15:19	1
7322	99	80	5,46	cyclictest	0	-21	swapper/1	00:15:19	1
7322	99	80	2,51	cyclictest	72	-1	kworker/1:1H	21:15:33	1
7322	99	80	22,52	cyclictest	30218	-21	cat	21:40:36	1
7318	99	80	7,18	cyclictest	8044	-21	apt-get	22:10:14	0
7318	99	80	26,20	cyclictest	503	-21	snmpd	22:35:27	0
7318	99	80	17,53	cyclictest	7309	-21	cyclictest	21:00:17	0
7322	99	79	8,43	cyclictest	0	-21	swapper/1	19:50:30	1
7322	99	79	6,44	cyclictest	0	-21	swapper/1	00:25:18	1
7322	99	79	5,45	cyclictest	0	-21	swapper/1	22:35:20	1
7322	99	79	5,45	cyclictest	0	-21	swapper/1	21:45:23	1
7322	99	79	5,45	cyclictest	0	-21	swapper/1	19:20:37	1
7322	99	79	2,55	cyclictest	21489	-21	runrttasks	22:45:52	1
7322	99	79	20,52	cyclictest	30347	-21	unixbench_multi	20:10:37	1
7318	99	79	7,25	cyclictest	20313	-21	apt-get	22:45:12	0
7318	99	79	7,23	cyclictest	6252	-21	apt-get	22:05:16	0
7318	99	79	7,19	cyclictest	25011	-21	apt-get	00:30:15	0
7318	99	79	6,19	cyclictest	20239	-21	apt-get	19:45:12	0
7318	99	79	6,18	cyclictest	705	-21	apt-get	21:50:13	0
7318	99	79	6,18	cyclictest	19769	-21	apt-get	00:15:13	0
7318	99	79	6,18	cyclictest	19769	-21	apt-get	00:15:13	0
7318	99	79	36,15	cyclictest	14859	-21	munin-run	22:30:00	0
7318	99	79	25,21	cyclictest	457	-21	dbus-daemon	00:00:00	0
7318	99	79	25,18	cyclictest	7934	-21	sort	19:10:24	0
7318	99	79	21,21	cyclictest	243	-21	systemd-journal	22:16:31	0
7318	99	79	21,18	cyclictest	13497	-21	df_inode	19:25:20	0
7322	99	78	5,44	cyclictest	0	-21	swapper/1	00:10:25	1
7322	99	78	2,53	cyclictest	562	-21	runrttasks	21:50:45	1
7322	99	78	2,53	cyclictest	15906	-21	runrttasks	21:01:14	1
7322	99	78	2,51	cyclictest	1881	-21	runrttasks	23:25:05	1
7318	99	78	7,22	cyclictest	31396	-21	apt-get	21:45:12	0
7318	99	78	7,18	cyclictest	27514	-21	apt-get	20:05:24	0
7318	99	78	7,18	cyclictest	243	-21	systemd-journal	21:41:25	0
7318	99	78	6,19	cyclictest	3865	-21	apt-get	20:30:15	0
7318	99	78	5,67	cyclictest	0	-21	swapper/0	19:30:27	0
7318	99	78	18,19	cyclictest	243	-21	systemd-journal	21:06:17	0
6770	2	78	19,47	sleep1	0	-21	swapper/1	19:06:06	1
7322	99	77	3,50	cyclictest	16104	-21	runrttasks	19:31:12	1
7322	99	77	2,49	cyclictest	22739	-21	runrttasks	00:21:12	1
7318	99	77	7,17	cyclictest	29075	-21	apt-get	23:10:13	0
7318	99	77	22,13	cyclictest	2628	-21	latency	20:25:24	0
7318	99	77	19,23	cyclictest	457	-21	dbus-daemon	20:40:00	0
7318	99	77	18,19	cyclictest	28379	-21	grep	23:05:36	0
7318	99	77	18,19	cyclictest	28379	-21	grep	23:05:36	0
7318	99	77	11,54	cyclictest	18055	-21	diskmemload	21:10:25	0
7322	99	76	2,52	cyclictest	562	-21	runrttasks	22:42:20	1
7318	99	76	6,18	cyclictest	2751	-21	apt-get	21:55:23	0
7318	99	76	3,38	cyclictest	26200	-21	strings	23:00:27	0
7318	99	76	22,18	cyclictest	22922	-21	sort	21:20:34	0
7318	99	76	21,20	cyclictest	27034	-21	cron	20:05:00	0
7318	99	76	21,20	cyclictest	27034	-21	cron	20:04:59	0
5582	2	76	28,36	sleep0	0	-21	swapper/0	19:05:15	0
7322	99	75	7,59	cyclictest	0	-21	swapper/1	19:45:12	1
7322	99	75	5,65	cyclictest	0	-21	swapper/1	20:39:55	1
7322	99	75	24,45	cyclictest	26006	-21	cat	00:30:37	1
7322	99	75	22,46	cyclictest	27423	-21	netstat	00:35:28	1
7322	99	75	11,54	cyclictest	0	-21	swapper/1	22:20:14	1
7318	99	75	7,18	cyclictest	29307	-21	apt-get	20:10:13	0
7318	99	75	6,34	cyclictest	0	-21	swapper/0	23:40:25	0
7318	99	75	3,44	cyclictest	243	-21	systemd-journal	23:51:50	0
7322	99	74	3,48	cyclictest	19803	-21	runrttasks	19:42:15	1
7322	99	74	23,44	cyclictest	7539	-21	entropy	23:40:21	1
7322	99	74	22,45	cyclictest	22417	-21	if_enp2s0	21:20:22	1
7318	99	74	7,22	cyclictest	5376	-21	apt-get	23:35:13	0
7318	99	74	7,22	cyclictest	18033	-21	apt-get	00:10:13	0
7318	99	74	7,21	cyclictest	4515	-21	apt-get	22:00:14	0
7318	99	74	7,20	cyclictest	342	-21	apt-get	20:20:14	0
7318	99	74	5,35	cyclictest	0	-21	swapper/0	00:20:37	0
7318	99	74	3,22	cyclictest	28417	-21	latency_hist	00:40:01	0
7322	99	73	6,57	cyclictest	7425	-21	apt-get	20:40:14	1
7322	99	73	21,46	cyclictest	15476	-21	if_err_enp3s0	22:30:25	1
7318	99	73	7,19	cyclictest	18504	-21	apt-get	19:40:15	0
7318	99	73	5,62	cyclictest	0	-21	swapper/0	19:50:34	0
7318	99	73	5,34	cyclictest	0	-21	swapper/0	00:25:33	0
7322	99	72	5,58	cyclictest	9159	-21	apt-get	20:45:12	1
7322	99	72	5,38	cyclictest	0	-21	swapper/1	20:50:23	1
7322	99	72	22,44	cyclictest	31389	-21	cat	23:15:23	1
7322	99	71	9,55	cyclictest	0	-21	swapper/1	19:25:13	1
7322	99	71	8,55	cyclictest	0	-21	swapper/1	22:05:14	1
7322	99	71	7,57	cyclictest	0	-21	swapper/1	23:00:13	1
7322	99	71	7,49	cyclictest	31043	-21	apt-get	20:15:12	1
7322	99	71	21,44	cyclictest	13854	-21	cat	22:25:25	1
7318	99	71	5,22	cyclictest	1895	-21	apt-get	23:25:15	0
7322	99	70	8,56	cyclictest	0	-21	swapper/1	22:15:11	1
7322	99	70	7,56	cyclictest	0	-21	swapper/1	19:10:11	1
7322	99	70	6,53	cyclictest	22053	-21	apt-get	22:50:12	1
7322	99	70	2,63	cyclictest	8041	-21	/usr/sbin/munin	22:10:21	1
7322	99	70	11,53	cyclictest	0	-21	swapper/1	19:55:14	1
7318	99	70	6,18	cyclictest	23793	-21	apt-get	22:55:14	0
7318	99	70	19,20	cyclictest	10517	-21	wc	23:50:01	0
7318	99	70	18,18	cyclictest	15935	-21	cat	22:30:36	0
7322	99	69	9,55	cyclictest	0	-21	swapper/1	21:10:42	1
7322	99	69	5,57	cyclictest	27514	-21	apt-get	20:05:16	1
7322	99	69	4,60	cyclictest	0	-21	swapper/1	00:00:16	1
7322	99	69	2,61	cyclictest	29072	-21	/usr/sbin/munin	23:10:15	1
7322	99	69	11,50	cyclictest	0	-21	swapper/1	21:55:26	1
7318	99	69	6,33	cyclictest	0	-21	swapper/0	22:20:28	0
7322	99	68	9,52	cyclictest	0	-21	swapper/1	20:55:23	1
7322	99	68	17,44	cyclictest	5868	-21	seq	22:02:48	1
7322	99	68	13,48	cyclictest	0	-21	swapper/1	23:05:11	1
7322	99	68	13,48	cyclictest	0	-21	swapper/1	23:05:11	1
7322	99	67	19,43	cyclictest	9105	-21	cpu	23:45:18	1
7322	99	67	18,44	cyclictest	2531	-21	if_enp3s0	20:25:22	1
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17