OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #3, slot #6

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 792 highest latencies:

System rack3slot6.osadl.org (updated Thu Sep 03, 2026 00:44:57)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
26896	99	94	13,74	cyclictest	0	-21	swapper/0	21:00:14	0
26405	2	94	20,28	sleep0	0	-21	swapper/0	19:06:43	0
26896	99	90	13,69	cyclictest	0	-21	swapper/0	21:25:15	0
26377	2	90	29,48	sleep1	0	-21	swapper/1	19:06:26	1
26896	99	89	7,67	cyclictest	349	-21	apt-get	22:25:13	0
26896	99	89	18,59	cyclictest	0	-21	swapper/0	20:15:13	0
26896	99	89	13,60	cyclictest	0	-21	swapper/0	21:45:13	0
26896	99	88	13,61	cyclictest	0	-21	swapper/0	00:30:14	0
26896	99	87	14,66	cyclictest	0	-21	swapper/0	22:35:15	0
26896	99	87	14,58	cyclictest	0	-21	swapper/0	21:15:15	0
26896	99	87	13,63	cyclictest	0	-21	swapper/0	21:55:12	0
26896	99	86	4,28	cyclictest	457	-21	dbus-daemon	19:20:00	0
26896	99	86	16,62	cyclictest	0	-21	swapper/0	22:50:12	0
26896	99	86	15,57	cyclictest	0	-21	swapper/0	20:05:17	0
26896	99	86	14,58	cyclictest	0	-21	swapper/0	20:10:10	0
26896	99	86	14,58	cyclictest	0	-21	swapper/0	19:40:13	0
26896	99	86	14,56	cyclictest	0	-21	swapper/0	23:20:15	0
1195	2	86	0,7	chrt	1106	-40	mandb	00:00:03	1
1195	2	86	0,7	chrt	1106	-40	mandb	00:00:02	1
26896	99	85	7,67	cyclictest	4087	-21	apt-get	19:35:24	0
26896	99	85	7,66	cyclictest	31940	-21	apt-get	23:55:13	0
26896	99	85	7,66	cyclictest	31940	-21	apt-get	23:55:13	0
26896	99	85	6,63	cyclictest	25474	-21	apt-get	22:05:23	0
26896	99	85	64,9	cyclictest	14	-21	ksoftirqd/0	20:20:13	0
26896	99	85	5,65	cyclictest	29310	-21	apt-get	22:15:22	0
26896	99	85	31,40	cyclictest	0	-21	swapper/0	19:10:15	0
26896	99	85	14,59	cyclictest	0	-21	swapper/0	23:50:12	0
26896	99	84	8,62	cyclictest	13853	-21	apt-get	00:35:13	0
26896	99	84	7,62	cyclictest	27242	-21	apt-get	20:40:13	0
26896	99	84	5,67	cyclictest	24910	-21	apt-get	23:35:15	0
26896	99	84	17,58	cyclictest	0	-21	swapper/0	23:15:10	0
26896	99	84	15,57	cyclictest	0	-21	swapper/0	22:55:13	0
26896	99	84	13,57	cyclictest	0	-21	swapper/0	20:25:11	0
26896	99	83	6,65	cyclictest	10375	-21	apt-get	00:25:13	0
26896	99	83	3,41	cyclictest	19991	-21	taskset	21:50:26	0
26896	99	83	15,60	cyclictest	0	-21	swapper/0	23:40:14	0
26896	99	83	14,55	cyclictest	0	-21	swapper/0	21:35:14	0
26896	99	83	12,60	cyclictest	0	-21	swapper/0	19:20:13	0
26896	99	83	12,57	cyclictest	0	-21	swapper/0	00:05:17	0
26896	99	82	6,61	cyclictest	12659	-21	apt-get	23:00:12	0
26896	99	82	14,59	cyclictest	0	-21	swapper/0	00:20:24	0
26896	99	82	14,54	cyclictest	0	-21	swapper/0	21:40:25	0
26896	99	82	13,57	cyclictest	0	-21	swapper/0	22:00:24	0
26896	99	82	13,54	cyclictest	0	-21	swapper/0	21:30:11	0
26896	99	82	12,57	cyclictest	0	-21	swapper/0	00:15:14	0
26896	99	81	7,63	cyclictest	28979	-21	apt-get	20:45:13	0
26896	99	81	7,61	cyclictest	32455	-21	apt-get	20:55:10	0
26896	99	81	6,64	cyclictest	27278	-21	apt-get	22:10:16	0
26896	99	81	26,47	cyclictest	15397	-21	sort	23:05:37	0
26896	99	81	20,50	cyclictest	0	-21	swapper/0	22:30:11	0
26896	99	80	6,61	cyclictest	2047	-21	apt-get	19:30:12	0
26896	99	80	14,52	cyclictest	0	-21	swapper/0	21:20:14	0
26901	99	79	9,38	cyclictest	0	-21	swapper/1	21:20:31	1
26896	99	79	8,61	cyclictest	23170	-21	apt-get	23:30:12	0
26896	99	79	6,67	cyclictest	0	-21	swapper/0	20:35:00	0
26896	99	79	6,63	cyclictest	31075	-21	apt-get	22:20:12	0
26896	99	79	6,57	cyclictest	28463	-21	apt-get	23:45:13	0
26896	99	79	4,22	cyclictest	243	-21	systemd-journal	00:01:19	0
26896	99	79	12,53	cyclictest	0	-21	swapper/0	21:05:15	0
26896	99	78	7,56	cyclictest	5322	-21	apt-get	21:10:13	0
26896	99	78	6,57	cyclictest	32734	-21	apt-get	19:25:14	0
26896	99	78	6,55	cyclictest	5683	-21	apt-get	22:40:14	0
26896	99	78	6,55	cyclictest	11120	-21	apt-get	19:55:14	0
26896	99	78	5,58	cyclictest	9380	-21	apt-get	19:50:12	0
26896	99	78	13,51	cyclictest	0	-21	swapper/0	00:10:12	0
26896	99	78	12,54	cyclictest	0	-21	swapper/0	19:45:15	0
26901	99	77	10,42	cyclictest	0	-21	swapper/1	21:40:17	1
26896	99	77	4,63	cyclictest	25425	-21	apt-get	20:35:12	0
26896	99	76	6,56	cyclictest	30717	-21	apt-get	20:50:13	0
26896	99	76	6,55	cyclictest	21431	-21	apt-get	23:25:15	0
26896	99	76	6,55	cyclictest	12876	-21	apt-get	20:00:12	0
26896	99	76	6,54	cyclictest	16197	-21	apt-get	23:10:18	0
26896	99	76	14,50	cyclictest	0	-21	swapper/0	22:45:12	0
27916	2	75	0,6	sleep1	25	1	rcuc/1	19:10:35	1
26901	99	75	31,29	cyclictest	26	-21	ksoftirqd/1	21:00:12	1
26901	99	73	31,29	cyclictest	26	-21	ksoftirqd/1	00:05:16	1
26901	99	71	7,38	cyclictest	0	-21	swapper/1	23:45:22	1
26901	99	71	41,19	cyclictest	17900	-21	apt-get	21:45:13	1
26901	99	71	39,15	cyclictest	26719	-21	apt-get	23:40:14	1
26901	99	70	42,13	cyclictest	17926	-21	apt-get	23:15:10	1
26901	99	70	39,17	cyclictest	21376	-21	apt-get	21:55:12	1
26901	99	69	39,20	cyclictest	12112	-21	apt-get	00:30:14	1
26901	99	69	31,27	cyclictest	26	-21	ksoftirqd/1	20:45:12	1
26901	99	68	40,15	cyclictest	21943	-21	apt-get	20:25:10	1
26901	99	68	39,17	cyclictest	6572	-21	apt-get	00:15:14	1
26901	99	68	39,14	cyclictest	30995	-21	apt-get	19:20:13	1
26901	99	68	38,17	cyclictest	16706	-21	apt-get	20:10:09	1
26901	99	68	36,19	cyclictest	26	-21	ksoftirqd/1	21:25:14	1
26901	99	68	32,30	cyclictest	562	-21	runrttasks	19:35:23	1
26901	99	67	40,17	cyclictest	5900	-21	apt-get	19:40:13	1
26901	99	67	40,17	cyclictest	3869	-21	apt-get	22:35:13	1
26901	99	67	37,17	cyclictest	26	-21	ksoftirqd/1	22:50:13	1
26901	99	66	40,16	cyclictest	14047	-21	apt-get	21:35:14	1
26901	99	66	40,14	cyclictest	23438	-21	apt-get	22:00:24	1
26901	99	66	39,17	cyclictest	7063	-21	apt-get	21:15:15	1
26901	99	66	37,18	cyclictest	19693	-21	apt-get	23:20:15	1
26901	99	66	35,16	cyclictest	2128	-21	apt-get	22:30:13	1
26901	99	65	36,14	cyclictest	20206	-21	apt-get	20:20:12	1
26901	99	65	33,17	cyclictest	8630	-21	apt-get	00:20:24	1
26901	99	65	29,25	cyclictest	30203	-21	apt-get	23:50:12	1
26901	99	64	40,14	cyclictest	14926	-21	apt-get	20:05:17	1
26901	99	64	38,15	cyclictest	7646	-21	apt-get	19:45:15	1
26901	99	64	30,29	cyclictest	0	-21	swapper/1	19:25:13	1
26901	99	63	35,16	cyclictest	27242	-21	apt-get	20:40:15	1
26901	99	63	33,16	cyclictest	29310	-21	apt-get	22:15:24	1
26901	99	63	27,25	cyclictest	26	-21	ksoftirqd/1	19:15:11	1
26901	99	62	37,14	cyclictest	10901	-21	apt-get	22:55:13	1
26901	99	62	32,20	cyclictest	12304	-21	apt-get	21:30:11	1
26901	99	62	31,21	cyclictest	0	-21	swapper/1	22:25:13	1
26901	99	62	31,15	cyclictest	23684	-21	apt-get	20:30:13	1
26901	99	62	30,27	cyclictest	0	-21	swapper/1	23:05:11	1
26901	99	62	30,27	cyclictest	0	-21	swapper/1	19:30:12	1
26901	99	62	29,27	cyclictest	0	-21	swapper/1	22:45:13	1
26901	99	62	24,26	cyclictest	26	-21	ksoftirqd/1	22:10:16	1
5561	2	61	0,6	chrt	0	-21	swapper/1	00:10:29	1
26901	99	61	34,17	cyclictest	3529	-21	apt-get	21:05:15	1
26901	99	61	33,17	cyclictest	1305	-21	apt-get	00:00:12	1
26901	99	61	31,16	cyclictest	18445	-21	apt-get	20:15:14	1
26901	99	61	30,25	cyclictest	0	-21	swapper/1	21:50:18	1
26901	99	61	28,27	cyclictest	0	-21	swapper/1	23:30:14	1
26901	99	60	33,16	cyclictest	13853	-21	apt-get	00:35:15	1
26901	99	60	30,24	cyclictest	0	-21	swapper/1	22:40:14	1
26901	99	60	29,25	cyclictest	0	-21	swapper/1	23:00:12	1
26901	99	59	32,13	cyclictest	16197	-21	apt-get	23:10:20	1
26901	99	59	30,24	cyclictest	0	-21	swapper/1	19:50:13	1
26901	99	59	12,41	cyclictest	0	-21	swapper/1	22:05:47	1
26901	99	58	27,25	cyclictest	0	-21	swapper/1	00:25:13	1
26901	99	58	27,24	cyclictest	0	-21	swapper/1	19:55:14	1
26901	99	58	25,18	cyclictest	0	-21	swapper/1	22:20:12	1
26901	99	58	24,29	cyclictest	31568	-21	sshd	20:50:34	1
26901	99	58	17,23	cyclictest	22170	-21	open_files	23:25:32	1
26901	99	56	28,21	cyclictest	0	-21	swapper/1	20:55:11	1
26901	99	56	21,29	cyclictest	0	-21	swapper/1	23:35:15	1
26901	99	56	21,29	cyclictest	0	-21	swapper/1	20:00:11	1
26901	99	54	19,26	cyclictest	0	-21	swapper/1	21:10:12	1
26901	99	54	14,23	cyclictest	25406	-21	date	20:35:01	1
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17