OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #3, slot #6

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of all highest latencies:

System rack3slot6.osadl.org (updated Thu Jul 30, 2026 00:44:59)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
27631	99	117	88,20	cyclictest	26	-21	ksoftirqd/1	22:35:13	1
31545	2	115	0,6	chrt	31546	-21	sendmail	19:20:01	1
27631	99	112	82,19	cyclictest	26	-21	ksoftirqd/1	19:30:14	1
27631	99	110	78,20	cyclictest	26	-21	ksoftirqd/1	20:45:14	1
27626	99	110	76,19	cyclictest	14	-21	ksoftirqd/0	19:15:14	0
27631	99	106	28,63	cyclictest	0	-21	swapper/1	22:30:13	1
27626	99	104	65,30	cyclictest	14	-21	ksoftirqd/0	20:00:11	0
27626	99	104	12,79	cyclictest	0	-21	swapper/0	00:35:16	0
27626	99	102	13,77	cyclictest	0	-21	swapper/0	00:05:17	0
27631	99	99	57,31	cyclictest	26	-21	ksoftirqd/1	21:00:13	1
27626	99	99	59,29	cyclictest	14	-21	ksoftirqd/0	22:05:12	0
27626	99	99	52,34	cyclictest	0	-21	swapper/0	00:20:12	0
27626	99	98	57,30	cyclictest	14	-21	ksoftirqd/0	22:40:15	0
27626	99	96	12,74	cyclictest	0	-21	swapper/0	22:35:15	0
27626	99	96	12,72	cyclictest	0	-21	swapper/0	20:05:15	0
27626	99	96	10,74	cyclictest	0	-21	swapper/0	23:15:10	0
27418	2	96	24,27	sleep0	0	-21	swapper/0	19:09:22	0
27626	99	95	11,71	cyclictest	0	-21	swapper/0	19:10:11	0
27626	99	94	11,70	cyclictest	0	-21	swapper/0	21:35:10	0
27626	99	94	10,72	cyclictest	0	-21	swapper/0	23:35:11	0
27631	99	93	45,35	cyclictest	26	-21	ksoftirqd/1	23:40:10	1
27626	99	93	43,39	cyclictest	14	-21	ksoftirqd/0	20:20:11	0
27626	99	93	13,68	cyclictest	0	-21	swapper/0	21:25:14	0
27626	99	93	12,68	cyclictest	0	-21	swapper/0	23:00:16	0
27626	99	93	12,68	cyclictest	0	-21	swapper/0	22:55:17	0
27626	99	93	11,70	cyclictest	0	-21	swapper/0	19:55:13	0
27626	99	93	11,69	cyclictest	0	-21	swapper/0	23:25:14	0
27626	99	92	8,68	cyclictest	503	-21	snmpd	21:50:09	0
27626	99	92	13,64	cyclictest	0	-21	swapper/0	22:00:13	0
27626	99	92	12,69	cyclictest	0	-21	swapper/0	23:40:09	0
27626	99	92	12,68	cyclictest	0	-21	swapper/0	19:40:14	0
27626	99	92	11,69	cyclictest	0	-21	swapper/0	23:50:13	0
27631	99	91	42,32	cyclictest	26	-21	ksoftirqd/1	19:55:15	1
27631	99	91	11,66	cyclictest	0	-21	swapper/1	00:00:17	1
27631	99	91	10,67	cyclictest	0	-21	swapper/1	21:05:14	1
27626	99	91	50,28	cyclictest	0	-21	swapper/0	21:00:14	0
27626	99	91	15,63	cyclictest	0	-21	swapper/0	22:20:13	0
27626	99	91	12,67	cyclictest	0	-21	swapper/0	22:45:12	0
27626	99	91	12,64	cyclictest	5587	-21	diskmemload	21:10:16	0
27626	99	91	11,69	cyclictest	0	-21	swapper/0	00:10:13	0
27626	99	91	11,67	cyclictest	0	-21	swapper/0	22:15:15	0
27626	99	91	11,67	cyclictest	0	-21	swapper/0	22:15:14	0
27631	99	90	44,32	cyclictest	26	-21	ksoftirqd/1	22:50:13	1
27631	99	90	13,62	cyclictest	0	-21	swapper/1	19:20:15	1
27631	99	90	10,65	cyclictest	0	-21	swapper/1	19:50:16	1
27626	99	90	14,65	cyclictest	0	-21	swapper/0	19:20:12	0
27626	99	90	12,68	cyclictest	0	-21	swapper/0	22:25:13	0
27626	99	90	10,25	cyclictest	0	-21	swapper/0	20:45:15	0
27626	99	90	10,23	cyclictest	0	-21	swapper/0	19:45:11	0
27631	99	89	37,34	cyclictest	26	-21	ksoftirqd/1	22:00:14	1
27631	99	89	11,61	cyclictest	0	-21	swapper/1	23:35:15	1
27626	99	89	13,65	cyclictest	0	-21	swapper/0	23:05:15	0
27626	99	89	13,64	cyclictest	0	-21	swapper/0	21:15:11	0
27626	99	89	12,65	cyclictest	0	-21	swapper/0	23:55:16	0
27626	99	89	11,66	cyclictest	0	-21	swapper/0	20:25:13	0
27626	99	89	11,66	cyclictest	0	-21	swapper/0	20:25:13	0
27626	99	89	10,67	cyclictest	0	-21	swapper/0	23:45:11	0
27626	99	89	10,66	cyclictest	0	-21	swapper/0	21:45:12	0
27626	99	89	10,24	cyclictest	0	-21	swapper/0	23:30:11	0
27626	99	88	36,44	cyclictest	0	-21	swapper/0	19:50:13	0
27626	99	88	15,61	cyclictest	0	-21	swapper/0	20:35:16	0
27626	99	88	10,65	cyclictest	0	-21	swapper/0	19:25:24	0
27631	99	87	6,62	cyclictest	5201	-21	runrttasks	22:40:14	1
27631	99	87	13,61	cyclictest	0	-21	swapper/1	21:15:13	1
27631	99	87	11,61	cyclictest	0	-21	swapper/1	00:30:16	1
27626	99	87	12,65	cyclictest	0	-21	swapper/0	19:35:15	0
27631	99	86	35,39	cyclictest	26	-21	ksoftirqd/1	22:10:15	1
27631	99	86	11,61	cyclictest	0	-21	swapper/1	23:30:12	1
27631	99	86	11,60	cyclictest	0	-21	swapper/1	00:25:12	1
27631	99	86	11,58	cyclictest	0	-21	swapper/1	00:05:16	1
27626	99	86	9,24	cyclictest	27671	-21	apt-get	20:40:14	0
27631	99	85	34,33	cyclictest	26	-21	ksoftirqd/1	20:15:15	1
27631	99	85	12,61	cyclictest	0	-21	swapper/1	20:50:15	1
27631	99	85	10,57	cyclictest	0	-21	swapper/1	19:45:12	1
27626	99	85	10,21	cyclictest	0	-21	swapper/0	00:15:14	0
27631	99	84	46,33	cyclictest	26	-21	ksoftirqd/1	23:55:19	1
27631	99	84	33,36	cyclictest	0	-21	swapper/1	22:15:12	1
27631	99	84	33,36	cyclictest	0	-21	swapper/1	22:15:12	1
27631	99	84	13,54	cyclictest	0	-21	swapper/1	20:35:14	1
27631	99	84	11,55	cyclictest	0	-21	swapper/1	23:25:11	1
27631	99	83	8,56	cyclictest	13922	-21	taskset	23:05:15	1
27631	99	83	11,58	cyclictest	0	-21	swapper/1	20:55:11	1
27626	99	83	6,21	cyclictest	15704	-21	apt-get	23:10:14	0
27631	99	82	7,55	cyclictest	0	-21	swapper/1	23:10:12	1
27631	99	82	35,33	cyclictest	0	-21	swapper/1	20:20:13	1
27631	99	82	34,36	cyclictest	0	-21	swapper/1	23:00:15	1
27626	99	82	6,21	cyclictest	16580	-21	apt-get	21:40:12	0
27626	99	82	20,20	cyclictest	457	-21	dbus-daemon	21:35:00	0
27631	99	81	37,38	cyclictest	26	-21	ksoftirqd/1	19:25:42	1
27631	99	81	32,34	cyclictest	0	-21	swapper/1	23:20:24	1
27626	99	81	22,17	cyclictest	8839	-21	aten2.4_r3power	22:50:17	0
27631	99	80	6,56	cyclictest	0	-21	swapper/1	20:25:12	1
27631	99	80	6,56	cyclictest	0	-21	swapper/1	20:25:12	1
27631	99	80	26,35	cyclictest	0	-21	swapper/1	22:25:14	1
27626	99	80	9,21	cyclictest	31141	-21	apt-get	20:50:12	0
27631	99	79	6,55	cyclictest	0	-21	swapper/1	21:55:11	1
27631	99	79	28,32	cyclictest	801	-21	gpm	21:30:25	1
27631	99	79	12,50	cyclictest	0	-21	swapper/1	22:05:11	1
27631	99	78	6,52	cyclictest	0	-21	swapper/1	20:10:15	1
27631	99	78	31,32	cyclictest	0	-21	swapper/1	20:05:13	1
27631	99	78	25,33	cyclictest	26	-21	ksoftirqd/1	20:30:14	1
27631	99	77	34,37	cyclictest	26	-21	ksoftirqd/1	21:35:16	1
27626	99	77	8,47	cyclictest	36	-21	kcompactd0	23:20:26	0
27626	99	77	8,23	cyclictest	18870	-21	apt-get	20:15:14	0
27631	99	76	30,33	cyclictest	0	-21	swapper/1	21:40:12	1
27631	99	76	30,33	cyclictest	0	-21	swapper/1	21:25:13	1
27626	99	76	8,23	cyclictest	9245	-21	apt-get	21:20:13	0
27626	99	76	3,18	cyclictest	22396	-21	missed_timers	21:55:25	0
27631	99	75	3,51	cyclictest	0	-21	swapper/1	21:45:14	1
27626	99	75	8,22	cyclictest	1133	-21	apt-get	00:00:16	0
27626	99	75	5,35	cyclictest	0	-21	swapper/0	19:30:27	0
27631	99	74	20,37	cyclictest	26	-21	ksoftirqd/1	21:50:11	1
27626	99	74	8,36	cyclictest	0	-21	swapper/0	00:30:25	0
27631	99	73	22,33	cyclictest	26	-21	ksoftirqd/1	00:20:13	1
27626	99	73	5,34	cyclictest	0	-21	swapper/0	20:10:06	0
27626	99	73	3,42	cyclictest	2772	-21	sed	22:30:36	0
27626	99	72	2,19	cyclictest	794	-21	/usr/sbin/munin	20:55:21	0
27626	99	72	17,14	cyclictest	14	-21	ksoftirqd/0	20:30:20	0
27631	99	71	2,48	cyclictest	0	-21	swapper/1	20:00:12	1
27626	99	70	8,19	cyclictest	27082	-21	apt-get	22:10:13	0
27626	99	70	28,10	cyclictest	0	-21	swapper/0	21:05:49	0
27439	2	70	33,26	sleep1	0	-21	swapper/1	19:09:35	1
27626	99	69	7,18	cyclictest	9895	-21	apt-get	00:25:12	0
27631	99	67	23,30	cyclictest	27615	-21	cyclictest	19:35:14	1
27631	99	67	12,34	cyclictest	0	-21	swapper/1	00:15:16	1
27631	99	66	15,31	cyclictest	0	-21	swapper/1	00:35:13	1
27631	99	66	14,31	cyclictest	0	-21	swapper/1	22:55:17	1
27631	99	66	13,32	cyclictest	0	-21	swapper/1	19:40:11	1
27631	99	65	8,34	cyclictest	885	-21	apache2	19:10:14	1
27631	99	65	14,33	cyclictest	0	-21	swapper/1	20:40:14	1
27631	99	65	14,30	cyclictest	0	-21	swapper/1	21:10:15	1
27631	99	65	14,30	cyclictest	0	-21	swapper/1	00:10:15	1
27631	99	65	12,35	cyclictest	0	-21	swapper/1	21:20:12	1
27631	99	64	14,30	cyclictest	0	-21	swapper/1	22:45:14	1
27631	99	64	13,31	cyclictest	0	-21	swapper/1	23:15:12	1
27631	99	63	13,31	cyclictest	0	-21	swapper/1	23:50:13	1
27631	99	62	13,30	cyclictest	0	-21	swapper/1	22:20:14	1
27631	99	62	12,32	cyclictest	0	-21	swapper/1	23:45:12	1
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17