OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #3, slot #1

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance

Characteristics of the highest latencies:

System rack3slot1.osadl.org (updated Tue Jul 21, 2026 12:43:26)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
26526	99	127	122	cyclictest	0	-21	swapper/3	06:43:44	3
26526	99	107	103	cyclictest	0	-21	swapper/3	08:03:51	3
26526	99	107	103	cyclictest	0	-21	swapper/3	06:09:33	3
26524	99	106	102	cyclictest	0	-21	swapper/1	09:05:26	1
26526	99	103	102	cyclictest	0	-21	swapper/3	04:00:51	3
26524	99	103	92	cyclictest	0	-21	swapper/1	06:25:32	1
26524	99	103	102	cyclictest	26314	-21	tune2fs	08:53:37	1
26524	99	103	102	cyclictest	0	-21	swapper/1	08:17:52	1
26524	99	103	102	cyclictest	0	-21	swapper/1	04:00:51	1
26524	99	93	92	cyclictest	0	-21	swapper/1	08:39:07	1
26524	99	93	92	cyclictest	0	-21	swapper/1	08:30:40	1
26524	99	93	92	cyclictest	0	-21	swapper/1	04:06:58	1
26526	99	88	82	cyclictest	631	-21	timerandwakeup	06:38:52	3
26526	99	88	72	cyclictest	0	-21	swapper/3	08:48:42	3
26526	99	88	72	cyclictest	0	-21	swapper/3	08:38:37	3
26526	99	88	72	cyclictest	0	-21	swapper/3	05:53:52	3
26524	99	88	77	cyclictest	0	-21	swapper/1	08:59:03	1
26526	99	87	83	cyclictest	0	-21	swapper/3	08:15:08	3
26526	99	87	82	cyclictest	20600	-21	md2_resync	09:08:58	3
26526	99	87	82	cyclictest	20600	-21	md2_resync	09:08:58	3
26526	99	87	82	cyclictest	0	-21	swapper/3	09:03:42	3
26526	99	87	82	cyclictest	0	-21	swapper/3	04:18:50	3
26526	99	87	82	cyclictest	0	-21	swapper/3	04:13:51	3
26526	99	87	82	cyclictest	0	-21	swapper/3	04:08:49	3
26526	99	87	82	cyclictest	0	-21	swapper/3	04:07:02	3
26526	99	87	72	cyclictest	0	-21	swapper/3	05:33:53	3
26526	99	86	82	cyclictest	0	-21	swapper/3	09:23:44	3
26526	99	86	82	cyclictest	0	-21	swapper/3	09:20:36	3
26526	99	86	82	cyclictest	0	-21	swapper/3	08:58:46	3
26526	99	86	82	cyclictest	0	-21	swapper/3	08:23:24	3
26526	99	86	82	cyclictest	0	-21	swapper/3	07:38:47	3
26526	99	86	82	cyclictest	0	-21	swapper/3	07:06:21	3
26526	99	86	82	cyclictest	0	-21	swapper/3	06:23:51	3
26526	99	86	82	cyclictest	0	-21	swapper/3	06:18:51	3
26526	99	86	82	cyclictest	0	-21	swapper/3	05:46:09	3
26526	99	86	82	cyclictest	0	-21	swapper/3	05:43:31	3
26526	99	86	82	cyclictest	0	-21	swapper/3	04:43:55	3
26526	99	86	82	cyclictest	0	-21	swapper/3	04:25:55	3
26526	99	86	71	cyclictest	0	-21	swapper/3	09:13:43	3
26524	99	86	85	cyclictest	0	-21	swapper/1	07:33:49	1
26526	99	85	72	cyclictest	0	-21	swapper/3	08:46:22	3
26524	99	85	84	cyclictest	31484	-21	taskset	06:38:15	1
26524	99	85	82	cyclictest	0	-21	swapper/1	04:53:26	1
26524	99	84	83	cyclictest	0	-21	swapper/1	05:18:46	1
26524	99	84	82	cyclictest	31399	-21	tune2fs	04:08:49	1
26524	99	84	72	cyclictest	0	-21	swapper/1	08:00:55	1
26524	99	84	72	cyclictest	0	-21	swapper/1	07:24:10	1
26524	99	84	72	cyclictest	0	-21	swapper/1	04:33:37	1
26526	99	83	82	cyclictest	0	-21	swapper/3	08:54:57	3
26526	99	83	82	cyclictest	0	-21	swapper/3	08:30:40	3
26526	99	83	82	cyclictest	0	-21	swapper/3	08:23:48	3
26526	99	83	82	cyclictest	0	-21	swapper/3	08:08:43	3
26526	99	83	82	cyclictest	0	-21	swapper/3	08:01:30	3
26526	99	83	82	cyclictest	0	-21	swapper/3	07:48:44	3
26526	99	83	82	cyclictest	0	-21	swapper/3	07:47:28	3
26526	99	83	82	cyclictest	0	-21	swapper/3	07:33:49	3
26526	99	83	82	cyclictest	0	-21	swapper/3	07:32:06	3
26526	99	83	82	cyclictest	0	-21	swapper/3	07:18:49	3
26526	99	83	82	cyclictest	0	-21	swapper/3	07:13:46	3
26526	99	83	82	cyclictest	0	-21	swapper/3	06:53:40	3
26526	99	83	82	cyclictest	0	-21	swapper/3	06:48:50	3
26526	99	83	82	cyclictest	0	-21	swapper/3	06:33:41	3
26526	99	83	82	cyclictest	0	-21	swapper/3	06:13:44	3
26526	99	83	82	cyclictest	0	-21	swapper/3	06:13:44	3
26526	99	83	82	cyclictest	0	-21	swapper/3	05:58:51	3
26526	99	83	82	cyclictest	0	-21	swapper/3	05:50:25	3
26526	99	83	82	cyclictest	0	-21	swapper/3	05:28:54	3
26526	99	83	82	cyclictest	0	-21	swapper/3	05:23:45	3
26526	99	83	82	cyclictest	0	-21	swapper/3	05:08:51	3
26526	99	83	82	cyclictest	0	-21	swapper/3	04:53:49	3
26526	99	83	82	cyclictest	0	-21	swapper/3	04:33:55	3
26526	99	83	82	cyclictest	0	-21	swapper/3	04:28:56	3
26526	99	83	71	cyclictest	0	-21	swapper/3	08:33:42	3
26524	99	83	82	cyclictest	1382	-17	auditd	07:13:27	1
26524	99	83	82	cyclictest	0	-21	swapper/1	09:23:46	1
26524	99	83	82	cyclictest	0	-21	swapper/1	09:18:43	1
26524	99	83	82	cyclictest	0	-21	swapper/1	09:16:06	1
26524	99	83	82	cyclictest	0	-21	swapper/1	09:08:53	1
26524	99	83	82	cyclictest	0	-21	swapper/1	09:08:53	1
26524	99	83	82	cyclictest	0	-21	swapper/1	08:43:33	1
26524	99	83	82	cyclictest	0	-21	swapper/1	08:33:46	1
26524	99	83	82	cyclictest	0	-21	swapper/1	08:23:48	1
26524	99	83	82	cyclictest	0	-21	swapper/1	08:21:46	1
26524	99	83	82	cyclictest	0	-21	swapper/1	08:08:46	1
26524	99	83	82	cyclictest	0	-21	swapper/1	07:38:49	1
26524	99	83	82	cyclictest	0	-21	swapper/1	07:29:46	1
26524	99	83	82	cyclictest	0	-21	swapper/1	07:08:44	1
26524	99	83	82	cyclictest	0	-21	swapper/1	07:03:42	1
26524	99	83	82	cyclictest	0	-21	swapper/1	06:58:48	1
26524	99	83	82	cyclictest	0	-21	swapper/1	06:53:40	1
26524	99	83	82	cyclictest	0	-21	swapper/1	06:48:50	1
26524	99	83	82	cyclictest	0	-21	swapper/1	06:18:47	1
26524	99	83	82	cyclictest	0	-21	swapper/1	06:13:44	1
26524	99	83	82	cyclictest	0	-21	swapper/1	06:13:44	1
26524	99	83	82	cyclictest	0	-21	swapper/1	06:08:51	1
26524	99	83	82	cyclictest	0	-21	swapper/1	06:00:32	1
26524	99	83	82	cyclictest	0	-21	swapper/1	05:48:52	1
26524	99	83	82	cyclictest	0	-21	swapper/1	05:38:53	1
26524	99	83	82	cyclictest	0	-21	swapper/1	05:28:54	1
26524	99	83	82	cyclictest	0	-21	swapper/1	05:08:53	1
26524	99	83	82	cyclictest	0	-21	swapper/1	04:58:55	1
26524	99	83	82	cyclictest	0	-21	swapper/1	04:53:49	1
26524	99	83	82	cyclictest	0	-21	swapper/1	04:45:59	1
26524	99	83	82	cyclictest	0	-21	swapper/1	04:42:30	1
26524	99	83	82	cyclictest	0	-21	swapper/1	04:13:55	1
26524	99	83	81	cyclictest	5781	-21	tune2fs	04:23:49	1
26524	99	83	81	cyclictest	32112	-21	aten2.4-expect	07:53:38	1
26524	99	83	81	cyclictest	30252	-21	tune2fs	07:48:40	1
26524	99	83	81	cyclictest	24258	-21	tune2fs	08:48:39	1
26524	99	83	81	cyclictest	16914	-21	aten2.4-expect	07:18:39	1
26524	99	83	72	cyclictest	0	-21	swapper/1	05:23:47	1
26524	99	83	71	cyclictest	7919	-21	tune2fs	04:28:49	1
26524	99	83	71	cyclictest	0	-21	swapper/1	04:18:48	1
26526	99	82	81	cyclictest	0	-21	swapper/3	04:53:26	3
26524	99	82	81	cyclictest	408	-21	md1_raid1	06:38:52	1
26524	99	82	81	cyclictest	12	1	ksoftirqd/1	06:28:41	1
26526	99	78	62	cyclictest	0	-21	swapper/3	07:08:37	3
26524	99	78	77	cyclictest	0	-21	swapper/1	08:03:26	1
26526	99	77	72	cyclictest	0	-21	swapper/3	05:19:35	3
26526	99	77	61	cyclictest	27417	-21	aten2.4_r3power	06:28:41	3
26526	99	77	61	cyclictest	0	-21	swapper/3	06:06:13	3
26526	99	76	72	cyclictest	0	-21	swapper/3	05:03:47	3
26526	99	75	71	cyclictest	0	-21	swapper/3	07:53:56	3
26526	99	75	71	cyclictest	0	-21	swapper/3	05:13:49	3
26526	99	73	72	cyclictest	0	-21	swapper/3	06:58:48	3
26526	99	73	72	cyclictest	0	-21	swapper/3	04:58:55	3
26524	99	73	72	cyclictest	0	-21	swapper/1	07:43:36	1
26524	99	73	72	cyclictest	0	-21	swapper/1	06:43:46	1
26524	99	73	72	cyclictest	0	-21	swapper/1	06:03:43	1
26524	99	73	72	cyclictest	0	-21	swapper/1	05:53:40	1
26524	99	73	72	cyclictest	0	-21	swapper/1	05:43:43	1
26524	99	73	72	cyclictest	0	-21	swapper/1	05:33:44	1
26524	99	73	72	cyclictest	0	-21	swapper/1	05:13:47	1
26524	99	73	72	cyclictest	0	-21	swapper/1	05:03:47	1
26526	99	72	71	cyclictest	0	-21	swapper/3	07:24:10	3
26526	99	72	61	cyclictest	0	-21	swapper/3	04:41:18	3
26525	99	72	71	cyclictest	0	-21	swapper/2	06:38:14	2
26525	99	72	71	cyclictest	0	-21	swapper/2	06:30:12	2
26525	99	72	71	cyclictest	0	-21	swapper/2	06:19:49	2
26525	99	72	71	cyclictest	0	-21	swapper/2	06:09:39	2
26525	99	72	71	cyclictest	0	-21	swapper/2	05:44:29	2
26525	99	72	71	cyclictest	0	-21	swapper/2	04:17:00	2
26525	99	71	70	cyclictest	0	-21	swapper/2	06:54:53	2
26525	99	71	70	cyclictest	0	-21	swapper/2	06:14:13	2
26525	99	71	70	cyclictest	0	-21	swapper/2	06:14:13	2
26525	99	70	69	cyclictest	0	-21	swapper/2	06:24:47	2
26525	99	70	69	cyclictest	0	-21	swapper/2	05:59:16	2
26523	99	70	41	cyclictest	0	-21	swapper/0	07:42:27	0
26523	99	70	41	cyclictest	0	-21	swapper/0	07:34:30	0
26525	99	69	68	cyclictest	0	-21	swapper/2	07:46:28	2
26525	99	69	68	cyclictest	0	-21	swapper/2	06:40:03	2
26523	99	69	65	cyclictest	0	-21	swapper/0	05:46:05	0
26523	99	69	52	cyclictest	0	-21	swapper/0	07:31:08	0
26525	99	68	67	cyclictest	0	-21	swapper/2	07:06:34	2
26525	99	68	67	cyclictest	0	-21	swapper/2	06:51:57	2
26525	99	67	66	cyclictest	0	-21	swapper/2	07:25:40	2
26525	99	67	66	cyclictest	0	-21	swapper/2	05:49:53	2
26523	99	67	59	cyclictest	0	-21	swapper/0	07:47:59	0
26523	99	67	51	cyclictest	0	-21	swapper/0	06:58:51	0
26523	99	67	49	cyclictest	0	-21	swapper/0	06:29:44	0
26525	99	66	65	cyclictest	0	-21	swapper/2	07:42:29	2
26525	99	66	65	cyclictest	0	-21	swapper/2	07:35:32	2
26525	99	66	65	cyclictest	0	-21	swapper/2	05:10:05	2
26523	99	66	62	cyclictest	0	-21	swapper/0	07:09:09	0
26523	99	66	62	cyclictest	0	-21	swapper/0	05:40:38	0
26523	99	66	62	cyclictest	0	-21	swapper/0	04:32:39	0
26523	99	66	51	cyclictest	0	-21	swapper/0	07:22:16	0
26523	99	66	51	cyclictest	0	-21	swapper/0	07:06:54	0
26523	99	66	51	cyclictest	0	-21	swapper/0	06:56:53	0
26523	99	66	51	cyclictest	0	-21	swapper/0	05:25:35	0
26523	99	66	41	cyclictest	50	50	irq/44-ahci	07:23:57	0
26523	99	66	41	cyclictest	0	-21	swapper/0	07:48:35	0
26523	99	66	41	cyclictest	0	-21	swapper/0	06:49:10	0
26525	99	65	64	cyclictest	0	-21	swapper/2	07:48:36	2
26525	99	65	64	cyclictest	0	-21	swapper/2	07:19:12	2
26525	99	65	61	cyclictest	0	-21	swapper/2	06:58:32	2
26525	99	65	52	cyclictest	0	-21	swapper/2	07:17:43	2
26525	99	65	51	cyclictest	0	-21	swapper/2	07:31:59	2
26525	99	65	51	cyclictest	0	-21	swapper/2	05:41:29	2
26523	99	65	64	cyclictest	0	-21	swapper/0	06:22:46	0
26523	99	65	41	cyclictest	0	-21	swapper/0	06:23:59	0
26525	99	64	63	cyclictest	0	-21	swapper/2	04:39:43	2
26525	99	64	51	cyclictest	0	-21	swapper/2	07:09:29	2
26523	99	64	60	cyclictest	0	-21	swapper/0	07:14:15	0
26523	99	64	59	cyclictest	0	-21	swapper/0	09:04:59	0
26523	99	64	59	cyclictest	0	-21	swapper/0	06:03:51	0
26525	99	63	62	cyclictest	0	-21	swapper/2	06:45:02	2
26525	99	63	62	cyclictest	0	-21	swapper/2	05:16:51	2
26525	99	63	52	cyclictest	0	-21	swapper/2	09:05:02	2
26525	99	63	51	cyclictest	0	-21	swapper/2	04:24:56	2
26525	99	62	61	cyclictest	0	-21	swapper/2	08:19:42	2
26525	99	62	61	cyclictest	0	-21	swapper/2	06:06:45	2
26525	99	62	61	cyclictest	0	-21	swapper/2	05:53:37	2
26525	99	62	61	cyclictest	0	-21	swapper/2	05:34:35	2
26525	99	62	61	cyclictest	0	-21	swapper/2	05:21:49	2
26525	99	62	61	cyclictest	0	-21	swapper/2	05:06:35	2
26525	99	62	61	cyclictest	0	-21	swapper/2	05:00:40	2
26525	99	62	61	cyclictest	0	-21	swapper/2	04:36:51	2
26525	99	62	61	cyclictest	0	-21	swapper/2	04:05:54	2
26525	99	62	51	cyclictest	0	-21	swapper/2	04:55:41	2
26525	99	62	51	cyclictest	0	-21	swapper/2	04:46:01	2
26523	99	62	61	cyclictest	0	-21	swapper/0	04:34:15	0
26523	99	62	51	cyclictest	0	-21	swapper/0	09:08:24	0
26523	99	62	51	cyclictest	0	-21	swapper/0	09:08:24	0
26523	99	62	51	cyclictest	0	-21	swapper/0	06:44:30	0
26523	99	62	51	cyclictest	0	-21	swapper/0	06:41:47	0
26523	99	62	51	cyclictest	0	-21	swapper/0	06:34:45	0
26523	99	62	51	cyclictest	0	-21	swapper/0	06:14:07	0
26523	99	62	51	cyclictest	0	-21	swapper/0	06:14:07	0
26523	99	62	51	cyclictest	0	-21	swapper/0	06:09:34	0
26523	99	62	51	cyclictest	0	-21	swapper/0	06:00:25	0
26523	99	62	51	cyclictest	0	-21	swapper/0	05:33:15	0
26523	99	62	51	cyclictest	0	-21	swapper/0	05:09:15	0
26523	99	62	51	cyclictest	0	-21	swapper/0	04:39:40	0
26523	99	62	51	cyclictest	0	-21	swapper/0	04:00:00	0
26525	99	61	60	cyclictest	0	-21	swapper/2	05:27:14	2
26523	99	61	60	cyclictest	0	-21	swapper/0	04:49:21	0
26523	99	61	50	cyclictest	0	-21	swapper/0	04:47:26	0
26525	99	60	59	cyclictest	0	-21	swapper/2	09:08:33	2
26525	99	60	59	cyclictest	0	-21	swapper/2	09:08:33	2
26525	99	60	59	cyclictest	0	-21	swapper/2	05:30:43	2
26523	99	60	59	cyclictest	0	-21	swapper/0	05:23:20	0
26525	99	59	58	cyclictest	0	-21	swapper/2	04:52:18	2
26525	99	59	58	cyclictest	0	-21	swapper/2	04:21:05	2
26525	99	59	58	cyclictest	0	-21	swapper/2	04:11:52	2
26523	99	59	58	cyclictest	0	-21	swapper/0	05:15:48	0
26523	99	59	58	cyclictest	0	-21	swapper/0	04:19:52	0
26525	99	58	57	cyclictest	0	-21	swapper/2	08:48:36	2
26523	99	58	57	cyclictest	0	-21	swapper/0	05:35:19	0
26523	99	58	31	cyclictest	0	-21	swapper/0	08:09:10	0
26525	99	57	56	cyclictest	0	-21	swapper/2	04:29:08	2
26525	99	57	52	cyclictest	18	1	ksoftirqd/2	08:26:04	2
26523	99	57	56	cyclictest	0	-21	swapper/0	05:57:38	0
26523	99	57	56	cyclictest	0	-21	swapper/0	05:50:10	0
26523	99	57	56	cyclictest	0	-21	swapper/0	05:02:47	0
26523	99	57	31	cyclictest	0	-21	swapper/0	07:54:05	0
26525	99	56	55	cyclictest	0	-21	swapper/2	08:34:29	2
26525	99	56	54	cyclictest	0	-21	swapper/2	04:03:21	2
26525	99	56	41	cyclictest	0	-21	swapper/2	07:57:53	2
26523	99	56	52	cyclictest	0	-21	swapper/0	04:09:40	0
26523	99	56	51	cyclictest	0	-21	swapper/0	04:54:23	0
26523	99	56	51	cyclictest	0	-21	swapper/0	04:06:08	0
26523	99	56	31	cyclictest	0	-21	swapper/0	09:16:12	0
26525	99	55	54	cyclictest	9850	-21	munin-node	08:13:48	2
26525	99	55	54	cyclictest	6555	-21	latency_hist	09:23:22	2
26525	99	55	54	cyclictest	3978	-21	runrttasks	09:14:12	2
26525	99	55	54	cyclictest	27311	-21	munin-node	08:53:44	2
26525	99	55	54	cyclictest	22479	-21	munin-node	08:43:42	2
26525	99	55	54	cyclictest	0	-21	swapper/2	09:23:46	2
26525	99	55	41	cyclictest	0	-21	swapper/2	08:07:11	2
26523	99	55	54	cyclictest	0	-21	swapper/0	08:28:38	0
26523	99	55	51	cyclictest	0	-21	swapper/0	04:14:06	0
26523	99	55	31	cyclictest	0	-21	swapper/0	09:23:14	0
26525	99	54	53	cyclictest	18	1	ksoftirqd/2	08:41:39	2
26525	99	54	52	cyclictest	50	50	irq/44-ahci	08:10:00	2
26523	99	54	43	cyclictest	0	-21	swapper/0	08:23:37	0
26523	99	54	37	cyclictest	0	-21	swapper/0	08:54:23	0
26525	99	53	52	cyclictest	50	50	irq/44-ahci	07:59:01	2
26525	99	53	52	cyclictest	0	-21	swapper/2	08:28:42	2
26525	99	53	41	cyclictest	0	-21	swapper/2	08:58:27	2
26523	99	53	52	cyclictest	760	50	irq/18-parport0	08:04:43	0
26523	99	53	52	cyclictest	0	-21	swapper/0	05:04:21	0
26523	99	53	52	cyclictest	0	-21	swapper/0	04:23:37	0
26523	99	52	51	cyclictest	760	50	irq/18-parport0	08:38:27	0
26523	99	52	51	cyclictest	392	-21	md2_raid1	09:27:23	0
26523	99	52	51	cyclictest	0	-21	swapper/0	09:01:13	0
26523	99	52	51	cyclictest	0	-21	swapper/0	08:48:36	0
26523	99	52	51	cyclictest	0	-21	swapper/0	08:44:29	0
26523	99	52	51	cyclictest	0	-21	swapper/0	08:35:41	0
26523	99	52	51	cyclictest	0	-21	swapper/0	07:58:38	0
26523	99	49	21	cyclictest	0	-21	swapper/0	08:16:50	0
26523	99	46	31	cyclictest	0	-21	swapper/0	08:20:12	0
50	50	14	0	irq/44-ahci	24656	-21	diskstats	03:53:49	2
23	1	14	0	ksoftirqd/3	20599	-21	md1_resync	03:55:56	3
50	50	13	0	irq/44-ahci	2725	-21	runrttasks	03:55:24	0
12	1	11	0	ksoftirqd/1	0	-21	swapper/1	03:54:18	1
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17