OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #3, slot #1

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance

Characteristics of the highest latencies:

System rack3slot1.osadl.org (updated Mon Dec 08, 2025 12:43:26)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
13114	99	104	102	cyclictest	7223	-21	tune2fs	23:10:11	1
13114	99	104	102	cyclictest	28975	-21	tune2fs	22:45:13	1
13116	99	103	92	cyclictest	0	-21	swapper/3	21:30:25	3
13116	99	103	102	cyclictest	0	-21	swapper/3	23:10:11	3
13116	99	103	102	cyclictest	0	-21	swapper/3	22:45:13	3
13114	99	103	102	cyclictest	28242	-21	munin-node	21:30:25	1
13116	99	89	72	cyclictest	0	-21	swapper/3	22:30:13	3
13116	99	87	82	cyclictest	0	-21	swapper/3	23:30:10	3
13114	99	87	81	cyclictest	15835	-21	tune2fs	23:30:10	1
13114	99	84	82	cyclictest	24065	-21	aten2.4-expect	21:20:14	1
13114	99	84	82	cyclictest	20420	-21	tune2fs	22:25:12	1
13116	99	83	82	cyclictest	0	-21	swapper/3	23:45:10	3
13116	99	83	82	cyclictest	0	-21	swapper/3	23:15:14	3
13116	99	83	82	cyclictest	0	-21	swapper/3	22:20:20	3
13116	99	83	82	cyclictest	0	-21	swapper/3	01:00:06	3
13114	99	83	82	cyclictest	13685	-21	tune2fs	20:55:16	1
13114	99	83	81	cyclictest	9766	-21	tune2fs	00:30:10	1
13114	99	83	81	cyclictest	9144	-21	aten2.4-expect	23:15:14	1
13114	99	83	81	cyclictest	5174	-21	tune2fs	21:50:14	1
13114	99	83	81	cyclictest	24949	-21	tune2fs	02:20:04	1
13114	99	83	81	cyclictest	24527	-21	tune2fs	22:35:12	1
13114	99	83	81	cyclictest	22408	-21	tune2fs	01:00:06	1
13114	99	83	81	cyclictest	22407	-21	tune2fs	23:45:10	1
13114	99	83	81	cyclictest	20379	-21	tune2fs	23:40:10	1
13114	99	83	81	cyclictest	18016	-21	tune2fs	22:20:20	1
13114	99	83	81	cyclictest	15864	-21	tune2fs	22:15:14	1
13114	99	83	81	cyclictest	13794	-21	tune2fs	22:10:10	1
13114	99	83	81	cyclictest	11725	-21	tune2fs	00:35:10	1
13114	99	83	81	cyclictest	0	-21	swapper/1	23:20:11	1
13116	99	82	81	cyclictest	0	-21	swapper/3	23:40:10	3
13116	99	82	81	cyclictest	0	-21	swapper/3	23:00:13	3
13116	99	82	81	cyclictest	0	-21	swapper/3	22:35:12	3
13116	99	82	81	cyclictest	0	-21	swapper/3	22:25:12	3
13116	99	82	81	cyclictest	0	-21	swapper/3	22:15:14	3
13116	99	82	81	cyclictest	0	-21	swapper/3	22:10:10	3
13116	99	82	81	cyclictest	0	-21	swapper/3	21:50:14	3
13116	99	82	81	cyclictest	0	-21	swapper/3	21:20:14	3
13116	99	82	81	cyclictest	0	-21	swapper/3	20:55:16	3
13116	99	82	81	cyclictest	0	-21	swapper/3	02:20:04	3
13116	99	82	81	cyclictest	0	-21	swapper/3	01:40:07	3
13116	99	82	81	cyclictest	0	-21	swapper/3	00:35:10	3
13116	99	82	81	cyclictest	0	-21	swapper/3	00:30:10	3
13116	99	82	81	cyclictest	0	-21	swapper/3	00:00:11	3
13114	99	82	81	cyclictest	7346	-21	tune2fs	01:40:07	1
13114	99	82	81	cyclictest	3062	-21	tune2fs	23:00:13	1
13114	99	82	81	cyclictest	29033	-21	tune2fs	00:00:11	1
13114	99	82	81	cyclictest	22530	-21	tune2fs	22:30:13	1
13114	99	82	81	cyclictest	11843	-21	tune2fs	01:50:07	1
13116	99	73	72	cyclictest	0	-21	swapper/3	22:58:06	3
13116	99	73	72	cyclictest	0	-21	swapper/3	01:54:49	3
13116	99	72	71	cyclictest	0	-21	swapper/3	23:25:14	3
13116	99	72	71	cyclictest	0	-21	swapper/3	23:20:11	3
13116	99	72	71	cyclictest	0	-21	swapper/3	21:35:16	3
13116	99	72	71	cyclictest	0	-21	swapper/3	00:55:14	3
13114	99	72	71	cyclictest	31029	-21	tune2fs	21:35:16	1
13114	99	72	71	cyclictest	0	-21	swapper/1	23:25:18	1
13114	99	72	71	cyclictest	0	-21	swapper/1	00:55:14	1
13116	99	69	62	cyclictest	0	-21	swapper/3	01:15:12	3
13116	99	68	62	cyclictest	0	-21	swapper/3	22:00:12	3
13114	99	68	63	cyclictest	0	-21	swapper/1	01:05:10	1
13114	99	68	61	cyclictest	9361	-21	tune2fs	22:00:12	1
13114	99	68	61	cyclictest	29854	-21	sendmail_mailqu	01:15:12	1
13114	99	68	61	cyclictest	18413	-21	tune2fs	00:50:07	1
13114	99	68	61	cyclictest	0	-21	swapper/1	21:45:10	1
13116	99	67	62	cyclictest	0	-21	swapper/3	21:15:17	3
13116	99	67	62	cyclictest	0	-21	swapper/3	01:55:09	3
13114	99	67	61	cyclictest	919	-21	tune2fs	22:55:11	1
13114	99	67	61	cyclictest	0	-21	swapper/1	22:40:10	1
13114	99	67	51	cyclictest	0	-21	swapper/1	21:10:20	1
13114	99	67	51	cyclictest	0	-21	swapper/1	02:10:05	1
13114	99	67	51	cyclictest	0	-21	swapper/1	00:15:09	1
13116	99	66	62	cyclictest	0	-21	swapper/3	21:42:54	3
13116	99	66	62	cyclictest	0	-21	swapper/3	21:12:50	3
13116	99	66	62	cyclictest	0	-21	swapper/3	02:05:13	3
13116	99	66	62	cyclictest	0	-21	swapper/3	02:00:11	3
13116	99	66	62	cyclictest	0	-21	swapper/3	01:09:51	3
13116	99	66	61	cyclictest	0	-21	swapper/3	22:40:10	3
13114	99	66	62	cyclictest	0	-21	swapper/1	21:05:15	1
13114	99	66	51	cyclictest	0	-21	swapper/1	23:54:36	1
13114	99	66	51	cyclictest	0	-21	swapper/1	23:35:17	1
13114	99	66	51	cyclictest	0	-21	swapper/1	22:05:16	1
13114	99	66	51	cyclictest	0	-21	swapper/1	21:55:13	1
13114	99	66	51	cyclictest	0	-21	swapper/1	21:25:20	1
13114	99	66	51	cyclictest	0	-21	swapper/1	21:15:16	1
13114	99	66	51	cyclictest	0	-21	swapper/1	21:00:20	1
13114	99	66	51	cyclictest	0	-21	swapper/1	02:05:15	1
13114	99	66	51	cyclictest	0	-21	swapper/1	02:00:03	1
13114	99	66	51	cyclictest	0	-21	swapper/1	01:59:49	1
13114	99	66	51	cyclictest	0	-21	swapper/1	01:25:09	1
13116	99	65	61	cyclictest	0	-21	swapper/3	21:45:10	3
13114	99	65	61	cyclictest	0	-21	swapper/1	21:42:54	1
13114	99	65	61	cyclictest	0	-21	swapper/1	00:28:18	1
13114	99	64	62	cyclictest	30977	-21	tune2fs	00:05:10	1
13116	99	63	62	cyclictest	0	-21	swapper/3	23:55:16	3
13116	99	63	62	cyclictest	0	-21	swapper/3	23:34:56	3
13116	99	63	62	cyclictest	0	-21	swapper/3	22:50:18	3
13116	99	63	62	cyclictest	0	-21	swapper/3	21:07:49	3
13116	99	63	62	cyclictest	0	-21	swapper/3	21:01:02	3
13116	99	63	62	cyclictest	0	-21	swapper/3	02:10:09	3
13116	99	63	62	cyclictest	0	-21	swapper/3	01:45:11	3
13116	99	63	62	cyclictest	0	-21	swapper/3	01:35:12	3
13116	99	63	62	cyclictest	0	-21	swapper/3	01:25:04	3
13116	99	63	62	cyclictest	0	-21	swapper/3	01:20:05	3
13116	99	63	62	cyclictest	0	-21	swapper/3	00:50:13	3
13116	99	63	62	cyclictest	0	-21	swapper/3	00:48:21	3
13116	99	63	62	cyclictest	0	-21	swapper/3	00:13:16	3
13116	99	63	62	cyclictest	0	-21	swapper/3	00:05:17	3
13114	99	63	62	cyclictest	0	-21	swapper/1	23:05:09	1
13114	99	63	62	cyclictest	0	-21	swapper/1	22:50:15	1
13114	99	63	62	cyclictest	0	-21	swapper/1	01:20:05	1
13114	99	63	62	cyclictest	0	-21	swapper/1	01:10:14	1
13114	99	63	61	cyclictest	3005	-21	tune2fs	01:30:04	1
13116	99	62	61	cyclictest	0	-21	swapper/3	23:50:08	3
13116	99	62	61	cyclictest	0	-21	swapper/3	23:05:11	3
13116	99	62	61	cyclictest	0	-21	swapper/3	22:05:15	3
13116	99	62	61	cyclictest	0	-21	swapper/3	21:25:11	3
13116	99	62	61	cyclictest	0	-21	swapper/3	01:30:04	3
13116	99	62	61	cyclictest	0	-21	swapper/3	00:25:08	3
13116	99	62	61	cyclictest	0	-21	swapper/3	00:20:07	3
13116	99	62	61	cyclictest	0	-21	swapper/3	00:15:09	3
13114	99	62	61	cyclictest	0	-21	swapper/1	23:55:16	1
13114	99	62	61	cyclictest	0	-21	swapper/1	02:15:10	1
13114	99	62	61	cyclictest	0	-21	swapper/1	01:45:03	1
13114	99	62	61	cyclictest	0	-21	swapper/1	00:48:21	1
13114	99	62	61	cyclictest	0	-21	swapper/1	00:20:07	1
13114	99	62	61	cyclictest	0	-21	swapper/1	00:13:16	1
13114	99	62	51	cyclictest	0	-21	swapper/1	01:35:06	1
13115	99	57	56	cyclictest	0	-21	swapper/2	22:35:10	2
13115	99	57	51	cyclictest	0	-21	swapper/2	23:16:47	2
13115	99	56	55	cyclictest	0	-21	swapper/2	01:05:04	2
13115	99	55	52	cyclictest	0	-21	swapper/2	22:05:45	2
13115	99	55	52	cyclictest	0	-21	swapper/2	01:35:53	2
13115	99	55	41	cyclictest	0	-21	swapper/2	00:12:27	2
13115	99	54	53	cyclictest	275	-21	usb-storage	21:56:54	2
13115	99	54	53	cyclictest	275	-21	usb-storage	02:08:56	2
13115	99	54	53	cyclictest	275	-21	usb-storage	00:05:39	2
13115	99	54	53	cyclictest	18	1	ksoftirqd/2	01:48:19	2
13115	99	54	53	cyclictest	0	-21	swapper/2	22:22:21	2
13115	99	54	53	cyclictest	0	-21	swapper/2	00:49:57	2
13115	99	54	52	cyclictest	0	-21	swapper/2	21:45:10	2
13115	99	54	51	cyclictest	0	-21	swapper/2	23:32:03	2
13115	99	54	42	cyclictest	275	1	usb-storage	00:03:29	2
13115	99	54	42	cyclictest	275	-21	usb-storage	23:03:07	2
13115	99	54	42	cyclictest	275	-21	usb-storage	01:42:07	2
13115	99	54	42	cyclictest	275	-21	usb-storage	00:18:28	2
13115	99	54	41	cyclictest	275	-21	usb-storage	23:23:28	2
13115	99	54	41	cyclictest	275	-21	usb-storage	21:42:20	2
13115	99	54	41	cyclictest	275	-21	usb-storage	21:12:33	2
13112	99	54	53	cyclictest	3	1	ksoftirqd/0	22:30:42	0
13116	99	53	52	cyclictest	0	-21	swapper/3	00:40:05	3
13115	99	53	52	cyclictest	275	-21	usb-storage	23:42:02	2
13115	99	53	52	cyclictest	0	-21	swapper/2	23:56:27	2
13115	99	53	52	cyclictest	0	-21	swapper/2	23:49:59	2
13115	99	53	52	cyclictest	0	-21	swapper/2	23:35:02	2
13115	99	53	52	cyclictest	0	-21	swapper/2	22:46:02	2
13115	99	53	52	cyclictest	0	-21	swapper/2	22:30:11	2
13115	99	53	52	cyclictest	0	-21	swapper/2	22:15:11	2
13115	99	53	52	cyclictest	0	-21	swapper/2	22:10:16	2
13115	99	53	52	cyclictest	0	-21	swapper/2	22:01:55	2
13115	99	53	52	cyclictest	0	-21	swapper/2	21:52:46	2
13115	99	53	52	cyclictest	0	-21	swapper/2	21:35:15	2
13115	99	53	52	cyclictest	0	-21	swapper/2	21:30:14	2
13115	99	53	52	cyclictest	0	-21	swapper/2	21:25:17	2
13115	99	53	52	cyclictest	0	-21	swapper/2	21:05:06	2
13115	99	53	52	cyclictest	0	-21	swapper/2	02:20:00	2
13115	99	53	52	cyclictest	0	-21	swapper/2	02:10:00	2
13115	99	53	52	cyclictest	0	-21	swapper/2	01:59:50	2
13115	99	53	52	cyclictest	0	-21	swapper/2	01:55:51	2
13115	99	53	52	cyclictest	0	-21	swapper/2	01:50:05	2
13115	99	53	52	cyclictest	0	-21	swapper/2	01:30:03	2
13115	99	53	52	cyclictest	0	-21	swapper/2	01:25:11	2
13115	99	53	52	cyclictest	0	-21	swapper/2	01:20:03	2
13115	99	53	52	cyclictest	0	-21	swapper/2	01:00:13	2
13115	99	53	52	cyclictest	0	-21	swapper/2	00:55:01	2
13115	99	53	52	cyclictest	0	-21	swapper/2	00:45:08	2
13115	99	53	52	cyclictest	0	-21	swapper/2	00:40:05	2
13115	99	53	52	cyclictest	0	-21	swapper/2	00:25:06	2
13115	99	53	52	cyclictest	0	-21	swapper/2	00:20:06	2
13115	99	53	41	cyclictest	275	-21	usb-storage	22:50:00	2
13115	99	53	41	cyclictest	275	-21	usb-storage	20:55:12	2
13115	99	53	41	cyclictest	0	-21	swapper/2	23:45:02	2
13115	99	53	41	cyclictest	0	-21	swapper/2	23:25:12	2
13115	99	53	41	cyclictest	0	-21	swapper/2	23:10:07	2
13115	99	53	41	cyclictest	0	-21	swapper/2	23:04:59	2
13115	99	53	41	cyclictest	0	-21	swapper/2	22:54:59	2
13115	99	53	41	cyclictest	0	-21	swapper/2	22:40:02	2
13115	99	53	41	cyclictest	0	-21	swapper/2	22:25:09	2
13115	99	53	41	cyclictest	0	-21	swapper/2	21:20:11	2
13115	99	53	41	cyclictest	0	-21	swapper/2	21:16:13	2
13115	99	53	41	cyclictest	0	-21	swapper/2	21:00:10	2
13115	99	53	41	cyclictest	0	-21	swapper/2	02:15:50	2
13115	99	53	41	cyclictest	0	-21	swapper/2	01:15:03	2
13115	99	53	41	cyclictest	0	-21	swapper/2	01:10:04	2
13115	99	53	41	cyclictest	0	-21	swapper/2	00:35:06	2
13115	99	53	41	cyclictest	0	-21	swapper/2	00:30:02	2
13114	99	53	31	cyclictest	0	-21	swapper/1	00:44:19	1
13112	99	53	52	cyclictest	745	50	irq/18-parport0	02:07:08	0
13112	99	53	52	cyclictest	745	50	irq/18-parport0	00:53:05	0
13112	99	53	52	cyclictest	275	-21	usb-storage	23:06:19	0
13112	99	53	52	cyclictest	275	-21	usb-storage	21:04:00	0
13112	99	53	52	cyclictest	0	-21	swapper/0	22:51:20	0
13112	99	53	52	cyclictest	0	-21	swapper/0	22:20:39	0
13112	99	53	52	cyclictest	0	-21	swapper/0	01:10:28	0
13112	99	53	52	cyclictest	0	-21	swapper/0	00:27:24	0
13112	99	53	52	cyclictest	0	-21	swapper/0	00:16:00	0
13112	99	53	52	cyclictest	0	-21	swapper/0	00:12:27	0
13112	99	53	41	cyclictest	0	-21	swapper/0	21:53:14	0
13116	99	52	51	cyclictest	0	-21	swapper/3	21:55:11	3
13116	99	52	51	cyclictest	0	-21	swapper/3	02:15:01	3
13116	99	52	51	cyclictest	0	-21	swapper/3	01:10:04	3
13112	99	52	51	cyclictest	745	50	irq/18-parport0	23:53:15	0
13112	99	52	51	cyclictest	745	50	irq/18-parport0	22:10:25	0
13112	99	52	51	cyclictest	745	50	irq/18-parport0	21:15:09	0
13112	99	52	51	cyclictest	275	-21	usb-storage	22:55:15	0
13112	99	52	51	cyclictest	275	-21	usb-storage	00:02:17	0
13112	99	52	51	cyclictest	0	-21	swapper/0	23:57:23	0
13112	99	52	51	cyclictest	0	-21	swapper/0	23:43:54	0
13112	99	52	51	cyclictest	0	-21	swapper/0	23:30:07	0
13112	99	52	51	cyclictest	0	-21	swapper/0	23:23:44	0
13112	99	52	51	cyclictest	0	-21	swapper/0	23:15:11	0
13112	99	52	51	cyclictest	0	-21	swapper/0	23:14:37	0
13112	99	52	51	cyclictest	0	-21	swapper/0	23:00:10	0
13112	99	52	51	cyclictest	0	-21	swapper/0	22:41:44	0
13112	99	52	51	cyclictest	0	-21	swapper/0	22:38:00	0
13112	99	52	51	cyclictest	0	-21	swapper/0	22:28:34	0
13112	99	52	51	cyclictest	0	-21	swapper/0	22:15:12	0
13112	99	52	51	cyclictest	0	-21	swapper/0	21:45:10	0
13112	99	52	51	cyclictest	0	-21	swapper/0	21:40:46	0
13112	99	52	51	cyclictest	0	-21	swapper/0	21:34:11	0
13112	99	52	51	cyclictest	0	-21	swapper/0	21:26:41	0
13112	99	52	51	cyclictest	0	-21	swapper/0	21:11:27	0
13112	99	52	51	cyclictest	0	-21	swapper/0	21:07:08	0
13112	99	52	51	cyclictest	0	-21	swapper/0	02:20:01	0
13112	99	52	51	cyclictest	0	-21	swapper/0	02:03:58	0
13112	99	52	51	cyclictest	0	-21	swapper/0	01:54:59	0
13112	99	52	51	cyclictest	0	-21	swapper/0	01:50:43	0
13112	99	52	51	cyclictest	0	-21	swapper/0	01:40:01	0
13112	99	52	51	cyclictest	0	-21	swapper/0	01:25:30	0
13112	99	52	51	cyclictest	0	-21	swapper/0	01:20:04	0
13112	99	52	51	cyclictest	0	-21	swapper/0	01:05:04	0
13112	99	52	51	cyclictest	0	-21	swapper/0	01:02:49	0
13112	99	52	51	cyclictest	0	-21	swapper/0	00:45:08	0
13112	99	52	51	cyclictest	0	-21	swapper/0	00:44:19	0
13112	99	52	51	cyclictest	0	-21	swapper/0	00:22:46	0
13112	99	52	51	cyclictest	0	-21	swapper/0	00:05:37	0
13112	99	51	50	cyclictest	0	-21	swapper/0	23:35:06	0
13112	99	51	50	cyclictest	0	-21	swapper/0	23:25:08	0
13112	99	51	50	cyclictest	0	-21	swapper/0	22:05:10	0
13112	99	51	50	cyclictest	0	-21	swapper/0	22:00:09	0
13112	99	51	50	cyclictest	0	-21	swapper/0	21:35:12	0
13112	99	51	50	cyclictest	0	-21	swapper/0	21:20:14	0
13112	99	51	50	cyclictest	0	-21	swapper/0	20:55:12	0
13112	99	51	50	cyclictest	0	-21	swapper/0	01:45:02	0
13112	99	51	50	cyclictest	0	-21	swapper/0	00:55:03	0
13112	99	51	50	cyclictest	0	-21	swapper/0	00:35:08	0
13112	99	51	50	cyclictest	0	-21	swapper/0	00:30:07	0
13112	99	48	47	cyclictest	0	-21	swapper/0	02:15:04	0
13112	99	45	41	cyclictest	0	-21	swapper/0	02:10:07	0
13112	99	45	41	cyclictest	0	-21	swapper/0	01:30:30	0
13112	99	44	41	cyclictest	0	-21	swapper/0	01:39:43	0
13112	99	44	41	cyclictest	0	-21	swapper/0	01:16:00	0
13112	99	43	42	cyclictest	0	-21	swapper/0	23:45:14	0
13112	99	43	42	cyclictest	0	-21	swapper/0	21:55:02	0
13112	99	43	41	cyclictest	1877	-21	hald-addon-stor	22:47:42	0
8	99	11	0	watchdog/0	12930	-40	yum-updatesd-he	20:54:50	0
18	1	11	0	ksoftirqd/2	0	-21	swapper/2	20:53:29	2
23	1	10	0	ksoftirqd/3	0	-21	swapper/3	20:51:25	3
12	1	10	0	ksoftirqd/1	0	-21	swapper/1	20:53:29	1
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17