OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #3, slot #1

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance

Characteristics of the 376 highest latencies:

System rack3slot1.osadl.org (updated Mon Dec 22, 2025 00:43:25)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
28020	99	125	123	cyclictest	0	-21	swapper/1	00:25:08	1
28020	99	108	92	cyclictest	0	-21	swapper/1	23:20:07	1
28020	99	108	92	cyclictest	0	-21	swapper/1	22:10:19	1
28020	99	108	102	cyclictest	12	1	ksoftirqd/1	21:09:58	1
28020	99	108	102	cyclictest	11251	-21	tune2fs	19:45:16	1
28020	99	104	103	cyclictest	0	-21	swapper/1	19:35:01	1
28022	99	103	102	cyclictest	21383	-21	munin-node	23:55:15	3
28020	99	103	92	cyclictest	0	-21	swapper/1	22:45:12	1
28020	99	103	102	cyclictest	0	-21	swapper/1	19:15:25	1
28020	99	93	92	cyclictest	0	-21	swapper/1	20:59:59	1
28020	99	88	82	cyclictest	0	-21	swapper/1	22:35:10	1
28020	99	88	81	cyclictest	2641	-21	tune2fs	21:55:10	1
28020	99	88	81	cyclictest	13393	-21	tune2fs	23:35:10	1
28020	99	87	81	cyclictest	15497	-21	tune2fs	21:10:14	1
28020	99	86	82	cyclictest	0	-21	swapper/1	23:25:14	1
28020	99	86	82	cyclictest	0	-21	swapper/1	23:05:15	1
28020	99	84	82	cyclictest	0	-21	swapper/1	20:45:23	1
28020	99	84	72	cyclictest	0	-21	swapper/1	22:34:55	1
28022	99	83	82	cyclictest	0	-21	swapper/3	23:50:15	3
28022	99	83	82	cyclictest	0	-21	swapper/3	23:40:15	3
28022	99	83	82	cyclictest	0	-21	swapper/3	23:35:10	3
28022	99	83	82	cyclictest	0	-21	swapper/3	23:15:14	3
28022	99	83	82	cyclictest	0	-21	swapper/3	23:00:14	3
28022	99	83	82	cyclictest	0	-21	swapper/3	22:55:13	3
28022	99	83	82	cyclictest	0	-21	swapper/3	22:50:15	3
28022	99	83	82	cyclictest	0	-21	swapper/3	22:49:55	3
28022	99	83	82	cyclictest	0	-21	swapper/3	22:40:15	3
28022	99	83	82	cyclictest	0	-21	swapper/3	22:30:16	3
28022	99	83	82	cyclictest	0	-21	swapper/3	22:25:10	3
28022	99	83	82	cyclictest	0	-21	swapper/3	22:25:10	3
28022	99	83	82	cyclictest	0	-21	swapper/3	22:20:17	3
28022	99	83	82	cyclictest	0	-21	swapper/3	22:15:20	3
28022	99	83	82	cyclictest	0	-21	swapper/3	22:10:06	3
28022	99	83	82	cyclictest	0	-21	swapper/3	22:00:18	3
28022	99	83	82	cyclictest	0	-21	swapper/3	21:55:16	3
28022	99	83	82	cyclictest	0	-21	swapper/3	21:45:02	3
28022	99	83	82	cyclictest	0	-21	swapper/3	21:40:01	3
28022	99	83	82	cyclictest	0	-21	swapper/3	21:35:01	3
28022	99	83	82	cyclictest	0	-21	swapper/3	21:25:21	3
28022	99	83	82	cyclictest	0	-21	swapper/3	21:15:20	3
28022	99	83	82	cyclictest	0	-21	swapper/3	21:10:21	3
28022	99	83	82	cyclictest	0	-21	swapper/3	20:59:55	3
28022	99	83	82	cyclictest	0	-21	swapper/3	20:40:21	3
28022	99	83	82	cyclictest	0	-21	swapper/3	20:35:24	3
28022	99	83	82	cyclictest	0	-21	swapper/3	20:25:21	3
28022	99	83	82	cyclictest	0	-21	swapper/3	20:24:50	3
28022	99	83	82	cyclictest	0	-21	swapper/3	19:55:01	3
28022	99	83	82	cyclictest	0	-21	swapper/3	19:45:16	3
28022	99	83	82	cyclictest	0	-21	swapper/3	19:39:43	3
28022	99	83	82	cyclictest	0	-21	swapper/3	19:35:01	3
28022	99	83	82	cyclictest	0	-21	swapper/3	19:25:18	3
28022	99	83	82	cyclictest	0	-21	swapper/3	19:25:18	3
28022	99	83	82	cyclictest	0	-21	swapper/3	19:10:03	3
28022	99	83	82	cyclictest	0	-21	swapper/3	00:35:05	3
28022	99	83	82	cyclictest	0	-21	swapper/3	00:30:13	3
28022	99	83	82	cyclictest	0	-21	swapper/3	00:25:08	3
28022	99	83	82	cyclictest	0	-21	swapper/3	00:10:14	3
28022	99	83	82	cyclictest	0	-21	swapper/3	00:05:04	3
28022	99	83	81	cyclictest	30841	-21	grep	20:30:13	3
28020	99	83	82	cyclictest	0	-21	swapper/1	23:55:10	1
28020	99	83	82	cyclictest	0	-21	swapper/1	23:50:12	1
28020	99	83	82	cyclictest	0	-21	swapper/1	23:45:09	1
28020	99	83	82	cyclictest	0	-21	swapper/1	23:40:06	1
28020	99	83	82	cyclictest	0	-21	swapper/1	23:30:10	1
28020	99	83	82	cyclictest	0	-21	swapper/1	23:10:06	1
28020	99	83	82	cyclictest	0	-21	swapper/1	23:00:15	1
28020	99	83	82	cyclictest	0	-21	swapper/1	22:55:17	1
28020	99	83	82	cyclictest	0	-21	swapper/1	22:40:10	1
28020	99	83	82	cyclictest	0	-21	swapper/1	22:25:17	1
28020	99	83	82	cyclictest	0	-21	swapper/1	22:25:17	1
28020	99	83	82	cyclictest	0	-21	swapper/1	22:20:08	1
28020	99	83	82	cyclictest	0	-21	swapper/1	22:15:14	1
28020	99	83	82	cyclictest	0	-21	swapper/1	22:05:11	1
28020	99	83	82	cyclictest	0	-21	swapper/1	21:50:03	1
28020	99	83	82	cyclictest	0	-21	swapper/1	21:45:02	1
28020	99	83	82	cyclictest	0	-21	swapper/1	21:40:14	1
28020	99	83	82	cyclictest	0	-21	swapper/1	21:35:15	1
28020	99	83	82	cyclictest	0	-21	swapper/1	21:30:14	1
28020	99	83	82	cyclictest	0	-21	swapper/1	21:25:21	1
28020	99	83	82	cyclictest	0	-21	swapper/1	21:20:16	1
28020	99	83	82	cyclictest	0	-21	swapper/1	21:15:11	1
28020	99	83	82	cyclictest	0	-21	swapper/1	21:00:21	1
28020	99	83	82	cyclictest	0	-21	swapper/1	20:50:14	1
28020	99	83	82	cyclictest	0	-21	swapper/1	20:40:12	1
28020	99	83	82	cyclictest	0	-21	swapper/1	20:35:24	1
28020	99	83	82	cyclictest	0	-21	swapper/1	20:30:16	1
28020	99	83	82	cyclictest	0	-21	swapper/1	20:25:21	1
28020	99	83	82	cyclictest	0	-21	swapper/1	20:20:21	1
28020	99	83	82	cyclictest	0	-21	swapper/1	20:15:14	1
28020	99	83	82	cyclictest	0	-21	swapper/1	20:05:21	1
28020	99	83	82	cyclictest	0	-21	swapper/1	20:00:19	1
28020	99	83	82	cyclictest	0	-21	swapper/1	19:55:23	1
28020	99	83	82	cyclictest	0	-21	swapper/1	19:50:15	1
28020	99	83	82	cyclictest	0	-21	swapper/1	19:40:15	1
28020	99	83	82	cyclictest	0	-21	swapper/1	19:40:15	1
28020	99	83	82	cyclictest	0	-21	swapper/1	19:35:17	1
28020	99	83	82	cyclictest	0	-21	swapper/1	19:25:18	1
28020	99	83	82	cyclictest	0	-21	swapper/1	19:25:18	1
28020	99	83	82	cyclictest	0	-21	swapper/1	19:10:24	1
28020	99	83	82	cyclictest	0	-21	swapper/1	19:10:03	1
28020	99	83	82	cyclictest	0	-21	swapper/1	00:35:05	1
28020	99	83	82	cyclictest	0	-21	swapper/1	00:30:13	1
28020	99	83	82	cyclictest	0	-21	swapper/1	00:20:14	1
28020	99	83	82	cyclictest	0	-21	swapper/1	00:15:14	1
28020	99	83	82	cyclictest	0	-21	swapper/1	00:10:05	1
28020	99	83	82	cyclictest	0	-21	swapper/1	00:05:04	1
28020	99	83	82	cyclictest	0	-21	swapper/1	00:00:15	1
28020	99	83	81	cyclictest	4854	-21	tune2fs	23:15:08	1
28020	99	83	81	cyclictest	4836	-21	tune2fs	22:00:12	1
28020	99	83	81	cyclictest	22164	-21	tune2fs	20:10:19	1
28022	99	82	81	cyclictest	0	-21	swapper/3	23:10:09	3
28022	99	82	81	cyclictest	0	-21	swapper/3	21:50:12	3
28022	99	82	81	cyclictest	0	-21	swapper/3	21:09:58	3
28022	99	82	81	cyclictest	0	-21	swapper/3	19:20:02	3
28020	99	76	72	cyclictest	0	-21	swapper/1	19:20:15	1
28022	99	73	72	cyclictest	0	-21	swapper/3	23:20:07	3
28022	99	73	72	cyclictest	0	-21	swapper/3	22:35:10	3
28022	99	73	72	cyclictest	0	-21	swapper/3	00:15:14	3
28020	99	73	72	cyclictest	0	-21	swapper/1	22:50:05	1
28022	99	72	71	cyclictest	0	-21	swapper/3	23:45:16	3
28022	99	72	71	cyclictest	0	-21	swapper/3	20:14:49	3
28022	99	72	71	cyclictest	0	-21	swapper/3	19:55:23	3
28022	99	66	41	cyclictest	0	-21	swapper/3	21:30:10	3
28022	99	65	62	cyclictest	0	-21	swapper/3	21:23:43	3
28022	99	63	62	cyclictest	0	-21	swapper/3	23:30:04	3
28022	99	63	62	cyclictest	0	-21	swapper/3	23:05:08	3
28022	99	63	62	cyclictest	0	-21	swapper/3	22:05:06	3
28022	99	63	62	cyclictest	0	-21	swapper/3	21:00:14	3
28022	99	63	62	cyclictest	0	-21	swapper/3	20:50:14	3
28022	99	63	62	cyclictest	0	-21	swapper/3	20:45:16	3
28022	99	63	62	cyclictest	0	-21	swapper/3	20:15:15	3
28022	99	63	62	cyclictest	0	-21	swapper/3	20:05:14	3
28022	99	63	62	cyclictest	0	-21	swapper/3	20:00:11	3
28022	99	63	62	cyclictest	0	-21	swapper/3	19:40:02	3
28022	99	63	62	cyclictest	0	-21	swapper/3	19:40:02	3
28022	99	63	62	cyclictest	0	-21	swapper/3	19:20:15	3
28022	99	63	62	cyclictest	0	-21	swapper/3	19:10:03	3
28022	99	63	62	cyclictest	0	-21	swapper/3	00:00:05	3
28022	99	63	51	cyclictest	0	-21	swapper/3	00:20:03	3
28022	99	63	41	cyclictest	0	-21	swapper/3	23:25:05	3
28021	99	62	61	cyclictest	0	-21	swapper/2	23:27:23	2
28019	99	62	61	cyclictest	0	-21	swapper/0	19:30:14	0
28021	99	61	60	cyclictest	0	-21	swapper/2	19:20:35	2
28021	99	59	58	cyclictest	0	-21	swapper/2	21:37:45	2
28021	99	59	47	cyclictest	0	-21	swapper/2	19:26:36	2
28021	99	59	47	cyclictest	0	-21	swapper/2	19:26:36	2
28019	99	58	51	cyclictest	0	-21	swapper/0	00:06:07	0
28021	99	57	41	cyclictest	0	-21	swapper/2	21:42:33	2
28019	99	57	50	cyclictest	0	-21	swapper/0	22:07:00	0
28021	99	56	51	cyclictest	0	-21	swapper/2	19:35:43	2
28021	99	56	50	cyclictest	0	-21	swapper/2	00:11:25	2
28021	99	56	45	cyclictest	0	-21	swapper/2	23:34:32	2
28021	99	55	53	cyclictest	50	50	irq/44-ahci	22:09:10	2
28019	99	55	48	cyclictest	0	-21	swapper/0	22:20:29	0
28021	99	54	53	cyclictest	32412	-21	telnet	21:50:10	2
28021	99	54	53	cyclictest	0	-21	swapper/2	23:46:00	2
28021	99	54	53	cyclictest	0	-21	swapper/2	21:20:25	2
28021	99	54	52	cyclictest	50	50	irq/44-ahci	23:56:39	2
28021	99	54	51	cyclictest	0	-21	swapper/2	23:24:43	2
28021	99	54	51	cyclictest	0	-21	swapper/2	20:55:16	2
28021	99	54	51	cyclictest	0	-21	swapper/2	00:27:10	2
28021	99	54	49	cyclictest	0	-21	swapper/2	00:15:51	2
28021	99	54	48	cyclictest	0	-21	swapper/2	22:21:23	2
28019	99	54	53	cyclictest	11689	-21	kworker/0:2	23:39:30	0
28019	99	54	53	cyclictest	0	-21	swapper/0	20:51:32	0
28021	99	53	52	cyclictest	50	50	irq/44-ahci	19:45:15	2
28021	99	53	52	cyclictest	50	50	irq/44-ahci	00:04:57	2
28021	99	53	52	cyclictest	29346	-21	kworker/2:2	21:55:28	2
28021	99	53	52	cyclictest	278	-21	usb-storage	22:42:00	2
28021	99	53	52	cyclictest	278	-21	usb-storage	22:15:45	2
28021	99	53	52	cyclictest	278	-21	usb-storage	21:46:04	2
28021	99	53	52	cyclictest	278	-21	usb-storage	21:01:28	2
28021	99	53	52	cyclictest	278	-21	usb-storage	20:14:20	2
28021	99	53	52	cyclictest	278	-21	usb-storage	19:31:01	2
28021	99	53	52	cyclictest	18	1	ksoftirqd/2	21:26:43	2
28021	99	53	52	cyclictest	0	-21	swapper/2	23:40:12	2
28021	99	53	52	cyclictest	0	-21	swapper/2	23:15:04	2
28021	99	53	52	cyclictest	0	-21	swapper/2	23:05:05	2
28021	99	53	52	cyclictest	0	-21	swapper/2	23:01:40	2
28021	99	53	52	cyclictest	0	-21	swapper/2	22:55:08	2
28021	99	53	52	cyclictest	0	-21	swapper/2	22:45:06	2
28021	99	53	52	cyclictest	0	-21	swapper/2	22:39:14	2
28021	99	53	52	cyclictest	0	-21	swapper/2	22:30:35	2
28021	99	53	52	cyclictest	0	-21	swapper/2	22:25:31	2
28021	99	53	52	cyclictest	0	-21	swapper/2	22:25:31	2
28021	99	53	52	cyclictest	0	-21	swapper/2	22:11:23	2
28021	99	53	52	cyclictest	0	-21	swapper/2	22:00:07	2
28021	99	53	52	cyclictest	0	-21	swapper/2	21:15:10	2
28021	99	53	52	cyclictest	0	-21	swapper/2	21:05:12	2
28021	99	53	52	cyclictest	0	-21	swapper/2	20:50:38	2
28021	99	53	52	cyclictest	0	-21	swapper/2	20:49:35	2
28021	99	53	52	cyclictest	0	-21	swapper/2	20:30:10	2
28021	99	53	52	cyclictest	0	-21	swapper/2	20:20:12	2
28021	99	53	52	cyclictest	0	-21	swapper/2	20:08:44	2
28021	99	53	52	cyclictest	0	-21	swapper/2	20:02:06	2
28021	99	53	52	cyclictest	0	-21	swapper/2	19:59:34	2
28021	99	53	52	cyclictest	0	-21	swapper/2	19:43:35	2
28021	99	53	52	cyclictest	0	-21	swapper/2	19:43:35	2
28021	99	53	52	cyclictest	0	-21	swapper/2	19:14:28	2
28021	99	53	52	cyclictest	0	-21	swapper/2	00:04:45	2
28021	99	53	51	cyclictest	50	50	irq/44-ahci	23:34:54	2
28021	99	53	51	cyclictest	50	50	irq/44-ahci	20:40:23	2
28021	99	53	51	cyclictest	50	50	irq/44-ahci	20:35:22	2
28021	99	53	51	cyclictest	30351	-21	telnet	19:15:15	2
28021	99	53	51	cyclictest	19242	-21	telnet	23:50:05	2
28021	99	53	41	cyclictest	0	-21	swapper/2	23:11:05	2
28021	99	53	41	cyclictest	0	-21	swapper/2	22:50:06	2
28019	99	53	52	cyclictest	760	50	irq/18-parport0	23:40:10	0
28019	99	53	52	cyclictest	760	50	irq/18-parport0	21:51:34	0
28019	99	53	52	cyclictest	760	50	irq/18-parport0	19:27:50	0
28019	99	53	52	cyclictest	760	50	irq/18-parport0	19:27:50	0
28019	99	53	52	cyclictest	760	50	irq/18-parport0	00:25:30	0
28019	99	53	52	cyclictest	3822	-21	kworker/0:0	21:20:25	0
28019	99	53	52	cyclictest	3	1	ksoftirqd/0	20:59:40	0
28019	99	53	52	cyclictest	3	1	ksoftirqd/0	00:01:01	0
28019	99	53	52	cyclictest	278	-21	usb-storage	22:30:49	0
28019	99	53	52	cyclictest	278	-21	usb-storage	20:28:51	0
28019	99	53	52	cyclictest	0	-21	swapper/0	23:56:39	0
28019	99	53	52	cyclictest	0	-21	swapper/0	23:31:52	0
28019	99	53	52	cyclictest	0	-21	swapper/0	23:17:19	0
28019	99	53	52	cyclictest	0	-21	swapper/0	23:13:17	0
28019	99	53	52	cyclictest	0	-21	swapper/0	22:55:42	0
28019	99	53	52	cyclictest	0	-21	swapper/0	22:49:56	0
28019	99	53	52	cyclictest	0	-21	swapper/0	22:45:06	0
28019	99	53	52	cyclictest	0	-21	swapper/0	22:39:18	0
28019	99	53	52	cyclictest	0	-21	swapper/0	21:55:06	0
28019	99	53	52	cyclictest	0	-21	swapper/0	21:15:26	0
28019	99	53	52	cyclictest	0	-21	swapper/0	20:38:15	0
28019	99	53	52	cyclictest	0	-21	swapper/0	20:12:46	0
28019	99	53	52	cyclictest	0	-21	swapper/0	19:43:12	0
28019	99	53	52	cyclictest	0	-21	swapper/0	19:43:12	0
28019	99	53	52	cyclictest	0	-21	swapper/0	19:35:59	0
28019	99	53	51	cyclictest	50	50	irq/44-ahci	23:49:57	0
28019	99	53	51	cyclictest	50	50	irq/44-ahci	22:44:10	0
28019	99	53	51	cyclictest	50	50	irq/44-ahci	21:28:11	0
28019	99	53	51	cyclictest	30234	-21	telnet	20:30:10	0
28019	99	53	51	cyclictest	26027	-21	telnet	20:20:12	0
28019	99	53	51	cyclictest	23693	-21	telnet	21:30:09	0
28019	99	53	51	cyclictest	11689	-21	kworker/0:2	23:45:18	0
28019	99	53	41	cyclictest	50	50	irq/44-ahci	00:22:25	0
28019	99	53	41	cyclictest	0	-21	swapper/0	21:49:34	0
28021	99	52	51	cyclictest	278	-21	usb-storage	20:15:40	2
28021	99	52	51	cyclictest	278	-21	usb-storage	00:34:52	2
28021	99	52	51	cyclictest	0	-21	swapper/2	21:30:09	2
28021	99	52	51	cyclictest	0	-21	swapper/2	21:10:06	2
28021	99	52	51	cyclictest	0	-21	swapper/2	20:26:27	2
28021	99	52	51	cyclictest	0	-21	swapper/2	19:50:43	2
28021	99	52	51	cyclictest	0	-21	swapper/2	00:30:26	2
28021	99	52	51	cyclictest	0	-21	swapper/2	00:20:01	2
28019	99	52	51	cyclictest	760	50	irq/18-parport0	19:56:03	0
28019	99	52	51	cyclictest	50	50	irq/44-ahci	21:10:49	0
28019	99	52	51	cyclictest	50	50	irq/44-ahci	20:15:50	0
28019	99	52	51	cyclictest	278	-21	usb-storage	00:11:57	0
28019	99	52	51	cyclictest	0	-21	swapper/0	23:25:05	0
28019	99	52	51	cyclictest	0	-21	swapper/0	23:20:12	0
28019	99	52	51	cyclictest	0	-21	swapper/0	23:05:05	0
28019	99	52	51	cyclictest	0	-21	swapper/0	23:00:00	0
28019	99	52	51	cyclictest	0	-21	swapper/0	22:25:08	0
28019	99	52	51	cyclictest	0	-21	swapper/0	22:25:08	0
28019	99	52	51	cyclictest	0	-21	swapper/0	22:15:05	0
28019	99	52	51	cyclictest	0	-21	swapper/0	22:10:53	0
28019	99	52	51	cyclictest	0	-21	swapper/0	22:00:07	0
28019	99	52	51	cyclictest	0	-21	swapper/0	21:40:45	0
28019	99	52	51	cyclictest	0	-21	swapper/0	21:35:30	0
28019	99	52	51	cyclictest	0	-21	swapper/0	21:05:12	0
28019	99	52	51	cyclictest	0	-21	swapper/0	21:00:12	0
28019	99	52	51	cyclictest	0	-21	swapper/0	20:46:35	0
28019	99	52	51	cyclictest	0	-21	swapper/0	20:40:12	0
28019	99	52	51	cyclictest	0	-21	swapper/0	20:05:42	0
28019	99	52	51	cyclictest	0	-21	swapper/0	20:03:36	0
28019	99	52	51	cyclictest	0	-21	swapper/0	19:45:11	0
28019	99	52	51	cyclictest	0	-21	swapper/0	19:21:52	0
28019	99	52	51	cyclictest	0	-21	swapper/0	19:15:26	0
28019	99	52	51	cyclictest	0	-21	swapper/0	19:10:04	0
28019	99	52	51	cyclictest	0	-21	swapper/0	00:35:42	0
28019	99	52	51	cyclictest	0	-21	swapper/0	00:30:28	0
28019	99	52	41	cyclictest	0	-21	swapper/0	00:15:30	0
28019	99	51	41	cyclictest	0	-21	swapper/0	19:53:35	0
28021	99	33	32	cyclictest	28047	-21	latency_hist	19:10:03	2
28019	99	32	31	cyclictest	0	-21	swapper/0	19:10:03	0
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17