OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #3, slot #1

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance

Characteristics of the 792 highest latencies:

System rack3slot1.osadl.org (updated Sun Apr 28, 2024 00:43:25)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
2986	99	123	122	cyclictest	0	-21	swapper/3	20:28:53	3
2984	99	109	103	cyclictest	0	-21	swapper/1	21:38:56	1
2986	99	108	92	cyclictest	0	-21	swapper/3	21:33:52	3
2986	99	107	92	cyclictest	0	-21	swapper/3	22:19:02	3
2986	99	106	102	cyclictest	0	-21	swapper/3	22:28:58	3
2986	99	104	102	cyclictest	7516	-21	tune2fs	21:48:54	3
2984	99	104	102	cyclictest	5704	-21	tune2fs	22:58:47	1
2986	99	101	95	cyclictest	0	-21	swapper/3	19:58:54	3
2986	99	96	92	cyclictest	0	-21	swapper/3	23:01:11	3
2986	99	96	92	cyclictest	0	-21	swapper/3	19:23:51	3
2986	99	96	82	cyclictest	0	-21	swapper/3	20:33:52	3
2986	99	95	92	cyclictest	0	-21	swapper/3	18:35:22	3
2984	99	94	92	cyclictest	13544	-21	tune2fs	20:48:54	1
2984	99	94	92	cyclictest	0	-21	swapper/1	21:48:54	1
2986	99	90	82	cyclictest	0	-21	swapper/3	19:54:02	3
2986	99	90	82	cyclictest	0	-21	swapper/3	17:59:11	3
2986	99	88	82	cyclictest	1388	-13	audispd	18:08:45	3
2986	99	88	82	cyclictest	0	-21	swapper/3	23:08:56	3
2986	99	88	82	cyclictest	0	-21	swapper/3	22:53:49	3
2986	99	88	82	cyclictest	0	-21	swapper/3	22:03:54	3
2986	99	88	82	cyclictest	0	-21	swapper/3	21:43:52	3
2986	99	88	82	cyclictest	0	-21	swapper/3	19:23:37	3
2986	99	88	82	cyclictest	0	-21	swapper/3	18:14:01	3
2986	99	88	72	cyclictest	0	-21	swapper/3	22:44:04	3
2986	99	88	72	cyclictest	0	-21	swapper/3	22:38:53	3
2986	99	88	72	cyclictest	0	-21	swapper/3	21:28:56	3
2986	99	88	72	cyclictest	0	-21	swapper/3	20:43:54	3
2986	99	88	72	cyclictest	0	-21	swapper/3	19:49:03	3
2986	99	88	72	cyclictest	0	-21	swapper/3	19:08:55	3
2986	99	88	72	cyclictest	0	-21	swapper/3	18:49:07	3
2986	99	88	72	cyclictest	0	-21	swapper/3	18:39:01	3
2986	99	87	82	cyclictest	0	-21	swapper/3	23:03:49	3
2986	99	87	82	cyclictest	0	-21	swapper/3	22:48:55	3
2986	99	87	82	cyclictest	0	-21	swapper/3	22:37:26	3
2986	99	87	82	cyclictest	0	-21	swapper/3	22:23:48	3
2986	99	87	82	cyclictest	0	-21	swapper/3	22:08:51	3
2986	99	87	82	cyclictest	0	-21	swapper/3	21:58:49	3
2986	99	87	82	cyclictest	0	-21	swapper/3	21:38:56	3
2986	99	87	82	cyclictest	0	-21	swapper/3	21:13:51	3
2986	99	87	82	cyclictest	0	-21	swapper/3	21:13:38	3
2986	99	87	82	cyclictest	0	-21	swapper/3	20:58:55	3
2986	99	87	82	cyclictest	0	-21	swapper/3	20:23:52	3
2986	99	87	82	cyclictest	0	-21	swapper/3	20:18:53	3
2986	99	87	82	cyclictest	0	-21	swapper/3	19:44:09	3
2986	99	87	82	cyclictest	0	-21	swapper/3	19:38:58	3
2986	99	87	82	cyclictest	0	-21	swapper/3	19:34:13	3
2986	99	87	82	cyclictest	0	-21	swapper/3	19:03:59	3
2986	99	87	82	cyclictest	0	-21	swapper/3	18:54:11	3
2986	99	87	82	cyclictest	0	-21	swapper/3	18:43:57	3
2986	99	87	82	cyclictest	0	-21	swapper/3	18:04:00	3
2986	99	87	82	cyclictest	0	-21	swapper/3	17:54:10	3
2986	99	87	82	cyclictest	0	-21	swapper/3	17:48:56	3
2986	99	87	72	cyclictest	0	-21	swapper/3	18:59:09	3
2986	99	87	72	cyclictest	0	-21	swapper/3	18:30:46	3
2986	99	86	82	cyclictest	0	-21	swapper/3	23:13:46	3
2986	99	86	82	cyclictest	0	-21	swapper/3	22:13:46	3
2986	99	86	82	cyclictest	0	-21	swapper/3	21:53:47	3
2986	99	86	82	cyclictest	0	-21	swapper/3	21:24:00	3
2986	99	86	82	cyclictest	0	-21	swapper/3	21:18:59	3
2986	99	86	82	cyclictest	0	-21	swapper/3	21:03:55	3
2986	99	86	82	cyclictest	0	-21	swapper/3	20:38:56	3
2986	99	86	82	cyclictest	0	-21	swapper/3	20:14:10	3
2986	99	86	82	cyclictest	0	-21	swapper/3	20:03:55	3
2986	99	86	82	cyclictest	0	-21	swapper/3	19:28:56	3
2986	99	86	82	cyclictest	0	-21	swapper/3	19:18:43	3
2986	99	86	82	cyclictest	0	-21	swapper/3	18:24:02	3
2986	99	86	81	cyclictest	0	-21	swapper/3	20:53:53	3
2984	99	86	84	cyclictest	15850	-21	tune2fs	20:53:52	1
2986	99	85	82	cyclictest	0	-21	swapper/3	20:49:04	3
2986	99	85	82	cyclictest	0	-21	swapper/3	20:08:59	3
2986	99	85	71	cyclictest	0	-21	swapper/3	18:23:43	3
2984	99	84	82	cyclictest	12088	-21	tune2fs	23:13:49	1
2984	99	83	82	cyclictest	0	-21	swapper/1	23:03:49	1
2984	99	83	82	cyclictest	0	-21	swapper/1	22:48:55	1
2984	99	83	82	cyclictest	0	-21	swapper/1	22:44:01	1
2984	99	83	82	cyclictest	0	-21	swapper/1	22:28:48	1
2984	99	83	82	cyclictest	0	-21	swapper/1	22:14:16	1
2984	99	83	82	cyclictest	0	-21	swapper/1	22:08:55	1
2984	99	83	82	cyclictest	0	-21	swapper/1	22:03:58	1
2984	99	83	82	cyclictest	0	-21	swapper/1	21:53:59	1
2984	99	83	82	cyclictest	0	-21	swapper/1	21:33:56	1
2984	99	83	82	cyclictest	0	-21	swapper/1	21:24:06	1
2984	99	83	82	cyclictest	0	-21	swapper/1	20:59:01	1
2984	99	83	82	cyclictest	0	-21	swapper/1	20:38:55	1
2984	99	83	82	cyclictest	0	-21	swapper/1	20:34:10	1
2984	99	83	82	cyclictest	0	-21	swapper/1	20:28:50	1
2984	99	83	82	cyclictest	0	-21	swapper/1	20:19:04	1
2984	99	83	82	cyclictest	0	-21	swapper/1	20:13:59	1
2984	99	83	82	cyclictest	0	-21	swapper/1	20:08:59	1
2984	99	83	82	cyclictest	0	-21	swapper/1	20:04:05	1
2984	99	83	82	cyclictest	0	-21	swapper/1	19:54:04	1
2984	99	83	82	cyclictest	0	-21	swapper/1	19:49:10	1
2984	99	83	82	cyclictest	0	-21	swapper/1	19:44:07	1
2984	99	83	82	cyclictest	0	-21	swapper/1	19:33:53	1
2984	99	83	82	cyclictest	0	-21	swapper/1	19:23:58	1
2984	99	83	82	cyclictest	0	-21	swapper/1	19:09:07	1
2984	99	83	82	cyclictest	0	-21	swapper/1	19:03:59	1
2984	99	83	82	cyclictest	0	-21	swapper/1	18:54:11	1
2984	99	83	82	cyclictest	0	-21	swapper/1	18:48:47	1
2984	99	83	82	cyclictest	0	-21	swapper/1	18:44:05	1
2984	99	83	82	cyclictest	0	-21	swapper/1	18:24:04	1
2984	99	83	82	cyclictest	0	-21	swapper/1	18:18:42	1
2984	99	83	82	cyclictest	0	-21	swapper/1	18:08:45	1
2984	99	83	82	cyclictest	0	-21	swapper/1	18:04:09	1
2984	99	83	82	cyclictest	0	-21	swapper/1	17:59:13	1
2984	99	83	82	cyclictest	0	-21	swapper/1	17:49:04	1
2984	99	83	81	cyclictest	9984	-21	tune2fs	23:08:50	1
2984	99	83	81	cyclictest	7114	-21	tune2fs	19:18:58	1
2984	99	83	81	cyclictest	26801	-21	tune2fs	21:18:57	1
2984	99	83	81	cyclictest	24706	-21	tune2fs	21:13:52	1
2984	99	83	81	cyclictest	21850	-21	tune2fs	18:39:01	1
2984	99	83	81	cyclictest	20310	-21	tune2fs	22:18:51	1
2984	99	83	81	cyclictest	15158	-21	aten2.4-expect	19:38:51	1
2984	99	82	81	cyclictest	0	-21	swapper/1	20:43:50	1
2984	99	73	72	cyclictest	0	-21	swapper/1	19:01:11	1
2984	99	72	71	cyclictest	0	-21	swapper/1	21:29:09	1
2984	99	72	71	cyclictest	0	-21	swapper/1	17:54:03	1
2986	99	67	51	cyclictest	0	-21	swapper/3	17:48:45	3
2984	99	65	62	cyclictest	0	-21	swapper/1	22:53:58	1
2984	99	64	63	cyclictest	5420	-21	tune2fs	21:43:52	1
2984	99	64	62	cyclictest	0	-21	swapper/1	22:34:19	1
2984	99	64	51	cyclictest	0	-21	swapper/1	20:23:50	1
2984	99	63	62	cyclictest	0	-21	swapper/1	22:38:47	1
2984	99	63	62	cyclictest	0	-21	swapper/1	22:23:46	1
2984	99	63	62	cyclictest	0	-21	swapper/1	21:58:47	1
2984	99	63	62	cyclictest	0	-21	swapper/1	21:08:49	1
2984	99	63	62	cyclictest	0	-21	swapper/1	19:58:50	1
2984	99	63	62	cyclictest	0	-21	swapper/1	19:28:52	1
2984	99	63	62	cyclictest	0	-21	swapper/1	19:13:50	1
2984	99	63	62	cyclictest	0	-21	swapper/1	18:33:44	1
2984	99	63	62	cyclictest	0	-21	swapper/1	18:29:30	1
2984	99	63	62	cyclictest	0	-21	swapper/1	18:22:14	1
2984	99	63	61	cyclictest	19704	-21	aten2.4-expect	21:03:48	1
2985	99	62	61	cyclictest	0	-21	swapper/2	21:38:46	2
2985	99	62	60	cyclictest	32261	-21	telnet	19:03:55	2
2985	99	61	60	cyclictest	19737	-21	telnet	21:03:49	2
2983	99	57	41	cyclictest	0	-21	swapper/0	23:00:15	0
2985	99	55	54	cyclictest	8105	-21	munin-node	23:03:54	2
2985	99	55	54	cyclictest	4912	-21	latency_hist	22:58:34	2
2985	99	55	54	cyclictest	16125	-21	munin-node	19:38:59	2
2985	99	55	54	cyclictest	13522	-21	munin-node	20:48:54	2
2985	99	55	54	cyclictest	13335	-21	sshd	23:16:13	2
2985	99	55	54	cyclictest	11699	-21	iostat	19:29:06	2
2985	99	55	53	cyclictest	18	1	ksoftirqd/2	21:26:01	2
2985	99	55	51	cyclictest	0	-21	swapper/2	22:39:12	2
2983	99	55	41	cyclictest	0	-21	swapper/0	18:54:18	0
2985	99	54	53	cyclictest	6760	-21	munin-node	19:18:57	2
2985	99	54	53	cyclictest	6618	-21	latency_hist	20:33:40	2
2985	99	54	53	cyclictest	6119	-21	missed_timers	22:58:51	2
2985	99	54	53	cyclictest	5938	-21	munin-node	21:43:58	2
2985	99	54	53	cyclictest	32220	-21	sshd	21:29:43	2
2985	99	54	53	cyclictest	27652	-21	munin-node	22:33:56	2
2985	99	54	53	cyclictest	24369	-21	munin-node	21:13:51	2
2985	99	54	53	cyclictest	21895	-21	fschecks_time	18:39:01	2
2985	99	54	53	cyclictest	19379	-21	munin-node	22:14:02	2
2985	99	54	53	cyclictest	18435	-21	iostat	20:58:54	2
2985	99	54	53	cyclictest	17926	-21	munin-node	18:24:01	2
2985	99	54	53	cyclictest	14676	-21	munin-node	22:03:56	2
2985	99	54	53	cyclictest	11704	-21	latency_hist	18:08:45	2
2985	99	54	41	cyclictest	275	-21	usb-storage	21:21:57	2
2985	99	54	41	cyclictest	0	-21	swapper/2	22:13:19	2
2983	99	54	53	cyclictest	0	-21	swapper/0	23:17:41	0
2983	99	54	42	cyclictest	745	50	irq/18-parport0	20:53:38	0
2983	99	54	21	cyclictest	0	-21	swapper/0	22:26:17	0
2985	99	53	52	cyclictest	27697	-21	sshd	20:05:47	2
2985	99	53	52	cyclictest	24466	-21	munin-node	22:28:46	2
2985	99	53	52	cyclictest	18	1	ksoftirqd/2	21:09:13	2
2985	99	53	52	cyclictest	18	1	ksoftirqd/2	18:49:21	2
2985	99	53	52	cyclictest	0	-21	swapper/2	23:08:45	2
2985	99	53	52	cyclictest	0	-21	swapper/2	21:48:49	2
2985	99	53	52	cyclictest	0	-21	swapper/2	20:55:35	2
2985	99	53	52	cyclictest	0	-21	swapper/2	20:38:49	2
2985	99	53	52	cyclictest	0	-21	swapper/2	20:23:49	2
2985	99	53	52	cyclictest	0	-21	swapper/2	19:58:50	2
2985	99	53	52	cyclictest	0	-21	swapper/2	19:34:03	2
2985	99	53	52	cyclictest	0	-21	swapper/2	18:58:55	2
2985	99	53	52	cyclictest	0	-21	swapper/2	17:58:57	2
2985	99	53	51	cyclictest	6741	-21	telnet	20:33:52	2
2985	99	53	51	cyclictest	3204	-21	telnet	17:48:53	2
2985	99	53	51	cyclictest	275	-21	usb-storage	22:49:48	2
2985	99	53	51	cyclictest	275	-21	usb-storage	22:19:55	2
2985	99	53	51	cyclictest	275	-21	usb-storage	21:35:54	2
2985	99	53	51	cyclictest	275	-21	usb-storage	20:13:40	2
2985	99	53	51	cyclictest	275	-21	usb-storage	19:27:01	2
2985	99	53	51	cyclictest	17403	-21	telnet	19:43:54	2
2985	99	53	42	cyclictest	0	-21	swapper/2	18:43:59	2
2985	99	53	41	cyclictest	29055	-21	open_files	18:54:07	2
2985	99	53	41	cyclictest	275	-21	usb-storage	19:49:28	2
2985	99	53	41	cyclictest	0	-21	swapper/2	21:58:47	2
2985	99	53	41	cyclictest	0	-21	swapper/2	21:56:16	2
2985	99	53	41	cyclictest	0	-21	swapper/2	19:17:38	2
2985	99	53	41	cyclictest	0	-21	swapper/2	18:33:52	2
2985	99	53	41	cyclictest	0	-21	swapper/2	18:31:47	2
2985	99	53	41	cyclictest	0	-21	swapper/2	18:22:48	2
2985	99	53	41	cyclictest	0	-21	swapper/2	18:15:54	2
2983	99	53	52	cyclictest	745	50	irq/18-parport0	23:11:01	0
2983	99	53	52	cyclictest	745	50	irq/18-parport0	21:50:48	0
2983	99	53	52	cyclictest	745	50	irq/18-parport0	20:29:05	0
2983	99	53	52	cyclictest	745	50	irq/18-parport0	19:39:27	0
2983	99	53	52	cyclictest	745	50	irq/18-parport0	18:24:42	0
2983	99	53	52	cyclictest	0	-21	swapper/0	23:03:39	0
2983	99	53	52	cyclictest	0	-21	swapper/0	22:43:44	0
2983	99	53	52	cyclictest	0	-21	swapper/0	22:33:40	0
2983	99	53	52	cyclictest	0	-21	swapper/0	21:21:09	0
2983	99	53	52	cyclictest	0	-21	swapper/0	21:12:31	0
2983	99	53	52	cyclictest	0	-21	swapper/0	20:53:48	0
2983	99	53	52	cyclictest	0	-21	swapper/0	19:26:09	0
2983	99	53	52	cyclictest	0	-21	swapper/0	18:51:35	0
2983	99	53	52	cyclictest	0	-21	swapper/0	18:48:17	0
2983	99	53	51	cyclictest	50	50	irq/44-ahci	22:30:34	0
2983	99	53	51	cyclictest	50	50	irq/44-ahci	17:59:23	0
2983	99	53	41	cyclictest	0	-21	swapper/0	22:58:32	0
2983	99	53	41	cyclictest	0	-21	swapper/0	22:51:22	0
2983	99	53	41	cyclictest	0	-21	swapper/0	22:39:08	0
2983	99	53	41	cyclictest	0	-21	swapper/0	22:16:07	0
2983	99	53	41	cyclictest	0	-21	swapper/0	22:08:48	0
2983	99	53	41	cyclictest	0	-21	swapper/0	22:03:54	0
2983	99	53	41	cyclictest	0	-21	swapper/0	21:54:24	0
2983	99	53	41	cyclictest	0	-21	swapper/0	21:33:47	0
2983	99	53	41	cyclictest	0	-21	swapper/0	21:29:17	0
2983	99	53	41	cyclictest	0	-21	swapper/0	21:24:57	0
2983	99	53	41	cyclictest	0	-21	swapper/0	20:58:49	0
2983	99	53	41	cyclictest	0	-21	swapper/0	20:43:50	0
2983	99	53	41	cyclictest	0	-21	swapper/0	20:39:49	0
2983	99	53	41	cyclictest	0	-21	swapper/0	20:08:51	0
2983	99	53	41	cyclictest	0	-21	swapper/0	20:03:52	0
2983	99	53	41	cyclictest	0	-21	swapper/0	20:02:16	0
2983	99	53	41	cyclictest	0	-21	swapper/0	19:54:42	0
2983	99	53	41	cyclictest	0	-21	swapper/0	19:49:48	0
2983	99	53	41	cyclictest	0	-21	swapper/0	19:43:43	0
2983	99	53	41	cyclictest	0	-21	swapper/0	19:38:07	0
2983	99	53	41	cyclictest	0	-21	swapper/0	19:31:17	0
2983	99	53	41	cyclictest	0	-21	swapper/0	19:18:54	0
2983	99	53	41	cyclictest	0	-21	swapper/0	19:08:52	0
2983	99	53	41	cyclictest	0	-21	swapper/0	19:04:46	0
2983	99	53	41	cyclictest	0	-21	swapper/0	18:58:54	0
2983	99	53	41	cyclictest	0	-21	swapper/0	18:39:05	0
2983	99	53	41	cyclictest	0	-21	swapper/0	18:35:49	0
2983	99	53	41	cyclictest	0	-21	swapper/0	18:20:14	0
2983	99	53	41	cyclictest	0	-21	swapper/0	18:13:56	0
2983	99	53	41	cyclictest	0	-21	swapper/0	18:09:24	0
2983	99	53	41	cyclictest	0	-21	swapper/0	18:03:55	0
2983	99	53	41	cyclictest	0	-21	swapper/0	17:53:55	0
2983	99	53	41	cyclictest	0	-21	swapper/0	17:48:53	0
2985	99	52	51	cyclictest	2000	-21	telnet	19:08:52	2
2985	99	52	51	cyclictest	11850	-21	telnet	18:08:57	2
2985	99	52	51	cyclictest	0	-21	swapper/2	22:43:36	2
2985	99	52	51	cyclictest	0	-21	swapper/2	22:23:51	2
2985	99	52	51	cyclictest	0	-21	swapper/2	20:43:44	2
2985	99	52	51	cyclictest	0	-21	swapper/2	20:20:45	2
2985	99	52	51	cyclictest	0	-21	swapper/2	19:53:50	2
2985	99	52	51	cyclictest	0	-21	swapper/2	17:53:57	2
2985	99	52	41	cyclictest	0	-21	swapper/2	20:13:55	2
2984	99	52	51	cyclictest	0	-21	swapper/1	17:48:45	1
2983	99	52	51	cyclictest	745	50	irq/18-parport0	21:14:15	0
2983	99	52	51	cyclictest	745	50	irq/18-parport0	20:37:27	0
2983	99	52	51	cyclictest	1877	-21	hald-addon-stor	21:59:00	0
2983	99	52	51	cyclictest	0	-21	swapper/0	21:43:44	0
2983	99	52	51	cyclictest	0	-21	swapper/0	21:38:52	0
2983	99	52	51	cyclictest	0	-21	swapper/0	21:03:49	0
2983	99	52	51	cyclictest	0	-21	swapper/0	20:23:49	0
2983	99	52	51	cyclictest	0	-21	swapper/0	20:21:21	0
2983	99	52	41	cyclictest	0	-21	swapper/0	19:18:24	0
2983	99	52	41	cyclictest	0	-21	swapper/0	18:28:56	0
2983	99	51	50	cyclictest	0	-21	swapper/0	22:19:06	0
2983	99	50	49	cyclictest	0	-21	swapper/0	20:13:56	0
2985	99	43	41	cyclictest	3024	-21	latency_hist	17:48:45	2
2983	99	42	41	cyclictest	0	-21	swapper/0	17:48:45	0
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17