OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #3, slot #1

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance

Characteristics of the highest latencies:

System rack3slot1.osadl.org (updated Wed Jul 15, 2026 12:43:26)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
21159	99	103	102	cyclictest	392	-21	md2_raid1	04:04:00	3
21159	99	103	102	cyclictest	0	-21	swapper/3	07:29:08	3
21144	99	103	102	cyclictest	0	-21	swapper/1	04:04:00	1
21144	99	95	93	cyclictest	13503	-21	tune2fs	07:29:08	1
21159	99	94	89	cyclictest	0	-21	swapper/3	05:09:13	3
21159	99	93	92	cyclictest	0	-21	swapper/3	09:14:10	3
21159	99	93	92	cyclictest	0	-21	swapper/3	08:44:09	3
21144	99	91	81	cyclictest	13196	-21	tune2fs	06:14:11	1
21144	99	89	81	cyclictest	2501	-21	tune2fs	07:04:07	1
21144	99	89	81	cyclictest	22160	-21	tune2fs	07:49:09	1
21144	99	89	81	cyclictest	15328	-21	tune2fs	05:04:12	1
21144	99	89	81	cyclictest	11222	-21	tune2fs	08:39:05	1
21144	99	89	81	cyclictest	10964	-21	tune2fs	06:09:09	1
21159	99	88	82	cyclictest	0	-21	swapper/3	06:49:14	3
21159	99	88	82	cyclictest	0	-21	swapper/3	06:14:11	3
21159	99	88	82	cyclictest	0	-21	swapper/3	04:34:23	3
21144	99	88	82	cyclictest	0	-21	swapper/1	08:04:09	1
21144	99	88	82	cyclictest	0	-21	swapper/1	07:22:04	1
21144	99	88	82	cyclictest	0	-21	swapper/1	06:49:11	1
21144	99	88	82	cyclictest	0	-21	swapper/1	05:58:27	1
21144	99	88	82	cyclictest	0	-21	swapper/1	04:34:23	1
21159	99	87	82	cyclictest	0	-21	swapper/3	08:39:05	3
21159	99	87	82	cyclictest	0	-21	swapper/3	07:49:09	3
21159	99	87	82	cyclictest	0	-21	swapper/3	06:09:09	3
21159	99	87	71	cyclictest	0	-21	swapper/3	04:49:18	3
21144	99	87	82	cyclictest	0	-21	swapper/1	08:49:10	1
21144	99	87	82	cyclictest	0	-21	swapper/1	06:19:16	1
21159	99	86	82	cyclictest	0	-21	swapper/3	07:21:59	3
21159	99	86	81	cyclictest	0	-21	swapper/3	07:04:07	3
21159	99	86	81	cyclictest	0	-21	swapper/3	05:04:12	3
21144	99	86	81	cyclictest	4903	-21	tune2fs	07:09:09	1
21144	99	84	82	cyclictest	30293	-21	tune2fs	04:24:15	1
21144	99	84	82	cyclictest	24438	-21	tune2fs	09:09:04	1
21144	99	84	82	cyclictest	24438	-21	tune2fs	09:09:04	1
21159	99	83	82	cyclictest	6423	-21	latency_hist	08:28:51	3
21159	99	83	82	cyclictest	0	-21	swapper/3	09:27:23	3
21159	99	83	82	cyclictest	0	-21	swapper/3	09:09:04	3
21159	99	83	82	cyclictest	0	-21	swapper/3	09:09:04	3
21159	99	83	82	cyclictest	0	-21	swapper/3	08:14:04	3
21159	99	83	82	cyclictest	0	-21	swapper/3	05:54:17	3
21159	99	83	82	cyclictest	0	-21	swapper/3	04:14:22	3
21144	99	83	82	cyclictest	0	-21	swapper/1	09:21:15	1
21144	99	83	82	cyclictest	0	-21	swapper/1	05:59:20	1
21144	99	83	82	cyclictest	0	-21	swapper/1	04:29:23	1
21144	99	83	81	cyclictest	8667	-21	aten2.4-expect	08:34:02	1
21144	99	83	81	cyclictest	30614	-21	tune2fs	06:54:08	1
21144	99	83	81	cyclictest	24255	-21	tune2fs	07:54:05	1
21144	99	83	81	cyclictest	23823	-21	tune2fs	05:24:09	1
21144	99	83	81	cyclictest	21856	-21	tune2fs	06:34:10	1
21144	99	83	81	cyclictest	19955	-21	tune2fs	07:44:08	1
21144	99	83	81	cyclictest	15555	-21	tune2fs	07:34:08	1
21144	99	83	71	cyclictest	0	-21	swapper/1	06:33:55	1
21159	99	82	81	cyclictest	21041	-21	cyclictest	07:09:09	3
21159	99	82	81	cyclictest	0	-21	swapper/3	09:29:04	3
21159	99	82	81	cyclictest	0	-21	swapper/3	08:49:05	3
21159	99	82	81	cyclictest	0	-21	swapper/3	08:34:02	3
21159	99	82	81	cyclictest	0	-21	swapper/3	07:54:05	3
21159	99	82	81	cyclictest	0	-21	swapper/3	07:44:08	3
21159	99	82	81	cyclictest	0	-21	swapper/3	07:34:08	3
21159	99	82	81	cyclictest	0	-21	swapper/3	06:54:08	3
21159	99	82	81	cyclictest	0	-21	swapper/3	06:34:10	3
21159	99	82	81	cyclictest	0	-21	swapper/3	06:33:55	3
21159	99	82	81	cyclictest	0	-21	swapper/3	05:24:09	3
21159	99	82	81	cyclictest	0	-21	swapper/3	04:24:15	3
21159	99	80	79	cyclictest	27722	-21	latency_hist	05:33:57	3
21159	99	79	72	cyclictest	0	-21	swapper/3	08:23:51	3
21159	99	78	72	cyclictest	0	-21	swapper/3	07:59:11	3
21159	99	78	72	cyclictest	0	-21	swapper/3	04:09:26	3
21159	99	76	72	cyclictest	0	-21	swapper/3	09:19:12	3
21159	99	76	72	cyclictest	0	-21	swapper/3	06:39:12	3
21159	99	76	72	cyclictest	0	-21	swapper/3	05:39:12	3
21159	99	76	71	cyclictest	0	-21	swapper/3	06:04:07	3
21159	99	76	71	cyclictest	0	-21	swapper/3	05:59:20	3
21159	99	76	71	cyclictest	0	-21	swapper/3	04:54:20	3
21144	99	76	74	cyclictest	647	-21	tune2fs	09:29:04	1
21159	99	74	73	cyclictest	18631	-21	vmstat	06:24:19	3
21159	99	74	72	cyclictest	0	-21	swapper/3	05:14:14	3
21159	99	73	72	cyclictest	0	-21	swapper/3	09:03:59	3
21159	99	73	72	cyclictest	0	-21	swapper/3	08:58:53	3
21159	99	73	72	cyclictest	0	-21	swapper/3	08:53:59	3
21159	99	73	72	cyclictest	0	-21	swapper/3	08:29:02	3
21159	99	73	72	cyclictest	0	-21	swapper/3	08:09:03	3
21159	99	73	72	cyclictest	0	-21	swapper/3	08:04:05	3
21159	99	73	72	cyclictest	0	-21	swapper/3	07:39:04	3
21159	99	73	72	cyclictest	0	-21	swapper/3	07:24:02	3
21159	99	73	72	cyclictest	0	-21	swapper/3	07:14:05	3
21159	99	73	72	cyclictest	0	-21	swapper/3	06:59:07	3
21159	99	73	72	cyclictest	0	-21	swapper/3	06:44:08	3
21159	99	73	72	cyclictest	0	-21	swapper/3	06:19:08	3
21159	99	73	72	cyclictest	0	-21	swapper/3	05:49:09	3
21159	99	73	72	cyclictest	0	-21	swapper/3	05:44:12	3
21159	99	73	72	cyclictest	0	-21	swapper/3	05:34:08	3
21159	99	73	72	cyclictest	0	-21	swapper/3	05:19:09	3
21159	99	73	72	cyclictest	0	-21	swapper/3	04:59:10	3
21159	99	73	72	cyclictest	0	-21	swapper/3	04:44:11	3
21159	99	73	72	cyclictest	0	-21	swapper/3	04:39:10	3
21159	99	73	72	cyclictest	0	-21	swapper/3	04:29:12	3
21159	99	73	72	cyclictest	0	-21	swapper/3	04:19:10	3
21144	99	73	72	cyclictest	0	-21	swapper/1	09:24:01	1
21144	99	73	72	cyclictest	0	-21	swapper/1	09:14:00	1
21144	99	73	72	cyclictest	0	-21	swapper/1	09:04:00	1
21144	99	73	72	cyclictest	0	-21	swapper/1	08:59:03	1
21144	99	73	72	cyclictest	0	-21	swapper/1	08:54:04	1
21144	99	73	72	cyclictest	0	-21	swapper/1	08:44:02	1
21144	99	73	72	cyclictest	0	-21	swapper/1	08:29:03	1
21144	99	73	72	cyclictest	0	-21	swapper/1	08:24:05	1
21144	99	73	72	cyclictest	0	-21	swapper/1	08:19:02	1
21144	99	73	72	cyclictest	0	-21	swapper/1	08:14:04	1
21144	99	73	72	cyclictest	0	-21	swapper/1	08:09:07	1
21144	99	73	72	cyclictest	0	-21	swapper/1	07:59:07	1
21144	99	73	72	cyclictest	0	-21	swapper/1	07:39:07	1
21144	99	73	72	cyclictest	0	-21	swapper/1	07:24:04	1
21144	99	73	72	cyclictest	0	-21	swapper/1	07:14:06	1
21144	99	73	72	cyclictest	0	-21	swapper/1	06:59:07	1
21144	99	73	72	cyclictest	0	-21	swapper/1	06:44:08	1
21144	99	73	72	cyclictest	0	-21	swapper/1	06:39:09	1
21144	99	73	72	cyclictest	0	-21	swapper/1	06:24:07	1
21144	99	73	72	cyclictest	0	-21	swapper/1	06:04:07	1
21144	99	73	72	cyclictest	0	-21	swapper/1	05:49:11	1
21144	99	73	72	cyclictest	0	-21	swapper/1	05:44:11	1
21144	99	73	72	cyclictest	0	-21	swapper/1	05:39:11	1
21144	99	73	72	cyclictest	0	-21	swapper/1	05:34:11	1
21144	99	73	72	cyclictest	0	-21	swapper/1	05:29:11	1
21144	99	73	72	cyclictest	0	-21	swapper/1	05:19:11	1
21144	99	73	72	cyclictest	0	-21	swapper/1	05:14:11	1
21144	99	73	72	cyclictest	0	-21	swapper/1	05:09:12	1
21144	99	73	72	cyclictest	0	-21	swapper/1	04:59:12	1
21144	99	73	72	cyclictest	0	-21	swapper/1	04:54:15	1
21144	99	73	72	cyclictest	0	-21	swapper/1	04:49:16	1
21144	99	73	72	cyclictest	0	-21	swapper/1	04:44:12	1
21144	99	73	72	cyclictest	0	-21	swapper/1	04:39:11	1
21144	99	73	72	cyclictest	0	-21	swapper/1	04:19:14	1
21144	99	73	72	cyclictest	0	-21	swapper/1	04:14:13	1
21144	99	73	72	cyclictest	0	-21	swapper/1	04:09:14	1
21154	99	62	61	cyclictest	0	-21	swapper/2	05:04:09	2
21134	99	60	59	cyclictest	0	-21	swapper/0	08:59:02	0
21134	99	60	59	cyclictest	0	-21	swapper/0	07:14:05	0
21154	99	59	58	cyclictest	0	-21	swapper/2	07:24:02	2
21154	99	58	57	cyclictest	0	-21	swapper/2	06:04:07	2
21134	99	58	41	cyclictest	0	-21	swapper/0	06:10:29	0
21134	99	56	55	cyclictest	0	-21	swapper/0	08:39:02	0
21154	99	55	54	cyclictest	0	-21	swapper/2	09:29:02	2
21154	99	54	53	cyclictest	0	-21	swapper/2	04:48:04	2
21134	99	54	53	cyclictest	0	-21	swapper/0	04:49:12	0
21134	99	54	42	cyclictest	760	50	irq/18-parport0	05:21:01	0
21134	99	54	42	cyclictest	760	50	irq/18-parport0	05:10:34	0
21154	99	53	52	cyclictest	50	50	irq/44-ahci	08:24:42	2
21154	99	53	52	cyclictest	0	-21	swapper/2	08:39:02	2
21154	99	53	52	cyclictest	0	-21	swapper/2	08:09:03	2
21154	99	53	52	cyclictest	0	-21	swapper/2	06:59:07	2
21154	99	53	52	cyclictest	0	-21	swapper/2	06:44:07	2
21154	99	53	52	cyclictest	0	-21	swapper/2	06:39:07	2
21154	99	53	52	cyclictest	0	-21	swapper/2	06:36:02	2
21154	99	53	52	cyclictest	0	-21	swapper/2	05:59:48	2
21154	99	53	52	cyclictest	0	-21	swapper/2	05:49:11	2
21154	99	53	52	cyclictest	0	-21	swapper/2	05:38:58	2
21154	99	53	52	cyclictest	0	-21	swapper/2	05:19:09	2
21154	99	53	52	cyclictest	0	-21	swapper/2	04:59:12	2
21154	99	53	52	cyclictest	0	-21	swapper/2	04:54:10	2
21154	99	53	52	cyclictest	0	-21	swapper/2	04:49:12	2
21154	99	53	52	cyclictest	0	-21	swapper/2	04:04:12	2
21154	99	53	51	cyclictest	27974	-21	telnet	06:49:06	2
21154	99	53	51	cyclictest	23770	-21	telnet	07:54:03	2
21154	99	53	51	cyclictest	2073	-21	telnet	07:04:05	2
21154	99	53	51	cyclictest	19596	-21	telnet	08:59:02	2
21154	99	53	51	cyclictest	15159	-21	telnet	07:34:06	2
21154	99	53	51	cyclictest	12839	-21	telnet	06:14:09	2
21154	99	53	41	cyclictest	0	-21	swapper/2	08:54:02	2
21154	99	53	41	cyclictest	0	-21	swapper/2	04:24:13	2
21134	99	53	52	cyclictest	760	50	irq/18-parport0	07:07:17	0
21134	99	53	52	cyclictest	0	-21	swapper/0	09:29:32	0
21134	99	53	52	cyclictest	0	-21	swapper/0	06:02:46	0
21134	99	53	52	cyclictest	0	-21	swapper/0	05:04:09	0
21134	99	53	51	cyclictest	50	50	irq/44-ahci	06:54:24	0
21134	99	53	41	cyclictest	760	50	irq/18-parport0	06:27:57	0
21134	99	53	41	cyclictest	760	50	irq/18-parport0	05:41:38	0
21134	99	53	41	cyclictest	760	50	irq/18-parport0	05:37:14	0
21134	99	53	41	cyclictest	760	50	irq/18-parport0	04:55:30	0
21134	99	53	41	cyclictest	0	-21	swapper/0	07:51:25	0
21154	99	52	51	cyclictest	8730	-21	telnet	08:34:02	2
21154	99	52	51	cyclictest	32222	-21	telnet	05:44:11	2
21154	99	52	51	cyclictest	21693	-21	telnet	09:04:00	2
21154	99	52	51	cyclictest	1805	-21	telnet	04:34:11	2
21154	99	52	51	cyclictest	17033	-21	telnet	05:09:10	2
21154	99	52	51	cyclictest	14975	-21	aten2.4_r3power	06:19:07	2
21154	99	52	51	cyclictest	0	-21	swapper/2	09:26:06	2
21154	99	52	51	cyclictest	0	-21	swapper/2	09:20:48	2
21154	99	52	51	cyclictest	0	-21	swapper/2	09:09:01	2
21154	99	52	51	cyclictest	0	-21	swapper/2	09:09:01	2
21154	99	52	51	cyclictest	0	-21	swapper/2	08:52:49	2
21154	99	52	51	cyclictest	0	-21	swapper/2	08:30:04	2
21154	99	52	51	cyclictest	0	-21	swapper/2	08:21:54	2
21154	99	52	51	cyclictest	0	-21	swapper/2	08:14:31	2
21154	99	52	51	cyclictest	0	-21	swapper/2	07:52:33	2
21154	99	52	51	cyclictest	0	-21	swapper/2	07:32:38	2
21154	99	52	51	cyclictest	0	-21	swapper/2	07:14:05	2
21154	99	52	51	cyclictest	0	-21	swapper/2	07:09:43	2
21154	99	52	51	cyclictest	0	-21	swapper/2	06:55:22	2
21154	99	52	51	cyclictest	0	-21	swapper/2	06:32:42	2
21154	99	52	51	cyclictest	0	-21	swapper/2	06:27:03	2
21154	99	52	51	cyclictest	0	-21	swapper/2	05:34:07	2
21154	99	52	51	cyclictest	0	-21	swapper/2	05:32:41	2
21154	99	52	51	cyclictest	0	-21	swapper/2	04:19:07	2
21154	99	52	51	cyclictest	0	-21	swapper/2	04:14:12	2
21134	99	52	51	cyclictest	760	50	irq/18-parport0	05:29:03	0
21134	99	52	51	cyclictest	760	50	irq/18-parport0	04:11:10	0
21134	99	52	51	cyclictest	0	-21	swapper/0	09:23:59	0
21134	99	52	51	cyclictest	0	-21	swapper/0	09:19:01	0
21134	99	52	51	cyclictest	0	-21	swapper/0	09:10:00	0
21134	99	52	51	cyclictest	0	-21	swapper/0	09:10:00	0
21134	99	52	51	cyclictest	0	-21	swapper/0	09:04:00	0
21134	99	52	51	cyclictest	0	-21	swapper/0	08:54:02	0
21134	99	52	51	cyclictest	0	-21	swapper/0	08:50:55	0
21134	99	52	51	cyclictest	0	-21	swapper/0	08:44:01	0
21134	99	52	51	cyclictest	0	-21	swapper/0	08:34:02	0
21134	99	52	51	cyclictest	0	-21	swapper/0	08:30:04	0
21134	99	52	51	cyclictest	0	-21	swapper/0	08:24:04	0
21134	99	52	51	cyclictest	0	-21	swapper/0	08:19:01	0
21134	99	52	51	cyclictest	0	-21	swapper/0	08:14:31	0
21134	99	52	51	cyclictest	0	-21	swapper/0	08:09:03	0
21134	99	52	51	cyclictest	0	-21	swapper/0	07:59:06	0
21134	99	52	51	cyclictest	0	-21	swapper/0	07:54:03	0
21134	99	52	51	cyclictest	0	-21	swapper/0	07:34:05	0
21134	99	52	51	cyclictest	0	-21	swapper/0	07:29:06	0
21134	99	52	51	cyclictest	0	-21	swapper/0	07:24:02	0
21134	99	52	51	cyclictest	0	-21	swapper/0	07:22:49	0
21134	99	52	51	cyclictest	0	-21	swapper/0	07:09:06	0
21134	99	52	51	cyclictest	0	-21	swapper/0	06:59:07	0
21134	99	52	51	cyclictest	0	-21	swapper/0	06:49:06	0
21134	99	52	51	cyclictest	0	-21	swapper/0	06:44:07	0
21134	99	52	51	cyclictest	0	-21	swapper/0	06:39:07	0
21134	99	52	51	cyclictest	0	-21	swapper/0	06:36:02	0
21134	99	52	51	cyclictest	0	-21	swapper/0	06:29:27	0
21134	99	52	51	cyclictest	0	-21	swapper/0	06:19:07	0
21134	99	52	51	cyclictest	0	-21	swapper/0	06:14:09	0
21134	99	52	51	cyclictest	0	-21	swapper/0	06:04:07	0
21134	99	52	51	cyclictest	0	-21	swapper/0	05:54:07	0
21134	99	52	51	cyclictest	0	-21	swapper/0	05:49:11	0
21134	99	52	51	cyclictest	0	-21	swapper/0	05:44:11	0
21134	99	52	51	cyclictest	0	-21	swapper/0	05:17:51	0
21134	99	52	51	cyclictest	0	-21	swapper/0	04:59:12	0
21134	99	52	51	cyclictest	0	-21	swapper/0	04:44:11	0
21134	99	52	51	cyclictest	0	-21	swapper/0	04:39:01	0
21134	99	52	51	cyclictest	0	-21	swapper/0	04:34:11	0
21134	99	52	51	cyclictest	0	-21	swapper/0	04:24:13	0
21134	99	52	51	cyclictest	0	-21	swapper/0	04:19:07	0
21134	99	52	51	cyclictest	0	-21	swapper/0	04:14:12	0
21134	99	52	51	cyclictest	0	-21	swapper/0	04:04:12	0
21134	99	48	41	cyclictest	0	-21	swapper/0	04:29:29	0
21154	99	46	41	cyclictest	0	-21	swapper/2	08:03:57	2
21154	99	45	42	cyclictest	50	50	irq/44-ahci	09:14:23	2
21154	99	45	41	cyclictest	0	-21	swapper/2	08:01:51	2
21154	99	44	42	cyclictest	50	50	irq/44-ahci	04:41:43	2
21154	99	44	42	cyclictest	278	-21	usb-storage	07:19:11	2
21154	99	44	42	cyclictest	18	1	ksoftirqd/2	04:09:47	2
21154	99	44	41	cyclictest	0	-21	swapper/2	08:46:01	2
21154	99	44	41	cyclictest	0	-21	swapper/2	06:09:34	2
21134	99	44	42	cyclictest	1382	-17	auditd	07:38:53	0
21154	99	43	42	cyclictest	0	-21	swapper/2	07:44:08	2
21154	99	43	42	cyclictest	0	-21	swapper/2	05:24:10	2
21154	99	43	42	cyclictest	0	-21	swapper/2	04:30:19	2
21154	99	43	41	cyclictest	50	50	irq/44-ahci	07:39:41	2
21154	99	43	41	cyclictest	50	50	irq/44-ahci	05:54:31	2
21154	99	43	31	cyclictest	0	-21	swapper/2	05:14:13	2
21134	99	43	41	cyclictest	760	50	irq/18-parport0	09:14:30	0
21134	99	43	41	cyclictest	760	50	irq/18-parport0	05:24:03	0
21134	99	43	41	cyclictest	50	50	irq/44-ahci	08:04:45	0
21134	99	43	31	cyclictest	0	-21	swapper/0	07:45:36	0
21134	99	22	21	cyclictest	0	-21	swapper/0	04:04:00	0
21144	99	21	20	cyclictest	0	-21	swapper/1	04:04:00	1
23	1	12	0	ksoftirqd/3	0	-21	swapper/3	03:59:16	3
18	1	11	0	ksoftirqd/2	0	-21	swapper/2	03:59:05	2
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17