OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #3, slot #1

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance

Characteristics of the 376 highest latencies:

System rack3slot1.osadl.org (updated Thu May 28, 2026 12:43:25)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
18143	99	138	132	cyclictest	3006	-40	yum-updatesd-he	06:40:04	1
18160	99	123	122	cyclictest	3013	-40	yum-updatesd-he	06:40:04	3
18160	99	113	112	cyclictest	0	-21	swapper/3	08:57:34	3
18143	99	108	102	cyclictest	14975	-21	tune2fs	09:37:31	1
18160	99	103	102	cyclictest	0	-21	swapper/3	05:27:41	3
18160	99	103	102	cyclictest	0	-21	swapper/3	05:02:41	3
18143	99	103	102	cyclictest	25107	-21	tune2fs	05:02:41	1
18160	99	95	94	cyclictest	0	-21	swapper/3	07:27:37	3
18160	99	94	92	cyclictest	3634	-21	aten2.4-expect	09:12:30	3
18160	99	93	92	cyclictest	0	-21	swapper/3	10:05:57	3
18160	99	93	92	cyclictest	0	-21	swapper/3	09:50:55	3
18160	99	93	92	cyclictest	0	-21	swapper/3	07:40:08	3
18143	99	93	91	cyclictest	9948	-21	aten2.4-expect	08:12:31	1
18143	99	89	83	cyclictest	8342	-21	tune2fs	06:52:38	1
18143	99	89	82	cyclictest	21524	-21	tune2fs	09:52:31	1
18143	99	89	82	cyclictest	21524	-21	tune2fs	09:52:31	1
18143	99	89	72	cyclictest	0	-21	swapper/1	08:42:33	1
18143	99	89	72	cyclictest	0	-21	swapper/1	04:57:44	1
18143	99	88	81	cyclictest	11960	-21	aten2.4-expect	05:47:37	1
18143	99	87	81	cyclictest	31586	-21	tune2fs	05:17:41	1
18143	99	87	72	cyclictest	0	-21	swapper/1	06:05:19	1
18143	99	86	82	cyclictest	0	-21	swapper/1	08:27:20	1
18143	99	86	82	cyclictest	0	-21	swapper/1	07:50:36	1
18143	99	86	82	cyclictest	0	-21	swapper/1	07:42:22	1
18143	99	86	82	cyclictest	0	-21	swapper/1	07:22:44	1
18143	99	86	82	cyclictest	0	-21	swapper/1	07:07:45	1
18143	99	86	82	cyclictest	0	-21	swapper/1	05:27:28	1
18143	99	86	72	cyclictest	0	-21	swapper/1	05:15:11	1
18160	99	85	82	cyclictest	0	-21	swapper/3	09:40:53	3
18143	99	85	82	cyclictest	0	-21	swapper/1	05:40:15	1
18143	99	85	81	cyclictest	0	-21	swapper/1	06:45:25	1
18160	99	84	72	cyclictest	0	-21	swapper/3	07:57:34	3
18143	99	84	81	cyclictest	0	-21	swapper/1	06:17:45	1
18143	99	84	71	cyclictest	23253	-21	tune2fs	07:27:37	1
18160	99	83	82	cyclictest	0	-21	swapper/3	10:07:29	3
18160	99	83	82	cyclictest	0	-21	swapper/3	09:55:55	3
18160	99	83	82	cyclictest	0	-21	swapper/3	09:55:55	3
18160	99	83	82	cyclictest	0	-21	swapper/3	09:17:39	3
18160	99	83	82	cyclictest	0	-21	swapper/3	08:42:42	3
18160	99	83	82	cyclictest	0	-21	swapper/3	08:37:33	3
18160	99	83	82	cyclictest	0	-21	swapper/3	08:32:38	3
18160	99	83	82	cyclictest	0	-21	swapper/3	08:28:44	3
18160	99	83	82	cyclictest	0	-21	swapper/3	08:22:34	3
18160	99	83	82	cyclictest	0	-21	swapper/3	08:07:22	3
18160	99	83	82	cyclictest	0	-21	swapper/3	07:52:34	3
18160	99	83	82	cyclictest	0	-21	swapper/3	07:45:35	3
18160	99	83	82	cyclictest	0	-21	swapper/3	07:07:41	3
18160	99	83	82	cyclictest	0	-21	swapper/3	06:52:45	3
18160	99	83	82	cyclictest	0	-21	swapper/3	06:42:37	3
18160	99	83	82	cyclictest	0	-21	swapper/3	06:35:24	3
18160	99	83	82	cyclictest	0	-21	swapper/3	06:30:23	3
18160	99	83	82	cyclictest	0	-21	swapper/3	06:22:47	3
18160	99	83	82	cyclictest	0	-21	swapper/3	06:17:25	3
18160	99	83	82	cyclictest	0	-21	swapper/3	06:07:45	3
18160	99	83	82	cyclictest	0	-21	swapper/3	06:07:26	3
18160	99	83	82	cyclictest	0	-21	swapper/3	05:57:47	3
18160	99	83	82	cyclictest	0	-21	swapper/3	05:50:17	3
18160	99	83	82	cyclictest	0	-21	swapper/3	05:42:44	3
18160	99	83	82	cyclictest	0	-21	swapper/3	05:32:48	3
18160	99	83	82	cyclictest	0	-21	swapper/3	05:07:44	3
18160	99	83	82	cyclictest	0	-21	swapper/3	04:47:29	3
18160	99	83	82	cyclictest	0	-21	swapper/3	04:47:29	3
18143	99	83	82	cyclictest	0	-21	swapper/1	09:47:37	1
18143	99	83	82	cyclictest	0	-21	swapper/1	09:27:39	1
18143	99	83	82	cyclictest	0	-21	swapper/1	08:37:42	1
18143	99	83	82	cyclictest	0	-21	swapper/1	08:28:44	1
18143	99	83	82	cyclictest	0	-21	swapper/1	08:07:42	1
18143	99	83	82	cyclictest	0	-21	swapper/1	08:07:22	1
18143	99	83	82	cyclictest	0	-21	swapper/1	07:57:22	1
18143	99	83	82	cyclictest	0	-21	swapper/1	07:20:31	1
18143	99	83	82	cyclictest	0	-21	swapper/1	07:15:30	1
18143	99	83	82	cyclictest	0	-21	swapper/1	06:57:45	1
18143	99	83	82	cyclictest	0	-21	swapper/1	06:17:25	1
18143	99	83	82	cyclictest	0	-21	swapper/1	06:07:45	1
18143	99	83	82	cyclictest	0	-21	swapper/1	05:55:18	1
18143	99	83	82	cyclictest	0	-21	swapper/1	05:07:44	1
18143	99	83	81	cyclictest	8476	-21	tune2fs	09:22:31	1
18143	99	83	81	cyclictest	4230	-21	tune2fs	09:12:32	1
18143	99	83	81	cyclictest	30192	-21	tune2fs	10:12:31	1
18143	99	83	81	cyclictest	27531	-21	tune2fs	06:22:41	1
18143	99	83	81	cyclictest	25708	-21	tune2fs	10:02:32	1
18160	99	82	81	cyclictest	0	-21	swapper/3	10:12:31	3
18160	99	82	81	cyclictest	0	-21	swapper/3	09:57:31	3
18160	99	82	81	cyclictest	0	-21	swapper/3	09:42:29	3
18160	99	82	81	cyclictest	0	-21	swapper/3	09:27:32	3
18160	99	82	81	cyclictest	0	-21	swapper/3	09:22:31	3
18160	99	82	81	cyclictest	0	-21	swapper/3	08:12:31	3
18160	99	82	81	cyclictest	0	-21	swapper/3	07:47:34	3
18160	99	82	81	cyclictest	0	-21	swapper/3	07:32:33	3
18160	99	82	81	cyclictest	0	-21	swapper/3	05:37:42	3
18143	99	82	81	cyclictest	3814	-21	tune2fs	05:27:41	1
18143	99	82	81	cyclictest	25020	-21	aten2.4-expect	07:32:33	1
18143	99	82	71	cyclictest	0	-21	swapper/1	07:45:35	1
18143	99	78	72	cyclictest	0	-21	swapper/1	08:47:34	1
18143	99	78	62	cyclictest	0	-21	swapper/1	08:52:32	1
18160	99	76	75	cyclictest	20523	-21	aten2.4-expect	04:52:40	3
18160	99	75	74	cyclictest	0	-21	swapper/3	09:02:30	3
18143	99	75	73	cyclictest	23547	-21	tune2fs	09:57:31	1
18160	99	74	73	cyclictest	28463	-21	proc_pri	08:52:40	3
18160	99	74	73	cyclictest	0	-21	swapper/3	07:12:34	3
18160	99	73	72	cyclictest	0	-21	swapper/3	09:32:32	3
18160	99	73	72	cyclictest	0	-21	swapper/3	09:07:30	3
18160	99	73	72	cyclictest	0	-21	swapper/3	08:47:36	3
18160	99	73	72	cyclictest	0	-21	swapper/3	08:17:32	3
18160	99	73	72	cyclictest	0	-21	swapper/3	08:17:32	3
18160	99	73	72	cyclictest	0	-21	swapper/3	08:07:33	3
18160	99	73	72	cyclictest	0	-21	swapper/3	07:22:40	3
18160	99	73	72	cyclictest	0	-21	swapper/3	07:17:32	3
18160	99	73	72	cyclictest	0	-21	swapper/3	07:02:36	3
18160	99	73	72	cyclictest	0	-21	swapper/3	06:57:37	3
18160	99	73	72	cyclictest	0	-21	swapper/3	06:47:29	3
18160	99	73	72	cyclictest	0	-21	swapper/3	06:17:41	3
18160	99	73	72	cyclictest	0	-21	swapper/3	05:52:38	3
18160	99	73	72	cyclictest	0	-21	swapper/3	05:22:44	3
18160	99	73	72	cyclictest	0	-21	swapper/3	05:17:43	3
18160	99	73	72	cyclictest	0	-21	swapper/3	05:12:37	3
18160	99	73	72	cyclictest	0	-21	swapper/3	04:57:42	3
18143	99	73	72	cyclictest	0	-21	swapper/1	07:57:42	1
18143	99	73	71	cyclictest	30080	-21	tune2fs	08:57:34	1
18143	99	72	71	cyclictest	0	-21	swapper/1	09:20:50	1
18143	99	72	71	cyclictest	0	-21	swapper/1	06:47:34	1
18143	99	72	71	cyclictest	0	-21	swapper/1	06:30:23	1
18143	99	72	71	cyclictest	0	-21	swapper/1	05:47:27	1
18143	99	69	68	cyclictest	21911	-21	sed	04:52:51	1
18143	99	68	61	cyclictest	27810	-21	tune2fs	10:07:29	1
18143	99	68	61	cyclictest	18642	-21	tune2fs	04:47:42	1
18143	99	68	61	cyclictest	18642	-21	tune2fs	04:47:42	1
18143	99	67	62	cyclictest	0	-21	swapper/1	09:02:33	1
18143	99	67	62	cyclictest	0	-21	swapper/1	05:57:41	1
18143	99	66	62	cyclictest	0	-21	swapper/1	09:42:39	1
18143	99	66	62	cyclictest	0	-21	swapper/1	09:32:58	1
18143	99	66	62	cyclictest	0	-21	swapper/1	09:07:19	1
18143	99	66	62	cyclictest	0	-21	swapper/1	08:17:21	1
18143	99	66	62	cyclictest	0	-21	swapper/1	08:17:21	1
18143	99	66	62	cyclictest	0	-21	swapper/1	05:32:47	1
18143	99	66	41	cyclictest	0	-21	swapper/1	06:34:00	1
18143	99	65	61	cyclictest	0	-21	swapper/1	08:32:33	1
18143	99	65	51	cyclictest	0	-21	swapper/1	07:02:35	1
18160	99	63	62	cyclictest	0	-21	swapper/3	04:47:29	3
18143	99	63	62	cyclictest	0	-21	swapper/1	04:47:29	1
18131	99	62	61	cyclictest	0	-21	swapper/0	05:57:38	0
18131	99	58	51	cyclictest	0	-21	swapper/0	08:03:49	0
18131	99	58	51	cyclictest	0	-21	swapper/0	07:26:10	0
18131	99	58	51	cyclictest	0	-21	swapper/0	06:11:13	0
18131	99	58	41	cyclictest	0	-21	swapper/0	10:09:52	0
18131	99	58	41	cyclictest	0	-21	swapper/0	08:37:16	0
18131	99	58	41	cyclictest	0	-21	swapper/0	08:13:55	0
18131	99	58	41	cyclictest	0	-21	swapper/0	07:41:28	0
18131	99	58	41	cyclictest	0	-21	swapper/0	07:08:45	0
18131	99	58	41	cyclictest	0	-21	swapper/0	06:35:08	0
18131	99	58	41	cyclictest	0	-21	swapper/0	06:03:57	0
18131	99	58	41	cyclictest	0	-21	swapper/0	05:04:22	0
18152	99	57	56	cyclictest	18	1	ksoftirqd/2	05:38:46	2
18131	99	57	50	cyclictest	0	-21	swapper/0	07:44:22	0
18131	99	57	41	cyclictest	0	-21	swapper/0	08:24:20	0
18131	99	57	41	cyclictest	0	-21	swapper/0	05:28:19	0
18152	99	56	54	cyclictest	5683	-21	munin-node	09:17:30	2
18152	99	56	54	cyclictest	0	-21	swapper/2	05:31:07	2
18152	99	56	51	cyclictest	0	-21	swapper/2	09:35:45	2
18152	99	55	54	cyclictest	31281	-21	telnet	05:17:39	2
18152	99	55	54	cyclictest	30368	-21	munin-node	06:27:44	2
18152	99	55	54	cyclictest	28290	-21	munin-node	05:07:49	2
18152	99	55	54	cyclictest	19390	-21	munin-node	07:17:39	2
18152	99	55	54	cyclictest	18	1	ksoftirqd/2	08:08:41	2
18152	99	55	54	cyclictest	18	1	ksoftirqd/2	07:15:45	2
18152	99	55	54	cyclictest	13248	-21	munin-node	07:02:42	2
18152	99	55	54	cyclictest	1164	-21	munin-node	07:52:33	2
18152	99	55	51	cyclictest	1876	50	hald-addon-stor	07:51:03	2
18152	99	55	51	cyclictest	0	-21	swapper/2	07:42:48	2
18131	99	55	54	cyclictest	0	-21	swapper/0	09:32:28	0
18131	99	55	48	cyclictest	0	-21	swapper/0	05:26:43	0
18131	99	55	21	cyclictest	0	-21	swapper/0	08:09:17	0
18152	99	54	53	cyclictest	9988	-21	latency_hist	09:27:18	2
18152	99	54	53	cyclictest	6558	-21	munin-node	08:02:36	2
18152	99	54	53	cyclictest	50	50	irq/44-ahci	05:43:11	2
18152	99	54	53	cyclictest	278	-21	usb-storage	09:11:34	2
18152	99	54	53	cyclictest	27339	-21	munin-node	06:22:40	2
18152	99	54	53	cyclictest	25002	-21	telnet	06:17:37	2
18152	99	54	53	cyclictest	24042	-21	munin-node	06:12:47	2
18152	99	54	53	cyclictest	21931	-21	munin-node	08:37:38	2
18152	99	54	53	cyclictest	21742	-21	munin-node	09:52:32	2
18152	99	54	53	cyclictest	21742	-21	munin-node	09:52:32	2
18152	99	54	53	cyclictest	20737	-21	munin-node	04:52:41	2
18152	99	54	53	cyclictest	19369	-21	smartctl	08:32:35	2
18152	99	54	53	cyclictest	18	1	ksoftirqd/2	10:07:22	2
18152	99	54	53	cyclictest	18	1	ksoftirqd/2	08:17:27	2
18152	99	54	53	cyclictest	18	1	ksoftirqd/2	08:17:27	2
18152	99	54	53	cyclictest	18	1	ksoftirqd/2	05:58:28	2
18152	99	54	53	cyclictest	11106	-21	munin-node	08:12:39	2
18152	99	54	53	cyclictest	1045	-21	munin-node	06:37:46	2
18152	99	54	53	cyclictest	0	-21	swapper/2	08:49:17	2
18152	99	54	52	cyclictest	50	50	irq/44-ahci	09:39:29	2
18152	99	54	52	cyclictest	50	50	irq/44-ahci	09:13:03	2
18152	99	54	52	cyclictest	50	50	irq/44-ahci	07:12:03	2
18152	99	54	52	cyclictest	50	50	irq/44-ahci	06:52:52	2
18152	99	54	52	cyclictest	50	50	irq/44-ahci	06:49:56	2
18152	99	54	52	cyclictest	50	50	irq/44-ahci	05:02:30	2
18152	99	54	52	cyclictest	18	1	ksoftirqd/2	10:15:52	2
18152	99	54	51	cyclictest	278	-21	usb-storage	04:50:04	2
18152	99	54	51	cyclictest	278	-21	usb-storage	04:50:04	2
18152	99	54	51	cyclictest	18	1	ksoftirqd/2	05:48:16	2
18152	99	54	51	cyclictest	0	-21	swapper/2	05:25:03	2
18152	99	54	42	cyclictest	278	-21	usb-storage	07:24:36	2
18152	99	54	41	cyclictest	0	-21	swapper/2	08:23:40	2
18131	99	54	41	cyclictest	1286	50	irq/46-eth0	09:08:21	0
18131	99	54	21	cyclictest	0	-21	swapper/0	06:14:17	0
18152	99	53	52	cyclictest	50	50	irq/44-ahci	07:28:04	2
18152	99	53	52	cyclictest	28263	-21	munin-node	08:52:38	2
18152	99	53	52	cyclictest	0	-21	swapper/2	09:57:22	2
18152	99	53	52	cyclictest	0	-21	swapper/2	09:47:41	2
18152	99	53	52	cyclictest	0	-21	swapper/2	09:42:27	2
18152	99	53	52	cyclictest	0	-21	swapper/2	09:27:29	2
18152	99	53	52	cyclictest	0	-21	swapper/2	09:02:30	2
18152	99	53	52	cyclictest	0	-21	swapper/2	07:57:31	2
18152	99	53	52	cyclictest	0	-21	swapper/2	07:32:32	2
18152	99	53	52	cyclictest	0	-21	swapper/2	06:42:48	2
18152	99	53	52	cyclictest	0	-21	swapper/2	06:32:26	2
18152	99	53	52	cyclictest	0	-21	swapper/2	06:02:31	2
18152	99	53	52	cyclictest	0	-21	swapper/2	05:32:34	2
18152	99	53	52	cyclictest	0	-21	swapper/2	04:57:40	2
18152	99	53	51	cyclictest	50	50	irq/44-ahci	06:57:41	2
18152	99	53	51	cyclictest	29405	-21	telnet	08:57:31	2
18152	99	53	51	cyclictest	27100	-21	munin-node	07:37:44	2
18152	99	53	51	cyclictest	23000	-21	telnet	08:42:31	2
18152	99	53	51	cyclictest	16429	-21	telnet	08:27:31	2
18152	99	53	41	cyclictest	0	-21	swapper/2	10:02:18	2
18152	99	53	41	cyclictest	0	-21	swapper/2	06:07:30	2
18152	99	53	41	cyclictest	0	-21	swapper/2	05:52:44	2
18152	99	53	41	cyclictest	0	-21	swapper/2	05:13:05	2
18131	99	53	52	cyclictest	760	50	irq/18-parport0	09:58:55	0
18131	99	53	52	cyclictest	760	50	irq/18-parport0	08:48:33	0
18131	99	53	52	cyclictest	760	50	irq/18-parport0	06:57:49	0
18131	99	53	52	cyclictest	760	50	irq/18-parport0	04:51:04	0
18131	99	53	52	cyclictest	760	50	irq/18-parport0	04:51:04	0
18131	99	53	52	cyclictest	278	-21	usb-storage	06:28:01	0
18131	99	53	52	cyclictest	20301	-21	kworker/0:0	10:12:18	0
18131	99	53	52	cyclictest	1876	-21	hald-addon-stor	06:53:12	0
18131	99	53	52	cyclictest	1876	-21	hald-addon-stor	06:47:32	0
18131	99	53	52	cyclictest	0	-21	swapper/0	09:43:23	0
18131	99	53	52	cyclictest	0	-21	swapper/0	09:29:24	0
18131	99	53	52	cyclictest	0	-21	swapper/0	07:32:36	0
18131	99	53	52	cyclictest	0	-21	swapper/0	07:16:13	0
18131	99	53	52	cyclictest	0	-21	swapper/0	07:02:34	0
18131	99	53	52	cyclictest	0	-21	swapper/0	06:47:18	0
18131	99	53	52	cyclictest	0	-21	swapper/0	06:38:20	0
18131	99	53	52	cyclictest	0	-21	swapper/0	05:52:37	0
18131	99	53	52	cyclictest	0	-21	swapper/0	05:10:44	0
18131	99	53	51	cyclictest	50	50	irq/44-ahci	07:53:08	0
18131	99	53	41	cyclictest	3	1	ksoftirqd/0	08:42:26	0
18131	99	53	41	cyclictest	0	-21	swapper/0	09:47:32	0
18131	99	53	41	cyclictest	0	-21	swapper/0	09:22:32	0
18131	99	53	41	cyclictest	0	-21	swapper/0	09:02:25	0
18131	99	53	41	cyclictest	0	-21	swapper/0	07:51:03	0
18131	99	53	41	cyclictest	0	-21	swapper/0	06:24:33	0
18131	99	53	41	cyclictest	0	-21	swapper/0	06:17:37	0
18131	99	53	41	cyclictest	0	-21	swapper/0	05:47:44	0
18131	99	53	41	cyclictest	0	-21	swapper/0	05:42:37	0
18131	99	53	41	cyclictest	0	-21	swapper/0	05:37:56	0
18131	99	53	41	cyclictest	0	-21	swapper/0	05:33:35	0
18131	99	53	41	cyclictest	0	-21	swapper/0	05:17:40	0
18131	99	53	41	cyclictest	0	-21	swapper/0	05:12:42	0
18131	99	53	21	cyclictest	0	-21	swapper/0	08:18:08	0
18131	99	53	21	cyclictest	0	-21	swapper/0	08:18:08	0
18131	99	52	51	cyclictest	760	50	irq/18-parport0	09:14:26	0
18131	99	52	51	cyclictest	760	50	irq/18-parport0	08:58:41	0
18131	99	52	51	cyclictest	760	50	irq/18-parport0	07:58:11	0
18131	99	52	51	cyclictest	0	-21	swapper/0	09:55:31	0
18131	99	52	51	cyclictest	0	-21	swapper/0	09:55:31	0
18131	99	52	51	cyclictest	0	-21	swapper/0	09:38:19	0
18131	99	52	51	cyclictest	0	-21	swapper/0	09:18:38	0
18131	99	52	51	cyclictest	0	-21	swapper/0	08:54:55	0
18131	99	52	51	cyclictest	0	-21	swapper/0	07:28:20	0
18131	99	52	51	cyclictest	0	-21	swapper/0	07:18:01	0
18131	99	52	51	cyclictest	0	-21	swapper/0	04:53:18	0
18131	99	52	41	cyclictest	0	-21	swapper/0	10:02:30	0
18131	99	52	41	cyclictest	0	-21	swapper/0	08:37:30	0
18131	99	52	41	cyclictest	0	-21	swapper/0	08:28:58	0
18131	99	52	41	cyclictest	0	-21	swapper/0	05:00:10	0
18152	99	43	42	cyclictest	0	-21	swapper/2	04:47:29	2
18131	99	42	41	cyclictest	0	-21	swapper/0	04:47:29	0
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17