OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #3, slot #6

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 376 highest latencies:

System rack3slot6.osadl.org (updated Wed Apr 22, 2026 00:45:04)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
2545	2	97	19,33	sleep1	0	-21	swapper/1	19:05:43	1
11145	2	90	0,6	sleep0	0	-21	swapper/0	22:30:26	0
6839	2	88	0,6	sleep0	6840	-21	ls	20:50:00	0
6839	2	88	0,6	sleep0	6840	-21	ls	20:50:00	0
3130	99	88	66,11	cyclictest	0	-21	swapper/0	00:05:14	0
3130	99	88	66,11	cyclictest	0	-21	swapper/0	00:05:14	0
3130	99	86	57,16	cyclictest	0	-21	swapper/0	23:55:11	0
3130	99	81	57,13	cyclictest	0	-21	swapper/0	22:50:13	0
23638	2	81	0,5	chrt	0	-21	swapper/1	23:05:26	1
3130	99	79	54,13	cyclictest	14	-21	ksoftirqd/0	23:35:14	0
1391	2	79	29,38	sleep0	0	-21	swapper/0	19:05:13	0
3130	99	78	56,10	cyclictest	14	-21	ksoftirqd/0	23:15:13	0
3130	99	78	55,11	cyclictest	14	-21	ksoftirqd/0	20:00:12	0
3130	99	78	55,11	cyclictest	14	-21	ksoftirqd/0	20:00:12	0
3130	99	78	48,24	cyclictest	14	-21	ksoftirqd/0	23:40:16	0
3130	99	77	59,10	cyclictest	14	-21	ksoftirqd/0	21:05:14	0
3130	99	77	53,12	cyclictest	14	-21	ksoftirqd/0	21:55:13	0
3134	99	73	56,10	cyclictest	26	-21	ksoftirqd/1	20:50:34	1
3130	99	73	46,13	cyclictest	14	-21	ksoftirqd/0	20:25:12	0
3130	99	72	46,14	cyclictest	14	-21	ksoftirqd/0	21:35:14	0
3130	99	72	43,16	cyclictest	14	-21	ksoftirqd/0	23:30:11	0
3130	99	72	41,17	cyclictest	14	-21	ksoftirqd/0	00:00:14	0
3134	99	71	44,13	cyclictest	26	-21	ksoftirqd/1	21:20:01	1
3134	99	71	31,8	cyclictest	26	-21	ksoftirqd/1	22:10:46	1
3130	99	69	40,16	cyclictest	0	-21	swapper/0	20:15:12	0
3130	99	69	38,18	cyclictest	0	-21	swapper/0	19:55:13	0
3130	99	68	40,18	cyclictest	14	-21	ksoftirqd/0	19:40:12	0
3130	99	68	40,17	cyclictest	14	-21	ksoftirqd/0	21:10:14	0
3130	99	68	38,17	cyclictest	14	-21	ksoftirqd/0	20:55:14	0
3130	99	68	38,17	cyclictest	0	-21	swapper/0	23:45:14	0
3130	99	68	38,17	cyclictest	0	-21	swapper/0	23:45:14	0
3130	99	67	38,19	cyclictest	14	-21	ksoftirqd/0	21:30:13	0
3130	99	67	38,18	cyclictest	0	-21	swapper/0	20:05:15	0
3130	99	67	38,16	cyclictest	0	-21	swapper/0	23:20:14	0
3130	99	66	39,17	cyclictest	14	-21	ksoftirqd/0	20:40:13	0
3130	99	66	37,17	cyclictest	0	-21	swapper/0	21:15:15	0
11508	2	66	0,8	sleep0	14	-21	ksoftirqd/0	21:00:29	0
3134	99	65	51,9	cyclictest	26	-21	ksoftirqd/1	23:52:07	1
3134	99	65	50,9	cyclictest	26	-21	ksoftirqd/1	23:55:16	1
3130	99	65	38,18	cyclictest	0	-21	swapper/0	19:20:22	0
3130	99	65	37,17	cyclictest	0	-21	swapper/0	22:00:12	0
3130	99	65	37,15	cyclictest	14	-21	ksoftirqd/0	23:10:13	0
3130	99	65	35,17	cyclictest	0	-21	swapper/0	19:15:15	0
3130	99	65	35,17	cyclictest	0	-21	swapper/0	19:15:15	0
3130	99	65	34,20	cyclictest	14	-21	ksoftirqd/0	22:35:15	0
3130	99	65	15,22	cyclictest	15108	-21	cron	00:15:00	0
3130	99	65	12,48	cyclictest	0	-21	swapper/0	22:25:14	0
3134	99	64	22,22	cyclictest	26	-21	ksoftirqd/1	20:45:01	1
3130	99	64	39,15	cyclictest	0	-21	swapper/0	23:00:11	0
3130	99	64	39,14	cyclictest	14	-21	ksoftirqd/0	22:15:14	0
3130	99	64	35,17	cyclictest	0	-21	swapper/0	00:35:16	0
3130	99	64	32,26	cyclictest	7362	-21	telnet	20:50:26	0
3134	99	63	23,22	cyclictest	26	-21	ksoftirqd/1	23:15:01	1
3130	99	63	36,18	cyclictest	14	-21	ksoftirqd/0	20:30:25	0
3130	99	63	34,18	cyclictest	0	-21	swapper/0	22:45:14	0
3130	99	63	32,18	cyclictest	0	-21	swapper/0	21:25:25	0
3134	99	62	38,9	cyclictest	26	-21	ksoftirqd/1	20:31:43	1
3134	99	62	31,24	cyclictest	26	-21	ksoftirqd/1	00:12:09	1
3134	99	62	17,31	cyclictest	26	-21	ksoftirqd/1	23:04:34	1
3130	99	62	41,16	cyclictest	14	-21	ksoftirqd/0	21:20:17	0
3130	99	62	39,19	cyclictest	14	-21	ksoftirqd/0	00:25:22	0
3130	99	62	37,19	cyclictest	14	-21	ksoftirqd/0	19:30:12	0
3130	99	62	32,17	cyclictest	0	-21	swapper/0	00:30:13	0
3134	99	61	39,7	cyclictest	26	-21	ksoftirqd/1	20:10:37	1
3134	99	61	38,9	cyclictest	26	-21	ksoftirqd/1	23:15:22	1
3134	99	61	38,8	cyclictest	26	-21	ksoftirqd/1	23:30:26	1
3134	99	61	38,8	cyclictest	26	-21	ksoftirqd/1	22:15:13	1
3134	99	61	38,8	cyclictest	26	-21	ksoftirqd/1	21:50:24	1
3134	99	61	38,8	cyclictest	26	-21	ksoftirqd/1	21:10:28	1
3134	99	61	38,8	cyclictest	26	-21	ksoftirqd/1	20:45:11	1
3134	99	61	38,8	cyclictest	26	-21	ksoftirqd/1	20:45:11	1
3134	99	61	38,8	cyclictest	26	-21	ksoftirqd/1	00:30:28	1
3134	99	61	38,8	cyclictest	26	-21	ksoftirqd/1	00:20:16	1
3134	99	61	36,9	cyclictest	26	-21	ksoftirqd/1	23:00:02	1
3134	99	61	20,20	cyclictest	26	-21	ksoftirqd/1	20:00:00	1
3134	99	61	13,31	cyclictest	12612	-21	head	21:05:16	1
3130	99	61	33,17	cyclictest	0	-21	swapper/0	22:55:25	0
3130	99	61	31,18	cyclictest	0	-21	swapper/0	22:40:12	0
3130	99	61	22,34	cyclictest	243	-21	systemd-journal	20:11:23	0
3130	99	61	20,16	cyclictest	16391	-21	aten2.4-expect	19:45:16	0
25280	2	61	2,16	sleep1	25385	-21	cpu	21:40:17	1
3134	99	60	38,8	cyclictest	26	-21	ksoftirqd/1	22:00:23	1
3134	99	60	38,7	cyclictest	26	-21	ksoftirqd/1	21:45:24	1
3134	99	60	37,9	cyclictest	26	-21	ksoftirqd/1	19:16:43	1
3134	99	60	37,9	cyclictest	26	-21	ksoftirqd/1	19:16:43	1
3134	99	60	37,8	cyclictest	26	-21	ksoftirqd/1	22:30:33	1
3134	99	60	37,8	cyclictest	26	-21	ksoftirqd/1	22:05:20	1
3134	99	60	37,8	cyclictest	26	-21	ksoftirqd/1	21:30:11	1
3134	99	60	37,8	cyclictest	26	-21	ksoftirqd/1	21:25:42	1
3134	99	60	37,8	cyclictest	26	-21	ksoftirqd/1	21:20:18	1
3134	99	60	37,8	cyclictest	26	-21	ksoftirqd/1	20:05:17	1
3134	99	60	37,8	cyclictest	26	-21	ksoftirqd/1	19:45:26	1
3134	99	60	37,8	cyclictest	26	-21	ksoftirqd/1	00:05:13	1
3134	99	60	37,8	cyclictest	26	-21	ksoftirqd/1	00:05:13	1
3134	99	60	37,8	cyclictest	26	-21	ksoftirqd/1	00:00:20	1
3134	99	60	37,7	cyclictest	26	-21	ksoftirqd/1	21:40:01	1
3134	99	60	36,9	cyclictest	26	-21	ksoftirqd/1	23:20:27	1
3134	99	60	36,8	cyclictest	26	-21	ksoftirqd/1	22:25:37	1
3134	99	60	36,8	cyclictest	26	-21	ksoftirqd/1	00:15:25	1
3134	99	60	35,9	cyclictest	26	-21	ksoftirqd/1	19:35:23	1
3134	99	60	33,7	cyclictest	26	-21	ksoftirqd/1	22:49:48	1
3130	99	60	28,23	cyclictest	457	-21	dbus-daemon	00:20:00	0
3134	99	59	38,8	cyclictest	26	-21	ksoftirqd/1	22:40:23	1
3134	99	59	37,8	cyclictest	26	-21	ksoftirqd/1	23:40:17	1
3134	99	59	37,8	cyclictest	26	-21	ksoftirqd/1	22:35:23	1
3134	99	59	37,8	cyclictest	26	-21	ksoftirqd/1	20:35:13	1
3134	99	59	37,8	cyclictest	26	-21	ksoftirqd/1	20:15:25	1
3134	99	59	37,8	cyclictest	26	-21	ksoftirqd/1	19:50:27	1
3134	99	59	37,7	cyclictest	26	-21	ksoftirqd/1	19:10:28	1
3134	99	59	37,10	cyclictest	26	-21	ksoftirqd/1	22:20:25	1
3134	99	59	36,9	cyclictest	26	-21	ksoftirqd/1	19:30:18	1
3134	99	59	36,8	cyclictest	26	-21	ksoftirqd/1	23:45:15	1
3134	99	59	36,8	cyclictest	26	-21	ksoftirqd/1	23:45:15	1
3134	99	59	36,8	cyclictest	26	-21	ksoftirqd/1	23:35:18	1
3134	99	59	36,8	cyclictest	26	-21	ksoftirqd/1	21:55:17	1
3134	99	59	36,8	cyclictest	26	-21	ksoftirqd/1	20:05:01	1
3134	99	59	36,8	cyclictest	26	-21	ksoftirqd/1	20:05:01	1
3134	99	59	36,8	cyclictest	26	-21	ksoftirqd/1	00:35:26	1
3134	99	59	36,8	cyclictest	26	-21	ksoftirqd/1	00:25:28	1
3134	99	59	35,9	cyclictest	26	-21	ksoftirqd/1	23:25:18	1
3134	99	59	35,8	cyclictest	26	-21	ksoftirqd/1	22:50:14	1
3134	99	59	35,8	cyclictest	26	-21	ksoftirqd/1	19:20:44	1
3130	99	59	29,19	cyclictest	0	-21	swapper/0	22:20:14	0
3134	99	58	36,8	cyclictest	26	-21	ksoftirqd/1	20:55:17	1
3134	99	58	36,8	cyclictest	26	-21	ksoftirqd/1	20:25:20	1
3134	99	58	36,8	cyclictest	26	-21	ksoftirqd/1	19:25:17	1
3134	99	58	35,8	cyclictest	26	-21	ksoftirqd/1	21:00:30	1
3134	99	58	35,7	cyclictest	26	-21	ksoftirqd/1	20:20:17	1
3130	99	58	35,13	cyclictest	0	-21	swapper/0	20:20:14	0
3130	99	58	30,17	cyclictest	0	-21	swapper/0	19:25:14	0
3130	99	58	30,16	cyclictest	0	-21	swapper/0	21:45:12	0
13810	2	58	0,7	sleep0	14	1	ksoftirqd/0	19:35:36	0
3134	99	57	34,8	cyclictest	26	-21	ksoftirqd/1	19:40:17	1
3130	99	57	27,17	cyclictest	14	-21	ksoftirqd/0	23:05:15	0
3130	99	57	19,33	cyclictest	3675	-21	aten2.4-expect	22:10:26	0
3130	99	56	21,31	cyclictest	243	-21	systemd-journal	21:41:41	0
3130	99	55	6,41	cyclictest	0	-21	swapper/0	21:50:34	0
3130	99	55	38,10	cyclictest	0	-21	swapper/0	20:35:32	0
3130	99	55	28,19	cyclictest	30981	-21	awk	23:25:32	0
3130	99	55	13,19	cyclictest	1531	-21	aten2.4-expect	22:05:17	0
3130	99	54	29,15	cyclictest	0	-21	swapper/0	00:20:12	0
3130	99	54	22,22	cyclictest	0	-21	swapper/0	23:50:31	0
3130	99	52	25,16	cyclictest	14	-21	ksoftirqd/0	19:50:14	0
3130	99	52	24,17	cyclictest	0	-21	swapper/0	19:10:13	0
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17