OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Sun Feb 08, 2026 12:43:33)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
2676879	2	59	49,7	sleep4	0	-21	swapper/4	07:03:35	4
2677007	2	53	19,30	sleep7	0	-21	swapper/7	07:05:23	7
2676843	2	53	43,7	sleep6	0	-21	swapper/6	07:03:05	6
2677356	99	51	44,5	cyclictest	2874369	-21	latency_hist	08:46:06	7
2677342	99	45	44,1	cyclictest	3360085	-21	kworker/u16:2+flush-8:0	12:31:05	4
2677342	99	45	44,1	cyclictest	3360085	-21	kworker/u16:2+flush-8:0	12:31:05	4
2677034	2	41	31,7	sleep5	0	-21	swapper/5	07:05:45	5
2677356	99	38	33,4	cyclictest	3300884	-21	latency_hist	11:46:05	7
2677342	99	37	35,2	cyclictest	2865100	-21	kworker/u16:2+flush-8:0	09:16:28	4
2937205	2	35	0,0	chrt	0	-21	swapper/6	09:16:05	6
2937205	2	35	0,0	chrt	0	-21	swapper/6	09:16:05	6
2677351	99	33	31,2	cyclictest	2998139	-21	kworker/u16:2+flush-8:0	10:16:32	6
2677356	99	32	27,4	cyclictest	2748752	-21	latency_hist	07:41:06	7
2677351	99	32	29,2	cyclictest	2949420	-21	kworker/u16:4+events_unbound	09:31:19	6
2677356	99	30	27,2	cyclictest	3191951	-21	latency_hist	11:01:06	7
2677356	99	29	28,0	cyclictest	0	-21	swapper/7	11:36:24	7
2677342	99	27	24,3	cyclictest	2787458	-21	kworker/u16:0+events_unbound	08:56:24	4
3142952	2	26	0,1	chrt	3142953	-21	rm	10:40:30	7
2677356	99	26	26,0	cyclictest	0	-21	swapper/7	09:56:38	7
2677356	99	26	22,3	cyclictest	2776823	-21	ntpq	07:51:32	7
2677342	99	26	24,2	cyclictest	2998139	-21	kworker/u16:2+events_unbound	10:06:32	4
2677342	99	26	24,2	cyclictest	2710092	-21	kworker/u16:2+flush-8:0	07:51:24	4
2677346	99	25	23,2	cyclictest	2926473	-21	kworker/u16:3+events_unbound	09:28:06	5
2677342	99	25	23,2	cyclictest	2865100	-21	kworker/u16:2+flush-8:0	09:06:45	4
2677342	99	25	23,2	cyclictest	2816624	-21	kworker/u16:3+events_unbound	08:31:19	4
2677356	99	24	23,0	cyclictest	0	-21	swapper/7	07:31:20	7
2677356	99	24	0,23	cyclictest	5512	-21	dbus-daemon	09:28:15	7
2677342	99	24	22,2	cyclictest	3203045	-21	kworker/u16:1+events_unbound	11:26:43	4
2677356	99	23	23,0	cyclictest	0	-21	swapper/7	09:11:29	7
2677356	99	23	23,0	cyclictest	0	-21	swapper/7	09:11:29	7
2677342	99	23	21,2	cyclictest	3119473	-21	kworker/u16:0+events_unbound	11:21:01	4
2677356	99	22	22,0	cyclictest	0	-21	swapper/7	10:21:28	7
2677356	99	22	20,1	cyclictest	3010121	-21	idleruntime-cro	09:46:05	7
2677351	99	22	20,2	cyclictest	3119473	-21	kworker/u16:0+flush-8:0	12:16:05	6
2677346	99	22	18,3	cyclictest	1184838	-21	kworker/u16:1+flush-8:0	07:26:23	5
2677342	99	22	20,2	cyclictest	3119473	-21	kworker/u16:0+events_unbound	12:31:30	4
2677342	99	22	20,2	cyclictest	2903169	-21	kworker/u16:1+events_unbound	09:56:40	4
2677342	99	22	20,2	cyclictest	2874575	-21	kworker/u16:4+flush-8:0	08:51:18	4
2677342	99	22	20,2	cyclictest	2787458	-21	kworker/u16:0+flush-8:0	08:26:30	4
2677356	99	21	21,0	cyclictest	0	-21	swapper/7	11:21:35	7
2677356	99	21	20,0	cyclictest	0	-21	swapper/7	10:11:27	7
2677356	99	21	19,1	cyclictest	2913055	-21	diskmemload	12:36:00	7
2677342	99	21	19,2	cyclictest	3203045	-21	kworker/u16:1+events_unbound	11:31:23	4
2677342	99	21	19,2	cyclictest	3203045	-21	kworker/u16:1+events_unbound	11:11:32	4
2677342	99	21	19,2	cyclictest	3119473	-21	kworker/u16:0+events_unbound	11:21:34	4
2677342	99	21	19,2	cyclictest	3087270	-21	kworker/u16:3+flush-8:0	10:56:33	4
2677342	99	21	19,2	cyclictest	2903169	-21	kworker/u16:1+flush-8:0	09:21:31	4
2677342	99	21	19,2	cyclictest	2816624	-21	kworker/u16:3+flush-8:0	08:21:24	4
2677342	99	21	18,2	cyclictest	2816624	-21	kworker/u16:3+flush-8:0	08:16:23	4
2677356	99	20	14,1	cyclictest	96	-21	ksoftirqd/7	11:56:34	7
2677342	99	20	18,2	cyclictest	3119473	-21	kworker/u16:0+events_unbound	12:06:25	4
2677342	99	20	18,2	cyclictest	3119473	-21	kworker/u16:0+events_unbound	11:51:19	4
2677342	99	20	18,2	cyclictest	2926473	-21	kworker/u16:3+events_unbound	09:11:35	4
2677342	99	20	18,2	cyclictest	2926473	-21	kworker/u16:3+events_unbound	09:11:35	4
2677342	99	20	18,2	cyclictest	2903169	-21	kworker/u16:1+flush-8:0	09:31:29	4
2677342	99	20	18,2	cyclictest	2787458	-21	kworker/u16:0+events_unbound	08:40:58	4
2677342	99	20	18,2	cyclictest	2768134	-21	kworker/u16:1+flush-8:0	08:01:23	4
2677342	99	20	18,2	cyclictest	1931995	-21	kworker/u16:5+flush-8:0	07:06:21	4
2677356	99	19	13,5	cyclictest	3313009	-21	latency_hist	11:51:05	7
2677356	99	19	0,18	cyclictest	0	-21	swapper/7	08:06:28	7
2677342	99	19	17,2	cyclictest	2903169	-21	kworker/u16:1+events_unbound	09:36:53	4
2677342	99	19	17,2	cyclictest	2865100	-21	kworker/u16:2+flush-8:0	08:46:21	4
2677356	99	18	14,3	cyclictest	1267	-21	snmpd	09:35:21	7
2677356	99	18	12,5	cyclictest	650	-21	systemd-journal	11:16:00	7
2677356	99	18	12,5	cyclictest	2729407	-21	latency_hist	07:31:07	7
2677346	99	18	16,2	cyclictest	3034672	-21	kworker/u16:0+events_unbound	10:01:21	5
2677346	99	18	16,2	cyclictest	2949420	-21	kworker/u16:4+events_unbound	09:31:31	5
2677342	99	18	16,2	cyclictest	2926473	-21	kworker/u16:3+events_unbound	09:51:50	4
2677342	99	18	16,2	cyclictest	2926473	-21	kworker/u16:3+events_unbound	09:51:50	4
2677342	99	18	16,2	cyclictest	2710092	-21	kworker/u16:2+flush-8:0	07:46:58	4
2677342	99	18	16,2	cyclictest	1184838	-21	kworker/u16:1+flush-8:0	07:26:34	4
2677342	99	18	11,3	cyclictest	3087270	-21	kworker/u16:3+events_unbound	10:55:02	4
2677356	99	17	7,9	cyclictest	0	-21	swapper/7	11:26:30	7
2677356	99	17	7,9	cyclictest	0	-21	swapper/7	10:06:31	7
2677356	99	17	4,12	cyclictest	0	-21	swapper/7	09:25:27	7
2677356	99	17	16,0	cyclictest	0	-21	swapper/7	08:16:21	7
2677356	99	17	15,1	cyclictest	2913055	-21	diskmemload	09:16:22	7
2677356	99	17	0,15	cyclictest	1267	-21	snmpd	08:36:05	7
2677351	99	17	14,3	cyclictest	2816624	-21	kworker/u16:3+flush-8:0	08:26:06	6
2677346	99	17	15,2	cyclictest	2926473	-21	kworker/u16:3+events_unbound	09:41:31	5
2677346	99	17	14,2	cyclictest	2903169	-21	kworker/u16:1+flush-8:0	09:56:05	5
2677346	99	17	14,2	cyclictest	2903169	-21	kworker/u16:1+flush-8:0	09:56:05	5
2677342	99	17	15,2	cyclictest	3119473	-21	kworker/u16:0+flush-8:0	12:16:30	4
2677342	99	17	15,2	cyclictest	2787458	-21	kworker/u16:0+flush-8:0	08:08:39	4
2677342	99	17	15,2	cyclictest	2687074	-21	kworker/u16:0+events_unbound	07:44:41	4
2677342	99	17	15,2	cyclictest	1184838	-21	kworker/u16:1+events_unbound	07:34:15	4
2677342	99	17	14,2	cyclictest	2903169	-21	kworker/u16:1+events_unbound	10:16:18	4
2677342	99	17	12,2	cyclictest	2903169	-21	kworker/u16:1+flush-8:0	09:01:22	4
2677356	99	16	7,7	cyclictest	0	-21	swapper/7	11:51:22	7
2677356	99	16	15,1	cyclictest	0	-21	swapper/7	11:31:30	7
2677356	99	16	12,3	cyclictest	1267	-21	snmpd	12:08:40	7
2677356	99	16	1,14	cyclictest	3406832	-21	ssh	12:26:30	7
2677356	99	16	1,14	cyclictest	3406832	-21	ssh	12:26:30	7
2677351	99	16	0,2	cyclictest	0	-21	swapper/6	08:11:31	6
2677346	99	16	15,1	cyclictest	0	-21	swapper/5	09:57:53	5
2677346	99	16	13,3	cyclictest	3360085	-21	kworker/u16:2+flush-8:0	12:16:46	5
2677346	99	16	12,3	cyclictest	3229467	-21	cat	11:16:21	5
2677346	99	16	0,15	cyclictest	3248081	-21	kthreadcore	11:21:26	5
2677342	99	16	14,2	cyclictest	3289216	-21	kworker/u16:3+flush-8:0	12:01:48	4
2677342	99	16	14,2	cyclictest	3241616	-21	kworker/u16:2+events_unbound	11:36:43	4
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17