OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Thu Mar 12, 2026 00:43:34)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
124886	2	104	0,1	sleep7	0	-21	swapper/7	20:59:20	7
495536	2	100	0,1	sleep7	495528	-21	ntp_offset	23:34:28	7
4089375	2	58	48,7	sleep5	0	-21	swapper/5	19:03:25	5
4089702	99	52	49,2	cyclictest	508128	-21	kworker/u16:0+events_unbound	00:04:15	5
4089343	2	51	41,7	sleep4	0	-21	swapper/4	19:03:00	4
4089702	99	49	46,2	cyclictest	344027	-21	kworker/u16:4+events_unbound	23:39:16	5
4089704	99	46	42,3	cyclictest	211757	-21	kworker/u16:3+flush-8:0	21:39:01	6
4089702	99	46	43,2	cyclictest	534533	-21	kworker/u16:1+events_unbound	23:54:13	5
4089702	99	45	43,2	cyclictest	343704	-21	kworker/u16:2+flush-8:0	22:49:00	5
4089702	99	44	42,2	cyclictest	486970	-21	kworker/u16:3+events_unbound	00:14:27	5
4089702	99	43	41,2	cyclictest	508128	-21	kworker/u16:0+events_unbound	00:19:28	5
4089702	99	43	40,2	cyclictest	508128	-21	kworker/u16:0+events_unbound	23:49:26	5
4089702	99	43	40,2	cyclictest	508128	-21	kworker/u16:0+events_unbound	23:49:26	5
4089702	99	42	40,2	cyclictest	401447	-21	kworker/u16:1+events_unbound	23:25:43	5
4089702	99	42	40,2	cyclictest	344027	-21	kworker/u16:4+events_unbound	22:59:32	5
4089702	99	42	39,2	cyclictest	534533	-21	kworker/u16:1+flush-8:0	00:29:10	5
4089702	99	42	39,2	cyclictest	534533	-21	kworker/u16:1+flush-8:0	00:29:10	5
4089702	99	41	39,2	cyclictest	508128	-21	kworker/u16:0+events_unbound	00:10:38	5
4089702	99	41	39,2	cyclictest	344027	-21	kworker/u16:4+events_unbound	23:19:32	5
4089702	99	41	39,2	cyclictest	344027	-21	kworker/u16:4+events_unbound	23:04:14	5
4089702	99	41	39,2	cyclictest	344027	-21	kworker/u16:4+events_unbound	22:52:38	5
4089702	99	41	38,2	cyclictest	61351	-21	kworker/u16:2+events_unbound	21:14:02	5
4089280	2	41	31,7	sleep6	0	-21	swapper/6	19:02:06	6
4089236	2	40	5,31	sleep7	0	-21	swapper/7	19:01:26	7
4089702	99	39	37,2	cyclictest	401447	-21	kworker/u16:1+events_unbound	22:58:18	5
4089702	99	39	37,2	cyclictest	272052	-21	kworker/u16:0+events_unbound	22:22:18	5
4089702	99	39	37,2	cyclictest	131092	-21	kworker/u16:0+events_unbound	21:28:30	5
4089702	99	39	37,2	cyclictest	131092	-21	kworker/u16:0+events_unbound	21:07:47	5
4089702	99	39	36,2	cyclictest	438503	-21	kworker/u16:2+events_unbound	23:29:09	5
4089704	99	38	35,2	cyclictest	24279	-21	kworker/u16:4+flush-8:0	20:29:26	6
4089702	99	38	36,2	cyclictest	91202	-21	kworker/u16:1+events_unbound	21:43:02	5
4089702	99	38	36,2	cyclictest	61351	-21	kworker/u16:2+events_unbound	21:34:43	5
4089702	99	38	36,2	cyclictest	606001	-21	kworker/u16:4+events_unbound	00:25:10	5
4089702	99	38	36,2	cyclictest	486970	-21	kworker/u16:3+events_unbound	00:01:57	5
4089702	99	38	36,2	cyclictest	438503	-21	kworker/u16:2+events_unbound	23:38:26	5
4089702	99	38	36,2	cyclictest	4169835	-21	kworker/u16:3+flush-8:0	19:54:34	5
4089702	99	38	36,2	cyclictest	401447	-21	kworker/u16:1+flush-8:0	23:09:46	5
4089702	99	38	36,2	cyclictest	401447	-21	kworker/u16:1+flush-8:0	23:09:46	5
4089702	99	38	36,2	cyclictest	401447	-21	kworker/u16:1+events_unbound	23:17:33	5
4089702	99	38	36,2	cyclictest	344027	-21	kworker/u16:4+flush-8:0	23:44:57	5
4089702	99	38	36,2	cyclictest	272052	-21	kworker/u16:0+events_unbound	22:17:14	5
4089702	99	38	36,2	cyclictest	258382	-21	kworker/u16:5+events_unbound	21:57:02	5
4089698	99	38	32,5	cyclictest	258382	-21	kworker/u16:5+flush-8:0	22:19:00	4
4089708	99	37	32,4	cyclictest	4141191	-21	latency_hist	19:29:00	7
4089704	99	37	35,2	cyclictest	91202	-21	kworker/u16:1+events_unbound	21:24:22	6
4089704	99	37	34,2	cyclictest	508128	-21	kworker/u16:0+flush-8:0	00:04:28	6
4089702	99	37	35,2	cyclictest	294078	-21	kworker/u16:1+events_unbound	22:11:47	5
4089702	99	37	35,2	cyclictest	258382	-21	kworker/u16:5+events_unbound	22:30:37	5
4089702	99	37	35,2	cyclictest	258382	-21	kworker/u16:5+events_unbound	22:30:37	5
4089702	99	37	35,2	cyclictest	247619	-21	kworker/u16:3+events_unbound	22:05:42	5
4089702	99	37	35,2	cyclictest	24279	-21	kworker/u16:4+flush-8:0	20:29:20	5
4089702	99	36	34,2	cyclictest	258382	-21	kworker/u16:5+flush-8:0	21:59:41	5
4089702	99	36	34,2	cyclictest	258382	-21	kworker/u16:5+events_unbound	22:40:37	5
4089702	99	36	33,2	cyclictest	61351	-21	kworker/u16:2+events_unbound	20:39:03	5
4089708	99	35	34,0	cyclictest	0	-21	swapper/7	21:20:06	7
4089702	99	35	33,2	cyclictest	131092	-21	kworker/u16:0+events_unbound	21:51:38	5
4089708	99	34	33,1	cyclictest	129346	-21	diskmemload	23:17:27	7
4089708	99	34	33,0	cyclictest	0	-21	swapper/7	23:30:35	7
4089704	99	34	32,2	cyclictest	344027	-21	kworker/u16:4+flush-8:0	23:18:55	6
4089702	99	34	33,1	cyclictest	344027	-21	kworker/u16:4+events_unbound	22:38:22	5
4089702	99	34	32,2	cyclictest	91202	-21	kworker/u16:1+events_unbound	21:21:14	5
4089702	99	34	32,2	cyclictest	61351	-21	kworker/u16:2+events_unbound	21:13:02	5
4089708	99	33	32,0	cyclictest	0	-21	swapper/7	22:35:05	7
4089702	99	33	31,2	cyclictest	258382	-21	kworker/u16:5+events_unbound	22:24:45	5
4089702	99	33	31,2	cyclictest	131092	-21	kworker/u16:0+events_unbound	21:47:53	5
4089698	99	33	30,2	cyclictest	91202	-21	kworker/u16:1+flush-8:0	21:49:34	4
4089708	99	32	27,4	cyclictest	650	-21	systemd-journal	21:29:00	7
4089704	99	32	30,2	cyclictest	272052	-21	kworker/u16:0+flush-8:0	22:28:55	6
4089704	99	32	29,2	cyclictest	258382	-21	kworker/u16:5+flush-8:0	22:34:00	6
4089704	99	32	29,2	cyclictest	258382	-21	kworker/u16:5+flush-8:0	22:34:00	6
4089708	99	31	30,0	cyclictest	0	-21	swapper/7	23:51:51	7
4089708	99	31	30,0	cyclictest	0	-21	swapper/7	23:51:51	7
4089702	99	31	28,2	cyclictest	4089470	-21	kworker/u16:1+events_unbound	19:19:02	5
4089708	99	30	29,0	cyclictest	0	-21	swapper/7	22:27:43	7
4089708	99	30	25,4	cyclictest	296077	-21	latency_hist	22:14:01	7
4089704	99	30	28,2	cyclictest	4339	-21	kworker/u16:2+flush-8:0	20:04:30	6
4089704	99	30	28,2	cyclictest	272052	-21	kworker/u16:0+flush-8:0	22:23:56	6
4089702	99	30	28,2	cyclictest	131092	-21	kworker/u16:0+flush-8:0	21:29:42	5
4089708	99	29	29,0	cyclictest	0	-21	swapper/7	22:49:31	7
4089708	99	29	29,0	cyclictest	0	-21	swapper/7	21:36:39	7
4089708	99	29	28,0	cyclictest	0	-21	swapper/7	23:05:20	7
4089708	99	29	26,2	cyclictest	151578	-21	ssh	21:11:04	7
4089704	99	29	27,2	cyclictest	91202	-21	kworker/u16:1+flush-8:0	21:39:39	6
4089704	99	29	27,2	cyclictest	534533	-21	kworker/u16:1+flush-8:0	00:19:43	6
4089708	99	27	27,0	cyclictest	0	-21	swapper/7	22:18:16	7
4089708	99	27	26,0	cyclictest	0	-21	swapper/7	20:43:31	7
4089708	99	27	25,1	cyclictest	627688	-21	ssh	00:30:31	7
4089708	99	27	25,1	cyclictest	627688	-21	ssh	00:30:31	7
4089698	99	27	25,2	cyclictest	343704	-21	kworker/u16:2+events_unbound	22:44:14	4
4089708	99	26	26,0	cyclictest	0	-21	swapper/7	23:46:19	7
4089708	99	26	26,0	cyclictest	0	-21	swapper/7	00:11:31	7
4089708	99	26	25,0	cyclictest	0	-21	swapper/7	19:32:58	7
4089708	99	26	25,0	cyclictest	0	-21	swapper/7	19:23:39	7
4089704	99	26	24,2	cyclictest	258382	-21	kworker/u16:5+flush-8:0	22:14:25	6
4089702	99	26	24,2	cyclictest	72285	-21	kworker/u16:0+events_unbound	20:35:11	5
4089702	99	26	23,2	cyclictest	72286	-21	kworker/u16:3+events_unbound	20:59:03	5
4089702	99	26	23,2	cyclictest	4089470	-21	kworker/u16:1+events_unbound	20:19:03	5
4089708	99	25	24,0	cyclictest	0	-21	swapper/7	20:37:22	7
4089708	99	25	24,0	cyclictest	0	-21	swapper/7	20:06:02	7
4089704	99	25	23,2	cyclictest	4051776	-21	kworker/u16:2+events_unbound	19:04:18	6
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17