OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack7slot7s.osadl.org (updated Sat Dec 20, 2025 12:43:47)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
18115	2	95	82,9	sleep2	0	-21	swapper/2	07:06:38	2
18066	2	94	81,9	sleep0	0	-21	swapper/0	07:06:00	0
18104	2	82	66,12	sleep1	0	-21	swapper/1	07:06:29	1
18241	2	79	67,8	sleep3	0	-21	swapper/3	07:08:16	3
18472	99	31	1,7	cyclictest	8867	-21	apt-get	09:05:14	3
18467	99	28	27,1	cyclictest	0	-21	swapper/1	09:19:36	1
18465	99	25	22,2	cyclictest	13	-21	ksoftirqd/0	12:00:00	0
18472	99	24	0,2	cyclictest	0	-21	swapper/3	10:10:13	3
18472	99	24	0,17	cyclictest	0	-21	swapper/3	08:30:16	3
18467	99	23	0,21	cyclictest	7899	-21	grep	11:15:27	1
18472	99	21	0,4	cyclictest	20382	-21	cut	10:35:17	3
18472	99	20	4,1	cyclictest	18211	-21	latency	10:30:20	3
18472	99	19	8,8	cyclictest	31257	-21	taskset	08:42:03	3
18472	99	19	3,2	cyclictest	11553	-21	wget	11:25:16	3
18472	99	18	0,3	cyclictest	26898	-21	needreboot	09:40:23	3
18470	99	17	6,7	cyclictest	423	-21	apt-get	12:10:13	2
18470	99	17	6,7	cyclictest	423	-21	apt-get	12:10:12	2
18472	99	16	6,9	cyclictest	12118	-21	cron	10:20:00	3
18472	99	16	6,9	cyclictest	12118	-21	cron	10:20:00	3
18472	99	16	3,4	cyclictest	225	-21	systemd-journal	12:25:01	3
18472	99	16	1,2	cyclictest	26997	-21	kernelversion	07:25:23	3
18472	99	16	12,3	cyclictest	39	-21	ksoftirqd/3	09:35:14	3
18472	99	16	0,15	cyclictest	24607	-21	head	11:50:26	3
18472	99	16	0,15	cyclictest	1435	-21	chrt	11:05:12	3
18472	99	16	0,15	cyclictest	0	-21	swapper/3	07:50:14	3
18467	99	16	0,15	cyclictest	0	-21	swapper/1	09:35:15	1
18472	99	15	9,5	cyclictest	5150	-21	wc	10:05:00	3
18472	99	15	9,5	cyclictest	22394	-21	tr	08:25:15	3
18472	99	15	8,6	cyclictest	31728	-21	uname	07:35:19	3
18472	99	15	6,8	cyclictest	14533	-21	munin-run	10:25:00	3
18472	99	15	0,14	cyclictest	12158	-21	taskset	08:02:02	3
18472	99	15	0,14	cyclictest	10161	-21	readlink	11:20:23	3
18472	99	14	9,4	cyclictest	27448	-21	grep	11:55:26	3
18472	99	14	9,4	cyclictest	15112	-21	grep	08:10:15	3
18472	99	14	8,5	cyclictest	3871	-21	grep	07:45:18	3
18472	99	14	8,5	cyclictest	32718	-21	wc	12:10:01	3
18472	99	14	8,5	cyclictest	24024	-21	taskset	10:44:09	3
18472	99	14	8,5	cyclictest	21255	-21	tr	11:45:16	3
18472	99	14	8,5	cyclictest	11353	-21	tr	12:30:24	3
18472	99	14	7,6	cyclictest	6159	-21	cat	10:05:19	3
18472	99	14	7,6	cyclictest	28837	-21	taskset	10:54:59	3
18472	99	14	7,6	cyclictest	2804	-21	taskset	08:50:21	3
18472	99	14	6,7	cyclictest	31193	-21	taskset	10:59:48	3
18472	99	14	6,7	cyclictest	24658	-21	tr	10:45:15	3
18472	99	14	6,7	cyclictest	10141	-21	cut	08:00:00	3
18472	99	14	3,4	cyclictest	225	-21	systemd-journal	08:19:59	3
18472	99	14	1,1	cyclictest	11914	-21	grep	09:10:18	3
18472	99	14	10,3	cyclictest	6740	-21	tr	09:00:14	3
18472	99	14	0,5	cyclictest	0	-21	swapper/3	08:20:14	3
18472	99	14	0,5	cyclictest	0	-21	swapper/3	08:20:14	3
18472	99	14	0,13	cyclictest	5939	-21	cron	11:15:00	3
18472	99	14	0,13	cyclictest	28930	-21	sed	07:30:18	3
18470	99	14	9,4	cyclictest	15450	-21	grep	09:15:27	2
18470	99	14	10,3	cyclictest	22642	-21	sed	11:45:28	2
18470	99	14	10,2	cyclictest	23721	-21	grep	07:20:16	2
18467	99	14	9,4	cyclictest	29126	-21	cut	09:45:19	1
18467	99	14	11,2	cyclictest	24095	-21	cron	10:45:01	1
18467	99	14	11,2	cyclictest	1219	-21	tail	09:55:18	1
18467	99	14	10,3	cyclictest	29392	-21	grep	12:00:22	1
18465	99	14	10,2	cyclictest	15894	-21	grep	10:25:20	0
18465	99	14	0,8	cyclictest	23165	-21	perf	09:35:00	0
18472	99	13	9,3	cyclictest	13755	-21	tail	08:05:21	3
18472	99	13	8,4	cyclictest	29491	-21	which	12:00:22	3
18472	99	13	8,4	cyclictest	15549	-21	chrt	11:30:44	3
18472	99	13	7,5	cyclictest	9182	-21	cat	12:25:27	3
18472	99	13	7,5	cyclictest	21400	-21	aten2_r7power_v	07:15:15	3
18472	99	13	7,5	cyclictest	15990	-21	sort	09:20:00	3
18472	99	13	6,6	cyclictest	864	-21	cut	07:40:15	3
18472	99	13	6,6	cyclictest	25932	-21	wc	07:25:00	3
18472	99	13	6,6	cyclictest	15027	-21	expr	10:25:16	3
18472	99	13	3,2	cyclictest	19653	-21	wc	09:25:21	3
18472	99	13	2,4	cyclictest	1715	-21	ps	12:10:23	3
18472	99	13	2,4	cyclictest	1715	-21	ps	12:10:23	3
18472	99	13	2,2	cyclictest	225	-21	systemd-journal	08:40:01	3
18472	99	13	1,4	cyclictest	19178	-21	cut	07:10:15	3
18472	99	13	1,3	cyclictest	31982	-21	cpuspeed_turbos	08:45:16	3
18472	99	13	1,2	cyclictest	225	-21	systemd-journal	11:44:59	3
18472	99	13	1,1	cyclictest	0	-21	swapper/3	09:50:21	3
18472	99	13	10,2	cyclictest	3570	-21	head	12:15:17	3
18472	99	13	10,2	cyclictest	1278	-21	sed	09:55:18	3
18472	99	13	0,3	cyclictest	225	-21	systemd-journal	09:00:01	3
18472	99	13	0,0	cyclictest	0	-21	swapper/3	12:35:16	3
18470	99	13	9,3	cyclictest	28230	-21	grep	08:35:24	2
18470	99	13	9,3	cyclictest	21208	-21	grep	10:35:26	2
18470	99	13	9,2	cyclictest	9588	-21	grep	11:20:17	2
18470	99	13	8,3	cyclictest	20917	-21	wc	09:30:00	2
18470	99	13	11,1	cyclictest	0	-21	swapper/2	10:15:15	2
18470	99	13	11,1	cyclictest	0	-21	swapper/2	10:15:15	2
18470	99	13	10,2	cyclictest	6288	-21	cut	11:15:14	2
18470	99	13	10,2	cyclictest	32159	-21	grep	11:00:18	2
18470	99	13	10,2	cyclictest	29275	-21	grep	09:45:20	2
18470	99	13	10,2	cyclictest	27286	-21	grep	07:25:26	2
18470	99	13	10,2	cyclictest	21003	-21	cron	07:14:59	2
18470	99	13	10,2	cyclictest	17042	-21	cat	10:30:01	2
18470	99	13	10,2	cyclictest	16153	-21	cut	08:10:22	2
18467	99	13	9,3	cyclictest	25855	-21	tr	08:30:24	1
18467	99	13	9,3	cyclictest	2446	-21	grep	08:50:19	1
18467	99	13	10,2	cyclictest	9501	-21	grep	11:20:17	1
18467	99	13	10,2	cyclictest	3010	-21	cat	11:05:26	1
18467	99	13	10,2	cyclictest	30059	-21	grep	07:30:28	1
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17