OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

x86 AMD Ryzen XXXX @3300 MHz, Linux 4.18.7-rt5 (Profile)

Latency plot of shadow in rack #c, slot #3

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rackcslot3s (updated Sun Jan 29, 2023 12:50:15)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
13833	2	670	653,13	sleep0	0	-21	swapper/0	07:09:11	0
11886	2	607	586,15	sleep7	0	-21	swapper/7	07:05:09	7
13651	2	601	581,15	sleep2	0	-21	swapper/2	07:06:32	2
13792	2	591	571,15	sleep4	0	-21	swapper/4	07:08:35	4
13798	2	494	475,15	sleep1	0	-21	swapper/1	07:08:40	1
13644	2	405	384,14	sleep6	0	-21	swapper/6	07:06:28	6
13856	2	398	376,17	sleep3	0	-21	swapper/3	07:09:30	3
13616	2	338	317,15	sleep5	0	-21	swapper/5	07:06:03	5
14336	99	62	59,2	cyclictest	14041	-21	cpuspeed_turbos	08:15:17	5
20799	2	57	0,0	sleep1	0	-21	swapper/1	08:30:01	1
14335	99	51	49,1	cyclictest	9280	-21	tr	08:05:16	4
14335	99	50	49,0	cyclictest	3236	-21	tr	12:20:18	4
14334	99	50	48,1	cyclictest	19316	-21	cut	10:40:18	3
14331	99	49	47,1	cyclictest	552	-21	tr	12:15:16	0
14337	99	47	46,0	cyclictest	3939	-21	latency_hist	07:55:00	6
14336	99	47	45,1	cyclictest	16519	-21	tr	08:20:16	5
14333	99	47	46,0	cyclictest	7930	-21	cron	11:25:00	2
14331	99	47	44,2	cyclictest	5530	-21	cron	11:20:00	0
14337	99	46	44,1	cyclictest	23061	-21	cpuspeed_turbos	09:40:16	6
14331	99	46	44,1	cyclictest	23429	-21	cron	10:50:00	0
14338	99	45	42,1	cyclictest	3153	-21	sh	11:15:00	7
14336	99	45	43,1	cyclictest	13582	-21	cut	11:35:19	5
14336	99	45	15,29	cyclictest	27276	-21	grep	12:00:32	5
14331	99	45	43,1	cyclictest	23470	-21	grep	11:55:20	0
14336	99	44	42,1	cyclictest	14640	-21	cpuspeed_turbos	07:10:17	5
14331	99	44	9,1	cyclictest	282	-21	systemd-journal	12:30:00	0
14331	99	44	42,1	cyclictest	19327	-21	tr	10:40:19	0
14333	99	43	40,2	cyclictest	28445	-21	tr	08:45:17	2
14338	99	42	19,1	cyclictest	19426	-21	apt	07:20:01	7
14334	99	42	40,1	cyclictest	924	-21	tr	08:55:21	3
14333	99	42	40,1	cyclictest	32389	-21	grep	08:50:31	2
14332	99	42	41,0	cyclictest	32402	-21	sh	12:15:00	1
14331	99	42	40,1	cyclictest	3327	-21	cpuspeed_turbos	09:00:19	0
14331	99	41	39,1	cyclictest	22470	-21	unixbench_multi	11:50:29	0
14338	99	40	38,1	cyclictest	19467	-21	head	11:45:24	7
14334	99	40	39,0	cyclictest	11168	-21	tr	11:30:18	3
14333	99	40	38,1	cyclictest	32179	-21	cpuspeed_turbos	07:45:14	2
14333	99	40	38,1	cyclictest	31995	-21	sh	10:00:00	2
14332	99	40	39,0	cyclictest	25245	-21	sh	12:00:00	1
14336	99	39	37,1	cyclictest	12191	-21	tr	10:25:21	5
14335	99	39	38,0	cyclictest	6933	-21	tr	08:00:18	4
14335	99	39	36,1	cyclictest	14039	-21	tr	08:15:17	4
14333	99	39	38,0	cyclictest	8740	-21	sed	10:15:29	2
14333	99	39	38,0	cyclictest	25247	-21	sh	12:00:00	2
14331	99	39	38,0	cyclictest	31422	-21	tr	11:05:18	0
14331	99	39	37,1	cyclictest	25702	-21	grep	08:40:01	0
14338	99	38	36,1	cyclictest	8736	-21	cut	11:25:22	7
14338	99	38	36,1	cyclictest	0	-21	swapper/7	08:20:01	7
14338	99	38	0,37	cyclictest	0	-21	swapper/7	10:40:00	7
14337	99	38	36,1	cyclictest	18664	-21	sh	10:40:00	6
14336	99	38	36,1	cyclictest	17481	-21	turbostat.cron	09:30:00	5
14335	99	38	37,0	cyclictest	13600	-21	cpuspeed_turbos	11:35:19	4
14333	99	38	36,1	cyclictest	24106	-21	grep	08:35:21	2
14333	99	38	0,1	cyclictest	0	-21	swapper/2	10:40:23	2
14331	99	38	36,1	cyclictest	8023	-21	cpuspeed_turbos	12:30:18	0
14338	99	37	35,1	cyclictest	17752	-21	timerwakeupswit	08:20:26	7
14338	99	37	35,1	cyclictest	0	-21	swapper/7	09:10:17	7
14337	99	37	36,0	cyclictest	19466	-21	grep	11:45:24	6
14337	99	37	35,1	cyclictest	15447	-21	grep	08:15:28	6
14336	99	37	35,1	cyclictest	27213	-21	cpuspeed_turbos	07:35:20	5
14336	99	37	35,1	cyclictest	2377	-21	basename	12:20:00	5
14335	99	37	36,0	cyclictest	18736	-21	cpuspeed_turbos	11:45:17	4
14335	99	37	3,18	cyclictest	0	-21	swapper/4	12:35:28	4
14334	99	37	35,1	cyclictest	31412	-21	cpuspeed_turbos	11:05:18	3
14334	99	37	35,1	cyclictest	23106	-21	wc	08:35:01	3
14334	99	37	0,36	cyclictest	0	-21	swapper/3	10:50:27	3
14333	99	37	35,1	cyclictest	21704	-21	tr	10:45:17	2
14332	99	37	36,0	cyclictest	20303	-21	grep	09:35:00	1
14332	99	37	35,1	cyclictest	32296	-21	proc_pri	11:05:29	1
14332	99	37	35,1	cyclictest	15922	-21	grep	07:10:29	1
14332	99	37	35,1	cyclictest	0	-21	swapper/1	09:15:21	1
14332	99	37	0,36	cyclictest	0	-21	swapper/1	09:20:27	1
14331	99	37	34,1	cyclictest	6760	-21	latency_hist	10:15:00	0
14331	99	37	0,1	cyclictest	0	-21	swapper/0	10:25:21	0
14338	99	36	35,0	cyclictest	24077	-21	tr	10:50:17	7
14338	99	36	34,1	cyclictest	28217	-21	tr	12:05:19	7
14338	99	36	34,1	cyclictest	23168	-21	cat	11:55:00	7
14338	99	36	34,1	cyclictest	12732	-21	cat	08:10:29	7
14338	99	36	34,1	cyclictest	12413	-21	cat	11:30:29	7
14338	99	36	34,1	cyclictest	0	-21	swapper/7	09:25:00	7
14337	99	36	34,1	cyclictest	24893	-21	grep	07:30:19	6
14336	99	36	34,1	cyclictest	8281	-21	unixbench_multi	08:00:30	5
14336	99	36	34,1	cyclictest	10667	-21	fgrep	10:20:27	5
14336	99	36	33,2	cyclictest	29253	-21	sed	09:50:28	5
14335	99	36	34,1	cyclictest	28273	-21	cpuspeed_turbos	12:05:18	4
14335	99	36	0,35	cyclictest	7360	-21	cpuspeed_turbos	10:15:17	4
14335	99	36	0,35	cyclictest	25544	-21	cpuspeed_turbos	09:45:18	4
14335	99	36	0,35	cyclictest	0	-21	swapper/4	07:15:02	4
14335	99	36	0,1	cyclictest	0	-21	swapper/4	09:10:24	4
14334	99	36	35,0	cyclictest	1689	-21	latency_hist	07:50:01	3
14334	99	36	35,0	cyclictest	0	-21	swapper/3	10:00:16	3
14333	99	36	28,7	cyclictest	21446	-21	unixbench_singl	07:20:28	2
14332	99	36	34,1	cyclictest	21654	-21	grep	10:45:17	1
14331	99	36	33,2	cyclictest	21268	-21	tr	08:30:20	0
14338	99	35	34,0	cyclictest	0	-21	swapper/7	08:35:27	7
14338	99	35	33,1	cyclictest	29447	-21	switchtime	08:45:27	7
14338	99	35	33,1	cyclictest	27804	-21	unixbench_multi	10:55:36	7
14338	99	35	33,1	cyclictest	19351	-21	tr	10:40:18	7
14337	99	35	34,0	cyclictest	31999	-21	latency_hist	10:00:00	6
14337	99	35	34,0	cyclictest	24083	-21	cpuspeed_turbos	10:50:17	6
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17