OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #0, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

x86 Intel Core i7-4960X @3600 MHz, Linux 5.15.2-rt19 (Profile)

Latency plot of shadow in rack #0, slot #3

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot3s.osadl.org (updated Mon Mar 20, 2023 00:44:21)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
187694	99	450	449,1	cyclictest	0	-21	swapper/4	20:24:55	4
187691	99	450	447,2	cyclictest	250977	-21	turbostat	21:00:00	3
187704	99	448	0,447	cyclictest	216772	-21	munin-run	20:00:00	7
187704	99	448	0,1	cyclictest	363170	-21	munin-run	23:00:01	7
187700	99	446	445,1	cyclictest	0	-21	swapper/6	20:24:55	6
187704	99	445	445,0	cyclictest	190360	-21	turbostat	19:19:55	7
187704	99	445	443,1	cyclictest	0	-21	swapper/7	19:35:00	7
187683	99	445	444,1	cyclictest	248110	-21	turbostat	20:59:55	0
187704	99	444	443,1	cyclictest	329379	-21	turbostat	22:29:55	7
187704	99	443	443,0	cyclictest	253822	-21	turbostat	21:09:55	7
187691	99	443	442,1	cyclictest	396958	-21	turbostat	23:35:00	3
187691	99	441	439,2	cyclictest	0	-21	swapper/3	21:34:55	3
187691	99	441	439,1	cyclictest	208246	-21	turbostat	19:45:00	3
187704	99	440	438,2	cyclictest	271253	-21	awk	21:24:59	7
187691	99	439	439,0	cyclictest	0	-21	swapper/3	20:44:55	3
187691	99	439	437,2	cyclictest	219665	-21	turbostat	20:05:00	3
187691	99	438	437,1	cyclictest	0	-21	swapper/3	22:14:56	3
187691	99	438	437,1	cyclictest	0	-21	swapper/3	21:29:54	3
187685	99	438	434,3	cyclictest	233900	-21	basename	20:30:00	1
187683	99	438	437,1	cyclictest	0	-21	swapper/0	20:24:55	0
187704	99	437	435,2	cyclictest	219665	-21	turbostat	20:09:55	7
187704	99	437	435,2	cyclictest	219665	-21	turbostat	20:09:55	7
187691	99	435	435,0	cyclictest	0	-21	swapper/3	23:39:55	3
187691	99	434	431,2	cyclictest	222488	-21	turbostat	20:10:00	3
187691	99	434	431,2	cyclictest	222488	-21	turbostat	20:10:00	3
187691	99	433	432,1	cyclictest	213943	-21	turbostat	19:55:00	3
187698	99	432	429,3	cyclictest	0	-21	swapper/5	20:30:00	5
187691	99	432	430,1	cyclictest	324510	-21	turbostat	22:20:00	3
187704	99	431	428,2	cyclictest	0	-21	swapper/7	21:05:00	7
187704	99	430	429,0	cyclictest	0	-21	swapper/7	23:25:00	7
187691	99	430	429,1	cyclictest	0	-21	swapper/3	23:09:55	3
187704	99	429	425,4	cyclictest	295468	-21	turbostat	21:54:55	7
187691	99	429	429,0	cyclictest	0	-21	swapper/3	21:24:54	3
187691	99	429	428,1	cyclictest	0	-21	swapper/3	20:14:56	3
187691	99	429	428,1	cyclictest	0	-21	swapper/3	20:14:55	3
187691	99	429	427,1	cyclictest	211109	-21	turbostat	19:50:00	3
187691	99	429	426,3	cyclictest	416301	-21	turbostat	23:55:00	3
187691	99	428	427,1	cyclictest	190360	-21	turbostat	19:15:00	3
187691	99	428	427,1	cyclictest	0	-21	swapper/3	21:14:55	3
187691	99	426	423,2	cyclictest	334183	-21	turbostat	22:30:00	3
187704	99	425	422,3	cyclictest	152048	-21	kworker/u16:3	22:40:00	7
187691	99	425	423,2	cyclictest	459892	-21	turbostat	00:40:00	3
187691	99	424	424,0	cyclictest	0	-21	swapper/3	20:34:55	3
187691	99	423	422,1	cyclictest	426026	-21	turbostat	00:05:00	3
187691	99	423	419,3	cyclictest	290629	-21	turbostat	21:45:01	3
187704	99	422	420,2	cyclictest	450204	-21	turbostat	00:34:55	7
187691	99	422	421,1	cyclictest	0	-21	swapper/3	19:24:55	3
187704	99	421	419,1	cyclictest	0	-21	swapper/7	23:20:00	7
187704	99	419	418,0	cyclictest	0	-21	swapper/7	00:10:01	7
187691	99	419	418,1	cyclictest	0	-21	swapper/3	20:24:55	3
187704	99	418	418,0	cyclictest	0	-21	swapper/7	22:50:00	7
187704	99	417	416,1	cyclictest	0	-21	swapper/7	22:35:00	7
187704	99	417	414,2	cyclictest	0	-21	swapper/7	19:40:00	7
187704	99	415	415,0	cyclictest	324510	-21	turbostat	22:24:55	7
187704	99	415	412,3	cyclictest	392155	-21	turbostat	23:34:55	7
187704	99	414	414,0	cyclictest	261526	-21	turbostat	21:19:54	7
187691	99	414	413,1	cyclictest	199729	-21	turbostat	19:30:00	3
187691	99	414	412,1	cyclictest	348704	-21	turbostat	22:45:00	3
187691	99	413	411,1	cyclictest	245280	-21	turbostat	20:50:00	3
187704	99	411	410,1	cyclictest	401805	-21	turbostat	23:44:55	7
187691	99	411	411,0	cyclictest	0	-21	swapper/3	20:39:56	3
187704	99	410	408,2	cyclictest	285818	-21	turbostat	21:44:55	7
187691	99	410	410,0	cyclictest	266421	-21	turbostat	21:19:59	3
187704	99	409	407,2	cyclictest	310032	-21	turbostat	22:09:55	7
187685	99	408	404,3	cyclictest	0	-21	swapper/1	22:55:00	1
187704	99	406	405,1	cyclictest	196852	-21	turbostat	19:29:55	7
187698	99	403	399,4	cyclictest	358327	-21	awk	22:55:01	5
187691	99	403	401,1	cyclictest	310032	-21	turbostat	22:05:00	3
187704	99	402	402,0	cyclictest	445396	-21	turbostat	00:29:55	7
187691	99	402	401,1	cyclictest	0	-21	swapper/3	20:54:55	3
187704	99	401	401,0	cyclictest	205426	-21	turbostat	19:44:55	7
187704	99	400	398,2	cyclictest	300325	-21	turbostat	21:59:55	7
187704	99	399	399,0	cyclictest	430841	-21	turbostat	00:14:55	7
187694	99	398	395,3	cyclictest	314850	-21	awk	22:10:00	4
187691	99	397	396,1	cyclictest	0	-21	swapper/3	22:34:55	3
187698	99	395	394,1	cyclictest	0	-21	swapper/5	20:00:00	5
187704	99	394	394,0	cyclictest	440520	-21	turbostat	00:24:55	7
187704	99	394	392,2	cyclictest	0	-21	swapper/7	21:30:00	7
187704	99	394	391,3	cyclictest	0	-21	swapper/7	23:05:00	7
187704	99	393	387,2	cyclictest	406625	-21	turbostat	23:49:55	7
187691	99	392	389,0	cyclictest	0	-21	swapper/3	21:49:55	3
187700	99	391	390,1	cyclictest	0	-21	swapper/6	22:10:00	6
187683	99	391	388,2	cyclictest	280954	-21	awk	21:35:00	0
187691	99	390	390,0	cyclictest	406625	-21	turbostat	23:45:00	3
187694	99	387	387,0	cyclictest	0	-21	swapper/4	19:59:55	4
187691	99	387	387,0	cyclictest	0	-21	swapper/3	00:09:55	3
187691	99	387	385,1	cyclictest	305135	-21	turbostat	22:00:00	3
187683	99	387	384,3	cyclictest	0	-21	swapper/0	22:10:00	0
187694	99	386	384,2	cyclictest	0	-21	swapper/4	20:30:00	4
187700	99	384	381,2	cyclictest	233891	-21	munin-run	20:30:00	6
187704	99	382	381,0	cyclictest	0	-21	swapper/7	20:45:00	7
187685	99	382	381,1	cyclictest	0	-21	swapper/1	23:25:00	1
187691	99	381	381,0	cyclictest	0	-21	swapper/3	22:49:55	3
187691	99	380	379,0	cyclictest	450204	-21	turbostat	00:30:01	3
187704	99	379	377,2	cyclictest	0	-21	swapper/7	22:02:59	7
187704	99	379	376,2	cyclictest	0	-21	swapper/7	21:00:00	7
187704	99	378	378,0	cyclictest	0	-21	swapper/7	23:13:59	7
187704	99	378	378,0	cyclictest	0	-21	swapper/7	22:44:59	7
187704	99	378	377,1	cyclictest	0	-21	swapper/7	23:07:59	7
187698	99	378	376,1	cyclictest	387313	-21	munin-run	23:25:00	5
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17