OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack8slot4s.osadl.org (updated Fri Dec 12, 2025 00:44:42)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
2922392	2	248	184,22	sleep1	0	-21	swapper/1	19:07:11	1
2922320	2	241	174,21	sleep6	0	-21	swapper/6	19:06:11	6
2922369	2	225	165,19	sleep7	0	-21	swapper/7	19:06:53	7
2922319	2	222	182,27	sleep5	0	-21	swapper/5	19:06:10	5
2922480	2	216	179,24	sleep4	0	-21	swapper/4	19:08:26	4
2922488	2	214	180,22	sleep3	0	-21	swapper/3	19:08:32	3
2922542	2	212	178,22	sleep0	0	-21	swapper/0	19:09:19	0
2922469	2	210	182,18	sleep2	0	-21	swapper/2	19:08:16	2
3010477	2	50	0,0	sleep5	0	-21	swapper/5	21:40:01	5
2922831	99	33	0,15	cyclictest	0	-21	swapper/7	21:12:32	7
2922810	99	32	31,1	cyclictest	3042125	-21	fwupd	22:33:34	0
2922822	99	31	30,1	cyclictest	0	-21	swapper/4	22:33:35	4
2922820	99	26	0,25	cyclictest	0	-21	swapper/3	21:29:59	3
2922825	99	25	0,24	cyclictest	0	-21	swapper/5	21:25:14	5
2922814	99	25	0,24	cyclictest	0	-21	swapper/1	23:15:11	1
2922822	99	24	0,1	cyclictest	0	-21	swapper/4	22:25:12	4
2922820	99	24	22,1	cyclictest	2958004	-21	munin-run	20:10:00	3
2922820	99	24	0,23	cyclictest	2963979	-21	munin-run	20:20:00	3
2922820	99	24	0,23	cyclictest	0	-21	swapper/3	20:34:59	3
2922816	99	24	0,23	cyclictest	0	-21	swapper/2	22:25:12	2
2922810	99	24	0,24	cyclictest	2955573	-21	apt-get	20:05:14	0
2922810	99	24	0,23	cyclictest	0	-21	swapper/0	21:00:15	0
2922831	99	23	22,1	cyclictest	0	-21	swapper/7	23:00:16	7
2922831	99	23	0,2	cyclictest	3080015	-21	idleruntime-cro	23:40:01	7
2922831	99	23	0,2	cyclictest	2963985	-21	perf	20:19:59	7
2922831	99	23	0,22	cyclictest	2998552	-21	turbostat	21:20:00	7
2922831	99	23	0,22	cyclictest	0	-21	swapper/7	20:55:01	7
2922829	99	23	23,0	cyclictest	0	-21	swapper/6	20:05:14	6
2922829	99	23	22,1	cyclictest	3051290	-21	apt-get	22:50:16	6
2922829	99	23	19,3	cyclictest	3080184	-21	latency_hist	23:40:01	6
2922829	99	23	0,22	cyclictest	0	-21	swapper/6	00:25:16	6
2922829	99	23	0,22	cyclictest	0	-21	swapper/6	00:15:14	6
2922825	99	23	0,2	cyclictest	2970130	-21	apt	20:30:13	5
2922825	99	23	0,22	cyclictest	3088815	-21	basename	23:55:00	5
2922825	99	23	0,22	cyclictest	0	-21	swapper/5	23:45:13	5
2922822	99	23	21,1	cyclictest	0	-21	swapper/4	21:20:01	4
2922822	99	23	19,4	cyclictest	2925547	-21	latency_hist	19:15:00	4
2922822	99	23	1,21	cyclictest	0	-21	swapper/4	00:20:13	4
2922822	99	23	1,1	cyclictest	0	-21	swapper/4	19:30:00	4
2922822	99	23	0,23	cyclictest	2940877	-21	basename	19:40:00	4
2922822	99	23	0,22	cyclictest	3079974	-21	munin-run	23:40:01	4
2922820	99	23	21,1	cyclictest	2958689	-21	apt-get	20:10:13	3
2922820	99	23	21,1	cyclictest	0	-21	swapper/3	00:35:00	3
2922820	99	23	0,22	cyclictest	3094987	-21	apt-get	00:05:14	3
2922820	99	23	0,22	cyclictest	2984009	-21	turbostat	20:55:00	3
2922820	99	23	0,22	cyclictest	0	-21	swapper/3	20:45:15	3
2922816	99	23	22,1	cyclictest	0	-21	swapper/2	21:00:00	2
2922816	99	23	0,22	cyclictest	0	-21	swapper/2	23:30:13	2
2922816	99	23	0,1	cyclictest	3016415	-21	apt-get	21:50:16	2
2922816	99	23	0,1	cyclictest	0	-21	swapper/2	00:34:59	2
2922814	99	23	21,1	cyclictest	3074681	-21	apt-get	23:30:13	1
2922814	99	23	21,1	cyclictest	0	-21	swapper/1	22:25:00	1
2922814	99	23	19,3	cyclictest	2943735	-21	idleruntime-cro	19:44:59	1
2922810	99	23	0,23	cyclictest	2978294	-21	munin-run	20:44:59	0
2922810	99	23	0,22	cyclictest	0	-21	swapper/0	22:40:13	0
2922810	99	23	0,22	cyclictest	0	-21	swapper/0	22:00:12	0
2922810	99	23	0,1	cyclictest	3114574	-21	cut	00:40:00	0
2922810	99	23	0,0	cyclictest	0	-21	swapper/0	22:50:16	0
2922831	99	22	21,1	cyclictest	3068006	-21	apt-get	23:20:01	7
2922831	99	22	21,1	cyclictest	0	-21	swapper/7	00:10:14	7
2922831	99	22	0,21	cyclictest	0	-21	swapper/7	23:15:12	7
2922831	99	22	0,1	cyclictest	3036951	-21	apt-get	22:25:13	7
2922829	99	22	21,1	cyclictest	3045579	-21	apt-get	22:40:13	6
2922829	99	22	0,22	cyclictest	3027686	-21	munin-run	22:10:00	6
2922829	99	22	0,21	cyclictest	3057019	-21	apt-get	23:00:16	6
2922829	99	22	0,21	cyclictest	3054140	-21	apt-get	22:55:14	6
2922829	99	22	0,21	cyclictest	0	-21	swapper/6	23:15:12	6
2922829	99	22	0,21	cyclictest	0	-21	swapper/6	21:45:13	6
2922829	99	22	0,21	cyclictest	0	-21	swapper/6	21:45:13	6
2922829	99	22	0,21	cyclictest	0	-21	swapper/6	20:25:15	6
2922829	99	22	0,21	cyclictest	0	-21	swapper/6	00:30:15	6
2922829	99	22	0,1	cyclictest	0	-21	swapper/6	19:40:14	6
2922829	99	22	0,0	cyclictest	0	-21	swapper/6	00:00:14	6
2922825	99	22	20,2	cyclictest	2930820	-21	taskset	19:20:20	5
2922825	99	22	0,22	cyclictest	3048013	-21	munin-run	22:45:01	5
2922825	99	22	0,22	cyclictest	0	-21	swapper/5	23:20:15	5
2922825	99	22	0,21	cyclictest	0	-21	swapper/5	22:15:15	5
2922825	99	22	0,21	cyclictest	0	-21	swapper/5	00:35:16	5
2922822	99	22	21,1	cyclictest	0	-21	swapper/4	20:20:14	4
2922822	99	22	19,2	cyclictest	3056591	-21	awk	23:00:01	4
2922822	99	22	19,2	cyclictest	3018828	-21	idleruntime-cro	21:55:00	4
2922822	99	22	0,1	cyclictest	0	-21	swapper/4	20:45:15	4
2922822	99	22	0,0	cyclictest	0	-21	swapper/4	22:20:13	4
2922820	99	22	21,1	cyclictest	0	-21	swapper/3	22:00:13	3
2922820	99	22	20,1	cyclictest	3039812	-21	apt-get	22:30:16	3
2922820	99	22	19,2	cyclictest	3100266	-21	dump-pmu-power	00:15:00	3
2922820	99	22	0,21	cyclictest	0	-21	swapper/3	22:15:14	3
2922820	99	22	0,21	cyclictest	0	-21	swapper/3	20:35:16	3
2922820	99	22	0,21	cyclictest	0	-21	swapper/3	19:20:15	3
2922820	99	22	0,0	cyclictest	0	-21	swapper/3	19:10:13	3
2922816	99	22	22,0	cyclictest	0	-21	swapper/2	22:30:16	2
2922816	99	22	21,1	cyclictest	3106024	-21	idleruntime-cro	00:25:00	2
2922816	99	22	21,1	cyclictest	3030962	-21	apt-get	22:15:15	2
2922816	99	22	0,21	cyclictest	3059869	-21	apt-get	23:05:13	2
2922816	99	22	0,21	cyclictest	2990008	-21	idleruntime-cro	21:04:59	2
2922816	99	22	0,21	cyclictest	0	-21	swapper/2	22:35:12	2
2922816	99	22	0,21	cyclictest	0	-21	swapper/2	22:10:12	2
2922816	99	22	0,21	cyclictest	0	-21	swapper/2	20:30:13	2
2922816	99	22	0,21	cyclictest	0	-21	swapper/2	20:10:13	2
2922816	99	22	0,21	cyclictest	0	-21	swapper/2	19:40:13	2
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17