OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rackeslot5s.osadl.org (updated Fri Dec 26, 2025 12:44:23)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
14757	99	25	23,1	cyclictest	29244	-21	lockfile-create	07:40:00	1
14761	99	24	0,0	cyclictest	0	-21	swapper/2	08:40:22	2
14757	99	24	0,23	cyclictest	0	-21	swapper/1	07:30:21	1
14753	99	24	0,23	cyclictest	0	-21	swapper/0	11:00:24	0
14753	99	24	0,1	cyclictest	0	-21	swapper/0	11:00:00	0
14763	99	23	23,0	cyclictest	0	-21	swapper/3	12:10:24	3
14763	99	23	21,1	cyclictest	0	-21	swapper/3	11:00:00	3
14763	99	23	20,2	cyclictest	10532	-21	munin-run	10:20:00	3
14761	99	23	21,1	cyclictest	26718	-21	perf	07:35:00	2
14761	99	23	20,2	cyclictest	19520	-21	perf	07:20:01	2
14761	99	23	0,2	cyclictest	3316	-21	latency_hist	10:05:00	2
14757	99	23	22,1	cyclictest	7102	-21	apt-get	08:00:24	1
14757	99	23	21,1	cyclictest	21280	-21	perf	08:30:00	1
14757	99	23	0,22	cyclictest	4287	-21	apt-get	07:55:21	1
14757	99	23	0,22	cyclictest	0	-21	swapper/1	11:35:25	1
14753	99	23	21,1	cyclictest	30888	-21	munin-run	08:50:01	0
14753	99	23	20,2	cyclictest	3681	-21	munin-plugin-st	09:00:01	0
14763	99	22	22,0	cyclictest	0	-21	swapper/3	08:40:22	3
14763	99	22	0,21	cyclictest	6416	-21	perf	08:00:01	3
14763	99	22	0,21	cyclictest	3314	-21	perf	10:05:00	3
14763	99	22	0,21	cyclictest	27762	-21	latency_hist	10:55:00	3
14763	99	22	0,21	cyclictest	10807	-21	apt-get	10:20:23	3
14763	99	22	0,1	cyclictest	30922	-21	turbostat	09:55:01	3
14763	99	22	0,1	cyclictest	26104	-21	cron	09:45:01	3
14761	99	22	20,1	cyclictest	8744	-21	apt-get	09:10:22	2
14761	99	22	20,1	cyclictest	1697	-21	apt-get	12:15:24	2
14761	99	22	20,1	cyclictest	0	-21	swapper/2	08:50:02	2
14761	99	22	19,2	cyclictest	14625	-21	dump-pmu-power	11:35:01	2
14761	99	22	0,21	cyclictest	30195	-21	sendmail-msp	11:00:00	2
14761	99	22	0,21	cyclictest	24512	-21	apt-get	11:55:23	2
14761	99	22	0,21	cyclictest	21274	-21	idleruntime-cro	08:30:00	2
14761	99	22	0,21	cyclictest	10528	-21	perf	10:20:00	2
14761	99	22	0,21	cyclictest	0	-21	swapper/2	08:35:22	2
14761	99	22	0,21	cyclictest	0	-21	swapper/2	08:15:21	2
14761	99	22	0,1	cyclictest	20918	-21	apt-get	09:35:02	2
14757	99	22	20,1	cyclictest	3600	-21	apt-get	10:05:24	1
14757	99	22	20,1	cyclictest	30471	-21	apt-get	11:00:24	1
14757	99	22	20,1	cyclictest	2569	-21	sh	11:10:01	1
14757	99	22	0,21	cyclictest	26408	-21	apt-get	09:45:23	1
14757	99	22	0,21	cyclictest	23728	-21	sendmail-msp	09:40:01	1
14757	99	22	0,21	cyclictest	22201	-21	apt-get	07:25:21	1
14757	99	22	0,21	cyclictest	0	-21	swapper/1	12:20:20	1
14757	99	22	0,21	cyclictest	0	-21	swapper/1	10:00:23	1
14757	99	22	0,21	cyclictest	0	-21	swapper/1	09:30:21	1
14757	99	22	0,21	cyclictest	0	-21	swapper/1	07:50:22	1
14757	99	22	0,1	cyclictest	905	-21	perf	10:00:01	1
14757	99	22	0,1	cyclictest	31448	-21	perf	12:10:01	1
14757	99	22	0,1	cyclictest	0	-21	swapper/1	11:35:01	1
14753	99	22	20,1	cyclictest	3953	-21	apt-get	09:00:23	0
14753	99	22	20,1	cyclictest	29322	-21	apt-get	12:05:22	0
14753	99	22	20,1	cyclictest	26708	-21	awk	07:35:00	0
14753	99	22	20,1	cyclictest	26656	-21	lockfile-create	12:00:01	0
14753	99	22	20,1	cyclictest	1410	-21	perf	12:15:01	0
14753	99	22	0,2	cyclictest	13720	-21	grep	09:20:00	0
14753	99	22	0,21	cyclictest	7662	-21	apt-get	11:20:24	0
14753	99	22	0,21	cyclictest	4016	-21	perf	07:55:01	0
14753	99	22	0,21	cyclictest	20866	-21	apt-get	10:40:22	0
14753	99	22	0,21	cyclictest	18528	-21	awk	09:30:00	0
14753	99	22	0,1	cyclictest	31198	-21	apt-get	09:55:23	0
14753	99	22	0,1	cyclictest	26927	-21	apt-get	12:00:21	0
14753	99	22	0,1	cyclictest	23685	-21	cron	09:40:01	0
14763	99	21	21,0	cyclictest	0	-21	swapper/3	09:30:21	3
14763	99	21	20,0	cyclictest	0	-21	swapper/3	12:20:20	3
14763	99	21	20,0	cyclictest	0	-21	swapper/3	11:15:21	3
14763	99	21	20,0	cyclictest	0	-21	swapper/3	11:15:21	3
14763	99	21	0,20	cyclictest	26727	-21	awk	07:35:00	3
14763	99	21	0,20	cyclictest	14911	-21	apt-get	11:35:25	3
14763	99	21	0,20	cyclictest	10837	-21	apt-get	09:15:02	3
14763	99	21	0,20	cyclictest	0	-21	swapper/3	12:30:23	3
14763	99	21	0,20	cyclictest	0	-21	swapper/3	09:20:23	3
14763	99	21	0,20	cyclictest	0	-21	swapper/3	09:05:21	3
14763	99	21	0,20	cyclictest	0	-21	swapper/3	08:00:24	3
14763	99	21	0,0	cyclictest	19159	-21	apt-get	08:25:24	3
14763	99	21	0,0	cyclictest	0	-21	swapper/3	10:35:02	3
14761	99	21	21,0	cyclictest	0	-21	swapper/2	10:10:22	2
14761	99	21	20,0	cyclictest	6344	-21	apt-get	09:05:21	2
14761	99	21	20,0	cyclictest	0	-21	swapper/2	07:45:22	2
14761	99	21	0,21	cyclictest	0	-21	swapper/2	07:55:21	2
14761	99	21	0,20	cyclictest	6513	-21	apt-get	12:25:24	2
14761	99	21	0,20	cyclictest	0	-21	swapper/2	11:35:25	2
14761	99	21	0,20	cyclictest	0	-21	swapper/2	08:20:23	2
14761	99	21	0,0	cyclictest	0	-21	swapper/2	12:20:20	2
14761	99	21	0,0	cyclictest	0	-21	swapper/2	11:50:22	2
14761	99	21	0,0	cyclictest	0	-21	swapper/2	10:40:22	2
14757	99	21	21,0	cyclictest	0	-21	swapper/1	12:25:24	1
14757	99	21	20,1	cyclictest	19719	-21	apt-get	11:45:22	1
14757	99	21	20,0	cyclictest	18450	-21	apt-get	10:35:23	1
14757	99	21	19,1	cyclictest	19453	-21	munin-run	11:45:00	1
14757	99	21	0,21	cyclictest	0	-21	swapper/1	09:10:22	1
14757	99	21	0,20	cyclictest	5239	-21	apt-get	11:15:21	1
14757	99	21	0,20	cyclictest	5239	-21	apt-get	11:15:21	1
14757	99	21	0,20	cyclictest	0	-21	swapper/1	12:35:20	1
14757	99	21	0,0	cyclictest	16765	-21	apt-get	08:20:23	1
14757	99	21	0,0	cyclictest	11888	-21	apt-get	08:10:23	1
14757	99	21	0,0	cyclictest	0	-21	swapper/1	09:15:02	1
14757	99	21	0,0	cyclictest	0	-21	swapper/1	07:45:22	1
14753	99	21	21,0	cyclictest	12447	-21	apt-get	11:30:21	0
14753	99	21	21,0	cyclictest	0	-21	swapper/0	10:30:22	0
14753	99	21	21,0	cyclictest	0	-21	swapper/0	08:50:23	0
14753	99	21	21,0	cyclictest	0	-21	swapper/0	08:35:22	0
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2020-09-17 - 11:29