OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #9, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack9slot2.osadl.org (updated Thu Nov 27, 2025 00:43:25)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
6808	2	316	280,24	sleep0	0	-21	swapper/0		0
6992	2	303	220,68	sleep3	0	-21	swapper/3		3
6340	2	279	241,24	sleep2	0	-21	swapper/2		2
7185	99	265	261,2	cyclictest	0	-21	swapper/3		3
6727	2	256	186,26	sleep1	0	-21	swapper/1		1
7182	99	235	231,2	cyclictest	0	-21	swapper/2		2
7176	99	168	164,2	cyclictest	0	-21	swapper/1		1
7170	99	154	149,3	cyclictest	18577	-21	strings		0
7399	2	55	0,3	sleep3	0	-21	swapper/3		3
25687	2	41	0,2	sleep3	0	-21	swapper/3		3
7185	99	36	14,19	cyclictest	9791	-21	sed		3
7185	99	36	12,21	cyclictest	20115	-21	wc		3
7185	99	35	2,9	cyclictest	4292	-21	apt-get		3
15725	2	35	0,2	sleep2	0	-21	swapper/2		2
7185	99	34	3,7	cyclictest	25878	-21	apt-get		3
7185	99	34	2,9	cyclictest	1933	-21	apt-get		3
7185	99	34	2,6	cyclictest	12055	-21	apt-get		3
7185	99	34	1,7	cyclictest	10421	-21	apt-get		3
7185	99	34	12,3	cyclictest	10172	-21	if_enp2s0		3
7185	99	34	10,4	cyclictest	26643	-21	expr		3
15244	2	34	0,2	sleep3	0	-21	swapper/3		3
7185	99	32	9,4	cyclictest	10136	-21	cron		3
7185	99	32	2,5	cyclictest	0	-21	swapper/3		3
7185	99	32	10,4	cyclictest	16151	-21	fw_packets		3
7185	99	31	9,3	cyclictest	12294	2	chrt		3
7185	99	31	2,8	cyclictest	13676	-21	apt-get		3
7185	99	31	2,6	cyclictest	6746	-21	apt-get		3
7185	99	31	2,6	cyclictest	637	-21	apt-get		3
7185	99	31	2,6	cyclictest	28311	-21	apt-get		3
7185	99	31	1,4	cyclictest	16111	-21	apt-get		3
7185	99	31	11,3	cyclictest	4035	-21	latency_hist		3
7185	99	31	11,3	cyclictest	24288	-21	cut		3
7185	99	31	11,3	cyclictest	18490	-21	cstates		3
7185	99	31	10,3	cyclictest	10696	-21	wc		3
7185	99	30	9,3	cyclictest	9546	-21	ls		3
7185	99	30	9,3	cyclictest	28143	-21	cat		3
7185	99	30	9,3	cyclictest	20422	-21	cpuspeed_turbos		3
7185	99	30	7,20	cyclictest	0	-21	swapper/3		3
7185	99	30	2,9	cyclictest	21801	-21	sort		3
7185	99	30	2,9	cyclictest	21801	-21	sort		3
7185	99	30	1,6	cyclictest	19779	-21	apt-get		3
7185	99	30	1,5	cyclictest	22916	-21	apt-get		3
7185	99	30	10,2	cyclictest	2798	-21	timerwakeupswit		3
7185	99	29	9,2	cyclictest	7117	-21	cpuspeed		3
7185	99	29	9,2	cyclictest	30778	-21	gltestperf		3
7185	99	29	8,3	cyclictest	28052	-21	latency_hist		3
7185	99	29	8,3	cyclictest	16645	-21	basename		3
7185	99	29	6,4	cyclictest	26742	-21	readlink		3
7185	99	29	5,5	cyclictest	2022	-21	sendmail		3
7185	99	29	5,4	cyclictest	509	-21	systemd-logind		3
7185	99	29	2,7	cyclictest	16811	-21	apt-get		3
7185	99	29	2,6	cyclictest	8788	-21	apt-get		3
7185	99	29	2,5	cyclictest	12050	-21	apt-get		3
7185	99	29	2,4	cyclictest	20183	-21	apt-get		3
7185	99	29	2,4	cyclictest	20183	-21	apt-get		3
7185	99	28	9,2	cyclictest	4805	-21	basename		3
7185	99	28	3,8	cyclictest	665	-21	runrttasks		3
7185	99	28	3,5	cyclictest	7363	-21	apt-get		3
7185	99	28	3,5	cyclictest	7363	-21	apt-get		3
7185	99	28	2,6	cyclictest	3591	-21	apt-get		3
7185	99	28	2,5	cyclictest	31962	-21	apt-get		3
7185	99	28	2,5	cyclictest	27903	-21	apt-get		3
7185	99	28	2,4	cyclictest	32358	-21	apt-get		3
7185	99	28	1,5	cyclictest	30068	-21	cron		3
6845	2	28	0,13	sleep2	28	-21	ksoftirqd/2		2
7185	99	27	3,7	cyclictest	665	-21	runrttasks		3
7185	99	27	3,7	cyclictest	665	-21	runrttasks		3
7185	99	27	2,6	cyclictest	14069	-21	apt-get		3
21857	2	27	0,12	sleep0	9	-21	ksoftirqd/0		0
7185	99	26	2,5	cyclictest	24234	-21	apt-get		3
7185	99	26	2,5	cyclictest	24234	-21	apt-get		3
7185	99	26	1,5	cyclictest	30338	-21	unixbench_multi		3
7182	99	26	8,9	cyclictest	28	-21	ksoftirqd/2		2
7182	99	26	8,9	cyclictest	28	-21	ksoftirqd/2		2
7185	99	25	8,2	cyclictest	665	-21	runrttasks		3
7185	99	25	3,4	cyclictest	3987	-21	apt-get		3
7182	99	25	4,6	cyclictest	28314	-21	taskset		2
7182	99	25	2,7	cyclictest	12125	-21	taskset		2
27520	2	25	0,2	sleep0	0	-21	swapper/0		0
7185	99	24	1,5	cyclictest	5759	-21	apt-get		3
7182	99	23	2,6	cyclictest	27708	-21	taskset		2
7182	99	23	2,18	cyclictest	19173	-21	grep		2
7182	99	23	2,18	cyclictest	19173	-21	grep		2
7182	99	23	1,8	cyclictest	0	-21	swapper/2		2
7182	99	23	1,6	cyclictest	23852	-21	taskset		2
7182	99	23	1,6	cyclictest	0	-21	swapper/2		2
7176	99	23	0,7	cyclictest	21403	-21	sed		1
7170	99	23	17,4	cyclictest	24676	-21	cron		0
8742	2	22	0,2	sleep2	0	-21	swapper/2		2
7185	99	22	15,4	cyclictest	2082	-21	systemd-journal		3
7182	99	22	3,17	cyclictest	30789	-21	cat		2
7182	99	22	1,18	cyclictest	25997	-21	munin-run		2
7182	99	22	1,18	cyclictest	25997	-21	munin-run		2
7182	99	22	0,9	cyclictest	32437	-21	apt-config		2
7176	99	22	9,7	cyclictest	4720	-21	apt-get		1
7176	99	22	16,4	cyclictest	505	-21	dbus-daemon		1
7176	99	22	16,4	cyclictest	20641	-21	basename		1
7176	99	22	16,4	cyclictest	20641	-21	basename		1
7176	99	22	13,5	cyclictest	505	-21	dbus-daemon		1
7170	99	22	15,4	cyclictest	3525	-21	sed		0
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2021-04-10 - 20:27