OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #a, slot #6

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system

Characteristics of the 792 highest latencies:

System rackaslot6.osadl.org (updated Tue Apr 28, 2026 18:45:37)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
163903	99	60	35,20	cyclictest	683734	-21	spawn		7
163898	99	57	1,44	cyclictest	0	-21	swapper/2		2
163898	99	57	1,44	cyclictest	0	-21	swapper/2		2
163896	99	57	2,31	cyclictest	524857	-21	kworker/u32:11+rpciod		0
163896	99	57	2,31	cyclictest	524857	-21	kworker/u32:11+rpciod		0
163898	99	53	10,41	cyclictest	970449	-21	spawn		2
163896	99	53	9,31	cyclictest	0	-21	swapper/0		0
163896	99	53	16,34	cyclictest	624023	-21	spawn		0
163900	99	52	17,32	cyclictest	1083397	-21	spawn		4
163903	99	50	23,6	cyclictest	524872	-21	execl		7
163900	99	50	36,12	cyclictest	522785	-21	Run		4
163896	99	49	29,14	cyclictest	524887	-21	execl		0
163902	99	48	8,27	cyclictest	802768	1	spawn		6
163899	99	47	41,4	cyclictest	802893	-21	spawn		3
163899	99	46	40,3	cyclictest	524848	-21	execl		3
163897	99	46	40,4	cyclictest	885372	-21	spawn		1
163897	99	46	40,4	cyclictest	524918	-21	execl		1
163896	99	46	3,2	cyclictest	525541	-21	fstime		0
163896	99	45	16,26	cyclictest	1441642	-21	grep		0
163896	99	45	16,26	cyclictest	1441642	-21	grep		0
163901	99	44	37,5	cyclictest	527588	-21	spawn		5
163900	99	44	12,29	cyclictest	524877	-21	execl		4
163903	99	43	14,6	cyclictest	509431	-21	sh		7
163901	99	43	16,8	cyclictest	225	50	irq/128-46000000.ethernet-tx2		5
163901	99	43	16,8	cyclictest	225	50	irq/128-46000000.ethernet-tx2		5
163897	99	43	37,4	cyclictest	522779	-21	Run		1
163902	99	42	4,26	cyclictest	171136	-21	kworker/u32:9+xprtiod		6
163902	99	42	35,4	cyclictest	524887	-21	execl		6
163902	99	42	1,27	cyclictest	1430765	-21	od		6
163902	99	42	1,27	cyclictest	1430765	-21	od		6
163899	99	42	18,5	cyclictest	1357147	-21	syscall		3
163899	99	42	18,5	cyclictest	1357147	-21	syscall		3
163896	99	42	3,15	cyclictest	139866	-21	kworker/u32:11+rpciod		0
163896	99	42	3,15	cyclictest	139866	-21	kworker/u32:11+rpciod		0
163902	99	41	1,34	cyclictest	0	-21	swapper/6		6
163901	99	41	34,5	cyclictest	524843	-21	execl		5
163898	99	41	4,33	cyclictest	524919	-21	execl		2
163903	99	40	23,14	cyclictest	1383703	-21	sh		7
163903	99	40	23,14	cyclictest	1383703	-21	sh		7
163900	99	40	7,8	cyclictest	231	50	irq/135-46000000.ethernet-rx0		4
163900	99	40	7,8	cyclictest	231	50	irq/135-46000000.ethernet-rx0		4
163896	99	40	1,30	cyclictest	145618	-21	kworker/u32:2+rpciod		0
163903	99	39	20,6	cyclictest	1	-21	systemd		7
163902	99	39	2,27	cyclictest	498032	-21	sh		6
163900	99	39	9,3	cyclictest	0	-21	swapper/4		4
163900	99	39	9,3	cyclictest	0	-21	swapper/4		4
163897	99	39	30,5	cyclictest	1382065	-21	multi.sh		1
163897	99	39	30,5	cyclictest	1382065	-21	multi.sh		1
163896	99	39	1,17	cyclictest	0	-21	swapper/0		0
163903	99	38	23,4	cyclictest	1444067	-21	sh		7
163903	99	38	23,4	cyclictest	1444067	-21	sh		7
163900	99	38	7,4	cyclictest	505957	-21	grep		4
163898	99	38	4,27	cyclictest	496452	-21	od		2
163898	99	38	11,15	cyclictest	524764	-21	kworker/u32:6+rpciod		2
163898	99	38	11,15	cyclictest	524764	-21	kworker/u32:6+rpciod		2
163896	99	38	1,28	cyclictest	1461103	-21	ntpq		0
163903	99	37	24,4	cyclictest	525543	-21	Run		7
163902	99	36	2,25	cyclictest	1440551	-21	multi.sh		6
163902	99	36	2,25	cyclictest	1440551	-21	multi.sh		6
163897	99	36	30,4	cyclictest	509524	-21	sh		1
163896	99	36	2,24	cyclictest	0	-21	swapper/0		0
163899	99	35	14,4	cyclictest	1447278	-21	sh		3
163899	99	35	14,4	cyclictest	1447278	-21	sh		3
163898	99	35	1,28	cyclictest	0	-21	swapper/2		2
163896	99	35	1,24	cyclictest	0	-21	swapper/0		0
163896	99	35	1,16	cyclictest	1457959	-21	ntpq		0
163901	99	34	28,3	cyclictest	514610	-21	sh		5
163898	99	34	1,22	cyclictest	0	-21	swapper/2		2
163896	99	34	1,24	cyclictest	0	-21	swapper/0		0
163901	99	33	29,2	cyclictest	1451140	-21	sh		5
163901	99	33	29,2	cyclictest	1451140	-21	sh		5
163900	99	33	29,3	cyclictest	525391	-21	Run		4
163896	99	33	14,12	cyclictest	164862	-21	ntpq		0
163896	99	33	14,11	cyclictest	0	-21	swapper/0		0
163902	99	32	1,9	cyclictest	231	50	irq/135-46000000.ethernet-rx0		6
163902	99	32	1,19	cyclictest	169042	-21	execl		6
163900	99	32	16,13	cyclictest	427428	-21	sh		4
163900	99	32	16,13	cyclictest	427428	-21	sh		4
163900	99	32	1,2	cyclictest	0	-21	swapper/4		4
163900	99	32	1,19	cyclictest	1457881	-21	ld		4
163897	99	32	27,3	cyclictest	1441327	-21	od		1
163897	99	32	27,3	cyclictest	1441327	-21	od		1
163902	99	31	1,18	cyclictest	0	-21	swapper/6		6
163900	99	31	1,2	cyclictest	0	-21	swapper/4		4
163900	99	31	1,2	cyclictest	0	-21	swapper/4		4
163899	99	31	25,3	cyclictest	522844	-21	Run		3
163899	99	31	25,3	cyclictest	149770	-21	kworker/u32:17+rpciod		3
163896	99	31	19,8	cyclictest	0	-21	swapper/0		0
163896	99	31	16,8	cyclictest	231	50	irq/135-46000000.ethernet-rx0		0
163896	99	31	16,8	cyclictest	231	50	irq/135-46000000.ethernet-rx0		0
163901	99	30	9,8	cyclictest	231	50	irq/135-46000000.ethernet-rx0		5
163900	99	30	1,2	cyclictest	0	-21	swapper/4		4
163900	99	30	1,2	cyclictest	0	-21	swapper/4		4
163900	99	30	0,2	cyclictest	0	-21	swapper/4		4
163898	99	30	4,16	cyclictest	0	-21	swapper/2		2
163898	99	30	1,3	cyclictest	0	-21	swapper/2		2
163896	99	30	3,2	cyclictest	0	-21	swapper/0		0
163896	99	30	3,2	cyclictest	0	-21	swapper/0		0
163900	99	29	1,2	cyclictest	0	-21	swapper/4		4
163898	99	29	1,2	cyclictest	1457312	-21	apt-get		2
163898	99	29	1,18	cyclictest	0	-21	swapper/2		2
163897	99	29	13,3	cyclictest	0	-21	swapper/1		1
163903	99	28	6,18	cyclictest	171613	-21	spawn		7
163903	99	28	6,18	cyclictest	171613	-21	spawn		7
163902	99	28	8,12	cyclictest	225	50	irq/128-46000000.ethernet-tx2		6
163902	99	28	17,8	cyclictest	0	-21	swapper/6		6
163900	99	28	1,2	cyclictest	0	-21	swapper/4		4
163898	99	28	15,9	cyclictest	0	-21	swapper/2		2
163898	99	28	15,9	cyclictest	0	-21	swapper/2		2
163902	99	27	14,11	cyclictest	199655	-21	spawn		6
163902	99	27	14,11	cyclictest	199655	-21	spawn		6
163902	99	27	14,10	cyclictest	0	-21	swapper/6		6
163902	99	27	0,20	cyclictest	1461810	-21	apt-get		6
163900	99	27	0,2	cyclictest	0	-21	swapper/4		4
163900	99	27	0,2	cyclictest	0	-21	swapper/4		4
163898	99	27	1,5	cyclictest	169784	-21	latency_hist		2
163898	99	27	1,1	cyclictest	0	-21	swapper/2		2
163898	99	27	1,19	cyclictest	1455562	-21	fschecks_time		2
163902	99	26	13,11	cyclictest	165559	-21	cat		6
163902	99	26	1,21	cyclictest	0	-21	swapper/6		6
163902	99	26	1,21	cyclictest	0	-21	swapper/6		6
163902	99	26	1,1	cyclictest	167569	-21	dhry2reg		6
163902	99	26	1,18	cyclictest	0	-21	swapper/6		6
163902	99	26	1,18	cyclictest	0	-21	swapper/6		6
163902	99	26	1,17	cyclictest	0	-21	swapper/6		6
163901	99	26	17,5	cyclictest	519806	-21	sh		5
163900	99	26	0,4	cyclictest	166691	-21	/usr/sbin/munin		4
163900	99	26	0,4	cyclictest	166691	-21	/usr/sbin/munin		4
163899	99	26	21,3	cyclictest	165445	-21	grep		3
163898	99	26	2,2	cyclictest	163877	-21	latency_hist		2
163898	99	26	20,4	cyclictest	164743	-21	if_eth0		2
163898	99	26	1,7	cyclictest	168329	-21	/usr/sbin/munin		2
163898	99	26	1,3	cyclictest	0	-21	swapper/2		2
163898	99	26	1,3	cyclictest	0	-21	swapper/2		2
163899	99	25	2,15	cyclictest	525456	-21	fstime		3
163898	99	25	1,18	cyclictest	0	-21	swapper/2		2
163898	99	25	1,18	cyclictest	0	-21	swapper/2		2
163897	99	25	3,18	cyclictest	0	-21	swapper/1		1
163897	99	25	2,18	cyclictest	0	-21	swapper/1		1
163897	99	25	19,4	cyclictest	1458720	-21	latency_hist		1
163903	99	24	1,4	cyclictest	0	-21	swapper/7		7
163901	99	24	20,2	cyclictest	1457239	-21	latency_hist		5
163899	99	24	5,16	cyclictest	0	-21	swapper/3		3
163897	99	24	3,2	cyclictest	231	50	irq/135-46000000.ethernet-rx0		1
163897	99	24	3,2	cyclictest	231	50	irq/135-46000000.ethernet-rx0		1
163897	99	24	20,2	cyclictest	525382	-21	Run		1
163896	99	24	9,4	cyclictest	164094	-21	latency_hist		0
163903	99	23	19,2	cyclictest	166456	-21	ntp_kernel_err		7
163903	99	23	19,2	cyclictest	166456	-21	ntp_kernel_err		7
163903	99	23	16,3	cyclictest	169896	-21	apt-get		7
163901	99	23	4,2	cyclictest	171613	-21	spawn		5
163901	99	23	4,2	cyclictest	171613	-21	spawn		5
163901	99	23	1,15	cyclictest	0	-21	swapper/5		5
163899	99	23	20,2	cyclictest	429008	-21	sh		3
163899	99	23	20,2	cyclictest	429008	-21	sh		3
163899	99	23	1,19	cyclictest	171146	-21	fstime		3
163903	99	22	17,2	cyclictest	0	-21	swapper/7		7
163903	99	22	1,17	cyclictest	0	-21	swapper/7		7
163903	99	22	1,17	cyclictest	0	-21	swapper/7		7
163903	99	22	1,17	cyclictest	0	-21	swapper/7		7
163903	99	22	1,17	cyclictest	0	-21	swapper/7		7
163901	99	22	18,2	cyclictest	0	-21	swapper/5		5
163901	99	22	17,3	cyclictest	1461677	-21	sed		5
163899	99	22	15,2	cyclictest	225	50	irq/128-46000000.ethernet-tx2		3
163899	99	22	1,17	cyclictest	0	-21	swapper/3		3
163897	99	22	19,2	cyclictest	1460520	-21	/usr/sbin/munin		1
163897	99	22	18,2	cyclictest	168099	-21	/usr/sbin/munin		1
163897	99	22	1,17	cyclictest	0	-21	swapper/1		1
163897	99	22	1,17	cyclictest	0	-21	swapper/1		1
163903	99	21	17,2	cyclictest	169042	-21	execl		7
163903	99	21	15,3	cyclictest	163954	-21	cat		7
163903	99	21	1,17	cyclictest	0	-21	swapper/7		7
163903	99	21	1,16	cyclictest	0	-21	swapper/7		7
163901	99	21	1,18	cyclictest	1457301	-21	acpi		5
163901	99	21	1,17	cyclictest	0	-21	swapper/5		5
163901	99	21	1,17	cyclictest	0	-21	swapper/5		5
163901	99	21	0,18	cyclictest	231	50	irq/135-46000000.ethernet-rx0		5
163901	99	21	0,18	cyclictest	1455323	-21	aten_rapower_en		5
163899	99	21	0,19	cyclictest	42	-21	rcuc/3		3
163897	99	21	2,16	cyclictest	164299	-21	apt-get		1
163897	99	21	1,17	cyclictest	0	-21	swapper/1		1
163897	99	21	1,17	cyclictest	0	-21	swapper/1		1
163897	99	21	1,17	cyclictest	0	-21	swapper/1		1
163903	99	20	1,16	cyclictest	0	-21	swapper/7		7
163901	99	20	15,3	cyclictest	1462296	-21	apt-get		5
163901	99	20	1,17	cyclictest	0	-21	swapper/5		5
163901	99	20	1,17	cyclictest	0	-21	swapper/5		5
163901	99	20	1,17	cyclictest	0	-21	swapper/5		5
163899	99	20	16,2	cyclictest	0	-21	swapper/3		3
163899	99	20	16,2	cyclictest	0	-21	swapper/3		3
163899	99	20	1,16	cyclictest	0	-21	swapper/3		3
163899	99	20	0,17	cyclictest	169042	-21	execl		3
163899	99	19	16,2	cyclictest	0	-21	swapper/3		3
163899	99	19	16,2	cyclictest	0	-21	swapper/3		3
163899	99	19	15,2	cyclictest	0	-21	swapper/3		3
163897	99	19	1,17	cyclictest	0	-21	swapper/1		1
163901	99	18	13,3	cyclictest	164173	-21	latency_hist		5
163900	99	16	2,5	cyclictest	231	50	irq/135-46000000.ethernet-rx0		4
163899	99	13	7,4	cyclictest	164165	-21	latency_hist		3
163897	99	13	8,4	cyclictest	163905	-21	latency_hist		1
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17