OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack7slot4s.osadl.org (updated Tue Jul 14, 2026 00:45:13)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
24092	99	62	3,50	cyclictest	24623	-21	ssh	22:30:12	2
24092	99	59	36,9	cyclictest	9935	-21	diskmemload	23:20:13	2
24092	99	49	25,17	cyclictest	24770	-21	apt-get	00:00:14	2
24072	99	49	2,36	cyclictest	19273	-21	ssh	22:45:12	0
24092	99	46	32,9	cyclictest	28372	-21	apt-get	21:20:16	2
24092	99	45	1,37	cyclictest	1481	-21	sshd	19:20:13	2
24092	99	44	34,3	cyclictest	5388	-21	apt-get	21:05:14	2
24072	99	44	3,8	cyclictest	13	1	rcu_preempt	21:10:12	0
24101	99	43	2,25	cyclictest	359	66	irq/58-eth0	21:45:13	3
24092	99	43	30,9	cyclictest	10308	-21	apt-get	21:10:13	2
24092	99	42	33,3	cyclictest	2772	-21	apt-get	19:55:15	2
24101	99	41	4,24	cyclictest	359	66	irq/58-eth0	22:25:13	3
24092	99	41	12,15	cyclictest	19553	-21	ssh	23:01:32	2
24084	99	41	3,27	cyclictest	6715	-21	ssh	23:50:13	1
24072	99	41	1,33	cyclictest	2956	-21	ssh	23:30:11	0
24101	99	40	2,22	cyclictest	359	66	irq/58-eth0	21:40:13	3
24072	99	40	20,9	cyclictest	12	-21	ksoftirqd/0	22:05:13	0
24072	99	40	1,8	cyclictest	1311	-21	ssh	00:05:15	0
24092	99	39	31,4	cyclictest	24283	-21	apt-get	19:10:11	2
24092	99	39	2,29	cyclictest	8465	-21	df	23:15:15	2
24101	99	38	11,15	cyclictest	359	66	irq/58-eth0	20:10:11	3
24092	99	38	4,14	cyclictest	0	-21	swapper/2	22:40:14	2
24092	99	38	31,3	cyclictest	15714	-21	apt-get	23:55:15	2
24092	99	38	12,6	cyclictest	9935	-21	diskmemload	00:17:27	2
24084	99	38	1,30	cyclictest	11959	-21	ssh	23:35:13	1
24092	99	37	17,12	cyclictest	0	-21	swapper/2	00:10:14	2
24072	99	37	2,26	cyclictest	13895	-21	ssh	23:00:14	0
24101	99	36	5,18	cyclictest	359	66	irq/58-eth0	00:05:14	3
24101	99	36	3,21	cyclictest	359	66	irq/58-eth0	22:45:10	3
24101	99	36	3,20	cyclictest	359	66	irq/58-eth0	22:50:13	3
24092	99	36	7,8	cyclictest	30105	-21	ntp_states	23:25:27	2
24092	99	36	7,15	cyclictest	31264	-21	usage_EthcTask	20:20:30	2
24092	99	36	6,14	cyclictest	4414	-21	irqcore	20:30:18	2
24092	99	36	16,15	cyclictest	0	-21	swapper/2	00:20:10	2
24084	99	36	1,29	cyclictest	10330	-21	ssh	21:10:12	1
24101	99	35	18,8	cyclictest	0	-21	swapper/3	23:55:13	3
24092	99	35	9,20	cyclictest	22820	-21	sh	23:05:13	2
24092	99	35	17,9	cyclictest	1299	-21	apt-get	00:05:17	2
24084	99	35	3,28	cyclictest	1212	-21	ssh	22:35:13	1
24101	99	34	4,24	cyclictest	359	66	irq/58-eth0	22:00:12	3
24092	99	34	8,16	cyclictest	0	-21	swapper/2	19:50:14	2
24092	99	34	7,13	cyclictest	21159	-21	ssh	21:33:27	2
24084	99	34	4,27	cyclictest	20925	-21	ssh	22:10:13	1
24084	99	34	3,27	cyclictest	19372	-21	ssh	21:15:15	1
24101	99	33	2,21	cyclictest	359	66	irq/58-eth0	00:20:11	3
24101	99	33	2,18	cyclictest	359	66	irq/58-eth0	23:20:12	3
24092	99	33	9,18	cyclictest	0	-21	swapper/2	22:05:13	2
24092	99	33	7,12	cyclictest	135	-21	systemd-journal	19:40:01	2
24092	99	33	7,12	cyclictest	135	-21	systemd-journal	19:40:01	2
24092	99	33	5,23	cyclictest	0	-21	swapper/2	22:17:14	2
24092	99	33	27,3	cyclictest	11948	-21	apt-get	23:35:15	2
24092	99	33	17,13	cyclictest	0	-21	swapper/2	22:20:13	2
24092	99	33	12,18	cyclictest	9935	-21	diskmemload	00:25:33	2
24084	99	33	3,26	cyclictest	22853	-21	ssh	21:35:13	1
24084	99	33	2,27	cyclictest	22815	-21	ssh	23:05:14	1
24092	99	32	9,6	cyclictest	23748	-21	kthreadcore	23:40:21	2
24092	99	32	9,6	cyclictest	23748	-21	kthreadcore	23:40:21	2
24092	99	32	7,10	cyclictest	14135	-21	ssh	23:53:34	2
24092	99	32	6,10	cyclictest	5235	-21	rm	23:48:44	2
24084	99	32	1,28	cyclictest	4789	-21	ssh	00:25:15	1
24072	99	32	4,19	cyclictest	13708	-21	apt-get	00:30:12	0
24072	99	32	2,17	cyclictest	4893	-21	ssh	21:25:13	0
24072	99	32	2,17	cyclictest	4893	-21	ssh	21:25:13	0
24072	99	32	18,12	cyclictest	12	-21	ksoftirqd/0	00:15:11	0
24101	99	31	3,23	cyclictest	359	66	irq/58-eth0	22:30:14	3
24101	99	31	2,20	cyclictest	359	66	irq/58-eth0	22:40:13	3
24101	99	31	15,10	cyclictest	0	-21	swapper/3	20:20:12	3
24092	99	31	5,9	cyclictest	0	-21	swapper/2	21:47:48	2
24092	99	31	5,7	cyclictest	0	-21	swapper/2	21:53:19	2
24092	99	31	3,15	cyclictest	167	-21	systemd-network	19:15:12	2
24092	99	31	2,11	cyclictest	930	-21	snmpd	22:12:13	2
24084	99	31	24,3	cyclictest	20	1	rcuc/1	19:50:14	1
24084	99	31	19,9	cyclictest	21	-21	ksoftirqd/1	19:20:14	1
24084	99	31	1,20	cyclictest	0	-21	swapper/1	21:05:13	1
24072	99	31	8,17	cyclictest	13871	-21	diskmemload	21:30:12	0
24072	99	31	2,22	cyclictest	21127	-21	ssh	23:40:14	0
24072	99	31	2,22	cyclictest	21127	-21	ssh	23:40:14	0
24072	99	31	1,19	cyclictest	22670	-21	ssh	00:35:13	0
24072	99	31	1,19	cyclictest	22670	-21	ssh	00:35:13	0
24101	99	30	4,22	cyclictest	359	66	irq/58-eth0	20:30:14	3
24101	99	30	1,19	cyclictest	359	66	irq/58-eth0	19:20:13	3
24092	99	30	8,8	cyclictest	17000	-21	cron	20:10:01	2
24092	99	30	6,10	cyclictest	14589	-21	ntp_192.168.114	19:30:25	2
24092	99	30	5,12	cyclictest	30960	-21	ssh	00:39:45	2
24092	99	30	5,12	cyclictest	30960	-21	ssh	00:39:44	2
24092	99	30	17,10	cyclictest	273	46	irq/40-dwc2_hso	21:35:14	2
24092	99	30	14,10	cyclictest	0	-21	swapper/2	20:40:13	2
24092	99	30	14,10	cyclictest	0	-21	swapper/2	20:40:13	2
24084	99	30	3,24	cyclictest	10276	-21	ssh	22:40:14	1
24084	99	30	2,25	cyclictest	13857	-21	ssh	21:30:11	1
24072	99	30	6,11	cyclictest	31811	-21	ssh	23:10:12	0
24072	99	30	2,22	cyclictest	2935	-21	ssh	22:00:12	0
24072	99	30	2,22	cyclictest	12076	-21	ssh	22:21:15	0
24072	99	30	1,5	cyclictest	13	1	rcu_preempt	19:50:14	0
24101	99	29	5,13	cyclictest	359	66	irq/58-eth0	19:30:14	3
24092	99	29	9,10	cyclictest	23435	-21	/usr/sbin/munin	21:15:32	2
24092	99	29	5,10	cyclictest	0	-21	swapper/2	21:42:17	2
24092	99	29	3,21	cyclictest	9935	-21	diskmemload	22:50:14	2
24092	99	29	15,11	cyclictest	0	-21	swapper/2	21:00:13	2
24092	99	29	1,20	cyclictest	4919	-21	ssh	21:25:15	2
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17