OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack7slot4s.osadl.org (updated Mon Aug 17, 2026 00:45:08)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
21437	99	56	40,12	cyclictest	359	66	irq/58-eth0	22:35:14	3
21420	99	54	4,9	cyclictest	23020	-21	apt-get	00:00:14	1
21409	99	51	19,21	cyclictest	12	-21	ksoftirqd/0	22:55:15	0
21409	99	46	29,10	cyclictest	4938	-21	apt-get	23:50:14	0
21429	99	45	3,35	cyclictest	930	-21	snmpd	23:30:14	2
21437	99	44	7,22	cyclictest	359	66	irq/58-eth0	22:30:14	3
21437	99	42	6,23	cyclictest	359	66	irq/58-eth0	23:35:14	3
21437	99	42	21,11	cyclictest	359	66	irq/58-eth0	00:35:13	3
21420	99	42	2,7	cyclictest	0	-21	swapper/1	20:50:13	1
21429	99	41	33,5	cyclictest	0	-21	swapper/2	21:00:14	2
21429	99	41	1,29	cyclictest	16604	-21	sh	21:15:13	2
21420	99	41	1,34	cyclictest	17295	-21	ssh	22:45:13	1
21420	99	41	1,12	cyclictest	19304	-21	apt-get	20:15:12	1
21437	99	40	1,21	cyclictest	359	66	irq/58-eth0	23:40:12	3
21420	99	40	7,31	cyclictest	22688	-21	ssh	22:30:15	1
21420	99	40	2,6	cyclictest	0	-21	swapper/1	00:25:14	1
21420	99	37	3,10	cyclictest	10366	-21	ssh	23:35:14	1
21420	99	37	1,7	cyclictest	1313	-21	apt-get	20:30:13	1
21409	99	37	9,11	cyclictest	17744	-21	ssh	22:09:10	0
21437	99	36	5,21	cyclictest	359	66	irq/58-eth0	21:50:13	3
21437	99	36	5,16	cyclictest	359	66	irq/58-eth0	23:00:15	3
21437	99	36	4,18	cyclictest	359	66	irq/58-eth0	00:00:15	3
21420	99	36	3,29	cyclictest	24105	-21	ssh	21:55:12	1
21420	99	36	1,9	cyclictest	19654	-21	grep	22:10:19	1
21420	99	36	1,12	cyclictest	11586	-21	sh	21:10:30	1
21409	99	36	29,3	cyclictest	25416	-21	apt-get	20:55:14	0
21409	99	36	18,11	cyclictest	13612	-21	ssh	22:25:17	0
21437	99	35	3,21	cyclictest	359	66	irq/58-eth0	21:30:14	3
21437	99	35	1,22	cyclictest	359	66	irq/58-eth0	00:15:13	3
21429	99	35	2,27	cyclictest	20320	-21	diskmemload	21:35:13	2
21420	99	35	2,6	cyclictest	0	-21	swapper/1	00:35:14	1
21420	99	35	1,9	cyclictest	25890	-21	irqcore	20:55:19	1
21420	99	35	1,9	cyclictest	14617	-21	cut	22:25:22	1
21420	99	35	1,8	cyclictest	29223	-21	sh	23:45:18	1
21420	99	35	12,20	cyclictest	13970	-21	/usr/sbin/munin	23:55:31	1
21420	99	35	1,15	cyclictest	12059	-21	rm	23:00:15	1
21420	99	35	1,11	cyclictest	5160	-21	fschecks_time	23:50:17	1
21420	99	35	1,10	cyclictest	31043	-21	kthreadcore	21:00:20	1
21409	99	35	28,3	cyclictest	20599	-21	apt-get	20:50:12	0
21409	99	35	24,8	cyclictest	0	-21	swapper/0	22:35:14	0
21437	99	34	4,18	cyclictest	359	66	irq/58-eth0	00:10:13	3
21437	99	34	12,15	cyclictest	359	66	irq/58-eth0	21:10:15	3
21429	99	34	19,12	cyclictest	0	-21	swapper/2	23:15:14	2
21420	99	34	3,5	cyclictest	0	-21	swapper/1	23:40:14	1
21420	99	34	2,9	cyclictest	4816	-21	timerandwakeup	22:00:29	1
21420	99	34	1,9	cyclictest	9359	-21	kthreadcore	00:10:20	1
21420	99	34	1,9	cyclictest	5144	-21	cut	22:20:19	1
21420	99	34	1,8	cyclictest	25460	-21	latency_hist	21:20:03	1
21420	99	34	1,8	cyclictest	14418	-21	ntp_192.168.114	20:40:24	1
21420	99	34	1,10	cyclictest	32616	-21	ssh	21:40:22	1
21420	99	34	0,9	cyclictest	31442	-21	tr	23:10:22	1
21409	99	34	7,10	cyclictest	16560	-21	ssh	22:44:38	0
21409	99	34	6,10	cyclictest	20829	-21	ssh	21:35:18	0
21437	99	33	4,23	cyclictest	359	66	irq/58-eth0	21:15:13	3
21437	99	33	3,17	cyclictest	359	66	irq/58-eth0	22:25:16	3
21420	99	33	2,10	cyclictest	2800	-21	fschecks_time	21:05:18	1
21420	99	33	1,9	cyclictest	28382	-21	cut	00:20:23	1
21420	99	33	1,9	cyclictest	16176	-21	irqcore	20:45:18	1
21420	99	33	1,9	cyclictest	12009	-21	kernelversion	21:30:20	1
21420	99	33	1,8	cyclictest	27768	-21	cut	19:15:19	1
21420	99	33	1,8	cyclictest	23577	-21	ssh	23:05:19	1
21420	99	33	1,8	cyclictest	15867	-21	fschecks_time	23:20:15	1
21420	99	33	1,8	cyclictest	11787	-21	cat	23:00:01	1
21420	99	33	1,7	cyclictest	14162	-21	cut	00:30:24	1
21420	99	33	1,11	cyclictest	25934	-21	hwlatdetect	21:20:16	1
21420	99	33	1,10	cyclictest	14674	-21	ls	21:50:01	1
21420	99	33	1,10	cyclictest	13370	-21	memory	22:05:24	1
21409	99	33	22,8	cyclictest	19377	-21	ssh	23:40:12	0
21409	99	33	20,8	cyclictest	6056	-21	ssh	21:45:14	0
21437	99	32	3,18	cyclictest	359	66	irq/58-eth0	20:30:13	3
21437	99	32	15,10	cyclictest	359	66	irq/58-eth0	22:50:13	3
21420	99	32	9,20	cyclictest	21	-21	ksoftirqd/1	19:50:13	1
21420	99	32	9,20	cyclictest	21	-21	ksoftirqd/1	19:50:13	1
21420	99	32	2,7	cyclictest	2817	-21	irqcore	21:25:22	1
21420	99	32	24,4	cyclictest	7124	-21	diskmemload	21:50:14	1
21420	99	32	2,10	cyclictest	17416	-21	ssh	00:15:14	1
21420	99	32	1,8	cyclictest	7128	-21	kthreadcore	20:35:21	1
21420	99	32	1,8	cyclictest	24763	-21	tr	19:45:23	1
21420	99	32	1,8	cyclictest	17881	-21	sendmail	19:40:01	1
21420	99	32	1,10	cyclictest	8603	-21	wc	20:00:31	1
21420	99	32	1,10	cyclictest	25404	-21	grep	19:10:29	1
21409	99	32	8,14	cyclictest	7124	-21	diskmemload	23:29:39	0
21409	99	32	7,10	cyclictest	27308	-21	ssh	23:07:16	0
21409	99	32	22,8	cyclictest	15055	-21	apt-get	21:50:14	0
21409	99	32	2,19	cyclictest	769	-21	ssh	22:00:14	0
21409	99	32	21,9	cyclictest	10354	-21	apt-get	23:35:15	0
21437	99	31	2,18	cyclictest	359	66	irq/58-eth0	20:40:13	3
21429	99	31	16,10	cyclictest	4919	-21	apt-get	20:00:15	2
21429	99	31	1,11	cyclictest	4544	-21	ssh	22:20:14	2
21420	99	31	4,5	cyclictest	15163	-21	kthreadcore	20:10:21	1
21420	99	31	2,14	cyclictest	26283	-21	ssh	22:50:13	1
21420	99	31	2,10	cyclictest	29029	-21	dpkg	21:40:01	1
21420	99	31	1,10	cyclictest	2432	-21	wc	19:20:28	1
21420	99	31	1,10	cyclictest	21811	-21	timerandwakeup	19:40:28	1
21437	99	30	3,17	cyclictest	359	66	irq/58-eth0	21:20:13	3
21437	99	30	15,10	cyclictest	359	66	irq/58-eth0	21:25:17	3
21429	99	30	5,15	cyclictest	273	46	irq/40-dwc2_hso	19:20:15	2
21429	99	30	4,13	cyclictest	273	46	irq/40-dwc2_hso	22:55:14	2
21429	99	30	2,18	cyclictest	32006	-21	ssh	00:05:16	2
21420	99	30	5,3	cyclictest	0	-21	swapper/1	22:41:16	1
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17