OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack7slot4s.osadl.org (updated Mon May 04, 2026 00:45:11)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
2852	99	59	34,8	cyclictest	21	-21	ksoftirqd/1	21:05:14	1
2863	99	58	3,29	cyclictest	9351	-21	ssh	23:45:13	2
2852	99	54	17,26	cyclictest	21	-21	ksoftirqd/1	22:10:14	1
2843	99	54	1,40	cyclictest	15974	-21	ssh	21:25:11	0
2863	99	51	21,19	cyclictest	630	-21	Async	09:36:31	2
2863	99	51	1,11	cyclictest	0	-21	swapper/2	20:50:14	2
2843	99	49	1,38	cyclictest	27439	-21	ssh	23:55:13	0
2876	99	48	5,22	cyclictest	359	66	irq/58-eth0	23:55:14	3
2852	99	47	21,17	cyclictest	21	-21	ksoftirqd/1	00:20:13	1
2876	99	46	4,28	cyclictest	359	66	irq/58-eth0	21:50:13	3
2876	99	46	20,15	cyclictest	359	66	irq/58-eth0	20:35:15	3
2843	99	46	4,39	cyclictest	21527	-21	ssh	22:40:15	0
2843	99	46	1,33	cyclictest	4160	-21	ssh	00:00:13	0
2876	99	43	18,20	cyclictest	359	66	irq/58-eth0	22:15:11	3
2863	99	43	1,27	cyclictest	1449	-21	ssh	21:35:15	2
2852	99	43	5,6	cyclictest	13	1	rcu_preempt	23:50:12	1
2852	99	43	3,5	cyclictest	13	1	rcu_preempt	19:15:12	1
2852	99	42	2,3	cyclictest	13	1	rcu_preempt	21:40:12	1
2863	99	41	7,30	cyclictest	2882	-21	unixbench_multi	00:15:28	2
2863	99	41	5,9	cyclictest	23467	-21	if_err_eth0	22:05:18	2
2852	99	41	6,26	cyclictest	2743	-21	ssh	00:35:13	1
2843	99	41	1,14	cyclictest	12	-21	ksoftirqd/0	22:20:10	0
2876	99	40	2,25	cyclictest	359	66	irq/58-eth0	22:20:13	3
2863	99	40	1,12	cyclictest	17729	-21	ssh	22:20:12	2
2843	99	40	12,20	cyclictest	12	-21	ksoftirqd/0	20:40:11	0
2863	99	39	3,27	cyclictest	7876	-21	ssh	00:20:12	2
2863	99	39	1,23	cyclictest	6842	-21	sshd	20:55:11	2
2863	99	39	1,23	cyclictest	6842	-21	sshd	20:55:11	2
2863	99	39	1,15	cyclictest	16850	-21	kthreadcore	23:30:20	2
2863	99	39	1,13	cyclictest	18430	-21	tr	22:55:20	2
2852	99	39	8,5	cyclictest	13	1	rcu_preempt	23:25:12	1
2843	99	39	1,33	cyclictest	31221	-21	ssh	00:15:12	0
2863	99	38	6,9	cyclictest	12325	-21	latency_hist	19:20:01	2
2863	99	38	4,10	cyclictest	273	46	irq/40-dwc2_hso	20:20:12	2
2863	99	38	2,28	cyclictest	28359	-21	sh	21:50:12	2
2863	99	38	1,9	cyclictest	777	-21	sendmail_mailqu	20:45:27	2
2863	99	38	1,9	cyclictest	26261	-21	grep	00:30:17	2
2863	99	38	1,9	cyclictest	22450	-21	cut	21:10:19	2
2863	99	38	1,8	cyclictest	21469	-21	usage_EthcTask	00:25:30	2
2863	99	38	1,10	cyclictest	29058	-21	grep	21:30:31	2
2843	99	38	21,11	cyclictest	2339	-21	sh	21:52:19	0
2863	99	37	1,9	cyclictest	7194	-21	kthreadcore	21:55:21	2
2863	99	37	1,9	cyclictest	7194	-21	kthreadcore	21:55:20	2
2863	99	37	1,14	cyclictest	5323	-21	kthreadcore	19:10:20	2
2863	99	37	1,14	cyclictest	31829	-21	sort	19:40:14	2
2863	99	37	1,14	cyclictest	12676	-21	fschecks_time	22:35:17	2
2852	99	37	1,33	cyclictest	5071	-21	ssh	21:55:13	1
2852	99	37	1,33	cyclictest	5071	-21	ssh	21:55:12	1
2843	99	37	1,28	cyclictest	3434	-21	ssh	22:30:14	0
2876	99	36	1,21	cyclictest	359	66	irq/58-eth0	22:55:12	3
2863	99	36	8,3	cyclictest	4868	-21	kthreadcore	22:30:20	2
2863	99	36	3,9	cyclictest	14149	-21	awk	21:40:28	2
2863	99	36	1,7	cyclictest	14514	-21	ntp_kernel_pll_	23:10:25	2
2843	99	36	4,22	cyclictest	23697	-21	ssh	23:35:15	0
2876	99	35	2,17	cyclictest	359	66	irq/58-eth0	00:35:14	3
2876	99	35	1,20	cyclictest	359	66	irq/58-eth0	22:05:12	3
2863	99	35	1,9	cyclictest	4368	-21	ssh	23:40:30	2
2863	99	35	1,9	cyclictest	21883	-21	ssh	23:50:24	2
2863	99	35	1,5	cyclictest	19745	-21	irqcore	21:45:17	2
2863	99	35	15,17	cyclictest	22079	-21	ssh	22:40:19	2
2863	99	34	3,6	cyclictest	15941	-21	ssh	21:25:10	2
2863	99	34	1,9	cyclictest	7719	-21	irqcore	22:50:18	2
2863	99	34	1,7	cyclictest	9063	-21	sort	19:50:15	2
2863	99	34	1,7	cyclictest	27586	-21	fschecks_count	23:55:16	2
2863	99	34	1,6	cyclictest	14996	-21	latency_hist	20:30:02	2
2863	99	34	1,13	cyclictest	16192	-21	idleruntime-cro	21:05:00	2
2863	99	34	1,11	cyclictest	29563	-21	kthreadcore	19:35:22	2
2863	99	34	1,10	cyclictest	10078	-21	kthreadcore	22:15:20	2
2852	99	34	2,24	cyclictest	0	-21	swapper/1	20:35:14	1
2843	99	34	9,20	cyclictest	12	-21	ksoftirqd/0	19:40:14	0
2843	99	34	4,9	cyclictest	12	-21	ksoftirqd/0	23:20:12	0
2843	99	34	13,16	cyclictest	12	-21	ksoftirqd/0	19:30:12	0
2876	99	33	9,6	cyclictest	0	-21	swapper/3	21:15:02	3
2863	99	33	4,21	cyclictest	1849	-21	ssh	23:05:14	2
2863	99	33	3,9	cyclictest	7911	-21	grep	00:00:27	2
2863	99	33	2,26	cyclictest	0	-21	swapper/2	22:13:11	2
2843	99	33	2,19	cyclictest	16204	-21	ssh	22:55:12	0
2863	99	32	5,15	cyclictest	273	46	irq/40-dwc2_hso	23:20:11	2
2863	99	32	1,11	cyclictest	4260	-21	find	20:15:28	2
2863	99	32	1,11	cyclictest	4260	-21	find	20:15:27	2
2863	99	32	1,10	cyclictest	9691	-21	unixbench_multi	23:25:29	2
2863	99	32	0,9	cyclictest	22133	-21	kthreadcore	20:35:21	2
2876	99	31	3,21	cyclictest	359	66	irq/58-eth0	22:00:12	3
2863	99	31	6,5	cyclictest	27027	-21	cat	00:10:32	2
2863	99	31	1,8	cyclictest	26713	-21	/usr/sbin/munin	22:25:29	2
2863	99	31	1,6	cyclictest	19912	-21	kthreadcore	19:25:20	2
2863	99	31	1,12	cyclictest	10806	-21	grep	21:20:27	2
2852	99	31	2,24	cyclictest	12485	-21	ssh	22:35:15	1
2852	99	31	2,22	cyclictest	25968	-21	ssh	00:30:13	1
2852	99	31	2,19	cyclictest	1428	-21	ssh	21:35:15	1
2843	99	31	2,21	cyclictest	23118	-21	ssh	22:05:11	0
2876	99	30	3,16	cyclictest	359	66	irq/58-eth0	22:45:12	3
2863	99	30	2,10	cyclictest	28215	-21	/usr/sbin/munin	20:10:12	2
2863	99	30	1,12	cyclictest	31020	-21	irqcore	22:45:18	2
2852	99	30	2,23	cyclictest	0	-21	swapper/1	23:00:13	1
2876	99	29	2,16	cyclictest	359	66	irq/58-eth0	21:40:12	3
2863	99	29	2,25	cyclictest	0	-21	swapper/2	23:38:11	2
2863	99	29	2,10	cyclictest	16992	-21	irqrtprio	21:05:21	2
2863	99	29	1,6	cyclictest	24938	-21	ssh	23:15:49	2
2863	99	29	1,10	cyclictest	12685	-21	fschecks_count	19:20:15	2
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17