OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack7slot4s.osadl.org (updated Mon Jul 13, 2026 00:45:07)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
6163	99	57	4,41	cyclictest	7778	-21	ssh	00:00:14	2
6155	99	55	1,43	cyclictest	26885	-21	ssh	22:05:12	1
6163	99	54	4,9	cyclictest	0	-21	swapper/2	20:15:13	2
6163	99	53	4,46	cyclictest	14793	-21	ssh	23:10:16	2
6155	99	53	2,39	cyclictest	1071	-21	ssh	21:15:13	1
6171	99	47	8,26	cyclictest	359	66	irq/58-eth0	21:45:12	3
6163	99	42	5,23	cyclictest	1064	-21	diskmemload	21:15:12	2
6163	99	42	12,25	cyclictest	273	46	irq/40-dwc2_hso	21:10:14	2
6163	99	42	12,21	cyclictest	30194	-21	readlink	19:35:01	2
6163	99	41	1,8	cyclictest	4305	-21	fschecks_time	23:40:17	2
6163	99	40	7,5	cyclictest	23090	-21	kthreadcore	22:20:21	2
6163	99	40	1,9	cyclictest	6435	-21	grep	19:10:16	2
6143	99	40	1,30	cyclictest	28220	-21	ssh	21:30:12	0
6163	99	39	16,16	cyclictest	27	-21	ksoftirqd/2	20:00:13	2
6163	99	39	1,11	cyclictest	14172	-21	kthreadcore	19:50:20	2
6155	99	39	3,28	cyclictest	11384	-21	ssh	00:20:15	1
6155	99	39	3,28	cyclictest	11384	-21	ssh	00:20:15	1
6155	99	39	1,31	cyclictest	7430	-21	ssh	22:30:14	1
6155	99	39	1,26	cyclictest	20267	-21	ssh	22:55:14	1
6143	99	39	1,28	cyclictest	22990	-21	ssh	21:45:12	0
6171	99	38	6,21	cyclictest	359	66	irq/58-eth0	21:35:13	3
6163	99	38	1,9	cyclictest	29460	-21	apt-get	00:30:15	2
6163	99	38	1,13	cyclictest	20464	-21	wc	22:35:29	2
6163	99	37	2,19	cyclictest	15815	-21	ssh	21:58:10	2
6163	99	37	1,9	cyclictest	7732	-21	kthreadcore	21:35:22	2
6163	99	37	1,9	cyclictest	3192	-21	latency_hist	22:10:01	2
6163	99	37	1,9	cyclictest	23035	-21	ps	21:05:28	2
6163	99	37	1,9	cyclictest	15986	-21	kthreadcore	21:40:21	2
6163	99	37	1,13	cyclictest	25697	-21	fschecks_time	22:40:16	2
6163	99	36	2,18	cyclictest	5883	-21	ssh	23:21:10	2
6163	99	36	2,18	cyclictest	20346	-21	ssh	00:25:12	2
6163	99	36	2,10	cyclictest	20106	-21	kthreadcore	20:30:21	2
6163	99	36	2,10	cyclictest	0	-21	swapper/2	23:45:20	2
6163	99	36	1,8	cyclictest	24103	-21	diskmemload	22:47:10	2
6163	99	36	1,12	cyclictest	31164	-21	ssh	23:55:14	2
6163	99	36	0,9	cyclictest	14277	-21	kthreadcore	20:25:18	2
6163	99	36	0,8	cyclictest	10480	-21	kthreadcore	19:45:21	2
6155	99	36	3,29	cyclictest	5824	-21	ssh	23:05:13	1
6171	99	35	3,21	cyclictest	359	66	irq/58-eth0	21:10:13	3
6163	99	35	6,5	cyclictest	21349	-21	sed	22:00:22	2
6163	99	35	1,8	cyclictest	2862	-21	kthreadcore	21:50:23	2
6163	99	35	1,11	cyclictest	13628	-21	latency	21:20:23	2
6163	99	35	1,10	cyclictest	9062	-21	if_eth1	20:20:18	2
6163	99	35	1,10	cyclictest	22361	-21	kthreadcore	21:25:24	2
6163	99	35	0,7	cyclictest	4916	-21	latency_hist	20:50:01	2
6143	99	35	2,23	cyclictest	15	1	rcuc/0	22:35:14	0
6171	99	34	6,19	cyclictest	359	66	irq/58-eth0	23:50:13	3
6163	99	34	5,6	cyclictest	28079	-21	kthreadcore	00:10:23	2
6163	99	34	2,12	cyclictest	19341	-21	latency_hist	21:05:01	2
6163	99	34	1,9	cyclictest	9589	-21	netstat	00:35:25	2
6163	99	34	1,9	cyclictest	29981	-21	kthreadcore	21:30:20	2
6163	99	34	1,7	cyclictest	18526	-21	/usr/sbin/munin	23:30:13	2
6163	99	34	1,15	cyclictest	27728	-21	ssh	23:35:17	2
6163	99	34	1,13	cyclictest	31227	-21	irqcore	22:25:16	2
6163	99	34	1,13	cyclictest	23635	-21	kthreadcore	22:55:22	2
6163	99	34	0,10	cyclictest	25879	-21	kthreadcore	20:35:22	2
6171	99	33	2,21	cyclictest	359	66	irq/58-eth0	19:35:12	3
6171	99	33	1,14	cyclictest	359	66	irq/58-eth0	22:45:13	3
6163	99	33	2,19	cyclictest	273	46	irq/40-dwc2_hso	23:15:14	2
6163	99	33	1,9	cyclictest	30734	-21	kthreadcore	20:40:23	2
6163	99	33	1,9	cyclictest	21762	-21	awk	20:00:01	2
6163	99	33	1,9	cyclictest	12700	-21	cut	19:15:20	2
6163	99	33	1,9	cyclictest	0	-21	swapper/2	19:40:18	2
6163	99	33	1,8	cyclictest	6188	-21	ssh	00:15:29	2
6163	99	33	0,9	cyclictest	23519	-21	irqcore	21:45:19	2
6155	99	33	26,4	cyclictest	0	-21	swapper/1	20:25:12	1
6171	99	32	20,6	cyclictest	0	-21	swapper/3	20:10:14	3
6163	99	32	6,4	cyclictest	10238	-21	irqrtprio	23:25:18	2
6163	99	32	2,4	cyclictest	22298	-21	diskstats	23:50:16	2
6163	99	32	1,8	cyclictest	9776	-21	latency_hist	20:55:01	2
6163	99	32	1,8	cyclictest	19623	-21	timerandwakeup	19:20:29	2
6163	99	32	1,10	cyclictest	16257	-21	users	00:20:35	2
6163	99	32	1,10	cyclictest	16257	-21	users	00:20:34	2
6163	99	31	7,21	cyclictest	0	-21	swapper/2	22:11:52	2
6163	99	31	1,3	cyclictest	0	-21	swapper/2	22:50:29	2
6163	99	31	1,10	cyclictest	23296	-21	ssh	00:07:37	2
6163	99	31	0,12	cyclictest	20083	-21	ssh	22:18:36	2
6143	99	31	3,15	cyclictest	22170	-21	ssh	23:50:12	0
6143	99	31	2,22	cyclictest	6018	-21	ssh	00:35:14	0
6143	99	31	1,3	cyclictest	13	1	rcu_preempt	22:25:11	0
6163	99	30	9,3	cyclictest	456	-21	ssh	23:00:27	2
6163	99	30	1,9	cyclictest	11731	-21	tr	20:55:22	2
6155	99	30	4,18	cyclictest	22981	-21	ssh	21:45:12	1
6163	99	29	2,6	cyclictest	0	-21	swapper/2	22:34:10	2
6163	99	29	1,9	cyclictest	8762	-21	kthreadcore	23:05:22	2
6163	99	29	1,9	cyclictest	2574	-21	latency_hist	19:40:01	2
6163	99	29	1,7	cyclictest	25418	-21	latency_hist	19:30:01	2
6163	99	29	1,7	cyclictest	25418	-21	latency_hist	19:30:01	2
6143	99	29	1,22	cyclictest	29282	-21	ssh	23:00:13	0
6163	99	28	5,5	cyclictest	1023	-21	kthreadcore	20:10:22	2
6155	99	28	3,20	cyclictest	13158	-21	ssh	23:45:14	1
6155	99	28	2,20	cyclictest	30933	-21	ssh	22:41:10	1
6155	99	28	2,19	cyclictest	15464	-21	ssh	23:26:46	1
6143	99	28	4,12	cyclictest	6135	-21	cyclictest	19:40:14	0
6143	99	28	13,13	cyclictest	24103	-21	diskmemload	23:30:13	0
6155	99	27	4,18	cyclictest	13	1	rcu_preempt	23:52:34	1
6155	99	27	2,19	cyclictest	7995	-21	ssh	22:46:42	1
6155	99	27	2,19	cyclictest	29241	-21	ssh	23:16:10	1
6155	99	27	2,19	cyclictest	16652	-21	sh	21:59:10	1
6143	99	27	2,18	cyclictest	8527	-21	ssh	21:35:26	0
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17