OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack7slot4s.osadl.org (updated Wed Sep 17, 2025 00:45:07)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
1074	99	59	24,18	cyclictest	253	66	irq/58-eth0	23:00:13	3
1066	99	56	14,8	cyclictest	13	1	rcu_preempt	21:50:13	2
1058	99	55	4,11	cyclictest	13	1	rcu_preempt	00:15:14	1
1049	99	54	1,8	cyclictest	0	-21	swapper/0	20:00:14	0
1049	99	52	1,2	cyclictest	18927	-21	diskmemload	23:15:12	0
1049	99	49	4,38	cyclictest	28599	-21	ssh	21:55:12	0
1049	99	49	2,41	cyclictest	0	-21	swapper/0	20:50:15	0
1074	99	47	34,10	cyclictest	253	66	irq/58-eth0	22:30:14	3
1074	99	47	3,29	cyclictest	253	66	irq/58-eth0	22:45:17	3
1058	99	47	1,40	cyclictest	8767	-21	ssh	22:25:16	1
1049	99	47	2,7	cyclictest	0	-21	swapper/0	22:50:13	0
1049	99	47	1,4	cyclictest	0	-21	swapper/0	22:45:16	0
1049	99	46	3,2	cyclictest	31314	-21	apt-get	00:20:14	0
1049	99	45	3,5	cyclictest	0	-21	swapper/0	19:35:17	0
1049	99	45	2,2	cyclictest	24958	-21	apt-get	22:55:14	0
1049	99	45	1,5	cyclictest	0	-21	swapper/0	00:05:14	0
1049	99	45	1,2	cyclictest	0	-21	swapper/0	21:15:14	0
1049	99	44	1,7	cyclictest	0	-21	swapper/0	19:55:17	0
1049	99	43	3,3	cyclictest	9280	-21	apt-get	23:45:11	0
1049	99	43	1,3	cyclictest	0	-21	swapper/0	19:45:13	0
1049	99	42	1,10	cyclictest	26261	-21	apt-get	20:10:12	0
1058	99	40	3,8	cyclictest	13	1	rcu_preempt	20:25:15	1
1058	99	40	1,32	cyclictest	4407	-21	ssh	22:00:14	1
1049	99	40	2,31	cyclictest	16421	-21	ssh	00:10:14	0
1049	99	40	1,7	cyclictest	0	-21	swapper/0	19:40:14	0
1049	99	40	1,2	cyclictest	0	-21	swapper/0	21:50:13	0
1074	99	39	13,14	cyclictest	253	66	irq/58-eth0	23:10:17	3
1049	99	39	1,4	cyclictest	0	-21	swapper/0	20:45:13	0
1049	99	39	0,4	cyclictest	27951	-21	apt-get	21:35:14	0
1049	99	38	1,30	cyclictest	4397	-21	ssh	22:00:13	0
1049	99	38	0,3	cyclictest	25180	-21	apt-get	22:35:16	0
1066	99	37	2,18	cyclictest	17827	-21	ssh	00:31:47	2
1058	99	37	10,23	cyclictest	21	-21	ksoftirqd/1	23:15:12	1
1049	99	37	2,27	cyclictest	16270	-21	ssh	22:10:10	0
1074	99	36	4,21	cyclictest	253	66	irq/58-eth0	22:10:10	3
1058	99	36	2,26	cyclictest	11582	-21	ssh	21:25:18	1
1049	99	36	2,3	cyclictest	17938	-21	apt-get	20:35:12	0
1049	99	36	1,3	cyclictest	14353	-21	apt-get	21:05:13	0
1049	99	36	1,3	cyclictest	0	-21	swapper/0	21:40:15	0
1049	99	36	1,3	cyclictest	0	-21	swapper/0	21:40:15	0
1049	99	36	1,2	cyclictest	23933	-21	apt-get	00:15:14	0
1066	99	35	2,28	cyclictest	25525	-21	ssh	00:38:46	2
1058	99	35	15,10	cyclictest	21	-21	ksoftirqd/1	00:25:14	1
1049	99	35	3,3	cyclictest	0	-21	swapper/0	21:45:14	0
1049	99	35	2,4	cyclictest	0	-21	swapper/0	20:55:14	0
1066	99	34	21,10	cyclictest	8773	-21	sh	22:25:17	2
1058	99	34	2,28	cyclictest	19713	-21	ssh	21:30:15	1
1049	99	34	5,25	cyclictest	20280	-21	tr	23:10:26	0
1049	99	34	2,2	cyclictest	21453	-21	apt-get	20:05:16	0
1049	99	34	1,9	cyclictest	26309	-21	cut	23:35:20	0
1049	99	34	1,8	cyclictest	2415	-21	cut	00:00:20	0
1049	99	34	1,8	cyclictest	19312	-21	cut	23:30:23	0
1049	99	34	1,4	cyclictest	0	-21	swapper/0	22:15:15	0
1049	99	34	1,3	cyclictest	17346	-21	apt-get	23:50:16	0
1049	99	34	1,3	cyclictest	0	-21	swapper/0	19:30:16	0
1049	99	34	1,10	cyclictest	786	-21	/usr/sbin/munin	23:20:24	0
1049	99	34	0,4	cyclictest	931	-21	apt-get	22:40:14	0
1049	99	34	0,4	cyclictest	8300	-21	apt-get	20:25:14	0
1049	99	33	5,25	cyclictest	8741	-21	cut	19:50:19	0
1049	99	33	5,25	cyclictest	8741	-21	cut	19:50:19	0
1049	99	33	1,9	cyclictest	32697	-21	ssh	22:20:06	0
1049	99	33	1,9	cyclictest	13290	-21	cut	21:25:25	0
1049	99	33	1,8	cyclictest	9878	-21	kthreadcore	23:05:18	0
1049	99	33	1,8	cyclictest	3389	-21	df	21:20:15	0
1049	99	33	1,8	cyclictest	23056	-21	irqcore	20:40:18	0
1049	99	33	1,8	cyclictest	19909	-21	fschecks_time	21:30:18	0
1049	99	33	1,7	cyclictest	9314	-21	grep	19:15:26	0
1049	99	33	1,4	cyclictest	0	-21	swapper/0	00:25:15	0
1049	99	33	11,20	cyclictest	4742	-21	unixbench_multi	23:00:29	0
1049	99	32	6,23	cyclictest	16729	-21	mailstats	20:30:26	0
1049	99	32	2,6	cyclictest	19598	-21	apt-get	00:35:14	0
1049	99	32	2,2	cyclictest	3521	-21	apt-get	20:20:14	0
1049	99	32	1,9	cyclictest	26223	-21	kthreadcore	23:55:19	0
1049	99	32	1,9	cyclictest	13191	-21	cat	23:25:30	0
1049	99	32	1,8	cyclictest	16381	-21	tr	19:25:17	0
1049	99	32	1,8	cyclictest	16381	-21	tr	19:25:17	0
1049	99	32	1,7	cyclictest	22090	-21	kthreadcore	21:10:23	0
1049	99	32	1,7	cyclictest	16669	-21	latency_hist	22:30:01	0
1049	99	32	1,7	cyclictest	12927	-21	sshd	22:05:46	0
1049	99	32	1,7	cyclictest	12927	-21	sshd	22:05:46	0
1049	99	32	1,7	cyclictest	10774	-21	sort	19:20:13	0
1049	99	32	1,4	cyclictest	16937	-21	ssh	22:30:12	0
1049	99	32	0,4	cyclictest	1171	-21	apt-get	19:10:14	0
1066	99	31	7,6	cyclictest	13	1	rcu_preempt	19:40:14	2
1066	99	31	5,19	cyclictest	218	46	irq/40-dwc2_hso	19:50:13	2
1066	99	31	5,19	cyclictest	218	46	irq/40-dwc2_hso	19:50:13	2
1066	99	31	5,18	cyclictest	218	46	irq/40-dwc2_hso	19:10:14	2
1058	99	31	15,10	cyclictest	0	-21	swapper/1	22:30:14	1
1049	99	31	1,7	cyclictest	1518	-21	ssh	23:40:20	0
1049	99	31	1,7	cyclictest	12826	-21	fschecks_time	00:30:17	0
1049	99	31	1,4	cyclictest	9472	-21	apt-get	21:00:16	0
1058	99	30	2,22	cyclictest	27963	-21	ssh	21:35:14	1
1049	99	30	1,6	cyclictest	2595	-21	head	20:15:31	0
1074	99	29	16,7	cyclictest	253	66	irq/58-eth0	23:05:13	3
1074	99	29	10,4	cyclictest	0	-21	swapper/3	23:23:08	3
1066	99	29	5,14	cyclictest	218	46	irq/40-dwc2_hso	21:00:16	2
1066	99	29	2,7	cyclictest	13	1	rcu_preempt	23:34:46	2
1066	99	29	17,9	cyclictest	872	-21	sendmail	23:40:01	2
1058	99	29	3,7	cyclictest	13	1	rcu_preempt	00:09:46	1
1058	99	29	3,15	cyclictest	21	-21	ksoftirqd/1	20:20:13	1
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17