OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack7slot4s.osadl.org (updated Mon May 04, 2026 12:45:10)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
14389	99	63	8,7	cyclictest	13	1	rcu_preempt	11:00:13	0
14389	99	62	7,11	cyclictest	13	1	rcu_preempt	08:45:14	0
14404	99	60	1,44	cyclictest	25490	-21	ssh	10:00:13	1
14418	99	56	17,8	cyclictest	13	1	rcu_preempt	07:50:12	2
14389	99	56	3,40	cyclictest	18366	-21	ssh	09:20:12	0
14418	99	53	9,9	cyclictest	13	1	rcu_preempt	11:15:13	2
14389	99	53	4,34	cyclictest	9395	-21	ssh	09:15:14	0
14418	99	52	2,41	cyclictest	19343	-21	ssh	11:45:13	2
14431	99	50	34,10	cyclictest	359	66	irq/58-eth0	12:15:14	3
14404	99	50	2,15	cyclictest	11613	-21	ssh	12:35:12	1
14404	99	48	4,6	cyclictest	13	1	rcu_preempt	11:15:13	1
14404	99	48	17,22	cyclictest	21	-21	ksoftirqd/1	09:45:13	1
14418	99	47	2,5	cyclictest	13	1	rcu_preempt	09:25:13	2
14431	99	46	1,28	cyclictest	359	66	irq/58-eth0	09:35:13	3
14431	99	44	35,6	cyclictest	359	66	irq/58-eth0	09:50:15	3
14404	99	44	1,34	cyclictest	21456	-21	ssh	11:10:13	1
14418	99	43	6,24	cyclictest	27	-21	ksoftirqd/2	11:05:14	2
14404	99	41	18,12	cyclictest	630	-21	Async	09:36:31	1
14404	99	39	4,18	cyclictest	29129	-21	ssh	10:20:14	1
14404	99	38	2,22	cyclictest	21	-21	ksoftirqd/1	07:25:12	1
14389	99	38	1,29	cyclictest	3963	-21	rm	09:30:13	0
14404	99	37	22,8	cyclictest	536	-21	usb-storage	12:00:11	1
14404	99	37	1,18	cyclictest	18057	-21	ssh	10:50:13	1
14404	99	36	16,14	cyclictest	21	-21	ksoftirqd/1	07:50:13	1
14389	99	36	2,27	cyclictest	22051	-21	ssh	09:40:13	0
14431	99	35	2,21	cyclictest	359	66	irq/58-eth0	11:05:15	3
14404	99	35	4,13	cyclictest	9060	-21	find	10:45:13	1
14404	99	35	2,29	cyclictest	16582	-21	ssh	09:55:13	1
14389	99	35	23,9	cyclictest	0	-21	swapper/0	08:05:12	0
14389	99	35	2,22	cyclictest	12	-21	ksoftirqd/0	09:35:13	0
14431	99	34	9,15	cyclictest	32	1	rcuc/3	08:45:14	3
14431	99	34	23,7	cyclictest	0	-21	swapper/3	12:05:12	3
14418	99	34	5,20	cyclictest	26	1	rcuc/2	10:15:13	2
14389	99	34	1,20	cyclictest	27364	-21	ssh	09:25:12	0
14431	99	33	17,10	cyclictest	359	66	irq/58-eth0	11:50:15	3
14404	99	33	1,27	cyclictest	10371	-21	sh	11:40:10	1
14389	99	33	1,23	cyclictest	32406	-21	diskmemload	12:20:12	0
14431	99	32	7,20	cyclictest	359	66	irq/58-eth0	12:35:12	3
14418	99	32	3,18	cyclictest	273	46	irq/40-dwc2_hso	08:05:13	2
14404	99	32	17,9	cyclictest	14380	-21	cyclictest	08:00:12	1
14431	99	31	3,17	cyclictest	359	66	irq/58-eth0	10:00:14	3
14431	99	31	2,18	cyclictest	359	66	irq/58-eth0	09:25:12	3
14431	99	31	1,17	cyclictest	359	66	irq/58-eth0	07:10:12	3
14404	99	31	3,20	cyclictest	14559	-21	ssh	10:30:13	1
14404	99	31	3,20	cyclictest	14559	-21	ssh	10:30:13	1
14389	99	31	13,14	cyclictest	12	-21	ksoftirqd/0	10:25:14	0
14431	99	30	2,13	cyclictest	359	66	irq/58-eth0	12:10:12	3
14404	99	30	2,20	cyclictest	22783	-21	ssh	12:05:14	1
14404	99	30	11,9	cyclictest	14380	-21	cyclictest	07:20:14	1
14389	99	30	2,22	cyclictest	8207	-21	ssh	12:31:23	0
14418	99	29	3,8	cyclictest	0	-21	swapper/2	12:31:23	2
14418	99	29	15,6	cyclictest	31612	-21	ssh	12:10:12	2
14418	99	29	10,9	cyclictest	27	-21	ksoftirqd/2	08:45:14	2
14389	99	29	1,24	cyclictest	12	-21	ksoftirqd/0	10:50:13	0
14389	99	29	1,20	cyclictest	12	-21	ksoftirqd/0	10:15:13	0
14431	99	28	3,18	cyclictest	0	-21	swapper/3	12:30:59	3
14431	99	28	14,8	cyclictest	359	66	irq/58-eth0	11:25:11	3
14418	99	28	3,16	cyclictest	273	46	irq/40-dwc2_hso	08:25:12	2
14418	99	28	2,15	cyclictest	273	46	irq/40-dwc2_hso	08:30:12	2
14418	99	28	18,7	cyclictest	27	-21	ksoftirqd/2	11:35:13	2
14418	99	28	18,7	cyclictest	27	-21	ksoftirqd/2	11:35:13	2
14404	99	28	2,20	cyclictest	31634	-21	ssh	12:10:14	1
14418	99	27	3,17	cyclictest	273	46	irq/40-dwc2_hso	07:35:14	2
14404	99	27	2,19	cyclictest	6141	-21	sed	12:30:24	1
14418	99	26	2,20	cyclictest	28238	-21	sh	11:50:15	2
14418	99	26	2,11	cyclictest	13013	-21	ssh	09:35:14	2
14418	99	26	2,10	cyclictest	13	1	rcu_preempt	07:15:14	2
14418	99	26	1,18	cyclictest	273	46	irq/40-dwc2_hso	10:20:13	2
14404	99	26	2,19	cyclictest	3613	-21	diskmemload	11:00:14	1
14404	99	26	19,3	cyclictest	0	-21	swapper/1	07:40:13	1
14389	99	26	7,12	cyclictest	29113	-21	ssh	10:20:13	0
14389	99	26	3,15	cyclictest	12	-21	ksoftirqd/0	11:20:12	0
14431	99	25	2,18	cyclictest	359	66	irq/58-eth0	10:35:13	3
14431	99	25	2,18	cyclictest	359	66	irq/58-eth0	10:35:13	3
14418	99	25	9,7	cyclictest	42	-21	kauditd	08:55:01	2
14418	99	25	3,13	cyclictest	2124	-21	sh	10:05:13	2
14418	99	25	11,11	cyclictest	14665	-21	memory	10:10:24	2
14404	99	25	14,7	cyclictest	7205	-21	cron	08:10:01	1
14389	99	25	2,18	cyclictest	819	-21	ssh	12:28:23	0
14431	99	24	6,16	cyclictest	33	-21	ksoftirqd/3	10:15:03	3
14431	99	24	4,15	cyclictest	0	-21	swapper/3	10:05:13	3
14431	99	24	3,14	cyclictest	0	-21	swapper/3	09:30:13	3
14418	99	24	1,8	cyclictest	18370	-21	sh	09:20:13	2
14418	99	24	1,19	cyclictest	12346	-21	cut	08:15:16	2
14418	99	24	11,6	cyclictest	21841	-21	ssh	11:10:19	2
14404	99	24	2,15	cyclictest	1983	-21	ssh	10:05:04	1
14389	99	24	1,17	cyclictest	27075	-21	ssh	10:55:14	0
14431	99	23	1,15	cyclictest	0	-21	swapper/3	10:55:13	3
14418	99	23	6,9	cyclictest	11312	-21	unixbench_singl	08:10:29	2
14418	99	23	2,16	cyclictest	10368	-21	apt-get	11:40:11	2
14418	99	23	11,5	cyclictest	23201	-21	ssh	11:28:29	2
14418	99	23	11,5	cyclictest	14817	-21	ssh	12:17:46	2
14418	99	23	11,10	cyclictest	17692	-21	ssh	09:19:43	2
14418	99	23	10,10	cyclictest	1	-21	systemd	12:36:00	2
14404	99	23	1,15	cyclictest	1407	-21	ssh	11:35:14	1
14404	99	23	1,15	cyclictest	1407	-21	ssh	11:35:14	1
14431	99	22	3,11	cyclictest	0	-21	swapper/3	10:30:12	3
14431	99	22	3,11	cyclictest	0	-21	swapper/3	10:30:12	3
14418	99	22	8,11	cyclictest	14698	-21	fschecks_count	10:30:15	2
14418	99	22	8,11	cyclictest	14698	-21	fschecks_count	10:30:15	2
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17