OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack7slot4s.osadl.org (updated Wed Jul 29, 2026 00:45:14)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
32523	99	75	13,7	cyclictest	13	1	rcu_preempt	20:20:12	0
32523	99	70	5,48	cyclictest	19168	-21	ssh	22:40:13	0
32523	99	70	5,48	cyclictest	19168	-21	ssh	22:40:12	0
32544	99	65	46,12	cyclictest	359	66	irq/58-eth0	23:40:12	3
32523	99	62	3,48	cyclictest	20594	-21	ssh	23:35:13	0
32544	99	57	32,10	cyclictest	359	66	irq/58-eth0	22:45:15	3
32524	99	51	3,10	cyclictest	13	1	rcu_preempt	22:20:13	1
32544	99	49	2,27	cyclictest	359	66	irq/58-eth0	23:45:15	3
32524	99	48	5,35	cyclictest	15057	-21	ssh	23:50:15	1
32524	99	47	6,3	cyclictest	13	1	rcu_preempt	00:30:11	1
32523	99	46	2,32	cyclictest	2928	-21	ssh	21:55:14	0
32523	99	46	2,32	cyclictest	2928	-21	ssh	21:55:13	0
32544	99	45	24,12	cyclictest	359	66	irq/58-eth0	22:20:14	3
32524	99	45	3,26	cyclictest	4116	-21	ssh	00:20:14	1
32535	99	42	2,25	cyclictest	27	-21	ksoftirqd/2	23:35:14	2
32524	99	41	8,18	cyclictest	21	-21	ksoftirqd/1	19:10:15	1
32524	99	39	3,31	cyclictest	18988	-21	ssh	21:10:15	1
32544	99	38	3,19	cyclictest	359	66	irq/58-eth0	22:55:14	3
32544	99	36	8,18	cyclictest	0	-21	swapper/3	21:10:14	3
32544	99	36	1,19	cyclictest	359	66	irq/58-eth0	21:20:14	3
32524	99	36	3,19	cyclictest	9628	-21	ssh	00:05:15	1
32523	99	36	9,19	cyclictest	12	-21	ksoftirqd/0	23:25:13	0
32523	99	36	21,9	cyclictest	22675	-21	rm	23:00:14	0
32523	99	36	1,28	cyclictest	27526	-21	ssh	00:15:13	0
32535	99	34	5,20	cyclictest	27	-21	ksoftirqd/2	21:55:14	2
32535	99	34	5,20	cyclictest	27	-21	ksoftirqd/2	21:55:13	2
32535	99	34	12,7	cyclictest	27	-21	ksoftirqd/2	23:15:14	2
32524	99	33	18,7	cyclictest	0	-21	swapper/1	23:45:16	1
32524	99	33	13,10	cyclictest	32522	-21	cyclictest	20:30:14	1
32544	99	32	5,20	cyclictest	359	66	irq/58-eth0	21:35:13	3
32535	99	32	2,25	cyclictest	5476	-21	ssh	00:37:48	2
32524	99	32	18,4	cyclictest	13	1	rcu_preempt	21:40:13	1
32524	99	31	1,19	cyclictest	30056	-21	ssh	22:10:14	1
32523	99	31	1,22	cyclictest	28064	-21	ssh	21:15:17	0
32535	99	30	3,18	cyclictest	273	46	irq/40-dwc2_hso	19:30:13	2
32524	99	30	8,14	cyclictest	0	-21	swapper/1	22:50:12	1
32524	99	30	4,21	cyclictest	20588	-21	ssh	23:35:14	1
32523	99	30	1,4	cyclictest	13	1	rcu_preempt	00:05:17	0
32535	99	29	5,17	cyclictest	273	46	irq/40-dwc2_hso	00:10:11	2
32535	99	29	2,17	cyclictest	273	46	irq/40-dwc2_hso	19:50:14	2
32524	99	29	2,18	cyclictest	8195	-21	ssh	23:10:14	1
32524	99	29	1,25	cyclictest	6658	-21	ssh	22:15:15	1
32544	99	28	3,18	cyclictest	0	-21	swapper/3	00:30:50	3
32544	99	28	16,4	cyclictest	0	-21	swapper/3	22:10:13	3
32535	99	28	2,20	cyclictest	495	-21	ssh	00:00:04	2
32535	99	28	1,7	cyclictest	27	-21	ksoftirqd/2	23:10:13	2
32523	99	28	3,18	cyclictest	10130	-21	ssh	22:35:12	0
32544	99	27	3,17	cyclictest	0	-21	swapper/3	00:23:49	3
32535	99	27	4,11	cyclictest	273	46	irq/40-dwc2_hso	20:00:11	2
32535	99	27	2,19	cyclictest	0	-21	swapper/2	20:20:13	2
32535	99	27	1,8	cyclictest	2627	-21	ssh	23:25:13	2
32523	99	27	2,20	cyclictest	12	-21	ksoftirqd/0	21:00:11	0
32523	99	27	2,17	cyclictest	22103	-21	ssh	00:30:14	0
32523	99	27	2,16	cyclictest	6639	-21	ssh	22:15:15	0
32523	99	27	1,18	cyclictest	17192	-21	ssh	21:45:05	0
32535	99	26	4,17	cyclictest	273	46	irq/40-dwc2_hso	22:25:12	2
32535	99	26	3,17	cyclictest	273	46	irq/40-dwc2_hso	23:00:14	2
32535	99	26	2,19	cyclictest	0	-21	swapper/2	20:15:15	2
32535	99	26	2,16	cyclictest	273	46	irq/40-dwc2_hso	22:35:11	2
32535	99	26	13,6	cyclictest	667	-21	ssh	21:15:47	2
32524	99	26	2,17	cyclictest	17347	-21	ssh	21:45:14	1
32524	99	26	14,6	cyclictest	630	-21	Async	09:36:31	1
32523	99	26	7,3	cyclictest	13	1	rcu_preempt	20:15:14	0
32544	99	25	2,14	cyclictest	359	66	irq/58-eth0	21:45:14	3
32544	99	25	12,9	cyclictest	359	66	irq/58-eth0	22:05:13	3
32544	99	25	1,16	cyclictest	359	66	irq/58-eth0	00:15:14	3
32535	99	25	2,16	cyclictest	0	-21	swapper/2	21:10:05	2
32524	99	25	2,20	cyclictest	13703	-21	ssh	22:55:14	1
32524	99	25	1,19	cyclictest	13554	-21	sh	21:25:13	1
32523	99	25	2,17	cyclictest	18480	-21	diskmemload	23:30:15	0
32523	99	25	1,3	cyclictest	13	1	rcu_preempt	20:45:34	0
32523	99	25	11,5	cyclictest	0	-21	swapper/0	22:55:30	0
32523	99	25	1,11	cyclictest	13	1	rcu_preempt	23:15:14	0
32544	99	24	9,10	cyclictest	0	-21	swapper/3	00:29:49	3
32535	99	24	9,4	cyclictest	27	-21	ksoftirqd/2	21:00:12	2
32535	99	24	2,7	cyclictest	0	-21	swapper/2	00:18:49	2
32535	99	24	2,6	cyclictest	0	-21	swapper/2	00:26:50	2
32535	99	24	12,9	cyclictest	24842	-21	cut	21:30:22	2
32535	99	24	12,9	cyclictest	24842	-21	cut	21:30:21	2
32535	99	24	10,6	cyclictest	10970	-21	cat	22:15:33	2
32535	99	24	0,13	cyclictest	4612	-21	cut	20:55:19	2
32524	99	24	2,18	cyclictest	24608	-21	ssh	22:25:12	1
32544	99	23	3,13	cyclictest	0	-21	swapper/3	21:25:13	3
32544	99	23	1,15	cyclictest	0	-21	swapper/3	21:55:14	3
32544	99	23	1,15	cyclictest	0	-21	swapper/3	21:55:14	3
32535	99	23	5,15	cyclictest	30308	-21	rm	22:10:16	2
32535	99	23	12,9	cyclictest	4100	-21	ssh	21:35:34	2
32535	99	23	12,9	cyclictest	0	-21	swapper/2	00:09:34	2
32535	99	23	1,13	cyclictest	4627	-21	ssh	21:20:13	2
32535	99	23	1,12	cyclictest	12169	-21	irqrtprio	23:30:19	2
32524	99	23	2,18	cyclictest	9932	-21	find	19:20:11	1
32524	99	23	2,17	cyclictest	2947	-21	ssh	21:55:15	1
32524	99	23	2,17	cyclictest	2947	-21	ssh	21:55:14	1
32524	99	23	1,18	cyclictest	26138	-21	ssh	23:20:16	1
32523	99	23	3,4	cyclictest	13	1	rcu_preempt	23:55:13	0
32523	99	23	3,4	cyclictest	13	1	rcu_preempt	23:55:12	0
32523	99	23	14,4	cyclictest	0	-21	swapper/0	22:45:28	0
32523	99	23	1,18	cyclictest	4752	-21	ssh	22:50:11	0
32544	99	22	3,11	cyclictest	359	66	irq/58-eth0	00:00:14	3
32544	99	22	2,13	cyclictest	0	-21	swapper/3	22:50:14	3
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17