OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack7slot4s.osadl.org (updated Thu Dec 11, 2025 00:45:03)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
27180	99	73	1,4	cyclictest	13	1	rcu_preempt	22:30:12	0
27193	99	64	5,46	cyclictest	14644	-21	ssh	21:30:15	1
27202	99	55	2,44	cyclictest	29156	-21	sh	00:15:13	2
27180	99	55	3,6	cyclictest	13	1	rcu_preempt	19:15:11	0
27193	99	49	4,35	cyclictest	27690	-21	ssh	23:35:14	1
27210	99	46	3,30	cyclictest	295	66	irq/58-eth0	22:15:12	3
27193	99	46	2,6	cyclictest	13	1	rcu_preempt	22:55:15	1
27202	99	44	12,14	cyclictest	27	-21	ksoftirqd/2	23:30:14	2
27180	99	44	1,33	cyclictest	9818	-21	ssh	22:45:14	0
27210	99	43	22,14	cyclictest	295	66	irq/58-eth0	23:20:14	3
27202	99	43	18,19	cyclictest	27	-21	ksoftirqd/2	22:51:24	2
27193	99	43	2,37	cyclictest	30384	-21	ssh	21:20:15	1
27210	99	42	18,18	cyclictest	32	1	rcuc/3	22:30:12	3
27202	99	42	2,32	cyclictest	14629	-21	cut	00:05:23	2
27210	99	41	3,26	cyclictest	295	66	irq/58-eth0	23:45:12	3
27210	99	41	2,26	cyclictest	295	66	irq/58-eth0	00:30:14	3
27210	99	41	2,23	cyclictest	295	66	irq/58-eth0	19:10:14	3
27180	99	41	1,30	cyclictest	26742	-21	sh	23:15:15	0
27210	99	40	4,25	cyclictest	295	66	irq/58-eth0	00:10:14	3
27210	99	40	16,14	cyclictest	295	66	irq/58-eth0	00:35:14	3
27210	99	39	3,24	cyclictest	295	66	irq/58-eth0	20:30:13	3
27202	99	37	6,12	cyclictest	24664	-21	ssh	22:15:12	2
27202	99	37	20,11	cyclictest	26	1	rcuc/2	19:50:12	2
27193	99	37	24,9	cyclictest	21	-21	ksoftirqd/1	20:45:13	1
27193	99	37	1,32	cyclictest	29158	-21	sh	00:15:14	1
27202	99	36	7,23	cyclictest	26	1	rcuc/2	23:55:11	2
27202	99	35	9,9	cyclictest	2630	-21	ssh	23:20:14	2
27202	99	35	21,10	cyclictest	13	1	rcu_preempt	21:05:12	2
27180	99	35	2,29	cyclictest	1434	-21	ssh	22:40:12	0
27210	99	34	1,26	cyclictest	295	66	irq/58-eth0	22:55:28	3
27202	99	34	1,24	cyclictest	16440	-21	ssh	22:10:16	2
27193	99	34	3,27	cyclictest	13688	-21	ssh	21:10:14	1
27193	99	34	2,24	cyclictest	5113	-21	ntp_states	22:40:24	1
27210	99	33	2,18	cyclictest	295	66	irq/58-eth0	22:35:14	3
27202	99	32	11,4	cyclictest	13	1	rcu_preempt	23:10:15	2
27193	99	32	3,26	cyclictest	11998	-21	ssh	23:45:13	1
27180	99	32	4,16	cyclictest	12	-21	ksoftirqd/0	20:40:12	0
27180	99	32	2,22	cyclictest	21628	-21	missed_timers	22:50:23	0
27180	99	32	1,23	cyclictest	23816	-21	ssh	21:55:12	0
27210	99	31	5,18	cyclictest	0	-21	swapper/3	23:30:12	3
27202	99	31	3,13	cyclictest	27	-21	ksoftirqd/2	20:00:15	2
27180	99	31	4,19	cyclictest	4493	-21	ssh	00:00:14	0
27202	99	30	3,22	cyclictest	0	-21	swapper/2	00:27:24	2
27202	99	30	3,20	cyclictest	31275	-21	ssh	21:40:15	2
27193	99	30	7,16	cyclictest	21	-21	ksoftirqd/1	20:35:13	1
27193	99	30	2,22	cyclictest	11112	-21	ssh	00:23:50	1
27193	99	30	2,21	cyclictest	622	-21	ssh	22:20:14	1
27180	99	30	1,4	cyclictest	13	1	rcu_preempt	23:25:15	0
27202	99	29	5,16	cyclictest	263	46	irq/40-dwc2_hso	20:35:14	2
27202	99	29	2,21	cyclictest	5470	-21	ssh	22:01:38	2
27202	99	29	2,21	cyclictest	5470	-21	ssh	22:01:38	2
27202	99	29	1,10	cyclictest	20910	-21	ssh	00:10:13	2
27180	99	29	9,16	cyclictest	15511	-21	awk	00:25:24	0
27210	99	28	21,4	cyclictest	295	66	irq/58-eth0	21:25:14	3
27210	99	28	1,18	cyclictest	295	66	irq/58-eth0	22:50:15	3
27202	99	28	1,15	cyclictest	8987	-21	ssh	22:25:15	2
27180	99	28	2,22	cyclictest	27538	-21	ssh	00:35:15	0
27180	99	28	2,20	cyclictest	11143	-21	ssh	00:23:57	0
27180	99	28	2,18	cyclictest	22915	-21	ssh	21:35:12	0
27180	99	28	2,17	cyclictest	20919	-21	ssh	00:10:14	0
27210	99	27	6,13	cyclictest	295	66	irq/58-eth0	22:45:14	3
27210	99	27	3,16	cyclictest	0	-21	swapper/3	00:07:24	3
27210	99	27	3,15	cyclictest	0	-21	swapper/3	22:41:46	3
27210	99	27	2,15	cyclictest	0	-21	swapper/3	21:55:12	3
27210	99	27	1,19	cyclictest	0	-21	swapper/3	20:40:12	3
27210	99	27	1,17	cyclictest	295	66	irq/58-eth0	21:30:15	3
27210	99	27	1,11	cyclictest	0	-21	swapper/3	20:55:14	3
27210	99	27	11,10	cyclictest	295	66	irq/58-eth0	21:10:14	3
27202	99	27	9,15	cyclictest	27	-21	ksoftirqd/2	22:30:12	2
27202	99	27	3,18	cyclictest	263	46	irq/40-dwc2_hso	23:25:15	2
27202	99	27	1,19	cyclictest	27769	-21	ls	23:35:14	2
27202	99	27	1,18	cyclictest	263	46	irq/40-dwc2_hso	19:20:15	2
27193	99	27	3,18	cyclictest	21	-21	ksoftirqd/1	23:05:14	1
27193	99	27	2,22	cyclictest	14630	-21	tr	00:05:24	1
27193	99	27	1,17	cyclictest	16425	-21	ssh	22:10:15	1
27193	99	27	11,10	cyclictest	21	-21	ksoftirqd/1	20:20:11	1
27210	99	26	2,18	cyclictest	0	-21	swapper/3	23:04:24	3
27193	99	26	3,18	cyclictest	19747	-21	ssh	00:30:13	1
27180	99	26	3,11	cyclictest	0	-21	swapper/0	22:55:15	0
27210	99	25	9,11	cyclictest	0	-21	swapper/3	23:35:24	3
27210	99	25	3,17	cyclictest	0	-21	swapper/3	23:07:24	3
27210	99	25	3,17	cyclictest	0	-21	swapper/3	21:20:16	3
27202	99	25	3,16	cyclictest	263	46	irq/40-dwc2_hso	21:00:14	2
27202	99	25	2,20	cyclictest	0	-21	swapper/2	20:40:13	2
27202	99	25	2,18	cyclictest	263	46	irq/40-dwc2_hso	22:20:14	2
27202	99	25	1,7	cyclictest	11697	-21	ls	00:25:01	2
27202	99	25	1,20	cyclictest	26758	-21	cut	20:50:17	2
27202	99	25	1,16	cyclictest	263	46	irq/40-dwc2_hso	21:45:11	2
27193	99	25	9,11	cyclictest	21	-21	ksoftirqd/1	21:58:24	1
27210	99	24	8,9	cyclictest	32	1	rcuc/3	21:45:12	3
27210	99	24	12,9	cyclictest	0	-21	swapper/3	22:10:31	3
27202	99	24	3,15	cyclictest	263	46	irq/40-dwc2_hso	23:15:14	2
27202	99	24	2,17	cyclictest	0	-21	swapper/2	22:05:24	2
27202	99	24	2,15	cyclictest	263	46	irq/40-dwc2_hso	21:35:14	2
27202	99	24	2,15	cyclictest	263	46	irq/40-dwc2_hso	20:25:16	2
27202	99	24	13,5	cyclictest	13297	-21	diskmemload	22:35:14	2
27202	99	24	1,16	cyclictest	263	46	irq/40-dwc2_hso	23:50:16	2
27202	99	24	1,16	cyclictest	263	46	irq/40-dwc2_hso	20:05:12	2
27202	99	24	0,15	cyclictest	13713	-21	ls	22:45:29	2
27193	99	24	2,17	cyclictest	21	-21	ksoftirqd/1	19:55:13	1
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17