OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack7slot4s.osadl.org (updated Sun Jul 05, 2026 00:45:14)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
32277	99	65	38,20	cyclictest	0	-21	swapper/2	00:35:13	2
32268	99	64	3,46	cyclictest	29116	-21	ssh	22:45:13	1
32257	99	64	4,39	cyclictest	22409	-21	ssh	21:30:12	0
32268	99	59	4,43	cyclictest	16059	-21	ssh	22:20:12	1
32268	99	57	2,8	cyclictest	13	1	rcu_preempt	23:30:13	1
32268	99	56	4,40	cyclictest	687	-21	ssh	23:05:14	1
32257	99	49	2,36	cyclictest	12144	-21	ssh	00:05:15	0
32285	99	46	5,28	cyclictest	359	66	irq/58-eth0	00:10:12	3
32268	99	46	18,10	cyclictest	13	1	rcu_preempt	22:05:12	1
32257	99	45	2,35	cyclictest	3084	-21	ssh	21:55:12	0
32268	99	43	2,37	cyclictest	21111	-21	ssh	00:10:12	1
32257	99	43	1,36	cyclictest	4149	-21	ssh	21:20:14	0
32277	99	42	1,22	cyclictest	10955	-21	ssh	22:35:13	2
32268	99	42	4,30	cyclictest	9700	-21	rm	23:10:09	1
32277	99	41	1,36	cyclictest	25121	-21	ssh	22:25:12	2
32257	99	41	1,30	cyclictest	18413	-21	ssh	21:10:14	0
32268	99	40	4,26	cyclictest	6980	-21	ssh	22:15:12	1
32268	99	40	1,31	cyclictest	4130	-21	ssh	21:20:13	1
32277	99	39	17,18	cyclictest	0	-21	swapper/2	20:05:12	2
32285	99	38	3,20	cyclictest	359	66	irq/58-eth0	21:10:14	3
32285	99	38	2,19	cyclictest	359	66	irq/58-eth0	00:25:11	3
32285	99	37	4,21	cyclictest	359	66	irq/58-eth0	23:25:12	3
32277	99	37	2,29	cyclictest	26606	-21	ssh	23:55:15	2
32268	99	37	4,21	cyclictest	21	-21	ksoftirqd/1	23:45:13	1
32268	99	37	4,21	cyclictest	21	-21	ksoftirqd/1	23:45:13	1
32257	99	37	4,22	cyclictest	15734	-21	ssh	00:25:12	0
32285	99	36	2,21	cyclictest	359	66	irq/58-eth0	23:05:15	3
32277	99	36	8,21	cyclictest	2656	-1	kworker/u9:1	00:00:15	2
32277	99	36	2,28	cyclictest	27985	-21	df_inode	23:20:14	2
32277	99	36	2,18	cyclictest	6973	-21	sh	22:30:49	2
32268	99	36	1,29	cyclictest	14849	-21	ssh	22:55:13	1
32285	99	35	4,24	cyclictest	33	-21	ksoftirqd/3	00:35:13	3
32285	99	35	3,16	cyclictest	359	66	irq/58-eth0	19:30:14	3
32277	99	35	2,3	cyclictest	0	-21	swapper/2	22:55:13	2
32277	99	35	1,28	cyclictest	23945	-21	ssh	23:00:13	2
32268	99	35	7,20	cyclictest	21	-21	ksoftirqd/1	23:40:12	1
32268	99	35	3,8	cyclictest	13	1	rcu_preempt	20:25:13	1
32268	99	35	1,24	cyclictest	30226	-21	ssh	22:10:12	1
32277	99	34	2,16	cyclictest	13635	-21	ssh	22:17:38	2
32277	99	34	2,16	cyclictest	10017	-21	ssh	23:26:12	2
32277	99	34	2,11	cyclictest	17574	-21	df	23:50:14	2
32268	99	34	7,16	cyclictest	27879	-21	ssh	23:20:12	1
32268	99	34	2,24	cyclictest	18432	-21	ssh	21:10:15	1
32268	99	34	11,15	cyclictest	630	-21	Async	09:36:31	1
32285	99	33	1,17	cyclictest	359	66	irq/58-eth0	21:40:12	3
32285	99	33	1,17	cyclictest	359	66	irq/58-eth0	21:40:11	3
32277	99	33	9,18	cyclictest	0	-21	swapper/2	22:22:38	2
32257	99	33	10,17	cyclictest	26306	-21	diskmemload	22:05:49	0
32285	99	32	6,23	cyclictest	359	66	irq/58-eth0	00:15:12	3
32277	99	32	14,13	cyclictest	536	-21	usb-storage	21:25:12	2
32277	99	32	1,24	cyclictest	18012	-21	diskmemload	22:45:13	2
32285	99	31	14,11	cyclictest	359	66	irq/58-eth0	22:52:46	3
32277	99	31	4,22	cyclictest	27	-21	ksoftirqd/2	00:30:15	2
32277	99	31	10,6	cyclictest	0	-21	swapper/2	23:15:12	2
32257	99	31	2,23	cyclictest	1797	-21	ssh	23:22:13	0
32285	99	30	2,17	cyclictest	359	66	irq/58-eth0	00:20:12	3
32285	99	30	2,17	cyclictest	359	66	irq/58-eth0	00:20:12	3
32285	99	30	12,10	cyclictest	359	66	irq/58-eth0	22:35:13	3
32277	99	30	3,18	cyclictest	273	46	irq/40-dwc2_hso	20:35:12	2
32268	99	30	14,10	cyclictest	0	-21	swapper/1	00:00:13	1
32257	99	30	2,22	cyclictest	29149	-21	ssh	23:00:56	0
32285	99	29	3,17	cyclictest	0	-21	swapper/3	21:50:11	3
32285	99	29	3,15	cyclictest	359	66	irq/58-eth0	23:35:15	3
32285	99	29	2,18	cyclictest	359	66	irq/58-eth0	22:05:13	3
32277	99	29	2,20	cyclictest	0	-21	swapper/2	19:20:14	2
32268	99	29	2,24	cyclictest	5775	-21	ssh	22:50:13	1
32257	99	29	2,21	cyclictest	22732	-21	sh	22:22:39	0
32257	99	29	1,22	cyclictest	31727	-21	ssh	23:40:14	0
32285	99	28	3,18	cyclictest	0	-21	swapper/3	23:23:12	3
32285	99	28	3,13	cyclictest	359	66	irq/58-eth0	23:10:10	3
32285	99	28	2,19	cyclictest	0	-21	swapper/3	23:18:14	3
32285	99	28	1,13	cyclictest	359	66	irq/58-eth0	00:05:14	3
32277	99	28	3,20	cyclictest	0	-21	swapper/2	23:11:14	2
32277	99	28	3,18	cyclictest	273	46	irq/40-dwc2_hso	21:10:14	2
32277	99	28	3,17	cyclictest	273	46	irq/40-dwc2_hso	20:55:14	2
32277	99	28	2,20	cyclictest	0	-21	swapper/2	19:45:12	2
32277	99	28	2,20	cyclictest	0	-21	swapper/2	19:30:15	2
32277	99	28	2,11	cyclictest	16545	-21	ssh	21:44:46	2
32277	99	28	2,11	cyclictest	16545	-21	ssh	21:44:46	2
32277	99	28	12,13	cyclictest	0	-21	swapper/2	22:05:12	2
32268	99	28	2,19	cyclictest	20	1	rcuc/1	22:30:15	1
32268	99	28	1,24	cyclictest	10939	-21	ssh	22:35:12	1
32257	99	28	2,20	cyclictest	31499	-21	ssh	21:35:13	0
32277	99	27	3,17	cyclictest	273	46	irq/40-dwc2_hso	21:00:12	2
32268	99	27	2,20	cyclictest	17578	-21	rm	23:50:14	1
32268	99	27	2,19	cyclictest	23965	-21	ssh	23:00:14	1
32257	99	27	2,20	cyclictest	19447	-21	ssh	22:39:51	0
32257	99	27	2,20	cyclictest	10374	-21	ssh	22:34:49	0
32285	99	26	9,9	cyclictest	359	66	irq/58-eth0	21:55:13	3
32285	99	26	3,16	cyclictest	0	-21	swapper/3	23:00:15	3
32285	99	26	2,4	cyclictest	0	-21	swapper/3	22:25:28	3
32285	99	26	1,8	cyclictest	0	-21	swapper/3	22:55:55	3
32277	99	26	3,10	cyclictest	273	46	irq/40-dwc2_hso	20:15:11	2
32277	99	26	3,10	cyclictest	273	46	irq/40-dwc2_hso	20:15:11	2
32277	99	26	2,16	cyclictest	11017	-21	grep	19:55:17	2
32277	99	26	13,8	cyclictest	11979	-1	kworker/u9:2	22:52:46	2
32277	99	26	1,17	cyclictest	0	-21	swapper/2	21:35:12	2
32277	99	26	0,22	cyclictest	2492	-21	find	20:20:11	2
32268	99	26	4,15	cyclictest	0	-21	swapper/1	00:25:13	1
32285	99	25	8,4	cyclictest	0	-21	swapper/3	22:20:41	3
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17