OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack7slot4s.osadl.org (updated Sun Aug 09, 2026 12:45:09)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
5458	99	67	2,9	cyclictest	21	-21	ksoftirqd/1	10:00:14	1
5467	99	58	30,7	cyclictest	13	1	rcu_preempt	10:00:15	2
5467	99	54	3,29	cyclictest	7970	-21	ssh	09:55:12	2
5443	99	53	4,10	cyclictest	12	-21	ksoftirqd/0	10:35:13	0
5467	99	48	1,33	cyclictest	9056	-21	ssh	12:20:13	2
5478	99	47	38,6	cyclictest	359	66	irq/58-eth0	09:35:13	3
5478	99	47	29,10	cyclictest	359	66	irq/58-eth0	09:50:13	3
5467	99	47	4,34	cyclictest	18527	-21	ssh	10:55:14	2
5443	99	47	3,11	cyclictest	12	-21	ksoftirqd/0	07:25:11	0
5458	99	46	4,7	cyclictest	21	-21	ksoftirqd/1	10:50:14	1
5458	99	46	4,3	cyclictest	13	1	rcu_preempt	12:20:12	1
5443	99	46	2,25	cyclictest	20033	-21	ssh	11:50:14	0
5467	99	44	19,10	cyclictest	13	1	rcu_preempt	11:45:11	2
5443	99	44	7,3	cyclictest	13	1	rcu_preempt	12:35:13	0
5467	99	42	28,9	cyclictest	16139	-21	ssh	11:30:00	2
5467	99	42	24,11	cyclictest	27	-21	ksoftirqd/2	10:35:13	2
5467	99	40	25,8	cyclictest	26	1	rcuc/2	10:23:34	2
5467	99	40	2,21	cyclictest	18036	-21	ssh	10:54:56	2
5478	99	39	20,11	cyclictest	359	66	irq/58-eth0	11:50:13	3
5478	99	38	23,11	cyclictest	0	-21	swapper/3	11:35:14	3
5458	99	37	4,3	cyclictest	13	1	rcu_preempt	11:45:12	1
5443	99	37	1,4	cyclictest	13	1	rcu_preempt	12:30:12	0
5478	99	36	3,20	cyclictest	359	66	irq/58-eth0	09:55:13	3
5467	99	36	2,28	cyclictest	25908	-21	ssh	09:28:27	2
5467	99	36	2,17	cyclictest	27602	-21	/usr/sbin/munin	09:30:28	2
5467	99	36	2,17	cyclictest	1937	-21	ssh	10:09:42	2
5467	99	36	12,13	cyclictest	27	-21	ksoftirqd/2	11:15:12	2
5458	99	36	9,4	cyclictest	13	1	rcu_preempt	10:20:11	1
5443	99	36	4,2	cyclictest	13	1	rcu_preempt	08:10:14	0
5478	99	35	8,22	cyclictest	359	66	irq/58-eth0	10:35:15	3
5478	99	35	5,22	cyclictest	359	66	irq/58-eth0	11:45:12	3
5478	99	35	2,25	cyclictest	359	66	irq/58-eth0	07:50:13	3
5478	99	35	17,12	cyclictest	359	66	irq/58-eth0	10:00:15	3
5458	99	35	1,28	cyclictest	14561	-21	ssh	12:05:13	1
5467	99	34	2,16	cyclictest	930	-21	snmpd	11:20:13	2
5478	99	33	3,25	cyclictest	359	66	irq/58-eth0	10:25:12	3
5478	99	33	1,21	cyclictest	359	66	irq/58-eth0	07:35:12	3
5478	99	33	1,18	cyclictest	359	66	irq/58-eth0	10:20:12	3
5458	99	33	6,19	cyclictest	31404	-21	ssh	09:50:13	1
5443	99	33	3,18	cyclictest	20517	-21	ssh	10:20:13	0
5443	99	33	2,7	cyclictest	13	1	rcu_preempt	08:30:13	0
5478	99	32	2,21	cyclictest	359	66	irq/58-eth0	08:55:12	3
5478	99	32	19,9	cyclictest	0	-21	swapper/3	09:05:12	3
5478	99	32	1,21	cyclictest	359	66	irq/58-eth0	11:25:14	3
5467	99	32	4,4	cyclictest	13	1	rcu_preempt	09:00:14	2
5467	99	32	3,12	cyclictest	11565	-21	ssh	10:15:14	2
5458	99	32	1,23	cyclictest	23549	-21	diskmemload	09:30:13	1
5458	99	32	1,22	cyclictest	22363	-21	ssh	09:45:13	1
5478	99	31	5,20	cyclictest	359	66	irq/58-eth0	09:15:14	3
5478	99	31	3,17	cyclictest	359	66	irq/58-eth0	10:10:11	3
5478	99	31	2,17	cyclictest	359	66	irq/58-eth0	12:25:14	3
5467	99	31	4,8	cyclictest	5431	-21	cyclictest	09:15:13	2
5467	99	31	2,23	cyclictest	29483	-21	ssh	10:25:12	2
5443	99	31	2,22	cyclictest	13348	-21	ssh	09:40:17	0
5443	99	31	17,10	cyclictest	0	-21	swapper/0	08:45:15	0
5443	99	31	1,22	cyclictest	9595	-21	ssh	10:50:13	0
5478	99	30	4,21	cyclictest	359	66	irq/58-eth0	09:20:14	3
5478	99	30	10,14	cyclictest	0	-21	swapper/3	10:55:14	3
5458	99	30	0,25	cyclictest	0	-21	swapper/1	12:25:13	1
5443	99	30	5,3	cyclictest	13	1	rcu_preempt	08:05:14	0
5443	99	30	5,15	cyclictest	23968	-21	ssh	09:10:15	0
5443	99	30	3,23	cyclictest	27537	-21	ssh	11:00:13	0
5443	99	30	3,23	cyclictest	27537	-21	ssh	11:00:12	0
5478	99	29	19,6	cyclictest	0	-21	swapper/3	12:20:12	3
5478	99	29	17,7	cyclictest	0	-21	swapper/3	11:05:14	3
5478	99	29	16,9	cyclictest	0	-21	swapper/3	10:05:14	3
5478	99	29	16,7	cyclictest	359	66	irq/58-eth0	11:30:13	3
5467	99	29	2,20	cyclictest	0	-21	swapper/2	07:35:14	2
5458	99	29	1,4	cyclictest	13	1	rcu_preempt	08:50:14	1
5458	99	29	13,4	cyclictest	21	-21	ksoftirqd/1	12:35:19	1
5443	99	29	1,18	cyclictest	73	-21	kswapd0	12:05:12	0
5478	99	28	3,15	cyclictest	359	66	irq/58-eth0	10:15:15	3
5467	99	28	3,14	cyclictest	273	46	irq/40-dwc2_hso	11:10:14	2
5467	99	28	2,19	cyclictest	273	46	irq/40-dwc2_hso	09:45:13	2
5458	99	28	8,3	cyclictest	21	-21	ksoftirqd/1	07:40:26	1
5458	99	28	4,6	cyclictest	21	-21	ksoftirqd/1	10:05:14	1
5458	99	28	2,20	cyclictest	16300	-21	ssh	09:59:41	1
5458	99	28	17,8	cyclictest	21	-21	ksoftirqd/1	11:00:12	1
5458	99	28	17,8	cyclictest	21	-21	ksoftirqd/1	11:00:11	1
5458	99	28	10,15	cyclictest	21	-21	ksoftirqd/1	10:30:13	1
5443	99	28	4,20	cyclictest	2610	-21	ssh	10:10:14	0
5443	99	28	11,11	cyclictest	0	-21	swapper/0	11:22:56	0
5478	99	27	20,2	cyclictest	0	-21	swapper/3	12:10:13	3
5478	99	27	11,11	cyclictest	0	-21	swapper/3	09:40:17	3
5467	99	27	3,19	cyclictest	0	-21	swapper/2	11:34:34	2
5467	99	27	2,19	cyclictest	0	-21	swapper/2	09:05:15	2
5467	99	27	2,15	cyclictest	273	46	irq/40-dwc2_hso	08:40:13	2
5467	99	27	1,11	cyclictest	12531	-21	ssh	09:39:36	2
5458	99	27	9,3	cyclictest	21	-21	ksoftirqd/1	08:45:31	1
5458	99	27	3,7	cyclictest	21	-21	ksoftirqd/1	07:50:12	1
5458	99	27	2,19	cyclictest	595	-21	ssh	10:45:12	1
5443	99	27	8,3	cyclictest	13	1	rcu_preempt	07:15:18	0
5443	99	27	2,19	cyclictest	26111	-21	open_inodes	09:45:27	0
5443	99	27	14,7	cyclictest	9577	-21	ssh	09:20:14	0
5443	99	27	14,3	cyclictest	12	-21	ksoftirqd/0	12:29:24	0
5443	99	27	10,11	cyclictest	0	-21	swapper/0	10:56:57	0
5478	99	26	4,15	cyclictest	0	-21	swapper/3	11:55:12	3
5478	99	26	3,17	cyclictest	0	-21	swapper/3	11:43:32	3
5478	99	26	3,16	cyclictest	0	-21	swapper/3	10:54:56	3
5478	99	26	3,16	cyclictest	0	-21	swapper/3	10:45:13	3
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17