OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack7slot4s.osadl.org (updated Sat Aug 29, 2026 00:45:09)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
31025	99	84	6,5	cyclictest	13	1	rcu_preempt	20:40:13	0
31052	99	77	42,22	cyclictest	359	66	irq/58-eth0	00:25:13	3
31033	99	74	20,11	cyclictest	13	1	rcu_preempt	20:45:12	1
31033	99	66	4,47	cyclictest	26673	-21	ssh	00:15:12	1
31025	99	64	6,41	cyclictest	2807	-21	ssh	22:50:15	0
31033	99	63	7,14	cyclictest	21	-21	ksoftirqd/1	19:30:13	1
31052	99	62	54,4	cyclictest	359	66	irq/58-eth0	21:45:13	3
31044	99	62	4,28	cyclictest	8223	-21	ssh	22:35:11	2
31025	99	61	3,45	cyclictest	6461	-21	ssh	21:40:11	0
31044	99	60	4,43	cyclictest	3174	-21	ssh	00:20:12	2
31044	99	60	4,43	cyclictest	3174	-21	ssh	00:20:11	2
31025	99	55	4,47	cyclictest	17610	-21	ssh	00:10:15	0
31033	99	54	5,38	cyclictest	24622	-21	ssh	23:20:13	1
31033	99	54	5,38	cyclictest	24622	-21	ssh	23:20:13	1
31052	99	49	4,32	cyclictest	359	66	irq/58-eth0	21:30:12	3
31052	99	49	27,13	cyclictest	359	66	irq/58-eth0	00:15:12	3
31025	99	49	2,40	cyclictest	26257	-21	ssh	22:45:14	0
31025	99	47	3,36	cyclictest	26338	-21	ssh	21:15:12	0
31025	99	47	2,34	cyclictest	28293	-21	ssh	23:40:12	0
31044	99	45	4,24	cyclictest	15431	-21	ssh	21:45:12	2
31025	99	45	2,37	cyclictest	14012	-21	ssh	23:50:13	0
31033	99	43	5,25	cyclictest	13644	-21	ssh	22:20:13	1
31052	99	41	3,26	cyclictest	359	66	irq/58-eth0	00:35:12	3
31044	99	41	15,19	cyclictest	27	-21	ksoftirqd/2	21:40:13	2
31025	99	41	2,17	cyclictest	8636	-21	ssh	00:05:13	0
31025	99	40	3,27	cyclictest	3184	-21	ssh	00:20:11	0
31025	99	40	3,27	cyclictest	3184	-21	ssh	00:20:11	0
31044	99	39	2,30	cyclictest	27178	-21	cat	23:55:30	2
31025	99	39	2,6	cyclictest	0	-21	swapper/0	22:43:29	0
31044	99	38	1,21	cyclictest	17181	-21	ssh	22:40:12	2
31033	99	38	2,32	cyclictest	2889	-21	ssh	21:20:12	1
31033	99	38	22,10	cyclictest	0	-21	swapper/1	00:00:13	1
31025	99	38	2,24	cyclictest	24411	-21	ssh	21:50:11	0
31044	99	37	4,25	cyclictest	0	-21	swapper/2	22:50:15	2
31033	99	37	4,25	cyclictest	4955	-21	sh	23:45:11	1
31025	99	37	4,20	cyclictest	12114	-21	ssh	00:25:14	0
31052	99	36	5,22	cyclictest	359	66	irq/58-eth0	23:40:12	3
31052	99	36	2,22	cyclictest	359	66	irq/58-eth0	22:15:12	3
31044	99	36	2,28	cyclictest	15967	-21	awk	22:55:29	2
31044	99	36	2,18	cyclictest	20455	-21	ssh	23:52:26	2
31033	99	35	15,15	cyclictest	21	-21	ksoftirqd/1	00:10:14	1
31025	99	35	8,5	cyclictest	31472	-21	date	22:30:02	0
31025	99	35	2,10	cyclictest	3892	-21	kthreadcore	23:25:22	0
31044	99	34	16,15	cyclictest	27	-21	ksoftirqd/2	23:25:13	2
31025	99	34	2,17	cyclictest	15440	-21	ssh	21:45:13	0
31025	99	34	10,4	cyclictest	4966	-21	ssh	22:15:17	0
31052	99	33	3,20	cyclictest	359	66	irq/58-eth0	00:00:15	3
31033	99	33	3,23	cyclictest	9998	-21	ssh	22:00:13	1
31025	99	33	1,8	cyclictest	31158	-21	apt-get	19:10:12	0
31025	99	33	1,29	cyclictest	22996	-21	sh	23:55:14	0
31025	99	33	1,11	cyclictest	13701	-21	missed_timers	23:30:22	0
31025	99	33	1,10	cyclictest	27669	-21	wc	22:10:01	0
31052	99	32	6,17	cyclictest	359	66	irq/58-eth0	22:50:14	3
31052	99	32	4,15	cyclictest	359	66	irq/58-eth0	19:20:12	3
31044	99	32	5,18	cyclictest	273	46	irq/40-dwc2_hso	23:15:12	2
31044	99	32	21,8	cyclictest	0	-21	swapper/2	21:30:13	2
31044	99	32	20,6	cyclictest	28299	-21	ssh	23:40:12	2
31033	99	32	4,17	cyclictest	29933	-21	ssh	23:05:14	1
31025	99	32	1,11	cyclictest	3081	-21	ssh	21:55:20	0
31025	99	32	1,11	cyclictest	17727	-21	munin-run	19:30:01	0
31025	99	32	1,11	cyclictest	1068	-21	kthreadcore	22:30:21	0
31025	99	32	0,11	cyclictest	15884	-21	ssh	21:25:29	0
31052	99	31	3,21	cyclictest	359	66	irq/58-eth0	21:20:10	3
31052	99	31	10,9	cyclictest	359	66	irq/58-eth0	21:25:13	3
31044	99	31	6,17	cyclictest	273	46	irq/40-dwc2_hso	20:20:12	2
31044	99	30	5,17	cyclictest	273	46	irq/40-dwc2_hso	20:05:10	2
31025	99	30	1,12	cyclictest	30462	-21	irqcore	00:35:17	0
31025	99	30	1,11	cyclictest	10911	-21	cut	22:00:19	0
31044	99	29	3,16	cyclictest	273	46	irq/40-dwc2_hso	19:10:11	2
31033	99	29	5,13	cyclictest	21	-21	ksoftirqd/1	19:55:13	1
31025	99	29	2,8	cyclictest	16910	-21	ls	22:40:01	0
31025	99	29	2,15	cyclictest	4956	-21	rm	23:45:11	0
31025	99	29	1,9	cyclictest	28209	-21	awk	23:20:24	0
31025	99	29	1,9	cyclictest	28209	-21	awk	23:20:24	0
31025	99	29	1,10	cyclictest	30001	-21	df	21:35:14	0
31052	99	28	3,18	cyclictest	0	-21	swapper/3	23:52:26	3
31052	99	28	3,16	cyclictest	0	-21	swapper/3	23:55:29	3
31052	99	28	13,10	cyclictest	359	66	irq/58-eth0	21:10:13	3
31052	99	28	1,13	cyclictest	359	66	irq/58-eth0	23:00:14	3
31044	99	28	3,16	cyclictest	273	46	irq/40-dwc2_hso	19:45:12	2
31044	99	28	2,22	cyclictest	930	-21	snmpd	22:00:13	2
31044	99	28	1,21	cyclictest	26536	-21	sh	00:15:05	2
31033	99	28	4,12	cyclictest	31014	-21	cyclictest	19:45:12	1
31033	99	28	1,18	cyclictest	1178	-21	rm	23:25:12	1
31025	99	28	2,6	cyclictest	0	-21	swapper/0	22:55:29	0
31052	99	27	1,12	cyclictest	359	66	irq/58-eth0	19:10:11	3
31044	99	27	5,16	cyclictest	273	46	irq/40-dwc2_hso	23:00:13	2
31044	99	27	3,19	cyclictest	0	-21	swapper/2	00:26:40	2
31044	99	27	2,16	cyclictest	273	46	irq/40-dwc2_hso	21:35:13	2
31033	99	27	9,12	cyclictest	630	-21	Async	09:36:31	1
31033	99	27	2,13	cyclictest	11955	-21	ssh	21:25:13	1
31025	99	27	3,6	cyclictest	0	-21	swapper/0	22:23:29	0
31052	99	26	9,11	cyclictest	359	66	irq/58-eth0	00:05:13	3
31052	99	26	12,9	cyclictest	359	66	irq/58-eth0	00:10:14	3
31044	99	26	3,16	cyclictest	273	46	irq/40-dwc2_hso	22:20:12	2
31044	99	26	2,15	cyclictest	273	46	irq/40-dwc2_hso	23:10:14	2
31044	99	26	1,16	cyclictest	273	46	irq/40-dwc2_hso	19:30:13	2
31033	99	26	4,10	cyclictest	22616	-21	ssh	22:25:14	1
31033	99	26	3,16	cyclictest	21	-21	ksoftirqd/1	21:05:12	1
31033	99	26	2,19	cyclictest	18371	-21	sh	22:57:29	1
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17