OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack7slot4s.osadl.org (updated Thu May 21, 2026 00:45:15)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
550	99	73	4,44	cyclictest	6652	-21	rm	00:20:11	2
550	99	67	3,32	cyclictest	4953	-21	ssh	21:20:13	2
550	99	67	3,32	cyclictest	4953	-21	ssh	21:20:13	2
542	99	65	20,8	cyclictest	13	1	rcu_preempt	23:15:14	1
542	99	64	5,47	cyclictest	15030	-21	ssh	22:55:11	1
550	99	62	5,45	cyclictest	7476	-21	ssh	22:15:12	2
562	99	58	34,10	cyclictest	359	66	irq/58-eth0	22:30:12	3
550	99	55	1,36	cyclictest	930	-21	snmpd	23:35:12	2
562	99	54	36,13	cyclictest	359	66	irq/58-eth0	21:30:10	3
550	99	54	2,32	cyclictest	27957	-21	ssh	23:20:11	2
542	99	49	12,24	cyclictest	21	-21	ksoftirqd/1	23:30:13	1
562	99	48	12,27	cyclictest	359	66	irq/58-eth0	22:20:12	3
542	99	48	4,7	cyclictest	13	1	rcu_preempt	20:40:12	1
550	99	46	21,17	cyclictest	13	1	rcu_preempt	20:20:13	2
550	99	46	0,38	cyclictest	9828	-21	ssh	23:10:13	2
542	99	45	4,30	cyclictest	13	1	rcu_preempt	19:45:11	1
550	99	43	23,11	cyclictest	27	-21	ksoftirqd/2	19:15:11	2
550	99	43	23,11	cyclictest	27	-21	ksoftirqd/2	19:15:11	2
550	99	43	1,34	cyclictest	2159	-21	ssh	22:30:12	2
531	99	41	1,37	cyclictest	20315	-21	sh	22:40:13	0
550	99	40	3,31	cyclictest	19249	-21	ssh	21:10:14	2
542	99	40	2,10	cyclictest	21	-21	ksoftirqd/1	21:15:12	1
531	99	40	2,33	cyclictest	29427	-21	ssh	22:45:13	0
562	99	38	5,15	cyclictest	359	66	irq/58-eth0	20:00:13	3
550	99	38	21,12	cyclictest	27	1	ksoftirqd/2	23:25:12	2
550	99	35	3,26	cyclictest	12702	-21	ssh	22:00:13	2
550	99	35	1,22	cyclictest	3601	-21	ssh	21:55:12	2
531	99	35	27,3	cyclictest	30832	-21	apt-get	20:15:13	0
562	99	34	1,19	cyclictest	359	66	irq/58-eth0	19:10:08	3
550	99	34	7,18	cyclictest	273	46	irq/40-dwc2_hso	20:25:12	2
542	99	34	4,10	cyclictest	21	-21	ksoftirqd/1	23:40:12	1
531	99	34	9,18	cyclictest	0	-21	swapper/0	22:20:11	0
531	99	34	26,3	cyclictest	26022	-21	apt-get	20:10:12	0
531	99	34	22,5	cyclictest	13959	-21	apt-get	21:05:03	0
531	99	34	21,8	cyclictest	30035	-21	apt-get	00:15:12	0
562	99	33	1,18	cyclictest	359	66	irq/58-eth0	00:05:13	3
550	99	33	3,23	cyclictest	31669	-21	ssh	23:40:11	2
550	99	33	15,10	cyclictest	521	-21	cyclictest	23:55:12	2
531	99	33	6,12	cyclictest	9447	-21	ssh	21:56:39	0
531	99	33	5,10	cyclictest	29400	-21	kthreadcore	20:45:20	0
531	99	33	3,25	cyclictest	8312	-21	ssh	23:45:13	0
531	99	33	3,25	cyclictest	8312	-21	ssh	23:45:12	0
550	99	32	4,5	cyclictest	13	1	rcu_preempt	20:05:15	2
550	99	32	4,5	cyclictest	13	1	rcu_preempt	20:05:15	2
550	99	32	17,9	cyclictest	27	-21	ksoftirqd/2	20:55:12	2
550	99	32	17,9	cyclictest	27	-21	ksoftirqd/2	20:55:12	2
531	99	32	20,8	cyclictest	722	-21	apt-get	19:10:07	0
531	99	32	15,12	cyclictest	31679	-21	ssh	23:40:12	0
562	99	31	13,12	cyclictest	359	66	irq/58-eth0	21:45:14	3
550	99	31	6,16	cyclictest	273	46	irq/40-dwc2_hso	21:15:14	2
550	99	31	5,15	cyclictest	273	46	irq/40-dwc2_hso	23:05:11	2
550	99	31	4,16	cyclictest	27	-21	ksoftirqd/2	19:25:13	2
550	99	31	2,22	cyclictest	17395	-21	ssh	23:50:14	2
542	99	31	4,23	cyclictest	21	-21	ksoftirqd/1	21:35:12	1
542	99	31	3,26	cyclictest	16512	-21	ssh	22:20:12	1
542	99	31	2,24	cyclictest	21019	-21	ssh	00:10:11	1
542	99	31	2,22	cyclictest	8830	-21	sh	21:40:12	1
531	99	31	6,10	cyclictest	14945	-21	ssh	23:10:51	0
531	99	31	5,9	cyclictest	13018	-21	switchtime	21:00:29	0
531	99	31	5,16	cyclictest	30864	-21	ssh	23:02:50	0
531	99	31	5,16	cyclictest	30864	-21	ssh	23:02:49	0
531	99	31	5,10	cyclictest	21111	-21	ssh	23:33:40	0
531	99	31	4,10	cyclictest	7588	-21	ssh	23:07:55	0
531	99	31	1,11	cyclictest	24932	-21	ssh	23:53:44	0
550	99	30	3,17	cyclictest	273	46	irq/40-dwc2_hso	19:45:13	2
542	99	30	3,9	cyclictest	21	-21	ksoftirqd/1	23:10:13	1
531	99	30	5,9	cyclictest	10928	-21	ls	22:15:22	0
531	99	30	4,8	cyclictest	26536	-21	ssh	00:11:16	0
531	99	30	22,3	cyclictest	4552	-21	apt-get	23:25:13	0
531	99	30	2,11	cyclictest	16686	-21	cat	00:25:18	0
531	99	30	18,9	cyclictest	30892	-21	ssh	22:10:12	0
531	99	30	1,20	cyclictest	12062	-21	ssh	00:05:15	0
562	99	29	1,17	cyclictest	359	66	irq/58-eth0	22:55:13	3
562	99	29	1,14	cyclictest	359	66	irq/58-eth0	00:00:13	3
550	99	29	5,13	cyclictest	273	46	irq/40-dwc2_hso	20:40:13	2
542	99	29	2,5	cyclictest	13	1	rcu_preempt	21:00:13	1
542	99	29	2,19	cyclictest	7492	-21	sh	22:15:12	1
542	99	29	16,8	cyclictest	14002	-21	ssh	21:25:12	1
531	99	29	8,17	cyclictest	12	-21	ksoftirqd/0	19:50:10	0
531	99	29	5,10	cyclictest	12416	-21	ssh	22:52:22	0
531	99	29	4,16	cyclictest	25572	-21	ssh	22:25:13	0
531	99	29	4,10	cyclictest	12766	-21	ssh	21:24:01	0
531	99	29	4,10	cyclictest	12766	-21	ssh	21:24:00	0
531	99	29	3,14	cyclictest	12	-21	ksoftirqd/0	23:35:12	0
531	99	29	2,9	cyclictest	16307	-21	ntp_192.168.114	22:00:23	0
531	99	29	2,21	cyclictest	11221	-21	ssh	22:35:14	0
531	99	29	21,4	cyclictest	3334	-21	apt-get	20:20:13	0
531	99	29	20,4	cyclictest	521	-21	cyclictest	20:50:11	0
562	99	28	9,11	cyclictest	359	66	irq/58-eth0	00:20:11	3
562	99	28	1,21	cyclictest	359	66	irq/58-eth0	22:10:13	3
550	99	28	2,21	cyclictest	6601	-21	ssh	00:36:10	2
550	99	28	2,20	cyclictest	0	-21	swapper/2	21:05:14	2
550	99	28	2,16	cyclictest	273	46	irq/40-dwc2_hso	19:30:12	2
550	99	28	2,11	cyclictest	5529	-21	ssh	00:19:10	2
550	99	28	11,13	cyclictest	22367	-21	cron	20:40:00	2
542	99	28	3,18	cyclictest	673	-21	ssh	23:05:13	1
531	99	28	5,7	cyclictest	6724	-21	ssh	00:00:25	0
531	99	28	5,14	cyclictest	4780	-21	netstat	00:35:22	0
531	99	28	0,11	cyclictest	29212	-21	usage_EthcTask	19:35:29	0
562	99	27	1,12	cyclictest	359	66	irq/58-eth0	21:10:14	3
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17