OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack7slot4s.osadl.org (updated Tue Aug 25, 2026 00:45:17)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
23032	99	52	2,30	cyclictest	13944	-21	ssh	22:25:13	2
23024	99	51	1,2	cyclictest	0	-21	swapper/1	22:05:14	1
23024	99	51	1,2	cyclictest	0	-21	swapper/1	22:05:13	1
23040	99	50	30,11	cyclictest	359	66	irq/58-eth0	23:40:16	3
23032	99	50	33,12	cyclictest	930	-21	snmpd	21:00:13	2
23032	99	50	23,16	cyclictest	0	-21	swapper/2	22:05:14	2
23032	99	50	23,16	cyclictest	0	-21	swapper/2	22:05:13	2
23016	99	50	1,23	cyclictest	31811	-21	ssh	22:35:12	0
23040	99	48	34,8	cyclictest	359	66	irq/58-eth0	21:55:14	3
23032	99	48	37,8	cyclictest	16302	-21	apt-get	21:50:13	2
23024	99	48	2,35	cyclictest	9239	-21	ssh	21:10:11	1
23024	99	48	1,44	cyclictest	0	-21	swapper/1	19:15:13	1
23032	99	47	27,11	cyclictest	6175	-21	ssh	00:35:11	2
23032	99	45	16,14	cyclictest	7416	-21	ssh	00:15:22	2
23032	99	45	15,11	cyclictest	30521	-21	ssh	23:48:52	2
23032	99	45	15,11	cyclictest	30521	-21	ssh	23:48:51	2
23024	99	45	2,37	cyclictest	2968	-21	apt-get	20:30:14	1
23024	99	45	1,6	cyclictest	0	-21	swapper/1	19:35:14	1
23024	99	45	0,3	cyclictest	3687	-21	apt-get	21:25:12	1
23024	99	45	0,3	cyclictest	3687	-21	apt-get	21:25:11	1
23040	99	44	0,39	cyclictest	0	-21	swapper/3	22:45:14	3
23032	99	44	30,9	cyclictest	12840	-21	apt-get	21:30:16	2
23024	99	44	2,36	cyclictest	19592	-21	apt-get	22:10:12	1
23032	99	43	18,14	cyclictest	0	-21	swapper/2	19:30:11	2
23032	99	43	14,24	cyclictest	0	-21	swapper/2	00:05:16	2
23024	99	43	2,36	cyclictest	0	-21	swapper/1	21:50:13	1
23024	99	43	1,6	cyclictest	0	-21	swapper/1	20:25:13	1
23032	99	42	3,27	cyclictest	25309	-21	ssh	21:55:13	2
23032	99	42	22,8	cyclictest	0	-21	swapper/2	19:40:12	2
23024	99	42	1,5	cyclictest	0	-21	swapper/1	20:40:13	1
23024	99	42	1,5	cyclictest	0	-21	swapper/1	20:40:12	1
23024	99	42	1,2	cyclictest	21833	-21	apt-get	00:05:15	1
23016	99	42	3,22	cyclictest	10725	-21	ssh	22:05:16	0
23016	99	42	3,22	cyclictest	10725	-21	ssh	22:05:15	0
23032	99	41	21,9	cyclictest	18670	-21	ssh	23:40:14	2
23032	99	41	12,13	cyclictest	3371	-21	ssh	22:18:32	2
23024	99	41	2,35	cyclictest	17260	-21	apt-get	22:45:14	1
23024	99	41	2,35	cyclictest	0	-21	swapper/1	23:40:15	1
23024	99	41	1,35	cyclictest	0	-21	swapper/1	19:40:12	1
23024	99	41	1,30	cyclictest	18253	-21	sh	21:15:15	1
23024	99	41	1,2	cyclictest	12776	-21	apt-get	00:00:12	1
23032	99	40	34,3	cyclictest	6370	-21	apt-get	20:00:13	2
23032	99	40	17,18	cyclictest	0	-21	swapper/2	20:40:13	2
23032	99	40	17,18	cyclictest	0	-21	swapper/2	20:40:12	2
23032	99	40	17,18	cyclictest	0	-21	swapper/2	00:00:12	2
23032	99	40	15,9	cyclictest	4673	-21	ssh	00:12:16	2
23032	99	40	11,15	cyclictest	16864	-21	rm	21:13:51	2
23024	99	40	2,35	cyclictest	0	-21	swapper/1	21:00:12	1
23024	99	40	1,34	cyclictest	15164	-21	apt-get	23:20:13	1
23024	99	40	1,2	cyclictest	0	-21	swapper/1	20:00:13	1
23032	99	39	18,17	cyclictest	0	-21	swapper/2	23:05:14	2
23032	99	39	15,8	cyclictest	1513	-21	ssh	22:53:56	2
23032	99	39	15,7	cyclictest	10335	-21	ssh	23:57:31	2
23032	99	38	18,15	cyclictest	0	-21	swapper/2	23:10:13	2
23032	99	38	13,18	cyclictest	0	-21	swapper/2	21:45:15	2
23024	99	38	1,4	cyclictest	0	-21	swapper/1	19:55:13	1
23032	99	37	14,9	cyclictest	21521	-21	ssh	23:22:22	2
23032	99	37	14,7	cyclictest	22873	-21	ssh	22:46:24	2
23024	99	37	2,29	cyclictest	9828	-21	apt-get	23:35:23	1
23024	99	37	1,33	cyclictest	20716	-21	apt-get	23:05:15	1
23024	99	37	1,2	cyclictest	0	-21	swapper/1	21:45:16	1
23016	99	37	6,25	cyclictest	22880	-21	ssh	22:30:14	0
23016	99	37	2,32	cyclictest	17281	-21	ssh	22:45:12	0
23032	99	36	30,2	cyclictest	11660	-21	apt-get	00:20:09	2
23032	99	36	20,12	cyclictest	5006	-21	ssh	22:20:13	2
23032	99	36	17,15	cyclictest	433	-21	latency_hist	23:30:03	2
23032	99	36	14,7	cyclictest	7556	-21	ssh	22:39:35	2
23024	99	36	1,32	cyclictest	0	-21	swapper/1	22:50:11	1
23024	99	36	1,2	cyclictest	4987	-21	apt-get	22:20:14	1
23024	99	36	1,2	cyclictest	28478	-21	apt-get	22:15:13	1
23024	99	36	1,2	cyclictest	0	-21	swapper/1	21:30:15	1
23024	99	36	1,22	cyclictest	0	-21	swapper/1	21:55:33	1
23016	99	36	14,19	cyclictest	12	-21	ksoftirqd/0	21:50:11	0
23032	99	35	8,11	cyclictest	0	-21	swapper/2	21:44:55	2
23032	99	35	19,13	cyclictest	8032	-21	/usr/sbin/munin	21:05:29	2
23032	99	35	19,13	cyclictest	8032	-21	/usr/sbin/munin	21:05:28	2
23032	99	35	18,8	cyclictest	1794	-21	/usr/sbin/munin	19:55:18	2
23032	99	35	18,14	cyclictest	21753	-21	/usr/sbin/munin	21:35:22	2
23032	99	35	18,14	cyclictest	0	-21	swapper/2	22:00:13	2
23032	99	35	17,9	cyclictest	25888	-21	df	20:20:14	2
23032	99	35	17,8	cyclictest	11212	-21	open_inodes	20:35:27	2
23032	99	35	16,16	cyclictest	23325	-21	grep	22:10:24	2
23032	99	35	10,17	cyclictest	0	-21	swapper/2	22:33:16	2
23024	99	35	1,30	cyclictest	30788	-21	/usr/sbin/munin	21:40:25	1
23024	99	35	0,31	cyclictest	11680	-21	ssh	00:20:10	1
23016	99	35	2,29	cyclictest	20698	-21	ssh	00:25:14	0
23016	99	35	1,30	cyclictest	20719	-21	ssh	23:05:13	0
23016	99	35	1,30	cyclictest	12786	-21	ssh	00:00:10	0
23032	99	34	9,9	cyclictest	0	-21	swapper/2	00:29:38	2
23032	99	34	20,11	cyclictest	0	-21	swapper/2	20:05:15	2
23032	99	34	18,8	cyclictest	12031	-21	ntp_states	22:40:26	2
23032	99	34	18,13	cyclictest	543	-21	ssh	23:51:40	2
23032	99	34	17,14	cyclictest	8587	-21	kthreadcore	23:15:22	2
23032	99	34	17,14	cyclictest	28296	-21	/usr/sbin/munin	19:45:30	2
23032	99	34	16,14	cyclictest	21056	-21	timerandwakeup	20:45:28	2
23032	99	34	16,14	cyclictest	1544	-21	ssh	21:22:51	2
23032	99	34	16,14	cyclictest	1544	-21	ssh	21:22:50	2
23032	99	34	13,8	cyclictest	10583	-21	ssh	21:27:53	2
23032	99	34	13,8	cyclictest	10583	-21	ssh	21:27:52	2
23024	99	34	3,21	cyclictest	21	-21	ksoftirqd/1	19:50:14	1
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17