OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack7slot7s.osadl.org (updated Tue Jun 30, 2026 00:43:50)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
3333	2	98	85,9	sleep2	0	-21	swapper/2	19:08:32	2
3424	2	90	78,8	sleep0	0	-21	swapper/0	19:09:42	0
3221	2	88	64,13	sleep1	0	-21	swapper/1	19:07:06	1
3317	2	85	69,10	sleep3	0	-21	swapper/3	19:08:21	3
15002	2	69	0,0	sleep3	0	-21	swapper/3	22:37:53	3
27961	2	61	0,0	sleep2	0	-21	swapper/2	00:21:34	2
20549	2	60	0,0	sleep0	0	-21	swapper/0	23:58:32	0
10076	2	51	0,0	sleep0	0	-21	swapper/0	22:54:41	0
3532	99	28	0,27	cyclictest	0	-21	swapper/1	23:24:54	1
3532	99	25	0,22	cyclictest	0	-21	swapper/1	21:52:35	1
3532	99	23	0,1	cyclictest	0	-21	swapper/1	19:50:16	1
25873	2	23	0,0	chrt	0	-21	swapper/3	22:05:27	3
3538	99	22	2,11	cyclictest	1394	-21	wget	22:30:14	3
3538	99	22	2,11	cyclictest	1394	-21	wget	22:30:14	3
3535	99	22	0,0	cyclictest	0	-21	swapper/2	19:20:14	2
3538	99	21	1,6	cyclictest	0	-21	swapper/3	22:56:40	3
3538	99	21	1,19	cyclictest	20724	-21	ntpq	23:00:20	3
3535	99	21	0,21	cyclictest	0	-21	swapper/2	22:08:16	2
3532	99	21	1,12	cyclictest	0	-21	swapper/1	20:25:14	1
3532	99	21	0,19	cyclictest	0	-21	swapper/1	22:48:11	1
3528	99	21	7,7	cyclictest	27381	-21	readlink	20:00:00	0
3538	99	20	0,19	cyclictest	0	-21	swapper/3	20:15:11	3
3535	99	20	0,1	cyclictest	16907	-21	ntp_states	22:00:21	2
3535	99	19	18,1	cyclictest	0	-21	swapper/2	21:27:08	2
3535	99	19	0,16	cyclictest	0	-21	swapper/2	23:32:24	2
3532	99	19	0,1	cyclictest	0	-21	swapper/1	22:05:17	1
3532	99	19	0,19	cyclictest	0	-21	swapper/1	00:25:45	1
3868	2	18	0,0	sleep3	0	-21	swapper/3	23:48:37	3
3538	99	18	17,1	cyclictest	0	-21	swapper/3	00:14:28	3
3535	99	18	0,1	cyclictest	0	-21	swapper/2	19:55:13	2
3535	99	18	0,18	cyclictest	0	-21	swapper/2	21:44:35	2
3535	99	18	0,16	cyclictest	0	-21	swapper/2	23:41:38	2
3538	99	17	0,17	cyclictest	6866	-21	kworker/3:0	00:30:53	3
3538	99	17	0,0	cyclictest	0	-21	swapper/3	23:18:12	3
3535	99	17	15,2	cyclictest	0	-21	swapper/2	23:53:49	2
3535	99	17	0,16	cyclictest	0	-21	swapper/2	22:45:15	2
3532	99	17	16,1	cyclictest	17680	-21	python	00:15:21	1
3532	99	17	16,1	cyclictest	0	-21	swapper/1	23:54:45	1
3532	99	17	0,16	cyclictest	0	-21	swapper/1	23:41:22	1
3528	99	17	3,8	cyclictest	29714	-21	awk	23:45:00	0
3538	99	16	1,15	cyclictest	22745	-21	kworker/3:1	23:08:19	3
3538	99	16	0,1	cyclictest	0	-21	swapper/3	21:40:22	3
3538	99	16	0,16	cyclictest	0	-21	swapper/3	00:25:03	3
3535	99	16	13,2	cyclictest	14213	-21	munin-run	20:40:00	2
3532	99	16	16,0	cyclictest	0	-21	swapper/1	21:32:05	1
3528	99	16	6,9	cyclictest	19641	-21	apache2	22:41:34	0
3528	99	16	0,1	cyclictest	0	-21	swapper/0	23:12:57	0
3538	99	15	5,7	cyclictest	28551	-21	diskmemload	00:17:37	3
3538	99	15	0,1	cyclictest	0	-21	swapper/3	23:34:33	3
3535	99	15	2,3	cyclictest	14	1	rcu_preempt	23:25:42	2
3535	99	15	0,15	cyclictest	0	-21	swapper/2	23:00:54	2
3528	99	15	0,2	cyclictest	0	-21	swapper/0	20:25:14	0
3528	99	15	0,15	cyclictest	0	-21	swapper/0	21:27:00	0
3538	99	14	5,6	cyclictest	1235	-21	ntpd	00:37:41	3
3538	99	14	3,1	cyclictest	0	-21	swapper/3	22:49:46	3
3538	99	14	12,1	cyclictest	29579	-21	kworker/3:0	21:10:43	3
3538	99	14	10,3	cyclictest	8536	-21	sh	21:35:44	3
3538	99	14	10,3	cyclictest	29682	-21	cron	23:44:59	3
3538	99	14	10,3	cyclictest	11056	-21	cut	19:25:15	3
3538	99	14	10,3	cyclictest	10652	-21	latency_hist	19:25:00	3
3538	99	14	0,4	cyclictest	0	-21	swapper/3	22:17:33	3
3535	99	14	9,4	cyclictest	9587	-21	wc	22:15:19	2
3535	99	14	0,13	cyclictest	0	-21	swapper/2	22:26:18	2
3535	99	14	0,13	cyclictest	0	-21	swapper/2	21:46:34	2
3532	99	14	0,1	cyclictest	28551	-21	diskmemload	21:45:54	1
3528	99	14	2,5	cyclictest	28321	-21	kworker/u8:3	22:34:14	0
3528	99	14	2,5	cyclictest	28321	-21	kworker/u8:3	22:34:14	0
3528	99	14	13,1	cyclictest	0	-21	swapper/0	23:08:21	0
3528	99	14	0,0	cyclictest	0	-21	swapper/0	22:06:23	0
3538	99	13	9,2	cyclictest	23357	-21	grep	19:50:15	3
3538	99	13	9,2	cyclictest	21990	-21	wc	23:20:22	3
3538	99	13	8,4	cyclictest	19080	-21	head	22:40:16	3
3538	99	13	4,6	cyclictest	0	-21	swapper/3	00:20:24	3
3538	99	13	3,7	cyclictest	0	-21	swapper/3	22:20:19	3
3538	99	13	12,1	cyclictest	39	-21	ksoftirqd/3	23:27:27	3
3538	99	13	10,2	cyclictest	4593	-21	cut	23:10:16	3
3538	99	13	10,2	cyclictest	32654	-21	wc	00:05:25	3
3538	99	13	10,2	cyclictest	27978	-21	wc	21:05:25	3
3538	99	13	10,2	cyclictest	14023	-21	cron	23:55:00	3
3538	99	13	10,2	cyclictest	1088	-21	cat	22:10:20	3
3535	99	13	8,3	cyclictest	0	-21	swapper/2	23:47:55	2
3535	99	13	7,4	cyclictest	32	-21	ksoftirqd/2	20:05:15	2
3535	99	13	12,1	cyclictest	0	-21	swapper/2	21:19:12	2
3535	99	13	1,1	cyclictest	1904	-21	diskstats	22:30:16	2
3535	99	13	1,1	cyclictest	1904	-21	diskstats	22:30:16	2
3535	99	13	10,2	cyclictest	9503	-21	cat	00:10:29	2
3535	99	13	10,2	cyclictest	849	-21	cat	00:05:32	2
3535	99	13	10,2	cyclictest	8343	-21	latency_hist	00:30:00	2
3535	99	13	10,2	cyclictest	4502	-21	cat	22:50:28	2
3535	99	13	10,2	cyclictest	20295	-21	grep	22:40:27	2
3535	99	13	10,2	cyclictest	1599	-21	cat	20:10:25	2
3535	99	13	0,1	cyclictest	0	-21	swapper/2	22:12:23	2
3535	99	13	0,13	cyclictest	0	-21	swapper/2	21:22:34	2
3535	99	13	0,13	cyclictest	0	-21	swapper/2	00:37:12	2
3532	99	13	10,2	cyclictest	30710	-21	ssh	21:10:26	1
3532	99	13	10,2	cyclictest	26720	-21	tr	21:05:14	1
3532	99	13	10,2	cyclictest	25488	-21	tr	19:55:14	1
3532	99	13	10,2	cyclictest	22311	-21	grep	20:55:17	1
3532	99	13	10,2	cyclictest	12909	-21	grep	23:15:18	1
3532	99	13	10,2	cyclictest	12111	-21	grep	19:25:22	1
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17