OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack7slot7s.osadl.org (updated Sun Jan 04, 2026 12:43:47)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
7272	2	94	69,9	sleep2	0	-21	swapper/2	07:07:31	2
7282	2	87	74,9	sleep0	0	-21	swapper/0	07:07:37	0
7442	2	84	68,12	sleep3	0	-21	swapper/3	07:09:42	3
7329	2	78	68,6	sleep1	0	-21	swapper/1	07:08:15	1
7562	99	23	11,5	cyclictest	225	-21	systemd-journal	08:54:24	3
7562	99	20	0,1	cyclictest	4935	-21	kworker/3:2	09:09:03	3
7562	99	19	0,1	cyclictest	4935	-21	kworker/3:2	09:11:23	3
7562	99	19	0,1	cyclictest	0	-21	swapper/3	11:15:14	3
7562	99	19	0,18	cyclictest	0	-21	swapper/3	12:05:14	3
7562	99	18	0,1	cyclictest	0	-21	swapper/3	11:35:14	3
7562	99	18	0,1	cyclictest	0	-21	swapper/3	11:35:13	3
7557	99	18	16,1	cyclictest	0	-21	swapper/1	09:50:27	1
7557	99	17	17,0	cyclictest	0	-21	swapper/1	10:00:15	1
7555	99	17	5,6	cyclictest	3432	-21	awk	11:30:00	0
7557	99	16	10,5	cyclictest	20477	-21	grep	07:35:18	1
7555	99	15	2,9	cyclictest	27762	-21	fschecks_count	10:05:16	0
7555	99	15	0,4	cyclictest	0	-21	swapper/0	09:40:15	0
7555	99	15	0,4	cyclictest	0	-21	swapper/0	09:40:14	0
7562	99	14	10,3	cyclictest	26089	-21	cron	11:10:01	3
7562	99	14	10,3	cyclictest	10945	-21	ls	11:45:01	3
7562	99	14	10,2	cyclictest	25245	-21	tail	07:45:17	3
7562	99	14	0,5	cyclictest	0	-21	swapper/3	09:55:14	3
7562	99	14	0,12	cyclictest	18264	-21	cut	09:45:18	3
7560	99	14	10,3	cyclictest	5513	-21	cat	10:25:20	2
7560	99	14	10,3	cyclictest	31347	-21	grep	12:25:17	2
7560	99	14	10,3	cyclictest	11289	-21	grep	11:45:14	2
7557	99	14	10,3	cyclictest	15197	-21	ls	11:50:27	1
7555	99	14	4,8	cyclictest	17366	-21	apache_accesses	07:30:10	0
7555	99	14	3,6	cyclictest	8672	-21	apt-get	11:40:14	0
7555	99	14	10,3	cyclictest	31033	-21	gzip	10:10:25	0
7555	99	14	10,2	cyclictest	32349	-21	ls	11:20:23	0
7555	99	14	10,2	cyclictest	24087	-21	grep	11:05:15	0
7562	99	13	9,3	cyclictest	22180	-21	cut	11:00:20	3
7562	99	13	9,3	cyclictest	18723	-21	cron	10:55:00	3
7562	99	13	8,3	cyclictest	29978	-21	grep	07:55:21	3
7562	99	13	10,2	cyclictest	9076	-21	wc	09:25:22	3
7562	99	13	10,2	cyclictest	3129	-21	wc	08:05:27	3
7562	99	13	10,2	cyclictest	24835	-21	cut	10:00:15	3
7562	99	13	10,2	cyclictest	16147	-21	dump-pmu-power	10:49:59	3
7562	99	13	10,2	cyclictest	13019	-21	grep	08:25:27	3
7560	99	13	10,2	cyclictest	799	-21	cron	11:25:01	2
7560	99	13	10,2	cyclictest	7294	-21	wc	11:35:21	2
7560	99	13	10,2	cyclictest	7294	-21	wc	11:35:21	2
7560	99	13	10,2	cyclictest	30657	-21	uname	10:10:21	2
7560	99	13	10,2	cyclictest	10657	-21	wc	10:35:27	2
7557	99	13	9,1	cyclictest	4532	-21	grep	12:35:28	1
7557	99	13	10,2	cyclictest	26438	-21	head	08:55:17	1
7557	99	13	10,2	cyclictest	2607	-21	cat	08:05:20	1
7557	99	13	10,2	cyclictest	23327	-21	wc	09:55:21	1
7557	99	13	10,2	cyclictest	1795	-21	turbostat	10:19:59	1
7557	99	13	10,2	cyclictest	1773	-21	sendmail_mailtr	09:10:23	1
7557	99	13	10,2	cyclictest	17276	-21	ls	11:55:26	1
7557	99	13	10,2	cyclictest	16252	-21	wc	07:25:22	1
7555	99	13	2,3	cyclictest	225	-21	systemd-journal	07:15:00	0
7555	99	13	10,2	cyclictest	7990	-21	grep	11:35:29	0
7555	99	13	10,2	cyclictest	7990	-21	grep	11:35:28	0
7555	99	13	10,2	cyclictest	3407	-21	wc	10:20:24	0
7555	99	13	10,2	cyclictest	21129	-21	sleep	11:00:00	0
7555	99	13	10,2	cyclictest	13725	-21	dump-pmu-power	10:45:00	0
7555	99	13	0,1	cyclictest	0	-21	swapper/0	10:15:15	0
7555	99	13	0,11	cyclictest	11966	-21	grep	11:45:18	0
7562	99	12	9,2	cyclictest	7776	-21	sh	09:25:01	3
7562	99	12	9,2	cyclictest	5693	-21	tr	10:25:22	3
7562	99	12	9,2	cyclictest	3386	-21	grep	12:35:16	3
7562	99	12	9,2	cyclictest	32197	-21	cut	10:15:14	3
7562	99	12	9,2	cyclictest	31361	-21	grep	11:20:14	3
7562	99	12	9,2	cyclictest	30832	-21	grep	10:10:22	3
7562	99	12	9,2	cyclictest	26346	-21	grep	12:15:20	3
7562	99	12	9,2	cyclictest	25068	-21	grep	12:10:25	3
7562	99	12	9,2	cyclictest	22171	-21	tr	08:45:23	3
7562	99	12	9,2	cyclictest	20555	-21	cron	12:05:00	3
7562	99	12	9,2	cyclictest	13886	-21	tr	11:50:16	3
7562	99	12	9,2	cyclictest	1308	-21	head	12:30:20	3
7562	99	12	8,3	cyclictest	5990	-21	cron	11:35:01	3
7562	99	12	8,3	cyclictest	5390	-21	apt	09:20:00	3
7562	99	12	8,2	cyclictest	15720	-21	tail	07:25:17	3
7562	99	12	8,2	cyclictest	11865	-21	tr	10:40:15	3
7562	99	12	7,3	cyclictest	9613	-21	cut	08:20:17	3
7562	99	12	4,4	cyclictest	225	-21	systemd-journal	09:04:59	3
7562	99	12	4,2	cyclictest	225	-21	systemd-journal	07:44:00	3
7560	99	12	9,2	cyclictest	8335	-21	grep	09:25:16	2
7560	99	12	9,2	cyclictest	4145	-21	cut	11:30:16	2
7560	99	12	9,2	cyclictest	32238	-21	grep	10:15:15	2
7560	99	12	9,2	cyclictest	30888	-21	grep	11:20:01	2
7560	99	12	9,2	cyclictest	28857	-21	head	10:05:27	2
7560	99	12	9,2	cyclictest	27073	-21	head	11:10:17	2
7560	99	12	9,2	cyclictest	25073	-21	grep	11:05:21	2
7560	99	12	9,2	cyclictest	20682	-21	uname	07:35:20	2
7560	99	12	9,2	cyclictest	15870	-21	sort	09:40:17	2
7560	99	12	9,2	cyclictest	15870	-21	sort	09:40:17	2
7560	99	12	9,2	cyclictest	14800	-21	wc	07:25:01	2
7560	99	12	9,2	cyclictest	13521	-21	grep	09:35:19	2
7560	99	12	9,2	cyclictest	1112	-21	grep	12:30:19	2
7560	99	12	8,3	cyclictest	3496	-21	sort	12:35:17	2
7560	99	12	8,3	cyclictest	25970	-21	tr	08:55:15	2
7560	99	12	8,3	cyclictest	10969	-21	cat	11:45:01	2
7557	99	12	9,2	cyclictest	6906	-21	tr	10:30:15	1
7557	99	12	9,2	cyclictest	6365	-21	grep	09:20:18	1
7557	99	12	9,2	cyclictest	2863	-21	head	10:20:18	1
7557	99	12	9,2	cyclictest	2830	-21	chrt	11:25:26	1
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17