OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack7slot7s.osadl.org (updated Thu Dec 18, 2025 12:43:47)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
21768	2	91	77,9	sleep0	0	-21	swapper/0	07:05:17	0
23022	2	88	76,8	sleep2	0	-21	swapper/2	07:06:43	2
23074	2	78	58,16	sleep1	0	-21	swapper/1	07:07:23	1
22998	2	77	64,9	sleep3	0	-21	swapper/3	07:06:23	3
23362	99	33	32,1	cyclictest	0	-21	swapper/1	08:00:28	1
23369	99	22	4,17	cyclictest	26116	-21	apt-get	11:45:14	3
23362	99	22	0,20	cyclictest	0	-21	swapper/1	08:05:12	1
23358	99	22	5,9	cyclictest	0	-21	swapper/0	07:20:14	0
23367	99	21	0,0	cyclictest	0	-21	swapper/2	07:43:40	2
23362	99	21	1,2	cyclictest	629	-21	cron	07:30:00	1
23367	99	20	0,20	cyclictest	0	-21	swapper/2	08:28:29	2
23362	99	20	0,19	cyclictest	0	-21	swapper/1	10:30:15	1
23358	99	20	5,8	cyclictest	20853	-21	munin-run	09:20:00	0
23358	99	20	0,18	cyclictest	0	-21	swapper/0	07:25:14	0
23367	99	19	0,1	cyclictest	0	-21	swapper/2	07:31:27	2
23367	99	19	0,18	cyclictest	0	-21	swapper/2	11:05:15	2
23369	99	18	0,15	cyclictest	0	-21	swapper/3	07:20:13	3
23362	99	18	0,12	cyclictest	0	-21	swapper/1	09:50:13	1
23369	99	17	17,0	cyclictest	0	-21	swapper/3	10:30:15	3
23358	99	17	0,14	cyclictest	0	-21	swapper/0	11:45:14	0
23362	99	16	9,6	cyclictest	31235	-21	wc	10:45:22	1
23367	99	15	0,9	cyclictest	0	-21	swapper/2	11:10:14	2
23362	99	15	8,5	cyclictest	475	-21	in:imuxsock	07:10:25	1
23362	99	15	0,14	cyclictest	0	-21	swapper/1	11:35:14	1
23358	99	15	0,8	cyclictest	28124	-21	sendmail	07:20:00	0
23369	99	14	10,3	cyclictest	11095	-21	sed	11:10:29	3
23367	99	14	10,3	cyclictest	27102	-21	idleruntime-cro	08:25:00	2
23367	99	14	10,2	cyclictest	31945	-21	cat	07:25:19	2
23367	99	14	10,2	cyclictest	26817	-21	grep	11:45:17	2
23367	99	14	10,2	cyclictest	1122	-21	wc	10:50:23	2
23367	99	14	10,2	cyclictest	1122	-21	wc	10:50:22	2
23367	99	14	0,14	cyclictest	0	-21	swapper/2	12:36:36	2
23362	99	14	9,3	cyclictest	15117	-21	cron	07:59:59	1
23362	99	14	10,3	cyclictest	20150	-21	ls	09:15:26	1
23362	99	14	10,2	cyclictest	32404	-21	grep	09:40:26	1
23362	99	14	10,2	cyclictest	28141	-21	grep	10:40:18	1
23362	99	14	10,2	cyclictest	2279	-21	cut	10:55:12	1
23362	99	14	0,12	cyclictest	10450	-21	wc	11:10:21	1
23358	99	14	6,7	cyclictest	4210	-21	munin-run	08:45:01	0
23358	99	14	2,9	cyclictest	16121	-21	latency_hist	09:10:01	0
23358	99	14	10,2	cyclictest	13866	-21	cron	11:20:01	0
4937	2	13	0,1	sleep1	0	-21	swapper/1	08:45:15	1
23369	99	13	9,3	cyclictest	28141	-21	grep	08:25:21	3
23369	99	13	9,3	cyclictest	27275	-21	awk	10:40:01	3
23369	99	13	9,3	cyclictest	1153	-21	tr	07:30:16	3
23369	99	13	3,9	cyclictest	22700	-21	apache2	07:55:24	3
23369	99	13	10,2	cyclictest	8707	-21	cut	07:45:18	3
23369	99	13	10,2	cyclictest	6503	-21	grep	12:10:18	3
23369	99	13	10,2	cyclictest	5362	-21	sleep	09:54:59	3
23369	99	13	10,2	cyclictest	32529	-21	sed	11:55:25	3
23369	99	13	10,2	cyclictest	17413	-21	sh	12:35:01	3
23369	99	13	10,2	cyclictest	12995	-21	head	11:15:22	3
23369	99	13	0,12	cyclictest	20871	-21	tr	09:20:01	3
23369	99	13	0,12	cyclictest	1805	-21	sh	08:40:01	3
23367	99	13	4,7	cyclictest	0	-21	swapper/2	09:45:17	2
23367	99	13	10,2	cyclictest	6505	-21	tail	12:10:18	2
23367	99	13	10,2	cyclictest	6242	-21	grep	11:00:28	2
23367	99	13	10,2	cyclictest	4140	-21	grep	12:05:18	2
23367	99	13	10,2	cyclictest	384	-21	grep	09:44:59	2
23367	99	13	10,2	cyclictest	23186	-21	grep	08:15:16	2
23367	99	13	10,2	cyclictest	19880	-21	cron	08:09:59	2
23367	99	13	10,2	cyclictest	18639	-21	sh	11:30:00	2
23367	99	13	10,2	cyclictest	16497	-21	grep	09:10:16	2
23367	99	13	10,2	cyclictest	15692	-21	grep	08:00:13	2
23367	99	13	10,2	cyclictest	14933	-21	head	11:20:17	2
23367	99	13	10,2	cyclictest	13741	-21	grep	07:55:18	2
23367	99	13	10,2	cyclictest	12738	-21	cron	07:54:59	2
23367	99	13	10,2	cyclictest	11358	-21	taskset	10:05:18	2
23362	99	13	9,3	cyclictest	2198	-21	cut	08:40:16	1
23362	99	13	9,3	cyclictest	20058	-21	cron	10:25:00	1
23362	99	13	9,2	cyclictest	614	-21	uname	08:35:20	1
23362	99	13	9,2	cyclictest	15600	-21	grep	12:30:13	1
23362	99	13	9,2	cyclictest	13665	-21	tail	12:25:16	1
23362	99	13	8,4	cyclictest	7877	-21	sed	08:50:21	1
23362	99	13	10,2	cyclictest	16810	-21	tr	11:25:16	1
23362	99	13	10,2	cyclictest	12838	-21	tr	09:00:22	1
23362	99	13	0,12	cyclictest	21255	-21	grep	08:10:21	1
23358	99	13	9,3	cyclictest	4527	-21	tr	07:35:22	0
23358	99	13	2,5	cyclictest	225	-21	systemd-journal	09:05:00	0
23358	99	13	12,1	cyclictest	0	-21	swapper/0	10:13:27	0
23358	99	13	10,2	cyclictest	6509	-21	grep	12:10:18	0
23358	99	13	10,2	cyclictest	5034	-21	grep	12:05:26	0
23358	99	13	10,2	cyclictest	31564	-21	tail	11:55:17	0
23369	99	12	9,2	cyclictest	7618	-21	head	08:50:18	3
23369	99	12	9,2	cyclictest	6408	-21	head	09:55:17	3
23369	99	12	9,2	cyclictest	6397	-21	grep	07:40:17	3
23369	99	12	9,2	cyclictest	5523	-21	sh	12:10:01	3
23369	99	12	9,2	cyclictest	28152	-21	cat	09:35:00	3
23369	99	12	9,2	cyclictest	25695	-21	grep	08:20:15	3
23369	99	12	9,2	cyclictest	24796	-21	grep	09:25:22	3
23369	99	12	9,2	cyclictest	22086	-21	cat	09:20:19	3
23369	99	12	9,2	cyclictest	19834	-21	sendmail-msp	12:40:00	3
23369	99	12	9,2	cyclictest	19351	-21	wc	08:05:24	3
23369	99	12	9,2	cyclictest	17508	-21	cron	08:04:59	3
23369	99	12	9,2	cyclictest	16414	-21	wc	11:25:00	3
23369	99	12	9,2	cyclictest	1569	-21	head	12:00:17	3
23369	99	12	9,2	cyclictest	15103	-21	tr	09:05:19	3
23369	99	12	9,2	cyclictest	14651	-21	grep	10:10:28	3
23369	99	12	9,2	cyclictest	13301	-21	grep	09:00:27	3
23369	99	12	9,2	cyclictest	12662	-21	cron	12:25:00	3
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17