OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack5slot4s.osadl.org (updated Fri Jul 17, 2026 00:44:14)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
1247292	2	105	0,3	chrt	584	-21	kworker/3:2+events	21:37:59	3
1096371	99	66	55,8	cyclictest	1257690	-21	latency_hist	21:50:00	0
1096371	99	57	49,6	cyclictest	1187617	-21	cp	20:40:01	0
1096372	99	55	48,5	cyclictest	1072020	-21	kworker/u8:1+flush-8:0	19:27:42	1
1096372	99	55	48,5	cyclictest	1072020	-21	kworker/u8:1+flush-8:0	19:27:41	1
1096371	99	55	45,7	cyclictest	1377359	-21	latency_hist	23:50:01	0
1096371	99	54	46,5	cyclictest	1108095	-21	latency_hist	19:20:00	0
1096371	99	53	46,5	cyclictest	1273385	-21	/usr/sbin/munin	22:07:42	0
1398543	2	52	0,4	chrt	0	-21	swapper/3	00:12:35	3
1096371	99	51	40,8	cyclictest	1182566	-21	latency_hist	20:35:00	0
1096371	99	50	42,6	cyclictest	1147935	-21	latency_hist	20:00:02	0
1096372	99	49	43,4	cyclictest	1327875	-21	kworker/u8:2+flush-8:0	23:17:32	1
1096372	99	49	41,6	cyclictest	1287629	-21	kworker/u8:1+flush-8:0	22:37:43	1
1096372	99	47	42,4	cyclictest	1072020	-21	kworker/u8:1+flush-8:0	19:37:42	1
1096372	99	46	42,3	cyclictest	1172191	-21	kworker/u8:3+flush-8:0	20:47:27	1
1096372	99	45	41,3	cyclictest	733430	-21	kworker/u8:0+flush-8:0	19:22:28	1
1096372	99	45	39,5	cyclictest	1172191	-21	kworker/u8:3+flush-8:0	20:42:28	1
1096377	99	44	35,6	cyclictest	281	-21	systemd-journal	00:30:01	2
1096372	99	44	39,4	cyclictest	1072020	-21	kworker/u8:1+flush-8:0	19:42:27	1
1096372	99	44	37,5	cyclictest	1218324	-21	apt-get	21:12:24	1
1096372	99	43	38,4	cyclictest	733430	-21	kworker/u8:0+flush-8:0	19:47:40	1
1096371	99	43	37,4	cyclictest	733430	-21	kworker/u8:0+flush-8:0	19:37:32	0
1096372	99	42	37,4	cyclictest	1072020	-21	kworker/u8:1+events_unbound	19:15:02	1
1096372	99	41	36,4	cyclictest	733430	-21	kworker/u8:0+flush-8:0	20:37:29	1
1096372	99	41	35,4	cyclictest	1366690	-21	kworker/u8:1+flush-8:0	00:22:43	1
1096372	99	41	35,4	cyclictest	1352812	-21	apt-get	23:27:21	1
1096372	99	41	30,8	cyclictest	281	-21	systemd-journal	00:35:01	1
1096371	99	41	37,3	cyclictest	1217913	-21	kworker/u8:2+flush-8:0	21:32:39	0
1096371	99	41	35,4	cyclictest	1361021	-21	kworker/u8:2+flush-8:0	00:27:28	0
1096377	99	40	34,4	cyclictest	1322489	-21	kworker/u8:0+events_unbound	23:57:41	2
1096377	99	40	34,4	cyclictest	1322489	-21	kworker/u8:0+events_unbound	23:57:41	2
1096372	99	40	35,4	cyclictest	733430	-21	kworker/u8:0+flush-8:0	20:22:45	1
1096371	99	40	34,5	cyclictest	1208228	-21	kworker/u8:0+events_unbound	21:22:46	0
1096371	99	40	30,8	cyclictest	1122229	-21	ntpq	19:32:36	0
1096372	99	39	35,3	cyclictest	1192946	-21	kworker/u8:1+flush-8:0	20:52:41	1
1096372	99	39	34,4	cyclictest	1118373	-21	kworker/u8:3+flush-8:0	19:52:29	1
1096371	99	39	31,6	cyclictest	1198070	-21	/usr/sbin/munin	20:52:35	0
1096371	99	39	31,5	cyclictest	281	-21	systemd-journal	23:40:00	0
1096377	99	38	33,4	cyclictest	1208228	-21	kworker/u8:0+flush-8:0	22:00:01	2
1096372	99	38	33,4	cyclictest	1118373	-21	kworker/u8:3+flush-8:0	19:57:27	1
1096372	99	38	32,4	cyclictest	1387215	-21	apt-get	00:00:00	1
1096372	99	38	32,4	cyclictest	1387215	-21	apt-get	00:00:00	1
1096372	99	38	32,4	cyclictest	1258120	-21	apt-get	21:52:24	1
1096372	99	38	30,5	cyclictest	281	-21	systemd-journal	23:10:40	1
1096371	99	38	31,5	cyclictest	1342292	-21	kworker/u8:1+events_unbound	23:22:27	0
1096372	99	37	30,5	cyclictest	1203240	-21	apt-get	20:57:23	1
1096372	99	37	29,6	cyclictest	1172191	-21	kworker/u8:3+events_unbound	21:02:26	1
1096371	99	37	32,4	cyclictest	1138023	-21	kworker/u8:2+flush-8:0	20:02:45	0
1096371	99	37	32,3	cyclictest	0	-21	swapper/0	00:00:00	0
1096371	99	37	32,3	cyclictest	0	-21	swapper/0	00:00:00	0
1096377	99	36	32,3	cyclictest	1217913	-21	kworker/u8:2+flush-8:0	21:27:43	2
1096372	99	36	32,3	cyclictest	1152909	-21	kworker/u8:1+flush-8:0	20:27:41	1
1096372	99	36	30,4	cyclictest	1361021	-21	kworker/u8:2+events_unbound	00:27:28	1
1096372	99	36	25,9	cyclictest	281	-21	systemd-journal	22:55:00	1
1096371	99	36	29,5	cyclictest	733430	-21	kworker/u8:0+flush-8:0	20:22:46	0
1096371	99	36	28,6	cyclictest	1357344	-21	apt-get	23:30:00	0
1096371	99	36	25,8	cyclictest	1262601	-21	latency_hist	21:55:01	0
1096377	99	35	24,8	cyclictest	1133322	-21	cp	19:45:01	2
1096372	99	35	31,3	cyclictest	733430	-21	kworker/u8:0+flush-8:0	19:32:30	1
1096372	99	35	28,5	cyclictest	1322489	-21	kworker/u8:0+events_unbound	00:02:42	1
1096372	99	35	28,5	cyclictest	1213361	-21	apt-get	21:07:22	1
1096371	99	35	28,5	cyclictest	1361021	-21	kworker/u8:2+events_unbound	00:37:26	0
1096371	99	35	24,8	cyclictest	1137257	-21	ntpq	19:47:35	0
1096372	99	34	29,4	cyclictest	1152909	-21	kworker/u8:1+events_unbound	20:12:27	1
1096372	99	34	29,3	cyclictest	1152909	-21	kworker/u8:1+events_unbound	20:20:00	1
1096372	99	34	27,5	cyclictest	1208228	-21	kworker/u8:0+events_unbound	22:17:42	1
1096371	99	34	30,3	cyclictest	1297757	-21	kworker/u8:3+events_unbound	23:02:42	0
1091637	2	34	4,9	sleep1	0	-21	swapper/1	19:06:32	1
1096377	99	33	23,7	cyclictest	1242885	-21	cat	21:35:00	2
1096372	99	33	29,3	cyclictest	1287629	-21	kworker/u8:1+events_unbound	22:42:23	1
1096372	99	33	29,3	cyclictest	1267301	-21	kworker/u8:2+flush-8:0	22:27:42	1
1096372	99	33	26,5	cyclictest	1332634	-21	latency_hist	23:05:01	1
1096372	99	33	26,5	cyclictest	1313253	-21	apt-get	22:47:24	1
1096372	99	33	26,5	cyclictest	1287629	-21	kworker/u8:1+events_unbound	22:22:27	1
1096372	99	33	25,5	cyclictest	281	-21	systemd-journal	23:49:41	1
1096371	99	33	28,4	cyclictest	1152909	-21	kworker/u8:1+flush-8:0	20:12:29	0
1096371	99	33	27,4	cyclictest	1361021	-21	kworker/u8:2+flush-8:0	00:17:32	0
1096371	99	33	27,4	cyclictest	1172191	-21	kworker/u8:3+events_unbound	21:05:47	0
1096372	99	32	26,5	cyclictest	1342292	-21	kworker/u8:1+events_unbound	23:30:02	1
1096372	99	32	23,7	cyclictest	1366670	-21	ntpq	23:37:38	1
1096372	99	32	22,1	cyclictest	16	1	rcu_preempt	22:05:02	1
1096371	99	32	27,4	cyclictest	1192946	-21	kworker/u8:1+flush-8:0	20:47:40	0
1096371	99	32	26,5	cyclictest	1366690	-21	kworker/u8:1+events_unbound	00:07:49	0
1096371	99	32	1,27	cyclictest	1222846	-21	true	21:15:00	0
1096371	99	32	1,27	cyclictest	1	-21	systemd	00:35:01	0
1096377	99	31	26,4	cyclictest	1207752	-21	sendmail	21:00:00	2
1096372	99	31	21,7	cyclictest	1337539	-21	latency_hist	23:10:01	1
1096371	99	31	26,4	cyclictest	733430	-21	kworker/u8:0+events_unbound	19:12:25	0
1096371	99	31	24,5	cyclictest	281	-21	systemd-journal	19:29:20	0
1096371	99	31	24,5	cyclictest	281	-21	systemd-journal	19:29:20	0
1096371	99	31	23,6	cyclictest	1293421	-21	/usr/sbin/munin	22:27:35	0
1096371	99	31	23,6	cyclictest	1243228	-21	/usr/sbin/munin	21:37:58	0
1096377	99	30	25,4	cyclictest	1192946	-21	kworker/u8:1+flush-8:0	21:00:06	2
1096377	99	30	25,4	cyclictest	1152909	-21	kworker/u8:1+flush-8:0	20:27:37	2
1096377	99	30	21,7	cyclictest	281	-21	systemd-journal	20:07:41	2
1096377	99	30	21,7	cyclictest	1096953	-21	fschecks_count	19:12:26	2
1096372	99	30	26,2	cyclictest	1237657	-21	latency_hist	21:30:00	1
1096372	99	30	25,4	cyclictest	1138023	-21	kworker/u8:2+flush-8:0	20:02:39	1
1096372	99	30	25,3	cyclictest	1097278	-21	irqcore	19:12:29	1
1096372	99	30	24,4	cyclictest	1287629	-21	kworker/u8:1+flush-8:0	22:57:29	1
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17