OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack5slot4s.osadl.org (updated Sat May 16, 2026 12:44:15)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
1979982	2	209	0,4	sleep3	0	-21	swapper/3	08:45:00	3
2057251	2	76	0,3	chrt	75	-21	kworker/3:1+events	10:00:57	3
2057251	2	76	0,3	chrt	75	-21	kworker/3:1+events	10:00:56	3
1883432	99	68	58,7	cyclictest	1929842	-21	latency_hist	07:55:01	1
1883432	99	65	59,5	cyclictest	2154010	-21	kworker/u8:3+events_unbound	11:56:06	1
1883432	99	64	56,6	cyclictest	1850527	-21	kworker/u8:3+flush-8:0	07:36:04	1
1883427	99	64	55,6	cyclictest	2094103	-21	latency_hist	10:40:01	0
1883432	99	62	54,6	cyclictest	2114214	-21	kworker/u8:2+flush-8:0	12:11:02	1
2167617	2	61	0,2	sleep3	48	1	ktimers/3	11:51:01	3
1883432	99	61	56,4	cyclictest	1945186	-21	kworker/u8:3+flush-8:0	08:36:07	1
1883432	99	61	54,5	cyclictest	1945186	-21	kworker/u8:3+events_unbound	09:46:03	1
1883427	99	60	54,4	cyclictest	1850527	-21	kworker/u8:3+flush-8:0	07:11:08	0
1883427	99	60	49,8	cyclictest	280	-21	systemd-journal	08:40:01	0
2164376	2	59	0,1	chrt	2163959	-21	/usr/sbin/munin	11:50:54	1
1883427	99	59	50,7	cyclictest	280	-21	systemd-journal	09:40:00	0
1883427	99	58	48,7	cyclictest	2174119	-21	diskstats	12:00:52	0
1883432	99	56	50,4	cyclictest	2108969	-21	kworker/u8:0+events_unbound	11:36:02	1
1883432	99	56	50,4	cyclictest	2094347	-21	kworker/u8:2+events_unbound	10:51:02	1
1883432	99	56	49,6	cyclictest	2147989	-21	kworker/u8:1+flush-8:0	12:21:04	1
1883432	99	55	51,3	cyclictest	2094347	-21	kworker/u8:2+events_unbound	10:41:07	1
1883432	99	55	50,4	cyclictest	2154010	-21	kworker/u8:3+flush-8:0	12:01:06	1
1883432	99	55	50,4	cyclictest	1945186	-21	kworker/u8:3+flush-8:0	09:16:06	1
1883432	99	55	50,4	cyclictest	1910743	-21	kworker/u8:1+flush-8:0	10:01:06	1
1883432	99	55	50,4	cyclictest	1910743	-21	kworker/u8:1+flush-8:0	10:01:05	1
1883432	99	55	50,4	cyclictest	1900387	-21	kworker/u8:2+flush-8:0	07:26:08	1
1883432	99	55	50,4	cyclictest	1850527	-21	kworker/u8:3+flush-8:0	07:41:07	1
1883435	99	53	48,3	cyclictest	1954861	-21	gpgv	08:20:01	2
1883432	99	53	47,4	cyclictest	1945186	-21	kworker/u8:3+flush-8:0	09:50:55	1
1883432	99	53	45,6	cyclictest	2182612	-21	ntpq	12:06:02	1
1883432	99	53	45,6	cyclictest	2182612	-21	ntpq	12:06:02	1
1883427	99	53	47,4	cyclictest	1900387	-21	kworker/u8:2+flush-8:0	07:26:03	0
1883432	99	52	48,3	cyclictest	1930304	-21	kworker/u8:0+events_unbound	09:10:53	1
1883432	99	52	47,3	cyclictest	1945186	-21	kworker/u8:3+events_unbound	08:41:07	1
1883432	99	52	46,4	cyclictest	1945186	-21	kworker/u8:3+events_unbound	08:26:04	1
1883432	99	52	45,6	cyclictest	2049629	-21	kworker/u8:2+flush-8:0	09:56:05	1
1883432	99	52	43,6	cyclictest	2208408	-21	latency_hist	12:35:01	1
1883432	99	51	47,3	cyclictest	2147989	-21	kworker/u8:1+events_unbound	11:46:08	1
1883432	99	51	45,5	cyclictest	2059063	-21	kworker/u8:0+flush-8:0	10:11:10	1
1883432	99	51	45,5	cyclictest	1910743	-21	kworker/u8:1+flush-8:0	09:06:09	1
1883427	99	51	44,4	cyclictest	1930304	-21	kworker/u8:0+flush-8:0	08:30:55	0
1883432	99	50	45,4	cyclictest	1910743	-21	kworker/u8:1+events_unbound	08:01:04	1
1883427	99	50	44,4	cyclictest	1930304	-21	kworker/u8:0+flush-8:0	08:51:05	0
1883427	99	50	40,7	cyclictest	2004698	-21	latency_hist	09:10:00	0
2104433	2	49	0,3	chrt	2104438	-21	aten2_r5power_v	10:50:52	2
1883432	99	49	44,3	cyclictest	1850527	-21	kworker/u8:3+events_unbound	07:21:02	1
1883432	99	49	43,4	cyclictest	1910743	-21	kworker/u8:1+flush-8:0	08:56:02	1
1883432	99	49	43,4	cyclictest	1910743	-21	kworker/u8:1+flush-8:0	08:16:05	1
1883432	99	49	42,5	cyclictest	2147989	-21	kworker/u8:1+flush-8:0	12:36:02	1
1883432	99	49	42,5	cyclictest	1910743	-21	kworker/u8:1+flush-8:0	09:31:03	1
1883427	99	49	39,7	cyclictest	2074237	-21	latency_hist	10:20:02	0
2112798	2	48	0,4	sleep0	0	-21	swapper/0	10:56:00	0
1883432	99	48	40,6	cyclictest	1930304	-21	kworker/u8:0+flush-8:0	08:11:09	1
1883427	99	48	38,7	cyclictest	1999783	-21	latency_hist	09:05:01	0
1883435	99	47	40,5	cyclictest	280	-21	systemd-journal	08:35:01	2
1883435	99	47	36,8	cyclictest	280	-21	systemd-journal	11:59:34	2
1883432	99	47	43,3	cyclictest	2147989	-21	kworker/u8:1+events_unbound	12:16:08	1
1883427	99	47	43,3	cyclictest	1910743	-21	kworker/u8:1+flush-8:0	07:51:08	0
1883432	99	46	42,3	cyclictest	2049629	-21	kworker/u8:2+flush-8:0	10:06:05	1
1883432	99	46	41,4	cyclictest	1930304	-21	kworker/u8:0+flush-8:0	09:40:55	1
1883432	99	46	37,7	cyclictest	1984184	-21	ntpq	08:46:05	1
1883432	99	45	40,4	cyclictest	1850527	-21	kworker/u8:3+events_unbound	07:30:52	1
1883432	99	45	40,4	cyclictest	1850527	-21	kworker/u8:3+events_unbound	07:30:51	1
1883432	99	45	40,3	cyclictest	2147989	-21	kworker/u8:1+events_unbound	12:25:52	1
1883432	99	45	40,3	cyclictest	1910743	-21	kworker/u8:1+events_unbound	09:25:52	1
1883432	99	45	40,3	cyclictest	1850527	-21	kworker/u8:3+flush-8:0	07:10:55	1
1883432	99	45	39,5	cyclictest	2108969	-21	kworker/u8:0+flush-8:0	11:21:02	1
1883432	99	45	38,5	cyclictest	2108969	-21	kworker/u8:0+flush-8:0	11:11:03	1
1883432	99	45	38,5	cyclictest	1930304	-21	kworker/u8:0+events_unbound	08:50:52	1
1883427	99	45	38,5	cyclictest	1930304	-21	kworker/u8:0+flush-8:0	09:15:55	0
2183716	2	44	33,6	sleep3	0	-21	swapper/3	12:10:35	3
1883432	99	44	38,5	cyclictest	1910743	-21	kworker/u8:1+flush-8:0	09:01:03	1
1883432	99	44	38,4	cyclictest	1930304	-21	kworker/u8:0+events_unbound	08:06:04	1
1883432	99	44	38,4	cyclictest	1850527	-21	kworker/u8:3+events_unbound	07:15:04	1
1883432	99	44	37,5	cyclictest	1930390	-21	apt-get	07:55:50	1
1883432	99	44	36,5	cyclictest	2133747	-21	latency_hist	11:20:01	1
1883427	99	44	35,6	cyclictest	1919972	-21	latency_hist	07:45:01	0
1883432	99	43	37,5	cyclictest	2108969	-21	kworker/u8:0+flush-8:0	11:01:04	1
1883432	99	43	37,4	cyclictest	1930304	-21	kworker/u8:0+events_unbound	09:35:53	1
1883432	99	43	36,5	cyclictest	1930304	-21	kworker/u8:0+flush-8:0	08:31:03	1
1883427	99	43	32,8	cyclictest	2124452	-21	diskstats	11:10:53	0
1883432	99	42	36,4	cyclictest	1930304	-21	kworker/u8:0+flush-8:0	09:20:58	1
1883427	99	42	37,4	cyclictest	1930304	-21	kworker/u8:0+flush-8:0	09:11:08	0
1883427	99	42	36,4	cyclictest	1850527	-21	kworker/u8:3+flush-8:0	07:21:08	0
1883432	99	41	34,5	cyclictest	1910743	-21	kworker/u8:1+flush-8:0	10:26:04	1
1883427	99	41	37,3	cyclictest	1850527	-21	kworker/u8:3+flush-8:0	07:46:05	0
1883432	99	40	35,4	cyclictest	2049629	-21	kworker/u8:2+flush-8:0	10:15:56	1
1883432	99	40	35,3	cyclictest	1900387	-21	kworker/u8:2+events_unbound	07:45:50	1
1883432	99	40	34,5	cyclictest	2084444	-21	kworker/u8:4+events_unbound	10:56:06	1
1883427	99	40	35,3	cyclictest	2154010	-21	kworker/u8:3+flush-8:0	11:50:53	0
1883427	99	40	33,5	cyclictest	280	-21	systemd-journal	12:31:42	0
1883427	99	40	32,6	cyclictest	2078317	-21	ntpq	10:21:01	0
1883427	99	40	31,6	cyclictest	2029381	-21	latency_hist	09:35:01	0
1883432	99	39	34,3	cyclictest	2114214	-21	kworker/u8:2+flush-8:0	11:05:57	1
1883432	99	39	32,4	cyclictest	1910743	-21	kworker/u8:1+flush-8:0	10:21:06	1
1883427	99	39	29,8	cyclictest	2018625	-21	ntpq	09:21:02	0
1883427	99	39	28,8	cyclictest	1954828	-21	latency_hist	08:20:00	0
2121970	2	38	0,4	sleep0	0	-21	swapper/0	11:05:59	0
1883432	99	38	29,7	cyclictest	2088316	-21	ntpq	10:31:02	1
1883427	99	38	31,5	cyclictest	1930304	-21	kworker/u8:0+events_unbound	09:41:05	0
1883427	99	38	28,8	cyclictest	280	-21	systemd-journal	07:19:59	0
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17