OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #0, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot4s.osadl.org (updated Fri Jun 12, 2026 00:46:18)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
1941689	2	383	363,13	sleep14	0	-21	swapper/14	19:08:11	6
1941774	2	361	325,13	sleep3	0	-21	swapper/3	19:09:21	9
1941753	2	359	340,13	sleep0	0	-21	swapper/0	19:09:02	0
1941556	2	359	319,14	sleep5	0	-21	swapper/5	19:06:26	11
1941761	2	355	335,13	sleep8	0	-21	swapper/8	19:09:10	14
1941612	2	353	333,13	sleep1	0	-21	swapper/1	19:07:10	1
1941763	2	350	330,13	sleep10	0	-21	swapper/10	19:09:12	2
1941498	2	350	330,13	sleep11	0	-21	swapper/11	19:05:43	3
1941822	2	349	328,14	sleep2	0	-21	swapper/2	19:09:52	8
1941645	2	349	327,14	sleep13	0	-21	swapper/13	19:07:38	5
1938346	2	349	328,14	sleep7	0	-21	swapper/7	19:05:07	13
1941560	2	348	328,13	sleep9	0	-21	swapper/9	19:06:30	15
1941529	2	348	338,7	sleep15	0	-21	swapper/15	19:06:03	7
1941730	2	347	327,13	sleep12	0	-21	swapper/12	19:08:42	4
1941716	2	347	329,12	sleep4	0	-21	swapper/4	19:08:34	10
1941698	2	347	329,11	sleep6	0	-21	swapper/6	19:08:19	12
1942255	99	41	40,1	cyclictest	2182013	-21	kworker/u32:3+events_unbound	23:05:09	8
1942255	99	40	39,1	cyclictest	0	-21	swapper/2	21:58:40	8
1942276	99	39	38,1	cyclictest	2067278	-21	kworker/u32:0+flush-259:0	21:45:16	15
1942255	99	39	0,31	cyclictest	1887251	-21	kworker/2:1+mm_percpu_wq	22:20:40	8
1942255	99	38	8,30	cyclictest	2273517	-21	kworker/u32:3+events_unbound	00:14:50	8
1942255	99	38	38,0	cyclictest	2061111	-21	ssh	21:21:30	8
1942255	99	38	37,1	cyclictest	1769	-21	arpwatch	00:19:19	8
1942255	99	37	36,1	cyclictest	1769	-21	arpwatch	00:07:57	8
1942255	99	37	35,2	cyclictest	0	-21	swapper/2	23:33:49	8
1942255	99	37	0,33	cyclictest	0	-21	swapper/2	23:47:48	8
1942255	99	37	0,33	cyclictest	0	-21	swapper/2	23:47:48	8
1942255	99	36	35,1	cyclictest	1769	-21	arpwatch	23:40:57	8
1942255	99	36	34,2	cyclictest	0	-21	swapper/2	00:26:12	8
1942255	99	36	34,1	cyclictest	1887251	-21	kworker/2:1-events	21:31:30	8
1942255	99	36	34,0	cyclictest	0	-21	swapper/2	19:50:10	8
1942255	99	36	33,3	cyclictest	0	-21	swapper/2	22:33:38	8
1942255	99	36	33,3	cyclictest	0	-21	swapper/2	22:25:13	8
1942255	99	36	33,3	cyclictest	0	-21	swapper/2	20:55:22	8
1942255	99	36	28,8	cyclictest	1887251	-21	kworker/2:1+mm_percpu_wq	21:10:02	8
1942255	99	35	4,31	cyclictest	0	-21	swapper/2	23:16:36	8
1942255	99	35	34,1	cyclictest	2185806	-21	kworker/u32:4+events_unbound	23:00:52	8
1942255	99	35	33,2	cyclictest	0	-21	swapper/2	23:29:08	8
1942255	99	35	33,2	cyclictest	0	-21	swapper/2	23:10:02	8
1942255	99	35	33,0	cyclictest	0	-21	swapper/2	22:40:16	8
1942255	99	35	32,3	cyclictest	0	-21	swapper/2	21:36:30	8
1942255	99	35	32,3	cyclictest	0	-21	swapper/2	00:34:59	8
1942255	99	35	32,2	cyclictest	1887251	-21	kworker/2:1-events	19:24:17	8
1942255	99	35	0,35	cyclictest	2049114	-21	kworker/2:2-mm_percpu_wq	21:18:03	8
1942255	99	35	0,33	cyclictest	2229460	-21	kworker/2:0+mm_percpu_wq	00:00:02	8
1942276	99	34	34,0	cyclictest	0	-21	swapper/9	21:16:32	15
1942255	99	34	33,1	cyclictest	1887251	-21	kworker/2:1-events	22:10:58	8
1942255	99	34	32,2	cyclictest	0	-21	swapper/2	23:52:51	8
1942255	99	34	32,2	cyclictest	0	-21	swapper/2	23:52:50	8
1942255	99	34	32,2	cyclictest	0	-21	swapper/2	21:53:30	8
1942255	99	34	32,0	cyclictest	0	-21	swapper/2	19:35:14	8
1942255	99	34	31,3	cyclictest	0	-21	swapper/2	20:09:04	8
1942255	99	34	31,3	cyclictest	0	-21	swapper/2	00:20:18	8
1942255	99	34	30,2	cyclictest	0	-21	swapper/2	22:57:31	8
1942255	99	34	28,3	cyclictest	1887251	-21	kworker/2:1+mm_percpu_wq	19:58:40	8
1942255	99	34	25,9	cyclictest	2067775	-21	ssh	21:26:52	8
1942255	99	34	1,31	cyclictest	2157421	-21	kworker/2:1-events	22:50:03	8
1942255	99	33	4,29	cyclictest	0	-21	swapper/2	23:24:57	8
1942255	99	33	31,2	cyclictest	0	-21	swapper/2	23:55:07	8
1942255	99	33	31,2	cyclictest	0	-21	swapper/2	23:55:06	8
1942255	99	33	31,2	cyclictest	0	-21	swapper/2	21:45:06	8
1942255	99	33	31,0	cyclictest	0	-21	swapper/2	22:35:10	8
1942255	99	33	31,0	cyclictest	0	-21	swapper/2	22:00:44	8
1942255	99	33	28,3	cyclictest	0	-21	swapper/2	20:25:20	8
1942255	99	33	2,29	cyclictest	2118732	-21	kworker/u32:3-nfsiod	22:16:39	8
1942255	99	33	1,1	cyclictest	42	1	ktimers/2	19:10:10	8
1942255	99	33	0,33	cyclictest	2121028	-21	StreamT~s		8
1942255	99	33	0,33	cyclictest	0	-21	swapper/2	23:35:52	8
1942255	99	33	0,31	cyclictest	2229460	-21	kworker/2:0-mm_percpu_wq	00:38:48	8
1942292	99	32	8,11	cyclictest	0	-21	swapper/14	22:27:51	6
1942255	99	32	32,0	cyclictest	1887251	-21	kworker/2:1-events	20:52:08	8
1942255	99	32	32,0	cyclictest	1887251	-21	kworker/2:1-events	20:52:08	8
1942255	99	32	32,0	cyclictest	0	-21	swapper/2	21:04:58	8
1942255	99	32	30,2	cyclictest	0	-21	swapper/2	20:45:14	8
1942255	99	32	30,2	cyclictest	0	-21	swapper/2	20:45:14	8
1942255	99	32	30,0	cyclictest	0	-21	swapper/2	22:45:47	8
1942255	99	32	30,0	cyclictest	0	-21	swapper/2	20:15:20	8
1942255	99	32	30,0	cyclictest	0	-21	swapper/2	19:45:20	8
1942255	99	32	0,32	cyclictest	0	-21	swapper/2	19:15:18	8
1942255	99	32	0,30	cyclictest	0	-21	swapper/2	21:41:25	8
1942255	99	32	0,29	cyclictest	0	-21	swapper/2	20:30:16	8
1942288	99	31	30,1	cyclictest	2271886	-21	ethtool	00:05:15	5
1942283	99	31	0,30	cyclictest	0	-21	swapper/11	21:40:47	3
1942281	99	31	0,30	cyclictest	0	-21	swapper/10	22:55:17	2
1942281	99	31	0,30	cyclictest	0	-21	swapper/10	22:19:21	2
1942281	99	31	0,30	cyclictest	0	-21	swapper/10	21:51:23	2
1942281	99	31	0,30	cyclictest	0	-21	swapper/10	21:05:28	2
1942281	99	31	0,1	cyclictest	0	-21	swapper/10	00:20:14	2
1942276	99	31	0,30	cyclictest	0	-21	swapper/9	23:52:31	15
1942276	99	31	0,30	cyclictest	0	-21	swapper/9	23:52:30	15
1942276	99	31	0,30	cyclictest	0	-21	swapper/9	19:44:29	15
1942274	99	31	30,1	cyclictest	1774	-21	snmpd	00:21:43	14
1942274	99	31	0,30	cyclictest	0	-21	swapper/8	22:46:34	14
1942274	99	31	0,30	cyclictest	0	-21	swapper/8	22:33:09	14
1942274	99	31	0,30	cyclictest	0	-21	swapper/8	20:33:45	14
1942274	99	31	0,30	cyclictest	0	-21	swapper/8	19:27:40	14
1942274	99	31	0,30	cyclictest	0	-21	swapper/8	00:00:36	14
1942271	99	31	30,1	cyclictest	0	-21	swapper/7	23:28:11	13
1942271	99	31	0,30	cyclictest	0	-21	swapper/7	19:55:14	13
1942267	99	31	30,1	cyclictest	1774	-21	snmpd	21:16:01	12
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17