OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #0, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot4s.osadl.org (updated Sun Sep 20, 2026 00:46:18)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
3482755	2	402	396,4	sleep2	0	-21	swapper/2	19:05:06	8
3486104	2	365	345,13	sleep3	0	-21	swapper/3	19:08:20	9
3486081	2	358	349,6	sleep4	0	-21	swapper/4	19:08:05	10
3486069	2	358	338,13	sleep9	0	-21	swapper/9	19:07:54	15
3486094	2	356	316,13	sleep0	0	-21	swapper/0	19:08:17	0
3485907	2	354	334,13	sleep6	0	-21	swapper/6	19:05:42	12
3485987	2	352	332,13	sleep7	0	-21	swapper/7	19:06:47	13
3485993	2	351	331,13	sleep11	0	-21	swapper/11	19:06:51	3
3485936	2	351	330,14	sleep15	0	-21	swapper/15	19:06:07	7
3485977	2	349	329,13	sleep14	0	-21	swapper/14	19:06:38	6
3485902	2	348	328,13	sleep1	0	-21	swapper/1	19:05:37	1
3485897	2	348	329,12	sleep13	0	-21	swapper/13	19:05:33	5
3486247	2	347	326,14	sleep10	0	-21	swapper/10	19:09:48	2
3485985	2	347	324,14	sleep5	0	-21	swapper/5	19:06:45	11
3486192	2	344	323,14	sleep8	0	-21	swapper/8	19:09:13	14
3486228	2	342	321,14	sleep12	0	-21	swapper/12	19:09:34	4
3486639	99	60	58,2	cyclictest	3517430	-21	kworker/u32:0+events_unbound	20:40:18	10
3486651	99	51	48,2	cyclictest	3574929	-21	kworker/u32:4+flush-259:0	21:47:01	14
3486632	99	47	45,2	cyclictest	3465221	-21	kworker/u32:3+flush-259:0	20:12:18	8
3486662	99	44	42,2	cyclictest	3719811	-21	kworker/u32:3+events_unbound	22:50:51	4
3486657	99	44	42,2	cyclictest	3574929	-21	kworker/u32:4+events_unbound	22:27:49	2
3486657	99	42	40,0	cyclictest	0	-21	swapper/10	19:18:25	2
3486657	99	42	40,0	cyclictest	0	-21	swapper/10	19:18:24	2
3486662	99	41	38,2	cyclictest	3782240	-21	kworker/u32:1+events_unbound	23:42:03	4
3486659	99	41	39,2	cyclictest	3750906	-21	kworker/u32:1+flush-259:0	23:21:44	3
3486636	99	40	37,2	cyclictest	3719948	-21	kworker/u32:4+flush-259:0	23:05:05	9
3486627	99	40	36,3	cyclictest	3468512	-21	/usr/sbin/httpd	22:45:15	0
3486657	99	39	37,2	cyclictest	3606187	-21	kworker/u32:3+events_unbound	21:28:56	2
3486653	99	39	38,0	cyclictest	3683872	-21	kworker/u32:2+flush-259:0	22:45:15	15
3486657	99	38	31,1	cyclictest	0	-21	swapper/10	19:20:02	2
3486657	99	38	31,1	cyclictest	0	-21	swapper/10	19:20:02	2
3486646	99	37	36,1	cyclictest	3598677	-21	kworker/u32:2+flush-259:0	22:00:27	12
3486639	99	37	34,1	cyclictest	0	-21	swapper/4	19:59:34	10
3486667	99	36	33,2	cyclictest	3606187	-21	kworker/u32:3+flush-259:0	21:25:16	5
3486667	99	34	32,1	cyclictest	3809435	-21	kworker/u32:6+events_unbound	00:23:02	5
3486653	99	34	33,1	cyclictest	3642231	-21	kworker/u32:0+flush-259:0	21:55:20	15
3486632	99	34	33,1	cyclictest	0	-21	swapper/2	23:30:51	8
3486632	99	34	32,2	cyclictest	3642231	-21	kworker/u32:0+events_unbound	22:02:14	8
3486632	99	34	32,2	cyclictest	0	-21	swapper/2	22:13:15	8
3486632	99	34	32,2	cyclictest	0	-21	swapper/2	22:13:15	8
3486669	99	33	7,25	cyclictest	0	-21	swapper/14	21:46:34	6
3486657	99	33	32,1	cyclictest	3801713	-21	kworker/u32:1+flush-259:0	00:25:45	2
3486657	99	33	31,2	cyclictest	3517430	-21	kworker/u32:0+flush-259:0	20:35:20	2
3486657	99	33	31,1	cyclictest	3622618	-21	kworker/u32:1+events_unbound	21:51:58	2
3486648	99	33	0,30	cyclictest	0	-21	swapper/7	21:50:15	13
3486632	99	33	29,4	cyclictest	3681975	-21	kworker/2:1-events	22:40:51	8
3486632	99	33	1,29	cyclictest	0	-21	swapper/2	21:00:27	8
3486632	99	33	0,30	cyclictest	0	-21	swapper/2	23:49:16	8
3486632	99	33	0,30	cyclictest	0	-21	swapper/2	22:30:03	8
3486630	99	33	0,30	cyclictest	0	-21	swapper/1	20:00:15	1
3486657	99	32	30,1	cyclictest	3752647	-21	kworker/10:0+events	00:10:03	2
3486657	99	32	29,1	cyclictest	3664185	-21	kworker/10:0+events	22:55:15	2
3486641	99	32	31,1	cyclictest	3642231	-21	kworker/u32:0+flush-259:0	22:15:18	11
3486641	99	32	31,1	cyclictest	3642231	-21	kworker/u32:0+flush-259:0	22:15:18	11
3486639	99	32	31,1	cyclictest	3855721	-21	kworker/u32:0+events_unbound	00:38:22	10
3486632	99	32	30,2	cyclictest	0	-21	swapper/2	22:05:39	8
3486632	99	32	30,0	cyclictest	0	-21	swapper/2	19:18:25	8
3486632	99	32	30,0	cyclictest	0	-21	swapper/2	19:18:25	8
3486632	99	32	28,3	cyclictest	3597335	-21	ntpq	21:15:21	8
3486632	99	32	0,30	cyclictest	3561180	-21	kworker/2:2+mm_percpu_wq	21:14:03	8
3486632	99	32	0,29	cyclictest	0	-21	swapper/2	23:41:15	8
3486632	99	32	0,29	cyclictest	0	-21	swapper/2	23:35:15	8
3486632	99	32	0,29	cyclictest	0	-21	swapper/2	22:29:39	8
3486632	99	32	0,29	cyclictest	0	-21	swapper/2	21:37:15	8
3486627	99	32	0,24	cyclictest	0	-21	swapper/0	19:16:34	0
3486627	99	32	0,24	cyclictest	0	-21	swapper/0	19:16:34	0
3486672	99	31	29,2	cyclictest	0	-21	swapper/15	19:18:25	7
3486672	99	31	29,2	cyclictest	0	-21	swapper/15	19:18:25	7
3486669	99	31	7,11	cyclictest	0	-21	swapper/14	19:34:46	6
3486669	99	31	30,1	cyclictest	1774	-21	snmpd	19:10:06	6
3486669	99	31	30,1	cyclictest	1774	-21	snmpd	19:10:06	6
3486662	99	31	0,30	cyclictest	0	-21	swapper/12	19:32:22	4
3486659	99	31	0,30	cyclictest	0	-21	swapper/11	23:18:41	3
3486657	99	31	30,1	cyclictest	3752647	-21	kworker/10:0+events	23:50:27	2
3486657	99	31	30,1	cyclictest	3752647	-21	kworker/10:0+events	23:46:30	2
3486657	99	31	30,1	cyclictest	3727363	-21	kworker/10:1+events	23:11:03	2
3486657	99	31	30,1	cyclictest	3727363	-21	kworker/10:1+events	23:09:41	2
3486657	99	31	30,1	cyclictest	3727363	-21	kworker/10:1+events	23:04:04	2
3486657	99	31	30,1	cyclictest	3664185	-21	kworker/10:0+events	22:39:39	2
3486657	99	31	30,1	cyclictest	3664185	-21	kworker/10:0+events	22:19:31	2
3486657	99	31	30,1	cyclictest	3664185	-21	kworker/10:0+events	22:19:31	2
3486657	99	31	30,1	cyclictest	3586292	-21	kworker/u32:0+events_unbound	21:15:17	2
3486657	99	31	30,1	cyclictest	3515194	-21	kworker/10:1+events	22:07:03	2
3486657	99	31	30,1	cyclictest	3515194	-21	kworker/10:1+events	22:04:52	2
3486657	99	31	30,1	cyclictest	3515194	-21	kworker/10:1+events	20:40:03	2
3486657	99	31	30,1	cyclictest	3515194	-21	kworker/10:1+events	20:16:27	2
3486657	99	31	29,1	cyclictest	3752647	-21	kworker/10:0+events	23:28:07	2
3486657	99	31	29,1	cyclictest	3515194	-21	kworker/10:1+events	20:20:03	2
3486657	99	31	29,1	cyclictest	3515194	-21	kworker/10:1+events	20:00:03	2
3486653	99	31	0,30	cyclictest	0	-21	swapper/9	23:50:14	15
3486653	99	31	0,30	cyclictest	0	-21	swapper/9	20:18:32	15
3486653	99	31	0,30	cyclictest	0	-21	swapper/9	00:30:01	15
3486653	99	31	0,30	cyclictest	0	-21	swapper/9	00:08:02	15
3486651	99	31	0,30	cyclictest	0	-21	swapper/8	20:18:38	14
3486648	99	31	0,28	cyclictest	0	-21	swapper/7	21:45:14	13
3486636	99	31	29,1	cyclictest	1774	-21	snmpd	23:30:32	9
3486636	99	31	28,3	cyclictest	3574929	-21	kworker/u32:4+flush-259:0	22:12:38	9
3486636	99	31	28,3	cyclictest	3574929	-21	kworker/u32:4+flush-259:0	22:12:37	9
3486636	99	31	0,30	cyclictest	0	-21	swapper/3	23:00:45	9
3486636	99	31	0,30	cyclictest	0	-21	swapper/3	21:28:20	9
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17