OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #0, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot4s.osadl.org (updated Sun Sep 13, 2026 00:46:20)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
3685114	2	397	377,13	sleep4	0	-21	swapper/4	19:06:12	10
3685213	2	364	308,12	sleep13	0	-21	swapper/13	19:07:09	5
3685339	2	363	322,15	sleep10	0	-21	swapper/10	19:08:42	2
3685355	2	356	318,30	sleep9	0	-21	swapper/9	19:08:57	15
3685364	2	355	335,13	sleep1	0	-21	swapper/1	19:09:06	1
3681919	2	354	334,13	sleep15	0	-21	swapper/15	19:05:03	7
3685204	2	353	334,12	sleep6	0	-21	swapper/6	19:07:02	12
3685118	2	353	333,13	sleep7	0	-21	swapper/7	19:06:15	13
3685354	2	351	332,13	sleep8	0	-21	swapper/8	19:08:57	14
3685331	2	351	331,13	sleep2	0	-21	swapper/2	19:08:35	8
3685079	2	350	331,12	sleep12	0	-21	swapper/12	19:05:48	4
3685363	2	349	330,13	sleep0	0	-21	swapper/0	19:09:05	0
3685173	2	349	329,13	sleep5	0	-21	swapper/5	19:06:45	11
3685290	2	347	328,13	sleep3	0	-21	swapper/3	19:08:03	9
3685343	2	346	325,14	sleep14	0	-21	swapper/14	19:08:47	6
3685130	2	344	324,13	sleep11	0	-21	swapper/11	19:06:18	3
3685879	99	86	84,1	cyclictest	3777082	-21	kworker/u32:4+events_unbound	21:15:27	2
4012512	2	59	0,0	sleep12	0	-21	swapper/12	00:05:13	4
3978396	2	56	0,0	sleep2	0	-21	swapper/2	23:39:08	8
3978396	2	56	0,0	sleep2	0	-21	swapper/2	23:39:08	8
3685891	99	55	53,1	cyclictest	3676721	-21	kworker/u32:3+events_unbound	19:25:22	6
3685859	99	55	54,1	cyclictest	4026971	-21	kworker/u32:5+flush-259:0	00:20:19	9
3685859	99	55	52,2	cyclictest	3971014	-21	kworker/u32:1+flush-259:0	23:33:36	9
3685859	99	55	52,2	cyclictest	3971014	-21	kworker/u32:1+flush-259:0	23:33:36	9
3685852	99	53	51,2	cyclictest	3642383	-21	kworker/u32:0+flush-259:0	19:30:23	1
3685891	99	49	47,1	cyclictest	3881997	-21	kworker/u32:3+events_unbound	23:00:24	6
3685879	99	49	47,2	cyclictest	4038927	-21	kworker/u32:0+events_unbound	00:32:34	2
3685879	99	48	47,1	cyclictest	3860232	-21	kworker/u32:1+events_unbound	22:37:22	2
3685859	99	48	46,2	cyclictest	4038927	-21	kworker/u32:0+flush-259:0	00:38:53	9
3685879	99	46	44,2	cyclictest	3702462	-21	kworker/u32:4+events_unbound	19:48:24	2
3685891	99	44	8,36	cyclictest	0	-21	swapper/14	22:16:49	6
3685891	99	43	7,11	cyclictest	0	-21	swapper/14	22:27:48	6
3685883	99	43	40,2	cyclictest	3741690	-21	kworker/u32:1+flush-259:0	20:20:19	3
3685891	99	42	38,3	cyclictest	3907015	-21	kworker/u32:0+flush-259:0	23:16:58	6
3685862	99	42	41,1	cyclictest	3907015	-21	kworker/u32:0+flush-259:0	22:47:31	10
3685862	99	42	41,1	cyclictest	3797421	-21	kworker/u32:2+flush-259:0	21:25:36	10
3685870	99	41	40,1	cyclictest	3714409	-21	kworker/u32:1+flush-259:0	19:50:15	13
3685859	99	40	37,2	cyclictest	3778261	-21	kworker/u32:5+events_unbound	21:12:53	9
3685879	99	39	37,1	cyclictest	3860232	-21	kworker/u32:1+events_unbound	22:24:34	2
3685857	99	39	35,2	cyclictest	3676721	-21	kworker/u32:3+flush-259:0	19:29:46	8
3685894	99	38	37,1	cyclictest	3676721	-21	kworker/u32:3+events_unbound	19:27:56	7
3685891	99	38	36,1	cyclictest	3783491	-21	kworker/u32:1+events_unbound	21:10:20	6
3685868	99	38	36,2	cyclictest	3741690	-21	kworker/u32:1+events_unbound	20:26:48	12
3685891	99	37	35,1	cyclictest	3774274	-21	kworker/u32:0+flush-259:0	22:00:48	6
3685891	99	37	1,5	cyclictest	0	-21	swapper/14	22:39:59	6
3685885	99	37	37,0	cyclictest	0	-21	swapper/12	19:10:01	4
3685885	99	37	36,1	cyclictest	3949993	-21	kworker/u32:2+events_unbound	23:26:00	4
3685883	99	37	34,2	cyclictest	3881997	-21	kworker/u32:3+events_unbound	23:28:34	3
3685870	99	37	37,0	cyclictest	0	-21	swapper/7	20:33:41	13
3685870	99	37	37,0	cyclictest	0	-21	swapper/7	20:33:41	13
3685851	99	37	36,1	cyclictest	3907015	-21	kworker/u32:0+flush-259:0	23:55:19	0
3685883	99	36	33,2	cyclictest	3732123	-21	kworker/u32:4+flush-259:0	20:09:27	3
3685870	99	36	36,0	cyclictest	0	-21	swapper/7	20:23:03	13
3685870	99	36	36,0	cyclictest	0	-21	swapper/7	00:02:09	13
3685885	99	35	5,30	cyclictest	3774274	-21	kworker/u32:0-nfsiod	21:18:29	4
3685870	99	35	35,0	cyclictest	0	-21	swapper/7	22:15:47	13
3685870	99	35	34,1	cyclictest	3907015	-21	kworker/u32:0+events_unbound	23:32:17	13
3685870	99	35	34,1	cyclictest	3907015	-21	kworker/u32:0+events_unbound	23:32:17	13
3685864	99	35	33,2	cyclictest	3714409	-21	kworker/u32:1+flush-259:0	19:55:22	11
3685851	99	35	33,1	cyclictest	4026971	-21	kworker/u32:5+events_unbound	00:35:07	0
3685851	99	35	32,2	cyclictest	3774274	-21	kworker/u32:0+events_unbound	22:16:53	0
3685891	99	34	31,2	cyclictest	3994430	-21	kworker/u32:2+flush-259:0	23:56:01	6
3685889	99	34	31,1	cyclictest	3702462	-21	kworker/u32:4+flush-259:0	19:32:20	5
3685879	99	34	33,1	cyclictest	3938635	-21	kworker/10:0+events	00:09:29	2
3685870	99	34	34,0	cyclictest	0	-21	swapper/7	21:53:55	13
3685870	99	34	34,0	cyclictest	0	-21	swapper/7	20:15:45	13
3685862	99	34	32,1	cyclictest	3899996	-21	kworker/u32:5+events_unbound	22:55:17	10
3685857	99	34	32,2	cyclictest	3676721	-21	kworker/u32:3+flush-259:0	19:20:27	8
3685851	99	34	32,1	cyclictest	3663012	-21	kworker/u32:4+events_unbound	19:15:08	0
3685885	99	33	33,0	cyclictest	0	-21	swapper/12	21:55:35	4
3685885	99	33	33,0	cyclictest	0	-21	swapper/12	20:44:41	4
3685885	99	33	33,0	cyclictest	0	-21	swapper/12	20:44:41	4
3685885	99	33	33,0	cyclictest	0	-21	swapper/12	20:17:11	4
3685885	99	33	33,0	cyclictest	0	-21	swapper/12	00:14:15	4
3685885	99	33	32,1	cyclictest	3881997	-21	kworker/u32:3+events_unbound	22:34:48	4
3685879	99	33	31,2	cyclictest	3848795	-21	kworker/u32:4+events_unbound	21:58:58	2
3685868	99	33	29,2	cyclictest	0	-21	swapper/6	23:35:12	12
3685868	99	33	29,2	cyclictest	0	-21	swapper/6	23:35:12	12
3685885	99	32	9,21	cyclictest	3828125	-21	daemon-init	21:40:15	4
3685885	99	32	32,0	cyclictest	0	-21	swapper/12	23:48:11	4
3685885	99	32	32,0	cyclictest	0	-21	swapper/12	21:21:49	4
3685885	99	32	32,0	cyclictest	0	-21	swapper/12	20:55:03	4
3685877	99	32	30,2	cyclictest	3815407	-21	kworker/u32:1+events_unbound	21:33:22	15
3685873	99	32	0,29	cyclictest	0	-21	swapper/8	23:50:17	14
3685870	99	32	0,32	cyclictest	87	1	ktimers/7	00:05:07	13
3685862	99	32	29,3	cyclictest	1774	-21	snmpd	20:51:39	10
3685857	99	32	29,3	cyclictest	1774	-21	snmpd	23:34:55	8
3685857	99	32	29,3	cyclictest	1774	-21	snmpd	23:34:55	8
3685857	99	32	29,3	cyclictest	1774	-21	snmpd	20:21:26	8
3685857	99	32	0,29	cyclictest	0	-21	swapper/2	22:56:03	8
3685857	99	32	0,29	cyclictest	0	-21	swapper/2	22:01:57	8
3685857	99	32	0,29	cyclictest	0	-21	swapper/2	21:46:01	8
3685857	99	32	0,29	cyclictest	0	-21	swapper/2	21:02:51	8
3685852	99	32	28,2	cyclictest	3774274	-21	kworker/u32:0+events_unbound	22:22:00	1
3685894	99	31	30,1	cyclictest	1774	-21	snmpd	19:23:26	7
3685891	99	31	7,13	cyclictest	0	-21	swapper/14	20:05:00	6
3685891	99	31	7,11	cyclictest	0	-21	swapper/14	20:59:56	6
3685891	99	31	7,11	cyclictest	0	-21	swapper/14	19:10:05	6
3685889	99	31	28,2	cyclictest	4020042	-21	kworker/u32:1+events_unbound	00:24:53	5
3685889	99	31	0,30	cyclictest	0	-21	swapper/13	23:10:17	5
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17