OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack8slot6s.osadl.org (updated Mon Aug 17, 2026 00:43:42)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
4330	99	1052	1050,2	cyclictest	5799	-21	kworker/0:2+events	22:36:07	0
4330	99	1052	1050,2	cyclictest	549	-21	kworker/0:0+events	22:56:08	0
4330	99	1051	1049,2	cyclictest	6535	-21	kworker/0:1+events	21:17:52	0
4330	99	1051	1049,2	cyclictest	549	-21	kworker/0:0+events	22:31:06	0
4330	99	1051	1048,3	cyclictest	22426	-21	kworker/0:0+events	22:20:43	0
4330	99	1050	1048,2	cyclictest	5799	-21	kworker/0:2+events	22:40:43	0
4330	99	1050	1048,2	cyclictest	5332	-21	kworker/0:0+events	22:06:06	0
4330	99	1050	1048,2	cyclictest	27175	-21	kworker/0:1+events	23:06:05	0
4330	99	1050	1048,2	cyclictest	17173	-21	kworker/0:1+events	22:15:52	0
4330	99	1050	1046,2	cyclictest	8794	-21	kworker/0:0+events	23:27:30	0
4330	99	1049	1047,2	cyclictest	3396	-21	kworker/0:2+events	23:04:10	0
4330	99	1049	1047,2	cyclictest	17199	-21	kworker/0:1+events	22:04:03	0
4330	99	1048	1046,2	cyclictest	6535	-21	kworker/0:1+events	21:26:36	0
4330	99	1048	1046,2	cyclictest	6535	-21	kworker/0:1+events	19:16:57	0
4330	99	1047	1045,2	cyclictest	26070	-21	kworker/0:0+events	21:21:29	0
4330	99	1047	1045,2	cyclictest	17173	-21	kworker/0:1+events	22:14:24	0
4330	99	1047	1045,2	cyclictest	10669	-21	kworker/0:2+events	22:26:07	0
4330	99	1046	1044,2	cyclictest	5438	-21	kworker/0:2+events	21:37:22	0
4330	99	1045	1043,2	cyclictest	10656	-21	kworker/0:0+events	21:32:34	0
4335	99	1044	1042,2	cyclictest	16088	-21	kworker/1:2+events	19:40:23	1
4339	99	1042	1041,1	cyclictest	24704	-21	kworker/2:1+events	20:03:02	2
4339	99	1041	1039,2	cyclictest	14512	-21	kworker/2:0+events	21:14:12	2
4343	99	1040	1038,2	cyclictest	19920	-21	turbostat	23:24:55	3
4335	99	1037	1036,1	cyclictest	18160	-21	kworker/1:1+events	19:48:41	1
4339	99	1035	1034,1	cyclictest	14512	-21	kworker/2:0+events	21:00:43	2
4339	99	1023	1021,2	cyclictest	14512	-21	kworker/2:0+events	23:49:33	2
4335	99	1023	1022,1	cyclictest	15637	-21	kworker/1:2+events	19:27:10	1
4339	99	1022	1021,1	cyclictest	14512	-21	kworker/2:0+events	23:35:43	2
4343	99	1020	1018,2	cyclictest	26459	-21	kworker/3:0+events	20:20:43	3
4335	99	1020	1018,2	cyclictest	15637	-21	kworker/1:2+events	19:30:20	1
4343	99	1019	1017,2	cyclictest	29810	-21	kworker/3:1+events	19:14:43	3
4335	99	1019	1017,2	cyclictest	18160	-21	kworker/1:1+events	20:37:35	1
4335	99	1017	1015,2	cyclictest	2430	-21	kworker/1:2+events	20:30:22	1
4335	99	1017	1014,3	cyclictest	18160	-21	kworker/1:1+events	19:58:31	1
4335	99	1017	1014,3	cyclictest	12016	-21	kworker/1:1+events	21:56:52	1
4343	99	1013	1012,1	cyclictest	17345	-21	kworker/3:1+events	00:18:49	3
4335	99	1010	1008,2	cyclictest	2430	-21	kworker/1:2+events	20:53:58	1
4343	99	1007	1005,2	cyclictest	17345	-21	kworker/3:1+events	00:36:33	3
4335	99	1005	1002,3	cyclictest	9976	-21	kworker/1:0+events	20:43:42	1
4335	99	1003	1000,2	cyclictest	18160	-21	kworker/1:1+events	19:50:23	1
4335	99	1002	999,2	cyclictest	15637	-21	kworker/1:2+events	19:21:04	1
4335	99	1001	999,2	cyclictest	2430	-21	kworker/1:2+events	20:48:31	1
4335	99	999	997,2	cyclictest	18160	-21	kworker/1:1+events	20:26:49	1
4335	99	999	997,2	cyclictest	14042	-21	kworker/1:1+events	19:35:41	1
4335	99	997	994,3	cyclictest	13071	-21	turbostat	21:39:56	1
4339	99	992	991,1	cyclictest	14512	-21	kworker/2:0+events	00:02:20	2
4343	99	991	989,2	cyclictest	10260	-21	turbostat	20:49:56	3
4343	99	985	983,2	cyclictest	22440	-21	turbostat	19:59:56	3
4343	99	981	979,2	cyclictest	17401	-21	kworker/3:2+events	20:19:33	3
4330	99	973	970,3	cyclictest	27889	-21	turbostat	21:54:56	0
4343	99	971	970,1	cyclictest	17963	-21	kworker/3:1+events	21:40:22	3
4343	99	969	968,1	cyclictest	30103	-21	kworker/3:0+events	22:45:21	3
4343	99	963	962,1	cyclictest	1354	-21	kworker/3:2+events	00:22:58	3
4343	99	963	961,2	cyclictest	27263	-21	turbostat	22:59:54	3
4343	99	961	959,1	cyclictest	17345	-21	kworker/3:1+events	23:15:21	3
4343	99	960	959,1	cyclictest	24343	-21	kworker/3:2+events	22:52:32	3
4343	99	959	956,1	cyclictest	1354	-21	kworker/3:2+events	00:14:03	3
4343	99	958	956,2	cyclictest	9994	-21	kworker/3:0+events	23:10:45	3
4343	99	956	955,1	cyclictest	17345	-21	kworker/3:1+events	00:08:17	3
4339	99	955	954,1	cyclictest	14512	-21	kworker/2:0+events	23:32:32	2
4343	99	954	953,1	cyclictest	17963	-21	kworker/3:1+events	21:53:50	3
4343	99	953	952,1	cyclictest	26459	-21	kworker/3:0+events	20:12:12	3
4339	99	950	948,2	cyclictest	14512	-21	kworker/2:0+pm	23:57:46	2
4343	99	947	946,1	cyclictest	17345	-21	kworker/3:1+events	00:33:58	3
4343	99	946	945,1	cyclictest	17345	-21	kworker/3:1+events	00:27:08	3
4343	99	943	942,1	cyclictest	17963	-21	kworker/3:1+events	21:47:55	3
4343	99	943	942,1	cyclictest	17963	-21	kworker/3:1+events	21:47:55	3
4343	99	942	941,1	cyclictest	26459	-21	kworker/3:0+events	20:07:53	3
4339	99	932	931,1	cyclictest	29775	-21	kworker/2:2+events	23:51:56	2
4343	99	931	928,3	cyclictest	5140	-21	turbostat	23:09:55	3
4339	99	923	922,1	cyclictest	14512	-21	kworker/2:0+events	20:58:58	2
4339	99	921	917,3	cyclictest	29775	-21	kworker/2:2+events	23:41:29	2
4339	99	915	913,2	cyclictest	14512	-21	kworker/2:0+events	21:07:22	2
4330	99	911	908,3	cyclictest	4232	-21	turbostat	19:14:55	0
4343	99	907	905,2	cyclictest	351	-21	turbostat	21:59:55	3
4343	99	893	891,2	cyclictest	24868	-21	turbostat	23:29:55	3
4330	99	880	876,4	cyclictest	27125	-21	turbostat	00:04:55	0
4343	99	877	874,3	cyclictest	20773	-21	turbostat	21:14:55	3
4343	99	873	872,1	cyclictest	25725	-21	turbostat	21:19:56	3
4343	99	866	864,2	cyclictest	12276	-21	turbostat	20:54:55	3
4330	99	856	853,3	cyclictest	24445	-21	turbostat	20:04:56	0
4343	99	687	684,3	cyclictest	13071	-21	turbostat	21:39:55	3
4343	99	683	682,1	cyclictest	24445	-21	turbostat	20:04:55	3
4335	99	668	664,4	cyclictest	27889	-21	turbostat	21:54:55	1
4343	99	626	622,4	cyclictest	12448	-21	turbostat	22:44:54	3
4339	99	623	621,2	cyclictest	24868	-21	turbostat	23:29:55	2
4335	99	613	611,1	cyclictest	32526	-21	turbostat	20:24:55	1
4339	99	595	594,1	cyclictest	12276	-21	turbostat	20:54:55	2
4330	99	588	585,3	cyclictest	351	-21	turbostat	21:59:55	0
4330	99	586	584,2	cyclictest	22373	-21	turbostat	22:54:55	0
4330	99	556	553,3	cyclictest	20773	-21	turbostat	21:14:55	0
4335	99	522	519,3	cyclictest	6277	-21	turbostat	19:19:55	1
4330	99	521	518,3	cyclictest	19920	-21	turbostat	23:24:54	0
4339	99	516	512,2	cyclictest	22440	-21	turbostat	19:59:56	2
4343	99	509	506,3	cyclictest	27125	-21	turbostat	00:04:55	3
4343	99	34	33,1	cyclictest	19154	-21	fwupd	23:50:28	3
4339	99	25	25,0	cyclictest	439	-21	hwrng	22:29:56	2
4335	99	22	22,0	cyclictest	439	-21	hwrng	22:36:06	1
4339	99	21	21,0	cyclictest	439	-21	hwrng	22:43:15	2
4343	99	20	19,1	cyclictest	678	50	irq/123-lan	22:31:33	3
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17