OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack8slot6s.osadl.org (updated Tue Aug 18, 2026 00:43:42)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
23072	99	1049	1048,1	cyclictest	6824	-21	kworker/3:0+events	19:50:44	3
23067	99	1047	1044,2	cyclictest	19178	-21	kworker/2:2+events	00:20:21	2
23072	99	1043	1042,1	cyclictest	30608	-21	kworker/3:2+events	21:50:42	3
23054	99	1040	1039,1	cyclictest	26869	-21	kworker/0:0+events	19:20:20	0
23072	99	1039	1038,1	cyclictest	22832	-21	kworker/3:0+events	22:10:53	3
23072	99	1033	1031,2	cyclictest	22832	-21	kworker/3:0+events	22:09:21	3
23067	99	1029	1028,1	cyclictest	25879	-21	kworker/2:2+events	20:31:30	2
23059	99	1028	1027,1	cyclictest	8075	-21	kworker/1:1+events	21:13:49	1
23054	99	1027	1026,1	cyclictest	26869	-21	kworker/0:0+events	22:19:01	0
23067	99	1026	1024,2	cyclictest	2261	-21	kworker/2:0+events	23:55:45	2
23067	99	1026	1024,2	cyclictest	18109	-21	kworker/2:1+events	23:47:34	2
23054	99	1026	1022,2	cyclictest	1140	-21	turbostat	22:44:55	0
23067	99	1025	1023,2	cyclictest	19178	-21	kworker/2:2+events	00:03:35	2
23067	99	1023	1021,2	cyclictest	19178	-21	kworker/2:2+events	00:16:27	2
23067	99	1021	1019,2	cyclictest	18109	-21	kworker/2:1+events	20:25:44	2
23067	99	1020	1018,2	cyclictest	18109	-21	kworker/2:1+events	23:53:40	2
23067	99	1019	1017,2	cyclictest	2261	-21	kworker/2:0+events	00:14:26	2
23054	99	1019	1018,1	cyclictest	30028	-21	kworker/0:2+events	19:19:54	0
23067	99	1016	1013,3	cyclictest	18109	-21	kworker/2:1+events	20:39:43	2
23067	99	1015	1013,2	cyclictest	22585	-21	kworker/2:0+events	20:19:45	2
23067	99	1015	1013,2	cyclictest	18109	-21	kworker/2:1+events	20:21:50	2
23054	99	1015	1014,1	cyclictest	21978	-21	kworker/0:3+events	22:37:23	0
23067	99	1014	1011,3	cyclictest	7967	-21	kworker/2:1+events	00:28:35	2
23067	99	1013	1011,2	cyclictest	29297	-21	kworker/2:1+events	19:25:42	2
23067	99	1013	1011,2	cyclictest	19178	-21	kworker/2:2+events	00:06:25	2
23059	99	1009	1006,2	cyclictest	20771	-21	kworker/1:0+events	00:37:21	1
23059	99	1009	1006,2	cyclictest	16485	-21	kworker/1:0+events	23:39:44	1
23072	99	1008	1007,1	cyclictest	13833	-21	kworker/3:1+events	23:01:08	3
23054	99	1005	1003,2	cyclictest	13928	-21	kworker/0:1+events	22:24:26	0
23067	99	1001	999,2	cyclictest	29297	-21	kworker/2:1+events	19:31:25	2
23072	99	1000	997,3	cyclictest	24076	-21	turbostat	23:39:55	3
23072	99	999	998,1	cyclictest	24550	-21	kworker/3:2+events	23:06:49	3
23067	99	999	997,2	cyclictest	27924	-21	kworker/2:0+events	20:42:24	2
23054	99	997	995,2	cyclictest	28246	-21	turbostat	20:44:55	0
23059	99	995	994,1	cyclictest	28536	-21	turbostat	22:39:55	1
23072	99	994	993,1	cyclictest	27761	-21	kworker/3:0+events	22:49:00	3
23072	99	991	991,0	cyclictest	30608	-21	kworker/3:2+events	21:39:02	3
23072	99	991	990,1	cyclictest	22832	-21	kworker/3:0+events	21:57:44	3
23059	99	987	986,1	cyclictest	6036	-21	kworker/1:0+events	21:09:22	1
23059	99	987	986,1	cyclictest	24626	-21	kworker/1:2+events	00:32:41	1
23059	99	983	982,1	cyclictest	8075	-21	kworker/1:1+events	21:16:05	1
23072	99	981	979,1	cyclictest	6679	-21	kworker/3:2+events	22:54:02	3
23072	99	980	980,0	cyclictest	13833	-21	kworker/3:1+events	23:27:14	3
23072	99	978	977,1	cyclictest	13833	-21	kworker/3:1+events	23:20:23	3
23072	99	978	977,1	cyclictest	13833	-21	kworker/3:1+events	23:20:23	3
23072	99	978	976,2	cyclictest	30272	-21	turbostat	20:49:54	3
23072	99	977	975,2	cyclictest	29068	-21	turbostat	19:29:55	3
23072	99	974	973,1	cyclictest	22832	-21	kworker/3:0+events	22:00:54	3
23072	99	974	973,1	cyclictest	13370	-21	kworker/3:1+events	21:47:14	3
23072	99	973	972,1	cyclictest	24550	-21	kworker/3:2+events	23:11:29	3
23072	99	972	971,1	cyclictest	3392	-21	kworker/3:1+events	19:44:23	3
23072	99	970	969,1	cyclictest	30608	-21	kworker/3:2+events	21:30:44	3
23072	99	970	969,1	cyclictest	14845	-21	kworker/3:0+events	19:38:19	3
23072	99	969	968,1	cyclictest	9291	-21	kworker/3:1+events	20:12:00	3
23072	99	969	968,1	cyclictest	18314	-21	kworker/3:2+events	23:31:06	3
23059	99	968	967,1	cyclictest	14536	-21	kworker/1:1+events	20:50:22	1
23072	99	967	964,3	cyclictest	27030	-21	turbostat	19:24:55	3
23059	99	966	965,1	cyclictest	32262	-21	kworker/1:2+events	20:59:24	1
23072	99	965	962,1	cyclictest	13833	-21	kworker/3:1+events	22:56:38	3
23059	99	965	965,0	cyclictest	16485	-21	kworker/1:0+events	21:22:54	1
23072	99	964	963,1	cyclictest	6824	-21	kworker/3:0+events	19:48:18	3
23072	99	963	963,0	cyclictest	8199	-21	kworker/3:0+events	21:44:05	3
23072	99	963	962,1	cyclictest	23170	-21	kworker/3:0+events	21:25:43	3
23072	99	962	961,1	cyclictest	5698	-21	kworker/3:3+events	19:57:13	3
23072	99	961	960,1	cyclictest	9291	-21	kworker/3:1+events	20:02:57	3
23072	99	961	960,1	cyclictest	6826	-21	kworker/3:1+events	22:40:22	3
23059	99	961	959,2	cyclictest	29029	-21	turbostat	23:44:55	1
23072	99	960	959,1	cyclictest	28954	-21	kworker/3:1+events	23:42:02	3
23054	99	960	958,2	cyclictest	20505	-21	kworker/0:0+events	22:32:54	0
23072	99	959	957,1	cyclictest	13833	-21	kworker/3:1+events	23:16:29	3
23059	99	956	955,1	cyclictest	6036	-21	kworker/1:0+events	21:02:45	1
23072	99	953	952,1	cyclictest	5698	-21	kworker/3:3+events	20:06:29	3
23067	99	946	942,4	cyclictest	18255	-21	turbostat	21:24:55	2
23072	99	942	940,2	cyclictest	8921	-21	turbostat	00:29:55	3
23054	99	934	931,3	cyclictest	21096	-21	turbostat	23:04:55	0
23054	99	922	921,1	cyclictest	20505	-21	kworker/0:0+events	22:26:36	0
23059	99	916	914,2	cyclictest	26023	-21	turbostat	23:09:55	1
23054	99	916	915,1	cyclictest	3032	-21	kworker/0:1+events	19:13:40	0
23059	99	909	906,3	cyclictest	31086	-21	turbostat	19:34:55	1
23072	99	897	895,2	cyclictest	16449	-21	turbostat	00:04:55	3
23054	99	872	870,2	cyclictest	9061	-21	turbostat	23:24:55	0
23054	99	872	870,2	cyclictest	9061	-21	turbostat	23:24:55	0
23054	99	856	854,2	cyclictest	15590	-21	turbostat	21:54:55	0
23072	99	852	849,3	cyclictest	31372	-21	turbostat	00:19:55	3
23067	99	681	678,3	cyclictest	16116	-21	turbostat	20:14:55	2
23072	99	661	659,2	cyclictest	31086	-21	turbostat	19:34:55	3
23072	99	643	642,1	cyclictest	18255	-21	turbostat	21:24:55	3
23059	99	642	639,3	cyclictest	19081	-21	turbostat	23:34:55	1
23067	99	574	573,1	cyclictest	27030	-21	turbostat	19:24:55	2
23067	99	567	566,1	cyclictest	29029	-21	turbostat	23:44:55	2
23072	99	550	547,3	cyclictest	28536	-21	turbostat	22:39:55	3
23072	99	536	535,1	cyclictest	18800	-21	turbostat	00:39:55	3
23059	99	535	532,3	cyclictest	30272	-21	turbostat	20:49:55	1
23072	99	501	500,1	cyclictest	28246	-21	turbostat	20:44:55	3
23059	99	497	496,1	cyclictest	8921	-21	turbostat	00:29:55	1
23054	99	348	346,2	cyclictest	3212	-21	turbostat	22:14:55	0
23067	99	25	25,0	cyclictest	439	-21	hwrng	00:37:41	2
23072	99	20	20,0	cyclictest	439	-21	hwrng	22:30:43	3
23067	99	19	19,0	cyclictest	439	-21	hwrng	23:21:55	2
23067	99	19	19,0	cyclictest	439	-21	hwrng	23:21:55	2
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17