OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack8slot6s.osadl.org (updated Thu Sep 17, 2026 00:43:41)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
9060	99	1052	1049,2	cyclictest	24565	-21	kworker/3:1+events	23:03:03	3
9058	99	1052	1051,1	cyclictest	7183	-21	kworker/2:0+events	00:08:36	2
9058	99	1052	1050,2	cyclictest	22149	-21	kworker/2:1+events	21:34:37	2
9058	99	1052	1050,2	cyclictest	22149	-21	kworker/2:1+events	19:59:45	2
9058	99	1051	1048,3	cyclictest	22149	-21	kworker/2:1+events	19:50:44	2
9058	99	1050	1048,2	cyclictest	31801	-21	kworker/2:2+events	21:54:02	2
9058	99	1050	1048,2	cyclictest	22149	-21	kworker/2:1+events	19:46:05	2
9058	99	1049	1047,2	cyclictest	22149	-21	kworker/2:1+events	20:01:27	2
9058	99	1046	1044,2	cyclictest	7183	-21	kworker/2:0+events	23:31:16	2
9058	99	1043	1041,2	cyclictest	1280	-21	kworker/2:1+events	23:29:09	2
9058	99	1042	1041,1	cyclictest	22149	-21	kworker/2:1+events	21:24:43	2
9060	99	1038	1036,2	cyclictest	29675	-21	turbostat	21:49:54	3
9058	99	1033	1031,2	cyclictest	31801	-21	kworker/2:2+events	21:58:28	2
9060	99	1029	1027,2	cyclictest	22843	-21	kworker/3:1+events	23:24:01	3
9058	99	1027	1024,3	cyclictest	26351	-21	turbostat	21:14:54	2
9053	99	1023	1021,2	cyclictest	16108	-21	kworker/1:0+events	22:15:44	1
9053	99	1022	1020,2	cyclictest	24639	-21	turbostat	21:44:54	1
9045	99	1021	1020,1	cyclictest	8684	-21	kworker/0:0+events	00:16:07	0
9045	99	1021	1020,1	cyclictest	11226	-21	kworker/0:1+events	19:17:57	0
9045	99	1021	1020,1	cyclictest	11226	-21	kworker/0:1+events	19:17:56	0
9045	99	1020	1019,1	cyclictest	20284	-21	kworker/0:0+events	19:26:26	0
9053	99	1016	1014,2	cyclictest	16108	-21	kworker/1:0+events	22:13:18	1
9045	99	1015	1014,1	cyclictest	19117	-21	kworker/0:0+events	22:32:54	0
9060	99	1012	1011,1	cyclictest	11320	-21	kworker/3:0+events	23:56:29	3
9053	99	1011	1009,2	cyclictest	19088	-21	turbostat	19:39:56	1
9045	99	1008	1006,1	cyclictest	31886	-21	kworker/0:2+events	00:26:27	0
9060	99	1006	1005,1	cyclictest	13399	-21	turbostat	20:44:55	3
9045	99	1006	1005,1	cyclictest	5384	-21	kworker/0:2+events	22:42:10	0
9045	99	1006	1004,2	cyclictest	7327	-21	turbostat	21:59:55	0
9060	99	1005	1003,2	cyclictest	3552	-21	kworker/3:2+pm	23:07:32	3
9045	99	1005	1003,2	cyclictest	8684	-21	kworker/0:0+events	00:01:26	0
9053	99	1003	1000,3	cyclictest	23681	-21	turbostat	23:19:55	1
9053	99	1000	999,1	cyclictest	2888	-21	kworker/1:2+events	22:24:36	1
9060	99	999	998,1	cyclictest	18836	-21	kworker/3:2+events	23:41:05	3
9060	99	996	996,0	cyclictest	11320	-21	kworker/3:0+events	23:50:21	3
9053	99	996	995,1	cyclictest	16108	-21	kworker/1:0+events	22:09:24	1
9060	99	995	994,1	cyclictest	28126	-21	kworker/3:2+events	23:19:41	3
9045	99	995	994,1	cyclictest	30936	-21	kworker/0:1+events	22:53:38	0
9053	99	994	993,1	cyclictest	12726	-21	kworker/1:1+events	22:02:54	1
9060	99	993	991,2	cyclictest	14584	-21	turbostat	00:14:56	3
9045	99	992	990,2	cyclictest	20823	-21	kworker/0:0+events	19:39:36	0
9045	99	991	990,1	cyclictest	19117	-21	kworker/0:0+events	22:36:03	0
9060	99	989	988,1	cyclictest	20526	-21	kworker/3:2+events	21:44:34	3
9045	99	989	988,1	cyclictest	29607	-21	kworker/0:1+events	21:48:25	0
9045	99	989	987,2	cyclictest	19117	-21	kworker/0:0+events	21:37:53	0
9060	99	988	987,1	cyclictest	24565	-21	kworker/3:1+events	22:55:20	3
9058	99	988	987,1	cyclictest	31801	-21	kworker/2:2+events	21:29:47	2
9058	99	985	983,2	cyclictest	4513	-21	turbostat	00:04:56	2
9045	99	985	984,1	cyclictest	10564	-21	kworker/0:1+events	00:38:35	0
9060	99	983	982,1	cyclictest	20526	-21	kworker/3:2+events	21:01:28	3
9060	99	982	981,1	cyclictest	8622	-21	kworker/3:0+events	20:56:36	3
9060	99	982	981,1	cyclictest	6565	-21	kworker/3:1+events	20:25:22	3
9060	99	982	981,1	cyclictest	6565	-21	kworker/3:1+events	20:24:26	3
9060	99	982	981,1	cyclictest	22843	-21	kworker/3:1+events	23:14:08	3
9060	99	981	980,1	cyclictest	6565	-21	kworker/3:1+events	20:35:21	3
9060	99	981	980,1	cyclictest	26010	-21	kworker/3:0+events	21:08:28	3
9053	99	981	980,1	cyclictest	16108	-21	kworker/1:0+events	20:45:45	1
9060	99	980	979,1	cyclictest	31186	-21	kworker/3:0+events	20:17:33	3
9060	99	980	979,1	cyclictest	22860	-21	kworker/3:1+events	19:44:13	3
9060	99	977	976,1	cyclictest	6565	-21	kworker/3:1+events	20:54:36	3
9060	99	977	976,1	cyclictest	24084	-21	kworker/3:1+events	23:46:59	3
9060	99	976	975,1	cyclictest	31186	-21	kworker/3:0+events	20:06:27	3
9060	99	975	974,1	cyclictest	6565	-21	kworker/3:1+events	20:31:28	3
9060	99	974	973,1	cyclictest	31186	-21	kworker/3:0+events	20:12:32	3
9060	99	973	973,0	cyclictest	26010	-21	kworker/3:0+events	21:10:21	3
9060	99	972	970,1	cyclictest	11320	-21	kworker/3:0+events	23:39:54	3
9045	99	972	971,1	cyclictest	20277	-21	kworker/0:1+events	00:23:40	0
9058	99	970	968,2	cyclictest	31801	-21	kworker/2:2+events	21:15:45	2
9060	99	968	966,2	cyclictest	14162	-21	turbostat	21:34:55	3
9060	99	957	955,2	cyclictest	32389	-21	turbostat	22:24:55	3
9053	99	949	947,2	cyclictest	9552	-21	turbostat	00:09:55	1
9053	99	937	935,2	cyclictest	6381	-21	turbostat	23:34:55	1
9045	99	921	919,2	cyclictest	5384	-21	kworker/0:2+events	22:45:22	0
9045	99	919	918,1	cyclictest	19117	-21	kworker/0:0+events	22:26:39	0
9045	99	917	915,2	cyclictest	830	-21	turbostat	20:14:55	0
9045	99	916	915,1	cyclictest	14672	-21	kworker/0:2+events	19:21:49	0
9045	99	916	915,1	cyclictest	14672	-21	kworker/0:2+events	19:21:48	0
9045	99	908	907,1	cyclictest	5202	-21	kworker/0:0+events	00:34:49	0
9053	99	903	901,2	cyclictest	24334	-21	turbostat	21:09:55	1
9060	99	900	898,2	cyclictest	4080	-21	turbostat	21:24:54	3
9045	99	900	899,1	cyclictest	20284	-21	kworker/0:0+events	19:31:16	0
9045	99	899	898,1	cyclictest	8878	-21	kworker/0:2+events	19:10:22	0
9045	99	895	893,2	cyclictest	17854	-21	turbostat	20:54:54	0
9045	99	889	888,1	cyclictest	3204	-21	turbostat	22:59:55	0
9045	99	874	871,3	cyclictest	11369	-21	turbostat	20:39:54	0
9045	99	872	869,3	cyclictest	29191	-21	turbostat	20:04:55	0
9045	99	869	866,3	cyclictest	31894	-21	turbostat	23:59:55	0
9053	99	868	866,2	cyclictest	30153	-21	turbostat	22:54:56	1
9058	99	864	862,2	cyclictest	15417	-21	turbostat	20:49:55	2
9045	99	864	862,2	cyclictest	13666	-21	turbostat	23:09:56	0
9053	99	857	854,3	cyclictest	28723	-21	turbostat	23:24:56	1
9058	99	701	697,4	cyclictest	28723	-21	turbostat	23:24:56	2
9058	99	698	694,4	cyclictest	29675	-21	turbostat	21:49:54	2
9060	99	696	693,3	cyclictest	29191	-21	turbostat	20:04:55	3
9045	99	694	691,3	cyclictest	24639	-21	turbostat	21:44:55	0
9060	99	688	687,1	cyclictest	9552	-21	turbostat	00:09:55	3
9045	99	678	676,2	cyclictest	14584	-21	turbostat	00:14:55	0
9060	99	676	675,1	cyclictest	6381	-21	turbostat	23:34:55	3
9045	99	653	651,2	cyclictest	14162	-21	turbostat	21:34:54	0
9053	99	645	642,3	cyclictest	7327	-21	turbostat	21:59:55	1
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17