OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack8slot6s.osadl.org (updated Sun Aug 16, 2026 00:43:42)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
22917	99	1052	1049,3	cyclictest	8628	-21	turbostat	19:59:55	0
22926	99	1051	1049,2	cyclictest	31101	-21	kworker/2:1+events	21:38:07	2
22926	99	1051	1049,2	cyclictest	30312	-21	kworker/2:0+events	21:46:27	2
22926	99	1051	1049,2	cyclictest	17606	-21	kworker/2:1+events	00:37:35	2
22926	99	1050	1048,2	cyclictest	31101	-21	kworker/2:1+events	21:43:05	2
22926	99	1050	1048,2	cyclictest	30312	-21	kworker/2:0+events	21:56:08	2
22926	99	1050	1048,2	cyclictest	30312	-21	kworker/2:0+events	21:52:56	2
22926	99	1050	1048,2	cyclictest	11089	-21	turbostat	23:59:55	2
22930	99	1049	1047,2	cyclictest	26887	-21	turbostat	19:24:55	3
22926	99	1048	1046,2	cyclictest	9034	-21	kworker/2:0+events	00:26:59	2
22926	99	1048	1046,2	cyclictest	17606	-21	kworker/2:1+events	22:25:42	2
22926	99	1047	1045,2	cyclictest	29345	-21	kworker/2:2+events	20:50:21	2
22917	99	1046	1042,3	cyclictest	24871	-21	turbostat	19:19:55	0
22930	99	1044	1042,2	cyclictest	4240	-21	kworker/3:1+events	22:11:05	3
22930	99	1043	1041,2	cyclictest	5069	-21	kworker/3:1+events	19:38:48	3
22926	99	1042	1040,2	cyclictest	19987	-21	kworker/2:1+events	19:27:35	2
22930	99	1041	1039,2	cyclictest	14135	-21	turbostat	00:34:55	3
22922	99	1038	1037,1	cyclictest	20329	-21	kworker/1:0+events	23:02:42	1
22930	99	1036	1035,1	cyclictest	14838	-21	kworker/3:1+events	22:34:22	3
22917	99	1036	1032,3	cyclictest	28917	-21	turbostat	19:29:54	0
22917	99	1035	1034,1	cyclictest	26103	-21	kworker/0:2+events	23:46:15	0
22922	99	1030	1029,1	cyclictest	24550	-21	kworker/1:1+events	19:18:58	1
22930	99	1026	1024,2	cyclictest	9510	-21	kworker/3:0+events	22:22:11	3
22922	99	1021	1020,1	cyclictest	30616	-21	kworker/1:2+events	21:04:18	1
22930	99	1020	1019,1	cyclictest	11750	-21	kworker/3:1+events	23:28:24	3
22926	99	1019	1017,2	cyclictest	11951	-21	turbostat	21:19:55	2
22922	99	1019	1016,3	cyclictest	15001	-21	turbostat	22:59:55	1
22917	99	1016	1014,2	cyclictest	19652	-21	turbostat	21:59:55	0
22930	99	1007	1004,1	cyclictest	3185	-21	kworker/3:2+events	23:17:29	3
22930	99	1007	1003,2	cyclictest	3185	-21	kworker/3:2+events	00:12:10	3
22930	99	1005	1004,1	cyclictest	30843	-21	turbostat	20:54:55	3
22930	99	1003	1002,1	cyclictest	17313	-21	kworker/3:1+events	21:34:38	3
22922	99	1003	1000,3	cyclictest	27073	-21	turbostat	21:34:55	1
22930	99	1001	1000,1	cyclictest	3185	-21	kworker/3:2+events	00:23:57	3
22922	99	1001	1000,1	cyclictest	5114	-21	kworker/1:2+events	22:47:11	1
22922	99	998	997,1	cyclictest	28575	-21	kworker/1:1+events	20:47:24	1
22930	99	995	994,1	cyclictest	27507	-21	kworker/3:2+events	22:39:02	3
22922	99	994	991,3	cyclictest	9110	-21	turbostat	00:29:55	1
22917	99	994	992,2	cyclictest	12664	-21	turbostat	20:09:55	0
22930	99	993	991,2	cyclictest	24106	-21	kworker/3:0+events	22:08:49	3
22922	99	993	992,1	cyclictest	1098	-21	kworker/1:0+events	21:05:45	1
22922	99	992	991,1	cyclictest	21294	-21	kworker/1:1+events	23:39:10	1
22922	99	990	989,1	cyclictest	5670	-21	kworker/1:0+events	19:13:42	1
22930	99	988	987,1	cyclictest	3185	-21	kworker/3:2+events	23:24:12	3
22922	99	987	986,1	cyclictest	6566	-21	kworker/1:1+events	21:13:37	1
22922	99	986	985,1	cyclictest	24755	-21	kworker/1:2+events	20:37:33	1
22922	99	985	984,1	cyclictest	28575	-21	kworker/1:1+events	20:42:31	1
22930	99	984	984,0	cyclictest	17313	-21	kworker/3:1+events	21:23:24	3
22930	99	984	983,1	cyclictest	2117	-21	kworker/3:1+events	22:40:53	3
22917	99	984	982,2	cyclictest	28937	-21	kworker/0:0+events	23:51:06	0
22930	99	983	982,1	cyclictest	3185	-21	kworker/3:2+events	00:18:12	3
22930	99	983	982,1	cyclictest	26862	-21	kworker/3:0+events	20:30:55	3
22930	99	983	982,1	cyclictest	26862	-21	kworker/3:0+events	19:30:22	3
22922	99	983	982,1	cyclictest	14508	-21	kworker/1:0+events	21:17:43	1
22917	99	983	982,1	cyclictest	32600	-21	kworker/0:1+events	23:14:43	0
22930	99	982	981,1	cyclictest	2117	-21	kworker/3:1+events	22:50:43	3
22930	99	982	981,1	cyclictest	17313	-21	kworker/3:1+events	21:27:57	3
22930	99	982	981,1	cyclictest	14641	-21	kworker/3:2+events	20:14:10	3
22922	99	982	981,1	cyclictest	30616	-21	kworker/1:2+events	20:56:05	1
22930	99	981	980,1	cyclictest	3185	-21	kworker/3:2+events	23:30:22	3
22930	99	981	980,1	cyclictest	3185	-21	kworker/3:2+events	00:06:05	3
22930	99	976	975,1	cyclictest	16160	-21	kworker/3:3+events	00:04:53	3
22930	99	976	975,1	cyclictest	15784	-21	kworker/3:0+events	22:59:12	3
22930	99	975	974,1	cyclictest	4240	-21	kworker/3:1+events	22:15:44	3
22930	99	975	974,1	cyclictest	26862	-21	kworker/3:0+events	20:26:06	3
22917	99	974	973,1	cyclictest	4833	-21	kworker/0:1+events	19:50:55	0
22930	99	973	973,0	cyclictest	24106	-21	kworker/3:0+events	22:02:54	3
22917	99	973	972,1	cyclictest	14238	-21	kworker/0:2+events	19:41:05	0
22917	99	973	972,1	cyclictest	10536	-21	kworker/0:1+events	23:59:21	0
22917	99	972	971,1	cyclictest	8264	-21	kworker/0:0+events	23:09:32	0
22930	99	969	968,1	cyclictest	26862	-21	kworker/3:0+events	20:06:27	3
22930	99	969	968,1	cyclictest	26862	-21	kworker/3:0+events	20:00:44	3
22917	99	969	967,2	cyclictest	32131	-21	turbostat	22:44:55	0
22930	99	968	967,1	cyclictest	26862	-21	kworker/3:0+events	20:16:50	3
22930	99	966	964,2	cyclictest	16922	-21	kworker/3:1+events	20:22:34	3
22917	99	964	963,1	cyclictest	28937	-21	kworker/0:0+events	23:41:58	0
22917	99	964	961,2	cyclictest	14238	-21	kworker/0:2+events	19:49:44	0
22926	99	934	932,2	cyclictest	23620	-21	turbostat	23:39:55	2
22926	99	931	927,4	cyclictest	2464	-21	turbostat	21:04:55	2
22926	99	930	928,2	cyclictest	24649	-21	turbostat	22:04:55	2
22930	99	921	919,2	cyclictest	20401	-21	turbostat	23:04:55	3
22926	99	911	909,2	cyclictest	22002	-21	turbostat	21:29:55	2
22930	99	897	896,1	cyclictest	6606	-21	turbostat	19:54:55	3
22922	99	895	891,4	cyclictest	16685	-21	turbostat	20:19:55	1
22917	99	883	882,1	cyclictest	5184	-21	turbostat	22:49:55	0
22930	99	875	873,2	cyclictest	28827	-21	turbostat	20:49:55	3
22926	99	864	863,1	cyclictest	8229	-21	turbostat	23:24:55	2
22917	99	853	850,2	cyclictest	7296	-21	turbostat	22:19:55	0
22930	99	696	694,2	cyclictest	11089	-21	turbostat	23:59:55	3
22917	99	670	667,2	cyclictest	20401	-21	turbostat	23:04:55	0
22930	99	665	663,2	cyclictest	11951	-21	turbostat	21:19:55	3
22926	99	655	653,2	cyclictest	27073	-21	turbostat	21:34:55	2
22930	99	645	642,3	cyclictest	5184	-21	turbostat	22:49:55	3
22930	99	640	638,2	cyclictest	30670	-21	turbostat	23:14:55	3
22917	99	615	611,4	cyclictest	486	-21	turbostat	19:39:55	0
22930	99	612	609,3	cyclictest	8628	-21	turbostat	19:59:55	3
22930	99	594	591,3	cyclictest	28917	-21	turbostat	19:29:55	3
22930	99	577	574,3	cyclictest	19652	-21	turbostat	21:59:55	3
22917	99	576	573,3	cyclictest	23620	-21	turbostat	23:39:55	0
22922	99	571	568,3	cyclictest	30843	-21	turbostat	20:54:55	1
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17