OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Fri Dec 26, 2025 12:43:32)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
885782	2	94	0,1	sleep7	885783	-21	cat	09:19:15	7
714595	2	65	0,1	sleep6	0	-21	swapper/6	07:49:24	6
619806	2	60	48,8	sleep5	0	-21	swapper/5	07:04:12	5
629076	2	59	48,8	sleep4	0	-21	swapper/4	07:09:00	4
619807	2	56	45,7	sleep6	0	-21	swapper/6	07:04:13	6
628810	2	53	17,31	sleep7	0	-21	swapper/7	07:05:18	7
629374	99	47	41,5	cyclictest	1117071	-21	latency_hist	11:19:03	7
629365	99	43	40,2	cyclictest	1166137	-21	kworker/u16:2+events_unbound	11:54:05	5
629374	99	42	34,7	cyclictest	855890	-21	latency_hist	09:04:03	7
629374	99	40	34,5	cyclictest	1262024	-21	latency_hist	12:34:03	7
629374	99	40	0,38	cyclictest	1223421	-21	latency_hist	12:14:03	7
629365	99	39	36,2	cyclictest	1043036	-21	kworker/u16:1+events_unbound	10:54:18	5
629365	99	38	35,2	cyclictest	1246487	-21	kworker/u16:4+events_unbound	12:29:21	5
629365	99	36	34,2	cyclictest	953186	-21	kworker/u16:2+flush-8:0	09:54:18	5
629365	99	36	34,2	cyclictest	953186	-21	kworker/u16:2+flush-8:0	09:54:18	5
629374	99	35	30,4	cyclictest	1175058	-21	latency_hist	11:49:03	7
629365	99	35	33,2	cyclictest	681996	-21	kworker/u16:1+events_unbound	08:53:14	5
629365	99	35	33,2	cyclictest	609737	-21	kworker/u16:0+events_unbound	07:42:14	5
629365	99	34	32,2	cyclictest	681996	-21	kworker/u16:1+events_unbound	08:01:15	5
629365	99	34	31,2	cyclictest	681996	-21	kworker/u16:1+events_unbound	08:14:21	5
629365	99	34	31,2	cyclictest	1088503	-21	kworker/u16:3+events_unbound	11:11:13	5
629365	99	34	31,2	cyclictest	1001490	-21	kworker/u16:0+events_unbound	10:24:05	5
629365	99	33	31,2	cyclictest	923935	-21	kworker/u16:3+events_unbound	09:44:19	5
629365	99	33	31,2	cyclictest	730192	-21	kworker/u16:4+events_unbound	08:05:14	5
629365	99	33	31,2	cyclictest	681996	-21	kworker/u16:1+events_unbound	07:48:14	5
629365	99	33	30,2	cyclictest	1088503	-21	kworker/u16:3+events_unbound	11:04:16	5
629365	99	32	30,2	cyclictest	1185849	-21	kworker/u16:1+events_unbound	12:17:22	5
629365	99	32	29,2	cyclictest	730192	-21	kworker/u16:4+events_unbound	08:24:21	5
629365	99	31	29,2	cyclictest	903436	-21	kworker/u16:1+flush-8:0	09:39:19	5
629365	99	31	29,2	cyclictest	865598	-21	kworker/u16:0+events_unbound	09:35:19	5
629365	99	31	29,2	cyclictest	798107	-21	kworker/u16:2+flush-8:0	08:38:14	5
629365	99	31	29,2	cyclictest	1059130	-21	kworker/u16:2+events_unbound	10:49:33	5
629365	99	31	28,2	cyclictest	681996	-21	kworker/u16:1+events_unbound	08:11:27	5
629365	99	31	28,2	cyclictest	662435	-21	kworker/u16:3+flush-8:0	07:29:34	5
629365	99	30	28,2	cyclictest	865686	-21	kworker/u16:3+events_unbound	09:12:14	5
629365	99	30	28,2	cyclictest	730192	-21	kworker/u16:4+flush-8:0	08:19:33	5
629365	99	30	28,2	cyclictest	730192	-21	kworker/u16:4+flush-8:0	08:19:33	5
629365	99	30	28,2	cyclictest	1242744	-21	kworker/u16:0+events_unbound	12:27:22	5
629365	99	30	28,2	cyclictest	1204179	-21	kworker/u16:3+events_unbound	12:23:14	5
629365	99	30	28,2	cyclictest	1204179	-21	kworker/u16:3+events_unbound	12:23:14	5
629365	99	30	28,2	cyclictest	1185849	-21	kworker/u16:1+events_unbound	12:10:22	5
629365	99	30	28,2	cyclictest	1125800	-21	kworker/u16:0+events_unbound	11:27:14	5
629365	99	30	28,2	cyclictest	1043036	-21	kworker/u16:1+events_unbound	11:44:05	5
629365	99	30	28,2	cyclictest	1043036	-21	kworker/u16:1+events_unbound	10:43:14	5
629365	99	30	26,3	cyclictest	662435	-21	kworker/u16:3+events_unbound	07:49:26	5
629365	99	29	27,2	cyclictest	730192	-21	kworker/u16:4+events_unbound	10:15:15	5
629365	99	29	27,2	cyclictest	681996	-21	kworker/u16:1+events_unbound	08:29:34	5
629365	99	29	26,2	cyclictest	1166137	-21	kworker/u16:2+flush-8:0	11:49:20	5
629369	99	28	25,2	cyclictest	865598	-21	kworker/u16:0+events_unbound	09:29:16	6
629369	99	28	25,2	cyclictest	845235	-21	kworker/u16:0+events_unbound	08:59:16	6
629369	99	28	25,2	cyclictest	609737	-21	kworker/u16:0+events_unbound	08:09:18	6
629369	99	28	23,2	cyclictest	1043036	-21	kworker/u16:1+events_unbound	10:49:19	6
629365	99	28	26,2	cyclictest	923935	-21	kworker/u16:3+events_unbound	10:38:15	5
629365	99	28	26,2	cyclictest	923935	-21	kworker/u16:3+events_unbound	09:59:31	5
629365	99	28	26,2	cyclictest	845235	-21	kworker/u16:0+events_unbound	08:59:22	5
629365	99	28	26,2	cyclictest	1204179	-21	kworker/u16:3+events_unbound	12:04:24	5
629365	99	28	26,2	cyclictest	1185849	-21	kworker/u16:1+events_unbound	12:01:13	5
629365	99	28	26,2	cyclictest	1001490	-21	kworker/u16:0+events_unbound	10:22:23	5
629365	99	28	25,2	cyclictest	609737	-21	kworker/u16:0+events_unbound	07:19:22	5
629365	99	27	25,2	cyclictest	798107	-21	kworker/u16:2+events_unbound	09:19:39	5
629365	99	27	25,2	cyclictest	609737	-21	kworker/u16:0+events_unbound	08:39:26	5
629365	99	27	25,2	cyclictest	609737	-21	kworker/u16:0+events_unbound	07:10:15	5
629365	99	27	25,2	cyclictest	1059130	-21	kworker/u16:2+events_unbound	11:24:01	5
629365	99	27	24,2	cyclictest	953186	-21	kworker/u16:2+events_unbound	10:10:14	5
629365	99	26	24,2	cyclictest	730192	-21	kworker/u16:4+events_unbound	09:26:22	5
629365	99	26	24,2	cyclictest	619493	-21	kworker/u16:1+events_unbound	07:17:15	5
1118066	2	26	0,1	chrt	1118067	-21	fschecks_count	11:19:17	7
629365	99	25	23,2	cyclictest	865598	-21	kworker/u16:0+events_unbound	09:53:51	5
629369	99	24	22,2	cyclictest	1001490	-21	kworker/u16:0+events_unbound	11:04:30	6
629369	99	24	21,2	cyclictest	1125800	-21	kworker/u16:0+events_unbound	11:59:15	6
629365	99	24	22,2	cyclictest	855881	-21	kworker/u16:1+events_unbound	09:05:23	5
629365	99	24	22,2	cyclictest	730192	-21	kworker/u16:4+events_unbound	10:04:26	5
629365	99	24	22,2	cyclictest	662435	-21	kworker/u16:3+flush-8:0	07:54:24	5
629365	99	24	22,2	cyclictest	609737	-21	kworker/u16:0+events_unbound	07:27:23	5
629365	99	24	22,2	cyclictest	1125800	-21	kworker/u16:0+flush-8:0	11:39:05	5
629369	99	23	20,2	cyclictest	1126923	-21	kworker/u16:4+events_unbound	11:54:18	6
629369	99	22	20,2	cyclictest	865598	-21	kworker/u16:0+flush-8:0	09:53:58	6
785202	2	21	0,0	sleep7	785204	-21	kthreadcore	08:24:26	7
629374	99	21	15,5	cyclictest	904376	-21	latency_hist	09:29:04	7
629369	99	21	19,2	cyclictest	923935	-21	kworker/u16:3+events_unbound	09:54:34	6
629369	99	21	19,2	cyclictest	923935	-21	kworker/u16:3+events_unbound	09:54:34	6
629369	99	21	19,2	cyclictest	923479	-21	kworker/u16:2+flush-8:0	09:38:59	6
629365	99	21	19,2	cyclictest	923935	-21	kworker/u16:3+events_unbound	10:44:15	5
629365	99	21	19,2	cyclictest	923935	-21	kworker/u16:3+events_unbound	10:44:15	5
629365	99	21	19,2	cyclictest	681996	-21	kworker/u16:1+events_unbound	08:54:17	5
629365	99	21	19,2	cyclictest	1125800	-21	kworker/u16:0+events_unbound	11:38:03	5
629365	99	21	18,2	cyclictest	865598	-21	kworker/u16:0+events_unbound	09:32:23	5
629362	99	21	18,2	cyclictest	730192	-21	kworker/u16:4+flush-8:0	10:04:29	4
629369	99	20	18,2	cyclictest	1166137	-21	kworker/u16:2+events_unbound	11:48:58	6
629365	99	20	18,2	cyclictest	923935	-21	kworker/u16:3+flush-8:0	10:33:54	5
629365	99	20	18,2	cyclictest	1125800	-21	kworker/u16:0+events_unbound	11:29:31	5
629362	99	20	17,3	cyclictest	798107	-21	kworker/u16:2+flush-8:0	08:34:22	4
629374	99	19	15,3	cyclictest	1267	-21	snmpd	11:56:15	7
629369	99	19	17,2	cyclictest	953186	-21	kworker/u16:2+flush-8:0	10:08:58	6
629365	99	19	17,2	cyclictest	798107	-21	kworker/u16:2+events_unbound	08:45:23	5
629365	99	19	17,2	cyclictest	1204179	-21	kworker/u16:3+flush-8:0	12:34:23	5
629365	99	19	17,2	cyclictest	1043036	-21	kworker/u16:1+events_unbound	11:14:34	5
629369	99	18	16,2	cyclictest	798107	-21	kworker/u16:2+flush-8:0	08:58:58	6
629369	99	18	16,2	cyclictest	1204179	-21	kworker/u16:3+events_unbound	12:24:18	6
629369	99	18	16,2	cyclictest	1043036	-21	kworker/u16:1+events_unbound	10:44:22	6
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17