OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Sat Nov 15, 2025 12:43:33)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
783804	2	63	53,7	sleep5	0	-21	swapper/5	15:23:22	5
784202	99	52	47,4	cyclictest	1151896	-21	latency_hist	18:23:58	7
774234	2	51	40,7	sleep6	0	-21	swapper/6	15:19:06	6
784202	99	48	43,4	cyclictest	970071	-21	latency_hist	16:53:59	7
784202	99	48	43,4	cyclictest	1061084	-21	latency_hist	17:38:58	7
784188	99	47	44,2	cyclictest	1060740	-21	kworker/u16:3+events_unbound	17:37:01	4
784194	99	45	43,2	cyclictest	3580970	-21	kworker/u16:0+flush-8:0	15:24:28	6
784194	99	44	42,2	cyclictest	879658	-21	kworker/u16:0+flush-8:0	16:13:58	6
784188	99	43	40,2	cyclictest	1272693	-21	kworker/u16:0+events_unbound	19:29:00	4
783545	2	43	9,30	sleep7	0	-21	swapper/7	15:19:47	7
784188	99	42	40,2	cyclictest	828561	-21	kworker/u16:2+events_unbound	15:55:49	4
784202	99	41	40,0	cyclictest	0	-21	swapper/7	18:14:30	7
784202	99	41	36,4	cyclictest	1343554	-21	latency_hist	19:58:58	7
784188	99	41	38,2	cyclictest	828561	-21	kworker/u16:2+events_unbound	16:24:01	4
784188	99	40	38,2	cyclictest	1151870	-21	kworker/u16:1+flush-8:0	18:40:49	4
784188	99	40	38,2	cyclictest	1141193	-21	kworker/u16:3+flush-8:0	18:31:48	4
784188	99	40	37,2	cyclictest	849108	-21	kworker/u16:3+flush-8:0	15:59:01	4
784188	99	39	37,2	cyclictest	939645	-21	kworker/u16:1+events_unbound	16:44:27	4
784188	99	39	37,2	cyclictest	1202004	-21	kworker/u16:5+flush-8:0	19:08:48	4
784188	99	39	37,2	cyclictest	1191924	-21	kworker/u16:0+flush-8:0	18:48:59	4
784188	99	39	37,2	cyclictest	1000263	-21	kworker/u16:1+flush-8:0	17:31:01	4
784194	99	38	36,2	cyclictest	1353292	-21	kworker/u16:4+flush-8:0	20:34:25	6
784188	99	38	36,2	cyclictest	808099	-21	kworker/u16:1+events_unbound	15:38:37	4
784188	99	38	36,2	cyclictest	808099	-21	kworker/u16:1+events_unbound	15:38:37	4
784188	99	38	36,2	cyclictest	1272693	-21	kworker/u16:0+flush-8:0	20:19:24	4
784202	99	37	32,4	cyclictest	1414173	-21	latency_hist	20:33:59	7
784194	99	37	35,2	cyclictest	819014	-21	kworker/u16:0+events_unbound	15:50:48	6
784194	99	37	35,2	cyclictest	1333146	-21	kworker/u16:3+events_unbound	19:56:08	6
784188	99	37	35,2	cyclictest	1040690	-21	kworker/u16:4+events_unbound	17:39:55	4
784202	99	36	0,34	cyclictest	1353626	-21	latency_hist	20:03:58	7
784194	99	36	34,2	cyclictest	879658	-21	kworker/u16:0+events_unbound	17:31:48	6
784194	99	36	34,2	cyclictest	1404074	-21	kworker/u16:1+flush-8:0	20:44:26	6
784194	99	36	34,2	cyclictest	1141193	-21	kworker/u16:3+events_unbound	18:20:49	6
784194	99	36	33,2	cyclictest	1151870	-21	kworker/u16:1+flush-8:0	18:44:08	6
784188	99	36	34,2	cyclictest	1202004	-21	kworker/u16:5+events_unbound	19:41:22	4
784188	99	36	34,2	cyclictest	1141193	-21	kworker/u16:3+flush-8:0	18:25:00	4
783657	2	36	26,7	sleep4	0	-21	swapper/4	15:21:20	4
784188	99	35	33,2	cyclictest	1404074	-21	kworker/u16:1+flush-8:0	20:53:45	4
784188	99	35	33,2	cyclictest	1202004	-21	kworker/u16:5+flush-8:0	19:01:48	4
784202	99	34	28,5	cyclictest	1212484	-21	latency_hist	18:53:59	7
784194	99	34	32,2	cyclictest	1000263	-21	kworker/u16:1+events_unbound	17:59:47	6
784194	99	34	31,2	cyclictest	1040690	-21	kworker/u16:4+events_unbound	17:39:08	6
784188	99	34	32,2	cyclictest	793869	-21	kworker/u16:4+events_unbound	15:25:49	4
784188	99	34	32,2	cyclictest	1313642	-21	kworker/u16:2+events_unbound	20:24:28	4
784194	99	33	31,2	cyclictest	990131	-21	kworker/u16:3+events_unbound	17:11:21	6
784194	99	33	31,2	cyclictest	808168	-21	kworker/u16:6+events_unbound	16:29:37	6
784194	99	33	31,2	cyclictest	1202004	-21	kworker/u16:5+flush-8:0	19:11:52	6
784188	99	33	31,2	cyclictest	990131	-21	kworker/u16:3+events_unbound	17:19:33	4
784202	99	32	27,4	cyclictest	1393994	-21	latency_hist	20:23:59	7
784194	99	32	30,2	cyclictest	828561	-21	kworker/u16:2+events_unbound	16:14:48	6
784194	99	32	30,2	cyclictest	828561	-21	kworker/u16:2+events_unbound	16:05:12	6
784194	99	32	30,2	cyclictest	793869	-21	kworker/u16:4+events_unbound	15:30:13	6
784194	99	32	30,2	cyclictest	1313642	-21	kworker/u16:2+events_unbound	20:27:22	6
784194	99	32	30,2	cyclictest	1272693	-21	kworker/u16:0+flush-8:0	20:19:14	6
784194	99	32	30,2	cyclictest	1272693	-21	kworker/u16:0+events_unbound	19:27:47	6
784188	99	32	30,2	cyclictest	1202004	-21	kworker/u16:5+events_unbound	18:58:48	4
784188	99	32	29,2	cyclictest	1353292	-21	kworker/u16:4+flush-8:0	20:34:24	4
784188	99	32	28,3	cyclictest	808029	-21	packagekitd	15:30:46	4
784202	99	31	30,0	cyclictest	0	-21	swapper/7	16:39:24	7
784194	99	31	29,2	cyclictest	990131	-21	kworker/u16:3+events_unbound	17:05:49	6
784194	99	31	29,2	cyclictest	939645	-21	kworker/u16:1+flush-8:0	16:49:30	6
784194	99	31	29,2	cyclictest	1313642	-21	kworker/u16:2+events_unbound	20:12:21	6
784194	99	31	29,2	cyclictest	1202004	-21	kworker/u16:5+events_unbound	18:58:01	6
784194	99	31	29,2	cyclictest	1141193	-21	kworker/u16:3+events_unbound	19:06:08	6
784194	99	31	29,2	cyclictest	1040690	-21	kworker/u16:4+events_unbound	17:54:56	6
784188	99	31	29,2	cyclictest	879658	-21	kworker/u16:0+events_unbound	17:08:48	4
784188	99	31	29,2	cyclictest	879658	-21	kworker/u16:0+events_unbound	16:16:49	4
784188	99	31	29,2	cyclictest	819014	-21	kworker/u16:0+flush-8:0	15:49:18	4
784188	99	31	29,2	cyclictest	1435931	-21	kworker/u16:3+flush-8:0	20:44:47	4
784188	99	31	29,2	cyclictest	1353292	-21	kworker/u16:4+flush-8:0	20:18:58	4
784188	99	31	29,2	cyclictest	1101202	-21	kworker/u16:0+flush-8:0	17:58:54	4
784188	99	31	28,3	cyclictest	828561	-21	kworker/u16:2+flush-8:0	15:44:21	4
784202	99	30	29,0	cyclictest	0	-21	swapper/7	19:19:37	7
784202	99	30	29,0	cyclictest	0	-21	swapper/7	19:19:37	7
784202	99	30	28,1	cyclictest	0	-21	swapper/7	18:09:24	7
784194	99	30	28,2	cyclictest	939645	-21	kworker/u16:1+events_unbound	16:59:33	6
784194	99	30	28,2	cyclictest	879658	-21	kworker/u16:0+flush-8:0	17:26:57	6
784194	99	30	28,2	cyclictest	849108	-21	kworker/u16:3+events_unbound	16:00:56	6
784194	99	30	28,2	cyclictest	828561	-21	kworker/u16:2+events_unbound	15:42:06	6
784194	99	30	28,2	cyclictest	1423940	-21	kworker/u16:2+flush-8:0	20:43:00	6
784194	99	30	28,2	cyclictest	1404074	-21	kworker/u16:1+events_unbound	20:30:47	6
784194	99	30	28,2	cyclictest	1353292	-21	kworker/u16:4+events_unbound	20:02:42	6
784194	99	30	28,2	cyclictest	1313642	-21	kworker/u16:2+events_unbound	20:16:04	6
784194	99	30	28,2	cyclictest	1232289	-21	kworker/u16:1+flush-8:0	19:45:09	6
784194	99	30	28,2	cyclictest	1202004	-21	kworker/u16:5+events_unbound	19:29:41	6
784194	99	30	28,2	cyclictest	1141193	-21	kworker/u16:3+events_unbound	18:18:01	6
784194	99	30	27,2	cyclictest	1242596	-21	kworker/u16:0+flush-8:0	19:14:08	6
784188	99	30	28,2	cyclictest	808168	-21	kworker/u16:6+flush-8:0	16:34:35	4
784188	99	30	28,2	cyclictest	1202004	-21	kworker/u16:5+events_unbound	19:20:47	4
784188	99	30	28,2	cyclictest	1202004	-21	kworker/u16:5+events_unbound	19:20:47	4
784188	99	30	28,2	cyclictest	1162196	-21	kworker/u16:2+events_unbound	18:34:48	4
784188	99	30	28,2	cyclictest	1162196	-21	kworker/u16:2+events_unbound	18:34:48	4
784194	99	29	27,2	cyclictest	956020	-21	kworker/u16:3+events_unbound	16:47:56	6
784194	99	29	27,2	cyclictest	879658	-21	kworker/u16:0+flush-8:0	16:28:08	6
784194	99	29	27,2	cyclictest	793869	-21	kworker/u16:4+flush-8:0	15:37:01	6
784194	99	29	27,2	cyclictest	793869	-21	kworker/u16:4+flush-8:0	15:37:01	6
784194	99	29	27,2	cyclictest	1333146	-21	kworker/u16:3+flush-8:0	19:52:12	6
784194	99	29	27,2	cyclictest	1141193	-21	kworker/u16:3+flush-8:0	18:38:05	6
784194	99	29	27,2	cyclictest	1141193	-21	kworker/u16:3+flush-8:0	18:38:05	6
784194	99	29	27,2	cyclictest	1141193	-21	kworker/u16:3+events_unbound	18:42:50	6
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17