OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Mon Mar 02, 2026 00:43:36)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
3501685	2	56	31,8	sleep6	0	-21	swapper/6	19:02:13	6
3501757	2	54	43,8	sleep4	0	-21	swapper/4	19:03:16	4
3502156	99	50	46,3	cyclictest	4140993	-21	kworker/u16:4+flush-8:0	23:55:10	6
3502156	99	47	44,2	cyclictest	3453710	-21	kworker/u16:0+flush-8:0	19:39:40	6
3502152	99	47	45,2	cyclictest	3756260	-21	kworker/u16:3+flush-8:0	21:35:08	5
3502160	99	45	40,4	cyclictest	30669	-21	latency_hist	00:29:40	7
3502156	99	45	42,2	cyclictest	3483229	-21	kworker/u16:1+flush-8:0	19:29:40	6
3502156	99	43	40,2	cyclictest	3553642	-21	kworker/u16:2+flush-8:0	20:54:40	6
3502152	99	42	40,2	cyclictest	4021707	-21	kworker/u16:1+events_unbound	23:20:42	5
3502152	99	42	40,2	cyclictest	3950103	-21	kworker/u16:0+events_unbound	22:42:35	5
3501684	2	42	32,7	sleep5	0	-21	swapper/5	19:02:12	5
3502152	99	41	39,2	cyclictest	4021707	-21	kworker/u16:1+events_unbound	23:39:37	5
3502152	99	41	39,2	cyclictest	3950103	-21	kworker/u16:0+events_unbound	23:06:46	5
3502152	99	41	39,2	cyclictest	3950103	-21	kworker/u16:0+events_unbound	23:06:46	5
3502152	99	41	39,2	cyclictest	3878160	-21	kworker/u16:2+events_unbound	22:32:06	5
3502152	99	41	39,2	cyclictest	3878160	-21	kworker/u16:2+events_unbound	22:27:54	5
3502152	99	41	39,2	cyclictest	3878160	-21	kworker/u16:2+events_unbound	22:27:54	5
3502160	99	40	35,4	cyclictest	3534398	-21	latency_hist	19:19:41	7
3502156	99	40	38,2	cyclictest	3950103	-21	kworker/u16:0+flush-8:0	23:00:25	6
3502156	99	39	37,2	cyclictest	3756260	-21	kworker/u16:3+events_unbound	21:36:30	6
3502152	99	39	37,2	cyclictest	3914171	-21	kworker/u16:3+events_unbound	23:01:58	5
3502152	99	39	37,2	cyclictest	3878160	-21	kworker/u16:2+events_unbound	22:24:15	5
3502152	99	39	37,2	cyclictest	3756260	-21	kworker/u16:3+events_unbound	21:55:30	5
3492656	2	39	3,30	sleep7	0	-21	swapper/7	18:59:42	7
3502156	99	38	35,2	cyclictest	3725664	-21	kworker/u16:1+flush-8:0	21:24:40	6
3502152	99	38	36,2	cyclictest	3950103	-21	kworker/u16:0+events_unbound	22:52:06	5
3502160	99	37	31,5	cyclictest	650	-21	systemd-journal	22:04:41	7
3502156	99	37	34,2	cyclictest	4150213	-21	kworker/u16:5+events_unbound	00:13:26	6
3502152	99	37	35,2	cyclictest	3878160	-21	kworker/u16:2+events_unbound	22:10:42	5
3502152	99	37	35,2	cyclictest	3830603	-21	kworker/u16:4+events_unbound	21:45:20	5
3502152	99	36	34,2	cyclictest	4116728	-21	kworker/u16:0+events_unbound	00:24:38	5
3502152	99	36	34,2	cyclictest	4034944	-21	kworker/u16:2+events_unbound	23:34:15	5
3502152	99	36	34,2	cyclictest	3914171	-21	kworker/u16:3+events_unbound	22:47:11	5
3502152	99	36	34,2	cyclictest	3725664	-21	kworker/u16:1+events_unbound	21:13:54	5
3502152	99	36	34,2	cyclictest	3553642	-21	kworker/u16:2+events_unbound	21:42:55	5
3502152	99	36	34,2	cyclictest	3553642	-21	kworker/u16:2+events_unbound	21:09:13	5
3502152	99	36	34,2	cyclictest	3553642	-21	kworker/u16:2+events_unbound	21:09:13	5
3502152	99	36	33,2	cyclictest	3553642	-21	kworker/u16:2+events_unbound	21:14:41	5
3502160	99	35	30,4	cyclictest	650	-21	systemd-journal	22:54:41	7
3502152	99	35	33,2	cyclictest	4116728	-21	kworker/u16:0+events_unbound	23:45:53	5
3502152	99	35	33,2	cyclictest	4116728	-21	kworker/u16:0+events_unbound	00:26:06	5
3502152	99	35	33,2	cyclictest	4021707	-21	kworker/u16:1+events_unbound	23:18:49	5
3502152	99	35	33,2	cyclictest	3950103	-21	kworker/u16:0+events_unbound	22:57:58	5
3502152	99	35	33,2	cyclictest	3553642	-21	kworker/u16:2+events_unbound	21:33:15	5
3502160	99	34	30,2	cyclictest	650	-21	systemd-journal	00:34:39	7
3502156	99	34	32,2	cyclictest	4140629	-21	kworker/u16:1+events_unbound	00:04:21	6
3502152	99	34	32,2	cyclictest	4140993	-21	kworker/u16:4+events_unbound	00:14:05	5
3502152	99	34	32,2	cyclictest	4034944	-21	kworker/u16:2+events_unbound	23:53:36	5
3502152	99	34	32,2	cyclictest	4021707	-21	kworker/u16:1+events_unbound	23:43:42	5
3502152	99	34	32,2	cyclictest	3914171	-21	kworker/u16:3+events_unbound	23:11:54	5
3502152	99	34	32,2	cyclictest	3553642	-21	kworker/u16:2+events_unbound	21:54:02	5
3502156	99	33	31,2	cyclictest	3553642	-21	kworker/u16:2+events_unbound	21:25:47	6
3502152	99	33	31,2	cyclictest	4116728	-21	kworker/u16:0+events_unbound	00:29:47	5
3502152	99	33	31,2	cyclictest	3553642	-21	kworker/u16:2+events_unbound	20:29:40	5
3502152	99	33	31,2	cyclictest	3553642	-21	kworker/u16:2+events_unbound	20:29:40	5
3502147	99	33	31,2	cyclictest	3853279	-21	kworker/u16:1+events_unbound	21:54:54	4
3502156	99	32	31,1	cyclictest	3830603	-21	kworker/u16:4+events_unbound	21:59:05	6
3502156	99	32	30,2	cyclictest	4140993	-21	kworker/u16:4+events_unbound	00:07:26	6
3502156	99	32	30,2	cyclictest	4021707	-21	kworker/u16:1+events_unbound	23:37:13	6
3502156	99	32	30,2	cyclictest	3853279	-21	kworker/u16:1+events_unbound	22:07:46	6
3502152	99	32	30,2	cyclictest	4116728	-21	kworker/u16:0+events_unbound	00:17:11	5
3502152	99	32	30,2	cyclictest	3950103	-21	kworker/u16:0+events_unbound	22:38:51	5
3502152	99	32	30,2	cyclictest	3853279	-21	kworker/u16:1+events_unbound	22:02:12	5
3502152	99	32	30,2	cyclictest	3792374	-21	kworker/u16:0+events_unbound	22:09:18	5
3502156	99	31	29,2	cyclictest	3914171	-21	kworker/u16:3+flush-8:0	22:25:43	6
3502156	99	31	29,2	cyclictest	3914171	-21	kworker/u16:3+flush-8:0	22:25:43	6
3502156	99	31	28,2	cyclictest	4116728	-21	kworker/u16:0+flush-8:0	00:24:41	6
3502152	99	31	29,2	cyclictest	4140993	-21	kworker/u16:4+flush-8:0	23:55:26	5
3502152	99	31	29,2	cyclictest	4140993	-21	kworker/u16:4+events_unbound	00:06:55	5
3502152	99	31	29,2	cyclictest	4140629	-21	kworker/u16:1+events_unbound	00:03:03	5
3502152	99	31	29,2	cyclictest	3453710	-21	kworker/u16:0+events_unbound	19:58:51	5
3502152	99	31	28,2	cyclictest	3725664	-21	kworker/u16:1+flush-8:0	20:59:58	5
3502147	99	31	29,2	cyclictest	4034944	-21	kworker/u16:2+flush-8:0	23:34:40	4
3502156	99	30	28,2	cyclictest	4021707	-21	kworker/u16:1+events_unbound	23:05:47	6
3502156	99	30	28,2	cyclictest	4021707	-21	kworker/u16:1+events_unbound	23:05:47	6
3502156	99	30	28,2	cyclictest	3914171	-21	kworker/u16:3+events_unbound	22:23:26	6
3502152	99	30	28,2	cyclictest	4034944	-21	kworker/u16:2+events_unbound	23:27:03	5
3502147	99	30	28,2	cyclictest	3950103	-21	kworker/u16:0+events_unbound	22:54:54	4
3502156	99	29	27,2	cyclictest	3983311	-21	kworker/u16:1+events_unbound	22:47:21	6
3502156	99	29	27,2	cyclictest	3725664	-21	kworker/u16:1+events_unbound	21:07:41	6
3502156	99	29	27,2	cyclictest	3725664	-21	kworker/u16:1+events_unbound	21:07:41	6
3502156	99	29	26,2	cyclictest	3878160	-21	kworker/u16:2+events_unbound	22:34:41	6
3502152	99	29	27,2	cyclictest	3725664	-21	kworker/u16:1+events_unbound	21:20:35	5
3502152	99	29	27,2	cyclictest	3534786	-21	kworker/u16:3+events_unbound	19:29:11	5
3502152	99	29	26,2	cyclictest	3686776	-21	kworker/u16:3+events_unbound	20:39:50	5
3502156	99	28	26,2	cyclictest	4034944	-21	kworker/u16:2+events_unbound	23:19:34	6
3502156	99	28	26,2	cyclictest	3950103	-21	kworker/u16:0+flush-8:0	22:55:26	6
3502156	99	28	26,2	cyclictest	3914171	-21	kworker/u16:3+flush-8:0	23:10:52	6
3502156	99	28	26,2	cyclictest	3853279	-21	kworker/u16:1+events_unbound	22:18:10	6
3502156	99	28	26,2	cyclictest	3853279	-21	kworker/u16:1+events_unbound	22:04:21	6
3502156	99	28	26,2	cyclictest	3756260	-21	kworker/u16:3+events_unbound	21:48:06	6
3502152	99	28	26,2	cyclictest	3756260	-21	kworker/u16:3+events_unbound	21:29:20	5
3502152	99	28	26,2	cyclictest	3553642	-21	kworker/u16:2+flush-8:0	19:44:50	5
3502152	99	28	26,2	cyclictest	3553642	-21	kworker/u16:2+events_unbound	20:11:10	5
3502152	99	28	26,2	cyclictest	3553642	-21	kworker/u16:2+events_unbound	20:07:10	5
3502156	99	27	25,2	cyclictest	4150213	-21	kworker/u16:5+flush-8:0	00:30:26	6
3502156	99	27	25,2	cyclictest	3914171	-21	kworker/u16:3+events_unbound	23:21:41	6
3502156	99	27	25,2	cyclictest	3553642	-21	kworker/u16:2+flush-8:0	20:24:40	6
3502156	99	27	23,3	cyclictest	3483229	-21	kworker/u16:1+flush-8:0	19:19:41	6
3502152	99	27	25,2	cyclictest	3878160	-21	kworker/u16:2+events_unbound	22:15:59	5
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17