OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Fri Apr 03, 2026 12:43:33)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
1834476	2	94	0,1	sleep7	1834478	-21	ssh	09:36:32	7
1516455	99	57	55,2	cyclictest	2079824	-21	kworker/u16:3+events_unbound	11:42:57	4
1516178	2	52	42,7	sleep4	0	-21	swapper/4	07:02:29	4
1516455	99	48	46,2	cyclictest	1942651	-21	kworker/u16:4+events_unbound	10:47:33	4
1516460	99	47	45,2	cyclictest	2079824	-21	kworker/u16:3+events_unbound	11:53:01	5
1516460	99	47	45,2	cyclictest	1896269	-21	kworker/u16:2+events_unbound	10:05:13	5
1516455	99	46	44,2	cyclictest	1701625	-21	kworker/u16:3+events_unbound	09:08:38	4
1516455	99	44	42,2	cyclictest	2079824	-21	kworker/u16:3+events_unbound	11:20:33	4
1516136	2	44	34,7	sleep5	0	-21	swapper/5	07:01:58	5
1516461	99	43	41,2	cyclictest	1673313	-21	kworker/u16:0+events_unbound	08:32:58	6
1516460	99	43	41,2	cyclictest	2079824	-21	kworker/u16:3+events_unbound	11:28:49	5
1516455	99	43	41,2	cyclictest	1871813	-21	kworker/u16:0+flush-8:0	10:08:09	4
1516460	99	42	40,2	cyclictest	2225135	-21	kworker/u16:1+events_unbound	12:30:17	5
1516460	99	42	40,2	cyclictest	2079824	-21	kworker/u16:3+events_unbound	12:10:13	5
1516460	99	42	40,2	cyclictest	1615716	-21	kworker/u16:1+events_unbound	09:09:54	5
1516460	99	42	39,2	cyclictest	1749763	-21	kworker/u16:4+events_unbound	09:34:14	5
1516460	99	42	38,3	cyclictest	2102635	-21	kworker/u16:5+flush-8:0	11:32:31	5
1516455	99	42	40,2	cyclictest	2102634	-21	kworker/u16:4+events_unbound	11:49:05	4
1516455	99	42	40,2	cyclictest	1896269	-21	kworker/u16:2+events_unbound	10:06:54	4
1515894	2	42	3,31	sleep7	0	-21	swapper/7	06:58:31	7
1516465	99	41	40,0	cyclictest	0	-21	swapper/7	11:58:02	7
1516465	99	41	36,4	cyclictest	1653847	-21	latency_hist	08:12:31	7
1516460	99	41	39,2	cyclictest	1995813	-21	kworker/u16:1+events_unbound	11:00:25	5
1516460	99	41	39,2	cyclictest	1995813	-21	kworker/u16:1+events_unbound	11:00:25	5
1516460	99	41	39,2	cyclictest	1749763	-21	kworker/u16:4+events_unbound	10:00:23	5
1516455	99	41	39,2	cyclictest	2226997	-21	kworker/u16:5+events_unbound	12:21:20	4
1516455	99	41	39,2	cyclictest	1995813	-21	kworker/u16:1+events_unbound	10:47:14	4
1516455	99	41	39,2	cyclictest	1749763	-21	kworker/u16:4+events_unbound	09:33:45	4
1516455	99	41	39,2	cyclictest	1615716	-21	kworker/u16:1+events_unbound	09:23:53	4
1516119	2	41	31,7	sleep6	0	-21	swapper/6	07:01:43	6
1516460	99	40	38,2	cyclictest	2226997	-21	kworker/u16:5+events_unbound	12:21:16	5
1516460	99	40	38,2	cyclictest	2153530	-21	kworker/u16:0+events_unbound	11:59:52	5
1516460	99	40	38,2	cyclictest	2079824	-21	kworker/u16:3+events_unbound	12:06:38	5
1516460	99	40	38,2	cyclictest	2079824	-21	kworker/u16:3+events_unbound	11:48:49	5
1516460	99	40	38,2	cyclictest	1995813	-21	kworker/u16:1+events_unbound	11:18:57	5
1516460	99	40	38,2	cyclictest	1942651	-21	kworker/u16:4+events_unbound	11:05:53	5
1516460	99	40	38,2	cyclictest	1942651	-21	kworker/u16:4+events_unbound	10:55:14	5
1516460	99	40	38,2	cyclictest	1896269	-21	kworker/u16:2+events_unbound	10:19:46	5
1516460	99	40	38,2	cyclictest	1896269	-21	kworker/u16:2+events_unbound	10:15:09	5
1516460	99	40	38,2	cyclictest	1835093	-21	kworker/u16:2+events_unbound	09:46:17	5
1516455	99	40	39,1	cyclictest	1896269	-21	kworker/u16:2+events_unbound	10:30:53	4
1516455	99	40	38,2	cyclictest	2225135	-21	kworker/u16:1+events_unbound	12:29:57	4
1516455	99	40	38,2	cyclictest	1995813	-21	kworker/u16:1+events_unbound	10:59:54	4
1516455	99	40	38,2	cyclictest	1995813	-21	kworker/u16:1+events_unbound	10:59:54	4
1516455	99	40	38,2	cyclictest	1932928	-21	kworker/u16:3+events_unbound	10:18:03	4
1516455	99	40	38,2	cyclictest	1786149	-21	kworker/u16:0+events_unbound	09:30:37	4
1516455	99	40	38,2	cyclictest	1615716	-21	kworker/u16:1+events_unbound	09:21:55	4
1773627	2	39	0,0	sleep7	1773628	-21	rm	09:12:04	7
1516460	99	39	37,2	cyclictest	2225135	-21	kworker/u16:1+events_unbound	12:25:32	5
1516460	99	39	37,2	cyclictest	2079824	-21	kworker/u16:3+events_unbound	11:39:13	5
1516460	99	39	37,2	cyclictest	2079824	-21	kworker/u16:3+events_unbound	11:39:13	5
1516460	99	39	37,2	cyclictest	1995813	-21	kworker/u16:1+events_unbound	10:46:01	5
1516460	99	39	37,2	cyclictest	1942651	-21	kworker/u16:4+events_unbound	11:09:09	5
1516460	99	39	37,2	cyclictest	1942651	-21	kworker/u16:4+events_unbound	10:51:53	5
1516460	99	39	37,2	cyclictest	1942651	-21	kworker/u16:4+events_unbound	10:23:33	5
1516460	99	39	37,2	cyclictest	1871813	-21	kworker/u16:0+flush-8:0	09:52:49	5
1516460	99	39	37,2	cyclictest	1871813	-21	kworker/u16:0+events_unbound	10:35:22	5
1516460	99	39	37,2	cyclictest	1749763	-21	kworker/u16:4+events_unbound	09:40:45	5
1516460	99	39	37,2	cyclictest	1615716	-21	kworker/u16:1+events_unbound	09:24:14	5
1516455	99	39	37,2	cyclictest	2153530	-21	kworker/u16:0+events_unbound	12:24:12	4
1516455	99	39	37,2	cyclictest	2102634	-21	kworker/u16:4+events_unbound	11:39:34	4
1516455	99	39	37,2	cyclictest	2102634	-21	kworker/u16:4+events_unbound	11:39:34	4
1516455	99	39	37,2	cyclictest	2079824	-21	kworker/u16:3+events_unbound	11:54:33	4
1516455	99	39	37,2	cyclictest	2079824	-21	kworker/u16:3+events_unbound	11:24:14	4
1516455	99	39	36,3	cyclictest	2079824	-21	kworker/u16:3+events_unbound	12:14:25	4
1516465	99	38	33,4	cyclictest	2251514	-21	latency_hist	12:27:30	7
1516455	99	38	36,2	cyclictest	1995813	-21	kworker/u16:1+events_unbound	11:13:33	4
1516455	99	38	36,2	cyclictest	1871813	-21	kworker/u16:0+events_unbound	10:38:26	4
1516455	99	38	36,2	cyclictest	1835093	-21	kworker/u16:2+flush-8:0	09:45:21	4
1516455	99	38	36,2	cyclictest	1835093	-21	kworker/u16:2+events_unbound	09:42:17	4
1516455	99	38	36,2	cyclictest	1701625	-21	kworker/u16:3+events_unbound	09:03:06	4
1516460	99	37	35,2	cyclictest	2079824	-21	kworker/u16:3+events_unbound	11:43:42	5
1516460	99	37	35,2	cyclictest	1995813	-21	kworker/u16:1+events_unbound	11:15:18	5
1516460	99	37	35,2	cyclictest	1749763	-21	kworker/u16:4+events_unbound	09:19:13	5
1516460	99	37	33,3	cyclictest	2102491	-21	fwupd	11:24:31	5
1516455	99	37	35,2	cyclictest	2102635	-21	kworker/u16:5+events_unbound	11:36:46	4
1516455	99	37	35,2	cyclictest	1942651	-21	kworker/u16:4+events_unbound	11:05:57	4
1516455	99	37	35,2	cyclictest	1942651	-21	kworker/u16:4+events_unbound	10:23:41	4
1516455	99	37	35,2	cyclictest	1871813	-21	kworker/u16:0+events_unbound	11:11:14	4
1516455	99	37	35,2	cyclictest	1871813	-21	kworker/u16:0+events_unbound	10:02:01	4
1516455	99	37	35,2	cyclictest	1835093	-21	kworker/u16:2+events_unbound	09:49:38	4
1516455	99	37	35,2	cyclictest	1615716	-21	kworker/u16:1+events_unbound	08:26:41	4
1516465	99	36	31,4	cyclictest	1548709	-21	latency_hist	07:17:31	7
1516465	99	36	30,5	cyclictest	1847356	-21	latency_hist	09:42:31	7
1516460	99	36	34,2	cyclictest	2225135	-21	kworker/u16:1+events_unbound	12:15:32	5
1516460	99	36	34,2	cyclictest	1871813	-21	kworker/u16:0+events_unbound	10:10:25	5
1516460	99	36	34,2	cyclictest	1739495	-21	kworker/u16:0+events_unbound	09:06:37	5
1516460	99	36	34,2	cyclictest	1701625	-21	kworker/u16:3+events_unbound	09:14:30	5
1516460	99	36	34,2	cyclictest	1615716	-21	kworker/u16:1+events_unbound	09:48:34	5
1516455	99	36	34,2	cyclictest	2102634	-21	kworker/u16:4+events_unbound	11:30:13	4
1516455	99	36	34,2	cyclictest	2079824	-21	kworker/u16:3+events_unbound	12:05:57	4
1516455	99	36	34,2	cyclictest	1739495	-21	kworker/u16:0+flush-8:0	08:57:53	4
1516455	99	36	34,2	cyclictest	1701625	-21	kworker/u16:3+events_unbound	09:14:10	4
1516455	99	36	34,2	cyclictest	1615716	-21	kworker/u16:1+events_unbound	10:15:34	4
1516460	99	35	33,2	cyclictest	1942651	-21	kworker/u16:4+events_unbound	10:41:26	5
1516460	99	35	33,2	cyclictest	1786149	-21	kworker/u16:0+events_unbound	09:30:49	5
1516460	99	35	33,2	cyclictest	1598411	-21	kworker/u16:2+events_unbound	08:28:50	5
1516455	99	35	33,2	cyclictest	2079824	-21	kworker/u16:3+events_unbound	12:10:54	4
1516455	99	35	33,2	cyclictest	1995813	-21	kworker/u16:1+flush-8:0	10:53:31	4
1516455	99	35	33,2	cyclictest	1749763	-21	kworker/u16:4+events_unbound	09:55:34	4
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17