OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Wed Feb 25, 2026 00:43:34)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
1383660	2	59	49,7	sleep5	0	-21	swapper/5	19:02:36	5
1383616	2	58	48,7	sleep6	0	-21	swapper/6	19:01:57	6
1384138	99	48	42,5	cyclictest	1502382	-21	latency_hist	20:05:01	7
1383527	2	47	37,7	sleep4	0	-21	swapper/4	19:00:42	4
1384138	99	44	0,42	cyclictest	1425962	-21	latency_hist	19:25:01	7
1384132	99	41	39,2	cyclictest	1502771	-21	kworker/u16:0+flush-8:0	20:15:24	5
1384133	99	40	38,1	cyclictest	1822402	-21	kworker/u16:3+flush-8:0	22:45:00	6
1384132	99	40	38,2	cyclictest	1398488	-21	kworker/u16:3+events_unbound	21:35:29	5
1384132	99	40	37,2	cyclictest	1822402	-21	kworker/u16:3+events_unbound	23:25:03	5
1383662	2	40	5,31	sleep7	0	-21	swapper/7	19:02:38	7
1384138	99	39	37,1	cyclictest	94	1	rcuc/7	00:04:55	7
1384132	99	38	36,2	cyclictest	1822402	-21	kworker/u16:3+events_unbound	23:09:19	5
1384132	99	37	35,2	cyclictest	2027347	-21	kworker/u16:4+events_unbound	00:17:31	5
1384132	99	37	35,2	cyclictest	1822402	-21	kworker/u16:3+events_unbound	23:51:18	5
1384132	99	37	35,2	cyclictest	1822402	-21	kworker/u16:3+events_unbound	23:51:18	5
1384132	99	36	34,2	cyclictest	1894616	-21	kworker/u16:1+events_unbound	23:16:23	5
1384132	99	36	34,2	cyclictest	1811718	-21	kworker/u16:2+events_unbound	22:28:28	5
1384132	99	36	34,2	cyclictest	1701611	-21	kworker/u16:0+events_unbound	21:52:03	5
1384132	99	36	34,2	cyclictest	1701611	-21	kworker/u16:0+events_unbound	21:52:03	5
1384132	99	36	34,2	cyclictest	1626751	-21	kworker/u16:2+events_unbound	21:17:26	5
1384132	99	35	33,2	cyclictest	1858716	-21	kworker/u16:0+events_unbound	23:02:46	5
1384132	99	34	32,2	cyclictest	2098983	-21	kworker/u16:3+events_unbound	00:29:50	5
1384132	99	34	32,2	cyclictest	2027347	-21	kworker/u16:4+events_unbound	00:22:15	5
1384132	99	34	32,2	cyclictest	1822402	-21	kworker/u16:3+events_unbound	22:52:36	5
1384132	99	34	32,2	cyclictest	1747614	-21	kworker/u16:5+events_unbound	22:04:03	5
1384132	99	34	32,2	cyclictest	1701611	-21	kworker/u16:0+events_unbound	21:42:35	5
1384132	99	34	32,2	cyclictest	1578974	-21	kworker/u16:1+events_unbound	21:12:35	5
1384132	99	34	32,2	cyclictest	1398488	-21	kworker/u16:3+events_unbound	21:46:06	5
1384133	99	33	31,2	cyclictest	1398488	-21	kworker/u16:3+flush-8:0	19:45:00	6
1384133	99	33	31,2	cyclictest	1398488	-21	kworker/u16:3+flush-8:0	19:45:00	6
1384132	99	33	31,2	cyclictest	1955112	-21	kworker/u16:2+events_unbound	23:30:35	5
1384132	99	33	31,2	cyclictest	1822402	-21	kworker/u16:3+events_unbound	23:22:32	5
1384132	99	33	31,2	cyclictest	1822402	-21	kworker/u16:3+events_unbound	00:03:19	5
1384132	99	33	31,2	cyclictest	1811718	-21	kworker/u16:2+flush-8:0	22:40:23	5
1384132	99	33	31,2	cyclictest	1626751	-21	kworker/u16:2+events_unbound	21:27:46	5
1384132	99	33	31,2	cyclictest	1398488	-21	kworker/u16:3+events_unbound	21:07:24	5
1384129	99	33	31,2	cyclictest	1760195	-21	kworker/u16:1+flush-8:0	22:05:15	4
1384138	99	32	26,5	cyclictest	1473743	-21	latency_hist	19:50:00	7
1384132	99	32	30,2	cyclictest	1918945	-21	kworker/u16:4+events_unbound	23:11:51	5
1384132	99	32	30,2	cyclictest	1918945	-21	kworker/u16:4+events_unbound	23:11:51	5
1384132	99	32	30,2	cyclictest	1811718	-21	kworker/u16:2+flush-8:0	22:45:43	5
1384132	99	32	30,2	cyclictest	1774337	-21	kworker/u16:0+events_unbound	22:23:51	5
1384132	99	32	30,2	cyclictest	1747614	-21	kworker/u16:5+events_unbound	22:10:22	5
1384132	99	32	30,2	cyclictest	1578974	-21	kworker/u16:1+events_unbound	21:20:20	5
1384132	99	32	28,2	cyclictest	1955112	-21	kworker/u16:2+events_unbound	23:43:29	5
1384138	99	31	27,3	cyclictest	124	-21	kswapd0	22:59:26	7
1384132	99	31	29,2	cyclictest	1955112	-21	kworker/u16:2+events_unbound	23:45:58	5
1384132	99	31	29,2	cyclictest	1894616	-21	kworker/u16:1+events_unbound	00:14:57	5
1384132	99	31	29,2	cyclictest	1843734	-21	kworker/u16:4+events_unbound	22:39:09	5
1384132	99	31	29,2	cyclictest	1578974	-21	kworker/u16:1+events_unbound	20:57:12	5
1384132	99	31	29,2	cyclictest	1398488	-21	kworker/u16:3+flush-8:0	19:30:25	5
1384129	99	31	29,2	cyclictest	1374710	-21	kworker/u16:1+flush-8:0	19:30:27	4
1384129	99	31	28,2	cyclictest	1894616	-21	kworker/u16:1+events_unbound	00:00:23	4
1384138	99	30	29,0	cyclictest	0	-21	swapper/7	21:05:18	7
1384133	99	30	28,2	cyclictest	1760195	-21	kworker/u16:1+flush-8:0	22:15:26	6
1384133	99	30	27,2	cyclictest	1760195	-21	kworker/u16:1+flush-8:0	22:05:06	6
1384133	99	30	27,2	cyclictest	1559963	-21	kworker/u16:0+flush-8:0	21:00:31	6
1384132	99	30	28,2	cyclictest	1822402	-21	kworker/u16:3+events_unbound	00:07:41	5
1384132	99	30	28,2	cyclictest	1733796	-21	kworker/u16:2+events_unbound	21:59:31	5
1384132	99	30	28,2	cyclictest	1578974	-21	kworker/u16:1+events_unbound	20:53:30	5
1384133	99	29	27,2	cyclictest	2098634	-21	kworker/u16:2+flush-8:0	00:25:54	6
1384133	99	29	27,2	cyclictest	1822402	-21	kworker/u16:3+events_unbound	23:26:07	6
1384132	99	29	27,2	cyclictest	1760195	-21	kworker/u16:1+flush-8:0	22:16:12	5
1384132	99	29	27,2	cyclictest	1626751	-21	kworker/u16:2+events_unbound	21:30:38	5
1384129	99	29	26,2	cyclictest	2098634	-21	kworker/u16:2+flush-8:0	00:25:15	4
1384133	99	28	26,2	cyclictest	2027347	-21	kworker/u16:4+flush-8:0	00:24:55	6
1384133	99	28	26,2	cyclictest	1733796	-21	kworker/u16:2+events_unbound	22:01:50	6
1384132	99	28	26,2	cyclictest	1894616	-21	kworker/u16:1+events_unbound	23:56:33	5
1384132	99	28	26,2	cyclictest	1747614	-21	kworker/u16:5+events_unbound	22:07:24	5
1384132	99	28	26,2	cyclictest	1578974	-21	kworker/u16:1+events_unbound	21:00:38	5
1384132	99	28	26,2	cyclictest	1398488	-21	kworker/u16:3+flush-8:0	19:16:07	5
1384132	99	28	26,2	cyclictest	1398488	-21	kworker/u16:3+events_unbound	19:38:39	5
1384132	99	28	25,2	cyclictest	1858716	-21	kworker/u16:0+events_unbound	22:55:09	5
1384129	99	28	26,2	cyclictest	1955112	-21	kworker/u16:2+flush-8:0	23:30:26	4
1384133	99	27	25,2	cyclictest	1955112	-21	kworker/u16:2+events_unbound	23:30:20	6
1384132	99	27	25,2	cyclictest	1435877	-21	kworker/u16:2+events_unbound	20:03:26	5
1384132	99	27	25,2	cyclictest	1435877	-21	kworker/u16:2+events_unbound	19:49:39	5
1384132	99	27	25,2	cyclictest	1398488	-21	kworker/u16:3+events_unbound	19:42:10	5
1384132	99	27	25,2	cyclictest	1398488	-21	kworker/u16:3+events_unbound	19:42:10	5
1384133	99	26	24,2	cyclictest	1822402	-21	kworker/u16:3+events_unbound	23:10:16	6
1384133	99	26	24,2	cyclictest	1822402	-21	kworker/u16:3+events_unbound	23:10:16	6
1384132	99	26	24,2	cyclictest	1822402	-21	kworker/u16:3+events_unbound	22:31:57	5
1384132	99	26	24,2	cyclictest	1822402	-21	kworker/u16:3+events_unbound	22:31:57	5
1384132	99	26	24,2	cyclictest	1355293	-21	kworker/u16:0+events_unbound	19:09:39	5
1384133	99	25	24,1	cyclictest	1858716	-21	kworker/u16:0+events_unbound	22:51:59	6
1384133	99	25	23,2	cyclictest	1578974	-21	kworker/u16:1+flush-8:0	21:49:55	6
1384132	99	25	23,2	cyclictest	2027347	-21	kworker/u16:4+events_unbound	00:31:53	5
1384132	99	25	23,2	cyclictest	2027347	-21	kworker/u16:4+events_unbound	00:31:53	5
1384132	99	25	23,2	cyclictest	1894616	-21	kworker/u16:1+events_unbound	23:36:05	5
1384132	99	25	23,2	cyclictest	1398488	-21	kworker/u16:3+events_unbound	20:09:10	5
1384133	99	24	22,2	cyclictest	1858716	-21	kworker/u16:0+events_unbound	22:46:06	6
1384132	99	24	22,2	cyclictest	1435877	-21	kworker/u16:2+events_unbound	19:58:27	5
1384132	99	24	22,2	cyclictest	1398488	-21	kworker/u16:3+events_unbound	20:13:38	5
1384132	99	24	22,2	cyclictest	1398488	-21	kworker/u16:3+events_unbound	19:53:38	5
1384132	99	24	22,2	cyclictest	1398488	-21	kworker/u16:3+events_unbound	19:24:11	5
1384132	99	24	22,2	cyclictest	1374710	-21	kworker/u16:1+flush-8:0	19:25:16	5
1384129	99	24	21,2	cyclictest	1398488	-21	kworker/u16:3+flush-8:0	19:25:26	4
1384138	99	23	22,0	cyclictest	0	-21	swapper/7	20:05:15	7
1384138	99	23	17,5	cyclictest	1906478	-21	latency_hist	23:04:59	7
1384133	99	23	22,1	cyclictest	1822402	-21	kworker/u16:3+events_unbound	00:00:27	6
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17