OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Sat Apr 18, 2026 00:43:33)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
1705355	99	42	0,40	cyclictest	2350466	-21	latency_hist	23:59:46	7
1705002	2	41	6,30	sleep7	0	-21	swapper/7	19:09:03	7
1704963	2	41	30,8	sleep4	0	-21	swapper/4	19:08:28	4
1705355	99	37	31,5	cyclictest	1824052	-21	latency_hist	20:09:48	7
1705353	99	37	35,2	cyclictest	1795380	-21	kworker/u16:0+flush-8:0	20:59:48	6
1705353	99	33	31,2	cyclictest	1872471	-21	kworker/u16:2+flush-8:0	21:19:47	6
1705348	99	31	29,2	cyclictest	1910492	-21	kworker/u16:3+flush-8:0	22:09:42	4
2180182	2	30	0,1	chrt	2180183	-21	ssh	22:48:09	7
1705355	99	30	29,0	cyclictest	0	-21	swapper/7	20:35:01	7
1705355	99	30	25,4	cyclictest	2314265	-21	latency_hist	23:44:47	7
1705029	2	30	20,7	sleep5	0	-21	swapper/5	19:09:25	5
1705021	2	30	19,8	sleep6	0	-21	swapper/6	19:09:18	6
1705353	99	29	27,2	cyclictest	1833815	-21	kworker/u16:3+events_unbound	20:45:10	6
2156458	2	27	0,1	chrt	2156459	-21	ssh	22:38:56	7
1705355	99	27	0,25	cyclictest	2386781	-21	latency_hist	00:14:47	7
1705353	99	27	25,2	cyclictest	1795380	-21	kworker/u16:0+flush-8:0	21:10:11	6
1705353	99	27	25,2	cyclictest	1776482	-21	kworker/u16:4+events_unbound	20:05:10	6
1705353	99	26	24,2	cyclictest	1910492	-21	kworker/u16:3+flush-8:0	22:05:09	6
1705348	99	26	24,2	cyclictest	2194963	-21	kworker/u16:0+events_unbound	22:57:56	4
1705348	99	26	24,2	cyclictest	2022000	-21	kworker/u16:1+events_unbound	22:29:41	4
1705355	99	25	20,4	cyclictest	2326321	-21	latency_hist	23:49:47	7
1705353	99	25	23,2	cyclictest	2290465	-21	kworker/u16:4+events_unbound	23:45:09	6
1705353	99	25	23,2	cyclictest	2034139	-21	kworker/u16:0+events_unbound	21:50:09	6
1705348	99	25	23,2	cyclictest	2106066	-21	kworker/u16:4+flush-8:0	22:30:01	4
1705353	99	24	22,2	cyclictest	2253936	-21	kworker/u16:2+events_unbound	23:35:13	6
1705348	99	24	21,2	cyclictest	2311819	-21	kworker/u16:0+flush-8:0	23:44:57	4
1705353	99	23	21,2	cyclictest	2155451	-21	kworker/u16:3+flush-8:0	23:24:41	6
1705348	99	23	21,2	cyclictest	2239423	-21	kworker/u16:1+flush-8:0	23:25:11	4
1705348	99	23	21,2	cyclictest	2239423	-21	kworker/u16:1+flush-8:0	23:25:11	4
1705348	99	23	21,2	cyclictest	2022000	-21	kworker/u16:1+events_unbound	22:22:21	4
1705355	99	22	0,20	cyclictest	2398955	-21	sed	00:19:47	7
1705353	99	22	20,2	cyclictest	2348220	-21	kworker/u16:2+events_unbound	00:25:01	6
1705353	99	22	20,2	cyclictest	2326515	-21	kworker/u16:1+events_unbound	00:20:10	6
1705353	99	22	20,2	cyclictest	2022000	-21	kworker/u16:1+flush-8:0	22:29:42	6
1705348	99	22	20,2	cyclictest	2290465	-21	kworker/u16:4+flush-8:0	23:44:42	4
1705353	99	21	19,2	cyclictest	2194963	-21	kworker/u16:0+events_unbound	23:04:42	6
1705353	99	21	19,2	cyclictest	2133146	-21	kworker/u16:2+flush-8:0	22:35:11	6
1705353	99	21	19,2	cyclictest	1910493	-21	kworker/u16:4+flush-8:0	21:40:18	6
1705348	99	21	19,2	cyclictest	2022000	-21	kworker/u16:1+flush-8:0	21:59:26	4
1705348	99	21	19,2	cyclictest	2022000	-21	kworker/u16:1+flush-8:0	21:59:26	4
1705348	99	21	19,2	cyclictest	1910492	-21	kworker/u16:3+events_unbound	21:46:40	4
1705348	99	21	19,2	cyclictest	1833815	-21	kworker/u16:3+flush-8:0	20:15:10	4
1705353	99	20	18,2	cyclictest	1872471	-21	kworker/u16:2+flush-8:0	21:30:09	6
1705353	99	20	18,2	cyclictest	1747758	-21	kworker/u16:1+flush-8:0	20:00:29	6
1705348	99	20	18,2	cyclictest	2311819	-21	kworker/u16:0+flush-8:0	00:04:46	4
1705348	99	20	18,2	cyclictest	1686375	-21	kworker/u16:2+flush-8:0	19:31:30	4
1705353	99	19	17,2	cyclictest	263001	-21	kworker/u16:1+events_unbound	19:15:10	6
1705353	99	19	17,2	cyclictest	2348220	-21	kworker/u16:2+events_unbound	00:03:57	6
1705353	99	19	17,2	cyclictest	1910492	-21	kworker/u16:3+flush-8:0	22:11:57	6
1705348	99	19	17,2	cyclictest	2311819	-21	kworker/u16:0+flush-8:0	23:54:42	4
1705348	99	19	17,2	cyclictest	2036333	-21	kworker/u16:2+events_unbound	22:14:52	4
1705348	99	19	17,2	cyclictest	1910492	-21	kworker/u16:3+flush-8:0	21:52:26	4
1705348	99	19	17,2	cyclictest	1795380	-21	kworker/u16:0+flush-8:0	21:12:10	4
1705355	99	18	14,3	cyclictest	1234	-21	snmpd	21:07:50	7
1705353	99	18	16,2	cyclictest	2348220	-21	kworker/u16:2+events_unbound	00:15:11	6
1705353	99	18	16,2	cyclictest	2106066	-21	kworker/u16:4+flush-8:0	22:20:13	6
1705353	99	18	15,2	cyclictest	2155451	-21	kworker/u16:3+events_unbound	23:29:49	6
1705348	99	18	16,2	cyclictest	2311819	-21	kworker/u16:0+events_unbound	00:05:09	4
1705348	99	18	16,2	cyclictest	2118179	-21	kworker/u16:0+events_unbound	22:45:19	4
1705348	99	18	16,2	cyclictest	1910492	-21	kworker/u16:3+events_unbound	22:10:05	4
1705348	99	18	16,2	cyclictest	1872471	-21	kworker/u16:2+flush-8:0	21:00:45	4
1705348	99	18	16,2	cyclictest	1776482	-21	kworker/u16:4+events_unbound	20:04:23	4
1705348	99	18	16,2	cyclictest	1686375	-21	kworker/u16:2+flush-8:0	19:35:26	4
1705355	99	17	0,1	cyclictest	0	-21	swapper/7	19:35:11	7
1705353	99	17	15,2	cyclictest	2326515	-21	kworker/u16:1+flush-8:0	00:14:42	6
1705353	99	17	15,2	cyclictest	1963513	-21	kworker/u16:1+flush-8:0	21:20:13	6
1705349	99	17	0,1	cyclictest	0	-21	swapper/5	00:10:08	5
1705348	99	17	15,2	cyclictest	999416	-21	kworker/u16:3+flush-8:0	19:19:48	4
1705348	99	17	15,2	cyclictest	2410955	-21	kworker/u16:4+flush-8:0	00:35:11	4
1705348	99	17	15,2	cyclictest	2326515	-21	kworker/u16:1+events_unbound	00:32:48	4
1705348	99	17	15,2	cyclictest	2253936	-21	kworker/u16:2+events_unbound	23:35:11	4
1705348	99	17	15,2	cyclictest	2179899	-21	kworker/u16:1+events_unbound	22:49:57	4
1705348	99	17	15,2	cyclictest	2133146	-21	kworker/u16:2+flush-8:0	22:35:13	4
1705348	99	17	15,2	cyclictest	1795380	-21	kworker/u16:0+flush-8:0	20:27:07	4
1705355	99	16	13,2	cyclictest	2205295	-21	latency_hist	22:59:47	7
1705355	99	16	12,3	cyclictest	1234	-21	snmpd	19:16:18	7
1705355	99	16	10,5	cyclictest	635	-21	systemd-journal	00:19:47	7
1705355	99	16	0,14	cyclictest	0	-21	swapper/7	21:45:11	7
1705348	99	16	14,2	cyclictest	999416	-21	kworker/u16:3+flush-8:0	19:45:20	4
1705348	99	16	14,2	cyclictest	2326515	-21	kworker/u16:1+flush-8:0	00:15:13	4
1705348	99	16	14,2	cyclictest	2311819	-21	kworker/u16:0+events_unbound	23:55:09	4
1705348	99	16	14,2	cyclictest	2194963	-21	kworker/u16:0+flush-8:0	23:02:26	4
1705348	99	16	14,2	cyclictest	2155451	-21	kworker/u16:3+flush-8:0	22:41:47	4
1705348	99	16	14,2	cyclictest	2155451	-21	kworker/u16:3+flush-8:0	22:41:47	4
1705348	99	16	14,2	cyclictest	2034139	-21	kworker/u16:0+flush-8:0	22:02:25	4
1705348	99	16	14,2	cyclictest	2034139	-21	kworker/u16:0+flush-8:0	22:02:25	4
1705348	99	16	14,2	cyclictest	1910493	-21	kworker/u16:4+flush-8:0	21:40:19	4
1705348	99	16	14,2	cyclictest	1910492	-21	kworker/u16:3+flush-8:0	21:32:01	4
1705348	99	16	14,2	cyclictest	1910492	-21	kworker/u16:3+flush-8:0	21:24:42	4
1705348	99	16	14,2	cyclictest	1872471	-21	kworker/u16:2+flush-8:0	21:05:01	4
1705348	99	16	14,2	cyclictest	1872471	-21	kworker/u16:2+events_unbound	20:46:30	4
1705348	99	16	14,2	cyclictest	1795380	-21	kworker/u16:0+flush-8:0	20:50:13	4
1705348	99	16	14,2	cyclictest	1747758	-21	kworker/u16:1+flush-8:0	20:12:21	4
1705348	99	16	14,2	cyclictest	1747758	-21	kworker/u16:1+events_unbound	20:33:01	4
1705348	99	16	14,2	cyclictest	1747758	-21	kworker/u16:1+events_unbound	20:33:01	4
1705348	99	16	14,2	cyclictest	1686375	-21	kworker/u16:2+events_unbound	19:13:29	4
1705348	99	16	14,2	cyclictest	1676168	-21	kworker/u16:0+events_unbound	19:22:01	4
1705355	99	15	9,5	cyclictest	2302134	-21	latency_hist	23:39:47	7
1705355	99	15	14,0	cyclictest	0	-21	swapper/7	00:25:04	7
1705353	99	15	13,2	cyclictest	2311819	-21	kworker/u16:0+flush-8:0	23:55:00	6
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17