OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Mon Apr 06, 2026 12:43:33)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
1277468	2	61	45,11	sleep4	0	-21	swapper/4	06:57:30	4
1277458	2	58	47,7	sleep5	0	-21	swapper/5	06:57:23	5
1286881	99	48	45,2	cyclictest	1481985	-21	kworker/u16:1+flush-8:0	09:57:49	4
1286881	99	45	43,2	cyclictest	1869430	-21	kworker/u16:2+events_unbound	11:27:48	4
1286890	99	44	38,5	cyclictest	1347874	-21	latency_hist	07:32:19	7
1286881	99	44	41,2	cyclictest	1750626	-21	kworker/u16:2+events_unbound	10:42:36	4
1286890	99	43	36,5	cyclictest	1519997	-21	latency_hist	09:02:19	7
1277750	2	41	3,31	sleep7	0	-21	swapper/7	06:57:34	7
1286890	99	40	39,0	cyclictest	0	-21	swapper/7	08:42:42	7
1279621	2	40	0,2	chrt	0	-21	swapper/6	06:57:40	6
1286881	99	37	34,3	cyclictest	1481985	-21	kworker/u16:1+events_unbound	10:17:41	4
1286881	99	35	33,2	cyclictest	1869430	-21	kworker/u16:2+events_unbound	11:47:48	4
1286881	99	35	33,2	cyclictest	1869430	-21	kworker/u16:2+events_unbound	11:34:25	4
1286881	99	34	32,2	cyclictest	1855980	-21	kworker/u16:0+events_unbound	11:17:52	4
1286881	99	34	32,2	cyclictest	1750626	-21	kworker/u16:2+events_unbound	10:52:53	4
1286881	99	33	31,2	cyclictest	1942893	-21	kworker/u16:1+events_unbound	12:02:53	4
1286881	99	33	31,2	cyclictest	1651217	-21	kworker/u16:0+events_unbound	10:22:52	4
1286881	99	33	30,2	cyclictest	1481985	-21	kworker/u16:1+events_unbound	10:32:36	4
1286883	99	32	29,2	cyclictest	1990863	-21	kworker/u16:3+flush-8:0	12:27:18	6
1286881	99	31	29,2	cyclictest	1990184	-21	kworker/u16:2+events_unbound	12:12:51	4
1286881	99	31	29,2	cyclictest	1914605	-21	kworker/u16:5+events_unbound	11:42:45	4
1286881	99	31	29,2	cyclictest	1481985	-21	kworker/u16:1+events_unbound	10:12:40	4
1286881	99	31	28,2	cyclictest	1750626	-21	kworker/u16:2+events_unbound	10:37:21	4
1286882	99	30	28,1	cyclictest	1750626	-21	kworker/u16:2+flush-8:0	11:07:18	5
1286881	99	30	28,2	cyclictest	1990863	-21	kworker/u16:3+events_unbound	12:27:48	4
1286881	99	30	28,2	cyclictest	1750626	-21	kworker/u16:2+events_unbound	11:02:53	4
1286881	99	30	28,2	cyclictest	1556857	-21	kworker/u16:5+events_unbound	09:27:41	4
1286881	99	30	28,2	cyclictest	1544664	-21	kworker/u16:2+flush-8:0	09:22:35	4
1813574	2	29	0,1	chrt	1813575	-21	kthreadcore	11:02:38	7
1286882	99	29	27,2	cyclictest	1357635	-21	kworker/u16:1+events_unbound	08:02:34	5
1286881	99	29	27,2	cyclictest	1811060	-21	kworker/u16:1+events_unbound	11:23:36	4
1286881	99	29	27,2	cyclictest	1544664	-21	kworker/u16:2+flush-8:0	09:37:49	4
1286881	99	29	27,2	cyclictest	1481985	-21	kworker/u16:1+events_unbound	09:48:40	4
1286881	99	29	26,2	cyclictest	1532135	-21	kworker/u16:4+flush-8:0	09:17:14	4
1286882	99	28	25,2	cyclictest	1318808	-21	kworker/u16:2+events_unbound	07:27:45	5
1286881	99	28	26,2	cyclictest	1942893	-21	kworker/u16:1+events_unbound	12:17:48	4
1806163	2	27	0,1	chrt	1806164	-21	grep	10:57:41	7
1286881	99	27	25,2	cyclictest	1942893	-21	kworker/u16:1+events_unbound	11:57:42	4
1286881	99	27	25,2	cyclictest	1869430	-21	kworker/u16:2+events_unbound	11:53:36	4
1286883	99	26	24,2	cyclictest	1295047	-21	kworker/u16:0+events_unbound	07:07:45	6
1286882	99	26	24,2	cyclictest	1328767	-21	kworker/u16:0+events_unbound	07:47:32	5
1286881	99	26	23,2	cyclictest	1651217	-21	kworker/u16:0+events_unbound	10:07:33	4
1286882	99	25	23,2	cyclictest	1463032	-21	kworker/u16:3+flush-8:0	08:52:45	5
1286881	99	25	24,1	cyclictest	1651217	-21	kworker/u16:0+events_unbound	10:02:46	4
1286881	99	25	23,2	cyclictest	1855980	-21	kworker/u16:0+events_unbound	11:37:37	4
1286881	99	25	23,2	cyclictest	1651217	-21	kworker/u16:0+events_unbound	10:30:48	4
1286881	99	25	23,2	cyclictest	1481985	-21	kworker/u16:1+flush-8:0	09:17:45	4
1286881	99	25	22,3	cyclictest	1424538	-21	kworker/u16:2+flush-8:0	08:57:44	4
1286881	99	25	22,2	cyclictest	1811060	-21	kworker/u16:1+events_unbound	11:07:29	4
1286881	99	25	22,2	cyclictest	1556857	-21	kworker/u16:5+events_unbound	09:52:32	4
1286883	99	24	21,2	cyclictest	1556857	-21	kworker/u16:5+flush-8:0	09:57:24	6
1286882	99	24	22,2	cyclictest	1396088	-21	kworker/u16:3+events_unbound	07:57:37	5
1286881	99	24	22,2	cyclictest	1295047	-21	kworker/u16:0+flush-8:0	07:07:37	4
1726707	2	23	0,1	chrt	0	-21	swapper/6	10:27:19	6
1435251	2	23	0,1	chrt	0	-21	swapper/5	08:17:35	5
1286883	99	23	20,2	cyclictest	1424538	-21	kworker/u16:2+flush-8:0	08:57:45	6
1286881	99	23	20,3	cyclictest	1318808	-21	kworker/u16:2+flush-8:0	07:22:20	4
1286890	99	22	16,5	cyclictest	1641561	-21	latency_hist	09:52:19	7
1286890	99	22	0,21	cyclictest	1653706	-21	latency_hist	09:57:19	7
1286881	99	22	20,2	cyclictest	1855980	-21	kworker/u16:0+events_unbound	12:25:29	4
1286881	99	22	20,2	cyclictest	1544664	-21	kworker/u16:2+events_unbound	09:42:38	4
1286881	99	22	20,2	cyclictest	1481985	-21	kworker/u16:1+events_unbound	09:04:37	4
1286881	99	22	20,2	cyclictest	1463032	-21	kworker/u16:3+events_unbound	08:47:41	4
1286881	99	22	19,2	cyclictest	1914605	-21	kworker/u16:5+events_unbound	12:07:21	4
1286881	99	22	19,2	cyclictest	1651217	-21	kworker/u16:0+events_unbound	10:47:29	4
1286890	99	21	16,4	cyclictest	650	-21	systemd-journal	07:47:19	7
1286883	99	21	20,1	cyclictest	1750960	-21	kworker/u16:4+events_unbound	10:39:09	6
1286881	99	21	19,2	cyclictest	1463032	-21	kworker/u16:3+flush-8:0	08:37:19	4
1286881	99	21	18,2	cyclictest	1328767	-21	kworker/u16:0+flush-8:0	08:12:36	4
1510575	2	20	0,1	chrt	1510417	-21	latency_hist	08:57:19	7
1286890	99	20	19,1	cyclictest	1519999	-21	diskmemload	11:49:20	7
1286890	99	20	15,4	cyclictest	1870170	-21	ssh	11:24:23	7
1286890	99	20	0,19	cyclictest	1617185	-21	latency_hist	09:42:19	7
1286882	99	20	18,2	cyclictest	1424538	-21	kworker/u16:2+events_unbound	08:27:33	5
1286882	99	20	18,2	cyclictest	1424538	-21	kworker/u16:2+events_unbound	08:27:33	5
1286881	99	20	18,2	cyclictest	1424538	-21	kworker/u16:2+events_unbound	08:27:45	4
1286881	99	20	18,2	cyclictest	1424538	-21	kworker/u16:2+events_unbound	08:27:45	4
1286890	99	19	17,1	cyclictest	1519999	-21	diskmemload	11:45:20	7
1286890	99	19	14,4	cyclictest	1787131	-21	latency_hist	10:52:19	7
1286881	99	19	17,2	cyclictest	1463032	-21	kworker/u16:3+flush-8:0	09:08:01	4
1286881	99	19	17,2	cyclictest	1463032	-21	kworker/u16:3+flush-8:0	09:08:01	4
1286890	99	18	18,0	cyclictest	0	-21	swapper/7	10:20:17	7
1286890	99	18	14,3	cyclictest	1267	-21	snmpd	11:07:29	7
1286890	99	18	0,17	cyclictest	0	-21	swapper/7	07:52:44	7
1286882	99	18	16,2	cyclictest	1463032	-21	kworker/u16:3+flush-8:0	09:12:56	5
1286890	99	17	4,5	cyclictest	1528936	-21	ssh	09:02:42	7
1286890	99	17	17,0	cyclictest	0	-21	swapper/7	09:35:28	7
1286890	99	17	13,3	cyclictest	1788421	-21	cat	10:52:34	7
1286890	99	17	0,15	cyclictest	650	-21	systemd-journal	10:22:19	7
1286883	99	17	14,2	cyclictest	1690255	-21	kworker/u16:2+flush-8:0	10:17:24	6
1286882	99	17	14,2	cyclictest	1556857	-21	kworker/u16:5+flush-8:0	09:27:36	5
1286882	99	17	0,16	cyclictest	1870165	-21	pool-/usr/libex	11:24:23	5
1286881	99	17	15,2	cyclictest	1750626	-21	kworker/u16:2+flush-8:0	11:13:16	4
1286881	99	17	15,2	cyclictest	1357635	-21	kworker/u16:1+events_unbound	08:02:38	4
1286890	99	16	16,0	cyclictest	0	-21	swapper/7	12:26:32	7
1286890	99	16	16,0	cyclictest	0	-21	swapper/7	12:15:49	7
1286890	99	16	16,0	cyclictest	0	-21	swapper/7	10:41:24	7
1286890	99	16	16,0	cyclictest	0	-21	swapper/7	09:07:45	7
1286890	99	16	16,0	cyclictest	0	-21	swapper/7	09:07:45	7
1286890	99	16	15,0	cyclictest	0	-21	swapper/7	07:59:05	7
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17