OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Wed Mar 11, 2026 12:43:34)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
1871397	2	95	0,1	sleep5	0	-21	swapper/5	11:09:21	5
1805238	2	94	0,1	sleep7	1805241	-21	cut	10:39:26	7
1336204	99	52	47,2	cyclictest	1368898	-21	kworker/u16:2+flush-8:0	07:44:03	6
1336204	99	47	44,2	cyclictest	1843381	-21	kworker/u16:3+flush-8:0	12:04:30	6
1335606	2	47	36,8	sleep5	0	-21	swapper/5	06:59:43	5
1335606	2	47	36,8	sleep5	0	-21	swapper/5	06:59:43	5
1712598	2	46	0,0	sleep7	1712599	-21	rm	10:02:23	7
1336205	99	45	0,43	cyclictest	650	-21	systemd-journal	08:59:02	7
1336204	99	45	43,2	cyclictest	1843380	-21	kworker/u16:0+flush-8:0	11:59:02	6
1336194	99	45	42,2	cyclictest	1843380	-21	kworker/u16:0+flush-8:0	12:29:02	4
1336204	99	44	42,2	cyclictest	1263625	-21	kworker/u16:1+flush-8:0	08:19:02	6
1335757	2	44	34,7	sleep6	0	-21	swapper/6	07:01:53	6
1335757	2	44	34,7	sleep6	0	-21	swapper/6	07:01:53	6
1336205	99	41	0,39	cyclictest	650	-21	systemd-journal	09:38:57	7
1336204	99	41	38,2	cyclictest	1326792	-21	kworker/u16:3+flush-8:0	07:19:02	6
1336194	99	41	37,3	cyclictest	1795466	-21	kworker/u16:2+flush-8:0	10:44:28	4
1327642	2	40	3,31	sleep7	0	-21	swapper/7	06:59:21	7
1327642	2	40	3,31	sleep7	0	-21	swapper/7	06:59:21	7
1336204	99	39	36,2	cyclictest	1843380	-21	kworker/u16:0+flush-8:0	12:14:37	6
1336204	99	39	36,2	cyclictest	1737606	-21	kworker/u16:1+flush-8:0	11:09:21	6
1336205	99	34	0,32	cyclictest	1473622	-21	latency_hist	08:14:02	7
1336204	99	34	32,2	cyclictest	1967016	-21	kworker/u16:1+flush-8:0	12:19:32	6
1335688	2	34	24,7	sleep4	0	-21	swapper/4	07:00:54	4
1335688	2	34	24,7	sleep4	0	-21	swapper/4	07:00:54	4
1336204	99	32	30,2	cyclictest	1737606	-21	kworker/u16:1+events_unbound	10:14:16	6
1336204	99	32	29,3	cyclictest	1593616	-21	kworker/u16:3+flush-8:0	09:26:29	6
1336204	99	32	29,3	cyclictest	1573653	-21	kworker/u16:0+events_unbound	09:49:28	6
1367506	2	30	0,0	chrt	1367489	-21	needreboot	07:14:27	7
1336204	99	30	28,2	cyclictest	1593616	-21	kworker/u16:3+events_unbound	10:04:20	6
1336204	99	28	25,2	cyclictest	1520656	-21	kworker/u16:1+flush-8:0	08:54:02	6
1893401	2	27	0,1	chrt	1893399	-21	ssh	11:18:21	7
1336205	99	27	25,2	cyclictest	124	-21	kswapd0	12:23:01	7
1336204	99	27	25,2	cyclictest	1484703	-21	kworker/u16:3+events_unbound	08:23:39	6
1336204	99	27	25,2	cyclictest	1454288	-21	kworker/u16:0+flush-8:0	08:09:33	6
1336204	99	27	25,2	cyclictest	1263625	-21	kworker/u16:1+flush-8:0	07:19:20	6
1336205	99	26	20,5	cyclictest	650	-21	systemd-journal	07:39:03	7
1336204	99	25	23,2	cyclictest	1593616	-21	kworker/u16:3+events_unbound	09:29:27	6
1802294	2	24	0,1	chrt	0	-21	swapper/4	10:39:23	4
1336205	99	24	19,4	cyclictest	650	-21	systemd-journal	11:28:57	7
1336204	99	24	22,2	cyclictest	1843380	-21	kworker/u16:0+events_unbound	11:59:32	6
1336205	99	23	19,3	cyclictest	650	-21	systemd-journal	08:33:57	7
1336205	99	23	0,22	cyclictest	6471	-21	gnome-software	09:26:15	7
1336204	99	23	21,2	cyclictest	1593616	-21	kworker/u16:3+flush-8:0	09:44:38	6
1336204	99	22	20,2	cyclictest	1843380	-21	kworker/u16:0+events_unbound	11:19:36	6
1336204	99	22	20,2	cyclictest	1326792	-21	kworker/u16:3+flush-8:0	07:34:33	6
1336205	99	21	17,3	cyclictest	1769739	-21	kthreadcore	10:24:25	7
1336204	99	21	19,2	cyclictest	1737606	-21	kworker/u16:1+flush-8:0	10:44:02	6
1738008	2	20	0,1	chrt	1738006	-21	ssh	10:14:09	7
1336205	99	20	15,4	cyclictest	2073727	-21	latency_hist	12:34:02	7
1336204	99	20	19,1	cyclictest	1795466	-21	kworker/u16:2+flush-8:0	10:44:25	6
1336204	99	20	18,2	cyclictest	1520656	-21	kworker/u16:1+flush-8:0	09:04:17	6
1336205	99	19	16,2	cyclictest	2057583	-21	kthreadcore	12:24:23	7
1336204	99	19	16,2	cyclictest	1263625	-21	kworker/u16:1+flush-8:0	07:24:28	6
1336204	99	19	16,2	cyclictest	1263625	-21	kworker/u16:1+flush-8:0	07:24:28	6
1336205	99	18	18,0	cyclictest	0	-21	swapper/7	10:30:14	7
1336204	99	18	16,2	cyclictest	1843380	-21	kworker/u16:0+flush-8:0	11:29:43	6
1336204	99	18	16,2	cyclictest	1843380	-21	kworker/u16:0+flush-8:0	11:29:43	6
1336205	99	17	5,4	cyclictest	2002816	-21	tune2fs	12:04:15	7
1336204	99	17	15,2	cyclictest	1879585	-21	kworker/u16:2+events_unbound	11:43:03	6
1336194	99	17	16,1	cyclictest	0	-21	swapper/4	08:29:44	4
1336194	99	17	16,1	cyclictest	0	-21	swapper/4	08:00:03	4
1336194	99	17	16,1	cyclictest	0	-21	swapper/4	07:40:35	4
1336204	99	16	14,2	cyclictest	1665523	-21	kworker/u16:4+flush-8:0	09:55:07	6
1336204	99	16	14,2	cyclictest	1573653	-21	kworker/u16:0+events_unbound	09:39:36	6
1336204	99	16	13,2	cyclictest	1484703	-21	kworker/u16:3+flush-8:0	08:34:02	6
1336194	99	16	15,1	cyclictest	0	-21	swapper/4	12:32:15	4
1336194	99	16	15,1	cyclictest	0	-21	swapper/4	09:00:28	4
1336194	99	16	15,1	cyclictest	0	-21	swapper/4	07:06:48	4
1336205	99	15	8,6	cyclictest	0	-21	swapper/7	09:29:26	7
1336205	99	15	13,2	cyclictest	1569210	-21	diskmemload	09:39:08	7
1336205	99	15	12,2	cyclictest	1267	-21	snmpd	11:40:09	7
1336204	99	15	13,2	cyclictest	1843380	-21	kworker/u16:0+events_unbound	11:44:31	6
1336204	99	15	13,2	cyclictest	1737606	-21	kworker/u16:1+events_unbound	10:36:52	6
1336204	99	15	13,2	cyclictest	1573653	-21	kworker/u16:0+events_unbound	10:24:02	6
1336204	99	15	12,2	cyclictest	1573653	-21	kworker/u16:0+flush-8:0	10:29:21	6
1336204	99	15	0,1	cyclictest	0	-21	swapper/6	11:34:15	6
1336199	99	15	14,1	cyclictest	0	-21	swapper/5	09:56:23	5
1336199	99	15	0,1	cyclictest	0	-21	swapper/5	09:25:01	5
1336199	99	15	0,15	cyclictest	0	-21	swapper/5	10:46:13	5
1336199	99	15	0,15	cyclictest	0	-21	swapper/5	09:46:37	5
1336199	99	15	0,0	cyclictest	0	-21	swapper/5	12:20:58	5
1336194	99	15	15,0	cyclictest	0	-21	swapper/4	10:51:39	4
1336194	99	15	14,1	cyclictest	0	-21	swapper/4	08:44:05	4
1336194	99	15	12,3	cyclictest	5755	-21	JS	21:50:16	4
1336205	99	14	9,4	cyclictest	1965697	-21	latency_hist	11:49:02	7
1336205	99	14	7,6	cyclictest	0	-21	swapper/7	11:55:35	7
1336205	99	14	7,6	cyclictest	0	-21	swapper/7	10:19:14	7
1336205	99	14	7,6	cyclictest	0	-21	swapper/7	10:09:18	7
1336205	99	14	7,6	cyclictest	0	-21	swapper/7	09:14:31	7
1336205	99	14	6,7	cyclictest	0	-21	swapper/7	12:15:01	7
1336205	99	14	6,7	cyclictest	0	-21	swapper/7	10:04:03	7
1336205	99	14	6,7	cyclictest	0	-21	swapper/7	09:10:11	7
1336205	99	14	5,8	cyclictest	0	-21	swapper/7	12:12:33	7
1336205	99	14	5,8	cyclictest	0	-21	swapper/7	12:12:33	7
1336205	99	14	5,8	cyclictest	0	-21	swapper/7	09:08:03	7
1336205	99	14	3,9	cyclictest	1991275	-21	irqcore	11:59:18	7
1336205	99	14	11,2	cyclictest	1491830	-21	ntpq	08:19:28	7
1336205	99	14	11,2	cyclictest	0	-21	swapper/7	11:09:02	7
1336205	99	14	10,3	cyclictest	1854766	-21	ntpq	10:59:33	7
1336205	99	14	0,13	cyclictest	1833701	-21	turbostat	10:54:01	7
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17