OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Wed May 27, 2026 12:43:34)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
970002	99	60	58,2	cyclictest	1553550	-21	kworker/u16:3+events_unbound	11:57:39	6
970002	99	60	58,2	cyclictest	1117978	-21	kworker/u16:2+events_unbound	10:42:40	6
970002	99	59	56,2	cyclictest	1553550	-21	kworker/u16:3+events_unbound	11:32:24	6
970002	99	59	56,2	cyclictest	1553550	-21	kworker/u16:3+events_unbound	11:32:24	6
969997	99	57	54,2	cyclictest	1553550	-21	kworker/u16:3+flush-8:0	12:22:36	5
969996	99	57	54,2	cyclictest	1469424	-21	kworker/u16:0+flush-8:0	10:57:25	4
969515	2	57	47,7	sleep4	0	-21	swapper/4	07:04:26	4
969996	99	54	52,2	cyclictest	1410795	-21	kworker/u16:1+flush-8:0	11:07:39	4
970007	99	53	48,4	cyclictest	1543672	-21	latency_hist	11:27:09	7
969996	99	53	50,2	cyclictest	1483141	-21	kworker/u16:2+flush-8:0	11:47:23	4
970002	99	52	50,2	cyclictest	1483141	-21	kworker/u16:2+flush-8:0	11:27:35	6
970002	99	52	50,2	cyclictest	1351540	-21	kworker/u16:0+events_unbound	10:32:40	6
969996	99	51	49,2	cyclictest	1469424	-21	kworker/u16:0+events_unbound	11:17:39	4
969996	99	51	48,2	cyclictest	1553550	-21	kworker/u16:3+flush-8:0	12:02:24	4
970002	99	50	48,2	cyclictest	1553550	-21	kworker/u16:3+events_unbound	12:22:40	6
970002	99	50	48,2	cyclictest	1483141	-21	kworker/u16:2+events_unbound	11:47:41	6
969996	99	49	47,2	cyclictest	1410795	-21	kworker/u16:1+events_unbound	11:27:23	4
969996	99	49	47,2	cyclictest	1313453	-21	kworker/u16:3+events_unbound	10:27:39	4
969996	99	49	47,2	cyclictest	1313453	-21	kworker/u16:3+events_unbound	10:12:11	4
970002	99	48	46,2	cyclictest	1483141	-21	kworker/u16:2+flush-8:0	11:17:41	6
969996	99	48	46,2	cyclictest	1351540	-21	kworker/u16:0+events_unbound	10:17:39	4
970002	99	47	45,2	cyclictest	1483141	-21	kworker/u16:2+flush-8:0	11:12:39	6
970002	99	47	45,2	cyclictest	1410795	-21	kworker/u16:1+events_unbound	10:37:39	6
969996	99	47	45,2	cyclictest	1117978	-21	kworker/u16:2+flush-8:0	10:02:36	4
969996	99	46	43,2	cyclictest	1351540	-21	kworker/u16:0+flush-8:0	10:22:25	4
970002	99	45	43,2	cyclictest	1351540	-21	kworker/u16:0+flush-8:0	10:17:37	6
969996	99	45	42,2	cyclictest	1567376	-21	kworker/u16:4+events_unbound	12:30:07	4
967533	2	45	3,30	sleep7	0	-21	swapper/7	07:02:36	7
970007	99	44	41,2	cyclictest	1277063	-21	perf	09:37:10	7
970007	99	44	41,2	cyclictest	1277063	-21	perf	09:37:10	7
970002	99	44	42,2	cyclictest	1553550	-21	kworker/u16:3+events_unbound	12:32:24	6
970002	99	44	42,2	cyclictest	1313453	-21	kworker/u16:3+events_unbound	10:02:38	6
969996	99	44	42,2	cyclictest	1553550	-21	kworker/u16:3+flush-8:0	11:32:44	4
969996	99	44	42,2	cyclictest	1553550	-21	kworker/u16:3+flush-8:0	11:32:44	4
969996	99	44	42,2	cyclictest	1553550	-21	kworker/u16:3+events_unbound	11:42:40	4
970002	99	43	41,2	cyclictest	1483141	-21	kworker/u16:2+events_unbound	11:22:30	6
970002	99	43	41,2	cyclictest	1447137	-21	kworker/u16:4+events_unbound	10:49:36	6
969996	99	43	41,2	cyclictest	1567376	-21	kworker/u16:4+flush-8:0	12:12:38	4
969996	99	43	41,2	cyclictest	1567376	-21	kworker/u16:4+flush-8:0	12:12:38	4
969996	99	43	41,2	cyclictest	1553550	-21	kworker/u16:3+events_unbound	12:32:43	4
969996	99	43	41,2	cyclictest	1553550	-21	kworker/u16:3+events_unbound	11:57:23	4
969996	99	43	41,2	cyclictest	1483141	-21	kworker/u16:2+events_unbound	11:13:19	4
969996	99	43	41,2	cyclictest	1410795	-21	kworker/u16:1+events_unbound	10:56:44	4
969996	99	43	41,2	cyclictest	1410795	-21	kworker/u16:1+events_unbound	10:56:44	4
970002	99	42	39,2	cyclictest	1483141	-21	kworker/u16:2+events_unbound	12:02:12	6
970002	99	41	39,2	cyclictest	1553550	-21	kworker/u16:3+flush-8:0	11:37:39	6
970002	99	41	39,2	cyclictest	1238478	-21	kworker/u16:5+flush-8:0	09:25:24	6
969996	99	41	38,2	cyclictest	1410795	-21	kworker/u16:1+events_unbound	11:02:12	4
970002	99	40	38,2	cyclictest	1567376	-21	kworker/u16:4+flush-8:0	12:12:59	6
970002	99	40	38,2	cyclictest	1567376	-21	kworker/u16:4+flush-8:0	12:12:59	6
970002	99	40	38,2	cyclictest	1325577	-21	kworker/u16:4+events_unbound	10:27:08	6
969996	99	40	38,2	cyclictest	1240836	-21	kworker/u16:0+flush-8:0	09:45:11	4
970002	99	39	37,2	cyclictest	1667770	-21	kworker/u16:0+flush-8:0	12:21:00	6
970002	99	39	37,2	cyclictest	1325577	-21	kworker/u16:4+flush-8:0	10:09:15	6
970002	99	39	37,2	cyclictest	1240836	-21	kworker/u16:0+events_unbound	09:45:16	6
969996	99	39	37,2	cyclictest	1483141	-21	kworker/u16:2+flush-8:0	11:56:15	4
969996	99	39	37,2	cyclictest	1447137	-21	kworker/u16:4+events_unbound	10:47:47	4
969996	99	39	37,2	cyclictest	1117978	-21	kworker/u16:2+flush-8:0	09:12:10	4
970002	99	38	36,2	cyclictest	1483141	-21	kworker/u16:2+flush-8:0	11:53:11	6
969996	99	38	36,2	cyclictest	1553550	-21	kworker/u16:3+events_unbound	12:25:55	4
969996	99	38	36,2	cyclictest	1483141	-21	kworker/u16:2+events_unbound	11:22:42	4
969996	99	38	36,2	cyclictest	1240836	-21	kworker/u16:0+events_unbound	09:37:08	4
969996	99	38	36,2	cyclictest	1240836	-21	kworker/u16:0+events_unbound	09:37:08	4
969996	99	38	36,2	cyclictest	1240836	-21	kworker/u16:0+events_unbound	09:29:52	4
969996	99	38	36,2	cyclictest	1117978	-21	kworker/u16:2+events_unbound	09:22:07	4
969996	99	38	35,2	cyclictest	1567376	-21	kworker/u16:4+flush-8:0	12:17:25	4
970007	99	37	32,4	cyclictest	1422479	-21	latency_hist	10:37:10	7
970002	99	37	35,2	cyclictest	1313453	-21	kworker/u16:3+flush-8:0	10:12:30	6
970002	99	37	34,2	cyclictest	969774	-21	kworker/u16:1+events_unbound	08:47:12	6
969996	99	37	35,2	cyclictest	1166111	-21	kworker/u16:4+flush-8:0	09:12:56	4
969996	99	37	35,2	cyclictest	1117978	-21	kworker/u16:2+events_unbound	09:53:00	4
970007	99	36	31,4	cyclictest	1031	-21	rs:main		7
970002	99	36	34,2	cyclictest	1469424	-21	kworker/u16:0+flush-8:0	10:56:11	6
970002	99	36	34,2	cyclictest	1469424	-21	kworker/u16:0+flush-8:0	10:56:11	6
970002	99	36	34,2	cyclictest	1278774	-21	kworker/u16:1+events_unbound	09:56:35	6
970002	99	36	34,2	cyclictest	1240836	-21	kworker/u16:0+flush-8:0	09:33:35	6
970002	99	36	34,2	cyclictest	1240836	-21	kworker/u16:0+flush-8:0	09:33:35	6
970002	99	36	34,2	cyclictest	1240836	-21	kworker/u16:0+events_unbound	09:50:51	6
969996	99	36	34,2	cyclictest	1553550	-21	kworker/u16:3+events_unbound	11:39:16	4
969996	99	36	34,2	cyclictest	1351540	-21	kworker/u16:0+flush-8:0	10:32:26	4
969996	99	36	34,2	cyclictest	1240836	-21	kworker/u16:0+flush-8:0	09:49:27	4
969996	99	36	34,2	cyclictest	1117978	-21	kworker/u16:2+events_unbound	09:59:15	4
970002	99	35	33,2	cyclictest	1483141	-21	kworker/u16:2+events_unbound	11:09:07	6
970002	99	35	33,2	cyclictest	1278774	-21	kworker/u16:1+events_unbound	09:41:00	6
969996	99	35	33,2	cyclictest	1567376	-21	kworker/u16:4+events_unbound	12:10:44	4
969996	99	35	33,2	cyclictest	1325577	-21	kworker/u16:4+flush-8:0	10:08:44	4
969403	2	35	25,7	sleep5	0	-21	swapper/5	07:02:50	5
970007	99	34	33,0	cyclictest	0	-21	swapper/7	10:30:06	7
970007	99	34	31,2	cyclictest	1346660	-21	ntpq	10:02:34	7
970002	99	34	32,2	cyclictest	1238478	-21	kworker/u16:5+events_unbound	09:30:08	6
970002	99	34	32,2	cyclictest	1225630	-21	kworker/u16:3+events_unbound	09:16:43	6
970002	99	34	32,2	cyclictest	1147034	-21	kworker/u16:3+flush-8:0	08:37:37	6
970002	99	34	32,2	cyclictest	1117978	-21	kworker/u16:2+flush-8:0	09:20:16	6
969996	99	34	32,2	cyclictest	969774	-21	kworker/u16:1+events_unbound	08:02:05	4
969996	99	34	32,2	cyclictest	1410795	-21	kworker/u16:1+events_unbound	10:44:12	4
970002	99	33	31,2	cyclictest	983619	-21	kworker/u16:2+flush-8:0	07:17:35	6
970002	99	33	31,2	cyclictest	1483141	-21	kworker/u16:2+events_unbound	12:11:00	6
969996	99	33	31,2	cyclictest	1166111	-21	kworker/u16:4+events_unbound	09:23:56	4
969996	99	33	31,2	cyclictest	1117978	-21	kworker/u16:2+events_unbound	10:41:55	4
969996	99	33	31,2	cyclictest	1117978	-21	kworker/u16:2+events_unbound	08:56:56	4
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17