OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Mon Jan 19, 2026 12:43:33)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
1454074	2	98	0,1	sleep7	1454075	-21	sh	10:35:14	7
996650	2	68	56,8	sleep5	0	-21	swapper/5	07:05:34	5
997060	99	58	57,1	cyclictest	124	-21	kswapd0	09:29:35	4
997060	99	58	57,1	cyclictest	124	-21	kswapd0	09:29:35	4
996670	2	58	48,7	sleep4	0	-21	swapper/4	07:05:49	4
996504	2	55	21,30	sleep7	0	-21	swapper/7	07:03:27	7
997062	99	47	45,2	cyclictest	1136845	-21	kworker/u16:0+events_unbound	09:12:26	5
996633	2	47	37,7	sleep6	0	-21	swapper/6	07:05:19	6
997062	99	46	44,2	cyclictest	1175531	-21	kworker/u16:3+events_unbound	09:52:26	5
997066	99	44	41,2	cyclictest	1415807	-21	kworker/u16:2+events_unbound	10:32:28	6
1429736	2	44	0,1	sleep6	0	-21	swapper/6	10:25:43	6
997066	99	42	40,2	cyclictest	1415807	-21	kworker/u16:2+flush-8:0	10:45:44	6
997066	99	42	40,2	cyclictest	1186917	-21	kworker/u16:1+events_unbound	09:26:52	6
997070	99	41	0,39	cyclictest	1344465	-21	sed	09:52:26	7
997066	99	41	39,2	cyclictest	1186917	-21	kworker/u16:1+events_unbound	09:15:32	6
997062	99	41	39,2	cyclictest	1539036	-21	kworker/u16:4+events_unbound	11:18:45	5
997062	99	41	39,2	cyclictest	1136845	-21	kworker/u16:0+events_unbound	10:47:25	5
997066	99	40	38,2	cyclictest	1415807	-21	kworker/u16:2+events_unbound	10:31:29	6
997066	99	40	37,2	cyclictest	1613389	-21	kworker/u16:2+events_unbound	12:32:28	6
997062	99	40	37,2	cyclictest	1175531	-21	kworker/u16:3+flush-8:0	09:29:31	5
997062	99	40	37,2	cyclictest	1175531	-21	kworker/u16:3+flush-8:0	09:29:31	5
997066	99	39	37,2	cyclictest	1553853	-21	kworker/u16:1+events_unbound	11:41:00	6
997066	99	39	37,2	cyclictest	1381348	-21	kworker/u16:5+events_unbound	10:09:56	6
997066	99	39	36,2	cyclictest	1614214	-21	kworker/u16:8+events_unbound	12:17:27	6
997062	99	39	36,2	cyclictest	1613389	-21	kworker/u16:2+events_unbound	12:07:27	5
997066	99	38	36,2	cyclictest	1539036	-21	kworker/u16:4+events_unbound	11:09:24	6
997066	99	38	36,2	cyclictest	1415807	-21	kworker/u16:2+events_unbound	10:39:44	6
997066	99	38	36,2	cyclictest	1186917	-21	kworker/u16:1+events_unbound	09:29:09	6
997066	99	38	36,2	cyclictest	1186917	-21	kworker/u16:1+events_unbound	09:29:09	6
997066	99	38	35,2	cyclictest	1553853	-21	kworker/u16:1+events_unbound	11:27:56	6
997062	99	38	36,2	cyclictest	1490873	-21	kworker/u16:3+flush-8:0	11:25:20	5
997060	99	38	36,2	cyclictest	1175531	-21	kworker/u16:3+flush-8:0	09:27:20	4
997066	99	37	35,2	cyclictest	1614214	-21	kworker/u16:8+events_unbound	12:16:16	6
997066	99	37	35,2	cyclictest	1175531	-21	kworker/u16:3+events_unbound	09:49:09	6
997066	99	37	35,2	cyclictest	1136845	-21	kworker/u16:0+events_unbound	10:48:53	6
997066	99	37	35,2	cyclictest	1136845	-21	kworker/u16:0+events_unbound	09:40:04	6
997066	99	37	34,2	cyclictest	1614214	-21	kworker/u16:8+flush-8:0	11:52:27	6
997062	99	37	35,2	cyclictest	1293233	-21	kworker/u16:4+events_unbound	10:02:57	5
997066	99	36	34,2	cyclictest	1627732	-21	kworker/u16:0+events_unbound	12:23:15	6
997066	99	36	34,2	cyclictest	1553853	-21	kworker/u16:1+flush-8:0	11:46:36	6
997066	99	36	34,2	cyclictest	1553853	-21	kworker/u16:1+flush-8:0	11:36:15	6
997066	99	36	34,2	cyclictest	1490873	-21	kworker/u16:3+events_unbound	11:03:44	6
997066	99	36	34,2	cyclictest	1415807	-21	kworker/u16:2+events_unbound	10:22:03	6
997066	99	36	34,2	cyclictest	1381348	-21	kworker/u16:5+events_unbound	10:16:01	6
997066	99	36	34,2	cyclictest	1356657	-21	kworker/u16:2+events_unbound	10:07:13	6
997066	99	36	34,2	cyclictest	1293233	-21	kworker/u16:4+flush-8:0	09:34:24	6
997066	99	36	34,2	cyclictest	1136845	-21	kworker/u16:0+events_unbound	09:12:01	6
997062	99	36	34,2	cyclictest	1575361	-21	kworker/u16:0+flush-8:0	11:35:03	5
997062	99	36	34,2	cyclictest	1097900	-21	kworker/u16:2+flush-8:0	08:38:26	5
997062	99	36	34,2	cyclictest	1020121	-21	kworker/u16:1+events_unbound	08:12:25	5
997062	99	36	33,2	cyclictest	1517274	-21	kworker/u16:2+events_unbound	11:07:27	5
997060	99	36	33,3	cyclictest	1575361	-21	kworker/u16:0+flush-8:0	11:32:44	4
997066	99	35	33,2	cyclictest	1614214	-21	kworker/u16:8+events_unbound	11:50:40	6
997066	99	35	33,2	cyclictest	1490873	-21	kworker/u16:3+events_unbound	11:20:31	6
997066	99	35	33,2	cyclictest	1344745	-21	kworker/u16:1+events_unbound	09:55:28	6
997066	99	35	33,2	cyclictest	1136845	-21	kworker/u16:0+events_unbound	09:43:56	6
997066	99	35	32,2	cyclictest	1627732	-21	kworker/u16:0+flush-8:0	12:02:28	6
997062	99	35	33,2	cyclictest	1415807	-21	kworker/u16:2+events_unbound	10:33:44	5
997062	99	35	33,2	cyclictest	1381348	-21	kworker/u16:5+flush-8:0	10:17:24	5
997062	99	35	33,2	cyclictest	1381348	-21	kworker/u16:5+flush-8:0	10:08:53	5
997062	99	35	33,2	cyclictest	1344745	-21	kworker/u16:1+events_unbound	09:55:36	5
997060	99	35	33,2	cyclictest	1613644	-21	kworker/u16:5+flush-8:0	11:39:24	4
997060	99	35	33,2	cyclictest	1613389	-21	kworker/u16:2+flush-8:0	12:17:43	4
997060	99	35	33,2	cyclictest	1381347	-21	kworker/u16:1+events_unbound	10:07:43	4
997060	99	35	32,2	cyclictest	1613389	-21	kworker/u16:2+flush-8:0	12:22:40	4
997070	99	34	29,4	cyclictest	1739593	-21	latency_hist	12:32:25	7
997062	99	34	32,2	cyclictest	1724778	-21	kworker/u16:3+flush-8:0	12:35:24	5
997062	99	34	32,2	cyclictest	1613389	-21	kworker/u16:2+events_unbound	11:48:25	5
997062	99	34	32,2	cyclictest	1293233	-21	kworker/u16:4+events_unbound	09:43:25	5
997062	99	34	32,2	cyclictest	1136845	-21	kworker/u16:0+events_unbound	10:48:00	5
997060	99	34	32,2	cyclictest	1136845	-21	kworker/u16:0+flush-8:0	10:47:47	4
997066	99	33	31,2	cyclictest	1627732	-21	kworker/u16:0+events_unbound	12:01:56	6
997066	99	33	31,2	cyclictest	1490873	-21	kworker/u16:3+events_unbound	11:23:19	6
997066	99	33	31,2	cyclictest	1356657	-21	kworker/u16:2+events_unbound	10:00:56	6
997062	99	33	31,2	cyclictest	1175531	-21	kworker/u16:3+events_unbound	09:42:03	5
997060	99	33	32,1	cyclictest	1136845	-21	kworker/u16:0+events_unbound	09:12:48	4
997066	99	32	30,2	cyclictest	1613389	-21	kworker/u16:2+events_unbound	12:29:59	6
997066	99	32	30,2	cyclictest	1490873	-21	kworker/u16:3+events_unbound	11:00:13	6
997066	99	32	30,2	cyclictest	1136845	-21	kworker/u16:0+events_unbound	11:14:08	6
997062	99	32	30,2	cyclictest	1175531	-21	kworker/u16:3+events_unbound	09:33:52	5
997062	99	32	30,2	cyclictest	1136845	-21	kworker/u16:0+events_unbound	10:23:27	5
997060	99	32	30,2	cyclictest	1415807	-21	kworker/u16:2+events_unbound	10:32:48	4
997070	99	31	30,0	cyclictest	0	-21	swapper/7	10:58:24	7
997066	99	31	29,2	cyclictest	1186917	-21	kworker/u16:1+events_unbound	09:20:43	6
997066	99	31	29,2	cyclictest	1136845	-21	kworker/u16:0+flush-8:0	10:55:07	6
997062	99	31	29,2	cyclictest	1712514	-21	kworker/u16:1+events_unbound	12:20:56	5
997062	99	31	29,2	cyclictest	1627732	-21	kworker/u16:0+events_unbound	12:13:36	5
997062	99	31	29,2	cyclictest	1415807	-21	kworker/u16:2+events_unbound	10:28:24	5
997062	99	31	29,2	cyclictest	1175531	-21	kworker/u16:3+events_unbound	09:17:12	5
997062	99	31	29,2	cyclictest	1136845	-21	kworker/u16:0+events_unbound	09:25:44	5
997060	99	31	29,2	cyclictest	1415807	-21	kworker/u16:2+events_unbound	10:27:52	4
997060	99	31	29,2	cyclictest	1136845	-21	kworker/u16:0+events_unbound	10:52:53	4
997070	99	30	29,1	cyclictest	0	-21	swapper/7	09:42:36	7
997070	99	30	28,1	cyclictest	1269076	-21	ssh	09:20:24	7
997070	99	30	25,4	cyclictest	1653459	-21	latency_hist	11:57:26	7
997062	99	30	28,2	cyclictest	1724778	-21	kworker/u16:3+flush-8:0	12:25:20	5
997062	99	30	28,2	cyclictest	1614214	-21	kworker/u16:8+flush-8:0	11:42:20	5
997062	99	30	28,2	cyclictest	1136845	-21	kworker/u16:0+flush-8:0	10:53:56	5
997062	99	30	28,2	cyclictest	1136845	-21	kworker/u16:0+events_unbound	09:19:05	5
997060	99	30	28,2	cyclictest	1539036	-21	kworker/u16:4+events_unbound	11:17:48	4
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17