OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Tue Nov 25, 2025 00:43:34)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
1485947	99	58	53,4	cyclictest	1954664	-21	latency_hist	07:13:21	7
1485301	2	57	47,7	sleep5	0	-21	swapper/5	03:19:41	5
1485322	2	56	46,7	sleep4	0	-21	swapper/4	03:19:56	4
1485947	99	53	48,4	cyclictest	2055733	-21	latency_hist	08:03:20	7
1485453	2	53	19,30	sleep7	0	-21	swapper/7	03:21:43	7
1485315	2	53	42,8	sleep6	0	-21	swapper/6	03:19:50	6
1485947	99	48	0,46	cyclictest	2075888	-21	latency_hist	08:13:21	7
1485947	99	45	44,0	cyclictest	0	-21	swapper/7	06:03:45	7
1485939	99	43	39,3	cyclictest	1671641	-21	kworker/u16:0+flush-8:0	05:08:39	4
1485943	99	40	38,2	cyclictest	1954432	-21	kworker/u16:0+flush-8:0	07:18:50	5
1485947	99	38	31,5	cyclictest	589235	-21	systemd-journal	06:48:21	7
1485947	99	38	31,5	cyclictest	589235	-21	systemd-journal	06:48:21	7
1485946	99	38	34,3	cyclictest	1415718	-21	kworker/u16:1+events_unbound	03:38:21	6
1485943	99	38	36,2	cyclictest	2005034	-21	kworker/u16:1+events_unbound	07:40:57	5
1485943	99	38	36,2	cyclictest	1399252	-21	kworker/u16:0+events_unbound	03:41:07	5
1485943	99	38	35,2	cyclictest	2085126	-21	kworker/u16:6+flush-8:0	08:18:22	5
1485947	99	37	36,0	cyclictest	0	-21	swapper/7	07:03:30	7
1485946	99	37	34,2	cyclictest	2005034	-21	kworker/u16:1+events_unbound	07:48:38	6
1485943	99	37	35,2	cyclictest	1671868	-21	kworker/u16:2+events_unbound	05:37:42	5
1485943	99	36	34,2	cyclictest	2065807	-21	kworker/u16:2+events_unbound	08:49:07	5
1485943	99	36	34,2	cyclictest	1712290	-21	kworker/u16:1+events_unbound	06:10:43	5
1485943	99	36	32,3	cyclictest	2005034	-21	kworker/u16:1+events_unbound	08:09:42	5
1485946	99	35	33,2	cyclictest	2015482	-21	kworker/u16:0+flush-8:0	08:18:20	6
1485946	99	35	33,2	cyclictest	2015482	-21	kworker/u16:0+flush-8:0	08:18:20	6
1485946	99	35	33,2	cyclictest	1930062	-21	kworker/u16:1+events_unbound	07:08:38	6
1485946	99	35	32,2	cyclictest	2055612	-21	kworker/u16:3+flush-8:0	08:03:34	6
1485943	99	35	33,2	cyclictest	2065807	-21	kworker/u16:2+flush-8:0	08:43:47	5
1485943	99	35	33,2	cyclictest	1863627	-21	kworker/u16:1+events_unbound	06:46:23	5
1485943	99	35	33,2	cyclictest	1863627	-21	kworker/u16:1+events_unbound	06:46:23	5
1485943	99	35	33,2	cyclictest	1823058	-21	kworker/u16:0+events_unbound	06:14:30	5
1485943	99	35	33,2	cyclictest	1802190	-21	kworker/u16:2+events_unbound	06:24:34	5
1485943	99	35	33,2	cyclictest	1783384	-21	kworker/u16:3+events_unbound	05:58:54	5
1485943	99	35	33,2	cyclictest	1783384	-21	kworker/u16:3+events_unbound	05:58:54	5
1485943	99	35	33,2	cyclictest	1712290	-21	kworker/u16:1+events_unbound	06:04:32	5
1485943	99	35	32,2	cyclictest	1954432	-21	kworker/u16:0+events_unbound	07:28:22	5
1485943	99	35	32,2	cyclictest	1954432	-21	kworker/u16:0+events_unbound	07:28:22	5
1485943	99	35	32,2	cyclictest	1671868	-21	kworker/u16:2+flush-8:0	04:58:21	5
1485943	99	34	32,2	cyclictest	1851857	-21	kworker/u16:3+events_unbound	06:54:45	5
1485943	99	34	32,2	cyclictest	1762685	-21	kworker/u16:0+flush-8:0	05:45:31	5
1485943	99	34	31,2	cyclictest	1570528	-21	kworker/u16:0+events_unbound	04:33:22	5
1485947	99	33	0,31	cyclictest	2126598	-21	latency_hist	08:38:21	7
1485946	99	33	31,2	cyclictest	1671868	-21	kworker/u16:2+flush-8:0	05:18:21	6
1485946	99	33	31,2	cyclictest	1671868	-21	kworker/u16:2+flush-8:0	05:18:21	6
1485943	99	33	31,2	cyclictest	2015482	-21	kworker/u16:0+events_unbound	08:27:31	5
1485943	99	33	31,2	cyclictest	1954432	-21	kworker/u16:0+events_unbound	07:33:45	5
1485943	99	33	31,2	cyclictest	1914607	-21	kworker/u16:5+events_unbound	07:04:45	5
1485943	99	33	31,2	cyclictest	1671868	-21	kworker/u16:2+events_unbound	05:16:23	5
1485943	99	33	31,2	cyclictest	1671868	-21	kworker/u16:2+events_unbound	05:16:23	5
1485943	99	33	31,2	cyclictest	1610914	-21	kworker/u16:1+events_unbound	04:28:44	5
1485943	99	33	31,2	cyclictest	1610914	-21	kworker/u16:1+events_unbound	04:28:44	5
1485943	99	33	31,2	cyclictest	1415718	-21	kworker/u16:1+events_unbound	03:32:31	5
1485943	99	33	30,2	cyclictest	1762685	-21	kworker/u16:0+events_unbound	05:51:24	5
1485943	99	32	30,2	cyclictest	1851857	-21	kworker/u16:3+events_unbound	06:52:03	5
1485943	99	32	30,2	cyclictest	1610914	-21	kworker/u16:1+events_unbound	04:21:48	5
1485943	99	32	30,2	cyclictest	1539407	-21	kworker/u16:5+events_unbound	03:55:31	5
1485939	99	32	30,2	cyclictest	1712290	-21	kworker/u16:1+flush-8:0	06:18:21	4
1485947	99	31	30,0	cyclictest	0	-21	swapper/7	05:46:21	7
1485943	99	31	29,2	cyclictest	1762685	-21	kworker/u16:0+flush-8:0	05:41:21	5
1485943	99	31	29,2	cyclictest	1671641	-21	kworker/u16:0+events_unbound	05:02:19	5
1485943	99	31	29,2	cyclictest	1399252	-21	kworker/u16:0+events_unbound	03:51:56	5
1485939	99	31	28,2	cyclictest	2005034	-21	kworker/u16:1+flush-8:0	08:08:46	4
1485939	99	31	27,3	cyclictest	1914607	-21	kworker/u16:5+flush-8:0	07:23:20	4
1485947	99	30	29,0	cyclictest	0	-21	swapper/7	07:31:55	7
1485947	99	30	29,0	cyclictest	0	-21	swapper/7	07:31:55	7
1485947	99	30	29,0	cyclictest	0	-21	swapper/7	05:57:26	7
1485947	99	30	29,0	cyclictest	0	-21	swapper/7	05:28:36	7
1485943	99	30	28,2	cyclictest	2005034	-21	kworker/u16:1+flush-8:0	07:43:22	5
1485943	99	30	28,2	cyclictest	1914607	-21	kworker/u16:5+events_unbound	07:24:55	5
1485943	99	30	28,2	cyclictest	1641365	-21	kworker/u16:3+events_unbound	04:43:30	5
1485943	99	30	28,2	cyclictest	1520262	-21	kworker/u16:3+events_unbound	03:44:40	5
1485943	99	30	28,2	cyclictest	1520262	-21	kworker/u16:3+events_unbound	03:44:40	5
1485939	99	30	27,2	cyclictest	2015482	-21	kworker/u16:0+events_unbound	08:48:34	4
1485939	99	30	27,2	cyclictest	1475431	-21	kworker/u16:2+flush-8:0	03:23:39	4
1485939	99	30	26,3	cyclictest	1475431	-21	kworker/u16:2+flush-8:0	03:33:19	4
1485947	99	29	28,0	cyclictest	0	-21	swapper/7	06:35:10	7
1485947	99	29	28,0	cyclictest	0	-21	swapper/7	05:20:20	7
1485947	99	29	23,5	cyclictest	1661751	-21	latency_hist	04:48:22	7
1485946	99	29	27,2	cyclictest	1399252	-21	kworker/u16:0+flush-8:0	03:28:23	6
1485943	99	29	27,2	cyclictest	2106804	-21	kworker/u16:1+events_unbound	08:31:02	5
1485943	99	29	27,2	cyclictest	2055612	-21	kworker/u16:3+flush-8:0	08:08:11	5
1485943	99	29	27,2	cyclictest	1989701	-21	kworker/u16:2+flush-8:0	07:49:11	5
1485943	99	29	27,2	cyclictest	1930062	-21	kworker/u16:1+events_unbound	07:08:42	5
1485943	99	29	27,2	cyclictest	1875531	-21	kworker/u16:0+events_unbound	06:36:21	5
1485943	99	29	27,2	cyclictest	1671641	-21	kworker/u16:0+events_unbound	05:25:31	5
1485943	99	29	27,2	cyclictest	1520262	-21	kworker/u16:3+events_unbound	03:59:47	5
1485943	99	29	26,2	cyclictest	1823755	-21	kworker/u16:4+flush-8:0	06:28:31	5
1485939	99	29	27,2	cyclictest	1671641	-21	kworker/u16:0+flush-8:0	05:23:50	4
1485947	99	28	27,0	cyclictest	0	-21	swapper/7	08:42:23	7
1485947	99	28	27,0	cyclictest	0	-21	swapper/7	06:30:59	7
1485946	99	28	26,2	cyclictest	1851857	-21	kworker/u16:3+events_unbound	06:18:50	6
1485946	99	28	26,2	cyclictest	1641365	-21	kworker/u16:3+events_unbound	04:38:21	6
1485946	99	28	25,2	cyclictest	2005034	-21	kworker/u16:1+flush-8:0	07:38:37	6
1485946	99	28	25,2	cyclictest	1539407	-21	kworker/u16:5+flush-8:0	04:03:38	6
1485943	99	28	26,2	cyclictest	2065807	-21	kworker/u16:2+events_unbound	08:41:02	5
1485943	99	28	26,2	cyclictest	2065807	-21	kworker/u16:2+events_unbound	08:35:26	5
1485943	99	28	26,2	cyclictest	1802190	-21	kworker/u16:2+events_unbound	05:54:32	5
1485943	99	28	26,2	cyclictest	1741727	-21	kworker/u16:4+events_unbound	05:32:11	5
1485943	99	28	26,2	cyclictest	1682337	-21	kworker/u16:3+events_unbound	05:12:39	5
1485943	99	28	26,2	cyclictest	1682337	-21	kworker/u16:3+events_unbound	05:04:28	5
1485943	99	28	26,2	cyclictest	1641616	-21	kworker/u16:4+events_unbound	04:42:15	5
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17