OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Sun Jun 07, 2026 12:43:34)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
2981340	2	96	0,1	sleep7	2981341	-21	grep	11:56:45	7
2333225	99	53	50,2	cyclictest	2991243	-21	kworker/u16:1+flush-8:0	12:11:43	4
2332700	2	53	41,8	sleep5	0	-21	swapper/5	07:03:14	5
2333232	99	46	41,4	cyclictest	2953811	-21	latency_hist	11:46:25	7
2333229	99	45	40,4	cyclictest	2567496	-21	kworker/u16:3+flush-8:0	10:06:26	5
2323776	2	43	32,7	sleep4	0	-21	swapper/4	07:01:28	4
2333232	99	42	37,4	cyclictest	2833182	-21	latency_hist	10:56:26	7
2333230	99	41	37,3	cyclictest	2323430	-21	kworker/u16:2+flush-8:0	07:46:26	6
2333230	99	41	37,3	cyclictest	2323430	-21	kworker/u16:2+flush-8:0	07:46:26	6
2333229	99	41	39,2	cyclictest	2881473	-21	kworker/u16:2+flush-8:0	11:27:00	5
2333229	99	41	39,2	cyclictest	2613512	-21	kworker/u16:0+events_unbound	10:31:59	5
2331935	2	41	3,31	sleep7	0	-21	swapper/7	07:01:55	7
2333230	99	38	36,2	cyclictest	2991243	-21	kworker/u16:1+flush-8:0	12:01:56	6
2333229	99	38	36,2	cyclictest	2881473	-21	kworker/u16:2+events_unbound	11:31:57	5
2333225	99	38	35,2	cyclictest	2529008	-21	kworker/u16:2+events_unbound	10:36:27	4
2333232	99	37	35,1	cyclictest	0	-21	swapper/7	10:31:41	7
2333229	99	37	34,2	cyclictest	3037682	-21	kworker/u16:0+flush-8:0	12:26:43	5
2333229	99	37	34,2	cyclictest	3037682	-21	kworker/u16:0+flush-8:0	12:26:43	5
2333225	99	36	33,2	cyclictest	2917900	-21	kworker/u16:4+flush-8:0	12:31:54	4
2333225	99	35	33,2	cyclictest	2442232	-21	kworker/u16:1+events_unbound	07:57:40	4
2325107	2	35	5,7	sleep6	0	-21	swapper/6	07:01:46	6
2333229	99	34	32,2	cyclictest	2881473	-21	kworker/u16:2+flush-8:0	12:36:25	5
2333225	99	34	32,2	cyclictest	2893941	-21	kworker/u16:0+events_unbound	11:51:59	4
2333225	99	34	32,2	cyclictest	2881473	-21	kworker/u16:2+flush-8:0	12:06:41	4
2333232	99	33	0,31	cyclictest	2869441	-21	systemd-run	11:11:25	7
2333232	99	33	0,31	cyclictest	2869441	-21	systemd-run	11:11:25	7
2333229	99	33	31,2	cyclictest	2529008	-21	kworker/u16:2+events_unbound	09:21:26	5
2333225	99	33	30,2	cyclictest	2843155	-21	kworker/u16:1+events_unbound	11:01:26	4
2333230	99	32	29,2	cyclictest	2423345	-21	kworker/u16:3+flush-8:0	07:57:33	6
2333229	99	32	30,2	cyclictest	2917900	-21	kworker/u16:4+events_unbound	11:46:58	5
2333229	99	32	30,2	cyclictest	2893941	-21	kworker/u16:0+events_unbound	11:51:39	5
2333229	99	32	30,2	cyclictest	2881473	-21	kworker/u16:2+events_unbound	11:56:58	5
2333229	99	32	30,2	cyclictest	2808972	-21	kworker/u16:3+events_unbound	11:03:35	5
2333229	99	32	30,2	cyclictest	2613512	-21	kworker/u16:0+events_unbound	09:26:13	5
2333225	99	32	30,2	cyclictest	2881473	-21	kworker/u16:2+events_unbound	11:26:58	4
2333225	99	32	30,2	cyclictest	2881473	-21	kworker/u16:2+events_unbound	11:21:59	4
2333225	99	32	29,2	cyclictest	2613512	-21	kworker/u16:0+events_unbound	10:16:23	4
2333229	99	31	29,2	cyclictest	2567496	-21	kworker/u16:3+events_unbound	09:50:47	5
2333229	99	31	29,2	cyclictest	2567496	-21	kworker/u16:3+events_unbound	09:50:47	5
2333225	99	31	29,2	cyclictest	2917900	-21	kworker/u16:4+events_unbound	11:36:40	4
2770359	2	30	0,1	chrt	2770360	-21	sh	10:26:52	7
2770359	2	30	0,1	chrt	2770360	-21	sh	10:26:52	7
2333232	99	30	28,1	cyclictest	0	-21	swapper/7	09:56:44	7
2333229	99	30	28,2	cyclictest	2917900	-21	kworker/u16:4+events_unbound	12:06:40	5
2333229	99	30	28,2	cyclictest	2529008	-21	kworker/u16:2+events_unbound	09:36:56	5
2333229	99	30	27,2	cyclictest	3037682	-21	kworker/u16:0+events_unbound	12:21:39	5
2333232	99	29	0,27	cyclictest	2919259	-21	irqcore	11:31:42	7
2333230	99	29	28,1	cyclictest	2529008	-21	kworker/u16:2+events_unbound	10:13:23	6
2333229	99	29	26,2	cyclictest	2773323	-21	kworker/u16:1+events_unbound	10:39:48	5
2333225	99	29	27,2	cyclictest	2773323	-21	kworker/u16:1+events_unbound	10:46:23	4
2333230	99	28	26,2	cyclictest	2442232	-21	kworker/u16:1+events_unbound	08:51:44	6
2333230	99	28	26,2	cyclictest	2323430	-21	kworker/u16:2+events_unbound	07:31:50	6
2333229	99	28	26,2	cyclictest	2917900	-21	kworker/u16:4+events_unbound	12:04:47	5
2333229	99	28	26,2	cyclictest	2881473	-21	kworker/u16:2+events_unbound	11:41:53	5
2333229	99	28	26,2	cyclictest	2881473	-21	kworker/u16:2+events_unbound	11:37:47	5
2333229	99	28	26,2	cyclictest	2567496	-21	kworker/u16:3+events_unbound	09:53:47	5
2333225	99	28	26,2	cyclictest	2917900	-21	kworker/u16:4+events_unbound	12:01:59	4
2333232	99	27	25,1	cyclictest	0	-21	swapper/7	09:11:49	7
2333229	99	27	25,2	cyclictest	2808972	-21	kworker/u16:3+events_unbound	10:51:24	5
2333225	99	27	25,2	cyclictest	2529008	-21	kworker/u16:2+flush-8:0	08:51:55	4
2333225	99	27	25,2	cyclictest	2529008	-21	kworker/u16:2+events_unbound	11:06:56	4
2333225	99	27	25,2	cyclictest	2529008	-21	kworker/u16:2+events_unbound	11:06:56	4
2333225	99	27	25,2	cyclictest	2529008	-21	kworker/u16:2+events_unbound	10:56:21	4
2333230	99	26	24,2	cyclictest	2442232	-21	kworker/u16:1+flush-8:0	08:26:26	6
2333230	99	26	24,2	cyclictest	2269992	-21	kworker/u16:3+flush-8:0	07:16:48	6
2333229	99	26	24,2	cyclictest	2881473	-21	kworker/u16:2+events_unbound	11:21:41	5
2333229	99	26	24,2	cyclictest	2529008	-21	kworker/u16:2+events_unbound	09:42:48	5
2333225	99	26	24,2	cyclictest	2991243	-21	kworker/u16:1+events_unbound	12:16:44	4
2333229	99	25	23,2	cyclictest	2808972	-21	kworker/u16:3+events_unbound	10:53:23	5
2333229	99	25	23,2	cyclictest	2529008	-21	kworker/u16:2+flush-8:0	11:06:47	5
2333229	99	25	23,2	cyclictest	2529008	-21	kworker/u16:2+flush-8:0	11:06:47	5
2333225	99	25	23,2	cyclictest	2529008	-21	kworker/u16:2+events_unbound	09:42:23	4
2333225	99	25	23,2	cyclictest	2500144	-21	kworker/u16:0+flush-8:0	09:06:51	4
2333225	99	25	23,2	cyclictest	2500144	-21	kworker/u16:0+flush-8:0	09:06:51	4
2333225	99	25	22,2	cyclictest	2423345	-21	kworker/u16:3+events_unbound	08:36:45	4
2333230	99	24	22,2	cyclictest	2567496	-21	kworker/u16:3+flush-8:0	09:11:42	6
2333230	99	24	20,3	cyclictest	2323430	-21	kworker/u16:2+flush-8:0	07:51:26	6
2333229	99	24	22,2	cyclictest	2613512	-21	kworker/u16:0+flush-8:0	10:01:21	5
2333229	99	24	22,2	cyclictest	2613512	-21	kworker/u16:0+events_unbound	10:28:53	5
2333229	99	24	22,2	cyclictest	2613512	-21	kworker/u16:0+events_unbound	10:28:53	5
2333225	99	24	23,1	cyclictest	2567496	-21	kworker/u16:3+events_unbound	09:39:44	4
2333225	99	24	22,2	cyclictest	2893941	-21	kworker/u16:0+events_unbound	11:56:28	4
2333232	99	23	20,1	cyclictest	0	-21	swapper/7	08:26:39	7
2333230	99	23	21,2	cyclictest	2500144	-21	kworker/u16:0+events_unbound	09:11:26	6
2333230	99	23	21,2	cyclictest	2500144	-21	kworker/u16:0+events_unbound	09:11:26	6
2333229	99	23	21,2	cyclictest	2808972	-21	kworker/u16:3+events_unbound	11:16:23	5
2333229	99	23	21,2	cyclictest	2529008	-21	kworker/u16:2+events_unbound	10:09:48	5
2333229	99	23	19,3	cyclictest	2442232	-21	kworker/u16:1+flush-8:0	08:56:53	5
2333225	99	23	21,2	cyclictest	2529008	-21	kworker/u16:2+events_unbound	10:18:36	4
2333225	99	23	20,2	cyclictest	2567496	-21	kworker/u16:3+events_unbound	09:56:27	4
2333225	99	23	20,2	cyclictest	2529008	-21	kworker/u16:2+events_unbound	10:26:27	4
2333225	99	23	20,2	cyclictest	2529008	-21	kworker/u16:2+events_unbound	10:26:27	4
2333225	99	23	20,2	cyclictest	2423345	-21	kworker/u16:3+events_unbound	08:16:39	4
2333230	99	22	20,2	cyclictest	2843155	-21	kworker/u16:1+events_unbound	11:11:41	6
2333230	99	22	20,2	cyclictest	2591702	-21	kworker/u16:4+flush-8:0	09:21:52	6
2333230	99	22	19,2	cyclictest	2881473	-21	kworker/u16:2+flush-8:0	11:46:56	6
2333230	99	22	18,3	cyclictest	2773323	-21	kworker/u16:1+events_unbound	10:41:45	6
2333229	99	22	20,2	cyclictest	2881473	-21	kworker/u16:2+events_unbound	12:16:28	5
2333229	99	22	20,2	cyclictest	2394852	-21	kworker/u16:0+flush-8:0	07:41:52	5
2333229	99	22	20,2	cyclictest	2394852	-21	kworker/u16:0+flush-8:0	07:41:52	5
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17