OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Fri Jul 17, 2026 12:43:29)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
898115	2	96	0,1	sleep7	898114	-21	nfs_client	11:10:14	7
1096905	2	95	0,1	sleep7	1096906	-21	rm	12:31:08	7
1032728	2	94	0,1	sleep4	0	-21	swapper/4	12:05:08	4
355271	2	56	21,30	sleep7	0	-21	swapper/7	07:07:18	7
355316	2	54	43,7	sleep5	0	-21	swapper/5	07:07:56	5
355728	99	42	40,2	cyclictest	572712	-21	kworker/u16:0+flush-8:0	12:15:00	4
355238	2	37	26,8	sleep4	0	-21	swapper/4	07:06:51	4
355733	99	36	34,2	cyclictest	582847	-21	kworker/u16:3+events_unbound	09:19:47	6
355728	99	36	34,2	cyclictest	572712	-21	kworker/u16:0+flush-8:0	11:44:59	4
355728	99	35	32,2	cyclictest	572712	-21	kworker/u16:0+flush-8:0	11:30:03	4
355733	99	34	32,2	cyclictest	765622	-21	kworker/u16:4+events_unbound	11:57:46	6
355733	99	34	32,2	cyclictest	572712	-21	kworker/u16:0+flush-8:0	12:34:59	6
355728	99	34	31,2	cyclictest	827542	-21	kworker/u16:1+events_unbound	10:59:55	4
355728	99	34	30,3	cyclictest	572712	-21	kworker/u16:0+flush-8:0	11:40:10	4
355733	99	32	30,2	cyclictest	572712	-21	kworker/u16:0+events_unbound	10:51:21	6
355733	99	32	30,2	cyclictest	1046532	-21	kworker/u16:1+events_unbound	12:18:58	6
355728	99	32	30,2	cyclictest	373502	-21	kworker/u16:3+flush-8:0	07:25:01	4
355299	2	32	21,8	sleep6	0	-21	swapper/6	07:07:41	6
355733	99	31	29,2	cyclictest	586769	-21	kworker/u16:4+events_unbound	09:09:20	6
355733	99	31	29,2	cyclictest	1046532	-21	kworker/u16:1+events_unbound	12:31:42	6
355733	99	31	25,6	cyclictest	765622	-21	kworker/u16:4+events_unbound	10:24:29	6
929377	2	30	0,1	chrt	929378	-21	kthreadcore	11:25:04	7
355733	99	30	28,2	cyclictest	572712	-21	kworker/u16:0+flush-8:0	09:34:11	6
355733	99	29	27,2	cyclictest	713434	-21	kworker/u16:3+events_unbound	10:19:22	6
355733	99	29	27,2	cyclictest	572712	-21	kworker/u16:0+flush-8:0	11:47:17	6
355733	99	29	27,2	cyclictest	572712	-21	kworker/u16:0+events_unbound	11:38:13	6
355733	99	29	27,2	cyclictest	572712	-21	kworker/u16:0+events_unbound	10:47:58	6
355733	99	29	27,2	cyclictest	1024947	-21	kworker/u16:6+events_unbound	12:20:10	6
355728	99	29	27,2	cyclictest	689593	-21	kworker/u16:2+flush-8:0	09:50:11	4
495936	2	28	0,1	chrt	0	-21	swapper/4	08:19:58	4
355733	99	28	26,2	cyclictest	960202	-21	kworker/u16:2+events_unbound	11:42:02	6
355733	99	28	26,2	cyclictest	827542	-21	kworker/u16:1+events_unbound	10:57:01	6
355733	99	28	26,2	cyclictest	765622	-21	kworker/u16:4+events_unbound	12:03:53	6
355733	99	28	26,2	cyclictest	713434	-21	kworker/u16:3+flush-8:0	10:08:49	6
355733	99	28	26,2	cyclictest	713434	-21	kworker/u16:3+flush-8:0	10:08:49	6
355733	99	28	26,2	cyclictest	626922	-21	kworker/u16:1+flush-8:0	09:46:13	6
355733	99	28	26,2	cyclictest	626922	-21	kworker/u16:1+events_unbound	09:36:42	6
355733	99	28	26,2	cyclictest	572712	-21	kworker/u16:0+events_unbound	12:12:04	6
355733	99	28	26,2	cyclictest	465900	-21	kworker/u16:1+events_unbound	08:23:01	6
355733	99	28	26,2	cyclictest	465900	-21	kworker/u16:1+events_unbound	08:04:47	6
355733	99	28	26,2	cyclictest	447792	-21	kworker/u16:4+events_unbound	08:19:36	6
355733	99	28	25,2	cyclictest	524611	-21	kworker/u16:3+events_unbound	08:39:59	6
355728	99	28	26,2	cyclictest	713434	-21	kworker/u16:3+events_unbound	10:07:15	4
355728	99	28	26,2	cyclictest	713434	-21	kworker/u16:3+events_unbound	10:07:15	4
355728	99	28	26,2	cyclictest	365474	-21	kworker/u16:2+flush-8:0	07:39:46	4
355728	99	28	25,2	cyclictest	572712	-21	kworker/u16:0+events_unbound	10:54:48	4
355733	99	27	25,2	cyclictest	765622	-21	kworker/u16:4+events_unbound	11:49:57	6
355733	99	27	25,2	cyclictest	765622	-21	kworker/u16:4+events_unbound	11:24:54	6
355733	99	27	25,2	cyclictest	765622	-21	kworker/u16:4+events_unbound	10:28:02	6
355733	99	27	25,2	cyclictest	572712	-21	kworker/u16:0+flush-8:0	11:10:56	6
355733	99	27	25,2	cyclictest	572712	-21	kworker/u16:0+flush-8:0	10:00:17	6
355733	99	27	25,2	cyclictest	572712	-21	kworker/u16:0+events_unbound	10:12:27	6
355733	99	27	25,2	cyclictest	418007	-21	kworker/u16:0+events_unbound	07:41:25	6
355733	99	27	25,2	cyclictest	345912	-21	kworker/u16:0+events_unbound	07:21:00	6
355733	99	27	25,2	cyclictest	345912	-21	kworker/u16:0+events_unbound	07:14:00	6
355728	99	27	25,2	cyclictest	778005	-21	kworker/u16:5+flush-8:0	10:38:51	4
1036327	2	27	0,1	chrt	1036328	-21	sh	12:08:08	7
738234	2	26	0,1	chrt	738232	-21	ssh	10:05:46	7
738234	2	26	0,1	chrt	738232	-21	ssh	10:05:46	7
355733	99	26	25,1	cyclictest	765622	-21	kworker/u16:4+events_unbound	10:37:46	6
355733	99	26	24,2	cyclictest	827542	-21	kworker/u16:1+events_unbound	10:59:49	6
355733	99	26	24,2	cyclictest	778005	-21	kworker/u16:5+flush-8:0	10:44:09	6
355733	99	26	24,2	cyclictest	713434	-21	kworker/u16:3+flush-8:0	09:55:46	6
355733	99	26	24,2	cyclictest	689593	-21	kworker/u16:2+events_unbound	09:51:41	6
355733	99	26	24,2	cyclictest	582847	-21	kworker/u16:3+events_unbound	09:11:06	6
355733	99	26	24,2	cyclictest	467777	-21	kworker/u16:2+flush-8:0	09:00:47	6
355733	99	26	24,2	cyclictest	447792	-21	kworker/u16:4+events_unbound	07:59:20	6
355733	99	26	24,2	cyclictest	365474	-21	kworker/u16:2+events_unbound	07:46:15	6
355733	99	26	23,2	cyclictest	345912	-21	kworker/u16:0+events_unbound	07:29:48	6
355728	99	26	24,2	cyclictest	689593	-21	kworker/u16:2+flush-8:0	10:03:20	4
355728	99	26	24,2	cyclictest	572712	-21	kworker/u16:0+flush-8:0	12:11:56	4
355737	99	25	24,0	cyclictest	0	-21	swapper/7	09:20:03	7
355733	99	25	23,2	cyclictest	765622	-21	kworker/u16:4+events_unbound	11:31:38	6
355733	99	25	23,2	cyclictest	765622	-21	kworker/u16:4+events_unbound	11:24:34	6
355733	99	25	23,2	cyclictest	586769	-21	kworker/u16:4+events_unbound	09:42:06	6
355733	99	25	23,2	cyclictest	572712	-21	kworker/u16:0+events_unbound	09:27:34	6
355733	99	25	23,2	cyclictest	465900	-21	kworker/u16:1+events_unbound	08:29:08	6
355733	99	25	23,2	cyclictest	447792	-21	kworker/u16:4+events_unbound	08:14:35	6
355733	99	25	23,2	cyclictest	365474	-21	kworker/u16:2+events_unbound	07:38:37	6
355733	99	25	23,2	cyclictest	345912	-21	kworker/u16:0+events_unbound	07:27:09	6
355728	99	25	23,2	cyclictest	765622	-21	kworker/u16:4+flush-8:0	11:35:19	4
355728	99	25	23,2	cyclictest	572712	-21	kworker/u16:0+flush-8:0	09:37:35	4
355728	99	25	23,2	cyclictest	572712	-21	kworker/u16:0+events_unbound	12:37:14	4
355728	99	25	22,2	cyclictest	572712	-21	kworker/u16:0+flush-8:0	10:49:46	4
355728	99	25	21,2	cyclictest	365474	-21	kworker/u16:2+events_unbound	07:40:01	4
355733	99	24	22,2	cyclictest	765622	-21	kworker/u16:4+events_unbound	11:16:49	6
355733	99	24	22,2	cyclictest	582847	-21	kworker/u16:3+events_unbound	09:17:25	6
355733	99	24	22,2	cyclictest	572712	-21	kworker/u16:0+events_unbound	12:26:41	6
355733	99	24	22,2	cyclictest	572712	-21	kworker/u16:0+events_unbound	10:33:26	6
355733	99	24	22,2	cyclictest	553654	-21	kworker/u16:4+events_unbound	08:54:00	6
355731	99	24	22,2	cyclictest	765622	-21	kworker/u16:4+events_unbound	11:14:56	5
355731	99	24	22,2	cyclictest	572712	-21	kworker/u16:0+flush-8:0	09:19:40	5
355731	99	24	22,2	cyclictest	572712	-21	kworker/u16:0+events_unbound	12:35:03	5
355728	99	24	22,2	cyclictest	960202	-21	kworker/u16:2+flush-8:0	11:59:55	4
355728	99	24	22,2	cyclictest	572712	-21	kworker/u16:0+flush-8:0	10:50:52	4
355728	99	24	22,2	cyclictest	572712	-21	kworker/u16:0+flush-8:0	09:45:22	4
355728	99	24	21,2	cyclictest	467777	-21	kworker/u16:2+flush-8:0	08:35:04	4
355737	99	23	22,0	cyclictest	0	-21	swapper/7	09:17:19	7
355733	99	23	21,2	cyclictest	572712	-21	kworker/u16:0+events_unbound	11:08:09	6
355733	99	23	21,2	cyclictest	495727	-21	kworker/u16:0+events_unbound	08:31:21	6
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17