OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Tue Mar 03, 2026 12:43:34)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
3360800	2	55	45,7	sleep5	0	-21	swapper/5	07:01:22	5
3361322	99	49	46,2	cyclictest	3666129	-21	kworker/u16:3+flush-8:0	10:34:35	6
3361322	99	49	46,2	cyclictest	3666129	-21	kworker/u16:3+flush-8:0	10:34:35	6
3361318	99	43	39,3	cyclictest	3920781	-21	kworker/u16:1+flush-8:0	11:44:39	5
3361316	99	43	41,2	cyclictest	3692732	-21	kworker/u16:1+flush-8:0	10:44:59	4
3360837	2	43	33,7	sleep6	0	-21	swapper/6	07:01:55	6
3360770	2	43	33,7	sleep4	0	-21	swapper/4	07:00:57	4
3361322	99	42	40,2	cyclictest	3787739	-21	kworker/u16:0+events_unbound	10:45:35	6
3361318	99	42	40,2	cyclictest	3451652	-21	kworker/u16:0+flush-8:0	07:54:34	5
3361322	99	41	39,2	cyclictest	3412890	-21	kworker/u16:3+flush-8:0	08:34:34	6
3360764	2	41	5,31	sleep7	0	-21	swapper/7	07:00:52	7
3361326	99	40	33,6	cyclictest	3994336	-21	latency_hist	11:49:34	7
3361322	99	40	38,2	cyclictest	3920781	-21	kworker/u16:1+flush-8:0	12:30:24	6
3361322	99	39	37,2	cyclictest	3596134	-21	kworker/u16:3+events_unbound	09:14:31	6
3361322	99	39	37,2	cyclictest	3596134	-21	kworker/u16:3+events_unbound	09:14:31	6
3361326	99	38	32,5	cyclictest	3813210	-21	latency_hist	10:34:34	7
3361322	99	38	36,2	cyclictest	3825404	-21	kworker/u16:4+flush-8:0	11:04:59	6
3361318	99	38	36,2	cyclictest	3451652	-21	kworker/u16:0+flush-8:0	08:04:34	5
3361322	99	37	35,2	cyclictest	3920781	-21	kworker/u16:1+events_unbound	11:48:39	6
3361322	99	37	35,2	cyclictest	3825404	-21	kworker/u16:4+events_unbound	11:36:45	6
3361322	99	37	35,2	cyclictest	3787739	-21	kworker/u16:0+events_unbound	10:30:44	6
3361322	99	37	34,2	cyclictest	3340991	-21	kworker/u16:4+flush-8:0	07:05:05	6
3361322	99	36	34,2	cyclictest	3881650	-21	kworker/u16:0+events_unbound	11:00:56	6
3361322	99	36	34,2	cyclictest	3692732	-21	kworker/u16:1+events_unbound	10:06:32	6
3361322	99	36	34,2	cyclictest	3666129	-21	kworker/u16:3+flush-8:0	09:45:07	6
3361322	99	36	34,2	cyclictest	3666129	-21	kworker/u16:3+events_unbound	10:29:20	6
3361318	99	36	33,2	cyclictest	3920781	-21	kworker/u16:1+flush-8:0	12:24:44	5
3361318	99	36	33,2	cyclictest	3920781	-21	kworker/u16:1+events_unbound	11:34:44	5
3361322	99	35	33,2	cyclictest	4018591	-21	kworker/u16:0+events_unbound	12:26:08	6
3361322	99	35	33,2	cyclictest	3859241	-21	kworker/u16:2+flush-8:0	11:32:24	6
3361322	99	35	33,2	cyclictest	3666129	-21	kworker/u16:3+events_unbound	10:03:44	6
3361318	99	35	33,2	cyclictest	3666129	-21	kworker/u16:3+flush-8:0	10:00:36	5
3361318	99	35	33,2	cyclictest	3666129	-21	kworker/u16:3+flush-8:0	09:34:34	5
3361322	99	34	32,2	cyclictest	3666129	-21	kworker/u16:3+events_unbound	10:42:28	6
3361322	99	34	32,2	cyclictest	3666129	-21	kworker/u16:3+events_unbound	09:52:32	6
3361322	99	34	32,2	cyclictest	3517793	-21	kworker/u16:2+events_unbound	09:41:39	6
3361322	99	34	32,2	cyclictest	3340991	-21	kworker/u16:4+flush-8:0	07:26:40	6
3361318	99	34	32,2	cyclictest	3692732	-21	kworker/u16:1+events_unbound	10:09:36	5
3919662	2	33	0,1	chrt	3919660	-21	ssh	11:15:18	7
3919662	2	33	0,1	chrt	3919660	-21	ssh	11:15:18	7
3361322	99	33	32,1	cyclictest	3982026	-21	kworker/u16:2+events_unbound	11:57:07	6
3361322	99	33	32,1	cyclictest	3982026	-21	kworker/u16:2+events_unbound	11:57:07	6
3361322	99	33	31,2	cyclictest	3982026	-21	kworker/u16:2+events_unbound	11:49:39	6
3361322	99	33	31,2	cyclictest	3825404	-21	kworker/u16:4+events_unbound	12:20:03	6
3361318	99	33	31,2	cyclictest	3825404	-21	kworker/u16:4+events_unbound	11:04:56	5
3361322	99	32	30,2	cyclictest	3958316	-21	kworker/u16:0+events_unbound	11:42:55	6
3361322	99	32	30,2	cyclictest	3825404	-21	kworker/u16:4+flush-8:0	11:25:12	6
3361322	99	32	30,2	cyclictest	3787739	-21	kworker/u16:0+events_unbound	10:49:36	6
3361322	99	32	30,2	cyclictest	3692732	-21	kworker/u16:1+flush-8:0	10:15:08	6
3361318	99	32	30,2	cyclictest	3666129	-21	kworker/u16:3+events_unbound	09:50:07	5
3361326	99	31	25,5	cyclictest	3595658	-21	latency_hist	09:04:35	7
3361322	99	31	29,2	cyclictest	3920781	-21	kworker/u16:1+flush-8:0	12:10:11	6
3361322	99	31	29,2	cyclictest	3859241	-21	kworker/u16:2+events_unbound	11:20:36	6
3361322	99	31	29,2	cyclictest	3619752	-21	kworker/u16:1+events_unbound	09:16:52	6
3361318	99	31	29,2	cyclictest	3982026	-21	kworker/u16:2+flush-8:0	11:50:07	5
3361318	99	31	28,2	cyclictest	3859241	-21	kworker/u16:2+flush-8:0	11:09:34	5
3361316	99	31	29,2	cyclictest	3361574	-21	kworker/u16:1+flush-8:0	07:09:56	4
3361316	99	31	29,2	cyclictest	3340991	-21	kworker/u16:4+flush-8:0	07:19:54	4
3361322	99	30	28,2	cyclictest	3920781	-21	kworker/u16:1+events_unbound	12:09:26	6
3361322	99	30	28,2	cyclictest	3825404	-21	kworker/u16:4+events_unbound	12:00:40	6
3361322	99	30	28,2	cyclictest	3825404	-21	kworker/u16:4+events_unbound	11:12:12	6
3361322	99	30	28,2	cyclictest	3692732	-21	kworker/u16:1+events_unbound	10:10:48	6
3361322	99	30	28,2	cyclictest	3451652	-21	kworker/u16:0+flush-8:0	09:25:11	6
3361322	99	30	28,2	cyclictest	3451652	-21	kworker/u16:0+events_unbound	09:22:24	6
3361322	99	29	27,2	cyclictest	3859241	-21	kworker/u16:2+events_unbound	11:15:25	6
3361322	99	29	27,2	cyclictest	3859241	-21	kworker/u16:2+events_unbound	11:15:25	6
3361322	99	29	27,2	cyclictest	3666129	-21	kworker/u16:3+flush-8:0	09:35:15	6
3361322	99	29	27,2	cyclictest	3619752	-21	kworker/u16:1+events_unbound	09:30:40	6
3361322	99	29	27,2	cyclictest	3451652	-21	kworker/u16:0+events_unbound	09:09:16	6
3361318	99	29	27,2	cyclictest	3692732	-21	kworker/u16:1+flush-8:0	10:05:24	5
3361318	99	29	25,3	cyclictest	3340991	-21	kworker/u16:4+flush-8:0	07:59:34	5
3361316	99	29	27,2	cyclictest	3920781	-21	kworker/u16:1+flush-8:0	11:59:49	4
3361316	99	29	27,2	cyclictest	3825404	-21	kworker/u16:4+events_unbound	11:24:58	4
3361316	99	29	27,2	cyclictest	3692732	-21	kworker/u16:1+events_unbound	10:04:45	4
3361326	99	28	26,1	cyclictest	0	-21	swapper/7	11:49:50	7
3361326	99	28	23,4	cyclictest	3740601	-21	cat	10:04:29	7
3361326	99	28	0,27	cyclictest	650	-21	systemd-journal	12:04:33	7
3361322	99	28	26,2	cyclictest	3920781	-21	kworker/u16:1+events_unbound	12:16:04	6
3361322	99	28	26,2	cyclictest	3692732	-21	kworker/u16:1+events_unbound	10:55:43	6
3361318	99	28	26,2	cyclictest	3920781	-21	kworker/u16:1+flush-8:0	11:55:31	5
3361318	99	28	26,2	cyclictest	3920781	-21	kworker/u16:1+flush-8:0	11:55:31	5
3361318	99	28	26,2	cyclictest	3666129	-21	kworker/u16:3+events_unbound	09:38:08	5
3361316	99	28	26,2	cyclictest	3451652	-21	kworker/u16:0+events_unbound	09:14:47	4
3361316	99	28	26,2	cyclictest	3340991	-21	kworker/u16:4+flush-8:0	07:50:00	4
3361322	99	27	25,2	cyclictest	3451652	-21	kworker/u16:0+events_unbound	09:55:43	6
3361322	99	27	25,2	cyclictest	3340991	-21	kworker/u16:4+flush-8:0	07:14:53	6
3361322	99	27	23,3	cyclictest	3462104	-21	diskstats	07:54:48	6
3361316	99	27	25,2	cyclictest	3920781	-21	kworker/u16:1+events_unbound	11:54:46	4
3361316	99	27	25,2	cyclictest	3920781	-21	kworker/u16:1+events_unbound	11:54:46	4
3361316	99	27	25,2	cyclictest	3787739	-21	kworker/u16:0+flush-8:0	10:50:02	4
3361316	99	27	25,2	cyclictest	3787739	-21	kworker/u16:0+flush-8:0	10:44:29	4
3361316	99	27	24,2	cyclictest	3340991	-21	kworker/u16:4+events_unbound	08:04:57	4
3931923	2	26	0,1	chrt	3931922	-21	ssh	11:20:39	7
3416736	2	26	0,1	chrt	0	-21	swapper/7	07:29:51	7
3361322	99	26	24,2	cyclictest	3692732	-21	kworker/u16:1+events_unbound	10:22:59	6
3361318	99	26	24,2	cyclictest	3666129	-21	kworker/u16:3+flush-8:0	10:35:08	5
3361318	99	26	24,2	cyclictest	3666129	-21	kworker/u16:3+flush-8:0	10:35:08	5
3361316	99	26	24,2	cyclictest	3825404	-21	kworker/u16:4+events_unbound	12:04:46	4
3361322	99	25	22,2	cyclictest	3517793	-21	kworker/u16:2+events_unbound	08:49:36	6
3361318	99	25	23,2	cyclictest	3920781	-21	kworker/u16:1+events_unbound	12:31:15	5
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17