OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Sat Apr 04, 2026 12:43:34)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
3201218	2	147	0,1	sleep7	3201219	-21	kthreadcore	09:52:48	7
2842698	99	65	60,4	cyclictest	3304672	-21	latency_hist	10:37:26	7
2842693	99	60	56,3	cyclictest	3171266	-21	kworker/u16:3+flush-8:0	09:42:27	6
2833252	2	59	48,7	sleep6	0	-21	swapper/6	06:57:31	6
2842177	2	52	42,7	sleep4	0	-21	swapper/4	06:59:21	4
2842693	99	50	48,2	cyclictest	2875384	-21	kworker/u16:1+flush-8:0	10:17:54	6
2842397	2	48	38,7	sleep5	0	-21	swapper/5	07:02:27	5
2842693	99	46	44,2	cyclictest	2875384	-21	kworker/u16:1+flush-8:0	10:17:22	6
2842693	99	43	40,2	cyclictest	2841486	-21	kworker/u16:0+flush-8:0	07:22:27	6
2842698	99	42	0,40	cyclictest	2970655	-21	latency_hist	08:07:27	7
2833262	2	40	3,31	sleep7	0	-21	swapper/7	06:57:40	7
2842698	99	38	0,36	cyclictest	3461544	-21	latency_hist	11:42:27	7
2842693	99	38	36,2	cyclictest	2913643	-21	kworker/u16:4+flush-8:0	08:07:22	6
2842698	99	37	31,5	cyclictest	3136018	-21	latency_hist	09:27:27	7
2842693	99	37	35,2	cyclictest	3268898	-21	kworker/u16:2+flush-8:0	11:17:38	6
2842698	99	36	30,5	cyclictest	2884548	-21	latency_hist	07:22:27	7
2842693	99	35	31,3	cyclictest	2990215	-21	kworker/u16:0+flush-8:0	08:27:53	6
2842681	99	34	32,2	cyclictest	2875384	-21	kworker/u16:1+flush-8:0	09:47:22	4
2842698	99	33	27,5	cyclictest	650	-21	systemd-journal	10:57:27	7
2842685	99	33	31,2	cyclictest	3268899	-21	kworker/u16:5+flush-8:0	11:24:57	5
2842681	99	33	30,2	cyclictest	3461966	-21	kworker/u16:2+flush-8:0	12:08:22	4
2842685	99	32	29,3	cyclictest	3304108	-21	kworker/u16:0+flush-8:0	11:08:21	5
2842698	99	31	26,4	cyclictest	3401137	-21	latency_hist	11:17:26	7
2842693	99	31	29,2	cyclictest	3316782	-21	kworker/u16:1+events_unbound	11:32:59	6
2842693	99	31	29,2	cyclictest	3087836	-21	kworker/u16:0+flush-8:0	09:32:57	6
2842698	99	30	29,0	cyclictest	0	-21	swapper/7	11:27:39	7
2842693	99	30	28,2	cyclictest	3087836	-21	kworker/u16:0+events_unbound	09:52:58	6
3076421	2	29	0,1	chrt	3076424	-21	cpuspeed_turbos	09:02:39	7
2842693	99	29	27,2	cyclictest	3304108	-21	kworker/u16:0+flush-8:0	10:47:45	6
2842693	99	29	27,2	cyclictest	3268899	-21	kworker/u16:5+events_unbound	12:02:26	6
2842693	99	29	27,2	cyclictest	3268899	-21	kworker/u16:5+events_unbound	12:02:26	6
2842693	99	29	27,2	cyclictest	2823752	-21	kworker/u16:3+flush-8:0	07:17:27	6
2842693	99	29	27,2	cyclictest	2823752	-21	kworker/u16:3+flush-8:0	07:17:27	6
2842693	99	29	26,2	cyclictest	3461966	-21	kworker/u16:2+flush-8:0	12:12:37	6
2842685	99	29	22,6	cyclictest	3304108	-21	kworker/u16:0+flush-8:0	10:42:26	5
2842685	99	29	22,6	cyclictest	3304108	-21	kworker/u16:0+flush-8:0	10:42:26	5
2842681	99	29	27,2	cyclictest	3268898	-21	kworker/u16:2+events_unbound	11:24:31	4
2842693	99	28	25,2	cyclictest	3304108	-21	kworker/u16:0+flush-8:0	10:37:43	6
2842693	99	28	25,2	cyclictest	3304108	-21	kworker/u16:0+flush-8:0	10:37:43	6
2842693	99	27	26,1	cyclictest	3461966	-21	kworker/u16:2+events_unbound	12:27:54	6
2842681	99	27	25,2	cyclictest	3018169	-21	kworker/u16:3+events_unbound	09:27:47	4
2842693	99	26	24,2	cyclictest	3316782	-21	kworker/u16:1+events_unbound	11:12:54	6
2842693	99	26	24,2	cyclictest	3268899	-21	kworker/u16:5+events_unbound	10:42:42	6
2842693	99	26	24,2	cyclictest	3171266	-21	kworker/u16:3+events_unbound	09:47:55	6
2842693	99	26	24,2	cyclictest	2875384	-21	kworker/u16:1+flush-8:0	08:42:53	6
2842681	99	26	22,2	cyclictest	2990215	-21	kworker/u16:0+events_unbound	08:27:45	4
2842698	99	25	23,1	cyclictest	3445424	-21	kthreadcore	11:32:50	7
2842698	99	25	20,4	cyclictest	3377078	-21	latency_hist	11:07:27	7
2842693	99	25	23,2	cyclictest	2875384	-21	kworker/u16:1+flush-8:0	08:37:59	6
2842693	99	25	23,2	cyclictest	2823752	-21	kworker/u16:3+flush-8:0	07:22:53	6
3549175	2	24	0,1	chrt	3549176	-21	kthreadcore	12:17:48	7
2842698	99	24	22,1	cyclictest	3410890	-21	ssh	11:18:10	7
2842698	99	24	19,4	cyclictest	650	-21	systemd-journal	10:07:21	7
2842693	99	24	22,2	cyclictest	2875384	-21	kworker/u16:1+flush-8:0	08:47:58	6
2842681	99	24	22,2	cyclictest	3018169	-21	kworker/u16:3+flush-8:0	09:22:45	4
2842698	99	23	17,4	cyclictest	2903671	-21	latency_hist	07:32:27	7
2842681	99	23	21,2	cyclictest	3018169	-21	kworker/u16:3+events_unbound	09:22:21	4
2842698	99	22	18,3	cyclictest	3423589	-21	pool-gnome-soft	11:24:27	7
2842698	99	22	17,4	cyclictest	650	-21	systemd-journal	10:32:27	7
2842681	99	22	20,2	cyclictest	3087836	-21	kworker/u16:0+flush-8:0	10:13:27	4
2842681	99	22	20,2	cyclictest	2875384	-21	kworker/u16:1+flush-8:0	08:02:22	4
2842698	99	21	20,1	cyclictest	0	-21	swapper/7	11:49:26	7
2842698	99	21	18,2	cyclictest	3506721	-21	ntpq	11:57:54	7
2842698	99	21	18,2	cyclictest	3506721	-21	ntpq	11:57:54	7
2842685	99	21	19,2	cyclictest	3316782	-21	kworker/u16:1+flush-8:0	10:58:21	5
2842681	99	21	19,2	cyclictest	3268899	-21	kworker/u16:5+events_unbound	11:41:18	4
2842698	99	20	19,0	cyclictest	0	-21	swapper/7	07:12:56	7
2842698	99	20	19,0	cyclictest	0	-21	swapper/7	07:12:56	7
2842685	99	20	15,4	cyclictest	3018169	-21	kworker/u16:3+flush-8:0	09:17:32	5
2842681	99	20	18,2	cyclictest	3525365	-21	kworker/u16:3+events_unbound	12:23:11	4
2842681	99	20	16,2	cyclictest	3316782	-21	kworker/u16:1+events_unbound	12:21:18	4
2842698	99	19	17,1	cyclictest	3496536	-21	ssh	11:55:14	7
2842698	99	19	12,6	cyclictest	0	-21	swapper/7	09:57:44	7
2842693	99	19	17,2	cyclictest	3220331	-21	kworker/u16:3+events_unbound	10:07:55	6
2842681	99	19	17,2	cyclictest	3268899	-21	kworker/u16:5+events_unbound	12:02:38	4
2842681	99	19	17,2	cyclictest	2841486	-21	kworker/u16:0+events_unbound	07:02:52	4
2842698	99	18	18,0	cyclictest	0	-21	swapper/7	10:40:23	7
2842698	99	18	18,0	cyclictest	0	-21	swapper/7	10:40:23	7
2842698	99	18	16,2	cyclictest	107	-21	kcompactd0	12:17:26	7
2842698	99	18	15,2	cyclictest	1267	-21	snmpd	07:45:33	7
2842698	99	18	13,4	cyclictest	2852095	-21	latency_hist	07:07:27	7
2842698	99	18	11,6	cyclictest	0	-21	swapper/7	09:12:28	7
2842693	99	18	16,2	cyclictest	3268898	-21	kworker/u16:2+events_unbound	10:52:50	6
2842693	99	18	16,2	cyclictest	2875384	-21	kworker/u16:1+flush-8:0	08:07:55	6
2842693	99	18	16,2	cyclictest	2823752	-21	kworker/u16:3+flush-8:0	07:47:46	6
2842681	99	18	16,2	cyclictest	3423928	-21	kworker/u16:6+flush-8:0	11:42:52	4
2842681	99	18	16,2	cyclictest	3316782	-21	kworker/u16:1+events_unbound	11:01:35	4
2842681	99	18	16,2	cyclictest	3316782	-21	kworker/u16:1+events_unbound	10:49:42	4
2842681	99	18	16,2	cyclictest	3304108	-21	kworker/u16:0+flush-8:0	10:42:48	4
2842681	99	18	16,2	cyclictest	3304108	-21	kworker/u16:0+events_unbound	11:17:47	4
2842681	99	18	16,2	cyclictest	3268899	-21	kworker/u16:5+events_unbound	11:56:58	4
2842681	99	18	16,2	cyclictest	3268898	-21	kworker/u16:2+events_unbound	10:39:25	4
2842681	99	18	16,2	cyclictest	3268898	-21	kworker/u16:2+events_unbound	10:39:25	4
2842681	99	18	16,2	cyclictest	3018169	-21	kworker/u16:3+events_unbound	09:02:58	4
2842681	99	18	16,2	cyclictest	2875384	-21	kworker/u16:1+flush-8:0	07:37:22	4
2842681	99	18	16,2	cyclictest	2865451	-21	kworker/u16:2+events_unbound	07:12:45	4
2842681	99	18	16,2	cyclictest	2865451	-21	kworker/u16:2+events_unbound	07:12:45	4
2842681	99	18	14,2	cyclictest	3304108	-21	kworker/u16:0+events_unbound	11:02:38	4
2842698	99	17	16,1	cyclictest	0	-21	swapper/7	09:49:40	7
2842698	99	17	15,1	cyclictest	2980679	-21	cpuspeed	08:12:39	7
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17