OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Tue Apr 07, 2026 00:43:34)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
4037242	2	57	47,7	sleep4	0	-21	swapper/4	19:01:35	4
4028165	2	57	46,7	sleep5	0	-21	swapper/5	18:57:19	5
4037579	99	56	50,5	cyclictest	521582	-21	latency_hist	00:07:17	7
4037244	2	51	41,7	sleep6	0	-21	swapper/6	19:01:37	6
4037579	99	45	40,4	cyclictest	125072	-21	latency_hist	21:22:17	7
4028176	2	39	3,31	sleep7	0	-21	swapper/7	18:57:29	7
4037579	99	38	33,4	cyclictest	389412	-21	latency_hist	23:12:17	7
4037570	99	36	33,2	cyclictest	197744	-21	kworker/u16:3+events_unbound	22:42:30	5
4037570	99	35	33,2	cyclictest	437749	-21	kworker/u16:3+flush-8:0	23:47:16	5
4037579	99	34	29,4	cyclictest	461410	-21	latency_hist	23:42:16	7
4037579	99	34	29,4	cyclictest	461410	-21	latency_hist	23:42:16	7
4037579	99	33	32,0	cyclictest	0	-21	swapper/7	23:32:30	7
4037574	99	33	31,2	cyclictest	77244	-21	kworker/u16:2+events_unbound	22:06:01	6
4037566	99	33	31,2	cyclictest	340483	-21	kworker/u16:0+events_unbound	23:37:34	4
4037566	99	33	31,2	cyclictest	340483	-21	kworker/u16:0+events_unbound	23:37:34	4
4037579	99	32	29,2	cyclictest	124	-21	kswapd0	00:12:30	7
4037574	99	32	30,2	cyclictest	461600	-21	kworker/u16:4+flush-8:0	23:51:24	6
4037574	99	32	30,2	cyclictest	340483	-21	kworker/u16:0+events_unbound	23:10:52	6
4037574	99	32	30,2	cyclictest	197744	-21	kworker/u16:3+flush-8:0	22:44:34	6
4037570	99	32	30,2	cyclictest	4037092	-21	kworker/u16:3+events_unbound	19:07:22	5
4037570	99	32	29,2	cyclictest	77244	-21	kworker/u16:2+events_unbound	22:12:30	5
4037566	99	32	29,2	cyclictest	461600	-21	kworker/u16:4+flush-8:0	00:02:22	4
461865	2	30	0,1	chrt	461866	-21	sh	23:42:24	7
4037579	99	30	25,4	cyclictest	101104	-21	latency_hist	21:12:17	7
4037574	99	30	28,2	cyclictest	3989139	-21	kworker/u16:1+events_unbound	20:51:39	6
4037574	99	29	27,2	cyclictest	3989139	-21	kworker/u16:1+events_unbound	20:41:43	6
4037574	99	29	27,2	cyclictest	3989139	-21	kworker/u16:1+events_unbound	20:28:34	6
4037574	99	29	27,2	cyclictest	3989139	-21	kworker/u16:1+events_unbound	19:30:02	6
4037574	99	29	27,2	cyclictest	124544	-21	kworker/u16:1+events_unbound	21:41:41	6
4037574	99	29	27,2	cyclictest	124544	-21	kworker/u16:1+events_unbound	21:41:41	6
4037570	99	29	27,2	cyclictest	461600	-21	kworker/u16:4+events_unbound	00:02:47	5
4037566	99	29	25,3	cyclictest	437749	-21	kworker/u16:3+flush-8:0	23:32:36	4
4037579	99	28	27,0	cyclictest	0	-21	swapper/7	23:57:32	7
4037574	99	28	26,2	cyclictest	77244	-21	kworker/u16:2+events_unbound	21:04:21	6
4037574	99	28	26,2	cyclictest	461600	-21	kworker/u16:4+events_unbound	00:07:01	6
4037574	99	28	26,2	cyclictest	4022578	-21	kworker/u16:2+events_unbound	20:35:38	6
4037574	99	28	26,2	cyclictest	3989139	-21	kworker/u16:1+events_unbound	20:45:38	6
4037574	99	28	26,2	cyclictest	329828	-21	kworker/u16:4+events_unbound	22:51:45	6
4037574	99	28	26,2	cyclictest	317781	-21	kworker/u16:1+flush-8:0	23:28:10	6
4037570	99	28	26,2	cyclictest	317781	-21	kworker/u16:1+flush-8:0	22:47:33	5
4037574	99	27	25,2	cyclictest	437749	-21	kworker/u16:3+events_unbound	00:22:08	6
4037574	99	27	25,2	cyclictest	437749	-21	kworker/u16:3+events_unbound	00:22:08	6
4037574	99	27	25,2	cyclictest	437748	-21	kworker/u16:2+events_unbound	23:42:57	6
4037574	99	27	25,2	cyclictest	4068929	-21	kworker/u16:0+events_unbound	19:14:30	6
4037574	99	27	25,2	cyclictest	4022578	-21	kworker/u16:2+events_unbound	20:25:42	6
4037574	99	27	25,2	cyclictest	4022578	-21	kworker/u16:2+events_unbound	20:10:10	6
4037574	99	27	25,2	cyclictest	4022578	-21	kworker/u16:2+events_unbound	20:01:47	6
4037574	99	27	25,2	cyclictest	3989139	-21	kworker/u16:1+events_unbound	20:19:46	6
4037574	99	27	24,2	cyclictest	124544	-21	kworker/u16:1+events_unbound	21:32:24	6
4037570	99	27	25,2	cyclictest	340483	-21	kworker/u16:0+events_unbound	23:17:40	5
4037566	99	27	26,1	cyclictest	317781	-21	kworker/u16:1+events_unbound	23:02:34	4
4037566	99	27	25,2	cyclictest	77244	-21	kworker/u16:2+flush-8:0	22:07:53	4
4037566	99	27	25,2	cyclictest	340483	-21	kworker/u16:0+events_unbound	23:07:40	4
4037566	99	27	23,2	cyclictest	437749	-21	kworker/u16:3+flush-8:0	23:52:17	4
4037579	99	26	21,4	cyclictest	19735	-21	latency_hist	20:32:17	7
4037574	99	26	24,2	cyclictest	475862	-21	kworker/u16:1+events_unbound	00:08:04	6
4037574	99	26	24,2	cyclictest	4037092	-21	kworker/u16:3+events_unbound	19:08:38	6
4037574	99	26	24,2	cyclictest	4037092	-21	kworker/u16:3+events_unbound	19:04:08	6
4037574	99	26	24,2	cyclictest	4022578	-21	kworker/u16:2+events_unbound	19:19:07	6
4037574	99	26	24,2	cyclictest	317781	-21	kworker/u16:1+flush-8:0	23:02:11	6
4037574	99	26	24,2	cyclictest	317781	-21	kworker/u16:1+flush-8:0	23:02:11	6
4037574	99	26	24,2	cyclictest	19917	-21	kworker/u16:4+events_unbound	21:01:43	6
4037574	99	26	24,2	cyclictest	19917	-21	kworker/u16:4+events_unbound	21:01:43	6
4037570	99	26	24,2	cyclictest	67446	-21	kworker/u16:0+flush-8:0	21:32:42	5
4037566	99	26	24,2	cyclictest	475862	-21	kworker/u16:1+flush-8:0	00:27:11	4
4037566	99	26	23,2	cyclictest	19917	-21	kworker/u16:4+events_unbound	20:47:35	4
4037579	99	25	22,2	cyclictest	218203	-21	ssh	21:57:41	7
4037579	99	25	0,1	cyclictest	0	-21	swapper/7	20:22:40	7
4037574	99	25	23,2	cyclictest	4098575	-21	kworker/u16:3+events_unbound	19:55:42	6
4037574	99	25	23,2	cyclictest	4098575	-21	kworker/u16:3+events_unbound	19:49:44	6
4037574	99	25	23,2	cyclictest	4098575	-21	kworker/u16:3+events_unbound	19:46:15	6
4037574	99	25	23,2	cyclictest	4068929	-21	kworker/u16:0+events_unbound	19:25:07	6
4037574	99	25	23,2	cyclictest	3989139	-21	kworker/u16:1+events_unbound	20:13:56	6
4037574	99	25	23,2	cyclictest	124544	-21	kworker/u16:1+events_unbound	21:53:09	6
4037570	99	25	23,2	cyclictest	4022578	-21	kworker/u16:2+flush-8:0	19:42:43	5
4037566	99	25	23,2	cyclictest	124544	-21	kworker/u16:1+flush-8:0	21:27:43	4
4037566	99	25	22,3	cyclictest	77244	-21	kworker/u16:2+flush-8:0	22:37:12	4
4037566	99	25	22,2	cyclictest	340483	-21	kworker/u16:0+events_unbound	22:57:29	4
4037566	99	25	22,2	cyclictest	340483	-21	kworker/u16:0+events_unbound	22:57:29	4
4037574	99	24	22,2	cyclictest	19917	-21	kworker/u16:4+events_unbound	20:54:59	6
4037566	99	24	22,2	cyclictest	461600	-21	kworker/u16:4+events_unbound	00:12:39	4
4037566	99	24	22,2	cyclictest	329828	-21	kworker/u16:4+events_unbound	23:12:30	4
4037566	99	24	22,2	cyclictest	124544	-21	kworker/u16:1+events_unbound	21:37:37	4
4037566	99	24	22,2	cyclictest	124544	-21	kworker/u16:1+events_unbound	21:37:37	4
4037566	99	24	21,2	cyclictest	4022578	-21	kworker/u16:2+events_unbound	20:22:32	4
4037566	99	24	21,2	cyclictest	340483	-21	kworker/u16:0+flush-8:0	23:17:21	4
4037579	99	23	22,0	cyclictest	0	-21	swapper/7	00:22:41	7
4037574	99	23	21,2	cyclictest	4022578	-21	kworker/u16:2+events_unbound	20:03:11	6
4037574	99	23	21,2	cyclictest	3989139	-21	kworker/u16:1+events_unbound	19:33:27	6
4037574	99	23	21,2	cyclictest	329828	-21	kworker/u16:4+flush-8:0	23:12:48	6
4037570	99	23	21,2	cyclictest	29695	-21	kworker/u16:5+flush-8:0	21:07:42	5
4037570	99	23	20,2	cyclictest	340483	-21	kworker/u16:0+events_unbound	23:27:29	5
4037566	99	23	21,2	cyclictest	567332	-21	kworker/u16:0+events_unbound	00:27:44	4
4037566	99	23	21,2	cyclictest	4022578	-21	kworker/u16:2+events_unbound	20:02:42	4
4037570	99	22	20,2	cyclictest	3989139	-21	kworker/u16:1+events_unbound	19:57:41	5
4037570	99	22	20,2	cyclictest	173522	-21	kworker/u16:5+events_unbound	22:37:29	5
4037574	99	21	19,2	cyclictest	67446	-21	kworker/u16:0+events_unbound	21:08:10	6
4037574	99	21	19,2	cyclictest	437748	-21	kworker/u16:2+events_unbound	23:36:57	6
4037574	99	21	19,2	cyclictest	4098575	-21	kworker/u16:3+events_unbound	19:39:03	6
4037570	99	21	19,2	cyclictest	124544	-21	kworker/u16:1+events_unbound	21:22:38	5
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17