OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Mon Dec 08, 2025 00:43:34)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
3792846	99	58	53,4	cyclictest	76423	-21	latency_hist	07:17:29	7
3792846	99	50	46,2	cyclictest	3846177	-21	ntpq	03:43:00	7
3792538	2	49	39,7	sleep5	0	-21	swapper/5	03:22:23	5
3792841	99	47	45,2	cyclictest	237415	-21	kworker/u16:0+events_unbound	08:41:31	6
3792262	2	47	37,7	sleep4	0	-21	swapper/4	03:18:37	4
3792846	99	44	43,0	cyclictest	0	-21	swapper/7	04:37:59	7
3792510	2	43	33,7	sleep6	0	-21	swapper/6	03:22:00	6
3792841	99	41	39,2	cyclictest	3816923	-21	kworker/u16:3+events_unbound	03:43:04	6
3792840	99	40	36,3	cyclictest	4098547	-21	kworker/u16:2+flush-8:0	06:57:45	5
3792531	2	39	5,30	sleep7	0	-21	swapper/7	03:22:17	7
3792846	99	38	37,0	cyclictest	0	-21	swapper/7	06:44:06	7
3792846	99	38	0,36	cyclictest	36149	-21	latency_hist	06:57:29	7
3792841	99	38	35,2	cyclictest	3846972	-21	kworker/u16:4+flush-8:0	05:22:55	6
3792846	99	37	30,5	cyclictest	589235	-21	systemd-journal	05:52:29	7
3792846	99	37	0,35	cyclictest	589235	-21	systemd-journal	08:51:38	7
3792841	99	37	35,2	cyclictest	4168955	-21	kworker/u16:3+events_unbound	06:36:40	6
3792846	99	36	35,0	cyclictest	0	-21	swapper/7	06:24:55	7
3792840	99	36	35,1	cyclictest	3370647	-21	kworker/u16:8+events_unbound	03:32:46	5
3792846	99	35	34,0	cyclictest	0	-21	swapper/7	07:38:39	7
3792846	99	35	34,0	cyclictest	0	-21	swapper/7	06:27:39	7
3792846	99	34	33,0	cyclictest	0	-21	swapper/7	04:31:49	7
3792841	99	34	32,2	cyclictest	4068465	-21	kworker/u16:1+flush-8:0	05:42:31	6
3792841	99	34	32,2	cyclictest	3846972	-21	kworker/u16:4+events_unbound	05:18:59	6
3792846	99	33	32,0	cyclictest	0	-21	swapper/7	08:42:57	7
3792846	99	33	32,0	cyclictest	0	-21	swapper/7	04:32:40	7
3792841	99	33	31,2	cyclictest	3846911	-21	kworker/u16:2+flush-8:0	03:47:57	6
3792841	99	33	31,2	cyclictest	227204	-21	kworker/u16:1+events_unbound	08:36:26	6
3792841	99	33	31,2	cyclictest	207157	-21	kworker/u16:3+flush-8:0	08:31:55	6
3792846	99	32	31,0	cyclictest	0	-21	swapper/7	07:05:12	7
3792846	99	32	31,0	cyclictest	0	-21	swapper/7	03:24:46	7
3792841	99	32	30,2	cyclictest	4138846	-21	kworker/u16:1+events_unbound	06:10:48	6
3792846	99	31	30,0	cyclictest	0	-21	swapper/7	06:47:31	7
3792841	99	31	29,2	cyclictest	86339	-21	kworker/u16:0+events_unbound	07:41:39	6
3792841	99	31	29,2	cyclictest	5313	-21	kworker/u16:1+events_unbound	07:58:02	6
3792841	99	31	29,2	cyclictest	4168955	-21	kworker/u16:3+events_unbound	07:26:05	6
3792841	99	31	29,2	cyclictest	4098547	-21	kworker/u16:2+events_unbound	06:56:44	6
3792841	99	31	29,2	cyclictest	4027913	-21	kworker/u16:0+events_unbound	06:06:44	6
3792841	99	31	29,2	cyclictest	3877082	-21	kworker/u16:1+flush-8:0	04:15:31	6
3792841	99	31	29,2	cyclictest	3816827	-21	kworker/u16:1+events_unbound	03:32:19	6
3792841	99	31	29,2	cyclictest	186906	-21	kworker/u16:2+flush-8:0	08:19:20	6
3792840	99	31	29,2	cyclictest	4098547	-21	kworker/u16:2+flush-8:0	05:57:55	5
3792840	99	31	29,2	cyclictest	3947682	-21	kworker/u16:2+flush-8:0	04:37:45	5
3792846	99	30	29,0	cyclictest	0	-21	swapper/7	08:32:30	7
3792841	99	30	28,2	cyclictest	86339	-21	kworker/u16:0+flush-8:0	07:34:43	6
3792841	99	30	28,2	cyclictest	5313	-21	kworker/u16:1+flush-8:0	06:48:59	6
3792841	99	30	28,2	cyclictest	4168955	-21	kworker/u16:3+events_unbound	06:32:07	6
3792841	99	30	28,2	cyclictest	4098547	-21	kworker/u16:2+events_unbound	07:11:55	6
3792841	99	30	28,2	cyclictest	4098547	-21	kworker/u16:2+events_unbound	05:49:44	6
3792841	99	30	28,2	cyclictest	4038693	-21	kworker/u16:6+events_unbound	05:42:07	6
3792841	99	30	28,2	cyclictest	4027913	-21	kworker/u16:0+events_unbound	06:19:11	6
3792841	99	30	28,2	cyclictest	3877082	-21	kworker/u16:1+events_unbound	04:48:08	6
3792841	99	30	28,2	cyclictest	3877082	-21	kworker/u16:1+events_unbound	04:02:23	6
3792841	99	30	28,2	cyclictest	246439	-21	kworker/u16:5+events_unbound	08:50:11	6
3792841	99	30	28,2	cyclictest	186906	-21	kworker/u16:2+flush-8:0	08:22:57	6
3792841	99	30	28,2	cyclictest	156713	-21	kworker/u16:2+events_unbound	08:02:39	6
3792840	99	30	28,2	cyclictest	3896478	-21	kworker/u16:2+flush-8:0	04:27:59	5
3792846	99	29	28,0	cyclictest	0	-21	swapper/7	06:12:45	7
3792846	99	29	28,0	cyclictest	0	-21	swapper/7	05:57:39	7
3792846	99	29	28,0	cyclictest	0	-21	swapper/7	04:53:55	7
3792841	99	29	27,2	cyclictest	5313	-21	kworker/u16:1+events_unbound	07:54:23	6
3792841	99	29	27,2	cyclictest	4098547	-21	kworker/u16:2+events_unbound	06:02:11	6
3792841	99	29	27,2	cyclictest	4038693	-21	kworker/u16:6+flush-8:0	05:32:11	6
3792841	99	29	27,2	cyclictest	3958196	-21	kworker/u16:5+events_unbound	04:43:52	6
3792841	99	29	27,2	cyclictest	3877082	-21	kworker/u16:1+events_unbound	04:09:23	6
3792841	99	29	27,2	cyclictest	3846972	-21	kworker/u16:4+events_unbound	04:59:23	6
3792841	99	29	27,2	cyclictest	116339	-21	kworker/u16:3+events_unbound	08:12:04	6
3792840	99	29	27,2	cyclictest	4038693	-21	kworker/u16:6+events_unbound	06:12:03	5
3792840	99	29	27,2	cyclictest	4038693	-21	kworker/u16:6+events_unbound	05:52:52	5
3792840	99	29	27,2	cyclictest	4027913	-21	kworker/u16:0+flush-8:0	05:32:55	5
3792841	99	28	26,2	cyclictest	4168955	-21	kworker/u16:3+flush-8:0	07:05:07	6
3792841	99	28	26,2	cyclictest	3846972	-21	kworker/u16:4+events_unbound	04:37:31	6
3792841	99	28	26,2	cyclictest	3816827	-21	kworker/u16:1+events_unbound	03:41:48	6
3792840	99	28	26,2	cyclictest	186906	-21	kworker/u16:2+flush-8:0	08:23:01	5
3792846	99	27	0,25	cyclictest	136850	-21	latency_hist	07:47:29	7
3792841	99	27	25,2	cyclictest	4168955	-21	kworker/u16:3+events_unbound	07:19:20	6
3792841	99	27	25,2	cyclictest	4098547	-21	kworker/u16:2+events_unbound	06:46:30	6
3792841	99	27	25,2	cyclictest	4038693	-21	kworker/u16:6+flush-8:0	05:56:15	6
3792841	99	27	25,2	cyclictest	3896478	-21	kworker/u16:2+flush-8:0	04:23:55	6
3792841	99	27	25,2	cyclictest	3877082	-21	kworker/u16:1+events_unbound	04:55:44	6
3792841	99	27	25,2	cyclictest	3846972	-21	kworker/u16:4+flush-8:0	05:14:45	6
3792841	99	27	25,2	cyclictest	246439	-21	kworker/u16:5+flush-8:0	08:43:40	6
3792841	99	27	24,2	cyclictest	3997893	-21	kworker/u16:2+flush-8:0	05:02:34	6
3792840	99	27	25,2	cyclictest	4138846	-21	kworker/u16:1+flush-8:0	06:12:59	5
3792840	99	27	25,2	cyclictest	4098547	-21	kworker/u16:2+events_unbound	06:37:27	5
3792840	99	27	25,2	cyclictest	4027913	-21	kworker/u16:0+events_unbound	06:24:08	5
3792846	99	26	25,0	cyclictest	0	-21	swapper/7	05:19:01	7
3792846	99	26	25,0	cyclictest	0	-21	swapper/7	04:19:39	7
3792846	99	26	25,0	cyclictest	0	-21	swapper/7	04:11:48	7
3792846	99	26	21,4	cyclictest	46182	-21	latency_hist	07:02:29	7
3792841	99	26	24,2	cyclictest	86339	-21	kworker/u16:0+events_unbound	07:28:59	6
3792841	99	26	24,2	cyclictest	5313	-21	kworker/u16:1+events_unbound	07:01:55	6
3792841	99	26	24,2	cyclictest	4168955	-21	kworker/u16:3+flush-8:0	07:17:19	6
3792841	99	26	24,2	cyclictest	4098547	-21	kworker/u16:2+events_unbound	06:39:03	6
3792841	99	26	24,2	cyclictest	4027913	-21	kworker/u16:0+events_unbound	06:26:30	6
3792841	99	26	24,2	cyclictest	186906	-21	kworker/u16:2+flush-8:0	08:13:58	6
3792840	99	26	24,2	cyclictest	4038693	-21	kworker/u16:6+flush-8:0	05:30:19	5
3792840	99	26	24,2	cyclictest	4027913	-21	kworker/u16:0+flush-8:0	06:17:45	5
3792840	99	26	24,2	cyclictest	156713	-21	kworker/u16:2+events_unbound	07:56:32	5
3792841	99	25	23,2	cyclictest	3846972	-21	kworker/u16:4+events_unbound	04:31:28	6
3792841	99	25	23,2	cyclictest	3836853	-21	kworker/u16:0+events_unbound	03:57:00	6
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17