OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Sat Nov 29, 2025 12:43:32)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
244551	2	58	48,7	sleep4	0	-21	swapper/4	15:23:00	4
244551	2	58	48,7	sleep4	0	-21	swapper/4	15:23:00	4
244499	2	58	48,7	sleep6	0	-21	swapper/6	15:22:22	6
244499	2	58	48,7	sleep6	0	-21	swapper/6	15:22:22	6
244861	99	54	0,52	cyclictest	27944	-21	gnome-software	19:29:34	7
244326	2	53	20,29	sleep7	0	-21	swapper/7	15:19:59	7
244326	2	53	20,29	sleep7	0	-21	swapper/7	15:19:59	7
244861	99	50	44,5	cyclictest	268863	-21	latency_hist	15:33:04	7
244854	99	49	46,2	cyclictest	883827	-21	kworker/u16:2+events_unbound	20:42:25	4
244856	99	48	44,3	cyclictest	280152	-21	iostat_ios	15:38:20	5
244861	99	47	42,4	cyclictest	773128	-21	latency_hist	19:43:03	7
244440	2	47	37,7	sleep5	0	-21	swapper/5	15:21:33	5
244440	2	47	37,7	sleep5	0	-21	swapper/5	15:21:33	5
244854	99	45	43,2	cyclictest	470293	-21	kworker/u16:4+events_unbound	19:21:25	4
244861	99	43	38,4	cyclictest	278943	-21	latency_hist	15:38:03	7
244854	99	43	41,2	cyclictest	288871	-21	kworker/u16:4+events_unbound	16:32:26	4
244861	99	42	0,40	cyclictest	540922	-21	latency_hist	17:48:04	7
244860	99	42	40,2	cyclictest	470293	-21	kworker/u16:4+events_unbound	19:53:22	6
244854	99	42	37,4	cyclictest	752517	-21	pool-/usr/libex	19:29:26	4
244854	99	41	39,2	cyclictest	288871	-21	kworker/u16:4+events_unbound	17:03:32	4
244861	99	40	34,5	cyclictest	732637	-21	latency_hist	19:23:02	7
244854	99	40	38,2	cyclictest	853576	-21	kworker/u16:4+events_unbound	20:32:25	4
244861	99	39	34,4	cyclictest	389783	-21	latency_hist	16:33:03	7
244860	99	38	36,2	cyclictest	601890	-21	kworker/u16:3+flush-8:0	19:58:21	6
244860	99	38	36,2	cyclictest	470293	-21	kworker/u16:4+flush-8:0	19:32:57	6
244854	99	38	36,2	cyclictest	540670	-21	kworker/u16:1+flush-8:0	17:58:29	4
244854	99	38	36,2	cyclictest	470293	-21	kworker/u16:4+events_unbound	18:48:37	4
244854	99	38	36,2	cyclictest	470293	-21	kworker/u16:4+events_unbound	18:48:37	4
244854	99	38	36,2	cyclictest	288871	-21	kworker/u16:4+events_unbound	16:57:21	4
244861	99	37	31,5	cyclictest	833682	-21	latency_hist	20:13:02	7
244854	99	37	35,2	cyclictest	793482	-21	kworker/u16:1+flush-8:0	20:12:57	4
244854	99	37	35,2	cyclictest	470293	-21	kworker/u16:4+flush-8:0	17:20:29	4
244861	99	36	31,4	cyclictest	702304	-21	latency_hist	19:08:03	7
244860	99	36	34,2	cyclictest	470293	-21	kworker/u16:4+flush-8:0	19:17:58	6
244856	99	36	32,3	cyclictest	3732922	-21	kworker/u16:3+events_unbound	15:58:18	5
244854	99	36	34,2	cyclictest	591286	-21	kworker/u16:1+events_unbound	18:16:14	4
244854	99	36	33,2	cyclictest	601890	-21	kworker/u16:3+flush-8:0	20:18:13	4
244854	99	36	33,2	cyclictest	601890	-21	kworker/u16:3+flush-8:0	20:18:13	4
244861	99	35	29,5	cyclictest	470387	-21	latency_hist	17:13:03	7
244856	99	35	32,2	cyclictest	853576	-21	kworker/u16:4+flush-8:0	20:43:03	5
244854	99	35	33,2	cyclictest	410144	-21	kworker/u16:0+events_unbound	16:43:30	4
244861	99	34	33,0	cyclictest	0	-21	swapper/7	19:58:21	7
244860	99	34	32,2	cyclictest	591286	-21	kworker/u16:1+flush-8:0	18:18:19	6
244854	99	34	32,2	cyclictest	359962	-21	kworker/u16:3+flush-8:0	16:21:06	4
244854	99	33	31,2	cyclictest	591286	-21	kworker/u16:1+events_unbound	18:39:24	4
244854	99	33	31,2	cyclictest	349307	-21	kworker/u16:2+flush-8:0	16:12:54	4
244854	99	33	31,2	cyclictest	288871	-21	kworker/u16:4+flush-8:0	15:42:21	4
244861	99	32	31,0	cyclictest	0	-21	swapper/7	15:57:51	7
244856	99	32	30,2	cyclictest	652320	-21	kworker/u16:0+flush-8:0	19:29:40	5
244854	99	32	30,2	cyclictest	3732922	-21	kworker/u16:3+events_unbound	15:55:14	4
244854	99	32	30,2	cyclictest	288871	-21	kworker/u16:4+events_unbound	16:07:31	4
244854	99	32	29,2	cyclictest	359962	-21	kworker/u16:3+flush-8:0	17:08:47	4
244861	99	31	0,29	cyclictest	631786	-21	systemd-run	18:33:03	7
244860	99	31	29,2	cyclictest	601890	-21	kworker/u16:3+flush-8:0	20:28:30	6
244860	99	31	29,2	cyclictest	410144	-21	kworker/u16:0+events_unbound	16:53:30	6
244854	99	31	29,2	cyclictest	652320	-21	kworker/u16:0+events_unbound	19:11:33	4
244854	99	31	29,2	cyclictest	288871	-21	kworker/u16:4+events_unbound	16:40:27	4
244854	99	31	29,2	cyclictest	288871	-21	kworker/u16:4+events_unbound	15:53:03	4
244854	99	31	28,3	cyclictest	601890	-21	kworker/u16:3+flush-8:0	18:23:38	4
244854	99	30	28,2	cyclictest	752576	-21	kworker/u16:6+events_unbound	19:57:01	4
244854	99	30	28,2	cyclictest	652320	-21	kworker/u16:0+flush-8:0	19:13:38	4
244854	99	30	28,2	cyclictest	601693	-21	kworker/u16:2+events_unbound	18:33:28	4
244854	99	30	28,2	cyclictest	591286	-21	kworker/u16:1+flush-8:0	18:20:25	4
244854	99	30	28,2	cyclictest	470293	-21	kworker/u16:4+events_unbound	19:24:58	4
244854	99	29	27,2	cyclictest	843623	-21	kworker/u16:2+flush-8:0	20:27:21	4
244854	99	29	27,2	cyclictest	793482	-21	kworker/u16:1+flush-8:0	20:01:07	4
244854	99	29	27,2	cyclictest	601693	-21	kworker/u16:2+flush-8:0	18:54:29	4
244854	99	29	27,2	cyclictest	540670	-21	kworker/u16:1+events_unbound	17:44:58	4
244854	99	29	27,2	cyclictest	359962	-21	kworker/u16:3+flush-8:0	17:25:36	4
244854	99	29	27,2	cyclictest	359962	-21	kworker/u16:3+flush-8:0	16:34:14	4
244854	99	29	27,2	cyclictest	359962	-21	kworker/u16:3+events_unbound	16:51:06	4
244860	99	28	26,2	cyclictest	883827	-21	kworker/u16:2+flush-8:0	20:36:26	6
244860	99	28	26,2	cyclictest	540670	-21	kworker/u16:1+events_unbound	17:58:27	6
244860	99	28	26,2	cyclictest	509802	-21	kworker/u16:0+flush-8:0	17:48:20	6
244854	99	28	26,2	cyclictest	470293	-21	kworker/u16:4+events_unbound	19:43:06	4
244854	99	28	26,2	cyclictest	288871	-21	kworker/u16:4+flush-8:0	16:58:33	4
244854	99	28	26,2	cyclictest	278675	-21	kworker/u16:1+events_unbound	15:59:13	4
244854	99	28	25,2	cyclictest	601890	-21	kworker/u16:3+flush-8:0	20:03:17	4
244860	99	27	25,2	cyclictest	883815	-21	kworker/u16:0+events_unbound	20:38:19	6
244860	99	27	25,2	cyclictest	783181	-21	kworker/u16:0+flush-8:0	20:07:59	6
244860	99	27	25,2	cyclictest	591286	-21	kworker/u16:1+flush-8:0	18:33:21	6
244860	99	27	25,2	cyclictest	470293	-21	kworker/u16:4+flush-8:0	19:22:58	6
244860	99	27	25,2	cyclictest	3732922	-21	kworker/u16:3+events_unbound	16:08:30	6
244860	99	27	25,2	cyclictest	188864	-21	kworker/u16:2+events_unbound	15:23:27	6
244856	99	27	25,2	cyclictest	864336	-21	kworker/u16:0+events_unbound	20:28:21	5
244856	99	27	25,2	cyclictest	652320	-21	kworker/u16:0+flush-8:0	19:18:19	5
244854	99	27	25,2	cyclictest	752576	-21	kworker/u16:6+events_unbound	19:52:06	4
244854	99	27	25,2	cyclictest	601693	-21	kworker/u16:2+events_unbound	18:32:04	4
244854	99	27	25,2	cyclictest	509802	-21	kworker/u16:0+events_unbound	18:13:01	4
244854	99	27	25,2	cyclictest	509802	-21	kworker/u16:0+events_unbound	17:42:14	4
244854	99	27	25,2	cyclictest	470293	-21	kworker/u16:4+events_unbound	19:41:34	4
244854	99	27	25,2	cyclictest	359962	-21	kworker/u16:3+events_unbound	17:15:31	4
244860	99	26	24,2	cyclictest	853576	-21	kworker/u16:4+flush-8:0	20:47:58	6
244860	99	26	24,2	cyclictest	652320	-21	kworker/u16:0+flush-8:0	19:27:57	6
244860	99	26	24,2	cyclictest	601890	-21	kworker/u16:3+events_unbound	20:08:21	6
244860	99	26	24,2	cyclictest	410144	-21	kworker/u16:0+events_unbound	16:58:27	6
244854	99	26	24,2	cyclictest	652320	-21	kworker/u16:0+flush-8:0	19:02:24	4
244854	99	26	24,2	cyclictest	470216	-21	kworker/u16:2+flush-8:0	17:32:26	4
244861	99	25	19,5	cyclictest	589235	-21	systemd-journal	17:27:22	7
244860	99	25	23,2	cyclictest	470293	-21	kworker/u16:4+flush-8:0	19:43:30	6
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17