OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Sat Mar 21, 2026 12:43:34)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
2163337	2	93	0,1	sleep4	0	-21	swapper/4	08:58:43	4
1931920	99	58	56,2	cyclictest	2406195	-21	kworker/u16:3+events_unbound	11:13:37	5
1931632	2	58	48,7	sleep5	0	-21	swapper/5	07:03:15	5
1931595	2	58	49,6	sleep4	0	-21	swapper/4	07:02:42	4
1931927	99	56	53,2	cyclictest	2450933	-21	awk	10:58:48	7
1931927	99	55	50,3	cyclictest	2346665	-21	ssh	10:18:40	7
1931927	99	55	50,3	cyclictest	2346665	-21	ssh	10:18:40	7
1931922	99	54	52,2	cyclictest	2237724	-21	kworker/u16:0+events_unbound	09:58:52	6
1931927	99	53	52,0	cyclictest	0	-21	swapper/7	10:28:51	7
1931922	99	52	50,2	cyclictest	2146080	-21	kworker/u16:0+flush-8:0	08:53:48	6
1931922	99	52	49,2	cyclictest	2201414	-21	kworker/u16:1+events_unbound	09:53:24	6
1931927	99	51	48,2	cyclictest	2535400	-21	smart_sda	11:33:53	7
1931922	99	51	49,2	cyclictest	2237724	-21	kworker/u16:0+flush-8:0	10:23:52	6
1931922	99	51	48,2	cyclictest	2598840	-21	kworker/u16:1+events_unbound	12:13:39	6
1931920	99	50	48,2	cyclictest	2274314	-21	kworker/u16:3+events_unbound	09:48:52	5
1931922	99	49	47,2	cyclictest	2406195	-21	kworker/u16:3+flush-8:0	11:58:51	6
1931922	99	49	47,2	cyclictest	2367940	-21	kworker/u16:6+events_unbound	10:33:53	6
1931920	99	49	47,2	cyclictest	2117044	-21	kworker/u16:1+events_unbound	08:53:22	5
1931927	99	48	46,1	cyclictest	2164878	-21	diskmemload	11:48:36	7
1931922	99	48	45,2	cyclictest	2155677	-21	kworker/u16:4+events_unbound	09:23:24	6
1931920	99	48	47,1	cyclictest	2201414	-21	kworker/u16:1+events_unbound	10:13:57	5
1931920	99	48	46,2	cyclictest	2644218	-21	kworker/u16:1+events_unbound	12:30:20	5
1931920	99	48	46,2	cyclictest	2367940	-21	kworker/u16:6+events_unbound	11:58:36	5
1931920	99	48	45,2	cyclictest	2237724	-21	kworker/u16:0+events_unbound	09:58:37	5
1931343	2	48	38,7	sleep6	0	-21	swapper/6	06:59:07	6
1931920	99	47	45,2	cyclictest	2406195	-21	kworker/u16:3+events_unbound	10:48:52	5
1931922	99	46	44,2	cyclictest	2237724	-21	kworker/u16:0+events_unbound	10:03:53	6
1931922	99	45	43,2	cyclictest	2367940	-21	kworker/u16:6+events_unbound	11:43:55	6
1931922	99	45	43,2	cyclictest	2237724	-21	kworker/u16:0+events_unbound	10:30:20	6
1931922	99	44	42,2	cyclictest	2644218	-21	kworker/u16:1+events_unbound	12:30:43	6
1931922	99	44	42,2	cyclictest	2406195	-21	kworker/u16:3+events_unbound	11:23:35	6
1931922	99	44	42,2	cyclictest	2406195	-21	kworker/u16:3+events_unbound	10:49:00	6
1931922	99	44	42,2	cyclictest	2367940	-21	kworker/u16:6+events_unbound	10:46:20	6
1931922	99	44	42,2	cyclictest	2237724	-21	kworker/u16:0+events_unbound	10:13:20	6
1931922	99	44	42,2	cyclictest	2146080	-21	kworker/u16:0+events_unbound	09:03:24	6
1931920	99	44	42,2	cyclictest	2470711	-21	kworker/u16:0+flush-8:0	11:48:39	5
1931920	99	44	42,2	cyclictest	2367940	-21	kworker/u16:6+events_unbound	11:23:37	5
1931927	99	43	42,0	cyclictest	0	-21	swapper/7	09:54:24	7
1931927	99	43	41,1	cyclictest	2164878	-21	diskmemload	09:53:16	7
1931922	99	43	41,2	cyclictest	2367940	-21	kworker/u16:6+events_unbound	11:15:40	6
1931922	99	43	40,2	cyclictest	2470711	-21	kworker/u16:0+events_unbound	11:48:24	6
1931927	99	42	41,0	cyclictest	0	-21	swapper/7	10:04:40	7
1931927	99	42	39,2	cyclictest	2620780	-21	ssh	12:10:12	7
1931922	99	42	40,2	cyclictest	2584297	-21	kworker/u16:0+events_unbound	12:12:40	6
1931922	99	42	40,2	cyclictest	2406195	-21	kworker/u16:3+events_unbound	11:21:40	6
1931922	99	42	40,2	cyclictest	2406195	-21	kworker/u16:3+events_unbound	10:56:08	6
1931922	99	42	40,2	cyclictest	2367940	-21	kworker/u16:6+events_unbound	11:05:40	6
1931920	99	42	39,2	cyclictest	2367940	-21	kworker/u16:6+flush-8:0	10:58:43	5
1931922	99	41	39,2	cyclictest	2644218	-21	kworker/u16:1+events_unbound	12:26:52	6
1931922	99	41	39,2	cyclictest	2584297	-21	kworker/u16:0+events_unbound	12:03:57	6
1931922	99	41	39,2	cyclictest	2274314	-21	kworker/u16:3+events_unbound	09:52:37	6
1931922	99	41	39,2	cyclictest	2201414	-21	kworker/u16:1+events_unbound	10:16:41	6
1931922	99	41	39,2	cyclictest	2201414	-21	kworker/u16:1+events_unbound	09:38:05	6
1931922	99	41	39,2	cyclictest	2155677	-21	kworker/u16:4+events_unbound	09:20:20	6
1931922	99	41	39,2	cyclictest	2155677	-21	kworker/u16:4+events_unbound	09:17:40	6
1931922	99	41	38,2	cyclictest	2632177	-21	kworker/u16:3+events_unbound	12:20:39	6
1931922	99	41	38,2	cyclictest	2632177	-21	kworker/u16:3+events_unbound	12:20:39	6
1931920	99	41	39,2	cyclictest	2632177	-21	kworker/u16:3+events_unbound	12:14:20	5
1931920	99	41	39,2	cyclictest	2367940	-21	kworker/u16:6+events_unbound	11:56:21	5
1923073	2	41	3,31	sleep7	0	-21	swapper/7	06:58:36	7
1931927	99	40	40,0	cyclictest	0	-21	swapper/7	09:47:48	7
1931927	99	40	38,1	cyclictest	2164878	-21	diskmemload	11:25:56	7
1931922	99	40	38,2	cyclictest	2406195	-21	kworker/u16:3+events_unbound	11:40:24	6
1931922	99	40	38,2	cyclictest	2406195	-21	kworker/u16:3+events_unbound	11:40:24	6
1931922	99	40	37,2	cyclictest	1932144	-21	kworker/u16:2+events_unbound	09:08:25	6
1931920	99	40	38,2	cyclictest	2237724	-21	kworker/u16:0+events_unbound	10:30:12	5
1931920	99	40	38,2	cyclictest	2201414	-21	kworker/u16:1+events_unbound	09:48:09	5
1931920	99	40	38,2	cyclictest	1932144	-21	kworker/u16:2+events_unbound	09:15:25	5
1931920	99	40	36,3	cyclictest	2237724	-21	kworker/u16:0+flush-8:0	10:38:22	5
1931927	99	39	38,0	cyclictest	0	-21	swapper/7	11:21:56	7
1931927	99	39	37,1	cyclictest	2657434	-21	ssh	12:26:23	7
1931927	99	39	37,1	cyclictest	2465025	-21	ssh	11:06:29	7
1931927	99	39	37,1	cyclictest	2440326	-21	ssh	10:55:13	7
1931922	99	39	37,2	cyclictest	2470711	-21	kworker/u16:0+events_unbound	11:35:27	6
1931922	99	39	37,2	cyclictest	2237724	-21	kworker/u16:0+events_unbound	10:40:45	6
1931922	99	39	37,2	cyclictest	2201414	-21	kworker/u16:1+events_unbound	10:22:04	6
1931922	99	39	37,2	cyclictest	2201414	-21	kworker/u16:1+events_unbound	10:22:04	6
1931920	99	39	37,2	cyclictest	2237724	-21	kworker/u16:0+events_unbound	10:27:20	5
1931920	99	39	37,2	cyclictest	2201414	-21	kworker/u16:1+events_unbound	10:08:01	5
1931920	99	39	37,2	cyclictest	1963989	-21	kworker/u16:1+events_unbound	07:58:23	5
1931927	99	38	37,0	cyclictest	0	-21	swapper/7	09:04:40	7
1931927	99	38	36,1	cyclictest	0	-21	swapper/7	11:16:00	7
1931927	99	38	35,2	cyclictest	2555235	-21	ssh	11:43:43	7
1931922	99	38	35,2	cyclictest	2406195	-21	kworker/u16:3+events_unbound	11:28:23	6
1931922	99	38	35,2	cyclictest	2406195	-21	kworker/u16:3+events_unbound	11:28:23	6
1931920	99	38	36,2	cyclictest	2598840	-21	kworker/u16:1+events_unbound	12:11:32	5
1931920	99	38	36,2	cyclictest	2584297	-21	kworker/u16:0+events_unbound	12:03:39	5
1931920	99	38	36,2	cyclictest	2367940	-21	kworker/u16:6+events_unbound	11:20:12	5
1931920	99	38	36,2	cyclictest	2367940	-21	kworker/u16:6+events_unbound	10:39:08	5
1931920	99	38	36,2	cyclictest	1932144	-21	kworker/u16:2+events_unbound	09:22:45	5
1931927	99	37	37,0	cyclictest	0	-21	swapper/7	10:51:44	7
1931927	99	37	36,0	cyclictest	0	-21	swapper/7	10:25:00	7
1931927	99	37	36,0	cyclictest	0	-21	swapper/7	10:17:44	7
1931927	99	37	35,1	cyclictest	2524066	-21	ssh	11:29:20	7
1931927	99	37	35,1	cyclictest	2524066	-21	ssh	11:29:20	7
1931927	99	37	35,1	cyclictest	0	-21	swapper/7	12:23:12	7
1931927	99	37	35,1	cyclictest	0	-21	swapper/7	12:23:12	7
1931922	99	37	35,2	cyclictest	2237724	-21	kworker/u16:0+events_unbound	09:40:20	6
1931922	99	37	35,2	cyclictest	2237724	-21	kworker/u16:0+events_unbound	09:40:20	6
1931920	99	37	35,2	cyclictest	2406195	-21	kworker/u16:3+events_unbound	10:57:12	5
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17