OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Fri Jul 31, 2026 00:43:33)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
3257682	2	65	0,1	sleep5	0	-21	swapper/5	22:54:15	5
2757382	2	56	46,7	sleep5	0	-21	swapper/5	19:04:35	5
2757381	2	56	46,7	sleep4	0	-21	swapper/4	19:04:34	4
2757576	2	55	20,30	sleep7	0	-21	swapper/7	19:07:17	7
2757995	99	40	37,2	cyclictest	125	-21	kswapd0	23:09:07	7
2757991	99	38	35,2	cyclictest	3077645	-21	kworker/u16:1+events_unbound	21:53:54	6
2757991	99	38	35,2	cyclictest	2927055	-21	kworker/u16:1+events_unbound	20:48:55	6
2757981	99	37	34,2	cyclictest	3224024	-21	kworker/u16:3+flush-8:0	22:58:53	4
2757991	99	36	34,2	cyclictest	3186821	-21	kworker/u16:0+events_unbound	23:02:55	6
2757991	99	36	34,2	cyclictest	2994823	-21	kworker/u16:4+events_unbound	22:21:35	6
2757995	99	35	34,0	cyclictest	0	-21	swapper/7	22:34:39	7
2757991	99	35	33,2	cyclictest	3308551	-21	kworker/u16:2+events_unbound	23:48:06	6
2757995	99	34	33,0	cyclictest	0	-21	swapper/7	22:43:54	7
2757995	99	34	33,0	cyclictest	0	-21	swapper/7	00:02:47	7
2757991	99	34	32,2	cyclictest	3308551	-21	kworker/u16:2+events_unbound	00:06:38	6
2757991	99	34	32,2	cyclictest	3006821	-21	kworker/u16:0+events_unbound	22:14:21	6
2757991	99	34	32,2	cyclictest	3006821	-21	kworker/u16:0+events_unbound	21:42:12	6
2757981	99	34	32,2	cyclictest	3308551	-21	kworker/u16:2+events_unbound	23:41:02	4
2757981	99	34	32,2	cyclictest	3237061	-21	kworker/u16:1+flush-8:0	23:23:53	4
2757981	99	34	32,2	cyclictest	3006821	-21	kworker/u16:0+flush-8:0	22:14:19	4
2757981	99	34	32,2	cyclictest	2994823	-21	kworker/u16:4+flush-8:0	21:33:53	4
2757517	2	34	24,7	sleep6	0	-21	swapper/6	19:06:28	6
2757991	99	33	31,2	cyclictest	3479457	-21	kworker/u16:3+flush-8:0	00:30:14	6
2757991	99	33	31,2	cyclictest	3479457	-21	kworker/u16:3+events_unbound	00:35:43	6
2757991	99	33	31,2	cyclictest	3441128	-21	kworker/u16:3+events_unbound	00:11:52	6
2757991	99	33	31,2	cyclictest	3237061	-21	kworker/u16:1+events_unbound	23:21:15	6
2757991	99	33	31,2	cyclictest	2956233	-21	kworker/u16:0+flush-8:0	21:04:20	6
2757981	99	33	31,2	cyclictest	3224024	-21	kworker/u16:3+flush-8:0	23:00:03	4
2757995	99	32	30,1	cyclictest	0	-21	swapper/7	00:13:42	7
2757991	99	32	30,2	cyclictest	3380264	-21	kworker/u16:1+events_unbound	00:00:54	6
2757991	99	32	30,2	cyclictest	3308551	-21	kworker/u16:2+events_unbound	23:31:52	6
2757991	99	32	30,2	cyclictest	3186821	-21	kworker/u16:0+events_unbound	22:32:08	6
2757981	99	32	30,2	cyclictest	3006821	-21	kworker/u16:0+flush-8:0	21:24:29	4
2757995	99	31	30,0	cyclictest	0	-21	swapper/7	21:41:59	7
2757995	99	31	27,2	cyclictest	3309561	-21	sh	23:18:16	7
2757995	99	31	27,2	cyclictest	3309561	-21	sh	23:18:16	7
2757991	99	31	29,2	cyclictest	3380264	-21	kworker/u16:1+events_unbound	00:20:50	6
2757991	99	31	29,2	cyclictest	3186821	-21	kworker/u16:0+flush-8:0	22:45:27	6
2757991	99	31	29,2	cyclictest	3006821	-21	kworker/u16:0+flush-8:0	21:51:11	6
2757991	99	31	28,2	cyclictest	2858975	-21	kworker/u16:3+flush-8:0	20:58:54	6
2757987	99	31	29,2	cyclictest	2956233	-21	kworker/u16:0+events_unbound	20:54:08	5
2757981	99	31	29,2	cyclictest	3416385	-21	kworker/u16:4+flush-8:0	00:27:36	4
2757981	99	31	29,2	cyclictest	3237061	-21	kworker/u16:1+flush-8:0	22:53:03	4
2757995	99	30	29,0	cyclictest	0	-21	swapper/7	22:55:45	7
2757995	99	30	29,0	cyclictest	0	-21	swapper/7	22:07:02	7
2757991	99	30	28,2	cyclictest	3237061	-21	kworker/u16:1+events_unbound	22:58:49	6
2757991	99	30	28,2	cyclictest	3186821	-21	kworker/u16:0+flush-8:0	22:36:35	6
2757981	99	30	28,2	cyclictest	3380264	-21	kworker/u16:1+events_unbound	23:52:04	4
2757981	99	30	28,2	cyclictest	3152950	-21	kworker/u16:1+events_unbound	22:19:03	4
2757995	99	29	27,1	cyclictest	0	-21	swapper/7	21:23:42	7
2757991	99	29	27,2	cyclictest	3237061	-21	kworker/u16:1+events_unbound	23:12:47	6
2757991	99	29	27,2	cyclictest	3211561	-21	kworker/u16:2+events_unbound	22:40:31	6
2757991	99	29	27,2	cyclictest	2858975	-21	kworker/u16:3+events_unbound	21:30:35	6
2757991	99	29	27,2	cyclictest	2858975	-21	kworker/u16:3+events_unbound	20:38:16	6
2757981	99	29	27,2	cyclictest	3432672	-21	kworker/u16:0+events_unbound	00:09:22	4
2757981	99	29	27,2	cyclictest	3308551	-21	kworker/u16:2+flush-8:0	00:18:23	4
2757981	99	29	27,2	cyclictest	3186821	-21	kworker/u16:0+flush-8:0	22:44:55	4
3114316	2	28	0,1	chrt	3114314	-21	ssh	21:55:27	7
2757995	99	28	27,0	cyclictest	0	-21	swapper/7	22:28:18	7
2757995	99	28	27,0	cyclictest	0	-21	swapper/7	20:20:08	7
2757995	99	28	27,0	cyclictest	0	-21	swapper/7	00:31:07	7
2757995	99	28	26,1	cyclictest	0	-21	swapper/7	22:02:59	7
2757991	99	28	26,2	cyclictest	3380264	-21	kworker/u16:1+events_unbound	00:26:12	6
2757991	99	28	26,2	cyclictest	2994823	-21	kworker/u16:4+events_unbound	21:10:02	6
2757991	99	28	25,2	cyclictest	2819793	-21	kworker/u16:2+events_unbound	19:54:26	6
2757987	99	28	26,2	cyclictest	3224024	-21	kworker/u16:3+events_unbound	22:59:26	5
2757987	99	28	26,2	cyclictest	2829712	-21	kworker/u16:1+events_unbound	19:49:07	5
2757987	99	28	26,2	cyclictest	2829712	-21	kworker/u16:1+events_unbound	19:49:07	5
2757991	99	27	25,2	cyclictest	3308551	-21	kworker/u16:2+events_unbound	23:50:17	6
2757991	99	27	25,2	cyclictest	3006821	-21	kworker/u16:0+events_unbound	22:11:06	6
2757991	99	27	25,2	cyclictest	2858975	-21	kworker/u16:3+flush-8:0	21:36:46	6
2757991	99	27	25,2	cyclictest	2858975	-21	kworker/u16:3+events_unbound	21:59:52	6
2757991	99	27	25,2	cyclictest	2858975	-21	kworker/u16:3+events_unbound	21:15:49	6
2757991	99	27	25,2	cyclictest	2858975	-21	kworker/u16:3+events_unbound	21:15:49	6
2757991	99	27	25,2	cyclictest	2780797	-21	kworker/u16:4+flush-8:0	20:27:08	6
2757981	99	27	25,2	cyclictest	3380264	-21	kworker/u16:1+flush-8:0	23:57:44	4
2757981	99	27	25,2	cyclictest	3308551	-21	kworker/u16:2+events_unbound	00:06:24	4
2757981	99	27	25,2	cyclictest	2994823	-21	kworker/u16:4+flush-8:0	22:32:04	4
2757995	99	26	25,0	cyclictest	0	-21	swapper/7	21:47:27	7
2757995	99	26	25,0	cyclictest	0	-21	swapper/7	00:22:14	7
2757995	99	26	24,1	cyclictest	2995141	-21	diskmemload	21:50:09	7
2757995	99	26	24,1	cyclictest	2995141	-21	diskmemload	00:18:37	7
2757995	99	26	0,24	cyclictest	125	-21	kswapd0	22:33:53	7
2757991	99	26	24,2	cyclictest	3343964	-21	kworker/u16:3+events_unbound	23:38:14	6
2757991	99	26	24,2	cyclictest	2819793	-21	kworker/u16:2+events_unbound	19:42:19	6
2757991	99	26	24,2	cyclictest	2780797	-21	kworker/u16:4+events_unbound	20:13:54	6
2757981	99	26	24,2	cyclictest	3380264	-21	kworker/u16:1+events_unbound	00:02:04	4
2757981	99	26	24,2	cyclictest	3343964	-21	kworker/u16:3+flush-8:0	23:36:03	4
2757981	99	26	24,2	cyclictest	3224024	-21	kworker/u16:3+flush-8:0	23:09:11	4
2757981	99	26	24,2	cyclictest	3224024	-21	kworker/u16:3+flush-8:0	23:04:23	4
2757981	99	26	23,2	cyclictest	2956233	-21	kworker/u16:0+flush-8:0	20:53:56	4
2757995	99	25	24,0	cyclictest	0	-21	swapper/7	21:06:07	7
2757995	99	25	24,0	cyclictest	0	-21	swapper/7	21:02:40	7
2757995	99	25	24,0	cyclictest	0	-21	swapper/7	19:15:01	7
2757995	99	25	21,2	cyclictest	3430791	-21	ssh	00:08:41	7
2757995	99	25	0,24	cyclictest	648	-21	systemd-journal	23:28:54	7
2757991	99	25	23,2	cyclictest	3380264	-21	kworker/u16:1+events_unbound	23:57:56	6
2757991	99	25	23,2	cyclictest	3308551	-21	kworker/u16:2+flush-8:0	23:27:01	6
2757991	99	25	23,2	cyclictest	3224024	-21	kworker/u16:3+events_unbound	23:15:51	6
2757991	99	25	23,2	cyclictest	3224024	-21	kworker/u16:3+events_unbound	23:15:51	6
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17