OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Sun Feb 01, 2026 00:43:34)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
3183622	99	53	52,1	cyclictest	3662142	-21	kworker/u16:4+flush-8:0	23:07:06	5
3183174	2	47	36,7	sleep6	0	-21	swapper/6	19:04:31	6
3183622	99	46	44,2	cyclictest	3390434	-21	kworker/u16:0+flush-8:0	21:47:01	5
6265	89	45	0,38	rtkit-daemon	0	-21	swapper/5	19:03:04	5
3183618	99	44	41,2	cyclictest	3528494	-21	kworker/u16:4+flush-8:0	22:16:36	4
3183214	2	41	1,36	sleep7	0	-21	swapper/7	19:05:04	7
3183622	99	40	38,2	cyclictest	3564868	-21	kworker/u16:0+flush-8:0	22:07:10	5
3183622	99	39	37,2	cyclictest	3821892	-21	kworker/u16:3+events_unbound	00:12:01	5
3183618	99	39	37,2	cyclictest	3564868	-21	kworker/u16:0+flush-8:0	22:31:36	4
3183622	99	38	36,2	cyclictest	3707865	-21	kworker/u16:0+events_unbound	23:51:32	5
3183622	99	38	35,2	cyclictest	3564868	-21	kworker/u16:0+events_unbound	22:36:54	5
3183622	99	37	35,2	cyclictest	3589481	-21	kworker/u16:2+events_unbound	22:22:05	5
3183622	99	37	35,2	cyclictest	3564868	-21	kworker/u16:0+flush-8:0	22:17:01	5
3183622	99	37	35,2	cyclictest	3564868	-21	kworker/u16:0+events_unbound	22:56:35	5
3183622	99	37	35,2	cyclictest	3174036	-21	kworker/u16:3+events_unbound	19:57:01	5
3183618	99	37	33,3	cyclictest	3734452	-21	kworker/u16:2+flush-8:0	23:36:53	4
3183622	99	36	34,2	cyclictest	3661260	-21	kworker/u16:3+flush-8:0	23:02:13	5
3183622	99	35	33,2	cyclictest	3890964	-21	kworker/u16:1+events_unbound	00:28:37	5
3183618	99	35	33,2	cyclictest	3539421	-21	kworker/u16:5+flush-8:0	22:03:10	4
3183622	99	34	32,2	cyclictest	3839415	-21	kworker/u16:4+events_unbound	00:03:21	5
3183624	99	33	31,1	cyclictest	3419089	-21	diskmemload	22:51:30	7
3183622	99	33	31,2	cyclictest	3890964	-21	kworker/u16:1+flush-8:0	00:31:56	5
3183622	99	33	31,2	cyclictest	3707865	-21	kworker/u16:0+events_unbound	23:58:32	5
3183622	99	33	31,2	cyclictest	3707865	-21	kworker/u16:0+events_unbound	23:37:37	5
3183622	99	32	30,2	cyclictest	3821892	-21	kworker/u16:3+events_unbound	00:22:57	5
3183622	99	32	30,2	cyclictest	3734452	-21	kworker/u16:2+events_unbound	23:34:33	5
3183622	99	32	30,2	cyclictest	3707865	-21	kworker/u16:0+flush-8:0	00:21:35	5
3183622	99	32	30,2	cyclictest	3662142	-21	kworker/u16:4+events_unbound	22:46:58	5
3183622	99	32	30,2	cyclictest	3661260	-21	kworker/u16:3+events_unbound	23:11:57	5
3183622	99	32	30,2	cyclictest	3293583	-21	kworker/u16:4+flush-8:0	21:41:36	5
3183279	2	32	7,8	sleep4	0	-21	swapper/4	19:05:57	4
3183622	99	31	29,2	cyclictest	3686048	-21	kworker/u16:1+events_unbound	22:58:05	5
3183622	99	31	29,2	cyclictest	3662142	-21	kworker/u16:4+flush-8:0	23:16:51	5
3183622	99	31	29,2	cyclictest	3390434	-21	kworker/u16:0+events_unbound	21:11:59	5
3183622	99	31	29,2	cyclictest	3174036	-21	kworker/u16:3+flush-8:0	19:12:06	5
3183622	99	31	29,2	cyclictest	3129571	-21	kworker/u16:2+events_unbound	21:28:50	5
3183622	99	31	29,2	cyclictest	3129571	-21	kworker/u16:2+events_unbound	21:24:18	5
3183622	99	30	28,2	cyclictest	3734452	-21	kworker/u16:2+events_unbound	23:53:29	5
3183622	99	30	28,2	cyclictest	3564868	-21	kworker/u16:0+events_unbound	22:42:25	5
3183622	99	30	28,2	cyclictest	3564868	-21	kworker/u16:0+events_unbound	22:42:25	5
3183622	99	30	28,2	cyclictest	3564868	-21	kworker/u16:0+events_unbound	22:34:41	5
3183624	99	29	28,0	cyclictest	0	-21	swapper/7	19:37:01	7
3183624	99	29	24,4	cyclictest	3235610	-21	latency_hist	19:31:36	7
3183622	99	29	27,2	cyclictest	3528494	-21	kworker/u16:4+events_unbound	21:57:30	5
3183622	99	29	27,2	cyclictest	3390434	-21	kworker/u16:0+events_unbound	21:21:26	5
3183618	99	29	27,2	cyclictest	3647670	-21	kworker/u16:1+events_unbound	22:41:30	4
3902947	2	28	0,1	chrt	3893725	-21	/usr/sbin/munin	00:27:10	7
3183622	99	28	27,1	cyclictest	3390434	-21	kworker/u16:0+events_unbound	21:42:52	5
3183622	99	28	26,2	cyclictest	3791873	-21	kworker/u16:1+events_unbound	23:44:06	5
3183622	99	28	26,2	cyclictest	3528494	-21	kworker/u16:4+events_unbound	22:14:53	5
3183622	99	28	26,2	cyclictest	3513750	-21	kworker/u16:1+events_unbound	21:53:49	5
3183622	99	28	26,2	cyclictest	3293583	-21	kworker/u16:4+events_unbound	21:32:50	5
3183622	99	28	26,2	cyclictest	3293583	-21	kworker/u16:4+events_unbound	20:57:02	5
3183622	99	28	26,2	cyclictest	3293583	-21	kworker/u16:4+events_unbound	20:46:31	5
3183622	99	28	26,2	cyclictest	3129571	-21	kworker/u16:2+flush-8:0	19:26:55	5
3183622	99	28	25,2	cyclictest	3707865	-21	kworker/u16:0+events_unbound	23:21:55	5
3183622	99	28	25,2	cyclictest	3707865	-21	kworker/u16:0+events_unbound	23:21:55	5
3183622	99	28	25,2	cyclictest	3539421	-21	kworker/u16:5+flush-8:0	22:26:41	5
3183618	99	28	26,2	cyclictest	3734452	-21	kworker/u16:2+flush-8:0	23:27:05	4
3183618	99	28	26,2	cyclictest	3564868	-21	kworker/u16:0+events_unbound	22:32:05	4
3183623	99	27	25,2	cyclictest	3390434	-21	kworker/u16:0+events_unbound	21:31:59	6
3183622	99	27	25,2	cyclictest	3821892	-21	kworker/u16:3+flush-8:0	00:06:56	5
3183622	99	27	25,2	cyclictest	3734452	-21	kworker/u16:2+events_unbound	23:26:52	5
3183622	99	27	25,2	cyclictest	3293583	-21	kworker/u16:4+events_unbound	21:09:37	5
3625226	2	26	0,1	chrt	3625227	-21	ssh	22:30:57	7
3183624	99	26	24,1	cyclictest	0	-21	swapper/7	20:26:49	7
3183618	99	26	22,3	cyclictest	3332284	-21	kworker/u16:1+flush-8:0	20:22:02	4
3183623	99	25	23,2	cyclictest	3129571	-21	kworker/u16:2+events_unbound	19:56:31	6
3183622	99	25	23,2	cyclictest	3245227	-21	kworker/u16:1+flush-8:0	19:37:05	5
3183618	99	25	23,2	cyclictest	2468144	-21	kworker/u16:0+flush-8:0	19:51:59	4
3183618	99	25	23,2	cyclictest	2468144	-21	kworker/u16:0+flush-8:0	19:22:05	4
3183618	99	25	22,2	cyclictest	3791873	-21	kworker/u16:1+flush-8:0	23:56:36	4
3183623	99	24	22,2	cyclictest	3707865	-21	kworker/u16:0+events_unbound	23:24:33	6
3183623	99	24	22,2	cyclictest	3707865	-21	kworker/u16:0+events_unbound	23:24:33	6
3183622	99	24	22,2	cyclictest	3174036	-21	kworker/u16:3+events_unbound	19:17:03	5
3183622	99	24	21,3	cyclictest	2468144	-21	kworker/u16:0+flush-8:0	20:01:51	5
3183618	99	24	23,1	cyclictest	3528494	-21	kworker/u16:4+flush-8:0	21:57:01	4
3183618	99	24	22,2	cyclictest	3845334	-21	kworker/u16:2+events_unbound	00:21:59	4
3183624	99	23	18,4	cyclictest	3917282	-21	latency_hist	00:36:35	7
3183623	99	23	21,2	cyclictest	3245227	-21	kworker/u16:1+events_unbound	19:41:32	6
3183618	99	23	21,2	cyclictest	3293583	-21	kworker/u16:4+flush-8:0	20:26:57	4
3183623	99	22	20,2	cyclictest	3539421	-21	kworker/u16:5+events_unbound	22:08:46	6
3183623	99	22	20,2	cyclictest	3129571	-21	kworker/u16:2+events_unbound	21:24:57	6
3183622	99	22	20,2	cyclictest	3539421	-21	kworker/u16:5+events_unbound	22:05:30	5
3183618	99	22	19,3	cyclictest	3564868	-21	kworker/u16:0+events_unbound	22:47:37	4
3183624	99	21	20,0	cyclictest	0	-21	swapper/7	21:41:57	7
3183623	99	21	19,2	cyclictest	3564868	-21	kworker/u16:0+events_unbound	22:55:52	6
3183623	99	21	19,2	cyclictest	3129571	-21	kworker/u16:2+events_unbound	21:14:14	6
3183622	99	21	19,2	cyclictest	3293583	-21	kworker/u16:4+events_unbound	20:47:34	5
3183618	99	21	19,2	cyclictest	3293583	-21	kworker/u16:4+flush-8:0	20:41:35	4
3183618	99	21	18,2	cyclictest	2468144	-21	kworker/u16:0+events_unbound	19:06:49	4
3183618	99	21	18,2	cyclictest	2468144	-21	kworker/u16:0+events_unbound	19:06:49	4
3183624	99	20	17,2	cyclictest	3293552	-21	munin-run	20:01:36	7
3183624	99	20	16,3	cyclictest	650	-21	systemd-journal	22:36:31	7
3183623	99	20	18,2	cyclictest	3707865	-21	kworker/u16:0+events_unbound	00:13:17	6
3183623	99	20	18,2	cyclictest	3661260	-21	kworker/u16:3+events_unbound	22:46:21	6
3183623	99	20	18,2	cyclictest	3661260	-21	kworker/u16:3+events_unbound	22:46:21	6
3183623	99	20	18,2	cyclictest	3390434	-21	kworker/u16:0+events_unbound	21:46:21	6
3183623	99	20	18,2	cyclictest	3174036	-21	kworker/u16:3+events_unbound	19:07:38	6
3183623	99	20	18,2	cyclictest	3174036	-21	kworker/u16:3+events_unbound	19:07:38	6
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17