OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Sun Apr 05, 2026 12:43:34)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
585644	2	98	0,1	sleep7	585646	-21	tr	11:37:46	7
585644	2	98	0,1	sleep7	585646	-21	tr	11:37:46	7
4161173	2	61	49,8	sleep6	0	-21	swapper/6	06:58:51	6
4161370	2	59	48,8	sleep5	0	-21	swapper/5	07:01:39	5
4161707	99	58	56,2	cyclictest	509404	-21	kworker/u16:3+events_unbound	12:07:37	4
4161707	99	53	51,2	cyclictest	509404	-21	kworker/u16:3+events_unbound	11:27:53	4
4161707	99	53	50,2	cyclictest	509404	-21	kworker/u16:3+events_unbound	11:42:40	4
4161707	99	51	48,2	cyclictest	601556	-21	kworker/u16:1+events_unbound	12:17:24	4
4161707	99	51	48,2	cyclictest	601556	-21	kworker/u16:1+events_unbound	12:17:24	4
4161310	2	51	41,7	sleep4	0	-21	swapper/4	07:00:49	4
4161707	99	50	48,2	cyclictest	358519	-21	kworker/u16:2+flush-8:0	10:42:41	4
4161717	99	49	46,2	cyclictest	38828	-21	kworker/u16:0+events_unbound	08:52:23	6
4161707	99	49	47,2	cyclictest	509404	-21	kworker/u16:3+events_unbound	12:12:54	4
4161721	99	48	46,1	cyclictest	453278	-21	switchtime	10:42:54	7
4161721	99	46	43,2	cyclictest	392598	-21	smart_sda	10:17:53	7
4161721	99	46	43,2	cyclictest	392598	-21	smart_sda	10:17:53	7
4161707	99	46	44,2	cyclictest	358519	-21	kworker/u16:2+events_unbound	10:36:33	4
4161707	99	46	44,2	cyclictest	213581	-21	kworker/u16:3+events_unbound	09:41:01	4
4161707	99	46	44,2	cyclictest	213581	-21	kworker/u16:3+events_unbound	09:41:01	4
4161707	99	45	43,2	cyclictest	509404	-21	kworker/u16:3+events_unbound	11:17:10	4
4161717	99	44	42,2	cyclictest	358519	-21	kworker/u16:2+events_unbound	11:42:50	6
4161707	99	44	42,2	cyclictest	554506	-21	kworker/u16:0+flush-8:0	11:37:52	4
4161707	99	44	42,2	cyclictest	554506	-21	kworker/u16:0+flush-8:0	11:37:52	4
4161721	99	43	40,2	cyclictest	67440	-21	awk	07:52:23	7
4161717	99	43	41,2	cyclictest	38828	-21	kworker/u16:0+flush-8:0	08:12:23	6
4161707	99	43	41,2	cyclictest	509404	-21	kworker/u16:3+events_unbound	11:57:51	4
4161707	99	43	41,2	cyclictest	274698	-21	kworker/u16:1+events_unbound	10:29:53	4
4161707	99	43	41,2	cyclictest	191801	-21	kworker/u16:2+events_unbound	09:20:05	4
4161717	99	42	40,2	cyclictest	671624	-21	kworker/u16:4+flush-8:0	12:27:22	6
4161707	99	42	40,2	cyclictest	601556	-21	kworker/u16:1+events_unbound	12:05:54	4
4161707	99	42	40,2	cyclictest	601556	-21	kworker/u16:1+events_unbound	11:52:56	4
4161707	99	42	40,2	cyclictest	509404	-21	kworker/u16:3+events_unbound	11:21:22	4
4161707	99	42	40,2	cyclictest	332733	-21	kworker/u16:0+events_unbound	10:05:46	4
4161717	99	41	38,3	cyclictest	274699	-21	kworker/u16:2+events_unbound	09:37:50	6
4161717	99	41	38,3	cyclictest	274699	-21	kworker/u16:2+events_unbound	09:37:50	6
4161707	99	41	39,2	cyclictest	601556	-21	kworker/u16:1+events_unbound	12:28:52	4
4161707	99	41	39,2	cyclictest	554506	-21	kworker/u16:0+events_unbound	11:50:57	4
4161707	99	41	39,2	cyclictest	554506	-21	kworker/u16:0+events_unbound	11:33:41	4
4161707	99	41	39,2	cyclictest	426663	-21	kworker/u16:0+events_unbound	10:38:42	4
4161707	99	41	39,2	cyclictest	274699	-21	kworker/u16:2+events_unbound	09:51:50	4
4161707	99	40	38,2	cyclictest	671624	-21	kworker/u16:4+events_unbound	12:26:13	4
4161707	99	40	38,2	cyclictest	528125	-21	kworker/u16:1+events_unbound	11:23:45	4
4161707	99	40	38,2	cyclictest	440846	-21	kworker/u16:3+events_unbound	10:49:29	4
4161707	99	40	38,2	cyclictest	426663	-21	kworker/u16:0+events_unbound	11:03:24	4
4161707	99	40	38,2	cyclictest	237731	-21	kworker/u16:0+events_unbound	09:25:21	4
4161707	99	40	38,2	cyclictest	213581	-21	kworker/u16:3+events_unbound	09:10:14	4
4161707	99	39	37,2	cyclictest	509404	-21	kworker/u16:3+events_unbound	11:10:49	4
4161707	99	39	37,2	cyclictest	406448	-21	kworker/u16:3+events_unbound	10:27:14	4
4161707	99	39	37,2	cyclictest	358519	-21	kworker/u16:2+events_unbound	10:59:33	4
4161707	99	39	37,2	cyclictest	358519	-21	kworker/u16:2+events_unbound	10:59:33	4
4161707	99	39	37,2	cyclictest	358519	-21	kworker/u16:2+events_unbound	10:11:26	4
4161707	99	39	37,2	cyclictest	274698	-21	kworker/u16:1+events_unbound	10:15:13	4
4161707	99	39	37,2	cyclictest	213581	-21	kworker/u16:3+events_unbound	09:13:22	4
4161295	2	39	5,30	sleep7	0	-21	swapper/7	07:00:36	7
4161707	99	38	36,2	cyclictest	332733	-21	kworker/u16:0+events_unbound	10:01:42	4
4161721	99	37	34,2	cyclictest	699763	-21	sed	12:27:38	7
4161721	99	37	0,35	cyclictest	462660	-21	kthreadcore	10:47:44	7
4161717	99	37	36,1	cyclictest	213581	-21	kworker/u16:3+flush-8:0	09:47:23	6
4161707	99	37	35,2	cyclictest	4133287	-21	kworker/u16:4+events_unbound	09:35:29	4
4161707	99	37	35,2	cyclictest	371367	-21	kworker/u16:0+events_unbound	10:18:42	4
4161707	99	37	35,2	cyclictest	371367	-21	kworker/u16:0+events_unbound	10:18:42	4
4161721	99	36	33,2	cyclictest	225594	-21	ssh	09:12:23	7
4161717	99	36	34,2	cyclictest	426663	-21	kworker/u16:0+events_unbound	11:07:57	6
4161717	99	36	33,2	cyclictest	426663	-21	kworker/u16:0+flush-8:0	11:12:28	6
4161717	99	36	33,2	cyclictest	191801	-21	kworker/u16:2+flush-8:0	09:17:29	6
4161707	99	36	34,2	cyclictest	4152306	-21	kworker/u16:0+events_unbound	07:21:58	4
4161721	99	35	33,1	cyclictest	611834	-21	ssh	11:49:13	7
4161721	99	35	0,34	cyclictest	284166	-21	rm	09:33:09	7
4161717	99	35	33,2	cyclictest	358519	-21	kworker/u16:2+events_unbound	11:37:49	6
4161717	99	35	33,2	cyclictest	358519	-21	kworker/u16:2+events_unbound	11:37:49	6
4161712	99	35	33,2	cyclictest	625668	-21	kworker/u16:2+flush-8:0	11:57:48	5
4161707	99	35	33,2	cyclictest	4152306	-21	kworker/u16:0+flush-8:0	07:22:49	4
4161707	99	35	33,2	cyclictest	38828	-21	kworker/u16:0+events_unbound	08:42:51	4
4161707	99	34	32,2	cyclictest	477305	-21	kworker/u16:1+events_unbound	10:53:18	4
4161707	99	34	32,2	cyclictest	3452001	-21	kworker/u16:1+events_unbound	07:44:33	4
4161707	99	34	32,2	cyclictest	237731	-21	kworker/u16:0+events_unbound	09:32:02	4
4161717	99	33	31,2	cyclictest	671624	-21	kworker/u16:4+events_unbound	12:27:45	6
4161707	99	33	31,2	cyclictest	4133287	-21	kworker/u16:4+events_unbound	09:03:26	4
4161707	99	33	31,2	cyclictest	213581	-21	kworker/u16:3+events_unbound	09:46:57	4
4161721	99	32	32,0	cyclictest	0	-21	swapper/7	11:47:06	7
4161721	99	32	31,0	cyclictest	0	-21	swapper/7	10:07:26	7
4161717	99	32	30,2	cyclictest	426663	-21	kworker/u16:0+flush-8:0	10:52:49	6
4161717	99	32	30,2	cyclictest	358519	-21	kworker/u16:2+flush-8:0	10:32:29	6
4161717	99	32	29,2	cyclictest	358519	-21	kworker/u16:2+flush-8:0	10:27:29	6
4161707	99	32	29,2	cyclictest	4133287	-21	kworker/u16:4+events_unbound	08:47:26	4
4161721	99	31	31,0	cyclictest	0	-21	swapper/7	10:54:14	7
4161721	99	31	30,1	cyclictest	0	-21	swapper/7	12:18:05	7
4161721	99	31	30,1	cyclictest	0	-21	swapper/7	12:18:05	7
4161721	99	31	29,1	cyclictest	659760	-21	ssh	12:08:34	7
4161717	99	31	29,2	cyclictest	554506	-21	kworker/u16:0+events_unbound	11:52:54	6
4161717	99	31	27,3	cyclictest	509404	-21	kworker/u16:3+flush-8:0	11:17:37	6
4161712	99	31	27,3	cyclictest	697059	-21	kworker/u16:0+flush-8:0	12:27:27	5
4161712	99	31	22,8	cyclictest	271331	-21	smart_sda	09:27:49	5
4161707	99	31	29,2	cyclictest	213581	-21	kworker/u16:3+events_unbound	09:53:45	4
4161721	99	30	25,4	cyclictest	359188	-21	latency_hist	10:07:23	7
4161717	99	30	27,2	cyclictest	440846	-21	kworker/u16:3+flush-8:0	10:42:23	6
4161717	99	30	27,2	cyclictest	426663	-21	kworker/u16:0+events_unbound	11:02:37	6
4161721	99	29	27,1	cyclictest	671753	-21	ssh	12:13:25	7
4161717	99	29	27,2	cyclictest	4133287	-21	kworker/u16:4+events_unbound	09:25:33	6
4161717	99	29	27,2	cyclictest	274699	-21	kworker/u16:2+events_unbound	09:33:38	6
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17