OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Sun Mar 01, 2026 12:43:34)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
910556	2	97	0,1	sleep7	910557	-21	tr	08:20:01	7
759730	99	65	62,2	cyclictest	1086862	-21	kworker/u16:5+flush-8:0	11:05:12	4
759730	99	63	61,2	cyclictest	1062701	-21	kworker/u16:0+events_unbound	09:59:56	4
759730	99	57	55,2	cyclictest	1062701	-21	kworker/u16:0+flush-8:0	11:30:11	4
759747	99	56	51,4	cyclictest	1413966	-21	latency_hist	11:59:41	7
759730	99	56	54,2	cyclictest	1062701	-21	kworker/u16:0+events_unbound	11:49:57	4
759730	99	55	53,2	cyclictest	1073368	-21	kworker/u16:2+events_unbound	09:35:12	4
759730	99	55	53,2	cyclictest	1014579	-21	kworker/u16:0+events_unbound	09:15:12	4
759730	99	53	51,2	cyclictest	1378031	-21	kworker/u16:2+events_unbound	12:24:56	4
759730	99	52	50,2	cyclictest	1062701	-21	kworker/u16:0+events_unbound	11:15:08	4
759740	99	51	48,2	cyclictest	792119	-21	kworker/u16:0+flush-8:0	07:45:08	6
759730	99	51	49,2	cyclictest	954606	-21	kworker/u16:1+events_unbound	08:45:11	4
759730	99	51	49,2	cyclictest	1245970	-21	kworker/u16:1+events_unbound	10:55:16	4
759730	99	50	48,2	cyclictest	983620	-21	kworker/u16:2+events_unbound	09:10:15	4
759730	99	49	47,2	cyclictest	849341	-21	kworker/u16:1+events_unbound	07:50:08	4
759730	99	49	47,2	cyclictest	1378031	-21	kworker/u16:2+events_unbound	12:05:06	4
759730	99	48	46,2	cyclictest	1062701	-21	kworker/u16:0+events_unbound	10:40:11	4
759730	99	48	45,2	cyclictest	792119	-21	kworker/u16:0+events_unbound	07:44:57	4
759730	99	48	45,2	cyclictest	1073368	-21	kworker/u16:2+events_unbound	10:29:52	4
759229	2	48	37,8	sleep4	0	-21	swapper/4	07:01:39	4
759730	99	47	45,2	cyclictest	1245970	-21	kworker/u16:1+events_unbound	11:10:15	4
759730	99	47	45,2	cyclictest	1086862	-21	kworker/u16:5+events_unbound	11:40:15	4
759730	99	47	45,2	cyclictest	1062701	-21	kworker/u16:0+flush-8:0	11:34:57	4
759730	99	47	44,2	cyclictest	751163	-21	kworker/u16:2+events_unbound	07:14:57	4
759730	99	47	44,2	cyclictest	751163	-21	kworker/u16:2+events_unbound	07:14:57	4
759740	99	46	44,2	cyclictest	983620	-21	kworker/u16:2+flush-8:0	09:00:07	6
759730	99	46	44,2	cyclictest	1040948	-21	kworker/u16:1+events_unbound	09:29:56	4
759730	99	45	43,2	cyclictest	1437007	-21	kworker/u16:3+flush-8:0	12:30:10	4
759730	99	45	43,2	cyclictest	1062701	-21	kworker/u16:0+events_unbound	11:00:13	4
759730	99	44	42,2	cyclictest	1086862	-21	kworker/u16:5+events_unbound	10:19:44	4
759730	99	44	42,2	cyclictest	1062701	-21	kworker/u16:0+events_unbound	10:53:27	4
759730	99	43	40,2	cyclictest	792119	-21	kworker/u16:0+events_unbound	07:39:58	4
759730	99	42	40,2	cyclictest	992983	-21	kworker/u16:4+events_unbound	09:09:32	4
759730	99	42	40,2	cyclictest	1086862	-21	kworker/u16:5+flush-8:0	11:25:24	4
759730	99	42	40,2	cyclictest	1062701	-21	kworker/u16:0+events_unbound	09:52:07	4
759738	99	41	39,1	cyclictest	811390	-21	kworker/u16:2+events_unbound	07:29:58	5
759730	99	41	39,2	cyclictest	1062701	-21	kworker/u16:0+events_unbound	11:56:52	4
759730	99	41	39,2	cyclictest	1062701	-21	kworker/u16:0+events_unbound	10:08:49	4
759730	99	41	39,2	cyclictest	1062701	-21	kworker/u16:0+events_unbound	10:08:49	4
759259	2	41	31,7	sleep6	0	-21	swapper/6	07:02:05	6
759730	99	40	38,2	cyclictest	1437007	-21	kworker/u16:3+events_unbound	12:21:43	4
759730	99	40	38,2	cyclictest	1233561	-21	kworker/u16:2+events_unbound	10:45:04	4
759730	99	40	38,2	cyclictest	1233561	-21	kworker/u16:2+events_unbound	10:45:04	4
759730	99	40	38,2	cyclictest	1062701	-21	kworker/u16:0+events_unbound	11:47:27	4
759730	99	40	38,2	cyclictest	1040948	-21	kworker/u16:1+events_unbound	09:25:52	4
759730	99	40	37,2	cyclictest	1062701	-21	kworker/u16:0+flush-8:0	10:09:52	4
759307	2	40	4,32	sleep7	0	-21	swapper/7	07:02:46	7
759730	99	39	37,2	cyclictest	1414007	-21	kworker/u16:1+flush-8:0	12:00:02	4
759730	99	39	37,2	cyclictest	1414007	-21	kworker/u16:1+events_unbound	12:18:27	4
759730	99	38	36,2	cyclictest	887907	-21	kworker/u16:3+events_unbound	09:24:08	4
759730	99	38	36,2	cyclictest	1086862	-21	kworker/u16:5+events_unbound	10:36:28	4
759730	99	38	36,2	cyclictest	1062701	-21	kworker/u16:0+events_unbound	09:46:36	4
759747	99	37	32,4	cyclictest	1449898	-21	latency_hist	12:14:42	7
759730	99	37	35,2	cyclictest	887907	-21	kworker/u16:3+flush-8:0	08:55:09	4
759730	99	37	35,2	cyclictest	1414007	-21	kworker/u16:1+events_unbound	12:09:47	4
759730	99	37	35,2	cyclictest	1086862	-21	kworker/u16:5+events_unbound	11:23:28	4
759730	99	37	35,2	cyclictest	1086862	-21	kworker/u16:5+events_unbound	10:19:03	4
759730	99	37	35,2	cyclictest	1062701	-21	kworker/u16:0+events_unbound	09:40:11	4
759730	99	36	34,2	cyclictest	811390	-21	kworker/u16:2+events_unbound	08:05:07	4
759730	99	36	33,2	cyclictest	811390	-21	kworker/u16:2+events_unbound	08:19:56	4
759419	2	36	26,7	sleep5	0	-21	swapper/5	07:04:20	5
759730	99	35	33,2	cyclictest	1086862	-21	kworker/u16:5+events_unbound	10:27:19	4
759740	99	34	32,2	cyclictest	740429	-21	kworker/u16:3+events_unbound	07:04:57	6
759738	99	34	32,2	cyclictest	1073368	-21	kworker/u16:2+flush-8:0	10:05:07	5
759738	99	34	32,2	cyclictest	1073368	-21	kworker/u16:2+flush-8:0	10:05:07	5
759730	99	34	32,2	cyclictest	1062701	-21	kworker/u16:0+events_unbound	09:59:19	4
759730	99	34	31,2	cyclictest	887907	-21	kworker/u16:3+flush-8:0	08:39:43	4
759747	99	31	29,1	cyclictest	871895	-21	kthreadcore	08:00:00	7
759747	99	31	25,5	cyclictest	964101	-21	latency_hist	08:49:42	7
759740	99	31	29,2	cyclictest	1073368	-21	kworker/u16:2+events_unbound	09:57:29	6
759730	99	30	28,2	cyclictest	887907	-21	kworker/u16:3+events_unbound	08:34:16	4
759730	99	30	28,2	cyclictest	878122	-21	kworker/u16:0+events_unbound	08:39:40	4
759730	99	30	27,2	cyclictest	954606	-21	kworker/u16:1+flush-8:0	08:49:57	4
759730	99	30	27,2	cyclictest	954606	-21	kworker/u16:1+flush-8:0	08:49:57	4
759740	99	28	26,2	cyclictest	887907	-21	kworker/u16:3+events_unbound	09:30:00	6
759740	99	27	25,2	cyclictest	1086862	-21	kworker/u16:5+flush-8:0	10:09:42	6
759740	99	27	25,2	cyclictest	1086862	-21	kworker/u16:5+flush-8:0	10:09:42	6
759730	99	27	25,2	cyclictest	887907	-21	kworker/u16:3+events_unbound	08:24:45	4
759730	99	27	25,2	cyclictest	67504	-21	kworker/u16:1+flush-8:0	07:10:10	4
1136612	2	27	0,0	chrt	1136610	-21	ssh	10:03:39	7
759740	99	25	23,2	cyclictest	1073368	-21	kworker/u16:2+events_unbound	10:15:12	6
759730	99	25	23,2	cyclictest	849341	-21	kworker/u16:1+events_unbound	07:55:09	4
759730	99	25	23,2	cyclictest	792119	-21	kworker/u16:0+flush-8:0	07:20:07	4
759740	99	24	22,2	cyclictest	992983	-21	kworker/u16:4+events_unbound	09:40:00	6
759740	99	24	22,2	cyclictest	954606	-21	kworker/u16:1+events_unbound	08:59:29	6
759740	99	24	22,2	cyclictest	878122	-21	kworker/u16:0+events_unbound	08:08:48	6
1218706	2	24	0,1	chrt	1218707	-21	cat	10:35:13	7
759740	99	23	21,2	cyclictest	67504	-21	kworker/u16:1+events_unbound	07:19:37	6
759740	99	23	21,2	cyclictest	67504	-21	kworker/u16:1+events_unbound	07:19:37	6
759738	99	23	21,2	cyclictest	954606	-21	kworker/u16:1+flush-8:0	08:49:59	5
759738	99	23	21,2	cyclictest	954606	-21	kworker/u16:1+flush-8:0	08:49:59	5
759730	99	23	21,2	cyclictest	811390	-21	kworker/u16:2+events_unbound	08:04:12	4
759747	99	22	22,0	cyclictest	0	-21	swapper/7	08:26:43	7
759740	99	22	19,2	cyclictest	1086862	-21	kworker/u16:5+events_unbound	10:39:42	6
759738	99	22	21,1	cyclictest	1245970	-21	kworker/u16:1+events_unbound	10:56:01	5
759730	99	22	20,2	cyclictest	740429	-21	kworker/u16:3+events_unbound	07:07:53	4
759747	99	21	20,0	cyclictest	0	-21	swapper/7	10:05:24	7
759747	99	21	20,0	cyclictest	0	-21	swapper/7	10:05:24	7
759740	99	21	19,2	cyclictest	1062701	-21	kworker/u16:0+flush-8:0	11:54:57	6
759738	99	21	19,2	cyclictest	1086862	-21	kworker/u16:5+flush-8:0	09:54:42	5
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17