OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Fri Mar 06, 2026 00:43:34)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
753648	2	99	0,1	sleep7	753646	-21	ssh	22:03:32	7
376020	99	57	55,2	cyclictest	704323	-21	kworker/u16:2+events_unbound	21:44:43	6
366556	2	57	46,7	sleep4	0	-21	swapper/4	18:59:32	4
376020	99	55	53,2	cyclictest	946047	-21	kworker/u16:2+events_unbound	00:09:58	6
376020	99	50	48,2	cyclictest	585470	-21	kworker/u16:4+events_unbound	21:59:42	6
376020	99	49	47,2	cyclictest	691957	-21	kworker/u16:1+flush-8:0	21:49:54	6
376020	99	49	47,2	cyclictest	1017892	-21	kworker/u16:1+flush-8:0	23:54:39	6
376020	99	48	46,2	cyclictest	850024	-21	kworker/u16:3+flush-8:0	23:44:54	6
376020	99	47	44,2	cyclictest	946047	-21	kworker/u16:2+events_unbound	00:14:57	6
376020	99	46	44,2	cyclictest	571230	-21	kworker/u16:3+events_unbound	21:09:56	6
376024	99	44	0,43	cyclictest	837882	-21	latency_hist	22:39:24	7
376020	99	43	41,2	cyclictest	718495	-21	kworker/u16:0+events_unbound	22:07:14	6
376020	99	42	40,2	cyclictest	718495	-21	kworker/u16:0+flush-8:0	22:19:39	6
376020	99	42	40,2	cyclictest	1053603	-21	kworker/u16:5+flush-8:0	00:24:39	6
376024	99	41	36,4	cyclictest	694137	-21	latency_hist	21:39:24	7
376020	99	41	39,2	cyclictest	585470	-21	kworker/u16:4+events_unbound	22:31:34	6
376020	99	41	39,2	cyclictest	585470	-21	kworker/u16:4+events_unbound	22:14:59	6
376020	99	41	39,2	cyclictest	585470	-21	kworker/u16:4+events_unbound	22:14:59	6
376020	99	41	39,2	cyclictest	585470	-21	kworker/u16:4+events_unbound	21:40:39	6
376020	99	41	39,2	cyclictest	585470	-21	kworker/u16:4+events_unbound	21:40:39	6
376020	99	41	39,2	cyclictest	1001944	-21	kworker/u16:0+flush-8:0	00:04:55	6
376020	99	40	38,2	cyclictest	883732	-21	kworker/u16:0+events_unbound	23:06:17	6
376020	99	40	38,2	cyclictest	718495	-21	kworker/u16:0+events_unbound	21:58:54	6
376020	99	40	38,2	cyclictest	585470	-21	kworker/u16:4+events_unbound	21:29:58	6
375620	2	40	30,7	sleep6	0	-21	swapper/6	19:02:55	6
376020	99	39	37,2	cyclictest	850024	-21	kworker/u16:3+events_unbound	23:12:13	6
376020	99	39	37,2	cyclictest	850024	-21	kworker/u16:3+events_unbound	22:51:02	6
376020	99	39	37,2	cyclictest	773724	-21	kworker/u16:3+events_unbound	22:13:06	6
376020	99	39	37,2	cyclictest	571230	-21	kworker/u16:3+events_unbound	21:17:14	6
376020	99	39	36,2	cyclictest	981694	-21	kworker/u16:4+events_unbound	23:40:17	6
376020	99	39	36,2	cyclictest	981694	-21	kworker/u16:4+events_unbound	23:40:17	6
375527	2	39	4,30	sleep7	0	-21	swapper/7	19:01:35	7
376020	99	38	36,2	cyclictest	946047	-21	kworker/u16:2+events_unbound	23:30:13	6
376020	99	38	36,2	cyclictest	898188	-21	kworker/u16:1+events_unbound	23:24:06	6
376020	99	38	36,2	cyclictest	850024	-21	kworker/u16:3+events_unbound	23:53:21	6
376020	99	37	35,2	cyclictest	946047	-21	kworker/u16:2+events_unbound	00:23:42	6
376020	99	37	35,2	cyclictest	946047	-21	kworker/u16:2+events_unbound	00:23:42	6
376020	99	37	35,2	cyclictest	883732	-21	kworker/u16:0+events_unbound	23:15:42	6
376020	99	37	35,2	cyclictest	826326	-21	kworker/u16:1+events_unbound	22:38:51	6
376020	99	37	35,2	cyclictest	800417	-21	kworker/u16:2+events_unbound	22:41:02	6
376020	99	37	35,2	cyclictest	800417	-21	kworker/u16:2+events_unbound	22:28:42	6
376020	99	37	35,2	cyclictest	571230	-21	kworker/u16:3+events_unbound	21:21:30	6
376020	99	37	35,2	cyclictest	552400	-21	kworker/u16:0+events_unbound	21:07:30	6
376020	99	37	35,2	cyclictest	1017892	-21	kworker/u16:1+events_unbound	00:29:48	6
376020	99	36	34,2	cyclictest	898188	-21	kworker/u16:1+events_unbound	23:34:53	6
376020	99	36	34,2	cyclictest	691957	-21	kworker/u16:1+events_unbound	21:37:22	6
376020	99	36	34,2	cyclictest	585470	-21	kworker/u16:4+events_unbound	21:27:51	6
376010	99	36	34,2	cyclictest	850024	-21	kworker/u16:3+events_unbound	23:39:50	4
376010	99	36	34,2	cyclictest	850024	-21	kworker/u16:3+events_unbound	23:39:50	4
376010	99	36	34,2	cyclictest	408254	-21	kworker/u16:4+events_unbound	19:19:49	4
376024	99	35	28,6	cyclictest	1017464	-21	latency_hist	23:54:24	7
376020	99	35	33,2	cyclictest	883732	-21	kworker/u16:0+events_unbound	22:58:18	6
376020	99	35	33,2	cyclictest	1001944	-21	kworker/u16:0+events_unbound	00:01:22	6
376010	99	35	32,3	cyclictest	418310	-21	kworker/u16:1+events_unbound	19:39:47	4
376024	99	34	30,3	cyclictest	124	-21	kswapd0	00:19:39	7
376024	99	34	30,3	cyclictest	124	-21	kswapd0	00:19:39	7
376020	99	34	32,2	cyclictest	946047	-21	kworker/u16:2+events_unbound	23:28:47	6
376020	99	34	32,2	cyclictest	850024	-21	kworker/u16:3+events_unbound	22:46:21	6
376020	99	34	32,2	cyclictest	533665	-21	kworker/u16:2+flush-8:0	20:44:24	6
376010	99	34	32,2	cyclictest	883732	-21	kworker/u16:0+events_unbound	23:15:16	4
376020	99	33	31,2	cyclictest	835074	-21	kworker/u16:5+events_unbound	23:02:42	6
376020	99	33	31,2	cyclictest	835074	-21	kworker/u16:5+events_unbound	23:02:42	6
376010	99	33	31,2	cyclictest	718495	-21	kworker/u16:0+events_unbound	21:58:28	4
376010	99	33	31,2	cyclictest	571230	-21	kworker/u16:3+events_unbound	21:27:59	4
376020	99	32	29,3	cyclictest	552400	-21	kworker/u16:0+flush-8:0	20:59:50	6
376020	99	32	29,3	cyclictest	552400	-21	kworker/u16:0+flush-8:0	20:59:50	6
376020	99	31	29,2	cyclictest	581191	-21	kworker/u16:1+events_unbound	20:47:06	6
376020	99	31	28,3	cyclictest	408254	-21	kworker/u16:4+flush-8:0	19:19:50	6
376010	99	31	29,2	cyclictest	691957	-21	kworker/u16:1+events_unbound	21:52:44	4
376010	99	31	29,2	cyclictest	3864650	-21	kworker/u16:3+events_unbound	19:09:39	4
376024	99	30	25,4	cyclictest	897767	-21	latency_hist	23:04:24	7
376024	99	30	25,4	cyclictest	897767	-21	latency_hist	23:04:24	7
376020	99	30	28,2	cyclictest	523594	-21	kworker/u16:1+events_unbound	20:34:44	6
376020	99	30	28,2	cyclictest	418310	-21	kworker/u16:1+events_unbound	19:59:41	6
376020	99	30	27,2	cyclictest	418310	-21	kworker/u16:1+events_unbound	19:54:34	6
376010	99	30	28,2	cyclictest	946047	-21	kworker/u16:2+events_unbound	23:45:03	4
376010	99	30	28,2	cyclictest	3864650	-21	kworker/u16:3+events_unbound	19:04:56	4
375563	2	30	20,7	sleep5	0	-21	swapper/5	19:02:05	5
376010	99	29	27,2	cyclictest	946047	-21	kworker/u16:2+events_unbound	00:14:51	4
376010	99	29	27,2	cyclictest	533665	-21	kworker/u16:2+events_unbound	20:39:54	4
376010	99	29	27,2	cyclictest	337536	-21	kworker/u16:2+events_unbound	20:13:47	4
376020	99	28	26,2	cyclictest	533665	-21	kworker/u16:2+events_unbound	20:24:48	6
376020	99	28	26,2	cyclictest	523594	-21	kworker/u16:1+events_unbound	20:19:06	6
376020	99	28	26,2	cyclictest	337536	-21	kworker/u16:2+events_unbound	20:13:06	6
376020	99	28	25,2	cyclictest	552400	-21	kworker/u16:0+flush-8:0	20:49:45	6
376010	99	28	26,2	cyclictest	571230	-21	kworker/u16:3+events_unbound	21:03:23	4
376010	99	28	26,2	cyclictest	571230	-21	kworker/u16:3+events_unbound	21:03:23	4
376010	99	28	26,2	cyclictest	552400	-21	kworker/u16:0+events_unbound	20:34:14	4
376010	99	28	26,2	cyclictest	491524	-21	kworker/u16:0+events_unbound	20:20:51	4
376010	99	28	26,2	cyclictest	1017892	-21	kworker/u16:1+events_unbound	00:09:50	4
376024	99	27	22,4	cyclictest	385479	-21	latency_hist	19:09:25	7
376024	99	27	20,6	cyclictest	610353	-21	latency_hist	21:04:24	7
376024	99	27	20,6	cyclictest	610353	-21	latency_hist	21:04:24	7
376020	99	27	25,2	cyclictest	337536	-21	kworker/u16:2+events_unbound	19:42:35	6
376011	99	27	23,4	cyclictest	571230	-21	kworker/u16:3+flush-8:0	21:29:50	5
376010	99	27	25,2	cyclictest	337536	-21	kworker/u16:2+events_unbound	19:49:10	4
376010	99	27	25,2	cyclictest	1053603	-21	kworker/u16:5+events_unbound	00:25:26	4
376020	99	26	24,2	cyclictest	458178	-21	kworker/u16:3+events_unbound	19:52:35	6
376020	99	25	23,2	cyclictest	408254	-21	kworker/u16:4+events_unbound	19:32:23	6
376020	99	25	23,2	cyclictest	337536	-21	kworker/u16:2+events_unbound	19:29:06	6
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17