OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Mon Feb 09, 2026 00:43:34)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
1368159	2	94	0,0	sleep6	0	-21	swapper/6	20:16:04	6
1368159	2	94	0,0	sleep6	0	-21	swapper/6	20:16:04	6
1404817	2	92	0,0	sleep4	0	-21	swapper/4	20:31:24	4
1228747	2	74	47,7	sleep4	0	-21	swapper/4	19:04:32	4
1228549	2	57	47,7	sleep5	0	-21	swapper/5	19:01:44	5
1220434	2	54	19,29	sleep7	0	-21	swapper/7	19:01:22	7
1229138	99	49	47,2	cyclictest	1435886	-21	kworker/u16:4+events_unbound	22:11:30	6
1229131	99	47	43,3	cyclictest	1837351	-21	kworker/u16:5+flush-8:0	00:31:03	4
1229142	99	45	38,6	cyclictest	1339206	-21	latency_hist	20:01:04	7
1229138	99	42	40,2	cyclictest	1610363	-21	kworker/u16:0+events_unbound	22:16:29	6
1229138	99	41	38,2	cyclictest	1683349	-21	kworker/u16:2+flush-8:0	22:51:09	6
1228749	2	41	31,7	sleep6	0	-21	swapper/6	19:04:34	6
1229138	99	40	37,2	cyclictest	1837351	-21	kworker/u16:5+events_unbound	00:31:06	6
1229138	99	39	37,2	cyclictest	1683349	-21	kworker/u16:2+events_unbound	23:10:14	6
1229138	99	39	36,2	cyclictest	1300920	-21	kworker/u16:4+flush-8:0	19:51:04	6
1229138	99	38	36,2	cyclictest	1884478	-21	kworker/u16:3+events_unbound	00:04:14	6
1229138	99	38	36,2	cyclictest	1683349	-21	kworker/u16:2+events_unbound	23:16:02	6
1229138	99	38	36,2	cyclictest	1584915	-21	kworker/u16:3+events_unbound	22:37:38	6
1229138	99	37	35,2	cyclictest	1683349	-21	kworker/u16:2+events_unbound	23:26:49	6
1229138	99	37	35,2	cyclictest	1683349	-21	kworker/u16:2+events_unbound	23:20:06	6
1229138	99	37	35,2	cyclictest	1670571	-21	kworker/u16:1+flush-8:0	23:46:03	6
1229138	99	37	35,2	cyclictest	1655769	-21	kworker/u16:5+events_unbound	22:23:06	6
1229138	99	37	35,2	cyclictest	1610363	-21	kworker/u16:0+events_unbound	22:36:02	6
1229138	99	36	34,2	cyclictest	1933495	-21	kworker/u16:1+events_unbound	00:22:13	6
1229138	99	36	34,2	cyclictest	1683349	-21	kworker/u16:2+flush-8:0	22:41:50	6
1229138	99	36	34,2	cyclictest	1498630	-21	kworker/u16:1+events_unbound	21:34:30	6
1229138	99	36	34,2	cyclictest	1435886	-21	kworker/u16:4+events_unbound	21:21:01	6
1229138	99	35	33,2	cyclictest	1670571	-21	kworker/u16:1+events_unbound	23:01:05	6
1229138	99	35	33,2	cyclictest	1610363	-21	kworker/u16:0+events_unbound	22:10:22	6
1229138	99	35	33,2	cyclictest	1584915	-21	kworker/u16:3+flush-8:0	22:01:54	6
1229138	99	35	32,3	cyclictest	1498630	-21	kworker/u16:1+flush-8:0	21:26:04	6
1229142	99	34	33,0	cyclictest	0	-21	swapper/7	19:31:19	7
1229138	99	34	32,2	cyclictest	1825032	-21	kworker/u16:3+flush-8:0	23:32:01	6
1229138	99	34	32,2	cyclictest	1707402	-21	kworker/u16:4+events_unbound	22:59:02	6
1229138	99	34	32,2	cyclictest	1707402	-21	kworker/u16:4+events_unbound	22:59:02	6
1229138	99	34	32,2	cyclictest	1584865	-21	kworker/u16:2+events_unbound	22:00:10	6
1229138	99	34	32,2	cyclictest	1498630	-21	kworker/u16:1+events_unbound	21:36:06	6
1229138	99	34	32,2	cyclictest	1498630	-21	kworker/u16:1+events_unbound	21:36:06	6
1229138	99	34	32,2	cyclictest	1435886	-21	kworker/u16:4+flush-8:0	21:11:46	6
1229138	99	33	31,2	cyclictest	1837351	-21	kworker/u16:5+flush-8:0	00:26:41	6
1229138	99	33	31,2	cyclictest	1837351	-21	kworker/u16:5+events_unbound	00:19:50	6
1229138	99	33	31,2	cyclictest	1837351	-21	kworker/u16:5+events_unbound	00:19:50	6
1229138	99	33	31,2	cyclictest	1670571	-21	kworker/u16:1+events_unbound	23:39:33	6
1229138	99	33	31,2	cyclictest	1670571	-21	kworker/u16:1+events_unbound	23:22:25	6
1229138	99	33	31,2	cyclictest	1435886	-21	kworker/u16:4+events_unbound	21:45:29	6
1229131	99	33	31,2	cyclictest	1209645	-21	kworker/u16:2+flush-8:0	19:06:20	4
1229142	99	32	30,1	cyclictest	1788612	-21	ssh	23:16:33	7
1229142	99	32	0,31	cyclictest	1935666	-21	latency_hist	00:21:03	7
1229142	99	32	0,31	cyclictest	1935666	-21	latency_hist	00:21:03	7
1229138	99	32	30,2	cyclictest	1884478	-21	kworker/u16:3+events_unbound	00:12:54	6
1229138	99	32	30,2	cyclictest	1884478	-21	kworker/u16:3+events_unbound	00:07:06	6
1229138	99	32	30,2	cyclictest	1585099	-21	kworker/u16:5+events_unbound	21:54:30	6
1229138	99	32	30,2	cyclictest	1498630	-21	kworker/u16:1+flush-8:0	21:46:09	6
1229131	99	32	30,2	cyclictest	1584915	-21	kworker/u16:3+flush-8:0	22:26:31	4
1229138	99	31	29,2	cyclictest	1610363	-21	kworker/u16:0+events_unbound	22:27:30	6
1229131	99	31	29,2	cyclictest	1209645	-21	kworker/u16:2+events_unbound	19:11:19	4
1229138	99	30	28,2	cyclictest	1875905	-21	kworker/u16:0+events_unbound	23:59:42	6
1229138	99	30	28,2	cyclictest	1851524	-21	kworker/u16:2+events_unbound	23:54:18	6
1229138	99	30	28,2	cyclictest	1837351	-21	kworker/u16:5+events_unbound	23:49:09	6
1229138	99	29	27,2	cyclictest	1435885	-21	kworker/u16:3+events_unbound	21:09:18	6
1229131	99	28	26,2	cyclictest	1683349	-21	kworker/u16:2+flush-8:0	22:51:19	4
1229131	99	28	26,2	cyclictest	1300731	-21	kworker/u16:0+flush-8:0	19:41:20	4
1229138	99	27	25,2	cyclictest	1670571	-21	kworker/u16:1+events_unbound	22:48:34	6
1229131	99	27	25,2	cyclictest	1933495	-21	kworker/u16:1+flush-8:0	00:21:34	4
1229131	99	27	25,2	cyclictest	1670571	-21	kworker/u16:1+events_unbound	22:46:29	4
1229131	99	27	25,2	cyclictest	1435886	-21	kworker/u16:4+events_unbound	21:51:17	4
1229142	99	26	25,0	cyclictest	0	-21	swapper/7	19:26:21	7
1229142	99	26	25,0	cyclictest	0	-21	swapper/7	19:26:21	7
1229142	99	26	23,2	cyclictest	1598197	-21	awk	22:01:04	7
1229142	99	26	23,2	cyclictest	1464819	-21	diskmemload	21:06:19	7
1229142	99	26	0,25	cyclictest	1488969	-21	latency_hist	21:16:04	7
1229142	99	26	0,24	cyclictest	1238648	-21	latency_hist	19:11:04	7
1229138	99	26	24,2	cyclictest	534501	-21	kworker/u16:1+flush-8:0	20:01:15	6
1229131	99	26	24,2	cyclictest	1837351	-21	kworker/u16:5+flush-8:0	00:11:32	4
1229138	99	25	23,2	cyclictest	1368332	-21	kworker/u16:2+flush-8:0	20:36:33	6
1229131	99	25	23,2	cyclictest	1670571	-21	kworker/u16:1+events_unbound	23:01:18	4
1229138	99	24	22,2	cyclictest	1511028	-21	kworker/u16:0+events_unbound	21:26:55	6
1229131	99	24	22,2	cyclictest	1435885	-21	kworker/u16:3+events_unbound	21:11:21	4
1229138	99	23	21,2	cyclictest	1209645	-21	kworker/u16:2+events_unbound	19:11:15	6
1229131	99	23	21,2	cyclictest	534501	-21	kworker/u16:1+events_unbound	19:36:26	4
1229131	99	23	20,2	cyclictest	1435886	-21	kworker/u16:4+events_unbound	22:01:17	4
1229142	99	22	18,3	cyclictest	124	-21	kswapd0	22:51:04	7
1229138	99	22	20,2	cyclictest	1300920	-21	kworker/u16:4+flush-8:0	19:51:15	6
1229138	99	22	20,2	cyclictest	1261534	-21	kworker/u16:3+flush-8:0	19:36:30	6
1229138	99	22	20,2	cyclictest	1209645	-21	kworker/u16:2+events_unbound	19:16:06	6
1229131	99	22	20,2	cyclictest	1300920	-21	kworker/u16:4+flush-8:0	20:15:59	4
1229131	99	22	20,2	cyclictest	1300920	-21	kworker/u16:4+flush-8:0	20:15:59	4
1229142	99	21	0,20	cyclictest	1537444	-21	latency_hist	21:36:04	7
1229138	99	21	19,2	cyclictest	534501	-21	kworker/u16:1+events_unbound	19:41:31	6
1229138	99	21	19,2	cyclictest	1300920	-21	kworker/u16:4+events_unbound	20:16:06	6
1229134	99	21	19,2	cyclictest	1945723	-21	kworker/u16:0+flush-8:0	00:31:14	5
1229134	99	21	18,2	cyclictest	1368332	-21	kworker/u16:2+flush-8:0	20:16:30	5
1229131	99	21	19,2	cyclictest	1387505	-21	kworker/u16:0+flush-8:0	20:51:26	4
1229131	99	21	19,2	cyclictest	1368332	-21	kworker/u16:2+flush-8:0	20:26:25	4
1229131	99	21	19,2	cyclictest	1368332	-21	kworker/u16:2+events_unbound	20:41:17	4
1229131	99	21	19,2	cyclictest	1300920	-21	kworker/u16:4+flush-8:0	19:56:27	4
1229131	99	21	19,2	cyclictest	1209645	-21	kworker/u16:2+events_unbound	19:21:33	4
1229131	99	21	18,2	cyclictest	1647119	-21	kworker/u16:4+events_unbound	22:21:22	4
1295194	2	20	0,1	chrt	1295195	-21	kthreadcore	19:36:23	7
1229142	99	20	16,3	cyclictest	1968701	-21	ssh	00:31:28	7
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17