OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Wed Nov 19, 2025 12:43:34)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
1266478	2	70	59,7	sleep6	0	-21	swapper/6	15:22:39	6
1266912	99	60	54,5	cyclictest	1553713	-21	latency_hist	17:43:42	7
1266912	99	59	54,4	cyclictest	1402005	-21	latency_hist	16:28:43	7
1266912	99	59	54,4	cyclictest	1402005	-21	latency_hist	16:28:43	7
1266504	2	58	48,7	sleep5	0	-21	swapper/5	15:23:02	5
1266324	2	58	48,7	sleep4	0	-21	swapper/4	15:20:36	4
1266905	99	53	50,2	cyclictest	1695085	-21	kworker/u16:2+events_unbound	19:18:42	5
1266901	99	51	49,2	cyclictest	1432129	-21	kworker/u16:3+flush-8:0	16:55:21	4
1266912	99	50	44,5	cyclictest	589235	-21	systemd-journal	18:38:42	7
1266901	99	50	48,2	cyclictest	1806375	-21	kworker/u16:0+flush-8:0	19:49:12	4
1266901	99	48	45,2	cyclictest	1406456	-21	kworker/u16:1+flush-8:0	16:34:21	4
1266901	99	48	45,2	cyclictest	1381630	-21	kworker/u16:3+events_unbound	16:28:21	4
1266901	99	48	45,2	cyclictest	1381630	-21	kworker/u16:3+events_unbound	16:28:21	4
1266901	99	47	45,2	cyclictest	1847271	-21	kworker/u16:2+events_unbound	20:26:21	4
1266901	99	47	44,2	cyclictest	1816541	-21	kworker/u16:1+events_unbound	19:53:45	4
1266901	99	46	44,2	cyclictest	1858372	-21	kworker/u16:1+events_unbound	20:16:21	4
1266901	99	46	44,2	cyclictest	1542855	-21	kworker/u16:4+events_unbound	17:44:22	4
1266901	99	46	44,2	cyclictest	1542855	-21	kworker/u16:4+events_unbound	17:42:21	4
1266901	99	46	44,2	cyclictest	1442338	-21	kworker/u16:0+events_unbound	18:13:20	4
1266901	99	45	43,2	cyclictest	1675242	-21	kworker/u16:0+events_unbound	18:56:21	4
1266901	99	45	43,2	cyclictest	1236043	-21	kworker/u16:4+flush-8:0	15:40:21	4
1266901	99	45	43,2	cyclictest	1236043	-21	kworker/u16:4+flush-8:0	15:40:21	4
1266905	99	44	40,3	cyclictest	1246153	-21	kworker/u16:0+flush-8:0	15:44:12	5
1266901	99	44	42,2	cyclictest	1406456	-21	kworker/u16:1+flush-8:0	17:23:20	4
1266901	99	44	41,2	cyclictest	1725412	-21	kworker/u16:1+events_unbound	19:28:44	4
1266901	99	44	41,2	cyclictest	1236043	-21	kworker/u16:4+flush-8:0	15:48:45	4
1266901	99	42	40,2	cyclictest	1846862	-21	kworker/u16:0+events_unbound	20:11:20	4
1266901	99	42	40,2	cyclictest	1846862	-21	kworker/u16:0+events_unbound	20:11:20	4
1266901	99	42	40,2	cyclictest	1725412	-21	kworker/u16:1+events_unbound	19:44:20	4
1266901	99	42	40,2	cyclictest	1675242	-21	kworker/u16:0+events_unbound	18:51:22	4
1266901	99	42	40,2	cyclictest	1503112	-21	kworker/u16:2+flush-8:0	17:29:16	4
1266901	99	42	39,2	cyclictest	1442338	-21	kworker/u16:0+events_unbound	18:15:21	4
1266901	99	40	38,2	cyclictest	1897606	-21	kworker/u16:4+flush-8:0	20:34:20	4
1266901	99	40	38,2	cyclictest	1786257	-21	kworker/u16:4+flush-8:0	19:43:20	4
1266901	99	40	38,2	cyclictest	1725412	-21	kworker/u16:1+events_unbound	19:14:45	4
1266901	99	40	38,2	cyclictest	1665007	-21	kworker/u16:1+flush-8:0	18:39:20	4
1266901	99	40	38,2	cyclictest	1665007	-21	kworker/u16:1+flush-8:0	18:39:20	4
1266901	99	39	37,2	cyclictest	1847271	-21	kworker/u16:2+flush-8:0	20:21:20	4
1266901	99	39	37,2	cyclictest	1593897	-21	kworker/u16:1+events_unbound	18:26:21	4
1266901	99	39	37,2	cyclictest	1452395	-21	kworker/u16:4+events_unbound	17:15:21	4
1266901	99	39	36,2	cyclictest	1267106	-21	kworker/u16:1+flush-8:0	15:44:09	4
1265640	2	39	1,32	sleep7	0	-21	swapper/7	15:19:19	7
1266905	99	38	34,3	cyclictest	1432129	-21	kworker/u16:3+flush-8:0	16:48:43	5
1266901	99	38	36,2	cyclictest	1351249	-21	kworker/u16:0+events_unbound	16:32:21	4
1266901	99	38	36,2	cyclictest	1341081	-21	kworker/u16:2+events_unbound	16:42:21	4
1266901	99	38	35,2	cyclictest	1746042	-21	kworker/u16:3+events_unbound	19:22:21	4
1266912	99	37	32,4	cyclictest	1341332	-21	latency_hist	15:58:43	7
1266901	99	37	35,2	cyclictest	1897606	-21	kworker/u16:4+flush-8:0	20:44:28	4
1813761	2	36	0,0	sleep7	1813760	-21	grep	19:49:05	7
1266901	99	36	34,2	cyclictest	1267106	-21	kworker/u16:1+events_unbound	16:00:22	4
1266901	99	36	33,2	cyclictest	1654721	-21	kworker/u16:3+events_unbound	19:08:44	4
1266901	99	35	33,2	cyclictest	1406456	-21	kworker/u16:1+flush-8:0	17:10:53	4
1266901	99	35	33,2	cyclictest	1406456	-21	kworker/u16:1+flush-8:0	17:10:53	4
1266901	99	35	33,2	cyclictest	1351249	-21	kworker/u16:0+events_unbound	16:17:30	4
1266912	99	34	28,5	cyclictest	1695407	-21	latency_hist	18:53:43	7
1266905	99	34	30,3	cyclictest	1907762	-21	kworker/u16:2+flush-8:0	20:38:57	5
1266901	99	34	31,2	cyclictest	1886702	-21	kworker/u16:3+flush-8:0	20:28:52	4
1266901	99	34	31,2	cyclictest	1442338	-21	kworker/u16:0+events_unbound	17:23:53	4
1266912	99	33	27,4	cyclictest	1432352	-21	latency_hist	16:43:43	7
1266905	99	33	30,2	cyclictest	1634529	-21	kworker/u16:4+flush-8:0	19:10:53	5
1266905	99	33	29,3	cyclictest	1341081	-21	kworker/u16:2+flush-8:0	15:59:53	5
1266901	99	33	31,2	cyclictest	1826764	-21	kworker/u16:4+events_unbound	20:05:52	4
1266901	99	33	30,2	cyclictest	1442338	-21	kworker/u16:0+events_unbound	18:03:45	4
1266912	99	32	31,0	cyclictest	0	-21	swapper/7	17:14:18	7
1266905	99	32	30,2	cyclictest	1614202	-21	kworker/u16:3+events_unbound	18:19:07	5
1266901	99	32	30,2	cyclictest	1695085	-21	kworker/u16:2+events_unbound	19:05:21	4
1266901	99	32	30,2	cyclictest	1503112	-21	kworker/u16:2+events_unbound	18:19:54	4
1266901	99	32	30,2	cyclictest	1351249	-21	kworker/u16:0+flush-8:0	16:19:20	4
1266901	99	31	29,2	cyclictest	1442338	-21	kworker/u16:0+events_unbound	17:53:20	4
1266905	99	30	26,3	cyclictest	1442338	-21	kworker/u16:0+flush-8:0	17:13:13	5
1266905	99	30	26,3	cyclictest	1442338	-21	kworker/u16:0+flush-8:0	17:13:13	5
1266901	99	30	28,2	cyclictest	1766123	-21	kworker/u16:0+flush-8:0	19:35:30	4
1266901	99	30	28,2	cyclictest	1281080	-21	kworker/u16:3+events_unbound	16:06:34	4
1266912	99	29	0,27	cyclictest	589235	-21	systemd-journal	18:47:20	7
1266901	99	29	27,2	cyclictest	1928095	-21	kworker/u16:1+flush-8:0	20:53:29	4
1266901	99	29	27,2	cyclictest	1442338	-21	kworker/u16:0+flush-8:0	17:37:45	4
1266901	99	29	27,2	cyclictest	1432129	-21	kworker/u16:3+events_unbound	16:53:22	4
1266901	99	29	27,2	cyclictest	1281080	-21	kworker/u16:3+flush-8:0	15:28:53	4
1266912	99	28	27,0	cyclictest	0	-21	swapper/7	20:51:52	7
1266912	99	28	26,1	cyclictest	0	-21	swapper/7	20:19:01	7
1266905	99	28	26,2	cyclictest	1897606	-21	kworker/u16:4+flush-8:0	20:48:57	5
1266905	99	28	25,2	cyclictest	1886702	-21	kworker/u16:3+events_unbound	20:28:57	5
1266905	99	27	25,2	cyclictest	1341081	-21	kworker/u16:2+flush-8:0	16:59:01	5
1266905	99	27	24,2	cyclictest	1776718	-21	kworker/u16:2+flush-8:0	19:34:03	5
1266905	99	27	23,3	cyclictest	1391777	-21	kworker/u16:4+flush-8:0	16:29:04	5
1266901	99	27	25,2	cyclictest	1432129	-21	kworker/u16:3+events_unbound	17:06:29	4
1266901	99	27	24,2	cyclictest	1654721	-21	kworker/u16:3+events_unbound	18:58:52	4
1266905	99	26	24,2	cyclictest	1897606	-21	kworker/u16:4+flush-8:0	20:34:05	5
1266905	99	26	24,2	cyclictest	1826764	-21	kworker/u16:4+flush-8:0	19:58:38	5
1266901	99	26	24,2	cyclictest	1675242	-21	kworker/u16:0+events_unbound	18:44:20	4
1266901	99	26	24,2	cyclictest	1236043	-21	kworker/u16:4+flush-8:0	15:37:09	4
1266912	99	25	24,0	cyclictest	0	-21	swapper/7	17:59:05	7
1266912	99	25	24,0	cyclictest	0	-21	swapper/7	16:09:03	7
1266905	99	25	22,2	cyclictest	1654721	-21	kworker/u16:3+flush-8:0	18:34:04	5
1266901	99	25	22,2	cyclictest	1695085	-21	kworker/u16:2+events_unbound	19:23:43	4
1266901	99	25	22,2	cyclictest	1695085	-21	kworker/u16:2+events_unbound	19:23:43	4
1266901	99	25	22,2	cyclictest	1644769	-21	kworker/u16:2+flush-8:0	18:29:08	4
1266912	99	24	18,5	cyclictest	1836940	-21	latency_hist	20:03:42	7
1266905	99	24	22,2	cyclictest	1634529	-21	kworker/u16:4+events_unbound	19:04:06	5
1266905	99	24	22,2	cyclictest	1542855	-21	kworker/u16:4+events_unbound	17:58:38	5
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17