OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Fri Sep 04, 2026 00:43:33)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
1559717	2	103	0,1	sleep7	1559718	-21	ssh	21:49:43	7
1540062	2	66	0,1	sleep7	1540063	-21	kthreadcore	21:39:50	7
1223259	2	54	44,7	sleep5	0	-21	swapper/5	19:08:04	5
1223639	99	45	43,2	cyclictest	1885964	-21	kworker/u16:2+events_unbound	00:04:41	5
1223337	2	44	33,7	sleep6	0	-21	swapper/6	19:09:10	6
1223643	99	41	35,6	cyclictest	1885964	-21	kworker/u16:2+events_unbound	00:07:00	6
1223639	99	41	39,2	cyclictest	1885964	-21	kworker/u16:2+events_unbound	00:19:52	5
1223136	2	40	5,30	sleep7	0	-21	swapper/7	19:06:22	7
1223645	99	39	35,2	cyclictest	1644253	-21	date	22:24:26	7
1223643	99	39	37,2	cyclictest	1558669	-21	kworker/u16:4+flush-8:0	00:10:00	6
1223643	99	39	37,2	cyclictest	1558669	-21	kworker/u16:4+flush-8:0	00:10:00	6
1223643	99	39	36,2	cyclictest	1286045	-21	kworker/u16:2+events_unbound	20:04:44	6
1223639	99	39	37,2	cyclictest	1324621	-21	kworker/u16:0+events_unbound	21:57:24	5
1223643	99	38	35,2	cyclictest	1509627	-21	kworker/u16:3+events_unbound	22:49:27	6
1223639	99	38	36,2	cyclictest	1925146	-21	kworker/u16:4+events_unbound	00:34:25	5
1223639	99	38	36,2	cyclictest	1827341	-21	kworker/u16:1+events_unbound	00:01:04	5
1223639	99	38	36,2	cyclictest	1730104	-21	kworker/u16:1+events_unbound	23:02:01	5
1223643	99	37	35,2	cyclictest	1730104	-21	kworker/u16:1+events_unbound	23:00:33	6
1223643	99	37	35,2	cyclictest	1558669	-21	kworker/u16:4+events_unbound	23:13:01	6
1223643	99	37	35,2	cyclictest	1441324	-21	kworker/u16:1+events_unbound	21:22:17	6
1223643	99	37	35,2	cyclictest	1441324	-21	kworker/u16:1+events_unbound	21:22:17	6
1223639	99	37	35,2	cyclictest	1925146	-21	kworker/u16:4+events_unbound	00:25:57	5
1223639	99	37	35,2	cyclictest	1570924	-21	kworker/u16:5+events_unbound	22:43:45	5
1223639	99	37	35,2	cyclictest	1558669	-21	kworker/u16:4+events_unbound	23:46:56	5
1223639	99	37	35,2	cyclictest	1441324	-21	kworker/u16:1+events_unbound	21:52:34	5
1223639	99	37	35,2	cyclictest	1324621	-21	kworker/u16:0+events_unbound	21:14:10	5
1223639	99	37	34,2	cyclictest	1304174	-21	kworker/u16:4+events_unbound	19:49:29	5
1223285	2	37	26,8	sleep4	0	-21	swapper/4	19:08:28	4
1223643	99	36	34,2	cyclictest	1570924	-21	kworker/u16:5+flush-8:0	22:49:08	6
1223643	99	36	34,2	cyclictest	1570924	-21	kworker/u16:5+flush-8:0	22:49:08	6
1223643	99	36	34,2	cyclictest	1558669	-21	kworker/u16:4+events_unbound	23:17:21	6
1223643	99	36	34,2	cyclictest	1441324	-21	kworker/u16:1+events_unbound	21:50:49	6
1223643	99	36	33,2	cyclictest	1912374	-21	kworker/u16:1+flush-8:0	00:34:41	6
1223639	99	36	34,2	cyclictest	1766661	-21	kworker/u16:0+events_unbound	00:14:25	5
1223639	99	36	34,2	cyclictest	1766661	-21	kworker/u16:0+events_unbound	00:14:25	5
1223639	99	36	34,2	cyclictest	1558669	-21	kworker/u16:4+events_unbound	22:37:20	5
1223639	99	36	34,2	cyclictest	1441324	-21	kworker/u16:1+events_unbound	21:31:40	5
1223639	99	36	34,2	cyclictest	1324621	-21	kworker/u16:0+events_unbound	22:01:32	5
1223643	99	35	33,2	cyclictest	1925146	-21	kworker/u16:4+events_unbound	00:21:25	6
1223643	99	35	33,2	cyclictest	1851822	-21	kworker/u16:3+events_unbound	23:57:17	6
1223643	99	35	33,2	cyclictest	1827341	-21	kworker/u16:1+events_unbound	23:46:23	6
1223643	99	35	33,2	cyclictest	1766661	-21	kworker/u16:0+events_unbound	23:35:32	6
1223643	99	35	33,2	cyclictest	1570924	-21	kworker/u16:5+events_unbound	22:59:01	6
1223643	99	35	33,2	cyclictest	1558669	-21	kworker/u16:4+events_unbound	23:21:16	6
1223643	99	35	33,2	cyclictest	1324621	-21	kworker/u16:0+events_unbound	21:33:58	6
1223639	99	35	33,2	cyclictest	1885964	-21	kworker/u16:2+events_unbound	00:36:32	5
1223639	99	35	33,2	cyclictest	1766661	-21	kworker/u16:0+flush-8:0	23:30:16	5
1223639	99	35	33,2	cyclictest	1570924	-21	kworker/u16:5+events_unbound	22:55:52	5
1223639	99	35	33,2	cyclictest	1570924	-21	kworker/u16:5+events_unbound	22:30:41	5
1223639	99	35	33,2	cyclictest	1558669	-21	kworker/u16:4+events_unbound	22:15:56	5
1223639	99	35	32,2	cyclictest	1412042	-21	kworker/u16:2+events_unbound	20:54:28	5
1223643	99	34	32,2	cyclictest	1925146	-21	kworker/u16:4+events_unbound	00:28:33	6
1223643	99	34	32,2	cyclictest	1690549	-21	kworker/u16:0+events_unbound	22:41:53	6
1223643	99	34	32,2	cyclictest	1570924	-21	kworker/u16:5+events_unbound	22:31:28	6
1223643	99	34	32,2	cyclictest	1558669	-21	kworker/u16:4+events_unbound	22:39:25	6
1223643	99	34	32,2	cyclictest	1509627	-21	kworker/u16:3+events_unbound	22:20:25	6
1223643	99	34	32,2	cyclictest	1449940	-21	kworker/u16:5+events_unbound	21:16:33	6
1223643	99	34	32,2	cyclictest	1324621	-21	kworker/u16:0+events_unbound	22:02:17	6
1223643	99	34	32,2	cyclictest	1324621	-21	kworker/u16:0+events_unbound	21:12:46	6
1223639	99	34	32,2	cyclictest	1690549	-21	kworker/u16:0+events_unbound	22:48:32	5
1223639	99	34	32,2	cyclictest	1690549	-21	kworker/u16:0+events_unbound	22:48:32	5
1223639	99	34	32,2	cyclictest	1558669	-21	kworker/u16:4+events_unbound	23:20:45	5
1223639	99	34	32,2	cyclictest	1558669	-21	kworker/u16:4+events_unbound	22:11:53	5
1223639	99	34	32,2	cyclictest	1509627	-21	kworker/u16:3+events_unbound	23:05:57	5
1223639	99	34	32,2	cyclictest	1509627	-21	kworker/u16:3+events_unbound	22:06:21	5
1223639	99	34	32,2	cyclictest	1509627	-21	kworker/u16:3+events_unbound	22:06:21	5
1223639	99	34	32,2	cyclictest	1449940	-21	kworker/u16:5+events_unbound	21:16:49	5
1223639	99	34	32,2	cyclictest	1324621	-21	kworker/u16:0+events_unbound	21:47:45	5
1223639	99	34	32,2	cyclictest	1324621	-21	kworker/u16:0+events_unbound	21:37:49	5
1223639	99	34	32,2	cyclictest	1324621	-21	kworker/u16:0+events_unbound	21:20:45	5
1223639	99	34	32,2	cyclictest	1324621	-21	kworker/u16:0+events_unbound	21:20:45	5
1223643	99	33	31,2	cyclictest	1827341	-21	kworker/u16:1+flush-8:0	00:00:33	6
1223643	99	33	31,2	cyclictest	1766661	-21	kworker/u16:0+events_unbound	23:27:41	6
1223643	99	33	31,2	cyclictest	1766661	-21	kworker/u16:0+events_unbound	23:27:41	6
1223643	99	33	31,2	cyclictest	1441324	-21	kworker/u16:1+flush-8:0	21:40:16	6
1223643	99	33	31,2	cyclictest	1324621	-21	kworker/u16:0+events_unbound	21:56:29	6
1223643	99	33	31,2	cyclictest	1324621	-21	kworker/u16:0+events_unbound	21:26:17	6
1223639	99	33	31,2	cyclictest	1570924	-21	kworker/u16:5+events_unbound	23:56:17	5
1223639	99	33	31,2	cyclictest	1570924	-21	kworker/u16:5+events_unbound	23:51:24	5
1223639	99	33	31,2	cyclictest	1570924	-21	kworker/u16:5+events_unbound	23:38:32	5
1223639	99	33	31,2	cyclictest	1570924	-21	kworker/u16:5+events_unbound	23:16:29	5
1223639	99	33	31,2	cyclictest	1509627	-21	kworker/u16:3+events_unbound	22:29:09	5
1223639	99	33	31,2	cyclictest	1324621	-21	kworker/u16:0+events_unbound	21:29:17	5
1223643	99	32	30,2	cyclictest	1766661	-21	kworker/u16:0+events_unbound	23:31:33	6
1223643	99	32	30,2	cyclictest	1324621	-21	kworker/u16:0+events_unbound	21:49:25	6
1223643	99	32	30,2	cyclictest	1324621	-21	kworker/u16:0+events_unbound	21:37:00	6
1223639	99	32	30,2	cyclictest	1570924	-21	kworker/u16:5+events_unbound	22:51:57	5
1223639	99	32	30,2	cyclictest	1286045	-21	kworker/u16:2+flush-8:0	20:05:01	5
1223643	99	31	29,2	cyclictest	1570924	-21	kworker/u16:5+events_unbound	22:18:20	6
1223643	99	31	29,2	cyclictest	1558669	-21	kworker/u16:4+events_unbound	22:13:33	6
1223639	99	31	29,2	cyclictest	1912374	-21	kworker/u16:1+events_unbound	00:15:53	5
1223639	99	31	29,2	cyclictest	1558669	-21	kworker/u16:4+events_unbound	23:41:24	5
1223639	99	31	29,2	cyclictest	1558669	-21	kworker/u16:4+events_unbound	23:25:44	5
1223639	99	31	29,2	cyclictest	1558669	-21	kworker/u16:4+events_unbound	23:25:44	5
1223643	99	30	28,2	cyclictest	1851822	-21	kworker/u16:3+flush-8:0	23:50:17	6
1223643	99	30	28,2	cyclictest	1766661	-21	kworker/u16:0+events_unbound	00:17:04	6
1223643	99	30	28,2	cyclictest	1558669	-21	kworker/u16:4+events_unbound	22:07:53	6
1223643	99	30	28,2	cyclictest	1558669	-21	kworker/u16:4+events_unbound	22:07:53	6
1223639	99	30	28,2	cyclictest	3977071	-21	kworker/u16:0+events_unbound	19:36:57	5
1223639	99	30	28,2	cyclictest	1392607	-21	kworker/u16:1+events_unbound	20:42:37	5
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17