OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Sat May 30, 2026 00:43:34)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
2193174	2	97	0,1	sleep7	2193175	-21	sh	22:08:30	7
1803276	2	52	38,9	sleep4	0	-21	swapper/4	19:03:18	4
2401960	2	47	0,4	sleep7	2401961	-21	tr	23:37:17	7
1803833	99	44	39,4	cyclictest	2074471	-21	latency_hist	21:22:00	7
1803819	99	41	39,2	cyclictest	1826531	-21	kworker/u16:4+events_unbound	19:22:23	4
1803308	2	39	28,7	sleep5	0	-21	swapper/5	19:03:43	5
1795177	2	39	1,31	sleep7	0	-21	swapper/7	19:02:16	7
1803819	99	37	33,3	cyclictest	2327778	-21	kworker/u16:0+flush-8:0	23:12:00	4
1803824	99	32	30,2	cyclictest	1961422	-21	kworker/u16:2+events_unbound	21:02:29	5
1803819	99	31	29,2	cyclictest	1826531	-21	kworker/u16:4+events_unbound	19:27:13	4
1803819	99	30	28,2	cyclictest	2312763	-21	kworker/u16:3+events_unbound	23:27:25	4
1803826	99	29	27,2	cyclictest	1826531	-21	kworker/u16:4+events_unbound	20:47:14	6
1803289	2	29	19,7	sleep6	0	-21	swapper/6	19:03:28	6
1803819	99	28	26,2	cyclictest	1804042	-21	kworker/u16:0+events_unbound	19:11:54	4
2508557	2	27	0,1	chrt	2508555	-21	ssh	00:21:41	7
1803819	99	27	25,2	cyclictest	2061461	-21	kworker/u16:0+events_unbound	21:35:05	4
1803819	99	27	25,2	cyclictest	1826531	-21	kworker/u16:4+events_unbound	20:15:33	4
1803819	99	27	25,2	cyclictest	1804042	-21	kworker/u16:0+events_unbound	19:12:06	4
1803819	99	26	24,2	cyclictest	2327778	-21	kworker/u16:0+flush-8:0	23:39:08	4
1803819	99	26	24,2	cyclictest	2327778	-21	kworker/u16:0+events_unbound	23:18:44	4
1803819	99	26	24,2	cyclictest	1999949	-21	kworker/u16:1+events_unbound	20:59:45	4
1803819	99	26	24,2	cyclictest	1826531	-21	kworker/u16:4+events_unbound	20:30:22	4
1803833	99	25	19,5	cyclictest	1855749	-21	latency_hist	19:32:01	7
1803819	99	25	23,2	cyclictest	2170976	-21	kworker/u16:3+flush-8:0	22:27:22	4
1803819	99	25	23,2	cyclictest	2061461	-21	kworker/u16:0+events_unbound	22:17:53	4
1803819	99	24	22,2	cyclictest	2327778	-21	kworker/u16:0+flush-8:0	23:22:45	4
1803819	99	24	22,2	cyclictest	2327778	-21	kworker/u16:0+events_unbound	00:12:13	4
1803819	99	24	22,2	cyclictest	2327778	-21	kworker/u16:0+events_unbound	00:12:13	4
1803819	99	24	22,2	cyclictest	2061461	-21	kworker/u16:0+flush-8:0	22:16:11	4
1803819	99	24	22,2	cyclictest	2061461	-21	kworker/u16:0+flush-8:0	22:16:11	4
1803819	99	24	22,2	cyclictest	2061461	-21	kworker/u16:0+flush-8:0	22:02:16	4
1803819	99	24	22,2	cyclictest	1793707	-21	kworker/u16:2+flush-8:0	20:07:23	4
1803819	99	24	22,2	cyclictest	1793707	-21	kworker/u16:2+events_unbound	19:41:46	4
1803819	99	23	21,2	cyclictest	2241046	-21	kworker/u16:2+events_unbound	22:40:20	4
1803819	99	23	21,2	cyclictest	1999949	-21	kworker/u16:1+flush-8:0	20:54:11	4
1803819	99	23	21,2	cyclictest	1826531	-21	kworker/u16:4+flush-8:0	21:15:57	4
1803819	99	23	21,2	cyclictest	1826531	-21	kworker/u16:4+events_unbound	19:47:18	4
1803819	99	22	20,2	cyclictest	2436310	-21	kworker/u16:3+events_unbound	23:54:24	4
1803819	99	22	20,2	cyclictest	2241046	-21	kworker/u16:2+events_unbound	22:35:13	4
1803819	99	22	20,2	cyclictest	1904344	-21	kworker/u16:1+flush-8:0	20:24:29	4
1803833	99	21	15,5	cyclictest	2544913	-21	latency_hist	00:37:00	7
1803824	99	21	19,2	cyclictest	2327778	-21	kworker/u16:0+events_unbound	00:00:21	5
1803824	99	21	19,2	cyclictest	2207117	-21	kworker/u16:1+events_unbound	22:35:50	5
1803824	99	21	19,2	cyclictest	2207117	-21	kworker/u16:1+events_unbound	22:24:02	5
1803824	99	21	19,2	cyclictest	2074859	-21	kworker/u16:2+events_unbound	21:35:42	5
1803819	99	21	19,2	cyclictest	2448784	-21	kworker/u16:4+events_unbound	00:03:07	4
1803819	99	21	19,2	cyclictest	2327778	-21	kworker/u16:0+flush-8:0	23:34:52	4
1803819	99	21	19,2	cyclictest	2327778	-21	kworker/u16:0+flush-8:0	23:34:52	4
1803819	99	21	19,2	cyclictest	2241046	-21	kworker/u16:2+flush-8:0	23:49:32	4
1803819	99	21	19,2	cyclictest	2061461	-21	kworker/u16:0+flush-8:0	21:52:23	4
1803819	99	21	19,2	cyclictest	2061461	-21	kworker/u16:0+events_unbound	22:23:17	4
1803819	99	21	19,2	cyclictest	1961422	-21	kworker/u16:2+events_unbound	20:48:46	4
1803819	99	21	19,2	cyclictest	1961422	-21	kworker/u16:2+events_unbound	20:39:37	4
1803819	99	21	19,2	cyclictest	1942309	-21	kworker/u16:0+events_unbound	20:19:41	4
1803819	99	21	19,2	cyclictest	1826531	-21	kworker/u16:4+events_unbound	20:34:30	4
1803819	99	21	18,2	cyclictest	1856131	-21	kworker/u16:0+events_unbound	19:42:04	4
1803824	99	20	18,2	cyclictest	2472860	-21	kworker/u16:3+events_unbound	00:20:58	5
1803824	99	20	18,2	cyclictest	2241046	-21	kworker/u16:2+events_unbound	00:36:21	5
1803824	99	20	18,2	cyclictest	2074859	-21	kworker/u16:2+events_unbound	22:04:55	5
1803824	99	20	18,2	cyclictest	1999949	-21	kworker/u16:1+events_unbound	21:31:59	5
1803824	99	20	18,2	cyclictest	1999949	-21	kworker/u16:1+events_unbound	21:31:59	5
1803819	99	20	18,2	cyclictest	2472860	-21	kworker/u16:3+events_unbound	00:24:47	4
1803819	99	20	18,2	cyclictest	2074859	-21	kworker/u16:2+events_unbound	21:37:53	4
1803819	99	20	18,2	cyclictest	2061461	-21	kworker/u16:0+events_unbound	22:07:34	4
1803819	99	20	18,2	cyclictest	2061461	-21	kworker/u16:0+events_unbound	21:26:47	4
1803819	99	20	18,2	cyclictest	1999949	-21	kworker/u16:1+flush-8:0	21:51:47	4
1803819	99	20	17,2	cyclictest	2448784	-21	kworker/u16:4+flush-8:0	23:57:25	4
1803824	99	19	17,2	cyclictest	2241046	-21	kworker/u16:2+events_unbound	23:51:34	5
1803824	99	19	17,2	cyclictest	2241046	-21	kworker/u16:2+events_unbound	23:19:21	5
1803824	99	19	17,2	cyclictest	2061461	-21	kworker/u16:0+events_unbound	22:18:34	5
1803819	99	19	17,2	cyclictest	2388077	-21	kworker/u16:1+flush-8:0	23:44:45	4
1803819	99	19	17,2	cyclictest	2241046	-21	kworker/u16:2+events_unbound	23:05:56	4
1803819	99	19	17,2	cyclictest	2207117	-21	kworker/u16:1+events_unbound	22:47:41	4
1803819	99	19	17,2	cyclictest	1961422	-21	kworker/u16:2+events_unbound	20:45:21	4
1803819	99	19	17,2	cyclictest	1856131	-21	kworker/u16:0+flush-8:0	19:32:24	4
1803819	99	19	17,2	cyclictest	1825912	-21	kworker/u16:1+events_unbound	19:17:13	4
1803819	99	19	17,2	cyclictest	1825912	-21	kworker/u16:1+events_unbound	19:17:13	4
1803833	99	18	17,0	cyclictest	0	-21	swapper/7	20:22:15	7
1803833	99	18	12,5	cyclictest	2472471	-21	latency_hist	00:06:59	7
1803824	99	18	16,2	cyclictest	2388077	-21	kworker/u16:1+events_unbound	23:44:45	5
1803824	99	18	16,2	cyclictest	2241046	-21	kworker/u16:2+events_unbound	00:14:49	5
1803824	99	18	16,2	cyclictest	2241046	-21	kworker/u16:2+events_unbound	00:14:49	5
1803824	99	18	16,2	cyclictest	1826531	-21	kworker/u16:4+events_unbound	21:16:35	5
1803824	99	18	16,2	cyclictest	1793707	-21	kworker/u16:2+events_unbound	19:33:26	5
1803819	99	18	16,2	cyclictest	2472860	-21	kworker/u16:3+events_unbound	00:08:45	4
1803819	99	18	16,2	cyclictest	2241046	-21	kworker/u16:2+events_unbound	00:27:51	4
1803819	99	18	16,2	cyclictest	1999949	-21	kworker/u16:1+events_unbound	21:04:44	4
1803819	99	18	16,2	cyclictest	1826531	-21	kworker/u16:4+flush-8:0	20:04:21	4
1803819	99	18	16,2	cyclictest	1826531	-21	kworker/u16:4+flush-8:0	20:04:21	4
1803833	99	17	15,1	cyclictest	0	-21	swapper/7	20:47:26	7
1803833	99	17	1,14	cyclictest	2312624	-21	ssh	22:57:22	7
1803833	99	17	0,15	cyclictest	2219154	-21	latency_hist	22:22:00	7
1803824	99	17	15,2	cyclictest	2241046	-21	kworker/u16:2+events_unbound	23:35:30	5
1803824	99	17	15,2	cyclictest	2241046	-21	kworker/u16:2+events_unbound	23:35:30	5
1803824	99	17	15,2	cyclictest	2241046	-21	kworker/u16:2+events_unbound	23:05:22	5
1803824	99	17	15,2	cyclictest	2074859	-21	kworker/u16:2+events_unbound	22:08:09	5
1803824	99	17	15,2	cyclictest	1999949	-21	kworker/u16:1+events_unbound	21:54:51	5
1803824	99	17	15,2	cyclictest	1793707	-21	kworker/u16:2+events_unbound	19:24:54	5
1803819	99	17	15,2	cyclictest	1826531	-21	kworker/u16:4+events_unbound	19:55:13	4
1803819	99	17	14,2	cyclictest	1904344	-21	kworker/u16:1+events_unbound	20:02:00	4
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17