OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Sat Aug 22, 2026 00:43:34)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
2251407	2	70	0,1	sleep7	2251405	-21	ssh	00:06:04	7
1582189	2	63	37,7	sleep4	0	-21	swapper/4	19:03:42	4
1582237	2	56	31,7	sleep5	0	-21	swapper/5	19:04:24	5
1573158	2	56	22,29	sleep7	0	-21	swapper/7	19:02:33	7
1582757	99	44	42,2	cyclictest	2032629	-21	kworker/u16:2+events_unbound	00:03:03	6
1582761	99	43	0,41	cyclictest	1879640	-21	latency_hist	21:32:29	7
1582761	99	40	35,4	cyclictest	2179936	-21	latency_hist	23:37:28	7
1582761	99	39	33,5	cyclictest	1927650	-21	latency_hist	21:52:28	7
1582757	99	39	37,2	cyclictest	2117948	-21	kworker/u16:4+events_unbound	23:37:53	6
1582757	99	37	35,2	cyclictest	2081535	-21	kworker/u16:3+events_unbound	22:54:04	6
1582757	99	37	35,2	cyclictest	2047692	-21	kworker/u16:0+events_unbound	23:23:03	6
1582757	99	37	31,6	cyclictest	2047692	-21	kworker/u16:0+events_unbound	23:12:07	6
1582757	99	36	34,2	cyclictest	2203687	-21	kworker/u16:0+events_unbound	00:08:43	6
1582757	99	36	34,2	cyclictest	2167932	-21	kworker/u16:3+events_unbound	23:51:35	6
1582757	99	36	34,2	cyclictest	1703279	-21	kworker/u16:1+flush-8:0	20:12:53	6
1582757	99	36	34,2	cyclictest	1563093	-21	kworker/u16:3+events_unbound	20:02:31	6
1582761	99	35	33,1	cyclictest	2162100	-21	kthreadcore	23:27:51	7
1582427	2	35	25,7	sleep6	0	-21	swapper/6	19:07:05	6
1582757	99	34	32,2	cyclictest	2227177	-21	kworker/u16:1+events_unbound	23:56:11	6
1582757	99	34	32,2	cyclictest	2032629	-21	kworker/u16:2+events_unbound	22:50:19	6
1582757	99	34	32,2	cyclictest	1819504	-21	kworker/u16:0+events_unbound	21:13:51	6
1582757	99	33	31,2	cyclictest	2167932	-21	kworker/u16:3+events_unbound	00:30:07	6
1582757	99	33	31,2	cyclictest	2158084	-21	kworker/u16:1+events_unbound	23:28:55	6
1582757	99	33	31,2	cyclictest	2117948	-21	kworker/u16:4+events_unbound	23:18:16	6
1582757	99	33	31,2	cyclictest	2032629	-21	kworker/u16:2+events_unbound	23:06:40	6
1582757	99	33	31,2	cyclictest	2032629	-21	kworker/u16:2+events_unbound	23:06:40	6
1582757	99	33	31,2	cyclictest	1703279	-21	kworker/u16:1+flush-8:0	20:07:51	6
1582757	99	32	30,2	cyclictest	2034976	-21	kworker/u16:4+events_unbound	22:42:07	6
1582757	99	32	30,2	cyclictest	1831641	-21	kworker/u16:2+events_unbound	21:20:55	6
1582757	99	32	30,2	cyclictest	1819504	-21	kworker/u16:0+events_unbound	21:25:07	6
1582757	99	32	29,2	cyclictest	2158084	-21	kworker/u16:1+flush-8:0	23:32:47	6
1582744	99	32	30,2	cyclictest	1553254	-21	kworker/u16:1+events_unbound	19:12:43	4
1582757	99	31	29,2	cyclictest	2203687	-21	kworker/u16:0+flush-8:0	00:17:27	6
1582757	99	31	29,2	cyclictest	2034976	-21	kworker/u16:4+flush-8:0	22:45:27	6
1582757	99	31	29,2	cyclictest	1903990	-21	kworker/u16:1+events_unbound	21:48:20	6
1582744	99	31	29,2	cyclictest	1703279	-21	kworker/u16:1+flush-8:0	20:22:57	4
1582761	99	30	25,4	cyclictest	2228044	-21	latency_hist	23:57:28	7
1582757	99	30	28,2	cyclictest	2288623	-21	kworker/u16:1+events_unbound	00:26:28	6
1582757	99	30	28,2	cyclictest	2288623	-21	kworker/u16:1+events_unbound	00:26:28	6
1582757	99	30	28,2	cyclictest	1962284	-21	kworker/u16:0+flush-8:0	22:31:23	6
1582757	99	30	28,2	cyclictest	1962284	-21	kworker/u16:0+events_unbound	22:33:15	6
1582757	99	30	28,2	cyclictest	1962284	-21	kworker/u16:0+events_unbound	22:13:19	6
1582761	99	29	24,4	cyclictest	2011762	-21	latency_hist	22:27:28	7
1582757	99	29	27,2	cyclictest	2047692	-21	kworker/u16:0+events_unbound	23:00:31	6
1582757	99	29	27,2	cyclictest	1927603	-21	kworker/u16:4+events_unbound	21:59:07	6
1582757	99	29	27,2	cyclictest	1877928	-21	kworker/u16:2+events_unbound	21:33:31	6
1582753	99	29	27,2	cyclictest	1927603	-21	kworker/u16:4+events_unbound	22:07:43	5
1582757	99	27	25,2	cyclictest	2081535	-21	kworker/u16:3+events_unbound	23:15:22	6
1582757	99	27	25,2	cyclictest	1877928	-21	kworker/u16:2+events_unbound	22:09:11	6
1582757	99	27	25,2	cyclictest	1865642	-21	kworker/u16:3+flush-8:0	21:56:49	6
1582757	99	27	21,6	cyclictest	2167932	-21	kworker/u16:3+events_unbound	00:36:22	6
1582753	99	27	25,2	cyclictest	1731428	-21	kworker/u16:2+flush-8:0	20:27:48	5
1582744	99	27	25,2	cyclictest	1903990	-21	kworker/u16:1+flush-8:0	21:52:23	4
1582757	99	26	24,2	cyclictest	2203687	-21	kworker/u16:0+events_unbound	00:17:54	6
1582757	99	26	24,2	cyclictest	1962284	-21	kworker/u16:0+events_unbound	22:26:48	6
1582753	99	26	24,2	cyclictest	2032629	-21	kworker/u16:2+flush-8:0	22:42:23	5
1582753	99	26	24,2	cyclictest	1819504	-21	kworker/u16:0+events_unbound	21:27:47	5
1582757	99	25	23,2	cyclictest	2158084	-21	kworker/u16:1+events_unbound	23:46:32	6
1582757	99	25	23,2	cyclictest	1962284	-21	kworker/u16:0+flush-8:0	22:05:47	6
1582757	99	25	23,2	cyclictest	1903990	-21	kworker/u16:1+events_unbound	21:46:50	6
1582757	99	25	23,2	cyclictest	1903990	-21	kworker/u16:1+events_unbound	21:46:50	6
1582757	99	25	23,2	cyclictest	1890103	-21	kworker/u16:0+events_unbound	21:38:40	6
1582757	99	25	23,2	cyclictest	1877928	-21	kworker/u16:2+events_unbound	22:21:18	6
1582753	99	25	23,2	cyclictest	1927603	-21	kworker/u16:4+events_unbound	22:12:44	5
1582753	99	25	23,2	cyclictest	1865642	-21	kworker/u16:3+flush-8:0	21:32:56	5
1582753	99	25	23,2	cyclictest	1553254	-21	kworker/u16:1+flush-8:0	19:17:54	5
1582753	99	25	22,2	cyclictest	1553254	-21	kworker/u16:1+events_unbound	19:12:47	5
1582744	99	25	23,2	cyclictest	2167932	-21	kworker/u16:3+flush-8:0	00:21:48	4
1582744	99	25	23,2	cyclictest	2081535	-21	kworker/u16:3+flush-8:0	22:55:57	4
1582744	99	25	23,2	cyclictest	1563093	-21	kworker/u16:3+events_unbound	20:09:47	4
2034120	2	24	0,1	chrt	2034118	-21	ssh	22:34:11	7
1582757	99	24	22,2	cyclictest	2203687	-21	kworker/u16:0+events_unbound	23:58:53	6
1582757	99	24	22,2	cyclictest	1731428	-21	kworker/u16:2+flush-8:0	20:37:28	6
1582757	99	24	22,2	cyclictest	1563093	-21	kworker/u16:3+flush-8:0	19:47:28	6
1582757	99	24	22,2	cyclictest	1534924	-21	kworker/u16:4+events_unbound	19:11:56	6
1582753	99	24	22,2	cyclictest	1927603	-21	kworker/u16:4+flush-8:0	22:02:58	5
1582753	99	24	22,2	cyclictest	1819504	-21	kworker/u16:0+events_unbound	21:12:52	5
1582761	99	23	20,2	cyclictest	125	-21	kswapd0	00:22:29	7
1582761	99	23	20,2	cyclictest	125	-21	kswapd0	00:22:29	7
1582757	99	23	21,2	cyclictest	1731428	-21	kworker/u16:2+events_unbound	20:47:16	6
1582757	99	23	21,2	cyclictest	1703279	-21	kworker/u16:1+events_unbound	20:22:54	6
1582757	99	23	21,2	cyclictest	1693030	-21	kworker/u16:0+events_unbound	20:40:24	6
1582753	99	23	21,2	cyclictest	1927603	-21	kworker/u16:4+events_unbound	22:02:23	5
1582744	99	23	21,2	cyclictest	2081535	-21	kworker/u16:3+events_unbound	23:16:11	4
1582744	99	23	21,2	cyclictest	1927603	-21	kworker/u16:4+events_unbound	22:16:07	4
1582744	99	23	21,2	cyclictest	1675552	-21	kworker/u16:4+events_unbound	19:58:27	4
1582757	99	22	20,2	cyclictest	1741702	-21	kworker/u16:3+events_unbound	21:10:49	6
1582753	99	22	20,2	cyclictest	2203687	-21	kworker/u16:0+flush-8:0	00:23:04	5
1582753	99	22	20,2	cyclictest	2203687	-21	kworker/u16:0+flush-8:0	00:23:04	5
1582753	99	22	20,2	cyclictest	1962284	-21	kworker/u16:0+flush-8:0	22:21:03	5
1582753	99	22	20,2	cyclictest	1741702	-21	kworker/u16:3+events_unbound	20:32:48	5
1582753	99	22	20,2	cyclictest	1703279	-21	kworker/u16:1+events_unbound	20:22:42	5
1582744	99	22	20,2	cyclictest	2203687	-21	kworker/u16:0+flush-8:0	23:53:02	4
1582744	99	22	20,2	cyclictest	2117948	-21	kworker/u16:4+events_unbound	23:42:52	4
1582744	99	22	20,2	cyclictest	1703279	-21	kworker/u16:1+flush-8:0	21:02:43	4
1582744	99	22	20,2	cyclictest	1693030	-21	kworker/u16:0+events_unbound	20:50:12	4
1582744	99	22	20,2	cyclictest	1675552	-21	kworker/u16:4+flush-8:0	20:12:45	4
1582744	99	22	20,2	cyclictest	1635272	-21	kworker/u16:1+events_unbound	19:41:11	4
1582744	99	22	20,2	cyclictest	1635272	-21	kworker/u16:1+events_unbound	19:41:11	4
1631161	2	21	0,1	chrt	1629986	-21	kthreadcore	19:27:49	7
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17