OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Wed Aug 05, 2026 00:43:36)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
995116	2	56	20,31	sleep7	0	-21	swapper/7	19:05:34	7
995322	2	51	40,8	sleep4	0	-21	swapper/4	19:08:28	4
995632	99	45	39,5	cyclictest	1509850	-21	latency_hist	23:03:34	7
995632	99	44	39,4	cyclictest	1377324	-21	latency_hist	22:08:34	7
6324	89	44	0,37	rtkit-daemon	0	-21	swapper/5	19:04:24	5
995628	99	41	39,2	cyclictest	1411812	-21	kworker/u16:3+flush-8:0	22:28:34	6
995314	2	41	31,7	sleep6	0	-21	swapper/6	19:08:22	6
995623	99	40	36,3	cyclictest	1546416	-21	kworker/u16:1+flush-8:0	23:38:34	5
995628	99	38	35,2	cyclictest	976102	-21	kworker/u16:0+events_unbound	20:48:36	6
995632	99	36	31,4	cyclictest	1244282	-21	latency_hist	21:13:34	7
995628	99	34	32,2	cyclictest	1341109	-21	kworker/u16:0+events_unbound	22:14:08	6
995628	99	34	31,2	cyclictest	1212883	-21	kworker/u16:1+events_unbound	21:13:37	6
995632	99	33	32,0	cyclictest	0	-21	swapper/7	00:30:19	7
995632	99	33	31,1	cyclictest	1497568	-21	ssh	22:58:08	7
995628	99	33	31,2	cyclictest	1642759	-21	kworker/u16:3+events_unbound	00:36:44	6
995623	99	32	29,2	cyclictest	1115500	-21	kworker/u16:1+events_unbound	20:18:52	5
995623	99	32	28,3	cyclictest	1304869	-21	kworker/u16:1+events_unbound	21:54:00	5
995618	99	32	29,3	cyclictest	1341109	-21	kworker/u16:0+flush-8:0	22:14:16	4
995623	99	31	28,2	cyclictest	1437558	-21	kworker/u16:1+flush-8:0	22:58:52	5
995632	99	30	29,0	cyclictest	0	-21	swapper/7	23:16:19	7
995632	99	30	29,0	cyclictest	0	-21	swapper/7	23:04:19	7
995632	99	30	29,0	cyclictest	0	-21	swapper/7	22:41:24	7
995632	99	30	25,4	cyclictest	1125467	-21	latency_hist	20:13:34	7
995632	99	30	25,4	cyclictest	1125467	-21	latency_hist	20:13:34	7
995628	99	30	28,2	cyclictest	1618674	-21	kworker/u16:2+events_unbound	23:53:23	6
995628	99	30	28,2	cyclictest	1389482	-21	kworker/u16:1+flush-8:0	22:19:12	6
995628	99	30	23,7	cyclictest	1546416	-21	kworker/u16:1+events_unbound	23:42:40	6
995623	99	30	27,2	cyclictest	1582146	-21	kworker/u16:0+events_unbound	23:43:48	5
995632	99	29	27,1	cyclictest	1076053	-21	/usr/sbin/munin	19:44:01	7
995628	99	29	27,2	cyclictest	1642759	-21	kworker/u16:3+events_unbound	00:12:44	6
995628	99	29	27,2	cyclictest	1582146	-21	kworker/u16:0+flush-8:0	23:35:44	6
995628	99	29	27,2	cyclictest	1498219	-21	kworker/u16:2+events_unbound	23:31:45	6
995628	99	29	27,2	cyclictest	1437558	-21	kworker/u16:1+events_unbound	22:54:24	6
995628	99	29	27,2	cyclictest	1341109	-21	kworker/u16:0+events_unbound	22:02:32	6
995632	99	28	27,0	cyclictest	0	-21	swapper/7	23:47:39	7
995632	99	28	27,0	cyclictest	0	-21	swapper/7	21:55:11	7
995628	99	28	26,2	cyclictest	1546416	-21	kworker/u16:1+events_unbound	23:53:40	6
995628	99	28	26,2	cyclictest	1546416	-21	kworker/u16:1+events_unbound	00:29:52	6
995628	99	28	26,2	cyclictest	1317361	-21	kworker/u16:2+events_unbound	22:35:07	6
995628	99	28	26,2	cyclictest	1304869	-21	kworker/u16:1+events_unbound	21:55:56	6
995628	99	28	26,2	cyclictest	1304869	-21	kworker/u16:1+events_unbound	21:40:45	6
995623	99	28	26,2	cyclictest	976102	-21	kworker/u16:0+events_unbound	21:39:05	5
995623	99	28	26,2	cyclictest	911388	-21	kworker/u16:1+events_unbound	19:24:05	5
995623	99	28	26,2	cyclictest	1037144	-21	kworker/u16:2+events_unbound	19:49:00	5
995632	99	27	26,0	cyclictest	0	-21	swapper/7	23:26:20	7
995632	99	27	26,0	cyclictest	0	-21	swapper/7	22:02:11	7
995632	99	27	25,1	cyclictest	1232271	-21	diskmemload	21:28:07	7
995628	99	27	25,2	cyclictest	1226754	-21	kworker/u16:2+events_unbound	21:29:43	6
995628	99	27	25,2	cyclictest	1212883	-21	kworker/u16:1+events_unbound	21:10:45	6
995623	99	27	25,2	cyclictest	1546416	-21	kworker/u16:1+flush-8:0	23:58:56	5
995632	99	26	25,0	cyclictest	0	-21	swapper/7	21:41:20	7
995632	99	26	25,0	cyclictest	0	-21	swapper/7	00:38:14	7
995632	99	26	22,2	cyclictest	1412994	-21	ssh	22:22:46	7
995628	99	26	24,2	cyclictest	1498219	-21	kworker/u16:2+events_unbound	23:06:01	6
995623	99	26	24,2	cyclictest	1461664	-21	kworker/u16:0+flush-8:0	22:48:50	5
995623	99	26	24,2	cyclictest	1341109	-21	kworker/u16:0+events_unbound	22:08:48	5
995623	99	26	24,2	cyclictest	1212883	-21	kworker/u16:1+flush-8:0	21:09:25	5
995623	99	26	23,2	cyclictest	1115500	-21	kworker/u16:1+events_unbound	20:28:48	5
1447806	2	26	0,1	chrt	1447807	-21	sshd-session	22:35:04	7
995632	99	25	23,1	cyclictest	0	-21	swapper/7	00:21:22	7
995632	99	25	23,1	cyclictest	0	-21	swapper/7	00:21:22	7
995628	99	25	23,2	cyclictest	1701244	-21	kworker/u16:4+events_unbound	00:22:44	6
995628	99	25	23,2	cyclictest	1701244	-21	kworker/u16:4+events_unbound	00:22:44	6
995628	99	25	23,2	cyclictest	1546416	-21	kworker/u16:1+events_unbound	00:24:16	6
995628	99	25	23,2	cyclictest	1546416	-21	kworker/u16:1+events_unbound	00:24:16	6
995628	99	25	23,2	cyclictest	1317361	-21	kworker/u16:2+flush-8:0	22:30:12	6
995628	99	25	23,2	cyclictest	1317361	-21	kworker/u16:2+flush-8:0	22:04:56	6
995632	99	24	23,0	cyclictest	0	-21	swapper/7	22:10:10	7
995632	99	24	23,0	cyclictest	0	-21	swapper/7	21:50:46	7
995632	99	24	22,1	cyclictest	1232271	-21	diskmemload	23:20:22	7
995632	99	24	22,1	cyclictest	0	-21	swapper/7	00:02:00	7
995632	99	24	21,2	cyclictest	1423487	-21	ssh	22:24:43	7
995628	99	24	22,2	cyclictest	1642759	-21	kworker/u16:3+events_unbound	00:15:45	6
995628	99	24	22,2	cyclictest	1437558	-21	kworker/u16:1+events_unbound	22:39:31	6
995628	99	24	22,2	cyclictest	1411812	-21	kworker/u16:3+flush-8:0	23:17:07	6
995628	99	24	22,2	cyclictest	1226754	-21	kworker/u16:2+flush-8:0	21:18:59	6
995632	99	23	22,0	cyclictest	0	-21	swapper/7	22:51:59	7
995632	99	23	22,0	cyclictest	0	-21	swapper/7	22:45:05	7
995632	99	23	22,0	cyclictest	0	-21	swapper/7	21:36:26	7
995632	99	23	22,0	cyclictest	0	-21	swapper/7	21:03:37	7
995632	99	23	22,0	cyclictest	0	-21	swapper/7	20:33:33	7
995632	99	23	22,0	cyclictest	0	-21	swapper/7	19:32:41	7
995632	99	23	22,0	cyclictest	0	-21	swapper/7	19:21:09	7
995632	99	23	21,1	cyclictest	0	-21	swapper/7	23:56:33	7
995632	99	23	21,1	cyclictest	0	-21	swapper/7	21:16:15	7
995628	99	23	21,2	cyclictest	976102	-21	kworker/u16:0+events_unbound	19:19:08	6
995628	99	23	21,2	cyclictest	1546416	-21	kworker/u16:1+flush-8:0	23:20:12	6
995628	99	23	21,2	cyclictest	1546416	-21	kworker/u16:1+events_unbound	23:44:59	6
995623	99	23	21,2	cyclictest	1618674	-21	kworker/u16:2+events_unbound	23:53:58	5
995623	99	23	21,2	cyclictest	1411812	-21	kworker/u16:3+events_unbound	22:33:47	5
1257402	2	23	0,1	sleep5	0	-21	swapper/5	21:18:48	5
995632	99	22	21,0	cyclictest	0	-21	swapper/7	23:42:05	7
995632	99	22	21,0	cyclictest	0	-21	swapper/7	23:32:20	7
995632	99	22	21,0	cyclictest	0	-21	swapper/7	00:17:02	7
995632	99	22	20,1	cyclictest	1399742	-21	ssh	22:15:21	7
995628	99	22	20,2	cyclictest	976102	-21	kworker/u16:0+events_unbound	21:28:01	6
995623	99	22	19,2	cyclictest	1317361	-21	kworker/u16:2+flush-8:0	22:18:34	5
995632	99	21	20,0	cyclictest	0	-21	swapper/7	23:37:53	7
995632	99	21	20,0	cyclictest	0	-21	swapper/7	00:05:46	7
995628	99	21	19,2	cyclictest	1618674	-21	kworker/u16:2+events_unbound	00:01:08	6
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17