OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Mon Feb 23, 2026 00:43:33)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
2966056	2	48	36,8	sleep4	0	-21	swapper/4	19:02:48	4
2966086	2	41	6,31	sleep7	0	-21	swapper/7	19:03:16	7
2966094	2	39	29,7	sleep6	0	-21	swapper/6	19:03:23	6
2966505	99	37	31,5	cyclictest	650	-21	systemd-journal	00:35:07	7
2966505	99	37	31,5	cyclictest	650	-21	systemd-journal	00:35:07	7
2966505	99	37	0,36	cyclictest	3477220	-21	latency_hist	23:00:08	7
2966496	99	37	34,2	cyclictest	2232101	-21	kworker/u16:2+events_unbound	20:50:33	5
2966496	99	36	32,3	cyclictest	2966709	-21	kworker/u16:1+flush-8:0	19:25:09	5
2966505	99	35	33,1	cyclictest	3309382	-21	ssh	21:50:23	7
2966505	99	34	28,5	cyclictest	650	-21	systemd-journal	23:20:07	7
2966501	99	33	31,2	cyclictest	3212089	-21	kworker/u16:0+flush-8:0	22:05:30	6
2966496	99	32	29,2	cyclictest	3511701	-21	kworker/u16:1+events_unbound	23:20:21	5
2966496	99	32	28,3	cyclictest	2893881	-21	kworker/u16:3+events_unbound	21:05:22	5
2966505	99	30	27,2	cyclictest	3538482	-21	ssh	23:25:22	7
2965931	2	30	4,8	sleep5	0	-21	swapper/5	19:01:05	5
2966501	99	28	25,2	cyclictest	3560671	-21	kworker/u16:3+events_unbound	23:35:22	6
2966501	99	28	25,2	cyclictest	3212089	-21	kworker/u16:0+flush-8:0	21:55:34	6
2966496	99	27	25,2	cyclictest	3560671	-21	kworker/u16:3+events_unbound	00:15:20	5
2966496	99	27	25,2	cyclictest	3187816	-21	kworker/u16:1+flush-8:0	21:10:45	5
2966496	99	27	25,2	cyclictest	3187816	-21	kworker/u16:1+flush-8:0	21:10:45	5
2966492	99	27	24,2	cyclictest	3402428	-21	kworker/u16:6+flush-8:0	22:40:33	4
2966501	99	25	22,2	cyclictest	2966709	-21	kworker/u16:1+flush-8:0	19:30:09	6
2966496	99	25	22,2	cyclictest	2232101	-21	kworker/u16:2+events_unbound	20:25:22	5
2966505	99	24	23,0	cyclictest	0	-21	swapper/7	22:20:23	7
2966505	99	24	19,4	cyclictest	3621697	-21	latency_hist	00:00:08	7
2966492	99	24	18,5	cyclictest	3296590	-21	kworker/u16:4+flush-8:0	22:15:18	4
2966501	99	23	21,2	cyclictest	3535829	-21	kworker/u16:0+flush-8:0	23:35:08	6
2966501	99	23	21,2	cyclictest	3535829	-21	kworker/u16:0+events_unbound	23:40:30	6
2966505	99	22	17,4	cyclictest	3561495	-21	latency_hist	23:35:08	7
2966501	99	22	18,3	cyclictest	3393194	-21	kworker/u16:2+flush-8:0	22:35:25	6
2966492	99	21	17,3	cyclictest	3646133	-21	kworker/u16:1+flush-8:0	00:25:35	4
3209775	2	20	0,1	chrt	3209772	-21	ssh	21:06:38	7
2966505	99	20	9,9	cyclictest	0	-21	swapper/7	21:45:29	7
2966496	99	20	19,1	cyclictest	3393194	-21	kworker/u16:2+events_unbound	22:38:43	5
2966496	99	20	17,2	cyclictest	3319286	-21	kworker/u16:5+flush-8:0	22:10:33	5
2966496	99	20	17,2	cyclictest	2893881	-21	kworker/u16:3+events_unbound	20:30:23	5
2966496	99	20	17,2	cyclictest	2893881	-21	kworker/u16:3+events_unbound	20:30:23	5
2966505	99	19	18,0	cyclictest	0	-21	swapper/7	22:35:26	7
2966505	99	19	13,5	cyclictest	3180528	-21	latency_hist	20:55:09	7
2966505	99	19	0,18	cyclictest	5747	-21	gnome-shell	20:35:18	7
2966501	99	19	17,2	cyclictest	2893881	-21	kworker/u16:3+flush-8:0	19:20:24	6
2966496	99	19	17,2	cyclictest	3296590	-21	kworker/u16:4+events_unbound	22:45:32	5
2966505	99	18	17,0	cyclictest	0	-21	swapper/7	22:10:54	7
2966505	99	18	12,5	cyclictest	3296197	-21	latency_hist	21:45:08	7
2966505	99	18	0,17	cyclictest	1267	-21	snmpd	23:46:40	7
2966505	99	18	0,17	cyclictest	0	-21	swapper/7	19:40:31	7
2966496	99	18	15,2	cyclictest	3027739	-21	kworker/u16:0+flush-8:0	19:50:34	5
3354267	2	17	0,0	sleep7	3354268	-21	rm	22:05:52	7
2966505	99	17	16,0	cyclictest	0	-21	swapper/7	23:10:38	7
2966505	99	17	16,0	cyclictest	0	-21	swapper/7	22:30:23	7
2966505	99	17	16,0	cyclictest	0	-21	swapper/7	20:10:33	7
2966505	99	17	11,5	cyclictest	3609670	-21	latency_hist	23:55:07	7
2966505	99	17	11,5	cyclictest	3235783	-21	latency_hist	21:20:08	7
2966501	99	17	14,2	cyclictest	3212089	-21	kworker/u16:0+events_unbound	22:00:21	6
2966501	99	17	13,3	cyclictest	3548105	-21	kworker/u16:2+flush-8:0	23:50:19	6
2966496	99	17	15,2	cyclictest	3560671	-21	kworker/u16:3+flush-8:0	00:05:28	5
2966496	99	17	14,2	cyclictest	3560671	-21	kworker/u16:3+events_unbound	00:30:26	5
2966496	99	17	14,2	cyclictest	3560671	-21	kworker/u16:3+events_unbound	00:30:26	5
2966496	99	17	14,2	cyclictest	3296590	-21	kworker/u16:4+flush-8:0	21:55:13	5
2966505	99	16	8,7	cyclictest	0	-21	swapper/7	22:15:26	7
2966505	99	16	7,7	cyclictest	0	-21	swapper/7	00:15:21	7
2966505	99	16	3,11	cyclictest	0	-21	swapper/7	20:05:32	7
2966505	99	16	15,0	cyclictest	0	-21	swapper/7	19:52:52	7
2966505	99	16	15,0	cyclictest	0	-21	swapper/7	19:25:05	7
2966505	99	16	13,2	cyclictest	0	-21	swapper/7	19:50:08	7
2966505	99	16	12,3	cyclictest	124	-21	kswapd0	22:50:21	7
2966501	99	16	14,2	cyclictest	3610147	-21	kworker/u16:4+flush-8:0	00:00:25	6
2966501	99	16	14,2	cyclictest	3187816	-21	kworker/u16:1+flush-8:0	21:35:03	6
2966501	99	16	14,2	cyclictest	2232101	-21	kworker/u16:2+events_unbound	21:10:33	6
2966501	99	16	14,2	cyclictest	2232101	-21	kworker/u16:2+events_unbound	21:10:33	6
2966505	99	15	6,8	cyclictest	0	-21	swapper/7	23:20:28	7
2966505	99	15	6,7	cyclictest	0	-21	swapper/7	23:35:32	7
2966505	99	15	6,7	cyclictest	0	-21	swapper/7	22:25:32	7
2966501	99	15	13,2	cyclictest	2232101	-21	kworker/u16:2+flush-8:0	21:35:34	6
2966501	99	15	13,2	cyclictest	2232101	-21	kworker/u16:2+events_unbound	20:15:24	6
2966496	99	15	13,2	cyclictest	3560671	-21	kworker/u16:3+events_unbound	00:25:31	5
2966492	99	15	13,1	cyclictest	0	-21	swapper/4	22:30:27	4
2966492	99	15	13,1	cyclictest	0	-21	swapper/4	19:35:27	4
2966492	99	15	0,1	cyclictest	0	-21	swapper/4	00:14:58	4
2966505	99	14	8,5	cyclictest	3170981	-21	latency_hist	20:50:08	7
2966505	99	14	7,6	cyclictest	0	-21	swapper/7	00:00:19	7
2966505	99	14	6,7	cyclictest	0	-21	swapper/7	21:34:51	7
2966505	99	14	6,7	cyclictest	0	-21	swapper/7	21:10:32	7
2966505	99	14	6,7	cyclictest	0	-21	swapper/7	21:10:32	7
2966505	99	14	5,8	cyclictest	1060	-21	runrttasks	21:57:53	7
2966505	99	14	5,8	cyclictest	0	-21	swapper/7	23:40:32	7
2966505	99	14	5,8	cyclictest	0	-21	swapper/7	23:08:09	7
2966505	99	14	5,7	cyclictest	0	-21	swapper/7	19:38:38	7
2966505	99	14	3,9	cyclictest	0	-21	swapper/7	21:35:33	7
2966505	99	14	14,0	cyclictest	0	-21	swapper/7	23:03:30	7
2966505	99	14	0,13	cyclictest	0	-21	swapper/7	20:55:31	7
2966501	99	14	13,1	cyclictest	0	-21	swapper/6	23:47:28	6
2966501	99	14	13,1	cyclictest	0	-21	swapper/6	21:42:27	6
2966501	99	14	0,1	cyclictest	0	-21	swapper/6	23:09:47	6
2966501	99	14	0,1	cyclictest	0	-21	swapper/6	22:18:33	6
2966501	99	14	0,1	cyclictest	0	-21	swapper/6	22:10:31	6
2966501	99	14	0,1	cyclictest	0	-21	swapper/6	00:10:39	6
2966501	99	14	0,14	cyclictest	0	-21	swapper/6	23:30:08	6
2966501	99	14	0,14	cyclictest	0	-21	swapper/6	23:14:50	6
2966496	99	14	13,1	cyclictest	0	-21	swapper/5	22:41:22	5
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17