OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Tue Aug 11, 2026 00:43:34)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
3944451	2	53	18,30	sleep7	0	-21	swapper/7	19:06:07	7
3944451	2	53	18,30	sleep7	0	-21	swapper/7	19:06:07	7
3944330	2	53	43,7	sleep4	0	-21	swapper/4	19:04:27	4
3944330	2	53	43,7	sleep4	0	-21	swapper/4	19:04:27	4
3944349	2	49	39,7	sleep5	0	-21	swapper/5	19:04:44	5
3944349	2	49	39,7	sleep5	0	-21	swapper/5	19:04:44	5
3944882	99	48	44,3	cyclictest	376480	-21	kworker/u16:1+flush-8:0	23:53:10	5
3944882	99	39	35,4	cyclictest	72894	-21	kworker/u16:1+flush-8:0	21:58:11	5
3944875	99	39	37,2	cyclictest	241304	-21	kworker/u16:1+events_unbound	22:50:05	4
3944400	2	39	29,7	sleep6	0	-21	swapper/6	19:05:25	6
3944400	2	39	29,7	sleep6	0	-21	swapper/6	19:05:25	6
3944875	99	35	33,2	cyclictest	4055218	-21	kworker/u16:3+events_unbound	20:03:35	4
3944875	99	35	33,2	cyclictest	3881316	-21	kworker/u16:1+flush-8:0	19:28:40	4
3944875	99	35	33,2	cyclictest	3881316	-21	kworker/u16:1+flush-8:0	19:28:40	4
3944875	99	35	33,2	cyclictest	361715	-21	kworker/u16:0+events_unbound	23:53:12	4
3944875	99	34	32,2	cyclictest	426907	-21	kworker/u16:1+flush-8:0	00:28:04	4
3944875	99	34	32,2	cyclictest	36954	-21	kworker/u16:3+events_unbound	22:09:08	4
3944875	99	34	32,2	cyclictest	253795	-21	kworker/u16:0+events_unbound	23:00:05	4
3944875	99	34	32,2	cyclictest	253795	-21	kworker/u16:0+events_unbound	23:00:05	4
3944888	99	33	27,5	cyclictest	648	-21	systemd-journal	21:18:11	7
3944883	99	32	30,2	cyclictest	4055218	-21	kworker/u16:3+flush-8:0	21:03:11	6
3944875	99	32	30,2	cyclictest	36954	-21	kworker/u16:3+events_unbound	22:47:05	4
3944875	99	32	30,2	cyclictest	327074	-21	kworker/u16:3+events_unbound	23:32:12	4
3944875	99	32	30,2	cyclictest	289739	-21	kworker/u16:1+events_unbound	23:19:24	4
3944875	99	32	30,2	cyclictest	241304	-21	kworker/u16:1+events_unbound	22:56:28	4
3944875	99	32	30,2	cyclictest	241304	-21	kworker/u16:1+events_unbound	22:56:28	4
3944875	99	32	30,2	cyclictest	170727	-21	kworker/u16:2+events_unbound	22:35:33	4
3944882	99	31	26,5	cyclictest	253795	-21	kworker/u16:0+flush-8:0	23:03:10	5
3944882	99	31	26,5	cyclictest	253795	-21	kworker/u16:0+flush-8:0	23:03:10	5
3944875	99	31	29,2	cyclictest	388992	-21	kworker/u16:4+events_unbound	00:22:29	4
3944875	99	31	29,2	cyclictest	388992	-21	kworker/u16:4+events_unbound	00:22:29	4
3944875	99	31	29,2	cyclictest	327074	-21	kworker/u16:3+events_unbound	23:27:05	4
3944875	99	31	28,2	cyclictest	3954162	-21	kworker/u16:3+events_unbound	19:33:13	4
3944875	99	31	28,2	cyclictest	36954	-21	kworker/u16:3+flush-8:0	22:23:43	4
3944888	99	30	24,5	cyclictest	4094285	-21	latency_hist	20:23:11	7
3944883	99	30	27,2	cyclictest	4113795	-21	kworker/u16:4+events_unbound	20:43:25	6
3944875	99	30	28,2	cyclictest	72894	-21	kworker/u16:1+events_unbound	22:02:57	4
3944875	99	30	28,2	cyclictest	4177273	-21	kworker/u16:0+events_unbound	21:10:46	4
3944883	99	29	27,2	cyclictest	4103819	-21	kworker/u16:2+flush-8:0	20:28:11	6
3944875	99	29	27,2	cyclictest	426907	-21	kworker/u16:1+events_unbound	00:34:40	4
3944875	99	29	27,2	cyclictest	4177273	-21	kworker/u16:0+events_unbound	21:21:04	4
3944875	99	29	27,2	cyclictest	4161991	-21	kworker/u16:1+events_unbound	20:58:12	4
3944875	99	29	27,2	cyclictest	361715	-21	kworker/u16:0+events_unbound	00:00:28	4
3944875	99	29	27,2	cyclictest	158715	-21	kworker/u16:0+flush-8:0	22:40:49	4
3944875	99	29	27,2	cyclictest	158715	-21	kworker/u16:0+events_unbound	22:22:00	4
3944875	99	29	27,2	cyclictest	158715	-21	kworker/u16:0+events_unbound	22:22:00	4
3944875	99	28	26,2	cyclictest	4016703	-21	kworker/u16:2+events_unbound	19:53:58	4
3944875	99	28	26,2	cyclictest	327074	-21	kworker/u16:3+flush-8:0	23:40:12	4
3944875	99	28	26,2	cyclictest	327074	-21	kworker/u16:3+flush-8:0	23:40:12	4
3944875	99	28	26,2	cyclictest	327074	-21	kworker/u16:3+events_unbound	23:36:00	4
410894	2	27	0,1	chrt	410893	-21	ssh	23:58:46	7
3944882	99	27	26,1	cyclictest	36954	-21	kworker/u16:3+events_unbound	22:35:45	5
3944882	99	27	25,2	cyclictest	36954	-21	kworker/u16:3+flush-8:0	21:59:13	5
3944875	99	27	25,2	cyclictest	388992	-21	kworker/u16:4+events_unbound	00:32:17	4
3944875	99	27	25,2	cyclictest	388992	-21	kworker/u16:4+events_unbound	00:09:04	4
3944875	99	27	25,2	cyclictest	3881316	-21	kworker/u16:1+flush-8:0	19:11:30	4
3944875	99	27	25,2	cyclictest	255591	-21	kworker/u16:2+flush-8:0	23:13:15	4
3944875	99	27	24,2	cyclictest	4055218	-21	kworker/u16:3+flush-8:0	20:18:14	4
3944888	99	26	21,4	cyclictest	648	-21	systemd-journal	22:58:11	7
3944888	99	26	21,4	cyclictest	648	-21	systemd-journal	22:58:11	7
3944888	99	26	20,5	cyclictest	4016664	-21	latency_hist	19:43:11	7
3944883	99	26	24,2	cyclictest	4026736	-21	kworker/u16:1+events_unbound	19:58:30	6
3944883	99	26	24,2	cyclictest	327074	-21	kworker/u16:3+events_unbound	23:43:28	6
3944882	99	26	24,2	cyclictest	361715	-21	kworker/u16:0+events_unbound	23:54:21	5
3944875	99	26	25,1	cyclictest	36954	-21	kworker/u16:3+events_unbound	21:49:36	4
3944875	99	26	24,2	cyclictest	72894	-21	kworker/u16:1+events_unbound	22:05:12	4
3944875	99	26	24,2	cyclictest	4177273	-21	kworker/u16:0+events_unbound	21:16:58	4
3944875	99	26	24,2	cyclictest	4065575	-21	kworker/u16:0+flush-8:0	20:09:01	4
3944875	99	26	24,2	cyclictest	4055218	-21	kworker/u16:3+events_unbound	20:47:58	4
3944875	99	26	24,2	cyclictest	3954162	-21	kworker/u16:3+events_unbound	19:20:58	4
3944875	99	26	24,2	cyclictest	376480	-21	kworker/u16:1+events_unbound	23:51:35	4
3944875	99	26	24,2	cyclictest	36954	-21	kworker/u16:3+events_unbound	22:15:02	4
3944875	99	26	24,2	cyclictest	22316	-21	kworker/u16:1+events_unbound	21:31:14	4
3944875	99	26	24,2	cyclictest	22316	-21	kworker/u16:1+events_unbound	21:24:56	4
3944875	99	26	24,2	cyclictest	192453	-21	kworker/u16:4+events_unbound	23:07:44	4
3944888	99	25	24,0	cyclictest	0	-21	swapper/7	23:38:52	7
3944888	99	25	24,0	cyclictest	0	-21	swapper/7	23:38:52	7
3944888	99	25	24,0	cyclictest	0	-21	swapper/7	21:30:42	7
3944888	99	25	24,0	cyclictest	0	-21	swapper/7	00:25:47	7
3944883	99	25	22,2	cyclictest	4177273	-21	kworker/u16:0+flush-8:0	21:13:17	6
3944883	99	25	22,2	cyclictest	36954	-21	kworker/u16:3+events_unbound	22:53:24	6
3944883	99	25	22,2	cyclictest	36954	-21	kworker/u16:3+events_unbound	22:53:24	6
3944882	99	25	23,2	cyclictest	253795	-21	kworker/u16:0+flush-8:0	22:55:45	5
3944882	99	25	23,2	cyclictest	253795	-21	kworker/u16:0+flush-8:0	22:55:45	5
3944875	99	25	23,2	cyclictest	4113795	-21	kworker/u16:4+events_unbound	20:56:01	4
3944875	99	25	23,2	cyclictest	361715	-21	kworker/u16:0+flush-8:0	00:04:47	4
3944888	99	24	23,0	cyclictest	0	-21	swapper/7	23:25:55	7
3944888	99	24	23,0	cyclictest	0	-21	swapper/7	22:23:11	7
3944888	99	24	23,0	cyclictest	0	-21	swapper/7	20:30:33	7
3944888	99	24	23,0	cyclictest	0	-21	swapper/7	00:06:30	7
3944888	99	24	19,4	cyclictest	648	-21	systemd-journal	19:58:12	7
3944883	99	24	22,2	cyclictest	4055218	-21	kworker/u16:3+flush-8:0	20:03:42	6
3944883	99	24	22,2	cyclictest	3954162	-21	kworker/u16:3+flush-8:0	19:18:07	6
3944883	99	24	21,2	cyclictest	4177273	-21	kworker/u16:0+events_unbound	21:58:33	6
3944882	99	24	22,2	cyclictest	255591	-21	kworker/u16:2+flush-8:0	23:23:35	5
3944882	99	24	21,2	cyclictest	327074	-21	kworker/u16:3+flush-8:0	23:43:32	5
3944875	99	24	22,2	cyclictest	4177273	-21	kworker/u16:0+flush-8:0	21:54:02	4
3944875	99	24	22,2	cyclictest	4093990	-21	kworker/u16:0+events_unbound	20:52:05	4
3944875	99	24	22,2	cyclictest	388992	-21	kworker/u16:4+events_unbound	00:17:32	4
3944875	99	24	22,2	cyclictest	192453	-21	kworker/u16:4+flush-8:0	23:12:08	4
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17