OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Mon Sep 07, 2026 12:43:33)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
63575	2	101	0,1	sleep7	63577	-21	sh	11:24:35	7
4042399	2	98	0,1	sleep7	4042401	-21	cut	09:59:32	7
3679954	2	57	46,8	sleep5	0	-21	swapper/5	07:06:39	5
3679487	2	44	3,30	sleep7	0	-21	swapper/7	07:04:48	7
3680086	2	43	32,8	sleep6	0	-21	swapper/6	07:08:32	6
3680037	2	43	31,8	sleep4	0	-21	swapper/4	07:07:50	4
3680417	99	40	37,2	cyclictest	4101464	-21	kworker/u16:3+events_unbound	10:59:29	5
3680418	99	39	37,2	cyclictest	17128	-21	kworker/u16:2+events_unbound	12:17:05	6
3680417	99	39	37,2	cyclictest	3907092	-21	kworker/u16:1+events_unbound	09:13:58	5
3680419	99	38	34,3	cyclictest	126	-21	kswapd0	11:06:30	7
3680418	99	38	36,2	cyclictest	80442	-21	kworker/u16:3+events_unbound	11:51:38	6
3680417	99	38	36,2	cyclictest	4101465	-21	kworker/u16:4+events_unbound	10:47:42	5
3680417	99	38	36,2	cyclictest	4028044	-21	kworker/u16:5+events_unbound	10:07:05	5
3680417	99	38	36,2	cyclictest	3966130	-21	kworker/u16:4+events_unbound	10:00:15	5
3680417	99	38	35,2	cyclictest	4003020	-21	kworker/u16:1+flush-8:0	09:49:27	5
3680418	99	37	35,2	cyclictest	4101465	-21	kworker/u16:4+flush-8:0	11:04:27	6
3680418	99	37	35,2	cyclictest	4003020	-21	kworker/u16:1+events_unbound	10:29:02	6
3680417	99	37	34,2	cyclictest	3680171	-21	kworker/u16:0+events_unbound	08:19:14	5
3680418	99	36	34,2	cyclictest	4101464	-21	kworker/u16:3+events_unbound	10:35:02	6
3680418	99	36	34,2	cyclictest	3941811	-21	kworker/u16:3+events_unbound	09:42:18	6
3680417	99	36	34,2	cyclictest	4086628	-21	kworker/u16:0+events_unbound	10:22:29	5
3680417	99	36	34,2	cyclictest	236548	-21	kworker/u16:1+events_unbound	12:36:33	5
3680418	99	35	33,2	cyclictest	80442	-21	kworker/u16:3+events_unbound	12:10:57	6
3680418	99	35	33,2	cyclictest	4101465	-21	kworker/u16:4+events_unbound	10:42:21	6
3680418	99	35	33,2	cyclictest	4003020	-21	kworker/u16:1+events_unbound	10:20:18	6
3680418	99	35	33,2	cyclictest	3941811	-21	kworker/u16:3+events_unbound	09:25:18	6
3680418	99	35	33,2	cyclictest	3858714	-21	kworker/u16:2+events_unbound	09:36:06	6
3680418	99	35	33,2	cyclictest	17128	-21	kworker/u16:2+events_unbound	11:25:58	6
3680418	99	35	33,2	cyclictest	117246	-21	kworker/u16:1+events_unbound	12:22:21	6
3680417	99	35	33,2	cyclictest	4101465	-21	kworker/u16:4+events_unbound	10:51:58	5
3680417	99	35	33,2	cyclictest	4101464	-21	kworker/u16:3+events_unbound	10:56:14	5
3680417	99	35	33,2	cyclictest	4101464	-21	kworker/u16:3+events_unbound	10:35:06	5
3680417	99	35	33,2	cyclictest	3858714	-21	kworker/u16:2+events_unbound	09:37:58	5
3680417	99	35	33,2	cyclictest	3848195	-21	kworker/u16:0+events_unbound	09:17:01	5
3680417	99	35	33,2	cyclictest	31375	-21	kworker/u16:0+events_unbound	11:25:37	5
3680418	99	34	32,2	cyclictest	4101464	-21	kworker/u16:3+events_unbound	11:02:25	6
3680418	99	34	32,2	cyclictest	3966130	-21	kworker/u16:4+events_unbound	09:55:57	6
3680418	99	34	32,2	cyclictest	3670188	-21	kworker/u16:1+events_unbound	07:29:40	6
3680418	99	34	32,2	cyclictest	216193	-21	kworker/u16:4+events_unbound	12:38:05	6
3680417	99	34	32,2	cyclictest	80442	-21	kworker/u16:3+events_unbound	12:12:21	5
3680417	99	34	32,2	cyclictest	4101465	-21	kworker/u16:4+events_unbound	10:42:10	5
3680417	99	34	32,2	cyclictest	4003020	-21	kworker/u16:1+events_unbound	10:18:04	5
3680417	99	34	32,2	cyclictest	3858714	-21	kworker/u16:2+events_unbound	09:40:05	5
3680417	99	34	32,2	cyclictest	3858714	-21	kworker/u16:2+events_unbound	09:32:45	5
3680417	99	34	32,2	cyclictest	31375	-21	kworker/u16:0+events_unbound	11:43:24	5
3680418	99	33	31,2	cyclictest	4003020	-21	kworker/u16:1+events_unbound	11:10:10	6
3680418	99	33	31,2	cyclictest	4003020	-21	kworker/u16:1+events_unbound	10:17:26	6
3680418	99	33	31,2	cyclictest	3761914	-21	kworker/u16:3+events_unbound	07:51:31	6
3680418	99	33	31,2	cyclictest	17128	-21	kworker/u16:2+events_unbound	11:42:17	6
3680418	99	33	30,2	cyclictest	3941811	-21	kworker/u16:3+events_unbound	09:44:21	6
3680417	99	33	31,2	cyclictest	4003020	-21	kworker/u16:1+events_unbound	11:12:53	5
3680417	99	33	31,2	cyclictest	3941811	-21	kworker/u16:3+events_unbound	09:58:54	5
3680417	99	33	31,2	cyclictest	3848195	-21	kworker/u16:0+events_unbound	09:25:41	5
3680417	99	33	31,2	cyclictest	3761914	-21	kworker/u16:3+flush-8:0	07:54:35	5
3680417	99	33	31,2	cyclictest	3742381	-21	kworker/u16:2+events_unbound	07:46:30	5
3680417	99	33	31,2	cyclictest	17128	-21	kworker/u16:2+events_unbound	11:50:40	5
3680417	99	33	31,2	cyclictest	17128	-21	kworker/u16:2+events_unbound	11:22:29	5
3680417	99	33	31,2	cyclictest	17128	-21	kworker/u16:2+events_unbound	11:08:54	5
3680418	99	32	30,2	cyclictest	4101464	-21	kworker/u16:3+events_unbound	10:29:57	6
3680418	99	32	30,2	cyclictest	4028044	-21	kworker/u16:5+events_unbound	10:01:05	6
3680418	99	32	30,2	cyclictest	4003020	-21	kworker/u16:1+events_unbound	09:52:26	6
3680418	99	32	30,2	cyclictest	3966130	-21	kworker/u16:4+events_unbound	09:31:04	6
3680418	99	32	30,2	cyclictest	3941811	-21	kworker/u16:3+events_unbound	09:19:45	6
3680418	99	32	30,2	cyclictest	3848195	-21	kworker/u16:0+events_unbound	09:10:21	6
3680418	99	32	30,2	cyclictest	31375	-21	kworker/u16:0+events_unbound	11:46:20	6
3680418	99	32	30,2	cyclictest	17227	-21	kworker/u16:5+events_unbound	11:16:14	6
3680417	99	32	30,2	cyclictest	52727	-21	kworker/u16:1+events_unbound	11:30:36	5
3680417	99	32	30,2	cyclictest	3761914	-21	kworker/u16:3+events_unbound	08:53:34	5
3680417	99	32	30,2	cyclictest	31375	-21	kworker/u16:0+events_unbound	11:35:37	5
3680417	99	32	30,2	cyclictest	17227	-21	kworker/u16:5+events_unbound	11:18:46	5
3680417	99	32	29,2	cyclictest	17128	-21	kworker/u16:2+events_unbound	12:24:28	5
3680419	99	31	28,2	cyclictest	5701	-21	gnome-shell	11:21:21	7
3680419	99	31	0,29	cyclictest	648	-21	systemd-journal	07:34:13	7
3680418	99	31	29,2	cyclictest	3848195	-21	kworker/u16:0+events_unbound	08:36:34	6
3680417	99	31	29,2	cyclictest	4101465	-21	kworker/u16:4+flush-8:0	10:29:28	5
3680417	99	31	29,2	cyclictest	3966130	-21	kworker/u16:4+events_unbound	10:12:24	5
3680417	99	31	29,2	cyclictest	117246	-21	kworker/u16:1+events_unbound	12:14:46	5
3680418	99	30	28,2	cyclictest	4003020	-21	kworker/u16:1+events_unbound	10:07:03	6
3680418	99	30	28,2	cyclictest	3848195	-21	kworker/u16:0+events_unbound	08:44:50	6
3680418	99	30	28,2	cyclictest	3761914	-21	kworker/u16:3+events_unbound	08:11:31	6
3680418	99	30	28,2	cyclictest	17128	-21	kworker/u16:2+events_unbound	12:30:32	6
3680417	99	30	28,2	cyclictest	80442	-21	kworker/u16:3+events_unbound	11:56:31	5
3680417	99	30	28,2	cyclictest	17128	-21	kworker/u16:2+events_unbound	11:47:52	5
3680419	99	29	0,27	cyclictest	4039865	-21	latency_hist	09:59:12	7
3680418	99	29	27,2	cyclictest	4003020	-21	kworker/u16:1+events_unbound	10:46:34	6
3680418	99	29	27,2	cyclictest	3848195	-21	kworker/u16:0+events_unbound	08:32:30	6
3680417	99	29	27,2	cyclictest	4101463	-21	kworker/u16:2+events_unbound	10:28:19	5
3680417	99	29	27,2	cyclictest	3941811	-21	kworker/u16:3+flush-8:0	09:19:38	5
3680417	99	29	27,2	cyclictest	3848195	-21	kworker/u16:0+events_unbound	09:05:31	5
3680417	99	29	27,2	cyclictest	3670188	-21	kworker/u16:1+events_unbound	07:18:30	5
3680417	99	29	27,2	cyclictest	17128	-21	kworker/u16:2+events_unbound	12:19:43	5
3680417	99	29	27,2	cyclictest	17128	-21	kworker/u16:2+events_unbound	12:05:52	5
3680418	99	28	26,2	cyclictest	80442	-21	kworker/u16:3+events_unbound	12:06:31	6
3680418	99	28	26,2	cyclictest	4028044	-21	kworker/u16:5+events_unbound	10:14:03	6
3680418	99	28	26,2	cyclictest	3848195	-21	kworker/u16:0+events_unbound	08:39:36	6
3680418	99	28	25,2	cyclictest	3829760	-21	kworker/u16:2+events_unbound	08:24:15	6
3680417	99	28	26,2	cyclictest	17128	-21	kworker/u16:2+events_unbound	12:30:26	5
3680418	99	27	25,2	cyclictest	4101464	-21	kworker/u16:3+flush-8:0	10:54:48	6
3680418	99	27	25,2	cyclictest	3858714	-21	kworker/u16:2+events_unbound	09:15:39	6
3680418	99	27	25,2	cyclictest	3761914	-21	kworker/u16:3+events_unbound	08:17:29	6
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17