OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #3, slot #4

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance

Characteristics of the 100 highest latencies:

System rack3slot4.osadl.org (updated Sun Feb 01, 2026 00:46:37)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
2820240	2	1449	1440,4	sleep5	0	-21	swapper/5	19:07:42	5
2819186	2	346	324,8	sleep3	2814577	-21	kworker/3:3+events@ dbs_work_handler	19:05:33	3
2820569	99	166	128,18	cyclictest	3025980	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	23:17:34	0
2820569	99	146	120,19	cyclictest	2816201	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	19:47:11	0
3098201	2	130	0,3	sleep5	0	-21	swapper/5	00:14:21	5
2820569	99	130	58,49	cyclictest	3044601	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:43:57	0
2820569	99	122	64,35	cyclictest	2911975	-21	kworker/u12:4+events_unbound@ flush_memcg_stats_dwork	22:26:56	0
2820569	99	121	61,36	cyclictest	2867018	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	21:34:16	0
2820582	99	120	107,6	cyclictest	3002365	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:57:32	3
2820569	99	119	72,40	cyclictest	3017850	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	23:24:17	0
2820582	99	118	105,7	cyclictest	3066677	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	23:52:39	3
2820569	99	118	92,19	cyclictest	2856715	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	20:37:15	0
2820569	99	118	62,34	cyclictest	2953614	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:51:11	0
2820569	99	117	90,19	cyclictest	3044601	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:45:10	0
2820569	99	117	59,36	cyclictest	2920146	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	21:51:23	0
2820569	99	116	70,39	cyclictest	2867018	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	21:07:20	0
2820578	99	115	96,7	cyclictest	3025980	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	23:47:35	2
2820569	99	115	58,35	cyclictest	3044601	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:37:23	0
2820569	99	114	56,34	cyclictest	2925915	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	22:01:04	0
2820569	99	114	56,34	cyclictest	2925915	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	22:01:03	0
2820591	99	113	107,3	cyclictest	0	-21	swapper/5	23:17:34	5
2820569	99	113	86,19	cyclictest	2946787	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:55:52	0
2820569	99	113	58,35	cyclictest	2867018	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	21:21:40	0
2820569	99	113	55,36	cyclictest	2816201	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	19:17:56	0
2820569	99	110	6,77	cyclictest	2839072	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	20:00:16	0
2820569	99	109	83,19	cyclictest	2816201	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	19:29:34	0
2820569	99	109	82,19	cyclictest	2995290	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:12:35	0
2820569	99	109	65,37	cyclictest	2874396	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	21:26:05	0
2820569	99	109	65,37	cyclictest	2874396	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	21:26:04	0
2820569	99	109	22,75	cyclictest	2915294	-21	kworker/0:0+pm@ pm_runtime_work	22:00:01	0
2820569	99	108	82,19	cyclictest	2946787	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:21:34	0
2820573	99	107	82,9	cyclictest	3017850	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	23:33:34	1
2820573	99	107	82,9	cyclictest	3017850	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	23:33:34	1
2820569	99	107	82,18	cyclictest	2836672	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	20:12:13	0
2820582	99	106	92,7	cyclictest	2953614	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:22:28	3
2820578	99	105	91,7	cyclictest	3017850	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	23:12:33	2
2820569	99	105	78,19	cyclictest	2839072	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	19:57:13	0
2820569	99	105	67,26	cyclictest	689	50	irq/527-eth0-Tx	00:15:29	0
2820591	99	104	99,2	cyclictest	0	-21	swapper/5	19:47:10	5
2820578	99	104	91,6	cyclictest	2946787	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:42:30	2
2820578	99	104	91,6	cyclictest	2946787	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:42:30	2
2820578	99	103	90,6	cyclictest	3078077	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	23:57:40	2
2820578	99	102	89,7	cyclictest	2856715	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	20:52:15	2
2820569	99	102	60,22	cyclictest	0	-21	swapper/0	20:57:05	0
2820569	99	102	60,22	cyclictest	0	-21	swapper/0	20:57:04	0
2820569	99	101	75,19	cyclictest	3117815	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:30:53	0
2820569	99	101	61,25	cyclictest	0	-21	swapper/0	20:25:20	0
2820569	99	101	1,86	cyclictest	0	-21	swapper/0	23:50:14	0
2896544	2	100	0,4	sleep5	0	-21	swapper/5	21:32:37	5
2820569	99	100	74,19	cyclictest	3035610	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:20:57	0
2820569	99	100	74,19	cyclictest	3035610	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:20:57	0
2820569	99	100	63,30	cyclictest	2867018	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	21:15:26	0
2820569	99	100	63,22	cyclictest	0	-21	swapper/0	22:14:48	0
2820569	99	100	55,23	cyclictest	0	-21	swapper/0	00:25:25	0
2820569	99	100	55,23	cyclictest	0	-21	swapper/0	00:25:25	0
2820578	99	99	85,7	cyclictest	3035610	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:24:36	2
2820578	99	99	85,7	cyclictest	3035610	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:24:35	2
2820573	99	99	86,6	cyclictest	3089714	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	00:07:40	1
2820573	99	99	86,6	cyclictest	2867018	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	21:07:18	1
2820569	99	99	73,19	cyclictest	3017850	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	23:25:11	0
2820569	99	99	73,19	cyclictest	3017850	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	23:25:11	0
2820569	99	99	60,15	cyclictest	2975834	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:46:46	0
2820569	99	99	60,15	cyclictest	2975834	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:46:46	0
2820569	99	99	1,80	cyclictest	0	-21	swapper/0	22:30:29	0
2820569	99	98	75,20	cyclictest	3002365	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:04:34	0
2820569	99	98	72,19	cyclictest	2946787	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:37:03	0
2820569	99	98	72,19	cyclictest	2856715	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	20:43:35	0
2820569	99	98	72,19	cyclictest	2856715	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	20:43:34	0
2820569	99	98	2,7	cyclictest	3093388	-21	ssh	00:10:17	0
2820569	99	97	18,68	cyclictest	691	50	irq/529-eth0-Tx	23:55:55	0
2820587	99	96	89,3	cyclictest	0	-21	swapper/4	19:30:19	4
2820582	99	96	82,7	cyclictest	2836672	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	20:07:12	3
2820582	99	96	80,8	cyclictest	3025980	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	23:47:39	3
2820578	99	96	84,6	cyclictest	3044601	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:42:38	2
2820573	99	96	83,6	cyclictest	2864955	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:11:33	1
2820569	99	96	71,18	cyclictest	3078077	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	00:03:54	0
2820569	99	96	64,23	cyclictest	690	50	irq/528-eth0-Tx	21:35:37	0
2820569	99	96	52,37	cyclictest	2816201	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	19:20:05	0
2820569	99	96	49,25	cyclictest	2839072	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	20:23:02	0
2820569	99	96	49,25	cyclictest	2839072	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	20:23:01	0
2820569	99	95	71,17	cyclictest	2869389	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	21:11:02	0
2820569	99	95	60,23	cyclictest	0	-21	swapper/0	20:15:12	0
2820578	99	94	80,7	cyclictest	2839072	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	20:32:14	2
2820573	99	94	80,7	cyclictest	3117815	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:34:33	1
2820569	99	94	69,18	cyclictest	2816201	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	19:38:35	0
2820569	99	94	69,18	cyclictest	2816201	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	19:38:34	0
2820569	99	94	69,18	cyclictest	2816201	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	19:10:41	0
2820582	99	93	78,8	cyclictest	2911975	-21	kworker/u12:4+events_unbound@ flush_memcg_stats_dwork	22:17:27	3
2820582	99	93	70,8	cyclictest	3017850	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	23:26:34	3
2820582	99	93	70,8	cyclictest	3017850	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	23:26:34	3
2820582	99	93	63,7	cyclictest	3111404	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	00:26:26	3
2820582	99	93	63,7	cyclictest	3111404	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	00:26:26	3
2820578	99	93	80,7	cyclictest	2816201	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	19:52:12	2
2820578	99	93	80,7	cyclictest	2816201	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	19:52:11	2
2820573	99	93	77,8	cyclictest	2839072	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	20:02:11	1
2820569	99	93	76,14	cyclictest	2953614	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:19:35	0
2820569	99	93	70,15	cyclictest	2862881	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	20:53:27	0
2820569	99	93	56,25	cyclictest	0	-21	swapper/0	22:40:19	0
2820569	99	93	56,25	cyclictest	0	-21	swapper/0	22:40:18	0
2820578	99	92	78,7	cyclictest	2816201	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	19:27:09	2
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17