OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #3, slot #4

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance

Characteristics of the 100 highest latencies:

System rack3slot4.osadl.org (updated Sun Dec 28, 2025 00:47:00)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
3168829	2	3391	3381,5	sleep5	0	-21	swapper/5	19:06:30	5
3168820	2	3312	3293,6	sleep3	3167071	-21	kworker/3:3+events@ dbs_work_handler	19:06:22	3
3169257	99	138	124,7	cyclictest	3189920	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	20:07:54	1
3169257	99	130	118,6	cyclictest	3336539	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:53:29	1
3169263	99	127	114,7	cyclictest	3199424	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	20:32:59	2
3169257	99	124	110,7	cyclictest	3244215	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	21:38:11	1
3169257	99	118	105,7	cyclictest	3355999	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	23:13:31	1
3169253	99	117	46,37	cyclictest	3401968	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:53:35	0
3169269	99	115	100,7	cyclictest	3336539	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:58:28	3
3169273	99	112	105,3	cyclictest	0	-21	swapper/4	23:43:39	4
3169263	99	108	96,6	cyclictest	3401968	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:58:41	2
3169273	99	107	94,8	cyclictest	3424239	-21	ssh	00:08:45	4
3169273	99	107	94,8	cyclictest	3424239	-21	ssh	00:08:44	4
3169263	99	107	93,7	cyclictest	3435990	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	00:28:46	2
3169263	99	107	93,7	cyclictest	3435990	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	00:28:45	2
3169257	99	104	90,7	cyclictest	3289847	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:44:33	1
3169253	99	103	48,17	cyclictest	15	-21	rcuc/0	22:05:28	0
3169253	99	103	48,17	cyclictest	15	-21	rcuc/0	22:05:28	0
3169257	99	102	91,7	cyclictest	3228934	-21	bingrep	21:20:28	1
3169278	99	101	95,3	cyclictest	0	-21	swapper/5	22:45:14	5
3169269	99	100	81,15	cyclictest	3281220	-21	bingrep	22:05:27	3
3169269	99	100	81,15	cyclictest	3281220	-21	bingrep	22:05:26	3
3169263	99	100	86,7	cyclictest	3211506	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	20:58:04	2
3169257	99	100	88,6	cyclictest	3185278	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	19:52:52	1
3169253	99	98	88,6	cyclictest	210	-21	kswapd0	23:20:11	0
3169257	99	97	88,6	cyclictest	210	-21	kswapd0	00:30:26	1
3169257	99	97	88,6	cyclictest	210	-21	kswapd0	00:30:26	1
3169257	99	97	84,7	cyclictest	3355999	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	23:28:35	1
3169257	99	97	84,7	cyclictest	3244215	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:15:36	1
3169257	99	97	84,7	cyclictest	3244215	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:15:36	1
3169273	99	96	81,9	cyclictest	3326917	-21	ssh	22:45:14	4
3169257	99	96	84,6	cyclictest	3289847	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:48:29	1
3169257	99	96	81,8	cyclictest	3336539	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	23:00:23	1
3169253	99	96	82,7	cyclictest	3349656	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:15:29	0
3169253	99	96	82,7	cyclictest	3173006	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	19:17:38	0
3169278	99	95	89,3	cyclictest	0	-21	swapper/5	22:52:04	5
3169278	99	95	89,3	cyclictest	0	-21	swapper/5	21:28:10	5
3405296	2	94	0,4	sleep2	0	-21	swapper/2	23:51:54	2
3169273	99	94	89,2	cyclictest	0	-21	swapper/4	21:18:07	4
3169263	99	94	84,4	cyclictest	3217798	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:53:14	2
3169257	99	94	80,7	cyclictest	3217798	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:26:01	1
3169253	99	94	85,5	cyclictest	15	-21	rcuc/0	22:25:06	0
3169253	99	94	85,5	cyclictest	15	-21	rcuc/0	22:25:06	0
3169273	99	93	86,3	cyclictest	0	-21	swapper/4	20:55:46	4
3169278	99	92	86,3	cyclictest	0	-21	swapper/5	00:39:19	5
3169257	99	92	77,8	cyclictest	3173006	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	19:35:13	1
3169278	99	90	85,2	cyclictest	0	-21	swapper/5	23:48:39	5
3169278	99	90	85,2	cyclictest	0	-21	swapper/5	21:18:07	5
3169278	99	90	82,3	cyclictest	0	-21	swapper/5	23:15:27	5
3169269	99	90	77,7	cyclictest	3244215	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:03:19	3
3169269	99	90	67,9	cyclictest	3217798	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:20:37	3
3169257	99	90	81,6	cyclictest	210	-21	kswapd0	23:45:14	1
3169273	99	89	70,14	cyclictest	3333502	-21	ssh	22:50:29	4
3169269	99	89	77,6	cyclictest	3251798	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	21:43:12	3
3169269	99	89	76,7	cyclictest	3289847	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:28:24	3
3169269	99	89	76,7	cyclictest	3289847	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:28:24	3
3169253	99	89	60,13	cyclictest	3430030	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	00:20:17	0
3169253	99	89	60,13	cyclictest	3430030	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	00:20:16	0
3169253	99	89	49,33	cyclictest	3349656	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:10:30	0
3169278	99	88	82,3	cyclictest	0	-21	swapper/5	23:50:58	5
3169273	99	88	75,8	cyclictest	3458298	-21	ssh	00:37:35	4
3169257	99	88	74,7	cyclictest	3289847	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:23:33	1
3169257	99	88	74,7	cyclictest	3289847	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:23:33	1
3169253	99	88	61,17	cyclictest	0	-21	swapper/0	23:35:44	0
3169273	99	87	81,3	cyclictest	0	-21	swapper/4	21:58:18	4
3169269	99	87	73,7	cyclictest	3376845	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:20:35	3
3169269	99	87	73,7	cyclictest	3376845	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:20:35	3
3169263	99	87	77,6	cyclictest	210	-21	kswapd0	00:05:28	2
3169263	99	87	77,6	cyclictest	210	-21	kswapd0	00:05:27	2
3169263	99	87	74,7	cyclictest	3308959	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:43:27	2
3169257	99	87	74,7	cyclictest	3199424	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	20:43:45	1
3169278	99	86	80,3	cyclictest	0	-21	swapper/5	22:17:35	5
3169278	99	86	80,3	cyclictest	0	-21	swapper/5	22:17:34	5
3169278	99	86	80,3	cyclictest	0	-21	swapper/5	00:05:45	5
3169278	99	86	80,3	cyclictest	0	-21	swapper/5	00:05:44	5
3169278	99	86	72,8	cyclictest	3173134	2	sleep5	19:18:12	5
3169273	99	86	79,2	cyclictest	0	-21	swapper/4	00:19:04	4
3169257	99	86	73,7	cyclictest	3376845	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:06:14	1
3169257	99	86	73,7	cyclictest	3376845	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:06:14	1
3169253	99	86	72,7	cyclictest	3217798	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:45:03	0
3169269	99	85	77,5	cyclictest	210	-21	kswapd0	23:10:00	3
3169257	99	85	73,6	cyclictest	3349656	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:18:34	1
3169257	99	85	71,7	cyclictest	3376845	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:24:23	1
3169257	99	85	71,7	cyclictest	3376845	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:24:23	1
3169253	99	85	71,7	cyclictest	3289847	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:34:28	0
3169278	99	84	77,3	cyclictest	0	-21	swapper/5	00:15:25	5
3169278	99	84	74,5	cyclictest	0	-21	swapper/5	00:27:27	5
3169278	99	84	74,5	cyclictest	0	-21	swapper/5	00:27:26	5
3169269	99	84	71,7	cyclictest	3376845	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:14:36	3
3169269	99	84	71,7	cyclictest	3376845	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:14:36	3
3169257	99	84	72,6	cyclictest	3261423	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	21:50:56	1
3169257	99	84	53,7	cyclictest	3401968	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:50:30	1
3169273	99	83	78,2	cyclictest	0	-21	swapper/4	20:02:53	4
3169273	99	83	78,2	cyclictest	0	-21	swapper/4	00:31:05	4
3169273	99	83	78,2	cyclictest	0	-21	swapper/4	00:31:04	4
3169269	99	83	71,6	cyclictest	3231038	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	21:32:07	3
3169269	99	83	69,7	cyclictest	3308959	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	23:03:29	3
3169263	99	83	69,7	cyclictest	3244215	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:11:05	2
3169263	99	83	69,7	cyclictest	3244215	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:11:04	2
3169257	99	83	70,7	cyclictest	3244215	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:25:35	1
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17