OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #3, slot #4

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance

Characteristics of the 100 highest latencies:

System rack3slot4.osadl.org (updated Thu Jan 15, 2026 00:46:35)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
4037908	2	3511	3493,6	sleep2	4018778	-21	kworker/2:2+events@ dbs_work_handler	19:08:15	2
4038208	99	146	37,90	cyclictest	4080539	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	20:55:29	0
4038208	99	139	42,81	cyclictest	48175	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:20:05	0
4038208	99	139	42,81	cyclictest	48175	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:20:04	0
4038208	99	133	28,62	cyclictest	4084548	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:17:18	0
4038208	99	127	37,79	cyclictest	4052364	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	19:45:15	0
4038208	99	124	52,53	cyclictest	73774	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	00:07:27	0
4038208	99	123	72,31	cyclictest	4066246	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	20:53:05	0
4038212	99	118	105,6	cyclictest	4084702	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	21:50:34	1
4038216	99	113	101,6	cyclictest	4038199	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	19:45:09	2
4038208	99	112	61,43	cyclictest	35876	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:10:30	0
4045413	2	111	0,5	sleep1	0	-21	swapper/1	19:25:35	1
4045413	2	111	0,5	sleep1	0	-21	swapper/1	19:25:34	1
4038208	99	110	1,81	cyclictest	0	-21	swapper/0	20:40:49	0
4038212	99	109	94,8	cyclictest	4084702	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	21:35:32	1
4038226	99	106	99,3	cyclictest	0	-21	swapper/4	22:30:41	4
4038226	99	105	98,3	cyclictest	0	-21	swapper/4	00:28:38	4
4038216	99	104	92,6	cyclictest	24273	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:32:36	2
4038221	99	103	62,9	cyclictest	67356	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:43:25	3
4038216	99	100	88,6	cyclictest	4163233	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:15:39	2
4038208	99	100	78,7	cyclictest	4124110	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:05:37	0
4038216	99	99	84,8	cyclictest	4189017	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	22:40:45	2
4038208	99	99	1,70	cyclictest	0	-21	swapper/0	20:05:33	0
4038226	99	98	93,3	cyclictest	0	-21	swapper/4	19:10:03	4
4038226	99	98	93,3	cyclictest	0	-21	swapper/4	19:10:03	4
4038216	99	98	84,7	cyclictest	114747	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	00:26:33	2
4038226	99	97	90,3	cyclictest	0	-21	swapper/4	21:45:33	4
4038226	99	97	86,6	cyclictest	4169524	-21	ssh	22:19:14	4
4038230	99	96	91,3	cyclictest	0	-21	swapper/5	21:40:31	5
4038226	99	95	89,3	cyclictest	0	-21	swapper/4	21:44:34	4
4038216	99	94	62,17	cyclictest	4106635	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	21:34:58	2
4038216	99	94	62,17	cyclictest	4106635	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	21:34:57	2
4038230	99	93	87,3	cyclictest	0	-21	swapper/5	23:35:54	5
4038216	99	92	79,7	cyclictest	35876	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:10:46	2
4038216	99	92	77,8	cyclictest	67356	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:51:55	2
4038212	99	92	64,8	cyclictest	73774	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:43:43	1
4038208	99	92	70,15	cyclictest	2369	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:43:41	0
4038208	99	92	56,33	cyclictest	24273	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:34:00	0
4038226	99	91	84,3	cyclictest	0	-21	swapper/4	19:23:27	4
4038212	99	91	76,8	cyclictest	4146550	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	22:10:38	1
4038221	99	90	76,7	cyclictest	4146550	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	22:00:36	3
4038208	99	90	71,16	cyclictest	4131154	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	23:06:31	0
4038208	99	90	62,8	cyclictest	4084548	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:40:31	0
4038216	99	89	75,7	cyclictest	4038199	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	19:31:34	2
4038208	99	89	64,6	cyclictest	2369	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:50:22	0
4038216	99	88	75,7	cyclictest	83626	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	23:58:06	2
4038212	99	88	74,7	cyclictest	60267	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	23:35:18	1
4038208	99	88	1,73	cyclictest	0	-21	swapper/0	00:18:26	0
4038226	99	87	81,3	cyclictest	0	-21	swapper/4	19:50:11	4
4038216	99	87	56,8	cyclictest	4070243	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	20:31:19	2
4038226	99	86	80,3	cyclictest	0	-21	swapper/4	23:20:17	4
4038226	99	86	80,3	cyclictest	0	-21	swapper/4	23:20:16	4
4038221	99	86	74,6	cyclictest	4066246	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	21:15:23	3
4038212	99	86	59,9	cyclictest	4131154	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	23:25:21	1
4038212	99	86	59,9	cyclictest	4131154	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	23:25:20	1
4038208	99	86	2,5	cyclictest	4183774	-21	ntpq	22:30:36	0
4038208	99	86	1,13	cyclictest	0	-21	swapper/0	00:23:48	0
4038208	99	86	1,13	cyclictest	0	-21	swapper/0	00:23:48	0
4038226	99	85	79,3	cyclictest	0	-21	swapper/4	23:40:22	4
4038226	99	85	73,7	cyclictest	102662	-21	ssh	00:03:09	4
4038226	99	85	72,8	cyclictest	142901	-21	ssh	00:36:02	4
4038221	99	85	57,6	cyclictest	14871	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	23:13:18	3
4038216	99	85	71,7	cyclictest	4131154	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	23:05:32	2
4038216	99	85	70,8	cyclictest	4163233	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:20:34	2
4038216	99	85	69,8	cyclictest	4131154	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:05:17	2
4038208	99	85	59,7	cyclictest	67356	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:35:23	0
4038208	99	85	1,62	cyclictest	4140141	-21	latency_hist	21:55:01	0
4038230	99	84	80,2	cyclictest	0	-21	swapper/5	19:20:04	5
4038221	99	84	72,6	cyclictest	24273	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:18:04	3
4038216	99	84	69,8	cyclictest	4092531	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	21:20:30	2
4038216	99	84	68,9	cyclictest	4092531	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	21:35:12	2
4139416	2	83	0,4	sleep3	37	-21	ksoftirqd/3	21:54:23	3
4038208	99	83	1,65	cyclictest	0	-21	swapper/0	20:35:27	0
4038230	99	82	77,2	cyclictest	0	-21	swapper/5	22:05:37	5
4038226	99	82	76,3	cyclictest	0	-21	swapper/4	19:39:24	4
4038226	99	82	76,3	cyclictest	0	-21	swapper/4	00:14:56	4
4038226	99	82	76,3	cyclictest	0	-21	swapper/4	00:14:55	4
4038216	99	82	69,6	cyclictest	4124110	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:00:50	2
4038208	99	82	2,6	cyclictest	4077737	-21	ntpq	20:45:35	0
4038208	99	82	2,6	cyclictest	4077737	-21	ntpq	20:45:35	0
4038208	99	82	1,57	cyclictest	0	-21	swapper/0	00:11:39	0
4038208	99	82	1,57	cyclictest	0	-21	swapper/0	00:11:39	0
4038226	99	81	66,9	cyclictest	44010	-21	ssh	23:15:05	4
4038221	99	81	69,6	cyclictest	67356	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:05:52	3
4038216	99	81	66,7	cyclictest	24273	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:27:06	2
4038216	99	81	66,7	cyclictest	24273	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:27:05	2
4038216	99	81	65,9	cyclictest	4189017	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	22:45:12	2
4038208	99	81	1,8	cyclictest	29998	-21	ssh	23:03:32	0
4038208	99	81	1,68	cyclictest	0	-21	swapper/0	21:08:35	0
4038208	99	81	1,66	cyclictest	0	-21	swapper/0	19:20:23	0
4038226	99	80	74,3	cyclictest	0	-21	swapper/4	23:06:36	4
4038226	99	80	74,3	cyclictest	0	-21	swapper/4	19:31:14	4
4038226	99	80	73,3	cyclictest	0	-21	swapper/4	00:15:22	4
4038226	99	80	69,6	cyclictest	4093571	-21	ssh	21:15:52	4
4038226	99	80	67,7	cyclictest	4140513	-21	ssh	21:55:14	4
4038216	99	80	66,7	cyclictest	73774	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:40:06	2
4038216	99	80	64,8	cyclictest	4124110	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	21:50:33	2
4038212	99	80	66,7	cyclictest	4066246	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	20:57:45	1
4038208	99	80	1,67	cyclictest	0	-21	swapper/0	23:54:08	0
4038208	99	80	1,5	cyclictest	4099620	-21	users	21:20:46	0
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17