OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #3, slot #4

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance

Characteristics of the 100 highest latencies:

System rack3slot4.osadl.org (updated Fri Jun 26, 2026 00:46:38)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
1591198	2	2469	2455,6	sleep1	1565197	-21	kworker/1:2+events@ dbs_work_handler	19:05:39	1
1591324	2	126	104,13	sleep4	0	-21	swapper/4	19:05:48	4
1591806	99	115	102,6	cyclictest	1571106	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	19:17:54	3
1591806	99	115	102,6	cyclictest	1571106	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	19:17:53	3
1591792	99	105	40,15	cyclictest	48	-21	rcuop/4	23:58:17	0
1591792	99	104	15,13	cyclictest	673	50	irq/527-eth0-Tx	23:15:21	0
1591806	99	102	90,6	cyclictest	1730579	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:58:31	3
1591792	99	102	48,32	cyclictest	210	-21	kswapd0	21:40:13	0
1591806	99	101	88,7	cyclictest	1621612	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	20:53:07	3
1591806	99	101	88,7	cyclictest	1621612	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	20:53:06	3
1591806	99	100	86,7	cyclictest	1870226	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:23:34	3
1591792	99	99	36,28	cyclictest	1730579	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	22:55:26	0
1591792	99	99	36,28	cyclictest	1730579	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	22:55:25	0
1591797	99	96	82,7	cyclictest	1686360	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	21:48:15	1
1591815	99	95	88,3	cyclictest	0	-21	swapper/5	00:08:33	5
1591797	99	94	81,7	cyclictest	1643193	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	21:38:15	1
1591797	99	94	81,7	cyclictest	1643193	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	21:38:15	1
1591792	99	93	71,15	cyclictest	1842293	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	00:08:33	0
1591806	99	92	79,6	cyclictest	1571106	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	19:27:55	3
1591792	99	92	32,14	cyclictest	673	50	irq/527-eth0-Tx	21:53:52	0
1591792	99	92	2,34	cyclictest	0	-21	swapper/0	23:00:37	0
1591792	99	92	2,34	cyclictest	0	-21	swapper/0	23:00:37	0
1591792	99	91	57,5	cyclictest	673	50	irq/527-eth0-Tx	00:15:47	0
1591792	99	91	57,5	cyclictest	673	50	irq/527-eth0-Tx	00:15:47	0
1591806	99	90	76,7	cyclictest	1615810	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	20:13:00	3
1591792	99	90	41,3	cyclictest	0	-21	swapper/0	23:35:41	0
1591797	99	89	75,7	cyclictest	1613787	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	20:08:01	1
1591797	99	89	75,7	cyclictest	1613787	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	20:08:00	1
1591792	99	89	47,15	cyclictest	0	-21	swapper/0	23:46:05	0
1591792	99	89	2,47	cyclictest	1894655	-21	sh	00:33:15	0
1591792	99	89	21,7	cyclictest	675	50	irq/529-eth0-Tx	21:20:00	0
1591797	99	87	74,6	cyclictest	1686360	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	21:55:17	1
1591792	99	87	63,17	cyclictest	1598456	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	19:42:57	0
1591792	99	87	45,4	cyclictest	675	50	irq/529-eth0-Tx	21:10:37	0
1591806	99	86	73,7	cyclictest	1637225	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	21:03:07	3
1591792	99	85	42,15	cyclictest	675	50	irq/529-eth0-Tx	23:05:38	0
1591792	99	85	36,17	cyclictest	1714055	-21	cat	22:10:01	0
1591792	99	85	2,40	cyclictest	676	50	irq/530-eth0-Tx	22:15:26	0
1591792	99	85	2,40	cyclictest	676	50	irq/530-eth0-Tx	22:15:25	0
1591815	99	84	79,2	cyclictest	0	-21	swapper/5	19:42:57	5
1591806	99	84	73,5	cyclictest	1863540	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	00:13:33	3
1591792	99	84	44,4	cyclictest	675	50	irq/529-eth0-Tx	21:20:31	0
1591792	99	83	73,6	cyclictest	210	-21	kswapd0	00:00:31	0
1591792	99	83	65,14	cyclictest	210	-21	kswapd0	23:10:12	0
1591792	99	83	2,46	cyclictest	0	-21	swapper/0	20:30:43	0
1591806	99	82	69,7	cyclictest	1870226	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:28:34	3
1591792	99	82	41,7	cyclictest	1634175	-21	grep	20:55:01	0
1591792	99	82	41,7	cyclictest	1634175	-21	grep	20:55:00	0
1591792	99	82	40,31	cyclictest	0	-21	swapper/0	19:45:28	0
1591810	99	81	76,2	cyclictest	0	-21	swapper/4	23:15:27	4
1591792	99	81	45,28	cyclictest	1731389	-21	kworker/0:2+pm@ pm_runtime_work	22:25:27	0
1591792	99	81	1,30	cyclictest	1863540	-21	kworker/u12:1+rpciod@ rpc_async_schedule	00:30:01	0
1591810	99	79	72,4	cyclictest	0	-21	swapper/4	23:06:49	4
1591792	99	79	32,17	cyclictest	0	-21	swapper/0	20:05:41	0
1591792	99	79	32,17	cyclictest	0	-21	swapper/0	20:05:40	0
1591797	99	78	67,5	cyclictest	1598456	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	20:02:59	1
1591792	99	78	5,35	cyclictest	0	-21	swapper/0	22:13:21	0
1591792	99	78	5,35	cyclictest	0	-21	swapper/0	22:13:20	0
1591815	99	77	71,3	cyclictest	0	-21	swapper/5	23:53:31	5
1591810	99	77	63,8	cyclictest	1643459	-21	diskmemload-4	23:03:03	4
1591810	99	77	63,8	cyclictest	1643459	-21	diskmemload-4	23:03:03	4
1591806	99	77	65,6	cyclictest	1696734	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:33:21	3
1591797	99	77	64,7	cyclictest	1730579	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:47:46	1
1591792	99	77	40,16	cyclictest	674	50	irq/528-eth0-Tx	00:12:09	0
1591792	99	77	1,37	cyclictest	1803955	-21	timerwakeupswit	23:20:43	0
1591810	99	76	70,3	cyclictest	0	-21	swapper/4	23:22:03	4
1591797	99	76	64,6	cyclictest	1790197	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:19:46	1
1591792	99	76	54,15	cyclictest	1730579	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:53:31	0
1591792	99	76	47,3	cyclictest	0	-21	swapper/0	20:40:01	0
1591792	99	76	47,3	cyclictest	0	-21	swapper/0	20:40:00	0
1591792	99	76	47,24	cyclictest	1635769	-1	kworker/0:2H-events_highpri	21:46:11	0
1591792	99	76	29,23	cyclictest	675	50	irq/529-eth0-Tx	21:57:42	0
1591797	99	75	62,7	cyclictest	1833396	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	23:56:11	1
1591792	99	75	23,5	cyclictest	1640088	-21	diskmemload	00:20:35	0
1591810	99	74	68,3	cyclictest	0	-21	swapper/4	19:45:35	4
1591810	99	74	61,8	cyclictest	1853454	-21	ssh	00:00:27	4
1591810	99	74	59,9	cyclictest	1882851	-21	ssh	00:24:03	4
1591806	99	74	62,6	cyclictest	1696734	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:08:17	3
1591797	99	74	62,6	cyclictest	1618924	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	20:23:02	1
1591797	99	74	61,6	cyclictest	1633945	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	20:58:06	1
1591810	99	73	66,3	cyclictest	0	-21	swapper/4	22:49:05	4
1591810	99	73	66,3	cyclictest	0	-21	swapper/4	21:05:35	4
1591797	99	73	59,7	cyclictest	1730579	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:44:51	1
1591797	99	73	59,7	cyclictest	1730579	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:44:50	1
1834985	2	72	0,4	chrt	0	-21	swapper/5	23:45:29	5
1687517	2	72	0,4	sleep3	0	-21	swapper/3	21:48:38	3
1591810	99	72	66,3	cyclictest	0	-21	swapper/4	20:25:26	4
1591797	99	72	58,7	cyclictest	1637225	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	21:23:10	1
1591792	99	72	52,5	cyclictest	210	-21	kswapd0	22:40:24	0
1591806	99	71	58,6	cyclictest	1686360	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	22:03:15	3
1591792	99	71	38,13	cyclictest	674	50	irq/528-eth0-Tx	20:59:37	0
1591806	99	70	63,5	cyclictest	1623926	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	20:38:04	3
1591806	99	70	63,5	cyclictest	1623926	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	20:38:03	3
1591792	99	70	48,15	cyclictest	1600910	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	19:37:57	0
1591810	99	69	65,2	cyclictest	0	-21	swapper/4	20:35:15	4
1591810	99	69	65,2	cyclictest	0	-21	swapper/4	20:35:14	4
1591810	99	69	57,6	cyclictest	1643459	-21	diskmemload-4	22:34:19	4
1591810	99	69	55,8	cyclictest	1694787	-21	ssh	21:54:39	4
1591810	99	69	54,9	cyclictest	1885129	-21	ssh	00:25:27	4
1591792	99	69	46,7	cyclictest	13	-21	ksoftirqd/0	00:35:27	0
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17