OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #3, slot #4

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance

Characteristics of the 100 highest latencies:

System rack3slot4.osadl.org (updated Mon Jun 29, 2026 00:46:34)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
399874	2	3023	2955,50	sleep0	398477	-21	kworker/0:3+events@ dbs_work_handler	19:05:31	0
400968	99	114	100,7	cyclictest	569374	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:48:12	3
400978	99	97	91,3	cyclictest	0	-21	swapper/5	00:08:30	5
400978	99	97	91,3	cyclictest	0	-21	swapper/5	00:08:30	5
400978	99	96	90,3	cyclictest	0	-21	swapper/5	22:13:05	5
400954	99	94	65,6	cyclictest	546171	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:02:09	0
400954	99	90	1,72	cyclictest	0	-21	swapper/0	00:23:56	0
400954	99	89	3,55	cyclictest	611181	-21	kworker/u12:0+rpciod@ rpc_async_schedule	23:28:20	0
400968	99	85	73,6	cyclictest	622411	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:33:23	3
400963	99	85	71,7	cyclictest	580769	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:58:16	2
400973	99	84	79,2	cyclictest	0	-21	swapper/4	22:40:13	4
400973	99	84	79,2	cyclictest	0	-21	swapper/4	22:40:13	4
400978	99	83	77,3	cyclictest	0	-21	swapper/5	23:08:17	5
400978	99	83	75,4	cyclictest	0	-21	swapper/5	23:35:45	5
400968	99	83	69,7	cyclictest	622411	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:52:21	3
400963	99	83	68,7	cyclictest	611181	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:28:22	2
400959	99	83	73,4	cyclictest	546171	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:03:16	1
400954	99	83	2,68	cyclictest	671170	-21	ssh	00:10:20	0
400954	99	83	2,68	cyclictest	671170	-21	ssh	00:10:20	0
400978	99	82	76,3	cyclictest	0	-21	swapper/5	22:35:27	5
400978	99	82	76,3	cyclictest	0	-21	swapper/5	22:35:27	5
400959	99	82	57,8	cyclictest	449727	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	21:27:57	1
400959	99	82	57,8	cyclictest	449727	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	21:27:56	1
400954	99	82	73,6	cyclictest	210	-21	kswapd0	22:35:27	0
400954	99	82	73,6	cyclictest	210	-21	kswapd0	22:35:27	0
400954	99	82	32,27	cyclictest	675	50	irq/529-eth0-Tx	22:45:32	0
400954	99	82	22,6	cyclictest	676	50	irq/530-eth0-Tx	23:20:33	0
400963	99	81	68,7	cyclictest	449727	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	21:32:58	2
400963	99	81	68,7	cyclictest	449727	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	21:32:58	2
400954	99	81	26,34	cyclictest	455542	-21	ssh	21:15:43	0
400954	99	81	26,34	cyclictest	455542	-21	ssh	21:15:43	0
400954	99	80	1,53	cyclictest	0	-21	swapper/0	21:55:34	0
400954	99	79	2,19	cyclictest	622411	-21	kworker/u12:3+rpciod@ rpc_async_schedule	00:25:30	0
400954	99	79	16,36	cyclictest	475987	-21	ssh	21:32:18	0
400954	99	79	16,36	cyclictest	475987	-21	ssh	21:32:18	0
400978	99	78	71,3	cyclictest	0	-21	swapper/5	23:55:25	5
400978	99	78	71,3	cyclictest	0	-21	swapper/5	23:55:25	5
400978	99	78	71,3	cyclictest	0	-21	swapper/5	21:40:19	5
400963	99	78	66,6	cyclictest	687490	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	00:38:46	2
400954	99	78	26,34	cyclictest	673	50	irq/527-eth0-Tx	23:15:18	0
704362	2	76	0,14	chrt	37	-21	ksoftirqd/3	00:36:39	3
400978	99	76	69,3	cyclictest	0	-21	swapper/5	00:30:23	5
400973	99	76	64,6	cyclictest	452793	-21	diskmemload-4	22:25:17	4
400954	99	76	9,5	cyclictest	673	50	irq/527-eth0-Tx	21:50:02	0
400954	99	76	2,5	cyclictest	550670	-21	ssh	22:32:37	0
400954	99	76	1,26	cyclictest	578082	-21	sh	22:55:11	0
400954	99	76	1,25	cyclictest	0	-21	swapper/0	21:25:27	0
400954	99	76	1,25	cyclictest	0	-21	swapper/0	21:25:26	0
400954	99	76	1,24	cyclictest	0	-21	swapper/0	00:30:26	0
400968	99	75	60,7	cyclictest	546171	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:18:19	3
400963	99	75	61,7	cyclictest	674216	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	00:23:35	2
400954	99	75	38,33	cyclictest	529566	-1	kworker/0:1H-mmc_complete	22:29:38	0
400954	99	75	1,61	cyclictest	0	-21	swapper/0	20:09:14	0
400978	99	74	67,4	cyclictest	0	-21	swapper/5	00:15:08	5
400978	99	74	67,4	cyclictest	0	-21	swapper/5	00:15:07	5
400973	99	74	63,6	cyclictest	676861	-21	ssh	00:15:07	4
400973	99	74	63,6	cyclictest	676861	-21	ssh	00:15:07	4
400954	99	74	23,25	cyclictest	498888	-21	mailstats	21:50:41	0
400954	99	74	14,8	cyclictest	675	50	irq/529-eth0-Tx	23:10:18	0
458928	2	73	0,4	sleep5	0	-21	swapper/5	21:19:04	5
458928	2	73	0,4	sleep5	0	-21	swapper/5	21:19:04	5
400968	99	73	59,7	cyclictest	443100	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	21:17:55	3
400968	99	73	59,7	cyclictest	443100	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	21:17:55	3
400954	99	73	14,5	cyclictest	674	50	irq/528-eth0-Tx	21:20:37	0
400954	99	73	14,5	cyclictest	674	50	irq/528-eth0-Tx	21:20:36	0
400954	99	73	11,7	cyclictest	516031	-21	date	22:05:02	0
400954	99	72	2,54	cyclictest	0	-21	swapper/0	22:05:26	0
400954	99	72	1,54	cyclictest	590629	-21	telnet	23:05:18	0
400968	99	71	63,5	cyclictest	587511	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	23:47:06	3
400963	99	71	58,7	cyclictest	449727	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	21:48:02	2
400954	99	71	25,30	cyclictest	415003	-21	seq	19:44:27	0
400954	99	71	1,67	cyclictest	0	-21	swapper/0	20:13:33	0
400954	99	71	1,5	cyclictest	674	50	irq/528-eth0-Tx	23:47:06	0
400973	99	70	61,5	cyclictest	526231	-21	ssh	22:13:05	4
400963	99	70	56,7	cyclictest	461510	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:03:03	2
400954	99	70	20,4	cyclictest	635797	-21	kworker/u12:1+rpciod@ rpc_async_schedule	00:00:25	0
400954	99	70	1,3	cyclictest	0	-21	swapper/0	20:32:09	0
400959	99	69	57,6	cyclictest	687490	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	00:28:36	1
400954	99	69	26,29	cyclictest	674	50	irq/528-eth0-Tx	22:10:20	0
400954	99	69	22,3	cyclictest	0	-21	swapper/0	19:25:19	0
400954	99	69	1,9	cyclictest	0	-21	swapper/0	21:40:36	0
400954	99	69	19,3	cyclictest	0	-21	swapper/0	23:50:12	0
400954	99	69	1,5	cyclictest	573242	-21	/usr/sbin/munin	22:50:40	0
400954	99	69	1,52	cyclictest	0	-21	swapper/0	19:45:19	0
400954	99	69	1,52	cyclictest	0	-21	swapper/0	19:45:19	0
400954	99	69	1,47	cyclictest	0	-21	swapper/0	23:36:09	0
400954	99	69	14,44	cyclictest	477889	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	21:38:14	0
400954	99	69	1,3	cyclictest	0	-21	swapper/0	22:24:49	0
400954	99	69	1,3	cyclictest	0	-21	swapper/0	21:08:07	0
400954	99	69	11,4	cyclictest	675	50	irq/529-eth0-Tx	22:40:12	0
400954	99	69	11,4	cyclictest	675	50	irq/529-eth0-Tx	22:40:12	0
400954	99	69	10,9	cyclictest	450887	-21	processes	21:10:41	0
400978	99	68	60,4	cyclictest	0	-21	swapper/5	22:50:59	5
400973	99	68	56,6	cyclictest	0	-21	swapper/4	23:15:46	4
400963	99	68	36,7	cyclictest	539392	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:30:33	2
400959	99	68	11,7	cyclictest	0	-21	swapper/1	00:10:01	1
400959	99	68	11,7	cyclictest	0	-21	swapper/1	00:10:00	1
400954	99	68	2,49	cyclictest	668206	-21	ssh	00:07:48	0
400954	99	68	2,49	cyclictest	668206	-21	ssh	00:07:48	0
400954	99	68	22,3	cyclictest	0	-21	swapper/0	23:58:09	0
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17