OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #3, slot #4

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance

Characteristics of the 100 highest latencies:

System rack3slot4.osadl.org (updated Fri Feb 06, 2026 00:46:44)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
2504212	2	135	0,6	sleep2	0	-21	swapper/2	22:57:52	2
2504212	2	135	0,6	sleep2	0	-21	swapper/2	22:57:51	2
2322436	99	124	111,7	cyclictest	2522731	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:12:42	2
2322430	99	123	107,8	cyclictest	2448326	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:27:35	1
2322425	99	119	2,91	cyclictest	2372010	-21	ssh	21:10:27	0
2322425	99	118	18,59	cyclictest	2563836	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	23:55:18	0
2322436	99	116	103,6	cyclictest	2529519	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	23:22:44	2
2322446	99	115	101,8	cyclictest	2423268	-21	ssh	21:52:27	4
2322446	99	115	101,8	cyclictest	2423268	-21	ssh	21:52:27	4
2322430	99	115	94,8	cyclictest	2376912	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	21:37:26	1
2322425	99	115	1,4	cyclictest	0	-21	swapper/0	20:55:25	0
2322425	99	114	26,48	cyclictest	690	50	irq/528-eth0-Tx	21:35:29	0
2322436	99	109	85,9	cyclictest	2609139	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:27:54	2
2322430	99	109	96,6	cyclictest	2599068	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	00:22:53	1
2322430	99	109	96,6	cyclictest	2599068	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	00:22:52	1
2322436	99	106	93,6	cyclictest	2357757	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	21:22:21	2
2322430	99	104	92,6	cyclictest	2538595	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	23:27:44	1
2322430	99	104	92,6	cyclictest	2538595	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	23:27:44	1
2322430	99	103	82,8	cyclictest	2386684	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:22:35	1
2322446	99	102	95,2	cyclictest	0	-21	swapper/4	23:52:48	4
2322425	99	100	4,67	cyclictest	2495161	-21	kworker/u12:1+rpciod@ rpc_async_schedule	22:55:29	0
2322425	99	100	4,67	cyclictest	2495161	-21	kworker/u12:1+rpciod@ rpc_async_schedule	22:55:29	0
2322436	99	99	69,7	cyclictest	2609139	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:38:34	2
2322425	99	99	31,60	cyclictest	2355116	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	20:30:35	0
2322425	99	99	31,60	cyclictest	2355116	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	20:30:35	0
2322425	99	99	2,6	cyclictest	2625885	-21	munin-run	00:35:01	0
2322425	99	98	1,80	cyclictest	2506640	-21	ssh	23:00:04	0
2322425	99	98	1,80	cyclictest	2506640	-21	ssh	23:00:04	0
2322425	99	98	1,70	cyclictest	0	-21	swapper/0	22:24:50	0
2322430	99	97	85,6	cyclictest	2386684	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:37:37	1
2322425	99	96	1,62	cyclictest	0	-21	swapper/0	22:31:39	0
2322441	99	94	72,9	cyclictest	2480083	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	22:40:53	3
2322441	99	94	72,9	cyclictest	2480083	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	22:40:52	3
2322436	99	94	80,7	cyclictest	2372806	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	21:11:35	2
2322430	99	94	80,7	cyclictest	2336419	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	20:02:10	1
2322430	99	94	80,7	cyclictest	2336419	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	20:02:10	1
2322425	99	94	58,7	cyclictest	2348491	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	20:15:37	0
2322425	99	93	72,7	cyclictest	2448326	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:17:34	0
2322425	99	93	53,14	cyclictest	0	-21	swapper/0	21:45:40	0
2322425	99	93	2,73	cyclictest	0	-21	swapper/0	22:07:31	0
2322425	99	93	1,81	cyclictest	0	-21	swapper/0	22:39:36	0
2322425	99	93	1,72	cyclictest	2353414	-21	if_eth1	20:25:23	0
2322425	99	92	71,9	cyclictest	690	50	irq/528-eth0-Tx	20:35:23	0
2322425	99	92	68,13	cyclictest	689	50	irq/527-eth0-Tx	23:07:41	0
2322425	99	92	63,17	cyclictest	690	50	irq/528-eth0-Tx	20:20:21	0
2322425	99	92	1,3	cyclictest	0	-21	swapper/0	23:28:23	0
2322425	99	92	1,3	cyclictest	0	-21	swapper/0	23:28:22	0
2322446	99	91	77,8	cyclictest	2468047	-21	ssh	22:28:15	4
2322441	99	91	78,6	cyclictest	2623575	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	00:32:53	3
2322436	99	91	77,7	cyclictest	2563836	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	00:30:16	2
2322425	99	91	73,4	cyclictest	689	50	irq/527-eth0-Tx	20:50:43	0
2322425	99	91	70,5	cyclictest	689	50	irq/527-eth0-Tx	22:00:20	0
2322425	99	91	1,79	cyclictest	0	-21	swapper/0	23:39:05	0
2322425	99	91	1,75	cyclictest	0	-21	swapper/0	21:25:10	0
2322425	99	91	1,73	cyclictest	2565755	-21	users	23:45:49	0
2322446	99	89	82,4	cyclictest	0	-21	swapper/4	19:42:07	4
2322430	99	88	76,6	cyclictest	2386684	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:26:34	1
2322430	99	88	74,7	cyclictest	2357757	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	20:45:34	1
2322430	99	88	74,7	cyclictest	2357757	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	20:45:34	1
2322425	99	88	70,4	cyclictest	690	50	irq/528-eth0-Tx	20:40:15	0
2322425	99	88	54,18	cyclictest	0	-21	swapper/0	23:10:30	0
2322425	99	88	1,75	cyclictest	2421385	-21	chrt	21:50:43	0
2322425	99	88	1,75	cyclictest	2421385	-21	chrt	21:50:42	0
2322425	99	88	1,72	cyclictest	2362888	-21	cat	20:50:01	0
2322425	99	88	1,72	cyclictest	2362888	-21	cat	20:50:01	0
2322430	99	87	73,7	cyclictest	2386684	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:55:23	1
2322430	99	87	73,7	cyclictest	2386684	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:55:23	1
2322430	99	87	73,7	cyclictest	2355116	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	20:33:34	1
2322430	99	87	73,7	cyclictest	2355116	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	20:33:34	1
2322425	99	87	2,71	cyclictest	2326677	-21	/usr/sbin/munin	19:20:45	0
2322425	99	87	1,74	cyclictest	15	-21	rcuc/0	20:10:41	0
2322446	99	86	79,3	cyclictest	0	-21	swapper/4	22:51:36	4
2322446	99	86	79,3	cyclictest	0	-21	swapper/4	22:51:35	4
2322436	99	86	72,7	cyclictest	2348491	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	20:15:35	2
2322436	99	86	64,6	cyclictest	2585773	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:07:23	2
2322430	99	86	74,6	cyclictest	2386684	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:43:34	1
2322430	99	86	72,7	cyclictest	2352327	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	20:41:45	1
2322425	99	86	59,17	cyclictest	689	50	irq/527-eth0-Tx	21:30:40	0
2322425	99	86	59,14	cyclictest	0	-21	swapper/0	21:55:19	0
2322425	99	86	59,14	cyclictest	0	-21	swapper/0	21:55:18	0
2322425	99	86	58,14	cyclictest	0	-21	swapper/0	19:45:38	0
2322425	99	86	54,17	cyclictest	0	-21	swapper/0	00:09:00	0
2322425	99	86	2,70	cyclictest	0	-21	swapper/0	19:40:33	0
2322425	99	86	1,70	cyclictest	2370854	-21	diskmemload	00:00:43	0
2322425	99	86	1,62	cyclictest	0	-21	swapper/0	22:49:50	0
2322446	99	85	77,4	cyclictest	0	-21	swapper/4	19:47:07	4
2322436	99	85	73,6	cyclictest	2395375	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	21:31:45	2
2322430	99	85	71,7	cyclictest	2522731	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:34:21	1
2322430	99	85	71,7	cyclictest	2522731	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:34:21	1
2322430	99	85	71,7	cyclictest	2495161	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:57:38	1
2322430	99	85	71,7	cyclictest	2495161	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:57:38	1
2322430	99	85	71,7	cyclictest	2324323	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	19:17:07	1
2322430	99	85	62,9	cyclictest	2386684	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:32:35	1
2322430	99	85	59,9	cyclictest	2434442	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	22:06:18	1
2322425	99	85	55,20	cyclictest	689	50	irq/527-eth0-Tx	19:25:15	0
2322425	99	85	55,20	cyclictest	689	50	irq/527-eth0-Tx	19:25:15	0
2322425	99	85	1,63	cyclictest	2370854	-21	diskmemload	21:42:12	0
2322436	99	84	71,6	cyclictest	2386684	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:11:35	2
2322430	99	84	72,6	cyclictest	2386684	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:47:27	1
2322430	99	84	71,7	cyclictest	2434442	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	22:16:24	1
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17