OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #3, slot #4

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance

Characteristics of the 100 highest latencies:

System rack3slot4.osadl.org (updated Mon Dec 22, 2025 00:46:46)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
1541448	2	2393	2377,6	sleep3	1538593	-21	kworker/3:1+events@ dbs_work_handler	19:05:22	3
1542952	99	163	96,50	cyclictest	1637445	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:28:13	0
1542957	99	137	124,7	cyclictest	1742566	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	23:43:28	1
1542952	99	135	31,82	cyclictest	1550721	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	19:38:39	0
1542952	99	131	31,65	cyclictest	1546652	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	19:46:36	0
1542970	99	130	125,2	cyclictest	0	-21	swapper/4	22:53:17	4
1542952	99	130	32,83	cyclictest	1662415	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:21:47	0
1542952	99	130	32,52	cyclictest	1812600	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	00:32:20	0
1542952	99	129	85,40	cyclictest	15	-21	rcuc/0	21:30:25	0
1542952	99	129	30,69	cyclictest	1585061	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	20:53:00	0
1542952	99	128	103,21	cyclictest	210	-21	kswapd0	22:35:17	0
1846260	2	124	0,4	sleep0	0	-21	swapper/0	00:40:00	0
1542952	99	122	1,75	cyclictest	1633523	-21	/usr/sbin/munin	21:45:11	0
1542952	99	118	87,23	cyclictest	1602691	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	21:28:01	0
1542952	99	118	46,52	cyclictest	692	50	irq/530-eth0-Tx	00:25:01	0
1542961	99	117	96,17	cyclictest	1615868	-21	bingrep	21:30:26	2
1542966	99	116	104,6	cyclictest	1760464	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:38:28	3
1542952	99	116	46,35	cyclictest	1568870	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	20:15:31	0
1542952	99	116	21,4	cyclictest	0	-21	swapper/0	00:10:01	0
1542952	99	116	21,4	cyclictest	0	-21	swapper/0	00:10:00	0
1542966	99	115	103,6	cyclictest	1702035	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	23:13:22	3
1542952	99	115	69,32	cyclictest	1662415	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:50:23	0
1542952	99	115	27,75	cyclictest	1779431	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:48:06	0
1542952	99	115	27,75	cyclictest	1779431	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:48:06	0
1542952	99	115	1,104	cyclictest	0	-21	swapper/0	19:57:55	0
1542952	99	115	1,104	cyclictest	0	-21	swapper/0	19:57:54	0
1542952	99	115	1,102	cyclictest	0	-21	swapper/0	19:22:50	0
1542975	99	114	108,3	cyclictest	0	-21	swapper/5	22:30:02	5
1542970	99	114	107,3	cyclictest	0	-21	swapper/4	00:13:43	4
1542966	99	114	102,6	cyclictest	1602691	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	21:48:03	3
1542952	99	114	8,13	cyclictest	1670715	-21	ssh	22:15:22	0
1542952	99	114	38,59	cyclictest	1779431	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:44:56	0
1542952	99	114	26,57	cyclictest	1550721	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	19:41:27	0
1542952	99	114	26,57	cyclictest	1550721	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	19:41:26	0
1542952	99	114	1,27	cyclictest	0	-21	swapper/0	23:15:38	0
1542957	99	112	103,6	cyclictest	210	-21	kswapd0	23:30:25	1
1542952	99	112	6,49	cyclictest	0	-21	swapper/0	22:30:14	0
1542952	99	111	2,102	cyclictest	691	50	irq/529-eth0-Tx	22:50:01	0
1542952	99	111	2,102	cyclictest	691	50	irq/529-eth0-Tx	22:50:01	0
1542952	99	110	1,98	cyclictest	0	-21	swapper/0	20:01:22	0
1542952	99	110	15,64	cyclictest	1546652	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	19:26:54	0
1542952	99	110	15,64	cyclictest	1546652	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	19:26:54	0
1542975	99	109	104,2	cyclictest	0	-21	swapper/5	21:30:25	5
1542966	99	109	96,6	cyclictest	1587128	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	21:07:56	3
1542952	99	109	26,3	cyclictest	0	-21	swapper/0	23:55:38	0
1542952	99	109	26,3	cyclictest	0	-21	swapper/0	23:55:38	0
1542970	99	108	98,6	cyclictest	1761747	-21	ssh	23:30:01	4
1542966	99	108	96,6	cyclictest	1637445	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:53:05	3
1542966	99	108	96,6	cyclictest	1637445	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:53:04	3
1542952	99	108	65,35	cyclictest	1602691	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:06:29	0
1542952	99	108	65,35	cyclictest	1602691	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:06:29	0
1542952	99	108	1,95	cyclictest	0	-21	swapper/0	20:11:39	0
1542952	99	107	83,16	cyclictest	1637445	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:13:10	0
1542952	99	107	1,95	cyclictest	0	-21	swapper/0	19:32:51	0
1542952	99	107	1,52	cyclictest	0	-21	swapper/0	20:43:17	0
1542975	99	106	100,3	cyclictest	0	-21	swapper/5	21:40:28	5
1542975	99	106	100,3	cyclictest	0	-21	swapper/5	21:40:27	5
1542970	99	106	95,6	cyclictest	1594844	-21	diskmemload-4	22:58:19	4
1542966	99	106	93,7	cyclictest	1742566	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	23:20:13	3
1542966	99	106	92,7	cyclictest	1806159	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:10:02	3
1542952	99	106	5,23	cyclictest	690	50	irq/528-eth0-Tx	23:32:33	0
1542970	99	105	100,2	cyclictest	0	-21	swapper/4	23:33:28	4
1542952	99	105	7,3	cyclictest	0	-21	swapper/0	21:55:34	0
1542952	99	105	2,54	cyclictest	0	-21	swapper/0	00:10:46	0
1542952	99	105	1,93	cyclictest	0	-21	swapper/0	19:17:15	0
1542970	99	103	97,3	cyclictest	0	-21	swapper/4	21:40:28	4
1542970	99	103	97,3	cyclictest	0	-21	swapper/4	21:40:28	4
1542961	99	103	90,7	cyclictest	1602691	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	21:23:00	2
1542952	99	103	2,96	cyclictest	0	-21	swapper/0	21:00:53	0
1542952	99	103	1,80	cyclictest	0	-21	swapper/0	20:23:57	0
1542952	99	103	1,47	cyclictest	0	-21	swapper/0	19:50:59	0
1542952	99	103	1,44	cyclictest	0	-21	swapper/0	20:34:45	0
1542966	99	102	88,7	cyclictest	1591391	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:17:59	3
1542952	99	102	1,78	cyclictest	0	-21	swapper/0	20:39:56	0
1542970	99	101	89,6	cyclictest	1594844	-21	diskmemload-4	23:56:34	4
1542970	99	101	89,6	cyclictest	1594844	-21	diskmemload-4	23:56:34	4
1542961	99	101	91,6	cyclictest	210	-21	kswapd0	23:25:07	2
1542952	99	101	46,3	cyclictest	0	-21	swapper/0	23:20:37	0
1542952	99	101	25,6	cyclictest	1602691	-21	kworker/u12:2+rpciod@ rpc_async_schedule	22:00:20	0
1542952	99	101	1,95	cyclictest	0	-21	swapper/0	21:09:37	0
1542952	99	101	1,75	cyclictest	0	-21	swapper/0	20:06:23	0
1542952	99	101	11,4	cyclictest	691	50	irq/529-eth0-Tx	23:25:37	0
1542966	99	100	87,7	cyclictest	1668991	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:18:10	3
1542952	99	100	52,7	cyclictest	1799036	-21	/usr/sbin/munin	00:00:26	0
1542952	99	100	1,76	cyclictest	0	-21	swapper/0	19:13:32	0
1542970	99	99	93,3	cyclictest	0	-21	swapper/4	00:23:43	4
1542952	99	98	22,4	cyclictest	692	50	irq/530-eth0-Tx	21:50:18	0
1542952	99	98	22,4	cyclictest	692	50	irq/530-eth0-Tx	21:50:18	0
1542975	99	97	90,3	cyclictest	0	-21	swapper/5	00:22:34	5
1542970	99	97	86,6	cyclictest	1594844	-21	diskmemload-4	22:30:02	4
1542952	99	97	2,5	cyclictest	1734238	-21	ssh	23:06:37	0
1542952	99	97	16,55	cyclictest	1725098	-21	latency_hist	23:00:01	0
1542957	99	96	88,5	cyclictest	22	-21	rcuc/1	22:15:16	1
1542957	99	96	87,6	cyclictest	210	-21	kswapd0	22:40:14	1
1542952	99	96	23,6	cyclictest	1769102	-21	sort	23:35:28	0
1542952	99	96	1,57	cyclictest	0	-21	swapper/0	20:59:15	0
1542952	99	96	1,52	cyclictest	1729750	-21	rm	23:03:24	0
1542952	99	96	1,3	cyclictest	0	-21	swapper/0	21:13:04	0
1542975	99	95	89,3	cyclictest	0	-21	swapper/5	23:35:35	5
1542970	99	95	90,2	cyclictest	0	-21	swapper/4	20:02:45	4
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17