OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #3, slot #4

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance

Characteristics of the 100 highest latencies:

System rack3slot4.osadl.org (updated Thu Apr 25, 2024 00:46:32)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
2965411	2	1810	1787,8	sleep0	2964061	-21	kworker/0:0+events@ dbs_work_handler	19:05:21	0
2966927	99	122	106,7	cyclictest	3175600	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	00:18:10	3
2966915	99	119	1,72	cyclictest	3064447	-21	kworker/0:2+events@ igb_watchdog_task	22:25:38	0
3036057	2	116	0,5	sleep1	0	-21	swapper/1	21:33:30	1
3036057	2	116	0,5	sleep1	0	-21	swapper/1	21:33:30	1
2966919	99	115	102,6	cyclictest	3006218	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	20:57:45	1
2966915	99	114	27,5	cyclictest	663	50	irq/530-eth0-Tx	21:50:17	0
2966915	99	114	27,5	cyclictest	663	50	irq/530-eth0-Tx	21:50:16	0
2966915	99	112	33,33	cyclictest	2987482	-21	kworker/u12:0+rpciod@ rpc_async_schedule	20:34:25	0
2966915	99	109	1,32	cyclictest	0	-21	swapper/0	21:40:47	0
2966915	99	106	3,73	cyclictest	2982737	-21	kworker/0:2+events@ igb_watchdog_task	20:10:24	0
2966927	99	101	87,7	cyclictest	2968811	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	19:57:14	3
2966915	99	99	10,70	cyclictest	13	-21	ksoftirqd/0	23:30:31	0
2966915	99	96	48,25	cyclictest	3166144	-21	kworker/0:2+events@ igb_watchdog_task	23:41:52	0
2966915	99	95	2,82	cyclictest	2973999	-21	kworker/0:0+events@ igb_watchdog_task	19:50:44	0
3173373	2	94	0,5	sleep1	0	-21	swapper/1	23:35:00	1
2992370	2	94	0,19	sleep2	30	-21	ksoftirqd/2	20:20:45	2
2966915	99	94	49,28	cyclictest	3004883	-1	kworker/0:1H-events_highpri	21:15:36	0
2966919	99	93	81,6	cyclictest	3051926	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:32:59	1
2966919	99	92	78,7	cyclictest	2968811	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	19:52:13	1
2966915	99	92	52,23	cyclictest	3121711	-21	kworker/0:1+events@ igb_watchdog_task	22:50:44	0
2966915	99	92	45,41	cyclictest	3106491	-1	kworker/0:2H-events_highpri	23:25:30	0
2966915	99	92	41,26	cyclictest	3121711	-21	kworker/0:1+events@ igb_watchdog_task	23:00:01	0
2966915	99	91	44,25	cyclictest	3064447	-21	kworker/0:2+events@ igb_watchdog_task	22:44:32	0
2966915	99	91	38,26	cyclictest	3210563	-21	kworker/0:1+events@ igb_watchdog_task	00:15:28	0
3236144	2	90	0,4	sleep3	0	-21	swapper/3	00:35:27	3
2966919	99	90	77,6	cyclictest	3117576	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:47:59	1
2966915	99	90	26,8	cyclictest	3197763	-21	grep	23:55:46	0
2966915	99	90	26,8	cyclictest	3197763	-21	grep	23:55:46	0
2966915	99	90	18,51	cyclictest	663	50	irq/530-eth0-Tx	00:25:41	0
2966915	99	89	40,24	cyclictest	3121711	-21	kworker/0:1+events@ igb_watchdog_task	23:15:30	0
2966938	99	88	82,2	cyclictest	0	-21	swapper/5	21:22:33	5
2966915	99	88	54,10	cyclictest	3160348	-21	ssh	23:22:53	0
2966915	99	87	63,16	cyclictest	3013977	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:17:52	0
2966915	99	87	46,25	cyclictest	3166144	-21	kworker/0:2+events@ igb_watchdog_task	23:35:46	0
2966915	99	87	27,4	cyclictest	3064447	-21	kworker/0:2+events@ igb_watchdog_task	22:10:01	0
2966915	99	87	24,41	cyclictest	13	-21	ksoftirqd/0	22:35:36	0
2966938	99	86	81,2	cyclictest	0	-21	swapper/5	21:42:41	5
2966927	99	86	73,6	cyclictest	3111496	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	23:03:02	3
2966915	99	86	52,14	cyclictest	2982737	-21	kworker/0:2+events@ igb_watchdog_task	21:10:29	0
2966915	99	86	37,26	cyclictest	2973999	-21	kworker/0:0+events@ igb_watchdog_task	19:40:26	0
2966927	99	85	71,7	cyclictest	2990165	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	20:47:26	3
2966927	99	85	71,7	cyclictest	2990165	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	20:47:25	3
2966915	99	85	39,26	cyclictest	2966446	-21	kworker/0:2+events@ igb_watchdog_task	19:25:02	0
2966933	99	84	70,7	cyclictest	3014402	-21	diskmemload-4	21:35:07	4
2966915	99	84	33,8	cyclictest	3123579	-21	latency_hist	22:50:02	0
2966927	99	83	69,7	cyclictest	3004986	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	20:57:26	3
2966915	99	83	42,16	cyclictest	3013977	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:05:03	0
2966915	99	83	1,40	cyclictest	56	-21	rcuop/5	21:23:24	0
2966915	99	82	1,34	cyclictest	2967874	-21	ntp_kernel_pll_	19:10:28	0
2966915	99	81	45,19	cyclictest	3064447	-21	kworker/0:2+events@ igb_watchdog_task	22:25:01	0
2966915	99	81	3,43	cyclictest	2982737	-21	kworker/0:2+events@ igb_watchdog_task	20:54:46	0
2966915	99	81	29,4	cyclictest	661	50	irq/528-eth0-Tx	21:35:25	0
2966927	99	80	66,7	cyclictest	2968811	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	19:47:37	3
2966927	99	80	66,7	cyclictest	2968811	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	19:47:37	3
2966915	99	80	51,15	cyclictest	3064447	-21	kworker/0:2+events@ igb_watchdog_task	22:30:41	0
2966924	99	79	67,5	cyclictest	2968811	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	19:32:10	2
2966915	99	79	60,15	cyclictest	210	-21	kswapd0	21:25:27	0
2966933	99	78	72,3	cyclictest	0	-21	swapper/4	23:33:07	4
2966919	99	78	65,6	cyclictest	3175600	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	23:36:03	1
2966915	99	78	7,35	cyclictest	662	50	irq/529-eth0-Tx	00:35:15	0
2966915	99	78	55,15	cyclictest	3051926	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:55:04	0
2966915	99	78	55,15	cyclictest	3051926	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:55:04	0
2966915	99	78	47,14	cyclictest	2982737	-21	kworker/0:2+events@ igb_watchdog_task	20:15:34	0
2966915	99	78	37,25	cyclictest	3121711	-21	kworker/0:1+events@ igb_watchdog_task	23:07:20	0
2966915	99	78	2,38	cyclictest	3184789	-21	kworker/0:0+events@ igb_watchdog_task	00:00:44	0
2966915	99	77	47,21	cyclictest	2968811	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	19:45:15	0
2966915	99	77	47,21	cyclictest	2968811	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	19:45:15	0
2966933	99	76	70,3	cyclictest	0	-21	swapper/4	20:50:40	4
2966927	99	76	62,7	cyclictest	3014684	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:32:35	3
2966927	99	76	62,7	cyclictest	3014684	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:32:34	3
2966927	99	76	62,7	cyclictest	2771225	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	19:14:18	3
2966915	99	76	32,4	cyclictest	663	50	irq/530-eth0-Tx	21:45:39	0
2966915	99	76	28,40	cyclictest	2968811	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	19:30:19	0
2966927	99	75	61,7	cyclictest	2990165	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	20:37:22	3
2966919	99	75	61,7	cyclictest	3051926	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:07:55	1
2966919	99	75	61,7	cyclictest	3006218	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	21:12:45	1
2966915	99	75	42,3	cyclictest	0	-21	swapper/0	00:08:54	0
2966915	99	75	1,37	cyclictest	0	-21	swapper/0	21:30:16	0
2966915	99	75	1,37	cyclictest	0	-21	swapper/0	21:30:15	0
2966938	99	74	68,2	cyclictest	0	-21	swapper/5	19:57:36	5
2966933	99	74	69,1	cyclictest	0	-21	swapper/4	23:46:25	4
2966927	99	74	60,7	cyclictest	3004986	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	21:12:30	3
2966915	99	74	42,22	cyclictest	660	50	irq/527-eth0-Tx	23:00:32	0
2966915	99	74	41,4	cyclictest	660	50	irq/527-eth0-Tx	20:05:38	0
2966915	99	74	41,4	cyclictest	660	50	irq/527-eth0-Tx	20:05:38	0
2966915	99	74	36,22	cyclictest	2982737	-21	kworker/0:2+events@ igb_watchdog_task	20:46:24	0
2966915	99	74	36,22	cyclictest	2982737	-21	kworker/0:2+events@ igb_watchdog_task	20:46:24	0
2966915	99	74	31,13	cyclictest	3184789	-21	kworker/0:0+events@ igb_watchdog_task	23:48:33	0
2966915	99	74	28,16	cyclictest	3064447	-21	kworker/0:2+events@ igb_watchdog_task	22:14:48	0
2966938	99	73	67,2	cyclictest	0	-21	swapper/5	19:20:30	5
2966927	99	73	59,7	cyclictest	2990165	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	20:27:21	3
2966927	99	73	59,7	cyclictest	2990165	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	20:27:21	3
2966915	99	73	29,5	cyclictest	3210563	-21	kworker/0:1+events@ igb_watchdog_task	00:10:24	0
2966915	99	72	41,4	cyclictest	660	50	irq/527-eth0-Tx	20:20:24	0
2966933	99	71	65,2	cyclictest	0	-21	swapper/4	00:07:58	4
2966919	99	71	56,7	cyclictest	3160504	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:50:09	1
2966919	99	71	56,7	cyclictest	3160504	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:50:08	1
2966915	99	71	52,15	cyclictest	2987482	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	20:42:23	0
2966915	99	71	31,7	cyclictest	13	-21	ksoftirqd/0	00:23:22	0
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17