OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #3, slot #4

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance

Characteristics of the 100 highest latencies:

System rack3slot4.osadl.org (updated Fri Apr 19, 2024 12:46:38)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
3008019	2	3571	3539,17	sleep4	0	-21	swapper/4	07:05:22	4
3009539	99	138	114,16	cyclictest	3139287	-21	kworker/u12:4+events_unbound@ flush_memcg_stats_dwork	10:43:42	0
3009555	99	131	117,7	cyclictest	3131202	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	10:35:04	2
3009578	99	130	115,9	cyclictest	3153859	-21	rm	10:43:42	4
3009539	99	121	93,20	cyclictest	3195323	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	12:20:38	0
3009544	99	119	105,7	cyclictest	3242946	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	12:05:34	1
3009539	99	115	67,40	cyclictest	3195323	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	12:10:33	0
3009566	99	112	95,8	cyclictest	3229735	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	11:55:20	3
3009555	99	109	94,7	cyclictest	3195323	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	11:25:13	2
3009539	99	107	63,24	cyclictest	3204854	-21	kworker/0:0+events@ igb_watchdog_task	12:10:01	0
3009539	99	106	82,15	cyclictest	3137526	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	10:28:41	0
3009539	99	106	28,31	cyclictest	0	-21	swapper/0	11:15:31	0
3009539	99	103	60,6	cyclictest	210	-21	kswapd0	11:40:06	0
3009539	99	103	30,6	cyclictest	663	50	irq/530-eth0-Tx	11:00:39	0
3009539	99	103	1,64	cyclictest	3050181	-21	kworker/0:0+events@ igb_watchdog_task	09:39:56	0
3009539	99	103	1,64	cyclictest	3050181	-21	kworker/0:0+events@ igb_watchdog_task	09:39:56	0
3009539	99	102	70,24	cyclictest	3078707	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	10:08:35	0
3009539	99	102	39,4	cyclictest	0	-21	swapper/0	11:25:00	0
3009578	99	101	94,3	cyclictest	0	-21	swapper/4	12:20:37	4
3009544	99	100	87,6	cyclictest	3045551	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	08:39:36	1
3009539	99	100	24,6	cyclictest	663	50	irq/530-eth0-Tx	12:30:32	0
3009539	99	97	44,16	cyclictest	3021619	-21	kworker/0:0+events@ igb_watchdog_task	07:55:28	0
3009539	99	97	44,16	cyclictest	3021619	-21	kworker/0:0+events@ igb_watchdog_task	07:55:28	0
3009539	99	97	32,41	cyclictest	663	50	irq/530-eth0-Tx	11:35:32	0
3009539	99	97	26,15	cyclictest	0	-21	swapper/0	11:50:14	0
3009539	99	96	69,19	cyclictest	3212753	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	11:45:24	0
3009566	99	95	82,6	cyclictest	3131202	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	10:38:43	3
3009539	99	95	45,18	cyclictest	13	-21	ksoftirqd/0	12:15:46	0
3009539	99	94	45,17	cyclictest	13	-21	ksoftirqd/0	11:10:16	0
3009539	99	93	21,34	cyclictest	2980203	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	07:45:10	0
3128637	2	91	0,6	sleep2	29	-21	rcuc/2	10:20:01	2
3009566	99	91	77,7	cyclictest	3137526	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	10:33:43	3
3009544	99	91	78,6	cyclictest	3114087	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	10:20:06	1
3009539	99	91	48,27	cyclictest	3139287	-21	kworker/u12:4+events_unbound@ flush_memcg_stats_dwork	10:45:07	0
3072190	2	90	0,2	sleep4	0	-21	swapper/4	09:24:55	4
3009578	99	90	83,3	cyclictest	0	-21	swapper/4	10:28:41	4
3009544	99	90	76,7	cyclictest	3155221	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	10:48:44	1
3009539	99	90	56,6	cyclictest	3050181	-21	kworker/0:0+events@ igb_watchdog_task	10:55:12	0
3009544	99	89	76,6	cyclictest	3009124	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	07:29:25	1
3009566	99	88	71,8	cyclictest	3195323	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	11:40:27	3
3009544	99	87	55,5	cyclictest	3251578	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	12:20:28	1
3009539	99	87	63,16	cyclictest	3039699	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	08:24:33	0
3009539	99	86	55,14	cyclictest	3050181	-21	kworker/0:0+events@ igb_watchdog_task	08:50:44	0
3009566	99	85	70,7	cyclictest	3102593	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	10:03:33	3
3009539	99	85	41,18	cyclictest	3197617	-21	kworker/0:2+events@ igb_watchdog_task	11:30:02	0
3009539	99	85	41,18	cyclictest	3197617	-21	kworker/0:2+events@ igb_watchdog_task	11:30:02	0
3009539	99	84	39,16	cyclictest	3050181	-21	kworker/0:0+events@ igb_watchdog_task	09:01:46	0
3009539	99	84	39,16	cyclictest	3050181	-21	kworker/0:0+events@ igb_watchdog_task	09:01:45	0
3009578	99	83	68,8	cyclictest	3249429	-21	ssh	12:10:33	4
3009555	99	83	69,7	cyclictest	3078707	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	09:38:29	2
3009555	99	83	69,7	cyclictest	3078707	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	09:38:29	2
3009555	99	83	68,7	cyclictest	3195323	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	11:55:32	2
3009555	99	83	35,16	cyclictest	3057928	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	09:24:13	2
3009539	99	83	64,15	cyclictest	3071151	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	09:45:20	0
3009539	99	83	53,15	cyclictest	3204854	-21	kworker/0:0+events@ igb_watchdog_task	12:25:43	0
3009539	99	83	30,25	cyclictest	13	-21	ksoftirqd/0	09:55:16	0
3225965	2	82	0,5	sleep1	0	-21	swapper/1	11:49:17	1
3009555	99	82	68,6	cyclictest	3125464	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	10:25:01	2
3009544	99	81	67,6	cyclictest	3021489	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	07:58:09	1
3009544	99	81	67,6	cyclictest	3021489	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	07:58:08	1
3009539	99	80	55,6	cyclictest	3114087	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	10:15:16	0
3009539	99	80	21,15	cyclictest	0	-21	swapper/0	09:25:18	0
3009578	99	79	73,2	cyclictest	0	-21	swapper/4	12:30:38	4
3009539	99	79	36,17	cyclictest	3050181	-21	kworker/0:0+events@ igb_watchdog_task	11:05:30	0
3009539	99	79	34,15	cyclictest	3050181	-21	kworker/0:0+events@ igb_watchdog_task	09:43:34	0
3009578	99	78	71,3	cyclictest	0	-21	swapper/4	11:45:24	4
3009539	99	78	38,16	cyclictest	3021619	-21	kworker/0:0+events@ igb_watchdog_task	08:05:42	0
3009578	99	77	71,3	cyclictest	0	-21	swapper/4	10:08:35	4
3009566	99	77	31,9	cyclictest	3139287	-21	kworker/u12:4+events_unbound@ flush_memcg_stats_dwork	11:02:37	3
3009539	99	77	46,4	cyclictest	3041784	-21	kworker/0:1+events@ igb_watchdog_task	08:35:38	0
3009539	99	77	37,15	cyclictest	3050181	-21	kworker/0:0+events@ igb_watchdog_task	09:50:20	0
3009539	99	77	35,17	cyclictest	3050181	-21	kworker/0:0+events@ igb_watchdog_task	10:20:10	0
3009539	99	76	48,5	cyclictest	3021619	-21	kworker/0:0+events@ igb_watchdog_task	08:10:43	0
3009539	99	76	46,5	cyclictest	3050181	-21	kworker/0:0+events@ igb_watchdog_task	10:10:38	0
3009539	99	76	30,16	cyclictest	3009098	-21	kworker/0:2+events@ igb_watchdog_task	07:30:01	0
3009578	99	75	64,6	cyclictest	3248180	-21	ssh	12:10:01	4
3009555	99	75	60,7	cyclictest	3139287	-21	kworker/u12:4+events_unbound@ flush_memcg_stats_dwork	11:10:10	2
3009539	99	75	52,7	cyclictest	3053840	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	09:09:40	0
3009539	99	75	52,7	cyclictest	3053840	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	09:09:39	0
3009539	99	75	44,5	cyclictest	3050181	-21	kworker/0:0+events@ igb_watchdog_task	09:17:52	0
3009539	99	75	44,5	cyclictest	3050181	-21	kworker/0:0+events@ igb_watchdog_task	09:17:52	0
3009566	99	74	61,6	cyclictest	3125464	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	10:25:07	3
3009566	99	74	59,7	cyclictest	3195323	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	12:15:35	3
3009544	99	74	60,7	cyclictest	3269291	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	12:30:30	1
3009544	99	74	59,7	cyclictest	3031541	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	08:03:11	1
3009544	99	74	38,7	cyclictest	3251578	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	12:39:43	1
3009544	99	74	38,7	cyclictest	3251578	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	12:39:42	1
3009539	99	74	47,4	cyclictest	3050181	-21	kworker/0:0+events@ igb_watchdog_task	09:10:48	0
3009539	99	74	47,4	cyclictest	3050181	-21	kworker/0:0+events@ igb_watchdog_task	09:10:47	0
3009539	99	74	31,14	cyclictest	0	-21	swapper/0	11:25:29	0
3009578	99	73	67,2	cyclictest	0	-21	swapper/4	09:35:09	4
3009578	99	73	67,2	cyclictest	0	-21	swapper/4	09:35:08	4
3009555	99	73	59,7	cyclictest	3251578	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	12:26:00	2
3009539	99	73	43,4	cyclictest	3204854	-21	kworker/0:0+events@ igb_watchdog_task	12:00:02	0
3009539	99	73	29,15	cyclictest	3021619	-21	kworker/0:0+events@ igb_watchdog_task	07:55:02	0
3217164	2	72	0,4	chrt	0	-21	swapper/5	11:40:33	5
3009578	99	72	66,3	cyclictest	0	-21	swapper/4	09:45:20	4
3009578	99	72	64,4	cyclictest	0	-21	swapper/4	08:24:33	4
3009586	99	71	64,3	cyclictest	0	-21	swapper/5	11:55:24	5
3009555	99	71	56,6	cyclictest	3216221	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	11:50:24	2
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17