OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #3, slot #4

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance

Characteristics of the 100 highest latencies:

System rack3slot4.osadl.org (updated Sat May 02, 2026 00:46:44)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
2712881	2	3568	3554,9	sleep5	0	-21	swapper/5	19:09:06	5
2712881	2	3568	3554,9	sleep5	0	-21	swapper/5	19:09:05	5
2712000	2	1467	1450,7	sleep1	2684955	-21	kworker/1:0+events@ dbs_work_handler	19:05:28	1
2712000	2	1467	1450,7	sleep1	2684955	-21	kworker/1:0+events@ dbs_work_handler	19:05:28	1
2713128	99	130	118,6	cyclictest	2743354	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	20:21:54	3
2713128	99	120	106,7	cyclictest	2936317	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	23:47:02	3
2713117	99	118	104,7	cyclictest	2724983	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	19:41:51	1
2713112	99	117	34,49	cyclictest	2921813	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:29:05	0
2713112	99	117	34,49	cyclictest	2921813	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:29:05	0
2713112	99	115	77,12	cyclictest	690	50	irq/528-eth0-Tx	00:10:33	0
2713128	99	110	96,7	cyclictest	2745530	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:45:35	3
2713128	99	101	85,8	cyclictest	2972298	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	00:05:33	3
2713112	99	101	78,16	cyclictest	2743366	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	20:28:24	0
2808995	2	98	0,7	sleep0	0	-21	swapper/0	21:48:46	0
2713112	99	96	2,88	cyclictest	2823192	-21	/usr/sbin/munin	22:00:16	0
2713112	99	95	46,18	cyclictest	2782253	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	21:28:57	0
2713112	99	95	46,18	cyclictest	2782253	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	21:28:56	0
2713112	99	95	1,79	cyclictest	0	-21	swapper/0	21:35:33	0
2713112	99	95	1,79	cyclictest	0	-21	swapper/0	21:35:32	0
2713112	99	94	81,4	cyclictest	690	50	irq/528-eth0-Tx	22:51:45	0
2713112	99	94	69,15	cyclictest	691	50	irq/529-eth0-Tx	20:05:28	0
2713112	99	93	75,4	cyclictest	690	50	irq/528-eth0-Tx	23:00:21	0
2713112	99	93	68,15	cyclictest	689	50	irq/527-eth0-Tx	21:57:48	0
2713112	99	93	1,80	cyclictest	0	-21	swapper/0	22:55:25	0
2713112	99	92	68,15	cyclictest	689	50	irq/527-eth0-Tx	00:22:13	0
2713112	99	92	68,15	cyclictest	689	50	irq/527-eth0-Tx	00:22:13	0
2713112	99	92	68,12	cyclictest	689	50	irq/527-eth0-Tx	20:10:32	0
2713112	99	92	67,15	cyclictest	689	50	irq/527-eth0-Tx	23:44:07	0
2713112	99	92	66,16	cyclictest	690	50	irq/528-eth0-Tx	22:31:39	0
2713112	99	92	66,16	cyclictest	690	50	irq/528-eth0-Tx	22:31:38	0
2713112	99	92	61,14	cyclictest	0	-21	swapper/0	19:25:36	0
2713112	99	92	61,14	cyclictest	0	-21	swapper/0	19:25:36	0
2713112	99	91	71,5	cyclictest	2754740	-21	memory	20:50:23	0
2713112	99	90	72,4	cyclictest	691	50	irq/529-eth0-Tx	00:15:40	0
2713112	99	90	69,9	cyclictest	690	50	irq/528-eth0-Tx	23:11:04	0
2713112	99	89	71,4	cyclictest	0	-21	swapper/0	19:17:23	0
2713112	99	89	1,76	cyclictest	0	-21	swapper/0	19:20:35	0
2713112	99	88	70,4	cyclictest	690	50	irq/528-eth0-Tx	23:45:15	0
2713112	99	88	67,6	cyclictest	2770597	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	21:18:17	0
2713112	99	88	65,12	cyclictest	690	50	irq/528-eth0-Tx	23:30:15	0
2713112	99	88	64,15	cyclictest	690	50	irq/528-eth0-Tx	00:37:29	0
2713112	99	88	64,15	cyclictest	690	50	irq/528-eth0-Tx	00:37:29	0
2713112	99	88	63,6	cyclictest	2760850	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	21:50:08	0
2713112	99	87	62,4	cyclictest	0	-21	swapper/0	00:35:01	0
2713112	99	87	2,70	cyclictest	0	-21	swapper/0	22:40:11	0
2713112	99	87	2,70	cyclictest	0	-21	swapper/0	22:40:11	0
2713112	99	86	67,4	cyclictest	689	50	irq/527-eth0-Tx	22:06:49	0
2713117	99	85	71,7	cyclictest	2915475	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:22:23	1
2713117	99	85	71,7	cyclictest	2915475	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:22:23	1
2713112	99	85	67,4	cyclictest	691	50	irq/529-eth0-Tx	23:23:53	0
2713112	99	85	67,4	cyclictest	691	50	irq/529-eth0-Tx	23:23:53	0
2713112	99	85	64,4	cyclictest	690	50	irq/528-eth0-Tx	20:45:38	0
2713112	99	85	48,18	cyclictest	0	-21	swapper/0	00:00:36	0
2713112	99	85	1,69	cyclictest	0	-21	swapper/0	19:35:33	0
2713112	99	84	67,3	cyclictest	0	-21	swapper/0	23:19:35	0
2713112	99	84	63,4	cyclictest	690	50	irq/528-eth0-Tx	20:30:31	0
2713112	99	84	62,4	cyclictest	689	50	irq/527-eth0-Tx	22:14:02	0
2713112	99	84	59,4	cyclictest	691	50	irq/529-eth0-Tx	00:05:19	0
2713112	99	84	56,14	cyclictest	0	-21	swapper/0	20:40:30	0
2713112	99	84	45,15	cyclictest	0	-21	swapper/0	21:40:05	0
2713112	99	83	65,4	cyclictest	0	-21	swapper/0	22:20:29	0
2713112	99	83	62,4	cyclictest	690	50	irq/528-eth0-Tx	21:30:44	0
2713112	99	83	62,4	cyclictest	690	50	irq/528-eth0-Tx	21:30:44	0
2713112	99	83	56,3	cyclictest	0	-21	swapper/0	22:45:32	0
2713112	99	83	3,6	cyclictest	2727289	-21	cat	19:45:00	0
2713112	99	83	2,3	cyclictest	0	-21	swapper/0	20:00:38	0
2713128	99	82	66,8	cyclictest	2980742	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:25:36	3
2713128	99	82	66,8	cyclictest	2980742	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:25:36	3
2713117	99	82	68,7	cyclictest	2729078	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	19:55:25	1
2713117	99	82	57,9	cyclictest	2885171	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:50:35	1
2713112	99	82	68,3	cyclictest	0	-21	swapper/0	21:00:32	0
2713112	99	82	61,3	cyclictest	0	-21	swapper/0	20:20:31	0
2713112	99	82	1,5	cyclictest	2867467	-21	ssh	22:35:28	0
2713132	99	81	67,7	cyclictest	2765252	-21	diskmemload-4	23:26:54	4
2713132	99	81	67,7	cyclictest	2765252	-21	diskmemload-4	23:26:53	4
2713128	99	81	67,7	cyclictest	2929109	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	23:37:23	3
2713123	99	81	70,5	cyclictest	2849916	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:35:11	2
2713123	99	81	67,7	cyclictest	2713707	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	19:31:50	2
2713112	99	81	52,13	cyclictest	0	-21	swapper/0	19:55:37	0
2713112	99	81	48,17	cyclictest	0	-21	swapper/0	19:49:54	0
2713112	99	81	46,15	cyclictest	0	-21	swapper/0	21:20:31	0
2713112	99	81	1,4	cyclictest	0	-21	swapper/0	21:10:43	0
2713128	99	80	67,6	cyclictest	2865448	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	23:17:23	3
2713128	99	80	54,8	cyclictest	2814465	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:41:38	3
2713128	99	80	54,8	cyclictest	2814465	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:41:37	3
2713123	99	80	67,6	cyclictest	2760850	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	21:30:34	2
2713123	99	80	67,6	cyclictest	2760850	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	21:30:33	2
2713123	99	80	47,8	cyclictest	2980742	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:20:45	2
2713123	99	80	47,8	cyclictest	2980742	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:20:45	2
2713112	99	80	65,4	cyclictest	690	50	irq/528-eth0-Tx	20:55:37	0
2713112	99	80	65,4	cyclictest	690	50	irq/528-eth0-Tx	20:55:36	0
2713112	99	80	54,16	cyclictest	689	50	irq/527-eth0-Tx	23:50:17	0
2713112	99	80	54,16	cyclictest	689	50	irq/527-eth0-Tx	23:50:17	0
2713112	99	80	1,53	cyclictest	2945609	-21	ssh	23:38:21	0
2713117	99	79	65,6	cyclictest	2729078	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	19:53:28	1
2713117	99	79	65,6	cyclictest	2729078	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	19:53:27	1
2713117	99	79	50,7	cyclictest	2823126	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	22:33:46	1
2713117	99	79	50,7	cyclictest	2823126	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	22:33:45	1
2713112	99	79	49,15	cyclictest	0	-21	swapper/0	22:15:27	0
2713112	99	79	49,15	cyclictest	0	-21	swapper/0	22:15:26	0
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17