OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #3, slot #4

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance

Characteristics of the 100 highest latencies:

System rack3slot4.osadl.org (updated Fri Oct 17, 2025 12:46:51)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
886275	99	146	131,8	cyclictest	907791	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	08:10:18	2
886275	99	146	131,8	cyclictest	907791	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	08:10:18	2
886275	99	144	130,7	cyclictest	1066525	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	11:20:56	2
886267	99	144	51,70	cyclictest	933713	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	09:10:13	0
886267	99	144	51,70	cyclictest	933713	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	09:10:13	0
886267	99	125	100,17	cyclictest	896064	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	08:10:20	0
886267	99	125	100,17	cyclictest	896064	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	08:10:20	0
886015	2	125	111,6	sleep1	885806	-21	kworker/1:3+events@ dbs_work_handler	07:08:46	1
886278	99	124	110,7	cyclictest	933713	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	09:25:37	3
886275	99	123	109,7	cyclictest	922754	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	08:50:26	2
886275	99	123	109,7	cyclictest	922754	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	08:50:26	2
886275	99	121	108,6	cyclictest	1081646	-21	kworker/u12:4+events_unbound@ flush_memcg_stats_dwork	11:41:01	2
886275	99	121	108,6	cyclictest	1081646	-21	kworker/u12:4+events_unbound@ flush_memcg_stats_dwork	11:41:00	2
886278	99	120	99,9	cyclictest	1088724	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	12:06:03	3
886278	99	120	99,9	cyclictest	1088724	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	12:06:02	3
886275	99	120	106,7	cyclictest	1033826	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	10:55:52	2
886267	99	120	94,7	cyclictest	896064	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	08:45:25	0
886267	99	120	82,16	cyclictest	1003656	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	11:10:55	0
886267	99	120	82,16	cyclictest	1003656	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	11:10:54	0
886267	99	120	81,17	cyclictest	951657	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	09:35:33	0
886278	99	118	106,6	cyclictest	945515	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	09:30:38	3
886278	99	118	105,6	cyclictest	896064	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	08:05:16	3
886278	99	118	105,6	cyclictest	896064	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	08:05:15	3
886283	99	116	110,3	cyclictest	0	-21	swapper/4	11:56:03	4
886275	99	114	101,7	cyclictest	945515	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	10:20:42	2
886278	99	113	99,7	cyclictest	935909	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	09:50:41	3
886278	99	110	98,6	cyclictest	1033826	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	10:50:52	3
886275	99	110	96,7	cyclictest	1088724	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	11:51:03	2
884983	2	109	2,6	sleep3	0	-21	swapper/3	07:05:21	3
886283	99	104	98,3	cyclictest	0	-21	swapper/4	08:40:24	4
886283	99	104	98,3	cyclictest	0	-21	swapper/4	08:40:24	4
886275	99	103	91,6	cyclictest	1125223	-21	kworker/u12:4+events_unbound@ flush_memcg_stats_dwork	12:26:07	2
886283	99	100	94,3	cyclictest	0	-21	swapper/4	11:05:53	4
886288	99	99	93,3	cyclictest	0	-21	swapper/5	11:56:02	5
886278	99	99	85,7	cyclictest	1066525	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	11:20:57	3
886275	99	99	86,7	cyclictest	1003656	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	11:15:56	2
886275	99	99	85,7	cyclictest	951657	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	09:55:38	2
963027	2	97	0,8	sleep1	963025	-21	ssh	09:40:36	1
886288	99	96	91,2	cyclictest	0	-21	swapper/5	09:15:37	5
886275	99	96	83,6	cyclictest	1081113	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	11:35:59	2
886275	99	96	83,6	cyclictest	1081113	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	11:35:58	2
886267	99	94	70,6	cyclictest	896064	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	08:20:20	0
886267	99	94	70,6	cyclictest	896064	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	08:20:20	0
886278	99	93	83,4	cyclictest	896064	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	08:40:22	3
886278	99	93	83,4	cyclictest	896064	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	08:40:21	3
886267	99	91	30,4	cyclictest	645	50	irq/530-eth0-Tx	10:13:56	0
886267	99	90	18,21	cyclictest	0	-21	swapper/0	10:50:00	0
886271	99	88	74,7	cyclictest	922754	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	09:20:38	1
886278	99	87	75,6	cyclictest	945515	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	10:20:44	3
1011911	2	87	0,5	chrt	0	-21	swapper/0	10:27:26	0
886267	99	86	10,7	cyclictest	1112801	-21	/usr/sbin/munin	12:05:20	0
886267	99	86	10,7	cyclictest	1112801	-21	/usr/sbin/munin	12:05:20	0
886288	99	85	81,2	cyclictest	0	-21	swapper/5	08:40:24	5
886288	99	85	81,2	cyclictest	0	-21	swapper/5	08:40:24	5
1134901	2	85	0,7	sleep1	1134898	-21	ssh	12:24:53	1
1036467	2	85	0,5	chrt	0	-21	swapper/0	10:50:34	0
886267	99	84	72,4	cyclictest	642	50	irq/527-eth0-Tx	09:21:27	0
886283	99	83	77,3	cyclictest	0	-21	swapper/4	09:20:30	4
886267	99	83	2,6	cyclictest	956408	-21	runrttasks	09:34:31	0
886288	99	82	77,2	cyclictest	0	-21	swapper/5	11:05:53	5
886283	99	81	74,3	cyclictest	0	-21	swapper/4	11:00:30	4
886283	99	81	74,3	cyclictest	0	-21	swapper/4	11:00:30	4
886278	99	81	72,4	cyclictest	1003656	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	11:10:56	3
886278	99	81	72,4	cyclictest	1003656	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	11:10:56	3
886275	99	81	67,7	cyclictest	997692	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	10:40:50	2
886267	99	81	28,4	cyclictest	644	50	irq/529-eth0-Tx	12:16:35	0
886278	99	80	68,6	cyclictest	1072746	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	11:25:57	3
886267	99	80	29,4	cyclictest	642	50	irq/527-eth0-Tx	09:25:35	0
886288	99	79	73,3	cyclictest	0	-21	swapper/5	09:35:37	5
886283	99	79	74,2	cyclictest	0	-21	swapper/4	11:10:55	4
886283	99	79	74,2	cyclictest	0	-21	swapper/4	11:10:54	4
886275	99	78	65,6	cyclictest	1125223	-21	kworker/u12:4+events_unbound@ flush_memcg_stats_dwork	12:16:05	2
886267	99	78	28,6	cyclictest	644	50	irq/529-eth0-Tx	09:15:41	0
941722	2	77	0,4	sleep1	0	-21	swapper/1	09:19:30	1
886283	99	77	73,2	cyclictest	0	-21	swapper/4	12:01:02	4
886283	99	77	73,2	cyclictest	0	-21	swapper/4	12:01:02	4
886283	99	77	70,3	cyclictest	0	-21	swapper/4	09:55:41	4
886275	99	77	68,6	cyclictest	1003656	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	10:32:57	2
886278	99	76	62,7	cyclictest	951657	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	09:48:58	3
886267	99	76	33,4	cyclictest	644	50	irq/529-eth0-Tx	09:56:52	0
886267	99	75	59,9	cyclictest	884495	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	07:15:07	0
886267	99	74	32,5	cyclictest	886487	-21	/usr/sbin/munin	07:10:38	0
1102741	2	74	0,19	sleep0	13	-21	ksoftirqd/0	11:55:23	0
886288	99	73	68,2	cyclictest	0	-21	swapper/5	08:35:29	5
886288	99	73	68,2	cyclictest	0	-21	swapper/5	08:35:29	5
886288	99	73	67,3	cyclictest	0	-21	swapper/5	09:00:29	5
886288	99	73	66,3	cyclictest	0	-21	swapper/5	11:35:15	5
886288	99	73	66,3	cyclictest	0	-21	swapper/5	11:35:14	5
886283	99	73	68,2	cyclictest	0	-21	swapper/4	11:30:17	4
886283	99	73	68,2	cyclictest	0	-21	swapper/4	11:30:16	4
886278	99	73	63,4	cyclictest	884495	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	07:15:05	3
886278	99	73	61,6	cyclictest	1123227	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	12:14:57	3
886278	99	73	61,6	cyclictest	1123227	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	12:14:56	3
886278	99	73	58,7	cyclictest	915918	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	08:30:22	3
886278	99	73	58,7	cyclictest	915918	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	08:30:22	3
886283	99	72	67,2	cyclictest	0	-21	swapper/4	10:10:40	4
1121867	2	72	0,14	sleep0	0	-21	swapper/0	12:13:04	0
1121867	2	72	0,14	sleep0	0	-21	swapper/0	12:13:03	0
886288	99	70	64,3	cyclictest	0	-21	swapper/5	08:05:18	5
886288	99	70	64,3	cyclictest	0	-21	swapper/5	08:05:17	5
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17