OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #3, slot #4

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance

Characteristics of the 100 highest latencies:

System rack3slot4.osadl.org (updated Sun Feb 22, 2026 00:46:39)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
3121224	2	2807	2785,9	sleep1	3120569	-21	kworker/1:1+events@ dbs_work_handler	19:05:17	1
3122668	2	2539	2529,5	sleep5	0	-21	swapper/5	19:08:35	5
3122931	99	146	84,55	cyclictest	3230648	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:02:02	0
3122931	99	138	88,43	cyclictest	3193610	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	21:45:21	0
3122931	99	138	88,43	cyclictest	3193610	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	21:45:21	0
3122931	99	136	57,54	cyclictest	3277421	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	22:32:03	0
3122931	99	132	59,57	cyclictest	3331769	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:19:40	0
3122931	99	130	68,55	cyclictest	3299426	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	22:55:13	0
3122931	99	130	54,49	cyclictest	3322872	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	23:08:38	0
3122931	99	130	54,49	cyclictest	3322872	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	23:08:38	0
3122936	99	129	116,7	cyclictest	3331769	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:33:27	1
3122931	99	129	52,41	cyclictest	3157168	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	21:19:28	0
3122931	99	128	66,54	cyclictest	3412465	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	00:25:20	0
3122931	99	128	51,41	cyclictest	3157168	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	21:10:16	0
3122931	99	128	51,41	cyclictest	3157168	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	21:10:16	0
3122931	99	127	65,54	cyclictest	3136035	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	19:45:40	0
3122931	99	127	65,54	cyclictest	3136035	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	19:45:39	0
3122931	99	126	54,37	cyclictest	3136035	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	19:53:58	0
3122931	99	126	1,110	cyclictest	0	-21	swapper/0	22:25:43	0
3122931	99	125	88,28	cyclictest	3140951	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	19:57:36	0
3122931	99	125	45,44	cyclictest	3311066	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:02:21	0
3122931	99	122	86,28	cyclictest	3116535	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	19:11:35	0
3122936	99	121	107,7	cyclictest	3375305	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	23:53:32	1
3122936	99	121	107,7	cyclictest	3375305	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	23:53:31	1
3197697	2	120	0,5	sleep4	3197695	-21	ssh	21:30:45	4
3122936	99	120	107,7	cyclictest	3331769	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:43:29	1
3122931	99	120	80,28	cyclictest	3291939	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:44:35	0
3122931	99	120	78,32	cyclictest	690	50	irq/528-eth0-Tx	22:10:21	0
3122931	99	120	44,39	cyclictest	3397015	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	00:35:33	0
3122931	99	120	44,39	cyclictest	3397015	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	00:35:32	0
3122931	99	120	42,41	cyclictest	3157168	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	21:05:29	0
3122931	99	119	55,56	cyclictest	3220535	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:05:14	0
3122931	99	117	81,27	cyclictest	3365826	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:53:52	0
3122931	99	117	81,27	cyclictest	3365826	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:53:51	0
3122931	99	117	80,29	cyclictest	3125384	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	19:22:30	0
3122931	99	117	75,35	cyclictest	3164456	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:23:25	0
3122931	99	117	44,37	cyclictest	3197482	-21	kworker/u12:4+events_unbound@ flush_memcg_stats_dwork	21:44:22	0
3122931	99	117	44,37	cyclictest	3197482	-21	kworker/u12:4+events_unbound@ flush_memcg_stats_dwork	21:44:22	0
3122936	99	116	101,7	cyclictest	3125384	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	19:37:34	1
3388674	2	114	0,5	sleep0	0	-21	swapper/0	23:57:15	0
3388674	2	114	0,5	sleep0	0	-21	swapper/0	23:57:14	0
3122931	99	114	76,30	cyclictest	3405369	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	00:15:28	0
3122931	99	114	42,37	cyclictest	3365826	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:11:14	0
3122931	99	111	76,14	cyclictest	690	50	irq/528-eth0-Tx	20:20:01	0
3122931	99	111	74,29	cyclictest	3311066	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:20:47	0
3122931	99	111	70,34	cyclictest	3171711	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	21:29:11	0
3122936	99	110	96,7	cyclictest	3284812	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:38:15	1
3122936	99	110	96,7	cyclictest	3284812	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:38:15	1
3122941	99	109	97,6	cyclictest	3151609	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	20:22:42	2
3122931	99	109	76,28	cyclictest	3331769	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:36:27	0
3122931	99	109	50,55	cyclictest	3136035	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	19:43:45	0
3411948	2	108	0,3	sleep2	0	-21	swapper/2	00:15:22	2
3122931	99	108	65,29	cyclictest	3365826	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:03:51	0
3122931	99	108	65,21	cyclictest	689	50	irq/527-eth0-Tx	20:20:22	0
3122931	99	108	1,85	cyclictest	3147352	-21	acpi	20:10:11	0
3122931	99	107	62,21	cyclictest	691	50	irq/529-eth0-Tx	21:35:13	0
3122931	99	107	62,21	cyclictest	691	50	irq/529-eth0-Tx	21:35:13	0
3122931	99	107	61,25	cyclictest	0	-21	swapper/0	23:25:29	0
3122931	99	106	26,26	cyclictest	3419899	-21	ssh	00:20:51	0
3122931	99	105	53,24	cyclictest	0	-21	swapper/0	21:00:01	0
3122931	99	105	52,37	cyclictest	3277421	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	22:39:15	0
3122931	99	105	52,37	cyclictest	3277421	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	22:39:15	0
3122931	99	104	49,27	cyclictest	0	-21	swapper/0	21:53:53	0
3122936	99	103	90,6	cyclictest	3151609	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	20:32:44	1
3122931	99	103	55,22	cyclictest	0	-21	swapper/0	19:25:29	0
3122931	99	103	55,22	cyclictest	0	-21	swapper/0	19:25:29	0
3122936	99	102	87,8	cyclictest	3350018	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	23:38:28	1
3122931	99	102	69,24	cyclictest	3144724	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	20:42:46	0
3122931	99	102	61,33	cyclictest	3245873	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	22:16:48	0
3122936	99	101	87,7	cyclictest	3265795	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:49:58	1
3122936	99	99	86,7	cyclictest	3171711	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	21:30:06	1
3122931	99	98	69,26	cyclictest	3265795	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:48:18	0
3122931	99	98	61,28	cyclictest	3331769	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:41:00	0
3122941	99	97	61,9	cyclictest	3311066	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:17:52	2
3122936	99	97	86,5	cyclictest	3125384	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	19:27:30	1
3122936	99	97	86,5	cyclictest	3125384	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	19:27:29	1
3122931	99	97	61,29	cyclictest	3397015	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	00:05:15	0
3122931	99	97	60,16	cyclictest	3220535	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:55:40	0
3122936	99	95	82,6	cyclictest	3237767	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:18:07	1
3122931	99	95	61,26	cyclictest	3164456	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:00:18	0
3122931	99	94	65,25	cyclictest	3265795	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	23:13:44	0
3122931	99	94	65,25	cyclictest	3265795	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	23:13:43	0
3122931	99	94	63,27	cyclictest	3365826	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:49:04	0
3122941	99	93	67,13	cyclictest	3397015	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	00:20:34	2
3122941	99	93	66,9	cyclictest	3220535	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:03:05	2
3122931	99	93	51,20	cyclictest	3157168	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	20:50:16	0
3122931	99	93	51,20	cyclictest	3157168	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	20:50:16	0
3122941	99	92	78,7	cyclictest	3206817	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	21:48:02	2
3122941	99	92	78,7	cyclictest	3206817	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	21:48:02	2
3122931	99	92	60,14	cyclictest	3144724	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	20:26:53	0
3122931	99	92	55,20	cyclictest	3125384	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	19:30:27	0
3122931	99	91	57,19	cyclictest	3397015	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	00:32:58	0
3122931	99	91	57,19	cyclictest	3397015	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	00:32:57	0
3122931	99	91	56,19	cyclictest	3125384	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	19:35:22	0
3122931	99	90	58,15	cyclictest	3140951	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	20:04:04	0
3122931	99	90	58,15	cyclictest	3140951	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	20:04:04	0
3122941	99	89	65,7	cyclictest	3230648	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:13:04	2
3122941	99	89	55,8	cyclictest	3322872	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	23:12:34	2
3122941	99	89	55,8	cyclictest	3322872	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	23:12:34	2
3122936	99	89	76,7	cyclictest	3193610	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	21:45:52	1
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17