OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #3, slot #4

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance

Characteristics of the 100 highest latencies:

System rack3slot4.osadl.org (updated Thu Apr 30, 2026 12:46:52)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
3302141	99	143	40,85	cyclictest	3479054	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	11:03:15	0
3302141	99	143	40,85	cyclictest	3479054	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	11:03:15	0
3302153	99	135	122,6	cyclictest	3426136	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	10:25:43	3
3302141	99	135	31,80	cyclictest	3522352	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	11:29:19	0
3302160	99	132	125,3	cyclictest	0	-21	swapper/4	11:00:53	4
3302160	99	132	125,3	cyclictest	0	-21	swapper/4	11:00:53	4
3302153	99	131	116,7	cyclictest	3377783	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	09:35:37	3
3302141	99	128	98,23	cyclictest	3466879	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	12:01:07	0
3302165	99	123	117,3	cyclictest	0	-21	swapper/5	10:40:46	5
3302150	99	123	90,9	cyclictest	3587879	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	12:21:11	2
3302141	99	122	2,63	cyclictest	3551559	-21	head	11:50:18	0
3302141	99	122	2,63	cyclictest	3551559	-21	head	11:50:17	0
3302141	99	121	97,16	cyclictest	3306030	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	08:05:17	0
3302141	99	121	97,16	cyclictest	3306030	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	08:05:17	0
3302141	99	120	30,7	cyclictest	3306030	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	07:20:28	0
3302150	99	119	105,7	cyclictest	3532352	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	11:36:03	2
3302141	99	119	32,7	cyclictest	3580039	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	12:35:45	0
3302141	99	117	22,3	cyclictest	0	-21	swapper/0	09:40:37	0
3302141	99	116	1,85	cyclictest	0	-21	swapper/0	08:29:27	0
3302141	99	116	1,85	cyclictest	0	-21	swapper/0	08:29:26	0
3302141	99	115	1,102	cyclictest	0	-21	swapper/0	08:43:25	0
3302141	99	115	1,102	cyclictest	0	-21	swapper/0	08:43:24	0
3302150	99	114	102,6	cyclictest	3440884	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	10:30:46	2
3302160	99	113	107,3	cyclictest	0	-21	swapper/4	08:10:17	4
3302160	99	112	99,8	cyclictest	3464566	-21	ssh	10:40:47	4
3302153	99	112	98,7	cyclictest	3310603	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	07:35:08	3
3302153	99	112	98,7	cyclictest	3310603	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	07:35:07	3
3302141	99	112	2,5	cyclictest	3426247	-21	find	10:10:44	0
3302165	99	111	105,3	cyclictest	0	-21	swapper/5	07:30:05	5
3302141	99	111	32,74	cyclictest	13	-21	ksoftirqd/0	08:00:01	0
3302165	99	109	103,3	cyclictest	0	-21	swapper/5	07:45:06	5
3302141	99	109	79,23	cyclictest	3496247	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	11:10:55	0
3302141	99	109	5,78	cyclictest	0	-21	swapper/0	12:05:41	0
3302141	99	109	37,27	cyclictest	689	50	irq/527-eth0-Tx	10:40:33	0
3302141	99	108	17,3	cyclictest	0	-21	swapper/0	11:35:34	0
3302141	99	108	1,24	cyclictest	3531008	-21	rm	11:34:01	0
3302141	99	107	2,54	cyclictest	0	-21	swapper/0	11:43:55	0
3302141	99	106	27,23	cyclictest	0	-21	swapper/0	11:15:45	0
3302141	99	106	1,83	cyclictest	0	-21	swapper/0	08:35:04	0
3302141	99	106	1,82	cyclictest	0	-21	swapper/0	09:09:09	0
3302141	99	106	1,82	cyclictest	0	-21	swapper/0	09:09:08	0
3302153	99	105	72,9	cyclictest	3540870	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	11:45:23	3
3302141	99	105	1,86	cyclictest	0	-21	swapper/0	08:58:49	0
3302141	99	105	1,86	cyclictest	0	-21	swapper/0	08:58:48	0
3302150	99	104	92,6	cyclictest	3355247	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	09:15:36	2
3302150	99	104	92,6	cyclictest	3355247	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	09:15:35	2
3302141	99	104	19,3	cyclictest	0	-21	swapper/0	10:04:27	0
3302141	99	104	19,3	cyclictest	0	-21	swapper/0	10:04:26	0
3302141	99	104	1,51	cyclictest	3399329	-21	ssh	09:50:12	0
3302141	99	104	1,51	cyclictest	3399329	-21	ssh	09:50:12	0
3302141	99	103	73,26	cyclictest	3397529	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	10:05:41	0
3302141	99	103	1,50	cyclictest	3577390	-21	ssh	12:10:30	0
3302146	99	102	90,6	cyclictest	3350445	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	09:17:35	1
3302146	99	102	90,6	cyclictest	3350445	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	09:17:34	1
3302141	99	102	66,16	cyclictest	3345147	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	09:25:37	0
3302141	99	102	1,83	cyclictest	0	-21	swapper/0	08:51:59	0
3302160	99	101	96,2	cyclictest	0	-21	swapper/4	10:57:35	4
3302160	99	101	96,2	cyclictest	0	-21	swapper/4	10:57:34	4
3302160	99	101	88,8	cyclictest	3556395	-21	ssh	11:54:10	4
3302160	99	101	88,8	cyclictest	3556395	-21	ssh	11:54:10	4
3302141	99	101	73,24	cyclictest	3573118	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	12:27:44	0
3302141	99	101	52,12	cyclictest	0	-21	swapper/0	08:48:38	0
3302141	99	100	1,66	cyclictest	0	-21	swapper/0	09:03:46	0
3302141	99	99	1,62	cyclictest	0	-21	swapper/0	08:30:25	0
3302141	99	99	1,5	cyclictest	48	-21	rcuop/4	09:39:29	0
3302146	99	97	75,9	cyclictest	3458477	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	10:41:35	1
3302141	99	97	2,9	cyclictest	0	-21	swapper/0	09:15:35	0
3302141	99	97	2,9	cyclictest	0	-21	swapper/0	09:15:34	0
3302141	99	97	28,21	cyclictest	692	50	irq/530-eth0-Tx	10:55:13	0
3302141	99	97	28,21	cyclictest	692	50	irq/530-eth0-Tx	10:55:12	0
3302141	99	97	2,56	cyclictest	3602807	-21	taskset	12:30:38	0
3302141	99	97	24,34	cyclictest	3471820	-21	env	10:46:46	0
3302141	99	97	24,34	cyclictest	3471820	-21	env	10:46:46	0
3302150	99	95	65,7	cyclictest	3426136	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	10:49:40	2
3302150	99	95	65,7	cyclictest	3426136	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	10:49:39	2
3302141	99	95	57,17	cyclictest	3306030	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	07:40:43	0
3302141	99	95	23,33	cyclictest	3584934	-21	chrt	12:16:11	0
3302150	99	94	81,6	cyclictest	3397529	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	10:10:42	2
3302150	99	93	79,7	cyclictest	3313954	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	08:40:23	2
3302150	99	93	79,7	cyclictest	3313954	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	08:40:22	2
3302150	99	93	62,9	cyclictest	3350445	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	09:35:40	2
3302141	99	92	57,27	cyclictest	3281620	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	07:25:19	0
3302146	99	91	82,6	cyclictest	3350445	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	09:44:35	1
3302160	99	90	86,2	cyclictest	0	-21	swapper/4	07:30:05	4
3302160	99	90	80,6	cyclictest	3471121	-21	ssh	10:46:02	4
3302160	99	90	80,6	cyclictest	3471121	-21	ssh	10:46:01	4
3302150	99	90	75,8	cyclictest	3563901	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	12:05:42	2
3302150	99	90	75,8	cyclictest	3306030	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	08:05:16	2
3302150	99	90	75,8	cyclictest	3306030	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	08:05:15	2
3302146	99	90	61,7	cyclictest	3466879	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	10:47:57	1
3302146	99	90	61,7	cyclictest	3466879	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	10:47:57	1
3302165	99	89	83,3	cyclictest	0	-21	swapper/5	11:46:05	5
3302165	99	89	83,3	cyclictest	0	-21	swapper/5	08:07:25	5
3302165	99	89	83,3	cyclictest	0	-21	swapper/5	08:07:24	5
3302160	99	89	84,3	cyclictest	0	-21	swapper/4	07:45:06	4
3302153	99	89	77,6	cyclictest	3466879	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	11:05:54	3
3302150	99	89	74,7	cyclictest	3466879	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	11:25:59	2
3302165	99	88	81,3	cyclictest	0	-21	swapper/5	09:57:03	5
3302165	99	88	81,3	cyclictest	0	-21	swapper/5	09:57:03	5
3302153	99	88	76,6	cyclictest	3414839	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	10:40:22	3
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17