OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #3, slot #4

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance

Characteristics of the 100 highest latencies:

System rack3slot4.osadl.org (updated Tue Feb 03, 2026 00:46:33)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
3446372	2	932	910,10	sleep2	3425100	-21	kworker/2:0+events@ dbs_work_handler	19:05:26	2
3447562	99	126	112,7	cyclictest	3510197	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	21:42:45	2
3447553	99	126	29,42	cyclictest	3488483	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	21:03:00	0
3447553	99	126	29,42	cyclictest	3488483	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	21:03:00	0
3447567	99	117	104,7	cyclictest	3479752	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	20:37:35	3
3447553	99	116	81,7	cyclictest	3697800	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	00:25:35	0
3447562	99	115	102,7	cyclictest	3638019	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:27:58	2
3447567	99	113	97,9	cyclictest	3510197	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:17:50	3
3447577	99	110	105,2	cyclictest	0	-21	swapper/5	21:57:47	5
3447553	99	110	75,7	cyclictest	3697800	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	00:08:02	0
3447562	99	108	94,7	cyclictest	3486042	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	21:12:39	2
3447562	99	106	93,7	cyclictest	3479752	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	20:42:34	2
3447562	99	106	93,7	cyclictest	3479752	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	20:42:34	2
3447553	99	106	71,7	cyclictest	3741484	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	00:32:38	0
3447562	99	103	89,7	cyclictest	3450641	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	20:22:30	2
3447567	99	99	85,7	cyclictest	3655196	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	23:19:35	3
3447567	99	99	85,7	cyclictest	3655196	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	23:19:34	3
3447571	99	98	84,8	cyclictest	3680303	-21	ssh	23:37:34	4
3447577	99	97	86,5	cyclictest	3490024	2	sleep5	20:52:36	5
3447553	99	95	35,37	cyclictest	3679889	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	23:40:03	0
3447553	99	95	24,45	cyclictest	3479752	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	20:36:19	0
3447571	99	93	80,7	cyclictest	3626989	-21	ssh	22:55:13	4
3447567	99	93	80,7	cyclictest	3466225	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	20:02:25	3
3447567	99	92	78,7	cyclictest	3750319	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	00:34:05	3
3447567	99	92	60,9	cyclictest	3510197	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:10:56	3
3447562	99	91	61,6	cyclictest	3557479	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	22:00:58	2
3447553	99	91	56,17	cyclictest	3447885	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	19:16:56	0
3447553	99	91	33,7	cyclictest	3423664	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	19:13:33	0
3447553	99	91	24,5	cyclictest	3710049	-21	ssh	00:00:54	0
3447553	99	91	24,5	cyclictest	3710049	-21	ssh	00:00:53	0
3447567	99	90	76,7	cyclictest	3587856	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	22:44:35	3
3447562	99	90	76,7	cyclictest	3611784	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	23:05:34	2
3447562	99	90	60,7	cyclictest	3488483	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	20:50:35	2
3651195	2	89	0,3	sleep1	0	-21	swapper/1	23:14:55	1
3447571	99	88	82,3	cyclictest	0	-21	swapper/4	00:25:15	4
3447571	99	88	78,5	cyclictest	3686878	-21	ssh	23:42:58	4
3447567	99	88	74,7	cyclictest	3450641	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	19:30:34	3
3447553	99	88	14,54	cyclictest	3447885	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	19:24:53	0
3447577	99	87	73,7	cyclictest	3467595	-21	chrt	19:57:35	5
3447567	99	87	75,6	cyclictest	3538656	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:46:55	3
3447567	99	87	74,6	cyclictest	3510197	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	21:51:33	3
3447567	99	87	61,9	cyclictest	3569126	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:26:14	3
3447553	99	87	29,37	cyclictest	3450641	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	19:51:00	0
3447567	99	86	73,7	cyclictest	3450641	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	20:09:35	3
3447553	99	86	33,29	cyclictest	690	50	irq/528-eth0-Tx	23:45:33	0
3447553	99	86	1,26	cyclictest	0	-21	swapper/0	22:54:59	0
3447577	99	85	79,3	cyclictest	0	-21	swapper/5	20:13:23	5
3447571	99	85	74,6	cyclictest	3613521	-21	ssh	22:44:31	4
3447571	99	85	72,7	cyclictest	3580333	-21	ssh	22:17:34	4
3447562	99	85	71,7	cyclictest	3741484	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	00:35:24	2
3447562	99	85	71,7	cyclictest	3450641	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	19:19:35	2
3447553	99	85	2,7	cyclictest	692	50	irq/530-eth0-Tx	21:50:14	0
3447577	99	84	78,3	cyclictest	0	-21	swapper/5	21:10:20	5
3447567	99	84	72,6	cyclictest	3510197	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	22:02:49	3
3447567	99	84	70,7	cyclictest	3450641	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	19:35:34	3
3447553	99	84	2,50	cyclictest	3703418	-21	timerandwakeup	23:55:43	0
3447571	99	83	69,7	cyclictest	0	-21	swapper/4	23:46:03	4
3447567	99	83	60,9	cyclictest	3734441	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	00:35:30	3
3447562	99	83	62,8	cyclictest	3638019	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:24:32	2
3447562	99	83	62,8	cyclictest	3638019	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	23:24:31	2
3447553	99	83	38,7	cyclictest	3714250	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	00:20:28	0
3447553	99	83	33,24	cyclictest	692	50	irq/530-eth0-Tx	21:20:32	0
3447577	99	82	76,3	cyclictest	0	-21	swapper/5	22:30:35	5
3447567	99	82	68,7	cyclictest	3496198	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:09:55	3
3447562	99	82	69,6	cyclictest	3569126	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:35:10	2
3447562	99	82	67,8	cyclictest	3423664	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	19:10:15	2
3447558	99	82	66,8	cyclictest	3510197	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	21:27:42	1
3447558	99	82	66,8	cyclictest	3510197	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	21:27:42	1
3447577	99	81	76,2	cyclictest	0	-21	swapper/5	23:41:05	5
3447577	99	81	75,3	cyclictest	0	-21	swapper/5	23:37:34	5
3447571	99	81	75,3	cyclictest	0	-21	swapper/4	22:45:50	4
3447571	99	81	75,3	cyclictest	0	-21	swapper/4	22:45:49	4
3447571	99	81	74,4	cyclictest	0	-21	swapper/4	21:55:33	4
3447571	99	81	68,7	cyclictest	3757276	-21	ssh	00:39:20	4
3447571	99	81	66,9	cyclictest	3704277	-21	ssh	23:56:28	4
3447567	99	81	67,8	cyclictest	3569126	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	23:10:15	3
3447567	99	81	66,8	cyclictest	3466225	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	20:10:15	3
3447562	99	81	68,7	cyclictest	3569126	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	22:11:35	2
3447553	99	81	44,17	cyclictest	0	-21	swapper/0	23:35:45	0
3447553	99	81	32,35	cyclictest	3471799	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	20:44:31	0
3447553	99	81	32,35	cyclictest	3471799	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	20:44:30	0
3447553	99	81	1,18	cyclictest	0	-21	swapper/0	00:10:18	0
3447553	99	81	1,18	cyclictest	0	-21	swapper/0	00:10:17	0
3447571	99	80	68,6	cyclictest	3464052	-21	taskset	19:50:05	4
3447571	99	80	67,7	cyclictest	3588388	-21	sh	22:24:24	4
3447571	99	80	67,7	cyclictest	3499649	-21	diskmemload-4	21:54:34	4
3447571	99	80	66,8	cyclictest	3643053	-21	ssh	23:07:34	4
3447567	99	80	68,6	cyclictest	3714655	-21	kworker/u12:4+events_unbound@ flush_memcg_stats_dwork	00:11:34	3
3447567	99	80	68,6	cyclictest	3714655	-21	kworker/u12:4+events_unbound@ flush_memcg_stats_dwork	00:11:34	3
3447567	99	80	66,7	cyclictest	3496198	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	21:21:39	3
3447567	99	80	64,8	cyclictest	3548199	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	21:55:42	3
3447567	99	80	59,8	cyclictest	3697800	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	00:16:47	3
3447567	99	80	59,8	cyclictest	3697800	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	00:16:47	3
3447562	99	80	51,7	cyclictest	3527667	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	21:36:02	2
3447553	99	80	29,7	cyclictest	13	-21	ksoftirqd/0	21:55:25	0
3447577	99	79	74,2	cyclictest	0	-21	swapper/5	00:25:15	5
3447577	99	79	73,3	cyclictest	0	-21	swapper/5	19:37:21	5
3447571	99	79	69,5	cyclictest	3737927	-21	ssh	00:23:44	4
3447571	99	79	66,7	cyclictest	3728438	-21	ssh	00:15:42	4
3447571	99	79	66,7	cyclictest	3728438	-21	ssh	00:15:41	4
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17