OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #3, slot #4

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance

Characteristics of the 100 highest latencies:

System rack3slot4.osadl.org (updated Sun Aug 16, 2026 12:46:41)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
1103584	2	2823	2803,6	sleep3	1100647	-21	kworker/3:3+events@ dbs_work_handler	07:05:22	3
1104841	2	2594	2566,10	sleep0	1103317	-21	kworker/0:0+events@ dbs_work_handler	07:08:58	0
1104753	2	1018	1003,7	sleep1	1104096	-21	kworker/1:2+events@ dbs_work_handler	07:07:46	1
1105081	99	157	58,15	cyclictest	1117148	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	07:49:06	0
1105081	99	112	21,63	cyclictest	0	-21	swapper/0	09:47:25	0
1105081	99	107	68,16	cyclictest	674	50	irq/528-eth0-Tx	10:15:25	0
1105081	99	107	68,16	cyclictest	674	50	irq/528-eth0-Tx	10:15:25	0
1105081	99	107	39,43	cyclictest	1151354	-1	kworker/0:1H-events_highpri	12:00:41	0
1105090	99	104	92,6	cyclictest	1222550	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	11:04:43	2
1105081	99	102	25,63	cyclictest	0	-21	swapper/0	11:15:45	0
1105081	99	102	25,63	cyclictest	0	-21	swapper/0	11:15:44	0
1105081	99	101	69,24	cyclictest	1333695	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	11:39:48	0
1105081	99	100	46,36	cyclictest	1151354	-1	kworker/0:1H-events_highpri	10:35:39	0
1105090	99	99	85,7	cyclictest	1374587	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	12:09:54	2
1105081	99	99	68,23	cyclictest	1214470	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	09:59:29	0
1105081	99	99	67,24	cyclictest	1235612	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	10:29:40	0
1105081	99	99	67,24	cyclictest	1235612	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	10:29:40	0
1105081	99	97	27,4	cyclictest	676	50	irq/530-eth0-Tx	12:20:32	0
1105081	99	94	57,16	cyclictest	1381917	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	12:10:14	0
1105081	99	92	59,16	cyclictest	1222550	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	10:14:39	0
1105081	99	92	59,16	cyclictest	1222550	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	10:14:39	0
1105081	99	92	55,16	cyclictest	1125772	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	08:15:18	0
1105090	99	90	77,6	cyclictest	1131294	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	08:14:12	2
1105090	99	90	77,6	cyclictest	1131294	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	08:14:11	2
1105085	99	90	76,7	cyclictest	1102632	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	07:39:07	1
1105081	99	89	1,63	cyclictest	13	-21	ksoftirqd/0	10:30:29	0
1105081	99	89	1,60	cyclictest	0	-21	swapper/0	10:45:04	0
1105090	99	87	74,6	cyclictest	1183828	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	09:39:27	2
1105090	99	87	74,6	cyclictest	1183828	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	09:39:27	2
1105081	99	87	31,25	cyclictest	15	-21	rcuc/0	12:05:32	0
1105081	99	87	26,29	cyclictest	1272212	-21	missed_timers	10:40:26	0
1105081	99	87	20,4	cyclictest	0	-21	swapper/0	11:27:04	0
1105081	99	85	27,25	cyclictest	0	-21	swapper/0	09:16:54	0
1105081	99	85	17,57	cyclictest	673	50	irq/527-eth0-Tx	09:30:37	0
1105085	99	84	70,7	cyclictest	1222550	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	10:54:41	1
1105081	99	84	2,49	cyclictest	13	-21	ksoftirqd/0	09:35:31	0
1105081	99	84	2,49	cyclictest	13	-21	ksoftirqd/0	09:35:30	0
1105085	99	83	71,6	cyclictest	1418884	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	12:39:59	1
1105081	99	83	2,58	cyclictest	1337326	-21	/usr/sbin/munin	11:30:32	0
1105081	99	83	2,4	cyclictest	1284742	-21	apt-get	10:50:18	0
1105081	99	83	24,24	cyclictest	0	-21	swapper/0	09:40:31	0
1105081	99	83	17,19	cyclictest	1358002	-21	ntpq	11:45:30	0
1105081	99	83	17,19	cyclictest	1358002	-21	ntpq	11:45:30	0
1105090	99	82	37,15	cyclictest	1349671	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	12:01:58	2
1105085	99	82	68,7	cyclictest	1102632	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	07:29:04	1
1105085	99	82	68,7	cyclictest	1102632	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	07:29:03	1
1105081	99	82	36,38	cyclictest	1349671	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	11:40:36	0
1105094	99	80	54,9	cyclictest	1138567	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	08:39:14	3
1105081	99	80	29,14	cyclictest	674	50	irq/528-eth0-Tx	09:15:01	0
1105081	99	80	24,28	cyclictest	13	-21	ksoftirqd/0	10:04:08	0
1105090	99	79	66,6	cyclictest	1151579	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	09:24:23	2
1105081	99	79	48,20	cyclictest	674	50	irq/528-eth0-Tx	08:30:31	0
1105081	99	79	48,20	cyclictest	674	50	irq/528-eth0-Tx	08:30:30	0
1105081	99	79	38,26	cyclictest	1173658	-21	grep	09:25:44	0
1105081	99	79	35,13	cyclictest	13	-21	ksoftirqd/0	11:10:32	0
1105081	99	79	35,13	cyclictest	13	-21	ksoftirqd/0	11:10:32	0
1105081	99	79	33,30	cyclictest	15	-21	rcuc/0	09:25:02	0
1105081	99	79	29,16	cyclictest	13	-21	ksoftirqd/0	12:15:14	0
1105081	99	79	27,5	cyclictest	13	-21	ksoftirqd/0	10:05:48	0
1344780	2	78	0,4	chrt	0	-21	swapper/5	11:35:25	5
1169800	2	78	0,3	sleep5	0	-21	swapper/5	09:23:26	5
1105090	99	78	63,7	cyclictest	1271377	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	10:44:41	2
1105081	99	78	6,49	cyclictest	15	-21	rcuc/0	12:30:33	0
1105081	99	78	6,49	cyclictest	15	-21	rcuc/0	12:30:32	0
1105081	99	78	45,24	cyclictest	673	50	irq/527-eth0-Tx	10:20:39	0
1105081	99	78	2,5	cyclictest	1299049	-21	ntpq	11:00:35	0
1105081	99	77	31,26	cyclictest	15	-21	rcuc/0	09:52:22	0
1105081	99	77	27,34	cyclictest	0	-21	swapper/0	11:59:38	0
1105099	99	76	64,2	cyclictest	0	-21	swapper/4	07:49:07	4
1105090	99	76	64,6	cyclictest	1283548	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	11:16:03	2
1105090	99	76	64,6	cyclictest	1283548	-21	kworker/u12:0+events_unbound@ flush_memcg_stats_dwork	11:16:03	2
1105090	99	76	62,7	cyclictest	1381917	-21	kworker/u12:3+events_unbound@ flush_memcg_stats_dwork	12:19:56	2
1105081	99	76	7,39	cyclictest	15	-21	rcuc/0	11:05:34	0
1105081	99	76	26,6	cyclictest	13	-21	ksoftirqd/0	12:35:44	0
1105081	99	76	24,37	cyclictest	13	-21	ksoftirqd/0	08:05:38	0
1105081	99	76	1,57	cyclictest	0	-21	swapper/0	11:50:31	0
1309473	2	75	0,5	sleep2	0	-21	swapper/2	11:08:55	2
1105099	99	75	70,2	cyclictest	0	-21	swapper/4	07:50:30	4
1105090	99	73	60,6	cyclictest	1102632	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	07:19:00	2
1105090	99	73	60,6	cyclictest	1102632	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	07:19:00	2
1105085	99	73	31,12	cyclictest	1333695	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	11:34:35	1
1105081	99	73	7,49	cyclictest	13	-21	ksoftirqd/0	10:55:39	0
1105081	99	73	53,16	cyclictest	210	-21	kswapd0	11:20:17	0
1105081	99	73	42,20	cyclictest	674	50	irq/528-eth0-Tx	07:35:17	0
1105081	99	73	40,23	cyclictest	1110054	-21	wc	07:20:25	0
1105081	99	73	33,24	cyclictest	0	-21	swapper/0	08:35:19	0
1105081	99	73	2,28	cyclictest	1409575	-21	ssh	12:25:17	0
1105090	99	70	57,6	cyclictest	1142667	-21	kworker/u12:2+events_unbound@ flush_memcg_stats_dwork	08:49:17	2
1105099	99	69	63,3	cyclictest	0	-21	swapper/4	08:05:51	4
1105090	99	68	54,7	cyclictest	1193659	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	09:52:45	2
1105085	99	68	54,7	cyclictest	1222550	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	10:24:39	1
1105081	99	68	48,16	cyclictest	1147428	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	08:54:19	0
1105081	99	68	1,3	cyclictest	0	-21	swapper/0	08:26:41	0
1158010	2	67	0,6	sleep3	36	-21	rcuc/3	09:14:54	3
1105099	99	67	62,2	cyclictest	0	-21	swapper/4	08:15:03	4
1105099	99	67	57,6	cyclictest	1382572	-21	ssh	12:04:53	4
1105099	99	67	55,7	cyclictest	1210620	-21	ssh	09:54:28	4
1105099	99	67	52,9	cyclictest	1131288	-21	taskset	08:13:35	4
1105099	99	67	52,9	cyclictest	1131288	-21	taskset	08:13:34	4
1105090	99	67	35,7	cyclictest	1222550	-21	kworker/u12:1+events_unbound@ flush_memcg_stats_dwork	11:20:31	2
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17