OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #3, slot #6

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 792 highest latencies:

System rack3slot6.osadl.org (updated Fri Aug 21, 2026 00:45:07)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
1557	2	96	41,38	sleep1	0	-21	swapper/1	19:05:28	1
1557	2	96	41,38	sleep1	0	-21	swapper/1	19:05:28	1
2528	99	90	5,78	cyclictest	0	-21	swapper/0	22:40:12	0
2532	99	89	14,61	cyclictest	0	-21	swapper/1	22:55:11	1
2532	99	88	69,12	cyclictest	26	-21	ksoftirqd/1	19:20:12	1
2532	99	88	20,58	cyclictest	0	-21	swapper/1	20:45:13	1
2532	99	88	15,62	cyclictest	0	-21	swapper/1	22:45:14	1
2532	99	86	13,62	cyclictest	0	-21	swapper/1	23:00:13	1
2532	99	85	7,66	cyclictest	18420	-21	apt-get	22:50:12	1
2532	99	85	5,67	cyclictest	12435	-21	apt-get	19:35:11	1
2528	99	85	33,44	cyclictest	4750	-21	python3	19:15:07	0
2532	99	84	6,66	cyclictest	7038	-21	apt-get	23:50:14	1
2532	99	84	6,65	cyclictest	8775	-21	apt-get	23:55:14	1
2532	99	84	6,64	cyclictest	18031	-21	apt-get	19:50:15	1
2532	99	84	17,56	cyclictest	0	-21	swapper/1	00:15:15	1
2532	99	84	15,60	cyclictest	0	-21	swapper/1	19:10:11	1
2532	99	84	15,58	cyclictest	0	-21	swapper/1	22:20:16	1
2532	99	84	14,57	cyclictest	0	-21	swapper/1	22:30:23	1
2532	99	84	14,56	cyclictest	0	-21	swapper/1	19:25:24	1
2532	99	84	13,61	cyclictest	0	-21	swapper/1	21:45:13	1
2528	99	84	29,48	cyclictest	32622	-21	sed	22:00:02	0
2532	99	83	6,64	cyclictest	9293	-21	apt-get	20:55:24	1
2532	99	83	15,57	cyclictest	0	-21	swapper/1	00:00:11	1
2532	99	83	14,53	cyclictest	0	-21	swapper/1	22:00:15	1
2532	99	83	13,60	cyclictest	0	-21	swapper/1	23:10:13	1
2532	99	83	13,60	cyclictest	0	-21	swapper/1	19:30:22	1
2532	99	82	7,64	cyclictest	5302	-21	apt-get	23:45:15	1
2532	99	82	7,63	cyclictest	12311	-21	apt-get	00:05:17	1
2532	99	82	7,60	cyclictest	1748	-21	apt-get	23:35:14	1
2532	99	82	7,60	cyclictest	1748	-21	apt-get	23:35:14	1
2532	99	82	6,61	cyclictest	9355	-21	apt-get	22:25:12	1
2532	99	82	25,51	cyclictest	481	-21	gdbus	21:15:00	1
2532	99	82	13,56	cyclictest	0	-21	swapper/1	22:15:12	1
2528	99	82	6,64	cyclictest	0	-21	swapper/0	23:05:13	0
2532	99	81	6,58	cyclictest	7254	-21	apt-get	20:50:13	1
2532	99	81	19,52	cyclictest	0	-21	swapper/1	20:40:15	1
2528	99	81	5,59	cyclictest	21924	-21	apt-get	23:00:13	0
2528	99	81	5,57	cyclictest	20157	-21	apt-get	22:55:11	0
2532	99	80	7,63	cyclictest	21375	-21	apt-get	00:30:14	1
2532	99	80	7,63	cyclictest	21375	-21	apt-get	00:30:14	1
2532	99	80	6,63	cyclictest	14094	-21	apt-get	00:10:14	1
2532	99	80	5,61	cyclictest	2015	-21	apt-get	22:05:13	1
2532	99	80	14,56	cyclictest	0	-21	swapper/1	23:30:14	1
2532	99	80	12,54	cyclictest	0	-21	swapper/1	00:20:23	1
2528	99	80	25,49	cyclictest	14783	-21	nfsd4	00:10:28	0
2532	99	79	7,61	cyclictest	21536	-21	apt-get	20:00:12	1
2532	99	79	6,64	cyclictest	23662	-21	apt-get	23:05:14	1
2532	99	79	6,62	cyclictest	14549	-21	apt-get	19:40:22	1
2532	99	79	14,59	cyclictest	0	-21	swapper/1	19:15:14	1
2528	99	79	4,59	cyclictest	16685	-21	apt-get	22:45:14	0
2532	99	78	8,61	cyclictest	30939	-21	apt-get	21:55:24	1
2532	99	78	7,59	cyclictest	28902	-21	apt-get	21:50:15	1
2532	99	78	6,62	cyclictest	23274	-21	apt-get	20:05:14	1
2532	99	78	6,56	cyclictest	4111	-21	apt-get	22:10:23	1
2532	99	78	6,56	cyclictest	23113	-21	apt-get	00:35:14	1
2532	99	78	26,46	cyclictest	457	-21	dbus-daemon	21:30:00	1
2532	99	78	2,20	cyclictest	13867	-21	ntp_states	22:35:31	1
2532	99	78	14,54	cyclictest	0	-21	swapper/1	23:20:13	1
2528	99	78	9,61	cyclictest	0	-21	swapper/0	00:35:14	0
2528	99	78	6,64	cyclictest	0	-21	swapper/0	21:50:14	0
2528	99	78	5,60	cyclictest	6622	-21	apt-get	19:20:12	0
2528	99	78	5,57	cyclictest	27163	-21	apt-get	21:45:13	0
2528	99	78	4,59	cyclictest	0	-21	swapper/0	22:32:14	0
2528	99	78	37,29	cyclictest	14	-21	ksoftirqd/0	20:15:22	0
2528	99	78	37,29	cyclictest	14	-21	ksoftirqd/0	20:15:22	0
2528	99	78	29,43	cyclictest	30440	-21	apt-get	20:25:15	0
2532	99	77	7,60	cyclictest	27100	-21	apt-get	20:15:22	1
2532	99	77	7,60	cyclictest	27100	-21	apt-get	20:15:22	1
2532	99	77	6,62	cyclictest	27192	-21	apt-get	23:15:13	1
2532	99	77	6,57	cyclictest	21574	-21	apt-get	21:30:14	1
2532	99	77	14,52	cyclictest	0	-21	swapper/1	20:35:11	1
2532	99	77	12,56	cyclictest	0	-21	swapper/1	21:05:13	1
2528	99	77	5,58	cyclictest	10696	-21	apt-get	19:30:22	0
2528	99	77	5,57	cyclictest	25452	-21	apt-get	23:10:13	0
2528	99	77	4,66	cyclictest	20364	-21	ntp_states	00:25:30	0
2528	99	77	40,25	cyclictest	14	-21	ksoftirqd/0	19:50:13	0
2528	99	77	22,48	cyclictest	457	-21	dbus-daemon	20:55:06	0
2532	99	76	8,53	cyclictest	23313	-21	apt-get	21:35:15	1
2532	99	76	8,53	cyclictest	23313	-21	apt-get	21:35:15	1
2532	99	76	7,53	cyclictest	19770	-21	apt-get	19:55:13	1
2532	99	76	7,53	cyclictest	19770	-21	apt-get	19:55:13	1
2532	99	76	6,55	cyclictest	14943	-21	apt-get	22:40:12	1
2532	99	76	23,45	cyclictest	243	-21	systemd-journal	21:40:05	1
2528	99	76	5,55	cyclictest	7616	-21	apt-get	22:20:16	0
2528	99	76	5,55	cyclictest	10568	-21	apt-get	00:00:11	0
2528	99	76	3,66	cyclictest	28757	-21	latency_hist	23:20:00	0
2231	2	76	19,48	sleep0	0	-21	swapper/0	19:08:53	0
2231	2	76	19,48	sleep0	0	-21	swapper/0	19:08:53	0
2532	99	75	7,52	cyclictest	11056	-21	apt-get	21:00:12	1
2532	99	75	6,57	cyclictest	28867	-21	apt-get	20:20:11	1
2532	99	75	6,55	cyclictest	30688	-21	apt-get	23:25:12	1
2528	99	75	6,57	cyclictest	3777	-21	apt-get	20:40:12	0
2528	99	75	6,55	cyclictest	2603	-21	apt-get	19:10:11	0
2528	99	75	6,53	cyclictest	0	-21	swapper/0	23:50:14	0
2528	99	75	6,52	cyclictest	5516	-21	apt-get	20:45:13	0
2528	99	75	33,36	cyclictest	14	-21	ksoftirqd/0	23:30:12	0
2528	99	75	26,43	cyclictest	7555	-21	latency_hist	22:20:02	0
2528	99	75	24,44	cyclictest	22009	-21	cat	21:30:24	0
2532	99	74	6,54	cyclictest	19638	-21	apt-get	00:25:15	1
2532	99	74	13,55	cyclictest	16750	-21	if_err_enp2s0	21:15:24	1
2528	99	74	5,53	cyclictest	15835	-21	apt-get	00:15:15	0
2532	99	73	7,53	cyclictest	32649	-21	apt-get	20:30:12	1
2532	99	73	7,51	cyclictest	3561	-21	apt-get	23:40:12	1
2532	99	73	6,52	cyclictest	25061	-21	apt-get	20:10:13	1
2532	99	73	6,52	cyclictest	25061	-21	apt-get	20:10:13	1
2528	99	73	8,56	cyclictest	0	-21	swapper/0	21:35:14	0
2528	99	73	8,56	cyclictest	0	-21	swapper/0	21:35:14	0
2528	99	73	6,53	cyclictest	8663	-21	apt-get	19:25:23	0
2528	99	73	5,52	cyclictest	12795	-21	apt-get	21:05:13	0
2528	99	73	4,49	cyclictest	0	-21	swapper/0	19:43:29	0
2528	99	73	20,47	cyclictest	2758	-21	cat	23:35:37	0
2528	99	73	20,47	cyclictest	2758	-21	cat	23:35:37	0
2532	99	72	6,53	cyclictest	16290	-21	apt-get	19:45:12	1
2528	99	72	7,57	cyclictest	0	-21	swapper/0	22:10:23	0
2528	99	72	6,58	cyclictest	0	-21	swapper/0	21:15:13	0
2528	99	72	5,55	cyclictest	32702	-21	apt-get	22:00:15	0
2528	99	72	5,55	cyclictest	0	-21	swapper/0	23:55:14	0
2528	99	72	4,62	cyclictest	25994	-21	memory	21:40:36	0
2528	99	72	2,65	cyclictest	13896	-21	taskset	19:38:55	0
2528	99	72	2,63	cyclictest	31159	-21	kernelversion	23:25:23	0
2528	99	72	22,45	cyclictest	8263	-21	unixbench-2d	20:50:35	0
2528	99	72	20,46	cyclictest	10979	-21	apt-key	22:30:02	0
2532	99	71	7,50	cyclictest	18087	-21	apt-get	21:20:11	1
2532	99	71	7,49	cyclictest	30903	-21	apt-get	20:25:24	1
2528	99	71	6,51	cyclictest	0	-21	swapper/0	22:50:12	0
2528	99	71	4,56	cyclictest	21375	-21	apt-get	00:30:15	0
2528	99	71	4,56	cyclictest	21375	-21	apt-get	00:30:15	0
2528	99	71	4,52	cyclictest	28949	-21	apt-get	23:20:13	0
2528	99	71	2,63	cyclictest	29241	-21	basename	20:20:22	0
2528	99	71	18,47	cyclictest	243	-21	systemd-journal	20:06:33	0
2528	99	70	5,60	cyclictest	0	-21	swapper/0	23:41:00	0
2528	99	70	5,36	cyclictest	0	-21	swapper/0	20:32:21	0
2528	99	70	4,57	cyclictest	17897	-21	apt-get	00:20:23	0
2528	99	70	4,56	cyclictest	1960	-21	apt-get	20:35:11	0
2528	99	70	2,63	cyclictest	14465	-21	runrttasks	22:37:33	0
2528	99	70	2,62	cyclictest	23154	-21	wc	20:05:01	0
2528	99	70	2,58	cyclictest	19835	-21	runrttasks	21:25:14	0
2528	99	70	24,39	cyclictest	25443	-21	if_enp2s0	20:10:22	0
2528	99	70	24,39	cyclictest	25443	-21	if_enp2s0	20:10:22	0
2528	99	69	6,54	cyclictest	0	-21	swapper/0	19:55:15	0
2528	99	69	6,54	cyclictest	0	-21	swapper/0	19:55:15	0
2528	99	69	6,51	cyclictest	0	-21	swapper/0	00:05:17	0
2528	99	69	6,49	cyclictest	0	-21	swapper/0	23:45:16	0
2528	99	69	5,56	cyclictest	0	-21	swapper/0	22:05:14	0
2528	99	69	2,62	cyclictest	17486	-21	runrttasks	19:46:04	0
2528	99	68	17,45	cyclictest	18535	-21	cat	21:20:23	0
2528	99	68	10,52	cyclictest	0	-21	swapper/0	21:00:13	0
2528	99	67	3,38	cyclictest	16153	-21	munin-run	21:15:00	0
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	3
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
none	08:08:41	2
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17