OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #e, slot #6

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rackeslot6.osadl.org (updated Wed Aug 27, 2025 00:44:55)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
1306636	99	110	100,7	cyclictest	1300697	-21	kworker/u8:1+flush-179:0	19:45:01	1
1306636	99	108	94,10	cyclictest	1407000	-21	latency_hist	00:20:02	1
1306636	99	108	94,10	cyclictest	1407000	-21	latency_hist	00:20:02	1
1306635	99	108	97,8	cyclictest	502	-21	rs:main		0
1306635	99	108	101,4	cyclictest	1398997	-21	kworker/u8:0+events_unbound	00:15:24	0
1306635	99	108	101,4	cyclictest	1398997	-21	kworker/u8:0+events_unbound	00:15:23	0
1306636	99	107	92,11	cyclictest	1379331	-21	apt-get	22:55:01	1
1306635	99	107	99,5	cyclictest	1393979	-21	kworker/u8:2+events_unbound	00:00:01	0
1306636	99	106	97,6	cyclictest	1300697	-21	kworker/u8:1+flush-179:0	19:30:02	1
1306635	99	105	94,8	cyclictest	502	-21	rs:main		0
1306635	99	105	83,20	cyclictest	1315013	-21	apt-get	19:35:14	0
1306635	99	105	83,20	cyclictest	1315013	-21	apt-get	19:35:13	0
1306635	99	104	93,8	cyclictest	1300697	-21	kworker/u8:1+flush-179:0	19:30:24	0
1306635	99	101	81,18	cyclictest	1353053	-21	latency_hist	21:35:01	0
1306635	99	99	94,3	cyclictest	14	-21	ksoftirqd/0	23:00:25	0
1306636	99	96	89,4	cyclictest	1393979	-21	kworker/u8:2+events_unbound	00:04:01	1
1306635	99	94	83,9	cyclictest	1394199	-21	apt-get	23:40:17	0
1306635	99	93	85,6	cyclictest	1365241	-21	apt-get	22:10:12	0
1306635	99	91	81,8	cyclictest	1397630	-21	apt-get	23:50:13	0
1306635	99	90	83,5	cyclictest	1359658	-21	kworker/u8:2+flush-179:0	22:05:01	0
1306635	99	90	79,8	cyclictest	1345091	-21	latency_hist	21:10:02	0
1306635	99	88	81,5	cyclictest	1412547	-21	apt-get	00:35:13	0
1306635	99	87	81,5	cyclictest	1322792	-21	kworker/u8:3+events_unbound	20:31:30	0
1306635	99	87	77,7	cyclictest	1329111	-21	latency_hist	20:20:01	0
1306635	99	86	74,9	cyclictest	1351463	-21	latency_hist	21:30:00	0
1306636	99	85	75,8	cyclictest	1383037	-21	apt-get	23:05:15	1
1306636	99	85	75,5	cyclictest	1405427	-21	kworker/u8:2+flush-179:0	00:20:23	1
1306635	99	85	78,5	cyclictest	1325953	-21	kworker/u8:1+flush-179:0	20:20:25	0
1306635	99	85	77,5	cyclictest	1358096	-21	latency_hist	21:50:00	0
1306636	99	84	71,10	cyclictest	1376399	-21	apt-get	22:45:12	1
1306636	99	83	72,8	cyclictest	1346676	-21	latency_hist	21:15:01	1
1306636	99	83	72,8	cyclictest	1321368	-21	/usr/sbin/munin	19:55:26	1
1306635	99	81	69,9	cyclictest	601706	-21	systemd-journal	19:57:35	0
1306636	99	80	71,7	cyclictest	1384911	-21	diskstats	23:10:21	1
1306636	99	80	70,7	cyclictest	1377708	-21	kworker/u8:0+flush-179:0	23:35:00	1
1306636	99	80	65,11	cyclictest	1312915	-21	apt-get	19:30:01	1
1306635	99	80	74,4	cyclictest	1382790	-21	kworker/u8:3+flush-179:0	23:10:26	0
1306635	99	79	71,6	cyclictest	1345602	-21	kworker/u8:0+flush-179:0	21:15:25	0
1306635	99	79	67,9	cyclictest	1343503	-21	latency_hist	21:05:00	0
1306635	99	78	70,5	cyclictest	1359658	-21	kworker/u8:2+flush-179:0	21:55:15	0
1306636	99	77	66,8	cyclictest	1393979	-21	kworker/u8:2+events_unbound	23:50:02	1
1306636	99	77	66,7	cyclictest	1355974	-21	kworker/u8:0+events_unbound	22:00:05	1
1306636	99	76	65,8	cyclictest	1369772	-21	latency_hist	22:25:01	1
1306636	99	76	64,9	cyclictest	601706	-21	systemd-journal	20:47:35	1
1306636	99	76	62,11	cyclictest	1330958	-21	apt-get	20:25:13	1
1306635	99	76	70,5	cyclictest	1382790	-21	kworker/u8:3+flush-179:0	23:35:23	0
1306635	99	76	65,9	cyclictest	1368440	-21	apt-get	22:20:15	0
1306636	99	75	66,6	cyclictest	1322792	-21	kworker/u8:3+flush-179:0	20:05:27	1
1306636	99	74	63,8	cyclictest	1368173	-21	latency_hist	22:20:02	1
1306636	99	74	62,9	cyclictest	1366572	-21	latency_hist	22:15:01	1
1306636	99	73	64,6	cyclictest	1332338	-21	kworker/u8:2+events_unbound	21:20:25	1
1306635	99	73	67,4	cyclictest	1377708	-21	kworker/u8:0+flush-179:0	22:50:18	0
1306636	99	72	61,8	cyclictest	1410697	-21	latency_hist	00:30:00	1
1306636	99	71	62,6	cyclictest	1387562	-21	latency_hist	23:20:01	1
1306636	99	71	57,11	cyclictest	601706	-21	systemd-journal	22:30:01	1
1306635	99	70	64,4	cyclictest	1398997	-21	kworker/u8:0+flush-179:0	00:05:00	0
1306635	99	70	63,5	cyclictest	1398997	-21	kworker/u8:0+flush-179:0	00:30:20	0
1306635	99	70	60,7	cyclictest	1306509	-21	latency_hist	19:10:02	0
1306641	99	69	38,29	cyclictest	0	-21	swapper/3	19:28:59	3
1306635	99	67	54,10	cyclictest	1402474	-21	apt-get	00:05:13	0
1306636	99	66	61,4	cyclictest	1353026	-21	kworker/u8:1+events_unbound	21:50:25	1
1306636	99	66	58,6	cyclictest	1382790	-21	kworker/u8:3+flush-179:0	23:35:25	1
1306636	99	66	57,6	cyclictest	1372927	-21	kworker/u8:2+flush-179:0	22:55:06	1
1306635	99	66	60,5	cyclictest	1312947	-21	kworker/u8:0+flush-179:0	19:40:36	0
1306635	99	66	59,5	cyclictest	1368219	-21	kworker/u8:3+events_unbound	22:35:25	0
1306636	99	65	54,8	cyclictest	601706	-21	systemd-journal	23:43:35	1
1306635	99	65	56,7	cyclictest	1392067	-21	sed	23:30:25	0
1306636	99	64	57,6	cyclictest	1398997	-21	kworker/u8:0+flush-179:0	00:30:23	1
1306635	99	64	54,7	cyclictest	1340558	-21	apt-get	20:55:11	0
1306635	99	64	54,7	cyclictest	1320607	-21	kworker/u8:2+flush-179:0	20:00:16	0
1306635	99	64	54,7	cyclictest	1320607	-21	kworker/u8:2+flush-179:0	20:00:16	0
1306636	99	62	53,7	cyclictest	1371334	-21	kworker/u8:1+events_unbound	22:30:28	1
1306635	99	62	55,4	cyclictest	1	-21	systemd	21:23:37	0
1306635	99	62	53,7	cyclictest	1332267	-21	kworker/u8:1+events_unbound	20:40:25	0
1306636	99	61	54,5	cyclictest	601706	-21	systemd-journal	19:36:35	1
1306636	99	61	54,5	cyclictest	601706	-21	systemd-journal	19:36:35	1
1306636	99	61	52,6	cyclictest	1400589	-40	mandb	00:00:00	1
1306636	99	61	49,10	cyclictest	1337367	-21	kworker/u8:3+flush-179:0	21:00:02	1
1306635	99	61	55,5	cyclictest	1368219	-21	kworker/u8:3+flush-179:0	22:45:23	0
1306635	99	61	47,11	cyclictest	1360046	-21	cat	21:55:01	0
1306636	99	60	50,7	cyclictest	1355974	-21	kworker/u8:0+events_unbound	21:55:01	1
1306636	99	60	49,7	cyclictest	502	-21	rs:main		1
1306635	99	60	54,4	cyclictest	1385991	-21	kworker/u8:1+events_unbound	23:15:18	0
1306635	99	60	54,4	cyclictest	1382790	-21	kworker/u8:3+flush-179:0	23:20:28	0
1306635	99	60	53,5	cyclictest	1322792	-21	kworker/u8:3+flush-179:0	20:25:17	0
1306636	99	59	51,6	cyclictest	1322792	-21	kworker/u8:3+flush-179:0	20:20:00	1
1306636	99	58	43,13	cyclictest	1359658	-21	kworker/u8:2+flush-179:0	22:05:23	1
1306635	99	58	52,4	cyclictest	1306578	-21	kworker/u8:2+events_unbound	19:20:26	0
1306635	99	58	51,5	cyclictest	1392309	-21	kworker/u8:1+flush-179:0	23:45:33	0
1306641	99	57	48,6	cyclictest	1372955	-21	turbostat	22:35:00	3
1306636	99	57	50,5	cyclictest	1404401	-21	tune2fs	00:10:17	1
1306635	99	57	52,4	cyclictest	1408827	-21	kworker/u8:3+flush-179:0	00:25:18	0
1306635	99	57	52,4	cyclictest	1368219	-21	kworker/u8:3+flush-179:0	22:25:16	0
1306635	99	57	50,6	cyclictest	1368219	-21	kworker/u8:3+events_unbound	22:40:27	0
1306635	99	57	49,5	cyclictest	1300697	-21	kworker/u8:1+events_unbound	19:50:20	0
1306636	99	56	51,4	cyclictest	1332338	-21	kworker/u8:2+flush-179:0	21:00:27	1
1306636	99	56	49,5	cyclictest	1368219	-21	kworker/u8:3+flush-179:0	22:40:22	1
1306636	99	56	47,7	cyclictest	1290979	-21	kworker/u8:0+flush-179:0	19:15:23	1
1306635	99	56	47,7	cyclictest	1389411	-21	apt-get	23:25:16	0
1306636	99	55	48,6	cyclictest	1320607	-21	kworker/u8:2+flush-179:0	20:00:23	1
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2020-10-04 - 12:21