OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #7, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system

Characteristics of the 792 highest latencies:

System rack7slot2.osadl.org (updated Sun Feb 01, 2026 00:44:10)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
1552374	99	89	36,42	cyclictest	1605780	-21	fschecks_time	21:35:19	0
1552376	99	83	35,10	cyclictest	0	-21	swapper/1	23:20:26	1
1552374	99	80	37,28	cyclictest	255	50	irq/23-42890000.ethernet	19:10:01	0
1552374	99	78	60,16	cyclictest	1554946	-21	kworker/u4:6+events_unbound	20:00:01	0
1552376	99	76	56,6	cyclictest	1651511	-21	kworker/u4:11+events_unbound	22:32:33	1
1552376	99	73	64,7	cyclictest	1552311	-21	kworker/u4:4+events_unbound	19:12:05	1
1552374	99	73	28,41	cyclictest	0	-21	swapper/0	23:40:02	0
1552376	99	71	54,5	cyclictest	1603161	-21	kworker/u4:4+events_unbound	21:42:24	1
1552376	99	70	65,3	cyclictest	1572980	-21	kworker/u4:4+events_unbound	21:07:17	1
1552376	99	69	60,7	cyclictest	1620974	-21	kworker/u4:3+events_unbound	22:02:27	1
1552374	99	69	37,29	cyclictest	1562925	-21	aten2__	19:50:20	0
1552374	99	69	33,31	cyclictest	1592367	-21	aten_r7power_cu	21:20:15	0
1552376	99	68	63,3	cyclictest	1576774	-21	kworker/u4:6+events_unbound	20:52:15	1
1552376	99	68	60,6	cyclictest	1647032	-21	kworker/u4:8+events_unbound	22:47:33	1
1552374	99	68	34,32	cyclictest	1569426	-21	aten_r7power_en	20:15:20	0
1552376	99	67	59,6	cyclictest	1720970	-21	kworker/u4:0+events_unbound	00:07:43	1
1552376	99	67	59,6	cyclictest	1720970	-21	kworker/u4:0+events_unbound	00:07:43	1
1552374	99	67	43,22	cyclictest	1689236	-21	diskmemload	23:10:26	0
1552374	99	67	42,21	cyclictest	1697030	-21	latency_hist	23:20:02	0
1552374	99	67	3,59	cyclictest	1675232	-21	latency_hist	22:55:01	0
1552376	99	66	58,6	cyclictest	1646427	-21	kworker/u4:2+events_unbound	22:42:33	1
1552374	99	66	58,6	cyclictest	1725326	-21	kworker/u4:7+events_unbound	00:20:58	0
1552374	99	65	32,30	cyclictest	0	-21	swapper/0	23:05:40	0
1552374	99	65	32,29	cyclictest	0	-21	swapper/0	21:30:42	0
1552374	99	65	3,19	cyclictest	1641272	-21	ntpq	22:15:32	0
1552374	99	65	30,33	cyclictest	1553918	-21	aten_r7power_cu	19:15:16	0
1552374	99	65	29,33	cyclictest	0	-21	swapper/0	22:05:35	0
1552376	99	64	55,7	cyclictest	1682036	-21	kworker/u4:10+events_unbound	23:02:34	1
1552374	99	64	31,30	cyclictest	0	-21	swapper/0	22:10:33	0
1552374	99	64	19,42	cyclictest	1715549	-21	switchtime	23:40:41	0
1552374	99	64	19,42	cyclictest	1715549	-21	switchtime	23:40:41	0
1552374	99	64	1,60	cyclictest	1752388	-21	aten_r7power_en	00:25:17	0
1552376	99	63	54,6	cyclictest	1552311	-21	kworker/u4:4+events_unbound	19:52:10	1
1552376	99	63	44,5	cyclictest	1585975	-21	kworker/u4:2+events_unbound	21:16:00	1
1552374	99	63	50,10	cyclictest	1552373	-21	cyclictest	22:36:05	0
1552374	99	63	33,28	cyclictest	1739080	-21	latency_hist	00:10:01	0
1552374	99	63	33,28	cyclictest	1739080	-21	latency_hist	00:10:01	0
1552374	99	63	32,29	cyclictest	1577205	-21	fschecks_count	20:45:18	0
1552374	99	63	31,29	cyclictest	0	-21	swapper/0	22:20:26	0
1552374	99	63	29,31	cyclictest	0	-21	swapper/0	22:55:29	0
1552374	99	63	1,15	cyclictest	1558288	-21	ntpq	19:30:29	0
1552374	99	63	1,12	cyclictest	1584363	-21	ntpq	21:10:36	0
1552374	99	62	31,29	cyclictest	1562723	-21	apt-key	19:50:01	0
1552374	99	62	29,30	cyclictest	0	-21	swapper/0	23:50:36	0
1552374	99	62	28,29	cyclictest	0	-21	swapper/0	22:40:32	0
1552374	99	62	2,7	cyclictest	1571258	-21	ntpq	20:20:29	0
1552374	99	62	2,7	cyclictest	1571258	-21	ntpq	20:20:28	0
1552374	99	62	2,16	cyclictest	1619321	-21	ntpq	21:50:30	0
1552374	99	62	21,38	cyclictest	0	-21	swapper/0	00:00:36	0
1552374	99	62	1,15	cyclictest	1649929	-21	ntpq	22:25:28	0
1552374	99	62	1,13	cyclictest	1582708	-21	ntpq	21:05:30	0
1552376	99	61	52,7	cyclictest	1725326	-21	kworker/u4:7+events_unbound	00:27:44	1
1552374	99	61	4,12	cyclictest	1559702	-21	ps	19:35:38	0
1552374	99	61	32,27	cyclictest	1552373	-21	cyclictest	00:30:25	0
1552374	99	61	32,26	cyclictest	1680913	-21	mailstats	23:00:37	0
1552374	99	61	28,30	cyclictest	0	-21	swapper/0	00:35:22	0
1552374	99	61	2,15	cyclictest	1614871	-21	ntpq	21:45:25	0
1552374	99	61	2,13	cyclictest	1668317	-21	ssh	22:46:02	0
1552374	99	61	21,38	cyclictest	1747622	-21	cron	00:20:00	0
1552374	99	61	2,11	cyclictest	1568026	-21	apt-get	20:10:19	0
1552376	99	60	51,6	cyclictest	1739247	-21	apt-get	00:10:09	1
1552376	99	60	47,10	cyclictest	0	-21	swapper/1	23:40:12	1
1552376	99	60	47,10	cyclictest	0	-21	swapper/1	23:40:11	1
1552374	99	60	27,30	cyclictest	0	-21	swapper/0	00:00:00	0
1552374	99	60	26,31	cyclictest	0	-21	swapper/0	22:30:18	0
1552374	99	60	12,45	cyclictest	0	-21	swapper/0	23:45:42	0
1552374	99	60	1,13	cyclictest	1581387	-21	ntpq	21:00:22	0
1552376	99	59	50,6	cyclictest	1552373	-21	cyclictest	22:20:19	1
1552376	99	59	50,5	cyclictest	0	-21	swapper/1	21:25:15	1
1552374	99	59	4,6	cyclictest	1555755	-21	ntpq	19:20:28	0
1552374	99	59	3,12	cyclictest	1610750	-21	ps	21:40:36	0
1552374	99	59	25,30	cyclictest	0	-21	swapper/0	21:15:31	0
1552374	99	59	23,32	cyclictest	0	-21	swapper/0	20:40:26	0
1552374	99	59	22,34	cyclictest	54	-21	kswapd0	22:00:32	0
1552374	99	59	2,15	cyclictest	1580125	-21	ntpq	20:55:26	0
1552374	99	59	2,14	cyclictest	1706598	-21	ntpq	23:30:29	0
1552376	99	58	49,7	cyclictest	1725327	-21	kworker/u4:9+events_unbound	23:52:40	1
1552374	99	58	3,9	cyclictest	1572463	-21	ntpq	20:25:27	0
1552374	99	58	17,39	cyclictest	1598621	-21	ssh	21:26:40	0
1552374	99	58	15,40	cyclictest	1697788	-21	kernelversion	23:20:22	0
1552376	99	57	43,11	cyclictest	0	-21	swapper/1	23:45:18	1
1552376	99	57	38,16	cyclictest	1728456	-21	mailstats	23:55:38	1
1552374	99	57	26,28	cyclictest	0	-21	swapper/0	20:10:01	0
1552374	99	57	15,39	cyclictest	1627055	-21	sed	22:00:01	0
1552374	99	57	11,11	cyclictest	1575248	-21	sed	20:35:35	0
1552376	99	56	48,6	cyclictest	1690836	-21	kworker/u4:8+events_unbound	23:17:37	1
1552376	99	56	46,7	cyclictest	1647034	-21	kworker/u4:10+events_unbound	22:27:30	1
1552374	99	56	4,8	cyclictest	1560893	-21	ntpq	19:40:28	0
1552374	99	56	3,6	cyclictest	1740204	-21	ps	00:10:33	0
1552374	99	56	25,28	cyclictest	0	-21	swapper/0	23:25:30	0
1552374	99	56	25,28	cyclictest	0	-21	swapper/0	20:00:17	0
1552374	99	56	2,13	cyclictest	1573404	-21	/usr/sbin/munin	20:30:18	0
1552376	99	55	45,7	cyclictest	1592897	-21	/usr/sbin/munin	21:20:22	1
1552374	99	55	25,27	cyclictest	0	-21	swapper/0	19:30:01	0
1552374	99	55	1,13	cyclictest	1553653	-21	users	19:10:42	0
1552376	99	54	46,6	cyclictest	1482819	-21	kworker/u4:8+events_unbound	20:15:00	1
1552376	99	53	45,6	cyclictest	1710260	-21	aten_r7power_po	23:35:15	1
1552376	99	53	44,6	cyclictest	1554598	-21	sendmail_mailqu	19:15:35	1
1552376	99	53	42,7	cyclictest	1629222	-21	kworker/u4:9+events_unbound	22:10:34	1
1552376	99	53	34,13	cyclictest	0	-21	swapper/1	00:30:30	1
1552374	99	53	1,13	cyclictest	1579028	-21	/usr/sbin/munin	20:50:39	0
1552376	99	52	43,6	cyclictest	1554946	-21	kworker/u4:6+events_unbound	19:35:00	1
1552376	99	51	42,6	cyclictest	0	-21	swapper/1	22:55:14	1
1552376	99	51	41,8	cyclictest	1732984	-21	unixbench_multi	00:00:38	1
1552376	99	51	38,10	cyclictest	1552535	-21	date	19:10:01	1
1552376	99	50	43,5	cyclictest	1743670	-21	aten_r7power_cu	00:15:16	1
1552376	99	50	41,7	cyclictest	1569396	-21	aten2__	20:15:19	1
1552376	99	50	31,15	cyclictest	0	-21	swapper/1	21:35:14	1
1552376	99	49	8,30	cyclictest	1632533	-21	ntpq	22:05:34	1
1552376	99	49	40,7	cyclictest	1576766	-21	systemd	20:42:14	1
1552376	99	49	40,6	cyclictest	1672379	-21	unixbench_multi	22:50:38	1
1552376	99	48	40,5	cyclictest	1748629	-21	ld	00:20:25	1
1552376	99	48	39,6	cyclictest	1584630	-21	timerwakeupswit	21:10:39	1
1552376	99	48	38,7	cyclictest	0	-21	swapper/1	22:35:25	1
1552376	99	47	7,26	cyclictest	1557495	-21	modprobe	19:30:00	1
1552376	99	47	5,30	cyclictest	1580099	-21	sshd	20:55:23	1
1552376	99	47	39,6	cyclictest	1570441	-21	kworker/u4:7+events_unbound	20:37:13	1
1552376	99	47	39,5	cyclictest	1571944	-21	aten2__	20:25:18	1
1552376	99	47	37,7	cyclictest	1602146	-21	timerandwakeup	21:30:40	1
1552376	99	47	37,7	cyclictest	0	-21	swapper/1	23:30:18	1
1552376	99	46	39,5	cyclictest	1623867	-21	mailstats	21:55:35	1
1552376	99	46	38,6	cyclictest	1561791	-21	fschecks_count	19:45:19	1
1552376	99	46	38,6	cyclictest	1554946	-21	kworker/u4:6+events_unbound	19:42:10	1
1552376	99	46	38,6	cyclictest	1482819	-21	kworker/u4:8+events_unbound	19:37:10	1
1552376	99	46	37,7	cyclictest	1566643	-21	latency_hist	20:05:02	1
1552376	99	46	36,8	cyclictest	1566876	-21	df_abs	20:05:17	1
1552376	99	45	36,6	cyclictest	1686445	-21	kworker/u4:7-rpciod	23:10:21	1
1552376	99	45	33,9	cyclictest	1619798	-21	diskmemload	21:50:40	1
1552376	99	44	33,8	cyclictest	0	-21	swapper/1	21:45:25	1
1552376	99	44	32,9	cyclictest	0	-21	swapper/1	22:15:24	1
1552376	99	43	31,7	cyclictest	0	-21	swapper/1	23:25:41	1
1552376	99	43	28,11	cyclictest	0	-21	swapper/1	19:55:19	1
1552376	99	42	35,5	cyclictest	1764941	-21	sendmail	00:40:01	1
1552376	99	42	33,6	cyclictest	1684341	-21	aten_r7power_en	23:05:19	1
1552376	99	42	33,6	cyclictest	0	-21	swapper/1	20:20:17	1
1552376	99	42	33,6	cyclictest	0	-21	swapper/1	20:20:16	1
1552376	99	41	30,8	cyclictest	1556094	-21	unixbench-2d	19:20:36	1
1552376	99	40	26,11	cyclictest	1578108	-21	apt-config	20:50:02	1
1552376	99	39	27,9	cyclictest	0	-21	swapper/1	21:00:16	1
1552376	99	37	26,7	cyclictest	0	-21	swapper/1	20:30:39	1
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17