OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #7, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system

Characteristics of the 792 highest latencies:

System rack7slot2.osadl.org (updated Sat May 30, 2026 00:44:09)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
1298883	99	122	109,9	cyclictest	1353951	-21	kworker/u4:7+events_unbound	21:42:13	0
1298885	99	105	90,5	cyclictest	1481334	-21	kworker/u4:12+events_unbound	00:03:14	1
1298883	99	102	87,6	cyclictest	1480688	-21	kworker/u4:9+events_unbound	00:13:21	0
1298883	99	94	84,7	cyclictest	1367291	-21	kworker/u4:8+events_unbound	21:57:15	0
1298883	99	89	82,5	cyclictest	1345698	-21	kworker/u4:1+events_unbound	21:32:07	0
1298883	99	88	80,6	cyclictest	1450775	-21	kworker/u4:4+events_unbound	23:32:34	0
1298883	99	85	77,6	cyclictest	1476861	-21	kworker/u4:8+events_unbound	00:18:23	0
1298883	99	84	76,6	cyclictest	1441886	-21	kworker/u4:7+events_unbound	23:22:32	0
1298883	99	83	75,6	cyclictest	1302698	-21	kworker/u4:3+events_unbound	20:47:00	0
1298883	99	82	73,6	cyclictest	1307893	-21	kworker/u4:15+events_unbound	21:17:03	0
1298885	99	81	74,4	cyclictest	1380028	-21	kworker/u4:2+events_unbound	22:07:17	1
1298883	99	81	62,11	cyclictest	1353952	-21	kworker/u4:9+events_unbound	21:37:09	0
1298883	99	79	67,9	cyclictest	1353949	-21	kworker/u4:5+events_unbound	21:47:14	0
1298883	99	79	66,8	cyclictest	1457147	-21	apt-get	23:35:15	0
1298883	99	78	71,5	cyclictest	1304056	-21	kworker/u4:10+events_unbound	19:41:48	0
1298883	99	76	67,7	cyclictest	1325756	-21	kworker/u4:2+events_unbound	21:30:01	0
1298883	99	75	68,5	cyclictest	1059055	-21	kworker/u4:6+events_unbound	19:10:02	0
1298883	99	75	67,6	cyclictest	1380028	-21	kworker/u4:2+events_unbound	22:10:00	0
1298883	99	73	66,5	cyclictest	1302697	-21	kworker/u4:2+events_unbound	19:26:43	0
1298883	99	72	65,5	cyclictest	15	1	ktimers/0	23:11:54	0
1298883	99	72	64,6	cyclictest	1406786	-21	kworker/u4:9+events_unbound	22:47:24	0
1298883	99	71	64,5	cyclictest	1367288	-21	kworker/u4:0+events_unbound	21:55:00	0
1298883	99	71	26,37	cyclictest	1400512	-21	cut	22:30:21	0
1298883	99	70	63,5	cyclictest	1454581	-21	kworker/u4:9+events_unbound	23:40:02	0
1298883	99	69	62,5	cyclictest	1301398	-21	kworker/u4:4+events_unbound	19:21:44	0
1298885	99	68	52,5	cyclictest	1450775	-21	kworker/u4:4+events_unbound	23:40:01	1
1298885	99	66	59,5	cyclictest	1406786	-21	kworker/u4:9+events_unbound	23:12:30	1
1298883	99	64	56,6	cyclictest	1300117	-21	kworker/u4:9+events_unbound	20:31:58	0
1298883	99	63	56,5	cyclictest	1304056	-21	kworker/u4:10+events_unbound	20:16:54	0
1298885	99	62	3,47	cyclictest	1466819	-21	ssh	23:45:34	1
1298883	99	62	52,7	cyclictest	1501000	-21	/usr/sbin/munin	00:25:13	0
1298883	99	62	49,9	cyclictest	378	-21	dbus-daemon	23:52:13	0
1298885	99	61	50,7	cyclictest	1353950	-21	kworker/u4:6+events_unbound	21:52:15	1
1298883	99	61	56,3	cyclictest	1059055	-21	kworker/u4:6+events_unbound	19:36:47	0
1298885	99	60	51,6	cyclictest	1411278	-21	kworker/u4:2+events_unbound	22:42:24	1
1298883	99	60	40,16	cyclictest	0	-21	swapper/0	23:45:14	0
1298885	99	59	51,6	cyclictest	1402294	-21	kworker/u4:5+events_unbound	22:32:23	1
1298883	99	59	47,8	cyclictest	1476861	-21	kworker/u4:8-rpciod	00:00:21	0
1298883	99	59	1,11	cyclictest	1496619	-21	apt-get	00:20:12	0
1298883	99	59	1,11	cyclictest	1496619	-21	apt-get	00:20:11	0
1298883	99	58	51,5	cyclictest	1304056	-21	kworker/u4:10+events_unbound	19:51:49	0
1298885	99	57	49,6	cyclictest	1384467	-21	kworker/u4:10+events_unbound	22:12:18	1
1298883	99	57	6,47	cyclictest	1478851	-21	sed	00:00:00	0
1298883	99	57	50,5	cyclictest	1327026	-21	kworker/u4:4+events_unbound	21:00:02	0
1298883	99	57	50,5	cyclictest	1302697	-21	kworker/u4:2+events_unbound	20:11:55	0
1298883	99	57	50,5	cyclictest	1301398	-21	kworker/u4:4+events_unbound	20:21:56	0
1298883	99	57	49,6	cyclictest	1376211	-21	kworker/u4:0+events_unbound	22:17:01	0
1298883	99	57	49,6	cyclictest	1376211	-21	kworker/u4:0+events_unbound	22:17:01	0
1298883	99	57	42,12	cyclictest	53	-21	kswapd0	22:55:11	0
1298885	99	56	48,5	cyclictest	1387412	-21	gltestperf	22:15:20	1
1298885	99	56	48,5	cyclictest	1387412	-21	gltestperf	22:15:19	1
1298885	99	56	47,7	cyclictest	1411278	-21	kworker/u4:2+events_unbound	22:57:26	1
1298883	99	56	49,5	cyclictest	1450773	-21	kworker/u4:2+events_unbound	23:29:59	0
1298885	99	55	47,6	cyclictest	0	-21	swapper/1	22:20:32	1
1298883	99	55	47,6	cyclictest	1441887	-21	kworker/u4:11+events_unbound	23:19:00	0
1298883	99	55	47,6	cyclictest	1415737	-21	kworker/u4:3+events_unbound	23:05:12	0
1298883	99	55	44,8	cyclictest	1513031	-21	sshd	00:38:47	0
1298883	99	55	42,8	cyclictest	1404494	-21	apt-get	22:35:13	0
1298885	99	54	47,5	cyclictest	1476861	-21	kworker/u4:8+events_unbound	00:05:38	1
1298885	99	54	47,5	cyclictest	1450776	-21	kworker/u4:5+events_unbound	23:27:33	1
1298885	99	54	45,7	cyclictest	1397880	-21	kworker/u4:7+events_unbound	22:37:24	1
1298883	99	54	47,5	cyclictest	1300117	-21	kworker/u4:9+events_unbound	19:56:50	0
1298885	99	53	47,4	cyclictest	1307893	-21	kworker/u4:15+events_unbound	21:20:01	1
1298883	99	53	44,6	cyclictest	1388940	-21	kworker/u4:15+events_unbound	22:23:00	0
1298883	99	53	40,8	cyclictest	0	-21	swapper/0	21:10:19	0
1298883	99	53	35,6	cyclictest	1402295	-21	kworker/u4:6+events_unbound	22:41:00	0
1298885	99	52	45,5	cyclictest	1418661	-21	sendmail_mailqu	22:50:37	1
1298885	99	52	45,5	cyclictest	1415722	-21	systemd	22:47:24	1
1298885	99	52	45,5	cyclictest	1328022	-21	cat	21:00:36	1
1298885	99	52	45,5	cyclictest	1323725	-21	fschecks_time	20:45:21	1
1298883	99	52	43,6	cyclictest	1427123	-21	/usr/sbin/munin	23:00:23	0
1298883	99	52	28,20	cyclictest	1316813	-21	kworker/u4:6-rpciod	21:05:00	0
1298883	99	52	2,18	cyclictest	1505571	-21	cpu	00:30:17	0
1298885	99	51	41,7	cyclictest	1430706	-21	apt-key	23:05:01	1
1298885	99	50	43,5	cyclictest	1501605	-21	kernelversion	00:25:24	1
1298885	99	50	43,5	cyclictest	1374383	-21	/usr/sbin/munin	22:00:23	1
1298885	99	50	39,8	cyclictest	0	-21	swapper/1	22:25:13	1
1298885	99	50	38,10	cyclictest	0	-21	swapper/1	21:20:14	1
1298883	99	50	4,38	cyclictest	1417737	-21	tr	22:50:14	0
1298883	99	50	43,5	cyclictest	1367291	-21	kworker/u4:8+events_unbound	22:04:01	0
1298883	99	50	23,23	cyclictest	15	1	ktimers/0	20:00:24	0
1298885	99	49	41,6	cyclictest	1300117	-21	kworker/u4:9+events_unbound	20:36:59	1
1298885	99	49	34,5	cyclictest	1304056	-21	kworker/u4:10+events_unbound	19:50:01	1
1298883	99	49	42,5	cyclictest	1059055	-21	kworker/u4:6+events_unbound	19:46:48	0
1298885	99	48	42,4	cyclictest	1493366	-21	timerandwakeup	00:15:37	1
1298885	99	48	42,4	cyclictest	1305317	-21	systemd	19:31:43	1
1298885	99	48	41,5	cyclictest	1444371	-21	fschecks_count	23:20:19	1
1298885	99	48	41,4	cyclictest	0	-21	swapper/1	20:40:14	1
1298885	99	48	40,6	cyclictest	1307893	-21	kworker/u4:15+events_unbound	20:01:50	1
1298885	99	48	39,7	cyclictest	1315525	-21	systemd	20:11:53	1
1298885	99	48	39,6	cyclictest	1500821	-21	latency_hist	00:25:02	1
1298885	99	48	39,6	cyclictest	1500821	-21	latency_hist	00:25:01	1
1298885	99	48	39,6	cyclictest	1478801	-21	sendmail	00:00:01	1
1298885	99	48	39,6	cyclictest	0	-21	swapper/1	21:35:35	1
1298885	99	48	38,8	cyclictest	37	-21	kcompactd0	21:50:01	1
1298885	99	47	41,4	cyclictest	1325747	-21	systemd	20:51:59	1
1298885	99	47	40,5	cyclictest	1357519	-21	unixbench_multi	21:40:40	1
1298885	99	47	40,5	cyclictest	1343506	-21	fschecks_time	21:25:24	1
1298885	99	47	37,6	cyclictest	1506176	-21	diskmemload	00:30:29	1
1298885	99	47	28,6	cyclictest	1301398	-21	kworker/u4:4+events_unbound	19:20:01	1
1298883	99	47	40,5	cyclictest	1336132	-21	kworker/u4:13+events_unbound	21:22:46	0
1298883	99	47	33,5	cyclictest	1367290	-21	kworker/u4:7+events_unbound	22:11:59	0
1298885	99	46	37,6	cyclictest	1430551	-21	cron	23:05:13	1
1298885	99	46	27,6	cyclictest	1454581	-21	kworker/u4:9+events_unbound	23:40:13	1
1298883	99	46	39,5	cyclictest	1395994	-21	df_abs	22:25:17	0
1298883	99	46	38,4	cyclictest	0	-21	swapper/0	21:05:16	0
1298883	99	46	37,6	cyclictest	1329624	-21	diskmemload	00:10:02	0
1298885	99	45	39,4	cyclictest	1347870	-21	gltestperf	21:30:18	1
1298885	99	45	38,5	cyclictest	1330282	-21	fschecks_count	21:10:25	1
1298885	99	45	38,5	cyclictest	1310911	-21	fschecks_count	19:55:20	1
1298885	99	45	38,4	cyclictest	1441883	-21	ssh	23:17:05	1
1298885	99	45	37,6	cyclictest	0	-21	swapper/1	21:55:28	1
1298885	99	45	35,7	cyclictest	1299878	-21	threads	19:10:36	1
1298883	99	45	36,6	cyclictest	1304294	-21	aten_r7power_cu	19:30:14	0
1298883	99	45	35,7	cyclictest	1322381	-21	fschecks_count	20:40:19	0
1298885	99	44	37,5	cyclictest	1314179	-21	unixbench_multi	20:05:40	1
1298885	99	44	36,6	cyclictest	1328543	-21	aten2__	21:05:16	1
1298885	99	44	34,8	cyclictest	317	-21	systemd-journal	00:13:20	1
1298883	99	44	33,8	cyclictest	1319389	-21	cron	20:30:00	0
1298885	99	43	37,4	cyclictest	0	-21	swapper/1	20:21:54	1
1298885	99	43	36,5	cyclictest	1513849	-21	idleruntime-cro	00:40:01	1
1298885	99	43	35,6	cyclictest	1303380	-21	tail	19:25:24	1
1298885	99	43	33,7	cyclictest	1470397	-21	aten_r7power_en	23:50:15	1
1298883	99	43	35,6	cyclictest	1300579	-21	fschecks_count	19:15:18	0
1298883	99	43	2,34	cyclictest	1314280	-21	modprobe	20:10:01	0
1298885	99	42	36,4	cyclictest	1452749	-21	apt-get	23:30:10	1
1298885	99	42	33,6	cyclictest	1309288	-21	latency_hist	19:50:01	1
1298883	99	42	24,6	cyclictest	1321917	-21	kworker/u4:8+events_unbound	20:51:58	0
1298885	99	41	35,4	cyclictest	1327058	-21	idleruntime-cro	20:59:59	1
1298883	99	41	32,6	cyclictest	1286179	-21	kworker/u4:8-rpciod	19:10:18	0
1298885	99	40	33,5	cyclictest	1315754	-21	aten2__	20:15:15	1
1298885	99	40	33,5	cyclictest	0	-21	swapper/1	19:36:45	1
1298885	99	40	32,4	cyclictest	1307908	-21	kworker/u4:9	19:45:00	1
1298885	99	40	30,7	cyclictest	1320674	-21	idleruntime-cro	20:35:01	1
1298883	99	40	30,6	cyclictest	0	-21	swapper/0	20:35:36	0
1298885	99	39	31,6	cyclictest	1302698	-21	kworker/u4:3+events_unbound	20:26:57	1
1298885	99	39	27,8	cyclictest	0	-21	swapper/1	19:20:35	1
1298885	99	32	24,5	cyclictest	0	-21	swapper/1	19:10:02	1
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17