OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #7, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system

Characteristics of the 376 highest latencies:

System rack7slot2.osadl.org (updated Wed Dec 10, 2025 00:44:10)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
743953	99	131	125,4	cyclictest	760742	-21	kworker/u4:10+events_unbound	20:51:57	1
743953	99	118	109,6	cyclictest	860622	-21	kworker/u4:9+events_unbound	22:52:28	1
743953	99	85	37,37	cyclictest	860622	-21	kworker/u4:9-rpciod	22:55:24	1
743951	99	85	81,2	cyclictest	798630	-21	kworker/u4:1+events_unbound	21:42:10	0
743951	99	82	72,7	cyclictest	763344	-21	kworker/u4:5+events_unbound	21:12:03	0
743951	99	82	33,29	cyclictest	797426	-21	mailstats	21:35:39	0
743951	99	82	33,29	cyclictest	797426	-21	mailstats	21:35:38	0
743951	99	81	67,8	cyclictest	911657	-21	kworker/u4:2+events_unbound	00:30:15	0
743951	99	77	68,6	cyclictest	840058	-21	kworker/u4:5+events_unbound	22:27:23	0
743953	99	76	66,6	cyclictest	781682	-21	kworker/u4:1+events_unbound	21:30:01	1
743953	99	75	67,5	cyclictest	840058	-21	kworker/u4:5+events_unbound	22:42:19	1
743953	99	75	59,6	cyclictest	835756	-21	kworker/u4:7+events_unbound	23:17:32	1
743953	99	74	64,7	cyclictest	920056	-21	kworker/u4:12+events_unbound	00:22:42	1
743951	99	74	28,27	cyclictest	0	-21	swapper/0	22:10:16	0
743953	99	73	41,28	cyclictest	751384	-21	sendmail_mailqu	19:35:36	1
743951	99	73	63,7	cyclictest	840058	-21	kworker/u4:5+events_unbound	22:42:27	0
743951	99	73	62,8	cyclictest	865019	-21	kworker/u4:4+events_unbound	23:22:36	0
743951	99	73	42,28	cyclictest	863391	-21	gltestperf	22:55:22	0
743953	99	72	61,8	cyclictest	931025	-21	kworker/u4:13+events_unbound	00:17:41	1
743953	99	72	61,8	cyclictest	931025	-21	kworker/u4:13+events_unbound	00:17:40	1
743953	99	72	2,16	cyclictest	820591	-21	ssh	22:04:31	1
743951	99	72	64,6	cyclictest	802850	-21	kworker/u4:2+events_unbound	22:07:22	0
743951	99	71	2,13	cyclictest	855069	-21	ssh	22:45:29	0
743953	99	70	61,6	cyclictest	840060	-21	kworker/u4:11+events_unbound	22:37:26	1
743953	99	70	34,31	cyclictest	753930	-21	ps	19:45:36	1
743953	99	70	34,31	cyclictest	753930	-21	ps	19:45:35	1
743951	99	69	60,7	cyclictest	755587	-21	kworker/u4:18+events_unbound	20:41:57	0
743951	99	69	33,33	cyclictest	753931	-21	grep	19:45:35	0
743951	99	69	33,33	cyclictest	753931	-21	grep	19:45:35	0
743953	99	68	37,28	cyclictest	925131	-21	apt-get	00:10:00	1
743951	99	68	37,28	cyclictest	772982	-21	gltestperf	21:00:21	0
743951	99	68	31,33	cyclictest	816729	-21	cron	22:00:01	0
743951	99	68	2,17	cyclictest	805429	-21	ntpq	21:45:28	0
743951	99	68	1,14	cyclictest	817055	-21	apt-get	22:00:13	0
743953	99	67	38,26	cyclictest	940795	-21	basename	00:30:22	1
743953	99	67	34,31	cyclictest	921430	-21	idleruntime-cro	00:05:00	1
743953	99	67	34,31	cyclictest	921430	-21	idleruntime-cro	00:04:59	1
743951	99	67	41,22	cyclictest	896836	-21	/usr/sbin/munin	23:35:18	0
743951	99	67	34,30	cyclictest	746947	-21	fschecks_count	19:20:18	0
743951	99	67	2,15	cyclictest	829978	-21	ssh	22:15:22	0
743951	99	66	42,21	cyclictest	0	-21	swapper/0	21:30:10	0
743951	99	66	11,48	cyclictest	914359	-21	ssh	23:55:30	0
743953	99	65	40,22	cyclictest	749207	-21	munin-plugin-st	19:30:00	1
743953	99	65	29,32	cyclictest	829993	-21	kernelversion	22:15:21	1
743951	99	65	6,9	cyclictest	0	-21	swapper/0	00:10:41	0
743951	99	65	55,7	cyclictest	749161	-21	kworker/u4:1+events_unbound	20:11:49	0
743951	99	65	33,29	cyclictest	845721	-21	cron	22:35:00	0
743951	99	65	33,29	cyclictest	0	-21	swapper/0	22:35:39	0
743951	99	65	30,32	cyclictest	867838	-21	awk	23:00:28	0
743951	99	64	7,51	cyclictest	944980	-21	ssh	00:35:27	0
743951	99	64	54,7	cyclictest	873435	-21	kworker/u4:11+events_unbound	23:07:33	0
743951	99	64	32,29	cyclictest	0	-21	swapper/0	21:20:12	0
743951	99	64	2,18	cyclictest	880099	-21	/usr/sbin/munin	23:15:23	0
743951	99	64	2,15	cyclictest	891380	-21	ssh	23:29:05	0
743951	99	64	2,15	cyclictest	891380	-21	ssh	23:29:04	0
743951	99	64	1,7	cyclictest	875771	-21	diskstats	23:10:24	0
743953	99	63	56,5	cyclictest	946277	-21	kworker/u4:5+events_unbound	00:37:46	1
743953	99	63	53,6	cyclictest	919500	-21	kworker/u4:11+events_unbound	00:25:01	1
743953	99	63	35,26	cyclictest	905493	-21	fschecks_time	23:45:24	1
743951	99	63	53,8	cyclictest	379	-21	dbus-daemon	20:46:56	0
743951	99	63	2,8	cyclictest	750630	-21	munin-node	19:35:11	0
743953	99	62	55,5	cyclictest	802854	-21	kworker/u4:9+events_unbound	21:45:01	1
743953	99	62	48,10	cyclictest	0	-21	swapper/1	22:45:25	1
743953	99	62	4,48	cyclictest	897253	-21	ssh	23:35:27	1
743953	99	62	32,28	cyclictest	875576	-21	aten_r7power_en	23:10:22	1
743953	99	62	28,32	cyclictest	808918	-21	aten_r7power_en	21:50:15	1
743951	99	62	40,20	cyclictest	925134	-21	munin-run	00:10:00	0
743951	99	62	2,13	cyclictest	910187	-21	ntpq	23:50:32	0
743951	99	62	18,40	cyclictest	901871	-21	sed	23:40:39	0
743953	99	61	51,8	cyclictest	749169	-21	kworker/u4:11+events_unbound	19:31:43	1
743953	99	61	29,30	cyclictest	0	-21	swapper/1	22:25:33	1
743951	99	61	4,14	cyclictest	937063	-21	apt-get	00:25:14	0
743951	99	61	35,23	cyclictest	0	-21	swapper/0	20:15:25	0
743951	99	61	2,9	cyclictest	746001	-21	ntpq	19:15:31	0
743951	99	61	2,8	cyclictest	918587	-21	ntpq	00:00:34	0
743951	99	61	2,8	cyclictest	918587	-21	ntpq	00:00:33	0
743951	99	61	2,15	cyclictest	766123	-21	apt-get	20:35:10	0
743953	99	60	54,3	cyclictest	911660	-21	kworker/u4:7+events_unbound	00:12:42	1
743953	99	60	35,23	cyclictest	901557	-21	ntpq	23:40:31	1
743953	99	60	3,15	cyclictest	783922	-21	apt-get	21:20:12	1
743953	99	60	28,29	cyclictest	0	-21	swapper/1	23:00:20	1
743953	99	60	2,18	cyclictest	884626	-21	ps	23:20:38	1
743953	99	60	2,15	cyclictest	764672	-21	idleruntime-cro	20:29:59	1
743953	99	60	12,27	cyclictest	915024	-21	ssh	23:55:48	1
743951	99	60	5,16	cyclictest	764671	-21	latency_hist	20:30:00	0
743951	99	60	3,11	cyclictest	770540	-21	ntpq	20:50:26	0
743951	99	60	2,16	cyclictest	808825	-21	apt-get	21:50:12	0
743951	99	60	13,8	cyclictest	834409	-21	sh	22:20:36	0
743953	99	59	51,6	cyclictest	827380	-21	kworker/u4:9+events_unbound	22:15:01	1
743951	99	59	4,9	cyclictest	933327	-21	/usr/sbin/munin	00:20:18	0
743951	99	59	34,20	cyclictest	0	-21	swapper/0	23:45:33	0
743951	99	59	31,26	cyclictest	751960	-21	aten2__	19:40:14	0
743953	99	58	48,7	cyclictest	745187	-21	kworker/u4:3+events_unbound	19:21:41	1
743953	99	58	34,21	cyclictest	821609	-21	gltestperf	22:05:22	1
743953	99	58	29,27	cyclictest	763344	-21	kworker/u4:5-rpciod	21:15:32	1
743953	99	58	23,31	cyclictest	745190	-21	kworker/u4:8+rpciod	19:15:18	1
743951	99	58	4,9	cyclictest	765939	-21	cron	20:35:00	0
743951	99	58	49,7	cyclictest	750436	-21	kworker/u4:13+events_unbound	19:51:47	0
743951	99	58	2,9	cyclictest	891965	-21	apt-get	23:30:04	0
743951	99	58	2,16	cyclictest	780526	-21	ntpq	21:15:38	0
743953	99	57	4,11	cyclictest	796420	-21	ssh	21:35:15	1
743953	99	57	4,11	cyclictest	796420	-21	ssh	21:35:14	1
743953	99	57	32,23	cyclictest	762027	-21	systemd	20:16:50	1
743953	99	57	26,27	cyclictest	0	-21	swapper/1	22:20:16	1
743953	99	57	23,31	cyclictest	0	-21	swapper/1	23:30:44	1
743951	99	57	9,5	cyclictest	758204	-21	cron	20:05:01	0
743951	99	57	48,7	cyclictest	379	-21	dbus-daemon	20:06:48	0
743951	99	57	48,7	cyclictest	379	-21	dbus-daemon	20:06:47	0
743951	99	57	3,10	cyclictest	859315	-21	ntpq	22:50:29	0
743951	99	57	1,10	cyclictest	929839	-21	ntpq	00:15:27	0
743951	99	57	1,10	cyclictest	929839	-21	ntpq	00:15:27	0
743953	99	56	47,7	cyclictest	745191	-21	kworker/u4:9+events_unbound	20:56:58	1
743953	99	56	32,22	cyclictest	775759	-21	gltestperf	21:10:21	1
743953	99	56	2,14	cyclictest	773141	-21	ntpq	21:00:26	1
743953	99	56	12,27	cyclictest	889887	-21	ssh	23:26:38	1
743953	99	56	12,27	cyclictest	889887	-21	ssh	23:26:37	1
743951	99	56	51,3	cyclictest	755586	-21	kworker/u4:17+events_unbound	19:56:46	0
743951	99	56	48,6	cyclictest	379	-21	dbus-daemon	19:31:42	0
743951	99	56	4,10	cyclictest	744713	-21	ntpq	19:10:29	0
743953	99	55	4,18	cyclictest	871601	-21	diskstats	23:05:24	1
743953	99	55	23,29	cyclictest	0	-21	swapper/1	23:50:28	1
743953	99	55	1,14	cyclictest	752516	-21	ntpq	19:40:23	1
743951	99	55	20,31	cyclictest	0	-21	swapper/0	19:25:35	0
743951	99	55	1,12	cyclictest	788117	-21	apt-get	21:25:15	0
743953	99	54	32,20	cyclictest	769795	-21	systemd	20:46:56	1
743953	99	54	3,11	cyclictest	842718	-21	ssh	22:30:32	1
743953	99	54	2,5	cyclictest	805338	-21	ntpq	21:45:25	1
743953	99	54	2,47	cyclictest	813747	-21	ssh	21:55:38	1
743953	99	54	2,14	cyclictest	794982	-21	ssh	21:33:05	1
743953	99	54	2,12	cyclictest	744647	-21	ntpq	19:10:24	1
743953	99	54	1,14	cyclictest	755171	-21	ntpq	19:50:35	1
743951	99	54	19,33	cyclictest	257	50	irq/23-42890000.ethernet	21:00:01	0
743951	99	54	1,11	cyclictest	774992	-21	latency_hist	21:10:00	0
743953	99	53	3,6	cyclictest	757683	-21	ntpq	20:00:29	1
743953	99	53	2,8	cyclictest	760224	-21	ntpq	20:10:27	1
743953	99	52	31,19	cyclictest	758740	-21	gltestperf	20:05:21	1
743953	99	52	31,19	cyclictest	758740	-21	gltestperf	20:05:21	1
743953	99	52	27,22	cyclictest	0	-21	swapper/1	20:30:16	1
743953	99	52	2,13	cyclictest	756427	-21	ntpq	19:55:30	1
743953	99	51	6,10	cyclictest	763412	-21	latency_hist	20:25:01	1
743953	99	51	4,7	cyclictest	766777	-21	ntpq	20:35:33	1
743953	99	50	2,6	cyclictest	774483	-21	ntpq	21:05:32	1
743953	99	50	15,32	cyclictest	0	-21	swapper/1	20:40:16	1
743951	99	50	2,13	cyclictest	762274	-21	apt-get	20:20:12	0
743951	99	50	2,11	cyclictest	744056	-21	latency_hist	19:10:02	0
743951	99	50	2,11	cyclictest	744056	-21	latency_hist	19:10:01	0
743953	99	42	1,10	cyclictest	743965	-21	sort	19:10:01	1
743953	99	42	1,10	cyclictest	743965	-21	sort	19:10:01	1
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17