OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rackeslot3s.osadl.org (updated Sat Aug 22, 2026 00:45:26)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
3190590	99	121	111,8	cyclictest	23	-21	ksoftirqd/1	20:55:02	1
3190587	99	103	99,2	cyclictest	0	-21	swapper/0	19:10:02	0
3190587	99	97	11,16	cyclictest	168	50	irq/134-8000000	22:00:03	0
3190587	99	73	10,45	cyclictest	169	50	irq/224-8000000	22:30:02	0
3190600	99	72	21,15	cyclictest	3251355	-21	ssh	23:05:37	3
3190600	99	71	21,15	cyclictest	3203416	-21	ssh	21:23:59	3
3190600	99	70	33,9	cyclictest	3291966	-21	ssh	00:32:03	3
3190600	99	69	50,15	cyclictest	3213157	-21	ssh	21:44:48	3
3190600	99	69	20,15	cyclictest	3193312	-21	cat	20:00:02	3
3190594	99	68	30,34	cyclictest	3283947	-21	sed	00:15:02	2
3190590	99	68	25,5	cyclictest	3229684	-21	sendmail	22:20:00	1
3190590	99	68	22,12	cyclictest	3467602	-21	systemd-journal	23:05:00	1
3190590	99	67	27,8	cyclictest	3235233	-21	sshd	22:31:19	1
3190590	99	67	22,5	cyclictest	309	-21	in:imuxsock	22:54:55	1
3190587	99	66	8,6	cyclictest	13	1	rcu_preempt	00:40:00	0
3190587	99	66	8,6	cyclictest	13	1	rcu_preempt	00:40:00	0
3190600	99	65	52,10	cyclictest	3222328	-21	ssh	22:04:18	3
3190600	99	65	25,13	cyclictest	3196815	-21	diskmemload	23:21:37	3
3190600	99	65	24,14	cyclictest	3271682	-21	kworker/u8:1	23:52:43	3
3190594	99	65	34,26	cyclictest	3248550	-21	ssh	23:00:00	2
3190594	99	65	2,6	cyclictest	0	-21	swapper/2	23:14:29	2
3190594	99	65	21,38	cyclictest	309	-21	in:imuxsock	19:35:00	2
3190590	99	65	25,7	cyclictest	3218009	-21	cat	21:55:02	1
3190594	99	64	2,19	cyclictest	3239115	-21	cron	22:40:00	2
3190590	99	64	31,29	cyclictest	3467602	-21	systemd-journal	20:50:00	1
3190590	99	64	24,7	cyclictest	23	-21	ksoftirqd/1	22:27:03	1
3190600	99	63	45,14	cyclictest	3264647	-21	ssh	23:34:17	3
3190600	99	63	43,16	cyclictest	3193931	-21	sshd	20:13:56	3
3190600	99	63	40,18	cyclictest	3237320	-21	ssh	22:35:46	3
3190600	99	63	24,11	cyclictest	0	-21	swapper/3	23:59:01	3
3190594	99	63	2,23	cyclictest	3269855	-21	sed	23:45:02	2
3190594	99	63	1,9	cyclictest	3278486	-21	ssh	00:03:19	2
3190594	99	63	1,9	cyclictest	3207718	-21	ssh	21:33:08	2
3190594	99	63	1,10	cyclictest	3284579	-21	ssh	00:16:09	2
3190590	99	63	18,13	cyclictest	3467602	-21	systemd-journal	19:19:59	1
3190587	99	63	5,33	cyclictest	169	50	irq/224-8000000	23:25:01	0
3190587	99	63	27,7	cyclictest	12	-21	ksoftirqd/0	22:04:06	0
3190600	99	62	51,9	cyclictest	3257076	-21	ssh	23:18:06	3
3190600	99	62	51,9	cyclictest	3257076	-21	ssh	23:18:06	3
3190600	99	62	50,9	cyclictest	3223817	-21	ssh	22:07:21	3
3190600	99	62	23,12	cyclictest	3213700	-21	ssh	21:45:38	3
3190594	99	62	37,5	cyclictest	3229746	-21	kworker/2:0	22:48:15	2
3190594	99	62	2,17	cyclictest	3239537	-21	ssh	22:40:20	2
3190594	99	62	2,15	cyclictest	3204525	-21	sshd	21:26:03	2
3190594	99	62	1,15	cyclictest	48	-21	kcompactd0	21:35:18	2
3190594	99	62	1,10	cyclictest	3194979	-21	sshd	20:34:56	2
3190590	99	62	19,11	cyclictest	23	-21	ksoftirqd/1	22:09:49	1
3190600	99	61	50,9	cyclictest	3253915	-21	ssh	23:11:10	3
3190600	99	61	48,10	cyclictest	3242182	-21	ssh	22:46:04	3
3190600	99	61	45,13	cyclictest	3293738	-21	ssh	00:35:42	3
3190600	99	61	45,13	cyclictest	3293738	-21	ssh	00:35:42	3
3190600	99	61	45,13	cyclictest	3270179	-21	ssh	23:45:30	3
3190600	99	61	45,13	cyclictest	3270179	-21	ssh	23:45:30	3
3190600	99	61	44,14	cyclictest	3219405	-21	ssh	21:58:06	3
3190600	99	61	43,14	cyclictest	3287402	-21	ssh	00:22:07	3
3190600	99	61	43,14	cyclictest	3261412	-21	ssh	23:27:17	3
3190600	99	61	43,14	cyclictest	3261412	-21	ssh	23:27:17	3
3190600	99	61	23,13	cyclictest	3282888	-21	ssh	00:12:43	3
3190600	99	61	19,14	cyclictest	364	-21	snmpd	21:34:22	3
3190594	99	61	2,7	cyclictest	3249156	-21	sshd	23:00:55	2
3190594	99	61	23,33	cyclictest	30	-21	ksoftirqd/2	22:30:29	2
3190594	99	61	2,12	cyclictest	3194178	-21	cron	20:17:01	2
3190594	99	61	1,9	cyclictest	3227791	-21	ssh	22:15:36	2
3190594	99	61	1,9	cyclictest	3217228	-21	ssh	21:53:26	2
3190594	99	61	1,12	cyclictest	3214815	-21	sh	21:48:18	2
3190594	99	61	1,10	cyclictest	3276261	-21	ssh	23:58:48	2
3190594	99	61	1,10	cyclictest	3274313	-21	ssh	23:54:43	2
3190590	99	61	4,18	cyclictest	3193093	-21	latency_hist	19:55:02	1
3190600	99	60	47,10	cyclictest	3267595	-21	ssh	23:40:04	3
3190600	99	60	45,11	cyclictest	3245840	-21	ssh	22:54:12	3
3190600	99	60	45,11	cyclictest	3230276	-21	ssh	22:20:45	3
3190600	99	60	44,13	cyclictest	3265271	-21	ssh	23:35:17	3
3190600	99	60	44,13	cyclictest	3265271	-21	ssh	23:35:17	3
3190600	99	60	43,13	cyclictest	3248225	-21	ssh	22:59:19	3
3190600	99	60	43,13	cyclictest	3233403	-21	ssh	22:27:37	3
3190600	99	60	43,13	cyclictest	3209421	-21	ssh	21:36:34	3
3190600	99	60	43,13	cyclictest	3198418	-21	ssh	21:13:16	3
3190600	99	60	42,14	cyclictest	3277372	-21	ssh	00:00:48	3
3190600	99	60	42,14	cyclictest	3226309	-21	ssh	22:12:38	3
3190600	99	60	41,15	cyclictest	3284863	-21	ssh	00:16:55	3
3190600	99	60	40,16	cyclictest	3194195	-21	sshd	20:19:54	3
3190594	99	60	3,24	cyclictest	3262635	-21	latency_hist	23:30:00	2
3190594	99	60	3,24	cyclictest	3262635	-21	latency_hist	23:30:00	2
3190594	99	60	31,24	cyclictest	3281230	-21	sh	00:09:21	2
3190594	99	60	1,9	cyclictest	3230857	-21	ssh	22:22:11	2
3190594	99	60	1,8	cyclictest	3257262	-21	ssh	23:18:29	2
3190594	99	60	1,8	cyclictest	3257262	-21	ssh	23:18:29	2
3190594	99	60	1,8	cyclictest	3233045	-21	ssh	22:26:45	2
3190594	99	60	1,12	cyclictest	3223414	-21	ssh	22:06:19	2
3190594	99	60	1,10	cyclictest	3264527	-21	ssh	23:34:02	2
3190590	99	60	19,12	cyclictest	3255557	-21	cron	23:15:01	1
3190587	99	60	17,7	cyclictest	12	-21	ksoftirqd/0	21:20:01	0
3190587	99	60	12,33	cyclictest	169	50	irq/224-8000000	20:45:03	0
3190600	99	59	47,9	cyclictest	3249946	-21	ssh	23:02:47	3
3190600	99	59	44,12	cyclictest	3289467	-21	ssh	00:26:31	3
3190600	99	59	43,13	cyclictest	3215976	-21	ssh	21:50:27	3
3190600	99	59	43,12	cyclictest	3235394	-21	ssh	22:31:40	3
3190600	99	59	43,12	cyclictest	3200479	-21	ssh	21:17:40	3
3190600	99	59	42,14	cyclictest	3228075	-21	ssh	22:16:12	3
3190594	99	59	1,8	cyclictest	3192048	-21	sshd	19:35:44	2
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2021-09-01 - 00:33