OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rackeslot3s.osadl.org (updated Sun May 10, 2026 00:45:36)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
3258217	99	101	2,4	cyclictest	13	1	rcu_preempt	22:50:02	1
3258214	99	80	3,55	cyclictest	0	-21	swapper/0	21:10:00	0
3258214	99	78	7,39	cyclictest	0	-21	swapper/0	21:05:01	0
3258214	99	78	10,31	cyclictest	1	-21	systemd	23:59:25	0
3258214	99	78	10,31	cyclictest	1	-21	systemd	23:59:24	0
3258214	99	77	1,73	cyclictest	0	-21	swapper/0	20:00:03	0
3258214	99	75	71,2	cyclictest	0	-21	swapper/0	00:05:02	0
3258214	99	75	71,2	cyclictest	0	-21	swapper/0	00:05:01	0
3258214	99	73	17,26	cyclictest	3323981	-21	ssh	23:14:34	0
3258214	99	71	14,20	cyclictest	3258262	-21	sed	19:10:02	0
3258214	99	70	13,21	cyclictest	3305616	-21	ssh	22:35:45	0
3258214	99	70	10,23	cyclictest	3331328	-21	ssh	23:29:57	0
3258214	99	70	10,23	cyclictest	3331328	-21	ssh	23:29:57	0
3258219	99	69	46,14	cyclictest	3269169	-21	kworker/2:1	23:19:17	2
3258214	99	68	11,18	cyclictest	3290666	-21	date	22:05:01	0
3258214	99	68	11,18	cyclictest	3290666	-21	date	22:05:01	0
3258214	99	68	0,34	cyclictest	0	-21	swapper/0	21:36:38	0
3258214	99	68	0,34	cyclictest	0	-21	swapper/0	21:36:38	0
3258214	99	67	15,16	cyclictest	3304031	-21	ssh	22:32:38	0
3258214	99	66	31,32	cyclictest	3312584	-21	ssh	22:50:17	0
3258214	99	66	16,46	cyclictest	3319916	-21	ssh	23:05:41	0
3258214	99	65	8,23	cyclictest	3349531	-21	ssh	00:07:48	0
3258214	99	65	31,31	cyclictest	3341764	-21	ssh	23:51:25	0
3258214	99	65	30,31	cyclictest	3309242	-21	ssh	22:43:31	0
3258214	99	64	30,31	cyclictest	3291338	-21	ssh	22:06:05	0
3258214	99	64	29,4	cyclictest	3276074	-21	sshd	21:34:24	0
3258214	99	64	15,14	cyclictest	3334150	-21	ssh	23:35:22	0
3258214	99	63	19,16	cyclictest	3353148	-21	sh	00:15:08	0
3258214	99	63	17,15	cyclictest	3327299	-21	ssh	23:21:06	0
3258214	99	63	15,15	cyclictest	3318630	-21	sh	23:03:09	0
3258214	99	63	10,37	cyclictest	3294599	-21	ssh	22:13:09	0
3258214	99	63	10,37	cyclictest	3294599	-21	ssh	22:13:08	0
3258219	99	62	44,7	cyclictest	3269169	-21	kworker/2:1	21:54:26	2
3258219	99	62	44,7	cyclictest	3269169	-21	kworker/2:1	21:54:26	2
3258214	99	62	41,6	cyclictest	168	50	irq/134-8000000	22:30:01	0
3258214	99	62	38,19	cyclictest	168	50	irq/134-8000000	23:00:01	0
3258214	99	62	32,4	cyclictest	3258378	-21	sshd	19:13:25	0
3258214	99	62	32,4	cyclictest	3258378	-21	sshd	19:13:24	0
3258214	99	62	15,13	cyclictest	3360940	-21	ssh	00:31:28	0
3258214	99	62	1,24	cyclictest	0	-21	swapper/0	21:47:23	0
3258214	99	61	9,35	cyclictest	3358189	-21	ssh	00:25:42	0
3258214	99	61	17,9	cyclictest	3273875	-21	ssh	21:29:58	0
3258214	99	61	17,9	cyclictest	3273875	-21	ssh	21:29:57	0
3258214	99	61	11,45	cyclictest	3331682	-21	ssh	23:30:13	0
3258214	99	61	11,45	cyclictest	3331682	-21	ssh	23:30:13	0
3258214	99	60	10,19	cyclictest	3338874	-21	ssh	23:45:08	0
3258214	99	60	10,19	cyclictest	3338874	-21	ssh	23:45:07	0
3258219	99	59	36,14	cyclictest	3246273	-21	kworker/2:3	22:59:14	2
3258214	99	58	36,5	cyclictest	168	50	irq/134-8000000	00:15:00	0
3258214	99	58	18,9	cyclictest	3312166	-21	ssh	22:49:52	0
3258219	99	57	50,3	cyclictest	3246273	-21	kworker/2:3	22:30:02	2
3258219	99	57	48,6	cyclictest	3329045	-21	kworker/2:0	00:26:48	2
3258214	99	57	6,30	cyclictest	13	1	rcu_preempt	00:25:01	0
3258214	99	57	6,30	cyclictest	13	1	rcu_preempt	00:25:00	0
3258214	99	57	4,17	cyclictest	3364938	-21	latency_hist	00:40:02	0
3258214	99	57	17,12	cyclictest	0	-21	swapper/0	23:43:45	0
3258219	99	56	47,6	cyclictest	3329045	-21	kworker/2:0	23:47:45	2
3258219	99	56	47,6	cyclictest	3329045	-21	kworker/2:0	23:47:44	2
3258214	99	56	22,29	cyclictest	168	50	irq/134-8000000	23:19:37	0
3258214	99	56	13,6	cyclictest	12	-21	ksoftirqd/0	21:21:03	0
3258214	99	56	13,13	cyclictest	3286382	-21	sh	21:55:47	0
3258219	99	55	46,5	cyclictest	3246273	-21	kworker/2:3	22:06:21	2
3258219	99	55	10,32	cyclictest	0	-21	swapper/2	22:00:01	2
3258224	99	54	37,12	cyclictest	1	-21	systemd	22:10:01	3
3258219	99	54	45,6	cyclictest	3269169	-21	kworker/2:1	22:04:50	2
3258219	99	54	45,6	cyclictest	3269169	-21	kworker/2:1	22:04:49	2
3258219	99	54	40,6	cyclictest	3246273	-21	kworker/2:3	22:15:44	2
3258219	99	54	27,9	cyclictest	3329045	-21	kworker/2:0	23:52:14	2
3258214	99	54	35,13	cyclictest	3467602	-21	systemd-journal	19:35:00	0
3258214	99	54	27,11	cyclictest	3284089	-21	ssh	21:50:56	0
3258214	99	54	27,11	cyclictest	3284089	-21	ssh	21:50:56	0
3258224	99	53	17,6	cyclictest	13	1	rcu_preempt	21:20:00	3
3258224	99	53	17,6	cyclictest	13	1	rcu_preempt	21:20:00	3
3258219	99	53	46,5	cyclictest	3246273	-21	kworker/2:3	22:31:44	2
3258219	99	53	45,5	cyclictest	3269169	-21	kworker/2:1	23:01:02	2
3258219	99	53	43,6	cyclictest	3329045	-21	kworker/2:0	00:09:30	2
3258219	99	53	43,6	cyclictest	3329045	-21	kworker/2:0	00:00:08	2
3258219	99	53	43,6	cyclictest	3329045	-21	kworker/2:0	00:00:08	2
3258219	99	53	35,8	cyclictest	3246273	-21	kworker/2:3	22:24:40	2
3258219	99	53	34,8	cyclictest	3269169	-21	kworker/2:1	22:47:04	2
3258214	99	53	15,6	cyclictest	12	-21	ksoftirqd/0	20:15:01	0
3258219	99	52	44,5	cyclictest	3246273	-21	kworker/2:3	22:36:45	2
3258219	99	52	44,5	cyclictest	3246273	-21	kworker/2:3	21:14:02	2
3258219	99	52	43,5	cyclictest	3246273	-21	kworker/2:3	22:44:24	2
3258219	99	52	42,7	cyclictest	3269169	-21	kworker/2:1	21:28:38	2
3258219	99	52	42,7	cyclictest	3269169	-21	kworker/2:1	21:28:37	2
3258219	99	52	42,6	cyclictest	3329045	-21	kworker/2:0	23:56:40	2
3258219	99	52	42,6	cyclictest	3329045	-21	kworker/2:0	23:56:39	2
3258219	99	52	27,9	cyclictest	3269169	-21	kworker/2:1	21:45:08	2
3258217	99	52	12,10	cyclictest	13	1	rcu_preempt	20:20:00	1
3258214	99	52	7,26	cyclictest	3259216	-21	kworker/0:2	20:00:02	0
3258214	99	52	33,11	cyclictest	3297831	-21	modprobe	22:19:59	0
3258214	99	52	1,36	cyclictest	3299100	-21	ssh	22:22:19	0
3258224	99	51	31,15	cyclictest	1	-21	systemd	00:00:05	3
3258224	99	51	31,15	cyclictest	1	-21	systemd	00:00:04	3
3258219	99	51	45,4	cyclictest	3329045	-21	kworker/2:0	00:10:19	2
3258219	99	51	44,4	cyclictest	3246273	-21	kworker/2:3	22:53:46	2
3258219	99	51	43,6	cyclictest	3269169	-21	kworker/2:1	23:34:43	2
3258219	99	51	43,6	cyclictest	3269169	-21	kworker/2:1	23:34:42	2
3258219	99	51	43,5	cyclictest	3329045	-21	kworker/2:0	00:33:38	2
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2021-09-01 - 00:33