OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rackeslot3s.osadl.org (updated Fri Aug 28, 2026 00:45:29)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
2305277	99	81	18,40	cyclictest	12	-21	ksoftirqd/0	00:12:09	0
2305277	99	81	18,40	cyclictest	12	-21	ksoftirqd/0	00:12:08	0
2305277	99	76	14,40	cyclictest	12	-21	ksoftirqd/0	23:59:59	0
2305286	99	75	22,12	cyclictest	2400114	-21	sshd	00:18:10	2
2305277	99	75	71,2	cyclictest	0	-21	swapper/0	19:15:01	0
2305277	99	72	8,42	cyclictest	12	-21	ksoftirqd/0	22:38:34	0
2305286	99	69	28,10	cyclictest	30	-21	ksoftirqd/2	21:23:22	2
2305286	99	69	28,10	cyclictest	30	-21	ksoftirqd/2	21:23:22	2
2305277	99	69	4,23	cyclictest	2306497	-21	modprobe	19:35:01	0
2305277	99	69	10,38	cyclictest	12	-21	ksoftirqd/0	21:23:57	0
2305277	99	69	10,38	cyclictest	12	-21	ksoftirqd/0	21:23:56	0
2305277	99	67	18,39	cyclictest	12	-21	ksoftirqd/0	22:32:21	0
2305277	99	66	17,40	cyclictest	12	-21	ksoftirqd/0	22:48:19	0
2305286	99	65	31,14	cyclictest	2405817	-21	latency_hist	00:30:02	2
2305277	99	65	18,43	cyclictest	12	-21	ksoftirqd/0	00:37:29	0
2305286	99	64	41,14	cyclictest	0	-21	swapper/2	00:15:02	2
2305286	99	64	41,14	cyclictest	0	-21	swapper/2	00:15:01	2
2305277	99	64	7,53	cyclictest	2374995	-21	sh	23:25:01	0
2305286	99	63	27,9	cyclictest	30	-21	ksoftirqd/2	22:55:03	2
2305277	99	62	27,3	cyclictest	2319274	1	ssh	21:26:17	0
2305286	99	61	25,12	cyclictest	30	-21	ksoftirqd/2	22:42:41	2
2305286	99	61	14,5	cyclictest	13	1	rcu_preempt	22:47:36	2
2305277	99	61	9,40	cyclictest	12	-21	ksoftirqd/0	23:34:33	0
2305277	99	61	15,37	cyclictest	12	-21	ksoftirqd/0	22:15:30	0
2305277	99	59	5,12	cyclictest	2325568	-21	unin-run	21:40:01	0
2305277	99	59	5,12	cyclictest	2325568	-21	unin-run	21:40:00	0
2305277	99	59	39,15	cyclictest	2306754	-21	modprobe	19:39:59	0
2305277	99	58	3,19	cyclictest	2334966	-21	cron	22:00:01	0
2305277	99	58	3,19	cyclictest	2334966	-21	cron	22:00:00	0
2305277	99	58	26,19	cyclictest	13	1	rcu_preempt	00:04:16	0
2305277	99	58	10,38	cyclictest	12	-21	ksoftirqd/0	23:10:44	0
2305277	99	57	28,13	cyclictest	120	-21	kswapd0	22:55:01	0
2305277	99	57	2,19	cyclictest	1	-21	systemd	23:00:00	0
2305277	99	57	21,32	cyclictest	12	-21	ksoftirqd/0	23:27:06	0
2305277	99	56	20,32	cyclictest	12	-21	ksoftirqd/0	22:42:49	0
2305286	99	55	25,10	cyclictest	275	-21	dbus-daemon	19:35:02	2
2305277	99	55	34,14	cyclictest	2310914	-21	systemctl	20:59:59	0
2305277	99	55	30,6	cyclictest	13	1	rcu_preempt	00:29:25	0
2305282	99	54	16,27	cyclictest	2363468	-21	ssh	23:00:02	1
2305277	99	54	24,15	cyclictest	2335919	-21	ssh	22:01:36	0
2305277	99	54	19,30	cyclictest	12	-21	ksoftirqd/0	22:28:21	0
2305277	99	54	1,48	cyclictest	2370609	-21	ssh	23:15:12	0
2305277	99	54	14,6	cyclictest	13	1	rcu_preempt	21:54:59	0
2305277	99	53	29,9	cyclictest	2366736	-21	ssh	23:07:14	0
2305277	99	53	26,12	cyclictest	2383076	-21	ssh	23:41:50	0
2305277	99	53	26,11	cyclictest	2386583	-21	ssh	23:49:31	0
2305277	99	53	23,4	cyclictest	13	1	rcu_preempt	00:24:11	0
2305277	99	53	22,26	cyclictest	275	-21	dbus-daemon	20:25:01	0
2305286	99	52	44,5	cyclictest	2310162	-21	kworker/2:2	23:40:22	2
2305286	99	52	43,5	cyclictest	2335554	-21	kworker/2:1	00:25:02	2
2305286	99	52	43,5	cyclictest	2309381	-21	kworker/2:3	21:11:54	2
2305286	99	52	41,6	cyclictest	2335554	-21	kworker/2:1	00:08:33	2
2305277	99	52	3,14	cyclictest	2311601	-21	cat	21:10:03	0
2305277	99	52	25,12	cyclictest	2363770	-21	ssh	23:00:38	0
2305277	99	52	24,13	cyclictest	2394593	-21	ssh	00:06:15	0
2305277	99	52	20,28	cyclictest	12	-21	ksoftirqd/0	21:10:48	0
2305277	99	51	2,44	cyclictest	2327947	-21	date	21:45:02	0
2305286	99	50	42,5	cyclictest	2310162	-21	kworker/2:2	23:15:01	2
2305286	99	50	42,5	cyclictest	2310162	-21	kworker/2:2	00:35:15	2
2305286	99	50	41,5	cyclictest	2310162	-21	kworker/2:2	21:59:17	2
2305286	99	50	41,5	cyclictest	2310162	-21	kworker/2:2	21:59:16	2
2305286	99	50	41,5	cyclictest	2219619	-21	kworker/2:1	20:10:02	2
2305286	99	50	40,6	cyclictest	2310162	-21	kworker/2:2	22:18:53	2
2305286	99	50	32,13	cyclictest	275	-21	dbus-daemon	21:35:01	2
2305282	99	50	4,3	cyclictest	13	1	rcu_preempt	23:52:31	1
2305282	99	50	3,5	cyclictest	13	1	rcu_preempt	23:50:00	1
2305282	99	50	2,13	cyclictest	2305706	-21	cron	19:20:00	1
2305277	99	50	4,38	cyclictest	0	-21	swapper/0	22:15:03	0
2305277	99	50	43,4	cyclictest	168	50	irq/134-8000000	23:55:01	0
2305288	99	49	6,8	cyclictest	37	-21	ksoftirqd/3	22:25:02	3
2305288	99	49	4,4	cyclictest	13	1	rcu_preempt	21:15:01	3
2305288	99	49	35,10	cyclictest	275	-21	dbus-daemon	23:55:01	3
2305286	99	49	41,5	cyclictest	2310162	-21	kworker/2:2	23:55:28	2
2305286	99	49	41,5	cyclictest	2310162	-21	kworker/2:2	23:38:46	2
2305286	99	49	41,5	cyclictest	2310162	-21	kworker/2:2	23:34:58	2
2305286	99	49	41,5	cyclictest	2310162	-21	kworker/2:2	23:16:08	2
2305286	99	49	41,5	cyclictest	2310162	-21	kworker/2:2	21:29:29	2
2305286	99	49	40,6	cyclictest	2335554	-21	kworker/2:1	23:07:02	2
2305286	99	49	1,10	cyclictest	2306265	-21	munin-run	19:30:01	2
2305282	99	49	5,3	cyclictest	13	1	rcu_preempt	21:50:38	1
2305282	99	49	4,4	cyclictest	13	1	rcu_preempt	00:40:00	1
2305282	99	49	2,8	cyclictest	0	-21	swapper/1	21:30:02	1
2305277	99	49	40,6	cyclictest	193	50	irq/34-mmc1	19:10:02	0
2305277	99	49	40,6	cyclictest	193	50	irq/34-mmc1	19:10:02	0
2305277	99	49	38,6	cyclictest	168	50	irq/134-8000000	21:19:59	0
2305277	99	49	36,10	cyclictest	169	50	irq/224-8000000	21:45:07	0
2305277	99	49	36,10	cyclictest	169	50	irq/224-8000000	21:45:06	0
2305277	99	49	3,5	cyclictest	13	1	rcu_preempt	21:32:17	0
2305277	99	49	31,14	cyclictest	2346963	-21	grep	22:25:02	0
2305277	99	49	25,4	cyclictest	13	1	rcu_preempt	22:10:00	0
2305288	99	48	31,12	cyclictest	1	-21	systemd	00:05:01	3
2305286	99	48	42,4	cyclictest	30	-21	ksoftirqd/2	22:35:03	2
2305286	99	48	40,5	cyclictest	2335554	-21	kworker/2:1	22:59:26	2
2305286	99	48	40,5	cyclictest	2310162	-21	kworker/2:2	23:45:05	2
2305286	99	48	40,5	cyclictest	2310162	-21	kworker/2:2	23:28:24	2
2305286	99	48	40,5	cyclictest	2219619	-21	kworker/2:1	19:15:01	2
2305282	99	48	6,8	cyclictest	23	-21	ksoftirqd/1	21:13:38	1
2305282	99	48	5,3	cyclictest	13	1	rcu_preempt	22:15:55	1
2305282	99	48	4,4	cyclictest	13	1	rcu_preempt	23:33:24	1
2305282	99	48	4,3	cyclictest	13	1	rcu_preempt	00:11:32	1
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2021-09-01 - 00:33