OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rackeslot3s.osadl.org (updated Mon Aug 10, 2026 00:45:21)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
3445504	99	107	99,6	cyclictest	12	-21	ksoftirqd/0	21:30:02	0
3445504	99	102	96,3	cyclictest	0	-21	swapper/0	00:00:03	0
3445504	99	94	83,7	cyclictest	3445506	-21	latency_hist	19:10:01	0
3445504	99	94	8,21	cyclictest	13	1	rcu_preempt	21:35:03	0
3445504	99	85	77,6	cyclictest	12	-21	ksoftirqd/0	22:20:02	0
3445509	99	78	1,5	cyclictest	13	1	rcu_preempt	21:40:02	1
3445518	99	73	63,8	cyclictest	37	-21	ksoftirqd/3	23:55:02	3
3445504	99	73	38,5	cyclictest	13	1	rcu_preempt	19:20:00	0
3445513	99	72	50,8	cyclictest	3456510	-21	kworker/2:1	23:38:01	2
3445509	99	71	30,11	cyclictest	23	-21	ksoftirqd/1	22:40:00	1
3445504	99	71	3,64	cyclictest	3482325	-21	ssh	22:15:03	0
3445504	99	70	2,6	cyclictest	0	-21	swapper/0	22:05:01	0
3445504	99	69	59,8	cyclictest	12	-21	ksoftirqd/0	23:00:03	0
3445504	99	69	50,13	cyclictest	193	50	irq/34-mmc1	21:35:02	0
3445504	99	69	29,9	cyclictest	3518763	-21	ssh	23:32:36	0
3445504	99	69	29,9	cyclictest	3518763	-21	ssh	23:32:36	0
3445504	99	67	6,5	cyclictest	13	1	rcu_preempt	22:00:01	0
3445504	99	66	31,15	cyclictest	3503474	-21	ssh	23:00:02	0
3445504	99	66	20,14	cyclictest	275	-21	dbus-daemon	22:55:01	0
3445504	99	66	10,5	cyclictest	13	1	rcu_preempt	23:50:01	0
3445509	99	65	3,39	cyclictest	13	1	rcu_preempt	22:20:03	1
3445504	99	65	50,9	cyclictest	169	50	irq/224-8000000	20:40:00	0
3445504	99	64	7,6	cyclictest	13	1	rcu_preempt	00:30:02	0
3445504	99	64	36,21	cyclictest	275	-21	dbus-daemon	20:55:00	0
3445504	99	63	4,5	cyclictest	13	1	rcu_preempt	22:31:14	0
3445504	99	63	3,55	cyclictest	0	-21	swapper/0	22:45:00	0
3445509	99	62	49,9	cyclictest	23	-21	ksoftirqd/1	00:10:00	1
3445509	99	62	49,9	cyclictest	23	-21	ksoftirqd/1	00:10:00	1
3445504	99	62	11,31	cyclictest	13	1	rcu_preempt	23:52:50	0
3445504	99	61	6,6	cyclictest	13	1	rcu_preempt	00:20:50	0
3445504	99	60	4,9	cyclictest	12	-21	ksoftirqd/0	21:55:01	0
3445504	99	60	4,29	cyclictest	3450914	-21	kworker/u8:2	21:00:04	0
3445513	99	58	5,6	cyclictest	13	1	rcu_preempt	00:00:00	2
3445504	99	58	37,15	cyclictest	1	-21	systemd	23:40:01	0
3445504	99	58	12,5	cyclictest	13	1	rcu_preempt	00:10:45	0
3445518	99	57	3,5	cyclictest	13	1	rcu_preempt	00:20:00	3
3445504	99	57	6,38	cyclictest	193	50	irq/34-mmc1	19:45:02	0
3445504	99	57	15,32	cyclictest	168	50	irq/134-8000000	22:49:14	0
3445504	99	56	9,6	cyclictest	13	1	rcu_preempt	21:05:00	0
3445504	99	56	7,10	cyclictest	12	-21	ksoftirqd/0	20:30:00	0
3445504	99	56	5,10	cyclictest	12	-21	ksoftirqd/0	20:35:00	0
3445504	99	56	5,10	cyclictest	12	-21	ksoftirqd/0	20:35:00	0
3445504	99	56	28,12	cyclictest	3487294	-21	ssh	22:25:26	0
3445513	99	55	3,15	cyclictest	3482364	-21	ssh	22:15:04	2
3445509	99	55	1,49	cyclictest	13	1	rcu_preempt	23:00:03	1
3445504	99	55	7,12	cyclictest	3479826	-21	unin-run	22:10:00	0
3445504	99	55	46,7	cyclictest	12	-21	ksoftirqd/0	23:07:23	0
3445504	99	55	1,39	cyclictest	3547339	-21	ssh	00:33:28	0
3445513	99	54	27,18	cyclictest	3444840	-21	kworker/2:0	21:58:58	2
3445504	99	54	43,7	cyclictest	15	1	rcuc/0	21:45:01	0
3445504	99	54	27,11	cyclictest	3510681	-21	ssh	23:15:10	0
3445504	99	54	2,5	cyclictest	13	1	rcu_preempt	00:15:43	0
3445504	99	54	18,18	cyclictest	168	50	irq/134-8000000	21:16:32	0
3445504	99	54	1,6	cyclictest	3485833	-21	modprobe	22:22:33	0
3445504	99	54	12,11	cyclictest	12	-21	ksoftirqd/0	20:05:02	0
3445513	99	53	45,5	cyclictest	3444840	-21	kworker/2:0	21:27:59	2
3445513	99	53	44,6	cyclictest	3444840	-21	kworker/2:0	22:07:36	2
3445513	99	53	35,13	cyclictest	1	-21	systemd	23:30:01	2
3445513	99	53	34,15	cyclictest	1	-21	systemd	21:10:00	2
3445513	99	53	23,7	cyclictest	3456510	-21	kworker/2:1	21:36:35	2
3445509	99	53	24,7	cyclictest	3487934	-21	sh	22:26:57	1
3445504	99	53	28,17	cyclictest	275	-21	dbus-daemon	23:15:00	0
3445504	99	53	24,14	cyclictest	3458695	-21	ssh	21:24:44	0
3445504	99	53	21,28	cyclictest	3470495	-21	cron	21:50:01	0
3445518	99	52	3,4	cyclictest	13	1	rcu_preempt	22:05:01	3
3445513	99	52	42,6	cyclictest	3444840	-21	kworker/2:0	22:31:52	2
3445513	99	52	41,6	cyclictest	3444840	-21	kworker/2:0	22:30:01	2
3445509	99	52	37,11	cyclictest	23	-21	ksoftirqd/1	00:00:01	1
3445509	99	52	35,12	cyclictest	275	-21	dbus-daemon	23:45:02	1
3445504	99	52	8,5	cyclictest	13	1	rcu_preempt	22:40:02	0
3445504	99	52	3,7	cyclictest	12	-21	ksoftirqd/0	20:45:01	0
3445504	99	52	15,7	cyclictest	12	-21	ksoftirqd/0	20:15:01	0
3445518	99	51	3,4	cyclictest	13	1	rcu_preempt	00:13:51	3
3445518	99	51	3,3	cyclictest	13	1	rcu_preempt	22:19:57	3
3445518	99	51	3,3	cyclictest	13	1	rcu_preempt	00:09:59	3
3445518	99	51	3,3	cyclictest	13	1	rcu_preempt	00:09:58	3
3445513	99	51	43,5	cyclictest	3456510	-21	kworker/2:1	00:15:53	2
3445513	99	51	43,5	cyclictest	3444840	-21	kworker/2:0	23:20:41	2
3445513	99	51	43,5	cyclictest	3444840	-21	kworker/2:0	23:20:40	2
3445513	99	51	35,12	cyclictest	3467602	-21	systemd-journal	21:00:00	2
3445513	99	51	35,12	cyclictest	275	-21	dbus-daemon	20:40:00	2
3445504	99	51	9,36	cyclictest	0	-21	swapper/0	00:10:03	0
3445504	99	51	9,36	cyclictest	0	-21	swapper/0	00:10:02	0
3445504	99	51	3,30	cyclictest	12	-21	ksoftirqd/0	23:27:36	0
3445504	99	51	24,11	cyclictest	3514798	-21	ssh	23:24:18	0
3445504	99	51	24,11	cyclictest	3514798	-21	ssh	23:24:18	0
3445504	99	51	1,9	cyclictest	12	-21	ksoftirqd/0	00:39:26	0
3445518	99	50	7,10	cyclictest	37	-21	ksoftirqd/3	19:20:00	3
3445518	99	50	5,3	cyclictest	13	1	rcu_preempt	22:56:12	3
3445518	99	50	4,3	cyclictest	13	1	rcu_preempt	23:50:30	3
3445518	99	50	3,3	cyclictest	13	1	rcu_preempt	22:00:02	3
3445518	99	50	2,3	cyclictest	13	1	rcu_preempt	21:32:43	3
3445518	99	50	1,4	cyclictest	13	1	rcu_preempt	22:29:30	3
3445513	99	50	42,5	cyclictest	3456510	-21	kworker/2:1	23:31:39	2
3445513	99	50	42,5	cyclictest	3456510	-21	kworker/2:1	23:31:38	2
3445513	99	50	42,5	cyclictest	3456510	-21	kworker/2:1	23:18:31	2
3445513	99	50	42,5	cyclictest	3456510	-21	kworker/2:1	23:08:34	2
3445513	99	50	42,5	cyclictest	3456510	-21	kworker/2:1	00:06:03	2
3445513	99	50	42,5	cyclictest	3456510	-21	kworker/2:1	00:06:02	2
3445513	99	50	42,5	cyclictest	3444840	-21	kworker/2:0	21:43:16	2
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2021-09-01 - 00:33