OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rackeslot3s.osadl.org (updated Wed Jul 01, 2026 00:45:23)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
3231807	99	95	2,86	cyclictest	3273924	-21	ssh	22:25:02	3
3231807	99	94	2,4	cyclictest	13	1	rcu_preempt	21:55:01	3
3231797	99	82	1,3	cyclictest	0	-21	swapper/0	21:40:03	0
3231807	99	71	5,33	cyclictest	3281059	-21	sshd	22:40:02	3
3231797	99	69	6,58	cyclictest	3271461	-21	ssh	22:20:03	0
3231797	99	69	17,45	cyclictest	169	50	irq/224-8000000	22:35:02	0
3231807	99	68	26,12	cyclictest	1	-21	systemd	23:50:01	3
3231805	99	67	38,6	cyclictest	3304730	-21	kworker/2:1	23:53:38	2
3231797	99	67	9,5	cyclictest	13	1	rcu_preempt	21:57:28	0
3231797	99	67	61,3	cyclictest	0	-21	swapper/0	21:10:02	0
3231797	99	67	61,3	cyclictest	0	-21	swapper/0	21:10:02	0
3231807	99	66	1,29	cyclictest	3304857	-21	ssh	23:30:02	3
3231797	99	65	53,7	cyclictest	169	50	irq/224-8000000	21:50:01	0
3231797	99	65	13,5	cyclictest	13	1	rcu_preempt	23:09:55	0
3231797	99	64	9,5	cyclictest	13	1	rcu_preempt	22:24:02	0
3231797	99	64	6,8	cyclictest	12	1	ksoftirqd/0	23:45:01	0
3231801	99	63	19,9	cyclictest	13	1	rcu_preempt	21:25:00	1
3231797	99	61	40,17	cyclictest	168	50	irq/134-8000000	23:00:01	0
3231797	99	60	4,14	cyclictest	12	-21	ksoftirqd/0	00:10:00	0
3231797	99	60	27,17	cyclictest	3241417	-21	ssh	21:16:57	0
3231797	99	60	17,39	cyclictest	12	-21	ksoftirqd/0	23:56:02	0
3231805	99	59	37,7	cyclictest	3240395	-21	kworker/2:0	22:04:57	2
3231797	99	59	47,7	cyclictest	169	50	irq/224-8000000	23:15:00	0
3231797	99	59	12,5	cyclictest	13	1	rcu_preempt	00:20:00	0
3231805	99	58	49,6	cyclictest	3319573	-21	kworker/2:2	00:19:48	2
3231797	99	58	8,9	cyclictest	12	-21	ksoftirqd/0	22:36:44	0
3231797	99	58	6,7	cyclictest	12	-21	ksoftirqd/0	22:55:00	0
3231797	99	58	28,26	cyclictest	3267891	-21	ssh	22:12:32	0
3231797	99	58	28,14	cyclictest	3336814	-21	ssh	00:37:08	0
3231797	99	58	28,14	cyclictest	3336814	-21	ssh	00:37:08	0
3231797	99	58	22,6	cyclictest	15	1	rcuc/0	19:20:01	0
3231805	99	57	49,5	cyclictest	3240395	-21	kworker/2:0	22:16:15	2
3231797	99	57	3,8	cyclictest	12	-21	ksoftirqd/0	22:50:02	0
3231797	99	57	28,14	cyclictest	3305596	-21	ssh	23:31:28	0
3231797	99	57	28,14	cyclictest	3305596	-21	ssh	23:31:28	0
3231797	99	57	28,14	cyclictest	3248617	-21	ssh	21:31:59	0
3231797	99	57	27,14	cyclictest	3291523	-21	ssh	23:02:07	0
3231797	99	57	1,40	cyclictest	3263630	-21	ssh	22:03:39	0
3231805	99	56	47,5	cyclictest	3319573	-21	kworker/2:2	00:21:45	2
3231805	99	56	47,5	cyclictest	3210759	-21	kworker/2:3	19:55:02	2
3231797	99	56	8,6	cyclictest	12	-21	ksoftirqd/0	23:48:37	0
3231797	99	56	27,13	cyclictest	3331297	-21	ssh	00:25:29	0
3231797	99	56	23,17	cyclictest	3243153	-21	ssh	21:20:32	0
3231797	99	56	2,11	cyclictest	3254760	-21	latency_hist	21:45:01	0
3231797	99	56	19,12	cyclictest	12	-21	ksoftirqd/0	21:30:00	0
3231797	99	56	10,5	cyclictest	13	1	rcu_preempt	22:43:06	0
3231805	99	55	46,6	cyclictest	3240395	-21	kworker/2:0	21:59:45	2
3231797	99	55	23,16	cyclictest	3240354	-21	ssh	21:14:59	0
3231797	99	55	22,29	cyclictest	12	-21	ksoftirqd/0	23:22:46	0
3231797	99	55	22,29	cyclictest	12	-21	ksoftirqd/0	23:22:46	0
3231797	99	55	1,38	cyclictest	3297791	-21	ssh	23:15:03	0
3231805	99	54	46,5	cyclictest	3240395	-21	kworker/2:0	23:17:08	2
3231805	99	54	45,6	cyclictest	3210759	-21	kworker/2:3	19:50:00	2
3231805	99	54	36,13	cyclictest	275	-21	dbus-daemon	23:40:00	2
3231801	99	54	45,6	cyclictest	23	-21	ksoftirqd/1	00:27:23	1
3231801	99	54	10,10	cyclictest	13	1	rcu_preempt	20:35:02	1
3231801	99	54	10,10	cyclictest	13	1	rcu_preempt	20:35:02	1
3231797	99	54	8,6	cyclictest	13	1	rcu_preempt	20:35:02	0
3231797	99	54	8,6	cyclictest	13	1	rcu_preempt	20:35:02	0
3231797	99	54	3,35	cyclictest	13	1	rcu_preempt	23:28:59	0
3231797	99	54	24,15	cyclictest	3265672	-21	ssh	22:08:01	0
3231797	99	54	10,5	cyclictest	13	1	rcu_preempt	00:13:27	0
3231807	99	53	33,15	cyclictest	1	-21	systemd	22:45:01	3
3231797	99	53	27,17	cyclictest	3234285	-21	cron	20:00:01	0
3231797	99	53	25,12	cyclictest	3335787	-21	ssh	00:35:03	0
3231797	99	53	2,36	cyclictest	3320428	-21	ssh	00:02:40	0
3231805	99	52	44,5	cyclictest	3326145	-21	kworker/2:3	00:33:02	2
3231805	99	52	43,5	cyclictest	3240395	-21	kworker/2:0	23:00:02	2
3231797	99	52	6,42	cyclictest	3232966	-21	sshd	19:30:02	0
3231797	99	52	25,12	cyclictest	3314869	-21	ssh	23:50:54	0
3231797	99	52	2,12	cyclictest	12	-21	ksoftirqd/0	20:45:02	0
3231797	99	52	14,12	cyclictest	12	-21	ksoftirqd/0	23:40:00	0
3231797	99	52	11,5	cyclictest	13	1	rcu_preempt	22:27:35	0
3231797	99	52	10,12	cyclictest	12	-21	ksoftirqd/0	20:05:01	0
3231805	99	51	43,5	cyclictest	3326145	-21	kworker/2:3	00:25:32	2
3231805	99	51	43,5	cyclictest	3240395	-21	kworker/2:0	23:11:01	2
3231805	99	51	43,5	cyclictest	3240395	-21	kworker/2:0	22:26:22	2
3231805	99	51	43,5	cyclictest	3240395	-21	kworker/2:0	21:50:07	2
3231805	99	51	42,6	cyclictest	3240395	-21	kworker/2:0	23:45:32	2
3231805	99	51	42,5	cyclictest	3319573	-21	kworker/2:2	00:09:59	2
3231805	99	51	40,7	cyclictest	3240395	-21	kworker/2:0	23:27:04	2
3231797	99	51	7,23	cyclictest	0	-21	swapper/0	20:45:03	0
3231797	99	51	25,11	cyclictest	3328972	-21	ssh	00:20:32	0
3231807	99	50	4,3	cyclictest	13	1	rcu_preempt	23:51:55	3
3231805	99	50	42,5	cyclictest	3319573	-21	kworker/2:2	00:11:14	2
3231805	99	50	42,5	cyclictest	3240395	-21	kworker/2:0	23:06:11	2
3231805	99	50	42,5	cyclictest	3240395	-21	kworker/2:0	22:46:05	2
3231805	99	50	42,5	cyclictest	3240395	-21	kworker/2:0	22:08:17	2
3231805	99	50	41,6	cyclictest	3240395	-21	kworker/2:0	23:21:26	2
3231805	99	50	41,6	cyclictest	3240395	-21	kworker/2:0	23:21:26	2
3231805	99	50	41,5	cyclictest	3240395	-21	kworker/2:0	00:04:32	2
3231805	99	50	40,6	cyclictest	3240395	-21	kworker/2:0	23:02:24	2
3231805	99	50	33,10	cyclictest	3252304	-21	cron	21:40:00	2
3231805	99	50	32,13	cyclictest	1	-21	systemd	20:30:00	2
3231797	99	50	22,12	cyclictest	3258417	-21	ssh	21:52:34	0
3231805	99	49	41,5	cyclictest	3240395	-21	kworker/2:0	22:40:01	2
3231805	99	49	41,5	cyclictest	3240395	-21	kworker/2:0	22:31:36	2
3231805	99	49	41,5	cyclictest	3240395	-21	kworker/2:0	22:21:42	2
3231805	99	49	40,6	cyclictest	3210759	-21	kworker/2:3	21:26:06	2
3231805	99	49	40,5	cyclictest	3240395	-21	kworker/2:0	21:35:01	2
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2021-09-01 - 00:33