OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack9slot4s (updated Wed Apr 29, 2026 00:59:03)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
30367	99	1187	1148,33	cyclictest	30501	2	sleep0	19:25:04	0
30368	99	1161	1131,9	cyclictest	4409	-21	cut	19:40:01	1
30368	99	1140	1131,6	cyclictest	0	-21	swapper/1	19:25:06	1
30073	2	551	507,29	sleep1	0	-21	swapper/1	19:22:28	1
29927	2	477	398,64	sleep0	0	-21	swapper/0	19:20:55	0
8520	2	88	0,9	sleep1	30368	99	cyclictest	22:19:15	1
29414	2	71	0,4	sleep0	0	-21	swapper/0	23:47:17	0
20653	2	66	0,8	sleep1	30368	99	cyclictest	23:05:12	1
4239	2	58	0,7	sleep0	30367	99	cyclictest	23:20:42	0
16719	2	56	0,8	sleep0	30367	99	cyclictest	20:10:24	0
30368	99	52	8,6	cyclictest	22	-21	ksoftirqd/1	21:50:15	1
30368	99	50	7,13	cyclictest	22	-21	ksoftirqd/1	00:26:15	1
30368	99	50	2,19	cyclictest	0	-21	swapper/1	00:39:46	1
30368	99	49	7,6	cyclictest	22	-21	ksoftirqd/1	21:59:09	1
30368	99	49	16,6	cyclictest	22	-21	ksoftirqd/1	21:36:08	1
30368	99	48	8,5	cyclictest	2225	-21	dbus-daemon	20:55:21	1
30368	99	48	8,12	cyclictest	22	-21	ksoftirqd/1	22:59:11	1
30368	99	48	5,6	cyclictest	22	-21	ksoftirqd/1	00:17:55	1
28134	2	48	1,28	sleep0	28236	-21	/usr/sbin/munin	22:05:28	0
30368	99	46	6,15	cyclictest	22	-21	ksoftirqd/1	00:00:35	1
30368	99	46	5,11	cyclictest	22	-21	ksoftirqd/1	00:12:33	1
30368	99	46	4,11	cyclictest	22	-21	ksoftirqd/1	23:13:51	1
30368	99	46	3,6	cyclictest	22	-21	ksoftirqd/1	22:48:38	1
30368	99	45	8,6	cyclictest	22	-21	ksoftirqd/1	21:45:25	1
30368	99	45	4,14	cyclictest	22	-21	ksoftirqd/1	23:52:02	1
30368	99	45	4,11	cyclictest	22	-21	ksoftirqd/1	22:28:13	1
30368	99	45	3,6	cyclictest	22	-21	ksoftirqd/1	23:24:33	1
30368	99	45	3,6	cyclictest	22	-21	ksoftirqd/1	22:31:20	1
30368	99	45	2,6	cyclictest	22	-21	ksoftirqd/1	21:31:06	1
30368	99	45	2,6	cyclictest	22	-21	ksoftirqd/1	21:22:44	1
30368	99	45	2,6	cyclictest	22	-21	ksoftirqd/1	00:23:03	1
30368	99	45	2,13	cyclictest	22	-21	ksoftirqd/1	22:10:35	1
30368	99	45	12,4	cyclictest	8	1	rcu_preempt	21:18:46	1
30368	99	44	9,6	cyclictest	22	-21	ksoftirqd/1	20:30:27	1
30368	99	44	8,5	cyclictest	9319	-21	/usr/sbin/munin	22:20:23	1
30368	99	44	3,6	cyclictest	22	-21	ksoftirqd/1	23:45:38	1
30368	99	44	3,6	cyclictest	22	-21	ksoftirqd/1	23:27:38	1
30368	99	44	2,6	cyclictest	22	-21	ksoftirqd/1	00:49:28	1
30368	99	44	17,4	cyclictest	22	-21	ksoftirqd/1	23:17:56	1
30368	99	44	13,6	cyclictest	22	-21	ksoftirqd/1	21:40:41	1
30368	99	44	13,5	cyclictest	22	-21	ksoftirqd/1	21:29:07	1
13605	2	44	0,7	sleep0	30367	99	cyclictest	00:38:05	0
30368	99	43	8,6	cyclictest	22	-21	ksoftirqd/1	23:30:22	1
30368	99	43	8,6	cyclictest	22	-21	ksoftirqd/1	00:44:25	1
30368	99	43	8,6	cyclictest	22	-21	ksoftirqd/1	00:30:28	1
30368	99	43	2,6	cyclictest	22	-21	ksoftirqd/1	21:12:07	1
23034	2	43	0,7	sleep1	30368	99	cyclictest	22:00:21	1
30368	99	42	9,5	cyclictest	22	-21	ksoftirqd/1	00:05:20	1
30368	99	42	8,6	cyclictest	6864	-21	idleruntime	19:45:17	1
30368	99	42	8,6	cyclictest	22	-21	ksoftirqd/1	20:20:09	1
30368	99	42	8,6	cyclictest	22	-21	ksoftirqd/1	20:14:24	1
30368	99	42	8,6	cyclictest	22	-21	ksoftirqd/1	19:34:16	1
30368	99	42	8,5	cyclictest	30471	-21	watchsnmpd	22:41:00	1
30368	99	42	8,5	cyclictest	22	-21	ksoftirqd/1	00:51:02	1
30368	99	42	7,5	cyclictest	25883	-21	irqstats	20:35:21	1
30368	99	42	6,6	cyclictest	28464	-21	sshd	20:40:35	1
30368	99	42	6,6	cyclictest	28464	-21	sshd	20:40:35	1
30368	99	42	6,6	cyclictest	22	-21	ksoftirqd/1	23:40:01	1
30368	99	42	6,6	cyclictest	22	-21	ksoftirqd/1	23:35:53	1
12979	2	42	0,8	sleep0	30367	99	cyclictest	23:30:21	0
30368	99	41	8,6	cyclictest	22	-21	ksoftirqd/1	19:41:04	1
30368	99	41	8,5	cyclictest	4736	-21	irqstats	23:55:20	1
30368	99	41	7,6	cyclictest	22	-21	ksoftirqd/1	23:00:04	1
30368	99	40	8,5	cyclictest	3286	-21	runrttasks	20:15:15	1
30368	99	40	24,4	cyclictest	22	-21	ksoftirqd/1	20:25:14	1
30368	99	40	1,4	cyclictest	0	-21	swapper/1	22:37:06	1
30368	99	39	9,4	cyclictest	8	1	rcu_preempt	22:05:24	1
30368	99	39	8,6	cyclictest	22	-21	ksoftirqd/1	22:50:14	1
30368	99	38	6,6	cyclictest	8827	-21	idleruntime	19:50:19	1
30368	99	38	3,5	cyclictest	22	-21	ksoftirqd/1	20:05:10	1
30368	99	37	7,6	cyclictest	13113	-21	smartctl-wrappe	20:00:27	1
30368	99	37	7,6	cyclictest	13113	-21	smartctl-wrappe	20:00:27	1
30368	99	37	3,6	cyclictest	22	-21	ksoftirqd/1	21:05:25	1
30368	99	37	23,4	cyclictest	22	-21	ksoftirqd/1	21:00:24	1
30367	99	37	2,11	cyclictest	0	-21	swapper/0	21:27:29	0
30367	99	37	1,24	cyclictest	0	-21	swapper/0	19:40:01	0
30368	99	36	6,10	cyclictest	22	-21	ksoftirqd/1	19:59:18	1
30368	99	36	22,4	cyclictest	22	-21	ksoftirqd/1	20:45:19	1
30367	99	36	1,23	cyclictest	19301	-21	/usr/sbin/munin	00:10:22	0
30367	99	36	1,22	cyclictest	28502	-21	taskset	23:13:01	0
30368	99	35	8,6	cyclictest	22	-21	ksoftirqd/1	20:51:22	1
30367	99	35	7,13	cyclictest	2697	-21	grep	21:40:01	0
30367	99	35	3,11	cyclictest	8	1	rcu_preempt	19:54:30	0
30367	99	35	1,10	cyclictest	4309	-21	sort	22:15:01	0
30367	99	35	10,12	cyclictest	13548	-21	/usr/sbin/munin	21:50:25	0
5957	2	34	1,4	sleep0	5968	-21	ssh	21:42:45	0
30367	99	34	4,4	cyclictest	8	1	rcu_preempt	00:00:23	0
30367	99	34	1,21	cyclictest	5076	-21	ntpq	21:05:25	0
30367	99	34	1,21	cyclictest	14853	-21	irqstats	22:25:21	0
29164	2	34	1,4	sleep0	29183	-21	ntp_states	00:20:25	0
19998	2	34	1,29	sleep0	19660	-21	/usr/sbin/munin	20:20:17	0
30367	99	33	5,4	cyclictest	8	1	rcu_preempt	23:09:03	0
30367	99	33	4,4	cyclictest	8	1	rcu_preempt	23:15:13	0
30367	99	33	4,4	cyclictest	8	1	rcu_preempt	22:21:49	0
30367	99	33	4,4	cyclictest	8	1	rcu_preempt	00:05:47	0
30367	99	33	3,14	cyclictest	22902	-21	ntp_states	23:40:26	0
30367	99	33	2,4	cyclictest	0	-21	swapper/0	22:46:41	0
30367	99	33	2,20	cyclictest	27091	-21	users	21:30:31	0
30367	99	33	1,20	cyclictest	10177	-21	basename	19:55:11	0
30367	99	33	1,19	cyclictest	919	-21	df_inode	00:25:13	0
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17