OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system

Characteristics of all highest latencies:

System rack3slot5s.osadl.org (updated Tue Jun 30, 2026 00:43:29)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
21466	2	139	0	sleep0	1216	50	irq/29-eth0_g0_	00:15:18	0
12	1	104	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	23:46:37	1
12	1	90	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	20:02:13	1
12	1	90	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	00:35:57	1
12	1	83	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	21:06:53	1
25361	2	73	0	sleep1	1216	50	irq/29-eth0_g0_	00:32:55	1
21727	2	65	0	sleep1	1216	50	irq/29-eth0_g0_	00:18:21	1
20769	2	56	0	sleep0	1216	50	irq/29-eth0_g0_	21:59:38	0
28580	2	50	0	sleep0	1217	50	irq/30-eth0_g0_	20:18:53	0
12	1	49	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	22:42:31	1
12	1	48	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	22:55:34	1
12	1	48	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	22:55:34	1
12	1	47	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	21:56:59	1
12	1	47	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	21:56:59	1
12	1	46	0	ksoftirqd/1	1216	50	irq/29-eth0_g0_	20:27:22	1
1216	50	45	0	irq/29-eth0_g0_	1523	-21	awk	22:53:48	0
1216	50	45	0	irq/29-eth0_g0_	1523	-21	awk	22:53:48	0
1216	50	44	0	irq/29-eth0_g0_	1964	-21	runrttasks	21:35:52	0
1216	50	44	0	irq/29-eth0_g0_	10178	-21	sed	19:03:50	0
29461	2	43	0	sleep0	1216	50	irq/29-eth0_g0_	22:36:02	0
1217	50	43	0	irq/30-eth0_g0_	1219	-21	kworker/1:2	23:54:39	1
1217	50	43	0	irq/30-eth0_g0_	1219	-21	kworker/1:2	23:54:39	1
1216	50	43	0	irq/29-eth0_g0_	3	1	ksoftirqd/0	23:58:43	0
1216	50	43	0	irq/29-eth0_g0_	1964	-21	runrttasks	23:50:18	0
12	1	43	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	21:22:35	1
12	1	43	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	21:22:35	1
12	1	43	0	ksoftirqd/1	1216	50	irq/29-eth0_g0_	21:35:33	1
1217	50	42	0	irq/30-eth0_g0_	13400	-21	/usr/sbin/munin	23:43:41	0
1217	50	42	0	irq/30-eth0_g0_	1219	-21	kworker/1:2	22:08:06	1
1217	50	42	0	irq/30-eth0_g0_	12	1	ksoftirqd/1	00:25:59	1
1216	50	42	0	irq/29-eth0_g0_	4454	-21	ntp_offset	20:53:48	0
1216	50	42	0	irq/29-eth0_g0_	13	-21	kworker/0:1	21:12:38	0
1216	50	42	0	irq/29-eth0_g0_	12	1	ksoftirqd/1	22:30:01	1
12	1	42	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	22:31:22	0
12	1	42	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	22:07:20	0
1216	50	41	0	irq/29-eth0_g0_	13	-21	kworker/0:1	23:02:43	0
10690	99	41	1	cyclictest	29225	-21	grep	22:33:52	1
1217	50	40	0	irq/30-eth0_g0_	8667	-21	memory	23:23:45	1
1217	50	40	0	irq/30-eth0_g0_	1958	-21	crond	19:05:29	1
1217	50	40	0	irq/30-eth0_g0_	1219	-21	kworker/1:2	23:31:53	1
1217	50	40	0	irq/30-eth0_g0_	1219	-21	kworker/1:2	23:16:51	1
1217	50	40	0	irq/30-eth0_g0_	1219	-21	kworker/1:2	21:24:45	1
1217	50	40	0	irq/30-eth0_g0_	12	1	ksoftirqd/1	23:38:14	1
1217	50	40	0	irq/30-eth0_g0_	12	1	ksoftirqd/1	23:04:02	1
1217	50	40	0	irq/30-eth0_g0_	12	1	ksoftirqd/1	00:03:18	1
1216	50	40	0	irq/29-eth0_g0_	5162	-21	proc_pri	23:08:48	0
1216	50	40	0	irq/29-eth0_g0_	30681	-21	missed_timers	20:28:45	0
1216	50	40	0	irq/29-eth0_g0_	3	1	ksoftirqd/0	21:50:56	0
1216	50	40	0	irq/29-eth0_g0_	13	-21	kworker/0:1	23:33:56	0
1216	50	40	0	irq/29-eth0_g0_	13	-21	kworker/0:1	23:07:24	0
1216	50	40	0	irq/29-eth0_g0_	13	-21	kworker/0:1	21:33:30	0
1216	50	40	0	irq/29-eth0_g0_	13	-21	kworker/0:1	21:26:41	0
12	1	40	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	23:20:11	0
12	1	40	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	22:48:47	0
12	1	40	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	19:18:33	0
12	1	40	0	ksoftirqd/1	1216	50	irq/29-eth0_g0_	20:51:00	1
12	1	40	0	ksoftirqd/1	1216	50	irq/29-eth0_g0_	20:51:00	1
10690	99	40	1	cyclictest	4445	2	sleep1	20:53:48	1
22927	2	39	0	sleep1	1216	50	irq/29-eth0_g0_	00:23:04	1
1217	50	39	0	irq/30-eth0_g0_	1219	-21	kworker/1:2	22:02:46	1
1217	50	39	0	irq/30-eth0_g0_	12	1	ksoftirqd/1	21:30:51	1
1216	50	39	0	irq/29-eth0_g0_	16330	-21	unixbench_multi	19:28:51	0
1216	50	39	0	irq/29-eth0_g0_	13	-21	kworker/0:1	23:41:50	0
1216	50	39	0	irq/29-eth0_g0_	13	-21	kworker/0:1	23:41:50	0
1216	50	39	0	irq/29-eth0_g0_	13	-21	kworker/0:1	23:30:43	0
1216	50	39	0	irq/29-eth0_g0_	13	-21	kworker/0:1	23:27:38	0
1216	50	39	0	irq/29-eth0_g0_	13	-21	kworker/0:1	23:13:32	0
1216	50	39	0	irq/29-eth0_g0_	13	-21	kworker/0:1	21:39:24	0
12	1	39	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	22:44:23	0
12	1	39	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	22:40:11	0
12	1	39	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	22:24:20	0
12	1	39	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	22:20:18	0
12	1	39	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	22:13:56	0
12	1	39	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	22:10:54	0
12	1	39	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	21:56:46	0
12	1	39	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	21:56:46	0
12	1	39	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	21:43:53	0
12	1	39	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	21:14:03	0
12	1	39	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	21:07:50	0
12	1	39	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	20:59:38	0
12	1	39	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	20:51:57	0
12	1	39	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	20:51:57	0
12	1	39	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	20:09:49	0
12	1	39	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	20:02:23	0
12	1	39	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	00:27:53	0
12	1	39	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	00:09:21	0
12	1	39	0	ksoftirqd/1	1216	50	irq/29-eth0_g0_	23:18:47	1
12	1	39	0	ksoftirqd/1	1216	50	irq/29-eth0_g0_	22:44:26	1
12	1	39	0	ksoftirqd/1	1216	50	irq/29-eth0_g0_	21:08:51	1
12	1	39	0	ksoftirqd/1	1216	50	irq/29-eth0_g0_	21:02:20	1
12	1	39	0	ksoftirqd/1	1216	50	irq/29-eth0_g0_	20:48:27	1
12	1	39	0	ksoftirqd/1	1216	50	irq/29-eth0_g0_	20:29:24	1
12	1	39	0	ksoftirqd/1	1216	50	irq/29-eth0_g0_	19:45:02	1
1217	50	38	0	irq/30-eth0_g0_	1219	-21	kworker/1:2	21:13:44	1
1217	50	38	0	irq/30-eth0_g0_	12	1	ksoftirqd/1	23:03:18	1
1217	50	38	0	irq/30-eth0_g0_	12	1	ksoftirqd/1	22:10:32	1
1217	50	38	0	irq/30-eth0_g0_	12	1	ksoftirqd/1	21:46:55	1
1216	50	38	0	irq/29-eth0_g0_	13	-21	kworker/0:1	21:18:35	0
1216	50	38	0	irq/29-eth0_g0_	13	-21	kworker/0:1	21:18:35	0
1216	50	38	0	irq/29-eth0_g0_	13	-21	kworker/0:1	00:35:27	0
1216	50	38	0	irq/29-eth0_g0_	13	-21	kworker/0:1	00:21:23	0
1216	50	38	0	irq/29-eth0_g0_	13	-21	kworker/0:1	00:04:32	0
1216	50	38	0	irq/29-eth0_g0_	1029	-21	grep	20:38:53	0
12	1	38	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	19:28:41	1
12	1	38	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	00:29:19	0
12	1	38	0	ksoftirqd/1	1216	50	irq/29-eth0_g0_	22:27:18	1
12	1	38	0	ksoftirqd/1	1216	50	irq/29-eth0_g0_	21:41:47	1
12	1	38	0	ksoftirqd/1	1216	50	irq/29-eth0_g0_	20:37:21	1
12	1	38	0	ksoftirqd/1	1216	50	irq/29-eth0_g0_	20:17:00	1
12	1	38	0	ksoftirqd/1	1216	50	irq/29-eth0_g0_	19:35:55	1
12	1	38	0	ksoftirqd/1	1216	50	irq/29-eth0_g0_	19:24:17	1
12	1	38	0	ksoftirqd/1	1216	50	irq/29-eth0_g0_	19:21:15	1
10690	99	38	2	cyclictest	898	-21	wc	22:53:30	1
10690	99	38	1	cyclictest	17754	2	sleep1	21:48:44	1
1216	50	37	0	irq/29-eth0_g0_	1964	-21	runrttasks	19:15:18	0
1216	50	37	0	irq/29-eth0_g0_	13	-21	kworker/0:1	23:55:30	0
1216	50	37	0	irq/29-eth0_g0_	13	-21	kworker/0:1	23:55:30	0
1216	50	37	0	irq/29-eth0_g0_	11074	-21	/usr/sbin/munin	19:08:46	0
1216	50	37	0	irq/29-eth0_g0_	0	-21	swapper/0	20:43:48	0
12	1	37	0	ksoftirqd/1	1216	50	irq/29-eth0_g0_	23:08:44	1
10690	99	37	1	cyclictest	28570	-21	unixbench_multi	20:18:53	1
10690	99	37	1	cyclictest	18839	2	sleep1	00:03:57	1
10690	99	37	1	cyclictest	1019	-21	sed	20:38:53	1
1217	50	36	0	irq/30-eth0_g0_	1219	-21	kworker/1:2	23:50:14	1
1217	50	36	0	irq/30-eth0_g0_	1219	-21	kworker/1:2	00:10:41	1
1216	50	36	0	irq/29-eth0_g0_	1964	-21	runrttasks	23:42:52	1
1216	50	36	0	irq/29-eth0_g0_	1964	-21	runrttasks	23:42:52	1
1216	50	36	0	irq/29-eth0_g0_	1964	-21	runrttasks	19:55:25	0
1216	50	36	0	irq/29-eth0_g0_	1964	-21	runrttasks	19:42:03	0
12	1	36	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	20:25:21	0
12	1	36	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	20:18:24	0
12	1	36	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	19:27:07	0
12	1	36	0	ksoftirqd/1	1216	50	irq/29-eth0_g0_	22:23:22	1
12	1	36	0	ksoftirqd/1	1216	50	irq/29-eth0_g0_	19:53:47	1
10690	99	36	2	cyclictest	25919	-21	taskset	20:08:47	1
10690	99	36	1	cyclictest	24402	2	sleep1	22:14:09	1
10690	99	36	1	cyclictest	21239	-21	grep	19:48:51	1
10690	99	36	1	cyclictest	11943	-21	cat	19:13:30	1
1216	50	35	0	irq/29-eth0_g0_	21020	-21	proc_pri	19:48:48	0
1216	50	35	0	irq/29-eth0_g0_	19721	-21	awk	19:43:46	0
1216	50	35	0	irq/29-eth0_g0_	17533	-21	grep	19:33:51	0
10690	99	35	2	cyclictest	24468	-21	irqstats	20:03:44	1
1216	50	34	0	irq/29-eth0_g0_	31828	-21	cat	20:33:43	0
1216	50	34	0	irq/29-eth0_g0_	0	-21	swapper/0	20:03:41	0
10690	99	34	1	cyclictest	12297	-21	seq	19:13:46	1
10690	99	33	1	cyclictest	19297	2	sleep1	19:43:30	1
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17