OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system

Characteristics of the 792 highest latencies:

System rack3slot5s.osadl.org (updated Sat Apr 18, 2026 00:43:30)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
2231	2	171	0	sleep1	1215	50	irq/30-eth0_g0_	23:39:40	1
358	2	149	0	sleep0	1215	50	irq/30-eth0_g0_	23:21:03	0
12	1	64	0	ksoftirqd/1	21032	2	sleep0	21:24:55	0
12	1	63	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	21:30:23	0
12	1	61	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	23:48:15	0
22208	2	59	0	sleep1	1215	50	irq/30-eth0_g0_	21:35:56	1
26826	2	58	0	sleep0	1214	50	irq/29-eth0_g0_	22:21:42	0
19791	2	58	0	sleep0	1214	50	irq/29-eth0_g0_	21:12:14	0
22742	2	54	0	chrt	1215	50	irq/30-eth0_g0_	21:41:14	1
7408	2	53	0	sleep0	1214	50	irq/29-eth0_g0_	00:30:09	0
12	1	50	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	22:15:32	1
12	1	48	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	21:11:38	1
12	1	47	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	21:58:47	0
12	1	46	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	21:42:46	0
12	1	46	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	23:56:57	1
12	1	46	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	21:32:58	1
12	1	46	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	20:40:48	1
12	1	45	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	00:10:13	1
12	1	44	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	22:27:11	0
1215	50	43	0	irq/30-eth0_g0_	1221	-21	kworker/1:2	23:34:46	1
1215	50	43	0	irq/30-eth0_g0_	1221	-21	kworker/1:2	23:21:32	1
1215	50	43	0	irq/30-eth0_g0_	1221	-21	kworker/1:2	00:34:22	1
12	1	43	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	23:38:00	0
12	1	43	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	21:53:32	0
1214	50	42	0	irq/29-eth0_g0_	3	1	ksoftirqd/0	00:28:26	0
1214	50	42	0	irq/29-eth0_g0_	13	-21	kworker/0:1	23:32:32	0
22502	2	41	0	sleep0	1214	50	irq/29-eth0_g0_	21:39:21	0
1214	50	41	0	irq/29-eth0_g0_	20126	-21	sshd	21:15:16	0
1214	50	41	0	irq/29-eth0_g0_	1911	-21	runrttasks	19:07:24	0
12	1	41	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	20:27:57	0
12	1	41	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	22:54:00	1
12	1	41	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	22:54:00	1
1215	50	40	0	irq/30-eth0_g0_	1221	-21	kworker/1:2	23:13:25	1
1215	50	40	0	irq/30-eth0_g0_	1221	-21	kworker/1:2	22:28:49	1
1215	50	40	0	irq/30-eth0_g0_	12	1	ksoftirqd/1	22:41:39	1
1215	50	40	0	irq/30-eth0_g0_	12	1	ksoftirqd/1	21:46:20	1
1214	50	40	0	irq/29-eth0_g0_	9468	-21	latency_hist	19:29:55	0
1214	50	40	0	irq/29-eth0_g0_	9468	-21	latency_hist	19:29:55	0
1214	50	40	0	irq/29-eth0_g0_	3	1	ksoftirqd/0	22:05:40	0
1214	50	40	0	irq/29-eth0_g0_	3	1	ksoftirqd/0	22:05:40	0
1214	50	40	0	irq/29-eth0_g0_	13	-21	kworker/0:1	22:32:35	0
1214	50	40	0	irq/29-eth0_g0_	13	-21	kworker/0:1	00:05:43	0
1214	50	40	0	irq/29-eth0_g0_	13	-21	kworker/0:1	00:05:43	0
1214	50	40	0	irq/29-eth0_g0_	12	1	ksoftirqd/1	22:00:42	1
12	1	40	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	23:52:10	0
12	1	40	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	20:12:32	0
1215	50	39	0	irq/30-eth0_g0_	1221	-21	kworker/1:2	22:57:13	1
1215	50	39	0	irq/30-eth0_g0_	1221	-21	kworker/1:2	21:29:26	1
1215	50	39	0	irq/30-eth0_g0_	12	1	ksoftirqd/1	23:47:31	1
1215	50	39	0	irq/30-eth0_g0_	12	1	ksoftirqd/1	23:29:15	1
1215	50	39	0	irq/30-eth0_g0_	12	1	ksoftirqd/1	23:29:15	1
1215	50	39	0	irq/30-eth0_g0_	0	-21	swapper/1	21:51:00	1
1214	50	39	0	irq/29-eth0_g0_	3	1	ksoftirqd/0	22:56:31	0
1214	50	39	0	irq/29-eth0_g0_	3	1	ksoftirqd/0	00:04:43	0
1214	50	39	0	irq/29-eth0_g0_	13	-21	kworker/0:1	23:01:04	0
1214	50	39	0	irq/29-eth0_g0_	13	-21	kworker/0:1	21:28:20	0
1214	50	39	0	irq/29-eth0_g0_	13	-21	kworker/0:1	00:15:22	0
1214	50	39	0	irq/29-eth0_g0_	12	1	ksoftirqd/1	23:42:13	0
1214	50	39	0	irq/29-eth0_g0_	12	1	ksoftirqd/1	21:16:20	1
12	1	39	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	23:17:12	0
12	1	39	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	23:10:29	0
12	1	39	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	22:53:37	0
12	1	39	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	22:53:37	0
12	1	39	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	22:46:26	0
12	1	39	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	22:41:48	0
12	1	39	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	22:12:51	0
12	1	39	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	22:01:19	0
12	1	39	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	20:44:26	0
12	1	39	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	20:08:05	0
12	1	39	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	19:39:20	0
12	1	39	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	19:21:44	0
12	1	39	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	00:14:31	0
12	1	39	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	20:39:29	1
1215	50	38	0	irq/30-eth0_g0_	31450	-21	chrt	23:07:37	1
1215	50	38	0	irq/30-eth0_g0_	1221	-21	kworker/1:2	22:33:31	1
1215	50	38	0	irq/30-eth0_g0_	1221	-21	kworker/1:2	22:08:45	1
1215	50	38	0	irq/30-eth0_g0_	1221	-21	kworker/1:2	22:08:45	1
1215	50	38	0	irq/30-eth0_g0_	12	1	ksoftirqd/1	22:38:27	1
1215	50	38	0	irq/30-eth0_g0_	12	1	ksoftirqd/1	21:21:39	1
1215	50	38	0	irq/30-eth0_g0_	12	1	ksoftirqd/1	19:58:26	1
1215	50	38	0	irq/30-eth0_g0_	12	1	ksoftirqd/1	00:07:16	1
1215	50	38	0	irq/30-eth0_g0_	12	1	ksoftirqd/1	00:07:16	1
1215	50	38	0	irq/30-eth0_g0_	0	-21	swapper/1	23:44:18	1
1215	50	38	0	irq/30-eth0_g0_	0	-21	swapper/1	23:16:08	1
1215	50	38	0	irq/30-eth0_g0_	0	-21	swapper/1	00:04:28	1
1214	50	38	0	irq/29-eth0_g0_	3	1	ksoftirqd/0	21:47:02	0
1214	50	38	0	irq/29-eth0_g0_	1911	-21	runrttasks	19:50:01	0
1214	50	38	0	irq/29-eth0_g0_	1911	-21	runrttasks	19:45:16	0
1214	50	38	0	irq/29-eth0_g0_	13	-21	kworker/0:1	00:20:01	0
12	1	38	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	23:05:38	0
12	1	38	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	21:04:22	0
12	1	38	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	00:39:38	0
12	1	38	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	00:39:38	0
1215	50	37	0	irq/30-eth0_g0_	31123	-21	taskset	23:04:41	1
1215	50	37	0	irq/30-eth0_g0_	1221	-21	kworker/1:2	00:39:29	1
1215	50	37	0	irq/30-eth0_g0_	1221	-21	kworker/1:2	00:39:29	1
1215	50	37	0	irq/30-eth0_g0_	1221	-21	kworker/1:2	00:20:03	1
1215	50	37	0	irq/30-eth0_g0_	12	1	ksoftirqd/1	22:20:08	1
1214	50	37	0	irq/29-eth0_g0_	13	-21	kworker/0:1	23:56:35	0
12	1	37	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	22:10:02	1
12	1	37	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	00:16:26	1
1215	50	36	0	irq/30-eth0_g0_	1221	-21	kworker/1:2	22:45:28	1
1214	50	36	0	irq/29-eth0_g0_	3	1	ksoftirqd/0	22:35:42	0
1214	50	36	0	irq/29-eth0_g0_	1911	-21	runrttasks	22:16:02	0
1214	50	36	0	irq/29-eth0_g0_	1911	-21	runrttasks	20:48:17	0
1214	50	36	0	irq/29-eth0_g0_	1911	-21	runrttasks	20:00:40	0
1214	50	36	0	irq/29-eth0_g0_	12	1	ksoftirqd/1	21:58:07	1
12	1	36	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	23:27:09	0
12	1	36	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	23:27:09	0
12	1	36	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	20:07:46	1
7441	99	35	2	cyclictest	1972	-21	lldpd	20:23:17	1
7441	99	35	1	cyclictest	15529	-21	cat	20:29:55	1
1215	50	35	0	irq/30-eth0_g0_	1221	-21	kworker/1:2	23:52:35	1
1215	50	35	0	irq/30-eth0_g0_	1221	-21	kworker/1:2	00:27:59	1
1214	50	35	0	irq/29-eth0_g0_	1911	-21	runrttasks	20:50:09	0
1214	50	35	0	irq/29-eth0_g0_	1911	-21	runrttasks	20:36:27	0
1214	50	35	0	irq/29-eth0_g0_	1911	-21	runrttasks	20:31:11	0
1214	50	35	0	irq/29-eth0_g0_	1911	-21	runrttasks	20:18:23	0
1214	50	35	0	irq/29-eth0_g0_	1911	-21	runrttasks	19:58:13	0
1214	50	35	0	irq/29-eth0_g0_	1911	-21	runrttasks	19:41:38	0
1214	50	35	0	irq/29-eth0_g0_	18523	-21	ls	20:59:56	0
7441	99	34	1	cyclictest	7498	-21	grep	19:09:56	1
7441	99	34	1	cyclictest	10463	2	sleep1	19:39:55	1
7441	99	34	1	cyclictest	0	-21	swapper/1	20:31:57	1
1214	50	34	0	irq/29-eth0_g0_	9995	-21	ls	19:34:55	0
1214	50	34	0	irq/29-eth0_g0_	9995	-21	ls	19:34:55	0
1214	50	34	0	irq/29-eth0_g0_	8449	-21	munin-run	19:19:56	0
12	1	34	0	ksoftirqd/1	19408	-21	taskset	21:08:42	1
7441	99	33	1	cyclictest	0	-21	swapper/1	21:02:10	1
7441	99	33	1	cyclictest	0	-21	swapper/1	19:29:55	1
7441	99	33	1	cyclictest	0	-21	swapper/1	19:29:55	1
1215	50	33	0	irq/30-eth0_g0_	1911	-21	runrttasks	20:20:07	0
12	1	33	0	ksoftirqd/1	8804	-21	taskset	19:23:11	1
12	1	33	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	19:47:46	1
7441	99	32	1	cyclictest	0	-21	swapper/1	20:54:09	1
7441	99	32	1	cyclictest	0	-21	swapper/1	20:00:42	1
7441	99	32	1	cyclictest	0	-21	swapper/1	19:31:55	1
7441	99	32	1	cyclictest	0	-21	swapper/1	19:31:55	1
7441	99	32	1	cyclictest	0	-21	swapper/1	19:12:57	1
1215	50	32	0	irq/30-eth0_g0_	17095	-21	taskset	20:45:06	1
12	1	32	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	20:18:15	1
7441	99	31	1	cyclictest	0	-21	swapper/1	19:18:13	1
1215	50	31	0	irq/30-eth0_g0_	10566	-21	sshd	19:40:15	1
1214	50	31	0	irq/29-eth0_g0_	18400	-21	runrttasks	20:58:37	1
1214	50	30	0	irq/29-eth0_g0_	11649	-21	runrttasks	19:51:10	1
1214	50	29	0	irq/29-eth0_g0_	13607	-21	runrttasks	20:10:29	1
1214	50	29	0	irq/29-eth0_g0_	0	-21	swapper/0	21:05:00	0
1214	50	29	0	irq/29-eth0_g0_	0	-21	swapper/0	19:13:58	0
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17