OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system

Characteristics of the 376 highest latencies:

System rack3slot5s.osadl.org (updated Sun May 03, 2026 00:43:30)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
26970	2	162	0	chrt	1215	50	irq/30-eth0_g0_	00:00:00	0
12	1	114	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	21:27:56	1
29800	2	111	0	chrt	1215	50	irq/30-eth0_g0_	00:28:33	1
28503	2	82	0	sleep0	1214	50	irq/29-eth0_g0_	00:15:13	0
12	1	80	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	20:55:26	0
28480	2	63	0	sleep1	1215	50	irq/30-eth0_g0_	00:14:57	1
12	1	57	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	23:05:44	1
17979	2	54	0	sleep0	1215	50	irq/30-eth0_g0_	22:31:17	0
12	1	48	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	23:51:30	1
12	1	48	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	22:10:37	1
12	1	47	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	21:11:59	1
12	1	47	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	19:36:57	1
1214	50	46	0	irq/29-eth0_g0_	19266	-21	latency_hist	22:44:21	0
12	1	46	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	22:46:22	1
12	1	46	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	22:46:22	1
12	1	46	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	22:53:39	1
12	1	46	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	19:28:15	1
12	1	45	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	22:37:31	0
1215	50	44	0	irq/30-eth0_g0_	1221	-21	kworker/1:2	22:58:49	1
12	1	44	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	22:27:35	1
12	1	44	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	23:10:16	1
12	1	44	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	23:01:17	0
12	1	44	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	20:29:40	1
12	1	44	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	00:19:25	1
1215	50	43	0	irq/30-eth0_g0_	1221	-21	kworker/1:2	23:56:22	1
1215	50	43	0	irq/30-eth0_g0_	1221	-21	kworker/1:2	23:24:59	1
1215	50	43	0	irq/30-eth0_g0_	1221	-21	kworker/1:2	21:56:32	1
12	1	43	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	00:30:06	0
1215	50	42	0	irq/30-eth0_g0_	1221	-21	kworker/1:2	23:40:23	1
1215	50	42	0	irq/30-eth0_g0_	1221	-21	kworker/1:2	00:34:37	1
12	1	42	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	20:28:27	0
12	1	42	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	21:14:24	1
1215	50	41	0	irq/30-eth0_g0_	1221	-21	kworker/1:2	22:19:50	1
1215	50	41	0	irq/30-eth0_g0_	1221	-21	kworker/1:2	22:00:49	1
1215	50	41	0	irq/30-eth0_g0_	1221	-21	kworker/1:2	22:00:49	1
1214	50	41	0	irq/29-eth0_g0_	3	1	ksoftirqd/0	23:42:23	0
1214	50	41	0	irq/29-eth0_g0_	1911	-21	runrttasks	21:15:50	0
1214	50	41	0	irq/29-eth0_g0_	12	1	ksoftirqd/1	23:17:21	0
1215	50	40	0	irq/30-eth0_g0_	1221	-21	kworker/1:2	23:45:35	1
1214	50	40	0	irq/29-eth0_g0_	3	1	ksoftirqd/0	22:27:05	0
1214	50	40	0	irq/29-eth0_g0_	13	-21	kworker/0:1	23:58:58	0
1214	50	40	0	irq/29-eth0_g0_	13	-21	kworker/0:1	21:55:34	0
1214	50	40	0	irq/29-eth0_g0_	13	-21	kworker/0:1	21:46:44	0
12	1	40	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	22:52:55	0
12	1	40	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	21:30:04	1
29948	99	39	1	cyclictest	8156	2	sleep1	20:54:21	1
1215	50	39	0	irq/30-eth0_g0_	3	1	ksoftirqd/0	00:36:39	0
1215	50	39	0	irq/30-eth0_g0_	23913	-21	taskset	23:29:44	1
1215	50	39	0	irq/30-eth0_g0_	1221	-21	kworker/1:2	00:10:30	1
1214	50	39	0	irq/29-eth0_g0_	1911	-21	runrttasks	21:21:11	0
1214	50	39	0	irq/29-eth0_g0_	1911	-21	runrttasks	19:07:45	0
1214	50	39	0	irq/29-eth0_g0_	13	-21	kworker/0:1	21:53:01	0
1214	50	39	0	irq/29-eth0_g0_	13	-21	kworker/0:1	21:39:20	0
1214	50	39	0	irq/29-eth0_g0_	13	-21	kworker/0:1	00:25:40	0
1214	50	39	0	irq/29-eth0_g0_	13	-21	kworker/0:1	00:10:02	0
12	1	39	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	23:36:25	0
12	1	39	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	23:25:03	0
12	1	39	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	23:20:04	0
12	1	39	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	23:09:57	0
12	1	39	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	22:18:33	0
12	1	39	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	22:11:02	0
12	1	39	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	21:59:29	0
12	1	39	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	21:59:29	0
12	1	39	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	21:43:54	0
12	1	39	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	21:12:43	0
12	1	39	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	20:53:34	0
12	1	39	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	20:18:34	0
12	1	39	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	20:01:00	0
12	1	39	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	19:45:16	0
12	1	39	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	19:16:34	0
1215	50	38	0	irq/30-eth0_g0_	3547	-21	taskset	20:08:49	1
1215	50	38	0	irq/30-eth0_g0_	1221	-21	kworker/1:2	23:21:06	1
1215	50	38	0	irq/30-eth0_g0_	1221	-21	kworker/1:2	23:16:10	1
1215	50	38	0	irq/30-eth0_g0_	1221	-21	kworker/1:2	21:35:23	1
1215	50	38	0	irq/30-eth0_g0_	1221	-21	kworker/1:2	21:20:41	1
1215	50	38	0	irq/30-eth0_g0_	12	1	ksoftirqd/1	22:36:28	1
1215	50	38	0	irq/30-eth0_g0_	12	1	ksoftirqd/1	21:48:05	1
1214	50	38	0	irq/29-eth0_g0_	3670	-21	date	20:09:22	0
1214	50	38	0	irq/29-eth0_g0_	1911	-21	runrttasks	21:33:51	0
1214	50	38	0	irq/29-eth0_g0_	13	-21	kworker/0:1	23:46:37	0
1214	50	38	0	irq/29-eth0_g0_	13	-21	kworker/0:1	23:04:32	0
1214	50	38	0	irq/29-eth0_g0_	13	-21	kworker/0:1	22:45:56	0
1214	50	38	0	irq/29-eth0_g0_	13	-21	kworker/0:1	22:45:56	0
1214	50	38	0	irq/29-eth0_g0_	13	-21	kworker/0:1	21:26:09	0
12	1	38	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	23:50:01	0
12	1	38	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	23:30:22	0
12	1	38	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	20:44:38	0
12	1	38	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	20:41:18	0
12	1	38	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	20:33:53	0
12	1	38	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	20:21:32	0
12	1	38	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	19:41:07	0
12	1	38	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	00:20:43	0
12	1	38	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	00:07:11	0
29948	99	37	1	cyclictest	8299	2	sleep1	20:55:26	1
29948	99	37	1	cyclictest	18244	-21	chrt	22:34:21	1
1215	50	37	0	irq/30-eth0_g0_	13044	-21	taskset	21:42:37	1
1215	50	37	0	irq/30-eth0_g0_	1221	-21	kworker/1:2	23:01:07	1
1215	50	37	0	irq/30-eth0_g0_	1221	-21	kworker/1:2	00:07:32	1
1215	50	37	0	irq/30-eth0_g0_	12	1	ksoftirqd/1	23:35:18	1
1215	50	37	0	irq/30-eth0_g0_	12	1	ksoftirqd/1	22:04:26	1
1215	50	37	0	irq/30-eth0_g0_	12	1	ksoftirqd/1	22:04:26	1
1215	50	37	0	irq/30-eth0_g0_	12	1	ksoftirqd/1	00:01:33	1
1214	50	37	0	irq/29-eth0_g0_	13	-21	kworker/0:1	22:56:09	0
12	1	37	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	00:32:41	1
12	1	37	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	21:51:43	1
29948	99	36	2	cyclictest	6620	-21	munin-run	20:39:21	1
29948	99	36	2	cyclictest	6620	-21	munin-run	20:39:21	1
29948	99	36	1	cyclictest	5647	-21	grep	20:29:22	1
29948	99	36	1	cyclictest	4101	-21	munin-run	20:14:21	1
29948	99	36	1	cyclictest	31497	-21	grep	19:24:22	1
1215	50	36	0	irq/30-eth0_g0_	1221	-21	kworker/1:2	22:39:26	1
1214	50	36	0	irq/29-eth0_g0_	1911	-21	runrttasks	22:21:23	0
1214	50	36	0	irq/29-eth0_g0_	1911	-21	runrttasks	22:04:52	0
1214	50	36	0	irq/29-eth0_g0_	1911	-21	runrttasks	22:04:52	0
12	1	36	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	19:33:43	0
29948	99	35	1	cyclictest	9202	-21	grep	21:04:22	1
29948	99	35	1	cyclictest	30453	-21	taskset	19:14:21	1
1215	50	35	0	irq/30-eth0_g0_	1911	-21	runrttasks	19:05:07	1
1214	50	35	0	irq/29-eth0_g0_	1911	-21	runrttasks	21:00:33	0
1214	50	35	0	irq/29-eth0_g0_	1911	-21	runrttasks	19:55:50	0
1214	50	35	0	irq/29-eth0_g0_	1911	-21	runrttasks	19:52:04	0
1214	50	35	0	irq/29-eth0_g0_	1911	-21	runrttasks	19:24:37	0
1214	50	35	0	irq/29-eth0_g0_	1911	-21	runrttasks	19:12:47	0
1214	50	35	0	irq/29-eth0_g0_	12	1	ksoftirqd/1	22:15:18	1
12	1	35	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	19:34:22	1
12	1	34	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	21:05:56	0
1214	50	33	0	irq/29-eth0_g0_	531	-21	latency_hist	19:39:22	0
1214	50	33	0	irq/29-eth0_g0_	4423	-21	runrttasks	20:17:15	1
1214	50	33	0	irq/29-eth0_g0_	1972	-21	lldpd	20:10:36	0
12	1	33	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	19:40:47	1
29948	99	32	1	cyclictest	0	-21	swapper/1	21:08:10	1
29948	99	32	1	cyclictest	0	-21	swapper/1	20:48:26	1
29948	99	32	1	cyclictest	0	-21	swapper/1	20:39:44	1
29948	99	32	1	cyclictest	0	-21	swapper/1	19:58:01	1
29948	99	32	1	cyclictest	0	-21	swapper/1	19:46:43	1
29948	99	32	1	cyclictest	0	-21	swapper/1	19:18:15	1
1214	50	32	0	irq/29-eth0_g0_	31464	-21	sh	19:24:21	0
1214	50	32	0	irq/29-eth0_g0_	3011	-21	runrttasks	20:03:27	1
1214	50	31	0	irq/29-eth0_g0_	0	-21	swapper/0	20:39:04	0
1214	50	31	0	irq/29-eth0_g0_	0	-21	swapper/0	20:39:04	0
12	1	31	0	ksoftirqd/1	1735	-21	taskset	19:50:34	1
1215	50	30	0	irq/30-eth0_g0_	5184	-21	latency_hist	20:24:22	1
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17