OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system

Characteristics of the 376 highest latencies:

System rack3slot5s.osadl.org (updated Thu Apr 30, 2026 00:43:29)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
12	1	104	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	21:38:45	1
11512	2	99	0	sleep0	1215	50	irq/30-eth0_g0_	23:06:08	0
11927	2	90	0	sleep1	1215	50	irq/30-eth0_g0_	23:10:04	1
12	1	85	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	23:55:22	0
12	1	84	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	23:54:28	1
1801	2	62	0	chrt	1215	50	irq/30-eth0_g0_	21:30:40	0
12	1	62	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	00:38:51	0
12	1	54	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	22:36:32	1
12	1	53	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	21:26:16	1
12	1	52	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	23:28:42	1
12	1	52	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	00:31:45	1
12	1	51	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	19:48:28	1
12	1	50	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	23:36:17	1
12	1	50	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	23:33:16	1
12	1	49	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	00:23:07	1
12	1	48	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	23:13:36	0
12	1	47	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	21:14:43	1
12	1	47	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	00:12:15	1
12	1	47	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	19:50:17	1
12	1	46	0	ksoftirqd/1	19808	99	cyclictest	22:53:35	1
12	1	46	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	21:11:04	0
12	1	46	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	00:00:52	1
12	1	46	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	21:48:32	1
12	1	46	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	20:59:29	1
12	1	45	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	21:40:34	1
12	1	45	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	20:09:03	1
12	1	45	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	20:09:03	1
12	1	45	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	00:27:49	0
1214	50	44	0	irq/29-eth0_g0_	1911	-21	runrttasks	00:15:40	0
12	1	44	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	22:44:48	1
20534	2	43	0	sleep1	1215	50	irq/30-eth0_g0_	00:34:35	1
1215	50	42	0	irq/30-eth0_g0_	1221	-21	kworker/1:2	22:56:17	1
1214	50	42	0	irq/29-eth0_g0_	12	1	ksoftirqd/1	22:25:41	0
1214	50	41	0	irq/29-eth0_g0_	13	-21	kworker/0:1	23:31:36	0
12	1	41	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	23:43:44	1
19808	99	40	10	cyclictest	0	-21	swapper/1	21:10:48	1
1215	50	40	0	irq/30-eth0_g0_	1221	-21	kworker/1:2	22:32:41	1
1215	50	40	0	irq/30-eth0_g0_	12	1	ksoftirqd/1	00:05:40	1
1214	50	40	0	irq/29-eth0_g0_	3	1	ksoftirqd/0	23:26:11	0
1214	50	40	0	irq/29-eth0_g0_	1911	-21	runrttasks	00:09:36	0
1214	50	40	0	irq/29-eth0_g0_	13	-21	kworker/0:1	23:49:15	0
1214	50	40	0	irq/29-eth0_g0_	13	-21	kworker/0:1	21:22:46	0
1214	50	40	0	irq/29-eth0_g0_	13	-21	kworker/0:1	21:22:46	0
12	1	40	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	23:44:16	0
12	1	40	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	22:53:43	0
12	1	40	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	20:20:48	0
12	1	40	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	22:04:56	1
19808	99	39	1	cyclictest	30495	-21	cat	20:54:28	1
1215	50	39	0	irq/30-eth0_g0_	1221	-21	kworker/1:2	23:01:32	1
1215	50	39	0	irq/30-eth0_g0_	1221	-21	kworker/1:2	23:01:32	1
1215	50	39	0	irq/30-eth0_g0_	0	-21	swapper/1	22:41:07	1
1214	50	39	0	irq/29-eth0_g0_	1911	-21	runrttasks	20:57:13	0
1214	50	39	0	irq/29-eth0_g0_	13	-21	kworker/0:1	23:59:37	0
1214	50	39	0	irq/29-eth0_g0_	13	-21	kworker/0:1	22:41:58	0
1214	50	39	0	irq/29-eth0_g0_	13	-21	kworker/0:1	22:39:18	0
1214	50	39	0	irq/29-eth0_g0_	13	-21	kworker/0:1	22:20:30	0
1214	50	39	0	irq/29-eth0_g0_	13	-21	kworker/0:1	21:46:35	0
12	1	39	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	23:01:16	0
12	1	39	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	23:01:16	0
12	1	39	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	22:12:31	0
12	1	39	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	22:05:45	0
12	1	39	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	22:00:26	0
12	1	39	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	21:50:50	0
12	1	39	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	21:14:29	0
12	1	39	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	21:04:29	0
12	1	39	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	20:40:28	0
12	1	39	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	20:36:38	0
12	1	39	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	19:56:19	0
12	1	39	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	19:52:50	0
12	1	39	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	19:35:21	0
12	1	39	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	19:30:41	0
12	1	39	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	19:25:44	0
12	1	39	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	19:12:53	0
12	1	39	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	00:06:59	0
12	1	39	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	19:59:17	1
19808	99	38	2	cyclictest	13323	-21	munin-run	23:24:28	1
19808	99	38	1	cyclictest	32019	-21	diskmemload	21:53:48	1
19808	99	38	1	cyclictest	15925	-21	latency_hist	23:49:29	1
1215	50	38	0	irq/30-eth0_g0_	12	1	ksoftirqd/1	21:20:33	1
1215	50	38	0	irq/30-eth0_g0_	12	1	ksoftirqd/1	21:20:33	1
1214	50	38	0	irq/29-eth0_g0_	13	-21	kworker/0:1	23:15:27	0
1214	50	38	0	irq/29-eth0_g0_	13	-21	kworker/0:1	21:57:33	0
1214	50	38	0	irq/29-eth0_g0_	13	-21	kworker/0:1	00:21:15	0
1214	50	38	0	irq/29-eth0_g0_	12	1	ksoftirqd/1	00:19:03	1
12	1	38	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	23:36:14	0
12	1	38	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	22:59:19	0
12	1	38	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	21:27:16	0
19808	99	37	7	cyclictest	1215	50	irq/30-eth0_g0_	00:25:06	1
19808	99	37	2	cyclictest	28004	-21	cat	20:29:29	1
1215	50	37	0	irq/30-eth0_g0_	21989	-21	chrt	19:30:17	1
1215	50	37	0	irq/30-eth0_g0_	1221	-21	kworker/1:2	23:05:52	1
1215	50	37	0	irq/30-eth0_g0_	1221	-21	kworker/1:2	21:55:18	1
1215	50	37	0	irq/30-eth0_g0_	12	1	ksoftirqd/1	22:19:57	1
1215	50	37	0	irq/30-eth0_g0_	12	1	ksoftirqd/1	22:17:41	1
1214	50	37	0	irq/29-eth0_g0_	3	1	ksoftirqd/0	22:15:12	0
1214	50	37	0	irq/29-eth0_g0_	1911	-21	runrttasks	20:04:37	0
1214	50	37	0	irq/29-eth0_g0_	1911	-21	runrttasks	20:04:37	0
1214	50	37	0	irq/29-eth0_g0_	13	-21	kworker/0:1	22:46:46	0
1214	50	37	0	irq/29-eth0_g0_	13	-21	kworker/0:1	21:42:45	0
1214	50	37	0	irq/29-eth0_g0_	13	-21	kworker/0:1	00:30:47	0
1214	50	37	0	irq/29-eth0_g0_	0	-21	swapper/0	19:06:01	0
12	1	37	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	21:00:20	0
12	1	37	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	21:36:19	0
19808	99	36	8	cyclictest	1215	50	irq/30-eth0_g0_	23:14:35	1
1215	50	36	0	irq/30-eth0_g0_	1221	-21	kworker/1:2	21:30:05	1
1215	50	36	0	irq/30-eth0_g0_	12	1	ksoftirqd/1	22:04:19	1
1214	50	36	0	irq/29-eth0_g0_	1911	-21	runrttasks	20:27:24	0
1214	50	36	0	irq/29-eth0_g0_	1911	-21	runrttasks	20:14:36	0
1214	50	36	0	irq/29-eth0_g0_	1911	-21	runrttasks	19:46:04	0
1214	50	36	0	irq/29-eth0_g0_	1911	-21	runrttasks	19:17:15	0
1214	50	36	0	irq/29-eth0_g0_	13	-21	kworker/0:1	23:22:06	0
1214	50	36	0	irq/29-eth0_g0_	13	-21	kworker/0:1	22:30:24	0
1214	50	36	0	irq/29-eth0_g0_	12	1	ksoftirqd/1	23:56:55	1
1214	50	36	0	irq/29-eth0_g0_	12	1	ksoftirqd/1	22:25:17	1
12	1	36	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	20:53:47	0
19808	99	35	1	cyclictest	32016	-21	latency_hist	21:09:28	1
19808	99	35	1	cyclictest	31539	-21	cat	21:04:29	1
19808	99	35	1	cyclictest	0	-21	swapper/1	19:18:19	1
1215	50	35	0	irq/30-eth0_g0_	12	1	ksoftirqd/1	22:14:08	1
1214	50	35	0	irq/29-eth0_g0_	1911	-21	runrttasks	23:51:52	0
12	1	35	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	20:47:14	0
19808	99	34	1	cyclictest	0	-21	swapper/1	20:10:09	1
19808	99	34	1	cyclictest	0	-21	swapper/1	20:01:53	1
19808	99	34	1	cyclictest	0	-21	swapper/1	19:25:02	1
19808	99	34	0	cyclictest	0	-21	swapper/1	20:36:22	1
1214	50	34	0	irq/29-eth0_g0_	1911	-21	runrttasks	20:10:52	0
1214	50	34	0	irq/29-eth0_g0_	1911	-21	runrttasks	20:01:21	0
1214	50	34	0	irq/29-eth0_g0_	1911	-21	runrttasks	19:41:38	0
19808	99	33	1	cyclictest	0	-21	swapper/1	20:30:03	1
19808	99	33	1	cyclictest	0	-21	swapper/1	19:10:25	1
1215	50	33	0	irq/30-eth0_g0_	26952	-21	sed	20:19:28	1
1214	50	33	0	irq/29-eth0_g0_	19660	-21	runrttasks	19:07:54	1
12	1	33	0	ksoftirqd/1	29639	-21	taskset	20:45:29	1
19808	99	32	27	cyclictest	29502	-21	latency_hist	20:44:28	1
19808	99	32	1	cyclictest	0	-21	swapper/1	19:41:41	1
19808	99	32	1	cyclictest	0	-21	swapper/1	19:35:52	1
1214	50	32	0	irq/29-eth0_g0_	28464	-21	wc	20:34:28	0
19808	99	31	1	cyclictest	0	-21	swapper/1	20:21:19	1
1215	50	31	0	irq/30-eth0_g0_	0	-21	swapper/1	19:20:20	1
1214	50	30	0	irq/29-eth0_g0_	21340	-21	latency_hist	19:24:28	0
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17