OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system

Characteristics of the highest latencies:

System rack3slot5s.osadl.org (updated Thu Dec 11, 2025 00:43:30)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
4385	2	147	0	sleep0	1214	50	irq/29-eth0_g0_	00:10:11	0
30879	2	84	0	sleep1	1215	50	irq/30-eth0_g0_	21:34:16	1
2094	2	84	0	sleep0	1214	50	irq/29-eth0_g0_	21:48:48	0
12	1	84	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	22:29:18	0
9178	2	83	0	sleep0	1215	50	irq/30-eth0_g0_	22:15:43	0
4065	2	76	0	sleep1	1215	50	irq/30-eth0_g0_	00:09:51	1
22934	2	61	0	sleep0	1214	50	irq/29-eth0_g0_	23:14:03	0
12	1	61	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	21:39:21	0
28438	2	60	0	sleep0	1215	50	irq/30-eth0_g0_	21:24:21	0
12	1	58	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	23:40:12	1
30307	2	56	0	sleep0	1214	50	irq/29-eth0_g0_	23:44:16	0
28944	2	53	0	sleep0	1215	50	irq/30-eth0_g0_	23:37:38	0
848	2	46	0	chrt	1215	50	irq/30-eth0_g0_	21:43:57	1
12950	2	46	0	sleep0	1214	50	irq/29-eth0_g0_	22:31:50	0
12	1	46	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	21:27:57	1
12	1	45	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	20:15:14	0
1215	50	44	0	irq/30-eth0_g0_	470	-21	kworker/1:1	22:16:51	1
1214	50	44	0	irq/29-eth0_g0_	3	1	ksoftirqd/0	23:49:57	0
1214	50	44	0	irq/29-eth0_g0_	3	1	ksoftirqd/0	23:49:57	0
12	1	44	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	19:40:33	1
1215	50	43	0	irq/30-eth0_g0_	3	1	ksoftirqd/0	19:30:19	0
1214	50	43	0	irq/29-eth0_g0_	24866	-21	sendmail_mailqu	23:20:20	0
1214	50	43	0	irq/29-eth0_g0_	13	-21	kworker/0:1	23:27:56	0
1214	50	43	0	irq/29-eth0_g0_	12	1	ksoftirqd/1	23:59:20	0
12	1	43	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	22:37:43	1
12	1	43	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	21:58:55	1
12	1	43	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	00:00:11	1
27591	2	42	0	sleep1	1214	50	irq/29-eth0_g0_	23:31:15	1
1215	50	42	0	irq/30-eth0_g0_	470	-21	kworker/1:1	21:46:29	1
1215	50	42	0	irq/30-eth0_g0_	12	1	ksoftirqd/1	23:48:04	1
1215	50	42	0	irq/30-eth0_g0_	12	1	ksoftirqd/1	23:48:04	1
1214	50	42	0	irq/29-eth0_g0_	3	1	ksoftirqd/0	22:00:50	0
1214	50	42	0	irq/29-eth0_g0_	1984	-21	runrttasks	19:09:07	0
1214	50	42	0	irq/29-eth0_g0_	13	-21	kworker/0:1	21:12:45	0
12	1	42	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	23:38:19	1
12	1	42	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	21:24:01	1
12	1	42	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	20:50:03	1
1215	50	41	0	irq/30-eth0_g0_	470	-21	kworker/1:1	23:57:05	1
1215	50	41	0	irq/30-eth0_g0_	470	-21	kworker/1:1	22:54:56	1
1215	50	41	0	irq/30-eth0_g0_	470	-21	kworker/1:1	22:13:21	1
1215	50	41	0	irq/30-eth0_g0_	12	1	ksoftirqd/1	00:34:24	1
1214	50	41	0	irq/29-eth0_g0_	7837	-21	grep	22:10:21	0
1214	50	41	0	irq/29-eth0_g0_	2105	-21	ntp_offset	00:00:14	0
1214	50	41	0	irq/29-eth0_g0_	13	-21	kworker/0:1	22:58:09	0
1214	50	41	0	irq/29-eth0_g0_	13	-21	kworker/0:1	22:58:09	0
1214	50	41	0	irq/29-eth0_g0_	13	-21	kworker/0:1	22:37:13	0
1214	50	41	0	irq/29-eth0_g0_	13	-21	kworker/0:1	21:57:12	0
1214	50	41	0	irq/29-eth0_g0_	12	1	ksoftirqd/1	23:33:16	0
12	1	41	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	22:47:31	0
12	1	41	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	00:06:07	0
1215	50	40	0	irq/30-eth0_g0_	470	-21	kworker/1:1	23:22:41	1
1215	50	40	0	irq/30-eth0_g0_	470	-21	kworker/1:1	00:10:10	1
1215	50	40	0	irq/30-eth0_g0_	12	1	ksoftirqd/1	23:11:12	1
1215	50	40	0	irq/30-eth0_g0_	12	1	ksoftirqd/1	23:02:17	1
1215	50	40	0	irq/30-eth0_g0_	12	1	ksoftirqd/1	22:58:13	1
1215	50	40	0	irq/30-eth0_g0_	12	1	ksoftirqd/1	22:58:13	1
1215	50	40	0	irq/30-eth0_g0_	12	1	ksoftirqd/1	22:04:08	1
1215	50	40	0	irq/30-eth0_g0_	10423	-21	df_abs	00:35:11	1
1214	50	40	0	irq/29-eth0_g0_	13	-21	kworker/0:1	23:16:02	0
1214	50	40	0	irq/29-eth0_g0_	13	-21	kworker/0:1	22:43:20	0
1214	50	40	0	irq/29-eth0_g0_	13	-21	kworker/0:1	22:23:13	0
1214	50	40	0	irq/29-eth0_g0_	13	-21	kworker/0:1	21:29:06	0
1214	50	40	0	irq/29-eth0_g0_	13	-21	kworker/0:1	00:26:51	0
12	1	40	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	00:37:46	0
12	1	40	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	22:48:44	1
12	1	40	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	20:32:56	1
12	1	40	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	20:20:12	1
15500	2	39	0	sleep0	1214	50	irq/29-eth0_g0_	20:30:15	0
1215	50	39	0	irq/30-eth0_g0_	470	-21	kworker/1:1	21:10:37	1
1215	50	39	0	irq/30-eth0_g0_	13179	-21	taskset	22:34:32	1
1214	50	39	0	irq/29-eth0_g0_	9111	-21	df	20:05:10	0
1214	50	39	0	irq/29-eth0_g0_	32660	-21	ntp_states	21:40:17	0
1214	50	39	0	irq/29-eth0_g0_	13	-21	kworker/0:1	22:54:46	0
1214	50	39	0	irq/29-eth0_g0_	13	-21	kworker/0:1	21:53:16	0
12	1	39	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	23:53:08	0
12	1	39	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	23:07:14	0
12	1	39	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	21:16:43	0
12	1	39	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	21:16:43	0
12	1	39	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	19:56:52	0
12	1	39	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	19:41:35	0
12	1	39	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	00:16:16	0
12	1	39	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	22:27:25	1
12	1	39	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	19:10:37	1
12	1	39	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	00:21:03	0
27813	99	38	2	cyclictest	30817	-21	ps	19:20:17	1
27813	99	38	1	cyclictest	24814	-21	diskmemload	23:05:15	1
27813	99	38	1	cyclictest	24814	-21	diskmemload	22:20:08	1
27813	99	38	1	cyclictest	1989	-21	/usr/sbin/munin	19:35:12	1
1215	50	38	0	irq/30-eth0_g0_	12	1	ksoftirqd/1	23:54:49	1
1214	50	38	0	irq/29-eth0_g0_	13	-21	kworker/0:1	21:31:22	0
1214	50	38	0	irq/29-eth0_g0_	0	-21	swapper/0	22:05:53	0
12	1	38	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	21:50:07	1
12	1	38	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	21:36:18	1
12	1	38	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	19:45:22	1
12	1	38	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	19:45:22	1
12	1	38	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	19:28:36	1
27813	99	37	1	cyclictest	9683	-21	unixbench_singl	20:05:19	1
27813	99	37	1	cyclictest	24318	-21	unixbench_multi	21:05:17	1
27813	99	37	1	cyclictest	1984	-21	runrttasks	21:04:46	1
27813	99	37	1	cyclictest	17935	-21	/usr/sbin/munin	20:40:13	1
27813	99	37	1	cyclictest	0	-21	swapper/1	00:24:43	1
1215	50	37	0	irq/30-eth0_g0_	470	-21	kworker/1:1	22:41:10	1
1215	50	37	0	irq/30-eth0_g0_	26950	-21	/usr/sbin/munin	19:05:11	1
1215	50	37	0	irq/30-eth0_g0_	12	1	ksoftirqd/1	21:18:59	1
1215	50	37	0	irq/30-eth0_g0_	12	1	ksoftirqd/1	21:18:59	1
1214	50	37	0	irq/29-eth0_g0_	3	1	ksoftirqd/0	21:09:57	0
1214	50	37	0	irq/29-eth0_g0_	22841	-21	ntpdc	21:00:15	0
1214	50	37	0	irq/29-eth0_g0_	20252	-21	taskset	23:01:18	0
1214	50	37	0	irq/29-eth0_g0_	13	-21	kworker/0:1	00:31:19	0
12	1	37	0	ksoftirqd/1	23812	-21	grep	23:15:21	1
12	1	37	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	20:37:45	1
12	1	37	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	20:37:45	1
27813	99	36	1	cyclictest	6211	-21	sort	22:05:13	1
27813	99	36	1	cyclictest	10474	-21	sort	20:10:12	1
27813	99	36	1	cyclictest	0	-21	swapper/1	20:48:04	1
1215	50	36	0	irq/30-eth0_g0_	470	-21	kworker/1:1	00:29:12	1
1215	50	36	0	irq/30-eth0_g0_	12	1	ksoftirqd/1	00:16:45	1
1214	50	36	0	irq/29-eth0_g0_	1984	-21	runrttasks	19:47:23	0
1214	50	36	0	irq/29-eth0_g0_	1984	-21	runrttasks	19:47:23	0
1214	50	36	0	irq/29-eth0_g0_	1984	-21	runrttasks	19:24:06	0
1214	50	36	0	irq/29-eth0_g0_	13	-21	kworker/0:1	20:45:17	0
12	1	36	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	20:58:51	0
12	1	36	0	ksoftirqd/1	1215	50	irq/30-eth0_g0_	19:19:19	0
1215	50	35	0	irq/30-eth0_g0_	0	-21	swapper/1	23:27:23	1
1215	50	35	0	irq/30-eth0_g0_	0	-21	swapper/1	20:55:21	1
1214	50	35	0	irq/29-eth0_g0_	2129	-21	lldpd	19:52:08	0
1214	50	35	0	irq/29-eth0_g0_	18089	-21	fgrep	20:40:16	0
1214	50	35	0	irq/29-eth0_g0_	10262	-21	df	20:10:09	0
27813	99	34	1	cyclictest	8051	-21	taskset	20:00:15	1
27813	99	34	1	cyclictest	0	-21	swapper/1	20:15:07	1
27813	99	34	1	cyclictest	0	-21	swapper/1	19:33:44	1
1214	50	34	0	irq/29-eth0_g0_	8390	-21	unixbench_multi	20:00:20	0
1214	50	34	0	irq/29-eth0_g0_	31644	-21	df	19:25:10	0
1214	50	34	0	irq/29-eth0_g0_	28591	-21	sed	19:10:19	0
1214	50	34	0	irq/29-eth0_g0_	1982	-21	awk	19:35:12	0
1214	50	34	0	irq/29-eth0_g0_	13198	-21	sendmail_mailst	20:20:16	0
1214	50	33	0	irq/29-eth0_g0_	0	-21	swapper/0	20:54:32	0
12	1	33	0	ksoftirqd/1	1214	50	irq/29-eth0_g0_	19:55:18	1
27813	99	32	1	cyclictest	0	-21	swapper/1	20:27:31	1
1215	50	32	0	irq/30-eth0_g0_	0	-21	swapper/1	19:50:09	1
1214	50	32	0	irq/29-eth0_g0_	1984	-21	runrttasks	20:35:57	0
1214	50	32	0	irq/29-eth0_g0_	1984	-21	runrttasks	20:35:57	0
1214	50	32	0	irq/29-eth0_g0_	15013	-21	munin-run	20:29:58	0
1215	50	31	0	irq/30-eth0_g0_	0	-21	swapper/1	19:15:08	1
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17