OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system

Characteristics of the highest latencies:

System rack3slot5s.osadl.org (updated Sun May 31, 2026 00:43:30)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
17208	2	106	0	sleep0	1216	50	irq/29-eth0_g0_	22:31:12	0
12	1	102	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	22:24:40	1
12	1	87	0	ksoftirqd/1	1216	50	irq/29-eth0_g0_	22:38:42	0
12	1	79	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	20:46:09	1
12	1	75	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	22:00:35	1
12	1	63	0	ksoftirqd/1	5546	2	sleep0	23:55:29	0
11988	2	59	0	sleep1	1217	50	irq/30-eth0_g0_	19:59:41	1
12	1	51	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	22:50:39	1
12	1	51	0	ksoftirqd/1	1216	50	irq/29-eth0_g0_	00:01:15	0
12	1	51	0	ksoftirqd/1	1216	50	irq/29-eth0_g0_	00:01:15	0
12	1	49	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	20:54:58	0
12	1	48	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	21:32:10	1
12	1	47	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	21:58:31	1
1217	50	46	0	irq/30-eth0_g0_	9666	2	sleep1	00:14:52	1
1216	50	46	0	irq/29-eth0_g0_	6602	-21	unixbench_multi	19:35:02	0
12	1	46	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	21:37:41	1
12	1	46	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	00:08:25	1
1217	50	45	0	irq/30-eth0_g0_	1219	-21	kworker/1:2	23:49:31	1
1216	50	45	0	irq/29-eth0_g0_	1964	-21	runrttasks	19:06:13	0
1216	50	45	0	irq/29-eth0_g0_	1964	-21	runrttasks	19:06:13	0
12	1	45	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	21:48:12	1
12	1	45	0	ksoftirqd/1	1216	50	irq/29-eth0_g0_	23:05:26	1
12	1	45	0	ksoftirqd/1	1216	50	irq/29-eth0_g0_	19:04:56	1
12	1	45	0	ksoftirqd/1	1216	50	irq/29-eth0_g0_	19:04:56	1
1216	50	44	0	irq/29-eth0_g0_	0	-21	swapper/0	23:04:57	0
12	1	44	0	ksoftirqd/1	1216	50	irq/29-eth0_g0_	23:56:17	1
1217	50	43	0	irq/30-eth0_g0_	1219	-21	kworker/1:2	00:13:50	1
1216	50	43	0	irq/29-eth0_g0_	12807	-21	unixbench_singl	00:25:05	0
12	1	43	0	ksoftirqd/1	1216	50	irq/29-eth0_g0_	23:50:04	1
1217	50	42	0	irq/30-eth0_g0_	1219	-21	kworker/1:2	23:01:36	1
1217	50	42	0	irq/30-eth0_g0_	1219	-21	kworker/1:2	22:58:51	1
1217	50	42	0	irq/30-eth0_g0_	1219	-21	kworker/1:2	22:45:05	1
1217	50	42	0	irq/30-eth0_g0_	1219	-21	kworker/1:2	22:14:12	1
1216	50	42	0	irq/29-eth0_g0_	3	1	ksoftirqd/0	21:24:58	0
1216	50	42	0	irq/29-eth0_g0_	3	1	ksoftirqd/0	21:24:58	0
1216	50	42	0	irq/29-eth0_g0_	1964	-21	runrttasks	23:11:51	0
1216	50	42	0	irq/29-eth0_g0_	12	1	ksoftirqd/1	22:41:25	0
12	1	42	0	ksoftirqd/1	1216	50	irq/29-eth0_g0_	00:30:32	1
32184	99	41	1	cyclictest	18287	-21	unixbench_singl	22:35:03	1
1217	50	41	0	irq/30-eth0_g0_	1219	-21	kworker/1:2	21:20:06	1
1216	50	41	0	irq/29-eth0_g0_	13	-21	kworker/0:1	23:41:23	0
1216	50	41	0	irq/29-eth0_g0_	13	-21	kworker/0:1	00:37:30	0
1216	50	41	0	irq/29-eth0_g0_	12	1	ksoftirqd/1	23:54:29	0
12	1	41	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	23:28:11	1
12	1	41	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	19:55:11	0
12	1	41	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	00:00:02	1
12	1	41	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	00:00:02	1
12	1	41	0	ksoftirqd/1	1216	50	irq/29-eth0_g0_	19:15:01	1
1217	50	40	0	irq/30-eth0_g0_	28362	2	sleep1	23:17:46	1
1216	50	40	0	irq/29-eth0_g0_	32183	99	cyclictest	23:33:05	0
1216	50	40	0	irq/29-eth0_g0_	31946	2	sleep1	23:31:45	1
1216	50	40	0	irq/29-eth0_g0_	3	1	ksoftirqd/0	22:58:09	0
1216	50	40	0	irq/29-eth0_g0_	12	1	ksoftirqd/1	21:54:46	0
1216	50	40	0	irq/29-eth0_g0_	12	1	ksoftirqd/1	21:41:04	1
12	1	40	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	23:47:17	0
12	1	40	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	23:28:29	0
12	1	40	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	21:36:58	0
12	1	40	0	ksoftirqd/1	1216	50	irq/29-eth0_g0_	22:42:21	1
12	1	40	0	ksoftirqd/1	1216	50	irq/29-eth0_g0_	19:45:01	1
32184	99	39	2	cyclictest	13075	-21	netstat	22:14:57	1
32184	99	39	2	cyclictest	13075	-21	netstat	22:14:57	1
1217	50	39	0	irq/30-eth0_g0_	1964	-21	runrttasks	23:42:13	1
1217	50	39	0	irq/30-eth0_g0_	1219	-21	kworker/1:2	22:30:43	1
1216	50	39	0	irq/29-eth0_g0_	30677	-21	irqstats	21:14:58	0
1216	50	39	0	irq/29-eth0_g0_	3	1	ksoftirqd/0	22:20:03	0
1216	50	39	0	irq/29-eth0_g0_	3	1	ksoftirqd/0	00:12:13	0
1216	50	39	0	irq/29-eth0_g0_	13	-21	kworker/0:1	23:35:00	0
1216	50	39	0	irq/29-eth0_g0_	13	-21	kworker/0:1	23:20:28	0
1216	50	39	0	irq/29-eth0_g0_	13	-21	kworker/0:1	22:11:12	0
1216	50	39	0	irq/29-eth0_g0_	13	-21	kworker/0:1	22:01:46	0
1216	50	39	0	irq/29-eth0_g0_	13	-21	kworker/0:1	21:23:29	0
12	1	39	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	23:02:22	0
12	1	39	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	22:53:59	0
12	1	39	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	22:47:33	0
12	1	39	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	22:07:11	0
12	1	39	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	21:40:17	0
12	1	39	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	21:30:28	0
12	1	39	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	20:32:43	0
12	1	39	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	19:54:23	0
12	1	39	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	19:33:09	0
12	1	39	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	19:20:17	0
32184	99	38	2	cyclictest	15655	-21	munin-run	20:14:41	1
31330	2	38	0	sleep1	1217	50	irq/30-eth0_g0_	21:16:56	1
1217	50	38	0	irq/30-eth0_g0_	1219	-21	kworker/1:2	23:39:31	1
1217	50	38	0	irq/30-eth0_g0_	1219	-21	kworker/1:2	00:38:03	1
1217	50	38	0	irq/30-eth0_g0_	1219	-21	kworker/1:2	00:20:34	1
1216	50	38	0	irq/29-eth0_g0_	3	1	ksoftirqd/0	22:29:06	0
1216	50	38	0	irq/29-eth0_g0_	1964	-21	runrttasks	19:28:29	0
1216	50	38	0	irq/29-eth0_g0_	13	-21	kworker/0:1	23:15:32	0
1216	50	38	0	irq/29-eth0_g0_	13	-21	kworker/0:1	21:10:57	0
1216	50	38	0	irq/29-eth0_g0_	13	-21	kworker/0:1	00:30:20	0
12	1	38	0	ksoftirqd/1	1216	50	irq/29-eth0_g0_	19:10:02	1
32184	99	37	23	cyclictest	0	-21	swapper/1	20:15:01	1
32184	99	37	1	cyclictest	11319	2	sleep1	22:08:53	1
1217	50	37	0	irq/30-eth0_g0_	0	-21	swapper/1	23:21:37	1
1217	50	37	0	irq/30-eth0_g0_	0	-21	swapper/1	21:11:00	1
1216	50	37	0	irq/29-eth0_g0_	3	1	ksoftirqd/0	21:47:35	0
1216	50	37	0	irq/29-eth0_g0_	22566	-21	grep	20:40:02	0
1216	50	37	0	irq/29-eth0_g0_	13	-21	kworker/0:1	22:16:02	0
1216	50	37	0	irq/29-eth0_g0_	13	-21	kworker/0:1	22:16:02	0
1216	50	37	0	irq/29-eth0_g0_	13	-21	kworker/0:1	21:52:40	0
1216	50	37	0	irq/29-eth0_g0_	13	-21	kworker/0:1	00:20:01	0
1216	50	37	0	irq/29-eth0_g0_	13	-21	kworker/0:1	00:08:48	0
1216	50	37	0	irq/29-eth0_g0_	12	1	ksoftirqd/1	22:27:02	1
12	1	37	0	ksoftirqd/1	1216	50	irq/29-eth0_g0_	21:07:22	1
32184	99	36	1	cyclictest	6177	-21	ethtool	19:34:54	1
32184	99	36	1	cyclictest	26725	-21	cat	20:59:41	1
32184	99	36	1	cyclictest	1608	-21	cut	21:29:42	1
32184	99	36	1	cyclictest	1608	-21	cut	21:29:42	1
1217	50	36	0	irq/30-eth0_g0_	1219	-21	kworker/1:2	23:12:43	1
1216	50	36	0	irq/29-eth0_g0_	29127	-21	latency_hist	21:09:41	0
1216	50	36	0	irq/29-eth0_g0_	20646	-21	/usr/sbin/munin	20:35:00	0
1216	50	36	0	irq/29-eth0_g0_	1964	-21	runrttasks	20:23:16	0
1216	50	36	0	irq/29-eth0_g0_	13952	-21	unixbench_multi	20:05:03	0
1216	50	36	0	irq/29-eth0_g0_	12759	-21	sed	20:00:02	0
1216	50	36	0	irq/29-eth0_g0_	12	1	ksoftirqd/1	21:50:08	1
12	1	36	0	ksoftirqd/1	1217	50	irq/30-eth0_g0_	00:16:44	0
32184	99	35	1	cyclictest	0	-21	swapper/1	19:59:54	1
1217	50	35	0	irq/30-eth0_g0_	22329	-21	ps	20:39:59	1
1217	50	35	0	irq/30-eth0_g0_	12707	-21	timerandwakeup	00:25:04	1
1216	50	35	0	irq/29-eth0_g0_	32740	-21	proc_pri	19:09:59	0
1216	50	35	0	irq/29-eth0_g0_	25457	-21	latency_hist	20:54:42	0
1216	50	35	0	irq/29-eth0_g0_	1964	-21	runrttasks	20:45:17	0
1216	50	35	0	irq/29-eth0_g0_	18424	-21	interrupts	20:24:56	0
1216	50	35	0	irq/29-eth0_g0_	1438	-21	awk	19:14:56	0
32184	99	34	1	cyclictest	13984	-21	unixbench_singl	20:05:03	1
32184	99	34	1	cyclictest	0	-21	swapper/1	19:52:34	1
1216	50	34	0	irq/29-eth0_g0_	32184	99	cyclictest	19:19:55	1
1216	50	34	0	irq/29-eth0_g0_	27431	-21	grep	21:00:03	0
1216	50	34	0	irq/29-eth0_g0_	15877	-21	df	20:14:53	0
1216	50	34	0	irq/29-eth0_g0_	15176	-21	sed	20:10:01	0
32184	99	33	1	cyclictest	20965	-21	cat	20:34:57	1
32184	99	33	1	cyclictest	0	-21	swapper/1	19:34:39	1
1216	50	33	0	irq/29-eth0_g0_	7827	-21	grep	19:40:02	0
1216	50	33	0	irq/29-eth0_g0_	1964	-21	runrttasks	19:45:26	0
12	1	33	0	ksoftirqd/1	1216	50	irq/29-eth0_g0_	20:59:57	1
32184	99	32	1	cyclictest	0	-21	swapper/1	20:24:05	1
32184	99	32	1	cyclictest	0	-21	swapper/1	19:44:41	1
1217	50	32	0	irq/30-eth0_g0_	4108	-21	sort	19:25:01	1
1217	50	32	0	irq/30-eth0_g0_	18894	-21	grep	20:25:03	1
12	1	31	0	ksoftirqd/1	24573	-21	latency	20:49:55	1
1217	50	30	0	irq/30-eth0_g0_	20135	-21	/usr/sbin/munin	20:30:01	1
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17