OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack7slot4s.osadl.org (updated Sat Aug 01, 2026 00:45:07)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
21151	99	64	23,19	cyclictest	21	-21	ksoftirqd/1	21:25:13	1
21160	99	60	8,5	cyclictest	13	1	rcu_preempt	20:35:14	2
21160	99	59	1,37	cyclictest	16262	-21	ssh	22:45:10	2
21151	99	56	1,44	cyclictest	15088	-21	ssh	00:35:12	1
21168	99	52	4,30	cyclictest	359	66	irq/58-eth0	23:05:13	3
21160	99	52	8,22	cyclictest	27	-21	ksoftirqd/2	22:10:13	2
21160	99	52	8,22	cyclictest	27	-21	ksoftirqd/2	22:10:12	2
21143	99	52	17,30	cyclictest	13	1	rcu_preempt	19:50:17	0
21151	99	51	9,4	cyclictest	13	1	rcu_preempt	22:15:14	1
21160	99	49	1,34	cyclictest	27177	-21	ssh	22:15:13	2
21160	99	47	1,36	cyclictest	0	-21	swapper/2	00:15:11	2
21168	99	46	23,17	cyclictest	359	66	irq/58-eth0	21:25:12	3
21160	99	46	1,29	cyclictest	20513	-21	ssh	00:20:17	2
21151	99	46	2,35	cyclictest	9188	-21	ssh	22:05:12	1
21143	99	46	6,36	cyclictest	5399	-21	ssh	23:15:14	0
21143	99	46	6,36	cyclictest	5399	-21	ssh	23:15:13	0
21143	99	45	19,21	cyclictest	12	-21	ksoftirqd/0	22:50:13	0
21143	99	44	33,6	cyclictest	6843	-21	diskmemload	22:25:11	0
21143	99	43	8,31	cyclictest	25349	-21	ssh	21:20:13	0
21168	99	42	17,15	cyclictest	359	66	irq/58-eth0	23:20:12	3
21143	99	42	5,9	cyclictest	13	1	rcu_preempt	23:10:13	0
21143	99	42	1,36	cyclictest	7340	-21	ssh	22:40:13	0
21143	99	41	2,31	cyclictest	23650	-21	ssh	21:55:15	0
21160	99	40	19,15	cyclictest	27	-21	ksoftirqd/2	22:20:12	2
21143	99	40	1,32	cyclictest	10867	-21	ssh	23:00:15	0
21160	99	39	16,13	cyclictest	21135	-21	cyclictest	20:15:12	2
21143	99	39	4,27	cyclictest	7326	-21	ssh	21:10:14	0
21151	99	38	1,8	cyclictest	13	1	rcu_preempt	00:25:13	1
21143	99	38	7,22	cyclictest	12	-21	ksoftirqd/0	19:20:15	0
21160	99	37	3,25	cyclictest	0	-21	swapper/2	22:51:57	2
21160	99	37	1,16	cyclictest	27090	-21	ssh	23:45:13	2
21151	99	37	6,19	cyclictest	27071	-21	ssh	23:45:13	1
21143	99	36	18,15	cyclictest	1829	-21	ssh	22:55:13	0
21160	99	35	2,17	cyclictest	19305	-21	ssh	22:27:33	2
21160	99	35	13,17	cyclictest	27	-21	ksoftirqd/2	21:35:11	2
21160	99	35	10,16	cyclictest	27	-21	ksoftirqd/2	20:25:13	2
21143	99	35	11,17	cyclictest	12	-21	ksoftirqd/0	21:40:13	0
21168	99	33	6,21	cyclictest	359	66	irq/58-eth0	23:55:14	3
21168	99	32	2,19	cyclictest	359	66	irq/58-eth0	22:55:13	3
21160	99	32	4,12	cyclictest	26	1	rcuc/2	19:50:18	2
21151	99	32	3,22	cyclictest	18107	-21	ssh	23:40:12	1
21151	99	32	1,25	cyclictest	19839	-21	sh	23:05:14	1
21168	99	31	2,19	cyclictest	359	66	irq/58-eth0	23:00:14	3
21160	99	31	2,22	cyclictest	32498	-21	ssh	23:30:14	2
21151	99	31	24,3	cyclictest	0	-21	swapper/1	21:30:12	1
21143	99	31	9,16	cyclictest	12	-21	ksoftirqd/0	19:10:12	0
21143	99	31	3,22	cyclictest	23468	-21	ssh	23:25:11	0
21168	99	30	5,15	cyclictest	359	66	irq/58-eth0	22:10:14	3
21168	99	30	5,15	cyclictest	359	66	irq/58-eth0	22:10:14	3
21168	99	30	1,17	cyclictest	359	66	irq/58-eth0	22:40:12	3
21160	99	30	3,21	cyclictest	0	-21	swapper/2	19:40:14	2
21160	99	30	3,18	cyclictest	28850	-21	ssh	23:10:15	2
21160	99	30	12,12	cyclictest	27	-21	ksoftirqd/2	00:30:14	2
21168	99	29	9,11	cyclictest	0	-21	swapper/3	00:00:11	3
21160	99	29	4,19	cyclictest	273	46	irq/40-dwc2_hso	23:50:13	2
21160	99	29	2,21	cyclictest	32612	-21	sh	00:07:43	2
21160	99	29	2,11	cyclictest	22034	-21	ssh	21:53:33	2
21151	99	29	2,22	cyclictest	31286	-21	ssh	22:51:56	1
21143	99	29	6,11	cyclictest	29456	-21	ssh	00:25:14	0
21143	99	29	4,20	cyclictest	7489	-21	kworker/u8:0	00:15:12	0
21160	99	28	6,17	cyclictest	0	-21	swapper/2	21:40:15	2
21160	99	28	3,12	cyclictest	273	46	irq/40-dwc2_hso	20:55:13	2
21160	99	28	2,19	cyclictest	0	-21	swapper/2	19:55:12	2
21160	99	28	1,15	cyclictest	7332	-21	ssh	21:10:14	2
21160	99	28	1,15	cyclictest	32663	-21	ssh	22:00:15	2
21143	99	28	2,5	cyclictest	13	1	rcu_preempt	19:25:14	0
21160	99	27	9,13	cyclictest	18110	-21	diskmemload	23:40:13	2
21160	99	27	1,20	cyclictest	273	46	irq/40-dwc2_hso	20:45:14	2
21151	99	27	2,19	cyclictest	844	-21	munin-node	19:55:13	1
21151	99	27	2,19	cyclictest	15531	-21	ssh	22:44:34	1
21151	99	27	2,18	cyclictest	30783	-21	ssh	22:35:13	1
21151	99	27	19,4	cyclictest	0	-21	swapper/1	22:00:15	1
21151	99	27	1,18	cyclictest	23477	-21	ssh	23:25:14	1
21143	99	27	3,14	cyclictest	12	-21	ksoftirqd/0	19:30:11	0
21143	99	27	2,9	cyclictest	0	-21	swapper/0	20:35:14	0
21143	99	27	2,19	cyclictest	3798	-21	ssh	22:20:14	0
21143	99	27	2,19	cyclictest	27763	-21	ssh	22:31:53	0
21143	99	27	2,19	cyclictest	14922	-21	ssh	22:06:35	0
21143	99	27	13,6	cyclictest	0	-21	swapper/0	23:55:14	0
21168	99	26	3,15	cyclictest	0	-21	swapper/3	22:06:35	3
21160	99	26	9,12	cyclictest	27	-21	ksoftirqd/2	22:06:47	2
21160	99	26	7,9	cyclictest	27	-21	ksoftirqd/2	23:05:13	2
21160	99	26	1,7	cyclictest	9614	-21	cut	20:05:17	2
21160	99	26	10,12	cyclictest	19434	-21	cron	23:05:02	2
21143	99	26	9,9	cyclictest	13	1	rcu_preempt	20:55:13	0
21143	99	26	2,18	cyclictest	25557	-21	ssh	22:13:34	0
21143	99	26	2,18	cyclictest	25557	-21	ssh	22:13:33	0
21168	99	25	4,13	cyclictest	0	-21	swapper/3	21:10:14	3
21168	99	25	2,17	cyclictest	0	-21	swapper/3	23:35:15	3
21168	99	25	1,18	cyclictest	0	-21	swapper/3	22:01:45	3
21160	99	25	9,12	cyclictest	0	-21	swapper/2	20:00:29	2
21160	99	25	3,13	cyclictest	273	46	irq/40-dwc2_hso	19:35:12	2
21160	99	25	12,11	cyclictest	27	-21	ksoftirqd/2	21:33:36	2
21160	99	25	11,9	cyclictest	6302	-21	switchtime	00:10:25	2
21160	99	25	1,15	cyclictest	27511	-21	ls	20:20:24	2
21151	99	25	2,17	cyclictest	6843	-21	diskmemload	00:07:44	1
21151	99	25	1,18	cyclictest	2457	-21	ssh	00:10:09	1
21143	99	25	7,15	cyclictest	12	-21	ksoftirqd/0	21:25:11	0
21143	99	25	11,11	cyclictest	20401	-21	ssh	21:15:30	0
21143	99	25	10,11	cyclictest	2109	-21	ssh	22:35:27	0
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17