OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rackeslot3s.osadl.org (updated Thu Sep 03, 2026 00:45:31)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
2781229	99	95	78,14	cyclictest	37	-21	ksoftirqd/3	23:40:02	3
2781218	99	95	27,14	cyclictest	2872031	-21	ssh	00:08:36	0
2781218	99	90	29,13	cyclictest	2821219	-21	ssh	22:20:46	0
2781218	99	89	26,11	cyclictest	2883278	-21	ssh	00:32:13	0
2781218	99	84	27,15	cyclictest	2795940	-21	ssh	21:27:46	0
2781218	99	84	26,11	cyclictest	2874834	-21	ssh	00:14:38	0
2781218	99	84	2,30	cyclictest	48	-21	kcompactd0	22:35:45	0
2781218	99	82	27,12	cyclictest	2868578	-21	ssh	00:01:04	0
2781218	99	82	1,36	cyclictest	2879300	-21	ssh	00:24:01	0
2781218	99	81	28,12	cyclictest	2861226	-21	ssh	23:45:20	0
2781218	99	81	26,11	cyclictest	2823686	-21	ssh	22:26:11	0
2781218	99	81	26,11	cyclictest	2823686	-21	ssh	22:26:11	0
2781218	99	80	24,11	cyclictest	2864498	-21	ssh	23:52:33	0
2781218	99	80	24,11	cyclictest	2864498	-21	ssh	23:52:33	0
2781218	99	79	42,32	cyclictest	168	50	irq/134-8000000	00:00:01	0
2781218	99	79	24,9	cyclictest	2843789	-21	ssh	23:08:54	0
2781218	99	79	24,9	cyclictest	2843789	-21	ssh	23:08:54	0
2781218	99	76	21,9	cyclictest	2820825	-21	sort	22:20:02	0
2781218	99	76	20,13	cyclictest	2853366	-21	ssh	23:29:15	0
2781218	99	75	3,26	cyclictest	169	50	irq/224-8000000	22:00:02	0
2781218	99	75	26,12	cyclictest	2807478	-21	ssh	21:51:55	0
2781218	99	75	25,14	cyclictest	2855185	-21	ssh	23:32:55	0
2781218	99	75	25,14	cyclictest	2806285	-21	ssh	21:49:42	0
2781218	99	74	25,13	cyclictest	2832906	-21	ssh	22:45:31	0
2781218	99	74	23,12	cyclictest	2831859	-21	ssh	22:43:38	0
2781218	99	74	23,12	cyclictest	2831859	-21	ssh	22:43:37	0
2781218	99	74	22,14	cyclictest	2860148	-21	ssh	23:43:25	0
2781218	99	74	22,14	cyclictest	2860148	-21	ssh	23:43:24	0
2781229	99	73	19,16	cyclictest	13	1	rcu_preempt	22:20:02	3
2781218	99	73	3,7	cyclictest	2781671	-21	sh	19:20:00	0
2781218	99	73	27,12	cyclictest	2825588	-21	ssh	22:30:03	0
2781218	99	73	27,12	cyclictest	2825588	-21	ssh	22:30:03	0
2781218	99	73	27,11	cyclictest	2793468	-21	ssh	21:22:34	0
2781218	99	73	25,13	cyclictest	2838064	-21	ssh	22:56:30	0
2781218	99	73	20,12	cyclictest	2800369	-21	ssh	21:37:04	0
2781227	99	72	13,19	cyclictest	2839716	-21	ssh	23:00:02	2
2781218	99	72	28,5	cyclictest	2782692	-21	cron	19:40:00	0
2781218	99	71	35,31	cyclictest	168	50	irq/134-8000000	21:20:00	0
2781218	99	71	24,13	cyclictest	2886488	-21	ssh	00:39:15	0
2781218	99	71	23,11	cyclictest	2844885	-21	ssh	23:10:52	0
2781218	99	71	22,10	cyclictest	2813246	-21	ssh	22:04:14	0
2781218	99	70	24,14	cyclictest	2788882	-21	ssh	21:12:59	0
2781218	99	68	26,11	cyclictest	2881974	-21	ssh	00:29:38	0
2781218	99	68	24,11	cyclictest	2837152	-21	ssh	22:54:54	0
2781218	99	68	12,6	cyclictest	168	50	irq/134-8000000	21:33:28	0
2781218	99	67	31,33	cyclictest	169	50	irq/224-8000000	20:30:03	0
2781218	99	67	1,4	cyclictest	2781922	-21	modprobe	19:21:29	0
2781218	99	66	62,2	cyclictest	0	-21	swapper/0	23:35:02	0
2781218	99	66	16,16	cyclictest	168	50	irq/134-8000000	22:07:58	0
2781218	99	65	19,15	cyclictest	2802479	-21	ssh	21:41:25	0
2781218	99	65	1,11	cyclictest	2849045	-21	sh	23:20:01	0
2781218	99	64	26,7	cyclictest	169	50	irq/224-8000000	00:20:01	0
2781218	99	64	2,4	cyclictest	2818331	-21	munin-run	22:15:01	0
2781227	99	63	40,5	cyclictest	2832498	-21	kworker/2:2	23:08:28	2
2781227	99	63	40,5	cyclictest	2832498	-21	kworker/2:2	23:08:27	2
2781218	99	63	28,6	cyclictest	2784015	-21	latency_hist	20:05:02	0
2781218	99	62	21,10	cyclictest	1	-21	systemd	21:00:00	0
2781229	99	61	6,9	cyclictest	37	-21	ksoftirqd/3	21:25:03	3
2781229	99	60	9,12	cyclictest	1	-21	systemd	19:40:00	3
2781229	99	60	16,21	cyclictest	36	1	rcuc/3	21:10:02	3
2781229	99	59	6,9	cyclictest	37	-21	ksoftirqd/3	21:45:01	3
2781229	99	59	11,15	cyclictest	37	-21	ksoftirqd/3	22:10:03	3
2781218	99	59	9,45	cyclictest	168	50	irq/134-8000000	23:24:26	0
2781229	99	58	6,10	cyclictest	37	-21	ksoftirqd/3	22:05:02	3
2781227	99	58	45,8	cyclictest	2784532	-21	kworker/2:1	22:10:00	2
2781218	99	58	17,17	cyclictest	168	50	irq/134-8000000	23:02:27	0
2781229	99	57	6,10	cyclictest	37	-21	ksoftirqd/3	21:00:02	3
2781229	99	57	5,11	cyclictest	37	-21	ksoftirqd/3	00:35:02	3
2781229	99	57	41,13	cyclictest	37	-21	ksoftirqd/3	00:00:00	3
2781218	99	57	1,30	cyclictest	0	-21	swapper/0	19:25:03	0
2781218	99	57	1,30	cyclictest	0	-21	swapper/0	19:25:03	0
2781229	99	56	1,8	cyclictest	150	-21	pr/ttyS2	23:15:02	3
2781229	99	56	10,6	cyclictest	13	1	rcu_preempt	21:51:45	3
2781227	99	56	9,33	cyclictest	310	-21	in:imklog	23:40:03	2
2781222	99	56	39,13	cyclictest	3467602	-21	systemd-journal	19:20:01	1
2781229	99	55	7,11	cyclictest	37	-21	ksoftirqd/3	22:40:33	3
2781229	99	55	7,11	cyclictest	37	-21	ksoftirqd/3	22:40:33	3
2781229	99	55	5,9	cyclictest	37	-21	ksoftirqd/3	21:14:11	3
2781229	99	55	2,8	cyclictest	37	-21	ksoftirqd/3	22:40:00	3
2781218	99	55	27,5	cyclictest	2785570	-21	munin-run	20:35:00	0
2781229	99	54	9,6	cyclictest	13	1	rcu_preempt	23:16:49	3
2781229	99	54	6,7	cyclictest	37	-21	ksoftirqd/3	22:06:49	3
2781229	99	54	6,10	cyclictest	37	-21	ksoftirqd/3	00:39:56	3
2781229	99	54	5,10	cyclictest	37	-21	ksoftirqd/3	00:12:52	3
2781229	99	54	13,6	cyclictest	13	1	rcu_preempt	00:15:59	3
2781227	99	54	45,5	cyclictest	2784532	-21	kworker/2:1	21:49:37	2
2781227	99	54	42,8	cyclictest	2784532	-21	kworker/2:1	23:44:07	2
2781227	99	54	42,8	cyclictest	2784532	-21	kworker/2:1	23:44:07	2
2781227	99	54	30,5	cyclictest	2784532	-21	kworker/2:1	22:41:00	2
2781227	99	54	30,5	cyclictest	2784532	-21	kworker/2:1	22:41:00	2
2781222	99	54	4,11	cyclictest	0	-21	swapper/1	00:00:02	1
2781229	99	53	18,11	cyclictest	37	-21	ksoftirqd/3	23:45:01	3
2781229	99	53	18,11	cyclictest	37	-21	ksoftirqd/3	23:45:01	3
2781229	99	53	13,4	cyclictest	13	1	rcu_preempt	22:30:01	3
2781229	99	53	13,4	cyclictest	13	1	rcu_preempt	22:30:00	3
2781227	99	53	44,6	cyclictest	2784532	-21	kworker/2:1	22:36:19	2
2781229	99	52	8,10	cyclictest	37	-21	ksoftirqd/3	23:08:42	3
2781229	99	52	8,10	cyclictest	37	-21	ksoftirqd/3	23:08:42	3
2781229	99	52	5,8	cyclictest	37	-21	ksoftirqd/3	23:30:52	3
2781229	99	52	4,10	cyclictest	37	-21	ksoftirqd/3	21:20:00	3
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2021-09-01 - 00:33