OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rackeslot3s.osadl.org (updated Fri May 08, 2026 12:45:39)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
2761979	99	92	1,88	cyclictest	0	-21	swapper/0	08:40:02	0
2761979	99	92	1,88	cyclictest	0	-21	swapper/0	08:40:01	0
2761992	99	84	29,25	cyclictest	2846527	-21	sh	11:52:14	3
2761992	99	83	1,50	cyclictest	2863520	-21	ssh	12:27:29	3
2761988	99	82	2,66	cyclictest	2789999	-21	rm	09:55:02	2
2761992	99	81	2,57	cyclictest	2852543	50	ssh	12:05:01	3
2761979	99	79	1,4	cyclictest	0	-21	swapper/0	09:20:03	0
2761992	99	78	2,43	cyclictest	2816897	-21	ssh	10:50:38	3
2761984	99	78	15,12	cyclictest	0	-21	swapper/1	12:40:02	1
2761992	99	77	2,42	cyclictest	2837999	-21	ssh	11:34:54	3
2761992	99	72	26,40	cyclictest	275	-21	dbus-daemon	12:00:01	3
2761992	99	72	26,40	cyclictest	275	-21	dbus-daemon	12:00:00	3
2761992	99	71	1,45	cyclictest	2785453	-21	ssh	09:45:22	3
2761992	99	71	1,45	cyclictest	2785453	-21	ssh	09:45:21	3
2761979	99	71	7,7	cyclictest	12	-21	ksoftirqd/0	10:12:14	0
2761992	99	69	1,42	cyclictest	2860107	-21	ssh	12:20:11	3
2761992	99	69	1,42	cyclictest	2860107	-21	ssh	12:20:10	3
2761992	99	68	27,36	cyclictest	2808393	-21	sh	10:33:20	3
2761992	99	68	22,42	cyclictest	37	-21	ksoftirqd/3	12:30:32	3
2761992	99	68	22,42	cyclictest	37	-21	ksoftirqd/3	12:30:31	3
2761992	99	67	1,62	cyclictest	2795132	-21	ssh	10:05:32	3
2761979	99	67	46,16	cyclictest	2814055	-21	latency_hist	10:45:02	0
2761992	99	66	2,35	cyclictest	2856430	-21	ssh	12:12:49	3
2761992	99	66	22,40	cyclictest	2869427	-21	run-parts	12:40:00	3
2761992	99	66	1,60	cyclictest	2805529	-21	sshd	10:27:15	3
2761992	99	66	1,60	cyclictest	2805529	-21	sshd	10:27:14	3
2761988	99	66	2,10	cyclictest	30	-21	ksoftirqd/2	12:40:01	2
2761992	99	65	22,40	cyclictest	37	-21	ksoftirqd/3	11:39:57	3
2761992	99	65	1,60	cyclictest	2810951	-21	ssh	10:38:37	3
2761992	99	65	1,60	cyclictest	2791990	-21	ssh	09:59:25	3
2761992	99	65	1,60	cyclictest	2791990	-21	ssh	09:59:24	3
2761979	99	65	33,28	cyclictest	12	-21	ksoftirqd/0	10:30:39	0
2761992	99	64	27,34	cyclictest	2843579	-21	sh	11:46:07	3
2761992	99	64	27,34	cyclictest	2843579	-21	sh	11:46:06	3
2761992	99	64	2,58	cyclictest	2827549	-21	ssh	11:12:59	3
2761992	99	64	2,57	cyclictest	2832309	-21	ssh	11:22:55	3
2761992	99	64	21,38	cyclictest	2789152	-21	sh	09:53:24	3
2761992	99	64	1,5	cyclictest	2803891	-21	ssh	10:23:56	3
2761992	99	64	1,59	cyclictest	2773522	-21	ssh	09:20:46	3
2761992	99	64	1,58	cyclictest	2841504	-21	ssh	11:41:47	3
2761992	99	64	1,58	cyclictest	2776697	-21	ssh	09:27:28	3
2761988	99	64	23,10	cyclictest	30	-21	ksoftirqd/2	09:30:01	2
2761992	99	63	19,40	cyclictest	37	-21	ksoftirqd/3	10:56:03	3
2761992	99	63	1,58	cyclictest	2852973	-21	ssh	12:05:29	3
2761992	99	63	1,58	cyclictest	2821764	-21	ssh	11:00:38	3
2761992	99	63	1,58	cyclictest	2800716	-21	ssh	10:17:24	3
2761992	99	63	1,58	cyclictest	2793553	-21	ssh	10:02:24	3
2761992	99	63	1,58	cyclictest	2780943	-21	ssh	09:36:13	3
2761992	99	63	1,58	cyclictest	2779693	-21	ssh	09:33:55	3
2761992	99	63	1,58	cyclictest	2768820	-21	ssh	09:11:09	3
2761992	99	63	1,58	cyclictest	2768820	-21	ssh	09:11:08	3
2761992	99	63	1,4	cyclictest	2813115	-21	ssh	10:42:54	3
2761984	99	63	8,8	cyclictest	23	-21	ksoftirqd/1	10:15:02	1
2761984	99	63	42,10	cyclictest	23	-21	ksoftirqd/1	10:05:00	1
2761984	99	63	15,12	cyclictest	23	-21	ksoftirqd/1	10:46:02	1
2761984	99	63	15,12	cyclictest	23	-21	ksoftirqd/1	10:46:01	1
2761992	99	62	2,55	cyclictest	2765304	-21	munin-run	08:15:01	3
2761992	99	62	2,55	cyclictest	2763232	-21	munin-run	07:35:00	3
2761992	99	62	1,8	cyclictest	2783752	-21	ssh	09:42:02	3
2761992	99	62	1,57	cyclictest	2858464	-21	ssh	12:17:07	3
2761992	99	62	1,57	cyclictest	2828722	-21	ssh	11:15:09	3
2761992	99	62	1,57	cyclictest	2824335	-21	ssh	11:06:05	3
2761992	99	62	1,57	cyclictest	2824335	-21	ssh	11:06:04	3
2761992	99	62	1,57	cyclictest	2814715	-21	ssh	10:46:04	3
2761992	99	62	1,57	cyclictest	2814715	-21	ssh	10:46:03	3
2761992	99	62	1,57	cyclictest	2771077	-21	ssh	09:15:56	3
2761992	99	62	1,56	cyclictest	2833892	-21	ssh	11:25:53	3
2761992	99	62	1,55	cyclictest	2766339	-21	munin-run	08:35:01	3
2761984	99	62	9,9	cyclictest	23	-21	ksoftirqd/1	10:21:25	1
2761979	99	62	22,9	cyclictest	2785097	-21	ls	09:45:02	0
2761979	99	62	13,44	cyclictest	168	50	irq/134-8000000	12:30:01	0
2761992	99	61	4,50	cyclictest	1	-21	systemd	08:05:01	3
2761992	99	61	2,55	cyclictest	2798986	-21	ssh	10:13:54	3
2761992	99	61	2,54	cyclictest	2766599	-21	munin-run	08:40:00	3
2761992	99	61	2,54	cyclictest	2766599	-21	munin-run	08:40:00	3
2761992	99	61	2,54	cyclictest	2763715	-21	sshd	07:40:14	3
2761992	99	61	2,54	cyclictest	2762980	-21	munin-run	07:30:02	3
2761992	99	61	2,54	cyclictest	2762980	-21	munin-run	07:30:01	3
2761979	99	61	48,10	cyclictest	3467602	-21	systemd-journal	11:25:02	0
2761992	99	60	1,55	cyclictest	2767597	-21	sshd	08:56:14	3
2761992	99	59	1,53	cyclictest	2768124	-21	sshd	09:05:14	3
2761992	99	59	1,53	cyclictest	2764169	-21	cat	07:50:02	3
2761988	99	59	8,8	cyclictest	30	-21	ksoftirqd/2	10:12:44	2
2761988	99	59	15,10	cyclictest	2833422	-21	ssh	11:25:01	2
2761988	99	59	12,18	cyclictest	2804347	-21	cron	10:24:59	2
2761979	99	59	20,34	cyclictest	168	50	irq/134-8000000	09:35:02	0
2761992	99	58	2,51	cyclictest	2764473	-21	sshd	07:55:15	3
2761992	99	58	2,51	cyclictest	2764473	-21	sshd	07:55:14	3
2761992	99	58	1,4	cyclictest	2762405	-21	sshd	07:17:14	3
2761988	99	58	15,8	cyclictest	30	-21	ksoftirqd/2	07:40:01	2
2761988	99	58	14,8	cyclictest	30	-21	ksoftirqd/2	11:13:24	2
2761979	99	58	8,5	cyclictest	13	1	rcu_preempt	11:48:48	0
2761979	99	58	8,5	cyclictest	13	1	rcu_preempt	11:48:47	0
2761979	99	58	18,31	cyclictest	168	50	irq/134-8000000	12:10:28	0
2761992	99	57	2,50	cyclictest	2765012	-21	sshd	08:07:14	3
2761992	99	57	1,51	cyclictest	48	-21	kcompactd0	08:20:45	3
2761992	99	57	1,51	cyclictest	2767126	-21	munin-run	08:50:00	3
2761992	99	57	1,51	cyclictest	2767126	-21	munin-run	08:50:00	3
2761992	99	57	1,51	cyclictest	2765506	-21	sshd	08:15:06	3
2761988	99	57	7,11	cyclictest	2854336	-21	ssh	12:08:37	2
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2021-09-01 - 00:33