OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rackeslot3s.osadl.org (updated Wed Mar 18, 2026 00:45:32)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
3650690	99	123	7,3	cyclictest	13	1	rcu_preempt	00:30:01	1
3650690	99	91	2,49	cyclictest	3713589	-21	sh	23:16:22	1
3650687	99	85	81,2	cyclictest	0	-21	swapper/0	21:25:02	0
3650697	99	81	15,39	cyclictest	3721792	-21	cron	23:35:02	2
3650687	99	79	5,69	cyclictest	3650695	-21	cat	19:10:01	0
3650687	99	79	5,69	cyclictest	3650695	-21	cat	19:10:01	0
3650697	99	72	1,36	cyclictest	3720280	-21	ssh	23:31:17	2
3650697	99	70	44,5	cyclictest	3644161	-21	kworker/2:3	20:35:02	2
3650697	99	70	44,5	cyclictest	3644161	-21	kworker/2:3	20:35:01	2
3650697	99	69	1,34	cyclictest	3704946	-21	ssh	22:56:57	2
3650690	99	69	23,41	cyclictest	3674127	-21	sh	21:48:32	1
3650697	99	67	23,39	cyclictest	30	-21	ksoftirqd/2	22:10:01	2
3650697	99	65	27,34	cyclictest	3662959	-21	sh	21:23:26	2
3650690	99	65	4,56	cyclictest	3651668	-21	cron	19:29:59	1
3650690	99	65	2,60	cyclictest	3690467	-21	ssh	22:25:01	1
3650697	99	64	25,34	cyclictest	3738291	-21	sh	00:11:36	2
3650697	99	64	25,34	cyclictest	3738291	-21	sh	00:11:35	2
3650697	99	64	22,37	cyclictest	1	-21	systemd	00:05:02	2
3650690	99	63	31,26	cyclictest	275	-21	dbus-daemon	21:40:01	1
3650690	99	63	2,55	cyclictest	3680211	-21	sshd	22:01:49	1
3650690	99	63	1,16	cyclictest	3687111	-21	ssh	22:17:18	1
3650687	99	63	18,40	cyclictest	168	50	irq/134-8000000	00:05:01	0
3650697	99	62	26,32	cyclictest	30	-21	ksoftirqd/2	23:07:09	2
3650697	99	62	24,35	cyclictest	30	-21	ksoftirqd/2	22:25:13	2
3650697	99	61	23,34	cyclictest	3653752	-21	latency_hist	20:10:00	2
3650690	99	61	22,35	cyclictest	3729105	-21	sh	23:50:54	1
3650690	99	61	22,34	cyclictest	3694857	-21	sh	22:34:54	1
3650690	99	61	1,16	cyclictest	3663522	-21	rm	21:24:55	1
3650690	99	61	1,15	cyclictest	3735763	-21	sh	00:05:53	1
3650690	99	61	1,15	cyclictest	3697404	-21	sh	22:40:08	1
3650697	99	60	2,52	cyclictest	3673244	-21	ssh	21:46:14	2
3650697	99	60	1,55	cyclictest	0	-21	swapper/2	20:30:02	2
3650697	99	60	1,54	cyclictest	3746881	-21	ssh	00:30:45	2
3650697	99	60	1,53	cyclictest	3743090	-21	ssh	00:22:32	2
3650697	99	60	1,53	cyclictest	3701787	-21	latency_hist	22:50:02	2
3650690	99	60	29,27	cyclictest	3667360	-21	sh	21:33:24	1
3650690	99	60	2,53	cyclictest	3676986	-21	cron	21:55:01	1
3650690	99	60	2,53	cyclictest	3664577	-21	sh	21:26:50	1
3650690	99	60	2,19	cyclictest	3657257	-21	ssh	21:10:23	1
3650690	99	60	19,36	cyclictest	1	-21	systemd	00:00:01	1
3650690	99	60	1,17	cyclictest	3740047	-21	ssh	00:15:24	1
3650697	99	59	29,25	cyclictest	3693883	-21	sh	22:32:27	2
3650697	99	59	25,30	cyclictest	3713762	-21	sh	23:16:48	2
3650697	99	59	1,54	cyclictest	3732888	-21	ssh	23:59:51	2
3650690	99	59	2,53	cyclictest	3733714	-21	ssh	00:01:19	1
3650690	99	59	2,52	cyclictest	3651138	-21	cron	19:20:00	1
3650690	99	59	1,54	cyclictest	3692152	-21	ssh	22:28:44	1
3650690	99	59	1,53	cyclictest	3746606	-21	ssh	00:30:06	1
3650690	99	59	1,18	cyclictest	3742148	-21	ssh	00:20:07	1
3650697	99	58	2,51	cyclictest	3679447	-21	cut	22:00:02	2
3650697	99	58	19,35	cyclictest	3740379	-21	sh	00:16:26	2
3650697	99	58	1,53	cyclictest	3724960	-21	ssh	23:41:42	2
3650697	99	58	1,52	cyclictest	3698622	-21	ssh	22:43:08	2
3650697	99	58	1,52	cyclictest	3668381	-21	ssh	21:35:16	2
3650690	99	58	28,26	cyclictest	3725397	-21	sh	23:42:45	1
3650690	99	58	2,52	cyclictest	3720639	-21	sh	23:32:07	1
3650690	99	58	2,52	cyclictest	3701454	-21	sh	22:49:37	1
3650690	99	58	2,52	cyclictest	3682997	-21	ssh	22:08:13	1
3650690	99	58	2,52	cyclictest	3671834	-21	ssh	21:43:17	1
3650690	99	58	1,52	cyclictest	3677688	-21	ssh	21:56:07	1
3650690	99	58	1,17	cyclictest	3656876	-21	munin-run	21:10:01	1
3650690	99	58	1,16	cyclictest	3727869	-21	ssh	23:48:20	1
3650687	99	58	26,12	cyclictest	3739859	-21	ssh	00:15:03	0
3650687	99	58	26,12	cyclictest	3739859	-21	ssh	00:15:02	0
3650697	99	57	44,7	cyclictest	3642836	-21	kworker/2:0	19:20:00	2
3650697	99	57	28,25	cyclictest	3718202	-21	sh	23:26:35	2
3650697	99	57	25,28	cyclictest	3735561	-21	sh	00:05:24	2
3650697	99	57	25,28	cyclictest	3711331	-21	sh	23:11:10	2
3650697	99	57	2,51	cyclictest	3702963	-21	ssh	22:52:40	2
3650697	99	57	2,51	cyclictest	3702963	-21	ssh	22:52:39	2
3650697	99	57	23,30	cyclictest	30	-21	ksoftirqd/2	22:37:00	2
3650697	99	57	1,52	cyclictest	3717166	-21	ssh	23:24:32	2
3650697	99	57	1,52	cyclictest	3717166	-21	ssh	23:24:31	2
3650697	99	57	1,52	cyclictest	3664099	-21	ssh	21:25:44	2
3650697	99	57	1,52	cyclictest	3659858	-21	ssh	21:16:26	2
3650697	99	57	1,51	cyclictest	3751095	-21	latency_hist	00:40:03	2
3650697	99	57	1,51	cyclictest	3684050	-21	ssh	22:10:21	2
3650697	99	57	1,51	cyclictest	3684050	-21	ssh	22:10:20	2
3650690	99	57	2,50	cyclictest	3684931	-21	ssh	22:12:34	1
3650690	99	57	2,50	cyclictest	3684931	-21	ssh	22:12:33	1
3650690	99	57	2,16	cyclictest	3738414	-21	sh	00:11:52	1
3650690	99	57	2,16	cyclictest	3738414	-21	sh	00:11:52	1
3650690	99	57	1,51	cyclictest	3722364	-21	ssh	23:35:50	1
3650690	99	57	1,51	cyclictest	3715385	-21	ssh	23:20:07	1
3650690	99	57	1,51	cyclictest	3715385	-21	ssh	23:20:07	1
3650687	99	57	2,39	cyclictest	3727703	-21	ssh	23:47:57	0
3650697	99	56	2,50	cyclictest	3744682	-21	sh	00:25:56	2
3650697	99	56	2,49	cyclictest	3657271	-21	sh	21:10:24	2
3650697	99	56	1,51	cyclictest	3729901	-21	ssh	23:52:57	2
3650697	99	56	1,51	cyclictest	3727669	-21	ssh	23:47:53	2
3650697	99	56	1,51	cyclictest	3686510	-21	ssh	22:15:55	2
3650697	99	56	1,51	cyclictest	3680788	-21	ssh	22:03:21	2
3650697	99	56	1,51	cyclictest	3676238	-21	ssh	21:53:05	2
3650690	99	56	2,50	cyclictest	3702086	-21	ssh	22:50:34	1
3650690	99	56	2,50	cyclictest	3702086	-21	ssh	22:50:34	1
3650690	99	56	2,49	cyclictest	3659967	-21	sshd	21:16:40	1
3650690	99	56	1,51	cyclictest	3695799	-21	ssh	22:36:48	1
3650690	99	56	1,50	cyclictest	3711654	-21	rm	23:11:57	1
3650690	99	56	1,18	cyclictest	3706070	-21	rm	22:59:47	1
3650687	99	56	52,2	cyclictest	0	-21	swapper/0	20:05:02	0
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2021-09-01 - 00:33