OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 792 highest latencies:

System rack2slot7s.osadl.org (updated Thu Mar 19, 2026 07:54:54)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
3525486	99	87	79,4	cyclictest	0	-21	swapper/2	05:50:45	2
3525486	99	87	79,4	cyclictest	0	-21	swapper/2	05:50:45	2
3525485	99	72	63,6	cyclictest	3762664	-21	sort	05:40:39	1
3525484	99	71	64,4	cyclictest	147673	-21	multi.sh	06:10:43	0
3525484	99	69	58,6	cyclictest	3761090	-21	sh	05:40:34	0
3525484	99	66	59,4	cyclictest	3832321	-21	execl	05:50:45	0
3525484	99	66	59,4	cyclictest	3832321	-21	execl	05:50:45	0
3525485	99	64	55,6	cyclictest	147420	-21	rm	06:10:42	1
3525485	99	60	30,6	cyclictest	74185	-21	sh	06:07:32	1
3525485	99	59	36,19	cyclictest	3832318	-21	execl	05:50:53	1
3525485	99	59	36,19	cyclictest	3832318	-21	execl	05:50:53	1
3525486	99	58	48,6	cyclictest	3761232	-21	sh	05:40:34	2
3525485	99	56	29,22	cyclictest	0	-21	swapper/1	05:50:01	1
3525486	99	55	42,9	cyclictest	147068	-21	sh	06:10:41	2
3525484	99	54	37,14	cyclictest	3831321	-21	execl	05:50:01	0
3525484	99	53	34,13	cyclictest	124	50	irq/32-eth0	05:05:37	0
3525486	99	52	43,5	cyclictest	130910	-21	grep	06:10:01	2
3525485	99	49	5,16	cyclictest	577	-21	mta-sts-daemon	06:15:40	1
3525484	99	49	41,5	cyclictest	123608	-21	multi.sh	06:09:40	0
3525486	99	48	39,6	cyclictest	3753897	-21	systemctl	05:40:00	2
3525485	99	46	39,3	cyclictest	0	-21	swapper/1	05:35:37	1
3525484	99	46	35,7	cyclictest	15	1	ktimers/0	06:35:37	0
3525484	99	46	28,15	cyclictest	3754363	-21	latency_hist	05:40:02	0
3525484	99	43	32,8	cyclictest	3530445	-21	execl	05:20:28	0
3525484	99	43	26,14	cyclictest	3834609	-21	Run	05:59:37	0
3525486	99	42	32,6	cyclictest	3831276	-21	Run	05:48:51	2
3525485	99	42	33,6	cyclictest	3718282	-21	unin-run	05:35:01	1
3525485	99	42	33,6	cyclictest	3718282	-21	unin-run	05:35:01	1
3525486	99	41	18,6	cyclictest	0	-21	swapper/2	06:00:41	2
3525484	99	41	31,6	cyclictest	15	1	ktimers/0	06:00:34	0
3525486	99	40	32,5	cyclictest	3834498	-21	fstime	05:58:28	2
3525485	99	40	31,5	cyclictest	3995605	-21	spawn	06:03:32	1
3525484	99	40	27,8	cyclictest	3534233	-21	ntpq	05:30:33	0
3525484	99	40	27,8	cyclictest	3534233	-21	ntpq	05:30:33	0
3525484	99	40	20,4	cyclictest	205647	-21	ntpq	06:25:39	0
3525485	99	39	31,4	cyclictest	0	-21	swapper/1	05:25:36	1
3525484	99	39	29,6	cyclictest	202536	-21	ntpq	06:15:36	0
3525485	99	38	30,5	cyclictest	207852	-21	latency_hist	06:35:02	1
3525485	99	38	29,5	cyclictest	3528457	-21	cron	05:15:00	1
3525484	99	38	17,15	cyclictest	0	-21	swapper/0	05:10:39	0
3525486	99	37	4,25	cyclictest	3530445	-21	execl	05:20:34	2
3525484	99	37	24,9	cyclictest	204048	-21	ntpq	06:20:33	0
3525484	99	37	18,15	cyclictest	207083	-21	ntpq	06:30:36	0
3525486	99	36	28,5	cyclictest	3719179	-21	latency_hist	05:35:02	2
3525486	99	36	28,5	cyclictest	3719179	-21	latency_hist	05:35:02	2
3525486	99	36	25,8	cyclictest	203958	-21	needreboot	06:20:29	2
3525485	99	36	1,28	cyclictest	3530445	-21	execl	05:20:34	1
3525484	99	36	23,9	cyclictest	1	-21	systemd	05:05:01	0
3525486	99	34	6,5	cyclictest	17	1	rcu_preempt	05:15:32	2
3525484	99	34	17,11	cyclictest	3532001	-21	fstime	05:25:33	0
3525486	99	33	26,4	cyclictest	208217	-21	fschecks_count	06:35:28	2
3525486	99	32	24,5	cyclictest	3525506	-21	latency_hist	05:05:01	2
3525486	99	32	2,22	cyclictest	207044	-21	sshd	06:30:34	2
3525486	99	31	25,3	cyclictest	0	-21	swapper/2	05:15:00	2
3525486	99	30	24,3	cyclictest	3524375	-21	kworker/u8:6+rpciod	05:10:01	2
3525485	99	30	5,16	cyclictest	577	-21	mta-sts-daemon	06:20:35	1
3525485	99	30	23,3	cyclictest	0	-21	swapper/1	05:15:38	1
3525484	99	30	15,8	cyclictest	3529438	-21	ntpq	05:15:33	0
3525485	99	29	20,6	cyclictest	3406575	-21	cups-browsed	06:35:41	1
3525485	99	29	20,4	cyclictest	0	-21	swapper/1	05:05:36	1
3525486	99	28	19,6	cyclictest	1165	-21	mta-sts-daemon	06:25:27	2
3525485	99	28	2,19	cyclictest	1319	-21	sendmail-mta	05:56:16	1
3525486	99	27	20,3	cyclictest	0	-21	swapper/2	05:25:25	2
3525486	99	24	2,17	cyclictest	202608	-21	ntpq	06:15:40	2
3525485	99	23	3,13	cyclictest	588	-21	gpm	06:25:33	1
3525487	99	19	3,8	cyclictest	0	-21	swapper/3	05:50:38	3
3525487	99	19	3,8	cyclictest	0	-21	swapper/3	05:50:38	3
3525487	99	19	3,7	cyclictest	0	-21	swapper/3	06:05:33	3
3525485	99	19	15,3	cyclictest	3520130	-21	kworker/u8:16-rpciod	05:05:01	1
3525487	99	18	4,9	cyclictest	0	-21	swapper/3	05:57:05	3
3525487	99	17	5,7	cyclictest	0	-21	swapper/3	06:10:40	3
3525487	99	17	2,9	cyclictest	0	-21	swapper/3	05:40:33	3
3525487	99	17	1,6	cyclictest	0	-21	swapper/3	06:25:32	3
3525487	99	16	4,5	cyclictest	0	-21	swapper/3	06:00:34	3
3525487	99	15	2,6	cyclictest	0	-21	swapper/3	05:35:41	3
3525487	99	15	1,9	cyclictest	0	-21	swapper/3	05:25:36	3
3525487	99	15	1,6	cyclictest	0	-21	swapper/3	05:15:34	3
3525487	99	14	2,5	cyclictest	0	-21	swapper/3	06:20:32	3
3525487	99	14	1,6	cyclictest	0	-21	swapper/3	05:10:37	3
3525487	99	13	2,7	cyclictest	0	-21	swapper/3	05:45:38	3
3525487	99	13	2,6	cyclictest	0	-21	swapper/3	05:20:35	3
3525487	99	13	2,5	cyclictest	0	-21	swapper/3	06:30:34	3
3525487	99	12	2,6	cyclictest	0	-21	swapper/3	06:35:33	3
3525487	99	11	1,6	cyclictest	0	-21	swapper/3	05:30:33	3
3525487	99	11	1,6	cyclictest	0	-21	swapper/3	05:30:33	3
3525487	99	11	1,6	cyclictest	0	-21	swapper/3	05:05:36	3
3525487	99	9	1,5	cyclictest	0	-21	swapper/3	06:15:25	3
3525487	99	7	1,3	cyclictest	0	-21	swapper/3	05:05:01	3
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17