OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #0, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot4s.osadl.org (updated Fri Aug 21, 2026 00:45:13)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
2749250	2	61	59,1	sleep0	0	-21	swapper/0	19:08:10	0
2749075	2	60	58,1	sleep3	0	-21	swapper/3	19:06:05	9
2749826	99	54	51,2	cyclictest	2998143	-21	kworker/u32:1+flush-259:0	23:10:18	3
2749795	99	54	53,1	cyclictest	3087420	-21	kworker/u32:2+events_unbound	00:25:21	1
2749113	2	54	31,2	sleep4	0	-21	swapper/4	19:06:38	10
4189	98	52	0,50	rtkit-daemon	0	-21	swapper/7	19:08:00	13
2749193	2	52	50,1	sleep6	0	-21	swapper/6	19:07:28	12
2749836	99	51	12,38	cyclictest	2972320	-21	ssh	22:43:58	6
2749832	99	48	46,2	cyclictest	2783893	-21	kworker/u32:3+events_unbound	20:08:26	5
2749815	99	47	45,2	cyclictest	2743640	-21	kworker/u32:2+flush-259:0	19:15:15	13
2749815	99	47	44,2	cyclictest	2982010	-21	kworker/u32:2+flush-259:0	23:05:12	13
2749838	99	46	46,0	cyclictest	0	-21	swapper/15	19:29:59	7
2749815	99	46	44,1	cyclictest	2906748	-21	kworker/u32:2+flush-259:0	22:15:15	13
2749167	2	45	42,2	sleep5	0	-21	swapper/5	19:07:11	11
2749832	99	44	43,1	cyclictest	3004673	-21	kworker/u32:0+events_unbound	00:15:20	5
2749815	99	44	43,1	cyclictest	2779589	-21	kworker/u32:0+flush-259:0	19:50:19	13
2749815	99	44	37,6	cyclictest	3087420	-21	kworker/u32:2+flush-259:0	00:21:38	13
2749810	99	44	41,2	cyclictest	3079933	-21	kworker/u32:3+flush-259:0	00:32:38	12
2749817	99	43	41,2	cyclictest	2796357	-21	kworker/u32:2+events_unbound	21:24:58	14
2749790	99	43	42,1	cyclictest	2958724	-21	kworker/u32:4+events_unbound	22:37:19	0
2749790	99	43	42,1	cyclictest	2958724	-21	kworker/u32:4+events_unbound	22:37:18	0
2749830	99	42	38,3	cyclictest	3004673	-21	kworker/u32:0+flush-259:0	00:10:11	4
2749282	2	42	38,2	sleep14	0	-21	swapper/14	19:08:40	6
2745901	2	41	39,1	sleep13	0	-21	swapper/13	19:05:12	5
2749817	99	40	37,3	cyclictest	3004673	-21	kworker/u32:0+flush-259:0	23:14:26	14
2749088	2	40	38,1	sleep15	0	-21	swapper/15	19:06:17	7
2749830	99	39	36,2	cyclictest	3099824	-21	kworker/u32:4+flush-259:0	00:35:15	4
2749807	99	39	35,3	cyclictest	2982010	-21	kworker/u32:2+flush-259:0	23:20:40	11
2749119	2	39	37,1	sleep9	0	-21	swapper/9	19:06:44	15
2749063	2	39	37,1	sleep11	0	-21	swapper/11	19:05:58	3
2749832	99	38	38,0	cyclictest	0	-21	swapper/13	21:45:52	5
2749810	99	38	36,2	cyclictest	3020445	-21	kworker/u32:1+events_unbound	00:00:25	12
2749838	99	37	35,2	cyclictest	2783893	-21	kworker/u32:3+flush-259:0	20:15:22	7
2749836	99	37	10,14	cyclictest	1161449	-21	thunderbird	21:16:06	6
2749832	99	37	37,0	cyclictest	0	-21	swapper/13	20:29:05	5
2749832	99	37	36,1	cyclictest	2900330	-21	kworker/u32:0+events_unbound	22:16:00	5
2749815	99	37	35,2	cyclictest	2808928	-21	kworker/u32:4+events_unbound	20:31:15	13
2749815	99	37	35,2	cyclictest	2747867	-21	kworker/u32:4+flush-259:0	19:32:18	13
2749810	99	37	34,2	cyclictest	2808928	-21	kworker/u32:4+events_unbound	20:15:31	12
2749805	99	37	35,1	cyclictest	2861690	-21	kworker/u32:4+events_unbound	21:16:10	10
2749800	99	37	36,0	cyclictest	0	-21	swapper/3	22:37:05	9
2749800	99	37	36,0	cyclictest	0	-21	swapper/3	22:37:04	9
2749361	2	37	34,2	sleep12	0	-21	swapper/12	19:09:43	4
2749832	99	36	35,1	cyclictest	3099824	-21	kworker/u32:4+flush-259:0	00:23:18	5
2749832	99	36	32,4	cyclictest	3578	-21	gnome-shell	22:01:56	5
2749820	99	36	35,1	cyclictest	2990696	-21	kworker/u32:0+events_unbound	23:04:04	15
2749815	99	36	36,0	cyclictest	0	-21	swapper/7	21:45:53	13
2749800	99	36	34,2	cyclictest	2732516	-21	kworker/u32:0+flush-259:0	19:32:08	9
2749797	99	36	35,1	cyclictest	2990696	-21	kworker/u32:0+events_unbound	23:04:11	8
2749790	99	36	34,2	cyclictest	2958724	-21	kworker/u32:4+flush-259:0	22:45:21	0
2749350	2	36	34,1	sleep1	0	-21	swapper/1	19:09:31	1
2749836	99	35	0,9	cyclictest	0	-21	swapper/14	22:00:03	6
2749836	99	35	0,10	cyclictest	0	-21	swapper/14	22:22:01	6
2749836	99	35	0,10	cyclictest	0	-21	swapper/14	19:26:16	6
2749815	99	35	34,1	cyclictest	3020445	-21	kworker/u32:1+flush-259:0	00:10:16	13
2749815	99	35	28,5	cyclictest	2900083	-21	kworker/u32:4+events_unbound	22:25:04	13
2749815	99	35	28,5	cyclictest	2900083	-21	kworker/u32:4+events_unbound	22:25:03	13
2749805	99	35	33,2	cyclictest	2871176	-21	kworker/u32:5+events_unbound	21:29:45	10
2749800	99	35	33,2	cyclictest	2796357	-21	kworker/u32:2+events_unbound	21:33:54	9
2749276	2	35	33,1	sleep8	0	-21	swapper/8	19:08:35	14
2749836	99	34	9,25	cyclictest	1161540	-21	WRScene~ilder#1	19:37:15	6
2749836	99	34	9,25	cyclictest	1161540	-21	WRScene~ilder#1	19:37:15	6
2749836	99	34	8,25	cyclictest	0	-21	swapper/14	20:32:10	6
2749836	99	34	0,9	cyclictest	0	-21	swapper/14	23:05:56	6
2749836	99	34	0,9	cyclictest	0	-21	swapper/14	22:54:58	6
2749836	99	34	0,9	cyclictest	0	-21	swapper/14	21:27:05	6
2749836	99	34	0,9	cyclictest	0	-21	swapper/14	20:43:09	6
2749830	99	34	34,0	cyclictest	3020445	-21	kworker/u32:1+events_unbound	23:51:33	4
2749830	99	34	30,3	cyclictest	2900083	-21	kworker/u32:4+flush-259:0	22:06:43	4
2749815	99	34	33,0	cyclictest	0	-21	swapper/7	23:50:14	13
2749215	2	34	31,2	sleep10	0	-21	swapper/10	19:07:48	2
2749138	2	34	32,1	sleep2	0	-21	swapper/2	19:06:52	8
2749836	99	33	29,4	cyclictest	1774	-21	snmpd	23:19:47	6
2749836	99	33	27,5	cyclictest	1774	-21	snmpd	20:23:36	6
2749832	99	33	32,1	cyclictest	2796357	-21	kworker/u32:2+events_unbound	21:14:58	5
2749832	99	33	32,1	cyclictest	2762899	-21	kworker/u32:1+flush-259:0	19:35:17	5
2749832	99	33	32,1	cyclictest	2762899	-21	kworker/u32:1+flush-259:0	19:35:17	5
2749832	99	33	31,2	cyclictest	2861690	-21	kworker/u32:4+flush-259:0	21:19:59	5
2749832	99	33	31,2	cyclictest	2775365	-21	kworker/u32:2+flush-259:0	19:45:17	5
2749832	99	33	31,2	cyclictest	2775365	-21	kworker/u32:2+flush-259:0	19:45:16	5
2749830	99	33	29,4	cyclictest	1774	-21	snmpd	20:19:25	4
2749826	99	33	29,3	cyclictest	3020445	-21	kworker/u32:1+events_unbound	23:27:32	3
2749822	99	33	32,1	cyclictest	2938776	-21	kworker/u32:1+flush-259:0	22:26:48	2
2749822	99	33	32,1	cyclictest	2938776	-21	kworker/u32:1+flush-259:0	22:26:47	2
2749817	99	33	1,8	cyclictest	1161449	-21	thunderbird	23:26:48	14
2749817	99	33	0,30	cyclictest	0	-21	swapper/8	20:30:06	14
2749810	99	33	30,2	cyclictest	2783893	-21	kworker/u32:3+flush-259:0	19:52:37	12
2749807	99	33	31,2	cyclictest	3004860	-21	latency_hist	23:10:00	11
2749805	99	33	33,0	cyclictest	0	-21	swapper/4	21:59:22	10
2749797	99	33	31,2	cyclictest	2747867	-21	kworker/u32:4+flush-259:0	19:26:50	8
2749797	99	33	0,33	cyclictest	782	-21	systemd-journal	21:57:48	8
2749797	99	33	0,31	cyclictest	3055491	-21	ksmtuned	23:48:58	8
2749797	99	33	0,30	cyclictest	0	-21	swapper/2	21:08:26	8
2749790	99	33	31,2	cyclictest	3020158	-21	iostat_ios	23:20:14	0
2749790	99	33	1,32	cyclictest	1161449	-21	thunderbird	00:34:15	0
2749838	99	32	30,2	cyclictest	3020445	-21	kworker/u32:1+flush-259:0	00:00:17	7
2749836	99	32	27,1	cyclictest	1774	-21	snmpd	22:46:31	6
2749830	99	32	31,1	cyclictest	3087420	-21	kworker/u32:2+flush-259:0	00:25:15	4
2749830	99	32	28,4	cyclictest	1774	-21	snmpd	21:19:03	4
2749817	99	32	2,1	cyclictest	2683944	-1	kworker/u33:2+i915_flip	20:44:22	14
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17