OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack5slot4s.osadl.org (updated Mon Jul 20, 2026 00:44:13)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
562918	2	176	0,3	sleep3	584	-21	kworker/3:2+events	19:52:25	3
829360	2	170	0,2	sleep3	0	-21	swapper/3	00:17:31	3
717080	2	66	0,5	sleep2	0	-21	swapper/2	22:27:29	2
519992	99	62	53,7	cyclictest	766042	-21	latency_hist	23:15:01	1
519989	99	58	47,8	cyclictest	586861	-21	latency_hist	20:15:00	0
519997	99	53	48,4	cyclictest	596774	-21	kworker/u8:3+flush-8:0	21:12:40	2
519997	99	53	48,3	cyclictest	617050	-21	kworker/u8:0+flush-8:0	21:57:41	2
519989	99	51	41,8	cyclictest	646861	-21	/usr/sbin/munin	21:17:40	0
519992	99	50	43,5	cyclictest	686129	-21	kworker/u8:1+flush-8:0	23:02:26	1
519989	99	50	45,4	cyclictest	701052	-21	kworker/u8:3+flush-8:0	23:07:39	0
699715	2	48	0,4	chrt	37	1	rcuc/2	22:07:30	2
519997	99	48	42,5	cyclictest	805719	-21	kworker/u8:1+flush-8:0	00:27:39	2
519992	99	48	39,6	cyclictest	576911	-21	kworker/u8:1+flush-8:0	20:37:27	1
519992	99	48	39,6	cyclictest	531725	-21	latency_hist	19:20:01	1
519997	99	46	36,8	cyclictest	681964	-21	diskstats	21:52:24	2
519989	99	46	42,3	cyclictest	370357	-21	kworker/u8:4+flush-8:0	19:27:40	0
519989	99	46	39,5	cyclictest	617247	-21	apt-get	20:47:20	0
519992	99	44	32,9	cyclictest	561201	-21	ntpq	19:47:37	1
519989	99	44	36,6	cyclictest	740051	-21	ntpq	22:47:33	0
519992	99	43	39,3	cyclictest	370357	-21	kworker/u8:4+events_unbound	19:37:24	1
519992	99	43	38,4	cyclictest	701052	-21	kworker/u8:3+flush-8:0	22:22:27	1
519992	99	43	38,4	cyclictest	701052	-21	kworker/u8:3+flush-8:0	22:22:27	1
519992	99	43	32,8	cyclictest	281	-21	systemd-journal	00:31:59	1
519992	99	43	32,8	cyclictest	281	-21	systemd-journal	00:31:59	1
519989	99	43	38,3	cyclictest	596774	-21	kworker/u8:3+flush-8:0	20:52:39	0
519989	99	43	34,7	cyclictest	612008	-21	/usr/sbin/munin	20:42:40	0
519989	99	43	34,6	cyclictest	830832	-21	latency_hist	00:20:01	0
519989	99	43	31,9	cyclictest	536702	-21	latency_hist	19:25:01	0
519997	99	42	38,3	cyclictest	776009	-21	kworker/u8:3+flush-8:0	00:07:42	2
519992	99	42	33,7	cyclictest	632047	-21	/usr/sbin/munin	21:02:34	1
519997	99	41	37,3	cyclictest	725795	-21	kworker/u8:0+events_unbound	22:37:39	2
519997	99	41	37,3	cyclictest	526989	-21	kworker/u8:2+events_unbound	19:17:38	2
519992	99	41	33,6	cyclictest	281	-21	systemd-journal	23:17:39	1
519992	99	41	31,7	cyclictest	581905	-21	latency_hist	20:10:01	1
519997	99	40	35,3	cyclictest	755932	-21	kworker/u8:2+events_unbound	23:25:03	2
519992	99	40	35,3	cyclictest	370357	-21	kworker/u8:4+flush-8:0	19:52:23	1
519992	99	40	34,4	cyclictest	596774	-21	kworker/u8:3+flush-8:0	21:07:44	1
519989	99	40	36,3	cyclictest	686129	-21	kworker/u8:1+flush-8:0	21:57:26	0
519989	99	40	32,6	cyclictest	766677	-21	/usr/sbin/munin	23:17:38	0
519992	99	39	30,7	cyclictest	745770	-21	apt-get	22:55:01	1
519989	99	39	35,3	cyclictest	576911	-21	kworker/u8:1+flush-8:0	20:37:39	0
519989	99	39	35,3	cyclictest	546772	-21	kworker/u8:0+flush-8:0	20:02:40	0
519989	99	39	35,3	cyclictest	370357	-21	kworker/u8:4+flush-8:0	19:57:38	0
519989	99	39	1,34	cyclictest	745814	-21	munin-run	22:55:01	0
519997	99	38	31,5	cyclictest	760850	-21	apt-get	23:10:00	2
519992	99	38	31,6	cyclictest	805719	-21	kworker/u8:1+events_unbound	00:12:41	1
519992	99	38	28,8	cyclictest	846079	-21	/usr/sbin/munin	00:37:33	1
519992	99	38	28,7	cyclictest	810847	-21	latency_hist	00:00:01	1
519989	99	38	34,3	cyclictest	617050	-21	kworker/u8:0+flush-8:0	22:07:43	0
519989	99	38	33,3	cyclictest	805719	-21	kworker/u8:1+flush-8:0	00:12:39	0
519989	99	37	31,5	cyclictest	701052	-21	kworker/u8:3+flush-8:0	22:12:37	0
519989	99	37	31,4	cyclictest	686129	-21	kworker/u8:1+events_unbound	22:32:37	0
519159	2	37	3,28	sleep2	0	-21	swapper/2	19:07:39	2
519997	99	36	31,4	cyclictest	546772	-21	kworker/u8:0+flush-8:0	20:27:29	2
519992	99	36	31,4	cyclictest	686129	-21	kworker/u8:1+flush-8:0	22:57:38	1
515252	2	36	3,9	sleep1	0	-21	swapper/1	19:06:22	1
519992	99	35	30,4	cyclictest	576911	-21	kworker/u8:1+flush-8:0	20:32:26	1
519989	99	35	30,4	cyclictest	596774	-21	kworker/u8:3+events_unbound	21:02:42	0
519989	99	35	28,6	cyclictest	617050	-21	kworker/u8:0+events_unbound	22:02:35	0
519989	99	35	28,5	cyclictest	805119	-21	ntpq	23:52:34	0
519989	99	35	24,9	cyclictest	592473	-21	diskstats	20:22:25	0
519992	99	34	29,4	cyclictest	617050	-21	kworker/u8:0+flush-8:0	21:27:10	1
519992	99	34	27,5	cyclictest	596774	-21	kworker/u8:3+flush-8:0	21:16:10	1
519997	99	33	26,5	cyclictest	681058	-21	apt-get	21:50:00	2
519992	99	33	27,4	cyclictest	370357	-21	kworker/u8:4+events_unbound	19:42:29	1
519992	99	33	25,6	cyclictest	536706	-21	apt-get	19:25:00	1
519989	99	33	26,5	cyclictest	686129	-21	kworker/u8:1+flush-8:0	22:17:30	0
519989	99	33	24,6	cyclictest	800960	-21	latency_hist	23:50:01	0
519989	99	33	22,9	cyclictest	556749	-21	apt-get	19:45:01	0
519997	99	32	24,6	cyclictest	572215	-21	cp	20:00:01	2
519992	99	32	28,3	cyclictest	776009	-21	kworker/u8:3+flush-8:0	00:22:26	1
519992	99	32	28,3	cyclictest	776009	-21	kworker/u8:3+events_unbound	23:55:01	1
519992	99	32	27,4	cyclictest	805719	-21	kworker/u8:1+flush-8:0	00:05:05	1
519992	99	32	26,5	cyclictest	576911	-21	kworker/u8:1+flush-8:0	20:12:43	1
519992	99	32	26,4	cyclictest	686129	-21	kworker/u8:1+flush-8:0	22:17:33	1
519992	99	32	26,4	cyclictest	370357	-21	kworker/u8:4+flush-8:0	19:57:25	1
519992	99	32	25,5	cyclictest	611625	-21	kworker/u8:2+events_unbound	21:52:39	1
519989	99	32	28,3	cyclictest	576911	-21	kworker/u8:1+flush-8:0	20:32:39	0
519989	99	32	27,4	cyclictest	755932	-21	kworker/u8:2+flush-8:0	23:22:36	0
519989	99	32	27,4	cyclictest	370357	-21	kworker/u8:4+flush-8:0	19:54:04	0
519989	99	32	27,3	cyclictest	370357	-21	kworker/u8:4+events_unbound	19:15:02	0
519989	99	32	26,4	cyclictest	796004	-21	latency_hist	23:45:01	0
519989	99	32	26,4	cyclictest	596774	-21	kworker/u8:3+events_unbound	21:07:42	0
519989	99	32	25,5	cyclictest	576911	-21	kworker/u8:1+events_unbound	20:17:21	0
519989	99	32	24,6	cyclictest	641922	-21	/usr/sbin/munin	21:12:41	0
519992	99	31	26,3	cyclictest	732108	-21	irqrtprio	22:42:28	1
519992	99	31	26,3	cyclictest	681457	-21	latency_hist	21:50:01	1
519992	99	31	25,4	cyclictest	830566	-21	unixbench_multi	00:17:41	1
519989	99	31	27,3	cyclictest	576911	-21	kworker/u8:1+flush-8:0	20:07:30	0
519989	99	31	26,4	cyclictest	611625	-21	kworker/u8:2+flush-8:0	21:42:27	0
519989	99	31	24,5	cyclictest	671180	-21	cron	21:40:00	0
519989	99	31	22,7	cyclictest	815029	-21	ntpq	00:02:33	0
519997	99	30	24,4	cyclictest	766145	-21	gpgv	23:15:00	2
519992	99	30	25,4	cyclictest	755932	-21	kworker/u8:2+flush-8:0	23:27:31	1
519989	99	30	26,3	cyclictest	546772	-21	kworker/u8:0+flush-8:0	19:47:40	0
519989	99	30	25,4	cyclictest	686129	-21	kworker/u8:1+events_unbound	22:57:40	0
519989	99	30	24,5	cyclictest	611625	-21	kworker/u8:2+events_unbound	20:57:37	0
519989	99	30	21,7	cyclictest	596774	-21	kworker/u8:3+flush-8:0	21:22:40	0
519989	99	30	21,7	cyclictest	596774	-21	kworker/u8:3+flush-8:0	21:22:40	0
519997	99	29	24,3	cyclictest	725795	-21	kworker/u8:0+events_unbound	22:45:07	2
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17