OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rackeslot4s.osadl.org (updated Thu Aug 20, 2026 00:44:04)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
2818506	99	57	51,4	cyclictest	2886654	-21	cat	20:15:01	0
2818509	99	49	43,4	cyclictest	2886144	-21	apt-get	20:15:00	1
2818506	99	49	45,3	cyclictest	2985513	-21	latency_hist	21:50:00	0
2818506	99	48	44,3	cyclictest	3137702	-21	apt-get	00:15:11	0
2818509	99	47	41,4	cyclictest	335557	-21	systemd-journal	19:35:22	1
2818512	99	45	41,3	cyclictest	3059540	-21	kworker/u8:1+flush-259:0	00:35:00	2
2818506	99	43	38,3	cyclictest	3049452	-21	apt-get	22:50:12	0
2818506	99	42	28,12	cyclictest	2881227	-21	apt-get	20:10:13	0
2818514	99	41	38,2	cyclictest	2891452	-21	kworker/u8:1+flush-259:0	20:55:25	3
2818514	99	41	36,4	cyclictest	3033305	-21	gpgv	22:35:01	3
2818509	99	41	34,5	cyclictest	335557	-21	systemd-journal	23:09:28	1
2818512	99	40	33,5	cyclictest	335557	-21	systemd-journal	20:15:00	2
2818512	99	39	34,3	cyclictest	509	-21	rs:main		2
2818512	99	38	33,4	cyclictest	2870631	-21	kworker/u8:3+flush-259:0	20:34:59	2
2818512	99	37	31,4	cyclictest	3070041	-21	/usr/sbin/munin	23:10:13	2
2818512	99	36	32,3	cyclictest	2870631	-21	kworker/u8:3+flush-259:0	20:30:00	2
2818512	99	36	32,2	cyclictest	2938122	-21	kworker/u8:0+events_unbound	21:05:18	2
2818509	99	36	31,3	cyclictest	335557	-21	systemd-journal	19:16:28	1
2818514	99	35	29,4	cyclictest	2996287	-21	latency_hist	22:00:00	3
2818512	99	35	28,5	cyclictest	2963947	-21	ntpq	21:25:23	2
2818509	99	35	30,4	cyclictest	2812961	-21	kworker/u8:4+flush-259:0	19:40:15	1
2818514	99	34	30,3	cyclictest	3043727	-21	gpgv	22:45:00	3
2818512	99	34	30,3	cyclictest	3106171	-21	/usr/sbin/munin	23:45:14	2
2818506	99	34	30,3	cyclictest	3048176	-21	kworker/u8:2+flush-259:0	23:15:15	0
2818506	99	34	27,5	cyclictest	2849832	-21	latency_hist	19:40:01	0
2818514	99	33	29,3	cyclictest	2839145	-21	kworker/u8:2+events_unbound	20:00:01	3
2818514	99	33	26,5	cyclictest	2907180	-21	latency_hist	20:35:01	3
2818509	99	33	27,4	cyclictest	335557	-21	systemd-journal	23:05:00	1
2818506	99	33	27,4	cyclictest	3153249	-21	apt-get	00:30:13	0
2818506	99	32	27,3	cyclictest	2975396	-21	apt-get	21:40:11	0
2818514	99	31	27,3	cyclictest	2849859	-21	kworker/u8:1+flush-259:0	20:09:59	3
2818514	99	31	26,3	cyclictest	2849453	-21	kworker/u8:0+events_unbound	20:15:18	3
2818512	99	31	26,4	cyclictest	2818447	-21	kworker/u8:1+flush-259:0	19:30:00	2
2818506	99	31	25,4	cyclictest	2958727	-21	ntpq	21:20:22	0
2818514	99	30	27,2	cyclictest	2812961	-21	kworker/u8:4+events_unbound	19:25:00	3
2818514	99	30	26,3	cyclictest	3142654	-21	kworker/u8:4+flush-259:0	00:25:31	3
2818514	99	30	25,4	cyclictest	3011790	-21	kworker/u8:1+events_unbound	22:39:28	3
2818509	99	30	26,3	cyclictest	3038694	-21	apt-get	22:40:09	1
2818506	99	30	26,3	cyclictest	3048176	-21	kworker/u8:2+flush-259:0	23:00:16	0
2818514	99	29	25,3	cyclictest	3059540	-21	kworker/u8:1+flush-259:0	23:20:23	3
2818512	99	29	23,4	cyclictest	3054788	-21	diskstats	22:55:14	2
2818506	99	29	23,5	cyclictest	3142654	-21	kworker/u8:4+flush-259:0	00:30:00	0
2818506	99	29	23,4	cyclictest	3033544	-21	/usr/sbin/munin	22:35:15	0
2818514	99	28	24,3	cyclictest	3059540	-21	kworker/u8:1+events_unbound	00:13:32	3
2818514	99	28	24,3	cyclictest	2948876	-21	kworker/u8:2+flush-259:0	21:35:00	3
2818514	99	28	24,3	cyclictest	2907207	-21	kworker/u8:2+events_unbound	20:36:28	3
2818514	99	28	22,5	cyclictest	2870878	-21	apt-get	20:00:14	3
2818512	99	28	22,4	cyclictest	3048176	-21	kworker/u8:2+flush-259:0	23:15:16	2
2818512	99	28	22,4	cyclictest	3027944	-21	/usr/sbin/munin	22:30:23	2
2818509	99	28	22,4	cyclictest	2907445	-21	apt-get	20:35:11	1
2818506	99	28	24,3	cyclictest	3121524	-21	kworker/u8:3+flush-259:0	00:25:00	0
2818506	99	28	22,4	cyclictest	3090735	-21	apt-get	23:30:08	0
2818514	99	27	24,2	cyclictest	3059540	-21	kworker/u8:1+flush-259:0	23:05:26	3
2818514	99	27	24,2	cyclictest	3048176	-21	kworker/u8:2+events_unbound	23:15:18	3
2818514	99	27	23,3	cyclictest	3059540	-21	kworker/u8:1+events_unbound	23:59:29	3
2818514	99	27	23,3	cyclictest	3048176	-21	kworker/u8:2+flush-259:0	23:40:00	3
2818514	99	27	23,3	cyclictest	2849453	-21	kworker/u8:0+events_unbound	20:40:24	3
2818512	99	27	23,3	cyclictest	3048176	-21	kworker/u8:2+flush-259:0	23:30:20	2
2818512	99	27	21,5	cyclictest	3027701	-21	latency_hist	22:30:00	2
2818512	99	27	21,4	cyclictest	3148111	-21	apt-get	00:25:11	2
2818512	99	27	20,5	cyclictest	3059540	-21	kworker/u8:1+flush-259:0	23:05:17	2
2818509	99	27	20,5	cyclictest	2886325	-21	cp	20:15:01	1
2818514	99	26	22,3	cyclictest	2990611	-21	kworker/u8:3+flush-259:0	22:05:48	3
2818514	99	26	22,3	cyclictest	2990611	-21	kworker/u8:3+events_unbound	22:00:48	3
2818514	99	26	22,3	cyclictest	2927838	-21	kworker/u8:3+events_unbound	21:00:25	3
2818514	99	26	22,3	cyclictest	2828825	-21	sh	19:20:00	3
2818514	99	26	21,4	cyclictest	2948876	-21	kworker/u8:2+events_unbound	21:15:39	3
2818514	99	26	21,4	cyclictest	2927838	-21	kworker/u8:3+flush-259:0	21:10:17	3
2818514	99	26	20,4	cyclictest	3044128	-21	cat	22:45:01	3
2818512	99	26	21,3	cyclictest	3111360	-21	apt-get	23:50:12	2
2818512	99	26	20,4	cyclictest	2880968	-21	latency_hist	20:10:01	2
2818512	99	26	18,6	cyclictest	3043997	-21	cp	22:45:01	2
2818509	99	26	21,3	cyclictest	2829087	-21	apt-get	19:20:11	1
2818506	99	26	20,4	cyclictest	3043612	-21	latency_hist	22:45:01	0
2818506	99	26	19,5	cyclictest	3105928	-21	latency_hist	23:45:00	0
2818514	99	25	22,2	cyclictest	3048176	-21	kworker/u8:2+flush-259:0	23:45:23	3
2818514	99	25	22,2	cyclictest	2870631	-21	kworker/u8:3+flush-259:0	20:25:24	3
2818514	99	25	21,3	cyclictest	2970162	-21	kworker/u8:1+events_unbound	21:39:28	3
2818514	99	25	21,3	cyclictest	2891452	-21	kworker/u8:1+events_unbound	20:53:28	3
2818514	99	25	20,3	cyclictest	2996309	-21	kworker/u8:0+events_unbound	22:15:17	3
2818512	99	25	21,3	cyclictest	2849453	-21	kworker/u8:0+flush-259:0	20:20:25	2
2818512	99	25	21,3	cyclictest	2812961	-21	kworker/u8:4+events_unbound	19:30:23	2
2818512	99	25	19,4	cyclictest	335557	-21	systemd-journal	23:37:28	2
2818509	99	25	22,2	cyclictest	2902061	-21	idleruntime-cro	20:30:00	1
2818509	99	25	20,4	cyclictest	2985625	-21	gpgv	21:50:00	1
2818509	99	25	20,3	cyclictest	3059540	-21	kworker/u8:1+flush-259:0	23:20:15	1
2818506	99	25	20,3	cyclictest	2991374	-21	apt-get	21:55:13	0
2818514	99	24	20,3	cyclictest	2927838	-21	kworker/u8:3+events_unbound	21:26:28	3
2818514	99	24	20,3	cyclictest	2927838	-21	kworker/u8:3+events_unbound	21:05:26	3
2818514	99	24	19,3	cyclictest	2980354	-21	kworker/u8:0+events_unbound	21:50:23	3
2818514	99	24	18,4	cyclictest	3059540	-21	kworker/u8:1+flush-259:0	00:15:20	3
2818512	99	24	18,4	cyclictest	2995378	-21	ntpq	21:55:22	2
2818509	99	24	20,3	cyclictest	335557	-21	systemd-journal	21:00:34	1
2818509	99	24	19,4	cyclictest	3017126	-21	gpgv	22:20:00	1
2818506	99	24	17,4	cyclictest	335557	-21	systemd-journal	23:05:27	0
2818514	99	23	20,2	cyclictest	3048176	-21	kworker/u8:2+events_unbound	00:01:31	3
2818512	99	23	19,3	cyclictest	2948876	-21	kworker/u8:2+flush-259:0	21:45:00	2
2818509	99	23	16,5	cyclictest	2943701	-21	latency_hist	21:10:01	1
2818506	99	23	19,3	cyclictest	2996309	-21	kworker/u8:0+flush-259:0	22:30:17	0
2818506	99	23	18,3	cyclictest	3126987	-21	apt-get	00:05:13	0
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2021-09-01 - 00:34