OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rackeslot4s.osadl.org (updated Wed Jan 21, 2026 00:44:05)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
2810082	99	56	53,2	cyclictest	3013800	-21	kworker/u8:1+flush-259:0	22:26:22	0
2810082	99	42	39,2	cyclictest	2888588	-21	kworker/u8:1+events_unbound	21:25:13	0
2810092	99	41	39,1	cyclictest	2893435	-21	fwupd	20:26:21	3
2810092	99	41	34,5	cyclictest	2899738	-21	diskstats	20:35:13	3
2810092	99	40	35,3	cyclictest	2857523	-21	/usr/sbin/munin	19:55:14	3
2810082	99	40	37,2	cyclictest	3075995	-21	kworker/u8:1+events_unbound	00:35:20	0
2810092	99	37	33,3	cyclictest	2868271	-21	apt-get	20:05:10	3
2810092	99	36	30,4	cyclictest	2951489	-21	base64	21:25:01	3
2810084	99	36	30,4	cyclictest	252	-21	systemd-journal	23:25:00	1
2810084	99	36	30,4	cyclictest	252	-21	systemd-journal	20:45:28	1
2810088	99	35	28,5	cyclictest	2972025	-21	latency_hist	21:45:01	2
2810082	99	35	32,2	cyclictest	3096571	-21	kworker/u8:0+events_unbound	23:50:15	0
2810092	99	34	27,5	cyclictest	509	-21	rs:main		3
2810084	99	34	28,5	cyclictest	2888588	-21	kworker/u8:1+flush-259:0	21:04:59	1
2810088	99	33	28,3	cyclictest	3013800	-21	kworker/u8:1+flush-259:0	22:30:17	2
2810082	99	33	30,2	cyclictest	3003672	-21	kworker/u8:3+events_unbound	23:20:01	0
2810082	99	33	30,2	cyclictest	2847193	-21	kworker/u8:0+events_unbound	20:10:23	0
2810082	99	33	30,2	cyclictest	2815229	-21	kworker/u8:4+events_unbound	19:30:14	0
2810092	99	32	25,5	cyclictest	3131727	-21	ntpq	00:15:21	3
2810088	99	32	26,4	cyclictest	2977652	-21	latency_hist	21:50:01	2
2810082	99	32	29,2	cyclictest	2847193	-21	kworker/u8:0+events_unbound	20:30:13	0
2810082	99	32	29,2	cyclictest	2756617	-21	kworker/u8:0+events_unbound	19:15:21	0
2810082	99	32	0,30	cyclictest	1	-21	systemd	22:19:30	0
2810084	99	31	26,3	cyclictest	2935506	-21	kworker/u8:3+flush-259:0	21:40:00	1
2810084	99	31	25,4	cyclictest	252	-21	systemd-journal	22:35:28	1
2810082	99	31	28,2	cyclictest	3075995	-21	kworker/u8:1+events_unbound	00:10:25	0
2810082	99	31	28,2	cyclictest	3075995	-21	kworker/u8:1+events_unbound	00:10:25	0
2810082	99	31	28,2	cyclictest	2888588	-21	kworker/u8:1+events_unbound	21:40:15	0
2810082	99	31	28,2	cyclictest	2868026	-21	kworker/u8:2+events_unbound	20:05:20	0
2810082	99	30	28,2	cyclictest	2868026	-21	kworker/u8:2+events_unbound	20:20:13	0
2810082	99	30	27,2	cyclictest	3013800	-21	kworker/u8:1+events_unbound	22:30:15	0
2810082	99	30	26,3	cyclictest	2988167	-21	kworker/u8:0+flush-259:0	22:10:19	0
2810092	99	29	26,2	cyclictest	2868026	-21	kworker/u8:2+flush-259:0	20:15:00	3
2810092	99	29	25,3	cyclictest	2988167	-21	kworker/u8:0+flush-259:0	22:15:20	3
2810088	99	29	22,4	cyclictest	2941083	-21	diskstats	21:15:14	2
2810082	99	29	26,2	cyclictest	2868026	-21	kworker/u8:2+events_unbound	20:26:20	0
2810082	99	29	26,2	cyclictest	2847193	-21	kworker/u8:0+events_unbound	20:15:20	0
2810082	99	29	25,3	cyclictest	2888588	-21	kworker/u8:1+flush-259:0	21:02:19	0
2810088	99	28	22,4	cyclictest	2862427	-21	turbostat	20:00:00	2
2810088	99	28	22,4	cyclictest	2841369	-21	latency_hist	19:40:01	2
2810084	99	28	24,3	cyclictest	3013800	-21	kworker/u8:1+flush-259:0	23:00:16	1
2810084	99	28	24,3	cyclictest	2799834	-21	kworker/u8:3+flush-259:0	19:25:20	1
2810082	99	28	25,2	cyclictest	2935506	-21	kworker/u8:3+flush-259:0	21:30:25	0
2810082	99	28	25,2	cyclictest	2888588	-21	kworker/u8:1+events_unbound	20:40:23	0
2810082	99	28	24,3	cyclictest	2830778	-21	kworker/u8:1+flush-259:0	20:00:14	0
2810082	99	28	24,3	cyclictest	2799834	-21	kworker/u8:3+events_unbound	19:45:16	0
2810088	99	27	23,3	cyclictest	3003673	-21	kworker/u8:4+flush-259:0	22:20:23	2
2810084	99	27	21,4	cyclictest	3096976	-21	diskstats	23:45:14	1
2810082	99	27	23,3	cyclictest	3055017	-21	kworker/u8:2+flush-259:0	23:55:25	0
2810082	99	27	23,3	cyclictest	2888588	-21	kworker/u8:1+events_unbound	20:35:02	0
2810082	99	27	23,3	cyclictest	2830778	-21	kworker/u8:1+events_unbound	19:55:14	0
2810082	99	27	20,5	cyclictest	2825640	-21	cp	19:25:00	0
2810092	99	26	20,4	cyclictest	2982784	-21	latency_hist	21:55:01	3
2810088	99	26	21,3	cyclictest	3013800	-21	kworker/u8:1+flush-259:0	22:50:01	2
2810084	99	26	20,4	cyclictest	2836210	-21	latency_hist	19:35:01	1
2810082	99	26	23,2	cyclictest	2888588	-21	kworker/u8:1+flush-259:0	21:20:26	0
2810082	99	26	22,3	cyclictest	3003672	-21	kworker/u8:3+events_unbound	23:45:21	0
2810082	99	26	22,3	cyclictest	2945833	-21	kworker/u8:0+events_unbound	21:45:22	0
2810092	99	25	22,2	cyclictest	2799834	-21	kworker/u8:3+flush-259:0	19:25:10	3
2810092	99	25	18,5	cyclictest	252	-21	systemd-journal	23:40:00	3
2810084	99	25	22,2	cyclictest	3075995	-21	kworker/u8:1+events_unbound	00:40:01	1
2810084	99	25	22,2	cyclictest	3055017	-21	kworker/u8:2+flush-259:0	23:35:10	1
2810084	99	25	21,3	cyclictest	3075995	-21	kworker/u8:1+flush-259:0	00:05:00	1
2810082	99	25	22,2	cyclictest	3122356	-21	kworker/u8:3+flush-259:0	00:30:21	0
2810082	99	25	22,2	cyclictest	3013800	-21	kworker/u8:1+events_unbound	22:50:21	0
2810082	99	25	21,3	cyclictest	2888588	-21	kworker/u8:1+events_unbound	20:45:18	0
2810082	99	25	18,5	cyclictest	2998268	-21	latency_hist	22:10:00	0
2810092	99	24	18,3	cyclictest	2987943	-21	latency_hist	22:00:01	3
2810084	99	24	21,2	cyclictest	2935506	-21	kworker/u8:3+events_unbound	21:30:15	1
2810084	99	24	20,3	cyclictest	2925529	-21	kworker/u8:2+events_unbound	21:05:19	1
2810084	99	24	20,3	cyclictest	2847193	-21	kworker/u8:0+events_unbound	20:52:15	1
2810082	99	24	21,2	cyclictest	3096571	-21	kworker/u8:0+events_unbound	00:00:26	0
2810082	99	24	20,3	cyclictest	3013800	-21	kworker/u8:1+events_unbound	22:38:31	0
2810092	99	23	18,4	cyclictest	2982808	-21	kworker/u8:2+flush-259:0	22:15:01	3
2810088	99	23	9,3	cyclictest	2882975	-21	unixbench_singl	20:15:40	2
2810088	99	23	17,4	cyclictest	509	-21	rs:main		2
2810084	99	23	20,2	cyclictest	2799834	-21	kworker/u8:3+flush-259:0	19:40:20	1
2810082	99	23	20,2	cyclictest	3003672	-21	kworker/u8:3+events_unbound	22:45:22	0
2810082	99	23	20,2	cyclictest	2904507	-21	kworker/u8:3+events_unbound	20:53:59	0
2810082	99	23	20,2	cyclictest	2830778	-21	kworker/u8:1+events_unbound	19:35:15	0
2810082	99	23	19,3	cyclictest	3055017	-21	kworker/u8:2+events_unbound	23:32:31	0
2810088	99	22	10,2	cyclictest	500	-21	systemd-logind	00:30:00	2
2810084	99	22	19,2	cyclictest	3055017	-21	kworker/u8:2+events_unbound	23:10:14	1
2810084	99	22	19,2	cyclictest	3003673	-21	kworker/u8:4+events_unbound	22:30:01	1
2810084	99	22	19,2	cyclictest	2888588	-21	kworker/u8:1+flush-259:0	20:40:16	1
2810084	99	22	18,3	cyclictest	2988167	-21	kworker/u8:0+events_unbound	22:17:29	1
2810084	99	22	17,3	cyclictest	2830778	-21	kworker/u8:1+flush-259:0	19:45:21	1
2810084	99	22	16,4	cyclictest	2825537	-21	latency_hist	19:25:01	1
2810082	99	22	19,2	cyclictest	3003672	-21	kworker/u8:3+events_unbound	23:12:43	0
2810082	99	22	19,2	cyclictest	2847193	-21	kworker/u8:0+events_unbound	20:55:13	0
2810082	99	22	18,3	cyclictest	3075995	-21	kworker/u8:1+events_unbound	00:09:30	0
2810082	99	22	17,3	cyclictest	3091652	-21	apt-get	23:40:12	0
2810092	99	21	16,4	cyclictest	3132885	-21	apt-get	00:20:12	3
2810092	99	21	16,3	cyclictest	3055017	-21	kworker/u8:2+flush-259:0	23:25:14	3
2810088	99	21	17,3	cyclictest	2815199	-21	idleruntime-cro	19:14:59	2
2810088	99	21	16,4	cyclictest	3071076	-21	apt-get	23:20:11	2
2810084	99	21	18,2	cyclictest	2810076	-21	cyclictest	21:50:00	1
2810084	99	21	17,3	cyclictest	3075995	-21	kworker/u8:1+events_unbound	00:15:15	1
2810084	99	21	17,3	cyclictest	3003672	-21	kworker/u8:3+flush-259:0	23:40:30	1
2810084	99	21	17,3	cyclictest	2935506	-21	kworker/u8:3+events_unbound	21:50:24	1
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2021-09-01 - 00:34