OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rackeslot4s.osadl.org (updated Sat Jul 11, 2026 12:44:07)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
732809	99	41	35,5	cyclictest	335557	-21	systemd-journal	09:10:01	3
732809	99	41	35,4	cyclictest	335557	-21	systemd-journal	07:30:00	3
732804	99	33	30,2	cyclictest	743289	-21	kworker/u8:4+events_unbound	07:36:15	2
732809	99	32	26,4	cyclictest	1040005	-21	latency_hist	12:05:01	3
732804	99	31	28,2	cyclictest	780263	-21	kworker/u8:3+flush-259:0	08:00:21	2
732804	99	31	28,2	cyclictest	780263	-21	kworker/u8:3+flush-259:0	08:00:20	2
732801	99	30	26,3	cyclictest	831585	-21	kworker/u8:0+events_unbound	09:20:02	1
732799	99	29	23,4	cyclictest	930662	-21	latency_hist	10:20:01	0
732801	99	28	26,2	cyclictest	800632	-21	kworker/u8:0+flush-259:0	08:25:01	1
732809	99	27	21,4	cyclictest	962296	-21	base64	10:50:02	3
732809	99	27	21,4	cyclictest	862739	-21	latency_hist	09:15:01	3
732804	99	27	23,3	cyclictest	1039645	-21	kworker/u8:1+flush-259:0	12:20:49	2
732804	99	26	22,3	cyclictest	831585	-21	kworker/u8:0+events_unbound	09:25:19	2
732801	99	26	23,2	cyclictest	780263	-21	kworker/u8:3+flush-259:0	08:10:00	1
732801	99	26	23,2	cyclictest	758558	-21	kworker/u8:1+flush-259:0	07:43:55	1
732801	99	26	20,4	cyclictest	1028800	-21	ntpq	11:50:21	1
732799	99	26	21,3	cyclictest	743059	-21	latency_hist	07:20:00	0
732809	99	25	18,5	cyclictest	1029692	-21	latency_hist	11:54:59	3
732801	99	25	21,3	cyclictest	925737	-21	kworker/u8:1+events_unbound	10:29:23	1
732804	99	24	21,2	cyclictest	852387	-21	kworker/u8:3+events_unbound	09:05:19	2
732804	99	24	18,4	cyclictest	998695	-21	latency_hist	11:25:00	2
732804	99	24	17,5	cyclictest	737889	-21	latency_hist	07:15:00	2
732801	99	24	21,2	cyclictest	831552	-21	apt-get	08:43:56	1
732799	99	24	19,4	cyclictest	873056	-21	latency_hist	09:25:01	0
732799	99	24	17,5	cyclictest	1034843	-21	latency_hist	11:59:59	0
732801	99	23	20,2	cyclictest	998721	-21	kworker/u8:2+events_unbound	11:30:24	1
732809	99	22	17,3	cyclictest	852360	-21	latency_hist	09:05:01	3
732801	99	22	19,2	cyclictest	727668	-21	kworker/u8:2+events_unbound	07:16:04	1
732799	99	22	19,2	cyclictest	857554	-21	kworker/u8:5+events_unbound	09:30:19	0
732799	99	22	17,3	cyclictest	335557	-21	systemd-journal	07:27:14	0
732809	99	21	19,1	cyclictest	888253	-21	fwupd	09:37:28	3
732809	99	21	18,2	cyclictest	1019234	-21	kworker/u8:3+events_unbound	12:10:00	3
732809	99	21	17,3	cyclictest	998721	-21	kworker/u8:2+flush-259:0	11:40:27	3
732809	99	21	15,4	cyclictest	877317	-21	ntpq	09:25:22	3
732804	99	21	18,2	cyclictest	920107	-21	kworker/u8:4+flush-259:0	10:15:29	2
732804	99	21	18,2	cyclictest	903986	-21	kworker/u8:3+events_unbound	10:00:00	2
732801	99	21	18,2	cyclictest	689192	-21	kworker/u8:0+events_unbound	07:10:01	1
732801	99	21	17,3	cyclictest	920107	-21	kworker/u8:4+events_unbound	11:15:23	1
732801	99	21	17,3	cyclictest	836899	-21	kworker/u8:5+events_unbound	08:50:22	1
732801	99	21	17,3	cyclictest	1008735	-21	kworker/u8:1+events_unbound	11:42:25	1
732799	99	21	19,2	cyclictest	790327	-21	kworker/u8:1+flush-259:0	08:13:25	0
732799	99	21	17,3	cyclictest	920107	-21	kworker/u8:4+events_unbound	10:55:13	0
732799	99	21	16,4	cyclictest	780263	-21	kworker/u8:3+flush-259:0	08:00:10	0
732799	99	21	16,4	cyclictest	780263	-21	kworker/u8:3+flush-259:0	08:00:10	0
732809	99	20	13,5	cyclictest	836044	-21	ntpq	08:45:24	3
732804	99	20	17,2	cyclictest	852387	-21	kworker/u8:3+events_unbound	09:18:16	2
732804	99	20	17,2	cyclictest	780263	-21	kworker/u8:3+events_unbound	08:05:17	2
732804	99	20	16,3	cyclictest	764361	-21	kworker/u8:2+events_unbound	08:17:15	2
732804	99	20	14,4	cyclictest	780127	-21	cat	07:55:00	2
732801	99	20	17,3	cyclictest	998721	-21	kworker/u8:2+events_unbound	11:46:24	1
732801	99	20	17,2	cyclictest	780263	-21	kworker/u8:3+flush-259:0	08:00:18	1
732801	99	20	17,2	cyclictest	780263	-21	kworker/u8:3+flush-259:0	08:00:18	1
732809	99	19	16,2	cyclictest	0	-21	swapper/3	07:25:01	3
732809	99	19	13,4	cyclictest	335557	-21	systemd-journal	11:35:38	3
732809	99	19	13,4	cyclictest	335557	-21	systemd-journal	07:41:14	3
732809	99	19	13,4	cyclictest	1004140	-21	apt-get	11:30:13	3
732809	99	19	12,5	cyclictest	993524	-21	latency_hist	11:20:01	3
732804	99	19	17,2	cyclictest	993271	-21	kworker/u8:0+events_unbound	11:29:25	2
732804	99	19	16,2	cyclictest	925292	-21	gpgv	10:15:01	2
732804	99	19	16,2	cyclictest	925292	-21	gpgv	10:15:00	2
732804	99	19	15,3	cyclictest	998721	-21	kworker/u8:2+flush-259:0	12:00:16	2
732804	99	19	15,3	cyclictest	903986	-21	kworker/u8:3+events_unbound	10:46:00	2
732804	99	19	14,4	cyclictest	983205	-21	latency_hist	11:10:00	2
732801	99	19	16,2	cyclictest	920107	-21	kworker/u8:4+flush-259:0	10:30:26	1
732801	99	19	16,2	cyclictest	831585	-21	kworker/u8:0+events_unbound	09:25:25	1
732801	99	19	16,2	cyclictest	774453	-21	kworker/u8:0+flush-259:0	07:55:56	1
732801	99	19	16,2	cyclictest	1060716	-21	kworker/u8:0+flush-259:0	12:35:25	1
732801	99	19	15,3	cyclictest	821545	-21	apt-get	08:35:12	1
732799	99	19	16,2	cyclictest	831585	-21	kworker/u8:0+flush-259:0	08:45:16	0
732799	99	19	16,2	cyclictest	821956	-21	kworker/u8:4+flush-259:0	08:45:00	0
732799	99	19	16,2	cyclictest	800632	-21	kworker/u8:0+events_unbound	08:25:15	0
732809	99	18	12,4	cyclictest	335557	-21	systemd-journal	07:56:14	3
732804	99	18	16,1	cyclictest	769341	-21	latency_hist	07:45:00	2
732804	99	18	15,2	cyclictest	920107	-21	kworker/u8:4+events_unbound	10:30:15	2
732804	99	18	15,2	cyclictest	716796	-21	kworker/u8:3+flush-259:0	07:25:23	2
732804	99	18	12,4	cyclictest	335557	-21	systemd-journal	07:21:12	2
732801	99	18	15,2	cyclictest	993271	-21	kworker/u8:0+flush-259:0	11:22:55	1
732801	99	18	15,2	cyclictest	920107	-21	kworker/u8:4+events_unbound	11:00:24	1
732801	99	18	15,2	cyclictest	903986	-21	kworker/u8:3+events_unbound	10:12:26	1
732801	99	18	15,2	cyclictest	903986	-21	kworker/u8:3+events_unbound	10:12:25	1
732801	99	18	15,2	cyclictest	1019234	-21	kworker/u8:3+events_unbound	12:15:19	1
732801	99	18	15,2	cyclictest	0	-21	swapper/1	09:00:00	1
732801	99	18	13,4	cyclictest	955660	-21	ntpq	10:40:23	1
732799	99	18	15,2	cyclictest	993531	-21	grep	11:20:00	0
732799	99	18	15,2	cyclictest	790327	-21	kworker/u8:1+flush-259:0	08:10:01	0
732799	99	18	15,2	cyclictest	780263	-21	kworker/u8:3+flush-259:0	08:00:01	0
732799	99	18	14,3	cyclictest	763823	-21	gpgv	07:40:00	0
732804	99	17	14,2	cyclictest	975962	-21	kthreadcore	11:00:20	2
732804	99	17	14,2	cyclictest	920107	-21	kworker/u8:4+flush-259:0	10:35:25	2
732804	99	17	13,3	cyclictest	877268	-21	kworker/u8:1+events_unbound	09:37:29	2
732804	99	17	13,3	cyclictest	1039645	-21	kworker/u8:1+events_unbound	12:05:24	2
732804	99	17	12,4	cyclictest	335557	-21	systemd-journal	09:14:14	2
732801	99	17	15,2	cyclictest	946165	-21	kworker/u8:0+events_unbound	10:48:25	1
732801	99	17	14,2	cyclictest	811603	-21	kworker/u8:3+flush-259:0	08:25:26	1
732801	99	17	14,2	cyclictest	1060716	-21	kworker/u8:0+events_unbound	12:34:26	1
732801	99	17	14,2	cyclictest	1039645	-21	kworker/u8:1+events_unbound	12:27:23	1
732801	99	17	12,4	cyclictest	878429	-21	/usr/sbin/munin	09:30:16	1
732799	99	17	14,2	cyclictest	893653	-21	kworker/u8:0+flush-259:0	09:50:24	0
732799	99	17	14,2	cyclictest	853647	-21	kthreadcore	09:05:18	0
732799	99	17	14,2	cyclictest	603	50	irq/132-enp0s29f1:rx-0	12:10:23	0
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2021-09-01 - 00:34