OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rackeslot3s.osadl.org (updated Fri Jan 09, 2026 00:45:07)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
3389941	99	112	1,3	cyclictest	0	-21	swapper/0	21:45:02	0
3389941	99	80	76,2	cyclictest	0	-21	swapper/0	21:50:02	0
3389941	99	80	72,2	cyclictest	0	-21	swapper/0	23:20:02	0
3389941	99	80	1,75	cyclictest	0	-21	swapper/0	22:10:02	0
3389941	99	79	75,2	cyclictest	0	-21	swapper/0	23:10:01	0
3389952	99	77	24,19	cyclictest	3403265	-21	cat	23:25:02	3
3389952	99	74	22,18	cyclictest	3393801	-21	ls	20:25:02	3
3389952	99	71	16,16	cyclictest	3393505	-21	munin-run	20:20:00	3
3389946	99	67	28,35	cyclictest	274	-21	cron	00:20:00	1
3389946	99	67	14,13	cyclictest	1	-21	systemd	20:05:01	1
3389946	99	66	2,15	cyclictest	3391411	-21	modprobe	19:40:01	1
3389949	99	64	49,11	cyclictest	29	1	rcuc/2	22:20:00	2
3389946	99	63	2,15	cyclictest	3395841	-21	true	21:05:01	1
3389952	99	62	37,18	cyclictest	238	-21	systemd-journal	19:55:00	3
3389946	99	62	24,4	cyclictest	3395052	-21	latency_hist	20:50:01	1
3389946	99	61	3,18	cyclictest	238	-21	systemd-journal	00:10:01	1
3389946	99	60	24,5	cyclictest	3392457	-21	latency_hist	20:00:02	1
3389946	99	60	2,19	cyclictest	3393520	-21	basename	20:20:00	1
3389946	99	60	2,17	cyclictest	3393005	-21	latency_hist	20:10:02	1
3389946	99	60	2,17	cyclictest	3393005	-21	latency_hist	20:10:02	1
3389946	99	60	15,6	cyclictest	309	-21	in:imuxsock	20:50:02	1
3389946	99	59	3,19	cyclictest	3394573	-21	cat	20:40:02	1
3389946	99	59	1,18	cyclictest	3390230	-21	latency_hist	19:15:01	1
3389946	99	59	10,13	cyclictest	1	-21	systemd	22:25:01	1
3389946	99	58	3,17	cyclictest	3399203	-21	grep	22:10:00	1
3389946	99	58	27,19	cyclictest	23	-21	ksoftirqd/1	23:30:02	1
3389946	99	58	21,12	cyclictest	23	-21	ksoftirqd/1	22:20:00	1
3389946	99	58	17,10	cyclictest	238	-21	systemd-journal	23:19:55	1
3389952	99	57	39,13	cyclictest	3395544	-21	cron	21:00:00	3
3389952	99	57	37,16	cyclictest	3396069	-21	sshd	21:09:55	3
3389946	99	57	8,9	cyclictest	0	-21	swapper/1	19:45:02	1
3389946	99	57	34,17	cyclictest	3404875	-21	cron	00:00:00	1
3389946	99	57	3,18	cyclictest	3402053	-21	munin-run	23:05:00	1
3389952	99	56	6,45	cyclictest	3394524	-21	sh	20:40:00	3
3389952	99	56	37,13	cyclictest	275	-21	dbus-daemon	00:05:00	3
3389949	99	56	47,6	cyclictest	30	-21	ksoftirqd/2	20:15:01	2
3389952	99	55	9,12	cyclictest	3396574	-21	cron	21:17:00	3
3389952	99	55	37,14	cyclictest	3400719	-21	sshd	22:38:09	3
3389952	99	55	37,13	cyclictest	3394248	-21	sshd	20:32:54	3
3389949	99	55	4,10	cyclictest	30	-21	ksoftirqd/2	21:40:00	2
3389949	99	55	2,9	cyclictest	30	-21	ksoftirqd/2	21:10:01	2
3389949	99	55	12,5	cyclictest	13	1	rcu_preempt	21:23:01	2
3389946	99	55	2,17	cyclictest	3390665	-21	sh	19:24:59	1
3389946	99	55	2,13	cyclictest	3406770	-21	modprobe	00:34:59	1
3389952	99	54	40,10	cyclictest	3406236	-21	sshd	00:23:55	3
3389952	99	54	39,10	cyclictest	3393216	-21	sshd	20:13:54	3
3389952	99	54	37,13	cyclictest	3397114	-21	sshd	21:28:01	3
3389952	99	54	36,15	cyclictest	3390642	-21	sshd	19:22:45	3
3389946	99	54	6,13	cyclictest	3404729	-21	grep	23:55:01	1
3389946	99	54	23,4	cyclictest	3397919	-21	/usr/sbin/munin	21:45:01	1
3389941	99	54	39,11	cyclictest	1	-21	systemd	00:20:01	0
3389952	99	53	46,4	cyclictest	3391951	-21	munin-run	19:50:01	3
3389952	99	53	46,4	cyclictest	3391951	-21	munin-run	19:50:01	3
3389952	99	53	38,11	cyclictest	3403576	-21	sshd	23:33:55	3
3389952	99	53	38,11	cyclictest	3400461	-21	sshd	22:31:55	3
3389952	99	53	38,11	cyclictest	3398399	-21	sshd	21:53:55	3
3389952	99	53	38,11	cyclictest	3398399	-21	sshd	21:53:55	3
3389952	99	53	38,11	cyclictest	3396319	-21	sshd	21:11:55	3
3389952	99	53	37,12	cyclictest	3405438	-21	sshd	00:07:56	3
3389952	99	53	33,15	cyclictest	3399464	-21	munin-run	22:15:01	3
3389946	99	53	3,17	cyclictest	3396620	-21	munin-run	21:20:01	1
3389946	99	53	23,4	cyclictest	3398857	-21	latency_hist	22:00:02	1
3389946	99	53	21,12	cyclictest	23	-21	ksoftirqd/1	22:05:01	1
3389952	99	52	39,9	cyclictest	3395006	-21	cat	20:45:02	3
3389952	99	52	39,9	cyclictest	3395006	-21	cat	20:45:02	3
3389952	99	52	38,10	cyclictest	3406491	-21	sshd	00:29:55	3
3389952	99	52	38,10	cyclictest	3406491	-21	sshd	00:29:55	3
3389952	99	52	37,11	cyclictest	3401758	-21	sshd	22:59:55	3
3389952	99	52	36,12	cyclictest	3406741	-21	sshd	00:33:56	3
3389949	99	52	43,6	cyclictest	0	-21	swapper/2	19:10:01	2
3389949	99	52	43,6	cyclictest	0	-21	swapper/2	19:10:01	2
3389949	99	52	31,16	cyclictest	238	-21	systemd-journal	19:20:01	2
3389949	99	52	31,16	cyclictest	238	-21	systemd-journal	19:20:00	2
3389946	99	52	21,5	cyclictest	3401016	-21	sh	22:45:01	1
3389952	99	51	44,4	cyclictest	3401022	-21	munin-run	22:45:01	3
3389952	99	51	38,9	cyclictest	3392971	-21	cron	20:10:00	3
3389952	99	51	38,9	cyclictest	3392971	-21	cron	20:10:00	3
3389952	99	51	37,11	cyclictest	3402032	-21	sshd	23:03:55	3
3389952	99	51	37,11	cyclictest	3398149	-21	sshd	21:48:55	3
3389952	99	51	37,10	cyclictest	3399969	-21	sshd	22:23:55	3
3389952	99	51	37,10	cyclictest	3391682	-21	sshd	19:44:55	3
3389952	99	51	25,20	cyclictest	3397390	-21	munin-run	21:35:00	3
3389952	99	51	25,19	cyclictest	3401526	-21	grep	22:55:00	3
3389952	99	51	24,22	cyclictest	3403889	-21	unin-run	23:40:02	3
3389952	99	51	24,22	cyclictest	3403889	-21	unin-run	23:40:01	3
3389949	99	51	33,13	cyclictest	275	-21	dbus-daemon	20:45:00	2
3389949	99	51	33,13	cyclictest	275	-21	dbus-daemon	20:45:00	2
3389946	99	51	4,15	cyclictest	3407020	-21	modprobe	00:40:00	1
3389946	99	51	31,16	cyclictest	23	-21	ksoftirqd/1	20:45:00	1
3389946	99	51	31,16	cyclictest	23	-21	ksoftirqd/1	20:45:00	1
3389946	99	51	18,5	cyclictest	3390916	-21	latency_hist	19:30:02	1
3389946	99	51	17,4	cyclictest	3393929	-21	latency_hist	20:25:02	1
3389946	99	51	13,11	cyclictest	23	-21	ksoftirqd/1	23:15:01	1
3389941	99	51	34,12	cyclictest	1	-21	systemd	23:35:01	0
3389941	99	51	19,23	cyclictest	15	1	rcuc/0	22:40:01	0
3389941	99	51	11,36	cyclictest	238	-21	systemd-journal	21:10:00	0
3389952	99	50	38,9	cyclictest	3400225	-21	sshd	22:29:55	3
3389952	99	50	38,9	cyclictest	3400225	-21	sshd	22:29:55	3
3389952	99	50	37,10	cyclictest	364	-21	snmpd	20:50:23	3
3389952	99	50	37,10	cyclictest	3401237	-21	sshd	22:45:56	3
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2021-09-01 - 00:33