OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rackeslot3s.osadl.org (updated Thu Jun 25, 2026 00:45:29)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
1261310	99	95	28,14	cyclictest	1360181	-21	ssh	00:23:53	0
1261310	99	95	12,61	cyclictest	169	50	irq/224-8000000	23:00:03	0
1261310	99	83	29,35	cyclictest	1336782	-21	latency_hist	23:35:02	0
1261322	99	82	13,41	cyclictest	1317726	-21	latency_hist	22:55:02	3
1261310	99	81	26,13	cyclictest	1354655	-21	ssh	00:12:11	0
1261310	99	78	17,44	cyclictest	168	50	irq/134-8000000	22:21:33	0
1261310	99	77	29,42	cyclictest	168	50	irq/134-8000000	22:45:00	0
1261310	99	76	28,16	cyclictest	1340098	-21	ssh	23:41:35	0
1261310	99	76	1,39	cyclictest	1333685	-21	ssh	23:28:18	0
1261310	99	75	27,7	cyclictest	1366905	-21	ssh	00:37:53	0
1261310	99	75	27,7	cyclictest	1366905	-21	ssh	00:37:53	0
1261310	99	74	26,13	cyclictest	1323768	-21	ssh	23:07:25	0
1261310	99	74	23,5	cyclictest	0	-21	swapper/0	00:28:17	0
1261310	99	74	23,5	cyclictest	0	-21	swapper/0	00:28:17	0
1261310	99	74	22,16	cyclictest	1356934	-21	ssh	00:16:52	0
1261322	99	73	4,4	cyclictest	13	1	rcu_preempt	22:00:02	3
1261322	99	73	4,4	cyclictest	13	1	rcu_preempt	22:00:01	3
1261310	99	73	25,12	cyclictest	1283032	-21	ssh	21:42:17	0
1261310	99	72	2,39	cyclictest	1327870	-21	ssh	23:15:53	0
1261310	99	72	1,5	cyclictest	1261787	-21	cron	19:20:01	0
1261310	99	72	1,5	cyclictest	1261787	-21	cron	19:20:00	0
1261310	99	71	28,13	cyclictest	1274058	-21	ssh	21:23:39	0
1261310	99	71	22,33	cyclictest	12	-21	ksoftirqd/0	23:52:49	0
1261310	99	71	19,9	cyclictest	1346863	-21	ssh	23:55:38	0
1261310	99	71	17,12	cyclictest	1363560	-21	ssh	00:30:37	0
1261322	99	70	27,38	cyclictest	1273590	-21	sh	21:22:31	3
1261310	99	70	3,6	cyclictest	1277242	-21	sed	21:30:03	0
1261310	99	70	25,14	cyclictest	1330242	-21	ssh	23:20:43	0
1261310	99	70	25,14	cyclictest	1292017	-21	ssh	22:01:02	0
1261310	99	70	24,14	cyclictest	1306135	-21	ssh	22:30:21	0
1261310	99	70	16,12	cyclictest	1350344	-21	ssh	00:03:08	0
1261310	99	69	24,12	cyclictest	1305923	-21	sed	22:30:03	0
1261310	99	69	24,12	cyclictest	1297975	-21	ssh	22:13:37	0
1261310	99	69	21,7	cyclictest	1289297	-21	ssh	21:55:25	0
1261310	99	69	21,7	cyclictest	1289297	-21	ssh	21:55:25	0
1261310	99	69	18,12	cyclictest	1270791	-21	ssh	21:16:42	0
1261310	99	69	18,12	cyclictest	1270791	-21	ssh	21:16:41	0
1261322	99	68	1,14	cyclictest	3467602	-21	systemd-journal	21:16:31	3
1261322	99	68	1,14	cyclictest	3467602	-21	systemd-journal	21:16:31	3
1261310	99	68	15,13	cyclictest	1351984	-21	ssh	00:06:24	0
1261322	99	67	2,13	cyclictest	1348253	-21	sshd	23:58:51	3
1261310	99	67	2,60	cyclictest	1300965	-21	sh	22:20:00	0
1261310	99	67	2,60	cyclictest	1300965	-21	sh	22:20:00	0
1261310	99	67	22,12	cyclictest	1308532	-21	ssh	22:35:24	0
1261310	99	67	1,62	cyclictest	0	-21	swapper/0	22:50:03	0
1261310	99	66	27,34	cyclictest	169	50	irq/224-8000000	21:35:00	0
1261322	99	65	28,33	cyclictest	1320271	-21	latency_hist	23:00:02	3
1261322	99	65	1,60	cyclictest	1324359	-21	ssh	23:08:42	3
1261310	99	65	1,37	cyclictest	1342260	-21	ssh	23:46:01	0
1261322	99	64	2,56	cyclictest	1341114	-21	apt-get	23:44:00	3
1261322	99	64	17,43	cyclictest	37	-21	ksoftirqd/3	23:29:22	3
1261310	99	64	22,5	cyclictest	309	-21	in:imuxsock	21:05:01	0
1261322	99	63	2,6	cyclictest	1265193	-21	munin-run	20:25:00	3
1261322	99	63	2,57	cyclictest	1311009	-21	ssh	22:40:28	3
1261322	99	63	25,35	cyclictest	1266697	-21	sshd	20:50:20	3
1261322	99	63	2,4	cyclictest	1304484	-21	ssh	22:27:05	3
1261322	99	63	2,18	cyclictest	1354551	-21	sh	00:11:57	3
1261322	99	63	2,18	cyclictest	1295119	-21	ssh	22:07:32	3
1261322	99	63	2,14	cyclictest	1284718	-21	sshd	21:45:42	3
1261322	99	63	1,8	cyclictest	1338198	-21	sshd	23:37:41	3
1261322	99	63	1,6	cyclictest	1335584	-21	ssh	23:32:14	3
1261322	99	63	1,14	cyclictest	1344854	-21	ssh	23:51:31	3
1261310	99	63	28,16	cyclictest	1295479	-21	ssh	22:08:24	0
1261310	99	63	13,13	cyclictest	275	-21	dbus-daemon	19:45:01	0
1261310	99	63	13,13	cyclictest	275	-21	dbus-daemon	19:45:00	0
1261322	99	62	2,19	cyclictest	1280732	-21	ssh	21:37:37	3
1261322	99	62	2,19	cyclictest	1280732	-21	ssh	21:37:37	3
1261322	99	62	1,9	cyclictest	1362522	-21	ssh	00:28:47	3
1261322	99	62	1,9	cyclictest	1362522	-21	ssh	00:28:47	3
1261322	99	62	1,9	cyclictest	1276349	-21	ssh	21:28:27	3
1261322	99	62	1,8	cyclictest	1310201	-21	ssh	22:39:19	3
1261322	99	62	1,7	cyclictest	1297219	-21	ssh	22:11:54	3
1261322	99	62	1,17	cyclictest	1366961	-21	sh	00:38:00	3
1261322	99	62	1,17	cyclictest	1366961	-21	sh	00:38:00	3
1261310	99	62	26,5	cyclictest	1288509	-21	sh	21:54:03	0
1261310	99	62	1,34	cyclictest	1268441	-21	sshd	21:11:42	0
1261310	99	62	1,27	cyclictest	0	-21	swapper/0	23:13:33	0
1261322	99	61	2,54	cyclictest	3467602	-21	systemd-journal	00:05:51	3
1261322	99	61	2,54	cyclictest	1322518	-21	munin-run	23:05:00	3
1261322	99	61	2,53	cyclictest	1292855	-21	ssh	22:02:54	3
1261322	99	61	1,8	cyclictest	1359517	-21	ssh	00:22:19	3
1261322	99	61	1,7	cyclictest	1302401	-21	ssh	22:22:46	3
1261322	99	61	1,53	cyclictest	1357547	-21	ssh	00:18:17	3
1261322	99	61	1,4	cyclictest	1307331	-21	ssh	22:33:12	3
1261322	99	60	2,5	cyclictest	1349044	-21	ssh	00:00:05	3
1261322	99	60	2,54	cyclictest	364	-21	snmpd	23:21:32	3
1261322	99	60	2,20	cyclictest	1326038	-21	ssh	23:12:13	3
1261322	99	60	1,8	cyclictest	1329371	-21	ssh	23:19:26	3
1261322	99	60	1,7	cyclictest	1314475	-21	ssh	22:48:02	3
1261322	99	60	1,6	cyclictest	1363674	-21	ssh	00:30:52	3
1261322	99	60	1,6	cyclictest	1283178	-21	ssh	21:42:35	3
1261310	99	60	19,9	cyclictest	1319035	-21	ssh	22:57:28	0
1261322	99	59	26,29	cyclictest	1265954	-21	latency_hist	20:40:02	3
1261322	99	59	2,5	cyclictest	1267280	-21	munin-run	21:05:01	3
1261322	99	59	2,52	cyclictest	1263831	-21	sshd	19:55:04	3
1261322	99	59	2,51	cyclictest	1263378	-21	munin-run	19:50:01	3
1261322	99	59	2,51	cyclictest	1263378	-21	munin-run	19:50:00	3
1261322	99	59	2,18	cyclictest	1	-21	systemd	21:00:01	3
1261322	99	59	1,4	cyclictest	1300730	-21	ssh	22:19:33	3
1261322	99	59	1,4	cyclictest	1300730	-21	ssh	22:19:33	3
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2021-09-01 - 00:33