OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rackeslot3s.osadl.org (updated Wed Mar 25, 2026 00:45:28)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
1619030	99	95	27,36	cyclictest	1624209	-21	latency_hist	20:50:02	2
1619030	99	95	27,36	cyclictest	1624209	-21	latency_hist	20:50:02	2
1619023	99	93	86,4	cyclictest	150	-21	pr/ttyS2	19:15:02	0
1619030	99	89	25,32	cyclictest	1658912	-21	sh	22:28:27	2
1619030	99	83	1,44	cyclictest	1662908	-21	ssh	22:37:54	2
1619030	99	81	2,4	cyclictest	13	1	rcu_preempt	00:00:03	2
1619030	99	81	2,4	cyclictest	13	1	rcu_preempt	00:00:03	2
1619030	99	79	2,45	cyclictest	1691065	-21	sh	23:43:32	2
1619030	99	79	2,45	cyclictest	1691065	-21	sh	23:43:32	2
1619030	99	79	11,33	cyclictest	1693732	-21	latency_hist	23:50:02	2
1619030	99	76	2,43	cyclictest	1706842	-21	date	00:20:03	2
1619030	99	71	2,31	cyclictest	1652539	-21	ssh	22:13:37	2
1619030	99	70	10,27	cyclictest	0	-21	swapper/2	00:23:01	2
1619030	99	69	1,64	cyclictest	1677933	-21	ssh	23:12:55	2
1619030	99	69	1,42	cyclictest	364	-21	snmpd	20:06:37	2
1619030	99	69	1,38	cyclictest	1625530	-21	ssh	21:10:09	2
1619030	99	69	1,38	cyclictest	1625530	-21	ssh	21:10:08	2
1619030	99	68	2,61	cyclictest	1679277	-21	ssh	23:15:42	2
1619030	99	68	1,63	cyclictest	1688821	-21	ssh	23:38:15	2
1619030	99	68	1,63	cyclictest	1684466	-21	ssh	23:28:02	2
1619030	99	67	3,59	cyclictest	1709485	-21	sshd	00:26:21	2
1619030	99	67	3,59	cyclictest	1709485	-21	sshd	00:26:21	2
1619030	99	66	2,59	cyclictest	1622350	-21	munin-run	20:15:02	2
1619030	99	66	1,61	cyclictest	1698311	-21	ssh	00:00:08	2
1619030	99	66	1,61	cyclictest	1630684	-21	ssh	21:22:29	2
1619030	99	66	1,60	cyclictest	1703592	-21	ssh	00:12:51	2
1619030	99	65	2,58	cyclictest	1640946	-21	sshd	21:46:20	2
1619030	99	65	24,37	cyclictest	1712023	-21	sh	00:32:23	2
1619030	99	65	1,60	cyclictest	1713735	-21	ssh	00:36:16	2
1619030	99	65	1,60	cyclictest	1686705	-21	ssh	23:33:19	2
1619030	99	65	1,60	cyclictest	1686705	-21	ssh	23:33:19	2
1619030	99	65	1,60	cyclictest	1676573	-21	ssh	23:09:52	2
1619030	99	65	1,60	cyclictest	1670056	-21	ssh	22:54:43	2
1619030	99	65	1,60	cyclictest	1665874	-21	ssh	22:44:58	2
1619030	99	65	1,60	cyclictest	1661012	-21	ssh	22:33:31	2
1619030	99	65	1,60	cyclictest	1649118	-21	ssh	22:05:23	2
1619030	99	65	1,60	cyclictest	1649118	-21	ssh	22:05:23	2
1619030	99	65	1,60	cyclictest	1628555	-21	ssh	21:17:30	2
1619030	99	64	2,58	cyclictest	1653394	-21	ssh	22:15:18	2
1619030	99	64	1,59	cyclictest	1701235	-21	ssh	00:07:17	2
1619030	99	64	1,59	cyclictest	1694107	-21	ssh	23:50:23	2
1619030	99	64	1,59	cyclictest	1681965	-21	ssh	23:22:07	2
1619030	99	64	1,59	cyclictest	1657092	-21	ssh	22:24:20	2
1619030	99	64	1,59	cyclictest	1647803	-21	ssh	22:02:29	2
1619030	99	64	1,59	cyclictest	1644912	-21	ssh	21:55:26	2
1619030	99	64	1,59	cyclictest	1643567	-21	ssh	21:52:37	2
1619030	99	64	1,59	cyclictest	1638588	-21	ssh	21:40:42	2
1619030	99	63	2,57	cyclictest	1666587	-21	ssh	22:46:09	2
1619030	99	63	1,58	cyclictest	1670501	-21	ssh	22:55:14	2
1619030	99	63	1,58	cyclictest	1636516	-21	ssh	21:35:53	2
1619030	99	63	1,58	cyclictest	1634702	-21	ssh	21:31:49	2
1619030	99	63	1,58	cyclictest	1632023	-21	ssh	21:25:16	2
1619030	99	63	1,58	cyclictest	1622551	-21	sshd	20:15:21	2
1619030	99	63	1,58	cyclictest	1620777	-21	sshd	19:43:20	2
1619030	99	63	1,58	cyclictest	1620777	-21	sshd	19:43:20	2
1619030	99	62	2,56	cyclictest	1672591	-21	ssh	23:00:06	2
1619030	99	62	1,57	cyclictest	1619991	-21	sshd	19:27:20	2
1619023	99	62	34,25	cyclictest	168	50	irq/134-8000000	00:40:02	0
1619030	99	61	1,56	cyclictest	1620247	-21	sshd	19:33:20	2
1619030	99	60	2,54	cyclictest	1624701	-21	sshd	20:59:20	2
1619030	99	59	2,52	cyclictest	364	-21	snmpd	20:40:32	2
1619030	99	59	1,54	cyclictest	1621025	-21	sshd	19:47:19	2
1619026	99	59	9,13	cyclictest	23	-21	ksoftirqd/1	22:58:54	1
1619023	99	59	2,3	cyclictest	13	1	rcu_preempt	00:18:35	0
1619030	99	58	2,51	cyclictest	364	-21	snmpd	19:35:18	2
1619030	99	58	2,51	cyclictest	1619525	-21	munin-run	19:20:02	2
1619030	99	58	1,53	cyclictest	48	-21	kcompactd0	20:25:46	2
1619030	99	58	1,53	cyclictest	1624433	-21	sshd	20:51:20	2
1619023	99	58	27,23	cyclictest	1619019	-21	cyclictest	19:20:01	0
1619023	99	58	19,35	cyclictest	168	50	irq/134-8000000	21:10:02	0
1619030	99	57	1,52	cyclictest	1622856	-21	sshd	20:23:19	2
1619026	99	57	17,10	cyclictest	23	-21	ksoftirqd/1	23:20:01	1
1619023	99	57	19,33	cyclictest	168	50	irq/134-8000000	00:20:43	0
1619030	99	56	1,51	cyclictest	1619775	-21	munin-run	19:25:01	2
1619026	99	56	9,5	cyclictest	13	1	rcu_preempt	22:54:56	1
1619026	99	56	10,11	cyclictest	23	-21	ksoftirqd/1	22:03:47	1
1619023	99	56	19,32	cyclictest	168	50	irq/134-8000000	22:53:27	0
1619023	99	56	1,3	cyclictest	13	1	rcu_preempt	00:09:13	0
1619030	99	55	2,48	cyclictest	48	-21	kcompactd0	20:36:18	2
1619030	99	55	1,50	cyclictest	3467562	-21	sshd	19:10:19	2
1619026	99	55	6,14	cyclictest	23	-21	ksoftirqd/1	23:25:07	1
1619026	99	55	11,10	cyclictest	23	-21	ksoftirqd/1	22:24:51	1
1619023	99	55	28,12	cyclictest	1674464	-21	ssh	23:04:59	0
1619023	99	55	25,15	cyclictest	1693498	-21	ssh	23:49:20	0
1619023	99	55	20,31	cyclictest	168	50	irq/134-8000000	23:15:35	0
1619023	99	55	19,31	cyclictest	168	50	irq/134-8000000	21:48:53	0
1619030	99	54	2,47	cyclictest	1621512	-21	date	19:55:02	2
1619030	99	54	2,47	cyclictest	1621512	-21	date	19:55:01	2
1619030	99	54	23,27	cyclictest	30	-21	ksoftirqd/2	19:55:02	2
1619030	99	54	1,49	cyclictest	1623429	-21	munin-run	20:35:00	2
1619030	99	54	1,48	cyclictest	364	-21	snmpd	21:06:03	2
1619026	99	54	14,9	cyclictest	23	-21	ksoftirqd/1	00:23:01	1
1619026	99	54	10,10	cyclictest	23	-21	ksoftirqd/1	00:33:32	1
1619023	99	54	4,46	cyclictest	0	-21	swapper/0	23:30:02	0
1619023	99	54	4,46	cyclictest	0	-21	swapper/0	23:30:02	0
1619023	99	54	27,12	cyclictest	1639583	-21	ssh	21:43:15	0
1619023	99	54	20,30	cyclictest	168	50	irq/134-8000000	23:22:14	0
1619023	99	54	20,30	cyclictest	168	50	irq/134-8000000	23:13:34	0
1619023	99	54	19,30	cyclictest	168	50	irq/134-8000000	22:04:26	0
1619023	99	54	18,31	cyclictest	168	50	irq/134-8000000	22:34:33	0
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2021-09-01 - 00:33