OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #a, slot #6

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system

Characteristics of the highest latencies:

System rackaslot6.osadl.org (updated Thu Oct 23, 2025 06:45:33)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
1553406	99	2213	10,2197	cyclictest	1880806	-21	Run	05:54:53	4
1553406	99	2213	10,2197	cyclictest	1880806	-21	Run	05:54:53	4
1553402	99	984	1,979	cyclictest	1554170	-21	Run	05:18:05	2
1553402	99	674	2,668	cyclictest	34	-21	rcuc/2	06:21:45	2
1553402	99	282	1,278	cyclictest	815	-21	snmpd	06:18:11	2
1553402	99	281	1,275	cyclictest	2845676	-21	rm	06:06:18	2
1553408	99	270	1,266	cyclictest	815	-21	snmpd	05:06:16	6
1553408	99	270	1,266	cyclictest	815	-21	snmpd	05:06:16	6
1553408	99	160	1,156	cyclictest	773	-21	avahi-daemon	06:13:18	6
1553402	99	90	1,86	cyclictest	1258	-21	in:imuxsock	05:10:32	2
1553402	99	78	0,75	cyclictest	773	-21	avahi-daemon	06:30:19	2
1553399	99	59	52,4	cyclictest	2480702	-21	spawn	05:59:17	1
1553399	99	59	52,4	cyclictest	2480702	-21	spawn	05:59:16	1
1553403	99	57	50,4	cyclictest	2370199	-21	spawn	05:58:45	3
1553403	99	57	50,4	cyclictest	2370199	-21	spawn	05:58:45	3
1553407	99	56	50,4	cyclictest	2480651	-21	spawn	05:59:20	5
1553407	99	56	50,4	cyclictest	2480651	-21	spawn	05:59:19	5
1553398	99	56	35,3	cyclictest	2855756	-21	cron	06:25:00	0
1553411	99	54	27,3	cyclictest	1880108	-21	execl	05:47:22	7
1553407	99	53	0,49	cyclictest	1879994	-21	sh	05:46:01	5
1553411	99	52	46,4	cyclictest	1881525	-21	spawn	05:58:18	7
1553411	99	52	46,4	cyclictest	1881525	-21	spawn	05:58:17	7
1553406	99	51	47,3	cyclictest	1978369	-21	spawn	05:58:03	4
1553406	99	51	47,3	cyclictest	1978369	-21	spawn	05:58:03	4
1553402	99	47	39,5	cyclictest	2480661	-21	spawn	05:59:21	2
1553402	99	47	39,5	cyclictest	2480661	-21	spawn	05:59:21	2
1553398	99	47	42,3	cyclictest	1888431	-21	spawn	05:57:55	0
1553398	99	47	42,3	cyclictest	1888431	-21	spawn	05:57:54	0
1553408	99	46	40,3	cyclictest	2510585	-21	spawn	05:59:04	6
1553408	99	46	40,3	cyclictest	2510585	-21	spawn	05:59:04	6
1553411	99	44	30,11	cyclictest	1863296	-21	grep	05:38:28	7
1553411	99	44	30,11	cyclictest	1863296	-21	grep	05:38:28	7
1553411	99	44	30,11	cyclictest	1863296	-21	grep	05:38:27	7
1553406	99	44	37,4	cyclictest	1880087	-21	execl	05:46:59	4
1553402	99	43	39,2	cyclictest	1880073	-21	execl	05:47:08	2
1553398	99	43	26,7	cyclictest	2785526	-21	expr	06:01:59	0
1553398	99	43	26,7	cyclictest	2785526	-21	expr	06:01:59	0
1553407	99	42	1,38	cyclictest	811	-21	systemd-network	05:22:11	5
1553398	99	42	1,20	cyclictest	1880749	-21	fstime	05:53:27	0
1553398	99	42	1,20	cyclictest	1880749	-21	fstime	05:53:27	0
1553403	99	41	34,4	cyclictest	2781167	-21	sh	06:01:50	3
1553403	99	41	34,4	cyclictest	2781167	-21	sh	06:01:49	3
1553411	99	40	25,4	cyclictest	2792745	-21	sh	06:02:17	7
1553411	99	40	25,4	cyclictest	2792745	-21	sh	06:02:17	7
1553398	99	39	33,4	cyclictest	1880135	-21	execl	05:47:57	0
1553403	99	38	31,4	cyclictest	1880087	-21	execl	05:46:52	3
1553402	99	38	31,4	cyclictest	2817489	-21	expr	06:03:25	2
1553402	99	38	31,4	cyclictest	2817489	-21	expr	06:03:25	2
1553399	99	38	28,6	cyclictest	2793945	-21	sh	06:02:20	1
1553399	99	38	28,6	cyclictest	2793945	-21	sh	06:02:20	1
1553408	99	37	32,3	cyclictest	2814979	-21	rm	06:03:19	6
1553408	99	37	32,3	cyclictest	2814979	-21	rm	06:03:19	6
1553406	99	37	32,3	cyclictest	2844568	-21	looper	06:06:00	4
1553398	99	37	19,14	cyclictest	1864820	-21	grep	05:38:41	0
1553398	99	37	19,14	cyclictest	1864820	-21	grep	05:38:40	0
1553398	99	37	19,14	cyclictest	1864820	-21	grep	05:38:40	0
1553398	99	37	17,14	cyclictest	2844431	-21	sh	06:05:53	0
1553408	99	36	30,4	cyclictest	1880112	-21	execl	05:47:22	6
1553407	99	36	28,5	cyclictest	2846585	-21	sort	06:06:26	5
1553407	99	36	28,5	cyclictest	1875256	-21	rm	05:40:34	5
1553398	99	36	12,10	cyclictest	0	-21	swapper/0	05:26:56	0
1553407	99	35	2,18	cyclictest	204	50	irq/117-46000000.ethernet-tx2	06:02:32	5
1553407	99	35	2,18	cyclictest	204	50	irq/117-46000000.ethernet-tx2	06:02:31	5
1553402	99	35	1,15	cyclictest	0	-21	swapper/2	05:35:08	2
1553402	99	35	1,15	cyclictest	0	-21	swapper/2	05:35:08	2
1553402	99	35	1,15	cyclictest	0	-21	swapper/2	05:35:08	2
1553399	99	35	29,4	cyclictest	1880085	-21	execl	05:46:48	1
1553408	99	34	29,3	cyclictest	1880776	-21	Run	05:54:50	6
1553408	99	34	29,3	cyclictest	1880776	-21	Run	05:54:50	6
1553402	99	34	0,30	cyclictest	36	-21	ksoftirqd/2	05:28:05	2
1553411	99	32	25,4	cyclictest	1880768	-21	Run	05:54:50	7
1553411	99	32	25,4	cyclictest	1880768	-21	Run	05:54:50	7
1553407	99	32	26,4	cyclictest	2855758	-21	cron	06:25:00	5
1553406	99	32	21,6	cyclictest	2806553	-21	sh	06:03:00	4
1553406	99	32	21,6	cyclictest	2806553	-21	sh	06:03:00	4
1553403	99	32	3,2	cyclictest	1540257	-21	kworker/u32:5+rpciod	05:40:14	3
1553402	99	32	27,3	cyclictest	1873578	-21	sh	05:40:20	2
1553399	99	32	27,3	cyclictest	2839989	-21	sh	06:05:10	1
1553399	99	32	26,4	cyclictest	1874296	-21	sh	05:40:26	1
1553398	99	32	17,13	cyclictest	1876940	-21	sh	05:40:48	0
1553408	99	31	25,4	cyclictest	2854003	-21	sh	06:08:00	6
1553407	99	31	0,19	cyclictest	0	-21	swapper/5	05:36:34	5
1553407	99	31	0,19	cyclictest	0	-21	swapper/5	05:36:33	5
1553407	99	31	0,19	cyclictest	0	-21	swapper/5	05:36:33	5
1553402	99	31	2,25	cyclictest	1880738	-21	fstime	05:53:28	2
1553402	99	31	2,25	cyclictest	1880738	-21	fstime	05:53:28	2
1553411	99	30	23,4	cyclictest	1879183	-21	Run	05:42:01	7
1553411	99	30	22,5	cyclictest	2844695	-21	sh	06:06:00	7
1553407	99	30	25,3	cyclictest	1880761	-21	Run	05:54:50	5
1553407	99	30	25,3	cyclictest	1880761	-21	Run	05:54:50	5
1553403	99	30	24,4	cyclictest	2851184	-21	sh	06:07:27	3
1553403	99	30	24,4	cyclictest	1869545	-21	sh	05:39:34	3
1553403	99	30	24,4	cyclictest	1869545	-21	sh	05:39:34	3
1553403	99	30	24,4	cyclictest	1869545	-21	sh	05:39:34	3
1553403	99	30	23,4	cyclictest	2855760	-21	cron	06:24:59	3
1553399	99	30	25,3	cyclictest	1880601	-21	Run	05:50:49	1
1553399	99	30	25,3	cyclictest	1880601	-21	Run	05:50:48	1
1553402	99	29	1,25	cyclictest	1560805	-21	spawn	05:31:03	2
1553402	99	29	1,25	cyclictest	1555184	-21	execl	05:20:35	2
1553398	99	29	17,5	cyclictest	1821616	-21	grep	05:35:00	0
1553398	99	29	1,18	cyclictest	0	-21	swapper/0	06:38:08	0
1553403	99	28	2,18	cyclictest	151	-21	kworker/3:3+events	05:32:21	3
1553403	99	28	17,4	cyclictest	1880735	-21	Run	05:54:03	3
1553403	99	28	17,4	cyclictest	1880735	-21	Run	05:54:02	3
1553398	99	28	2,12	cyclictest	0	-21	swapper/0	06:20:02	0
1553411	99	27	23,2	cyclictest	2855522	-21	latency_hist	06:20:01	7
1553407	99	27	10,7	cyclictest	1554787	-21	execl	05:20:00	5
1553406	99	27	21,4	cyclictest	1863139	-21	wc	05:38:25	4
1553406	99	27	21,4	cyclictest	1863139	-21	wc	05:38:25	4
1553406	99	27	21,4	cyclictest	1863139	-21	wc	05:38:25	4
1553406	99	27	21,3	cyclictest	1875907	-21	tee	05:40:39	4
1553403	99	27	5,19	cyclictest	1554797	-21	latency_hist	05:20:01	3
1553399	99	27	6,4	cyclictest	1644724	-21	spawn	05:31:34	1
1553408	99	26	19,4	cyclictest	1874592	-21	sh	05:40:28	6
1553406	99	26	21,3	cyclictest	2855072	-21	latency_hist	06:15:00	4
1553403	99	26	15,3	cyclictest	0	-21	swapper/3	05:15:00	3
1553402	99	26	0,22	cyclictest	0	-21	swapper/2	06:13:08	2
1553402	99	26	0,22	cyclictest	0	-21	swapper/2	05:05:01	2
1553402	99	26	0,22	cyclictest	0	-21	swapper/2	05:05:01	2
1553399	99	26	22,2	cyclictest	1823446	-21	sort	05:35:07	1
1553399	99	26	22,2	cyclictest	1823446	-21	sort	05:35:06	1
1553399	99	26	22,2	cyclictest	1823446	-21	sort	05:35:06	1
1553408	99	25	3,13	cyclictest	0	-21	swapper/6	05:26:58	6
1553408	99	25	12,6	cyclictest	1862695	-21	sh	05:38:22	6
1553408	99	25	12,6	cyclictest	1862695	-21	sh	05:38:22	6
1553408	99	25	12,6	cyclictest	1862695	-21	sh	05:38:21	6
1553398	99	25	11,7	cyclictest	0	-21	swapper/0	05:20:16	0
1553411	99	24	0,20	cyclictest	1555666	-21	fstime	05:28:45	7
1553403	99	24	1,11	cyclictest	0	-21	swapper/3	05:28:28	3
1553402	99	24	0,21	cyclictest	0	-21	swapper/2	05:06:59	2
1553402	99	24	0,21	cyclictest	0	-21	swapper/2	05:06:59	2
1553398	99	24	17,4	cyclictest	1554158	-21	cron	05:14:59	0
1553398	99	24	1,11	cyclictest	0	-21	swapper/0	05:05:02	0
1553398	99	24	1,11	cyclictest	0	-21	swapper/0	05:05:01	0
1553411	99	23	4,16	cyclictest	1731989	-21	spawn	05:32:17	7
1553408	99	23	0,14	cyclictest	0	-21	swapper/6	06:33:48	6
1553407	99	23	1,18	cyclictest	0	-21	swapper/5	05:07:53	5
1553407	99	23	1,18	cyclictest	0	-21	swapper/5	05:07:52	5
1553407	99	23	0,20	cyclictest	1555657	-21	fstime	05:27:47	5
1553406	99	23	0,20	cyclictest	1555646	-21	fstime	05:26:57	4
1553402	99	23	0,20	cyclictest	0	-21	swapper/2	06:39:23	2
1553398	99	23	0,3	cyclictest	210	50	irq/124-46000000.ethernet-rx0	06:13:46	0
1553408	99	22	19,1	cyclictest	1596672	-21	spawn	05:31:16	6
1553408	99	22	18,2	cyclictest	2857296	-21	latency_hist	06:40:01	6
1553408	99	22	15,4	cyclictest	2855379	-21	sendmail	06:20:00	6
1553407	99	22	17,3	cyclictest	1554231	-21	sh	05:15:00	5
1553407	99	22	1,18	cyclictest	1713979	-21	spawn	05:31:59	5
1553407	99	22	1,18	cyclictest	0	-21	swapper/5	06:34:07	5
1553407	99	22	1,17	cyclictest	0	-21	swapper/5	06:26:43	5
1553407	99	22	0,19	cyclictest	0	-21	swapper/5	06:35:20	5
1553406	99	22	17,2	cyclictest	0	-21	swapper/4	06:20:00	4
1553399	99	22	1,18	cyclictest	1555184	-21	execl	05:20:58	1
1553398	99	22	1,10	cyclictest	0	-21	swapper/0	06:29:59	0
1553411	99	21	1,3	cyclictest	1554535	-21	latency_hist	05:15:01	7
1553408	99	21	16,2	cyclictest	0	-21	swapper/6	05:21:59	6
1553408	99	21	0,17	cyclictest	66	-21	rcuc/6	05:20:01	6
1553407	99	21	1,17	cyclictest	0	-21	swapper/5	06:15:31	5
1553407	99	21	1,17	cyclictest	0	-21	swapper/5	06:10:31	5
1553406	99	21	16,3	cyclictest	1624463	-21	spawn	05:31:24	4
1553406	99	21	1,17	cyclictest	0	-21	swapper/4	06:37:16	4
1553406	99	21	1,17	cyclictest	0	-21	swapper/4	05:08:38	4
1553406	99	21	1,17	cyclictest	0	-21	swapper/4	05:08:37	4
1553406	99	21	1,16	cyclictest	0	-21	swapper/4	05:20:20	4
1553403	99	21	1,17	cyclictest	0	-21	swapper/3	06:11:19	3
1553399	99	21	1,17	cyclictest	0	-21	swapper/1	06:16:47	1
1553399	99	21	1,17	cyclictest	0	-21	swapper/1	06:10:57	1
1553399	99	21	1,16	cyclictest	0	-21	swapper/1	05:15:30	1
1553398	99	21	1,4	cyclictest	1554797	-21	sh	05:20:00	0
1553398	99	21	1,16	cyclictest	0	-21	swapper/0	05:05:52	0
1553398	99	21	1,16	cyclictest	0	-21	swapper/0	05:05:52	0
1553411	99	20	1,16	cyclictest	0	-21	swapper/7	06:36:40	7
1553411	99	20	1,16	cyclictest	0	-21	swapper/7	05:20:09	7
1553411	99	20	1,16	cyclictest	0	-21	swapper/7	05:19:32	7
1553408	99	20	0,17	cyclictest	0	-21	swapper/6	06:27:45	6
1553408	99	20	0,17	cyclictest	0	-21	swapper/6	06:20:53	6
1553406	99	20	1,17	cyclictest	0	-21	swapper/4	06:33:00	4
1553403	99	20	1,16	cyclictest	0	-21	swapper/3	06:15:04	3
1553403	99	20	0,9	cyclictest	0	-21	swapper/3	06:33:50	3
1553403	99	20	0,18	cyclictest	210	50	irq/124-46000000.ethernet-rx0	05:23:53	3
1553403	99	20	0,17	cyclictest	0	-21	swapper/3	06:35:44	3
1553403	99	20	0,17	cyclictest	0	-21	swapper/3	06:25:44	3
1553403	99	20	0,17	cyclictest	0	-21	swapper/3	05:06:47	3
1553403	99	20	0,17	cyclictest	0	-21	swapper/3	05:06:47	3
1553399	99	20	1,16	cyclictest	0	-21	swapper/1	06:22:12	1
1553399	99	20	0,17	cyclictest	0	-21	swapper/1	06:30:15	1
1553399	99	20	0,17	cyclictest	0	-21	swapper/1	06:25:51	1
1553399	99	20	0,17	cyclictest	0	-21	swapper/1	05:26:29	1
1553399	99	20	0,17	cyclictest	0	-21	swapper/1	05:10:35	1
1553399	99	20	0,17	cyclictest	0	-21	swapper/1	05:06:07	1
1553399	99	20	0,17	cyclictest	0	-21	swapper/1	05:06:07	1
1553398	99	20	0,7	cyclictest	0	-21	swapper/0	06:32:08	0
1553411	99	19	13,4	cyclictest	811	-21	systemd-network	06:27:00	7
1553411	99	19	1,2	cyclictest	0	-21	swapper/7	05:05:02	7
1553411	99	19	1,2	cyclictest	0	-21	swapper/7	05:05:01	7
1553411	99	19	1,16	cyclictest	0	-21	swapper/7	06:31:35	7
1553408	99	19	0,17	cyclictest	0	-21	swapper/6	05:10:09	6
1553406	99	19	0,17	cyclictest	0	-21	swapper/4	06:26:29	4
1553406	99	19	0,17	cyclictest	0	-21	swapper/4	05:10:03	4
1553406	99	19	0,16	cyclictest	0	-21	swapper/4	06:20:13	4
1553406	99	19	0,16	cyclictest	0	-21	swapper/4	05:15:05	4
1553403	99	19	0,15	cyclictest	159	-1	pr/ttyS2	05:05:00	3
1553403	99	19	0,15	cyclictest	159	-1	pr/ttyS2	05:05:00	3
1553402	99	19	0,18	cyclictest	0	-21	swapper/2	06:25:21	2
1553399	99	19	0,17	cyclictest	0	-21	swapper/1	06:35:02	1
1553411	99	18	14,2	cyclictest	0	-21	swapper/7	06:20:04	7
1553411	99	18	0,15	cyclictest	0	-21	swapper/7	06:11:15	7
1553411	99	18	0,15	cyclictest	0	-21	swapper/7	05:07:00	7
1553411	99	18	0,15	cyclictest	0	-21	swapper/7	05:07:00	7
1553399	99	17	12,3	cyclictest	1553465	-21	latency_hist	05:05:01	1
1553399	99	17	12,3	cyclictest	1553465	-21	latency_hist	05:05:01	1
1553408	99	16	0,1	cyclictest	0	-21	swapper/6	05:05:02	6
1553408	99	16	0,1	cyclictest	0	-21	swapper/6	05:05:01	6
1553407	99	16	13,2	cyclictest	0	-21	swapper/5	05:05:01	5
1553407	99	16	13,2	cyclictest	0	-21	swapper/5	05:05:01	5
1553406	99	15	13,1	cyclictest	0	-21	swapper/4	05:05:01	4
1553406	99	15	13,1	cyclictest	0	-21	swapper/4	05:05:01	4
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17