OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #7, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system

Characteristics of the 792 highest latencies:

System rack7slot2.osadl.org (updated Mon Mar 23, 2026 00:44:11)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
4173842	99	113	104,7	cyclictest	158950	-21	kworker/u4:10+events_unbound	00:13:26	0
4173842	99	110	102,6	cyclictest	158948	-21	kworker/u4:6+events_unbound	00:08:25	0
4173845	99	107	99,5	cyclictest	79585	-21	kworker/u4:9+events_unbound	22:41:42	1
4173842	99	104	97,5	cyclictest	27008	-21	kworker/u4:0+events_unbound	22:06:36	0
4173842	99	103	93,7	cyclictest	53	-21	kswapd0	23:15:08	0
4173845	99	99	91,5	cyclictest	189632	-21	kworker/u4:3+events_unbound	00:33:59	1
4173845	99	96	88,6	cyclictest	158950	-21	kworker/u4:10+events_unbound	00:18:49	1
4173842	99	96	91,3	cyclictest	4493	-21	kworker/u4:17+events_unbound	20:46:28	0
4173842	99	95	87,6	cyclictest	71273	-21	kworker/u4:0+events_unbound	22:31:42	0
4173842	99	94	80,6	cyclictest	48951	-21	kworker/u4:1+events_unbound	23:10:03	0
4173842	99	94	80,6	cyclictest	48951	-21	kworker/u4:1+events_unbound	23:10:03	0
4173842	99	93	85,6	cyclictest	4189400	-21	kworker/u4:4+events_unbound	20:16:28	0
4173845	99	91	86,3	cyclictest	48951	-21	kworker/u4:1+events_unbound	22:56:44	1
4173842	99	91	86,3	cyclictest	136691	-21	kworker/u4:5+events_unbound	23:41:53	0
4173842	99	91	79,5	cyclictest	4190678	-21	kworker/u4:9+events_unbound	21:01:29	0
4173842	99	91	79,5	cyclictest	4190678	-21	kworker/u4:9+events_unbound	21:01:29	0
4173842	99	90	66,19	cyclictest	53	-21	kswapd0	21:45:07	0
4173842	99	88	80,6	cyclictest	123939	-21	kworker/u4:11+events_unbound	23:21:50	0
4173842	99	87	79,6	cyclictest	35413	-21	kworker/u4:5+events_unbound	21:36:32	0
4173845	99	85	77,6	cyclictest	79585	-21	kworker/u4:9+events_unbound	22:46:44	1
4173842	99	85	73,6	cyclictest	35414	-21	kworker/u4:8+events_unbound	22:16:36	0
4173842	99	82	75,5	cyclictest	31534	-21	kworker/u4:4+events_unbound	22:01:35	0
4173842	99	82	74,6	cyclictest	3190	-21	kworker/u4:5+events_unbound	21:26:30	0
4173842	99	81	73,6	cyclictest	4193254	-21	kworker/u4:7+events_unbound	20:36:29	0
4173842	99	73	66,5	cyclictest	4175070	-21	kworker/u4:15+events_unbound	19:16:20	0
4173842	99	73	61,7	cyclictest	53	-21	kswapd0	23:50:00	0
4173845	99	72	61,8	cyclictest	185144	-21	kworker/u4:2+events_unbound	00:30:01	1
4173845	99	71	63,6	cyclictest	53	-21	kswapd0	00:05:01	1
4173842	99	71	63,5	cyclictest	53	-21	kswapd0	23:10:01	0
4173845	99	69	61,6	cyclictest	40641	-21	kworker/u4:12+events_unbound	21:51:33	1
4173845	99	69	61,6	cyclictest	106165	-21	kworker/u4:0+events_unbound	23:11:50	1
4173845	99	69	61,6	cyclictest	106165	-21	kworker/u4:0+events_unbound	23:11:50	1
4173842	99	68	61,5	cyclictest	4177737	-21	kworker/u4:0+events_unbound	19:56:26	0
4173845	99	67	59,6	cyclictest	4185559	-21	kworker/u4:8+events_unbound	20:10:00	1
4173845	99	66	59,5	cyclictest	35414	-21	kworker/u4:8+events_unbound	22:26:39	1
4173842	99	65	57,6	cyclictest	4189400	-21	kworker/u4:4+events_unbound	20:11:26	0
4173842	99	65	52,10	cyclictest	15	1	ktimers/0	21:50:29	0
4173845	99	63	57,4	cyclictest	4190673	-21	kworker/u4:2+events_unbound	20:30:00	1
4173845	99	63	56,5	cyclictest	4176341	-21	kworker/u4:4+events_unbound	19:20:01	1
4173845	99	62	54,6	cyclictest	101718	-21	kworker/u4:9+events_unbound	23:06:46	1
4173842	99	62	2,28	cyclictest	4179043	-21	apt-get	19:30:10	0
4173845	99	61	55,4	cyclictest	7032	-21	kworker/u4:4+events_unbound	20:55:00	1
4173842	99	61	48,8	cyclictest	53	-21	kswapd0	22:25:01	0
4173842	99	61	43,15	cyclictest	101718	-21	kworker/u4:9+events_unbound	22:57:43	0
4173845	99	60	53,5	cyclictest	145656	-21	kworker/u4:11+events_unbound	23:53:06	1
4173845	99	60	44,5	cyclictest	4494	-21	kworker/u4:18+events_unbound	22:11:37	1
4173842	99	60	55,3	cyclictest	158950	-21	kworker/u4:10+events_unbound	00:19:00	0
4173842	99	60	48,10	cyclictest	53	-21	kswapd0	22:25:07	0
4173842	99	60	44,6	cyclictest	101718	-21	kworker/u4:9+events_unbound	23:04:59	0
4173845	99	59	52,5	cyclictest	36087	-21	kworker/u4:9+events_unbound	21:40:02	1
4173845	99	59	51,6	cyclictest	4172560	-21	kworker/u4:3+events_unbound	19:46:23	1
4173845	99	58	51,5	cyclictest	106166	-21	kworker/u4:2+events_unbound	23:01:45	1
4173845	99	58	49,6	cyclictest	4177741	-21	kworker/u4:8+events_unbound	19:25:00	1
4173842	99	58	51,5	cyclictest	4189400	-21	kworker/u4:4+events_unbound	20:26:27	0
4173842	99	58	51,5	cyclictest	4173774	-21	kworker/u4:9+events_unbound	19:11:19	0
4173842	99	58	49,7	cyclictest	53	-21	kswapd0	00:25:01	0
4173845	99	57	53,3	cyclictest	9603	-21	kworker/u4:23-rpciod	21:27:05	1
4173842	99	57	49,6	cyclictest	4177738	-21	kworker/u4:1+events_unbound	19:41:22	0
4173845	99	56	47,7	cyclictest	166111	-21	ntp_states	00:05:32	1
4173842	99	56	49,5	cyclictest	4193254	-21	kworker/u4:7+events_unbound	20:31:28	0
4173842	99	56	48,6	cyclictest	136689	-21	kworker/u4:1+events_unbound	23:51:59	0
4173842	99	56	46,6	cyclictest	187374	-21	/usr/sbin/munin	00:30:16	0
4173842	99	56	24,23	cyclictest	256	50	irq/23-42890000.ethernet	19:35:09	0
4173845	99	55	47,6	cyclictest	3196	-21	kworker/u4:14+events_unbound	21:21:31	1
4173842	99	55	9,36	cyclictest	157306	-21	ssh	23:55:31	0
4173842	99	55	44,7	cyclictest	11487	-21	/usr/sbin/munin	21:10:17	0
4173845	99	54	47,5	cyclictest	40642	-21	kworker/u4:13+events_unbound	21:56:34	1
4173845	99	54	47,5	cyclictest	40642	-21	kworker/u4:13+events_unbound	21:56:34	1
4173845	99	53	45,6	cyclictest	4485	-21	systemd	20:41:27	1
4173845	99	53	10,38	cyclictest	11881	-21	ssh	21:10:24	1
4173842	99	53	39,7	cyclictest	256	50	irq/23-42890000.ethernet	00:00:25	0
4173842	99	53	15,11	cyclictest	17	1	rcu_preempt	00:35:27	0
4173842	99	53	1,21	cyclictest	4176342	-21	kworker/u4:5+nfsiod	19:20:10	0
4173845	99	52	45,5	cyclictest	57660	-21	sshd	22:01:34	1
4173845	99	52	44,6	cyclictest	4172560	-21	kworker/u4:3+events_unbound	19:31:22	1
4173845	99	52	42,7	cyclictest	130563	-21	fschecks_time	23:25:21	1
4173842	99	52	42,7	cyclictest	101532	-21	ssh	22:51:43	0
4173845	99	51	43,6	cyclictest	167238	-21	kworker/u4:12+events_unbound	00:21:07	1
4173842	99	51	43,6	cyclictest	4185559	-21	kworker/u4:8+events_unbound	20:20:03	0
4173842	99	51	36,12	cyclictest	38446	-21	fschecks_count	21:40:20	0
4173845	99	50	45,4	cyclictest	141158	-21	kworker/u4:4+events_unbound	23:41:59	1
4173845	99	50	43,5	cyclictest	4177743	-21	kworker/u4:11+events_unbound	20:01:25	1
4173845	99	50	43,5	cyclictest	160611	-21	sh	00:00:01	1
4173845	99	50	42,6	cyclictest	101717	-21	kworker/u4:7+events_unbound	23:20:01	1
4173845	99	50	41,7	cyclictest	136691	-21	kworker/u4:5+events_unbound	23:35:37	1
4173845	99	50	41,7	cyclictest	136691	-21	kworker/u4:5+events_unbound	23:35:37	1
4173845	99	50	40,7	cyclictest	42271	-21	apt-get	21:45:08	1
4173845	99	50	33,6	cyclictest	115104	-21	kworker/u4:3+events_unbound	23:31:50	1
4173845	99	50	33,6	cyclictest	115104	-21	kworker/u4:3+events_unbound	23:31:50	1
4173842	99	50	37,7	cyclictest	31534	-21	kworker/u4:4+events_unbound	21:34:41	0
4173845	99	49	42,5	cyclictest	4178829	-21	timerwakeupswit	19:25:37	1
4173845	99	49	41,5	cyclictest	86203	-21	latency_hist	22:35:01	1
4173842	99	49	43,5	cyclictest	17858	-21	kworker/u4:9+events_unbound	21:17:00	0
4173842	99	49	42,5	cyclictest	39975	-21	kworker/u4:10+events_unbound	21:56:00	0
4173842	99	49	42,5	cyclictest	39975	-21	kworker/u4:10+events_unbound	21:56:00	0
4173842	99	49	42,5	cyclictest	183517	-21	/usr/sbin/munin	00:25:26	0
4173842	99	49	42,5	cyclictest	130534	-21	diskmemload	23:25:21	0
4173842	99	49	41,6	cyclictest	3192	-21	kworker/u4:8+events_unbound	21:21:59	0
4173842	99	49	40,7	cyclictest	4184751	-21	fschecks_count	19:50:19	0
4173842	99	49	27,19	cyclictest	0	-21	swapper/0	19:25:35	0
4173845	99	48	42,4	cyclictest	27013	-21	kworker/u4:6+events_unbound	21:30:20	1
4173845	99	48	41,5	cyclictest	8764	-21	fschecks_count	21:00:20	1
4173845	99	48	41,5	cyclictest	8764	-21	fschecks_count	21:00:20	1
4173845	99	48	41,5	cyclictest	126941	-21	timerwakeupswit	23:20:41	1
4173845	99	48	40,5	cyclictest	0	-21	swapper/1	20:10:23	1
4173845	99	48	38,7	cyclictest	77542	-21	latency_hist	22:25:02	1
4173845	99	47	41,4	cyclictest	53	-21	kswapd0	22:20:01	1
4173845	99	47	39,5	cyclictest	4174407	-21	gltestperf	19:10:22	1
4173842	99	47	40,6	cyclictest	101717	-21	kworker/u4:7+events_unbound	23:33:42	0
4173842	99	47	40,6	cyclictest	101717	-21	kworker/u4:7+events_unbound	23:33:42	0
4173842	99	47	39,6	cyclictest	139052	-21	aten_r7power_en	23:35:15	0
4173842	99	47	39,6	cyclictest	139052	-21	aten_r7power_en	23:35:15	0
4173842	99	47	36,8	cyclictest	0	-21	swapper/0	22:40:24	0
4173845	99	46	40,4	cyclictest	170757	-21	threads	00:10:38	1
4173845	99	46	39,5	cyclictest	4181755	-21	kworker/u4:13	19:40:01	1
4173845	99	46	38,6	cyclictest	192378	-21	/usr/sbin/munin	00:35:28	1
4173845	99	46	38,5	cyclictest	0	-21	swapper/1	22:35:22	1
4173842	99	46	39,5	cyclictest	3192	-21	kworker/u4:8+events_unbound	20:55:01	0
4173842	99	46	36,7	cyclictest	86567	-21	aten_r7power_po	22:35:15	0
4173845	99	45	39,4	cyclictest	3190	-21	kworker/u4:5+events_unbound	21:20:02	1
4173845	99	45	37,6	cyclictest	64277	-21	latency_hist	22:10:01	1
4173845	99	45	37,6	cyclictest	1866	-21	munin-run	20:35:01	1
4173842	99	45	37,6	cyclictest	4177738	-21	kworker/u4:1+events_unbound	19:45:03	0
4173842	99	45	36,7	cyclictest	64675	-21	aten_r7power_po	22:10:17	0
4173845	99	44	37,5	cyclictest	100273	-21	ssh	22:50:28	1
4173845	99	44	35,7	cyclictest	4186902	-21	sendmail	20:00:01	1
4173842	99	44	37,5	cyclictest	4190674	-21	kworker/u4:3+events_unbound	20:40:42	0
4173842	99	44	35,7	cyclictest	96221	-21	sendmail_mailqu	22:45:38	0
4173845	99	43	36,5	cyclictest	33970	-21	fschecks_time	21:35:18	1
4173845	99	43	34,6	cyclictest	5892	-21	latency_hist	20:50:06	1
4173842	99	43	37,4	cyclictest	10573	-21	cat	21:05:37	0
4173842	99	43	33,7	cyclictest	4189060	-21	wc	20:05:36	0
4173842	99	43	21,19	cyclictest	0	-21	swapper/0	20:00:19	0
4173845	99	42	34,5	cyclictest	0	-21	swapper/1	19:50:27	1
4173845	99	42	31,9	cyclictest	148836	-21	sendmail_mailtr	23:45:43	1
4173845	99	41	33,6	cyclictest	3376	-21	latency_hist	20:40:02	1
4173845	99	40	31,6	cyclictest	0	-21	swapper/1	20:20:38	1
4173842	99	40	24,7	cyclictest	4492	-21	kworker/u4:16+events_unbound	20:51:37	0
4173845	99	38	29,7	cyclictest	11050	-21	latency_hist	21:10:01	1
4173845	99	37	31,4	cyclictest	0	-21	swapper/1	20:15:27	1
4173845	99	37	28,6	cyclictest	0	-21	swapper/1	19:45:00	1
4173845	99	37	25,10	cyclictest	0	-21	swapper/1	21:00:01	1
4173842	99	33	4,9	cyclictest	4173818	-21	munin-run	19:09:59	0
4173845	99	31	17,5	cyclictest	17	1	rcu_preempt	19:10:00	1
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17