OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #7, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system

Characteristics of the highest latencies:

System rack7slot2.osadl.org (updated Sun Nov 16, 2025 00:44:05)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
3160042	99	90	80,7	cyclictest	3253302	-21	kworker/u4:0+events_unbound	22:52:57	0
3160044	99	88	39,38	cyclictest	3203009	-21	aten_r7power_en	21:25:16	1
3160042	99	87	82,3	cyclictest	3300951	-21	kworker/u4:14+events_unbound	23:38:13	0
3160044	99	86	79,5	cyclictest	3137722	-21	kworker/u4:1+events_unbound	19:12:07	1
3160044	99	85	39,29	cyclictest	3323715	-21	timerandwakeup	23:55:42	1
3160044	99	83	31,28	cyclictest	0	-21	swapper/1	23:20:32	1
3160042	99	83	77,4	cyclictest	3236764	-21	kworker/u4:4+events_unbound	22:12:48	0
3160042	99	82	73,6	cyclictest	3269096	-21	kworker/u4:3+events_unbound	22:58:02	0
3160042	99	81	72,6	cyclictest	3240873	-21	kworker/u4:1+events_unbound	22:22:50	0
3160042	99	81	72,6	cyclictest	3191007	-21	kworker/u4:4+events_unbound	21:07:34	0
3160042	99	80	4,27	cyclictest	3195936	-21	ntpq	21:15:35	0
3160044	99	79	26,36	cyclictest	0	-21	swapper/1	22:35:21	1
3160042	99	79	72,5	cyclictest	3193223	-21	kworker/u4:10+events_unbound	21:13:14	0
3160042	99	79	31,46	cyclictest	3185019	-21	fschecks_count	20:45:22	0
3160044	99	77	35,22	cyclictest	3240045	-21	unixbench_multi	22:10:43	1
3160044	99	77	24,41	cyclictest	0	-21	swapper/1	22:25:33	1
3160042	99	77	36,38	cyclictest	3295103	-21	latency_hist	23:20:02	0
3160042	99	76	68,5	cyclictest	3201009	-21	kworker/u4:6+events_unbound	21:48:20	0
3160042	99	76	27,43	cyclictest	3331212	-21	ssh	00:05:32	0
3160044	99	75	65,8	cyclictest	3174255	-21	kworker/u4:13+events_unbound	20:32:23	1
3160044	99	75	45,27	cyclictest	0	-21	swapper/1	23:13:06	1
3160042	99	74	64,7	cyclictest	3149868	-21	kworker/u4:5+events_unbound	19:42:15	0
3160042	99	74	3,29	cyclictest	3283938	-21	ssh	23:05:31	0
3160042	99	74	20,6	cyclictest	3215390	-21	ssh	21:40:28	0
3160044	99	73	29,22	cyclictest	0	-21	swapper/1	23:25:45	1
3160042	99	72	62,7	cyclictest	3157500	-21	kworker/u4:4+events_unbound	19:47:56	0
3160044	99	71	65,4	cyclictest	3312894	-21	kworker/u4:0+events_unbound	23:53:46	1
3160044	99	71	62,6	cyclictest	3335528	-21	kworker/u4:3+events_unbound	00:18:18	1
3160044	99	71	29,21	cyclictest	0	-21	swapper/1	23:05:25	1
3160042	99	71	30,38	cyclictest	0	-21	swapper/0	00:30:38	0
3160044	99	70	35,32	cyclictest	0	-21	swapper/1	00:00:17	1
3160042	99	70	24,4	cyclictest	0	-21	swapper/0	00:15:23	0
3160044	99	69	4,37	cyclictest	3202459	-21	ssh	21:24:45	1
3160042	99	69	60,7	cyclictest	3312894	-21	kworker/u4:0+events_unbound	00:13:50	0
3160042	99	69	60,6	cyclictest	3157501	-21	kworker/u4:6+events_unbound	19:27:12	0
3160042	99	69	34,32	cyclictest	3231856	-21	timerwakeupswit	22:00:42	0
3160044	99	68	35,30	cyclictest	3236100	-21	/usr/sbin/munin	22:05:43	1
3160044	99	68	3,36	cyclictest	3334603	-21	ntpq	00:10:34	1
3160044	99	68	2,41	cyclictest	3313133	-21	ssh	23:42:22	1
3160044	99	68	2,39	cyclictest	3332163	-21	ssh	00:06:28	1
3160044	99	68	1,43	cyclictest	3294683	-21	ssh	23:19:30	1
3160042	99	68	32,33	cyclictest	3203948	-21	switchtime	21:25:38	0
3160044	99	67	1,41	cyclictest	3318213	-21	ssh	23:49:19	1
3160042	99	67	35,28	cyclictest	0	-21	swapper/0	23:00:26	0
3160044	99	66	35,28	cyclictest	3279547	-21	sort	23:00:20	1
3160042	99	66	57,6	cyclictest	3171582	-21	kworker/u4:10+events_unbound	20:27:22	0
3160042	99	66	32,32	cyclictest	3199654	-21	ntp_kernel_err	21:20:25	0
3160042	99	66	3,11	cyclictest	3182009	-21	modprobe	20:35:00	0
3160042	99	66	29,34	cyclictest	53	-21	kswapd0	00:20:34	0
3160042	99	65	55,7	cyclictest	3179418	-21	kworker/u4:4+events_unbound	20:23:04	0
3160042	99	65	3,10	cyclictest	3191061	-21	diskmemload	21:35:00	0
3160042	99	65	3,10	cyclictest	3191061	-21	diskmemload	21:35:00	0
3160042	99	65	30,32	cyclictest	3312003	-21	timerandwakeup	23:40:40	0
3160042	99	65	28,33	cyclictest	0	-21	swapper/0	21:00:27	0
3160042	99	65	25,7	cyclictest	0	-21	swapper/0	23:30:34	0
3160042	99	65	18,42	cyclictest	0	-21	swapper/0	23:30:00	0
3160044	99	64	56,6	cyclictest	3253304	-21	kworker/u4:6+events_unbound	22:48:33	1
3160044	99	64	53,7	cyclictest	53	-21	kswapd0	00:25:22	1
3160044	99	64	3,38	cyclictest	3192881	-21	ssh	21:11:11	1
3160044	99	64	1,39	cyclictest	3271827	-21	ssh	22:50:26	1
3160042	99	64	29,33	cyclictest	3298934	-21	modprobe	23:25:00	0
3160042	99	64	29,32	cyclictest	0	-21	swapper/0	19:40:00	0
3160042	99	64	29,32	cyclictest	0	-21	swapper/0	19:40:00	0
3160042	99	64	27,34	cyclictest	0	-21	swapper/0	22:05:44	0
3160044	99	63	54,7	cyclictest	3212917	-21	kworker/u4:0+events_unbound	21:42:41	1
3160044	99	63	29,31	cyclictest	0	-21	swapper/1	23:30:26	1
3160044	99	63	1,56	cyclictest	3161479	-21	apt-get	19:15:16	1
3160044	99	63	13,29	cyclictest	3211398	-21	memory	21:35:23	1
3160042	99	63	5,13	cyclictest	3162589	-21	modprobe	19:20:00	0
3160042	99	63	29,31	cyclictest	3318559	-21	cron	23:50:00	0
3160042	99	63	28,32	cyclictest	0	-21	swapper/0	19:20:37	0
3160042	99	63	24,8	cyclictest	3288466	-21	sed	23:10:42	0
3160044	99	62	57,3	cyclictest	3179418	-21	kworker/u4:4+events_unbound	20:22:22	1
3160044	99	62	53,6	cyclictest	3157501	-21	kworker/u4:6+events_unbound	19:27:51	1
3160044	99	62	33,26	cyclictest	3240875	-21	kworker/u4:5+rpciod	22:30:39	1
3160044	99	62	30,29	cyclictest	3231654	-21	sh	22:00:40	1
3160044	99	62	1,37	cyclictest	3310515	-21	ssh	23:39:48	1
3160044	99	62	12,29	cyclictest	0	-21	swapper/1	21:45:34	1
3160042	99	62	52,7	cyclictest	3170299	-21	kworker/u4:0+events_unbound	19:57:18	0
3160042	99	62	29,30	cyclictest	0	-21	swapper/0	22:25:32	0
3160042	99	62	22,5	cyclictest	3350578	-21	apt-get	00:35:17	0
3160042	99	61	29,29	cyclictest	0	-21	swapper/0	20:00:35	0
3160042	99	61	28,31	cyclictest	3227272	-21	kernelversion	21:55:25	0
3160042	99	61	27,31	cyclictest	3236764	-21	kworker/u4:4-rpciod	22:30:33	0
3160042	99	61	13,11	cyclictest	3262831	-21	sh	22:40:00	0
3160044	99	60	37,21	cyclictest	3180753	-21	idleruntime-cro	20:30:00	1
3160044	99	60	2,35	cyclictest	3208919	-21	ssh	21:32:08	1
3160044	99	60	2,35	cyclictest	3208919	-21	ssh	21:32:08	1
3160044	99	60	1,34	cyclictest	3177015	-21	/usr/sbin/munin	20:15:24	1
3160042	99	60	5,13	cyclictest	3171625	-21	modprobe	19:55:00	0
3160042	99	60	4,23	cyclictest	3211435	-21	ntpq	21:35:24	0
3160042	99	60	26,31	cyclictest	0	-21	swapper/0	20:40:19	0
3160042	99	60	25,31	cyclictest	0	-21	swapper/0	20:05:20	0
3160042	99	60	25,30	cyclictest	3236764	-21	kworker/u4:4-rpciod	22:45:39	0
3160044	99	59	51,6	cyclictest	3245035	-21	kworker/u4:3+events_unbound	22:18:27	1
3160044	99	59	51,6	cyclictest	3245035	-21	kworker/u4:3+events_unbound	22:18:27	1
3160044	99	59	3,49	cyclictest	3167216	-21	ntpq	19:35:29	1
3160044	99	59	3,49	cyclictest	3167216	-21	ntpq	19:35:29	1
3160044	99	59	2,52	cyclictest	3176346	-21	ntpq	20:10:31	1
3160042	99	59	4,17	cyclictest	3325517	-21	sshd	23:58:46	0
3160042	99	59	3,23	cyclictest	3327265	-21	ntpq	00:00:33	0
3160042	99	59	27,4	cyclictest	3243687	-21	ntp_states	22:15:33	0
3160042	99	59	27,4	cyclictest	3243687	-21	ntp_states	22:15:33	0
3160042	99	59	1,20	cyclictest	3263849	-21	ntpq	22:40:29	0
3160044	99	58	26,30	cyclictest	0	-21	swapper/1	21:55:28	1
3160044	99	58	1,35	cyclictest	3248715	-21	ssh	22:21:28	1
3160044	99	58	11,42	cyclictest	1060	-21	snmpd	20:00:11	1
3160044	99	58	10,36	cyclictest	3265110	-21	ssh	22:41:53	1
3160042	99	57	26,5	cyclictest	3335528	-21	kworker/u4:3-rpciod	00:25:33	0
3160042	99	57	24,5	cyclictest	0	-21	swapper/0	21:50:30	0
3160042	99	57	23,6	cyclictest	0	-21	swapper/0	19:10:40	0
3160044	99	56	44,8	cyclictest	0	-21	swapper/1	21:50:14	1
3160044	99	56	33,20	cyclictest	0	-21	swapper/1	19:22:48	1
3160044	99	56	27,26	cyclictest	3190369	-21	gltestperf	21:05:21	1
3160044	99	56	2,50	cyclictest	3171120	-21	ntpq	19:50:30	1
3160042	99	56	46,8	cyclictest	3187147	-21	systemd	20:53:09	0
3160044	99	55	4,45	cyclictest	3341704	-21	ssh	00:20:34	1
3160044	99	55	3,47	cyclictest	3186655	-21	ntpq	20:50:29	1
3160044	99	55	32,20	cyclictest	3347942	-21	fschecks_time	00:30:23	1
3160044	99	55	2,50	cyclictest	3168560	-21	ntpq	19:40:33	1
3160044	99	55	1,30	cyclictest	3183499	-21	apt-get	20:40:15	1
3160044	99	54	2,46	cyclictest	3195914	-21	ntpq	21:15:33	1
3160044	99	54	2,32	cyclictest	3352898	-21	ssh	00:39:37	1
3160044	99	54	19,32	cyclictest	3185341	-21	ntpq	20:45:29	1
3160042	99	54	4,14	cyclictest	3320118	-21	users	23:50:41	0
3160044	99	53	2,47	cyclictest	3275319	-21	ssh	22:55:20	1
3160044	99	53	2,47	cyclictest	3189292	-21	ntpq	21:00:33	1
3160042	99	53	45,6	cyclictest	3171582	-21	kworker/u4:10+events_unbound	20:17:21	0
3160044	99	52	2,47	cyclictest	3165341	-21	/usr/sbin/munin	19:30:41	1
3160044	99	52	1,49	cyclictest	3175066	-21	ntpq	20:05:30	1
3160044	99	52	1,48	cyclictest	3182755	-21	ntpq	20:35:28	1
3160044	99	52	1,47	cyclictest	3169211	-21	/usr/sbin/munin	19:45:42	1
3160042	99	52	23,6	cyclictest	3165905	-21	ntp_kernel_err	19:30:24	0
3160044	99	51	2,45	cyclictest	3172347	-21	ntpq	19:55:29	1
3160044	99	50	23,24	cyclictest	0	-21	swapper/1	20:55:22	1
3160042	99	48	39,6	cyclictest	3162548	-21	kworker/u4:14+events_unbound	20:13:04	0
3160044	99	47	1,40	cyclictest	3160141	-21	latency_hist	19:10:01	1
3160042	99	45	35,8	cyclictest	3187704	-21	gltestperf	20:55:25	0
3160042	99	39	30,6	cyclictest	0	-21	swapper/0	20:40:00	0
3160042	99	34	23,6	cyclictest	0	-21	swapper/0	19:10:01	0
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17