OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #7, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system

Characteristics of the 792 highest latencies:

System rack7slot2.osadl.org (updated Wed Jun 03, 2026 12:44:09)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
3118747	99	115	101,8	cyclictest	3224165	-21	kworker/u4:9+events_unbound	10:39:59	0
3118748	99	104	96,6	cyclictest	3105871	-21	kworker/u4:1+events_unbound	07:34:25	1
3118747	99	98	90,6	cyclictest	3215731	-21	kworker/u4:13+events_unbound	11:35:06	0
3118747	99	97	87,7	cyclictest	3175224	-21	kworker/u4:0+events_unbound	09:39:49	0
3118748	99	91	85,4	cyclictest	3224167	-21	kworker/u4:15+events_unbound	10:34:57	1
3118747	99	90	83,5	cyclictest	3175227	-21	kworker/u4:6+events_unbound	10:17:00	0
3118747	99	87	77,7	cyclictest	3224162	-21	kworker/u4:4+events_unbound	10:35:00	0
3118747	99	86	79,5	cyclictest	3129008	-21	kworker/u4:10+events_unbound	08:09:38	0
3118748	99	84	74,7	cyclictest	3121235	-21	kworker/u4:5+events_unbound	08:20:00	1
3118747	99	83	76,5	cyclictest	3254942	-21	kworker/u4:0+events_unbound	11:15:03	0
3118747	99	82	74,6	cyclictest	3140938	-21	kworker/u4:16+events_unbound	08:49:42	0
3118747	99	82	74,6	cyclictest	3140938	-21	kworker/u4:16+events_unbound	08:49:42	0
3118747	99	79	66,6	cyclictest	3140934	-21	kworker/u4:12+events_unbound	08:44:41	0
3118747	99	77	65,5	cyclictest	3116151	-21	kworker/u4:13+events_unbound	07:24:25	0
3118747	99	77	58,8	cyclictest	256	50	irq/23-42890000.ethernet	08:20:12	0
3118748	99	76	35,37	cyclictest	0	-21	swapper/1	09:40:36	1
3118747	99	75	69,4	cyclictest	3197451	-21	kworker/u4:2+events_unbound	10:04:55	0
3118747	99	73	64,7	cyclictest	3229358	-21	kworker/u4:1+events_unbound	10:44:59	0
3118748	99	72	41,27	cyclictest	3126752	-21	aten_r7power_en	07:40:21	1
3118747	99	72	68,3	cyclictest	3129011	-21	kworker/u4:15+events_unbound	07:49:32	0
3118748	99	70	35,33	cyclictest	3235865	-21	fschecks_count	10:45:17	1
3118747	99	70	65,3	cyclictest	3122546	-21	kworker/u4:14+events_unbound	07:34:27	0
3118747	99	70	62,6	cyclictest	3121236	-21	kworker/u4:7+events_unbound	07:54:32	0
3118748	99	69	62,5	cyclictest	3286269	-21	kworker/u4:9+events_unbound	11:45:09	1
3118748	99	68	6,12	cyclictest	3232022	-21	ssh	10:40:33	1
3118748	99	68	35,30	cyclictest	3131372	-21	timerwakeupswit	07:55:37	1
3118748	99	68	32,32	cyclictest	3138445	-21	latency_hist	08:25:01	1
3118748	99	68	31,33	cyclictest	0	-21	swapper/1	08:00:04	1
3118748	99	68	31,33	cyclictest	0	-21	swapper/1	08:00:04	1
3118747	99	68	60,6	cyclictest	3121237	-21	kworker/u4:8+events_unbound	07:19:25	0
3118748	99	67	36,25	cyclictest	0	-21	swapper/1	11:20:35	1
3118748	99	67	33,31	cyclictest	0	-21	swapper/1	11:55:26	1
3118748	99	67	26,38	cyclictest	0	-21	swapper/1	12:05:30	1
3118748	99	67	1,58	cyclictest	3311127	-21	ssh	12:10:27	1
3118748	99	66	33,31	cyclictest	3302617	-21	sendmail_mailqu	12:00:36	1
3118748	99	66	1,15	cyclictest	1060	-21	sshd	12:27:39	1
3118747	99	66	44,12	cyclictest	3220297	-21	kworker/u4:0+events_unbound	10:28:20	0
3118748	99	65	33,29	cyclictest	3151346	-21	timerandwakeup	09:10:38	1
3118748	99	65	33,29	cyclictest	3151346	-21	timerandwakeup	09:10:37	1
3118748	99	65	23,5	cyclictest	3208792	-21	ssh	10:15:12	1
3118747	99	65	51,12	cyclictest	3170643	-21	kworker/u4:10+events_unbound	09:41:45	0
3118748	99	64	37,24	cyclictest	3173860	-21	switchtime	09:35:37	1
3118748	99	64	34,28	cyclictest	3315207	-21	fschecks_time	12:15:19	1
3118748	99	64	34,27	cyclictest	0	-21	swapper/1	11:00:42	1
3118748	99	64	33,29	cyclictest	3190543	-21	latency_hist	09:55:01	1
3118748	99	64	32,29	cyclictest	3283851	-21	latency_hist	11:40:02	1
3118748	99	64	29,31	cyclictest	0	-21	swapper/1	11:05:27	1
3118748	99	64	28,32	cyclictest	0	-21	swapper/1	11:45:01	1
3118748	99	64	28,31	cyclictest	0	-21	swapper/1	08:05:24	1
3118748	99	64	26,34	cyclictest	0	-21	swapper/1	12:20:17	1
3118748	99	63	33,27	cyclictest	0	-21	swapper/1	10:35:09	1
3118748	99	63	30,29	cyclictest	0	-21	swapper/1	10:15:02	1
3118748	99	63	21,8	cyclictest	53	-21	kswapd0	11:50:32	1
3118747	99	63	2,20	cyclictest	3131860	-21	apt-get	08:00:20	0
3118747	99	63	2,20	cyclictest	3131860	-21	apt-get	08:00:19	0
3118748	99	62	30,28	cyclictest	0	-21	swapper/1	09:25:30	1
3118748	99	62	29,29	cyclictest	0	-21	swapper/1	12:30:28	1
3118748	99	62	24,32	cyclictest	0	-21	swapper/1	09:00:15	1
3118747	99	62	54,6	cyclictest	3325961	-21	kworker/u4:5+events_unbound	12:30:18	0
3118747	99	62	49,9	cyclictest	53	-21	kswapd0	12:05:09	0
3118748	99	61	30,28	cyclictest	0	-21	swapper/1	09:30:26	1
3118748	99	61	28,29	cyclictest	0	-21	swapper/1	09:45:23	1
3118748	99	61	27,31	cyclictest	3119918	-21	unixbench_singl	07:10:38	1
3118748	99	61	25,34	cyclictest	0	-21	swapper/1	10:55:15	1
3118748	99	61	25,32	cyclictest	0	-21	swapper/1	08:35:39	1
3118748	99	61	25,32	cyclictest	0	-21	swapper/1	08:25:18	1
3118748	99	60	4,20	cyclictest	3199843	-21	ssh	10:05:15	1
3118748	99	60	3,17	cyclictest	3276047	-21	ntpq	11:30:34	1
3118748	99	60	30,27	cyclictest	0	-21	swapper/1	09:15:22	1
3118748	99	60	28,30	cyclictest	3272187	-21	vmstat	11:25:43	1
3118748	99	60	24,33	cyclictest	0	-21	swapper/1	08:55:36	1
3118748	99	60	23,34	cyclictest	0	-21	swapper/1	12:35:29	1
3118747	99	60	52,6	cyclictest	3215731	-21	kworker/u4:13+events_unbound	11:25:16	0
3118747	99	60	52,6	cyclictest	3121236	-21	kworker/u4:7+events_unbound	07:29:24	0
3118748	99	59	5,14	cyclictest	3258343	-21	ntpq	11:10:31	1
3118748	99	59	31,25	cyclictest	0	-21	swapper/1	10:00:17	1
3118748	99	59	30,26	cyclictest	0	-21	swapper/1	08:30:25	1
3118748	99	59	26,29	cyclictest	0	-21	swapper/1	10:20:21	1
3118748	99	59	25,30	cyclictest	0	-21	swapper/1	10:50:32	1
3118748	99	59	24,32	cyclictest	0	-21	swapper/1	07:20:00	1
3118748	99	59	23,31	cyclictest	0	-21	swapper/1	10:30:02	1
3118747	99	59	52,5	cyclictest	3294570	-21	kworker/u4:2+events_unbound	12:00:00	0
3118747	99	59	41,6	cyclictest	3197454	-21	kworker/u4:5+events_unbound	10:46:45	0
3118748	99	58	29,25	cyclictest	3215731	-21	kworker/u4:13-xprtiod	11:15:36	1
3118748	99	58	26,30	cyclictest	3159743	-21	grep	09:20:19	1
3118747	99	58	48,7	cyclictest	3135769	-21	acpi	08:15:13	0
3118747	99	58	48,7	cyclictest	3135769	-21	acpi	08:15:12	0
3118747	99	58	46,7	cyclictest	3215731	-21	kworker/u4:13+events_unbound	11:13:45	0
3118748	99	57	5,15	cyclictest	3135695	-21	munin-run	08:15:06	1
3118748	99	57	5,15	cyclictest	3135695	-21	munin-run	08:15:06	1
3118748	99	57	26,29	cyclictest	0	-21	swapper/1	08:44:39	1
3118748	99	57	18,9	cyclictest	0	-21	swapper/1	07:45:22	1
3118748	99	56	28,26	cyclictest	0	-21	swapper/1	08:45:36	1
3118748	99	56	28,26	cyclictest	0	-21	swapper/1	08:45:36	1
3118747	99	56	48,5	cyclictest	3323970	-21	/usr/sbin/munin	12:25:18	0
3118747	99	56	48,5	cyclictest	3316930	-21	kworker/u4:1+events_unbound	12:21:00	0
3118748	99	55	26,27	cyclictest	0	-21	swapper/1	07:35:20	1
3118748	99	55	23,29	cyclictest	0	-21	swapper/1	07:29:59	1
3118747	99	55	47,6	cyclictest	3179827	-21	kworker/u4:9+events_unbound	09:49:50	0
3118747	99	55	47,6	cyclictest	3161470	-21	kworker/u4:10+events_unbound	09:23:45	0
3118748	99	54	23,28	cyclictest	0	-21	swapper/1	09:55:17	1
3118748	99	54	20,8	cyclictest	0	-21	swapper/1	08:50:18	1
3118747	99	54	47,5	cyclictest	3156853	-21	kworker/u4:0+events_unbound	09:17:59	0
3118747	99	54	44,6	cyclictest	0	-21	swapper/0	11:00:39	0
3118747	99	54	37,15	cyclictest	3336703	-21	sendmail-msp	12:40:01	0
3118748	99	53	21,28	cyclictest	0	-21	swapper/1	09:05:36	1
3118747	99	53	46,5	cyclictest	3211166	-21	kworker/u4:2+events_unbound	10:23:46	0
3118747	99	53	45,6	cyclictest	3179827	-21	kworker/u4:9+events_unbound	09:50:59	0
3118747	99	52	44,6	cyclictest	3286269	-21	kworker/u4:9+events_unbound	11:46:00	0
3118747	99	52	42,7	cyclictest	3310716	-21	/usr/sbin/munin	12:10:19	0
3118747	99	51	8,21	cyclictest	3149920	-21	diskmemload	09:25:28	0
3118747	99	51	44,5	cyclictest	3286269	-21	kworker/u4:9+events_unbound	11:50:03	0
3118747	99	51	43,6	cyclictest	3175227	-21	kworker/u4:6+events_unbound	10:13:00	0
3118748	99	50	19,4	cyclictest	0	-21	swapper/1	07:50:38	1
3118747	99	50	43,6	cyclictest	3316934	-21	kworker/u4:7+events_unbound	12:20:00	0
3118747	99	50	42,6	cyclictest	3121235	-21	kworker/u4:5+events_unbound	07:39:04	0
3118747	99	50	36,8	cyclictest	3295220	-21	kworker/u4:6-rpciod	12:00:19	0
3118747	99	50	29,17	cyclictest	0	-21	swapper/0	08:30:11	0
3118747	99	50	26,7	cyclictest	0	-21	swapper/0	10:55:27	0
3118747	99	49	44,4	cyclictest	3127741	-21	kworker/u4:3+events_unbound	09:00:04	0
3118747	99	48	41,5	cyclictest	3240999	-21	/usr/sbin/munin	10:50:37	0
3118747	99	48	38,7	cyclictest	3254515	-21	unixbench_singl	11:05:41	0
3118747	99	48	28,18	cyclictest	3200120	-21	fschecks_count	10:05:19	0
3118747	99	47	39,6	cyclictest	3140938	-21	kworker/u4:16+events_unbound	08:50:02	0
3118747	99	47	37,6	cyclictest	3215731	-21	kworker/u4:13-rpciod	11:30:28	0
3118747	99	47	36,8	cyclictest	3168945	-21	ntp_kernel_err	09:30:24	0
3118747	99	47	36,7	cyclictest	3141404	-21	fschecks_count	08:35:20	0
3118747	99	47	27,5	cyclictest	3150304	-21	grep	09:10:15	0
3118747	99	47	27,5	cyclictest	3150304	-21	grep	09:10:15	0
3118747	99	46	38,6	cyclictest	3284706	-21	sh	11:40:24	0
3118747	99	46	34,6	cyclictest	3215731	-21	kworker/u4:13+events_unbound	11:20:45	0
3118747	99	45	14,23	cyclictest	0	-21	swapper/0	09:58:04	0
3118747	99	44	37,6	cyclictest	3127741	-21	kworker/u4:3+events_unbound	07:43:45	0
3118747	99	44	37,5	cyclictest	3137044	-21	dpkg	08:20:01	0
3118747	99	44	36,6	cyclictest	3131608	-21	idleruntime-cro	08:00:01	0
3118747	99	44	33,9	cyclictest	3139621	-21	munin-run	08:30:00	0
3118747	99	44	33,7	cyclictest	3146044	-21	kworker/u4:17-rpciod	09:05:35	0
3118747	99	43	10,24	cyclictest	17	1	rcu_preempt	07:10:20	0
3118747	99	41	30,8	cyclictest	0	-21	swapper/0	09:00:01	0
3118748	99	40	31,7	cyclictest	3121885	-21	head	07:20:21	1
3118748	99	35	24,8	cyclictest	3118674	-21	kworker/u4:2+events_unbound	07:10:00	1
3118747	99	30	20,7	cyclictest	3109805	-21	kworker/u4:6+rpciod	07:10:01	0
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17