OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Thu Jul 09, 2026 12:43:33)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
4163106	2	96	0,1	sleep5	0	-21	swapper/5	12:30:10	5
3790426	2	96	0,1	sleep7	3790427	-21	cat	10:00:09	7
3424429	2	72	62,7	sleep5	0	-21	swapper/5	07:06:04	5
3675024	2	62	0,1	sleep7	3675025	-21	cat	09:14:49	7
3424638	2	60	48,7	sleep4	0	-21	swapper/4	07:09:00	4
3424661	2	58	23,30	sleep7	0	-21	swapper/7	07:09:19	7
3424982	99	43	39,4	cyclictest	4058194	-21	kworker/u16:3+flush-8:0	11:54:49	5
3424994	99	42	0,40	cyclictest	4120030	-21	latency_hist	12:14:48	7
3424986	99	42	40,2	cyclictest	3922454	-21	kworker/u16:2+events_unbound	11:24:39	6
3424986	99	40	37,2	cyclictest	3661585	-21	kworker/u16:5+events_unbound	09:35:03	6
3424986	99	39	37,2	cyclictest	3814484	-21	kworker/u16:2+events_unbound	10:26:23	6
3424994	99	38	32,5	cyclictest	3623536	-21	latency_hist	08:49:49	7
3424986	99	38	36,2	cyclictest	3814484	-21	kworker/u16:2+events_unbound	10:18:38	6
3424986	99	37	35,2	cyclictest	4058194	-21	kworker/u16:3+events_unbound	12:07:14	6
3424986	99	37	35,2	cyclictest	3996430	-21	kworker/u16:4+events_unbound	11:47:27	6
3424986	99	37	35,2	cyclictest	3661585	-21	kworker/u16:5+events_unbound	09:18:11	6
3424986	99	37	34,2	cyclictest	3506390	-21	kworker/u16:2+flush-8:0	08:45:04	6
3424986	99	37	34,2	cyclictest	2677098	-21	kworker/u16:3+events_unbound	07:19:51	6
3424986	99	36	34,2	cyclictest	4114882	-21	kworker/u16:2+events_unbound	12:28:06	6
3424986	99	36	34,2	cyclictest	4058194	-21	kworker/u16:3+events_unbound	11:59:19	6
3424986	99	36	34,2	cyclictest	4045163	-21	kworker/u16:1+events_unbound	12:20:39	6
3424986	99	36	34,2	cyclictest	3833007	-21	kworker/u16:1+events_unbound	10:49:11	6
3424986	99	36	34,2	cyclictest	3459255	-21	kworker/u16:1+events_unbound	09:43:43	6
4053641	2	34	0,1	sleep4	0	-21	swapper/4	11:45:13	4
3424994	99	34	32,1	cyclictest	3943557	-21	ssh	11:00:14	7
3424986	99	34	32,2	cyclictest	3860334	-21	kworker/u16:3+events_unbound	11:31:43	6
3424986	99	34	32,2	cyclictest	3786106	-21	kworker/u16:4+events_unbound	10:52:23	6
3424986	99	34	32,2	cyclictest	3633509	-21	kworker/u16:3+events_unbound	09:10:39	6
3696487	2	33	0,5	chrt	3696481	-21	ssh	09:20:18	7
3424986	99	33	31,2	cyclictest	4154627	-21	kworker/u16:0+events_unbound	12:32:03	6
3424986	99	33	31,2	cyclictest	3956865	-21	kworker/u16:0+events_unbound	11:07:26	6
3424986	99	33	31,2	cyclictest	3661585	-21	kworker/u16:5+events_unbound	09:23:26	6
3424613	2	33	10,8	sleep6	0	-21	swapper/6	07:08:37	6
3424986	99	32	30,2	cyclictest	3459255	-21	kworker/u16:1+events_unbound	09:00:22	6
3424986	99	31	29,2	cyclictest	3487549	-21	kworker/u16:3+events_unbound	07:39:51	6
3424986	99	31	29,2	cyclictest	3459255	-21	kworker/u16:1+events_unbound	10:01:32	6
3701463	2	30	0,1	chrt	3701464	-21	kthreadcore	09:25:07	7
3424994	99	30	0,28	cyclictest	908	-21	dbus-daemon	07:29:49	7
3424986	99	30	28,2	cyclictest	4154627	-21	kworker/u16:0+events_unbound	12:36:29	6
3424986	99	30	28,2	cyclictest	3956865	-21	kworker/u16:0+flush-8:0	12:11:58	6
3424986	99	30	28,2	cyclictest	3922454	-21	kworker/u16:2+flush-8:0	11:00:30	6
3424986	99	30	28,2	cyclictest	3860334	-21	kworker/u16:3+events_unbound	11:29:42	6
3424986	99	30	28,2	cyclictest	3595571	-21	kworker/u16:0+events_unbound	08:40:12	6
3762746	2	29	0,1	chrt	3762748	-21	irqcore	09:50:08	7
3641153	2	29	0,1	chrt	3641154	-21	kthreadcore	08:55:10	7
3424986	99	29	27,2	cyclictest	3956865	-21	kworker/u16:0+flush-8:0	12:00:23	6
3424986	99	29	27,2	cyclictest	3833007	-21	kworker/u16:1+flush-8:0	10:24:06	6
3424986	99	29	27,2	cyclictest	3786106	-21	kworker/u16:4+events_unbound	10:10:16	6
3424986	99	29	27,2	cyclictest	3699477	-21	kworker/u16:2+events_unbound	09:27:28	6
3424986	99	29	27,2	cyclictest	3661585	-21	kworker/u16:5+flush-8:0	09:31:27	6
3424986	99	29	27,2	cyclictest	3661585	-21	kworker/u16:5+events_unbound	09:54:28	6
3424986	99	29	27,2	cyclictest	3467749	-21	kworker/u16:2+events_unbound	07:32:59	6
3424986	99	29	27,2	cyclictest	3459255	-21	kworker/u16:1+events_unbound	09:56:51	6
3424986	99	28	26,2	cyclictest	3459255	-21	kworker/u16:1+events_unbound	07:27:15	6
3424986	99	28	26,2	cyclictest	2677098	-21	kworker/u16:3+events_unbound	07:17:15	6
3424986	99	27	25,2	cyclictest	4120214	-21	kworker/u16:4+flush-8:0	12:17:15	6
3424986	99	27	25,2	cyclictest	4120214	-21	kworker/u16:4+flush-8:0	12:17:15	6
3424986	99	27	25,2	cyclictest	3860334	-21	kworker/u16:3+flush-8:0	10:41:10	6
3424986	99	27	25,2	cyclictest	3506390	-21	kworker/u16:2+events_unbound	08:09:00	6
3424986	99	27	25,2	cyclictest	3435024	-21	kworker/u16:0+flush-8:0	07:35:19	6
3424986	99	27	25,2	cyclictest	2677098	-21	kworker/u16:3+events_unbound	07:14:44	6
3424986	99	26	24,2	cyclictest	3922454	-21	kworker/u16:2+events_unbound	11:16:09	6
3424986	99	26	24,2	cyclictest	3506390	-21	kworker/u16:2+events_unbound	08:23:59	6
3424986	99	26	24,2	cyclictest	3459255	-21	kworker/u16:1+flush-8:0	08:59:47	6
3424986	99	26	24,2	cyclictest	3459255	-21	kworker/u16:1+flush-8:0	08:25:03	6
3424986	99	25	24,1	cyclictest	3833007	-21	kworker/u16:1+events_unbound	10:38:40	6
3424986	99	25	23,2	cyclictest	3996430	-21	kworker/u16:4+events_unbound	11:38:42	6
3424986	99	25	23,2	cyclictest	3860334	-21	kworker/u16:3+events_unbound	11:39:56	6
3424986	99	25	23,2	cyclictest	3833007	-21	kworker/u16:1+events_unbound	10:58:24	6
3424986	99	25	23,2	cyclictest	3786106	-21	kworker/u16:4+flush-8:0	10:31:32	6
3424986	99	25	23,2	cyclictest	3739563	-21	kworker/u16:0+events_unbound	09:46:37	6
3424986	99	25	23,2	cyclictest	3575073	-21	kworker/u16:4+events_unbound	08:33:59	6
3424986	99	25	23,2	cyclictest	3459255	-21	kworker/u16:1+events_unbound	08:17:16	6
3424986	99	24	22,2	cyclictest	3459255	-21	kworker/u16:1+flush-8:0	08:10:02	6
3424986	99	24	22,2	cyclictest	3459255	-21	kworker/u16:1+flush-8:0	07:59:59	6
3424986	99	24	22,2	cyclictest	3459255	-21	kworker/u16:1+flush-8:0	07:59:59	6
3424986	99	24	22,2	cyclictest	3459255	-21	kworker/u16:1+events_unbound	10:05:17	6
3424994	99	23	22,0	cyclictest	0	-21	swapper/7	12:36:15	7
3424986	99	23	20,2	cyclictest	3459255	-21	kworker/u16:1+flush-8:0	07:50:04	6
3424994	99	22	20,1	cyclictest	4130974	-21	ssh	12:17:20	7
3424994	99	22	20,1	cyclictest	4130974	-21	ssh	12:17:20	7
3424994	99	22	20,1	cyclictest	3822690	-21	ssh	10:13:59	7
3424994	99	22	19,2	cyclictest	3476541	-21	ntpq	07:30:15	7
3424994	99	22	0,21	cyclictest	908	-21	dbus-daemon	08:04:49	7
3424994	99	22	0,21	cyclictest	908	-21	dbus-daemon	08:04:49	7
3424986	99	22	20,2	cyclictest	3459255	-21	kworker/u16:1+flush-8:0	08:53:08	6
3424982	99	22	20,2	cyclictest	3415013	-21	kworker/u16:4+flush-8:0	07:24:49	5
4146757	2	21	0,1	chrt	4146759	-21	sh	12:25:07	7
3424994	99	21	19,1	cyclictest	0	-21	swapper/7	10:18:23	7
3424994	99	21	19,1	cyclictest	0	-21	swapper/7	09:35:14	7
3424994	99	21	18,2	cyclictest	4133639	-21	hddtemp_smartct	12:20:04	7
3424994	99	21	18,2	cyclictest	3745776	-21	ntpq	09:40:18	7
3424986	99	21	19,2	cyclictest	4057695	-21	kworker/u16:2+flush-8:0	11:51:04	6
3424986	99	21	19,2	cyclictest	3459255	-21	kworker/u16:1+flush-8:0	07:55:17	6
3424982	99	21	18,2	cyclictest	3786106	-21	kworker/u16:4+flush-8:0	10:44:55	5
3424994	99	20	19,0	cyclictest	0	-21	swapper/7	09:19:14	7
3424986	99	20	17,2	cyclictest	3651803	-21	kworker/u16:4+events_unbound	09:04:59	6
3424982	99	20	16,4	cyclictest	3860334	-21	kworker/u16:3+flush-8:0	11:19:49	5
3424994	99	19	18,0	cyclictest	0	-21	swapper/7	07:15:18	7
3424994	99	19	17,1	cyclictest	0	-21	swapper/7	11:57:35	7
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17