OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Wed Mar 11, 2026 00:43:34)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
2754883	99	58	56,2	cyclictest	2745456	-21	kworker/u16:1+flush-8:0	21:09:35	6
2754405	2	54	44,7	sleep5	0	-21	swapper/5	19:01:19	5
2754885	99	50	44,4	cyclictest	3360446	-21	latency_hist	23:39:03	7
2754451	2	48	37,8	sleep4	0	-21	swapper/4	19:01:58	4
2754417	2	47	36,8	sleep6	0	-21	swapper/6	19:01:28	6
2754874	99	46	44,2	cyclictest	3336401	-21	kworker/u16:3+events_unbound	23:44:38	4
2754883	99	42	40,2	cyclictest	3372471	-21	kworker/u16:1+events_unbound	23:46:10	6
2754883	99	41	39,2	cyclictest	3240613	-21	kworker/u16:2+events_unbound	22:54:50	6
2754883	99	41	39,2	cyclictest	3213644	-21	kworker/u16:5+events_unbound	22:52:06	6
2754883	99	41	39,2	cyclictest	3059634	-21	kworker/u16:0+events_unbound	22:32:22	6
2754883	99	41	39,2	cyclictest	2745456	-21	kworker/u16:1+events_unbound	21:30:54	6
2754874	99	41	38,2	cyclictest	2939815	-21	kworker/u16:2+events_unbound	21:49:06	4
2754883	99	40	38,2	cyclictest	3372471	-21	kworker/u16:1+events_unbound	23:52:37	6
2754883	99	40	38,2	cyclictest	3336401	-21	kworker/u16:3+events_unbound	23:57:38	6
2754883	99	40	38,2	cyclictest	3025250	-21	kworker/u16:3+events_unbound	22:03:50	6
2754883	99	40	38,2	cyclictest	2745456	-21	kworker/u16:1+events_unbound	21:19:06	6
2754874	99	40	37,2	cyclictest	3336401	-21	kworker/u16:3+events_unbound	00:14:05	4
2754883	99	39	37,2	cyclictest	3336401	-21	kworker/u16:3+events_unbound	00:03:49	6
2754883	99	39	37,2	cyclictest	3213644	-21	kworker/u16:5+events_unbound	23:03:30	6
2754883	99	39	37,2	cyclictest	3059634	-21	kworker/u16:0+events_unbound	21:36:34	6
2754883	99	39	37,2	cyclictest	2745456	-21	kworker/u16:1+flush-8:0	19:14:05	6
2754883	99	39	36,2	cyclictest	3025250	-21	kworker/u16:3+events_unbound	21:39:14	6
2754883	99	39	36,2	cyclictest	2745456	-21	kworker/u16:1+events_unbound	20:24:06	6
2754883	99	39	36,2	cyclictest	2745456	-21	kworker/u16:1+events_unbound	20:24:06	6
2754874	99	39	36,2	cyclictest	3025250	-21	kworker/u16:3+events_unbound	22:04:07	4
2754874	99	39	36,2	cyclictest	2745456	-21	kworker/u16:1+flush-8:0	21:04:14	4
2754874	99	39	36,2	cyclictest	2745456	-21	kworker/u16:1+flush-8:0	21:04:14	4
2745989	2	39	2,31	sleep7	0	-21	swapper/7	18:59:20	7
2754885	99	38	35,2	cyclictest	3005970	-21	kthreadcore	21:09:28	7
2754885	99	38	33,4	cyclictest	3228147	-21	latency_hist	22:44:04	7
2754885	99	38	0,37	cyclictest	124	-21	kswapd0	23:29:18	7
2754883	99	38	36,2	cyclictest	3213644	-21	kworker/u16:5+events_unbound	22:40:13	6
2754883	99	38	36,2	cyclictest	3204071	-21	kworker/u16:3+events_unbound	23:06:30	6
2754883	99	38	36,2	cyclictest	3204071	-21	kworker/u16:3+events_unbound	23:06:30	6
2754883	99	38	36,2	cyclictest	2939815	-21	kworker/u16:2+events_unbound	21:52:18	6
2754883	99	38	35,2	cyclictest	3336401	-21	kworker/u16:3+events_unbound	00:14:13	6
2754874	99	38	36,2	cyclictest	3228169	-21	kworker/u16:0+events_unbound	23:40:14	4
2754883	99	37	35,2	cyclictest	3431186	-21	kworker/u16:1+events_unbound	00:07:50	6
2754883	99	37	35,2	cyclictest	3336401	-21	kworker/u16:3+events_unbound	00:20:55	6
2754883	99	37	35,2	cyclictest	3228169	-21	kworker/u16:0+events_unbound	23:27:58	6
2754883	99	37	35,2	cyclictest	3059634	-21	kworker/u16:0+events_unbound	22:10:58	6
2754883	99	37	35,2	cyclictest	2909999	-21	kworker/u16:4+events_unbound	21:05:39	6
2754883	99	37	35,2	cyclictest	2909999	-21	kworker/u16:4+events_unbound	21:05:39	6
2754883	99	37	34,2	cyclictest	3228169	-21	kworker/u16:0+events_unbound	23:35:17	6
2754874	99	37	35,2	cyclictest	3396668	-21	kworker/u16:4+events_unbound	00:07:38	4
2754874	99	37	35,2	cyclictest	3155915	-21	kworker/u16:2+events_unbound	22:24:46	4
2754874	99	37	35,2	cyclictest	3155915	-21	kworker/u16:2+events_unbound	22:24:46	4
2754874	99	37	35,2	cyclictest	3025250	-21	kworker/u16:3+events_unbound	22:13:01	4
2754874	99	37	35,2	cyclictest	3025250	-21	kworker/u16:3+events_unbound	21:43:59	4
2754874	99	37	34,2	cyclictest	3106619	-21	kworker/u16:4+events_unbound	21:54:07	4
2754883	99	36	35,1	cyclictest	3059634	-21	kworker/u16:0+events_unbound	22:27:19	6
2754883	99	36	35,1	cyclictest	3059634	-21	kworker/u16:0+events_unbound	22:27:19	6
2754883	99	36	34,2	cyclictest	3431186	-21	kworker/u16:1+events_unbound	00:11:50	6
2754883	99	36	34,2	cyclictest	3276638	-21	kworker/u16:1+events_unbound	23:13:30	6
2754883	99	36	34,2	cyclictest	3228169	-21	kworker/u16:0+events_unbound	23:43:53	6
2754883	99	36	34,2	cyclictest	3131874	-21	kworker/u16:1+events_unbound	22:44:38	6
2754883	99	36	34,2	cyclictest	3059634	-21	kworker/u16:0+events_unbound	21:58:49	6
2754874	99	36	34,2	cyclictest	3059634	-21	kworker/u16:0+events_unbound	21:37:25	4
2754883	99	35	33,2	cyclictest	3456893	-21	kworker/u16:0+events_unbound	00:25:54	6
2754883	99	35	33,2	cyclictest	3456893	-21	kworker/u16:0+events_unbound	00:25:54	6
2754883	99	35	33,2	cyclictest	3336401	-21	kworker/u16:3+events_unbound	00:30:49	6
2754883	99	35	33,2	cyclictest	3213644	-21	kworker/u16:5+events_unbound	22:37:50	6
2754883	99	35	33,2	cyclictest	3204071	-21	kworker/u16:3+events_unbound	23:16:08	6
2754883	99	35	33,2	cyclictest	3059634	-21	kworker/u16:0+events_unbound	22:20:17	6
2754883	99	35	33,2	cyclictest	3059634	-21	kworker/u16:0+events_unbound	22:18:23	6
2754883	99	35	33,2	cyclictest	3025250	-21	kworker/u16:3+events_unbound	22:07:01	6
2754883	99	35	33,2	cyclictest	3025250	-21	kworker/u16:3+events_unbound	21:25:47	6
2754883	99	35	33,2	cyclictest	2745456	-21	kworker/u16:1+events_unbound	21:17:11	6
2754883	99	35	32,2	cyclictest	2735913	-21	kworker/u16:2+events_unbound	19:24:14	6
2754874	99	35	33,2	cyclictest	3396668	-21	kworker/u16:4+events_unbound	23:54:54	4
2754874	99	35	33,2	cyclictest	3213644	-21	kworker/u16:5+events_unbound	23:02:23	4
2754883	99	34	32,2	cyclictest	3323723	-21	kworker/u16:2+events_unbound	23:33:34	6
2754879	99	34	32,2	cyclictest	3228169	-21	kworker/u16:0+events_unbound	00:04:26	5
2754874	99	34	32,2	cyclictest	3323723	-21	kworker/u16:2+events_unbound	23:25:37	4
2754874	99	34	32,2	cyclictest	3276638	-21	kworker/u16:1+events_unbound	23:16:22	4
2754874	99	34	32,2	cyclictest	3228169	-21	kworker/u16:0+events_unbound	23:11:03	4
2754874	99	34	32,2	cyclictest	3025250	-21	kworker/u16:3+events_unbound	22:03:05	4
2754885	99	33	31,1	cyclictest	0	-21	swapper/7	22:09:18	7
2754874	99	33	31,2	cyclictest	3323723	-21	kworker/u16:2+events_unbound	23:38:37	4
2754874	99	33	31,2	cyclictest	3204071	-21	kworker/u16:3+events_unbound	22:49:53	4
2754874	99	33	31,2	cyclictest	3059634	-21	kworker/u16:0+events_unbound	22:22:06	4
2754874	99	33	31,2	cyclictest	2745456	-21	kworker/u16:1+flush-8:0	21:19:38	4
2754883	99	32	30,2	cyclictest	2939815	-21	kworker/u16:2+events_unbound	21:48:27	6
2754883	99	32	30,2	cyclictest	2939815	-21	kworker/u16:2+events_unbound	21:48:27	6
2754874	99	32	30,2	cyclictest	2745456	-21	kworker/u16:1+events_unbound	20:09:30	4
2754874	99	32	29,2	cyclictest	2999496	-21	kworker/u16:0+flush-8:0	21:14:14	4
3300036	2	31	0,1	chrt	3300037	-21	sh	23:13:37	7
2754883	99	31	29,2	cyclictest	3228169	-21	kworker/u16:0+events_unbound	23:19:53	6
2754883	99	31	29,2	cyclictest	2745456	-21	kworker/u16:1+events_unbound	20:06:07	6
2754883	99	31	28,2	cyclictest	2735913	-21	kworker/u16:2+events_unbound	19:19:07	6
2754874	99	31	29,2	cyclictest	3213644	-21	kworker/u16:5+events_unbound	22:55:23	4
2754874	99	31	28,2	cyclictest	3131874	-21	kworker/u16:1+events_unbound	22:14:13	4
2754874	99	30	28,2	cyclictest	3456893	-21	kworker/u16:0+events_unbound	00:25:50	4
2754874	99	30	28,2	cyclictest	3456893	-21	kworker/u16:0+events_unbound	00:25:50	4
2754874	99	30	28,2	cyclictest	3431186	-21	kworker/u16:1+events_unbound	00:10:38	4
2754874	99	30	28,2	cyclictest	3131874	-21	kworker/u16:1+events_unbound	22:46:51	4
2754874	99	30	28,2	cyclictest	2999496	-21	kworker/u16:0+events_unbound	21:11:45	4
2754883	99	29	27,2	cyclictest	2745456	-21	kworker/u16:1+flush-8:0	19:29:50	6
2754874	99	29	27,2	cyclictest	3372471	-21	kworker/u16:1+events_unbound	23:52:29	4
2754874	99	29	27,2	cyclictest	3204071	-21	kworker/u16:3+events_unbound	23:05:13	4
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17