OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Thu Jun 11, 2026 00:43:34)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
591515	99	62	60,2	cyclictest	1087690	-21	kworker/u16:4+events_unbound	23:06:42	5
591515	99	62	60,2	cyclictest	1087690	-21	kworker/u16:4+events_unbound	23:06:42	5
591515	99	61	59,2	cyclictest	1257984	-21	kworker/u16:5+flush-8:0	00:06:37	5
591515	99	53	51,2	cyclictest	1173952	-21	kworker/u16:2+events_unbound	23:41:43	5
591515	99	53	50,2	cyclictest	1101201	-21	kworker/u16:0+flush-8:0	23:26:41	5
591515	99	49	47,2	cyclictest	945872	-21	kworker/u16:2+flush-8:0	22:06:42	5
591106	2	49	38,8	sleep5	0	-21	swapper/5	19:04:23	5
591515	99	47	45,2	cyclictest	1087690	-21	kworker/u16:4+flush-8:0	23:01:24	5
591515	99	44	42,2	cyclictest	1257984	-21	kworker/u16:5+flush-8:0	00:16:41	5
591515	99	44	41,2	cyclictest	980062	-21	kworker/u16:3+events_unbound	22:11:14	5
591515	99	43	41,2	cyclictest	796679	-21	kworker/u16:3+events_unbound	21:55:58	5
591515	99	43	41,2	cyclictest	613568	-21	kworker/u16:0+events_unbound	19:16:14	5
591515	99	42	40,2	cyclictest	1126422	-21	kworker/u16:1+flush-8:0	23:31:41	5
590960	2	42	4,32	sleep7	0	-21	swapper/7	19:02:16	7
591515	99	41	39,2	cyclictest	1173952	-21	kworker/u16:2+flush-8:0	00:11:42	5
591515	99	41	39,2	cyclictest	1126422	-21	kworker/u16:1+events_unbound	23:24:53	5
591515	99	41	38,2	cyclictest	919630	-21	kworker/u16:0+flush-8:0	22:26:45	5
591515	99	40	38,2	cyclictest	1026924	-21	kworker/u16:2+events_unbound	22:41:36	5
591524	99	39	32,6	cyclictest	796891	-21	latency_hist	20:51:12	7
591515	99	39	37,2	cyclictest	768476	-21	kworker/u16:1+events_unbound	22:37:43	5
591515	99	39	37,2	cyclictest	768476	-21	kworker/u16:1+events_unbound	22:04:30	5
591515	99	39	37,2	cyclictest	1234042	-21	kworker/u16:4+flush-8:0	00:01:32	5
591515	99	39	37,2	cyclictest	1101201	-21	kworker/u16:0+flush-8:0	23:16:41	5
591515	99	39	36,2	cyclictest	671820	-21	kworker/u16:2+events_unbound	20:46:14	5
591515	99	37	35,2	cyclictest	980062	-21	kworker/u16:3+events_unbound	22:19:37	5
591515	99	37	35,2	cyclictest	919630	-21	kworker/u16:0+flush-8:0	21:47:17	5
591515	99	37	35,2	cyclictest	919630	-21	kworker/u16:0+flush-8:0	21:47:17	5
591515	99	37	35,2	cyclictest	919630	-21	kworker/u16:0+events_unbound	22:22:30	5
591515	99	37	35,2	cyclictest	919630	-21	kworker/u16:0+events_unbound	21:58:38	5
591515	99	37	35,2	cyclictest	768476	-21	kworker/u16:1+events_unbound	21:32:50	5
591515	99	37	35,2	cyclictest	768476	-21	kworker/u16:1+events_unbound	21:22:29	5
591515	99	37	35,2	cyclictest	1173952	-21	kworker/u16:2+events_unbound	23:59:09	5
591515	99	37	35,2	cyclictest	1126422	-21	kworker/u16:1+events_unbound	23:14:53	5
591515	99	37	35,2	cyclictest	1101201	-21	kworker/u16:0+events_unbound	23:53:42	5
591515	99	36	34,2	cyclictest	768476	-21	kworker/u16:1+flush-8:0	21:30:34	5
591515	99	36	34,2	cyclictest	768476	-21	kworker/u16:1+events_unbound	22:33:41	5
591515	99	36	34,2	cyclictest	768476	-21	kworker/u16:1+events_unbound	21:37:18	5
591515	99	36	34,2	cyclictest	768476	-21	kworker/u16:1+events_unbound	21:11:44	5
591515	99	36	34,2	cyclictest	1173952	-21	kworker/u16:2+events_unbound	00:33:25	5
591515	99	36	34,2	cyclictest	1173952	-21	kworker/u16:2+events_unbound	00:29:38	5
591515	99	36	34,2	cyclictest	1173952	-21	kworker/u16:2+events_unbound	00:29:38	5
591515	99	35	33,2	cyclictest	768476	-21	kworker/u16:1+events_unbound	22:52:38	5
591515	99	35	33,2	cyclictest	768476	-21	kworker/u16:1+events_unbound	21:08:26	5
591515	99	35	33,2	cyclictest	768476	-21	kworker/u16:1+events_unbound	21:08:26	5
591515	99	35	33,2	cyclictest	671820	-21	kworker/u16:2+events_unbound	19:55:50	5
591515	99	35	33,2	cyclictest	1054362	-21	kworker/u16:3+events_unbound	22:50:45	5
591133	2	35	25,7	sleep4	0	-21	swapper/4	19:04:46	4
591524	99	34	27,6	cyclictest	635	-21	systemd-journal	22:16:12	7
591515	99	34	32,2	cyclictest	816221	-21	kworker/u16:4+events_unbound	21:18:30	5
591515	99	34	32,2	cyclictest	1054362	-21	kworker/u16:3+events_unbound	23:49:34	5
591515	99	34	32,2	cyclictest	1054362	-21	kworker/u16:3+events_unbound	23:49:34	5
591515	99	33	31,2	cyclictest	768476	-21	kworker/u16:1+flush-8:0	21:42:02	5
591515	99	33	31,2	cyclictest	671820	-21	kworker/u16:2+flush-8:0	20:36:32	5
591515	99	33	31,2	cyclictest	1173952	-21	kworker/u16:2+events_unbound	23:40:33	5
591165	2	33	23,7	sleep6	0	-21	swapper/6	19:05:12	6
591515	99	32	30,2	cyclictest	768476	-21	kworker/u16:1+flush-8:0	22:57:53	5
591515	99	32	30,2	cyclictest	652514	-21	kworker/u16:0+flush-8:0	19:36:39	5
591515	99	32	30,2	cyclictest	1173952	-21	kworker/u16:2+flush-8:0	00:22:25	5
591515	99	30	28,2	cyclictest	768476	-21	kworker/u16:1+events_unbound	20:44:22	5
591515	99	30	28,2	cyclictest	652514	-21	kworker/u16:0+events_unbound	19:57:22	5
591524	99	29	28,0	cyclictest	0	-21	swapper/7	19:06:35	7
591524	99	29	24,4	cyclictest	1329942	-21	latency_hist	00:36:12	7
591515	99	29	26,2	cyclictest	671820	-21	kworker/u16:2+events_unbound	20:06:14	5
591520	99	28	26,2	cyclictest	700882	-21	kworker/u16:4+events_unbound	20:24:38	6
591515	99	28	26,2	cyclictest	768476	-21	kworker/u16:1+events_unbound	21:03:22	5
591524	99	27	26,0	cyclictest	0	-21	swapper/7	21:46:37	7
591524	99	27	26,0	cyclictest	0	-21	swapper/7	21:46:37	7
591520	99	27	25,2	cyclictest	671820	-21	kworker/u16:2+events_unbound	20:15:46	6
591520	99	27	25,2	cyclictest	1087690	-21	kworker/u16:4+flush-8:0	23:03:45	6
591515	99	27	25,2	cyclictest	652514	-21	kworker/u16:0+events_unbound	20:01:21	5
1279949	2	27	0,1	sleep4	0	-21	swapper/4	00:12:09	4
1259518	2	27	0,0	chrt	1259519	-21	irqcore	00:06:27	7
591524	99	26	21,4	cyclictest	825784	-21	latency_hist	21:06:12	7
591520	99	26	24,2	cyclictest	571932	-21	kworker/u16:1+events_unbound	19:44:17	6
591520	99	26	24,2	cyclictest	1173952	-21	kworker/u16:2+events_unbound	00:28:24	6
591520	99	26	24,2	cyclictest	1173952	-21	kworker/u16:2+events_unbound	00:28:24	6
591520	99	26	24,2	cyclictest	1101201	-21	kworker/u16:0+events_unbound	23:14:00	6
591515	99	26	24,2	cyclictest	700882	-21	kworker/u16:4+events_unbound	20:24:50	5
591515	99	26	24,2	cyclictest	671820	-21	kworker/u16:2+flush-8:0	20:26:13	5
591515	99	26	24,2	cyclictest	671820	-21	kworker/u16:2+events_unbound	20:57:14	5
591515	99	26	24,2	cyclictest	613568	-21	kworker/u16:0+events_unbound	19:13:51	5
591515	99	26	24,2	cyclictest	571932	-21	kworker/u16:1+events_unbound	19:36:02	5
591512	99	26	24,2	cyclictest	671820	-21	kworker/u16:2+flush-8:0	20:16:38	4
1101131	2	26	0,1	chrt	1101129	-21	ssh	22:59:35	7
591524	99	25	24,0	cyclictest	0	-21	swapper/7	20:51:30	7
591524	99	25	20,4	cyclictest	1197916	-21	latency_hist	23:41:12	7
591515	99	25	23,2	cyclictest	671820	-21	kworker/u16:2+events_unbound	20:13:51	5
591515	99	25	23,2	cyclictest	652514	-21	kworker/u16:0+events_unbound	20:34:50	5
591520	99	24	22,2	cyclictest	796679	-21	kworker/u16:3+events_unbound	20:50:49	6
591520	99	24	22,2	cyclictest	571932	-21	kworker/u16:1+events_unbound	19:55:29	6
591515	99	24	22,2	cyclictest	671820	-21	kworker/u16:2+events_unbound	19:44:22	5
591524	99	23	22,0	cyclictest	0	-21	swapper/7	19:31:38	7
591520	99	23	21,2	cyclictest	919630	-21	kworker/u16:0+events_unbound	22:32:05	6
591520	99	23	21,2	cyclictest	796679	-21	kworker/u16:3+events_unbound	21:22:44	6
591520	99	23	21,2	cyclictest	671820	-21	kworker/u16:2+events_unbound	20:16:27	6
591520	99	23	21,2	cyclictest	571932	-21	kworker/u16:1+events_unbound	19:30:06	6
591515	99	23	21,2	cyclictest	768476	-21	kworker/u16:1+flush-8:0	20:53:22	5
591515	99	23	21,2	cyclictest	671820	-21	kworker/u16:2+flush-8:0	19:49:22	5
591515	99	23	21,2	cyclictest	671820	-21	kworker/u16:2+flush-8:0	19:49:22	5
591512	99	23	21,2	cyclictest	537523	-21	kworker/u16:3+events_unbound	19:11:15	4
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17