OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Mon Jun 01, 2026 00:43:34)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
358114	2	68	0,1	sleep5	76	-21	ksoftirqd/5	20:02:15	5
242044	2	48	38,7	sleep4	0	-21	swapper/4	19:05:18	4
609577	2	46	0,0	sleep7	609578	-21	sh	22:02:05	7
242449	99	45	43,2	cyclictest	883921	-21	kworker/u16:2+events_unbound	23:57:53	5
242449	99	44	42,2	cyclictest	597082	-21	kworker/u16:0+events_unbound	22:27:24	5
242449	99	41	39,2	cyclictest	813785	-21	kworker/u16:1+events_unbound	23:30:14	5
233555	2	39	3,30	sleep7	0	-21	swapper/7	19:02:08	7
242459	99	35	32,2	cyclictest	786923	-21	ssh	23:12:17	7
242449	99	35	33,2	cyclictest	729789	-21	kworker/u16:0+events_unbound	23:19:02	5
937060	2	34	0,1	chrt	937058	-21	ssh	00:21:25	7
242459	99	33	27,5	cyclictest	635	-21	systemd-journal	22:26:52	7
242459	99	33	27,5	cyclictest	313384	-21	latency_hist	19:41:52	7
242449	99	32	30,2	cyclictest	862165	-21	kworker/u16:0+events_unbound	23:52:50	5
242449	99	32	30,2	cyclictest	476826	-21	kworker/u16:3+events_unbound	22:05:03	5
242459	99	31	30,0	cyclictest	0	-21	swapper/7	19:27:07	7
242453	99	31	29,2	cyclictest	642739	-21	kworker/u16:1+events_unbound	22:17:18	6
242449	99	31	29,2	cyclictest	883921	-21	kworker/u16:2+events_unbound	00:21:01	5
242449	99	31	29,2	cyclictest	862165	-21	kworker/u16:0+events_unbound	23:49:50	5
242449	99	31	29,2	cyclictest	693396	-21	kworker/u16:4+events_unbound	23:03:02	5
242449	99	31	29,2	cyclictest	693396	-21	kworker/u16:4+events_unbound	23:03:02	5
242449	99	31	29,2	cyclictest	693396	-21	kworker/u16:4+events_unbound	22:57:02	5
242449	99	31	29,2	cyclictest	409190	-21	kworker/u16:1+events_unbound	21:25:19	5
242459	99	30	29,0	cyclictest	0	-21	swapper/7	21:47:05	7
242459	99	30	29,0	cyclictest	0	-21	swapper/7	20:22:22	7
242449	99	30	28,2	cyclictest	729789	-21	kworker/u16:0+events_unbound	23:16:37	5
242449	99	30	28,2	cyclictest	642739	-21	kworker/u16:1+events_unbound	22:54:14	5
242449	99	30	28,2	cyclictest	536740	-21	kworker/u16:4+events_unbound	21:38:03	5
242449	99	30	28,2	cyclictest	351364	-21	kworker/u16:2+events_unbound	21:51:34	5
242459	99	29	23,5	cyclictest	657197	-21	latency_hist	22:21:52	7
242449	99	29	27,2	cyclictest	883921	-21	kworker/u16:2+events_unbound	00:32:01	5
242449	99	29	27,2	cyclictest	729789	-21	kworker/u16:0+events_unbound	23:11:34	5
242449	99	29	27,2	cyclictest	351364	-21	kworker/u16:2+events_unbound	21:36:39	5
242449	99	29	27,2	cyclictest	351364	-21	kworker/u16:2+events_unbound	21:09:02	5
242449	99	29	26,2	cyclictest	476826	-21	kworker/u16:3+flush-8:0	21:41:53	5
242449	99	29	25,3	cyclictest	693396	-21	kworker/u16:4+events_unbound	23:43:29	5
242449	99	29	25,3	cyclictest	693396	-21	kworker/u16:4+events_unbound	23:43:29	5
242459	99	28	26,1	cyclictest	0	-21	swapper/7	21:37:18	7
242459	99	28	22,5	cyclictest	635	-21	systemd-journal	23:46:51	7
242459	99	28	22,5	cyclictest	635	-21	systemd-journal	23:46:51	7
242449	99	28	26,2	cyclictest	693396	-21	kworker/u16:4+events_unbound	22:43:22	5
242449	99	28	26,2	cyclictest	409190	-21	kworker/u16:1+events_unbound	21:20:51	5
242449	99	28	26,2	cyclictest	351364	-21	kworker/u16:2+flush-8:0	22:09:01	5
241979	2	28	17,8	sleep5	0	-21	swapper/5	19:04:23	5
242459	99	27	26,0	cyclictest	0	-21	swapper/7	23:27:20	7
242459	99	27	26,0	cyclictest	0	-21	swapper/7	23:07:06	7
242459	99	27	26,0	cyclictest	0	-21	swapper/7	21:27:15	7
242459	99	27	25,1	cyclictest	0	-21	swapper/7	00:07:12	7
242449	99	27	26,1	cyclictest	790331	-21	kworker/u16:2+events_unbound	23:26:19	5
242449	99	27	25,2	cyclictest	886169	-21	kworker/u16:3+events_unbound	00:02:54	5
242449	99	27	25,2	cyclictest	883921	-21	kworker/u16:2+events_unbound	00:25:30	5
242449	99	27	25,2	cyclictest	883921	-21	kworker/u16:2+events_unbound	00:25:30	5
242459	99	26	25,0	cyclictest	0	-21	swapper/7	21:42:20	7
242459	99	26	25,0	cyclictest	0	-21	swapper/7	20:37:17	7
242459	99	26	22,3	cyclictest	512496	-21	latency_hist	21:21:52	7
242449	99	26	24,2	cyclictest	883921	-21	kworker/u16:2+events_unbound	00:15:05	5
242449	99	26	24,2	cyclictest	813785	-21	kworker/u16:1+events_unbound	23:37:20	5
242449	99	26	24,2	cyclictest	597082	-21	kworker/u16:0+events_unbound	22:34:50	5
242449	99	26	24,2	cyclictest	476826	-21	kworker/u16:3+events_unbound	22:47:51	5
242449	99	26	22,3	cyclictest	265712	-21	kworker/u16:3+flush-8:0	19:17:19	5
233029	2	26	16,7	sleep6	0	-21	swapper/6	19:01:59	6
242459	99	25	24,0	cyclictest	0	-21	swapper/7	19:47:08	7
242459	99	25	22,2	cyclictest	418857	-21	sort	20:36:52	7
242449	99	25	23,2	cyclictest	813785	-21	kworker/u16:1+events_unbound	23:36:49	5
242449	99	25	23,2	cyclictest	813785	-21	kworker/u16:1+events_unbound	00:10:49	5
242449	99	25	23,2	cyclictest	476826	-21	kworker/u16:3+events_unbound	22:38:58	5
242449	99	25	23,2	cyclictest	476826	-21	kworker/u16:3+events_unbound	22:19:49	5
242449	99	25	23,2	cyclictest	351364	-21	kworker/u16:2+events_unbound	21:14:42	5
242449	99	24	22,2	cyclictest	597082	-21	kworker/u16:0+events_unbound	21:57:42	5
242449	99	24	22,2	cyclictest	409190	-21	kworker/u16:1+events_unbound	21:28:58	5
242459	99	23	21,1	cyclictest	946225	-21	ssh	00:22:15	7
242459	99	23	21,1	cyclictest	946225	-21	ssh	00:22:15	7
242459	99	23	21,1	cyclictest	0	-21	swapper/7	20:02:12	7
242453	99	23	21,2	cyclictest	693396	-21	kworker/u16:4+events_unbound	22:47:20	6
242453	99	23	20,2	cyclictest	265712	-21	kworker/u16:3+flush-8:0	19:27:18	6
242449	99	23	21,2	cyclictest	597082	-21	kworker/u16:0+events_unbound	22:26:37	5
242449	99	23	21,2	cyclictest	597082	-21	kworker/u16:0+events_unbound	22:26:37	5
242449	99	23	18,2	cyclictest	813785	-21	kworker/u16:1+flush-8:0	00:31:51	5
242444	99	23	21,2	cyclictest	597082	-21	kworker/u16:0+events_unbound	22:29:23	4
242444	99	23	20,2	cyclictest	693396	-21	kworker/u16:4+events_unbound	00:16:54	4
242459	99	22	0,20	cyclictest	124	-21	kswapd0	23:17:17	7
242449	99	22	20,2	cyclictest	536740	-21	kworker/u16:4+events_unbound	21:55:50	5
242449	99	22	20,2	cyclictest	409190	-21	kworker/u16:1+flush-8:0	21:02:21	5
242449	99	22	20,2	cyclictest	179595	-21	kworker/u16:1+events_unbound	19:16:38	5
242459	99	21	19,1	cyclictest	0	-21	swapper/7	20:12:17	7
242453	99	21	19,2	cyclictest	813785	-21	kworker/u16:1+events_unbound	23:27:06	6
242453	99	21	18,2	cyclictest	790331	-21	kworker/u16:2+flush-8:0	23:21:52	6
242449	99	21	19,2	cyclictest	418656	-21	kworker/u16:3+flush-8:0	20:47:21	5
242444	99	21	19,2	cyclictest	476826	-21	kworker/u16:3+events_unbound	21:33:06	4
242444	99	21	19,2	cyclictest	351364	-21	kworker/u16:2+events_unbound	21:57:51	4
242444	99	21	19,2	cyclictest	265712	-21	kworker/u16:3+events_unbound	19:37:05	4
242444	99	21	18,2	cyclictest	597082	-21	kworker/u16:0+events_unbound	22:31:56	4
242459	99	20	19,0	cyclictest	0	-21	swapper/7	23:22:05	7
242459	99	20	19,0	cyclictest	0	-21	swapper/7	00:32:21	7
242459	99	20	15,4	cyclictest	838014	-21	latency_hist	23:36:51	7
242453	99	20	18,2	cyclictest	456370	-21	kworker/u16:0+events_unbound	21:02:10	6
242453	99	20	17,2	cyclictest	0	-21	swapper/6	21:47:06	6
242453	99	20	16,3	cyclictest	642739	-21	kworker/u16:1+flush-8:0	22:37:08	6
242449	99	20	18,2	cyclictest	242206	-21	kworker/u16:0+events_unbound	20:16:38	5
242444	99	20	18,2	cyclictest	790331	-21	kworker/u16:2+events_unbound	23:43:37	4
242444	99	20	18,2	cyclictest	790331	-21	kworker/u16:2+events_unbound	23:43:37	4
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17