OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Sat Mar 07, 2026 12:43:34)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
733401	2	72	0,1	sleep4	0	-21	swapper/4	10:34:40	4
276936	2	63	51,8	sleep6	0	-21	swapper/6	07:00:55	6
276944	2	58	48,7	sleep5	0	-21	swapper/5	07:01:02	5
277461	99	55	51,3	cyclictest	979376	-21	kworker/u16:0+flush-8:0	12:19:47	4
277476	99	52	48,3	cyclictest	764645	-21	sed	10:49:43	7
276889	2	52	42,7	sleep4	0	-21	swapper/4	07:00:13	4
277461	99	49	47,2	cyclictest	568856	-21	kworker/u16:5+flush-8:0	10:19:55	4
277476	99	47	0,45	cyclictest	570503	-21	date	09:29:18	7
277476	99	43	0,41	cyclictest	991355	-21	latency_hist	12:24:18	7
277461	99	42	40,2	cyclictest	568855	-21	kworker/u16:3+flush-8:0	09:29:18	4
277476	99	39	34,4	cyclictest	606609	-21	latency_hist	09:44:19	7
277461	99	38	35,2	cyclictest	500769	-21	kworker/u16:1+events_unbound	09:49:31	4
276242	2	38	3,30	sleep7	0	-21	swapper/7	06:59:52	7
277476	99	36	31,4	cyclictest	630700	-21	latency_hist	09:54:18	7
277461	99	36	33,2	cyclictest	809025	-21	kworker/u16:3+flush-8:0	11:44:23	4
736740	2	34	0,1	chrt	736741	-21	unixbench_singl	10:34:51	7
277469	99	34	32,2	cyclictest	809025	-21	kworker/u16:3+flush-8:0	11:09:59	5
277461	99	34	31,3	cyclictest	809025	-21	kworker/u16:3+flush-8:0	11:34:13	4
277461	99	34	31,2	cyclictest	631090	-21	kworker/u16:2+events_unbound	10:09:31	4
277461	99	34	31,2	cyclictest	568856	-21	kworker/u16:5+events_unbound	10:04:32	4
277476	99	32	0,30	cyclictest	824613	-21	sed	11:14:36	7
277469	99	32	30,2	cyclictest	568856	-21	kworker/u16:5+events_unbound	10:09:39	5
277461	99	32	30,2	cyclictest	568855	-21	kworker/u16:3+flush-8:0	10:44:54	4
277469	99	31	29,2	cyclictest	205325	-21	kworker/u16:0+events_unbound	07:14:52	5
277469	99	31	29,2	cyclictest	205325	-21	kworker/u16:0+events_unbound	07:04:44	5
277476	99	29	29,0	cyclictest	0	-21	swapper/7	11:46:01	7
277476	99	29	25,2	cyclictest	376577	-21	sed	07:54:18	7
277461	99	29	27,2	cyclictest	631090	-21	kworker/u16:2+flush-8:0	11:04:46	4
844536	2	28	0,1	sleep5	76	-21	ksoftirqd/5	11:19:53	5
844536	2	28	0,1	sleep5	76	-21	ksoftirqd/5	11:19:53	5
277469	99	28	26,2	cyclictest	500769	-21	kworker/u16:1+events_unbound	09:09:35	5
277469	99	28	26,2	cyclictest	462204	-21	kworker/u16:3+events_unbound	08:49:45	5
277461	99	28	25,2	cyclictest	979376	-21	kworker/u16:0+flush-8:0	12:29:13	4
277476	99	27	25,1	cyclictest	610179	-21	ssh	09:44:38	7
277476	99	27	25,1	cyclictest	510147	-21	diskmemload	09:29:39	7
277476	99	27	1,24	cyclictest	481487	-21	grep	08:49:18	7
277476	99	27	0,25	cyclictest	347817	-21	latency_hist	07:39:19	7
570427	2	26	0,0	sleep6	0	-21	swapper/6	09:29:18	6
277469	99	26	24,2	cyclictest	319601	-21	kworker/u16:2+flush-8:0	07:24:34	5
277469	99	26	24,2	cyclictest	277672	-21	kworker/u16:3+flush-8:0	07:09:23	5
277469	99	26	24,2	cyclictest	277672	-21	kworker/u16:3+flush-8:0	07:09:23	5
277469	99	26	23,2	cyclictest	386029	-21	kworker/u16:3+events_unbound	08:19:47	5
277469	99	26	23,2	cyclictest	3757659	-21	kworker/u16:1+events_unbound	08:24:38	5
277461	99	25	23,2	cyclictest	386029	-21	kworker/u16:3+flush-8:0	08:04:18	4
850309	2	24	0,1	chrt	850312	-21	kthreadcore	11:24:36	7
277469	99	24	22,2	cyclictest	773590	-21	kworker/u16:1+events_unbound	11:54:39	5
277469	99	24	22,2	cyclictest	568855	-21	kworker/u16:3+events_unbound	09:39:31	5
277469	99	24	22,2	cyclictest	319601	-21	kworker/u16:2+flush-8:0	07:54:46	5
277469	99	24	22,2	cyclictest	319601	-21	kworker/u16:2+flush-8:0	07:54:46	5
277469	99	24	22,2	cyclictest	277672	-21	kworker/u16:3+flush-8:0	07:29:47	5
277476	99	23	21,1	cyclictest	534673	-21	ssh	09:14:30	7
277469	99	23	21,2	cyclictest	462204	-21	kworker/u16:3+events_unbound	08:39:41	5
277469	99	23	21,2	cyclictest	3757659	-21	kworker/u16:1+flush-8:0	08:09:36	5
688055	2	22	0,1	chrt	688054	-21	ntpq	10:14:47	7
277476	99	22	17,4	cyclictest	667018	-21	latency_hist	10:09:18	7
277469	99	22	20,2	cyclictest	706709	-21	kworker/u16:0+events_unbound	11:06:19	5
277469	99	22	20,2	cyclictest	568856	-21	kworker/u16:5+events_unbound	10:14:40	5
277469	99	22	20,2	cyclictest	500769	-21	kworker/u16:1+events_unbound	09:59:31	5
277461	99	22	20,2	cyclictest	3757659	-21	kworker/u16:1+flush-8:0	08:09:41	4
277476	99	21	13,7	cyclictest	0	-21	swapper/7	11:59:18	7
277461	99	21	20,1	cyclictest	433937	-21	kworker/u16:0+events_unbound	08:29:33	4
277476	99	20	0,19	cyclictest	847222	-21	latency_hist	11:24:18	7
277476	99	20	0,19	cyclictest	847222	-21	latency_hist	11:24:18	7
277471	99	20	0,2	cyclictest	675234	-21	grep	10:09:38	6
277469	99	20	18,2	cyclictest	706709	-21	kworker/u16:0+flush-8:0	11:24:59	5
277469	99	20	18,2	cyclictest	631090	-21	kworker/u16:2+events_unbound	10:26:43	5
277469	99	20	18,2	cyclictest	568856	-21	kworker/u16:5+events_unbound	10:09:05	5
277469	99	20	18,2	cyclictest	568855	-21	kworker/u16:3+events_unbound	10:48:48	5
277469	99	20	18,2	cyclictest	490794	-21	kworker/u16:0+flush-8:0	08:59:47	5
277461	99	20	17,2	cyclictest	500769	-21	kworker/u16:1+flush-8:0	09:34:40	4
277461	99	20	16,3	cyclictest	809025	-21	kworker/u16:3+flush-8:0	11:55:54	4
713424	2	19	0,1	sleep7	713423	-21	ssh	10:26:09	7
277476	99	19	15,3	cyclictest	1267	-21	snmpd	08:58:09	7
277476	99	19	0,18	cyclictest	1003502	-21	cat	12:29:18	7
277471	99	19	0,19	cyclictest	0	-21	swapper/6	09:09:44	6
277469	99	19	17,2	cyclictest	895248	-21	kworker/u16:4+events_unbound	11:53:15	5
277469	99	19	17,2	cyclictest	500769	-21	kworker/u16:1+events_unbound	09:34:46	5
277469	99	19	17,2	cyclictest	3757659	-21	kworker/u16:1+flush-8:0	07:34:47	5
277469	99	19	14,4	cyclictest	568759	-21	packagekit-dnf-	09:26:20	5
277476	99	18	15,2	cyclictest	1267	-21	snmpd	09:12:54	7
277476	99	18	11,6	cyclictest	0	-21	swapper/7	12:29:44	7
277476	99	18	11,6	cyclictest	0	-21	swapper/7	12:04:35	7
277469	99	18	16,2	cyclictest	931342	-21	kworker/u16:0+events_unbound	12:05:16	5
277469	99	18	16,2	cyclictest	895248	-21	kworker/u16:4+events_unbound	11:59:42	5
277469	99	18	16,2	cyclictest	706709	-21	kworker/u16:0+events_unbound	11:39:42	5
277469	99	18	16,2	cyclictest	568855	-21	kworker/u16:3+events_unbound	09:45:23	5
277469	99	18	15,2	cyclictest	500769	-21	kworker/u16:1+flush-8:0	09:04:29	5
277469	99	18	15,2	cyclictest	277672	-21	kworker/u16:3+events_unbound	07:19:37	5
277461	99	18	16,2	cyclictest	568855	-21	kworker/u16:3+flush-8:0	10:42:33	4
277461	99	18	16,2	cyclictest	568855	-21	kworker/u16:3+flush-8:0	10:42:33	4
277461	99	18	16,2	cyclictest	490794	-21	kworker/u16:0+flush-8:0	08:59:43	4
277461	99	18	15,2	cyclictest	568855	-21	kworker/u16:3+events_unbound	09:44:40	4
277476	99	17	15,1	cyclictest	775626	-21	sh	10:54:24	7
277476	99	17	13,3	cyclictest	1267	-21	snmpd	09:00:33	7
277476	99	17	0,16	cyclictest	892232	-21	sh	11:39:48	7
277471	99	17	0,1	cyclictest	0	-21	swapper/6	08:34:34	6
277469	99	17	15,2	cyclictest	809025	-21	kworker/u16:3+events_unbound	11:35:20	5
277469	99	17	15,2	cyclictest	706709	-21	kworker/u16:0+events_unbound	10:53:55	5
277469	99	17	15,2	cyclictest	568856	-21	kworker/u16:5+flush-8:0	09:49:40	5
277469	99	17	15,2	cyclictest	568855	-21	kworker/u16:3+events_unbound	10:31:03	5
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17