OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #0, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack0slot2.osadl.org (updated Sun Jun 28, 2026 12:43:34)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
1056856	2	52	42,7	sleep4	0	-21	swapper/4	07:08:04	4
1057238	99	49	47,2	cyclictest	788447	-21	kworker/u16:1+flush-8:0	09:24:15	5
1057236	99	44	42,2	cyclictest	1307675	-21	kworker/u16:4+events_unbound	10:54:17	4
1057236	99	44	42,2	cyclictest	1307675	-21	kworker/u16:4+events_unbound	10:45:45	4
1056916	2	43	9,29	sleep7	0	-21	swapper/7	07:08:55	7
1057236	99	42	40,2	cyclictest	1368487	-21	kworker/u16:1+events_unbound	09:46:38	4
1057236	99	39	37,2	cyclictest	1443448	-21	kworker/u16:5+events_unbound	10:20:58	4
1057236	99	38	36,2	cyclictest	1653106	-21	kworker/u16:3+events_unbound	11:58:45	4
1056905	2	38	28,7	sleep5	0	-21	swapper/5	07:08:45	5
1057236	99	37	35,2	cyclictest	1776629	-21	kworker/u16:2+events_unbound	12:37:13	4
1057236	99	37	35,2	cyclictest	1368487	-21	kworker/u16:1+events_unbound	10:02:26	4
1057236	99	37	35,2	cyclictest	1307675	-21	kworker/u16:4+events_unbound	11:21:37	4
1057236	99	37	33,3	cyclictest	1527793	-21	kworker/u16:0+events_unbound	10:53:09	4
1057238	99	36	34,2	cyclictest	1653107	-21	kworker/u16:5+events_unbound	11:51:25	5
1057238	99	36	34,2	cyclictest	1320297	-21	kworker/u16:2+events_unbound	10:23:26	5
1057236	99	36	34,2	cyclictest	788447	-21	kworker/u16:1+events_unbound	09:28:14	4
1057236	99	36	34,2	cyclictest	1307675	-21	kworker/u16:4+events_unbound	12:14:13	4
1057236	99	36	34,2	cyclictest	1307675	-21	kworker/u16:4+events_unbound	12:05:49	4
1057236	99	36	34,2	cyclictest	1307675	-21	kworker/u16:4+events_unbound	10:41:29	4
1057236	99	36	34,2	cyclictest	1307675	-21	kworker/u16:4+events_unbound	10:16:13	4
1057236	99	35	33,2	cyclictest	1761463	-21	kworker/u16:0+events_unbound	12:24:38	4
1057236	99	35	33,2	cyclictest	1761463	-21	kworker/u16:0+events_unbound	12:19:12	4
1057236	99	35	33,2	cyclictest	1653106	-21	kworker/u16:3+events_unbound	12:00:30	4
1057236	99	35	33,2	cyclictest	1566607	-21	kworker/u16:2+events_unbound	11:16:14	4
1057236	99	35	33,2	cyclictest	1443448	-21	kworker/u16:5+events_unbound	10:34:15	4
1057236	99	35	33,2	cyclictest	1443448	-21	kworker/u16:5+events_unbound	10:34:15	4
1057236	99	35	33,2	cyclictest	1368487	-21	kworker/u16:1+events_unbound	11:11:45	4
1057236	99	35	33,2	cyclictest	1368487	-21	kworker/u16:1+events_unbound	11:11:45	4
1057236	99	35	33,2	cyclictest	1320297	-21	kworker/u16:2+events_unbound	09:21:14	4
1057236	99	35	33,2	cyclictest	1307675	-21	kworker/u16:4+flush-8:0	10:59:57	4
1057236	99	35	33,2	cyclictest	1307675	-21	kworker/u16:4+events_unbound	09:56:26	4
1056775	2	35	25,7	sleep6	0	-21	swapper/6	07:06:54	6
1057238	99	34	32,2	cyclictest	1368487	-21	kworker/u16:1+events_unbound	11:13:42	5
1057238	99	34	32,2	cyclictest	1368487	-21	kworker/u16:1+events_unbound	11:13:42	5
1057236	99	34	33,1	cyclictest	1505301	-21	kworker/u16:2+events_unbound	10:36:46	4
1057236	99	34	32,2	cyclictest	1566607	-21	kworker/u16:2+events_unbound	11:07:50	4
1057236	99	34	32,2	cyclictest	1307675	-21	kworker/u16:4+events_unbound	11:32:14	4
1057236	99	34	32,2	cyclictest	1307675	-21	kworker/u16:4+events_unbound	09:17:37	4
1057236	99	33	31,2	cyclictest	1653107	-21	kworker/u16:5+events_unbound	11:50:42	4
1057236	99	33	31,2	cyclictest	1641083	-21	kworker/u16:0+events_unbound	11:35:45	4
1057236	99	33	31,2	cyclictest	1368487	-21	kworker/u16:1+flush-8:0	10:24:41	4
1057236	99	33	31,2	cyclictest	1368487	-21	kworker/u16:1+events_unbound	11:29:02	4
1057236	99	33	31,2	cyclictest	1320297	-21	kworker/u16:2+events_unbound	09:40:42	4
1057236	99	33	31,2	cyclictest	1320297	-21	kworker/u16:2+events_unbound	09:38:57	4
1057236	99	33	31,2	cyclictest	1320297	-21	kworker/u16:2+events_unbound	09:32:18	4
1057236	99	33	31,2	cyclictest	1307675	-21	kworker/u16:4+events_unbound	09:50:54	4
1057246	99	32	26,5	cyclictest	1285481	-21	latency_hist	09:04:15	7
1057246	99	32	0,30	cyclictest	650	-21	systemd-journal	08:34:15	7
1057246	99	32	0,30	cyclictest	1813185	-21	latency_hist	12:39:15	7
1057238	99	32	30,2	cyclictest	1527793	-21	kworker/u16:0+flush-8:0	10:49:25	5
1057236	99	32	30,2	cyclictest	1641083	-21	kworker/u16:0+events_unbound	11:46:44	4
1057236	99	32	30,2	cyclictest	1418666	-21	kworker/u16:3+events_unbound	10:04:54	4
1057236	99	32	30,2	cyclictest	1368487	-21	kworker/u16:1+events_unbound	10:09:48	4
1057236	99	32	30,2	cyclictest	1307675	-21	kworker/u16:4+events_unbound	12:29:49	4
1057236	99	32	30,2	cyclictest	1307675	-21	kworker/u16:4+events_unbound	12:29:49	4
1057246	99	31	26,4	cyclictest	1542010	-21	latency_hist	10:49:15	7
1057238	99	31	29,2	cyclictest	1690163	-21	kworker/u16:1+events_unbound	12:16:24	5
1057238	99	31	29,2	cyclictest	1418666	-21	kworker/u16:3+events_unbound	10:06:26	5
1057238	99	31	28,2	cyclictest	1368487	-21	kworker/u16:1+events_unbound	11:34:15	5
1057236	99	31	29,2	cyclictest	1653106	-21	kworker/u16:3+events_unbound	11:44:09	4
1057236	99	31	29,2	cyclictest	1079631	-21	kworker/u16:4+flush-8:0	07:19:16	4
1057238	99	30	29,1	cyclictest	1566607	-21	kworker/u16:2+events_unbound	11:27:34	5
1057238	99	30	28,2	cyclictest	1394006	-21	kworker/u16:0+flush-8:0	09:53:25	5
1057238	99	30	28,2	cyclictest	1307675	-21	kworker/u16:4+events_unbound	10:14:49	5
1057238	99	30	27,2	cyclictest	1800894	-21	kworker/u16:5+flush-8:0	12:34:25	5
1057236	99	30	28,2	cyclictest	788447	-21	kworker/u16:1+flush-8:0	08:49:16	4
1057236	99	30	27,2	cyclictest	1761463	-21	kworker/u16:0+events_unbound	12:19:16	4
1057238	99	29	27,2	cyclictest	1505301	-21	kworker/u16:2+events_unbound	10:39:14	5
1057238	99	29	27,2	cyclictest	1368487	-21	kworker/u16:1+events_unbound	10:01:14	5
1057238	99	29	27,2	cyclictest	1307675	-21	kworker/u16:4+events_unbound	10:59:14	5
1057236	99	29	27,2	cyclictest	1285520	-21	kworker/u16:0+flush-8:0	09:10:17	4
1057236	99	29	26,3	cyclictest	1110180	-21	kworker/u16:2+flush-8:0	07:34:30	4
1057236	99	29	26,3	cyclictest	1110180	-21	kworker/u16:2+flush-8:0	07:34:30	4
1057238	99	28	26,2	cyclictest	1653107	-21	kworker/u16:5+events_unbound	11:56:06	5
1057238	99	28	26,2	cyclictest	1368487	-21	kworker/u16:1+events_unbound	10:09:18	5
1057238	99	28	26,2	cyclictest	1307675	-21	kworker/u16:4+flush-8:0	09:29:15	5
1057238	99	28	25,2	cyclictest	1307675	-21	kworker/u16:4+events_unbound	11:04:25	5
1057246	99	27	23,3	cyclictest	1677693	-21	cat	11:44:10	7
1057238	99	27	25,2	cyclictest	1690163	-21	kworker/u16:1+events_unbound	12:28:25	5
1057238	99	27	25,2	cyclictest	1641083	-21	kworker/u16:0+events_unbound	11:48:58	5
1057238	99	27	25,2	cyclictest	1307675	-21	kworker/u16:4+events_unbound	10:45:24	5
1057238	99	27	24,2	cyclictest	1505301	-21	kworker/u16:2+events_unbound	10:39:28	5
1057238	99	26	24,2	cyclictest	1714804	-21	kworker/u16:0+events_unbound	12:03:09	5
1057238	99	26	24,2	cyclictest	1368487	-21	kworker/u16:1+events_unbound	10:26:13	5
1057238	99	26	24,2	cyclictest	1307675	-21	kworker/u16:4+flush-8:0	10:29:37	5
1057238	99	26	24,2	cyclictest	1307675	-21	kworker/u16:4+flush-8:0	10:29:37	5
1057238	99	26	24,2	cyclictest	1307675	-21	kworker/u16:4+events_unbound	12:29:45	5
1057238	99	26	24,2	cyclictest	1307675	-21	kworker/u16:4+events_unbound	12:29:45	5
1057238	99	26	24,2	cyclictest	1307675	-21	kworker/u16:4+events_unbound	12:07:25	5
1057238	99	26	24,2	cyclictest	1307675	-21	kworker/u16:4+events_unbound	11:03:13	5
1057238	99	26	24,2	cyclictest	1285520	-21	kworker/u16:0+flush-8:0	09:09:58	5
1057238	99	26	22,3	cyclictest	1616843	-21	wc	11:19:26	5
1057243	99	25	22,2	cyclictest	1320297	-21	kworker/u16:2+events_unbound	10:09:22	6
1057238	99	25	23,2	cyclictest	1320297	-21	kworker/u16:2+events_unbound	09:42:14	5
1057243	99	24	22,2	cyclictest	788447	-21	kworker/u16:1+flush-8:0	07:14:34	6
1057238	99	24	22,2	cyclictest	1307675	-21	kworker/u16:4+events_unbound	09:56:38	5
1057238	99	24	22,2	cyclictest	1129856	-21	kworker/u16:3+flush-8:0	07:44:41	5
1057236	99	24	22,2	cyclictest	1195944	-21	kworker/u16:3+events_unbound	08:14:51	4
1057236	99	24	22,2	cyclictest	1099967	-21	kworker/u16:0+events_unbound	07:29:15	4
1057236	99	24	21,2	cyclictest	788447	-21	kworker/u16:1+events_unbound	08:54:29	4
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17