OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #3, slot #1

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance

Characteristics of the 792 highest latencies:

System rack3slot1.osadl.org (updated Sun Nov 02, 2025 12:43:27)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
8230	99	104	102	cyclictest	32557	-21	tune2fs	00:52:43	1
8230	99	104	102	cyclictest	21736	-21	tune2fs	21:57:52	1
8232	99	103	102	cyclictest	0	-21	swapper/3	22:27:55	3
8232	99	103	102	cyclictest	0	-21	swapper/3	21:57:52	3
8232	99	103	102	cyclictest	0	-21	swapper/3	01:37:41	3
8232	99	93	92	cyclictest	0	-21	swapper/3	01:49:18	3
8230	99	93	81	cyclictest	0	-21	swapper/1	02:27:42	1
8232	99	92	91	cyclictest	0	-21	swapper/3	00:52:43	3
8230	99	85	73	cyclictest	12691	-21	aten2.4-expect	22:52:45	1
8232	99	84	82	cyclictest	0	-21	swapper/3	23:07:49	3
8232	99	84	72	cyclictest	0	-21	swapper/3	02:02:52	3
8230	99	84	83	cyclictest	0	-21	swapper/1	21:57:35	1
8230	99	84	82	cyclictest	23887	-21	tune2fs	22:02:48	1
8232	99	83	82	cyclictest	0	-21	swapper/3	23:57:46	3
8232	99	83	82	cyclictest	0	-21	swapper/3	23:42:45	3
8232	99	83	82	cyclictest	0	-21	swapper/3	23:37:47	3
8232	99	83	82	cyclictest	0	-21	swapper/3	23:27:55	3
8232	99	83	82	cyclictest	0	-21	swapper/3	23:27:32	3
8232	99	83	82	cyclictest	0	-21	swapper/3	23:03:04	3
8232	99	83	82	cyclictest	0	-21	swapper/3	22:47:49	3
8232	99	83	82	cyclictest	0	-21	swapper/3	22:42:52	3
8232	99	83	82	cyclictest	0	-21	swapper/3	22:38:01	3
8232	99	83	82	cyclictest	0	-21	swapper/3	22:17:48	3
8232	99	83	82	cyclictest	0	-21	swapper/3	22:12:53	3
8232	99	83	82	cyclictest	0	-21	swapper/3	22:02:48	3
8232	99	83	82	cyclictest	0	-21	swapper/3	21:57:35	3
8232	99	83	82	cyclictest	0	-21	swapper/3	21:42:59	3
8232	99	83	82	cyclictest	0	-21	swapper/3	21:37:51	3
8232	99	83	82	cyclictest	0	-21	swapper/3	21:27:51	3
8232	99	83	82	cyclictest	0	-21	swapper/3	02:52:42	3
8232	99	83	82	cyclictest	0	-21	swapper/3	02:47:42	3
8232	99	83	82	cyclictest	0	-21	swapper/3	02:42:47	3
8232	99	83	82	cyclictest	0	-21	swapper/3	02:30:32	3
8232	99	83	82	cyclictest	0	-21	swapper/3	02:22:42	3
8232	99	83	82	cyclictest	0	-21	swapper/3	02:12:48	3
8232	99	83	82	cyclictest	0	-21	swapper/3	02:07:41	3
8232	99	83	82	cyclictest	0	-21	swapper/3	01:42:48	3
8232	99	83	82	cyclictest	0	-21	swapper/3	01:32:27	3
8232	99	83	82	cyclictest	0	-21	swapper/3	01:17:48	3
8232	99	83	82	cyclictest	0	-21	swapper/3	01:07:51	3
8232	99	83	82	cyclictest	0	-21	swapper/3	01:02:44	3
8232	99	83	82	cyclictest	0	-21	swapper/3	00:50:18	3
8232	99	83	82	cyclictest	0	-21	swapper/3	00:42:52	3
8232	99	83	82	cyclictest	0	-21	swapper/3	00:32:51	3
8232	99	83	82	cyclictest	0	-21	swapper/3	00:27:52	3
8232	99	83	82	cyclictest	0	-21	swapper/3	00:22:53	3
8232	99	83	82	cyclictest	0	-21	swapper/3	00:19:51	3
8232	99	83	82	cyclictest	0	-21	swapper/3	00:12:43	3
8232	99	83	82	cyclictest	0	-21	swapper/3	00:12:30	3
8230	99	83	82	cyclictest	19922	-21	tune2fs	01:37:41	1
8230	99	83	82	cyclictest	0	-21	swapper/1	23:37:47	1
8230	99	83	82	cyclictest	0	-21	swapper/1	23:27:55	1
8230	99	83	82	cyclictest	0	-21	swapper/1	23:27:32	1
8230	99	83	82	cyclictest	0	-21	swapper/1	22:49:58	1
8230	99	83	82	cyclictest	0	-21	swapper/1	22:42:56	1
8230	99	83	82	cyclictest	0	-21	swapper/1	22:38:03	1
8230	99	83	82	cyclictest	0	-21	swapper/1	22:27:58	1
8230	99	83	82	cyclictest	0	-21	swapper/1	22:12:53	1
8230	99	83	82	cyclictest	0	-21	swapper/1	21:42:59	1
8230	99	83	82	cyclictest	0	-21	swapper/1	02:52:42	1
8230	99	83	82	cyclictest	0	-21	swapper/1	02:47:47	1
8230	99	83	82	cyclictest	0	-21	swapper/1	02:32:42	1
8230	99	83	82	cyclictest	0	-21	swapper/1	02:22:42	1
8230	99	83	82	cyclictest	0	-21	swapper/1	02:07:44	1
8230	99	83	82	cyclictest	0	-21	swapper/1	02:05:30	1
8230	99	83	82	cyclictest	0	-21	swapper/1	01:42:48	1
8230	99	83	82	cyclictest	0	-21	swapper/1	01:32:42	1
8230	99	83	82	cyclictest	0	-21	swapper/1	01:30:24	1
8230	99	83	82	cyclictest	0	-21	swapper/1	01:17:54	1
8230	99	83	82	cyclictest	0	-21	swapper/1	00:47:53	1
8230	99	83	82	cyclictest	0	-21	swapper/1	00:37:52	1
8230	99	83	82	cyclictest	0	-21	swapper/1	00:32:53	1
8230	99	83	82	cyclictest	0	-21	swapper/1	00:12:30	1
8230	99	83	81	cyclictest	25967	-21	tune2fs	22:07:48	1
8230	99	83	81	cyclictest	23575	-21	aten2.4-expect	23:17:44	1
8230	99	83	81	cyclictest	0	-21	swapper/1	01:25:45	1
8230	99	83	71	cyclictest	17501	-21	tune2fs	23:03:04	1
8232	99	82	81	cyclictest	0	-21	swapper/3	23:17:44	3
8232	99	82	81	cyclictest	0	-21	swapper/3	23:12:47	3
8232	99	82	81	cyclictest	0	-21	swapper/3	22:22:48	3
8232	99	82	81	cyclictest	0	-21	swapper/3	22:07:48	3
8232	99	82	81	cyclictest	0	-21	swapper/3	02:32:41	3
8232	99	82	81	cyclictest	0	-21	swapper/3	00:57:44	3
8232	99	82	81	cyclictest	0	-21	swapper/3	00:02:48	3
8230	99	82	81	cyclictest	8931	-21	tune2fs	23:57:46	1
8230	99	82	81	cyclictest	15563	-21	tune2fs	00:12:43	1
8230	99	82	81	cyclictest	0	-21	swapper/1	01:47:48	1
8230	99	82	71	cyclictest	0	-21	swapper/1	02:42:47	1
8232	99	80	79	cyclictest	0	-21	swapper/3	22:32:47	3
8230	99	75	73	cyclictest	21852	-21	tune2fs	23:12:47	1
8230	99	74	72	cyclictest	2347	-21	tune2fs	00:57:44	1
8232	99	73	72	cyclictest	0	-21	swapper/3	23:52:43	3
8232	99	73	72	cyclictest	0	-21	swapper/3	02:17:46	3
8232	99	73	72	cyclictest	0	-21	swapper/3	01:52:48	3
8232	99	73	72	cyclictest	0	-21	swapper/3	01:12:43	3
8232	99	73	72	cyclictest	0	-21	swapper/3	00:37:52	3
8232	99	73	71	cyclictest	12961	-21	aten2.4-expect	01:22:40	3
8230	99	73	72	cyclictest	0	-21	swapper/1	23:42:45	1
8230	99	73	72	cyclictest	0	-21	swapper/1	21:27:48	1
8230	99	73	72	cyclictest	0	-21	swapper/1	00:47:29	1
8230	99	73	71	cyclictest	4486	-21	tune2fs	23:47:45	1
8230	99	73	71	cyclictest	11119	-21	tune2fs	00:02:48	1
8230	99	73	71	cyclictest	0	-21	swapper/1	22:22:48	1
8230	99	73	71	cyclictest	0	-21	swapper/1	21:39:01	1
8230	99	73	71	cyclictest	0	-21	swapper/1	01:07:46	1
8230	99	73	71	cyclictest	0	-21	swapper/1	00:17:43	1
8232	99	72	71	cyclictest	0	-21	swapper/3	23:35:05	3
8232	99	72	71	cyclictest	0	-21	swapper/3	22:52:55	3
8232	99	72	71	cyclictest	0	-21	swapper/3	21:47:59	3
8232	99	72	71	cyclictest	0	-21	swapper/3	21:34:48	3
8232	99	72	71	cyclictest	0	-21	swapper/3	01:32:45	3
8230	99	72	71	cyclictest	0	-21	swapper/1	23:32:56	1
8230	99	72	71	cyclictest	0	-21	swapper/1	23:07:44	1
8230	99	72	71	cyclictest	0	-21	swapper/1	22:57:46	1
8230	99	72	71	cyclictest	0	-21	swapper/1	22:32:47	1
8230	99	72	71	cyclictest	0	-21	swapper/1	22:17:46	1
8230	99	72	71	cyclictest	0	-21	swapper/1	21:47:49	1
8230	99	72	71	cyclictest	0	-21	swapper/1	21:32:57	1
8230	99	72	71	cyclictest	0	-21	swapper/1	02:37:37	1
8230	99	72	71	cyclictest	0	-21	swapper/1	02:17:38	1
8230	99	72	71	cyclictest	0	-21	swapper/1	02:12:43	1
8230	99	72	71	cyclictest	0	-21	swapper/1	01:57:42	1
8230	99	72	71	cyclictest	0	-21	swapper/1	01:02:48	1
8230	99	72	71	cyclictest	0	-21	swapper/1	00:27:51	1
8230	99	72	71	cyclictest	0	-21	swapper/1	00:22:47	1
8232	99	64	62	cyclictest	0	-21	swapper/3	02:02:26	3
8232	99	63	62	cyclictest	0	-21	swapper/3	23:47:32	3
8232	99	63	62	cyclictest	0	-21	swapper/3	22:57:46	3
8232	99	63	62	cyclictest	0	-21	swapper/3	02:37:36	3
8231	99	63	41	cyclictest	0	-21	swapper/2	02:32:39	2
8230	99	63	62	cyclictest	0	-21	swapper/1	23:52:43	1
8230	99	63	62	cyclictest	0	-21	swapper/1	01:52:26	1
8230	99	63	61	cyclictest	0	-21	swapper/1	01:12:28	1
8229	99	63	62	cyclictest	0	-21	swapper/0	23:02:48	0
8231	99	62	61	cyclictest	0	-21	swapper/2	00:02:44	2
8229	99	61	60	cyclictest	0	-21	swapper/0	02:37:37	0
8231	99	59	57	cyclictest	25675	-21	telnet	23:22:43	2
8231	99	55	41	cyclictest	0	-21	swapper/2	23:27:43	2
8231	99	55	41	cyclictest	0	-21	swapper/2	23:13:44	2
8231	99	55	41	cyclictest	0	-21	swapper/2	22:42:57	2
8231	99	55	41	cyclictest	0	-21	swapper/2	22:13:18	2
8231	99	55	41	cyclictest	0	-21	swapper/2	02:40:06	2
8231	99	55	41	cyclictest	0	-21	swapper/2	01:52:02	2
8229	99	55	54	cyclictest	0	-21	swapper/0	01:27:42	0
8231	99	54	53	cyclictest	5035	-21	latency	22:32:52	2
8231	99	54	51	cyclictest	0	-21	swapper/2	21:53:23	2
8231	99	54	41	cyclictest	0	-21	swapper/2	01:10:01	2
8229	99	54	52	cyclictest	745	50	irq/18-parport0	23:37:09	0
8229	99	54	52	cyclictest	745	50	irq/18-parport0	22:44:09	0
8229	99	54	52	cyclictest	50	50	irq/44-ahci	23:21:47	0
8229	99	54	52	cyclictest	0	-21	swapper/0	00:12:40	0
8229	99	54	42	cyclictest	1388	-13	audispd	22:02:34	0
8231	99	53	52	cyclictest	275	-21	usb-storage	00:58:24	2
8231	99	53	52	cyclictest	0	-21	swapper/2	01:53:20	2
8231	99	53	51	cyclictest	50	50	irq/44-ahci	02:22:51	2
8231	99	53	41	cyclictest	8642	-21	telnet	02:27:37	2
8231	99	53	41	cyclictest	1747	-21	telnet	22:27:46	2
8231	99	53	41	cyclictest	12843	-21	telnet	01:22:39	2
8231	99	53	41	cyclictest	0	-21	swapper/2	23:57:51	2
8231	99	53	41	cyclictest	0	-21	swapper/2	23:52:42	2
8231	99	53	41	cyclictest	0	-21	swapper/2	23:47:43	2
8231	99	53	41	cyclictest	0	-21	swapper/2	23:46:38	2
8231	99	53	41	cyclictest	0	-21	swapper/2	23:37:45	2
8231	99	53	41	cyclictest	0	-21	swapper/2	23:32:48	2
8231	99	53	41	cyclictest	0	-21	swapper/2	23:17:43	2
8231	99	53	41	cyclictest	0	-21	swapper/2	23:07:43	2
8231	99	53	41	cyclictest	0	-21	swapper/2	23:02:34	2
8231	99	53	41	cyclictest	0	-21	swapper/2	22:57:45	2
8231	99	53	41	cyclictest	0	-21	swapper/2	22:52:46	2
8231	99	53	41	cyclictest	0	-21	swapper/2	22:47:47	2
8231	99	53	41	cyclictest	0	-21	swapper/2	22:37:54	2
8231	99	53	41	cyclictest	0	-21	swapper/2	22:22:45	2
8231	99	53	41	cyclictest	0	-21	swapper/2	22:17:47	2
8231	99	53	41	cyclictest	0	-21	swapper/2	22:07:46	2
8231	99	53	41	cyclictest	0	-21	swapper/2	22:02:45	2
8231	99	53	41	cyclictest	0	-21	swapper/2	21:57:49	2
8231	99	53	41	cyclictest	0	-21	swapper/2	21:47:48	2
8231	99	53	41	cyclictest	0	-21	swapper/2	21:42:47	2
8231	99	53	41	cyclictest	0	-21	swapper/2	21:37:49	2
8231	99	53	41	cyclictest	0	-21	swapper/2	21:32:42	2
8231	99	53	41	cyclictest	0	-21	swapper/2	21:27:48	2
8231	99	53	41	cyclictest	0	-21	swapper/2	02:52:36	2
8231	99	53	41	cyclictest	0	-21	swapper/2	02:47:37	2
8231	99	53	41	cyclictest	0	-21	swapper/2	02:42:36	2
8231	99	53	41	cyclictest	0	-21	swapper/2	02:17:36	2
8231	99	53	41	cyclictest	0	-21	swapper/2	02:12:30	2
8231	99	53	41	cyclictest	0	-21	swapper/2	02:07:38	2
8231	99	53	41	cyclictest	0	-21	swapper/2	02:02:29	2
8231	99	53	41	cyclictest	0	-21	swapper/2	01:57:41	2
8231	99	53	41	cyclictest	0	-21	swapper/2	01:42:41	2
8231	99	53	41	cyclictest	0	-21	swapper/2	01:37:32	2
8231	99	53	41	cyclictest	0	-21	swapper/2	01:32:39	2
8231	99	53	41	cyclictest	0	-21	swapper/2	01:27:41	2
8231	99	53	41	cyclictest	0	-21	swapper/2	01:17:44	2
8231	99	53	41	cyclictest	0	-21	swapper/2	01:13:19	2
8231	99	53	41	cyclictest	0	-21	swapper/2	01:02:41	2
8231	99	53	41	cyclictest	0	-21	swapper/2	00:52:41	2
8231	99	53	41	cyclictest	0	-21	swapper/2	00:47:44	2
8231	99	53	41	cyclictest	0	-21	swapper/2	00:42:41	2
8231	99	53	41	cyclictest	0	-21	swapper/2	00:37:44	2
8231	99	53	41	cyclictest	0	-21	swapper/2	00:32:48	2
8231	99	53	41	cyclictest	0	-21	swapper/2	00:27:41	2
8231	99	53	41	cyclictest	0	-21	swapper/2	00:22:44	2
8231	99	53	41	cyclictest	0	-21	swapper/2	00:17:42	2
8231	99	53	41	cyclictest	0	-21	swapper/2	00:12:40	2
8231	99	53	41	cyclictest	0	-21	swapper/2	00:07:41	2
8229	99	53	52	cyclictest	745	50	irq/18-parport0	23:38:07	0
8229	99	53	52	cyclictest	0	-21	swapper/0	23:52:43	0
8229	99	53	52	cyclictest	0	-21	swapper/0	23:22:43	0
8229	99	53	52	cyclictest	0	-21	swapper/0	23:07:41	0
8229	99	53	52	cyclictest	0	-21	swapper/0	21:52:48	0
8229	99	53	52	cyclictest	0	-21	swapper/0	21:37:49	0
8229	99	53	52	cyclictest	0	-21	swapper/0	21:35:39	0
8229	99	53	52	cyclictest	0	-21	swapper/0	02:12:37	0
8229	99	53	52	cyclictest	0	-21	swapper/0	02:07:38	0
8229	99	53	52	cyclictest	0	-21	swapper/0	01:37:39	0
8229	99	53	52	cyclictest	0	-21	swapper/0	00:57:42	0
8229	99	53	52	cyclictest	0	-21	swapper/0	00:52:40	0
8229	99	53	52	cyclictest	0	-21	swapper/0	00:42:42	0
8229	99	53	52	cyclictest	0	-21	swapper/0	00:22:42	0
8229	99	53	52	cyclictest	0	-21	swapper/0	00:02:45	0
8229	99	53	51	cyclictest	19434	-21	telnet	02:52:36	0
8229	99	53	41	cyclictest	745	50	irq/18-parport0	23:27:39	0
8229	99	53	41	cyclictest	745	50	irq/18-parport0	22:50:28	0
8229	99	53	41	cyclictest	0	-21	swapper/0	21:43:41	0
8229	99	53	41	cyclictest	0	-21	swapper/0	02:22:48	0
8229	99	52	51	cyclictest	6422	-21	telnet	01:07:41	0
8229	99	52	51	cyclictest	27953	-21	telnet	01:57:40	0
8229	99	52	51	cyclictest	0	-21	swapper/0	23:50:54	0
8229	99	52	51	cyclictest	0	-21	swapper/0	23:42:52	0
8229	99	52	51	cyclictest	0	-21	swapper/0	23:12:45	0
8229	99	52	51	cyclictest	0	-21	swapper/0	22:52:46	0
8229	99	52	51	cyclictest	0	-21	swapper/0	22:34:21	0
8229	99	52	51	cyclictest	0	-21	swapper/0	22:27:35	0
8229	99	52	51	cyclictest	0	-21	swapper/0	22:22:49	0
8229	99	52	51	cyclictest	0	-21	swapper/0	22:07:54	0
8229	99	52	51	cyclictest	0	-21	swapper/0	21:49:01	0
8229	99	52	51	cyclictest	0	-21	swapper/0	21:27:52	0
8229	99	52	51	cyclictest	0	-21	swapper/0	02:51:55	0
8229	99	52	51	cyclictest	0	-21	swapper/0	02:32:48	0
8229	99	52	51	cyclictest	0	-21	swapper/0	02:27:43	0
8229	99	52	51	cyclictest	0	-21	swapper/0	02:17:45	0
8229	99	52	51	cyclictest	0	-21	swapper/0	02:02:44	0
8229	99	52	51	cyclictest	0	-21	swapper/0	01:42:44	0
8229	99	52	51	cyclictest	0	-21	swapper/0	01:33:04	0
8229	99	52	51	cyclictest	0	-21	swapper/0	01:23:21	0
8229	99	52	51	cyclictest	0	-21	swapper/0	01:12:40	0
8229	99	52	51	cyclictest	0	-21	swapper/0	01:02:45	0
8229	99	52	51	cyclictest	0	-21	swapper/0	00:47:43	0
8229	99	52	51	cyclictest	0	-21	swapper/0	00:37:48	0
8229	99	52	51	cyclictest	0	-21	swapper/0	00:32:42	0
8229	99	52	51	cyclictest	0	-21	swapper/0	00:27:45	0
8229	99	52	51	cyclictest	0	-21	swapper/0	00:17:50	0
8229	99	52	51	cyclictest	0	-21	swapper/0	00:10:15	0
8229	99	52	51	cyclictest	0	-21	swapper/0	00:01:50	0
8229	99	52	50	cyclictest	0	-21	swapper/0	22:19:10	0
8229	99	51	50	cyclictest	0	-21	swapper/0	22:40:22	0
8229	99	51	50	cyclictest	0	-21	swapper/0	02:42:40	0
8229	99	51	50	cyclictest	0	-21	swapper/0	01:52:40	0
8229	99	51	50	cyclictest	0	-21	swapper/0	01:47:48	0
8229	99	51	50	cyclictest	0	-21	swapper/0	01:17:35	0
8229	99	51	41	cyclictest	0	-21	swapper/0	22:13:20	0
8229	99	49	48	cyclictest	0	-21	swapper/0	22:57:54	0
8229	99	48	47	cyclictest	0	-21	swapper/0	22:04:09	0
8	99	13	0	watchdog/0	7916	-40	yum-updatesd-he	21:25:49	0
21	99	11	0	migration/3	6037	-21	munin-node	21:22:47	3
50	50	9	0	irq/44-ahci	0	-21	swapper/2	21:24:30	2
12	1	8	0	ksoftirqd/1	2996	-21	runrttasks	21:22:50	1
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17