OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #f, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails

Click here to display the system's profile data.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rackfslot7s.osadl.org (updated Sat Apr 04, 2026 00:44:16)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
176591	99	3172	15,3152	cyclictest	218999	-21	kworker/3:0+events	21:25:13	3
176576	99	3007	15,2988	cyclictest	205703	-21	kworker/0:0+events	21:14:59	0
176591	99	2949	14,2932	cyclictest	255418	-21	kworker/3:1+events	23:15:16	3
176582	99	2831	8,2819	cyclictest	242577	-21	kworker/1:0+events	22:35:13	1
176591	99	2817	13,2799	cyclictest	208786	-21	kworker/3:1+events	20:55:16	3
176586	99	2776	13,2759	cyclictest	179664	-21	kworker/2:0+events	19:45:01	2
176586	99	2761	13,2744	cyclictest	237768	-21	kworker/2:0+events	22:50:00	2
176586	99	2737	13,2720	cyclictest	257230	-21	kworker/2:1+events	23:35:12	2
176586	99	2717	7,2706	cyclictest	179664	-21	kworker/2:0+events	20:05:17	2
176586	99	2717	12,2701	cyclictest	239488	-21	kworker/2:1+events	22:25:17	2
176576	99	2713	13,2697	cyclictest	279709	-21	kworker/0:2+events	00:35:19	0
176591	99	2696	12,2679	cyclictest	263556	-21	kworker/3:1+events	23:40:11	3
176582	99	2696	12,2681	cyclictest	247215	-21	kworker/1:1+events	23:10:00	1
176576	99	2680	13,2664	cyclictest	177918	-21	kworker/0:1+events	19:15:01	0
176582	99	2640	13,2624	cyclictest	178172	-21	kworker/1:3+events	19:30:15	1
176582	99	2634	13,2616	cyclictest	247215	-21	kworker/1:1+events	23:20:17	1
176582	99	2624	13,2607	cyclictest	203017	-21	kworker/1:1+events	20:50:16	1
176591	99	2609	12,2593	cyclictest	202581	-21	kworker/3:1+events	20:31:52	3
176582	99	2580	12,2564	cyclictest	264938	-21	kworker/1:2+events	23:42:26	1
176582	99	2569	8,2559	cyclictest	178172	-21	kworker/1:3+events	19:20:18	1
176591	99	2552	10,2538	cyclictest	270287	-21	kworker/3:2+events	00:03:01	3
176591	99	2531	12,2515	cyclictest	207115	-21	kworker/3:0+events	20:50:40	3
176586	99	2526	9,2513	cyclictest	257230	-21	kworker/2:1+events	23:28:21	2
176591	99	2521	12,2505	cyclictest	253800	-21	kworker/3:2+events	23:14:55	3
176591	99	2501	11,2486	cyclictest	270013	-21	kworker/3:1+events	23:58:49	3
176582	99	2498	12,2483	cyclictest	224520	-21	kworker/1:2+events	21:39:13	1
176576	99	2496	12,2481	cyclictest	258589	-21	kworker/0:1+events	23:25:16	0
176586	99	2494	11,2479	cyclictest	242755	-21	kworker/2:1+events	22:39:19	2
176582	99	2491	12,2475	cyclictest	267152	-21	kworker/1:1+events	23:56:31	1
176582	99	2490	12,2474	cyclictest	183193	-21	kworker/1:1+events	19:44:15	1
176582	99	2487	12,2472	cyclictest	187992	-21	kworker/1:2+events	19:53:11	1
176582	99	2485	11,2471	cyclictest	231338	-21	kworker/1:1+events	22:11:58	1
176582	99	2484	11,2469	cyclictest	201011	-21	kworker/1:0+events	20:33:47	1
176576	99	2482	12,2466	cyclictest	267565	-21	kworker/0:2+events	23:56:54	0
176582	99	2478	12,2462	cyclictest	178172	-21	kworker/1:3+events	19:27:15	1
176582	99	2472	11,2459	cyclictest	216615	-21	kworker/1:0+events	21:20:45	1
176586	99	2467	11,2453	cyclictest	237768	-21	kworker/2:0+events	22:20:21	2
176582	99	2465	12,2450	cyclictest	227842	-21	kworker/1:2+events	21:56:44	1
176586	99	2462	13,2446	cyclictest	264854	-21	kworker/2:2+events	23:52:42	2
176586	99	2452	12,2436	cyclictest	268613	-21	kworker/2:1+events	23:56:08	2
176586	99	2451	10,2437	cyclictest	219903	-21	kworker/2:2+events	21:48:21	2
176591	99	2448	10,2434	cyclictest	202581	-21	kworker/3:1+events	20:39:29	3
176586	99	2448	12,2432	cyclictest	210570	-21	kworker/2:2+events	20:59:28	2
176586	99	2447	12,2432	cyclictest	179664	-21	kworker/2:0+events	20:02:38	2
176576	99	2444	12,2428	cyclictest	210636	-21	kworker/0:1+events	21:00:15	0
176576	99	2436	12,2421	cyclictest	234527	-21	kworker/0:0+events	22:21:05	0
176582	99	2435	12,2419	cyclictest	240966	-21	kworker/1:1+events	22:29:29	1
176591	99	2429	12,2414	cyclictest	208786	-21	kworker/3:1+events	20:49:31	3
176591	99	2428	10,2415	cyclictest	215343	-21	kworker/3:1+events	21:32:46	3
176591	99	2426	11,2411	cyclictest	279618	-21	kworker/3:1+events	00:33:46	3
176591	99	2423	12,2408	cyclictest	223071	-21	kworker/3:0+events	21:39:37	3
176591	99	2423	12,2407	cyclictest	276343	-21	kworker/3:2+events	00:26:32	3
176586	99	2423	12,2407	cyclictest	183373	-21	kworker/2:1+events	19:34:30	2
176591	99	2421	12,2406	cyclictest	239519	-21	kworker/3:0+events	22:27:56	3
176591	99	2421	11,2406	cyclictest	191585	-21	kworker/3:2+events	19:57:18	3
176591	99	2421	11,2406	cyclictest	186314	-21	kworker/3:1+events	19:41:18	3
176591	99	2420	12,2404	cyclictest	223071	-21	kworker/3:0+events	21:44:37	3
176591	99	2419	12,2403	cyclictest	279618	-21	kworker/3:1+events	00:37:35	3
176591	99	2419	11,2405	cyclictest	216965	-21	kworker/3:2+events	21:19:37	3
176586	99	2415	12,2399	cyclictest	201922	-21	kworker/2:2+events	20:32:38	2
176586	99	2414	12,2399	cyclictest	207296	-21	kworker/2:0+events	20:51:49	2
176591	99	2413	12,2398	cyclictest	265276	-21	kworker/3:3+events	23:46:36	3
176591	99	2413	12,2397	cyclictest	181654	-21	kworker/3:0+events	19:26:28	3
176591	99	2412	10,2399	cyclictest	252214	-21	kworker/3:0+events	23:02:05	3
176591	99	2409	11,2395	cyclictest	196026	-21	kworker/3:0+events	20:12:26	3
176582	99	2409	12,2393	cyclictest	193098	-21	kworker/1:1+events	20:10:15	1
176586	99	2407	12,2392	cyclictest	197928	-21	kworker/2:0+events	20:26:58	2
176586	99	2402	12,2387	cyclictest	237768	-21	kworker/2:0+events	22:55:41	2
176582	99	2400	9,2387	cyclictest	271777	-21	kworker/1:0+events	00:10:54	1
176586	99	2394	12,2378	cyclictest	279493	-21	kworker/2:2+events	00:36:49	2
176586	99	2394	11,2379	cyclictest	223130	-21	kworker/2:0+events	21:35:47	2
176586	99	2392	12,2378	cyclictest	227990	-21	kworker/2:1+events	22:05:03	2
176591	99	2388	10,2374	cyclictest	265276	-21	kworker/3:3+events	00:07:52	3
176586	99	2388	11,2374	cyclictest	237768	-21	kworker/2:0+events	22:41:55	2
176586	99	2387	11,2373	cyclictest	227990	-21	kworker/2:1+events	21:58:37	2
176586	99	2385	9,2374	cyclictest	260538	-21	kworker/2:0+events	00:00:19	2
176586	99	2384	12,2369	cyclictest	215016	-21	kworker/2:1+events	21:19:15	2
176586	99	2384	11,2370	cyclictest	215016	-21	kworker/2:1+events	21:13:55	2
176586	99	2382	11,2367	cyclictest	197928	-21	kworker/2:0+events	20:23:33	2
176591	99	2372	10,2359	cyclictest	245823	-21	kworker/3:1+events	22:44:13	3
176586	99	2367	12,2351	cyclictest	273047	-21	kworker/2:1+events	00:06:44	2
176576	99	2367	12,2352	cyclictest	188246	-21	kworker/0:3+events	19:50:57	0
176586	99	2366	11,2351	cyclictest	179664	-21	kworker/2:0+events	19:26:05	2
176586	99	2361	9,2348	cyclictest	207296	-21	kworker/2:0+events	21:00:59	2
176582	99	2357	12,2342	cyclictest	271777	-21	kworker/1:0+events	00:30:00	1
176582	99	2357	12,2342	cyclictest	227842	-21	kworker/1:2+events	22:01:39	1
176586	99	2352	11,2337	cyclictest	179664	-21	kworker/2:0+events	19:51:20	2
176586	99	2350	12,2335	cyclictest	201922	-21	kworker/2:2+events	20:42:35	2
176586	99	2349	12,2334	cyclictest	204067	-21	kworker/2:1+events	20:37:38	2
176582	99	2349	12,2334	cyclictest	264938	-21	kworker/1:2+events	23:50:47	1
176586	99	2348	12,2333	cyclictest	281128	-21	kworker/2:0+events	00:33:00	2
176591	99	2344	11,2330	cyclictest	233046	-21	kworker/3:2+events	22:09:18	3
176586	99	2343	11,2328	cyclictest	251010	-21	kworker/2:2+events	23:16:26	2
176576	99	2340	12,2324	cyclictest	239302	-21	kworker/0:1+events	22:51:33	0
176576	99	2336	12,2322	cyclictest	199573	-21	kworker/0:1+events	20:30:00	0
176582	99	2335	12,2323	cyclictest	254111	-21	kworker/1:2+events	23:10:47	1
176576	99	2334	12,2319	cyclictest	190267	-21	kworker/0:0+events	20:15:55	0
176576	99	2331	12,2315	cyclictest	269950	-21	kworker/0:1+events	00:11:17	0
176576	99	2329	12,2314	cyclictest	247772	-21	kworker/0:2+events	23:12:40	0
176576	99	2328	12,2312	cyclictest	241087	-21	kworker/0:2+events	22:30:14	0
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2024-07-29 - 23:00