OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #f, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails

Click here to display the system's profile data.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rackfslot7s.osadl.org (updated Mon May 11, 2026 00:44:17)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
988875	99	3439	16,3419	cyclictest	1025597	-21	kworker/3:0+events	21:10:15	3
988873	99	3062	15,3044	cyclictest	992447	-21	kworker/2:3+events	19:20:00	2
988875	99	3049	13,3031	cyclictest	1069029	-21	kworker/3:1+events	23:20:18	3
988875	99	2987	14,2971	cyclictest	1009986	-21	kworker/3:2+events	20:25:16	3
988865	99	2952	13,2934	cyclictest	1065657	-21	kworker/0:2+events	23:10:17	0
988875	99	2903	14,2885	cyclictest	1067388	-21	kworker/3:0+events	23:15:22	3
988875	99	2873	14,2853	cyclictest	1009986	-21	kworker/3:2+events	20:35:13	3
988873	99	2858	12,2842	cyclictest	1083775	-21	kworker/2:1+events	00:30:18	2
988873	99	2850	14,2833	cyclictest	1051730	-21	kworker/2:1+events	22:30:01	2
988875	99	2817	13,2800	cyclictest	1045323	-21	kworker/3:2+events	22:05:13	3
988868	99	2816	13,2798	cyclictest	997097	-21	kworker/1:0+events	19:40:16	1
988865	99	2805	12,2789	cyclictest	1080349	-21	kworker/0:0+events	00:05:17	0
988875	99	2765	13,2748	cyclictest	1034104	-21	kworker/3:3+events	21:50:47	3
988875	99	2765	13,2748	cyclictest	1034104	-21	kworker/3:3+events	21:50:47	3
988875	99	2762	13,2745	cyclictest	1073859	-21	kworker/3:1+events	23:33:21	3
988865	99	2724	13,2709	cyclictest	1033838	-21	kworker/0:2+events	21:50:01	0
988865	99	2724	13,2709	cyclictest	1033838	-21	kworker/0:2+events	21:50:01	0
988865	99	2720	12,2702	cyclictest	1074039	-21	kworker/0:2+events	23:35:14	0
988873	99	2702	11,2687	cyclictest	1025807	-21	kworker/2:2+events	21:05:20	2
988873	99	2680	13,2664	cyclictest	1069088	-21	kworker/2:1+events	23:30:17	2
988865	99	2669	13,2653	cyclictest	991515	-21	kworker/0:2+events	19:30:17	0
988868	99	2651	13,2635	cyclictest	992293	-21	kworker/1:1+events	19:35:17	1
988868	99	2644	13,2628	cyclictest	1027324	-21	kworker/1:1+events	21:25:01	1
988875	99	2634	12,2619	cyclictest	1056218	-21	kworker/3:0+events	22:40:18	3
988865	99	2631	13,2614	cyclictest	1000295	-21	kworker/0:0+events	19:44:02	0
988865	99	2628	13,2611	cyclictest	1078568	-21	kworker/0:2+events	23:53:28	0
988865	99	2624	12,2609	cyclictest	1033838	-21	kworker/0:2+events	21:46:11	0
988873	99	2594	12,2579	cyclictest	1057901	-21	kworker/2:2+events	22:44:26	2
988875	99	2588	13,2573	cyclictest	1088200	-21	kworker/3:0+events	00:35:19	3
988868	99	2584	12,2568	cyclictest	1067203	-21	kworker/1:2+events	23:24:09	1
988868	99	2578	12,2562	cyclictest	1024395	-21	kworker/1:2+events	21:01:53	1
988868	99	2576	12,2560	cyclictest	1072406	-21	kworker/1:0+events	23:28:46	1
988865	99	2574	9,2562	cyclictest	1080349	-21	kworker/0:0+events	00:00:21	0
988875	99	2568	12,2552	cyclictest	1001919	-21	kworker/3:2+events	19:49:41	3
988875	99	2568	12,2552	cyclictest	1001919	-21	kworker/3:2+events	19:49:41	3
988868	99	2567	12,2553	cyclictest	1005217	-21	kworker/1:1+events	20:00:24	1
988868	99	2563	12,2548	cyclictest	1042003	-21	kworker/1:2+events	21:54:09	1
988868	99	2563	12,2548	cyclictest	1042003	-21	kworker/1:2+events	21:54:09	1
988875	99	2559	9,2546	cyclictest	1019338	-21	kworker/3:2+events	20:58:49	3
988868	99	2559	10,2545	cyclictest	1059319	-21	kworker/1:1+events	22:48:41	1
988875	99	2553	11,2540	cyclictest	1034104	-21	kworker/3:3+events	21:40:01	3
988868	99	2553	12,2537	cyclictest	1034613	-21	kworker/1:2+events	21:31:13	1
988868	99	2550	12,2535	cyclictest	1016263	-21	kworker/1:0+events	20:38:37	1
988868	99	2549	12,2533	cyclictest	1064327	-21	kworker/1:0+events	23:08:22	1
988868	99	2549	12,2533	cyclictest	1062677	-21	kworker/1:1+events	23:04:13	1
988868	99	2546	12,2532	cyclictest	992293	-21	kworker/1:1+events	19:22:16	1
988868	99	2545	12,2530	cyclictest	1086834	-21	kworker/1:0+events	00:25:47	1
988873	99	2544	12,2528	cyclictest	1050101	-21	kworker/2:0+events	22:45:12	2
988865	99	2536	9,2523	cyclictest	1043633	-21	kworker/0:2+events	22:06:21	0
988875	99	2535	12,2520	cyclictest	1056218	-21	kworker/3:0+events	22:38:50	3
988865	99	2535	12,2520	cyclictest	1054609	-21	kworker/0:2+events	22:38:28	0
988875	99	2533	10,2519	cyclictest	1065717	-21	kworker/3:2+events	23:13:01	3
988875	99	2532	3,2525	cyclictest	1043663	-21	kworker/3:1+events	21:59:30	3
988875	99	2532	12,2516	cyclictest	1034104	-21	kworker/3:3+events	21:30:49	3
988875	99	2531	7,2520	cyclictest	1009986	-21	kworker/3:2+events	20:31:00	3
988875	99	2531	12,2516	cyclictest	992047	-21	kworker/3:2+events	19:18:32	3
988875	99	2531	12,2516	cyclictest	1037137	-21	kworker/3:0+events	21:37:00	3
988868	99	2531	12,2516	cyclictest	1027324	-21	kworker/1:1+events	21:13:58	1
988865	99	2531	10,2517	cyclictest	1027140	-21	kworker/0:2+events	21:20:26	0
988875	99	2530	12,2515	cyclictest	1049797	-21	kworker/3:2+events	22:16:36	3
988865	99	2530	11,2515	cyclictest	1040338	-21	kworker/0:0+events	22:02:35	0
988865	99	2529	12,2514	cyclictest	1016022	-21	kworker/0:1+events	20:34:26	0
988873	99	2528	12,2512	cyclictest	1069088	-21	kworker/2:1+events	23:52:42	2
988868	99	2528	11,2515	cyclictest	1080141	-21	kworker/1:2+events	00:15:08	1
988865	99	2528	12,2512	cyclictest	1088232	-21	kworker/0:2+events	00:18:54	0
988865	99	2528	10,2514	cyclictest	1024237	-21	kworker/0:1+events	21:00:44	0
988875	99	2527	12,2512	cyclictest	1025597	-21	kworker/3:0+events	21:17:00	3
988865	99	2527	8,2515	cyclictest	1010015	-21	kworker/0:1+events	20:22:40	0
988865	99	2526	12,2510	cyclictest	1091520	-21	kworker/0:2+events	00:31:06	0
988865	99	2524	12,2508	cyclictest	1033838	-21	kworker/0:2+events	21:29:42	0
988865	99	2519	12,2504	cyclictest	1022636	-21	kworker/0:2+events	20:55:01	0
988875	99	2518	12,2502	cyclictest	1078869	-21	kworker/3:0+events	23:58:51	3
988865	99	2518	12,2503	cyclictest	1059624	-21	kworker/0:0+events	22:50:37	0
988865	99	2518	11,2503	cyclictest	1010015	-21	kworker/0:1+events	20:15:55	0
988875	99	2514	11,2499	cyclictest	1093027	-21	kworker/3:1+events	00:34:32	3
988868	99	2512	12,2496	cyclictest	997097	-21	kworker/1:0+events	19:45:54	1
988868	99	2512	12,2496	cyclictest	997097	-21	kworker/1:0+events	19:45:54	1
988875	99	2511	8,2499	cyclictest	1088200	-21	kworker/3:0+events	00:22:43	3
988875	99	2510	12,2494	cyclictest	997212	-21	kworker/3:1+events	19:35:39	3
988875	99	2509	12,2493	cyclictest	1059502	-21	kworker/3:0+events	22:49:26	3
988875	99	2507	12,2491	cyclictest	1051412	-21	kworker/3:1+events	22:24:43	3
988868	99	2507	12,2492	cyclictest	1030582	-21	kworker/1:2+events	21:24:16	1
988865	99	2505	11,2491	cyclictest	1011613	-21	kworker/0:2+events	20:25:40	0
988875	99	2504	12,2488	cyclictest	1051412	-21	kworker/3:1+events	22:31:10	3
988868	99	2500	10,2487	cyclictest	1089793	-21	kworker/1:1+events	00:32:58	1
988868	99	2499	5,2490	cyclictest	1053158	-21	kworker/1:0+events	22:32:17	1
988875	99	2498	12,2482	cyclictest	1067388	-21	kworker/3:0+events	23:25:42	3
988868	99	2498	10,2485	cyclictest	1072406	-21	kworker/1:0+events	23:34:04	1
988873	99	2497	12,2482	cyclictest	995512	-21	kworker/2:1+events	19:27:39	2
988875	99	2495	7,2484	cyclictest	1077202	-21	kworker/3:2+events	23:44:20	3
988868	99	2495	12,2480	cyclictest	1043601	-21	kworker/1:1+events	22:06:00	1
988868	99	2495	12,2479	cyclictest	1080141	-21	kworker/1:2+events	23:54:16	1
988868	99	2494	12,2480	cyclictest	1016263	-21	kworker/1:0+events	20:49:17	1
988868	99	2494	12,2478	cyclictest	1034613	-21	kworker/1:2+events	21:43:31	1
988865	99	2493	12,2478	cyclictest	991515	-21	kworker/0:2+events	19:21:34	0
988873	99	2492	12,2476	cyclictest	1006605	-21	kworker/2:2+events	20:11:02	2
988868	99	2492	12,2478	cyclictest	1006844	-21	kworker/1:0+events	20:12:10	1
988868	99	2492	12,2476	cyclictest	1022368	-21	kworker/1:1+events	20:54:38	1
988868	99	2491	12,2476	cyclictest	1000382	-21	kworker/1:1+events	19:53:07	1
988868	99	2491	12,2475	cyclictest	1056455	-21	kworker/1:0+events	22:54:24	1
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2024-07-29 - 23:00