You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-17 - 19:01

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #f, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails
  
Click here to display the system's profile data.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackfslot7s.osadl.org (updated Fri Apr 17, 2026 00:44:19)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
217892199323515,3215cyclictest2267194-21kworker/2:2+events23:50:162
217892599288714,2869cyclictest2213135-21kworker/3:1+events20:55:173
217891499288111,2866cyclictest2242108-21kworker/0:2+events22:25:190
217892199282613,2809cyclictest2273343-21kworker/2:0+events00:05:202
217892599280213,2787cyclictest2242406-21kworker/3:0+events22:45:193
217892599274013,2723cyclictest2224204-21kworker/3:2+events21:30:113
217891899273413,2719cyclictest2250483-21kworker/1:1+events22:50:171
21789189927309,2717cyclictest2190367-21kworker/1:1+events20:20:171
217892599271313,2696cyclictest2224204-21kworker/3:2+events21:27:113
21789189927109,2698cyclictest2234480-21kworker/1:1+events22:00:131
217891899268013,2663cyclictest2191989-21kworker/1:2+events19:49:231
21789219926798,2667cyclictest2200226-21kworker/2:0+events20:25:162
217891899267813,2661cyclictest2220773-21kworker/1:0+events21:31:201
217891899264513,2629cyclictest2236126-21kworker/1:2+events22:35:011
217892199262713,2611cyclictest2237630-21kworker/2:2+events22:10:212
217892599262213,2606cyclictest2213135-21kworker/3:1+events21:15:013
217892599259713,2581cyclictest2275025-21kworker/3:1+events00:10:013
217892199257412,2558cyclictest2273343-21kworker/2:0+events00:11:092
217892199257412,2558cyclictest2273343-21kworker/2:0+events00:11:082
217892199257112,2555cyclictest2221049-21kworker/2:1+events21:53:032
217892199257112,2555cyclictest2221049-21kworker/2:1+events21:53:022
217892199256811,2553cyclictest2229199-21kworker/2:2+events21:48:022
217892199256712,2551cyclictest2281578-21kworker/2:0+events00:22:512
217892199256412,2548cyclictest2192108-21kworker/2:2+events19:52:022
217892199255910,2545cyclictest2232644-21kworker/2:0+events21:56:492
217892199254812,2532cyclictest2273343-21kworker/2:0+events00:02:182
217892199252912,2514cyclictest2257038-21kworker/2:2+events23:15:192
217892199251411,2499cyclictest2161016-21kworker/2:1+events19:23:242
217892599251212,2498cyclictest2185725-21kworker/3:2+events19:51:173
217892199249812,2482cyclictest2237630-21kworker/2:2+events22:21:272
217892199249612,2481cyclictest2258670-21kworker/2:1+events23:35:262
217892199249312,2478cyclictest2212874-21kworker/2:2+events21:01:172
217891499249312,2479cyclictest2180444-21kworker/0:0+events19:32:090
217892199249212,2477cyclictest2232644-21kworker/2:0+events22:02:532
217892199249012,2474cyclictest2221049-21kworker/2:1+events21:24:542
217892199248813,2471cyclictest2188725-21kworker/2:1+events19:48:132
217892199248812,2474cyclictest2200226-21kworker/2:0+events20:22:122
217892199248711,2472cyclictest2193930-21kworker/2:1+events20:09:582
217892199248512,2469cyclictest2257038-21kworker/2:2+events23:08:072
217892199248511,2471cyclictest2222671-21kworker/2:2+events21:32:062
217892199248412,2469cyclictest2253814-21kworker/2:1+events23:04:212
217892599248312,2468cyclictest2250331-21kworker/3:2+events22:54:063
217892199248312,2468cyclictest2251842-21kworker/2:2+events22:54:292
217892199248111,2466cyclictest2187093-21kworker/2:0+events19:33:142
217892599247812,2463cyclictest2242406-21kworker/3:0+events22:28:403
217892599247811,2463cyclictest2199985-21kworker/3:0+events20:12:173
217892599247810,2465cyclictest2284723-21kworker/3:1+events00:31:593
217892199247811,2463cyclictest2251842-21kworker/2:2+events22:56:432
217892199247712,2461cyclictest2221049-21kworker/2:1+events21:36:112
217892199247711,2462cyclictest2242166-21kworker/2:1+events22:30:532
217892199247612,2461cyclictest2182183-21kworker/2:0+events19:19:342
217892199247611,2461cyclictest2245394-21kworker/2:2+events22:44:092
217892199247310,2461cyclictest2237630-21kworker/2:2+events22:15:452
21789189924739,2461cyclictest2278683-21kworker/1:1+events00:18:221
217892199247212,2457cyclictest2217811-21kworker/2:0+events21:16:312
217892199247211,2458cyclictest2200226-21kworker/2:0+events20:16:312
217892599247112,2455cyclictest2235981-21kworker/3:2+events22:18:013
217892599247012,2455cyclictest2250331-21kworker/3:2+events22:55:593
21789219924699,2456cyclictest2247172-21kworker/2:1+events22:48:462
217892199246912,2454cyclictest2200226-21kworker/2:0+events20:45:282
217892199246412,2449cyclictest2185427-21kworker/2:2+events19:26:252
217891899246112,2445cyclictest2183918-21kworker/1:0+events19:25:151
217891899245012,2434cyclictest2271868-21kworker/1:2+events00:06:071
217892199244612,2431cyclictest2212874-21kworker/2:2+events21:08:322
217892199243813,2422cyclictest2279689-21kworker/2:2+events00:19:062
217891899243712,2422cyclictest2271868-21kworker/1:2+events23:57:231
217891899243712,2422cyclictest2271868-21kworker/1:2+events23:57:231
217891899243512,2421cyclictest2280788-21kworker/1:0+events00:25:071
217891899243512,2420cyclictest2190367-21kworker/1:1+events20:11:531
21789189924347,2423cyclictest2276899-21kworker/1:3+events00:13:241
21789189924347,2423cyclictest2276899-21kworker/1:3+events00:13:241
217891899243310,2420cyclictest2193617-21kworker/1:0+events19:54:411
217891899243212,2417cyclictest2230984-21kworker/1:1+events21:52:401
217891899243212,2417cyclictest2230984-21kworker/1:1+events21:52:391
217892599242611,2411cyclictest2185725-21kworker/3:2+events20:06:513
21789189924259,2412cyclictest2211459-21kworker/1:2+events20:51:541
217891899242312,2408cyclictest2209827-21kworker/1:0+events20:44:191
217892599242112,2406cyclictest2263789-21kworker/3:1+events23:46:033
217892599242112,2406cyclictest2242406-21kworker/3:0+events22:37:173
217892599242111,2406cyclictest2235981-21kworker/3:2+events22:13:483
217892599241911,2405cyclictest2244048-21kworker/3:1+events22:33:533
217892599241811,2404cyclictest2284723-21kworker/3:1+events00:36:393
217891899241312,2397cyclictest2255589-21kworker/1:0+events23:20:151
217892599240912,2394cyclictest2227813-21kworker/3:2+events21:42:383
217891899240912,2393cyclictest2204700-21kworker/1:1+events20:29:031
217892599240611,2392cyclictest2213135-21kworker/3:1+events21:20:203
217892599240512,2389cyclictest2235981-21kworker/3:2+events22:09:353
217892599240412,2391cyclictest2218729-21kworker/3:0+events21:19:113
217892599240212,2386cyclictest2265376-21kworker/3:2+events23:38:523
217892199240210,2390cyclictest2192108-21kworker/2:2+events19:55:052
217892599240012,2386cyclictest2187329-21kworker/3:1+events19:35:573
217892599239912,2385cyclictest2214737-21kworker/3:2+events21:00:333
217892599239912,2383cyclictest2231637-21kworker/3:0+events21:54:113
217892599239912,2383cyclictest2231637-21kworker/3:0+events21:54:103
21789189923999,2387cyclictest2219175-21kworker/1:1+events21:13:501
217892599239812,2382cyclictest2263789-21kworker/3:1+events23:40:473
217892599239810,2385cyclictest2263789-21kworker/3:1+events23:53:393
217892599239810,2384cyclictest2213135-21kworker/3:1+events21:06:593
217892599239711,2383cyclictest2255562-21kworker/3:2+events23:06:583
217892599239711,2383cyclictest2209941-21kworker/3:0+events20:53:013
217892599239711,2383cyclictest2187329-21kworker/3:1+events19:44:013
217892599239611,2382cyclictest2218729-21kworker/3:0+events21:38:273
217892599239610,2382cyclictest2185725-21kworker/3:2+events20:00:473
217892599239512,2380cyclictest2257249-21kworker/3:0+events23:33:343
217892599239512,2380cyclictest2180204-21kworker/3:0+events19:29:283
217892599239412,2379cyclictest2253936-21kworker/3:1+events23:03:363
217892599239411,2379cyclictest2279901-21kworker/3:0+events00:20:343
217892599239212,2376cyclictest2206453-21kworker/3:1+events20:38:143
217892599239210,2380cyclictest2185725-21kworker/3:2+events19:45:103
217892599239112,2376cyclictest2257249-21kworker/3:0+events23:18:443
217892599238912,2374cyclictest2257249-21kworker/3:0+events23:29:243
217892599238912,2373cyclictest2234333-21kworker/3:1+events21:58:463
21789189923899,2376cyclictest2260943-21kworker/1:2+events23:29:001
217891899238912,2374cyclictest2224266-21kworker/1:2+events21:42:151
217892599238612,2368cyclictest2180204-21kworker/3:0+events19:31:233
217891899238612,2370cyclictest2273555-21kworker/1:1+events00:00:241
217891899238512,2370cyclictest2234480-21kworker/1:1+events22:11:281
217892599238112,2366cyclictest2180204-21kworker/3:0+events19:24:073
217891899238112,2366cyclictest2206331-21kworker/1:2+events20:35:331
217892599237711,2363cyclictest2278267-21kworker/3:2+events00:13:483
217892599237711,2363cyclictest2278267-21kworker/3:2+events00:13:483
217891899237711,2364cyclictest2260943-21kworker/1:2+events23:30:081
217891899237512,2360cyclictest2227783-21kworker/1:0+events21:59:291
217892199237311,2358cyclictest2283211-21kworker/2:2+events00:38:322
217891499237010,2356cyclictest2205101-21kworker/0:0+events20:42:050
217892199236810,2355cyclictest2198543-21kworker/2:2+events20:14:342
217892199236712,2351cyclictest2258670-21kworker/2:1+events23:33:562
217891899236610,2353cyclictest2196916-21kworker/1:0+events20:09:341
217892599236312,2348cyclictest2248681-21kworker/3:1+events22:44:343
217892199236312,2347cyclictest2206391-21kworker/2:1+events20:36:172
217891899236311,2348cyclictest2190367-21kworker/1:1+events19:43:141
217891899236112,2345cyclictest2276899-21kworker/1:3+events00:22:061
217892599236012,2344cyclictest2281921-21kworker/3:3+events00:27:493
217891899235412,2339cyclictest2263517-21kworker/1:1+events23:49:531
217891899235312,2338cyclictest2222400-21kworker/1:1+events21:24:081
217891899235312,2338cyclictest2180503-21kworker/1:3+events19:16:361
217891899235311,2338cyclictest2211459-21kworker/1:2+events21:01:421
217892199235212,2337cyclictest2221049-21kworker/2:1+events21:40:452
21789189923529,2339cyclictest2253981-21kworker/1:2+events22:59:241
217891899235210,2338cyclictest2276899-21kworker/1:3+events00:39:181
217891899235112,2336cyclictest2245334-21kworker/1:0+events22:38:251
217891899235011,2336cyclictest2257221-21kworker/1:2+events23:10:261
217891899235010,2336cyclictest2250483-21kworker/1:1+events23:01:381
217891899234912,2334cyclictest2195223-21kworker/1:2+events20:02:391
217891899234712,2332cyclictest2245334-21kworker/1:0+events22:41:071
217891899234711,2333cyclictest2268643-21kworker/1:0+events23:53:151
217891499234712,2332cyclictest2227931-21kworker/0:1+events22:20:430
217892199234611,2331cyclictest2196693-21kworker/2:0+events20:04:562
217891899234612,2332cyclictest2234480-21kworker/1:1+events22:05:061
217892199234510,2332cyclictest2242166-21kworker/2:1+events22:26:492
217891899233811,2323cyclictest2211459-21kworker/1:2+events21:18:481
217891499233612,2322cyclictest2271952-21kworker/0:2+events00:01:340
217891499233610,2322cyclictest2216298-21kworker/0:1+events21:07:450
217892199233512,2320cyclictest2188725-21kworker/2:1+events19:43:372
217891899233411,2320cyclictest2172358-21kworker/1:2+events19:36:191
217892199232611,2312cyclictest2212874-21kworker/2:2+events21:12:202
21789219923256,2315cyclictest2203693-21kworker/2:2+events20:31:192
217891499232412,2309cyclictest2193531-21kworker/0:0+events20:01:540
21789259923218,2309cyclictest2275025-21kworker/3:1+events00:03:023
21789189923166,2307cyclictest2267015-21kworker/1:2+events23:39:371
217891499231512,2301cyclictest2187123-21kworker/0:2+events19:40:100
217892599230210,2289cyclictest2271719-21kworker/3:0+events23:59:203
217892599230210,2289cyclictest2271719-21kworker/3:0+events23:59:193
21789259922939,2280cyclictest2206453-21kworker/3:1+events20:46:143
217892599228812,2273cyclictest2201775-21kworker/3:1+events20:19:553
217892199228710,2273cyclictest2257038-21kworker/2:2+events23:13:032
217892199227510,2262cyclictest2258670-21kworker/2:1+events23:44:122
217892199227411,2260cyclictest2257038-21kworker/2:2+events23:22:332
217892199227411,2258cyclictest2267194-21kworker/2:2+events23:45:432
217892199227310,2259cyclictest2200226-21kworker/2:0+events20:40:102
217892199227211,2258cyclictest2212874-21kworker/2:2+events20:55:402
217892199227211,2258cyclictest2188725-21kworker/2:1+events19:39:252
217892199227210,2258cyclictest2245394-21kworker/2:2+events22:35:442
217891499226711,2253cyclictest2188937-21kworker/0:1+events19:53:120
217892199226211,2248cyclictest2222671-21kworker/2:2+events21:26:242
21789259922598,2247cyclictest2260412-21kworker/3:1+events23:21:233
217891499225510,2242cyclictest2268550-21kworker/0:0+events23:44:580
217892599224911,2235cyclictest2194482-21kworker/3:0+events19:58:073
217892199224812,2233cyclictest2258670-21kworker/2:1+events23:57:012
217892199224812,2233cyclictest2258670-21kworker/2:1+events23:57:012
217892599222511,2211cyclictest2199985-21kworker/3:0+events20:29:253
217892599221710,2204cyclictest2208041-21kworker/3:2+events20:43:133
217891899221712,2201cyclictest2242138-21kworker/1:1+events22:24:081
217891499221410,2202cyclictest2278473-21kworker/0:0+events00:30:070
21789189922038,2191cyclictest2222400-21kworker/1:1+events21:25:411
217891899220211,2188cyclictest2236126-21kworker/1:2+events22:29:031
217892599219811,2184cyclictest2235981-21kworker/3:2+events22:03:593
217892599219810,2185cyclictest2255562-21kworker/3:2+events23:11:553
21789259921969,2184cyclictest2206453-21kworker/3:1+events20:34:003
217892599219611,2181cyclictest2199985-21kworker/3:0+events20:21:043
217892599219511,2181cyclictest2180204-21kworker/3:0+events19:15:283
21789219921916,2182cyclictest2258670-21kworker/2:1+events23:28:372
217892599217711,2162cyclictest2278267-21kworker/3:2+events00:16:273
217891899216810,2154cyclictest2236126-21kworker/1:2+events22:16:531
217891899216611,2152cyclictest2204700-21kworker/1:1+events20:45:511
21789189921658,2153cyclictest2258815-21kworker/1:1+events23:19:081
217891899216411,2150cyclictest2216358-21kworker/1:0+events21:05:501
217891899216411,2149cyclictest2227783-21kworker/1:0+events21:48:271
217891899216310,2151cyclictest2183918-21kworker/1:0+events19:30:591
21789219921627,2152cyclictest2211029-21kworker/2:1+events20:54:532
21789189921628,2150cyclictest2172358-21kworker/1:2+events19:23:451
217891899216210,2148cyclictest2213108-21kworker/1:0+events20:59:041
21789189921619,2149cyclictest2227783-21kworker/1:0+events21:39:361
217891899215911,2145cyclictest2276899-21kworker/1:3+events00:30:281
217891899215910,2146cyclictest2267015-21kworker/1:2+events23:41:321
217891899215910,2146cyclictest2255589-21kworker/1:0+events23:07:441
217891899215810,2144cyclictest2178462-21kworker/1:1+events19:12:501
217891899215511,2140cyclictest2203097-21kworker/1:0+events20:30:571
217891899215411,2140cyclictest2248853-21kworker/1:2+events22:48:221
21789189921529,2140cyclictest2199179-21kworker/1:2+events20:16:551
217891899215211,2137cyclictest2190367-21kworker/1:1+events19:56:131
217892199213314,2116cyclictest2283211-21kworker/2:2+events00:32:452
217892599212211,2108cyclictest2229438-21kworker/3:1+events21:47:393
217892199212110,2108cyclictest2232644-21kworker/2:0+events22:05:512
21789149921199,2106cyclictest2220868-21kworker/0:0+events21:39:580
217891499211910,2106cyclictest2209798-21kworker/0:2+events21:04:220
217891499211811,2104cyclictest2265667-21kworker/0:1+events00:17:360
217891499211710,2104cyclictest2222521-21kworker/0:1+events21:23:450
217891499211611,2102cyclictest2257335-21kworker/0:0+events23:26:210
217891499211611,2102cyclictest2224418-21kworker/0:2+events21:30:340
217892199211010,2096cyclictest2161016-21kworker/2:1+events19:12:292
217891499210810,2095cyclictest2227931-21kworker/0:1+events23:12:180
21789149921069,2094cyclictest2209798-21kworker/0:2+events21:17:180
21789149921056,2096cyclictest2263548-21kworker/0:2+events23:35:500
217891499209110,2078cyclictest2201656-21kworker/0:1+events20:32:050
217892199208610,2073cyclictest2281578-21kworker/2:0+events00:27:022
217891499205310,2039cyclictest2265667-21kworker/0:1+events23:54:210
21789259920446,2035cyclictest2239173-21kworker/3:1+events22:23:223
217892599202910,2016cyclictest2180204-21kworker/3:0+events19:13:563
21789149919609,1948cyclictest2220868-21kworker/0:0+events21:28:420
21789149919468,1935cyclictest2196605-21kworker/0:2+events20:17:410
21789149919337,1922cyclictest2284938-21kworker/0:1+events00:38:550
21789149918802,1875cyclictest2171983-21kworker/0:2+events19:16:590
21786762148102,26sleep00-21swapper/019:09:420
2178647214693,16sleep30-21swapper/319:09:213
21786112145111,28sleep20-21swapper/219:08:522
21786102140109,26sleep10-21swapper/119:08:511
2178914998380,2cyclictest2278473-21kworker/0:0+pm00:29:560
2178914994239,2cyclictest2187123-21kworker/0:2+pm19:40:000
2178914992720,5cyclictest2252270-21awk22:55:000
2178914992617,7cyclictest2270465-21awk23:49:590
2178914992518,5cyclictest740-21snmpd22:10:330
2178914992514,4cyclictest2278022-21sed00:10:210
2178914992514,4cyclictest2278022-21sed00:10:210
217891499238,7cyclictest0-21swapper/000:20:020
217891499225,4cyclictest2198942-21kworker/u16:1+events_unbound20:35:210
2178914992216,4cyclictest740-21snmpd20:51:540
217891499212,9cyclictest2197913-21gcc20:05:160
2178914992110,8cyclictest2255712-21idleruntime-cro23:05:000
217891499209,8cyclictest2254132-21sendmail23:00:000
217891499203,11cyclictest2237331-21users22:05:200
2178914992010,7cyclictest2273627-40ogrotate23:59:590
2178914992010,7cyclictest2273627-40ogrotate00:00:000
217891499199,8cyclictest2195287-21munin-run20:00:000
217891499198,7cyclictest2245629-21munin-run22:35:000
217891499197,8cyclictest2234538-21cron22:00:000
2178914991911,5cyclictest2204113-21meminfo20:25:150
217891499188,8cyclictest2240878-21sh22:20:000
217891499188,8cyclictest2200298-21idleruntime-cro20:15:010
217891499187,8cyclictest2257278-21munin-run23:10:000
217891499184,9cyclictest2259870-21ntpq23:15:180
217891499183,5cyclictest2228746-21ld21:40:170
2178914991811,3cyclictest2219531-21perf21:14:590
2178914991810,6cyclictest0-21swapper/022:38:530
217891499179,5cyclictest2214786-21idleruntime-cro21:00:000
217891499174,4cyclictest2183468-21ntpq19:20:180
2178914991711,4cyclictest2211364-21runrttasks20:46:470
217891499170,15cyclictest2202350-21df_inode20:20:120
217891499168,5cyclictest2276881-21idleruntime-cro00:09:590
2178914991614,1cyclictest2231638-21/usr/sbin/munin21:50:190
2178914991614,1cyclictest2231638-21/usr/sbin/munin21:50:190
2178914991610,4cyclictest2247749-21apt-get22:40:110
217891499160,14cyclictest2231124-21cron21:50:010
217891499158,4cyclictest2235756-21users22:00:220
217891499151,2cyclictest0-21swapper/019:25:420
217891499150,13cyclictest2265108-21runrttasks23:30:560
217891499147,4cyclictest2180379-21awk19:15:000
217891499141,2cyclictest622-21gdbus22:45:070
217891499140,2cyclictest2260953-21/usr/sbin/munin23:20:110
217891499136,5cyclictest2191352-21chrt19:45:180
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional