You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-06-16 - 20:07

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #f, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails
  
Click here to display the system's profile data.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackfslot7s.osadl.org (updated Mon Jun 16, 2025 00:44:18)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
226952999299212,2975cyclictest2338268-21kworker/3:0+events22:45:223
226952499289414,2875cyclictest2338091-21kworker/2:1+events22:50:212
226951799281813,2799cyclictest2301183-21kworker/0:1+events20:50:140
226951799276613,2748cyclictest2310791-21kworker/0:1+events21:35:190
226952999271613,2699cyclictest2276013-21kworker/3:2+events20:05:183
226952999271212,2695cyclictest2276013-21kworker/3:2+events20:15:133
226952199267913,2664cyclictest2302732-21kworker/1:0+events21:20:011
226951799266912,2654cyclictest2325075-21kworker/0:2+events22:15:170
226952999249812,2483cyclictest2287197-21kworker/3:0+events20:10:173
22695299924958,2484cyclictest2296479-21kworker/3:1+events20:35:003
226951799248612,2471cyclictest2347877-21kworker/0:2+events23:14:030
226951799244910,2436cyclictest2372015-21kworker/0:0+events00:30:160
226951799244412,2428cyclictest2354051-21kworker/0:1+events23:39:230
226951799243811,2424cyclictest2364166-21kworker/0:1+events00:07:010
226951799242512,2409cyclictest2306312-21kworker/0:0+events21:06:140
226952199242412,2410cyclictest2282393-21kworker/1:1+events19:55:201
226952999242012,2403cyclictest2314112-21kworker/3:0+events21:30:183
226951799241311,2399cyclictest2367009-21kworker/0:0+events00:16:590
226951799240511,2391cyclictest2297900-21kworker/0:0+events20:47:320
226952999238812,2374cyclictest2314112-21kworker/3:0+events21:40:123
226952199236212,2348cyclictest2350958-21kworker/1:2+events23:40:081
22695219923568,2344cyclictest2326800-21kworker/1:2+events22:10:191
226952199235112,2336cyclictest2279191-21kworker/1:0+events19:45:251
226952199233610,2323cyclictest2277590-21kworker/1:1+events19:38:541
226952199231512,2300cyclictest2290193-21kworker/1:1+events20:17:521
22695179923119,2299cyclictest2322299-21kworker/0:0+events22:01:080
22695219922879,2275cyclictest2288471-21kworker/1:2+events20:38:491
22695219922879,2275cyclictest2288471-21kworker/1:2+events20:38:481
226952199227811,2263cyclictest2275984-21kworker/1:0+events19:31:171
226952199227311,2258cyclictest2330095-21kworker/1:1+events22:32:451
226951799226310,2250cyclictest2279161-21kworker/0:0+events19:43:520
226952199226111,2246cyclictest2326800-21kworker/1:2+events22:08:461
226951799225611,2242cyclictest2268952-21kworker/0:1+events19:16:300
226951799224411,2230cyclictest2326768-21kworker/0:1+events22:33:320
22695179922429,2229cyclictest2359177-21kworker/0:0+events23:48:280
226951799224211,2227cyclictest2372015-21kworker/0:0+events00:36:230
22695179922409,2228cyclictest2354051-21kworker/0:1+events23:44:410
226951799223811,2224cyclictest2308022-21kworker/0:2+events21:14:170
226951799223811,2223cyclictest2279161-21kworker/0:0+events19:46:100
226951799223810,2225cyclictest2315567-21kworker/0:0+events21:43:160
22695179922378,2226cyclictest2302793-21kworker/0:2+events20:55:340
226951799223711,2222cyclictest2333236-21kworker/0:0+events22:38:270
226951799223710,2223cyclictest2359177-21kworker/0:0+events23:50:220
22695179922369,2224cyclictest2279161-21kworker/0:0+events20:09:070
22695179922369,2223cyclictest2275865-21kworker/0:1+events19:33:560
226951799223510,2222cyclictest2339831-21kworker/0:0+events22:47:150
226951799223411,2220cyclictest2302793-21kworker/0:2+events21:00:540
226951799223410,2221cyclictest2279161-21kworker/0:0+events20:26:170
226951799223311,2219cyclictest2331604-21kworker/0:2+events22:26:190
226951799223311,2219cyclictest2331604-21kworker/0:2+events22:26:180
22695179922309,2217cyclictest2343074-21kworker/0:2+events22:58:000
226951799222910,2215cyclictest2279161-21kworker/0:0+events20:19:240
226951799222211,2207cyclictest2279161-21kworker/0:0+events19:59:320
22695299922216,2212cyclictest2368904-21kworker/3:2+events00:33:423
22695179922198,2208cyclictest2339831-21kworker/0:0+events22:54:100
226951799221910,2206cyclictest2275865-21kworker/0:1+events19:29:450
22695179922179,2204cyclictest2354051-21kworker/0:1+events23:34:030
226951799221410,2201cyclictest2299526-21kworker/0:2+events20:42:570
226951799221110,2198cyclictest2270662-21kworker/0:2+events19:23:180
226951799221110,2197cyclictest2344695-21kworker/0:1+events23:04:090
226951799220911,2196cyclictest2310791-21kworker/0:1+events21:23:020
226951799220710,2194cyclictest2295057-21kworker/0:1+events20:33:510
226952999220110,2188cyclictest2368904-21kworker/3:2+events00:21:343
226952499219611,2182cyclictest2334561-21kworker/2:0+events22:37:422
226951799219610,2183cyclictest2368646-21kworker/0:1+events00:21:110
226952999219210,2179cyclictest2328193-21kworker/3:1+events22:11:293
226951799219010,2178cyclictest2347877-21kworker/0:2+events23:16:420
226952999218911,2174cyclictest2352714-21kworker/3:1+events23:43:573
22695299921867,2175cyclictest2277650-21kworker/3:0+events19:38:313
226951799218611,2172cyclictest2277560-21kworker/0:2+events19:35:280
226952999218511,2171cyclictest2323603-21kworker/3:1+events21:58:303
226952999218411,2169cyclictest2325254-21kworker/3:0+events22:31:383
226952999218011,2166cyclictest2293215-21kworker/3:2+events20:37:393
226952999218011,2166cyclictest2293215-21kworker/3:2+events20:37:393
226952999217610,2163cyclictest2344578-21kworker/3:1+events23:04:543
226951799217510,2162cyclictest2288647-21kworker/0:1+events20:20:340
226952999217311,2159cyclictest2293215-21kworker/3:2+events20:42:373
226952999217210,2159cyclictest2352714-21kworker/3:1+events23:47:453
226952999216710,2153cyclictest2298168-21kworker/3:0+events20:59:463
226952999216510,2153cyclictest2298168-21kworker/3:0+events21:10:493
226952999216311,2149cyclictest2329854-21kworker/3:2+events22:17:333
226952999216211,2147cyclictest2325254-21kworker/3:0+events22:06:533
226952999216111,2147cyclictest2298168-21kworker/3:0+events21:05:323
226952199216010,2147cyclictest2350958-21kworker/1:2+events23:31:221
226951799216010,2147cyclictest2349461-21kworker/0:0+events23:25:590
226952199215810,2144cyclictest2307910-21kworker/1:2+events21:09:401
226952999215710,2144cyclictest2333343-21kworker/3:2+events22:39:133
226952199215710,2139cyclictest2347725-21kworker/1:1+events23:19:251
226952999215511,2140cyclictest2351110-21kworker/3:0+events23:28:173
226952999215511,2140cyclictest2309215-21kworker/3:1+events21:16:353
226952999215510,2142cyclictest2342951-21kworker/3:0+events23:05:393
226952199215511,2140cyclictest2330095-21kworker/1:1+events22:24:011
226952199215411,2140cyclictest2362345-21kworker/1:0+events00:12:431
226952199215411,2139cyclictest2350958-21kworker/1:2+events23:39:001
226952199215411,2139cyclictest2297868-21kworker/1:1+events20:46:451
226952999215311,2138cyclictest2344578-21kworker/3:1+events23:19:023
226952999215310,2140cyclictest2287197-21kworker/3:0+events20:20:583
22695219921539,2140cyclictest2297868-21kworker/1:1+events20:41:071
226952199215311,2138cyclictest2332968-21kworker/1:0+events22:36:331
22695299921529,2140cyclictest2364017-21kworker/3:0+events00:04:443
22695299921519,2138cyclictest2357725-21kworker/3:2+events23:59:023
22695299921514,2143cyclictest2276013-21kworker/3:2+events19:32:503
226952199215111,2137cyclictest2279191-21kworker/1:0+events19:43:281
226952199215111,2136cyclictest2272672-21kworker/1:2+events19:26:211
226952199215110,2138cyclictest2363987-21kworker/1:1+events00:07:471
226952999215011,2136cyclictest2298168-21kworker/3:0+events21:02:033
226952499215010,2137cyclictest2282210-21kworker/2:0+events19:51:572
22695219921477,2137cyclictest2321882-21kworker/1:1+events21:55:501
22695219921449,2131cyclictest2302732-21kworker/1:0+events21:01:171
226952199213911,2125cyclictest2370511-21kworker/1:1+events00:32:311
226952199213911,2125cyclictest2288471-21kworker/1:2+events20:22:281
226952199213811,2124cyclictest2350958-21kworker/1:2+events23:22:091
226952199213811,2123cyclictest2272672-21kworker/1:2+events19:51:101
226952199213711,2123cyclictest2274324-21kworker/1:1+events19:24:251
226952199213513,2118cyclictest2314142-21kworker/1:1+events21:39:081
226952199213310,2119cyclictest2360745-21kworker/1:2+events23:57:361
226951799213310,2120cyclictest2320608-21kworker/0:2+events21:51:160
226952199213011,2115cyclictest2341766-21kworker/1:0+events23:05:171
226952199212811,2113cyclictest2343570-21kworker/1:2+events23:14:251
226952499211811,2103cyclictest2363956-21kworker/2:1+events00:06:162
226952499211611,2101cyclictest2306311-21kworker/2:1+events21:17:432
226952499211510,2102cyclictest2368828-21kworker/2:1+events00:20:472
226952499211310,2100cyclictest2315580-21kworker/2:0+events21:37:582
226952999211211,2098cyclictest2298168-21kworker/3:0+events20:50:363
22695219921127,2102cyclictest2332968-21kworker/1:0+events22:27:061
22695219921127,2102cyclictest2332968-21kworker/1:0+events22:27:061
226952499210411,2090cyclictest2331377-21kworker/2:2+events22:30:302
226952499210311,2090cyclictest2302936-21kworker/2:2+events20:58:372
226952499210211,2088cyclictest2363956-21kworker/2:1+events00:13:062
226952499210211,2088cyclictest2318908-21kworker/2:2+events21:47:052
22695299921019,2089cyclictest2314112-21kworker/3:0+events21:37:133
22695219921019,2089cyclictest2363987-21kworker/1:1+events00:03:571
22695219920949,2082cyclictest2282393-21kworker/1:1+events20:04:311
226951799208610,2073cyclictest2370410-21kworker/0:2+events00:29:090
226951799208610,2072cyclictest2326768-21kworker/0:1+events22:11:060
22695179920858,2074cyclictest2285386-21kworker/0:1+events20:02:350
22695179920837,2073cyclictest2359177-21kworker/0:0+events00:04:190
226951799208210,2069cyclictest2360776-21kworker/0:1+events23:56:530
22695179920814,2073cyclictest2336991-21kworker/0:1+events22:44:120
226951799208010,2067cyclictest2344695-21kworker/0:1+events23:22:330
226951799208010,2067cyclictest2329950-21kworker/0:0+events22:20:130
22695179920798,2068cyclictest2297900-21kworker/0:0+events20:37:150
22695179920798,2068cyclictest2297900-21kworker/0:0+events20:37:150
226951799207810,2064cyclictest2326768-21kworker/0:1+events22:09:090
22695179920779,2065cyclictest2339831-21kworker/0:0+events23:06:040
22695179920769,2063cyclictest2367009-21kworker/0:0+events00:12:200
226951799207610,2063cyclictest2279161-21kworker/0:0+events20:12:110
22695299920741,2070cyclictest2357725-21kworker/3:2+events23:53:483
226952999207411,2060cyclictest2365652-21kworker/3:1+events00:17:433
22695249920728,2061cyclictest2372054-21kworker/2:2+events00:37:072
226951799207210,2059cyclictest2315567-21kworker/0:0+events21:34:100
22695219920688,2057cyclictest2341766-21kworker/1:0+events23:03:221
226952999206510,2052cyclictest2352714-21kworker/3:1+events23:37:063
22695299920637,2053cyclictest2309215-21kworker/3:1+events21:26:073
22695249920635,2055cyclictest2315580-21kworker/2:0+events21:33:242
226952999206210,2049cyclictest2329854-21kworker/3:2+events22:21:443
22695219920585,2049cyclictest2307910-21kworker/1:2+events21:10:261
226952199205811,2044cyclictest2338206-21kworker/1:0+events22:46:301
226952199205510,2042cyclictest2288471-21kworker/1:2+events20:30:281
226952999205110,2038cyclictest2320281-21kworker/3:1+events21:49:233
226952199205110,2038cyclictest2362345-21kworker/1:0+events00:38:381
22695179920487,2039cyclictest2318918-21kworker/0:1+events21:57:220
22695299920468,2036cyclictest2364017-21kworker/3:0+events00:10:073
226952999204310,2030cyclictest2325254-21kworker/3:0+events22:25:113
226952999204310,2030cyclictest2325254-21kworker/3:0+events22:25:113
22695219920439,2031cyclictest2312481-21kworker/1:2+events21:26:291
22695249920415,2033cyclictest2298057-21kworker/2:1+events20:40:212
226952199204010,2027cyclictest2317386-21kworker/1:2+events21:41:441
22695299920369,2024cyclictest2276013-21kworker/3:2+events19:59:103
226952199203310,2020cyclictest2319108-21kworker/1:0+events22:04:121
22695299920328,2021cyclictest2274382-21kworker/3:1+events19:24:013
226952499203210,2019cyclictest2306311-21kworker/2:1+events21:25:432
22695219920319,2019cyclictest2314142-21kworker/1:1+events21:33:461
22695299920307,2019cyclictest2300058-21kworker/3:1+events20:46:233
226952199202810,2015cyclictest2367256-21kworker/1:1+events00:18:521
22695299920247,2013cyclictest2349418-21kworker/3:2+events23:33:173
226952999202410,2011cyclictest2276013-21kworker/3:2+events19:50:243
22695299920224,2015cyclictest2282121-21kworker/3:1+events19:46:583
22695299920223,2016cyclictest2344578-21kworker/3:1+events23:12:103
22695299920218,2010cyclictest2272734-21kworker/3:0+events19:26:443
226952999202110,2008cyclictest2272734-21kworker/3:0+events19:19:083
22695219920218,2009cyclictest2288471-21kworker/1:2+events20:10:411
22695299920208,2009cyclictest2370783-21kworker/3:1+events00:29:543
22695179920209,2007cyclictest2318918-21kworker/0:1+events21:49:460
22695179920198,2008cyclictest2310791-21kworker/0:1+events21:17:220
226952999201810,2005cyclictest2368904-21kworker/3:2+events00:35:593
226952999201810,2005cyclictest2321942-21kworker/3:0+events21:52:273
226952999201810,2005cyclictest2294797-21kworker/3:0+events20:28:333
22695299920172,2012cyclictest2342951-21kworker/3:0+events22:59:313
22695299920168,2005cyclictest2323603-21kworker/3:1+events22:01:293
22695219920146,2005cyclictest2319108-21kworker/1:0+events21:45:111
226952999201210,1999cyclictest2309215-21kworker/3:1+events21:21:333
22695299920117,2001cyclictest2285478-21kworker/3:1+events20:01:033
22695179920119,1999cyclictest2279161-21kworker/0:0+events19:52:410
22695179920088,1997cyclictest2312570-21kworker/0:2+events21:27:150
22695249920069,1994cyclictest2322122-21kworker/2:2+events21:56:142
22695219920069,1994cyclictest2336289-21kworker/1:2+events22:42:391
226952199200610,1993cyclictest2297868-21kworker/1:1+events20:52:101
22695219920059,1993cyclictest2340130-21kworker/1:1+events22:50:431
22695179920059,1993cyclictest2268952-21kworker/0:1+events19:14:160
226952999200410,1990cyclictest2364017-21kworker/3:0+events00:08:583
22695219920009,1988cyclictest2284234-21kworker/1:0+events20:06:501
22695219919999,1987cyclictest2270631-21kworker/1:0+events19:17:131
226952199199910,1986cyclictest2352594-21kworker/1:0+events23:28:401
22695219919985,1990cyclictest2370511-21kworker/1:1+events00:29:311
22695219919977,1988cyclictest2362345-21kworker/1:0+events00:23:291
226952199199710,1984cyclictest2288471-21kworker/1:2+events20:27:471
226952199199610,1983cyclictest2297868-21kworker/1:1+events20:55:581
226952199199510,1982cyclictest2340130-21kworker/1:1+events22:55:201
226952199199510,1982cyclictest2312481-21kworker/1:2+events21:22:391
226952499199411,1980cyclictest2283834-21kworker/2:1+events20:12:572
22695219919889,1976cyclictest2359149-21kworker/1:0+events23:48:061
226952199198710,1974cyclictest2326800-21kworker/1:2+events22:19:481
22695249919869,1974cyclictest2311659-21kworker/2:0+events21:21:112
22695219919866,1976cyclictest2359149-21kworker/1:0+events23:51:091
22695219919826,1974cyclictest2317386-21kworker/1:2+events21:50:101
226952499197310,1960cyclictest2294807-21kworker/2:0+events20:32:002
22695249919719,1959cyclictest2288972-21kworker/2:0+events20:15:582
22695299919699,1957cyclictest2349418-21kworker/3:2+events23:23:413
226952499196910,1957cyclictest2278885-21kworker/2:1+events19:42:192
22695249919639,1950cyclictest2365598-21kworker/2:2+events00:19:162
22695249919627,1952cyclictest2331377-21kworker/2:2+events22:29:012
22695249919627,1952cyclictest2331377-21kworker/2:2+events22:29:002
22695249919576,1948cyclictest2329028-21kworker/2:0+events22:17:102
22695249919527,1942cyclictest2341560-21kworker/2:2+events22:58:462
22695249919527,1942cyclictest2294807-21kworker/2:0+events20:29:192
22695249919509,1938cyclictest2343163-21kworker/2:1+events23:00:412
22695249919507,1940cyclictest2265827-21kworker/2:2+events19:13:522
226952499195010,1937cyclictest2265827-21kworker/2:2+events19:19:302
226952999194810,1935cyclictest2333343-21kworker/3:2+events22:51:083
226952499194810,1935cyclictest2336155-21kworker/2:2+events22:46:542
22695219919487,1938cyclictest2270631-21kworker/1:0+events19:13:061
22695249919479,1935cyclictest2372054-21kworker/2:2+events00:32:102
22695249919476,1938cyclictest2344778-21kworker/2:0+events23:10:392
226952499194610,1933cyclictest2320512-21kworker/2:0+events21:52:022
22695299919438,1931cyclictest2333343-21kworker/3:2+events22:44:353
22695249919438,1932cyclictest2347974-21kworker/2:2+events23:20:352
226952499194310,1930cyclictest2344778-21kworker/2:0+events23:17:072
22695249919419,1928cyclictest2351469-21kworker/2:0+events23:35:572
226952499194110,1928cyclictest2318908-21kworker/2:2+events21:44:492
22695299919289,1916cyclictest2270691-21kworker/3:1+events19:13:293
22695249919236,1914cyclictest2272849-21kworker/2:1+events19:22:102
22695249919159,1903cyclictest2331377-21kworker/2:2+events22:20:572
226952499189410,1881cyclictest2274498-21kworker/2:0+events19:28:142
22695249918839,1871cyclictest2302936-21kworker/2:2+events21:08:322
226952499188310,1871cyclictest2301317-21kworker/2:0+events21:01:402
22695249918809,1868cyclictest2347974-21kworker/2:2+events23:26:432
22695249918809,1868cyclictest2323724-21kworker/2:0+events22:04:342
226952499187910,1868cyclictest2368828-21kworker/2:1+events00:25:482
22695249918699,1858cyclictest2359130-21kworker/2:0+events00:00:122
22695249918139,1802cyclictest2355887-21kworker/2:2+events23:48:502
22690952149104,26sleep30-21swapper/319:08:003
22689082142110,27sleep20-21swapper/219:05:402
22691022140110,25sleep00-21swapper/019:08:050
22690702139111,22sleep10-21swapper/119:07:421
226952499198,8cyclictest2354406-21idleruntime-cro23:35:002
226952499191,2cyclictest2292153-21taskset20:20:052
226952999168,5cyclictest89133-21systemd-journal19:43:403
226952499168,6cyclictest2298243-21systemctl20:40:012
226952499168,6cyclictest2298243-21systemctl20:40:002
226952499167,6cyclictest2357848-21awk23:45:002
226952499165,9cyclictest2285580-21sendmail19:59:592
2269524991610,4cyclictest43-21ksoftirqd/220:09:582
226952499160,4cyclictest2301939-21/usr/sbin/munin20:50:162
226952499157,5cyclictest2282428-21awk19:49:592
226952499157,5cyclictest2278136-21df_inode19:35:152
226952499155,7cyclictest2326995-21basename22:10:002
2269524991511,3cyclictest2362406-21taskset23:58:122
226952499151,13cyclictest2359876-21df23:50:172
2269524991511,3cyclictest2328402-21runrttasks22:13:112
226952499151,13cyclictest2300078-21/usr/sbin/munin20:45:192
2269524991412,1cyclictest0-21swapper/219:32:212
226952499140,13cyclictest2285834-21/usr/sbin/munin20:00:162
2269524991311,1cyclictest2308238-21/usr/sbin/munin21:10:222
226952499130,4cyclictest2346146-21taskset23:05:492
226952499125,4cyclictest660-21rs:main2
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional