You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-01 - 11:56

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #f, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails
  
Click here to display the system's profile data.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackfslot7s.osadl.org (updated Mon Dec 01, 2025 00:44:17)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
410602799275013,2733cyclictest10810-21kworker/1:0+events00:25:161
410603599273913,2722cyclictest4152381-21kworker/3:0+events22:05:193
410602599272413,2706cyclictest10931-21kworker/0:0+events00:35:150
410602799269311,2678cyclictest4136619-21kworker/1:1+events21:10:191
410602599257512,2559cyclictest4117270-21kworker/0:1+events20:35:160
410602599257512,2559cyclictest4117270-21kworker/0:1+events20:35:160
410602799248812,2474cyclictest4152627-21kworker/1:2+events21:45:371
410602799248612,2471cyclictest4115594-21kworker/1:2+events19:39:201
410602799248512,2470cyclictest4117210-21kworker/1:1+events20:30:151
410602599242712,2412cyclictest4136295-21kworker/0:2+events20:41:470
410602799241211,2398cyclictest4133304-21kworker/1:2+events20:55:181
410603599240311,2388cyclictest4157295-21kworker/3:1+events21:58:543
410603599240112,2386cyclictest4117036-21kworker/3:2+events19:41:333
410603599239912,2384cyclictest4170222-21kworker/3:1+events22:42:273
410603599239912,2384cyclictest4170222-21kworker/3:1+events22:42:273
410603099238312,2367cyclictest4163625-21kworker/2:1+events22:21:162
410602799237011,2355cyclictest4158835-21kworker/1:0+events22:11:261
410603599233411,2320cyclictest4193778-21kworker/3:1+events23:43:273
41060359923324,2325cyclictest4133545-21kworker/3:2+events20:44:033
410602799230511,2290cyclictest4152627-21kworker/1:2+events21:40:181
41060359922889,2275cyclictest4110943-21kworker/3:1+events19:32:083
410602799228612,2271cyclictest4178258-21kworker/1:0+events23:05:221
410603099228512,2269cyclictest4176685-21kworker/2:1+events22:54:202
410602799227712,2263cyclictest2032-21kworker/1:2+events00:00:081
410602799227411,2260cyclictest4173341-21kworker/1:2+events23:11:071
410602799227311,2259cyclictest4194016-21kworker/1:1+events23:50:111
410602799227210,2258cyclictest4151016-21kworker/1:0+events21:29:431
410602799226911,2255cyclictest4194016-21kworker/1:1+events23:44:571
410602799226911,2255cyclictest4133304-21kworker/1:2+events21:04:061
410602799226811,2254cyclictest9138-21kworker/1:2+events00:17:171
410602799226711,2252cyclictest4117210-21kworker/1:1+events20:22:121
410602799226710,2254cyclictest4113846-21kworker/1:0+events19:43:031
410602799226610,2253cyclictest4147727-21kworker/1:0+events21:21:391
410602799226511,2251cyclictest4158835-21kworker/1:0+events21:53:581
410602599226512,2250cyclictest4117270-21kworker/0:1+events20:13:510
410602799226411,2250cyclictest4136619-21kworker/1:1+events20:48:101
410602799226411,2250cyclictest10810-21kworker/1:0+events00:23:441
410602799226410,2251cyclictest4173341-21kworker/1:2+events22:57:481
410602799226310,2250cyclictest4134912-21kworker/1:0+events20:43:171
410602799226111,2247cyclictest4136619-21kworker/1:1+events20:50:241
410602799226110,2247cyclictest4151016-21kworker/1:0+events21:32:201
410603599225511,2240cyclictest4166911-21kworker/3:2+events22:20:323
410602799225511,2240cyclictest4168685-21kworker/1:2+events22:25:501
410602799225010,2237cyclictest4136619-21kworker/1:1+events21:06:491
410603099224611,2232cyclictest4189095-21kworker/2:1+events23:30:532
410602799224411,2229cyclictest4121893-21kworker/1:0+events20:01:161
410602599223710,2224cyclictest15345-21kworker/0:1+events00:34:090
410603099223611,2221cyclictest4184533-21kworker/2:2+events23:20:172
41060259922358,2224cyclictest4155601-21kworker/0:2+events21:42:340
410602799223211,2218cyclictest4192240-21kworker/1:2+events23:38:491
410602599223210,2220cyclictest4146010-21kworker/0:0+events21:20:100
410602599223110,2218cyclictest4117270-21kworker/0:1+events20:31:480
410602599222711,2212cyclictest4102275-21kworker/0:0+events19:30:150
410602599222710,2214cyclictest4102275-21kworker/0:0+events19:43:490
410603599222511,2210cyclictest4133545-21kworker/3:2+events20:36:493
410603599222511,2210cyclictest4133545-21kworker/3:2+events20:36:493
410603599222010,2207cyclictest4193778-21kworker/3:1+events23:54:233
41060259922209,2208cyclictest4102275-21kworker/0:0+events19:37:270
41060359922187,2208cyclictest4128615-21kworker/3:1+events20:31:243
410602599221810,2205cyclictest4176653-21kworker/0:0+events22:52:240
41060359922169,2204cyclictest4118625-21kworker/3:1+events20:12:413
410602799221611,2201cyclictest4173341-21kworker/1:2+events23:22:321
410602599221310,2201cyclictest4184504-21kworker/0:1+events23:15:160
410603599221211,2198cyclictest4136324-21kworker/3:0+events20:45:533
410603599221010,2197cyclictest4157295-21kworker/3:1+events21:54:203
410603599221010,2196cyclictest4152381-21kworker/3:0+events21:42:123
410603599220912,2194cyclictest4193778-21kworker/3:1+events00:02:253
410603599220810,2195cyclictest4152381-21kworker/3:0+events22:19:453
410603599220610,2193cyclictest4110943-21kworker/3:1+events19:38:583
41060259922048,2193cyclictest4147604-21kworker/0:2+events21:26:390
410602599220410,2191cyclictest3189-21kworker/0:2+events23:56:400
410603599220311,2189cyclictest4144703-21kworker/3:1+events21:25:273
410602599220311,2189cyclictest3189-21kworker/0:2+events00:16:310
410602599220310,2190cyclictest13706-21kworker/0:2+events00:26:500
410603599220210,2189cyclictest4141177-21kworker/3:2+events20:58:453
410603599220111,2187cyclictest4170222-21kworker/3:1+events22:38:193
410603599219911,2185cyclictest4848-21kworker/3:0+events23:59:233
41060279921988,2187cyclictest4173341-21kworker/1:2+events22:51:151
410602599219710,2184cyclictest10931-21kworker/0:0+events00:21:510
410602599219511,2181cyclictest4117270-21kworker/0:1+events19:59:450
410603599219211,2178cyclictest4118625-21kworker/3:1+events20:03:333
410602799218211,2168cyclictest4158835-21kworker/1:0+events22:03:471
41060279921769,2164cyclictest4131694-21kworker/1:0+events20:29:061
41060359921758,2164cyclictest4152381-21kworker/3:0+events21:48:173
410602599217110,2158cyclictest4189217-21kworker/0:1+events23:36:120
410602799216810,2154cyclictest4162083-21kworker/1:1+events22:08:431
410603099216611,2152cyclictest1549-21kworker/2:2+events23:46:502
410603099216511,2151cyclictest4116600-21kworker/2:2+events19:41:112
410603099216511,2151cyclictest16827-21kworker/2:1+events00:37:122
41060359921649,2152cyclictest4142833-21kworker/3:0+events21:11:493
410603099216410,2151cyclictest4171704-21kworker/2:1+events22:41:422
410603099216410,2151cyclictest4171704-21kworker/2:1+events22:41:422
41060359921637,2153cyclictest4125377-21kworker/3:2+events20:19:093
410603599216311,2149cyclictest4110943-21kworker/3:1+events19:27:383
410603599216211,2148cyclictest4166911-21kworker/3:2+events22:28:283
410603099216211,2148cyclictest4174926-21kworker/2:0+events23:12:122
410602799216210,2149cyclictest4133304-21kworker/1:2+events20:38:441
410602799216210,2149cyclictest4133304-21kworker/1:2+events20:38:431
41060309921607,2150cyclictest4181220-21kworker/2:1+events23:06:282
41060309921599,2147cyclictest4174926-21kworker/2:0+events22:46:162
410603099215710,2144cyclictest4091317-21kworker/2:2+events19:22:182
410603099215411,2141cyclictest4189095-21kworker/2:1+events00:05:032
410603099215410,2141cyclictest4170104-21kworker/2:0+events22:28:042
410603599215311,2139cyclictest4144703-21kworker/3:1+events21:34:133
41060309921519,2139cyclictest4179530-21kworker/2:2+events23:00:302
410602599215110,2138cyclictest4163417-21kworker/0:0+events22:09:550
410603599214811,2134cyclictest4170222-21kworker/3:1+events22:34:523
41060259921447,2134cyclictest4128525-21kworker/0:0+events20:23:000
410603099213710,2123cyclictest4128759-21kworker/2:0+events20:22:362
410602799213610,2124cyclictest4727-21kworker/1:0+events00:13:061
41060309921307,2119cyclictest7116-21kworker/2:0+events00:20:192
410603599212910,2116cyclictest4173776-21kworker/3:2+events22:58:113
410603099212910,2116cyclictest4158804-21kworker/2:2+events22:03:022
41060259921298,2118cyclictest4102275-21kworker/0:0+events19:53:400
410603599212811,2114cyclictest4144703-21kworker/3:1+events21:16:433
410603599212610,2113cyclictest4173776-21kworker/3:2+events23:01:143
410603599212511,2111cyclictest4182966-21kworker/3:2+events23:12:333
410603599212510,2111cyclictest4118625-21kworker/3:1+events19:50:583
410603099212511,2111cyclictest4153919-21kworker/2:1+events21:38:492
410602799211810,2105cyclictest4171615-21kworker/1:1+events22:35:151
41060259921186,2108cyclictest4170342-21kworker/0:2+events22:31:500
410602799211710,2104cyclictest4173341-21kworker/1:2+events22:46:401
41060359921139,2101cyclictest4184564-21kworker/3:0+events23:25:123
41060309921089,2096cyclictest5176-21kworker/2:3+events00:10:022
41060279921088,2097cyclictest4194016-21kworker/1:1+events23:47:591
41060279921079,2095cyclictest4189338-21kworker/1:0+events23:33:121
41060279921077,2097cyclictest4121893-21kworker/1:0+events19:59:241
410603099210611,2092cyclictest4184533-21kworker/2:2+events23:28:572
41060279921059,2092cyclictest4118868-21kworker/1:2+events19:50:361
41060279921049,2092cyclictest14486-21kworker/1:2+events00:34:531
410603099210311,2089cyclictest4134822-21kworker/2:1+events20:40:392
410602799210310,2091cyclictest4167066-21kworker/1:1+events22:20:081
41060279921028,2091cyclictest4149352-21kworker/1:1+events21:38:021
410602799210210,2089cyclictest4123690-21kworker/1:2+events20:06:361
410603099210111,2088cyclictest7116-21kworker/2:0+events00:16:542
41060279921016,2093cyclictest4173341-21kworker/1:2+events23:00:061
41060279921008,2089cyclictest4113846-21kworker/1:0+events19:47:371
410603599209911,2085cyclictest4162439-21kworker/3:1+events22:11:023
410602599209911,2085cyclictest4102275-21kworker/0:0+events19:49:270
410602799209811,2084cyclictest10810-21kworker/1:0+events00:36:491
410602599209811,2084cyclictest4168657-21kworker/0:1+events22:49:440
410602799209710,2084cyclictest4173341-21kworker/1:2+events22:44:001
410602799209710,2084cyclictest4173341-21kworker/1:2+events22:43:591
410602799209710,2084cyclictest4125639-21kworker/1:0+events20:14:341
410602799209610,2083cyclictest4158835-21kworker/1:0+events21:55:521
410602599209611,2082cyclictest4152530-21kworker/0:1+events21:59:410
410602799209410,2081cyclictest6562-21kworker/1:1+events00:06:351
41060279920939,2081cyclictest4107333-21kworker/1:1+events19:15:511
41060359920888,2077cyclictest4125377-21kworker/3:2+events20:21:033
41060279920885,2080cyclictest4097899-21kworker/1:0+events19:22:431
410602799208610,2073cyclictest4158835-21kworker/1:0+events22:18:391
410602599208310,2070cyclictest4176653-21kworker/0:0+events23:48:440
41060259920819,2069cyclictest4136295-21kworker/0:2+events20:46:390
41060259920808,2068cyclictest4179029-21kworker/0:2+events22:59:440
410602599208010,2066cyclictest4182750-21kworker/0:2+events23:09:100
41060259920798,2067cyclictest6443-21kworker/0:1+events00:04:400
410602599207910,2066cyclictest3189-21kworker/0:2+events00:11:100
410603099207811,2064cyclictest4141267-21kworker/2:1+events21:01:472
41060259920786,2069cyclictest4163417-21kworker/0:0+events22:17:060
41060259920776,2068cyclictest4187499-21kworker/0:2+events23:26:180
41060259920756,2066cyclictest4137920-21kworker/0:0+events20:51:330
41060259920747,2064cyclictest4176653-21kworker/0:0+events23:40:210
41060259920744,2067cyclictest4176653-21kworker/0:0+events23:01:360
41060259920739,2061cyclictest4152530-21kworker/0:1+events21:52:060
41060259920739,2061cyclictest4147604-21kworker/0:2+events21:31:370
41060259920738,2062cyclictest4139502-21kworker/0:1+events21:10:420
41060359920728,2058cyclictest9046-21kworker/3:0+events00:14:143
410602599207210,2059cyclictest4173246-21kworker/0:0+events22:40:570
410602599207210,2059cyclictest4173246-21kworker/0:0+events22:40:570
410602599207110,2058cyclictest3189-21kworker/0:2+events00:06:590
410603599207010,2057cyclictest4144703-21kworker/3:1+events21:22:023
41060259920704,2063cyclictest4147604-21kworker/0:2+events21:19:230
41060359920699,2057cyclictest4139576-21kworker/3:0+events20:53:233
41060359920698,2057cyclictest4173776-21kworker/3:2+events22:51:373
41060359920695,2061cyclictest4181188-21kworker/3:0+events23:05:453
410603599206910,2056cyclictest4118625-21kworker/3:1+events19:56:463
410602599206910,2056cyclictest4117270-21kworker/0:1+events20:17:590
410602599206910,2056cyclictest4112332-21kworker/0:2+events19:28:440
41060359920689,2055cyclictest9046-21kworker/3:0+events00:23:213
41060359920684,2061cyclictest4182966-21kworker/3:2+events23:24:263
410603599206810,2055cyclictest4190539-21kworker/3:2+events23:48:223
41060259920689,2056cyclictest4168657-21kworker/0:1+events22:36:480
41060259920689,2056cyclictest4152530-21kworker/0:1+events21:47:310
41060279920679,2055cyclictest4117210-21kworker/1:1+events20:16:511
41060259920673,2060cyclictest4117270-21kworker/0:1+events20:09:150
410602599206710,2054cyclictest4160555-21kworker/0:2+events22:10:390
410602599206610,2053cyclictest4122073-21kworker/0:2+events20:00:320
410603599206510,2052cyclictest4184564-21kworker/3:0+events23:19:063
410603599206510,2052cyclictest4176352-21kworker/3:0+events22:47:043
41060259920658,2054cyclictest4163417-21kworker/0:0+events22:27:210
410602599206510,2051cyclictest4160555-21kworker/0:2+events22:00:450
41060359920648,2053cyclictest4190539-21kworker/3:2+events23:38:273
410603599206410,2051cyclictest12401-21kworker/3:2+events00:26:043
410602599206410,2051cyclictest4187499-21kworker/0:2+events23:23:170
410603599206210,2049cyclictest10713-21kworker/3:1+events00:36:253
41060359920614,2054cyclictest6502-21kworker/3:2+events00:09:403
410603599206110,2049cyclictest4152381-21kworker/3:0+events22:00:023
410603599205810,2045cyclictest4128615-21kworker/3:1+events20:27:333
41060259920583,2051cyclictest4176653-21kworker/0:0+events23:50:580
410603599205610,2043cyclictest4152381-21kworker/3:0+events21:39:123
41060359920559,2043cyclictest4137978-21kworker/3:1+events21:02:573
410603099205510,2042cyclictest4147821-21kworker/2:2+events21:19:472
41060259920557,2045cyclictest4176653-21kworker/0:0+events23:14:060
410603099205210,2039cyclictest4136560-21kworker/2:0+events21:07:582
41060259920527,2042cyclictest4166938-21kworker/0:2+events22:22:020
410603599204910,2036cyclictest4142833-21kworker/3:0+events21:05:393
410603599204710,2033cyclictest4188005-21kworker/3:1+events23:34:203
410603599204610,2033cyclictest15285-21kworker/3:0+events00:33:243
41060359920459,2033cyclictest4110943-21kworker/3:1+events19:25:003
410602599204310,2030cyclictest4150835-21kworker/0:0+events21:37:380
410603099204210,2029cyclictest4112602-21kworker/2:0+events19:33:162
41060359920377,2027cyclictest4114199-21kworker/3:0+events19:46:023
41060279920379,2025cyclictest4111088-21kworker/1:1+events19:30:391
41060279920299,2016cyclictest4158835-21kworker/1:0+events22:34:051
41060279920289,2016cyclictest4184269-21kworker/1:1+events23:15:381
41060359920275,2019cyclictest9046-21kworker/3:0+events00:16:083
41060259920245,2016cyclictest4142744-21kworker/0:2+events21:05:160
41060279920209,2008cyclictest4111088-21kworker/1:1+events19:26:071
41060359920169,2004cyclictest4107301-21kworker/3:0+events19:17:243
410603099201510,2002cyclictest10560-21kworker/2:1+events00:29:072
410603599201110,1998cyclictest4125377-21kworker/3:2+events20:08:513
41060309920119,1999cyclictest4120319-21kworker/2:2+events20:17:162
41060309920118,2000cyclictest4153919-21kworker/2:1+events21:57:232
410603099201110,1998cyclictest4091317-21kworker/2:2+events19:19:402
410603099201010,1997cyclictest4130538-21kworker/2:1+events20:26:242
41060309920099,1997cyclictest4171704-21kworker/2:1+events22:37:352
41060309920093,2003cyclictest4133105-21kworker/2:2+events20:49:162
410603099200910,1995cyclictest4189095-21kworker/2:1+events23:53:382
41060259920099,1997cyclictest4109381-21kworker/0:1+events19:20:270
410603099200810,1995cyclictest1549-21kworker/2:2+events23:57:512
41060259920089,1996cyclictest4102275-21kworker/0:0+events19:13:560
410603099200710,1994cyclictest4112602-21kworker/2:0+events19:47:122
41060309920069,1994cyclictest4158804-21kworker/2:2+events22:08:222
41060309920056,1996cyclictest4136560-21kworker/2:0+events21:14:482
41060259920057,1995cyclictest4102275-21kworker/0:0+events19:16:150
410603099200410,1991cyclictest4153919-21kworker/2:1+events21:43:422
41060259920047,1994cyclictest4139502-21kworker/0:1+events21:02:340
41060309920038,1992cyclictest5176-21kworker/2:3+events00:02:022
410603099200110,1989cyclictest4174926-21kworker/2:0+events22:58:582
41060259920019,1988cyclictest4189217-21kworker/0:1+events23:32:260
410603099200010,1986cyclictest4144869-21kworker/2:1+events21:27:012
410603099199910,1986cyclictest4128759-21kworker/2:0+events20:34:542
41060259919995,1991cyclictest4128525-21kworker/0:0+events20:26:470
41060309919988,1987cyclictest4118719-21kworker/2:1+events19:52:322
41060309919987,1988cyclictest4181220-21kworker/2:1+events23:19:302
410603099199810,1985cyclictest4133105-21kworker/2:2+events20:37:362
410603099199810,1985cyclictest4133105-21kworker/2:2+events20:37:362
41060359919979,1985cyclictest4107301-21kworker/3:0+events19:14:433
41060309919979,1985cyclictest4168473-21kworker/2:2+events22:30:412
41060309919978,1985cyclictest4163625-21kworker/2:1+events22:15:392
41060309919976,1988cyclictest4189095-21kworker/2:1+events23:40:452
41060309919975,1989cyclictest4123603-21kworker/2:1+events20:14:122
41060309919967,1985cyclictest4121954-21kworker/2:0+events20:03:082
410603099199410,1981cyclictest4144869-21kworker/2:1+events21:23:562
41060309919888,1977cyclictest4190823-21kworker/2:0+events23:36:342
41060279919838,1972cyclictest4727-21kworker/1:0+events23:59:001
41060309919828,1971cyclictest12340-21kworker/2:2+events00:34:302
41060309919736,1964cyclictest4123603-21kworker/2:1+events20:06:112
41060279919697,1959cyclictest4097899-21kworker/1:0+events19:12:221
410603099196510,1952cyclictest4110822-21kworker/2:1+events19:29:542
41060279919639,1951cyclictest4146126-21kworker/1:2+events21:17:301
410603099193710,1924cyclictest4158804-21kworker/2:2+events22:13:212
41060309919358,1924cyclictest4091317-21kworker/2:2+events19:14:202
41060309919352,1930cyclictest4121954-21kworker/2:0+events19:59:022
41060309919327,1922cyclictest4110822-21kworker/2:1+events19:37:042
41060309919288,1916cyclictest4155751-21kworker/2:0+events21:47:532
41060309919209,1908cyclictest4153919-21kworker/2:1+events21:52:282
41060309919143,1908cyclictest4136560-21kworker/2:0+events20:59:072
41057092157111,25sleep30-21swapper/319:08:483
41055162143114,23sleep10-21swapper/119:06:221
41057482142109,28sleep20-21swapper/219:09:202
41056102138106,27sleep00-21swapper/019:07:320
4106030992015,3cyclictest4139782-21idleruntime-cro20:55:002
4106027991610,5cyclictest4189175-21chrt23:27:521
410603099151,10cyclictest4152710-21munin-run21:35:012
410602599150,4cyclictest4140981-21/usr/sbin/munin20:55:220
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional