You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-27 - 11:59

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #f, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails
  
Click here to display the system's profile data.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackfslot7s.osadl.org (updated Thu Nov 27, 2025 00:44:16)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
222519499273713,2720cyclictest2249943-21kworker/0:0+events20:40:130
222520099269013,2674cyclictest2314224-21kworker/2:1+events23:55:112
222520099266411,2649cyclictest2284785-21kworker/2:1+events22:20:192
222520099264113,2624cyclictest2275152-21kworker/2:2+events21:50:192
222519499259712,2581cyclictest2229942-21kworker/0:0+events19:35:130
222519499249312,2477cyclictest2249943-21kworker/0:0+events20:27:450
222519499247612,2460cyclictest2323448-21kworker/0:2+events00:37:510
222519499247511,2461cyclictest2238076-21kworker/0:1+events19:54:190
222519499244712,2431cyclictest2278578-21kworker/0:1+events22:05:160
222520499242311,2408cyclictest2228411-21kworker/3:2+events19:21:483
222519499240911,2395cyclictest2276970-21kworker/0:0+events22:00:190
22251949924058,2395cyclictest2236441-21kworker/0:0+events19:45:110
222520099239812,2383cyclictest2305817-21kworker/2:1+events23:43:262
222519499239412,2381cyclictest2238076-21kworker/0:1+events20:20:110
222520099234412,2330cyclictest2288312-21kworker/2:1+events22:40:142
222519499234010,2326cyclictest2264365-21kworker/0:0+events21:16:250
222519499232912,2313cyclictest2255490-21kworker/0:0+events20:50:120
222520499232612,2311cyclictest2302350-21kworker/3:2+events23:17:233
222520099231312,2298cyclictest2310718-21kworker/2:2+events23:50:212
222520499231212,2296cyclictest2264037-21kworker/3:0+events21:25:153
222520099231212,2297cyclictest2243278-21kworker/2:1+events20:10:212
222519499231110,2299cyclictest2262528-21kworker/0:1+events21:20:140
222519899231011,2296cyclictest2220085-21kworker/1:1+events20:30:491
222519499229111,2276cyclictest2293093-21kworker/0:2+events22:45:140
222519499228512,2271cyclictest2273455-21kworker/0:2+events21:45:210
222519499227910,2265cyclictest2226282-21kworker/0:1+events19:17:400
222520099227510,2262cyclictest2310718-21kworker/2:2+events23:39:592
222519499227311,2258cyclictest2238076-21kworker/0:1+events20:01:580
222519499227211,2258cyclictest2268571-21kworker/0:1+events21:51:050
222519499227211,2258cyclictest2226282-21kworker/0:1+events19:24:500
222519499227211,2257cyclictest2294762-21kworker/0:1+events22:52:530
222519499227110,2258cyclictest2291462-21kworker/0:0+events22:44:300
222519499227010,2257cyclictest2257091-21kworker/0:1+events21:02:460
222519499227010,2257cyclictest2257091-21kworker/0:1+events21:02:460
222519499226811,2254cyclictest2278578-21kworker/0:1+events21:59:560
222519499226611,2251cyclictest2291462-21kworker/0:0+events22:59:230
222519499226511,2251cyclictest2323448-21kworker/0:2+events00:17:210
222519499226510,2251cyclictest2280264-21kworker/0:2+events22:19:330
22251949922627,2252cyclictest2243101-21kworker/0:2+events20:12:370
222519499226011,2246cyclictest2297541-21kworker/0:2+events23:38:040
222519499226011,2246cyclictest2229942-21kworker/0:0+events19:32:580
222519499225511,2241cyclictest2321847-21kworker/0:0+events00:22:120
222519499224911,2235cyclictest2251539-21kworker/0:2+events20:39:520
222519499224910,2235cyclictest2304277-21kworker/0:1+events23:18:550
222519499224811,2234cyclictest2315290-21kworker/0:1+events00:08:560
222519499224811,2233cyclictest2297541-21kworker/0:2+events23:13:570
22251949922478,2236cyclictest2238076-21kworker/0:1+events19:55:520
222519499223610,2223cyclictest2289768-21kworker/0:1+events22:36:480
22251949922349,2221cyclictest2283020-21kworker/0:0+events22:29:550
222519499222511,2211cyclictest2264365-21kworker/0:0+events21:26:220
222519499222111,2207cyclictest2262528-21kworker/0:1+events21:12:390
222519499220610,2193cyclictest2289768-21kworker/0:1+events22:33:440
222520099219911,2185cyclictest2251814-21kworker/2:1+events20:42:542
22252049921737,2163cyclictest2273428-21kworker/3:1+events21:43:503
222520499215911,2145cyclictest2291311-21kworker/3:2+events22:38:203
222519499215411,2140cyclictest2234762-21kworker/0:2+events19:41:220
22251949921499,2137cyclictest2297541-21kworker/0:2+events23:20:280
222520499213811,2124cyclictest2315517-21kworker/3:0+events00:05:543
222520499213811,2124cyclictest2264037-21kworker/3:0+events21:14:563
222520499213611,2122cyclictest2320280-21kworker/3:1+events00:30:343
222519499213410,2121cyclictest2271776-21kworker/0:0+events21:41:570
222520499213011,2116cyclictest2293031-21kworker/3:1+events22:47:313
222520499213011,2116cyclictest2286467-21kworker/3:0+events22:24:113
222520499212711,2113cyclictest2325377-21kworker/3:2+events00:25:383
222520499212711,2113cyclictest2312316-21kworker/3:2+events23:44:583
22251949921277,2117cyclictest2280264-21kworker/0:2+events22:14:520
222520499212611,2112cyclictest2224722-21kworker/3:1+events19:29:073
222520499212411,2110cyclictest2273428-21kworker/3:1+events21:45:443
22251949921249,2112cyclictest2315290-21kworker/0:1+events23:54:280
22252009921196,2110cyclictest2294420-21kworker/2:0+events22:54:022
222520099211911,2105cyclictest2283079-21kworker/2:2+events22:13:202
222519499211910,2106cyclictest2305935-21kworker/0:0+events23:27:430
222520499211811,2103cyclictest2250275-21kworker/3:1+events20:36:063
222520099211711,2102cyclictest2321243-21kworker/2:2+events00:19:102
222520099211611,2102cyclictest2239640-21kworker/2:2+events19:58:322
222520099211610,2103cyclictest2230063-21kworker/2:0+events19:28:432
222519899211611,2102cyclictest2220085-21kworker/1:1+events20:14:541
22251949921158,2104cyclictest2286496-21kworker/0:1+events22:22:150
222519899211411,2101cyclictest2302288-21kworker/1:2+events23:18:311
222519899211411,2100cyclictest2275089-21kworker/1:2+events22:04:311
22251989921139,2101cyclictest2284689-21kworker/1:2+events22:19:101
222519499211310,2100cyclictest2309177-21kworker/0:0+events23:46:310
22251949921119,2098cyclictest2246546-21kworker/0:0+events20:18:430
222519499211110,2098cyclictest2291462-21kworker/0:0+events23:03:590
22251949921109,2098cyclictest2251539-21kworker/0:2+events20:45:150
22251949921105,2102cyclictest2268571-21kworker/0:1+events21:31:420
222519499211010,2097cyclictest2300805-21kworker/0:1+events23:07:500
22252009921089,2096cyclictest2271863-21kworker/2:1+events22:00:412
22251949921087,2098cyclictest2297541-21kworker/0:2+events23:34:370
222519499210810,2095cyclictest2251539-21kworker/0:2+events20:58:330
222519499210810,2095cyclictest2226282-21kworker/0:1+events19:12:470
222520499210710,2093cyclictest2305874-21kworker/3:1+events23:32:183
22251949921078,2096cyclictest2315290-21kworker/0:1+events00:02:250
222520099210611,2092cyclictest2288312-21kworker/2:1+events22:49:252
222520099210611,2092cyclictest2236147-21kworker/2:0+events19:42:332
222520499210311,2089cyclictest2241243-21kworker/3:2+events20:06:323
222520099210311,2089cyclictest2307509-21kworker/2:0+events23:33:282
222520099210311,2088cyclictest2302112-21kworker/2:0+events23:12:012
222520099210311,2088cyclictest2299348-21kworker/2:2+events23:04:232
22252049921029,2090cyclictest2239579-21kworker/3:1+events19:58:523
222520499210211,2088cyclictest2233266-21kworker/3:0+events19:43:403
222520499210211,2087cyclictest2244764-21kworker/3:1+events20:22:263
222520099210211,2090cyclictest2302112-21kworker/2:0+events23:20:052
222520099210211,2088cyclictest2230063-21kworker/2:0+events19:24:282
222519899210211,2088cyclictest2297659-21kworker/1:2+events23:02:281
222519499210210,2088cyclictest2326736-21kworker/0:1+events00:32:030
222519499210110,2088cyclictest2321847-21kworker/0:0+events00:26:250
222520499210011,2085cyclictest2305874-21kworker/3:1+events23:59:463
222520499209911,2085cyclictest2250275-21kworker/3:1+events20:44:293
222519899209911,2085cyclictest2253884-21kworker/1:2+events20:56:351
222519499209810,2085cyclictest2315290-21kworker/0:1+events23:57:280
222520099209710,2083cyclictest2239640-21kworker/2:2+events20:15:172
22251949920979,2085cyclictest2309177-21kworker/0:0+events23:43:500
222519899209411,2080cyclictest2304338-21kworker/1:1+events23:21:381
22252049920927,2082cyclictest2296266-21kworker/3:0+events22:55:333
222520099209010,2078cyclictest2255781-21kworker/2:0+events21:00:052
222520099209010,2078cyclictest2255781-21kworker/2:0+events21:00:042
22252049920896,2079cyclictest2249556-21kworker/3:2+events20:26:403
222519899208711,2074cyclictest2231608-21kworker/1:2+events19:31:481
222519499208010,2067cyclictest2249943-21kworker/0:0+events20:31:330
22252009920789,2067cyclictest2263948-21kworker/2:0+events21:35:062
22251949920759,2063cyclictest2243101-21kworker/0:2+events20:06:540
222520499207110,2058cyclictest2305874-21kworker/3:1+events23:50:003
222520499206911,2055cyclictest2256973-21kworker/3:0+events21:02:223
222520499206911,2055cyclictest2256973-21kworker/3:0+events21:02:223
22251949920686,2059cyclictest2226282-21kworker/0:1+events19:26:470
22252009920659,2053cyclictest2236147-21kworker/2:0+events19:50:272
22252049920548,2043cyclictest2288055-21kworker/3:2+events22:26:283
22251949920539,2041cyclictest2321847-21kworker/0:0+events00:11:540
222520499204811,2034cyclictest2273428-21kworker/3:1+events22:10:153
22251949920447,2034cyclictest2268571-21kworker/0:1+events21:37:010
222520499203811,2024cyclictest2302350-21kworker/3:2+events23:27:203
22251989920297,2019cyclictest2233405-21kworker/1:0+events19:43:181
22252009920259,2013cyclictest2316761-21kworker/2:0+events00:07:002
22252009920258,2014cyclictest2239640-21kworker/2:2+events20:20:562
222520499202010,2007cyclictest2233266-21kworker/3:0+events19:50:523
22252009920178,2005cyclictest2243278-21kworker/2:1+events20:09:122
22252049920146,2005cyclictest2322126-21kworker/3:2+events00:15:003
222520499201010,1997cyclictest2280051-21kworker/3:2+events22:07:113
22252049920097,1995cyclictest2244764-21kworker/3:1+events20:19:503
22251989920085,1999cyclictest2286855-21kworker/1:1+events22:45:371
222520499200611,1992cyclictest2267535-21kworker/3:1+events21:32:053
22252049920056,1995cyclictest2320280-21kworker/3:1+events00:16:103
22252009920041,2000cyclictest2226341-21kworker/2:1+events19:14:592
222520499200110,1988cyclictest2302350-21kworker/3:2+events23:12:483
22252049919949,1982cyclictest2315517-21kworker/3:0+events00:03:593
22252009919949,1984cyclictest2268784-21kworker/2:2+events21:40:032
222520499198810,1974cyclictest2288055-21kworker/3:2+events22:30:193
22252049919878,1976cyclictest2310688-21kworker/3:0+events23:39:343
22252009919879,1975cyclictest2286708-21kworker/2:0+events22:34:082
222520499198510,1972cyclictest2325377-21kworker/3:2+events00:36:403
222520499198510,1972cyclictest2231670-21kworker/3:2+events19:34:263
22252049919848,1973cyclictest2297873-21kworker/3:1+events23:01:413
22252049919848,1972cyclictest2250275-21kworker/3:1+events20:49:483
22251989919838,1972cyclictest2286855-21kworker/1:1+events22:40:381
222520499198210,1969cyclictest2256973-21kworker/3:0+events21:06:123
22252049919819,1969cyclictest2297873-21kworker/3:1+events23:09:193
22252049919816,1972cyclictest2283620-21kworker/3:2+events22:19:573
22252049919794,1972cyclictest2250275-21kworker/3:1+events20:34:353
22252009919794,1972cyclictest2267196-21kworker/2:1+events21:27:332
22252049919789,1966cyclictest2264037-21kworker/3:0+events21:37:463
22252049919759,1963cyclictest2256973-21kworker/3:0+events20:52:273
222520499197510,1962cyclictest2224722-21kworker/3:1+events19:18:013
22252009919755,1967cyclictest2263948-21kworker/2:0+events21:23:462
22252049919748,1963cyclictest2323720-21kworker/3:0+events00:24:523
22252049919739,1961cyclictest2265624-21kworker/3:2+events21:19:493
22252049919739,1961cyclictest2239579-21kworker/3:1+events20:01:123
22252049919736,1964cyclictest2305874-21kworker/3:1+events23:52:593
222520499197310,1960cyclictest2253761-21kworker/3:2+events20:56:133
222520099197310,1961cyclictest2230063-21kworker/2:0+events19:31:232
22252049919725,1964cyclictest2289647-21kworker/3:0+events22:43:453
222520499197210,1959cyclictest2233266-21kworker/3:0+events19:39:503
222520499197110,1958cyclictest2241243-21kworker/3:2+events20:12:163
22251989919718,1960cyclictest2268510-21kworker/1:2+events21:33:351
22252049919708,1959cyclictest2275056-21kworker/3:2+events21:52:393
22252049919706,1961cyclictest2233266-21kworker/3:0+events19:47:043
22252009919698,1958cyclictest2321243-21kworker/2:2+events00:32:502
22252049919666,1957cyclictest2304245-21kworker/3:0+events23:21:143
222520099196410,1950cyclictest2277301-21kworker/2:0+events21:59:112
222520099196310,1950cyclictest2304120-21kworker/2:2+events23:26:582
222520099196210,1950cyclictest2254094-21kworker/2:2+events20:49:032
22252009919619,1949cyclictest2273485-21kworker/2:0+events21:47:382
222520099196010,1947cyclictest2323388-21kworker/2:0+events00:20:442
22251989919607,1950cyclictest2248147-21kworker/1:0+events20:42:091
222519899196010,1947cyclictest2220085-21kworker/1:1+events20:20:331
22252009919595,1951cyclictest2296059-21kworker/2:2+events22:55:102
222520099195910,1946cyclictest2271863-21kworker/2:1+events22:09:512
222520099195910,1945cyclictest2283079-21kworker/2:2+events22:26:072
22251989919597,1950cyclictest2286855-21kworker/1:1+events23:05:101
222519899195910,1946cyclictest2307724-21kworker/1:0+events23:53:211
222520099195810,1945cyclictest2325424-21kworker/2:1+events00:28:392
22251989919585,1950cyclictest2248147-21kworker/1:0+events20:50:351
222519899195810,1945cyclictest2314213-21kworker/1:1+events00:13:501
222519899195410,1941cyclictest2253884-21kworker/1:2+events20:47:091
22251989919519,1939cyclictest2233405-21kworker/1:0+events19:37:341
222519899195110,1938cyclictest2320481-21kworker/1:0+events00:24:071
22252009919508,1938cyclictest2239640-21kworker/2:2+events20:04:142
22252009919507,1940cyclictest2263948-21kworker/2:0+events21:16:482
22251989919509,1938cyclictest2220085-21kworker/1:1+events19:28:191
22252009919499,1937cyclictest2260431-21kworker/2:2+events21:05:502
222520099194910,1936cyclictest2312345-21kworker/2:0+events23:48:522
22252049919487,1938cyclictest2280051-21kworker/3:2+events22:04:083
22251989919489,1935cyclictest2286855-21kworker/1:1+events22:57:271
22252049919479,1935cyclictest2276943-21kworker/3:0+events21:58:473
22252009919478,1936cyclictest2251814-21kworker/2:1+events20:39:292
22251989919479,1935cyclictest2220085-21kworker/1:1+events19:20:391
22252009919448,1933cyclictest2299348-21kworker/2:2+events23:08:112
22251989919399,1927cyclictest2258568-21kworker/1:1+events21:25:361
222520099193810,1925cyclictest2221411-21kworker/2:0+events19:15:472
22251989919355,1927cyclictest2273338-21kworker/1:1+events21:48:461
22252009919349,1922cyclictest2286708-21kworker/2:0+events22:37:562
222519899193310,1919cyclictest2280171-21kworker/1:1+events22:13:431
22252009919322,1927cyclictest2263948-21kworker/2:0+events21:14:322
222519899193211,1919cyclictest2284689-21kworker/1:2+events22:25:221
22251989919219,1909cyclictest2273338-21kworker/1:1+events21:50:421
222520099191810,1905cyclictest2321243-21kworker/2:2+events00:14:132
22252049919169,1904cyclictest2264037-21kworker/3:0+events21:22:593
22251989919143,1908cyclictest2302288-21kworker/1:2+events23:35:231
22251989919079,1895cyclictest2292789-21kworker/1:2+events22:50:351
22252049919022,1897cyclictest2224722-21kworker/3:1+events19:13:543
22252049919019,1889cyclictest2294634-21kworker/3:2+events22:54:473
22251989918977,1887cyclictest2270226-21kworker/1:0+events21:41:331
222519899189710,1884cyclictest2302288-21kworker/1:2+events23:55:591
22252009918969,1884cyclictest2263948-21kworker/2:0+events21:32:502
22252009918963,1890cyclictest2321243-21kworker/2:2+events00:35:312
22251989918927,1882cyclictest2286855-21kworker/1:1+events22:36:031
22252009918904,1883cyclictest2239640-21kworker/2:2+events20:32:202
22252009918869,1874cyclictest2233305-21kworker/2:1+events19:39:042
222520099188410,1871cyclictest2304120-21kworker/2:2+events23:16:362
22252009918828,1871cyclictest2237778-21kworker/2:1+events19:47:492
22252009918819,1869cyclictest2255781-21kworker/2:0+events20:55:262
22252009918819,1869cyclictest2255781-21kworker/2:0+events20:53:122
22251989918819,1869cyclictest2325254-21kworker/1:2+events00:32:271
222519899187510,1862cyclictest2299470-21kworker/1:0+events23:10:531
22251949918708,1859cyclictest2260639-21kworker/0:0+events21:05:250
22251989918528,1841cyclictest2220085-21kworker/1:1+events19:11:371
22252009918518,1840cyclictest2316761-21kworker/2:0+events00:04:232
22252009918419,1829cyclictest2239640-21kworker/2:2+events20:28:542
22252009918309,1818cyclictest2283079-21kworker/2:2+events22:17:592
22251989918249,1812cyclictest2223124-21kworker/1:2+events19:15:231
222519899181010,1797cyclictest2246483-21kworker/1:2+events20:16:511
22246992170139,28sleep30-21swapper/319:07:023
22246292145116,24sleep10-21swapper/119:06:071
22245752145118,22sleep20-21swapper/219:05:282
22248072141109,27sleep00-21swapper/019:08:230
222519899232,19cyclictest2311622-21ldconfig23:40:151
2225198992215,4cyclictest0-21swapper/119:50:001
2225198992212,8cyclictest726-21mta-sts-daemon21:10:001
222519899221,16cyclictest2252550-21systemctl20:39:591
222519899211,14cyclictest2239146-21ntpq19:50:181
222519899201,16cyclictest2265711-21munin-run21:20:001
222519899200,7cyclictest2277820-21ntpq21:55:151
222519899200,19cyclictest2329485-21/usr/sbin/munin00:35:211
2225198991811,5cyclictest2320327-21sh00:10:001
222519899181,13cyclictest2250323-21sh20:30:001
222519899171,8cyclictest2317997-21ntpq00:00:181
2225198991711,4cyclictest745-21runrttasks19:56:131
222519899171,14cyclictest0-21swapper/123:30:001
222519899171,13cyclictest2270724-21apt-get21:35:111
222519899171,12cyclictest0-21swapper/121:10:141
2225198991710,5cyclictest2242286-21hddtemp_smartct20:00:161
222519899170,14cyclictest2289990-21perf22:35:001
222519899163,11cyclictest2323754-21munin-run00:20:001
2225198991612,2cyclictest2308441-21ntpq23:30:181
222519899161,13cyclictest2244954-21awk20:10:011
222519899160,3cyclictest2259385-21/usr/sbin/munin21:00:201
222519899160,3cyclictest2259385-21/usr/sbin/munin21:00:201
222519899160,14cyclictest2286841-21awk22:25:001
222519899160,13cyclictest2281061-21ntpq22:05:181
222519899159,4cyclictest2267201-21chrt21:23:501
222519899150,13cyclictest2313995-21idleruntime-cro23:50:001
222519899150,12cyclictest2326006-21meminfo00:25:171
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional