You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-10-17 - 10:51

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #f, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails
  
Click here to display the system's profile data.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackfslot7s.osadl.org (updated Fri Oct 17, 2025 00:44:17)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
190947899292112,2905cyclictest2006981-21kworker/2:2+events00:35:212
190947899287012,2854cyclictest1997522-21kworker/2:1+events23:55:192
190946999283913,2822cyclictest1992697-21kworker/0:1+events23:40:160
190946999271611,2701cyclictest1941081-21kworker/0:0+events21:05:000
190948399268413,2667cyclictest1945696-21kworker/3:1+events21:15:183
190948399267213,2656cyclictest1960858-21kworker/3:1+events22:00:013
190947199257212,2556cyclictest1951936-21kworker/1:0+events21:30:341
190947199255712,2543cyclictest1986165-21kworker/1:1+events23:20:451
190947199254012,2524cyclictest1967108-21kworker/1:0+events22:56:511
190946999250511,2490cyclictest1978382-21kworker/0:2+events22:56:290
190947199249212,2476cyclictest1926913-21kworker/1:0+events20:08:091
190947199248812,2473cyclictest1973654-21kworker/1:1+events22:42:491
190946999248812,2474cyclictest1984628-21kworker/0:1+events23:30:400
190947199248710,2474cyclictest1934801-21kworker/1:0+events20:30:571
190947199248612,2472cyclictest1951936-21kworker/1:0+events21:40:111
190947199248611,2471cyclictest1982932-21kworker/1:2+events23:19:341
190947199248512,2470cyclictest1957621-21kworker/1:0+events21:56:321
190947199248512,2469cyclictest1987798-21kworker/1:0+events23:25:441
190947199248512,2469cyclictest1987798-21kworker/1:0+events23:25:431
190947199248412,2468cyclictest1944228-21kworker/1:1+events21:04:131
19094719924752,2470cyclictest2002349-21kworker/1:1+events00:10:361
190947199246911,2454cyclictest1953542-21kworker/1:1+events21:39:481
190946999246310,2449cyclictest2000688-21kworker/0:1+events00:05:170
190947199246212,2446cyclictest1987798-21kworker/1:0+events23:37:581
190947899245911,2444cyclictest1960644-21kworker/2:2+events22:00:452
190946999245512,2441cyclictest1963925-21kworker/0:0+events22:30:190
190947199245212,2437cyclictest1995837-21kworker/1:2+events23:49:001
190947199245112,2436cyclictest1945845-21kworker/1:0+events21:15:401
190946999245011,2437cyclictest1984628-21kworker/0:1+events23:25:220
190946999245011,2437cyclictest1984628-21kworker/0:1+events23:25:220
190947899244910,2435cyclictest1982750-21kworker/2:0+events23:07:522
190947199244911,2435cyclictest1961148-21kworker/1:2+events22:39:041
190946999244912,2433cyclictest1967327-21kworker/0:1+events22:24:390
190947199244812,2432cyclictest1945845-21kworker/1:0+events21:05:211
190946999244812,2432cyclictest1963925-21kworker/0:0+events22:27:420
190946999244712,2431cyclictest1942686-21kworker/0:1+events20:54:260
190946999244710,2434cyclictest1953947-21kworker/0:1+events21:49:430
190946999244611,2432cyclictest1981239-21kworker/0:0+events23:04:030
190946999244611,2432cyclictest1951513-21kworker/0:2+events21:36:220
190946999244610,2432cyclictest1918800-21kworker/0:1+events19:51:500
190946999244312,2428cyclictest1948844-21kworker/0:2+events21:11:290
190946999244112,2426cyclictest1937420-21kworker/0:1+events20:46:300
19094839924348,2423cyclictest1948916-21kworker/3:0+events21:20:363
190946999242912,2413cyclictest1928705-21kworker/0:2+events20:20:200
19094719924279,2414cyclictest1967108-21kworker/1:0+events22:29:351
190946999242512,2409cyclictest1991062-21kworker/0:2+events23:37:370
190946999242412,2408cyclictest1987583-21kworker/0:2+events23:23:490
190946999241710,2403cyclictest1946027-21kworker/0:1+events21:07:180
190947199241411,2400cyclictest1942535-21kworker/1:0+events20:53:181
190946999241012,2394cyclictest1959130-21kworker/0:0+events21:56:560
190946999240912,2395cyclictest2002379-21kworker/0:2+events00:12:510
190948399240812,2393cyclictest1933135-21kworker/3:2+events20:27:083
19094839923949,2381cyclictest1921965-21kworker/3:0+events19:51:283
190947899238412,2368cyclictest1939580-21kworker/2:1+events20:44:182
190947899238411,2370cyclictest1968520-21kworker/2:0+events22:38:192
190948399237712,2363cyclictest1982954-21kworker/3:1+events23:10:113
190947899237511,2361cyclictest2006981-21kworker/2:2+events00:27:192
190947899237011,2356cyclictest1923508-21kworker/2:1+events19:59:262
190947899236111,2348cyclictest1962332-21kworker/2:1+events22:07:072
190947899236012,2345cyclictest1962332-21kworker/2:1+events22:21:342
190948399235912,2344cyclictest1965261-21kworker/3:0+events22:13:593
190948399235912,2343cyclictest1913688-21kworker/3:1+events19:25:193
190948399235811,2343cyclictest1984561-21kworker/3:2+events23:29:343
190948399235811,2343cyclictest1984561-21kworker/3:2+events23:29:333
190947899235812,2344cyclictest1942503-21kworker/2:2+events20:55:342
190947199235711,2343cyclictest1981267-21kworker/1:1+events23:12:041
190948399235511,2340cyclictest1953734-21kworker/3:2+events21:40:573
190948399235312,2337cyclictest2003847-21kworker/3:0+events00:27:423
190947899235212,2336cyclictest1982750-21kworker/2:0+events23:20:212
190948399235112,2336cyclictest1920366-21kworker/3:2+events19:45:243
190948399235012,2335cyclictest1918754-21kworker/3:0+events19:43:083
190948399235011,2338cyclictest1959250-21kworker/3:2+events22:01:083
190947899235012,2335cyclictest2000899-21kworker/2:3+events00:05:422
190948399234912,2334cyclictest1995649-21kworker/3:1+events23:47:043
19094789923499,2337cyclictest1995928-21kworker/2:0+events23:50:272
190948399234812,2333cyclictest1913688-21kworker/3:1+events19:33:593
190948399234812,2332cyclictest1956983-21kworker/3:1+events21:54:173
190948399234812,2332cyclictest1945696-21kworker/3:1+events21:29:033
190947899234812,2333cyclictest1951575-21kworker/2:1+events21:35:132
190948399234710,2334cyclictest1973302-21kworker/3:2+events22:40:343
190947199234510,2332cyclictest1953542-21kworker/1:1+events21:47:281
190948399234312,2327cyclictest1976533-21kworker/3:1+events22:48:533
190948399234111,2326cyclictest1948916-21kworker/3:0+events21:11:523
190948399233912,2324cyclictest1976533-21kworker/3:1+events22:54:133
190948399233812,2324cyclictest1925170-21kworker/3:0+events20:10:043
190948399233412,2318cyclictest1941281-21kworker/3:0+events21:02:223
190947199232112,2305cyclictest1929112-21kworker/1:1+events20:15:261
190946999232010,2306cyclictest1976678-21kworker/0:1+events22:50:250
190947899230911,2295cyclictest1939580-21kworker/2:1+events21:02:022
190946999230111,2287cyclictest1910681-21kworker/0:2+events19:17:250
190948399229911,2284cyclictest1945696-21kworker/3:1+events21:31:213
190948399229911,2284cyclictest1927282-21kworker/3:2+events20:05:123
190948399229710,2283cyclictest2007084-21kworker/3:2+events00:23:283
190946999229512,2280cyclictest2007012-21kworker/0:2+events00:33:020
190947199229412,2279cyclictest2008696-21kworker/1:0+events00:38:241
190947899227911,2265cyclictest1989543-21kworker/2:2+events23:33:462
190948399227511,2261cyclictest1976533-21kworker/3:1+events23:02:573
190946999227511,2260cyclictest1970186-21kworker/0:2+events22:35:180
19094789922739,2260cyclictest1925114-21kworker/2:2+events20:00:132
190947199227212,2257cyclictest1913792-21kworker/1:0+events19:37:031
190947199226711,2252cyclictest1961148-21kworker/1:2+events22:13:141
190947199226711,2252cyclictest1923391-21kworker/1:0+events19:54:281
190947199226611,2251cyclictest1925025-21kworker/1:1+events20:01:221
190947199226511,2251cyclictest1947455-21kworker/1:2+events21:14:311
190947199226510,2253cyclictest1934801-21kworker/1:0+events20:35:111
190948399226411,2249cyclictest1923565-21kworker/3:1+events19:57:563
190947199226411,2250cyclictest1967108-21kworker/1:0+events22:31:521
190947199226410,2251cyclictest1921728-21kworker/1:2+events20:28:181
190947199226310,2250cyclictest1967108-21kworker/1:0+events23:05:561
19094839922629,2250cyclictest1929876-21kworker/3:1+events20:23:203
19094719922629,2250cyclictest1938073-21kworker/1:1+events20:40:321
190947199226211,2248cyclictest1919466-21kworker/1:1+events19:42:231
190947199226210,2248cyclictest1950690-21kworker/1:1+events21:20:581
19094719922618,2250cyclictest1977981-21kworker/1:2+events22:51:551
190947199226111,2246cyclictest1951936-21kworker/1:0+events21:27:301
190947899226010,2247cyclictest1978575-21kworker/2:1+events23:03:192
19094719922608,2250cyclictest2008696-21kworker/1:0+events00:30:011
19094719922608,2249cyclictest1961148-21kworker/1:2+events22:06:031
190947199226011,2246cyclictest1967108-21kworker/1:0+events22:47:451
190947199226011,2246cyclictest1921728-21kworker/1:2+events19:58:181
190947199226011,2245cyclictest1995837-21kworker/1:2+events23:51:101
190947199226010,2247cyclictest1919466-21kworker/1:1+events19:48:481
190947199225910,2246cyclictest1913792-21kworker/1:0+events19:26:471
190947199225811,2244cyclictest1940934-21kworker/1:2+events20:57:301
190947199225810,2245cyclictest1906356-21kworker/1:1+events19:20:491
190948399225712,2242cyclictest1953734-21kworker/3:2+events21:48:143
190947199225711,2243cyclictest2006909-21kworker/1:1+events00:28:521
190947199225710,2244cyclictest1915389-21kworker/1:1+events19:34:451
190947199225611,2242cyclictest1987798-21kworker/1:0+events23:32:131
19094719922547,2244cyclictest1933163-21kworker/1:1+events20:24:051
190947199225412,2239cyclictest1938073-21kworker/1:1+events20:49:531
190947199225411,2239cyclictest1995837-21kworker/1:2+events23:57:371
190946999225210,2239cyclictest1974987-21kworker/0:0+events22:44:450
190948399225011,2235cyclictest1925170-21kworker/3:0+events20:01:003
190946999224913,2233cyclictest1924813-21kworker/0:2+events19:56:480
190946999224410,2232cyclictest1981239-21kworker/0:0+events23:15:070
190947199224211,2228cyclictest1961148-21kworker/1:2+events22:22:421
190947199224210,2228cyclictest2002478-21kworker/1:2+events00:23:521
190946999224210,2229cyclictest1953947-21kworker/0:1+events21:51:150
19094719922399,2227cyclictest1961148-21kworker/1:2+events22:02:411
190948399223812,2223cyclictest1997328-21kworker/3:3+events00:16:423
190946999223811,2224cyclictest1989450-21kworker/0:0+events23:58:470
190946999223710,2224cyclictest1915299-21kworker/0:2+events19:38:330
190946999223611,2222cyclictest1946027-21kworker/0:1+events21:18:180
190946999223210,2219cyclictest1956718-21kworker/0:0+events21:41:190
19094839922289,2215cyclictest1997328-21kworker/3:3+events00:07:103
19094699922286,2218cyclictest1933198-21kworker/0:0+events20:31:210
190946999222711,2213cyclictest1970186-21kworker/0:2+events22:45:320
190946999222511,2211cyclictest1951513-21kworker/0:2+events21:25:580
190946999222510,2212cyclictest1933198-21kworker/0:0+events20:37:500
190946999222410,2210cyclictest1911759-21kworker/0:1+events19:30:330
190946999222311,2208cyclictest1951513-21kworker/0:2+events21:24:030
19094699922219,2208cyclictest1931423-21kworker/0:1+events20:27:320
190946999222011,2207cyclictest1953106-21kworker/0:0+events21:30:100
19094699922189,2205cyclictest1922265-21kworker/0:0+events20:05:330
190946999221510,2202cyclictest1981239-21kworker/0:0+events23:09:020
190946999221410,2201cyclictest1926762-21kworker/0:1+events20:04:260
19094699922138,2202cyclictest1989450-21kworker/0:0+events00:03:020
190946999221311,2200cyclictest1913677-21kworker/0:0+events19:26:040
190946999221211,2198cyclictest1965562-21kworker/0:2+events22:17:230
190947199220710,2193cyclictest1921728-21kworker/1:2+events20:13:531
190946999220211,2188cyclictest1928705-21kworker/0:2+events20:15:480
190948399220011,2186cyclictest1929876-21kworker/3:1+events20:34:473
190948399220010,2186cyclictest1936435-21kworker/3:2+events20:47:593
190946999220010,2187cyclictest1918800-21kworker/0:1+events19:41:130
190946999219510,2182cyclictest1963925-21kworker/0:0+events22:10:560
190947899218912,2174cyclictest1934923-21kworker/2:0+events20:47:142
190948399218611,2171cyclictest1947281-21kworker/3:2+events21:08:263
19094789921859,2172cyclictest1944197-21kworker/2:0+events21:17:542
190947899218512,2170cyclictest1928653-21kworker/2:1+events20:22:122
19094699921859,2173cyclictest2007012-21kworker/0:2+events00:24:380
190947899218411,2170cyclictest1962332-21kworker/2:1+events22:25:472
190947899218311,2168cyclictest1954148-21kworker/2:0+events21:57:202
190948399218111,2166cyclictest1966866-21kworker/3:1+events22:37:103
190947899217011,2155cyclictest2003684-21kworker/2:0+events00:22:182
190947899216810,2154cyclictest1984536-21kworker/2:2+events23:17:412
190948399216710,2154cyclictest1939672-21kworker/3:1+events20:53:403
190947899216711,2153cyclictest1986489-21kworker/2:1+events23:28:472
190947899216711,2153cyclictest1986489-21kworker/2:1+events23:28:472
190947899216711,2153cyclictest1965718-21kworker/2:2+events22:19:152
190947899216610,2152cyclictest1928653-21kworker/2:1+events20:18:282
190947899216511,2151cyclictest1968520-21kworker/2:0+events22:59:312
190947899216511,2151cyclictest1908906-21kworker/2:1+events19:44:392
190947899216510,2152cyclictest2008582-21kworker/2:1+events00:30:472
190947899216510,2152cyclictest1976384-21kworker/2:2+events22:48:302
190947899216510,2152cyclictest1915121-21kworker/2:0+events19:27:342
19094789921649,2153cyclictest1916752-21kworker/2:2+events19:35:092
19094789921649,2151cyclictest1921815-21kworker/2:2+events19:52:142
190947899216411,2150cyclictest1925114-21kworker/2:2+events20:08:552
190947899216411,2149cyclictest1956869-21kworker/2:2+events21:45:562
190948399216313,2147cyclictest2002247-21kworker/3:1+events00:11:233
190948399216311,2149cyclictest1929876-21kworker/3:1+events20:18:493
190947899216310,2149cyclictest2003684-21kworker/2:0+events00:17:042
19094719921626,2153cyclictest1987798-21kworker/1:0+events23:42:111
19094699921587,2148cyclictest1989450-21kworker/0:0+events23:50:490
190948399215710,2144cyclictest1984561-21kworker/3:2+events23:41:483
190946999215610,2142cyclictest1918800-21kworker/0:1+events19:48:250
190948399215411,2139cyclictest1998917-21kworker/3:0+events23:59:343
190948399215410,2141cyclictest1966866-21kworker/3:1+events22:20:253
190948399215310,2140cyclictest1960858-21kworker/3:1+events22:07:293
19094719921539,2140cyclictest2002478-21kworker/1:2+events00:15:571
19094839921529,2140cyclictest1911823-21kworker/3:0+events19:20:263
190948399215211,2137cyclictest2003847-21kworker/3:0+events00:31:543
19094839921518,2140cyclictest1970098-21kworker/3:0+events22:29:563
190948399215111,2138cyclictest1966866-21kworker/3:1+events22:30:423
19094839921508,2139cyclictest1997328-21kworker/3:3+events23:52:423
19094839921508,2139cyclictest1981357-21kworker/3:0+events23:17:173
190948399214911,2136cyclictest1976533-21kworker/3:1+events22:57:133
190947199214510,2132cyclictest1999312-21kworker/1:3+events00:02:391
19094839921429,2129cyclictest2010334-21kworker/3:2+events00:39:343
190948399214211,2128cyclictest1986198-21kworker/3:1+events23:30:193
190948399214010,2126cyclictest1918754-21kworker/3:0+events19:39:183
19094839921389,2126cyclictest1992517-21kworker/3:0+events23:37:123
190948399213111,2116cyclictest1998917-21kworker/3:0+events00:03:463
190946999212910,2116cyclictest1939427-21kworker/0:2+events20:42:030
190947899212711,2113cyclictest1928653-21kworker/2:1+events20:29:502
190946999212710,2114cyclictest1922265-21kworker/0:0+events20:10:480
190948399212611,2112cyclictest1959250-21kworker/3:2+events22:19:393
190947899212511,2111cyclictest1954148-21kworker/2:0+events21:42:292
19094839921219,2109cyclictest1986198-21kworker/3:1+events23:22:393
190948399212111,2106cyclictest1941281-21kworker/3:0+events20:59:453
190948399211710,2104cyclictest1936435-21kworker/3:2+events20:36:193
190947899211311,2099cyclictest1976384-21kworker/2:2+events22:50:482
190947899210810,2095cyclictest1959039-21kworker/2:1+events21:53:292
190948399210511,2091cyclictest1945696-21kworker/3:1+events21:35:583
190947899210311,2089cyclictest1982750-21kworker/2:0+events23:13:582
19094789920926,2083cyclictest1994206-21kworker/2:1+events23:45:572
19094699920909,2078cyclictest1941081-21kworker/0:0+events20:59:000
19094699920889,2076cyclictest1995628-21kworker/0:2+events23:47:270
190946999208110,2068cyclictest2005286-21kworker/0:0+events00:19:210
19094719920798,2067cyclictest1961148-21kworker/1:2+events22:18:541
190947899207610,2063cyclictest1950598-21kworker/2:2+events21:20:142
19094719920759,2062cyclictest1967108-21kworker/1:0+events23:02:331
190946999207510,2062cyclictest2007012-21kworker/0:2+events00:28:050
19094699920749,2062cyclictest1957439-21kworker/0:2+events22:01:550
190947199207010,2057cyclictest1906356-21kworker/1:1+events19:18:341
190946999206710,2053cyclictest1982966-21kworker/0:2+events23:12:500
19094699920637,2054cyclictest1960740-21kworker/0:1+events22:08:370
19094699920617,2051cyclictest1910681-21kworker/0:2+events19:13:100
190947899205710,2044cyclictest1908906-21kworker/2:1+events19:32:062
19094699920517,2041cyclictest1911759-21kworker/0:1+events19:23:050
190947899204810,2036cyclictest1999131-21kworker/2:2+events00:00:422
190947899204310,2030cyclictest2003684-21kworker/2:0+events00:12:312
190948399204210,2029cyclictest1936435-21kworker/3:2+events20:43:553
19094719920429,2030cyclictest1997492-21kworker/1:0+events00:09:051
19094789920188,2007cyclictest1939580-21kworker/2:1+events20:50:382
19094789920159,2004cyclictest1934923-21kworker/2:0+events20:36:422
190947899201310,2001cyclictest1944197-21kworker/2:0+events21:12:132
19094789920108,1999cyclictest1944197-21kworker/2:0+events21:06:082
19094789920032,1998cyclictest1908906-21kworker/2:1+events19:17:022
190947899200310,1990cyclictest1989543-21kworker/2:2+events23:42:332
190947899200110,1988cyclictest1908906-21kworker/2:1+events19:14:422
190947899200010,1987cyclictest1918902-21kworker/2:0+events19:47:402
190948399199910,1982cyclictest1910725-21kworker/3:3+events19:13:333
19094719919988,1987cyclictest1906356-21kworker/1:1+events19:12:461
190947899199210,1979cyclictest1951575-21kworker/2:1+events21:25:362
19094839919878,1976cyclictest1910725-21kworker/3:3+events19:16:383
19094839919715,1963cyclictest1978133-21kworker/3:2+events23:05:363
190947899194410,1931cyclictest1963987-21kworker/2:0+events22:12:072
190947899194110,1928cyclictest1925114-21kworker/2:2+events20:11:572
19090132185152,30sleep30-21swapper/319:06:453
19092642155115,34sleep10-21swapper/119:09:551
19090892152106,26sleep20-21swapper/219:07:402
19092632141108,27sleep00-21swapper/019:09:540
190947899207,10cyclictest1935619-21/usr/sbin/munin20:30:142
190947899180,16cyclictest797-21snmpd22:43:372
190947899179,6cyclictest1-21systemd22:35:012
1909471991711,4cyclictest1959035-21seq21:51:451
1909478991610,4cyclictest1913565-21sshd19:21:332
190947899157,5cyclictest2479533-21systemd-journal21:31:372
1909478991410,2cyclictest1991987-21grep23:35:172
190946999135,5cyclictest2010888-21sh00:35:180
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional