You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-26 - 20:27

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #f, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails
  
Click here to display the system's profile data.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackfslot7s.osadl.org (updated Wed Nov 26, 2025 00:44:16)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
17533569928069,2793cyclictest1791526-21kworker/2:0+events21:10:182
175335299266813,2651cyclictest1820555-21kworker/1:1+events22:50:101
175336099263113,2614cyclictest1823657-21kworker/3:1+events23:00:183
175335299261413,2599cyclictest1830162-21kworker/1:0+events23:10:161
175335299259412,2577cyclictest1767815-21kworker/1:2+events20:20:121
175335099258712,2572cyclictest1817265-21kworker/0:0+events22:45:180
175336099257912,2564cyclictest1847713-21kworker/3:2+events00:05:193
17533529925559,2542cyclictest1850755-21kworker/1:1+events00:35:191
175335299248810,2475cyclictest1767815-21kworker/1:2+events19:58:441
175335299248710,2474cyclictest1783799-21kworker/1:0+events20:50:151
175336099248612,2471cyclictest1820317-21kworker/3:2+events22:45:413
175335299247912,2463cyclictest1758454-21kworker/1:2+events19:30:131
175335299247911,2465cyclictest1794719-21kworker/1:1+events21:25:301
175335299245311,2439cyclictest1783799-21kworker/1:0+events21:04:181
175335299240710,2393cyclictest1798211-21kworker/1:2+events21:45:161
175335299238112,2366cyclictest1798211-21kworker/1:2+events21:33:531
17533529923769,2365cyclictest1804642-21kworker/1:1+events21:55:061
175336099235512,2338cyclictest1809323-21kworker/3:0+events22:04:173
175335099235412,2339cyclictest1834866-21kworker/0:0+events23:53:310
175335699234311,2328cyclictest1809228-21kworker/2:2+events22:25:142
175335099232411,2309cyclictest1831615-21kworker/0:1+events23:18:100
17533569923229,2309cyclictest1759873-21kworker/2:2+events19:26:472
175336099231512,2300cyclictest1823657-21kworker/3:1+events22:58:033
175335299231011,2296cyclictest1780598-21kworker/1:2+events20:41:561
175335299228311,2268cyclictest1828524-21kworker/1:1+events23:05:161
175335299227211,2258cyclictest1818897-21kworker/1:2+events23:03:451
175335299227211,2258cyclictest1788629-21kworker/1:2+events21:21:211
175335299226910,2256cyclictest1761750-21kworker/1:0+events19:46:111
175335299226711,2253cyclictest1746834-21kworker/1:1+events19:14:191
175335299226710,2254cyclictest1831762-21kworker/1:1+events23:16:421
175335299226710,2254cyclictest1798211-21kworker/1:2+events21:38:031
17533529922669,2254cyclictest1850755-21kworker/1:1+events00:33:491
175335299226611,2252cyclictest1817201-21kworker/1:0+events22:29:471
17533529922659,2253cyclictest1780598-21kworker/1:2+events20:33:511
175335299226511,2251cyclictest1830162-21kworker/1:0+events23:20:281
175335299226410,2250cyclictest1780598-21kworker/1:2+events20:38:071
175335299226210,2248cyclictest1798211-21kworker/1:2+events21:51:181
175336099225811,2245cyclictest1806119-21kworker/3:1+events22:05:033
175335099225711,2243cyclictest1825579-21kworker/0:2+events22:59:550
17533609922568,2245cyclictest1783739-21kworker/3:1+events20:47:573
175336099225511,2240cyclictest1769539-21kworker/3:1+events20:04:053
17533609922548,2243cyclictest1845967-21kworker/3:1+events23:58:033
17533609922526,2243cyclictest1758421-21kworker/3:1+events19:23:463
175335299225010,2237cyclictest1818897-21kworker/1:2+events22:38:071
175335299224911,2235cyclictest1793118-21kworker/1:0+events21:12:111
17533529922419,2229cyclictest1763418-21kworker/1:2+events19:41:101
175335299223710,2224cyclictest1767815-21kworker/1:2+events20:15:561
17533609922357,2225cyclictest1830128-21kworker/3:1+events23:08:453
175335299223511,2221cyclictest1837977-21kworker/1:2+events23:39:541
175335699222610,2213cyclictest1759873-21kworker/2:2+events19:37:312
175335099222510,2212cyclictest1817265-21kworker/0:0+events23:10:400
175335699222211,2207cyclictest1850641-21kworker/2:0+events00:18:092
175335699222210,2209cyclictest1798152-21kworker/2:1+events21:44:312
175335699221910,2206cyclictest1799784-21kworker/2:2+events21:36:352
17533569922176,2207cyclictest1841349-21kworker/2:1+events00:03:032
175335699221710,2204cyclictest1759873-21kworker/2:2+events19:31:462
175335699221611,2202cyclictest1835018-21kworker/2:1+events23:30:002
17533569922157,2205cyclictest1777363-21kworker/2:0+events20:22:492
17533569922148,2203cyclictest1850641-21kworker/2:0+events00:29:552
17533569922148,2203cyclictest1841349-21kworker/2:1+events23:46:412
17533529922129,2200cyclictest1788629-21kworker/1:2+events21:08:261
175335299220610,2193cyclictest1817201-21kworker/1:0+events22:33:551
175335299220111,2188cyclictest1850755-21kworker/1:1+events00:25:011
175335699219911,2185cyclictest1828703-21kworker/2:0+events23:06:492
17533569921979,2185cyclictest1772583-21kworker/2:1+events20:19:492
17533569921967,2187cyclictest1795311-21kworker/2:2+events21:25:072
175335699219611,2182cyclictest1798152-21kworker/2:1+events22:05:482
175335699219610,2183cyclictest1766570-21kworker/2:0+events19:53:242
175335699219510,2182cyclictest1825308-21kworker/2:0+events22:54:182
175335699219411,2180cyclictest1820284-21kworker/2:2+events22:42:142
175335699219110,2178cyclictest1815879-21kworker/2:0+events22:36:332
175335299218910,2175cyclictest1769448-21kworker/1:1+events20:12:051
175335099218911,2174cyclictest1855842-21kworker/0:2+events00:32:160
175335699218310,2170cyclictest1826969-21kworker/2:1+events23:11:032
175335099217811,2165cyclictest1836652-21kworker/0:3+events23:35:200
175335299217611,2162cyclictest1812344-21kworker/1:1+events22:18:431
17533569921719,2159cyclictest1810860-21kworker/2:0+events22:13:012
175335099216710,2154cyclictest1775892-21kworker/0:0+events20:29:390
17533609921658,2154cyclictest1804435-21kworker/3:0+events21:48:163
175335299216510,2152cyclictest1809591-21kworker/1:2+events22:08:031
175335699216410,2151cyclictest1841349-21kworker/2:1+events23:58:492
175335699216211,2148cyclictest1775736-21kworker/2:2+events20:32:392
17533509921627,2152cyclictest1806376-21kworker/0:2+events22:15:420
175335099216211,2149cyclictest1772678-21kworker/0:1+events20:15:100
175335099216110,2148cyclictest1828403-21kworker/0:1+events23:02:350
17533569921607,2150cyclictest1785492-21kworker/2:2+events20:50:392
17533569921606,2151cyclictest1815879-21kworker/2:0+events22:30:322
175335699216011,2146cyclictest1775736-21kworker/2:2+events20:39:392
175335699216010,2147cyclictest1803133-21kworker/2:2+events21:47:312
175335299215910,2146cyclictest1852362-21kworker/1:2+events00:22:441
175335099215911,2145cyclictest1839819-21kworker/0:1+events23:40:160
17533509921588,2147cyclictest1772678-21kworker/0:1+events20:11:430
175335099215710,2144cyclictest1817265-21kworker/0:0+events22:43:010
175335099215510,2142cyclictest1793385-21kworker/0:0+events21:27:020
175335099215411,2140cyclictest1847860-21kworker/0:1+events00:09:280
17533509921536,2144cyclictest1788598-21kworker/0:1+events21:11:470
175335099215311,2139cyclictest1752962-21kworker/0:2+events19:33:200
175335099215310,2140cyclictest1757008-21kworker/0:1+events19:24:070
175335099215211,2138cyclictest1763320-21kworker/0:1+events20:00:410
175335099215210,2139cyclictest1855842-21kworker/0:2+events00:38:020
175335099215011,2136cyclictest1771013-21kworker/0:2+events20:05:350
17533529921478,2135cyclictest1837977-21kworker/1:2+events23:31:531
175335099214510,2132cyclictest1791612-21kworker/0:2+events21:24:440
175335099214310,2130cyclictest1782115-21kworker/0:0+events20:59:200
175335699214110,2128cyclictest1743563-21kworker/2:1+events19:22:122
17533509921408,2129cyclictest1793385-21kworker/0:0+events21:51:390
175335099213711,2123cyclictest1817265-21kworker/0:0+events22:37:440
175335099213610,2123cyclictest1830009-21kworker/0:2+events23:26:100
175335699213510,2122cyclictest1826969-21kworker/2:1+events22:58:272
17533569921294,2121cyclictest1798152-21kworker/2:1+events21:55:522
175335099212911,2114cyclictest1766515-21kworker/0:2+events19:54:110
175335099212611,2111cyclictest1812524-21kworker/0:0+events22:14:340
17533529921218,2110cyclictest1850755-21kworker/1:1+events00:14:231
175336099212011,2106cyclictest1810982-21kworker/3:2+events22:11:543
175335299212010,2106cyclictest1807844-21kworker/1:0+events22:00:501
175335299211910,2106cyclictest1746834-21kworker/1:1+events19:16:131
175336099211810,2105cyclictest1836441-21kworker/3:2+events23:34:563
175335299211710,2104cyclictest1845908-21kworker/1:0+events23:57:401
175335299211710,2104cyclictest1820555-21kworker/1:1+events22:44:551
175335299211610,2103cyclictest1778871-21kworker/1:0+events20:26:551
175336099211510,2102cyclictest1820317-21kworker/3:2+events22:43:233
175336099211310,2100cyclictest1800100-21kworker/3:2+events21:57:263
175335299211310,2100cyclictest1839728-21kworker/1:1+events00:03:241
17533609921127,2102cyclictest1806119-21kworker/3:1+events22:29:243
17533609921108,2098cyclictest1800100-21kworker/3:2+events21:52:263
17533529921109,2098cyclictest1758454-21kworker/1:2+events19:26:251
175335099211010,2097cyclictest1777647-21kworker/0:2+events20:47:330
17533529921098,2098cyclictest1767815-21kworker/1:2+events20:07:071
175335699210810,2095cyclictest1787130-21kworker/2:0+events20:56:422
17533529921087,2098cyclictest1801421-21kworker/1:1+events21:40:421
175335299210810,2095cyclictest1809591-21kworker/1:2+events22:11:071
17533529921078,2096cyclictest1841408-21kworker/1:0+events23:47:511
175335299210710,2094cyclictest1830162-21kworker/1:0+events23:29:151
175335299210610,2093cyclictest1845908-21kworker/1:0+events00:08:171
17533509921068,2095cyclictest1803015-21kworker/0:2+events21:43:460
175336099210511,2091cyclictest1838276-21kworker/3:0+events23:37:403
17533529921059,2093cyclictest1812344-21kworker/1:1+events22:22:091
175335099210510,2092cyclictest1817265-21kworker/0:0+events22:29:020
175336099210410,2091cyclictest1806119-21kworker/3:1+events22:31:393
17533529921049,2092cyclictest1839728-21kworker/1:1+events23:50:311
17533529921047,2094cyclictest1746834-21kworker/1:1+events19:21:491
17533529921036,2093cyclictest1767815-21kworker/1:2+events20:04:281
17533529921033,2097cyclictest1783799-21kworker/1:0+events20:56:201
175335299210310,2090cyclictest1767815-21kworker/1:2+events19:54:321
175336099210210,2089cyclictest1853924-21kworker/3:1+events00:23:073
175336099210210,2089cyclictest1774473-21kworker/3:2+events20:19:043
17533529921028,2091cyclictest1760113-21kworker/1:1+events19:39:001
17533609921019,2089cyclictest1812910-21kworker/3:0+events22:21:463
17533609921019,2089cyclictest1800100-21kworker/3:2+events21:37:193
175336099210111,2087cyclictest1828464-21kworker/3:0+events23:17:483
17533609921007,2089cyclictest1761857-21kworker/3:2+events19:38:373
17533609920989,2086cyclictest1766242-21kworker/3:0+events19:58:223
17533529920989,2086cyclictest1845908-21kworker/1:0+events00:17:031
17533609920969,2084cyclictest1836441-21kworker/3:2+events23:40:383
17533609920967,2087cyclictest1841111-21kworker/3:1+events23:50:073
17533529920968,2085cyclictest1782180-21kworker/1:1+events20:49:311
175335299209410,2081cyclictest1839728-21kworker/1:1+events23:41:021
17533609920927,2082cyclictest1769539-21kworker/3:1+events20:31:113
175336099209110,2078cyclictest1793210-21kworker/3:1+events21:23:353
175335699208810,2075cyclictest1857237-21kworker/2:1+events00:34:562
175336099208610,2073cyclictest1822027-21kworker/3:0+events22:53:113
175335099208411,2070cyclictest1791612-21kworker/0:2+events21:08:470
175336099208011,2066cyclictest1801861-21kworker/3:1+events21:41:053
175336099208011,2066cyclictest1796728-21kworker/3:0+events21:30:263
17533609920759,2063cyclictest1793210-21kworker/3:1+events21:19:243
175335699207410,2061cyclictest1775736-21kworker/2:2+events20:27:192
175335299207010,2056cyclictest1818897-21kworker/1:2+events22:56:141
175335699206910,2056cyclictest1769357-21kworker/2:2+events19:59:552
175335699206810,2055cyclictest1826969-21kworker/2:1+events23:01:032
175335699206810,2055cyclictest1759873-21kworker/2:2+events19:44:162
175335699206810,2054cyclictest1805973-21kworker/2:0+events21:53:112
175335699206810,2054cyclictest1785492-21kworker/2:2+events21:04:392
17533569920679,2055cyclictest1791526-21kworker/2:0+events21:20:112
175335699206710,2054cyclictest1766570-21kworker/2:0+events20:06:432
17533609920667,2055cyclictest1806119-21kworker/3:1+events22:15:183
17533569920668,2055cyclictest1764880-21kworker/2:1+events19:48:272
17533569920659,2053cyclictest1814070-21kworker/2:1+events22:21:232
17533509920655,2057cyclictest1817265-21kworker/0:0+events23:06:030
17533569920639,2051cyclictest1775736-21kworker/2:2+events20:41:332
17533569920636,2054cyclictest1831586-21kworker/2:2+events23:21:362
17533569920622,2057cyclictest1780565-21kworker/2:0+events20:46:262
175335099206211,2048cyclictest1801331-21kworker/0:1+events21:39:570
17533569920618,2050cyclictest1850641-21kworker/2:0+events00:11:422
17533569920617,2051cyclictest1810860-21kworker/2:0+events22:17:142
17533569920616,2052cyclictest1855574-21kworker/2:2+events00:36:282
175335099205710,2044cyclictest1806376-21kworker/0:2+events22:06:340
17533609920519,2039cyclictest1783739-21kworker/3:1+events20:51:243
175336099204911,2035cyclictest1836441-21kworker/3:2+events23:28:523
17533569920499,2037cyclictest1831586-21kworker/2:2+events23:18:552
17533529920489,2035cyclictest1794719-21kworker/1:1+events21:19:011
17533509920479,2035cyclictest1782115-21kworker/0:0+events20:53:190
17533609920468,2035cyclictest1844286-21kworker/3:0+events00:12:513
17533609920467,2036cyclictest1854037-21kworker/3:3+events00:28:493
17533569920468,2035cyclictest1836703-21kworker/2:2+events23:32:392
17533509920468,2035cyclictest1817265-21kworker/0:0+events22:34:400
17533609920458,2034cyclictest1844286-21kworker/3:0+events00:19:163
17533609920457,2035cyclictest1761857-21kworker/3:2+events19:43:293
175336099204310,2030cyclictest1782082-21kworker/3:0+events20:41:103
17533569920422,2037cyclictest1839611-21kworker/2:0+events23:43:402
17533529920418,2030cyclictest1818897-21kworker/1:2+events22:47:531
175335699204010,2027cyclictest1766570-21kworker/2:0+events20:01:032
17533509920358,2024cyclictest1780506-21kworker/0:1+events20:33:280
17533569920349,2022cyclictest1743563-21kworker/2:1+events19:12:252
17533609920319,2019cyclictest1793210-21kworker/3:1+events21:14:273
17533569920317,2021cyclictest1809228-21kworker/2:2+events22:03:302
175335699203110,2018cyclictest1850641-21kworker/2:0+events00:21:112
17533609920292,2025cyclictest1769539-21kworker/3:1+events20:25:063
17533609920289,2016cyclictest1857413-21kworker/3:0+events00:39:113
17533609920288,2017cyclictest1828464-21kworker/3:0+events23:21:593
17533609920288,2017cyclictest1750094-21kworker/3:0+events19:15:273
17533609920287,2018cyclictest1845967-21kworker/3:1+events00:01:323
17533569920287,2018cyclictest1799784-21kworker/2:2+events21:34:162
17533609920279,2015cyclictest1769539-21kworker/3:1+events20:09:263
17533609920268,2015cyclictest1790345-21kworker/3:0+events21:06:333
17533609920239,2011cyclictest1796728-21kworker/3:0+events21:28:313
175335099202010,2007cyclictest1830009-21kworker/0:2+events23:22:220
175335099201910,2005cyclictest1772678-21kworker/0:1+events20:23:100
17533569920157,2005cyclictest1839611-21kworker/2:0+events23:38:462
17533509920147,2004cyclictest1752962-21kworker/0:2+events19:36:190
175335099200810,1995cyclictest1752962-21kworker/0:2+events19:18:050
17533509920059,1993cyclictest1836652-21kworker/0:3+events23:33:260
175335099200410,1991cyclictest1834866-21kworker/0:0+events00:01:090
175335099200410,1991cyclictest1777647-21kworker/0:2+events20:43:490
175335099200310,1990cyclictest1806376-21kworker/0:2+events21:57:470
17533509920028,1991cyclictest1769385-21kworker/0:0+events19:59:310
175336099200111,1987cyclictest1766242-21kworker/3:0+events19:49:373
175335099200010,1986cyclictest1793385-21kworker/0:0+events21:31:100
17533569919989,1986cyclictest1841349-21kworker/2:1+events23:51:162
17533509919988,1988cyclictest1763320-21kworker/0:1+events19:50:000
175335099199810,1985cyclictest1791612-21kworker/0:2+events21:18:390
17533509919978,1987cyclictest1834866-21kworker/0:0+events23:55:030
17533509919977,1987cyclictest1852330-21kworker/0:2+events00:21:330
175335099199510,1982cyclictest1788598-21kworker/0:1+events21:01:140
17533509919948,1983cyclictest1757008-21kworker/0:1+events19:27:120
17533509919946,1985cyclictest1777647-21kworker/0:2+events20:37:200
17533509919929,1980cyclictest1842712-21kworker/0:2+events23:48:370
175335699198510,1972cyclictest1789925-21kworker/2:1+events21:06:122
175335099198310,1970cyclictest1763320-21kworker/0:1+events19:43:520
17533569919822,1977cyclictest1743563-21kworker/2:1+events19:19:352
17533609919649,1951cyclictest1780448-21kworker/3:2+events20:36:553
17533609919589,1946cyclictest1772796-21kworker/3:0+events20:14:003
175336099195210,1939cyclictest1788661-21kworker/3:1+events21:03:543
17533509919522,1947cyclictest1806376-21kworker/0:2+events22:22:330
17533569919514,1944cyclictest1841349-21kworker/2:1+events00:06:022
17533609919489,1936cyclictest1750094-21kworker/3:0+events19:28:403
17533609919479,1935cyclictest1836441-21kworker/3:2+events23:45:343
17533609919479,1935cyclictest1785341-21kworker/3:0+events20:58:133
17533509919439,1931cyclictest1834866-21kworker/0:0+events00:13:590
175336099193510,1922cyclictest1806119-21kworker/3:1+events22:36:573
17533509919257,1915cyclictest1834866-21kworker/0:0+events00:25:230
17533509919219,1909cyclictest1806376-21kworker/0:2+events22:03:540
17533569919178,1906cyclictest1789925-21kworker/2:1+events21:17:522
175335099191710,1904cyclictest1752962-21kworker/0:2+events19:14:420
17533509919168,1905cyclictest1834866-21kworker/0:0+events00:17:250
175336099189210,1879cyclictest1766242-21kworker/3:0+events19:50:213
17533609918819,1869cyclictest1750094-21kworker/3:0+events19:10:523
17533609918799,1867cyclictest1830128-21kworker/3:1+events23:12:123
17533609918724,1865cyclictest1855635-21kworker/3:1+events00:33:253
17533509918377,1827cyclictest1793385-21kworker/0:0+events21:49:230
17529972155110,25sleep10-21swapper/119:08:291
17530672152112,24sleep30-21swapper/319:09:223
17530442147100,26sleep00-21swapper/019:09:040
17528072146104,38sleep20-21swapper/219:06:062
175336099116114,1cyclictest1753445-21kworker/u16:4+events_unbound19:34:503
175336099115114,1cyclictest1772896-21kworker/u16:3+events_unbound20:24:453
1753350993128,2cyclictest1817265-21kworker/0:0+pm22:54:550
1753356991411,2cyclictest1773357-21/usr/sbin/munin20:10:182
175335699125,4cyclictest1822575-21cat22:45:122
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional