You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-23 - 10:39

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #f, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails
  
Click here to display the system's profile data.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackfslot7s.osadl.org (updated Tue Dec 23, 2025 00:44:16)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
71646399266613,2650cyclictest767201-21kworker/2:2+events21:50:002
71646899257515,2557cyclictest796242-21kworker/3:2+events23:18:093
71646899251812,2503cyclictest772243-21kworker/3:1+events22:05:133
7164639924947,2484cyclictest744918-21kworker/2:0+events20:54:542
71646399248912,2473cyclictest724314-21kworker/2:1+events19:35:172
71646399248311,2469cyclictest716163-21kworker/2:1+events19:25:472
71645499247212,2457cyclictest733735-21kworker/0:0+events20:12:400
71645499246212,2447cyclictest738668-21kworker/0:2+events20:20:180
71646899245211,2438cyclictest790210-21kworker/3:0+events23:00:163
71646399242712,2413cyclictest761102-21kworker/2:0+events21:45:002
71646899241511,2401cyclictest783534-21kworker/3:1+events22:45:393
71646399240511,2391cyclictest749774-21kworker/2:2+events21:12:422
71646399239812,2384cyclictest749774-21kworker/2:2+events21:00:152
71646899239111,2376cyclictest735399-21kworker/3:0+events20:25:123
71646399238710,2374cyclictest759416-21kworker/2:2+events21:23:212
71645499237212,2356cyclictest802833-21kworker/0:0+events23:50:180
71646899234412,2329cyclictest744886-21kworker/3:0+events20:41:343
71645499232512,2310cyclictest806379-21kworker/0:1+events23:59:020
71646399232411,2309cyclictest804595-21kworker/2:2+events23:52:582
71645499231812,2304cyclictest726004-21kworker/0:2+events19:40:160
71646399231710,2305cyclictest778794-21kworker/2:1+events22:25:062
71645499231712,2302cyclictest757999-21kworker/0:2+events21:45:210
71646399231411,2299cyclictest782004-21kworker/2:0+events22:44:552
71646399231410,2301cyclictest810733-21kworker/2:2+events00:03:372
7164639923088,2296cyclictest738699-21kworker/2:1+events20:23:202
7164639923079,2295cyclictest740379-21kworker/2:2+events20:31:392
71646399230410,2291cyclictest814049-21kworker/2:2+events00:25:452
7164639923039,2291cyclictest757759-21kworker/2:1+events21:25:382
71646399230010,2288cyclictest740379-21kworker/2:2+events20:25:352
71646399229811,2284cyclictest814049-21kworker/2:2+events00:17:032
71646399229811,2284cyclictest790060-21kworker/2:1+events23:30:482
71646399229611,2282cyclictest729236-21kworker/2:2+events19:50:112
71646399229511,2280cyclictest794938-21kworker/2:2+events23:20:492
71646399229510,2282cyclictest791713-21kworker/2:2+events23:07:542
71646399229510,2281cyclictest735307-21kworker/2:2+events20:08:302
7164639922929,2279cyclictest794938-21kworker/2:2+events23:16:592
71646399229211,2278cyclictest814049-21kworker/2:2+events00:23:332
71646399229211,2278cyclictest786782-21kworker/2:1+events22:50:362
71646399229111,2277cyclictest772063-21kworker/2:0+events22:04:512
71646399228510,2272cyclictest802864-21kworker/2:0+events23:42:112
71646399228411,2270cyclictest746539-21kworker/2:1+events20:43:282
71646399228311,2269cyclictest773694-21kworker/2:2+events22:17:052
71646899228010,2267cyclictest773850-21kworker/3:0+events22:10:133
71646899228010,2267cyclictest773850-21kworker/3:0+events22:10:133
7164549922599,2247cyclictest776908-21kworker/0:0+events22:21:170
71646399225610,2243cyclictest730837-21kworker/2:1+events19:59:192
71646899224111,2227cyclictest801146-21kworker/3:0+events23:38:003
7164689922309,2218cyclictest796242-21kworker/3:2+events23:28:283
71646899222811,2214cyclictest775436-21kworker/3:1+events22:15:123
71646899222511,2211cyclictest770562-21kworker/3:2+events22:28:103
71646899222510,2212cyclictest735399-21kworker/3:0+events20:17:373
7164689922249,2212cyclictest764013-21kworker/3:0+events22:00:153
71646899222411,2210cyclictest770562-21kworker/3:2+events22:32:483
7164689922229,2210cyclictest757968-21kworker/3:2+events21:21:043
71646399222210,2209cyclictest820072-21kworker/2:0+events00:37:162
71646899222111,2207cyclictest724466-21kworker/3:0+events19:36:023
71646899222011,2206cyclictest794313-21kworker/3:1+events23:12:243
71646899221811,2204cyclictest711365-21kworker/3:1+events19:33:473
71646899221711,2203cyclictest764013-21kworker/3:0+events21:42:213
71646899221511,2201cyclictest818706-21kworker/3:0+events00:35:413
71646399221310,2200cyclictest809100-21kworker/2:0+events23:59:232
71645699220912,2194cyclictest743601-21kworker/1:0+events21:01:451
71646899220511,2191cyclictest751427-21kworker/3:1+events21:05:273
71646899220511,2191cyclictest735399-21kworker/3:0+events20:11:333
7164689922039,2191cyclictest807746-21kworker/3:1+events00:07:033
71646899220210,2188cyclictest801146-21kworker/3:0+events23:55:163
71646899220011,2186cyclictest764013-21kworker/3:0+events21:50:483
71646899219911,2185cyclictest807746-21kworker/3:1+events00:21:393
71646399219611,2182cyclictest763038-21kworker/2:1+events22:12:312
71646399219611,2182cyclictest763038-21kworker/2:1+events22:12:312
71645499218811,2175cyclictest778853-21kworker/0:2+events22:55:040
71645499218410,2170cyclictest778853-21kworker/0:2+events22:45:160
7164689921749,2162cyclictest804530-21kworker/3:1+events23:47:583
71645699215711,2143cyclictest765632-21kworker/1:0+events21:51:331
71646899214810,2135cyclictest817082-21kworker/3:2+events00:28:033
7164549921479,2135cyclictest757999-21kworker/0:2+events21:20:180
7164569921469,2133cyclictest743601-21kworker/1:0+events20:45:231
71645499214411,2130cyclictest801086-21kworker/0:2+events23:39:080
7164549921439,2131cyclictest817227-21kworker/0:1+events00:29:350
71645499214311,2129cyclictest820133-21kworker/0:0+events00:37:380
71645499214311,2129cyclictest748265-21kworker/0:2+events20:57:320
71646399214110,2128cyclictest807385-21kworker/2:1+events00:12:042
7164549921419,2129cyclictest778853-21kworker/0:2+events22:26:390
71645499214111,2129cyclictest778853-21kworker/0:2+events22:35:030
71645499214111,2127cyclictest802833-21kworker/0:0+events23:44:090
71646399214011,2126cyclictest770440-21kworker/2:2+events21:58:002
7164549921409,2128cyclictest772032-21kworker/0:0+events22:04:040
71645499214011,2126cyclictest721059-21kworker/0:1+events19:29:110
7164549921399,2127cyclictest724404-21kworker/0:0+events19:36:250
71645699213810,2126cyclictest783302-21kworker/1:0+events23:15:041
71646899213611,2122cyclictest741950-21kworker/3:2+events20:34:443
7164549921359,2123cyclictest730540-21kworker/0:1+events20:02:450
71646399213411,2120cyclictest722709-21kworker/2:2+events19:31:522
71646899213210,2119cyclictest785159-21kworker/3:0+events22:53:143
7164569921319,2119cyclictest770680-21kworker/1:2+events22:06:001
71646399212811,2114cyclictest775303-21kworker/2:0+events22:20:552
71646899212710,2114cyclictest788586-21kworker/3:2+events22:59:323
71645499212611,2113cyclictest796577-21kworker/0:1+events23:26:320
71646399212110,2108cyclictest757759-21kworker/2:1+events21:17:142
71645499211910,2105cyclictest750142-21kworker/0:1+events21:19:310
71646399211610,2103cyclictest783942-21kworker/2:2+events22:46:022
7164639921155,2107cyclictest714582-21kworker/2:2+events19:19:262
71645499211310,2100cyclictest759687-21kworker/0:0+events21:29:480
7164639921129,2099cyclictest763038-21kworker/2:1+events22:07:552
71646399211110,2098cyclictest818471-21kworker/2:1+events00:31:052
71646399211110,2098cyclictest802864-21kworker/2:0+events23:48:212
71646399211110,2098cyclictest761102-21kworker/2:0+events21:36:432
7164639921109,2098cyclictest749774-21kworker/2:2+events21:07:242
71646399210910,2096cyclictest790060-21kworker/2:1+events23:11:152
71646399210910,2096cyclictest778794-21kworker/2:1+events22:37:182
7164639921076,2098cyclictest735307-21kworker/2:2+events20:13:042
7164689921043,2098cyclictest807746-21kworker/3:1+events00:19:463
7164689921028,2091cyclictest797904-21kworker/3:0+events23:22:193
7164689921019,2089cyclictest746774-21kworker/3:2+events20:53:233
7164639921017,2090cyclictest810733-21kworker/2:2+events00:07:272
7164639920989,2086cyclictest799658-21kworker/2:0+events23:29:392
7164639920988,2087cyclictest790060-21kworker/2:1+events23:37:152
71646399209710,2084cyclictest744918-21kworker/2:0+events20:39:392
71645699209710,2084cyclictest783302-21kworker/1:0+events22:38:031
7164639920969,2084cyclictest744918-21kworker/2:0+events20:56:452
71646399209510,2083cyclictest722709-21kworker/2:2+events19:40:402
7164639920933,2087cyclictest735307-21kworker/2:2+events20:18:002
71645499208711,2073cyclictest746571-21kworker/0:0+events20:43:510
71646899207511,2060cyclictest753108-21kworker/3:0+events21:29:023
71646899207510,2062cyclictest801146-21kworker/3:0+events23:41:013
71645499207310,2060cyclictest773633-21kworker/0:1+events22:15:590
7164689920716,2062cyclictest807746-21kworker/3:1+events00:02:263
7164549920644,2057cyclictest794817-21kworker/0:0+events23:16:120
71645499205910,2046cyclictest746571-21kworker/0:0+events20:49:350
7164689920589,2046cyclictest778277-21kworker/3:0+events22:23:353
7164639920559,2043cyclictest716163-21kworker/2:1+events19:22:252
7164639920507,2040cyclictest778794-21kworker/2:1+events22:34:412
7164549920505,2042cyclictest773633-21kworker/0:1+events22:08:180
7164639920489,2036cyclictest768811-21kworker/2:0+events21:52:402
71646399204810,2036cyclictest725942-21kworker/2:0+events20:00:052
7164639920479,2035cyclictest788443-21kworker/2:0+events22:57:392
7164689920449,2032cyclictest733765-21kworker/3:2+events20:06:583
7164689920447,2035cyclictest790210-21kworker/3:0+events23:05:123
7164639920436,2034cyclictest744918-21kworker/2:0+events20:46:552
7164639920435,2035cyclictest725942-21kworker/2:0+events19:46:242
71646899204110,2028cyclictest807746-21kworker/3:1+events00:30:193
71646899204010,2027cyclictest770562-21kworker/3:2+events22:40:423
7164689920377,2027cyclictest753108-21kworker/3:0+events21:19:103
71645499203511,2021cyclictest808987-21kworker/0:2+events00:04:250
7164689920349,2022cyclictest767351-21kworker/3:2+events21:49:143
7164549920347,2024cyclictest737032-21kworker/0:1+events20:15:440
7164689920318,2020cyclictest751427-21kworker/3:1+events21:02:293
7164689920304,2023cyclictest770562-21kworker/3:2+events22:35:463
7164639920299,2018cyclictest790060-21kworker/2:1+events23:03:202
71646899202810,2015cyclictest770562-21kworker/3:2+events21:59:323
7164689920278,2016cyclictest726097-21kworker/3:2+events19:51:193
7164689920269,2014cyclictest746774-21kworker/3:2+events20:56:243
7164689920254,2018cyclictest757968-21kworker/3:2+events21:35:343
71646899202510,2012cyclictest807746-21kworker/3:1+events00:12:273
71646899202410,2011cyclictest726097-21kworker/3:2+events19:47:533
7164689920229,2010cyclictest711365-21kworker/3:1+events19:16:473
7164689920228,2011cyclictest722829-21kworker/3:2+events19:29:573
7164689920215,2013cyclictest806110-21kworker/3:2+events23:50:413
71646899202110,2008cyclictest754707-21kworker/3:2+events21:12:213
71646899202110,2008cyclictest738604-21kworker/3:1+events20:47:393
71646899202110,2008cyclictest729366-21kworker/3:0+events19:55:523
71646899202010,2007cyclictest721235-21kworker/3:0+events19:23:553
7164689920196,2010cyclictest735399-21kworker/3:0+events20:21:493
7164689920182,2012cyclictest761464-21kworker/3:1+events21:34:023
7164689920179,2005cyclictest732141-21kworker/3:1+events20:01:133
7164549920114,2004cyclictest778853-21kworker/0:2+events22:50:590
71646899200810,1996cyclictest796242-21kworker/3:2+events23:31:363
7164549920079,1995cyclictest757999-21kworker/0:2+events21:37:490
71645499200311,1989cyclictest796577-21kworker/0:1+events23:22:430
71645499199311,1979cyclictest801086-21kworker/0:2+events23:49:080
71646899198410,1971cyclictest738604-21kworker/3:1+events20:37:003
7164549919835,1975cyclictest812116-21kworker/0:0+events00:13:560
7164639919807,1970cyclictest714582-21kworker/2:2+events19:14:502
7164549919798,1968cyclictest812116-21kworker/0:0+events00:23:570
7164689919776,1968cyclictest717538-21kworker/3:0+events19:14:273
7164689919749,1962cyclictest726097-21kworker/3:2+events19:44:513
71645499197410,1961cyclictest812116-21kworker/0:0+events00:06:180
71645699197210,1959cyclictest719461-21kworker/1:1+events19:19:491
7164569919704,1963cyclictest715945-21kworker/1:0+events19:30:201
7164549919709,1958cyclictest789942-21kworker/0:0+events23:01:250
7164569919629,1950cyclictest761161-21kworker/1:1+events21:34:481
7164569919619,1949cyclictest748232-21kworker/1:1+events20:54:071
7164569919618,1950cyclictest755728-21kworker/1:1+events21:15:231
7164569919614,1954cyclictest783302-21kworker/1:0+events23:02:371
7164569919607,1950cyclictest729407-21kworker/1:0+events20:04:161
7164569919597,1949cyclictest798851-21kworker/1:0+events23:46:021
7164569919597,1949cyclictest780455-21kworker/1:1+events22:32:021
7164549919597,1949cyclictest791563-21kworker/0:1+events23:11:360
7164549919597,1949cyclictest757999-21kworker/0:2+events21:54:130
71645499195811,1945cyclictest726004-21kworker/0:2+events19:56:150
7164569919577,1947cyclictest812295-21kworker/1:2+events00:08:581
7164549919577,1947cyclictest740291-21kworker/0:1+events20:27:530
7164569919536,1944cyclictest743601-21kworker/1:0+events21:10:281
7164549919539,1941cyclictest750142-21kworker/0:1+events21:09:430
7164549919537,1943cyclictest757999-21kworker/0:2+events21:32:520
7164569919517,1941cyclictest746421-21kworker/1:2+events20:42:211
71645499195110,1938cyclictest796577-21kworker/0:1+events23:31:570
71645499195110,1938cyclictest757999-21kworker/0:2+events21:57:140
7164569919509,1938cyclictest732541-21kworker/1:1+events20:13:501
71645699195010,1937cyclictest818738-21kworker/1:2+events00:33:001
71645499195010,1937cyclictest721059-21kworker/0:1+events19:33:010
7164569919499,1937cyclictest741829-21kworker/1:1+events20:28:591
7164549919496,1940cyclictest730540-21kworker/0:1+events20:05:040
7164549919489,1936cyclictest812116-21kworker/0:0+events00:19:210
7164549919488,1937cyclictest757999-21kworker/0:2+events22:12:540
7164549919488,1937cyclictest757999-21kworker/0:2+events22:12:540
71645499194810,1935cyclictest738668-21kworker/0:2+events20:31:170
71645499194710,1934cyclictest789942-21kworker/0:0+events23:05:590
71645499194710,1934cyclictest717659-21kworker/0:2+events19:20:350
7164549919468,1935cyclictest764189-21kworker/0:0+events21:40:260
7164549919466,1937cyclictest750142-21kworker/0:1+events21:11:580
7164549919449,1932cyclictest815570-21kworker/0:2+events00:31:500
7164549919438,1932cyclictest730540-21kworker/0:1+events19:51:410
71646399194110,1928cyclictest761102-21kworker/2:0+events21:32:282
71645499193310,1920cyclictest783473-21kworker/0:1+events22:44:080
71645699193210,1919cyclictest775093-21kworker/1:1+events22:11:461
71645699193210,1919cyclictest775093-21kworker/1:1+events22:11:461
71645699193210,1919cyclictest743601-21kworker/1:0+events21:21:511
71645499192810,1915cyclictest717659-21kworker/0:2+events19:14:040
7164549919267,1916cyclictest738668-21kworker/0:2+events20:39:160
7164549919128,1901cyclictest746571-21kworker/0:0+events20:52:390
7164569919047,1894cyclictest783433-21kworker/1:3+events22:41:531
7164569919029,1889cyclictest714292-21kworker/1:2+events19:10:121
7164569918989,1886cyclictest798851-21kworker/1:0+events00:29:121
7164569918969,1884cyclictest727632-21kworker/1:2+events19:50:561
71645699189510,1882cyclictest791654-21kworker/1:2+events23:06:201
7164569918949,1882cyclictest808926-21kworker/1:1+events23:56:481
71645699189410,1881cyclictest798851-21kworker/1:0+events23:40:171
7164569918939,1881cyclictest794999-21kworker/1:1+events23:25:251
71645699188710,1874cyclictest735741-21kworker/1:2+events20:30:301
7164569918848,1873cyclictest768841-21kworker/1:1+events21:59:541
7164549918825,1874cyclictest726004-21kworker/0:2+events19:46:010
7164569918816,1872cyclictest759386-21kworker/1:2+events21:29:251
7164569918816,1872cyclictest741829-21kworker/1:1+events20:35:521
7164569918813,1875cyclictest798851-21kworker/1:0+events23:31:111
7164549918818,1870cyclictest781766-21kworker/0:0+events22:34:190
7164569918809,1868cyclictest723250-21kworker/1:1+events19:43:411
7164569918748,1863cyclictest759386-21kworker/1:2+events21:40:501
7164569918179,1805cyclictest801026-21kworker/1:2+events23:35:211
7164549918069,1794cyclictest706386-21kworker/0:0+events19:17:320
71645699169164,4cyclictest783302-21kworker/1:0+pm23:14:541
7160002147111,31sleep20-21swapper/219:07:462
7161752143114,25sleep30-21swapper/319:10:003
7161242142106,30sleep10-21swapper/119:09:221
7158662140107,27sleep00-21swapper/019:06:040
716456994238,4cyclictest777110-21kworker/1:2+pm22:29:541
716456993315,16cyclictest846-10wireplumber21:50:001
716456993225,4cyclictest707-21mta-sts-daemon00:15:191
716456992813,13cyclictest810907-21sh00:05:001
71645699281,25cyclictest0-21swapper/123:20:211
716456992712,14cyclictest0-21swapper/119:20:161
71645699261,15cyclictest763590-21ntpq21:35:171
71645699251,18cyclictest788094-21/usr/sbin/munin22:50:201
71645699250,17cyclictest806619-21diskstats23:50:151
71645699241,20cyclictest790290-21basename23:00:001
71645699241,14cyclictest0-21swapper/122:20:211
716456992411,12cyclictest739086-21/usr/sbin/munin20:20:191
716456992411,11cyclictest777042-21idleruntime-cro22:20:001
71645699240,14cyclictest771009-21taskset22:00:151
71645699240,12cyclictest738007-21ntpq20:15:171
71645699231,15cyclictest754176-21ntpq21:05:181
71645699221,18cyclictest814080-21awk00:15:011
71645699221,16cyclictest728233-21diskstats19:45:121
71645699220,15cyclictest731382-21df_inode19:55:161
71645699210,20cyclictest0-21swapper/120:05:171
71645699210,13cyclictest0-21swapper/119:35:201
71645699201,17cyclictest751748-21cat21:00:001
71645699201,16cyclictest821989-21idleruntime-cro00:40:001
71645699201,10cyclictest786022-21diskstats22:45:181
71645699191,15cyclictest816475-21ntpq00:20:191
71645699180,10cyclictest721903-21diskstats19:25:131
71645499159,4cyclictest753147-21sshd21:02:480
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional