You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-12 - 12:52

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #f, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails
  
Click here to display the system's profile data.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackfslot7s.osadl.org (updated Sat Jul 12, 2025 00:44:18)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
20377919931195,3110cyclictest2053589-21kworker/0:0+events20:40:000
203779999296113,2943cyclictest2048962-21kworker/2:1+events20:10:182
203779999296113,2943cyclictest2048962-21kworker/2:1+events20:10:172
203779599284913,2832cyclictest2050651-21kworker/1:2+events20:05:191
203780199278013,2764cyclictest2140184-21kworker/3:1+events00:35:183
203779599277813,2761cyclictest2052284-21kworker/1:0+events19:58:261
203779599277813,2761cyclictest2052284-21kworker/1:0+events19:58:261
203779999275513,2740cyclictest2068375-21kworker/2:1+events21:10:182
203780199274413,2727cyclictest2101732-21kworker/3:0+events22:45:183
20377959927415,2732cyclictest2109911-21kworker/1:1+events23:15:131
203779599271912,2703cyclictest2071682-21kworker/1:2+events21:05:191
203780199269913,2682cyclictest2081211-21kworker/3:0+events21:29:123
203779599269413,2675cyclictest2082996-21kworker/1:1+events21:35:141
203779199265713,2639cyclictest2121125-21kworker/0:1+events23:45:100
203779999265313,2636cyclictest2130630-21kworker/2:0+events00:20:162
203779599261611,2601cyclictest2093737-21kworker/1:2+events22:02:381
203779599257912,2563cyclictest2134028-21kworker/1:0+events00:10:141
203779199257011,2555cyclictest2095330-21kworker/0:2+events22:10:570
203779599255610,2544cyclictest2079468-21kworker/1:0+events21:20:091
203779599255511,2540cyclictest2082996-21kworker/1:1+events21:31:051
203779599254912,2533cyclictest2120918-21kworker/1:1+events23:30:211
203779599254712,2531cyclictest2071682-21kworker/1:2+events20:58:561
203779599254211,2529cyclictest2088933-21kworker/1:2+events21:50:241
203779599254112,2526cyclictest2079468-21kworker/1:0+events21:27:411
203779599251312,2497cyclictest2042724-21kworker/1:1+events19:26:341
203779599250714,2489cyclictest2116432-21kworker/1:0+events23:35:141
203780199250312,2487cyclictest2092052-21kworker/3:1+events22:00:203
203779599249812,2482cyclictest2092112-21kworker/1:0+events21:59:341
203779599249712,2482cyclictest2070083-21kworker/1:0+events20:50:541
203779599249611,2482cyclictest2109911-21kworker/1:1+events22:53:561
203780199249512,2479cyclictest2079540-21kworker/3:2+events21:20:523
203779199249511,2480cyclictest2053589-21kworker/0:0+events20:01:520
203779599249412,2479cyclictest2047223-21kworker/1:1+events19:39:021
203779599249312,2477cyclictest2042724-21kworker/1:1+events19:24:211
203779199249312,2479cyclictest2085893-21kworker/0:2+events21:56:090
203779599249112,2475cyclictest2068231-21kworker/1:1+events20:43:461
203780199249012,2474cyclictest2050708-21kworker/3:0+events20:22:313
203779599248912,2473cyclictest2141978-21kworker/1:0+events00:36:271
203779599248812,2474cyclictest2124104-21kworker/1:2+events23:41:471
203779599248811,2473cyclictest2109911-21kworker/1:1+events22:58:051
203779599248811,2473cyclictest2047223-21kworker/1:1+events19:40:551
203780199248712,2471cyclictest2087645-21kworker/3:1+events21:49:193
203779599248712,2471cyclictest2047223-21kworker/1:1+events19:48:521
203779599248711,2472cyclictest2109911-21kworker/1:1+events23:10:351
203779599248612,2471cyclictest2050651-21kworker/1:2+events19:54:361
203779199248111,2468cyclictest2045714-21kworker/0:1+events19:45:050
203779599247711,2463cyclictest2082996-21kworker/1:1+events21:42:011
203779599247412,2458cyclictest2056018-21kworker/1:0+events20:15:391
203779599247312,2458cyclictest2082996-21kworker/1:1+events21:46:131
203780199247210,2459cyclictest2029459-21kworker/3:0+events19:31:063
203780199247012,2454cyclictest2101732-21kworker/3:0+events22:43:043
203779599246712,2453cyclictest2070083-21kworker/1:0+events21:00:481
203779599246612,2451cyclictest2056018-21kworker/1:0+events20:11:031
203779599246612,2451cyclictest2056018-21kworker/1:0+events20:11:031
20378019924649,2452cyclictest2049114-21kworker/3:1+events19:47:223
203779599245511,2441cyclictest2060350-21kworker/1:2+events20:22:531
203779199245312,2437cyclictest2064972-21kworker/0:2+events20:33:110
203779599245112,2436cyclictest2044381-21kworker/1:2+events19:32:381
203779199245012,2435cyclictest2101942-21kworker/0:1+events22:32:260
203779599244910,2435cyclictest2093737-21kworker/1:2+events22:21:351
203779599243912,2423cyclictest2093737-21kworker/1:2+events22:12:061
203780199243612,2420cyclictest2050708-21kworker/3:0+events20:05:443
203779199243512,2420cyclictest2089127-21kworker/0:0+events21:53:280
203780199243412,2418cyclictest2134410-21kworker/3:0+events00:15:193
203779999242412,2409cyclictest2048962-21kworker/2:1+events20:19:062
203780199242112,2405cyclictest2076344-21kworker/3:1+events21:12:123
203780199242110,2407cyclictest2061793-21kworker/3:1+events20:38:053
203779199241312,2398cyclictest2078201-21kworker/0:1+events21:30:410
203779999241012,2395cyclictest2040928-21kworker/2:2+events19:31:512
203779599240611,2391cyclictest2138663-21kworker/1:0+events00:25:191
203780199239911,2385cyclictest2061793-21kworker/3:1+events20:43:233
203779599239912,2383cyclictest2113098-21kworker/1:0+events23:07:141
203779199239711,2382cyclictest2060442-21kworker/0:2+events20:20:150
203780199239511,2380cyclictest2113221-21kworker/3:1+events23:16:193
203780199239312,2378cyclictest2061793-21kworker/3:1+events20:34:433
203780199239212,2376cyclictest2130600-21kworker/3:0+events00:03:283
203780199239211,2379cyclictest2086030-21kworker/3:0+events21:55:013
203780199239112,2376cyclictest2111596-21kworker/3:2+events23:04:583
20378019923898,2378cyclictest2076344-21kworker/3:1+events21:09:543
203780199238910,2376cyclictest2071530-21kworker/3:2+events20:51:383
203780199238910,2376cyclictest2061793-21kworker/3:1+events20:28:153
203780199238812,2373cyclictest2079540-21kworker/3:2+events21:17:503
203780199238812,2373cyclictest2047469-21kworker/3:0+events19:41:183
203780199238711,2373cyclictest2140184-21kworker/3:1+events00:29:553
203780199238610,2373cyclictest2082816-21kworker/3:1+events21:38:173
203780199238512,2369cyclictest2029459-21kworker/3:0+events19:17:123
203779199238511,2370cyclictest2078201-21kworker/0:1+events21:21:120
20378019923849,2371cyclictest2082816-21kworker/3:1+events21:33:433
203780199238110,2368cyclictest2055461-21kworker/3:1+events20:03:243
203779599237910,2365cyclictest2037365-21kworker/1:2+events19:13:051
203780199237311,2358cyclictest2049114-21kworker/3:1+events19:54:133
203780199237211,2357cyclictest2093678-21kworker/3:2+events22:05:153
20377999923719,2359cyclictest2066723-21kworker/2:0+events20:44:312
203779999237111,2356cyclictest2063493-21kworker/2:1+events20:37:192
203780199236711,2353cyclictest2124123-21kworker/3:2+events23:39:513
203779199234812,2332cyclictest2101942-21kworker/0:1+events22:37:010
203779999234712,2331cyclictest2045982-21kworker/2:0+events19:42:262
203779199234312,2327cyclictest2089127-21kworker/0:0+events21:49:400
203779999234112,2325cyclictest2071620-21kworker/2:2+events20:58:092
203779999234012,2325cyclictest2068375-21kworker/2:1+events20:48:392
203779599233411,2319cyclictest2126033-21kworker/1:1+events00:01:571
203779599233212,2316cyclictest2066483-21kworker/1:0+events20:37:421
203779999233012,2314cyclictest2048962-21kworker/2:1+events20:07:372
20377999923299,2316cyclictest2065078-21kworker/2:2+events20:33:582
203779999232812,2314cyclictest2125700-21kworker/2:3+events23:50:352
203779999232812,2313cyclictest2040928-21kworker/2:2+events19:17:572
203779199232810,2315cyclictest2041135-21kworker/0:2+events19:31:300
203779199232712,2311cyclictest2137306-21kworker/0:1+events00:21:030
203779199232712,2311cyclictest2058833-21kworker/0:1+events20:15:160
203779999232612,2311cyclictest2048962-21kworker/2:1+events19:47:442
203779999232612,2310cyclictest2140178-21kworker/2:2+events00:37:132
203779199232611,2312cyclictest2109733-21kworker/0:1+events22:52:040
203779199232610,2312cyclictest2130539-21kworker/0:0+events00:02:200
203779999232512,2309cyclictest2042635-21kworker/2:0+events19:26:572
203779199232512,2310cyclictest2141795-21kworker/0:2+events00:32:160
203779199232512,2310cyclictest2084142-21kworker/0:0+events21:42:240
203779199232512,2309cyclictest2117874-21kworker/0:0+events23:22:000
203779199232411,2310cyclictest2050833-21kworker/0:2+events19:58:500
203779199232411,2310cyclictest2050833-21kworker/0:2+events19:58:490
203779999232311,2308cyclictest2076448-21kworker/2:2+events21:09:322
203779999232310,2310cyclictest2052161-21kworker/2:0+events19:53:492
203779199232212,2306cyclictest2104808-21kworker/0:2+events22:44:340
203779199232212,2306cyclictest2071383-21kworker/0:1+events21:08:000
203779999232112,2306cyclictest2071620-21kworker/2:2+events20:53:342
203779199232111,2307cyclictest2048994-21kworker/0:0+events19:44:190
203779999232012,2305cyclictest2060231-21kworker/2:0+events20:27:522
203779999232010,2306cyclictest2068375-21kworker/2:1+events21:01:122
203779999231912,2303cyclictest2140178-21kworker/2:2+events00:34:102
203779199231912,2303cyclictest2138965-21kworker/0:0+events00:26:500
203779999231812,2303cyclictest2060231-21kworker/2:0+events20:22:072
203779999231812,2302cyclictest2096966-21kworker/2:1+events23:04:112
203779999231812,2302cyclictest2089234-21kworker/2:2+events21:51:342
203779999231712,2301cyclictest2084289-21kworker/2:2+events21:34:512
203779999231710,2303cyclictest2113158-21kworker/2:0+events23:23:342
203779199231712,2301cyclictest2053589-21kworker/0:0+events20:48:170
203779199231711,2302cyclictest2085893-21kworker/0:2+events22:00:440
203779999231510,2302cyclictest2079649-21kworker/2:1+events21:19:452
203779999231411,2300cyclictest2108199-21kworker/2:0+events22:52:272
203779999231312,2298cyclictest2125700-21kworker/2:3+events23:56:332
203779199231211,2297cyclictest2133927-21kworker/0:1+events00:09:060
203779599231012,2295cyclictest2120918-21kworker/1:1+events23:28:021
203780199230512,2289cyclictest2111596-21kworker/3:2+events23:09:523
203779199230211,2287cyclictest2053589-21kworker/0:0+events20:28:380
203779199230111,2287cyclictest2111330-21kworker/0:0+events23:07:590
203779599229712,2281cyclictest2058594-21kworker/1:1+events20:34:201
203779199229611,2282cyclictest2093704-21kworker/0:0+events22:27:230
203780199228110,2268cyclictest2050708-21kworker/3:0+events20:14:303
203780199228110,2268cyclictest2050708-21kworker/3:0+events20:14:293
203780199227711,2263cyclictest2113221-21kworker/3:1+events23:25:493
203779999227611,2261cyclictest2052161-21kworker/2:0+events19:56:542
203779999227611,2261cyclictest2052161-21kworker/2:0+events19:56:542
203779599227411,2260cyclictest2130479-21kworker/1:2+events23:58:511
203779599227211,2258cyclictest2100365-21kworker/1:1+events22:37:241
203779599227112,2255cyclictest2075098-21kworker/1:1+events21:11:291
203779599227010,2256cyclictest2102896-21kworker/1:2+events23:01:291
203779599227010,2256cyclictest2102896-21kworker/1:2+events22:42:191
203779599226710,2254cyclictest2041104-21kworker/1:0+events19:19:281
203779599226710,2253cyclictest2082996-21kworker/1:1+events22:06:441
203779599226611,2252cyclictest2126033-21kworker/1:1+events00:08:211
203779599226610,2253cyclictest2102896-21kworker/1:2+events22:31:171
203779599226610,2253cyclictest2058594-21kworker/1:1+events20:29:431
20377959922659,2253cyclictest2102896-21kworker/1:2+events22:48:201
203779599226511,2250cyclictest2116432-21kworker/1:0+events23:23:111
203779599226413,2248cyclictest2126033-21kworker/1:1+events23:51:221
203779999226311,2249cyclictest2140178-21kworker/2:2+events00:27:132
203779599226111,2246cyclictest2135699-21kworker/1:2+events00:33:481
203779599225610,2242cyclictest2070083-21kworker/1:0+events20:49:031
203779599225511,2241cyclictest2096903-21kworker/1:0+events22:27:471
20378019922438,2232cyclictest2101732-21kworker/3:0+events22:53:353
20377919922337,2222cyclictest2117874-21kworker/0:0+events23:27:410
203779199223310,2220cyclictest2068345-21kworker/0:2+events20:44:540
20377959922328,2222cyclictest2096903-21kworker/1:0+events22:15:061
203779999222712,2212cyclictest2101742-21kworker/2:0+events22:32:032
203779199222311,2209cyclictest2039068-21kworker/0:1+events19:18:400
20377999922209,2207cyclictest2101742-21kworker/2:0+events22:29:432
203779199221812,2202cyclictest2109733-21kworker/0:1+events22:58:490
203779999221711,2203cyclictest2079649-21kworker/2:1+events21:29:342
203780199221611,2201cyclictest2130600-21kworker/3:0+events23:59:593
203779999220511,2190cyclictest2125700-21kworker/2:3+events00:04:132
203780199220411,2189cyclictest2132510-21kworker/3:2+events00:05:443
203780199220210,2188cyclictest2096889-21kworker/3:2+events22:24:183
203779199220111,2187cyclictest2116274-21kworker/0:2+events23:14:270
203779999219811,2183cyclictest2113158-21kworker/2:0+events23:26:342
20378019921968,2184cyclictest2086030-21kworker/3:0+events21:43:103
203780199219311,2179cyclictest2103322-21kworker/3:2+events22:39:403
203780199219211,2178cyclictest2029459-21kworker/3:0+events19:23:133
20378019921909,2177cyclictest2111596-21kworker/3:2+events22:59:593
203780199219011,2176cyclictest2096889-21kworker/3:2+events22:15:533
203780199219011,2176cyclictest2069929-21kworker/3:0+events20:46:263
20378019921888,2177cyclictest2074743-21kworker/3:2+events21:02:413
20378019921869,2174cyclictest2101732-21kworker/3:0+events22:33:353
20378019921869,2173cyclictest2044118-21kworker/3:2+events19:37:073
203780199218611,2171cyclictest2050708-21kworker/3:0+events19:57:183
203780199218611,2171cyclictest2050708-21kworker/3:0+events19:57:173
20378019921859,2173cyclictest2050708-21kworker/3:0+events20:19:523
20378019921858,2173cyclictest2073131-21kworker/3:1+events20:59:413
203780199218511,2171cyclictest2120887-21kworker/3:0+events23:42:103
203780199218510,2171cyclictest2029459-21kworker/3:0+events19:29:593
203780199218311,2168cyclictest2086030-21kworker/3:0+events21:50:493
203780199218211,2169cyclictest2113221-21kworker/3:1+events23:20:063
203780199218211,2168cyclictest2113221-21kworker/3:1+events23:10:583
203779999218211,2167cyclictest2116180-21kworker/2:2+events23:17:052
203780199218110,2167cyclictest2134410-21kworker/3:0+events00:12:353
20377999921809,2168cyclictest2093673-21kworker/2:2+events22:10:352
203780199217911,2165cyclictest2134410-21kworker/3:0+events00:23:243
203779999217710,2163cyclictest2076448-21kworker/2:2+events21:20:302
203780199217411,2160cyclictest2134410-21kworker/3:0+events00:31:063
203779999217311,2159cyclictest2133814-21kworker/2:1+events00:07:582
203779999217010,2157cyclictest2135484-21kworker/2:2+events00:15:422
203780199216811,2153cyclictest2127406-21kworker/3:1+events23:53:143
203779999216310,2150cyclictest2130630-21kworker/2:0+events00:11:482
203779199216311,2149cyclictest2143398-21kworker/0:0+events00:37:360
203779999213110,2117cyclictest2082574-21kworker/2:0+events21:44:182
20377919921319,2118cyclictest2123970-21kworker/0:2+events23:37:110
203779199213112,2116cyclictest2041135-21kworker/0:2+events19:26:140
203779999213011,2116cyclictest2098597-21kworker/2:2+events22:19:382
203779999213011,2115cyclictest2040928-21kworker/2:2+events19:37:552
203779999212811,2113cyclictest2098597-21kworker/2:2+events22:22:452
203779199212611,2113cyclictest2074877-21kworker/0:0+events21:15:110
203779199212511,2111cyclictest2093704-21kworker/0:0+events22:07:060
203779999212411,2111cyclictest2082574-21kworker/2:0+events21:45:062
203780199212211,2107cyclictest2092052-21kworker/3:1+events22:14:203
20377999921219,2108cyclictest2095269-21kworker/2:0+events22:07:522
203779999212111,2106cyclictest2093673-21kworker/2:2+events22:02:142
203779999212110,2108cyclictest2042635-21kworker/2:0+events19:23:592
203779199212110,2107cyclictest2071383-21kworker/0:1+events20:55:290
203779999212011,2105cyclictest2082574-21kworker/2:0+events21:36:242
203779199212011,2106cyclictest2084142-21kworker/0:0+events21:37:550
203779199212010,2106cyclictest2045714-21kworker/0:1+events19:37:310
20378019921199,2106cyclictest2120887-21kworker/3:0+events23:46:203
20377999921198,2107cyclictest2103477-21kworker/2:2+events22:38:342
203779999211911,2105cyclictest2127283-21kworker/2:0+events23:49:022
203779199211913,2102cyclictest2096931-21kworker/0:1+events22:16:150
20377919921189,2106cyclictest2057172-21kworker/0:2+events20:12:120
20377919921189,2106cyclictest2057172-21kworker/0:2+events20:12:120
20377919921188,2107cyclictest2045714-21kworker/0:1+events19:51:550
203779199211811,2103cyclictest2093704-21kworker/0:0+events22:21:130
203779199211710,2104cyclictest2041135-21kworker/0:2+events19:21:180
203779999211611,2102cyclictest2122761-21kworker/2:2+events23:41:232
203779199211611,2102cyclictest2121125-21kworker/0:1+events23:57:180
203779199211611,2102cyclictest2121125-21kworker/0:1+events23:53:360
203779999211511,2101cyclictest2096966-21kworker/2:1+events23:08:212
20377919921159,2103cyclictest2068345-21kworker/0:2+events21:00:250
203779199211511,2101cyclictest2111330-21kworker/0:0+events23:01:540
203779999211411,2099cyclictest2048962-21kworker/2:1+events20:01:292
20377919921148,2103cyclictest2127314-21kworker/0:2+events00:17:140
203779199211411,2100cyclictest2125579-21kworker/0:0+events23:44:050
203779199211311,2099cyclictest2116274-21kworker/0:2+events23:18:580
203779599210910,2096cyclictest2126033-21kworker/1:1+events00:17:361
203779999210811,2093cyclictest2113158-21kworker/2:0+events23:12:322
203779199210811,2094cyclictest2037368-21kworker/0:0+events19:14:580
203779199210711,2092cyclictest2127314-21kworker/0:2+events00:14:090
203779999210511,2090cyclictest2090861-21kworker/2:1+events21:58:482
203779599210510,2091cyclictest2124104-21kworker/1:2+events23:48:171
203779199210511,2091cyclictest2117874-21kworker/0:0+events23:32:590
203779599210310,2090cyclictest2053891-21kworker/1:1+events20:04:341
203779199210310,2089cyclictest2068345-21kworker/0:2+events20:50:320
203779599210210,2088cyclictest2135699-21kworker/1:2+events00:21:501
20377959921015,2092cyclictest2075098-21kworker/1:1+events21:16:421
203779199209310,2081cyclictest2081028-21kworker/0:2+events21:25:030
20378019920645,2056cyclictest2120887-21kworker/3:0+events23:31:303
203779999206410,2051cyclictest2037512-21kworker/2:1+events19:12:192
203779999204710,2033cyclictest2096966-21kworker/2:1+events22:49:492
20378019920297,2019cyclictest2029459-21kworker/3:0+events19:14:343
203780199202110,2009cyclictest2100129-21kworker/3:1+events22:25:033
203779199197610,1963cyclictest2076509-21kworker/0:2+events21:11:510
20377999919597,1950cyclictest2103477-21kworker/2:2+events22:40:032
203779999195810,1945cyclictest2096966-21kworker/2:1+events22:58:262
20377919919538,1942cyclictest2106458-21kworker/0:0+events22:46:280
203779199195210,1941cyclictest2053589-21kworker/0:0+events20:08:010
20377999919496,1940cyclictest2121094-21kworker/2:1+events23:31:522
20373102155112,24sleep00-21swapper/019:06:170
20373492144110,29sleep20-21swapper/219:06:462
2037308214395,22sleep30-21swapper/319:06:153
20375372142112,25sleep10-21swapper/119:09:111
203779999261,7cyclictest2124343-21tr23:40:002
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional