You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-07 - 12:13

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #f, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails
  
Click here to display the system's profile data.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackfslot7s.osadl.org (updated Tue Jan 06, 2026 00:44:17)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
324814099579027,5759cyclictest3239495-21kworker/2:0+events19:11:092
324814399578927,5758cyclictest3238141-21kworker/3:2+events19:11:323
324813399576328,5730cyclictest3246218-21kworker/0:1+events19:12:160
32481369927959,2781cyclictest3276860-21kworker/1:2+events20:40:161
324813699269413,2677cyclictest3260952-21kworker/1:2+events19:50:141
324813699265813,2641cyclictest3342843-21kworker/1:0+events00:15:181
324814099263711,2624cyclictest3247870-21kworker/2:2+events19:15:202
324814099259712,2581cyclictest3299260-21kworker/2:1+events22:00:172
324813399254911,2534cyclictest3313620-21kworker/0:2+events22:40:000
324813399252612,2512cyclictest3320407-21kworker/0:1+events23:25:180
324813399248711,2472cyclictest3278548-21kworker/0:0+events21:15:180
324814099230210,2288cyclictest3267291-21kworker/2:2+events20:14:162
324813399229911,2284cyclictest3262519-21kworker/0:0+events20:00:530
32481409922977,2287cyclictest3318164-21kworker/2:2+events22:55:342
324813399226911,2254cyclictest3260895-21kworker/0:2+events19:53:120
324813399226711,2253cyclictest3308822-21kworker/0:0+events22:23:140
324813399222911,2214cyclictest3262519-21kworker/0:0+events20:07:250
32481339922228,2211cyclictest3320407-21kworker/0:1+events23:23:430
32481409922209,2208cyclictest3326402-21kworker/2:1+events23:13:082
32481369921808,2169cyclictest3244339-21kworker/1:0+events19:13:251
324814099215411,2140cyclictest3267291-21kworker/2:2+events20:22:192
32481439921449,2132cyclictest3259408-21kworker/3:1+events19:46:063
324814099212110,2107cyclictest3256070-21kworker/2:2+events19:46:302
324814099212010,2107cyclictest3318164-21kworker/2:2+events22:51:262
324814099211910,2106cyclictest3319819-21kworker/2:0+events23:05:322
324814099211910,2105cyclictest3282887-21kworker/2:1+events20:58:282
324814099211910,2105cyclictest3282887-21kworker/2:1+events20:58:282
324814399211510,2102cyclictest3347389-21kworker/3:1+events00:36:313
324813399211510,2101cyclictest3302351-21kworker/0:2+events22:08:130
324814099211410,2102cyclictest3268924-21kworker/2:1+events20:15:032
324814099211410,2101cyclictest3331337-21kworker/2:2+events23:46:042
32481409921138,2101cyclictest3314813-21kworker/2:1+events22:44:362
324813399211310,2100cyclictest3337797-21kworker/0:1+events23:52:110
32481409921129,2100cyclictest3261138-21kworker/2:1+events20:05:272
32481409921125,2104cyclictest3332973-21kworker/2:1+events23:43:512
32481409921119,2099cyclictest3253162-21kworker/2:1+events19:37:462
32481339921118,2100cyclictest3324968-21kworker/0:0+events23:17:160
324813399211110,2098cyclictest3340724-21kworker/0:2+events00:07:160
324813399211110,2098cyclictest3330187-21kworker/0:0+events23:31:170
32481409921103,2104cyclictest3329619-21kworker/2:1+events23:23:202
324814099211010,2097cyclictest3297653-21kworker/2:0+events21:44:352
32481369921106,2101cyclictest3315022-21kworker/1:2+events22:39:371
32481339921108,2099cyclictest3330187-21kworker/0:0+events00:01:120
32481339921105,2102cyclictest3324968-21kworker/0:0+events23:12:210
324813399211010,2097cyclictest3286763-21kworker/0:2+events21:24:010
32481409921098,2097cyclictest3308517-21kworker/2:0+events22:45:452
324813699210810,2095cyclictest3348898-21kworker/1:2+events00:21:471
32481339921047,2094cyclictest3262519-21kworker/0:0+events20:11:080
324814099210310,2091cyclictest3319819-21kworker/2:0+events23:00:072
324814099210310,2091cyclictest3319819-21kworker/2:0+events23:00:072
32481339921039,2091cyclictest3308822-21kworker/0:0+events22:32:020
32481369921009,2088cyclictest3291189-21kworker/1:0+events21:22:541
32481339921008,2089cyclictest3297474-21kworker/0:1+events21:42:390
324813699209910,2085cyclictest3337556-21kworker/1:1+events23:47:131
32481339920999,2087cyclictest3253430-21kworker/0:0+events19:41:550
32481409920979,2085cyclictest3286444-21kworker/2:2+events21:33:092
32481409920975,2089cyclictest3282887-21kworker/2:1+events21:10:212
32481409920974,2090cyclictest3331337-21kworker/2:2+events00:02:002
324814399209510,2082cyclictest3282980-21kworker/3:1+events21:14:573
324814099209510,2083cyclictest3290091-21kworker/2:1+events21:25:082
324814099209411,2080cyclictest3344269-21kworker/2:1+events00:09:102
32481409920928,2081cyclictest3286444-21kworker/2:2+events21:08:032
32481339920908,2079cyclictest3282919-21kworker/0:2+events21:00:240
32481339920888,2077cyclictest3278548-21kworker/0:0+events20:50:520
32481339920878,2076cyclictest3344332-21kworker/0:0+events00:28:130
32481409920864,2079cyclictest3256070-21kworker/2:2+events19:44:342
32481409920844,2077cyclictest3278383-21kworker/2:1+events20:44:242
32481339920816,2072cyclictest3286763-21kworker/0:2+events21:36:160
324813399208110,2067cyclictest3262519-21kworker/0:0+events20:16:330
32481409920787,2068cyclictest3306911-21kworker/2:2+events22:18:372
32481409920768,2064cyclictest3301003-21kworker/2:2+events21:57:122
32481339920768,2065cyclictest3351163-21kworker/0:1+events00:36:070
32481339920767,2066cyclictest3330187-21kworker/0:0+events23:56:360
324813699207510,2062cyclictest3249589-21kworker/1:2+events19:25:591
324813399207410,2061cyclictest3270289-21kworker/0:2+events20:21:560
32481339920739,2061cyclictest3336197-21kworker/0:2+events23:48:430
32481339920727,2061cyclictest3291067-21kworker/0:1+events21:25:330
32481409920719,2059cyclictest3331337-21kworker/2:2+events23:39:432
32481339920717,2061cyclictest3342670-21kworker/0:1+events00:12:590
324813399207110,2058cyclictest3316467-21kworker/0:0+events22:48:240
32481339920689,2056cyclictest3286763-21kworker/0:2+events21:30:510
32481409920679,2054cyclictest3340692-21kworker/2:1+events23:58:322
324813699206710,2054cyclictest3331486-21kworker/1:2+events23:37:501
324813399206610,2053cyclictest3253430-21kworker/0:0+events19:38:080
324814099206410,2051cyclictest3308517-21kworker/2:0+events22:33:582
32481409920637,2053cyclictest3299260-21kworker/2:1+events21:50:202
324814399205710,2044cyclictest3308790-21kworker/3:0+events22:21:403
32481339920568,2045cyclictest3282919-21kworker/0:2+events20:57:200
32481339920568,2045cyclictest3282919-21kworker/0:2+events20:57:190
32481339920555,2047cyclictest3311969-21kworker/0:1+events22:44:590
324813399205510,2042cyclictest3334400-21kworker/0:1+events23:40:510
32481369920549,2042cyclictest3308546-21kworker/1:0+events22:43:251
32481339920543,2048cyclictest3246218-21kworker/0:1+events19:24:040
324814099205110,2038cyclictest3350540-21kworker/2:0+events00:37:402
324813399205111,2037cyclictest3273541-21kworker/0:0+events20:36:530
324813399205010,2038cyclictest3332792-21kworker/0:2+events23:35:090
32481409920497,2039cyclictest3331337-21kworker/2:2+events00:10:422
32481369920489,2036cyclictest3331486-21kworker/1:2+events23:42:421
32481409920479,2035cyclictest3286444-21kworker/2:2+events21:35:292
32481339920478,2036cyclictest3295854-21kworker/0:0+events21:52:150
324814099204610,2033cyclictest3286444-21kworker/2:2+events21:15:412
324813699204610,2033cyclictest3272037-21kworker/1:0+events20:25:401
32481409920458,2034cyclictest3290091-21kworker/2:1+events21:21:452
32481409920457,2035cyclictest3280014-21kworker/2:2+events20:53:322
324814099204510,2032cyclictest3308517-21kworker/2:0+events22:35:492
32481369920459,2033cyclictest3328141-21kworker/1:0+events23:21:281
32481339920453,2038cyclictest3323040-21kworker/0:2+events23:04:460
32481339920453,2038cyclictest3323040-21kworker/0:2+events23:04:460
32481409920449,2032cyclictest3239495-21kworker/2:0+events19:27:292
32481409920449,2031cyclictest3256070-21kworker/2:2+events19:52:302
32481339920449,2032cyclictest3262519-21kworker/0:0+events19:58:310
32481409920434,2036cyclictest3339102-21kworker/2:0+events23:54:452
324814099204310,2030cyclictest3299260-21kworker/2:1+events22:08:372
32481369920438,2032cyclictest3276860-21kworker/1:2+events21:00:471
32481369920424,2035cyclictest3334671-21kworker/1:0+events23:54:231
32481339920429,2030cyclictest3253430-21kworker/0:0+events19:30:310
32481339920427,2032cyclictest3302351-21kworker/0:2+events22:03:400
32481339920419,2029cyclictest3278548-21kworker/0:0+events20:46:170
32481339920418,2030cyclictest3351163-21kworker/0:1+events00:31:590
32481409920409,2028cyclictest3247870-21kworker/2:2+events19:23:392
32481339920393,2033cyclictest3270289-21kworker/0:2+events20:26:230
32481439920389,2026cyclictest3261011-21kworker/3:0+events19:58:563
32481339920384,2031cyclictest3299080-21kworker/0:2+events21:48:030
32481439920379,2025cyclictest3262713-21kworker/3:2+events20:00:303
32481339920377,2027cyclictest3307211-21kworker/0:2+events22:15:560
32481409920364,2029cyclictest3278383-21kworker/2:1+events20:45:522
32481409920359,2023cyclictest3308517-21kworker/2:0+events22:24:012
32481409920358,2024cyclictest3319819-21kworker/2:0+events23:30:562
32481369920359,2023cyclictest3292850-21kworker/1:1+events21:32:001
324813699203410,2021cyclictest3350570-21kworker/1:0+events00:29:201
32481369920338,2022cyclictest3276860-21kworker/1:2+events20:55:261
32481369920338,2022cyclictest3276860-21kworker/1:2+events20:55:261
32481369920338,2021cyclictest3316630-21kworker/1:1+events22:51:031
32481339920337,2023cyclictest3318191-21kworker/0:2+events22:52:550
32481409920329,2020cyclictest3267291-21kworker/2:2+events20:37:412
32481369920329,2020cyclictest3328141-21kworker/1:0+events23:18:011
32481369920329,2020cyclictest3308546-21kworker/1:0+events22:32:481
32481409920313,2024cyclictest3256070-21kworker/2:2+events19:32:282
324814099203110,2018cyclictest3308517-21kworker/2:0+events22:27:532
32481369920319,2019cyclictest3318410-21kworker/1:2+events23:08:351
32481369920319,2019cyclictest3294489-21kworker/1:2+events21:35:061
324813399203110,2018cyclictest3345962-21kworker/0:2+events00:19:070
32481409920309,2018cyclictest3346024-21kworker/2:0+events00:17:342
32481369920309,2018cyclictest3350570-21kworker/1:0+events00:32:211
32481339920303,2024cyclictest3286763-21kworker/0:2+events21:10:430
32481369920296,2020cyclictest3329858-21kworker/1:1+events23:29:471
32481369920294,2022cyclictest3308546-21kworker/1:0+events22:46:081
32481369920294,2022cyclictest3302226-21kworker/1:2+events21:57:591
324813699202910,2016cyclictest3331486-21kworker/1:2+events23:31:411
32481409920289,2016cyclictest3297653-21kworker/2:0+events21:47:402
32481369920275,2019cyclictest3275260-21kworker/1:0+events20:38:031
32481369920272,2022cyclictest3267112-21kworker/1:0+events20:09:191
32481369920259,2013cyclictest3294489-21kworker/1:2+events21:48:481
32481369920254,2018cyclictest3270384-21kworker/1:2+events20:21:101
32481339920259,2014cyclictest3249847-21kworker/0:3+events19:19:080
32481409920242,2019cyclictest3319819-21kworker/2:0+events23:15:482
32481339920237,2013cyclictest3316467-21kworker/0:0+events22:55:110
32481369920096,2000cyclictest3292850-21kworker/1:1+events22:05:571
32481369920077,1997cyclictest3273631-21kworker/1:1+events20:34:161
32481369920075,1999cyclictest3321444-21kworker/1:0+events22:59:201
32481339920079,1995cyclictest3303973-21kworker/0:0+events22:11:190
32481339920068,1995cyclictest3278548-21kworker/0:0+events21:06:310
32481339920029,1990cyclictest3260895-21kworker/0:2+events19:48:440
32481409919998,1988cyclictest3268924-21kworker/2:1+events20:27:522
32481369919999,1987cyclictest3254437-21kworker/1:0+events19:43:251
32481339919993,1993cyclictest3320407-21kworker/0:1+events23:07:260
32481339919993,1993cyclictest3249847-21kworker/0:3+events19:29:010
32481409919949,1982cyclictest3331337-21kworker/2:2+events23:29:232
32481409919948,1983cyclictest3267291-21kworker/2:2+events20:30:322
32481369919944,1987cyclictest3321444-21kworker/1:0+events23:00:531
32481369919944,1987cyclictest3321444-21kworker/1:0+events23:00:531
32481439919908,1981cyclictest3323194-21kworker/3:0+events23:15:013
32481439919889,1976cyclictest3286595-21kworker/3:2+events21:17:353
32481369919879,1975cyclictest3342843-21kworker/1:0+events00:08:031
32481369919879,1975cyclictest3305666-21kworker/1:2+events22:15:101
32481339919828,1971cyclictest3300687-21kworker/0:1+events21:58:220
32481369919818,1970cyclictest3334671-21kworker/1:0+events23:55:511
32481339919789,1966cyclictest3275143-21kworker/0:2+events20:42:310
32481439919775,1969cyclictest3251657-21kworker/3:2+events19:28:373
32481409919779,1965cyclictest3347747-21kworker/2:1+events00:22:072
32481409919759,1965cyclictest3256070-21kworker/2:2+events19:55:102
32481439919747,1964cyclictest3347389-21kworker/3:1+events00:24:473
32481409919706,1961cyclictest3282887-21kworker/2:1+events21:01:102
324813699196910,1956cyclictest3283070-21kworker/1:1+events21:13:251
32481409919689,1958cyclictest3350540-21kworker/2:0+events00:30:062
32481369919669,1954cyclictest3268710-21kworker/1:1+events20:17:451
324814399196510,1951cyclictest3273425-21kworker/3:0+events20:54:403
32481369919649,1952cyclictest3249589-21kworker/1:2+events19:19:531
32481339919647,1954cyclictest3275143-21kworker/0:2+events20:34:370
32481369919638,1952cyclictest3276860-21kworker/1:2+events20:52:221
324814399196210,1949cyclictest3286595-21kworker/3:2+events21:09:593
32481369919629,1950cyclictest3254437-21kworker/1:0+events19:38:541
32481369919618,1950cyclictest3254437-21kworker/1:0+events19:31:181
32481369919614,1953cyclictest3350570-21kworker/1:0+events00:35:221
324814399196010,1947cyclictest3302287-21kworker/3:2+events22:03:173
32481439919599,1947cyclictest3349045-21kworker/3:0+events00:26:203
32481439919598,1948cyclictest3247737-21kworker/3:0+events19:20:173
32481439919593,1953cyclictest3294549-21kworker/3:1+events21:36:373
32481369919598,1948cyclictest3267112-21kworker/1:0+events20:13:291
324814399195810,1945cyclictest3313273-21kworker/3:0+events22:41:093
32481369919557,1945cyclictest3259293-21kworker/1:1+events19:56:151
32481369919549,1942cyclictest3344452-21kworker/1:2+events00:12:361
32481439919539,1940cyclictest3325174-21kworker/3:2+events23:11:123
32481369919517,1941cyclictest3305666-21kworker/1:2+events22:11:421
32481439919509,1938cyclictest3313273-21kworker/3:0+events22:47:183
324814399195010,1937cyclictest3305574-21kworker/3:1+events22:31:173
324814399195010,1937cyclictest3273425-21kworker/3:0+events20:44:013
32481439919499,1936cyclictest3269665-21kworker/3:1+events20:16:133
32481409919489,1937cyclictest3256070-21kworker/2:2+events20:00:062
32481369919479,1935cyclictest3326509-21kworker/1:1+events23:12:451
324813699194710,1934cyclictest3276860-21kworker/1:2+events20:49:431
32481439919466,1937cyclictest3313273-21kworker/3:0+events22:56:413
32481439919455,1937cyclictest3271829-21kworker/3:2+events20:29:023
32481409919448,1933cyclictest3347747-21kworker/2:1+events00:26:432
32481439919434,1936cyclictest3305574-21kworker/3:1+events22:13:593
32481369919439,1931cyclictest3259293-21kworker/1:1+events20:01:151
324814399194210,1929cyclictest3305574-21kworker/3:1+events22:25:333
324814399194210,1929cyclictest3286595-21kworker/3:2+events21:41:083
32481409919425,1934cyclictest3306911-21kworker/2:2+events22:12:062
32481369919399,1927cyclictest3292850-21kworker/1:1+events22:26:421
32481339919386,1929cyclictest3347690-21kworker/0:1+events00:24:030
32481439919359,1925cyclictest3265303-21kworker/3:1+events20:05:043
32481369919279,1915cyclictest3292850-21kworker/1:1+events21:28:571
324814399192310,1910cyclictest3331278-21kworker/3:0+events23:37:053
32481369919218,1910cyclictest3292850-21kworker/1:1+events22:04:491
32481439919192,1914cyclictest3286595-21kworker/3:2+events21:31:363
324814399191710,1904cyclictest3261011-21kworker/3:0+events19:52:073
32481439919158,1904cyclictest3273425-21kworker/3:0+events20:47:513
32481439919152,1910cyclictest3271829-21kworker/3:2+events20:30:523
324814399191510,1902cyclictest3326999-21kworker/3:1+events23:27:083
32481369919103,1904cyclictest3286713-21kworker/1:2+events21:19:271
32481439919079,1895cyclictest3269665-21kworker/3:1+events20:23:473
32481339919035,1895cyclictest3308822-21kworker/0:0+events22:29:470
32481369919027,1892cyclictest3294489-21kworker/1:2+events21:41:551
32481439919019,1889cyclictest3317243-21kworker/3:1+events22:53:183
32481439918969,1884cyclictest3297442-21kworker/3:0+events21:51:063
32481369918959,1883cyclictest3292850-21kworker/1:1+events22:20:321
32481439918929,1880cyclictest3273425-21kworker/3:0+events20:38:273
32481439918919,1879cyclictest3331278-21kworker/3:0+events00:03:063
32481439918892,1884cyclictest3345753-21kworker/3:2+events00:17:113
32481439918849,1872cyclictest3262713-21kworker/3:2+events20:11:553
32481439918845,1876cyclictest3345753-21kworker/3:2+events00:13:463
324814399188410,1871cyclictest3326999-21kworker/3:1+events23:33:153
32481439918839,1872cyclictest3314992-21kworker/3:2+events22:38:513
32481439918836,1874cyclictest3286595-21kworker/3:2+events21:25:563
32481369918833,1877cyclictest3259293-21kworker/1:1+events19:48:221
32481439918823,1876cyclictest3331278-21kworker/3:0+events00:05:453
32481439918813,1875cyclictest3286595-21kworker/3:2+events21:45:443
32481439918779,1865cyclictest3280180-21kworker/3:2+events20:56:573
32481439918779,1865cyclictest3280180-21kworker/3:2+events20:56:573
32481439918768,1865cyclictest3297442-21kworker/3:0+events21:56:043
32481439918758,1864cyclictest3347389-21kworker/3:1+events00:30:293
32481369918744,1867cyclictest3299322-21kworker/1:0+events21:54:111
32481439918726,1863cyclictest3254494-21kworker/3:1+events19:30:543
32481369918587,1848cyclictest3337556-21kworker/1:1+events00:04:351
324814399184410,1831cyclictest3334579-21kworker/3:2+events23:41:583
32481439918357,1825cyclictest3291279-21kworker/3:0+events21:24:473
32481439918189,1806cyclictest3302287-21kworker/3:2+events22:18:153
32481439918169,1804cyclictest3331278-21kworker/3:0+events23:46:493
32481439918168,1805cyclictest3317243-21kworker/3:1+events23:03:153
32481439918168,1805cyclictest3317243-21kworker/3:1+events23:03:153
324814399181510,1802cyclictest3251657-21kworker/3:2+events19:35:313
32481439918149,1802cyclictest3329678-21kworker/3:2+events23:24:543
32481439918125,1804cyclictest3251657-21kworker/3:2+events19:43:483
32481439918119,1799cyclictest3303943-21kworker/3:0+events22:07:053
32481439918079,1795cyclictest3249497-21kworker/3:1+events19:17:153
32481439918055,1797cyclictest3323194-21kworker/3:0+events23:06:193
32481439918036,1794cyclictest3282980-21kworker/3:1+events21:03:043
32481369918039,1791cyclictest3246248-21kworker/1:1+events19:22:301
32481369917899,1777cyclictest3284697-21kworker/1:0+events21:06:091
3247917214292,46sleep00-21swapper/019:09:370
32479102141111,25sleep30-21swapper/319:09:333
32477192141111,25sleep10-21swapper/119:07:071
3247603214095,15sleep20-21swapper/219:05:402
324814399170,2cyclictest3339707-21apt-get23:55:123
324814399150,14cyclictest3338955-21/usr/sbin/munin23:50:203
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional