You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-21 - 05:28

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #f, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails
  
Click here to display the system's profile data.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackfslot7s.osadl.org (updated Fri Nov 21, 2025 00:44:17)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
356892999332315,3303cyclictest3606102-21kworker/2:1+events21:15:192
356892199271513,2698cyclictest3654619-21kworker/0:1+events00:05:010
356892199271513,2698cyclictest3654619-21kworker/0:1+events00:05:010
356892999270013,2681cyclictest3579161-21kworker/2:0+events20:05:132
356892499268512,2671cyclictest3601334-21kworker/1:2+events20:50:171
356892499250411,2489cyclictest3612692-21kworker/1:0+events21:29:101
356892999250111,2486cyclictest3654529-21kworker/2:1+events23:40:122
356892999250012,2484cyclictest3577431-21kworker/2:2+events19:50:012
356892499249312,2477cyclictest3641666-21kworker/1:1+events22:55:581
356892199249012,2476cyclictest3627752-21kworker/0:2+events22:15:180
35689349924829,2470cyclictest3624040-21kworker/3:0+events22:07:203
356892499246312,2447cyclictest3601334-21kworker/1:2+events21:00:151
356892499246312,2447cyclictest3582353-21kworker/1:0+events19:55:181
356892499241612,2401cyclictest3667237-21kworker/1:2+events00:20:131
356893499239911,2385cyclictest3614544-21kworker/3:2+events21:40:143
356893499235010,2337cyclictest3574273-21kworker/3:1+events19:36:163
356892199232612,2310cyclictest3611337-21kworker/0:1+events21:20:170
356893499231411,2300cyclictest3644821-21kworker/3:0+events23:24:093
356893499230110,2288cyclictest3616652-21kworker/3:0+events21:39:063
356893499227311,2260cyclictest3667119-21kworker/3:2+events00:20:593
35689249922698,2258cyclictest3656070-21kworker/1:1+events23:52:541
356892499226310,2250cyclictest3616060-21kworker/1:1+events21:38:421
35689349922588,2247cyclictest3660999-21kworker/3:1+events00:11:523
356892499225811,2243cyclictest3572407-21kworker/1:1+events19:44:441
356892499225810,2244cyclictest3575542-21kworker/1:2+events19:36:391
35689349922579,2244cyclictest3603214-21kworker/3:1+events20:58:183
356893499225711,2242cyclictest3639821-21kworker/3:0+events22:47:553
356893499225611,2242cyclictest3654343-21kworker/3:0+events23:51:443
356893499225510,2242cyclictest3627482-21kworker/3:2+events22:28:433
356893499225211,2238cyclictest3583881-21kworker/3:0+events19:57:383
356893499225110,2238cyclictest3654343-21kworker/3:0+events23:36:023
356893499225110,2237cyclictest3643122-21kworker/3:1+events23:01:153
356892499225110,2238cyclictest3667237-21kworker/1:2+events00:33:511
356893499224511,2231cyclictest3644821-21kworker/3:0+events23:14:393
356893499224211,2228cyclictest3578916-21kworker/3:2+events20:19:383
356893499224110,2228cyclictest3611247-21kworker/3:0+events21:21:033
356892499223910,2225cyclictest3617938-21kworker/1:0+events21:43:181
356892499223910,2225cyclictest3594776-21kworker/1:2+events20:32:031
356892499222610,2212cyclictest3590229-21kworker/1:1+events20:27:501
356892999221910,2205cyclictest3644993-21kworker/2:0+events23:07:432
35689249922159,2203cyclictest3635042-21kworker/1:2+events22:36:021
356893499221111,2196cyclictest3654343-21kworker/3:0+events23:31:263
356893499220211,2189cyclictest3574273-21kworker/3:1+events19:40:533
356892499219910,2186cyclictest3582353-21kworker/1:0+events20:00:391
356892199219811,2184cyclictest3638486-21kworker/0:2+events23:06:100
356892199219710,2183cyclictest3654619-21kworker/0:1+events23:55:570
35689219921958,2184cyclictest3633561-21kworker/0:1+events22:35:150
356892199219510,2182cyclictest3638486-21kworker/0:2+events22:53:150
356892199219510,2182cyclictest3604774-21kworker/0:0+events21:02:550
356892999219411,2180cyclictest3633621-21kworker/2:1+events22:30:142
356892499218710,2173cyclictest3670796-21kworker/1:1+events00:27:231
356892199218711,2173cyclictest3625871-21kworker/0:1+events22:07:430
35689219921858,2174cyclictest3638486-21kworker/0:2+events23:18:310
356892199218211,2168cyclictest3633561-21kworker/0:1+events22:46:440
35689219921819,2169cyclictest3594963-21kworker/0:1+events20:33:560
356892199218111,2167cyclictest3604774-21kworker/0:0+events21:07:260
356892199217610,2163cyclictest3580756-21kworker/0:1+events19:47:210
356892999216711,2153cyclictest3577431-21kworker/2:2+events19:39:212
35689219921649,2152cyclictest3593349-21kworker/0:0+events20:45:220
356892199216311,2149cyclictest3585258-21kworker/0:0+events20:00:180
35689299921599,2147cyclictest3609850-21kworker/2:2+events21:25:142
35689219921598,2148cyclictest3604774-21kworker/0:0+events21:13:250
356892199215310,2140cyclictest3670574-21kworker/0:2+events00:31:550
35689249921437,2132cyclictest3636652-21kworker/1:1+events22:43:381
35689349921418,2129cyclictest3590199-21kworker/3:1+events20:21:343
35689249921307,2120cyclictest3582353-21kworker/1:0+events19:50:231
356892999212911,2114cyclictest3588656-21kworker/2:1+events20:13:072
356892999212811,2114cyclictest3642941-21kworker/2:2+events22:57:062
356892999212711,2113cyclictest3649335-21kworker/2:0+events23:21:302
356892999212510,2112cyclictest3667029-21kworker/2:0+events00:22:282
35689249921247,2114cyclictest3614280-21kworker/1:2+events21:53:341
356892999212211,2108cyclictest3572905-21kworker/2:0+events19:26:222
356892499212010,2107cyclictest3656070-21kworker/1:1+events23:47:291
356892499212010,2107cyclictest3590229-21kworker/1:1+events20:16:341
356892499211910,2106cyclictest3643616-21kworker/1:2+events23:04:401
356892999211811,2104cyclictest3649335-21kworker/2:0+events23:30:152
356893499211710,2104cyclictest3624040-21kworker/3:0+events22:04:143
356893499211710,2104cyclictest3624040-21kworker/3:0+events22:04:143
35689349921159,2103cyclictest3672244-21kworker/3:0+events00:34:593
356893499211510,2101cyclictest3598045-21kworker/3:1+events20:48:023
35689249921159,2103cyclictest3580664-21kworker/1:2+events19:45:301
35689349921149,2102cyclictest3570334-21kworker/3:2+events19:27:563
356893499211410,2101cyclictest3624040-21kworker/3:0+events22:13:003
356893499211410,2101cyclictest3578916-21kworker/3:2+events19:53:503
35689249921149,2102cyclictest3638335-21kworker/1:0+events23:07:221
35689349921138,2101cyclictest3620863-21kworker/3:1+events21:51:143
356893499211310,2100cyclictest3596326-21kworker/3:2+events20:31:413
35689249921138,2102cyclictest3590229-21kworker/1:1+events20:22:451
356892499211310,2100cyclictest3632228-21kworker/1:0+events22:30:371
356892499211310,2100cyclictest3627332-21kworker/1:2+events22:11:081
356892499211310,2100cyclictest3607635-21kworker/1:1+events21:13:481
35689349921126,2102cyclictest3590199-21kworker/3:1+events20:14:413
356893499211210,2099cyclictest3643122-21kworker/3:1+events22:58:583
35689349921118,2100cyclictest3670637-21kworker/3:1+events00:26:383
356893499211110,2098cyclictest3636922-21kworker/3:2+events22:44:473
356893499211110,2098cyclictest3614544-21kworker/3:2+events21:31:263
356893499211110,2098cyclictest3596326-21kworker/3:2+events20:42:393
35689249921117,2101cyclictest3588627-21kworker/1:2+events20:12:431
356892499211110,2098cyclictest3607635-21kworker/1:1+events21:21:251
35689349921109,2098cyclictest3614544-21kworker/3:2+events21:56:383
35689249921108,2098cyclictest3657819-21kworker/1:0+events00:10:211
35689349921099,2097cyclictest3578916-21kworker/3:2+events20:05:583
35689249921098,2099cyclictest3647882-21kworker/1:2+events23:20:031
35689349921079,2095cyclictest3651341-21kworker/3:1+events23:29:513
356892999210711,2093cyclictest3630293-21kworker/2:0+events22:23:462
35689249921078,2096cyclictest3665546-21kworker/1:1+events00:16:251
35689249921069,2094cyclictest3649674-21kworker/1:0+events23:25:421
356892499210610,2093cyclictest3569059-21kworker/1:0+events19:19:101
356892499210610,2092cyclictest3661269-21kworker/1:2+events00:01:591
356892499210610,2092cyclictest3661269-21kworker/1:2+events00:01:591
35689249921059,2093cyclictest3628960-21kworker/1:1+events22:16:281
35689249921058,2094cyclictest3646185-21kworker/1:1+events23:14:151
35689249921056,2096cyclictest3596389-21kworker/1:0+events20:43:501
356892499210510,2092cyclictest3638335-21kworker/1:0+events22:49:261
356892499210510,2091cyclictest3594776-21kworker/1:2+events20:37:161
356892499210410,2091cyclictest3614280-21kworker/1:2+events21:34:081
356892999210311,2089cyclictest3602934-21kworker/2:2+events20:57:312
35689249921037,2093cyclictest3654435-21kworker/1:2+events23:42:331
35689249921036,2094cyclictest3639924-21kworker/1:2+events22:52:061
35689349921029,2089cyclictest3612902-21kworker/3:1+events21:29:553
35689349921027,2092cyclictest3660999-21kworker/3:1+events00:00:303
35689349921027,2092cyclictest3660999-21kworker/3:1+events00:00:293
356893499210210,2089cyclictest3635259-21kworker/3:1+events22:34:303
35689249921029,2090cyclictest3601334-21kworker/1:2+events21:05:551
356892499210210,2089cyclictest3622405-21kworker/1:0+events21:57:461
356892499210010,2087cyclictest3596389-21kworker/1:0+events20:49:321
35689249920997,2089cyclictest3622405-21kworker/1:0+events22:03:261
35689249920997,2089cyclictest3622405-21kworker/1:0+events22:03:261
35689349920983,2091cyclictest3582295-21kworker/3:1+events19:49:153
356893499209810,2085cyclictest3629064-21kworker/3:1+events22:17:383
356892499209810,2085cyclictest3622405-21kworker/1:0+events22:09:381
356892499209810,2084cyclictest3603467-21kworker/1:1+events20:55:591
35689349920975,2087cyclictest3665487-21kworker/3:0+events00:07:423
356893499209610,2082cyclictest3648061-21kworker/3:2+events23:19:393
356892499209510,2082cyclictest3619633-21kworker/1:1+events21:49:451
35689349920949,2082cyclictest3605067-21kworker/3:0+events21:11:573
356892499209310,2079cyclictest3647882-21kworker/1:2+events23:16:091
356893499209110,2078cyclictest3605067-21kworker/3:0+events21:07:023
356892999208810,2075cyclictest3597952-21kworker/2:1+events20:53:202
35689219920858,2074cyclictest3591685-21kworker/0:2+events20:25:540
35689349920847,2074cyclictest3578916-21kworker/3:2+events20:04:243
356892199208210,2069cyclictest3641727-21kworker/0:0+events22:57:290
356892499208010,2067cyclictest3628960-21kworker/1:1+events22:24:561
356892199207810,2065cyclictest3660788-21kworker/0:2+events00:12:140
35689349920765,2068cyclictest3605067-21kworker/3:0+events21:03:393
356892999207510,2062cyclictest3579161-21kworker/2:0+events19:58:442
35689349920749,2062cyclictest3601366-21kworker/3:2+events20:51:043
356893499207310,2060cyclictest3598045-21kworker/3:1+events20:39:113
356892199207310,2061cyclictest3617772-21kworker/0:1+events21:45:100
35689219920704,2063cyclictest3621013-21kworker/0:2+events21:55:510
35689349920699,2057cyclictest3568530-21kworker/3:1+events19:18:243
35689249920698,2058cyclictest3654435-21kworker/1:2+events23:34:301
35689219920619,2049cyclictest3611337-21kworker/0:1+events21:26:490
35689249920588,2047cyclictest3562114-21kworker/1:2+events19:13:501
356892499205810,2047cyclictest3649674-21kworker/1:0+events23:35:381
356893499205610,2042cyclictest3609642-21kworker/3:1+events21:15:423
356892499205310,2040cyclictest3632228-21kworker/1:0+events22:26:301
356893499204910,2036cyclictest3668748-21kworker/3:0+events00:18:423
35689349920486,2039cyclictest3572613-21kworker/3:0+events19:24:053
356893499204810,2035cyclictest3614544-21kworker/3:2+events21:46:183
35689349920459,2033cyclictest3641428-21kworker/3:2+events22:52:293
35689299920452,2040cyclictest3592210-21kworker/2:1+events20:23:562
35689249920439,2031cyclictest3657819-21kworker/1:0+events23:59:001
35689219920439,2031cyclictest3575571-21kworker/0:1+events19:37:510
356892199204210,2028cyclictest3660788-21kworker/0:2+events00:05:450
35689349920418,2030cyclictest3654343-21kworker/3:0+events23:42:093
35689349920415,2033cyclictest3577140-21kworker/3:0+events19:32:033
35689249920403,2034cyclictest3569059-21kworker/1:0+events19:21:471
35689219920409,2028cyclictest3644614-21kworker/0:1+events23:01:390
356892199204010,2028cyclictest3575571-21kworker/0:1+events19:30:090
35689219920398,2028cyclictest3579133-21kworker/0:0+events19:40:300
35689219920387,2028cyclictest3621013-21kworker/0:2+events22:02:200
35689219920387,2028cyclictest3621013-21kworker/0:2+events22:02:190
356892199203810,2025cyclictest3593349-21kworker/0:0+events20:24:440
35689349920379,2026cyclictest3590199-21kworker/3:1+events20:25:073
35689299920377,2026cyclictest3654529-21kworker/2:1+events23:38:412
356892199203710,2024cyclictest3594963-21kworker/0:1+events20:35:480
35689299920365,2028cyclictest3612631-21kworker/2:0+events21:30:202
35689219920368,2027cyclictest3617772-21kworker/0:1+events21:50:080
356892199203610,2024cyclictest3585258-21kworker/0:0+events20:10:480
35689249920359,2024cyclictest3675207-21kworker/1:1+events00:38:251
356892199203310,2019cyclictest3670574-21kworker/0:2+events00:29:150
356892199203210,2019cyclictest3612961-21kworker/0:2+events21:33:000
356892999203110,2018cyclictest3630293-21kworker/2:0+events22:19:342
35689219920317,2021cyclictest3627752-21kworker/0:2+events22:12:390
356892199203110,2018cyclictest3647851-21kworker/0:1+events23:24:560
356892199203010,2017cyclictest3585258-21kworker/0:0+events20:08:110
35689219920298,2018cyclictest3593349-21kworker/0:0+events20:52:580
35689219920295,2022cyclictest3630323-21kworker/0:0+events22:26:520
35689219920289,2016cyclictest3659120-21kworker/0:0+events23:49:480
35689219920286,2019cyclictest3651054-21kworker/0:0+events23:34:060
356892199202810,2015cyclictest3604774-21kworker/0:0+events21:16:500
35689219920278,2016cyclictest3612961-21kworker/0:2+events21:42:070
356892199202710,2015cyclictest3654619-21kworker/0:1+events23:52:070
356892999202610,2012cyclictest3633621-21kworker/2:1+events22:36:502
35689219920267,2016cyclictest3647851-21kworker/0:1+events23:27:120
35689249920243,2018cyclictest3582353-21kworker/1:0+events20:08:561
35689219920249,2011cyclictest3667297-21kworker/0:1+events00:16:470
356892199202410,2011cyclictest3654619-21kworker/0:1+events23:41:220
356892199202110,2008cyclictest3672213-21kworker/0:1+events00:39:080
356892199202010,2007cyclictest3651054-21kworker/0:0+events23:38:190
356892199201710,2004cyclictest3636863-21kworker/0:0+events22:40:320
356892199201610,2004cyclictest3577494-21kworker/0:2+events19:51:540
35689219920108,1999cyclictest3573934-21kworker/0:0+events19:23:430
35689219920107,2000cyclictest3665668-21kworker/0:0+events00:24:230
35689299920088,1997cyclictest3599640-21kworker/2:2+events20:44:582
35689249920048,1993cyclictest3575542-21kworker/1:2+events19:34:221
35689299919994,1992cyclictest3583630-21kworker/2:1+events19:54:552
356893499199111,1978cyclictest3654343-21kworker/3:0+events23:45:123
35689219919899,1977cyclictest3647851-21kworker/0:1+events23:13:060
35689299919889,1976cyclictest3659362-21kworker/2:0+events23:57:282
35689299919859,1973cyclictest3656162-21kworker/2:2+events23:54:042
356892999198410,1970cyclictest3579161-21kworker/2:0+events20:16:562
356892999198310,1971cyclictest3592210-21kworker/2:1+events20:30:082
35689219919829,1970cyclictest3585258-21kworker/0:0+events19:58:000
356892999198110,1967cyclictest3607932-21kworker/2:0+events21:11:122
35689249919789,1966cyclictest3572407-21kworker/1:1+events19:26:461
35689349919759,1963cyclictest3632017-21kworker/3:0+events22:36:253
35689249919759,1963cyclictest3657819-21kworker/1:0+events00:06:551
35689349919749,1962cyclictest3560300-21kworker/3:0+events19:11:303
35689219919745,1966cyclictest3594963-21kworker/0:1+events20:58:410
356892999197310,1960cyclictest3612631-21kworker/2:0+events21:37:122
356892999197310,1960cyclictest3575508-21kworker/2:1+events19:32:252
35689219919736,1964cyclictest3635196-21kworker/0:2+events22:34:070
35689299919728,1961cyclictest3606102-21kworker/2:1+events21:05:302
35689299919725,1964cyclictest3647017-21kworker/2:1+events23:11:122
35689299919724,1964cyclictest3625902-21kworker/2:0+events22:05:222
356892999197210,1959cyclictest3656162-21kworker/2:2+events23:49:012
35689349919717,1962cyclictest3644821-21kworker/3:0+events23:05:043
35689299919715,1963cyclictest3656162-21kworker/2:2+events00:09:372
356892999197110,1957cyclictest3602934-21kworker/2:2+events21:01:262
35689219919716,1963cyclictest3630323-21kworker/0:0+events22:21:070
35689299919698,1958cyclictest3597952-21kworker/2:1+events20:49:102
356892999196710,1956cyclictest3579161-21kworker/2:0+events19:40:062
35689299919649,1952cyclictest3615911-21kworker/2:2+events21:55:302
356892999196310,1950cyclictest3627536-21kworker/2:1+events22:14:322
35689219919637,1954cyclictest3566995-21kworker/0:2+events19:25:120
35689299919628,1951cyclictest3579161-21kworker/2:0+events20:39:352
35689299919625,1954cyclictest3633621-21kworker/2:1+events22:41:432
356892999196210,1949cyclictest3650994-21kworker/2:2+events23:29:062
35689219919629,1951cyclictest3594963-21kworker/0:1+events20:43:260
35689299919618,1950cyclictest3672305-21kworker/2:1+events00:32:192
35689299919612,1956cyclictest3667029-21kworker/2:0+events00:11:292
356892999196110,1948cyclictest3630293-21kworker/2:0+events22:25:422
35689299919604,1953cyclictest3640195-21kworker/2:0+events22:54:472
356892999195810,1945cyclictest3615911-21kworker/2:2+events21:44:482
35689299919576,1948cyclictest3563747-21kworker/2:2+events19:20:192
35689219919564,1949cyclictest3566995-21kworker/0:2+events19:16:530
356892999195210,1938cyclictest3579161-21kworker/2:0+events20:29:232
356892999195110,1938cyclictest3621210-21kworker/2:1+events21:53:102
356893499194610,1933cyclictest3632017-21kworker/3:0+events22:23:233
35689219919464,1939cyclictest3591685-21kworker/0:2+events20:19:140
356893499194310,1930cyclictest3673966-21kworker/3:1+events00:36:073
35689299919409,1928cyclictest3570217-21kworker/2:1+events19:18:472
356892999193710,1924cyclictest3609850-21kworker/2:2+events21:20:392
35689349919366,1927cyclictest3660999-21kworker/3:1+events23:56:433
35689299919179,1905cyclictest3670604-21kworker/2:2+events00:28:522
35689299919165,1908cyclictest3673906-21kworker/2:0+events00:36:542
35689299919158,1905cyclictest3621210-21kworker/2:1+events22:00:032
35689299919158,1905cyclictest3621210-21kworker/2:1+events22:00:022
35689299919047,1896cyclictest3656162-21kworker/2:2+events00:00:072
35689299919047,1896cyclictest3656162-21kworker/2:2+events00:00:062
356892999190210,1891cyclictest3662600-21kworker/2:1+events00:15:382
35689299918988,1886cyclictest3586880-21kworker/2:2+events20:03:402
35689299918954,1888cyclictest3619561-21kworker/2:0+events21:45:332
35689219918949,1882cyclictest3565376-21kworker/0:0+events19:13:270
35689299918939,1882cyclictest3642941-21kworker/2:2+events23:15:012
35689299918936,1884cyclictest3563747-21kworker/2:2+events19:13:042
35689299918927,1882cyclictest3638517-21kworker/2:2+events22:46:222
35689219918869,1874cyclictest3612961-21kworker/0:2+events21:37:330
356892999185010,1837cyclictest3633621-21kworker/2:1+events23:00:082
35686852157114,25sleep10-21swapper/119:09:301
35686172142108,28sleep30-21swapper/319:08:363
35685062140108,26sleep20-21swapper/219:07:152
35686792139110,24sleep00-21swapper/019:09:250
356892499136,5cyclictest3610088-21apt-get21:15:121
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional