You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-30 - 21:47

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #f, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails
  
Click here to display the system's profile data.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackfslot7s.osadl.org (updated Sun Nov 30, 2025 00:44:16)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
363072399281413,2797cyclictest3700161-21kworker/0:0+events22:55:220
363072399279010,2775cyclictest3648880-21kworker/0:2+events20:10:140
363072399272813,2710cyclictest3653366-21kworker/0:0+events20:20:140
363072399271312,2699cyclictest3687360-21kworker/0:2+events22:15:230
363072699269011,2677cyclictest3669506-21kworker/1:1+events21:50:021
363073199265611,2641cyclictest3640540-21kworker/2:1+events19:40:232
363073399257713,2560cyclictest3659950-21kworker/3:1+events20:50:183
363073399256612,2551cyclictest3690527-21kworker/3:0+events22:32:273
363073399256512,2550cyclictest3648759-21kworker/3:2+events20:05:163
36307269925329,2519cyclictest3687199-21kworker/1:2+events22:25:111
36307339925228,2510cyclictest3719103-21kworker/3:2+events23:45:173
363072699247710,2463cyclictest3647286-21kworker/1:2+events20:11:231
363073399247612,2461cyclictest3647041-21kworker/3:0+events19:57:343
363072699247412,2460cyclictest3653462-21kworker/1:2+events20:40:221
363073199246412,2448cyclictest3644234-21kworker/2:1+events20:05:392
363073199245112,2435cyclictest3695204-21kworker/2:0+events22:40:522
363073199244712,2431cyclictest3715978-21kworker/2:1+events23:37:442
363072699244612,2430cyclictest3709761-21kworker/1:2+events23:39:121
363073199244512,2429cyclictest3706408-21kworker/2:2+events23:27:302
363073399244411,2430cyclictest3651701-21kworker/3:0+events20:19:073
363073199244412,2429cyclictest3656596-21kworker/2:1+events20:30:552
363072699243212,2416cyclictest3653462-21kworker/1:2+events21:08:061
363072399240512,2389cyclictest3664448-21kworker/0:1+events20:54:290
363073399240111,2387cyclictest3703362-21kworker/3:2+events23:00:163
363072699239911,2384cyclictest3664540-21kworker/1:1+events20:55:371
363072399239812,2382cyclictest3706348-21kworker/0:0+events23:04:470
363072699239312,2378cyclictest3709761-21kworker/1:2+events23:16:341
363072399238810,2375cyclictest3656718-21kworker/0:1+events20:32:280
363073399238312,2368cyclictest3647041-21kworker/3:0+events20:00:163
363073399238211,2368cyclictest3730512-21kworker/3:1+events00:25:583
363073399238012,2365cyclictest3709550-21kworker/3:0+events23:27:543
363072399236911,2355cyclictest3637277-21kworker/0:1+events19:31:140
363072699236210,2349cyclictest3669506-21kworker/1:1+events21:57:371
363073399235212,2337cyclictest3630397-21kworker/3:1+events19:20:333
363072699235212,2336cyclictest3719163-21kworker/1:0+events23:45:381
363072699235112,2336cyclictest3687199-21kworker/1:2+events22:22:351
363072399234912,2333cyclictest3683706-21kworker/0:0+events22:08:300
363072699234712,2332cyclictest3650214-21kworker/1:1+events20:15:591
363073399232711,2312cyclictest3720734-21kworker/3:1+events23:55:223
363073399232510,2313cyclictest3713219-21kworker/3:1+events23:40:413
36307269923247,2315cyclictest3680216-21kworker/1:0+events21:45:061
363072399231912,2304cyclictest3685686-21kworker/0:1+events22:03:340
363073399231310,2299cyclictest3722620-21kworker/3:2+events00:03:383
363072699230911,2294cyclictest3720795-21kworker/1:2+events23:53:311
363073199228310,2271cyclictest3732204-21kworker/2:1+events00:30:082
363073399228210,2269cyclictest3662814-21kworker/3:2+events20:57:073
363072399227710,2264cyclictest3673981-21kworker/0:1+events21:24:020
363073399227511,2261cyclictest3672478-21kworker/3:1+events21:19:543
363073399227510,2262cyclictest3701552-21kworker/3:0+events22:53:483
363073199227510,2262cyclictest3690795-21kworker/2:1+events22:19:562
363073399227411,2259cyclictest3713219-21kworker/3:1+events23:30:343
363073399227311,2260cyclictest3690527-21kworker/3:0+events22:35:083
363073399227311,2258cyclictest3687036-21kworker/3:1+events22:09:393
363073399227310,2260cyclictest3636953-21kworker/3:0+events19:34:393
363073399227210,2258cyclictest3644101-21kworker/3:1+events19:50:163
363073399227111,2257cyclictest3651701-21kworker/3:0+events20:27:063
363073399227110,2258cyclictest3687036-21kworker/3:1+events22:11:343
363073399227011,2256cyclictest3636953-21kworker/3:0+events19:48:453
363073399227010,2257cyclictest3681936-21kworker/3:0+events22:02:073
36307339922698,2258cyclictest3730512-21kworker/3:1+events00:23:403
363073399226811,2254cyclictest3674455-21kworker/3:2+events21:37:033
363073399226811,2254cyclictest3662814-21kworker/3:2+events21:12:403
36307339922679,2254cyclictest3730512-21kworker/3:1+events00:34:433
363073399226711,2252cyclictest3717455-21kworker/3:0+events23:52:223
363073399226710,2254cyclictest3703362-21kworker/3:2+events23:09:423
363073399226411,2250cyclictest3680333-21kworker/3:1+events21:47:443
363073399226411,2250cyclictest3656567-21kworker/3:2+events20:36:363
363073399226410,2251cyclictest3727310-21kworker/3:1+events00:11:333
363073399226410,2251cyclictest3636953-21kworker/3:0+events19:43:463
363073399226011,2246cyclictest3681936-21kworker/3:0+events21:50:463
363073399225510,2242cyclictest3703362-21kworker/3:2+events23:17:433
363073399225411,2240cyclictest3717455-21kworker/3:0+events23:38:073
363073399225411,2240cyclictest3672478-21kworker/3:1+events21:26:443
363073399225411,2239cyclictest3664477-21kworker/3:0+events21:03:333
363073199225310,2240cyclictest3711652-21kworker/2:1+events23:24:522
363072699225110,2238cyclictest3642120-21kworker/1:1+events19:48:211
363073399224811,2234cyclictest3703362-21kworker/3:2+events23:12:023
363073199224710,2234cyclictest3725659-21kworker/2:2+events00:02:562
363073199224710,2234cyclictest3715978-21kworker/2:1+events23:33:572
363072399224711,2232cyclictest3642029-21kworker/0:2+events19:45:410
363073399224510,2232cyclictest3690527-21kworker/3:0+events22:24:283
36307319922459,2232cyclictest3695204-21kworker/2:0+events22:28:152
363072399224412,2229cyclictest3709611-21kworker/0:1+events23:18:300
363073199223810,2225cyclictest3636921-21kworker/2:1+events19:27:032
363073399223711,2222cyclictest3630397-21kworker/3:1+events19:29:193
36307319922379,2225cyclictest3659920-21kworker/2:1+events20:52:332
363073199223711,2223cyclictest3701580-21kworker/2:2+events22:55:462
36307339922368,2224cyclictest3630397-21kworker/3:1+events19:14:543
363073199223611,2221cyclictest3648610-21kworker/2:0+events20:02:122
363073199223610,2222cyclictest3677217-21kworker/2:0+events21:34:232
363073199223310,2219cyclictest3723949-21kworker/2:1+events00:13:272
363072699223210,2219cyclictest3669506-21kworker/1:1+events21:27:301
363073199223011,2216cyclictest3717604-21kworker/2:0+events23:41:532
363072699223010,2217cyclictest3642120-21kworker/1:1+events19:53:421
363073199222810,2215cyclictest3679014-21kworker/2:1+events21:51:102
363073199222711,2212cyclictest3687216-21kworker/2:2+events22:08:522
363073199222711,2212cyclictest3669421-21kworker/2:2+events21:20:182
363073199222611,2212cyclictest3650247-21kworker/2:2+events20:20:592
363073199222611,2211cyclictest3666166-21kworker/2:2+events21:01:142
363073199222510,2212cyclictest3671291-21kworker/2:0+events21:19:312
363072699222410,2211cyclictest3655176-21kworker/1:1+events20:26:431
363073199222311,2209cyclictest3638937-21kworker/2:2+events19:36:312
363073199222310,2210cyclictest3701580-21kworker/2:2+events22:51:282
363072699222312,2207cyclictest3724279-21kworker/1:2+events00:32:031
363073199222110,2208cyclictest3735455-21kworker/2:0+events00:37:022
363072399222011,2206cyclictest3698476-21kworker/0:2+events22:38:580
363073199221911,2205cyclictest3651851-21kworker/2:1+events20:13:422
363073199221911,2204cyclictest3658261-21kworker/2:0+events20:44:302
363072699221912,2204cyclictest3714252-21kworker/1:0+events23:29:271
363072699221911,2205cyclictest3724279-21kworker/1:2+events00:16:031
36307239922189,2207cyclictest3727190-21kworker/0:1+events00:10:060
36307319922179,2206cyclictest3725659-21kworker/2:2+events00:05:102
36307319922179,2206cyclictest3725659-21kworker/2:2+events00:05:102
36307319922169,2204cyclictest3695204-21kworker/2:0+events22:34:452
363072399221611,2202cyclictest3673981-21kworker/0:1+events21:32:280
363072399221511,2201cyclictest3692225-21kworker/0:0+events22:24:490
36307319922131,2209cyclictest3679014-21kworker/2:1+events21:49:382
363073199221310,2199cyclictest3720955-21kworker/2:0+events23:58:462
363072399221312,2198cyclictest3660030-21kworker/0:0+events21:01:360
363073399221210,2199cyclictest3632103-21kworker/3:0+events19:18:403
363072399220410,2191cyclictest3687360-21kworker/0:2+events22:13:310
363072399220310,2190cyclictest3727190-21kworker/0:1+events00:29:470
363072399220310,2190cyclictest3725568-21kworker/0:2+events00:08:100
363072399220310,2190cyclictest3725568-21kworker/0:2+events00:08:100
363072399220310,2190cyclictest3639381-21kworker/0:0+events19:36:540
36307239922027,2192cyclictest3683706-21kworker/0:0+events21:58:430
363072399220211,2187cyclictest3722966-21kworker/0:1+events23:59:100
363072399220210,2189cyclictest3639381-21kworker/0:0+events20:07:560
363072399220210,2188cyclictest3673981-21kworker/0:1+events21:28:580
36307239922019,2188cyclictest3655025-21kworker/0:2+events20:26:180
363072399220110,2188cyclictest3645583-21kworker/0:2+events19:56:020
363073199219910,2186cyclictest3697110-21kworker/2:2+events22:36:192
363072399219911,2185cyclictest3712860-21kworker/0:0+events00:00:400
363072399219710,2184cyclictest3712860-21kworker/0:0+events23:35:510
36307239921969,2184cyclictest3727190-21kworker/0:1+events00:24:500
36307239921848,2173cyclictest3712860-21kworker/0:0+events23:41:290
363072399218210,2169cyclictest3712860-21kworker/0:0+events23:46:490
36307239921797,2168cyclictest3660030-21kworker/0:0+events21:05:520
363072399217611,2162cyclictest3700161-21kworker/0:0+events22:53:030
36307319921758,2163cyclictest3708216-21kworker/2:1+events23:10:522
363072399217311,2159cyclictest3693961-21kworker/0:2+events22:25:570
363072399217210,2159cyclictest3704727-21kworker/0:1+events23:05:530
363072399217111,2157cyclictest3658321-21kworker/0:2+events20:39:150
363072699216910,2156cyclictest3687199-21kworker/1:2+events22:36:431
363072699216711,2153cyclictest3724279-21kworker/1:2+events00:10:291
363072699216711,2153cyclictest3715913-21kworker/1:1+events00:08:321
363072699216711,2153cyclictest3715913-21kworker/1:1+events00:08:321
363072399216610,2153cyclictest3631899-21kworker/0:2+events19:15:170
363072699216510,2152cyclictest3647286-21kworker/1:2+events20:06:481
363072699216411,2150cyclictest3683736-21kworker/1:0+events22:04:211
363072399216410,2151cyclictest3673981-21kworker/0:1+events21:45:300
363072399216410,2151cyclictest3670783-21kworker/0:2+events21:12:170
36307269921637,2153cyclictest3706436-21kworker/1:2+events23:07:261
363072699216311,2148cyclictest3711362-21kworker/1:1+events23:22:371
363072699216310,2150cyclictest3704817-21kworker/1:1+events22:58:251
36307269921629,2149cyclictest3675758-21kworker/1:2+events21:34:011
363072699216211,2148cyclictest3715913-21kworker/1:1+events00:04:011
363072699216211,2148cyclictest3635617-21kworker/1:2+events19:28:101
363073199216110,2148cyclictest3642297-21kworker/2:2+events19:46:482
363072699216111,2147cyclictest3614175-21kworker/1:1+events19:32:211
363073399216010,2147cyclictest3724246-21kworker/3:0+events00:07:473
363073399216010,2147cyclictest3724246-21kworker/3:0+events00:07:473
36307269921609,2149cyclictest3687199-21kworker/1:2+events22:10:031
36307269921608,2150cyclictest3614175-21kworker/1:1+events19:35:021
363072699215911,2144cyclictest3669506-21kworker/1:1+events21:42:491
363072699215910,2146cyclictest3724279-21kworker/1:2+events00:29:231
363072699215811,2142cyclictest3708095-21kworker/1:0+events23:13:101
363072699215711,2144cyclictest3695534-21kworker/1:0+events22:30:101
363072699215710,2144cyclictest3700008-21kworker/1:1+events22:46:341
36307339921553,2149cyclictest3638846-21kworker/3:2+events19:36:093
363072699215311,2138cyclictest3662877-21kworker/1:0+events20:49:091
36307269921526,2143cyclictest3695534-21kworker/1:0+events22:41:141
363072399214010,2127cyclictest3727190-21kworker/0:1+events00:19:260
363072699213711,2123cyclictest3709761-21kworker/1:2+events23:32:261
36307319921268,2114cyclictest3650247-21kworker/2:2+events20:27:272
363073199212411,2110cyclictest3630333-21kworker/2:0+events19:32:422
36307269921249,2112cyclictest3683736-21kworker/1:0+events22:07:001
363072699212110,2108cyclictest3735169-21kworker/1:1+events00:37:481
36307239921199,2107cyclictest3709611-21kworker/0:1+events23:14:200
363073399211810,2104cyclictest3651701-21kworker/3:0+events20:21:213
36307339921168,2105cyclictest3697141-21kworker/3:1+events22:48:273
36307269921167,2105cyclictest3645615-21kworker/1:0+events19:57:111
363073399211510,2102cyclictest3732142-21kworker/3:0+events00:35:303
36307339921114,2104cyclictest3656567-21kworker/3:2+events20:33:563
36307339921099,2097cyclictest3672478-21kworker/3:1+events21:22:553
363073399210910,2095cyclictest3662814-21kworker/3:2+events21:07:423
363073399210810,2096cyclictest3678714-21kworker/3:0+events21:40:083
363072399210811,2094cyclictest3709611-21kworker/0:1+events23:21:530
363073399210411,2090cyclictest3693842-21kworker/3:1+events22:26:203
36307319920999,2086cyclictest3730482-21kworker/2:2+events00:21:452
363072699209911,2085cyclictest3675758-21kworker/1:2+events21:39:231
36307339920949,2082cyclictest3659950-21kworker/3:1+events20:46:023
363072699209111,2077cyclictest3722649-21kworker/1:0+events23:56:071
36307339920906,2081cyclictest3648759-21kworker/3:2+events20:14:043
36307319920899,2077cyclictest3664508-21kworker/2:0+events21:08:502
363072699208911,2075cyclictest3689073-21kworker/1:1+events22:18:481
363073199208710,2074cyclictest3706408-21kworker/2:2+events23:54:372
36307319920869,2074cyclictest3706408-21kworker/2:2+events23:04:252
36307319920859,2073cyclictest3659920-21kworker/2:1+events20:48:222
36307269920846,2074cyclictest3666139-21kworker/1:0+events21:03:561
363072399208410,2071cyclictest3635647-21kworker/0:2+events19:27:470
36307319920825,2074cyclictest3634377-21kworker/2:2+events19:23:352
363073199208210,2069cyclictest3650247-21kworker/2:2+events20:16:222
36307319920818,2069cyclictest3679014-21kworker/2:1+events21:56:512
363073199208110,2067cyclictest3642297-21kworker/2:2+events19:51:472
36307319920804,2073cyclictest3679014-21kworker/2:1+events21:41:172
36307339920789,2065cyclictest3674455-21kworker/3:2+events21:32:033
36307319920785,2070cyclictest3703176-21kworker/2:0+events23:06:182
363073199207710,2064cyclictest3644234-21kworker/2:1+events19:56:242
36307319920764,2069cyclictest3689135-21kworker/2:0+events22:12:202
36307339920728,2061cyclictest3703362-21kworker/3:2+events22:58:023
363072399207110,2058cyclictest3661811-21kworker/0:2+events20:45:390
363073399206810,2055cyclictest3728909-21kworker/3:2+events00:15:393
363073199206810,2055cyclictest3658261-21kworker/2:0+events20:37:202
363073199206710,2054cyclictest3659920-21kworker/2:1+events20:56:222
36307239920679,2055cyclictest3732237-21kworker/0:2+events00:30:530
36307239920667,2056cyclictest3700161-21kworker/0:0+events22:45:270
36307239920649,2052cyclictest3650522-21kworker/0:1+events20:16:470
36307319920629,2050cyclictest3706408-21kworker/2:2+events23:16:122
36307339920617,2050cyclictest3681936-21kworker/3:0+events21:56:043
363073199206110,2048cyclictest3673892-21kworker/2:1+events21:27:062
36307269920618,2049cyclictest3653462-21kworker/1:2+events20:20:361
363072699205910,2046cyclictest3638057-21kworker/1:0+events19:44:331
363072399205911,2044cyclictest3643981-21kworker/0:1+events19:50:390
36307319920587,2048cyclictest3723949-21kworker/2:1+events00:16:262
363073199205610,2043cyclictest3675550-21kworker/2:2+events21:35:562
36307239920554,2048cyclictest3670783-21kworker/0:2+events21:16:510
36307239920553,2049cyclictest3695663-21kworker/0:1+events22:31:400
36307239920535,2045cyclictest3732237-21kworker/0:2+events00:36:400
363072399205010,2037cyclictest3622402-21kworker/0:0+events19:24:210
36307239920479,2034cyclictest3639381-21kworker/0:0+events19:44:100
36307339920469,2034cyclictest3651701-21kworker/3:0+events20:43:453
36307269920468,2035cyclictest3653462-21kworker/1:2+events20:31:181
363072399204510,2032cyclictest3677340-21kworker/0:0+events21:36:400
36307239920449,2032cyclictest3639381-21kworker/0:0+events20:00:400
363072399204410,2031cyclictest3677340-21kworker/0:0+events21:40:310
36307239920439,2031cyclictest3622402-21kworker/0:0+events19:14:320
36307239920429,2030cyclictest3660030-21kworker/0:0+events20:42:130
363072399204210,2029cyclictest3660030-21kworker/0:0+events20:59:430
363073399203710,2024cyclictest3690527-21kworker/3:0+events22:17:193
363072399202010,2007cyclictest3712860-21kworker/0:0+events23:29:030
36307269920178,2006cyclictest3662877-21kworker/1:0+events20:51:241
36307269920159,2002cyclictest3672448-21kworker/1:2+events21:22:331
36307319920139,2001cyclictest3687216-21kworker/2:2+events22:22:132
36307269920129,2000cyclictest3703210-21kworker/1:0+events22:52:391
36307269920127,2002cyclictest3666139-21kworker/1:0+events21:10:251
36307319920104,2003cyclictest3679014-21kworker/2:1+events22:02:522
363073199200810,1995cyclictest3698783-21kworker/2:1+events22:49:362
36307269920087,1998cyclictest3656779-21kworker/1:0+events20:36:561
36307269920087,1998cyclictest3631867-21kworker/1:0+events19:22:041
36307269920067,1996cyclictest3715913-21kworker/1:1+events23:43:221
363072699200610,1993cyclictest3704817-21kworker/1:1+events23:02:321
36307239919996,1990cyclictest3720617-21kworker/0:2+events23:50:550
363072699199610,1983cyclictest3724279-21kworker/1:2+events00:23:171
363072699199410,1981cyclictest3645615-21kworker/1:0+events20:04:291
36307239919928,1981cyclictest3695663-21kworker/0:1+events22:43:110
36307239919807,1970cyclictest3714466-21kworker/0:1+events23:31:420
36307239919725,1964cyclictest3680631-21kworker/0:2+events21:51:530
36307269919485,1940cyclictest3631867-21kworker/1:0+events19:16:011
36307339919427,1933cyclictest3697141-21kworker/3:1+events22:40:073
36307319919335,1925cyclictest3727432-21kworker/2:0+events00:27:062
36307269919319,1918cyclictest3672448-21kworker/1:2+events21:18:461
36307319919299,1917cyclictest3630333-21kworker/2:0+events19:16:442
36307269919174,1910cyclictest3614175-21kworker/1:1+events19:11:531
36307319918839,1871cyclictest3630333-21kworker/2:0+events19:11:082
36302942193165,23sleep30-21swapper/319:07:483
36304662152125,22sleep10-21swapper/119:09:581
36304342148119,24sleep00-21swapper/019:09:330
36302342143102,30sleep20-21swapper/219:07:032
363073199170,16cyclictest3720864-21chrt23:49:462
363073399161,13cyclictest3712992-21munin-run23:25:003
363073199120,1cyclictest0-21swapper/221:10:422
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional