You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-07 - 05:00

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #f, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails
  
Click here to display the system's profile data.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackfslot7s.osadl.org (updated Fri Mar 06, 2026 00:44:16)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
320894799272312,2709cyclictest3281524-21kworker/3:2+events23:00:233
320894799271813,2702cyclictest3200315-21kworker/3:2+events19:35:183
320893799271213,2696cyclictest3302645-21kworker/0:0+events00:00:000
320894099268513,2669cyclictest3289805-21kworker/1:0+events23:30:131
320893799260212,2587cyclictest3305834-21kworker/0:2+events00:10:170
320893799260112,2585cyclictest3254398-21kworker/0:2+events21:40:250
320894099257612,2560cyclictest3286309-21kworker/1:2+events23:12:081
32089409925619,2549cyclictest3293368-21kworker/1:1+events23:44:411
320894799254510,2531cyclictest3287999-21kworker/3:2+events23:15:163
320894099253911,2525cyclictest3315511-21kworker/1:0+events00:39:181
320893799253612,2520cyclictest3261409-21kworker/0:2+events21:59:530
320894099251712,2501cyclictest3231552-21kworker/1:1+events20:17:311
320894799249912,2483cyclictest3301069-21kworker/3:2+events23:54:163
320894099249912,2483cyclictest3218622-21kworker/1:0+events19:40:151
320894099249612,2481cyclictest3283077-21kworker/1:0+events23:08:011
320894099249612,2480cyclictest3286309-21kworker/1:2+events23:15:391
320894099249411,2479cyclictest3313911-21kworker/1:1+events00:32:301
320894099249112,2475cyclictest3237927-21kworker/1:2+events20:37:201
320894099248912,2474cyclictest3257591-21kworker/1:2+events21:53:041
320894099248312,2469cyclictest3307440-21kworker/1:0+events00:16:511
320894099248212,2468cyclictest3279856-21kworker/1:1+events22:50:011
320894099248012,2465cyclictest3268902-21kworker/1:0+events22:20:201
320894099247811,2463cyclictest3249691-21kworker/1:1+events21:18:111
320894099247512,2459cyclictest3265668-21kworker/1:1+events22:07:481
320894799247212,2456cyclictest3268978-21kworker/3:2+events22:15:413
320894799247010,2457cyclictest3259565-21kworker/3:1+events21:48:253
320894799247010,2456cyclictest3268978-21kworker/3:2+events22:34:273
320894799246512,2449cyclictest3236284-21kworker/3:2+events20:32:023
320893799245611,2443cyclictest3296443-21kworker/0:0+events23:50:030
320894499245010,2437cyclictest3284555-21kworker/2:2+events23:10:192
320893799245012,2435cyclictest3281491-21kworker/0:1+events22:56:090
320894799244812,2432cyclictest3246647-21kworker/3:2+events21:18:333
32089479924448,2433cyclictest3286366-21kworker/3:1+events23:10:403
320894799244311,2428cyclictest3225820-21kworker/3:1+events20:06:073
320894799244311,2428cyclictest3225820-21kworker/3:1+events20:06:063
320893799244311,2429cyclictest3302645-21kworker/0:0+events00:07:590
320893799244211,2428cyclictest3254398-21kworker/0:2+events21:35:540
320893799243912,2424cyclictest3275249-21kworker/0:1+events22:42:220
320893799243912,2423cyclictest3254398-21kworker/0:2+events21:29:080
320893799243812,2422cyclictest3294813-21kworker/0:1+events23:39:420
320893799243712,2421cyclictest3267330-21kworker/0:0+events22:32:140
320894799243611,2422cyclictest3276666-21kworker/3:0+events22:39:003
320893799243610,2423cyclictest3239719-21kworker/0:0+events20:45:390
320893799243511,2421cyclictest3291510-21kworker/0:2+events23:27:550
320893799243510,2421cyclictest3283173-21kworker/0:0+events23:06:530
320893799243112,2415cyclictest3225222-21kworker/0:2+events20:12:580
320893799242612,2411cyclictest3259243-21kworker/0:1+events22:06:180
320893799242412,2409cyclictest3279795-21kworker/0:2+events22:53:540
320894499242312,2407cyclictest3297889-21kworker/2:1+events23:46:572
320893799242211,2408cyclictest3302645-21kworker/0:0+events00:29:310
320893799242012,2405cyclictest3261409-21kworker/0:2+events21:51:070
320893799241812,2403cyclictest3243238-21kworker/0:0+events20:59:430
320894799240511,2391cyclictest3308806-21kworker/3:1+events00:37:253
320894499240512,2389cyclictest3230164-21kworker/2:1+events20:41:282
320893799240212,2387cyclictest3208121-21kworker/0:2+events19:32:180
320894799240112,2385cyclictest3246647-21kworker/3:2+events21:10:153
320893799240111,2387cyclictest3208121-21kworker/0:2+events19:21:380
320893799240011,2387cyclictest3296443-21kworker/0:0+events23:45:040
320894099239512,2379cyclictest3247948-21kworker/1:2+events21:22:191
320894799239212,2376cyclictest3281524-21kworker/3:2+events22:54:393
320893799239211,2377cyclictest3239719-21kworker/0:0+events20:42:120
32089479923918,2379cyclictest3222104-21kworker/3:0+events19:57:463
320894799239112,2375cyclictest3267388-21kworker/3:1+events22:09:413
320894799238712,2372cyclictest3216870-21kworker/3:1+events19:43:423
320894799238711,2373cyclictest3284739-21kworker/3:0+events23:08:463
320894799238512,2369cyclictest3239810-21kworker/3:1+events20:44:093
320894799238311,2370cyclictest3256023-21kworker/3:0+events21:40:053
32089379923839,2370cyclictest3234869-21kworker/0:0+events20:28:580
320894799237912,2363cyclictest3200315-21kworker/3:2+events19:28:293
320894799237612,2360cyclictest3276666-21kworker/3:0+events22:41:363
320894499237612,2360cyclictest3307671-21kworker/2:2+events00:19:532
320894499237611,2361cyclictest3278254-21kworker/2:1+events22:44:412
320894799237512,2359cyclictest3243272-21kworker/3:1+events21:01:393
320894499237511,2360cyclictest3294602-21kworker/2:0+events23:44:172
320894799236812,2354cyclictest3231734-21kworker/3:1+events20:25:043
320894799236611,2352cyclictest3294541-21kworker/3:2+events23:31:443
320894499236512,2350cyclictest3259006-21kworker/2:1+events21:58:222
320894799236412,2349cyclictest3281524-21kworker/3:2+events22:58:513
320894799236311,2348cyclictest3299428-21kworker/3:0+events00:03:233
320894099236212,2348cyclictest3213725-21kworker/1:1+events19:56:591
320894799236112,2345cyclictest3246647-21kworker/3:2+events21:29:323
320894099236112,2346cyclictest3281461-21kworker/1:2+events22:57:181
320894499235912,2344cyclictest3310404-21kworker/2:1+events00:27:112
320893799235911,2345cyclictest3225222-21kworker/0:2+events20:04:360
320894799235812,2343cyclictest3279887-21kworker/3:1+events22:49:403
320894799235811,2343cyclictest3216870-21kworker/3:1+events19:46:163
320894799235711,2343cyclictest3268978-21kworker/3:2+events22:22:403
320894499235311,2339cyclictest3270405-21kworker/2:1+events22:22:172
32089449923499,2337cyclictest3270405-21kworker/2:1+events22:25:212
320893799234811,2333cyclictest3226841-21kworker/0:1+events20:06:310
320893799234811,2333cyclictest3226841-21kworker/0:1+events20:06:310
320894799234710,2334cyclictest3243272-21kworker/3:1+events20:54:213
320894499234711,2334cyclictest3249487-21kworker/2:0+events21:15:092
320894799234412,2329cyclictest3289864-21kworker/3:1+events23:23:423
320893799234212,2326cyclictest3212162-21kworker/0:0+events19:18:350
32089449923417,2330cyclictest3287759-21kworker/2:1+events23:33:142
320894499234012,2325cyclictest3230164-21kworker/2:1+events20:24:402
320894499234012,2325cyclictest3216993-21kworker/2:1+events19:41:022
320894499234011,2326cyclictest3231773-21kworker/2:0+events20:19:462
320894499233912,2324cyclictest3233441-21kworker/2:2+events20:25:502
320894499233912,2323cyclictest3212097-21kworker/2:0+events19:37:112
320894499233812,2323cyclictest3257105-21kworker/2:0+events21:41:112
320894499233811,2323cyclictest3296200-21kworker/2:2+events23:37:002
320894499233710,2324cyclictest3302550-21kworker/2:1+events00:09:092
32089379923379,2325cyclictest3220005-21kworker/0:2+events19:44:270
320894499233610,2322cyclictest3273856-21kworker/2:2+events22:31:272
320894099233112,2316cyclictest3267297-21kworker/1:2+events22:16:511
32089379923299,2317cyclictest3283173-21kworker/0:0+events23:04:590
320894499232812,2313cyclictest3241530-21kworker/2:2+events20:51:172
32089479923259,2312cyclictest3259565-21kworker/3:1+events21:58:463
320894099232412,2309cyclictest3220363-21kworker/1:2+events19:46:411
320894099232410,2310cyclictest3307440-21kworker/1:0+events00:27:341
320894799232212,2307cyclictest3312465-21kworker/3:2+events00:31:213
320893799232111,2306cyclictest3307470-21kworker/0:1+events00:21:030
320894099232012,2304cyclictest3228326-21kworker/1:0+events20:45:161
320893799231711,2305cyclictest3302645-21kworker/0:0+events00:01:510
320894099231611,2302cyclictest3208448-21kworker/1:0+events19:31:111
32089379923158,2304cyclictest3215475-21kworker/0:0+events19:35:400
320894099231412,2299cyclictest3270911-21kworker/1:1+events22:27:381
320894499230711,2293cyclictest3243392-21kworker/2:1+events21:06:522
320894099230511,2291cyclictest3208448-21kworker/1:0+events19:22:001
320894099230412,2289cyclictest3211041-21kworker/1:2+events19:19:201
32089449923039,2292cyclictest3230164-21kworker/2:1+events20:30:062
320894799229911,2284cyclictest3254333-21kworker/3:1+events21:33:363
320894499229711,2282cyclictest3276636-21kworker/2:0+events22:48:312
320894099228711,2273cyclictest3281461-21kworker/1:2+events23:01:561
320893799228012,2266cyclictest3310747-21kworker/0:2+events00:30:130
320893799227412,2258cyclictest3259243-21kworker/0:1+events21:45:220
320894799227310,2259cyclictest3302675-21kworker/3:1+events23:58:523
320894099227011,2256cyclictest3213725-21kworker/1:1+events19:52:231
320894499226811,2254cyclictest3230164-21kworker/2:1+events20:14:082
320894099226711,2252cyclictest3254367-21kworker/1:0+events21:34:241
32089479922647,2253cyclictest3272219-21kworker/3:0+events22:26:523
320894099226211,2248cyclictest3257591-21kworker/1:2+events21:43:261
320894099226111,2246cyclictest3293368-21kworker/1:1+events23:38:091
320894099226110,2248cyclictest3275220-21kworker/1:0+events22:35:141
320894099226010,2246cyclictest3243732-21kworker/1:1+events21:00:311
320894099225810,2245cyclictest3307440-21kworker/1:0+events00:23:451
320894099225810,2245cyclictest3301270-21kworker/1:2+events00:07:361
320894099225611,2242cyclictest3293368-21kworker/1:1+events23:53:301
320894099225411,2239cyclictest3257591-21kworker/1:2+events21:49:111
32089409922539,2241cyclictest3247948-21kworker/1:2+events21:13:391
320894099225311,2238cyclictest3279856-21kworker/1:1+events22:47:431
320894099225111,2237cyclictest3235101-21kworker/1:1+events20:31:151
320894099225111,2237cyclictest3216965-21kworker/1:2+events19:36:261
320894099225011,2236cyclictest3228326-21kworker/1:0+events20:11:051
320894099225010,2237cyclictest3239870-21kworker/1:1+events20:43:441
320894099224810,2235cyclictest3257591-21kworker/1:2+events21:56:271
320894099224810,2234cyclictest3246286-21kworker/1:0+events21:06:301
320894799224312,2228cyclictest3225820-21kworker/3:1+events20:03:063
320894099223810,2226cyclictest3208448-21kworker/1:0+events19:25:041
320893799223810,2225cyclictest3223527-21kworker/0:0+events19:59:160
320893799223710,2224cyclictest3223527-21kworker/0:0+events19:53:550
320893799223610,2222cyclictest3294813-21kworker/0:1+events23:43:540
320893799223511,2221cyclictest3251054-21kworker/0:2+events21:16:410
32089379922339,2221cyclictest3248095-21kworker/0:0+events21:09:050
320893799223311,2219cyclictest3291510-21kworker/0:2+events23:34:000
320893799223211,2218cyclictest3259243-21kworker/0:1+events22:04:050
320893799223210,2219cyclictest3307470-21kworker/0:1+events00:16:280
32089479922319,2219cyclictest3312465-21kworker/3:2+events00:29:073
320893799223110,2218cyclictest3267330-21kworker/0:0+events22:29:340
320894799223012,2214cyclictest3265732-21kworker/3:0+events22:13:233
320893799222911,2214cyclictest3248095-21kworker/0:0+events21:32:300
320893799222910,2216cyclictest3289620-21kworker/0:0+events23:19:250
320893799222910,2215cyclictest3246502-21kworker/0:1+events21:13:150
320894499222811,2215cyclictest3235051-21kworker/2:0+events20:35:032
320893799222811,2214cyclictest3243238-21kworker/0:0+events21:02:010
320893799222810,2215cyclictest3251054-21kworker/0:2+events21:20:250
32089409922263,2219cyclictest3228326-21kworker/1:0+events20:09:101
32089409922263,2219cyclictest3228326-21kworker/1:0+events20:09:101
320893799222610,2212cyclictest3281491-21kworker/0:1+events23:21:210
320894099222411,2210cyclictest3267297-21kworker/1:2+events22:13:011
32089379922239,2212cyclictest3275249-21kworker/0:1+events22:45:030
320894499222111,2206cyclictest3262505-21kworker/2:0+events21:53:272
320893799222111,2207cyclictest3208121-21kworker/0:2+events19:29:140
320893799221711,2203cyclictest3261409-21kworker/0:2+events22:12:400
320894099221610,2202cyclictest3301270-21kworker/1:2+events00:02:391
320894099221510,2202cyclictest3298036-21kworker/1:0+events23:57:411
32089479922149,2201cyclictest3200315-21kworker/3:2+events19:31:553
32089379922129,2199cyclictest3231882-21kworker/0:2+events20:34:390
320893799221211,2198cyclictest3208121-21kworker/0:2+events19:14:460
32089379922099,2197cyclictest3312648-21kworker/0:1+events00:35:330
32089379922089,2196cyclictest3220005-21kworker/0:2+events19:47:260
320893799220810,2196cyclictest3226841-21kworker/0:1+events20:20:090
320894099220211,2187cyclictest3254367-21kworker/1:0+events21:38:311
32089449921987,2187cyclictest3265876-21kworker/2:0+events22:05:322
320893799219710,2183cyclictest3231882-21kworker/0:2+events20:52:020
320894799218911,2175cyclictest3305983-21kworker/3:0+events00:14:333
320894799218610,2172cyclictest3243272-21kworker/3:1+events21:07:143
320894799218311,2169cyclictest3291996-21kworker/3:0+events23:35:553
320894499218110,2167cyclictest3224950-21kworker/2:0+events19:58:332
320894499217911,2165cyclictest3287759-21kworker/2:1+events23:23:182
320894799217811,2163cyclictest3304350-21kworker/3:2+events00:09:543
320894499217812,2162cyclictest3276636-21kworker/2:0+events22:54:162
320894799216911,2155cyclictest3296145-21kworker/3:1+events23:47:203
320894799216911,2155cyclictest3210298-21kworker/3:1+events19:23:323
32089379921679,2155cyclictest3226841-21kworker/0:1+events20:17:520
320894799216611,2152cyclictest3252735-21kworker/3:0+events21:23:063
320894799216510,2152cyclictest3291996-21kworker/3:0+events23:27:333
320894799216510,2152cyclictest3241409-21kworker/3:2+events20:47:543
320894799216411,2150cyclictest3210298-21kworker/3:1+events19:18:133
320894799216410,2150cyclictest3241409-21kworker/3:2+events20:57:263
32089449921649,2152cyclictest3222222-21kworker/2:1+events19:51:382
320894799216311,2149cyclictest3296145-21kworker/3:1+events23:41:363
320894799216210,2149cyclictest3254333-21kworker/3:1+events21:37:253
320894499215713,2141cyclictest3287759-21kworker/2:1+events23:26:252
32089479921569,2144cyclictest3220186-21kworker/3:2+events20:15:153
320894499215411,2139cyclictest3305925-21kworker/2:0+events00:24:092
320894099215311,2138cyclictest3270911-21kworker/1:1+events22:32:591
320894799215211,2138cyclictest3231734-21kworker/3:1+events20:35:263
320894499215210,2139cyclictest3243392-21kworker/2:1+events21:14:002
320894499215111,2137cyclictest3276636-21kworker/2:0+events23:15:592
320894499215111,2137cyclictest3270405-21kworker/2:1+events22:19:342
32089449921508,2139cyclictest3251296-21kworker/2:2+events21:21:322
320893799214410,2131cyclictest3267330-21kworker/0:0+events22:23:020
320894499214211,2127cyclictest3243392-21kworker/2:1+events21:03:312
320894499214111,2127cyclictest3310404-21kworker/2:1+events00:38:102
320894499214110,2128cyclictest3294602-21kworker/2:0+events00:01:072
320894499214110,2128cyclictest3259006-21kworker/2:1+events22:01:242
320894499214011,2126cyclictest3300888-21kworker/2:2+events23:51:592
320894499214011,2126cyclictest3267473-21kworker/2:2+events22:10:452
320894499213911,2125cyclictest3305925-21kworker/2:0+events00:10:402
32089479921389,2126cyclictest3304350-21kworker/3:2+events00:15:433
320894799213811,2123cyclictest3220186-21kworker/3:2+events20:20:333
320894499213710,2124cyclictest3260897-21kworker/2:2+events21:47:392
320894499213710,2124cyclictest3251296-21kworker/2:2+events21:32:072
32089449921358,2124cyclictest3281658-21kworker/2:1+events22:55:232
320894099213110,2118cyclictest3276903-21kworker/1:2+events22:43:541
320894499213011,2116cyclictest3228297-21kworker/2:0+events20:08:462
320894499213011,2116cyclictest3228297-21kworker/2:0+events20:08:452
320894099212011,2106cyclictest3293368-21kworker/1:1+events23:48:071
320894099211710,2104cyclictest3265668-21kworker/1:1+events22:03:191
320894499211611,2102cyclictest3212097-21kworker/2:0+events19:48:342
320894099211410,2101cyclictest3208448-21kworker/1:0+events19:13:391
320894499211012,2095cyclictest3257105-21kworker/2:0+events21:36:402
32089449921089,2096cyclictest3284555-21kworker/2:2+events23:05:442
320894499210610,2093cyclictest3239778-21kworker/2:0+events20:48:392
32089449921019,2089cyclictest3252943-21kworker/2:0+events21:26:062
32089409920999,2087cyclictest3301270-21kworker/1:2+events00:11:501
320894099209811,2084cyclictest3249691-21kworker/1:1+events21:27:371
320894799209511,2081cyclictest3259565-21kworker/3:1+events21:52:403
320894099209411,2080cyclictest3228326-21kworker/1:0+events20:23:541
320894099209311,2079cyclictest3225040-21kworker/1:2+events20:03:511
32089409920929,2080cyclictest3241842-21kworker/1:2+events20:52:491
320893799208710,2074cyclictest3272069-21kworker/0:2+events22:37:310
320893799208210,2069cyclictest3269099-21kworker/0:1+events22:18:460
32089449920723,2066cyclictest3312371-21kworker/2:2+events00:32:062
32089379920718,2060cyclictest3231882-21kworker/0:2+events20:37:410
320894099206110,2048cyclictest3241842-21kworker/1:2+events20:56:391
320894499204910,2036cyclictest3273856-21kworker/2:2+events22:35:592
32089479920338,2022cyclictest3230012-21kworker/3:0+events20:13:453
320894799201710,2004cyclictest3262538-21kworker/3:2+events22:02:563
32089479920139,2000cyclictest3222104-21kworker/3:0+events19:54:173
32089479920129,2000cyclictest3200315-21kworker/3:2+events19:14:023
320894799200010,1987cyclictest3310588-21kworker/3:0+events00:24:313
32089449919969,1984cyclictest3241530-21kworker/2:2+events20:55:542
320894499199510,1982cyclictest3212097-21kworker/2:0+events19:33:262
320894499199410,1982cyclictest3276636-21kworker/2:0+events23:00:462
320894499198910,1977cyclictest3209059-21kworker/2:1+events19:16:422
32089449919823,1975cyclictest3212097-21kworker/2:0+events19:26:132
32089449919819,1969cyclictest3222222-21kworker/2:1+events20:00:472
32089379919716,1962cyclictest3286398-21kworker/0:2+events23:13:190
320894499196910,1956cyclictest3294602-21kworker/2:0+events23:56:562
320894099193510,1922cyclictest3233382-21kworker/1:2+events20:25:271
32089449917997,1789cyclictest3206878-21kworker/2:2+events19:10:162
32081842179151,23sleep30-21swapper/319:06:253
32082462144106,32sleep00-21swapper/019:07:120
32081332142111,26sleep20-21swapper/219:05:452
32083412138105,28sleep10-21swapper/119:08:251
320894099160,5cyclictest3293034-21taskset23:26:191
320894099154,2cyclictest191rcu_preempt23:20:231
320894499148,4cyclictest3213223-21ld19:20:212
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional