You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-16 - 08:41

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #f, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails
  
Click here to display the system's profile data.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackfslot7s.osadl.org (updated Sun Nov 16, 2025 00:44:17)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
120669799365217,3629cyclictest1237403-21kworker/1:2+events21:05:011
120669999283513,2818cyclictest1277326-21kworker/2:0+events22:50:172
120669799281313,2795cyclictest1271194-21kworker/1:2+events22:45:191
12067039927568,2745cyclictest1295675-21kworker/3:2+events23:45:003
120669399273313,2717cyclictest1234277-21kworker/0:1+events20:35:200
120669399273112,2715cyclictest1290852-21kworker/0:0+events23:30:160
12066999926579,2644cyclictest1213266-21kworker/2:1+events19:30:172
120669399263413,2618cyclictest1264504-21kworker/0:0+events22:30:260
120669799257312,2558cyclictest1308289-21kworker/1:0+events00:34:061
120669799256012,2544cyclictest1245107-21kworker/1:1+events21:10:201
12066979925515,2543cyclictest1237403-21kworker/1:2+events20:51:381
120669799254812,2533cyclictest1229290-21kworker/1:0+events20:43:381
120669799254812,2533cyclictest1229290-21kworker/1:0+events20:43:381
120669999250912,2494cyclictest1269759-21kworker/2:2+events23:00:162
120670399250811,2494cyclictest1256755-21kworker/3:0+events21:47:313
120669399250712,2492cyclictest1300216-21kworker/0:0+events00:00:230
120669799249812,2482cyclictest1264362-21kworker/1:1+events22:09:301
120669799249412,2479cyclictest1254834-21kworker/1:0+events21:42:361
120669799249412,2479cyclictest1221370-21kworker/1:2+events20:20:531
120669399249212,2477cyclictest1264504-21kworker/0:0+events22:25:290
120669399249012,2476cyclictest1279084-21kworker/0:2+events23:05:060
12066979924839,2471cyclictest1306630-21kworker/1:2+events00:22:221
120669399247310,2460cyclictest1274130-21kworker/0:0+events22:56:240
120669799245810,2444cyclictest1214781-21kworker/1:2+events19:36:051
12067039924425,2434cyclictest1237432-21kworker/3:2+events20:48:563
120670399244212,2426cyclictest1222933-21kworker/3:1+events20:04:543
120670399243210,2419cyclictest1290699-21kworker/3:1+events23:38:123
120669799242513,2409cyclictest1264362-21kworker/1:1+events22:11:461
120669799242412,2409cyclictest1221370-21kworker/1:2+events20:05:191
120670399242312,2408cyclictest1300274-21kworker/3:2+events00:24:173
120670399242311,2409cyclictest1224842-21kworker/3:0+events20:52:013
120670399242111,2406cyclictest1297339-21kworker/3:1+events23:54:393
120670399242111,2406cyclictest1285371-21kworker/3:0+events23:25:193
120670399241710,2404cyclictest1292393-21kworker/3:0+events23:47:403
120669799241112,2395cyclictest1239066-21kworker/1:1+events20:58:101
120669399240912,2393cyclictest1229348-21kworker/0:2+events20:40:140
120669399240912,2393cyclictest1229348-21kworker/0:2+events20:40:140
120670399240112,2385cyclictest1292393-21kworker/3:0+events23:33:183
12066939924009,2389cyclictest1264504-21kworker/0:0+events22:20:080
120670399239311,2379cyclictest1264593-21kworker/3:1+events22:09:513
120670399237310,2360cyclictest1300274-21kworker/3:2+events00:13:133
120669799236811,2354cyclictest1245107-21kworker/1:1+events21:53:301
120670399236610,2352cyclictest1268055-21kworker/3:1+events22:24:433
120669799236612,2350cyclictest1271194-21kworker/1:2+events22:34:381
120669799236210,2348cyclictest1295708-21kworker/1:2+events23:46:311
120669799233711,2323cyclictest1300585-21kworker/1:3+events00:03:041
120669799233710,2324cyclictest1274403-21kworker/1:1+events22:42:581
120669999233512,2320cyclictest1301873-21kworker/2:0+events00:09:492
120669799233512,2320cyclictest1280536-21kworker/1:1+events23:01:021
120669399233310,2319cyclictest1262541-21kworker/0:2+events22:05:390
120669799233112,2315cyclictest1245107-21kworker/1:1+events21:20:331
120669799233012,2315cyclictest1267903-21kworker/1:0+events22:22:051
120669799232612,2311cyclictest1282144-21kworker/1:2+events23:07:231
120669799232512,2310cyclictest1277296-21kworker/1:0+events22:54:521
120669399232312,2308cyclictest1260924-21kworker/0:1+events22:10:380
120669399232012,2305cyclictest1222991-21kworker/0:1+events20:07:350
120669799231812,2303cyclictest1280536-21kworker/1:1+events22:58:431
120669999231710,2304cyclictest1292486-21kworker/2:0+events23:36:422
120669799231712,2302cyclictest1271194-21kworker/1:2+events22:39:581
120669799231710,2304cyclictest1227691-21kworker/1:1+events20:14:261
120670399231610,2302cyclictest1283790-21kworker/3:1+events23:18:023
120669799231611,2302cyclictest1245107-21kworker/1:1+events21:29:391
120669799231512,2302cyclictest1306630-21kworker/1:2+events00:25:271
120669399231512,2300cyclictest1254709-21kworker/0:2+events21:52:020
12066999923107,2300cyclictest1264408-21kworker/2:0+events22:11:232
120669799231012,2295cyclictest1285414-21kworker/1:2+events23:15:181
120669999230712,2292cyclictest1213266-21kworker/2:1+events19:39:042
120669399229112,2276cyclictest1262541-21kworker/0:2+events22:02:330
120670399229010,2277cyclictest1288840-21kworker/3:2+events23:24:343
120669399228812,2273cyclictest1239099-21kworker/0:0+events21:11:510
120669399228810,2275cyclictest1290852-21kworker/0:0+events23:28:470
12067039922836,2274cyclictest1235757-21kworker/3:1+events20:41:003
12067039922836,2274cyclictest1235757-21kworker/3:1+events20:40:593
120669799227111,2257cyclictest1262514-21kworker/1:2+events22:03:411
120670399226911,2255cyclictest1264593-21kworker/3:1+events22:13:383
120669799226910,2256cyclictest1245107-21kworker/1:1+events21:49:451
120670399226711,2253cyclictest1216414-21kworker/3:1+events19:47:063
120669799226710,2253cyclictest1295708-21kworker/1:2+events23:51:551
120669399226711,2253cyclictest1215023-21kworker/0:0+events19:36:270
120669399226211,2248cyclictest1283607-21kworker/0:0+events23:13:480
120669399226211,2248cyclictest1283607-21kworker/0:0+events23:13:480
120669799226111,2247cyclictest1229290-21kworker/1:0+events20:33:481
120670399225811,2244cyclictest1248521-21kworker/3:0+events21:27:003
120669399225810,2245cyclictest1234277-21kworker/0:1+events20:59:420
120669399225710,2244cyclictest1287102-21kworker/0:1+events23:56:310
120669399225611,2242cyclictest1206110-21kworker/0:1+events19:48:130
120669399225511,2240cyclictest1309994-21kworker/0:0+events00:34:530
120669399225511,2240cyclictest1254709-21kworker/0:2+events21:38:030
120669799225311,2238cyclictest1219765-21kworker/1:1+events19:51:181
120669399225311,2239cyclictest1239099-21kworker/0:0+events21:00:290
120669399225211,2239cyclictest1229348-21kworker/0:2+events20:30:030
120669399225111,2237cyclictest1239099-21kworker/0:0+events20:53:100
120669399225010,2237cyclictest1239099-21kworker/0:0+events21:18:140
12066939922476,2238cyclictest1210067-21kworker/0:2+events19:21:100
120670399224110,2228cyclictest1214749-21kworker/3:0+events19:50:333
12066939922368,2225cyclictest1216626-21kworker/0:2+events19:44:240
120669799223312,2218cyclictest1308289-21kworker/1:0+events00:37:321
120670399222710,2213cyclictest1302176-21kworker/3:3+events00:07:103
120670399222510,2212cyclictest1292393-21kworker/3:0+events00:04:133
120670399222510,2212cyclictest1251744-21kworker/3:2+events21:35:213
120670399222411,2210cyclictest1251744-21kworker/3:2+events21:44:303
120670399222311,2209cyclictest1272583-21kworker/3:1+events22:33:293
120670399222310,2210cyclictest1283790-21kworker/3:1+events23:13:273
120670399222310,2210cyclictest1283790-21kworker/3:1+events23:13:273
120669799222212,2207cyclictest1218097-21kworker/1:0+events19:58:511
120669799222211,2207cyclictest1229290-21kworker/1:0+events20:25:491
120670399222111,2207cyclictest1230888-21kworker/3:1+events20:22:243
120669399222110,2208cyclictest1298615-21kworker/0:2+events00:06:030
120670399222010,2207cyclictest1279210-21kworker/3:3+events23:01:243
120670399222010,2207cyclictest1224842-21kworker/3:0+events20:55:313
120670399221910,2206cyclictest1300274-21kworker/3:2+events23:58:033
120670399221910,2206cyclictest1274252-21kworker/3:0+events22:43:223
120670399221910,2206cyclictest1230888-21kworker/3:1+events20:25:243
120670399221811,2203cyclictest1224842-21kworker/3:0+events21:09:093
120670399221810,2204cyclictest1274252-21kworker/3:0+events22:38:503
120670399221712,2202cyclictest1300274-21kworker/3:2+events00:27:243
120670399221711,2203cyclictest1216414-21kworker/3:1+events19:37:543
120670399221311,2200cyclictest1277481-21kworker/3:2+events22:50:413
120669799220711,2193cyclictest1248309-21kworker/1:2+events21:17:301
120670399220611,2192cyclictest1245224-21kworker/3:2+events21:17:523
120670399220210,2189cyclictest1214749-21kworker/3:0+events19:55:523
120669799220210,2189cyclictest1208088-21kworker/1:1+events19:31:041
120670399220010,2187cyclictest1248521-21kworker/3:0+events21:24:433
120669399219911,2184cyclictest1206110-21kworker/0:1+events19:29:330
12067039921988,2187cyclictest1211875-21kworker/3:2+events19:30:423
120670399219711,2183cyclictest1266448-21kworker/3:0+events22:15:113
12067039921968,2185cyclictest1211875-21kworker/3:2+events19:25:243
120669999219612,2181cyclictest1259750-21kworker/2:2+events21:56:322
120670399219411,2180cyclictest1256755-21kworker/3:0+events21:50:303
12066999921887,2178cyclictest1208302-21kworker/2:3+events19:20:272
120670399218711,2173cyclictest1232549-21kworker/3:2+events20:32:203
12067039921858,2174cyclictest1275874-21kworker/3:1+events22:48:453
120669799218011,2166cyclictest1298914-21kworker/1:1+events00:13:581
120670399217911,2165cyclictest1311438-21kworker/3:1+events00:32:593
120670399217710,2163cyclictest1224842-21kworker/3:0+events20:16:423
12066939921719,2159cyclictest1283607-21kworker/0:0+events23:16:270
120669799216410,2150cyclictest1267903-21kworker/1:0+events22:19:001
120669999215811,2144cyclictest1254634-21kworker/2:1+events22:21:182
120669999215810,2145cyclictest1280600-21kworker/2:0+events23:19:352
120669999215411,2140cyclictest1240611-21kworker/2:1+events21:06:552
120670399215210,2139cyclictest1243508-21kworker/3:1+events21:03:503
120670399215011,2136cyclictest1300274-21kworker/3:2+events00:36:003
120669999214912,2134cyclictest1272645-21kworker/2:0+events22:37:172
120669999214510,2131cyclictest1216595-21kworker/2:0+events19:40:152
120669399214111,2127cyclictest1311646-21kworker/0:2+events00:36:460
12066939921335,2125cyclictest1287102-21kworker/0:1+events23:35:550
120669999212911,2115cyclictest1227722-21kworker/2:2+events20:26:592
120669799212611,2112cyclictest1237403-21kworker/1:2+events20:47:481
120669799212512,2110cyclictest1295708-21kworker/1:2+events23:56:531
12066999921205,2112cyclictest1227722-21kworker/2:2+events20:36:302
120669399212010,2107cyclictest1281987-21kworker/0:1+events23:03:370
120669999211711,2104cyclictest1213266-21kworker/2:1+events19:45:352
120669999211611,2102cyclictest1280600-21kworker/2:0+events22:59:292
120669999211611,2102cyclictest1257968-21kworker/2:0+events21:49:022
12066999921148,2103cyclictest1287287-21kworker/2:2+events23:43:342
120669999211411,2101cyclictest1240611-21kworker/2:1+events20:55:062
120669999211411,2100cyclictest1305372-21kworker/2:2+events00:19:212
12066939921139,2101cyclictest1206110-21kworker/0:1+events19:18:330
12066999921109,2097cyclictest1289234-21kworker/2:1+events23:29:532
120669799210910,2096cyclictest1208088-21kworker/1:1+events19:41:001
120669399210910,2096cyclictest1253108-21kworker/0:1+events21:42:140
120669999210810,2095cyclictest1308258-21kworker/2:1+events00:27:462
120669999210711,2093cyclictest1303726-21kworker/2:1+events00:10:572
120669399210711,2093cyclictest1287102-21kworker/0:1+events23:51:100
12067039921068,2095cyclictest1246920-21kworker/3:1+events21:14:553
12066999921069,2094cyclictest1240611-21kworker/2:1+events21:03:072
120669399210610,2093cyclictest1239099-21kworker/0:0+events21:24:210
120669399210610,2093cyclictest1227661-21kworker/0:0+events20:17:030
120669399210610,2093cyclictest1222991-21kworker/0:1+events20:12:330
120670399210510,2092cyclictest1248521-21kworker/3:0+events21:32:413
120669799210511,2090cyclictest1285414-21kworker/1:2+events23:24:121
120669399210511,2091cyclictest1285208-21kworker/0:2+events23:24:570
120670399210410,2091cyclictest1206169-21kworker/3:1+events19:22:203
120669999210411,2090cyclictest1254634-21kworker/2:1+events22:06:482
120669999210410,2092cyclictest1243688-21kworker/2:2+events21:12:142
12066999921039,2091cyclictest1308258-21kworker/2:1+events00:39:252
120669999210310,2090cyclictest1227722-21kworker/2:2+events20:18:092
120669799210311,2089cyclictest1290730-21kworker/1:0+events23:31:221
120669799210311,2089cyclictest1280536-21kworker/1:1+events23:11:111
120669799210311,2089cyclictest1280536-21kworker/1:1+events23:11:111
12066939921039,2091cyclictest1295071-21kworker/0:0+events23:45:240
120669399210311,2089cyclictest1306906-21kworker/0:1+events00:29:570
120669999210211,2088cyclictest1266269-21kworker/2:2+events22:16:432
12066979921027,2092cyclictest1306630-21kworker/1:2+events00:17:491
12066939921019,2089cyclictest1218518-21kworker/0:0+events19:59:570
120669399210111,2087cyclictest1229348-21kworker/0:2+events20:22:460
120669799210013,2085cyclictest1210414-21kworker/1:2+events19:25:011
120669799210011,2086cyclictest1245107-21kworker/1:1+events21:58:251
120669799210011,2085cyclictest1294055-21kworker/1:0+events23:44:181
120669399210010,2087cyclictest1298615-21kworker/0:2+events00:13:360
12066979920997,2088cyclictest1200014-21kworker/1:0+events19:17:021
120669799209910,2086cyclictest1245107-21kworker/1:1+events21:33:041
120669399209910,2086cyclictest1295071-21kworker/0:0+events23:42:010
12066939920989,2086cyclictest1229348-21kworker/0:2+events20:49:200
120669399209811,2084cyclictest1227661-21kworker/0:0+events20:28:310
120669999209711,2084cyclictest1287287-21kworker/2:2+events23:56:102
120669399209711,2082cyclictest1239099-21kworker/0:0+events21:32:190
120669799209611,2082cyclictest1280536-21kworker/1:1+events23:26:271
12066939920968,2085cyclictest1266478-21kworker/0:2+events22:15:350
12066939920967,2086cyclictest1222991-21kworker/0:1+events20:00:440
120669399209610,2083cyclictest1274130-21kworker/0:0+events22:54:050
12066999920953,2089cyclictest1238795-21kworker/2:2+events20:47:252
12066999920949,2082cyclictest1301873-21kworker/2:0+events00:21:582
120669799209310,2080cyclictest1218097-21kworker/1:0+events19:45:581
120669799209210,2079cyclictest1267903-21kworker/1:0+events22:25:511
12066939920919,2079cyclictest1274130-21kworker/0:0+events22:47:120
120669999209010,2077cyclictest1297309-21kworker/2:0+events23:53:062
12066939920896,2080cyclictest1256551-21kworker/0:0+events21:58:020
12066939920888,2077cyclictest1245076-21kworker/0:1+events21:09:340
120669999208511,2071cyclictest1280600-21kworker/2:0+events23:13:032
120669999208511,2071cyclictest1280600-21kworker/2:0+events23:13:032
120669399208511,2071cyclictest1215023-21kworker/0:0+events19:34:540
120669399208010,2067cyclictest1275782-21kworker/0:1+events22:43:450
120669799207911,2065cyclictest1242244-21kworker/1:0+events21:08:471
12066939920788,2067cyclictest1270474-21kworker/0:2+events22:36:330
12066999920778,2066cyclictest1235991-21kworker/2:0+events20:54:212
12066999920729,2060cyclictest1256246-21kworker/2:2+events21:43:422
120670399207010,2057cyclictest1282045-21kworker/3:0+events23:05:303
120670399206910,2056cyclictest1261040-21kworker/3:1+events22:02:113
120670399206910,2056cyclictest1261040-21kworker/3:1+events21:59:093
12067039920686,2060cyclictest1224842-21kworker/3:0+events20:10:143
12067039920679,2055cyclictest1224842-21kworker/3:0+events20:37:593
12067039920676,2059cyclictest1302176-21kworker/3:3+events00:15:083
12067039920659,2053cyclictest1275874-21kworker/3:1+events22:59:053
12066979920639,2051cyclictest1221370-21kworker/1:2+events20:01:521
120669799206310,2050cyclictest1298914-21kworker/1:1+events00:08:181
120669999205810,2044cyclictest1269759-21kworker/2:2+events22:28:082
12067039920559,2043cyclictest1216414-21kworker/3:1+events19:40:373
12067039920526,2043cyclictest1269725-21kworker/3:2+events22:27:443
120669399205010,2037cyclictest1254709-21kworker/0:2+events21:46:010
120669799204410,2031cyclictest1221370-21kworker/1:2+events20:18:311
12066979920347,2024cyclictest1206408-21kworker/1:2+events19:11:391
12066939920329,2020cyclictest1206110-21kworker/0:1+events19:14:400
12066939920308,2019cyclictest1298615-21kworker/0:2+events00:20:060
12066939920228,2010cyclictest1305208-21kworker/0:0+events00:18:340
12067039920157,2005cyclictest1206169-21kworker/3:1+events19:18:083
120669999200811,1993cyclictest1213266-21kworker/2:1+events19:50:082
12066979920008,1989cyclictest1229290-21kworker/1:0+events20:36:051
12066999919948,1983cyclictest1274371-21kworker/2:1+events22:40:432
120669999199410,1981cyclictest1254634-21kworker/2:1+events21:51:382
120670399199310,1980cyclictest1206169-21kworker/3:1+events19:13:523
120669799197611,1961cyclictest1294055-21kworker/1:0+events23:39:201
12066999919627,1952cyclictest1240611-21kworker/2:1+events21:25:302
120669999196210,1949cyclictest1232520-21kworker/2:0+events20:31:362
12066999919609,1948cyclictest1234159-21kworker/2:1+events20:43:152
12066999919609,1948cyclictest1234159-21kworker/2:1+events20:43:142
12066999919603,1954cyclictest1269759-21kworker/2:2+events22:46:502
12066999919599,1949cyclictest1269759-21kworker/2:2+events22:32:422
12066999919599,1947cyclictest1253048-21kworker/2:0+events21:36:312
120669999195910,1946cyclictest1240611-21kworker/2:1+events21:23:332
12066999919589,1946cyclictest1225112-21kworker/2:0+events20:11:242
120669999195210,1939cyclictest1211004-21kworker/2:0+events19:26:542
12066999919519,1939cyclictest1229442-21kworker/2:1+events20:22:002
120669999195110,1938cyclictest1208302-21kworker/2:3+events19:18:562
120669999195110,1938cyclictest1198384-21kworker/2:0+events19:13:292
120669999195010,1937cyclictest1251141-21kworker/2:2+events21:31:342
12066939919488,1937cyclictest1218518-21kworker/0:0+events19:52:510
12066999919476,1938cyclictest1287287-21kworker/2:2+events23:49:372
12066999919474,1940cyclictest1213266-21kworker/2:1+events20:06:492
12066999919466,1937cyclictest1287287-21kworker/2:2+events23:33:412
120669999194610,1933cyclictest1287287-21kworker/2:2+events00:01:542
12066979919444,1937cyclictest1245107-21kworker/1:1+events21:36:081
120669799194310,1930cyclictest1210414-21kworker/1:2+events19:22:431
12066999919414,1934cyclictest1309903-21kworker/2:2+events00:30:432
12066939919019,1889cyclictest1250112-21kworker/0:2+events21:27:240
12061292185154,26sleep20-21swapper/219:06:042
12063462141110,26sleep10-21swapper/119:08:471
12062382141109,26sleep00-21swapper/019:07:270
12062412140106,28sleep30-21swapper/319:07:303
1206699992621,3cyclictest1248618-21idleruntime-cro21:20:012
120669999204,6cyclictest1224939-21latency_hist20:05:002
120669999193,8cyclictest1261659-21cstates22:00:162
120669999164,3cyclictest1283149-21ld23:05:162
1206699991613,2cyclictest608-21systemd-logind23:25:002
120669999160,6cyclictest1223496-21cat20:00:002
1206703991511,2cyclictest0-21swapper/320:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional