You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-04 - 05:02

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #f, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails
  
Click here to display the system's profile data.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackfslot7s.osadl.org (updated Thu Dec 04, 2025 00:44:16)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
133137199301015,2991cyclictest1381916-21kworker/0:0+events21:45:000
13313769927745,2764cyclictest1373852-21kworker/2:1+events21:25:122
133137499268412,2668cyclictest1407701-21kworker/1:0+events23:10:171
133138399267712,2661cyclictest1414120-21kworker/3:2+events23:25:153
133137699266113,2645cyclictest1371144-21kworker/2:0+events21:10:152
133137699258012,2565cyclictest1358125-21kworker/2:2+events20:40:152
13313839925309,2517cyclictest1414120-21kworker/3:2+events23:35:193
133137199251011,2494cyclictest1372275-21kworker/0:1+events21:15:170
133137699249812,2482cyclictest1356526-21kworker/2:1+events20:34:122
133137499249812,2482cyclictest1350113-21kworker/1:2+events20:05:181
133137499249612,2480cyclictest1351751-21kworker/1:0+events20:10:181
133137199249612,2480cyclictest1428303-21kworker/0:0+events00:08:450
133138399249012,2475cyclictest1390234-21kworker/3:1+events22:28:393
133138399248412,2468cyclictest1402948-21kworker/3:2+events22:55:423
133137199248311,2470cyclictest1309997-21kworker/0:2+events19:25:000
133137199247711,2463cyclictest1393163-21kworker/0:1+events22:25:150
133137199247112,2455cyclictest1348054-21kworker/0:0+events19:58:040
133137699246110,2448cyclictest1375600-21kworker/2:0+events21:40:482
133137199241912,2404cyclictest1393163-21kworker/0:1+events22:37:460
133138399241012,2396cyclictest1412369-21kworker/3:0+events23:20:183
133137199237610,2363cyclictest1433550-21kworker/0:1+events00:25:310
133138399235612,2341cyclictest1338053-21kworker/3:1+events19:37:343
133137699235012,2334cyclictest1410550-21kworker/2:1+events23:37:332
133137499232710,2314cyclictest1402857-21kworker/1:1+events22:55:191
133137699232311,2309cyclictest1398026-21kworker/2:2+events22:54:132
133138399231112,2297cyclictest1370975-21kworker/3:0+events21:20:143
133137499230710,2295cyclictest1430115-21kworker/1:0+events00:15:091
133137699230511,2291cyclictest1393558-21kworker/2:2+events22:20:202
133137199229010,2278cyclictest1388721-21kworker/0:0+events22:10:040
133138399228711,2273cyclictest1428333-21kworker/3:1+events00:12:303
133137199228612,2270cyclictest1418486-21kworker/0:0+events23:39:270
133138399228310,2269cyclictest1356798-21kworker/3:0+events20:34:353
133137499227810,2264cyclictest1394885-21kworker/1:0+events22:34:431
133137699227510,2262cyclictest1341731-21kworker/2:0+events19:42:092
133137199227510,2261cyclictest1343245-21kworker/0:1+events19:45:130
133137199227211,2258cyclictest1349971-21kworker/0:1+events20:09:070
133138399227011,2255cyclictest1361200-21kworker/3:1+events20:37:133
133138399226911,2255cyclictest1369389-21kworker/3:2+events21:07:583
133138399226911,2254cyclictest1346197-21kworker/3:0+events20:01:313
133138399226910,2256cyclictest1430053-21kworker/3:0+events00:17:313
133137199226811,2254cyclictest1401074-21kworker/0:0+events22:59:330
133138399226710,2253cyclictest1428333-21kworker/3:1+events00:07:593
13313719922679,2254cyclictest1369564-21kworker/0:0+events21:12:110
13313719922678,2255cyclictest1349971-21kworker/0:1+events20:14:510
133138399226611,2252cyclictest1391862-21kworker/3:2+events22:19:343
133137699226610,2253cyclictest1410550-21kworker/2:1+events23:34:572
133137199226410,2251cyclictest1383873-21kworker/0:2+events22:08:300
133137199226310,2250cyclictest1438077-21kworker/0:1+events00:37:440
133137699226210,2248cyclictest1364412-21kworker/2:1+events21:05:212
133137199226210,2249cyclictest1364476-21kworker/0:2+events20:48:080
13313839922619,2249cyclictest1430053-21kworker/3:0+events00:32:043
133137199226111,2247cyclictest1431663-21kworker/0:2+events00:33:340
133138399226011,2246cyclictest1426710-21kworker/3:2+events00:04:353
133138399226011,2246cyclictest1426710-21kworker/3:2+events00:04:343
133137699226010,2246cyclictest1410550-21kworker/2:1+events23:14:552
133137199226010,2247cyclictest1361229-21kworker/0:0+events20:37:590
133138399225911,2245cyclictest1425055-21kworker/3:0+events23:57:463
133138399225810,2244cyclictest1334691-21kworker/3:0+events19:23:263
13313839922579,2244cyclictest1386892-21kworker/3:0+events22:09:173
133138399225710,2244cyclictest1414120-21kworker/3:2+events23:33:243
13313839922546,2245cyclictest1397868-21kworker/3:0+events22:43:523
133137699225411,2240cyclictest1385206-21kworker/2:0+events21:54:542
133137699225411,2239cyclictest1413965-21kworker/2:2+events23:24:312
133137699225411,2239cyclictest1385206-21kworker/2:0+events22:17:152
133137699225410,2241cyclictest1406423-21kworker/2:2+events23:01:062
133138399225310,2240cyclictest1399565-21kworker/3:2+events22:38:093
133137699225311,2239cyclictest1421304-21kworker/2:1+events23:49:412
133137699225310,2240cyclictest1436711-21kworker/2:0+events00:34:432
133138399225211,2238cyclictest1354931-21kworker/3:1+events20:28:553
13313769922529,2240cyclictest1375600-21kworker/2:0+events21:24:272
13313769922526,2243cyclictest1344940-21kworker/2:0+events19:59:352
133138399225110,2238cyclictest1386892-21kworker/3:0+events22:13:073
133138399224910,2236cyclictest1346197-21kworker/3:0+events19:57:173
133138399224810,2234cyclictest1361200-21kworker/3:1+events21:01:563
13313769922489,2235cyclictest1423514-21kworker/2:0+events23:57:222
133137199224811,2233cyclictest1367838-21kworker/0:2+events21:00:470
133137199223810,2225cyclictest1412153-21kworker/0:2+events23:51:370
133137199223710,2224cyclictest1426678-21kworker/0:2+events00:01:560
133137199223710,2224cyclictest1426678-21kworker/0:2+events00:01:560
133137199223410,2221cyclictest1337943-21kworker/0:0+events19:38:440
133138399223211,2218cyclictest1382194-21kworker/3:2+events21:53:233
133138399223210,2219cyclictest1420254-21kworker/3:2+events23:54:413
133138399222711,2213cyclictest1338053-21kworker/3:1+events19:31:573
133137699222510,2212cyclictest1369284-21kworker/2:2+events21:15:412
13313749922237,2212cyclictest1377237-21kworker/1:1+events21:44:141
13313719922156,2206cyclictest1409274-21kworker/0:1+events23:17:590
133138399221211,2198cyclictest1370975-21kworker/3:0+events21:12:323
13313839922068,2195cyclictest1354931-21kworker/3:1+events20:22:283
133137199220610,2193cyclictest1353233-21kworker/0:0+events20:16:480
13313719922059,2193cyclictest1349971-21kworker/0:1+events20:03:230
13313749921968,2185cyclictest1389274-21kworker/1:0+events22:07:221
133137499219511,2181cyclictest1383662-21kworker/1:2+events22:21:081
133137199219310,2180cyclictest1361229-21kworker/0:0+events20:41:210
133137499219011,2175cyclictest1351751-21kworker/1:0+events20:42:281
133137699218611,2172cyclictest1351572-21kworker/2:0+events20:15:142
13313749921857,2176cyclictest1418786-21kworker/1:0+events23:55:021
133137699218311,2169cyclictest1367629-21kworker/2:0+events20:59:152
133137499218311,2169cyclictest1412121-21kworker/1:2+events23:24:081
133137499218211,2167cyclictest1436658-21kworker/1:3+events00:35:291
133137699218110,2168cyclictest1385206-21kworker/2:0+events21:56:242
133137499218011,2166cyclictest1357861-21kworker/1:1+events20:31:341
133137499218010,2167cyclictest1409574-21kworker/1:1+events23:15:391
13313749921798,2168cyclictest1375479-21kworker/1:2+events21:21:451
133137499217911,2165cyclictest1362822-21kworker/1:1+events21:08:221
133137499217910,2166cyclictest1409574-21kworker/1:1+events23:35:401
133137499217810,2164cyclictest1385295-21kworker/1:0+events21:53:001
133137499217510,2162cyclictest1362822-21kworker/1:1+events20:58:281
13313769921748,2163cyclictest1336430-21kworker/2:1+events19:23:042
133137499217411,2160cyclictest1415359-21kworker/1:0+events23:28:001
133137499217311,2159cyclictest1425116-21kworker/1:1+events00:05:221
13313769921689,2156cyclictest1351572-21kworker/2:0+events20:08:452
13313749921673,2161cyclictest1378891-21kworker/1:2+events21:38:521
133137499216610,2153cyclictest1343025-21kworker/1:2+events19:47:551
13313769921629,2149cyclictest1421304-21kworker/2:1+events23:51:142
13313839921617,2151cyclictest1327993-21kworker/3:1+events19:12:343
133137499216010,2147cyclictest1420400-21kworker/1:2+events23:49:191
13313719921588,2147cyclictest1404248-21kworker/0:1+events22:52:150
13313839921579,2145cyclictest1409099-21kworker/3:2+events23:08:253
13313749921558,2143cyclictest1348236-21kworker/1:0+events20:03:011
13313769921506,2141cyclictest1425178-21kworker/2:2+events00:09:082
13313749921416,2132cyclictest1370912-21kworker/1:2+events21:12:561
13313769921409,2128cyclictest1388507-21kworker/2:1+events22:04:202
133137699213410,2121cyclictest1364412-21kworker/2:1+events20:48:552
13313719921349,2122cyclictest1388721-21kworker/0:0+events22:20:430
133137199213311,2119cyclictest1385471-21kworker/0:0+events21:57:330
13313839921328,2120cyclictest1402948-21kworker/3:2+events22:51:093
133137699213010,2117cyclictest1385206-21kworker/2:0+events22:30:342
133137499212810,2115cyclictest1336637-21kworker/1:1+events19:32:391
133137499212510,2112cyclictest1409574-21kworker/1:1+events23:41:181
13313769921239,2111cyclictest1406423-21kworker/2:2+events23:06:542
133138399212110,2108cyclictest1358917-21kworker/3:2+events20:40:363
133137699212110,2108cyclictest1334720-21kworker/2:0+events19:19:442
133137199211910,2106cyclictest1362850-21kworker/0:1+events20:59:380
133137199211810,2104cyclictest1421826-21kworker/0:0+events23:59:170
133138399211710,2104cyclictest1397868-21kworker/3:0+events22:46:583
133138399211610,2103cyclictest1433401-21kworker/3:1+events00:23:163
13313749921167,2106cyclictest1351751-21kworker/1:0+events20:26:431
133137499211610,2103cyclictest1334810-21kworker/1:0+events19:18:131
13313839921159,2103cyclictest1416909-21kworker/3:0+events23:46:373
133138399211510,2102cyclictest1407459-21kworker/3:1+events23:02:173
133138399211510,2102cyclictest1382194-21kworker/3:2+events21:46:573
133138399211510,2102cyclictest1349941-21kworker/3:1+events20:06:293
133137699211510,2102cyclictest1336430-21kworker/2:1+events19:28:522
133138399211410,2101cyclictest1380318-21kworker/3:1+events21:41:113
133137199211410,2101cyclictest1362850-21kworker/0:1+events20:54:150
133138399211310,2100cyclictest1433401-21kworker/3:1+events00:27:033
133138399211310,2100cyclictest1370975-21kworker/3:0+events21:29:023
133137699211310,2100cyclictest1417218-21kworker/2:2+events23:43:342
133137199211310,2101cyclictest1367838-21kworker/0:2+events21:05:460
133137199211310,2100cyclictest1404248-21kworker/0:1+events23:00:440
13313839921129,2100cyclictest1338053-21kworker/3:1+events19:46:243
133138399211210,2099cyclictest1380318-21kworker/3:1+events21:38:293
13313719921127,2102cyclictest1330542-21kworker/0:0+events19:18:350
133137199211210,2099cyclictest1412153-21kworker/0:2+events23:21:490
133137199211210,2099cyclictest1399442-21kworker/0:2+events22:41:160
13313839921118,2100cyclictest1405759-21kworker/3:0+events23:11:033
133137199211110,2098cyclictest1420169-21kworker/0:1+events23:44:220
133137199211110,2098cyclictest1407611-21kworker/0:2+events23:05:450
13313839921099,2097cyclictest1346197-21kworker/3:0+events20:13:433
13313839921098,2098cyclictest1372609-21kworker/3:2+events21:18:193
133137199210910,2096cyclictest1421826-21kworker/0:0+events23:48:550
133137199210712,2091cyclictest1383873-21kworker/0:2+events21:54:100
13313839921059,2093cyclictest1334691-21kworker/3:0+events19:28:073
133137199210510,2092cyclictest1373975-21kworker/0:2+events21:25:590
133137199210510,2092cyclictest1354838-21kworker/0:2+events20:22:530
133138399210410,2091cyclictest1343264-21kworker/3:2+events19:50:533
133137499210410,2092cyclictest1409574-21kworker/1:1+events23:30:161
13313769921032,2097cyclictest1429932-21kworker/2:0+events00:13:172
133137699210311,2088cyclictest1429932-21kworker/2:0+events00:20:122
13313769921029,2089cyclictest1356526-21kworker/2:1+events20:25:352
133137699210210,2089cyclictest1390143-21kworker/2:2+events22:08:552
133137699210111,2087cyclictest1428245-21kworker/2:1+events00:16:432
133138399210010,2087cyclictest1334691-21kworker/3:0+events19:16:433
13313769921008,2089cyclictest1373852-21kworker/2:1+events21:33:532
13313719921008,2088cyclictest1387096-21kworker/0:1+events22:03:090
133137699209910,2086cyclictest1415573-21kworker/2:0+events23:29:532
133137699209910,2086cyclictest1402918-21kworker/2:1+events22:58:452
13313769920984,2090cyclictest1385206-21kworker/2:0+events22:26:262
13313839920978,2086cyclictest1390234-21kworker/3:1+events22:21:303
133137699209711,2084cyclictest1398026-21kworker/2:2+events22:44:162
133137699209710,2084cyclictest1431569-21kworker/2:2+events00:26:422
13313839920968,2085cyclictest1353263-21kworker/3:2+events20:17:093
133137699209611,2083cyclictest1398026-21kworker/2:2+events22:35:062
133137699209610,2084cyclictest1401312-21kworker/2:0+events22:45:032
133137699209610,2083cyclictest1425178-21kworker/2:2+events00:02:182
133137699209610,2083cyclictest1425178-21kworker/2:2+events00:02:182
13313749920969,2084cyclictest1367594-21kworker/1:0+events21:01:321
13313719920968,2085cyclictest1401074-21kworker/0:0+events22:45:250
133137199209610,2083cyclictest1377325-21kworker/0:0+events21:33:100
13313839920959,2082cyclictest1361200-21kworker/3:1+events20:49:403
133137699209511,2081cyclictest1354809-21kworker/2:2+events20:20:352
133137199209510,2082cyclictest1412153-21kworker/0:2+events23:28:430
133137699209411,2081cyclictest1358125-21kworker/2:2+events20:50:032
133137699209310,2080cyclictest1339721-21kworker/2:2+events19:51:152
13313719920936,2084cyclictest1375660-21kworker/0:1+events21:38:060
133137199209310,2080cyclictest1393163-21kworker/0:1+events22:18:500
13313719920899,2077cyclictest1358226-21kworker/0:1+events20:33:490
133138399208810,2075cyclictest1341579-21kworker/3:0+events19:42:333
13313769920867,2077cyclictest1338061-21kworker/2:0+events19:30:022
133137199208010,2067cyclictest1339779-21kworker/0:2+events19:34:340
13313769920799,2067cyclictest1360226-21kworker/2:0+events20:36:492
13313769920747,2063cyclictest1410550-21kworker/2:1+events23:19:072
133137699207411,2060cyclictest1347124-21kworker/2:1+events20:04:552
13313749920728,2061cyclictest1338916-21kworker/1:0+events19:44:271
13313769920719,2059cyclictest1369284-21kworker/2:2+events21:04:122
133137199207110,2058cyclictest1428303-21kworker/0:0+events00:12:530
13313839920669,2054cyclictest1377023-21kworker/3:2+events21:31:173
13313719920664,2059cyclictest1428303-21kworker/0:0+events00:21:220
133137199205711,2043cyclictest1396453-21kworker/0:0+events22:31:440
13313769920486,2039cyclictest1391773-21kworker/2:1+events22:14:362
13313839920479,2036cyclictest1361200-21kworker/3:1+events20:58:533
13313839920477,2037cyclictest1390234-21kworker/3:1+events22:32:053
13313839920469,2034cyclictest1361200-21kworker/3:1+events20:52:203
13313719920468,2035cyclictest1428303-21kworker/0:0+events00:19:010
13313839920459,2033cyclictest1416909-21kworker/3:0+events23:43:133
13313839920449,2032cyclictest1435012-21kworker/3:2+events00:37:003
13313839920439,2032cyclictest1388631-21kworker/3:2+events22:04:433
13313839920439,2031cyclictest1386892-21kworker/3:0+events21:58:393
13313769920438,2033cyclictest1375600-21kworker/2:0+events21:35:022
133137199204310,2030cyclictest1401074-21kworker/0:0+events23:13:450
13313749920428,2031cyclictest1372547-21kworker/1:0+events21:16:481
133137199204110,2028cyclictest1336333-21kworker/0:1+events19:23:500
133137499204010,2026cyclictest1351751-21kworker/1:0+events20:20:591
13313719920409,2028cyclictest1375660-21kworker/0:1+events21:42:200
13313749920396,2030cyclictest1428456-21kworker/1:2+events00:14:251
133137199203910,2025cyclictest1369564-21kworker/0:0+events21:22:090
13313749920389,2026cyclictest1377237-21kworker/1:1+events21:46:111
13313749920387,2027cyclictest1383662-21kworker/1:2+events22:19:131
13313749920369,2024cyclictest1372547-21kworker/1:0+events21:32:001
133137499203510,2022cyclictest1332801-21kworker/1:2+events19:24:381
13313719920347,2025cyclictest1339779-21kworker/0:2+events19:43:170
13313769920333,2027cyclictest1339721-21kworker/2:2+events19:46:002
133137499203210,2019cyclictest1404785-21kworker/1:2+events23:04:361
13313719920329,2020cyclictest1330542-21kworker/0:0+events19:14:510
13313749920317,2021cyclictest1343025-21kworker/1:2+events19:51:381
133137499203110,2019cyclictest1430115-21kworker/1:0+events00:22:081
133137499203110,2018cyclictest1351751-21kworker/1:0+events20:45:281
13313749920307,2020cyclictest1436558-21kworker/1:0+events00:32:481
133137499203010,2017cyclictest1420400-21kworker/1:2+events23:53:541
13313769920299,2017cyclictest1315254-21kworker/2:1+events19:12:572
133137699202910,2016cyclictest1435074-21kworker/2:1+events00:39:392
13313749920299,2017cyclictest1394885-21kworker/1:0+events22:48:531
13313749920297,2019cyclictest1426857-21kworker/1:0+events00:03:511
13313749920297,2019cyclictest1426857-21kworker/1:0+events00:03:501
13313769920279,2015cyclictest1382407-21kworker/2:1+events21:47:202
13313749920272,2022cyclictest1428456-21kworker/1:2+events00:25:551
133137499202710,2013cyclictest1383662-21kworker/1:2+events21:56:041
13313749920256,2016cyclictest1389274-21kworker/1:0+events22:14:121
133137499202010,2007cyclictest1383662-21kworker/1:2+events22:00:561
13313749920187,2008cyclictest1351751-21kworker/1:0+events20:36:261
13313749920089,1996cyclictest1402857-21kworker/1:1+events22:50:231
13313749920086,1999cyclictest1393198-21kworker/1:1+events22:27:101
13313839919999,1987cyclictest1410731-21kworker/3:1+events23:18:463
133137499199410,1981cyclictest1344782-21kworker/1:1+events19:58:271
13313749919859,1973cyclictest1332801-21kworker/1:2+events19:28:291
133137499198210,1969cyclictest1407701-21kworker/1:0+events23:08:031
13313749919769,1964cyclictest1328260-21kworker/1:1+events19:14:051
13313749919758,1964cyclictest1394885-21kworker/1:0+events22:42:261
133137699197310,1960cyclictest1347124-21kworker/2:1+events20:12:132
13313749919678,1956cyclictest1394885-21kworker/1:0+events22:36:131
133137499196510,1952cyclictest1372547-21kworker/1:0+events21:28:161
13313749919539,1941cyclictest1362822-21kworker/1:1+events20:50:491
13313749919449,1932cyclictest1336637-21kworker/1:1+events19:35:211
13313769919427,1932cyclictest1336430-21kworker/2:1+events19:36:502
13295172153125,23sleep30-21swapper/319:06:303
1329648214298,41sleep10-21swapper/119:08:071
13295922142111,26sleep00-21swapper/019:07:260
13291812142110,25sleep20-21swapper/219:05:202
133137199199,4cyclictest1346733-10gpg-connect-age19:55:000
133137199170,16cyclictest1416129-21df23:30:120
133137499130,12cyclictest1353631-21/usr/sbin/munin20:15:141
133137199138,3cyclictest1357625-21find20:25:180
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional