You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-09-19 - 05:43

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #f, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails
  
Click here to display the system's profile data.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackfslot7s.osadl.org (updated Fri Sep 19, 2025 00:44:17)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
126186399394513,3927cyclictest1263405-21kworker/1:3+events19:30:001
126187499331015,3290cyclictest1258022-21kworker/3:1+events19:30:223
126186099303814,3021cyclictest1268253-21kworker/0:2+events19:35:170
126187499286210,2847cyclictest1344165-21kworker/3:0+events23:45:183
126186399283514,2815cyclictest1334806-21kworker/1:1+events23:15:121
126186399271213,2694cyclictest1337390-21kworker/1:0+events23:10:151
126187499269113,2675cyclictest1332764-21kworker/3:2+events23:05:003
126187499265113,2634cyclictest1273339-21kworker/3:0+events19:55:203
126186099264313,2628cyclictest1312276-21kworker/0:1+events22:00:170
126187499258212,2566cyclictest1329578-21kworker/3:1+events22:51:443
126186099258112,2565cyclictest1268253-21kworker/0:2+events19:33:240
126186399256712,2552cyclictest1287414-21kworker/1:0+events21:10:141
126186099255712,2542cyclictest1350741-21kworker/0:0+events00:06:440
126186099254012,2525cyclictest1279718-21kworker/0:2+events20:06:490
126187499253512,2520cyclictest1282894-21kworker/3:1+events20:17:473
126186399252913,2513cyclictest1360438-21kworker/1:0+events00:25:231
126186899252812,2512cyclictest1270073-21kworker/2:2+events19:43:102
126186099250012,2484cyclictest1339630-21kworker/0:1+events23:19:010
126187499249812,2482cyclictest1273339-21kworker/3:0+events19:51:073
126187499249712,2481cyclictest1362265-21kworker/3:1+events00:30:453
126187499249712,2481cyclictest1306539-21kworker/3:1+events21:33:133
126186099249712,2481cyclictest1317149-21kworker/0:0+events22:29:370
126187499249612,2481cyclictest1349587-21kworker/3:2+events23:55:583
126186099249211,2478cyclictest1262977-21kworker/0:0+events19:22:250
126187499249112,2476cyclictest1321107-21kworker/3:1+events22:28:283
126187499249012,2474cyclictest1300806-21kworker/3:2+events21:14:013
126187499249012,2474cyclictest1271479-21kworker/3:2+events19:45:483
126186099249011,2475cyclictest1292695-21kworker/0:2+events20:46:500
126187499248912,2474cyclictest1308826-21kworker/3:2+events21:52:393
126186099248912,2473cyclictest1334367-21kworker/0:2+events22:59:380
126186099248911,2474cyclictest1284224-21kworker/0:2+events20:34:220
12618609924879,2475cyclictest1310670-21kworker/0:2+events21:49:160
126186099248711,2473cyclictest1324982-21kworker/0:1+events22:39:300
126186099248710,2474cyclictest1297723-21kworker/0:2+events21:06:250
126186099248412,2469cyclictest1366742-21kworker/0:2+events00:36:490
126186099248412,2468cyclictest1362237-21kworker/0:1+events00:27:440
126187499248210,2469cyclictest1300806-21kworker/3:2+events21:26:403
126186899248212,2466cyclictest1326352-21kworker/2:1+events22:35:192
126186099248110,2467cyclictest1276392-21kworker/0:1+events19:55:430
126187499247912,2464cyclictest1332764-21kworker/3:2+events23:12:533
126186099247712,2462cyclictest1357007-21kworker/0:1+events00:12:310
126187499247412,2458cyclictest1313725-21kworker/3:1+events21:57:373
126186399246711,2452cyclictest1323488-21kworker/1:2+events22:30:231
126186399246711,2452cyclictest1323488-21kworker/1:2+events22:30:231
126186399246312,2447cyclictest1277904-21kworker/1:2+events19:58:001
126187499245612,2440cyclictest1357099-21kworker/3:1+events00:18:093
126186099245112,2436cyclictest1289200-21kworker/0:0+events20:36:120
126186399244611,2431cyclictest1287414-21kworker/1:0+events20:32:061
12618689924409,2427cyclictest1355517-21kworker/2:1+events00:05:122
126186899243912,2423cyclictest1264897-21kworker/2:1+events19:20:302
126186899243512,2419cyclictest1353863-21kworker/2:2+events00:03:402
126186399243312,2417cyclictest1320171-21kworker/1:0+events22:19:441
126186399243110,2417cyclictest1334806-21kworker/1:1+events23:23:411
126186899242212,2407cyclictest1320253-21kworker/2:1+events22:15:502
126186099242110,2408cyclictest1326590-21kworker/0:2+events22:40:170
126187499241010,2397cyclictest1276841-21kworker/3:2+events20:05:183
126186099239711,2382cyclictest1296065-21kworker/0:1+events21:03:480
12618749923938,2382cyclictest1292393-21kworker/3:2+events20:52:533
126186399239312,2377cyclictest1354157-21kworker/1:3+events00:02:511
126186899239212,2376cyclictest1334421-21kworker/2:1+events23:06:302
126186399239012,2374cyclictest1290616-21kworker/1:1+events21:01:531
126186399239011,2376cyclictest1350808-21kworker/1:2+events23:50:161
126186899238910,2375cyclictest1285849-21kworker/2:1+events20:35:292
126186399238812,2372cyclictest1290616-21kworker/1:1+events20:53:151
126187499238712,2371cyclictest1290741-21kworker/3:0+events20:40:243
126186899238712,2371cyclictest1271682-21kworker/2:1+events19:57:132
126186099238712,2371cyclictest1342402-21kworker/0:2+events23:39:130
126186399238412,2370cyclictest1287414-21kworker/1:0+events21:15:101
126186399237813,2361cyclictest1287414-21kworker/1:0+events20:47:121
126186399237812,2363cyclictest1354157-21kworker/1:3+events00:06:201
126187499237512,2360cyclictest1318727-21kworker/3:0+events22:15:293
126186399236712,2351cyclictest1299770-21kworker/1:2+events21:23:591
126186899236512,2351cyclictest1363826-21kworker/2:2+events00:30:002
126186899236411,2350cyclictest1331189-21kworker/2:0+events22:48:402
126186399236012,2345cyclictest1348754-21kworker/1:0+events23:43:451
126186899235912,2344cyclictest1285849-21kworker/2:1+events20:25:432
126186399235611,2342cyclictest1362569-21kworker/1:2+events00:35:401
126187499235511,2341cyclictest1357099-21kworker/3:1+events00:10:563
126186899235512,2340cyclictest1336041-21kworker/2:2+events23:20:352
126186899235312,2337cyclictest1318578-21kworker/2:2+events22:11:192
126186399235312,2338cyclictest1277904-21kworker/1:2+events20:05:411
126186399235311,2338cyclictest1281204-21kworker/1:1+events20:26:481
126186099235311,2339cyclictest1324982-21kworker/0:1+events22:34:550
126186099235311,2339cyclictest1324982-21kworker/0:1+events22:34:550
126187499235212,2336cyclictest1355398-21kworker/3:0+events00:04:033
126186899235212,2337cyclictest1276579-21kworker/2:0+events20:02:112
126186899235212,2336cyclictest1352270-21kworker/2:1+events23:54:492
126186899235210,2339cyclictest1266538-21kworker/2:0+events19:29:372
126186399235212,2336cyclictest1287414-21kworker/1:0+events20:56:571
126186399235212,2336cyclictest1287414-21kworker/1:0+events20:56:561
12618689923516,2341cyclictest1295294-21kworker/2:2+events20:52:302
126186899235112,2336cyclictest1362200-21kworker/2:0+events00:28:312
126186399235110,2338cyclictest1290616-21kworker/1:1+events21:05:411
126186399235110,2338cyclictest1273250-21kworker/1:1+events19:53:481
12618689923509,2337cyclictest1293696-21kworker/2:1+events20:57:192
12618689923509,2337cyclictest1293696-21kworker/2:1+events20:57:192
126186899235012,2335cyclictest1353863-21kworker/2:2+events00:21:372
126186899235012,2334cyclictest1343955-21kworker/2:1+events23:26:202
126186399235012,2334cyclictest1290616-21kworker/1:1+events20:41:101
126186399235012,2334cyclictest1289018-21kworker/1:2+events20:35:501
126186899234911,2334cyclictest1297661-21kworker/2:2+events21:07:582
126186399234912,2335cyclictest1309437-21kworker/1:2+events21:45:051
126186399234912,2334cyclictest1334806-21kworker/1:1+events23:33:551
126186399234912,2334cyclictest1334806-21kworker/1:1+events23:02:401
126186399234912,2334cyclictest1284812-21kworker/1:2+events20:23:491
126186399234912,2334cyclictest1273250-21kworker/1:1+events19:46:121
12618689923489,2336cyclictest1271682-21kworker/2:1+events19:39:032
126186899234712,2332cyclictest1339523-21kworker/2:1+events23:17:512
12618639923476,2337cyclictest1305524-21kworker/1:0+events21:54:571
126186399234712,2332cyclictest1331126-21kworker/1:1+events22:48:171
12618749923464,2339cyclictest1340918-21kworker/3:1+events23:25:343
12618639923469,2334cyclictest1269934-21kworker/1:1+events19:31:531
126186399233712,2322cyclictest1354157-21kworker/1:3+events00:17:461
126187499232812,2313cyclictest1282894-21kworker/3:1+events20:47:573
126186899232812,2313cyclictest1341195-21kworker/2:0+events23:41:282
126187499232712,2312cyclictest1295915-21kworker/3:0+events21:02:173
126186399232712,2311cyclictest1299770-21kworker/1:2+events21:29:201
126186099232710,2314cyclictest1303861-21kworker/0:0+events21:28:330
126187499232612,2311cyclictest1308826-21kworker/3:2+events22:04:303
126187499232612,2310cyclictest1321107-21kworker/3:1+events22:35:423
126187499232512,2309cyclictest1300806-21kworker/3:2+events21:19:003
126187499232411,2310cyclictest1352170-21kworker/3:1+events23:53:183
126186099232312,2307cyclictest1324982-21kworker/0:1+events23:06:510
126186099232212,2307cyclictest1292695-21kworker/0:2+events20:56:340
126186099232212,2307cyclictest1292695-21kworker/0:2+events20:56:340
126186899232110,2307cyclictest1287505-21kworker/2:2+events20:32:292
126186099232112,2305cyclictest1299352-21kworker/0:0+events21:10:380
126186099231712,2301cyclictest1337686-21kworker/0:2+events23:11:010
126187499231412,2298cyclictest1332764-21kworker/3:2+events22:57:453
126187499231411,2299cyclictest1365128-21kworker/3:2+events00:36:263
126187499230912,2294cyclictest1300806-21kworker/3:2+events21:20:563
126187499230912,2294cyclictest1287356-21kworker/3:2+events20:28:403
126187499230912,2293cyclictest1344165-21kworker/3:0+events23:33:323
126187499230712,2292cyclictest1318727-21kworker/3:0+events22:20:523
126187499230712,2291cyclictest1342552-21kworker/3:2+events23:24:493
126187499230512,2290cyclictest1281262-21kworker/3:0+events20:14:463
126187499230311,2288cyclictest1271479-21kworker/3:2+events19:37:343
126186099230312,2289cyclictest1348873-21kworker/0:1+events23:46:020
126186399230212,2287cyclictest1350808-21kworker/1:2+events23:56:231
126186099230113,2285cyclictest1342402-21kworker/0:2+events23:34:410
126186099230111,2287cyclictest1317149-21kworker/0:0+events22:17:250
126186099229210,2279cyclictest1331037-21kworker/0:0+events22:46:440
126186899229112,2275cyclictest1280024-21kworker/2:2+events20:11:432
126186899228711,2272cyclictest1347175-21kworker/2:1+events23:38:042
126186099228412,2269cyclictest1336058-21kworker/0:0+events23:20:590
126186099228111,2267cyclictest1302225-21kworker/0:1+events21:33:360
126187499227410,2261cyclictest1329578-21kworker/3:1+events22:43:153
126186099227211,2258cyclictest1350741-21kworker/0:0+events23:51:250
126187499226810,2254cyclictest1282894-21kworker/3:1+events20:34:003
126186099226711,2253cyclictest1324982-21kworker/0:1+events23:01:080
126186899226612,2251cyclictest1280024-21kworker/2:2+events20:07:582
126186099226612,2250cyclictest1312276-21kworker/0:1+events22:05:140
126186099226611,2251cyclictest1303861-21kworker/0:0+events21:23:360
126186099226610,2253cyclictest1365159-21kworker/0:0+events00:34:110
126186399226412,2249cyclictest1357134-21kworker/1:0+events00:12:531
126187499226010,2247cyclictest1306539-21kworker/3:1+events21:38:073
126186099226010,2247cyclictest1271721-21kworker/0:0+events20:01:050
126186399225912,2244cyclictest1305524-21kworker/1:0+events21:34:201
126186399225712,2241cyclictest1334806-21kworker/1:1+events23:06:091
12618609922546,2245cyclictest1302225-21kworker/0:1+events21:42:000
126186099225310,2239cyclictest1324982-21kworker/0:1+events22:53:140
126186099225310,2239cyclictest1297723-21kworker/0:2+events21:15:540
126186399225211,2237cyclictest1271571-21kworker/1:2+events19:42:481
126186899225011,2236cyclictest1273350-21kworker/2:0+events19:48:042
126186099224910,2236cyclictest1350741-21kworker/0:0+events23:58:170
12618609922449,2232cyclictest1262977-21kworker/0:0+events19:15:130
126186399224312,2228cyclictest1281204-21kworker/1:1+events20:17:251
126186099224110,2228cyclictest1319929-21kworker/0:2+events22:12:060
126186399223911,2224cyclictest1279599-21kworker/1:0+events20:04:091
126186099223812,2222cyclictest1271721-21kworker/0:0+events19:40:120
126187499223413,2217cyclictest1297597-21kworker/3:2+events21:06:493
126187499222811,2214cyclictest1295915-21kworker/3:0+events20:58:503
126187499222811,2214cyclictest1295915-21kworker/3:0+events20:58:503
126186899222811,2213cyclictest1327957-21kworker/2:2+events22:42:522
126186099222410,2211cyclictest1336058-21kworker/0:0+events23:25:590
126186099222311,2209cyclictest1317149-21kworker/0:0+events22:24:170
126186399222212,2206cyclictest1334806-21kworker/1:1+events23:35:481
126186899222112,2205cyclictest1302830-21kworker/2:1+events21:35:052
126187499221710,2204cyclictest1312063-21kworker/3:0+events21:47:443
126186899221711,2203cyclictest1315343-21kworker/2:2+events22:03:002
126186899220310,2189cyclictest1300937-21kworker/2:0+events21:19:232
126186099220311,2189cyclictest1295324-21kworker/0:0+events20:54:020
126186899220110,2187cyclictest1353863-21kworker/2:2+events23:59:252
126186399220111,2187cyclictest1323488-21kworker/1:2+events22:40:401
126186399220110,2187cyclictest1323488-21kworker/1:2+events22:26:581
126186099219511,2181cyclictest1302225-21kworker/0:1+events21:37:220
126186399218611,2173cyclictest1281204-21kworker/1:1+events20:10:341
12618639921746,2165cyclictest1323488-21kworker/1:2+events22:36:271
12618689921709,2158cyclictest1313709-21kworker/2:1+events21:58:242
126186099217010,2157cyclictest1312276-21kworker/0:1+events21:50:240
126186899216911,2154cyclictest1331189-21kworker/2:0+events22:59:592
12618639921697,2159cyclictest1258021-21kworker/1:1+events19:22:471
126186899216811,2154cyclictest1305405-21kworker/2:0+events21:29:422
126186099216811,2153cyclictest1310670-21kworker/0:2+events21:58:480
126186399216511,2150cyclictest1316752-21kworker/1:1+events22:01:501
12618689921649,2151cyclictest1331189-21kworker/2:0+events23:01:302
126186899216411,2150cyclictest1318578-21kworker/2:2+events22:30:462
126186899216411,2150cyclictest1318578-21kworker/2:2+events22:30:452
126186899216410,2151cyclictest1302830-21kworker/2:1+events21:32:492
12618639921649,2151cyclictest1318343-21kworker/1:2+events22:13:591
12618689921637,2153cyclictest1316996-21kworker/2:0+events22:09:482
12618639921639,2152cyclictest1327892-21kworker/1:0+events22:55:041
12618639921637,2153cyclictest1316752-21kworker/1:1+events22:05:381
126186099216311,2149cyclictest1285910-21kworker/0:1+events20:24:580
12618689921628,2151cyclictest1310454-21kworker/2:0+events21:45:262
126186899216211,2148cyclictest1337654-21kworker/2:0+events23:12:322
126186399216211,2148cyclictest1360438-21kworker/1:0+events00:32:151
126186399216210,2149cyclictest1313909-21kworker/1:2+events21:59:321
126186899216011,2145cyclictest1320253-21kworker/2:1+events22:22:462
126186399216010,2147cyclictest1334806-21kworker/1:1+events23:28:381
126186899215911,2145cyclictest1310454-21kworker/2:0+events21:53:022
126186899215710,2144cyclictest1289135-21kworker/2:0+events20:48:422
12618639921579,2144cyclictest1307260-21kworker/1:1+events21:37:451
126186399215711,2143cyclictest1321811-21kworker/1:1+events22:23:561
126186399215511,2142cyclictest1305524-21kworker/1:0+events21:40:261
126186899215411,2139cyclictest1299276-21kworker/2:1+events21:11:442
126186399215411,2139cyclictest1268067-21kworker/1:0+events19:36:491
126186899215311,2139cyclictest1353863-21kworker/2:2+events00:18:322
126186899215311,2139cyclictest1282597-21kworker/2:0+events20:19:182
126186899215310,2140cyclictest1296001-21kworker/2:0+events21:03:012
126186899215110,2139cyclictest1280024-21kworker/2:2+events20:20:022
126186899215110,2138cyclictest1256809-21kworker/2:2+events19:15:562
126186399215111,2137cyclictest1331126-21kworker/1:1+events22:51:201
126187499215010,2137cyclictest1332764-21kworker/3:2+events23:08:223
126186399214210,2129cyclictest1350808-21kworker/1:2+events23:49:301
126187499213210,2118cyclictest1276841-21kworker/3:2+events20:01:493
126187499213112,2116cyclictest1317054-21kworker/3:1+events22:09:023
126186399213011,2116cyclictest1263405-21kworker/1:3+events19:16:191
126186399212911,2116cyclictest1358770-21kworker/1:1+events00:20:051
12618639921267,2116cyclictest1259902-21kworker/1:0+events19:14:261
126186899212411,2110cyclictest1302830-21kworker/2:1+events21:24:232
126186099212311,2108cyclictest1274725-21kworker/0:2+events19:49:330
126187499211911,2105cyclictest1273339-21kworker/3:0+events19:44:393
126187499211910,2106cyclictest1362265-21kworker/3:1+events00:26:113
126187499211910,2106cyclictest1358740-21kworker/3:2+events00:20:273
126187499211711,2103cyclictest1282894-21kworker/3:1+events20:38:533
126186899211710,2104cyclictest1274976-21kworker/2:2+events19:54:352
126187499211210,2099cyclictest1318727-21kworker/3:0+events22:13:353
126186099211210,2099cyclictest1290797-21kworker/0:1+events20:42:410
12618689921117,2101cyclictest1365410-21kworker/2:0+events00:37:112
126186099210710,2093cyclictest1284224-21kworker/0:2+events20:18:090
126187499210611,2092cyclictest1326316-21kworker/3:2+events22:34:113
126187499210611,2092cyclictest1326316-21kworker/3:2+events22:34:103
126187499210611,2090cyclictest1347428-21kworker/3:1+events23:43:003
126186099210611,2091cyclictest1342402-21kworker/0:2+events23:44:080
126187499210511,2091cyclictest1337568-21kworker/3:0+events23:17:293
126187499210511,2091cyclictest1266823-21kworker/3:2+events19:26:343
126187499210511,2091cyclictest1264869-21kworker/3:0+events19:18:373
126187499210510,2091cyclictest1357099-21kworker/3:1+events00:09:013
126187499210411,2090cyclictest1344165-21kworker/3:0+events23:39:353
126187499210411,2090cyclictest1281262-21kworker/3:0+events20:22:203
126187499210210,2089cyclictest1308826-21kworker/3:2+events21:43:093
126186099210211,2088cyclictest1353930-21kworker/0:2+events00:00:320
126186099210211,2088cyclictest1285910-21kworker/0:1+events20:27:340
126186099210011,2085cyclictest1271721-21kworker/0:0+events19:52:380
126186099209411,2080cyclictest1282592-21kworker/0:1+events20:12:520
126186099209411,2080cyclictest1268253-21kworker/0:2+events19:28:520
126186099209410,2082cyclictest1360345-21kworker/0:0+events00:20:500
126186899209110,2078cyclictest1329589-21kworker/2:1+events22:52:292
126186099207910,2066cyclictest1258267-21kworker/0:2+events19:14:020
126186899207310,2059cyclictest1256809-21kworker/2:2+events19:12:522
126186099202510,2012cyclictest1358646-21kworker/0:2+events00:15:540
12618689920094,2001cyclictest1292096-21kworker/2:2+events20:44:122
126186899200810,1994cyclictest1318578-21kworker/2:2+events22:28:512
12618689919985,1990cyclictest1302830-21kworker/2:1+events21:42:242
12618689919968,1985cyclictest1341195-21kworker/2:0+events23:49:082
12618749919479,1935cyclictest1329578-21kworker/3:1+events22:49:483
126187499183110,1818cyclictest1259961-21kworker/3:2+events19:10:363
12615312174143,28sleep00-21swapper/019:09:200
1261449214895,35sleep10-21swapper/119:08:171
12602522144111,27sleep20-21swapper/219:05:062
12613432140106,28sleep30-21swapper/319:06:593
1261874991812,2cyclictest191rcu_preempt19:25:003
126186899189,2cyclictest1358883-21perf00:14:592
126186899170,4cyclictest1345633-21taskset23:31:402
126186899150,13cyclictest1269487-21ntpq19:30:192
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional