You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-26 - 13:03

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #f, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails
  
Click here to display the system's profile data.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackfslot7s.osadl.org (updated Wed Aug 26, 2026 00:44:16)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
13801439931799,3165cyclictest1438686-21kworker/1:2+events22:30:191
138014399309713,3079cyclictest1379776-21kworker/1:1+events19:25:181
138014899290512,2889cyclictest1380871-21kworker/2:0+events19:20:162
138014399280513,2790cyclictest1379776-21kworker/1:1+events20:30:191
138014399280412,2789cyclictest1420576-21kworker/1:1+events21:25:121
138014899279913,2782cyclictest1380871-21kworker/2:0+events19:35:132
138013999278711,2772cyclictest1391428-21kworker/0:2+events20:05:190
138013999278111,2765cyclictest1412273-21kworker/0:1+events21:00:110
138014899274212,2728cyclictest1429780-21kworker/2:1+events21:45:002
138013999273913,2722cyclictest1471692-21kworker/0:0+events00:05:190
138014899273711,2721cyclictest1473461-21kworker/2:1+events00:10:182
138013999273613,2719cyclictest1464086-21kworker/0:1+events23:30:210
138013999273613,2719cyclictest1464086-21kworker/0:1+events23:30:210
138014899270810,2694cyclictest1429780-21kworker/2:1+events21:55:172
138014899269813,2681cyclictest1385080-21kworker/2:2+events19:30:152
138014899269713,2680cyclictest1413819-21kworker/2:0+events21:02:052
138015199266913,2652cyclictest1482939-21kworker/3:2+events00:35:183
138013999265912,2643cyclictest1439371-21kworker/0:0+events22:15:210
13801489926089,2594cyclictest1400974-21kworker/2:1+events20:50:122
138014899258613,2569cyclictest1452178-21kworker/2:0+events22:55:202
138014399258011,2565cyclictest1455673-21kworker/1:2+events23:49:371
138014399254812,2531cyclictest1420576-21kworker/1:1+events21:22:101
138014899254113,2526cyclictest1434511-21kworker/2:0+events22:05:292
138014399254110,2527cyclictest1407284-21kworker/1:0+events20:40:131
138013999252312,2508cyclictest1412273-21kworker/0:1+events21:14:110
138014399251511,2501cyclictest1438686-21kworker/1:2+events22:15:441
138013999251212,2496cyclictest1429991-21kworker/0:0+events21:44:380
138013999251110,2497cyclictest1405771-21kworker/0:2+events20:31:510
138014399250912,2495cyclictest1445734-21kworker/1:0+events22:40:031
138013999250912,2493cyclictest1399200-21kworker/0:0+events20:37:330
13801439925066,2497cyclictest1450670-21kworker/1:0+events22:50:221
13801489925054,2498cyclictest1400974-21kworker/2:1+events20:35:182
138013999250511,2490cyclictest1462228-21kworker/0:2+events23:46:590
138014399250212,2488cyclictest1481407-21kworker/1:1+events00:35:421
138014399250110,2488cyclictest1448773-21kworker/1:1+events23:06:471
138013999249912,2485cyclictest1434906-21kworker/0:0+events22:05:050
138015199249611,2481cyclictest1397820-21kworker/3:0+events20:12:543
138014399249612,2482cyclictest1473709-21kworker/1:3+events00:01:271
138014899249212,2477cyclictest1395995-21kworker/2:2+events19:59:332
138014899249012,2475cyclictest1440924-21kworker/2:1+events22:29:562
138014399248612,2470cyclictest1413699-21kworker/1:1+events20:55:331
13801439924809,2467cyclictest1412086-21kworker/1:2+events20:46:421
138014399248011,2467cyclictest1455673-21kworker/1:2+events23:35:151
13801399924808,2468cyclictest1454085-21kworker/0:0+events23:29:140
138014899247712,2461cyclictest1473461-21kworker/2:1+events00:01:072
138013999247512,2459cyclictest1460573-21kworker/0:1+events23:19:230
138014399247412,2458cyclictest1415424-21kworker/1:0+events21:00:561
138014899247112,2455cyclictest1445792-21kworker/2:2+events22:35:582
138014899247012,2454cyclictest1473461-21kworker/2:1+events23:58:502
138013999246512,2449cyclictest1388113-21kworker/0:0+events19:37:300
138014899246312,2447cyclictest1418652-21kworker/2:2+events21:20:182
138013999245712,2441cyclictest1402423-21kworker/0:1+events20:18:080
138013999245510,2442cyclictest1471692-21kworker/0:0+events00:11:290
138014899245211,2437cyclictest1413819-21kworker/2:0+events21:08:572
138013999245212,2436cyclictest1424778-21kworker/0:1+events21:27:050
138014899244712,2431cyclictest1484516-21kworker/2:1+events00:34:102
138014899244412,2428cyclictest1391575-21kworker/2:0+events19:46:152
138014899244311,2428cyclictest1395995-21kworker/2:2+events20:04:572
138013999244312,2427cyclictest1391428-21kworker/0:2+events19:54:370
138015199244212,2426cyclictest1460968-21kworker/3:2+events23:56:093
138013999244112,2425cyclictest1448868-21kworker/0:0+events22:44:190
138014899243711,2422cyclictest1399284-21kworker/2:0+events20:09:092
138014399243711,2422cyclictest1431588-21kworker/1:0+events21:53:211
13801399924299,2416cyclictest1431611-21kworker/0:2+events21:49:570
138014399242710,2413cyclictest1429961-21kworker/1:1+events21:49:341
138013999242612,2411cyclictest1412273-21kworker/0:1+events21:16:050
138013999242612,2410cyclictest1434906-21kworker/0:0+events22:01:190
138013999242512,2410cyclictest1471692-21kworker/0:0+events23:54:350
13801439924248,2413cyclictest1455673-21kworker/1:2+events00:19:261
138013999242412,2409cyclictest1447245-21kworker/0:2+events22:37:500
138015199242311,2410cyclictest1399416-21kworker/3:2+events20:20:033
138013999242312,2408cyclictest1473402-21kworker/0:2+events00:00:430
138013999242312,2408cyclictest1452419-21kworker/0:1+events22:56:300
138014399242212,2406cyclictest1455673-21kworker/1:2+events23:40:311
138014399242211,2407cyclictest1423269-21kworker/1:0+events21:37:471
138014399242112,2405cyclictest1455673-21kworker/1:2+events23:12:091
138013999242112,2406cyclictest1484574-21kworker/0:0+events00:37:560
13801439924209,2409cyclictest1392910-21kworker/1:0+events19:50:021
138014399242012,2405cyclictest1455673-21kworker/1:2+events00:06:031
138013999242012,2404cyclictest1481346-21kworker/0:2+events00:28:330
138013999242012,2404cyclictest1381635-21kworker/0:1+events19:23:230
138013999242011,2406cyclictest1437737-21kworker/0:1+events22:10:500
138015199241911,2405cyclictest1386536-21kworker/3:0+events19:55:453
138014899241912,2404cyclictest1402604-21kworker/2:2+events20:21:362
138014399241911,2404cyclictest1462196-21kworker/1:0+events23:26:371
138013999241912,2404cyclictest1391428-21kworker/0:2+events19:47:230
138014399241812,2404cyclictest1455673-21kworker/1:2+events23:20:101
138014399241811,2404cyclictest1423269-21kworker/1:0+events21:40:501
138013999241812,2403cyclictest1431611-21kworker/0:2+events21:54:310
138013999241812,2402cyclictest1402423-21kworker/0:1+events20:24:390
138013999241811,2404cyclictest1462228-21kworker/0:2+events23:40:540
138013999241811,2404cyclictest1399200-21kworker/0:0+events20:46:200
138014399241710,2404cyclictest1438686-21kworker/1:2+events22:11:131
138013999241712,2402cyclictest1444403-21kworker/0:0+events22:32:540
138014399241612,2401cyclictest1412086-21kworker/1:2+events20:54:041
138014399241611,2401cyclictest1445734-21kworker/1:0+events22:36:201
138013999241612,2401cyclictest1448868-21kworker/0:0+events22:51:060
138014399241211,2398cyclictest1379776-21kworker/1:1+events20:26:111
138014399241011,2396cyclictest1417079-21kworker/1:2+events21:09:441
138013999240811,2393cyclictest1454085-21kworker/0:0+events23:12:540
13801439924067,2395cyclictest1444131-21kworker/1:1+events22:27:391
13801439924055,2397cyclictest1479745-21kworker/1:0+events00:31:341
138014399240211,2387cyclictest1379776-21kworker/1:1+events19:42:471
138014899240012,2384cyclictest1413819-21kworker/2:0+events21:11:132
138014899239911,2384cyclictest1476715-21kworker/2:2+events00:16:262
138013999239912,2385cyclictest1454085-21kworker/0:0+events23:21:180
138014899239612,2381cyclictest1426572-21kworker/2:2+events21:50:202
138014899239312,2378cyclictest1440924-21kworker/2:1+events22:52:162
138014899239212,2377cyclictest1457334-21kworker/2:1+events23:11:442
138014899239011,2376cyclictest1440924-21kworker/2:1+events22:23:002
138014899238912,2373cyclictest1440924-21kworker/2:1+events22:43:102
138014899238812,2373cyclictest1452178-21kworker/2:0+events23:34:062
138014899238812,2373cyclictest1452178-21kworker/2:0+events23:34:052
138014899238811,2373cyclictest1400974-21kworker/2:1+events20:28:492
138014899238712,2372cyclictest1452178-21kworker/2:0+events23:08:402
138014399238712,2372cyclictest1481407-21kworker/1:1+events00:25:291
138014899238512,2370cyclictest1465386-21kworker/2:2+events23:38:162
138014899238512,2370cyclictest1389866-21kworker/2:1+events19:42:022
138014899238511,2371cyclictest1424716-21kworker/2:1+events21:28:392
138014899238111,2367cyclictest1434511-21kworker/2:0+events22:12:432
138013999238112,2365cyclictest1450729-21kworker/0:2+events23:04:260
138013999238011,2366cyclictest1421966-21kworker/0:0+events21:21:250
138014899237611,2362cyclictest1400974-21kworker/2:1+events20:30:422
138014399237511,2361cyclictest1448773-21kworker/1:1+events22:59:341
138014899237412,2359cyclictest1420304-21kworker/2:1+events21:15:422
138014899237311,2359cyclictest1437947-21kworker/2:2+events22:19:592
138014399236911,2355cyclictest1448773-21kworker/1:1+events22:46:361
138013999236512,2349cyclictest1413851-21kworker/0:2+events20:54:260
138014899236312,2347cyclictest1452178-21kworker/2:0+events23:28:502
138014899236111,2347cyclictest1426572-21kworker/2:2+events21:41:122
138013999236010,2347cyclictest1417110-21kworker/0:2+events21:30:590
138014899235811,2343cyclictest1468640-21kworker/2:0+events23:52:402
138013999235810,2344cyclictest1478111-21kworker/0:2+events00:16:050
138014399235710,2343cyclictest1401064-21kworker/1:2+events20:19:191
138014899235310,2339cyclictest1426572-21kworker/2:2+events21:32:052
138014899235211,2337cyclictest1486124-21kworker/2:2+events00:38:422
138013999234812,2332cyclictest1468874-21kworker/0:1+events23:55:210
138014899233912,2324cyclictest1426572-21kworker/2:2+events21:37:012
138014899233812,2322cyclictest1440924-21kworker/2:1+events22:33:192
13801519923358,2325cyclictest1481347-21kworker/3:0+events00:30:033
138014399233512,2320cyclictest1379776-21kworker/1:1+events19:15:261
13801489923329,2319cyclictest1478348-21kworker/2:0+events00:20:562
138014399233211,2318cyclictest1429961-21kworker/1:1+events22:03:161
13801439923318,2320cyclictest1417079-21kworker/1:2+events21:19:551
138013999233111,2316cyclictest1431611-21kworker/0:2+events22:21:100
138015199232212,2307cyclictest1429749-21kworker/3:2+events21:51:043
138014399231611,2301cyclictest1392910-21kworker/1:0+events19:59:101
13801519923139,2301cyclictest1432956-21kworker/3:0+events22:13:533
138015199230712,2292cyclictest1448870-21kworker/3:1+events22:42:483
138014899230112,2285cyclictest1400974-21kworker/2:1+events20:43:412
138013999228911,2275cyclictest1391428-21kworker/0:2+events20:02:150
138013999228811,2274cyclictest1415455-21kworker/0:0+events20:59:250
138014399228511,2270cyclictest1455673-21kworker/1:2+events23:15:341
138015199228410,2270cyclictest1394590-21kworker/3:2+events20:02:383
138014399227510,2262cyclictest1438686-21kworker/1:2+events22:23:231
138014899225911,2245cyclictest1452178-21kworker/2:0+events23:01:052
138014399225411,2240cyclictest1429961-21kworker/1:1+events22:05:511
138014399224511,2231cyclictest1405683-21kworker/1:2+events20:36:261
138014399224412,2229cyclictest1455673-21kworker/1:2+events00:20:341
138013999223611,2222cyclictest1378320-21kworker/0:2+events19:15:010
138013999223411,2220cyclictest1412273-21kworker/0:1+events21:08:110
138013999223410,2221cyclictest1481346-21kworker/0:2+events00:33:050
138013999223111,2216cyclictest1471692-21kworker/0:0+events00:21:180
138013999223110,2218cyclictest1388113-21kworker/0:0+events19:34:050
13801439922269,2214cyclictest1424866-21kworker/1:2+events21:31:201
13801399922268,2215cyclictest1399200-21kworker/0:0+events20:29:360
138013999222610,2212cyclictest1431611-21kworker/0:2+events21:58:180
138014399222511,2211cyclictest1448773-21kworker/1:1+events23:04:501
138014399222511,2211cyclictest1404021-21kworker/1:0+events20:23:301
138013999222511,2211cyclictest1381635-21kworker/0:1+events19:29:280
138013999222510,2212cyclictest1428278-21kworker/0:2+events21:38:100
138015199222411,2211cyclictest1407061-21kworker/3:2+events20:35:403
13801439922248,2212cyclictest1388052-21kworker/1:2+events19:39:221
138014399222411,2210cyclictest1379776-21kworker/1:1+events20:09:561
138013999222411,2209cyclictest1448868-21kworker/0:0+events22:47:210
138013999222310,2209cyclictest1408765-21kworker/0:1+events20:40:350
138014399222010,2206cyclictest1392910-21kworker/1:0+events19:48:331
13801439922194,2212cyclictest1471753-21kworker/1:0+events23:54:121
138014399221910,2206cyclictest1455673-21kworker/1:2+events00:11:511
138013999221810,2204cyclictest1391428-21kworker/0:2+events19:57:160
138014899221711,2202cyclictest1476715-21kworker/2:2+events00:09:322
138014399221711,2203cyclictest1379776-21kworker/1:1+events19:14:171
13801439922169,2203cyclictest1433179-21kworker/1:2+events21:59:041
138014399220710,2193cyclictest1399467-21kworker/1:0+events20:11:011
138014899220511,2191cyclictest1413819-21kworker/2:0+events20:55:582
138013999220311,2188cyclictest1400801-21kworker/0:2+events20:12:320
138014399220111,2187cyclictest1455673-21kworker/1:2+events23:30:421
138014399220111,2187cyclictest1455673-21kworker/1:2+events23:30:421
138015199219810,2185cyclictest1452360-21kworker/3:2+events22:58:483
138013999219711,2184cyclictest1457305-21kworker/0:1+events23:07:560
138013999219311,2179cyclictest1388113-21kworker/0:0+events19:43:100
138014899219211,2179cyclictest1476715-21kworker/2:2+events00:25:052
138014899219110,2177cyclictest1400974-21kworker/2:1+events20:14:272
138014399218611,2173cyclictest1418716-21kworker/1:0+events21:13:261
138014899218211,2168cyclictest1380871-21kworker/2:0+events19:14:392
138014399218210,2169cyclictest1379776-21kworker/1:1+events19:33:411
13801399921797,2169cyclictest1465683-21kworker/0:0+events23:36:210
138014899217711,2162cyclictest1400974-21kworker/2:1+events20:47:292
138014899217611,2161cyclictest1460603-21kworker/2:1+events23:23:142
138014899216610,2153cyclictest1449076-21kworker/2:2+events22:48:252
138014899216511,2151cyclictest1434511-21kworker/2:0+events22:02:512
138014899216310,2150cyclictest1465386-21kworker/2:2+events23:42:042
138014899216210,2149cyclictest1468640-21kworker/2:0+events23:47:212
138014899216111,2146cyclictest1460603-21kworker/2:1+events23:19:472
13801489921578,2145cyclictest1400974-21kworker/2:1+events20:19:412
138014899215711,2142cyclictest1380871-21kworker/2:0+events19:27:562
138014899215111,2137cyclictest1378333-21kworker/2:1+events19:16:542
138015199214711,2132cyclictest1421835-21kworker/3:0+events21:19:313
138014399213111,2116cyclictest1379776-21kworker/1:1+events20:04:101
138015199212011,2106cyclictest1404724-21kworker/3:1+events20:28:063
13801519921188,2107cyclictest1455571-21kworker/3:1+events23:10:133
13801399920986,2089cyclictest1431611-21kworker/0:2+events22:26:510
138014399207610,2063cyclictest1383756-21kworker/1:0+events19:24:311
13801439920696,2060cyclictest1471753-21kworker/1:0+events23:59:591
13801519920688,2057cyclictest1379977-21kworker/3:2+events19:21:033
13801489920269,2014cyclictest1391575-21kworker/2:0+events19:50:492
138015199201410,2001cyclictest1445595-21kworker/3:1+events22:35:353
138015199200810,1995cyclictest1381692-21kworker/3:0+events19:18:453
13801399920024,1995cyclictest1376703-21kworker/0:0+events19:13:310
138015199198311,1969cyclictest1473624-21kworker/3:1+events00:09:093
13801519919748,1963cyclictest1465367-21kworker/3:0+events23:33:423
13801519919748,1963cyclictest1465367-21kworker/3:0+events23:33:423
13801519919619,1949cyclictest1402632-21kworker/3:0+events20:34:563
13801519919604,1953cyclictest1453939-21kworker/3:0+events23:04:063
13801519919499,1937cyclictest1401032-21kworker/3:1+events20:16:153
13801519919237,1913cyclictest1402632-21kworker/3:0+events20:50:363
138015199190610,1893cyclictest1460968-21kworker/3:2+events23:25:313
138015199157153,2cyclictest1481347-21kworker/3:0+pm00:29:533
13798892149105,36sleep20-21swapper/219:08:412
13798222145110,29sleep30-21swapper/319:07:503
13796912142110,27sleep10-21swapper/119:06:081
13798292141109,27sleep00-21swapper/019:07:550
138015199276,8cyclictest1436942-21hddtemp_smartct22:05:153
1380151992315,6cyclictest1477242-21cstates00:10:123
138015199215,4cyclictest1460549-21taskset23:18:093
1380151992113,6cyclictest1457563-21awk23:09:593
1380151992017,1cyclictest1450214-21ntpq22:45:173
138015199199,8cyclictest1462506-21cat23:25:013
138015199195,7cyclictest1416163-21irqstats21:00:153
138015199194,7cyclictest1409676-21meminfo20:40:173
1380151991915,3cyclictest865-10wireplumber00:15:183
1380151991914,3cyclictest0-21swapper/322:25:003
138015199191,15cyclictest739-21runrttasks20:55:213
138015199187,8cyclictest1388434-21idleruntime-cro19:35:003
138015199187,7cyclictest1399559-21date20:10:003
1380151991813,3cyclictest1425482-21apt-key21:30:003
138015199180,4cyclictest1439858-21/usr/sbin/munin22:15:223
138015199180,16cyclictest1422409-21/usr/sbin/munin21:20:163
138015199176,9cyclictest1381687-21sh19:15:003
138015199176,5cyclictest1452292-21sshd22:51:593
138015199175,7cyclictest1466706-21python323:35:203
138015199175,11cyclictest1470308-21chrt23:48:033
138015199174,2cyclictest1429333-21ntpq21:40:183
138015199173,6cyclictest1392439-21/usr/sbin/munin19:45:203
1380151991714,2cyclictest710-21mta-sts-daemon21:35:203
138015199169,5cyclictest1444274-21chrt22:28:553
138015199169,5cyclictest1411128-21apt-get20:45:133
138015199168,7cyclictest0-21swapper/321:30:013
138015199168,6cyclictest1418123-21awk21:05:283
138015199167,7cyclictest1419566-21ntp_kernel_pll_21:10:193
138015199167,6cyclictest1435664-21ntpq22:00:183
1380151991613,2cyclictest1386873-21systemd-run19:30:003
1380151991612,3cyclictest1434787-21runrttasks21:59:203
1380151991610,4cyclictest1471784-21sshd23:51:363
138015199160,4cyclictest1468575-21runrttasks23:41:043
138015199160,10cyclictest1389078-21meminfo19:35:163
138015199158,5cyclictest1431324-21chrt21:45:263
138015199156,7cyclictest1445063-21missed_timers22:30:183
1380151991513,1cyclictest0-21swapper/319:50:193
1380151991510,3cyclictest0-21swapper/300:05:013
138015199150,2cyclictest1480741-21sed00:20:173
138015199142,8cyclictest1390263-21/usr/sbin/munin19:40:113
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional