You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-25 - 18:20

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #f, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails
  
Click here to display the system's profile data.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackfslot7s.osadl.org (updated Thu Jun 25, 2026 00:44:18)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
139266599568022,5650cyclictest1426421-21kworker/0:0+events21:14:590
139267399413918,4116cyclictest1487113-21kworker/2:0+events00:10:192
139266899402319,3998cyclictest1443398-21kworker/1:0+events21:55:141
139267399360317,3582cyclictest1437502-21kworker/2:0+events21:30:112
139266899360218,3579cyclictest1458703-21kworker/1:1+events22:40:111
139266899342216,3403cyclictest1461773-21kworker/1:0+events22:55:191
139266899342216,3403cyclictest1461773-21kworker/1:0+events22:55:191
139267399342116,3401cyclictest1397753-21kworker/2:1+events20:00:422
139266599333517,3308cyclictest1430390-21kworker/0:1+events21:07:420
139266599320813,3190cyclictest1434069-21kworker/0:1+events21:20:170
139266599312815,3109cyclictest1426421-21kworker/0:0+events21:50:160
139268199307215,3052cyclictest1457004-21kworker/3:1+events22:30:013
139266599305814,3038cyclictest1410423-21kworker/0:1+events20:10:150
139266899305314,3035cyclictest1397686-21kworker/1:1+events19:25:161
13926739930113,3003cyclictest1424673-21kworker/2:0+events20:50:152
13926659930088,2996cyclictest1458762-21kworker/0:2+events22:49:590
139266899300514,2987cyclictest1418359-21kworker/1:1+events20:43:471
13926659929999,2986cyclictest1410423-21kworker/0:1+events20:20:560
139267399298414,2965cyclictest1449056-21kworker/2:1+events22:11:372
139266899298414,2966cyclictest1435727-21kworker/1:2+events21:45:251
139267399298214,2964cyclictest1408689-21kworker/2:0+events19:58:472
139267399298210,2968cyclictest1435639-21kworker/2:2+events21:23:382
139267399298114,2963cyclictest1437502-21kworker/2:0+events21:38:112
139266899298114,2963cyclictest1429297-21kworker/1:0+events21:01:391
139266899298113,2963cyclictest1476183-21kworker/1:2+events23:39:161
139267399297714,2959cyclictest1447405-21kworker/2:0+events22:01:422
139266899297614,2958cyclictest1418359-21kworker/1:1+events20:48:441
139266599297414,2956cyclictest1423224-21kworker/0:1+events20:58:360
139266599297014,2952cyclictest1426421-21kworker/0:0+events21:32:520
139266599297012,2954cyclictest1435785-21kworker/0:2+events21:27:070
13926689929620,2957cyclictest1450598-21kworker/1:0+events22:11:141
139266599293913,2922cyclictest1463904-21kworker/0:0+events22:54:350
139266599293714,2919cyclictest1426421-21kworker/0:0+events22:13:570
139267399293013,2912cyclictest1410378-21kworker/2:2+events20:13:402
139266599292713,2910cyclictest1399141-21kworker/0:0+events19:35:140
139266899291213,2895cyclictest1458940-21kworker/1:3+events22:39:251
139266599290814,2889cyclictest1490520-21kworker/0:0+events00:18:450
139266599287913,2860cyclictest1469708-21kworker/0:2+events23:14:060
13926689928787,2867cyclictest1447252-21kworker/1:1+events22:05:521
139266899287013,2853cyclictest1415070-21kworker/1:1+events20:18:381
139267399285913,2842cyclictest1397753-21kworker/2:1+events19:44:202
13926739928579,2844cyclictest1463548-21kworker/2:0+events23:40:212
139267399285714,2839cyclictest1427938-21kworker/2:2+events21:08:072
139266899285113,2834cyclictest1485489-21kworker/1:2+events00:02:241
139266899285113,2834cyclictest1437616-21kworker/1:1+events21:32:301
139266599285112,2835cyclictest1423224-21kworker/0:1+events20:46:230
139267399284613,2831cyclictest1426272-21kworker/2:1+events20:55:092
139266899284513,2828cyclictest1424823-21kworker/1:2+events20:52:281
139266899284511,2830cyclictest1412010-21kworker/1:0+events20:24:421
139266899284413,2827cyclictest1447252-21kworker/1:1+events22:18:551
139268199284314,2825cyclictest1407241-21kworker/3:0+events20:05:393
139266899284314,2826cyclictest1465028-21kworker/1:2+events23:05:111
139267399284213,2825cyclictest1477885-21kworker/2:1+events23:38:532
139266899284213,2825cyclictest1427670-21kworker/1:1+events21:18:431
139267399284013,2822cyclictest1450687-21kworker/2:2+events22:37:302
13926739928399,2826cyclictest1450687-21kworker/2:2+events22:30:222
13926659928349,2821cyclictest1419968-21kworker/0:2+events20:38:470
139266599283310,2819cyclictest1390642-21kworker/0:1+events19:19:320
139267399283013,2813cyclictest1471288-21kworker/2:2+events23:12:562
139267399283013,2813cyclictest1431175-21kworker/2:0+events21:15:232
139266899282913,2812cyclictest1405271-21kworker/1:2+events19:49:391
139266899282913,2812cyclictest1399334-21kworker/1:3+events19:34:051
139267399282813,2810cyclictest1439228-21kworker/2:2+events21:58:192
139267399282613,2809cyclictest1415283-21kworker/2:1+events20:25:512
139267399282513,2808cyclictest1487113-21kworker/2:0+events00:08:282
139267399282513,2808cyclictest1449056-21kworker/2:1+events22:09:412
139267399282513,2808cyclictest1421363-21kworker/2:0+events20:40:422
139267399282513,2808cyclictest1413674-21kworker/2:0+events20:17:312
139266899282511,2810cyclictest1435727-21kworker/1:2+events21:25:361
139266899282413,2807cyclictest1497051-21kworker/1:0+events00:36:411
139266899282413,2807cyclictest1455198-21kworker/1:0+events22:31:541
139266899282413,2807cyclictest1396035-21kworker/1:0+events19:20:191
139267399282313,2806cyclictest1463548-21kworker/2:0+events22:49:392
139266599282313,2805cyclictest1483886-21kworker/0:2+events23:55:120
139267399282213,2805cyclictest1427938-21kworker/2:2+events21:02:462
139267399282210,2808cyclictest1457064-21kworker/2:0+events22:28:502
139267399282013,2803cyclictest1437502-21kworker/2:0+events21:27:292
139267399282013,2803cyclictest1431175-21kworker/2:0+events21:10:242
139266899282011,2807cyclictest1427670-21kworker/1:1+events21:05:031
139266899282011,2804cyclictest1405271-21kworker/1:2+events20:09:291
139267399281911,2804cyclictest1460265-21kworker/2:0+events22:40:562
139266899281814,2802cyclictest1439316-21kworker/1:0+events21:35:061
139266899281813,2801cyclictest1402471-21kworker/1:0+events19:55:191
13926739928179,2804cyclictest1461919-21kworker/2:1+events22:52:132
139266599281713,2800cyclictest1416698-21kworker/0:0+events20:27:230
139267399281613,2799cyclictest1402501-21kworker/2:0+events19:39:292
139266599281613,2797cyclictest1426421-21kworker/0:0+events21:16:280
139267399281513,2799cyclictest1450687-21kworker/2:2+events22:15:052
139266899281513,2798cyclictest1488196-21kworker/1:0+events00:19:301
139266899281513,2798cyclictest1397686-21kworker/1:1+events19:40:581
139266899281512,2799cyclictest1435727-21kworker/1:2+events21:23:151
139267399281313,2796cyclictest1474581-21kworker/2:1+events23:27:482
139267399281313,2796cyclictest1423042-21kworker/2:2+events20:47:332
139266599281313,2795cyclictest1488918-21kworker/0:1+events00:11:040
139266899281213,2795cyclictest1472840-21kworker/1:0+events23:26:391
13926659928129,2799cyclictest1455289-21kworker/0:2+events22:26:310
13926659928129,2799cyclictest1426421-21kworker/0:0+events21:36:130
139266599281213,2795cyclictest1421597-21kworker/0:0+events20:43:020
139266599281113,2794cyclictest1469708-21kworker/0:2+events23:07:300
139266599281110,2797cyclictest1426421-21kworker/0:0+events22:15:280
139266599281012,2794cyclictest1390642-21kworker/0:1+events19:27:320
139266899280913,2794cyclictest1492094-21kworker/1:1+events00:27:541
139266899280913,2792cyclictest1435727-21kworker/1:2+events21:53:431
139266899280913,2791cyclictest1418359-21kworker/1:1+events20:29:211
139266599280913,2792cyclictest1488918-21kworker/0:1+events00:21:470
139266599280913,2792cyclictest1397538-21kworker/0:2+events19:34:280
139266599280912,2793cyclictest1458762-21kworker/0:2+events22:43:330
139266599280813,2793cyclictest1469708-21kworker/0:2+events23:25:100
139266599280813,2792cyclictest1471349-21kworker/0:1+events23:15:150
139267399280713,2790cyclictest1482406-21kworker/2:1+events23:55:332
139266599280713,2789cyclictest1483886-21kworker/0:2+events00:06:080
139267399280613,2788cyclictest1463548-21kworker/2:0+events23:05:332
139267399280613,2788cyclictest1405728-21kworker/2:2+events19:48:542
139266599280613,2789cyclictest1458762-21kworker/0:2+events22:39:490
139267399280513,2788cyclictest1471288-21kworker/2:2+events23:17:332
139266599280513,2788cyclictest1397538-21kworker/0:2+events19:42:050
139266599280511,2790cyclictest1407210-21kworker/0:2+events19:57:160
139267399280413,2789cyclictest1439228-21kworker/2:2+events21:41:172
139267399280213,2785cyclictest1415283-21kworker/2:1+events20:35:482
139266899280113,2786cyclictest1412010-21kworker/1:0+events20:35:031
13926739927999,2786cyclictest1400572-21kworker/2:2+events19:32:312
139266899279811,2783cyclictest1465028-21kworker/1:2+events22:53:241
139267399279713,2782cyclictest1410378-21kworker/2:2+events20:06:032
139266599279713,2780cyclictest1419968-21kworker/0:2+events20:51:220
139266599279713,2780cyclictest1390642-21kworker/0:1+events19:22:570
139266899279612,2782cyclictest1435727-21kworker/1:2+events21:40:071
139266599279613,2779cyclictest1428092-21kworker/0:2+events21:04:170
139266899279511,2782cyclictest1479393-21kworker/1:1+events23:56:181
139266899279511,2780cyclictest1402471-21kworker/1:0+events19:39:531
139266599279513,2777cyclictest1410423-21kworker/0:1+events20:02:580
139266599279313,2776cyclictest1477554-21kworker/0:0+events23:45:150
139266899279213,2775cyclictest1481010-21kworker/1:0+events23:49:041
139267399278913,2772cyclictest1390547-21kworker/2:0+events19:18:032
139268199278810,2774cyclictest1413452-21kworker/3:1+events20:21:193
139266599278713,2770cyclictest1452056-21kworker/0:1+events22:30:460
139267399278612,2770cyclictest1444438-21kworker/2:3+events21:54:502
139266899278613,2769cyclictest1471194-21kworker/1:1+events23:11:231
139267399278413,2767cyclictest1463548-21kworker/2:0+events23:31:412
139266899278413,2767cyclictest1474491-21kworker/1:1+events23:30:311
139266599278413,2767cyclictest1410423-21kworker/0:1+events20:15:580
139266899278312,2767cyclictest1447252-21kworker/1:1+events22:02:511
139266899278212,2766cyclictest1492094-21kworker/1:1+events00:31:191
139266899278013,2763cyclictest1455198-21kworker/1:0+events22:24:391
139266899278013,2763cyclictest1432468-21kworker/1:2+events21:13:511
139266899277912,2763cyclictest1418359-21kworker/1:1+events20:32:211
13926659927798,2767cyclictest1462064-21kworker/0:1+events22:59:300
13926659927798,2767cyclictest1462064-21kworker/0:1+events22:59:300
139267399277813,2761cyclictest1463548-21kworker/2:0+events22:56:062
139267399277813,2761cyclictest1463548-21kworker/2:0+events22:56:052
139267399277713,2760cyclictest1490366-21kworker/2:2+events00:20:172
139267399277710,2762cyclictest1397753-21kworker/2:1+events19:51:102
139266899277713,2760cyclictest1488196-21kworker/1:0+events00:08:481
139267399277612,2760cyclictest1395946-21kworker/2:2+events19:20:422
139267399277513,2758cyclictest1490366-21kworker/2:2+events00:15:422
139267399277513,2758cyclictest1415283-21kworker/2:1+events20:23:582
139268199277412,2758cyclictest1479451-21kworker/3:2+events23:52:513
139266599277413,2757cyclictest1477554-21kworker/0:0+events23:32:050
139267399277313,2756cyclictest1493630-21kworker/2:1+events00:28:412
139267399277213,2755cyclictest1415283-21kworker/2:1+events20:31:582
139267399277113,2754cyclictest1465150-21kworker/2:2+events23:04:242
139267399277113,2754cyclictest1465150-21kworker/2:2+events23:04:232
139267399277110,2757cyclictest1390547-21kworker/2:0+events19:29:512
139266899277113,2754cyclictest1466782-21kworker/1:1+events23:02:531
139266899277113,2754cyclictest1466782-21kworker/1:1+events23:02:521
139266899277113,2754cyclictest1412259-21kworker/1:3+events20:14:031
139266899277113,2754cyclictest1405271-21kworker/1:2+events19:54:101
139266599276913,2752cyclictest1426421-21kworker/0:0+events21:43:120
139267399276813,2753cyclictest1463548-21kworker/2:0+events23:47:092
139267399276813,2751cyclictest1493630-21kworker/2:1+events00:37:022
139267399276813,2751cyclictest1484125-21kworker/2:2+events00:02:472
139266599276713,2750cyclictest1419968-21kworker/0:2+events20:33:540
139266599276713,2750cyclictest1407210-21kworker/0:2+events20:06:480
139266599276711,2752cyclictest1477554-21kworker/0:0+events23:38:320
139266599276412,2748cyclictest1426421-21kworker/0:0+events21:46:310
139266599276313,2746cyclictest1447221-21kworker/0:2+events21:58:390
139266599276313,2746cyclictest1426421-21kworker/0:0+events22:08:550
139266599276313,2746cyclictest1407210-21kworker/0:2+events19:54:350
139266599276212,2746cyclictest1496810-21kworker/0:1+events00:31:430
139266599276013,2743cyclictest1462064-21kworker/0:1+events23:01:220
139266599276013,2743cyclictest1462064-21kworker/0:1+events23:01:210
139266599275913,2742cyclictest1447221-21kworker/0:2+events22:00:330
139266899275713,2740cyclictest1479393-21kworker/1:1+events23:51:211
139266899275311,2738cyclictest1410349-21kworker/1:1+events20:02:381
139266599274313,2726cyclictest1472989-21kworker/0:0+events23:22:500
139266599272213,2705cyclictest1493796-21kworker/0:0+events00:25:570
139267399271713,2700cyclictest1471288-21kworker/2:2+events23:20:122
139266899271012,2695cyclictest1447252-21kworker/1:1+events22:25:241
139268199269213,2675cyclictest1448906-21kworker/3:2+events22:07:463
139266899268812,2672cyclictest1393997-21kworker/1:1+events19:18:251
139268199268313,2665cyclictest1437650-21kworker/3:0+events21:35:523
139268199268113,2664cyclictest1394198-21kworker/3:1+events19:35:373
139266899267913,2661cyclictest1427670-21kworker/1:1+events20:59:221
139267399267712,2661cyclictest1493630-21kworker/2:1+events00:30:582
139268199265413,2637cyclictest1492240-21kworker/3:1+events00:28:173
139268199264613,2629cyclictest1487349-21kworker/3:0+events00:24:033
139268199264213,2625cyclictest1487349-21kworker/3:0+events00:13:243
139268199264212,2626cyclictest1455137-21kworker/3:2+events22:40:343
139268199264113,2624cyclictest1416669-21kworker/3:2+events21:20:383
139268199264113,2624cyclictest1413452-21kworker/3:1+events20:11:023
139268199264013,2623cyclictest1442428-21kworker/3:1+events21:49:513
139268199263913,2622cyclictest1416669-21kworker/3:2+events20:34:413
139268199263813,2621cyclictest1452342-21kworker/3:0+events22:20:483
139268199263813,2621cyclictest1402350-21kworker/3:2+events19:52:193
139268199263713,2622cyclictest1440621-21kworker/3:2+events21:43:343
139268199263713,2620cyclictest1450538-21kworker/3:1+events22:18:073
139268199263613,2619cyclictest1495185-21kworker/3:2+events00:34:013
139266599263512,2619cyclictest1452056-21kworker/0:1+events22:22:430
139268199263413,2618cyclictest1455137-21kworker/3:2+events22:47:433
139266899263313,2616cyclictest1474491-21kworker/1:1+events23:22:271
139266899262913,2612cyclictest1465028-21kworker/1:2+events23:18:421
139266899261212,2596cyclictest1476183-21kworker/1:2+events23:41:511
139268199260113,2584cyclictest1474698-21kworker/3:1+events23:39:593
139266599259512,2579cyclictest1477554-21kworker/0:0+events23:43:200
139268199259210,2579cyclictest1472809-21kworker/3:0+events23:24:463
139268199258711,2573cyclictest1463458-21kworker/3:1+events23:13:423
139266599257713,2560cyclictest1483886-21kworker/0:2+events23:51:430
139266899256512,2549cyclictest1488196-21kworker/1:0+events00:13:461
139267399255711,2542cyclictest1450687-21kworker/2:2+events22:23:532
139267399255012,2534cyclictest1484125-21kworker/2:2+events23:54:472
139267399255012,2534cyclictest1442474-21kworker/2:0+events21:46:082
139266899254912,2533cyclictest1392279-21kworker/1:2+events19:14:431
13926689925448,2532cyclictest1492094-21kworker/1:1+events00:22:551
139266899254212,2526cyclictest1463369-21kworker/1:1+events22:49:151
139266599253612,2521cyclictest1399141-21kworker/0:0+events19:46:380
139266599253210,2518cyclictest1479562-21kworker/0:1+events00:00:540
139266599253011,2515cyclictest1498558-21kworker/0:0+events00:39:430
139267399247712,2462cyclictest1390547-21kworker/2:0+events19:12:482
139266599244812,2433cyclictest1390642-21kworker/0:1+events19:12:250
139268199241812,2403cyclictest1479451-21kworker/3:2+events23:56:413
13926819924148,2402cyclictest1479451-21kworker/3:2+events23:48:423
139268199241412,2398cyclictest1444369-21kworker/3:0+events22:02:053
13926819924105,2401cyclictest1435608-21kworker/3:1+events21:28:133
139268199241012,2395cyclictest1490428-21kworker/3:2+events00:15:193
139268199238512,2369cyclictest1470063-21kworker/3:2+events23:16:033
139268199234012,2325cyclictest1485520-21kworker/3:1+events00:04:193
139268199217411,2160cyclictest1390761-21kworker/3:0+events19:17:413
139268199188510,1873cyclictest1390761-21kworker/3:0+events19:10:063
13924652160117,25sleep10-21swapper/119:09:581
13922582151108,23sleep30-21swapper/319:07:243
13923402150104,26sleep00-21swapper/019:08:260
13921272140107,26sleep20-21swapper/219:05:432
1392681995243,5cyclictest593-21dbus-daemon22:00:013
139268199392,30cyclictest1409072-21readlink20:00:003
139268199311,25cyclictest1438085-21taskset21:30:113
1392681993025,3cyclictest1481074-21idleruntime-cro23:45:013
139268199293,22cyclictest1460525-21sendmail-msp22:39:593
139268199291,23cyclictest1400879-21latency_hist19:35:003
139268199281,25cyclictest0-21swapper/320:59:593
139268199281,24cyclictest1410649-21idleruntime-cro20:05:003
139268199281,23cyclictest1466835-21awk23:00:003
139268199281,23cyclictest1466835-21awk22:59:593
139268199281,21cyclictest14446792sleep321:50:113
139268199281,19cyclictest1487692-21apt-get00:05:113
139268199271,21cyclictest1452329-21perf-exec22:15:003
139268199271,21cyclictest1424912-21awk20:50:003
1392681992622,2cyclictest1476335-21dump-pmu-power23:30:003
139268199262,19cyclictest1470007-21runrttasks23:09:593
139268199261,21cyclictest1458925-21idleruntime-cro22:35:003
139268199261,21cyclictest1432803-21perf21:15:003
139268199261,21cyclictest1431228-21basename21:09:593
139268199261,21cyclictest1415421-21tr20:20:003
139268199260,20cyclictest0-21swapper/321:00:133
1392681992521,2cyclictest1418571-21idleruntime-cro20:30:003
139268199251,21cyclictest1477909-21munin-run23:34:593
139268199251,21cyclictest1405914-21perf-exec19:50:013
139268199251,20cyclictest1498758-21apt-get00:40:003
1392681992417,5cyclictest1403294-21cat19:40:143
139268199241,19cyclictest1421741-21awk20:40:003
139268199240,21cyclictest1423274-21taskset20:44:393
139268199221,18cyclictest1433584-21ntpq21:15:203
139268199212,14cyclictest0-21swapper/319:20:113
139268199211,16cyclictest0-21swapper/319:25:113
139268199201,15cyclictest1468416-21munin-run23:05:003
139268199201,15cyclictest1468416-21munin-run23:05:003
139268199191,15cyclictest739-21runrttasks20:50:183
139268199191,14cyclictest1465370-21latency_hist22:55:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional