You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-12 - 13:33

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot4s.osadl.org (updated Fri Jun 12, 2026 00:44:14)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
290961421980,2sleep3471rcuc/321:20:403
27748482760,3chrt2774851-21unixbench_singl19:05:512
28306392720,5sleep20-21swapper/220:00:372
2775370997048,19cyclictest3075346-21apt-get00:10:000
29053542630,3sleep375-21kworker/3:1-events21:15:433
2775373996355,6cyclictest3019634-21ntpq23:10:391
2775373996255,5cyclictest3025168-21kworker/u8:0+flush-8:000:35:421
2775370996242,17cyclictest2875660-21ntpq20:45:400
2775374996139,19cyclictest3065457-21cp00:00:012
2775370995848,8cyclictest2856971-21/usr/sbin/munin20:30:370
2775373995751,4cyclictest2915245-21kworker/u8:0+flush-8:021:50:301
2775373995745,9cyclictest2915170-21ntpq21:25:401
29322692560,4chrt481ktimers/321:45:363
2775370995645,8cyclictest2955107-21ntpq22:05:430
2775373995550,3cyclictest2965757-21kworker/u8:1+flush-8:022:35:311
2775373995447,5cyclictest2950698-21kworker/u8:3+flush-8:022:10:471
2775373995447,5cyclictest2870764-21ntpq20:40:431
2775370995443,8cyclictest3065419-21cp00:00:000
2775373995347,3cyclictest2782424-21kworker/u8:3+flush-8:020:55:371
2775373995246,4cyclictest2851987-21kworker/u8:2+events_unbound20:30:451
2775370995240,9cyclictest2779365-21ntpq19:10:410
2775373995145,4cyclictest3025168-21kworker/u8:0+events_unbound00:05:451
2775373995144,5cyclictest2780272-21latency_hist19:15:011
30417042500,5chrt0-21swapper/323:35:363
27979792500,4chrt0-21swapper/019:30:340
2775374995041,6cyclictest2961332-21cat22:15:002
2775373995044,5cyclictest2915245-21kworker/u8:0+flush-8:022:30:361
2775373994943,4cyclictest3035318-21kworker/u8:3+flush-8:023:30:461
2775373994942,5cyclictest3025168-21kworker/u8:0+events_unbound00:25:441
2775373994939,7cyclictest3035318-21kworker/u8:3+events_unbound23:40:411
2775373994842,4cyclictest3025504-21apt-get23:20:041
2775373994839,7cyclictest2826982-21latency_hist20:00:011
2775370994835,10cyclictest3014724-21ntpq23:05:420
2775370994835,10cyclictest3014724-21ntpq23:05:410
2775373994743,3cyclictest2896971-21kworker/u8:1+flush-8:021:20:311
2775373994742,3cyclictest2915245-21kworker/u8:0+flush-8:022:20:331
2775373994741,4cyclictest3015831-21kworker/u8:2+events_unbound23:35:431
2775373994738,6cyclictest2905304-21ntpq21:15:411
2775373994737,7cyclictest2782424-21kworker/u8:3+flush-8:021:05:401
2775373994640,4cyclictest3000435-21kworker/u8:1+flush-8:023:50:361
2775373994640,4cyclictest2782424-21kworker/u8:3+flush-8:020:10:471
2775373994638,5cyclictest2896971-21kworker/u8:1+events_unbound21:35:431
2775373994637,6cyclictest2995801-21latency_hist22:50:011
2775373994539,4cyclictest2826621-21kworker/u8:0+events_unbound20:50:421
2775373994539,3cyclictest2950698-21kworker/u8:3+flush-8:022:05:471
2775373994538,4cyclictest3000435-21kworker/u8:1+events_unbound22:55:311
2775373994535,7cyclictest1704890-21systemd-journal22:20:001
2775370994539,5cyclictest2782424-21kworker/u8:3+flush-8:020:15:430
2775374994428,14cyclictest2876880-21aten2_r5power_p20:50:292
2775373994440,3cyclictest2826621-21kworker/u8:0+flush-8:020:00:441
2775373994438,4cyclictest2926267-21kworker/u8:2+flush-8:021:40:361
2775370994437,5cyclictest3100281-21latency_hist00:35:010
2775374994333,8cyclictest1704890-21systemd-journal21:18:372
2775373994338,4cyclictest2826621-21kworker/u8:0+events_unbound20:05:441
2775373994337,4cyclictest3095015-21kworker/u8:1+events_unbound00:30:401
2775373994336,5cyclictest3015831-21kworker/u8:2+events_unbound23:15:461
2775373994332,7cyclictest1704890-21systemd-journal00:19:401
2775374994232,8cyclictest1704890-21systemd-journal00:10:412
2775373994237,4cyclictest2926267-21kworker/u8:2+events_unbound21:45:311
2775373994237,4cyclictest2782424-21kworker/u8:3+flush-8:020:20:311
2775373994237,3cyclictest2826621-21kworker/u8:0+flush-8:021:00:361
2775373994237,3cyclictest2826621-21kworker/u8:0+flush-8:021:00:351
2775373994235,5cyclictest3030990-21apt-get23:25:261
2775373994234,6cyclictest2915245-21kworker/u8:0+flush-8:021:30:361
2775373994234,5cyclictest1952109-21kworker/u8:1+flush-8:019:25:471
2775373994135,4cyclictest2915245-21kworker/u8:0+events_unbound23:05:441
2775373994135,4cyclictest2915245-21kworker/u8:0+events_unbound23:05:441
2775370994134,5cyclictest3025168-21kworker/u8:0+flush-8:023:20:360
2775370994131,7cyclictest3095344-21latency_hist00:30:000
2775370994129,9cyclictest2852288-21diskstats20:25:310
2775373994033,5cyclictest2782424-21kworker/u8:3+events_unbound19:15:381
2775373994029,9cyclictest3089448-21ntpq00:20:401
2775370994035,3cyclictest2915245-21kworker/u8:0+flush-8:022:40:470
277537099402,32cyclictest2856754-21kworker/0:0+events21:50:170
2775374993924,13cyclictest2787669-21cpuspeed_turbos19:20:312
2775374993924,13cyclictest2787669-21cpuspeed_turbos19:20:302
2775374993918,18cyclictest1704890-21systemd-journal20:09:592
2775373993935,3cyclictest2782424-21kworker/u8:3+flush-8:020:45:441
2775373993932,5cyclictest2926267-21kworker/u8:2+flush-8:021:55:421
2775370993931,6cyclictest3005969-21/usr/sbin/munin23:00:410
2775370993825,10cyclictest2802528-21diskstats19:35:310
2775370993817,19cyclictest2856754-21kworker/0:0+events21:35:390
277537099381,32cyclictest3045593-21kworker/0:1+events00:10:080
2775374993722,13cyclictest3095677-21aten2_r5power_e00:30:272
2775373993731,4cyclictest3065738-21apt-get00:00:271
2775373993730,4cyclictest1704890-21systemd-journal22:43:401
2775370993729,6cyclictest1704890-21systemd-journal23:32:580
2775370993729,6cyclictest1704890-21systemd-journal00:05:000
2775370993727,8cyclictest2910254-21ntpq21:20:430
277537499361,28cyclictest2978839-21kthreadcore22:30:432
277537499360,21cyclictest28100672kthreadcore19:40:382
2775373993631,3cyclictest2926267-21kworker/u8:2+flush-8:022:00:371
2775373993630,4cyclictest2782424-21kworker/u8:3+flush-8:019:30:301
2775373993629,4cyclictest3008369-21kworker/u8:3+events_unbound23:00:461
2775373993628,6cyclictest2801975-21kworker/u8:0+events_unbound19:40:411
277537099363,31cyclictest3030730-21kworker/0:0+events23:45:430
2775370993628,6cyclictest2950118-21ntpq22:00:400
27749812364,10sleep10-21swapper/119:07:081
2775374993521,12cyclictest2896939-21fschecks_time21:10:302
277537499351,28cyclictest2952874-21kthreadcore22:05:392
2775373993530,4cyclictest3065034-21kworker/u8:2+flush-8:000:10:431
2775373993529,4cyclictest2851987-21kworker/u8:2+events_unbound20:35:291
2775373993528,5cyclictest1952109-21kworker/u8:1+flush-8:019:20:431
2775373993528,5cyclictest1952109-21kworker/u8:1+flush-8:019:20:431
2775370993525,8cyclictest2930176-21ntpq21:40:440
28334822340,5sleep3451irq_work/320:05:353
277537499342,25cyclictest2911464-21apt-get21:25:282
2775374993419,13cyclictest2940880-21apt-get21:55:002
2775374993411,11cyclictest0-21swapper/223:50:402
2775374993410,8cyclictest28568062latency_hist20:30:012
2775373993429,3cyclictest2926267-21kworker/u8:2+events_unbound22:50:381
2775373993429,3cyclictest2817324-21kworker/u8:1+events_unbound19:50:481
2775370993426,6cyclictest1704890-21systemd-journal23:30:010
2775370993424,8cyclictest2827252-21/usr/sbin/munin20:00:380
2775370993424,8cyclictest2815983-21ntpq19:45:410
2775374993319,12cyclictest2971580-21cpuspeed_turbos22:25:292
277537499331,25cyclictest2916376-21apt-get21:30:302
2775370993310,22cyclictest3090422-21kworker/0:0+events00:35:430
2775374993226,4cyclictest3025504-21apt-get23:20:012
277537499322,22cyclictest2986191-21apt-get22:40:292
277537099329,18cyclictest2856754-21kworker/0:0+events20:55:010
277537099328,22cyclictest2775248-21kworker/0:1+events19:50:470
2775370993223,6cyclictest2981215-21apt-get22:35:270
277537099322,25cyclictest2995821-21kworker/0:1+events22:55:010
2775370993212,15cyclictest2856754-21kworker/0:0+events22:15:030
2775370993210,21cyclictest3030730-21kworker/0:0+events23:35:550
277537499312,23cyclictest3085753-21apt-get00:20:282
277537499312,23cyclictest2871867-21apt-get20:45:272
2775374993119,10cyclictest2867222-21fschecks_count20:40:312
2775373993125,5cyclictest2782424-21kworker/u8:3+flush-8:020:15:421
2775373993125,4cyclictest2801975-21kworker/u8:0+flush-8:019:36:411
277537099319,16cyclictest2856754-21kworker/0:0+events21:10:270
277537099318,22cyclictest2886459-21kworker/0:1+events21:05:270
2775370993126,4cyclictest2782424-21kworker/u8:3+flush-8:021:25:370
2775370993126,3cyclictest2833161-21kthreadcore20:05:360
277537099312,28cyclictest2856754-21kworker/0:0+events20:55:370
277537099312,27cyclictest3005749-21kworker/0:2+events23:15:310
2775370993122,7cyclictest1704890-21systemd-journal23:15:000
2775370993122,7cyclictest1704890-21systemd-journal19:40:350
2775370993110,19cyclictest2856754-21kworker/0:0+events21:20:000
2775370993110,16cyclictest2995821-21kworker/0:1+events22:55:280
2775374993025,3cyclictest2862010-21aten2_r5power_c20:35:292
2775374993015,12cyclictest0-21swapper/221:45:352
2775374993010,18cyclictest3055472-21cron23:50:002
277537499300,18cyclictest3043036-21/usr/sbin/munin23:35:382
277537099302,26cyclictest3045593-21kworker/0:1+events00:20:370
2775370993020,8cyclictest2837167-21/usr/sbin/munin20:10:440
2775370993012,16cyclictest2856754-21kworker/0:0+events21:45:280
2775370993011,17cyclictest2856754-21kworker/0:0+events22:10:310
2775370993011,14cyclictest3045593-21kworker/0:1+events23:50:290
2775370993010,15cyclictest2856754-21kworker/0:0+events20:40:270
2775374992925,3cyclictest2891967-21fschecks_time21:05:302
2775374992916,11cyclictest0-21swapper/200:10:012
2775374992914,13cyclictest2961721-21uniq22:15:322
277537499291,22cyclictest2840981-21ntpq20:10:432
277537499290,25cyclictest3065722-21/usr/sbin/munin00:00:262
2775373992925,2cyclictest0-21swapper/119:50:001
277537099295,19cyclictest2775248-21kworker/0:1+events19:30:000
2775370992925,3cyclictest2896971-21kworker/u8:1+flush-8:021:30:050
277537099292,26cyclictest2775248-21kworker/0:1+events19:15:320
277537099291,27cyclictest2856754-21kworker/0:0+events21:55:330
277537499288,17cyclictest3081223-21awk00:15:322
277537499287,17cyclictest3006800-21sed23:00:352
277537499282,21cyclictest3049605-21ntpq23:40:412
2775374992821,5cyclictest3026281-21diskstats23:20:322
277537499281,24cyclictest2946803-21iwlist22:00:332
277537499281,23cyclictest2981734-21/usr/sbin/munin22:35:362
277537499281,23cyclictest2887429-21/usr/sbin/munin21:00:342
277537499281,23cyclictest2887429-21/usr/sbin/munin21:00:342
2775374992812,14cyclictest2944603-21cut21:55:362
277537499280,24cyclictest3005216-21/usr/sbin/munin22:55:502
2775373992824,3cyclictest3000435-21kworker/u8:1+flush-8:023:45:461
2775373992821,6cyclictest2926267-21kworker/u8:2+flush-8:022:25:411
277537099287,19cyclictest3045593-21kworker/0:1+events00:16:190
277537099286,17cyclictest2775248-21kworker/0:1+events19:55:150
2775370992811,15cyclictest2856754-21kworker/0:0+events20:35:490
277537099280,26cyclictest2856754-21kworker/0:0+events22:30:430
277537099280,26cyclictest2856754-21kworker/0:0+events22:20:460
277537499279,14cyclictest3101081-21irqcore00:35:352
277537499278,15cyclictest2966605-21tr22:20:312
2775374992724,2cyclictest2801566-21unixbench_multi19:30:452
2775374992723,2cyclictest2792469-21aten2_r5power_p19:25:302
277537499271,24cyclictest2925366-21/usr/sbin/munin21:35:462
277537499271,22cyclictest3011673-21ps23:05:352
277537499271,22cyclictest3011673-21ps23:05:352
277537499271,22cyclictest2815973-21ntpq19:45:412
2775374992712,13cyclictest3000371-21unixbench_singl22:50:492
277537499270,25cyclictest2821895-21chrt19:53:062
277537499270,24cyclictest2993402-21cut22:45:382
277537499270,24cyclictest2806088-21awk19:35:422
277537499270,23cyclictest3031715-21grep23:25:352
277537499270,23cyclictest2843021-21/usr/sbin/munin20:15:352
277537499270,23cyclictest2826352-21sort19:55:452
2775373992723,2cyclictest2854211-21kthreadcore20:25:371
277537099279,16cyclictest2856754-21kworker/0:0+events22:25:350
2775370992717,8cyclictest2886966-21diskstats21:00:310
2775370992717,8cyclictest2886966-21diskstats21:00:300
277537499268,15cyclictest2777726-21tr19:10:382
2775374992623,2cyclictest3091015-21fschecks_time00:25:322
2775374992622,2cyclictest2926345-21aten2_r5power_c21:40:302
277537499262,19cyclictest2885579-21ntpq20:55:422
2775374992613,10cyclictest2847802-21irqrtprio20:20:352
277537499261,21cyclictest3035927-21runrttasks23:30:272
277537499260,22cyclictest2910513-21mailstats21:20:472
2775373992619,5cyclictest2851987-21kworker/u8:2+flush-8:021:10:411
277537099267,17cyclictest2856754-21kworker/0:0+events22:45:310
27752042264,12sleep01-21systemd19:10:000
277537499251,21cyclictest3019626-21ntpq23:10:392
277537499251,21cyclictest2857231-21seq20:30:292
277537499250,22cyclictest2780988-21sort19:15:032
277537099258,16cyclictest3030730-21kworker/0:0+events23:45:000
2775373992420,3cyclictest3035318-21kworker/u8:3+flush-8:023:55:401
277537099247,16cyclictest2775248-21kworker/0:1+events19:20:370
277537099247,16cyclictest2775248-21kworker/0:1+events19:20:370
277537099228,13cyclictest2827018-21kworker/0:2+events20:20:130
29503502200,2sleep3481ktimers/322:00:453
27703602182,7sleep30-21swapper/319:05:003
28679412170,3chrt75-21kworker/3:1-events20:40:343
31008002144,6sleep30-21swapper/300:35:313
30793642142,8sleep30-21swapper/300:10:403
28838822142,7sleep30-21swapper/320:55:363
30704982132,6sleep30-21swapper/300:05:023
30531212132,7sleep30-21swapper/323:45:393
30047982132,7sleep30-21swapper/322:55:443
29969792133,7sleep30-21swapper/322:50:383
29710932132,6sleep30-21swapper/322:25:023
29399382134,5sleep30-21swapper/321:50:393
28459692133,6sleep30-21swapper/320:15:443
28081492133,5sleep30-21swapper/319:40:373
30804782122,5sleep30-21swapper/300:15:013
30666252122,6sleep30-21swapper/300:00:363
30612012122,6sleep30-21swapper/323:55:353
30603582122,6sleep30-21swapper/323:54:393
30455802122,6sleep30-21swapper/323:40:023
30404592122,6sleep30-21swapper/323:33:193
30313612122,6sleep30-21swapper/323:25:333
30260062122,6sleep30-21swapper/323:20:283
30240032122,6sleep30-21swapper/323:15:373
30161932122,6sleep30-21swapper/323:10:333
30057262122,5sleep30-21swapper/323:00:013
29956052122,6sleep30-21swapper/322:47:403
29908292122,6sleep30-21swapper/322:45:003
29850552122,6sleep30-21swapper/322:35:453
29805932122,6sleep30-21swapper/322:30:553
29702112122,6sleep30-21swapper/322:20:453
29552022122,6sleep30-21swapper/322:05:483
29452582122,6sleep30-21swapper/321:55:443
29210802122,5sleep30-21swapper/321:35:023
29199282122,6sleep30-21swapper/321:30:403
29002782122,6sleep30-21swapper/321:10:383
28953622122,6sleep30-21swapper/321:05:393
28866912122,6sleep30-21swapper/321:00:273
28866912122,6sleep30-21swapper/321:00:273
28716702122,5sleep30-21swapper/320:45:023
28642582122,6sleep30-21swapper/320:35:393
28574522122,6sleep30-21swapper/320:30:333
28520142122,7sleep30-21swapper/320:25:263
28468262122,5sleep30-21swapper/320:20:023
28371962122,6sleep30-21swapper/320:10:313
28318312122,6sleep30-21swapper/320:03:363
28220652122,6sleep30-21swapper/319:55:023
28152562123,6sleep30-21swapper/319:45:393
28063772122,6sleep30-21swapper/319:35:463
28013432122,6sleep30-21swapper/319:30:463
27967102122,7sleep30-21swapper/319:26:163
27917592122,6sleep30-21swapper/319:20:553
27917592122,6sleep30-21swapper/319:20:543
27824742122,6sleep30-21swapper/319:15:293
27760382122,6sleep30-21swapper/319:10:353
30955622112,5sleep30-21swapper/300:30:023
30915832112,5sleep30-21swapper/300:25:373
30862632112,5sleep30-21swapper/300:20:343
30147342112,5sleep30-21swapper/323:05:433
30147342112,5sleep30-21swapper/323:05:423
29613722112,4sleep30-21swapper/322:15:023
29607402112,6sleep30-21swapper/322:12:543
29267542112,5sleep30-21swapper/321:40:353
29114062112,5sleep30-21swapper/321:25:023
28767692112,5sleep30-21swapper/320:50:283
28173512112,5sleep30-21swapper/319:50:273
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional