You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-24 - 03:47

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack5slot4s.osadl.org (updated Mon Aug 24, 2026 00:44:14)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3081254996151,7cyclictest275-21systemd-journal19:37:371
3081254995749,5cyclictest3411039-21latency_hist00:40:011
3081250995750,5cyclictest3173323-21diskstats20:42:380
3081254995448,4cyclictest2954217-21kworker/u8:0+events_unbound20:10:011
3081250995343,7cyclictest3081094-21latency_hist19:10:010
3081254995248,3cyclictest3221716-21kworker/u8:0+events_unbound22:27:381
3081254995247,4cyclictest3221716-21kworker/u8:0+flush-8:022:02:391
3081250995142,6cyclictest3306918-21latency_hist22:55:010
3081250995142,6cyclictest2954217-21kworker/u8:0+flush-8:019:22:380
3081250995142,6cyclictest2954217-21kworker/u8:0+flush-8:019:22:380
31728122500,2chrt291ktimers/120:40:011
3081254994841,5cyclictest3272166-21base6422:20:011
3081254994839,7cyclictest3197852-21kworker/u8:1+flush-8:021:07:481
3081254994742,4cyclictest1773283-21kworker/u8:3+flush-8:021:32:391
3081254994742,3cyclictest3296863-21kworker/u8:1+events_unbound23:12:371
3081254994740,5cyclictest2954217-21kworker/u8:0+flush-8:021:02:491
3081254994641,4cyclictest3371220-21kworker/u8:2+events_unbound00:22:421
3081250994638,5cyclictest3395974-21apt-get00:25:030
3081250994637,6cyclictest1-21systemd19:42:370
3081254994541,3cyclictest3261969-21kworker/u8:1+flush-8:022:22:381
3081254994540,4cyclictest3281449-21kworker/u8:3+events_unbound22:57:451
3081254994538,5cyclictest1773283-21kworker/u8:3+flush-8:021:17:471
3081250994538,5cyclictest3395971-21latency_hist00:25:010
3081257994423,18cyclictest3197624-21cat21:05:002
3081254994440,3cyclictest3187729-21kworker/u8:2+events_unbound22:42:451
3081254994439,4cyclictest1773283-21kworker/u8:3+events_unbound19:27:521
3081254994436,6cyclictest3405316-21ntpq00:32:491
3081254994436,6cyclictest2954217-21kworker/u8:0+flush-8:020:22:501
3081254994431,9cyclictest3167447-21latency_hist20:35:011
3081254994338,4cyclictest3366097-21kworker/u8:1+events_unbound00:17:371
3081254994337,4cyclictest1773283-21kworker/u8:3+flush-8:021:22:541
3081250994335,6cyclictest275-21systemd-journal21:37:370
3081250994335,5cyclictest3147635-21latency_hist20:15:010
3081254994238,3cyclictest1773283-21kworker/u8:3+events_unbound20:27:381
3081250994234,6cyclictest2954217-21kworker/u8:0+flush-8:020:22:400
3081250994232,8cyclictest3167443-21apt-get20:35:000
3081254994136,4cyclictest3034507-21kworker/u8:2+flush-8:019:17:371
3081254994135,4cyclictest2954217-21kworker/u8:0+events_unbound20:57:391
3081254994036,3cyclictest2954217-21kworker/u8:0+events_unbound19:47:501
3081254994035,3cyclictest3371220-21kworker/u8:2+events_unbound00:27:471
3081254994035,3cyclictest3221716-21kworker/u8:0+flush-8:023:07:391
3081254994033,5cyclictest3093289-21kworker/u8:1+flush-8:019:52:471
3081257993924,13cyclictest3158233-21fschecks_count20:27:382
3081254993934,4cyclictest3221716-21kworker/u8:0+flush-8:022:12:501
3081254993934,3cyclictest3221716-21kworker/u8:0+flush-8:021:52:411
3081254993933,5cyclictest3093289-21kworker/u8:1+flush-8:020:02:541
3081254993933,4cyclictest3128108-21apt-get19:57:331
3081250993931,6cyclictest3202605-21latency_hist21:10:010
3081257993829,7cyclictest275-21systemd-journal00:31:042
3081254993833,4cyclictest3335877-21kworker/u8:0+flush-8:023:57:421
3081254993833,4cyclictest3335877-21kworker/u8:0+flush-8:023:57:421
3081254993833,4cyclictest3221716-21kworker/u8:0+flush-8:022:32:371
3081254993833,3cyclictest2954217-21kworker/u8:0+events_unbound19:32:471
3081254993828,7cyclictest275-21systemd-journal21:27:371
3081250993832,4cyclictest3221716-21kworker/u8:0+flush-8:022:17:460
3081250993830,6cyclictest3365511-21ntpq23:52:470
3081250993828,8cyclictest3102979-21latency_hist19:30:000
3081257993723,12cyclictest3107650-21unixbench_multi19:32:552
3081254993732,3cyclictest3258135-21irqrtprio22:07:411
3081254993731,5cyclictest3371220-21kworker/u8:2+flush-8:000:02:501
3081254993731,5cyclictest3093289-21kworker/u8:1+flush-8:019:42:381
3081254993731,4cyclictest3316797-21kworker/u8:2+flush-8:023:37:491
31251172360,6sleep3451irq_work/319:52:443
3081254993630,5cyclictest3316797-21kworker/u8:2+flush-8:023:42:531
3081254993630,5cyclictest1773283-21kworker/u8:3+flush-8:021:42:491
3081254993630,5cyclictest1773283-21kworker/u8:3+flush-8:021:42:491
3081254993630,4cyclictest2954217-21kworker/u8:0+flush-8:020:12:541
3081250993631,3cyclictest3221716-21kworker/u8:0+events_unbound22:27:420
3081250993631,3cyclictest1773283-21kworker/u8:3+events_unbound21:42:540
3081250993631,3cyclictest1773283-21kworker/u8:3+events_unbound21:42:540
33708622350,3sleep2371rcuc/223:57:542
33708622350,3sleep2371rcuc/223:57:542
3081257993519,14cyclictest3326378-21unixbench_multi23:12:542
3081254993529,5cyclictest3296863-21kworker/u8:1+flush-8:022:52:481
3081254993529,4cyclictest2954217-21kworker/u8:0+flush-8:020:47:501
3081254993519,12cyclictest275-21systemd-journal20:45:001
3081250993527,6cyclictest275-21systemd-journal22:25:000
3081257993422,11cyclictest3218204-21fschecks_time21:27:392
3081257993420,12cyclictest3287325-21latency_hist22:35:012
3081254993427,4cyclictest3237211-21kworker/u8:1+flush-8:021:57:381
3081250993426,6cyclictest3138456-21diskstats20:07:370
3081250993423,9cyclictest3132758-21runrttasks20:00:010
3081257993317,14cyclictest3396007-21apt-config00:25:012
3081257993310,21cyclictest3201935-21ntpq21:07:482
3081254993328,3cyclictest3324699-21kworker/u8:3+events_unbound23:32:391
3081254993328,3cyclictest3187729-21kworker/u8:2+flush-8:022:37:411
3081254993327,4cyclictest3296863-21kworker/u8:1+events_unbound23:20:001
3081254993327,4cyclictest3296863-21kworker/u8:1+events_unbound23:02:541
3081250993328,4cyclictest3316797-21kworker/u8:2+flush-8:023:38:080
3081250993328,4cyclictest3221716-21kworker/u8:0+flush-8:022:02:070
3081250993328,3cyclictest1773283-21kworker/u8:3+flush-8:021:17:370
3081250993327,4cyclictest3281449-21kworker/u8:3+flush-8:022:41:080
3081257993228,3cyclictest3337036-21aten2_r5power_c23:27:362
3081257993216,14cyclictest3362092-21fschecks_time23:52:382
3081254993227,3cyclictest3083769-21kthreadcore19:12:441
3081254993224,6cyclictest3324699-21kworker/u8:3+flush-8:023:27:491
3081254993222,8cyclictest3366097-21kworker/u8:1+flush-8:000:07:531
3081250993224,6cyclictest275-21systemd-journal20:02:370
3081250993223,7cyclictest3292071-21latency_hist22:40:010
3081250993223,7cyclictest3182664-21cp20:50:000
3081250993222,7cyclictest3371190-21latency_hist00:00:000
3081250993222,7cyclictest3371190-21latency_hist00:00:000
3081250993221,8cyclictest3187716-21latency_hist20:55:010
308125799312,23cyclictest3276598-21ntpq22:22:502
3081257993118,11cyclictest0-21swapper/219:50:002
3081257993117,12cyclictest3347274-21fschecks_time23:37:412
3081257993117,12cyclictest3293250-21cut22:42:412
3081257993116,13cyclictest0-21swapper/222:15:012
3081254993125,4cyclictest3187729-21kworker/u8:2+flush-8:021:37:481
3081254993124,5cyclictest2954217-21kworker/u8:0+events_unbound19:22:471
3081254993124,5cyclictest2954217-21kworker/u8:0+events_unbound19:22:471
3081250993126,4cyclictest2954217-21kworker/u8:0+flush-8:019:48:090
3081250993125,4cyclictest3366097-21kworker/u8:1+flush-8:000:32:590
3081250993123,6cyclictest3207520-21latency_hist21:15:010
307663623124,4sleep23071127-21kworker/2:2+events19:07:332
3081257993020,8cyclictest3355730-21ntpq23:42:502
3081257993017,11cyclictest3156715-21grep20:22:472
3081257993013,15cyclictest0-21swapper/200:27:472
308125799301,11cyclictest3332088-21apt-get23:22:342
3081254993024,4cyclictest0-21swapper/123:25:001
3081254993023,5cyclictest3296863-21kworker/u8:1+flush-8:022:47:471
3081254993022,5cyclictest275-21systemd-journal00:15:001
3081250993026,3cyclictest2954217-21kworker/u8:0+flush-8:019:12:080
3081250993025,3cyclictest3296863-21kworker/u8:1+events_unbound23:02:460
3081250993024,5cyclictest3034507-21kworker/u8:2+flush-8:019:19:270
3081250993024,4cyclictest3321445-21unixbench_multi23:07:540
3081257992924,3cyclictest3207242-21unixbench_multi21:12:532
308125799292,22cyclictest3227769-21apt-get21:37:362
308125799292,10cyclictest3277683-21apt-get22:27:362
3081257992917,11cyclictest3306696-21unixbench_singl22:52:552
3081257992917,11cyclictest3251727-21unixbench_multi21:57:562
3081257992916,11cyclictest0-21swapper/219:17:472
3081257992915,12cyclictest0-21swapper/200:20:012
3081257992910,17cyclictest3311846-21sendmail23:00:002
3081254992923,5cyclictest3187729-21kworker/u8:2+events_unbound21:12:491
3081250992924,3cyclictest3391087-21systemd-run00:20:010
3081250992924,3cyclictest3246820-21unixbench_multi21:52:540
3081250992922,5cyclictest3296863-21kworker/u8:1+flush-8:023:17:380
3081250992922,5cyclictest275-21systemd-journal19:32:370
3081250992920,6cyclictest3242149-21latency_hist21:50:000
3081250992919,8cyclictest3342230-21diskstats23:32:350
3081250992917,10cyclictest3381848-21diskstats00:12:390
308125799284,21cyclictest0-21swapper/220:35:032
308125799282,6cyclictest3297466-21apt-get22:47:342
308125799282,5cyclictest3320966-21ntpq23:07:482
308125799282,5cyclictest3241377-21ntpq21:47:452
3081257992824,2cyclictest3133102-21aten2_r5power_e20:02:372
3081257992813,13cyclictest0-21swapper/220:47:482
3081257992812,14cyclictest3287633-21aten2_r5power_e22:37:352
308125799281,12cyclictest3375508-21ntpq00:02:492
3081257992810,16cyclictest3377608-21tr00:07:452
3081254992822,3cyclictest0-21swapper/119:10:001
3081254992817,9cyclictest3241392-21ntpq21:47:461
3081250992823,4cyclictest3237211-21kworker/u8:1+flush-8:021:56:080
3081250992822,4cyclictest3324699-21kworker/u8:3+flush-8:023:27:540
308125799277,7cyclictest3095730-21tr19:22:432
308125799277,7cyclictest3095730-21tr19:22:432
308125799276,6cyclictest3407683-21tr00:37:432
308125799273,18cyclictest3192009-21ntpq20:57:512
308125799271,9cyclictest3382495-21/usr/sbin/munin00:12:432
308125799271,8cyclictest3312663-21/usr/sbin/munin23:02:402
308125799271,8cyclictest3127474-21/usr/sbin/munin19:52:552
3081257992718,7cyclictest3167443-21apt-get20:35:002
308125799271,5cyclictest3207964-21apt-get21:17:342
3081257992711,14cyclictest0-21swapper/219:37:352
3081257992710,15cyclictest3260368-21tr22:07:432
308125799270,9cyclictest3176950-21/usr/sbin/munin20:42:502
308125799270,7cyclictest3143116-21munin-node20:12:332
3081254992721,4cyclictest3316797-21kworker/u8:2+flush-8:023:47:481
3081254992721,4cyclictest3183353-21grep20:52:371
3081250992723,2cyclictest0-21swapper/000:40:000
3081250992722,4cyclictest3366097-21kworker/u8:1+flush-8:000:08:050
3081250992722,4cyclictest20823-21cron23:25:000
3081250992722,3cyclictest3316797-21kworker/u8:2+flush-8:023:43:050
3081250992722,3cyclictest3286859-21unixbench_multi22:32:520
3081250992721,5cyclictest3221716-21kworker/u8:0+flush-8:022:12:400
3081250992716,9cyclictest275-21systemd-journal20:40:000
308125799265,7cyclictest3344736-21tr23:32:412
308125799261,9cyclictest3186998-21ntpq20:52:482
308125799261,8cyclictest3222977-21/usr/sbin/munin21:32:372
308125799261,7cyclictest3243104-21/usr/sbin/munin21:52:402
308125799261,6cyclictest3271324-21ntpq22:17:472
3081257992613,11cyclictest0-21swapper/223:17:352
3081257992612,12cyclictest0-21swapper/220:17:402
308125799260,8cyclictest3217217-21/usr/sbin/munin21:22:552
308125799260,7cyclictest3085845-21/usr/sbin/munin19:12:542
3081254992622,3cyclictest3324699-21kworker/u8:3+flush-8:023:52:501
3081250992620,5cyclictest2954217-21kworker/u8:0+flush-8:019:37:520
33021132250,6chrt451irq_work/322:50:013
308125799257,3cyclictest3102708-21grep19:27:542
308125799257,15cyclictest3356883-21aten2_r5power_e23:47:332
308125799253,7cyclictest3130884-21tr19:57:442
3081257992522,2cyclictest3138238-21aten2_r5power_c20:07:352
308125799250,8cyclictest3114453-21tr19:42:432
3081254992518,5cyclictest275-21systemd-journal20:15:371
3081250992519,4cyclictest3324699-21kworker/u8:3+events_unbound23:12:480
3081250992516,7cyclictest3212917-21/usr/sbin/munin21:22:460
308125799241,8cyclictest3252767-21/usr/sbin/munin22:02:382
3081250992422,1cyclictest0-21swapper/021:02:530
3081250992420,3cyclictest3281449-21kworker/u8:3+flush-8:022:47:520
3081250992418,5cyclictest2954217-21kworker/u8:0+events_unbound19:55:010
3081250992417,5cyclictest2954217-21kworker/u8:0+flush-8:020:15:360
3081250992416,6cyclictest3187729-21kworker/u8:2+flush-8:022:07:470
3081257992320,2cyclictest3232921-21cpuspeed_turbos21:42:372
3081257992320,2cyclictest3232921-21cpuspeed_turbos21:42:372
3081250992318,4cyclictest3324699-21kworker/u8:3+flush-8:000:01:180
3081250992318,4cyclictest3316797-21kworker/u8:2+events_unbound23:47:440
3081250992318,3cyclictest3296863-21kworker/u8:1+flush-8:022:57:400
3081250992316,5cyclictest3227440-21ls21:35:000
308125099228,5cyclictest0-21swapper/020:55:480
3081250992218,2cyclictest0-21swapper/021:30:000
3081250992114,5cyclictest3157966-21apt-get20:27:340
30804412193,9sleep30-21swapper/319:07:503
34053662134,5sleep30-21swapper/300:32:513
33654532132,7sleep30-21swapper/323:52:463
33426942132,7sleep30-21swapper/323:32:403
32871212132,6sleep30-21swapper/322:35:013
32767602132,7sleep30-21swapper/322:22:533
31869662132,7sleep30-21swapper/320:52:483
31525912132,6sleep30-21swapper/320:20:023
31418962132,7sleep30-21swapper/320:07:463
31276822132,6sleep30-21swapper/319:55:013
31179912132,6sleep30-21swapper/319:45:023
34087382122,6sleep30-21swapper/300:37:453
33959582121,7sleep30-21swapper/300:25:013
33819692122,6sleep30-21swapper/300:12:413
33478202122,6sleep30-21swapper/323:37:453
33365852122,5sleep30-21swapper/323:25:013
33356592122,6sleep30-21swapper/323:22:443
33315972122,6sleep30-21swapper/323:19:483
33210622122,6sleep30-21swapper/323:07:523
33118572122,5sleep30-21swapper/323:00:013
33082792122,6sleep30-21swapper/322:57:403
33021132122,6sleep30-21swapper/322:50:023
32875782122,6sleep30-21swapper/322:37:343
32772662122,5sleep30-21swapper/322:25:023
32679372122,6sleep30-21swapper/322:17:393
32520382122,6sleep30-21swapper/322:00:023
32474962122,6sleep30-21swapper/321:57:333
32469022122,6sleep30-21swapper/321:52:553
32330742122,6sleep30-21swapper/321:42:403
32330742122,6sleep30-21swapper/321:42:403
32282962122,6sleep30-21swapper/321:37:433
32272822122,6sleep30-21swapper/321:34:383
32181092122,6sleep30-21swapper/321:27:393
32125132122,5sleep30-21swapper/321:20:013
32078932122,6sleep30-21swapper/321:17:003
32027492122,5sleep30-21swapper/321:10:023
31929112122,5sleep30-21swapper/321:00:023
31776122122,6sleep30-21swapper/320:45:023
31735472122,6sleep30-21swapper/320:42:413
31634522122,6sleep30-21swapper/320:32:423
31599342122,6sleep30-21swapper/320:27:443
31520992122,6sleep30-21swapper/320:17:533
31440612123,5sleep30-21swapper/320:12:433
31332902122,6sleep30-21swapper/320:02:403
31069482122,6sleep30-21swapper/319:32:463
30829142122,7sleep30-21swapper/319:12:443
34006472112,5sleep30-21swapper/300:27:533
33863902112,6sleep30-21swapper/300:16:063
33764922112,6sleep30-21swapper/300:05:543
33761142112,5sleep30-21swapper/300:04:063
33710572112,6sleep30-21swapper/323:58:323
33710572112,6sleep30-21swapper/323:58:323
33577472112,5sleep30-21swapper/323:47:413
33516552112,5sleep30-21swapper/323:40:023
33221252112,5sleep30-21swapper/323:12:333
32925342112,6sleep30-21swapper/322:42:363
32622172112,5sleep30-21swapper/322:10:373
32573852112,6sleep30-21swapper/322:07:323
32372162112,5sleep30-21swapper/321:45:013
31978892112,5sleep30-21swapper/321:05:363
31881062113,5sleep30-21swapper/320:57:183
31157422112,6sleep30-21swapper/319:42:463
31086172112,5sleep30-21swapper/319:37:403
30983062112,6sleep30-21swapper/319:25:523
30932952112,5sleep30-21swapper/319:20:103
30932952112,5sleep30-21swapper/319:20:103
30893862112,6sleep30-21swapper/319:17:413
31679082102,5sleep30-21swapper/320:35:103
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional