You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-23 - 11:13

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot4s.osadl.org (updated Sat May 23, 2026 00:44:13)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
28590662950,4sleep00-21swapper/020:32:300
2772090996050,7cyclictest1704890-21systemd-journal20:31:582
2772086995951,6cyclictest2969327-21apt-key22:25:011
27721552560,3sleep1281rcuc/119:10:011
2772084995644,9cyclictest2953551-21ntpq22:05:570
2772084995549,4cyclictest2762099-21kworker/u8:4+flush-8:020:25:580
2772086995447,5cyclictest2807906-21kworker/u8:2+events_unbound22:10:581
2772086995443,8cyclictest2938778-21ntpq21:51:001
29467922530,2sleep375-21kworker/3:1+events22:00:533
2772086995348,3cyclictest2969245-21kworker/u8:0+events_unbound23:00:471
2772084995346,5cyclictest2798462-21kworker/u8:1+flush-8:019:35:480
2772084995342,8cyclictest2776081-21ntpq19:10:570
2772086995248,3cyclictest3029893-21kworker/u8:0+flush-8:000:06:011
2772086995246,4cyclictest2807906-21kworker/u8:2+events_unbound22:15:581
2772086995245,4cyclictest3064510-21kworker/u8:4+flush-8:000:00:441
2772086995145,5cyclictest3029893-21kworker/u8:0+events_unbound00:15:011
2772086995143,5cyclictest1704890-21systemd-journal20:32:231
2772086995141,8cyclictest2847252-21ntpq20:20:571
2772084995143,6cyclictest2807692-21ntpq19:40:550
2772086995045,4cyclictest2771923-21kworker/u8:0+flush-8:019:21:041
2772086995043,6cyclictest2899006-21kworker/u8:3+flush-8:021:35:531
2772086995040,7cyclictest2838520-21/usr/sbin/munin20:15:561
2772084995044,4cyclictest2828369-21kworker/u8:0+flush-8:020:46:010
2772090994842,4cyclictest2807906-21kworker/u8:2+flush-8:020:50:482
2772086994843,4cyclictest2899006-21kworker/u8:3+flush-8:021:15:481
2772084994840,7cyclictest3064510-21kworker/u8:4+flush-8:000:00:500
2772086994742,3cyclictest2807906-21kworker/u8:2+events_unbound22:50:581
2772086994741,4cyclictest2798462-21kworker/u8:1+flush-8:019:40:541
2772084994743,3cyclictest2807906-21kworker/u8:2+events_unbound22:33:580
29965892460,5chrt0-21swapper/222:50:512
2772086994636,8cyclictest2973573-21ntpq22:25:581
2772086994636,8cyclictest2908614-21ntpq21:20:581
2772086994636,8cyclictest2868526-21ntpq20:40:531
2772086994541,3cyclictest2969245-21kworker/u8:0+flush-8:022:45:471
2772086994539,4cyclictest2667740-21kworker/u8:2+flush-8:019:10:491
2772086994538,5cyclictest3015142-21apt-get23:10:441
2772086994535,8cyclictest2948694-21ntpq22:00:571
2772086994440,3cyclictest3029893-21kworker/u8:0+flush-8:023:25:491
2772086994439,4cyclictest3029893-21kworker/u8:0+flush-8:023:40:461
2772086994438,4cyclictest2667740-21kworker/u8:2+flush-8:019:15:491
2772086994436,6cyclictest3084507-21apt-get00:20:431
2772086994435,7cyclictest2984927-21/usr/sbin/munin22:40:561
2772086994435,7cyclictest2802773-21ntpq19:35:581
2772084994435,7cyclictest1704890-21systemd-journal20:05:000
2772084994435,7cyclictest1704890-21systemd-journal20:05:000
2772086994338,4cyclictest2828369-21kworker/u8:0+flush-8:021:11:011
2772086994338,4cyclictest2762099-21kworker/u8:4+flush-8:020:05:481
2772086994338,3cyclictest3069521-21kworker/u8:1+events_unbound00:35:001
2772086994335,6cyclictest2762099-21kworker/u8:4+flush-8:020:00:571
2772086994335,6cyclictest2762099-21kworker/u8:4+flush-8:020:00:571
2772086994333,8cyclictest2918695-21ntpq21:30:561
2772084994339,3cyclictest2807906-21kworker/u8:2+flush-8:020:40:590
2772084994338,4cyclictest2828369-21kworker/u8:0+events_unbound20:56:170
2772084994333,7cyclictest1704890-21systemd-journal21:05:000
2772084994332,8cyclictest1704890-21systemd-journal19:20:010
2772086994236,4cyclictest2969245-21kworker/u8:0+flush-8:022:30:591
2772086994235,5cyclictest1704890-21systemd-journal22:06:581
2772084994237,4cyclictest2807906-21kworker/u8:2+flush-8:019:55:470
2772084994237,4cyclictest2762099-21kworker/u8:4+flush-8:019:31:030
2772084994237,3cyclictest2979362-21kworker/u8:1+flush-8:022:35:530
2772084994237,3cyclictest2807906-21kworker/u8:2+events_unbound20:50:450
2772084994233,6cyclictest2793724-21latency_hist19:30:010
2772086994136,4cyclictest2807906-21kworker/u8:2+flush-8:023:55:501
2772086994135,4cyclictest3029893-21kworker/u8:0+flush-8:000:35:541
2772086994135,4cyclictest2807906-21kworker/u8:2+flush-8:021:00:561
2772084994135,5cyclictest2807906-21kworker/u8:2+flush-8:020:10:580
2772084994134,5cyclictest2807906-21kworker/u8:2+flush-8:023:51:000
2772090994030,7cyclictest2909888-21cat21:25:012
2772086994032,6cyclictest1704890-21systemd-journal20:59:591
2772084994036,3cyclictest2969245-21kworker/u8:0+events_unbound22:40:060
2772086993933,4cyclictest3029893-21kworker/u8:0+flush-8:023:30:471
2772086993933,3cyclictest2807906-21kworker/u8:2+flush-8:019:45:531
2772086993932,6cyclictest2807906-21kworker/u8:2+flush-8:023:21:011
2772086993932,5cyclictest2869710-21/usr/sbin/munin20:45:551
2772086993931,6cyclictest3029893-21kworker/u8:0+flush-8:023:46:021
2772084993934,4cyclictest2899006-21kworker/u8:3+flush-8:022:26:020
2772084993933,4cyclictest2807906-21kworker/u8:2+events_unbound21:26:020
2772084993834,3cyclictest2807906-21kworker/u8:2+flush-8:021:45:580
2772084993833,4cyclictest2807906-21kworker/u8:2+flush-8:021:20:490
2772084993829,6cyclictest2848155-21latency_hist20:25:000
2772084993828,8cyclictest2963445-21ntpq22:15:570
2772086993733,3cyclictest2762099-21kworker/u8:4+flush-8:020:27:091
2772086993730,5cyclictest2762099-21kworker/u8:4+events_unbound19:55:561
2772086993728,7cyclictest2979749-21apt-get22:35:451
2772084993731,4cyclictest2828369-21kworker/u8:0+flush-8:020:16:000
2772084993731,4cyclictest2807906-21kworker/u8:2+flush-8:021:55:480
2772084993730,5cyclictest2894936-21diskstats21:10:470
2772084993730,5cyclictest2828369-21kworker/u8:0+events_unbound20:35:570
2772084993630,5cyclictest2899006-21kworker/u8:3+flush-8:021:41:090
2772084993628,6cyclictest3044786-21/usr/sbin/munin23:40:530
2772084993628,6cyclictest3029893-21kworker/u8:0+flush-8:000:11:000
2772084993627,7cyclictest1704890-21systemd-journal21:54:370
2772086993530,3cyclictest3024707-21idleruntime-cro23:19:591
2772086993530,3cyclictest2807906-21kworker/u8:2+flush-8:023:50:561
2772086993529,4cyclictest3069521-21kworker/u8:1+flush-8:000:25:521
2772084993531,3cyclictest2813552-21cat19:50:000
2772084993530,4cyclictest3029893-21kworker/u8:0+flush-8:000:05:490
2772086993429,4cyclictest2762099-21kworker/u8:4+flush-8:019:51:111
2772086993429,3cyclictest2899006-21kworker/u8:3+events_unbound21:50:001
2772084993429,4cyclictest3029893-21kworker/u8:0+flush-8:023:49:510
2772090993328,4cyclictest2807906-21kworker/u8:2+flush-8:023:06:012
2772086993327,4cyclictest2771923-21kworker/u8:0+flush-8:019:25:541
2772086993325,6cyclictest2910002-21apt-get21:25:441
2772084993328,3cyclictest201rcuc/022:50:000
2772084993325,6cyclictest3090362-21ps00:25:520
2772084993324,7cyclictest3020051-21/usr/sbin/munin23:15:540
2772086993226,4cyclictest3069521-21kworker/u8:1+events_unbound00:15:581
2772084993228,3cyclictest2807906-21kworker/u8:2+flush-8:022:51:010
2772084993227,4cyclictest3029893-21kworker/u8:0+events_unbound23:26:020
2772090993126,3cyclictest2899006-21kworker/u8:3+flush-8:022:20:542
2772086993126,4cyclictest2828369-21kworker/u8:0+events_unbound20:36:051
2772086993126,4cyclictest2807906-21kworker/u8:2+events_unbound21:55:571
2772086993125,5cyclictest2807906-21kworker/u8:2+flush-8:022:55:561
2772086993125,4cyclictest2807906-21kworker/u8:2+events_unbound21:40:481
2772086993120,9cyclictest2894148-21apt-get21:10:011
2772084993126,4cyclictest3069521-21kworker/u8:1+flush-8:000:16:000
2772084993126,3cyclictest2894314-21latency_hist21:10:010
2772084993126,3cyclictest2807906-21kworker/u8:2+flush-8:021:31:000
2772086993027,1cyclictest3029893-21kworker/u8:0+events_unbound23:36:031
2772086993025,4cyclictest2807906-21kworker/u8:2+flush-8:020:11:081
2772084993026,3cyclictest2807906-21kworker/u8:2+flush-8:023:21:030
2772084993024,4cyclictest2969245-21kworker/u8:0+events_unbound23:00:510
2772090992925,2cyclictest2804471-21irqcore19:40:492
2772090992924,4cyclictest2858279-21kworker/u8:1+flush-8:021:06:022
2772084992923,4cyclictest3069521-21kworker/u8:1+events_unbound00:35:560
2772084992923,4cyclictest2807906-21kworker/u8:2+events_unbound19:50:580
2772084992920,7cyclictest3039813-21/usr/sbin/munin23:35:550
277126922916,7sleep30-21swapper/319:06:033
2772090992821,5cyclictest1704890-21systemd-journal23:40:002
2772090992818,7cyclictest2990037-21cat22:45:012
2772084992823,3cyclictest2899006-21kworker/u8:3+flush-8:021:20:000
2772090992724,2cyclictest2915025-21aten2_r5power_v21:30:442
2772086992723,3cyclictest3014449-21unixbench_multi23:06:031
2772084992723,3cyclictest2899006-21kworker/u8:3+events_unbound22:24:190
2772084992721,4cyclictest3029893-21kworker/u8:0+flush-8:023:30:430
277209099267,3cyclictest27738022tr19:10:522
2772090992623,2cyclictest3019374-21unixbench_multi23:11:032
2772090992622,2cyclictest2879717-21aten2_r5power_v20:55:472
2772090992621,3cyclictest3098618-21unixbench_singl00:31:032
2772090992621,3cyclictest2920091-21cpuspeed_turbos21:35:482
2772090992617,7cyclictest3005027-21cat23:00:012
2772086992622,3cyclictest2771923-21kworker/u8:0+events_unbound19:34:401
2772086992620,4cyclictest2828369-21kworker/u8:0+flush-8:020:50:571
2772084992621,4cyclictest2969245-21kworker/u8:0+events_unbound22:56:410
2772084992620,4cyclictest2807906-21kworker/u8:2+flush-8:022:11:050
2772084992619,5cyclictest3069521-21kworker/u8:1+flush-8:000:20:560
2772090992522,2cyclictest3103571-21unixbench_singl00:36:042
2772090992521,2cyclictest2874003-21unixbench_singl20:46:032
2772090992521,2cyclictest2799253-21fschecks_count19:35:482
2772090992521,2cyclictest0-21swapper/221:19:592
2772084992521,2cyclictest0-21swapper/020:06:020
2772084992517,5cyclictest2788788-21latency_hist19:25:000
2772090992416,6cyclictest1704890-21systemd-journal20:30:002
2772090992320,2cyclictest2929889-21aten2_r5power_p21:45:432
2772090992320,2cyclictest2910087-21aten2_r5power_p21:25:462
2772084992319,3cyclictest2899006-21kworker/u8:3+flush-8:021:36:050
2772084992313,8cyclictest3010435-21diskstats23:05:470
2772090992219,2cyclictest2814083-21cpuspeed_turbos19:50:462
2772090992218,2cyclictest2789317-21cpuspeed_turbos19:25:472
2772090992217,3cyclictest3074860-21fschecks_count00:10:482
2772090992217,3cyclictest2945220-21fschecks_count22:00:482
2772090992217,3cyclictest2818984-21cpuspeed_turbos19:55:462
2772090992213,7cyclictest2994813-21cron22:50:002
2772084992218,2cyclictest882-21mta-sts-daemon23:58:350
2772084992217,3cyclictest3029893-21kworker/u8:0+events_unbound00:32:140
277209099214,8cyclictest161rcu_preempt23:20:492
2772090992116,4cyclictest3088780-21unixbench_singl00:21:042
2772090992115,4cyclictest2807906-21kworker/u8:2+events_unbound23:55:482
2772090992111,2cyclictest0-21swapper/220:20:582
277209099210,8cyclictest27721582latency_hist19:10:012
2772084992116,3cyclictest2969245-21kworker/u8:0+events_unbound23:10:470
2772090992017,2cyclictest2940035-21aten2_r5power_e21:55:462
2772090992016,2cyclictest0-21swapper/222:10:562
2772090992015,4cyclictest2984386-21latency_hist22:40:012
2772090992015,3cyclictest2959740-21aten2_r5power_e22:15:452
2772090992015,3cyclictest0-21swapper/220:15:002
2772090992013,5cyclictest3048499-21needreboot23:40:532
2772090992012,6cyclictest2864419-21cron20:40:002
2772090992012,6cyclictest2809168-21fschecks_count19:45:492
2772090992011,7cyclictest3030282-21fschecks_time23:25:482
2772090992011,7cyclictest2939277-21unixbench_singl21:51:062
2772090992011,7cyclictest2798240-21unixbench_multi19:31:032
2772090992010,8cyclictest3038281-21tr23:30:532
27716792202,12sleep0876-21mta-sts-daemon19:07:190
2772090991915,2cyclictest0-21swapper/222:26:022
2772090991915,2cyclictest0-21swapper/219:20:482
2772090991914,3cyclictest2869022-21unixbench_multi20:41:022
2772090991912,5cyclictest1704890-21systemd-journal23:49:392
2772084991914,4cyclictest2945128-21cpuspeed_turbos22:00:470
2772090991814,2cyclictest0-21swapper/223:00:582
2772090991814,2cyclictest0-21swapper/200:05:012
2772090991812,4cyclictest3054996-21fschecks_time23:50:472
2772090991812,4cyclictest3020426-21fschecks_time23:15:482
277209099179,6cyclictest2953175-21tr22:05:542
277209099179,6cyclictest2887596-21tr21:00:522
277209099179,6cyclictest2828919-21fschecks_count20:05:482
277209099178,7cyclictest2777275-21sed19:15:022
2772090991714,2cyclictest2894798-21aten2_r5power_v21:10:462
2772090991712,4cyclictest3069521-21kworker/u8:1+flush-8:000:06:032
2772090991712,3cyclictest0-21swapper/200:25:522
2772090991711,4cyclictest2842805-21unixbench_singl20:16:042
2772090991710,6cyclictest2824007-21fschecks_count20:00:482
2772090991710,6cyclictest2824007-21fschecks_count20:00:472
2772090991710,5cyclictest2928287-21tr21:40:542
2772090991610,4cyclictest2975037-21fschecks_count22:30:472
277209099156,7cyclictest3079863-21fschecks_time00:15:482
29887522142,8sleep30-21swapper/322:40:593
28129232143,7sleep30-21swapper/319:46:033
30537572132,7sleep30-21swapper/323:46:033
30292652133,6sleep30-21swapper/323:21:033
30046782132,7sleep30-21swapper/323:00:013
29636682132,7sleep30-21swapper/322:16:033
28931062132,7sleep30-21swapper/321:05:573
28583522134,5sleep30-21swapper/320:31:413
28375472132,7sleep30-21swapper/320:11:033
28036862132,6sleep30-21swapper/319:40:023
27769772132,6sleep30-21swapper/319:15:023
30842472122,6sleep30-21swapper/300:20:023
30698402122,6sleep30-21swapper/300:05:493
30645512122,6sleep30-21swapper/300:00:403
30593392122,6sleep30-21swapper/323:55:013
30556002122,6sleep30-21swapper/323:50:533
30488892122,6sleep30-21swapper/323:41:013
30440322122,6sleep30-21swapper/323:36:043
30204832122,6sleep30-21swapper/323:15:503
30195312122,6sleep30-21swapper/323:12:043
30142152122,6sleep30-21swapper/323:06:023
29951322122,6sleep30-21swapper/322:50:453
29799912122,6sleep30-21swapper/322:35:483
29693572122,5sleep30-21swapper/322:25:023
29645922122,6sleep30-21swapper/322:20:433
29538332122,6sleep30-21swapper/322:06:033
29441702122,6sleep30-21swapper/321:56:053
29287032122,6sleep30-21swapper/321:41:003
29201822122,6sleep30-21swapper/321:35:493
29152832122,6sleep30-21swapper/321:30:493
29100062122,6sleep30-21swapper/321:25:473
29092672122,6sleep30-21swapper/321:22:383
29039762122,6sleep30-21swapper/321:16:403
28945682122,5sleep30-21swapper/321:10:023
28848242122,6sleep30-21swapper/321:00:473
28755512122,6sleep30-21swapper/320:50:523
28695692122,6sleep30-21swapper/320:45:013
28601752122,6sleep30-21swapper/320:35:513
28488512122,6sleep30-21swapper/320:25:483
28432612122,6sleep30-21swapper/320:20:023
28290452122,6sleep30-21swapper/320:05:503
28026962122,6sleep30-21swapper/319:35:563
27877442122,6sleep30-21swapper/319:20:563
27765302122,6sleep30-21swapper/319:11:053
31038382112,6sleep30-21swapper/300:39:043
30943762112,5sleep30-21swapper/300:30:453
30894142112,5sleep30-21swapper/300:25:443
30833732112,6sleep30-21swapper/300:16:013
30744892112,5sleep30-21swapper/300:10:023
30351162112,5sleep30-21swapper/323:30:463
30341952112,5sleep30-21swapper/323:26:053
30055962112,5sleep30-21swapper/323:00:493
29901192112,6sleep30-21swapper/322:45:243
29747462112,5sleep30-21swapper/322:30:433
29584572112,5sleep30-21swapper/322:10:573
29342392112,6sleep30-21swapper/321:47:383
28820722112,6sleep30-21swapper/320:56:103
28685792112,5sleep30-21swapper/320:40:573
28422522112,5sleep30-21swapper/320:15:573
28237092112,5sleep30-21swapper/320:00:403
28237092112,5sleep30-21swapper/320:00:393
28186872112,6sleep30-21swapper/319:55:023
28138582112,5sleep30-21swapper/319:50:453
27939472112,5sleep30-21swapper/319:30:023
27898532112,5sleep30-21swapper/319:25:533
29348972102,5sleep30-21swapper/321:50:073
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional