You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-16 - 21:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack2slot7s.osadl.org (updated Sat May 16, 2026 12:43:04)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2845134995446,5cyclictest2979858-21timerandwakeup11:11:031
2845134995446,5cyclictest2979858-21timerandwakeup11:11:031
2845135995347,3cyclictest0-21swapper/209:25:532
2845135995347,3cyclictest0-21swapper/209:25:532
2845135995243,6cyclictest2949816-21munin-plugin-st10:35:002
2845135995243,6cyclictest2848163-21kworker/u8:6-nfsiod08:19:242
284513499495,34cyclictest2898724-21ssh09:31:001
284513499495,34cyclictest2898724-21ssh09:31:001
2845134994837,8cyclictest2889244-21idleruntime-cro09:20:011
2845133994737,7cyclictest2894460-21needreboot09:25:500
2845133994737,7cyclictest2894460-21needreboot09:25:500
2845135994640,3cyclictest0-21swapper/209:40:002
2845134994639,3cyclictest0-21swapper/111:15:571
2845134994639,3cyclictest0-21swapper/111:15:571
284513399465,33cyclictest2947236-21ssh10:30:590
2845134994537,5cyclictest421-21dbus-daemon12:15:021
2845134994535,6cyclictest3018546-21latency_hist12:00:031
2845134994535,6cyclictest3018546-21latency_hist12:00:031
284513499451,33cyclictest2922822-21ssh10:00:541
284513399453,30cyclictest2902629-21ssh09:35:550
2845133994514,24cyclictest624-21lldpd08:45:530
2845135994436,5cyclictest2863186-21systemd-run08:10:012
2845134994438,3cyclictest0-21swapper/110:21:001
2845135994338,2cyclictest0-21swapper/211:20:012
2845135994338,2cyclictest0-21swapper/211:20:012
2845134994337,3cyclictest0-21swapper/108:00:011
2845134994336,3cyclictest0-21swapper/112:00:511
2845133994336,4cyclictest2980066-21unixbench_singl11:11:040
2845133994336,4cyclictest2980066-21unixbench_singl11:11:040
2845133994334,6cyclictest695-21cron10:55:010
284513399433,33cyclictest2955380-21ssh10:41:050
2845135994234,4cyclictest0-21swapper/210:25:562
2845135994233,6cyclictest2858666-21grep07:55:012
2845135994233,6cyclictest261-21systemd-journal12:20:022
2845135994231,8cyclictest3018402-21systemd-run12:00:012
2845135994231,8cyclictest3018402-21systemd-run12:00:012
2845134994234,5cyclictest2961950-21ssh10:50:011
2845134994234,5cyclictest2867709-21systemd-run08:25:011
2845134994234,5cyclictest2867709-21systemd-run08:25:011
284513399427,27cyclictest428-21mta-sts-daemon08:05:530
2845133994231,8cyclictest3050610-21ssh12:40:010
2845135994133,5cyclictest3039969-21ld12:25:592
2845135994133,4cyclictest0-21swapper/210:00:012
2845133994134,4cyclictest2974051-21sh11:05:010
2845133994132,6cyclictest420-21avahi-daemon11:51:560
2845135994030,8cyclictest3026455-21ssh12:10:022
2845135994030,8cyclictest3002346-21systemd-run11:40:012
2845134994034,3cyclictest0-21swapper/109:35:541
2845134994031,6cyclictest2854206-21systemd-run07:40:021
2845134994030,7cyclictest3036027-21ssh12:21:011
2845134994029,7cyclictest2001030-21cups-browsed11:29:101
2845133994031,5cyclictest151ktimers/008:20:520
2845133994031,5cyclictest151ktimers/008:20:520
2845133994030,8cyclictest3023638-21needreboot12:05:510
2845133994027,6cyclictest12350irq/32-eth011:15:580
2845133994027,6cyclictest12350irq/32-eth011:15:580
2845133994021,15cyclictest3031777-21ssh12:15:530
2845135993931,5cyclictest2901301-21systemd-run09:35:012
2845135993931,5cyclictest2901301-21systemd-run09:35:012
2845135993930,6cyclictest0-21swapper/210:40:012
2845135993927,9cyclictest547-21gmain10:18:532
2845135993927,9cyclictest547-21gmain10:18:532
284513599392,29cyclictest2967386-21ssh10:55:572
2845134993932,3cyclictest0-21swapper/112:35:011
2845134993929,6cyclictest2918575-21missed_timers09:55:481
2845133993932,5cyclictest1-21systemd09:45:010
2845133993928,4cyclictest0-21swapper/010:20:010
2845133993928,4cyclictest0-21swapper/010:20:010
2845133993924,12cyclictest2866172-21munin-plugin-st08:20:000
2845135993831,4cyclictest2872036-21unixbench_multi08:36:062
2845135993830,5cyclictest2860175-21sendmail08:00:012
2845134993830,5cyclictest3047431-21sh12:35:451
2845133993832,3cyclictest0-21swapper/008:05:010
2845133993829,6cyclictest2891131-21unixbench_multi09:21:050
2845133993829,6cyclictest2001030-21cups-browsed12:10:560
2845133993828,5cyclictest0-21swapper/011:40:010
2845135993730,4cyclictest3022403-21cron12:05:012
2845135993729,5cyclictest2867703-21systemd-run08:25:012
2845135993729,5cyclictest2867703-21systemd-run08:25:012
2845135993728,5cyclictest2978111-21idleruntime-cro11:10:012
2845134993731,3cyclictest0-21swapper/112:15:591
2845134993730,3cyclictest0-21swapper/111:50:551
2845134993729,4cyclictest0-21swapper/109:15:011
2845134993728,5cyclictest0-21swapper/111:46:101
284513399372,27cyclictest2975398-21ssh11:05:540
284513399372,26cyclictest2943044-21ssh10:25:520
284513399371,30cyclictest2987661-21rm11:21:010
2845135993630,3cyclictest0-21swapper/211:35:012
2845135993627,5cyclictest2889215-21ssh09:20:012
284513499363,29cyclictest2995514-21ssh11:30:521
2845134993629,4cyclictest0-21swapper/111:10:021
2845134993629,3cyclictest0-21swapper/111:01:011
2845134993628,5cyclictest2875109-21unixbench_singl08:46:041
2845134993628,3cyclictest0-21swapper/110:25:551
2845134993624,8cyclictest420-21avahi-daemon10:55:321
284513499362,26cyclictest2910684-21ssh09:45:541
2845134993622,10cyclictest2915158-21timerandwakeup09:51:051
2845134993618,3cyclictest171rcu_preempt10:05:551
2845133993630,4cyclictest0-21swapper/012:01:050
2845133993629,4cyclictest2918831-21ntp_offset09:55:540
2845133993627,6cyclictest2001030-21cups-browsed07:20:450
2845133993613,18cyclictest3039824-21ld12:25:530
2845135993528,3cyclictest0-21swapper/211:55:012
2845135993527,5cyclictest2991804-21mailstats11:26:022
2845135993527,4cyclictest0-21swapper/212:10:562
284513499358,5cyclictest0-21swapper/109:41:001
2845134993528,3cyclictest0-21swapper/111:20:541
2845134993527,5cyclictest2963567-21mailstats10:51:031
2845134993527,5cyclictest2881091-21unixbench_singl09:06:061
2845134993527,5cyclictest2869187-21idleruntime-cro08:30:001
2845134993527,4cyclictest0-21swapper/109:21:061
2845134993526,6cyclictest2001030-21cups-browsed11:44:571
284513499352,29cyclictest2894714-21rm09:25:591
284513499352,29cyclictest2894714-21rm09:25:591
284513399356,19cyclictest12350irq/32-eth009:31:000
284513399356,19cyclictest12350irq/32-eth009:31:000
284513399354,27cyclictest2886590-21ssh09:15:550
2845133993528,4cyclictest2998293-21ssh11:35:010
2845133993528,4cyclictest0-21swapper/009:00:020
2845133993527,5cyclictest1051-21mta-sts-daemon11:30:020
2845133993524,7cyclictest12350irq/32-eth011:55:510
2845133993524,7cyclictest12350irq/32-eth011:55:510
284513399352,26cyclictest2939145-21ssh10:20:580
2845133993519,12cyclictest3007278-21fschecks_count11:45:500
2845135993428,3cyclictest0-21swapper/209:46:062
2845135993428,3cyclictest0-21swapper/207:20:012
2845135993426,5cyclictest3044461-21tail12:31:062
2845135993426,5cyclictest2988105-21unixbench_multi11:21:072
2845135993425,6cyclictest2881180-21diskmemload10:01:032
2845135993423,8cyclictest2955905-21sh10:41:142
2845135993423,4cyclictest1055-21mta-sts-daemon07:40:422
2845135993422,9cyclictest2863978-21needreboot08:10:512
2845135993422,8cyclictest2926733-21missed_timers10:05:522
284513599342,22cyclictest3048028-21ssh12:35:592
284513499343,22cyclictest2999715-21ssh11:35:591
2845134993427,3cyclictest0-21swapper/110:15:561
2845134993427,3cyclictest0-21swapper/110:15:561
2845134993422,9cyclictest2951021-21missed_timers10:35:511
284513399344,25cyclictest2927041-21ssh10:06:010
284513399343,24cyclictest2882521-21ssh09:10:540
2845133993425,5cyclictest2881180-21diskmemload11:40:500
2845133993425,5cyclictest0-21swapper/009:50:010
2845133993424,7cyclictest2967877-21unixbench_multi10:56:070
2845133993424,5cyclictest0-21swapper/012:30:550
2845133993421,9cyclictest421-21dbus-daemon07:30:010
2845133993421,9cyclictest421-21dbus-daemon07:30:010
2845135993325,5cyclictest2882895-21switchtime09:11:042
2845135993324,6cyclictest421-21dbus-daemon09:25:012
2845135993322,8cyclictest2971175-21missed_timers11:00:502
2845134993325,4cyclictest0-21swapper/112:05:591
2845134993324,4cyclictest0-21swapper/110:41:011
2845134993323,7cyclictest2879104-21ntp_offset09:00:571
284513399334,10cyclictest14-21ksoftirqd/008:50:570
2845133993326,4cyclictest0-21swapper/010:35:570
2845133993321,8cyclictest2854160-21idleruntime-cro07:40:010
2845133993320,9cyclictest0-21swapper/008:36:060
2845133993315,13cyclictest2915345-21/usr/sbin/munin09:51:060
2845135993225,4cyclictest2870685-21systemd-run08:35:012
2845135993225,4cyclictest1054-21mta-sts-daemon09:40:542
2845135993225,3cyclictest0-21swapper/210:45:492
2845135993223,5cyclictest2879698-21kthreadd09:05:012
284513599322,23cyclictest2963342-21ssh10:50:572
2845135993218,10cyclictest0-21swapper/210:20:522
2845134993226,3cyclictest0-21swapper/110:15:011
2845134993224,5cyclictest439-21systemd-logind07:10:011
2845134993224,5cyclictest3042534-21grep12:30:001
2845134993222,8cyclictest2869985-21needreboot08:30:531
2845133993227,3cyclictest12450irq/33-eth010:10:540
2845133993215,13cyclictest421-21dbus-daemon08:00:010
2845135993121,8cyclictest2915047-21sh09:51:042
2845134993123,5cyclictest2873601-21unixbench_singl08:41:041
284513499312,23cyclictest2947256-21ssh10:31:001
2845134993119,9cyclictest548-21gdbus07:24:591
284513399317,4cyclictest191rcuc/007:15:560
2845133993123,5cyclictest2922859-21ntpq10:00:560
284513399312,15cyclictest2870820-21wc08:35:020
2845133993118,9cyclictest2959261-21ssh10:45:570
2845133993118,10cyclictest2854720-21fschecks_time07:40:450
2845133993115,14cyclictest151ktimers/007:50:550
2845135993021,6cyclictest3036221-21switchtime12:21:052
284513499304,4cyclictest1055-21mta-sts-daemon08:35:591
2845134993023,5cyclictest271ktimers/107:40:501
2845134993022,5cyclictest2876666-21grep08:55:011
284513499300,5cyclictest2845132-21cyclictest08:00:551
284513399305,5cyclictest151ktimers/008:10:590
2845133993023,4cyclictest2846632-21systemd-run07:15:010
2845133993015,13cyclictest0-21swapper/012:25:010
2845133993010,5cyclictest14-21ksoftirqd/009:05:530
2845135992922,4cyclictest2846632-21systemd-run07:15:012
284513599292,23cyclictest3007731-21ssh11:45:592
2845135992922,3cyclictest0-21swapper/210:10:582
2845135992922,3cyclictest0-21swapper/209:10:012
2845135992920,6cyclictest2852662-21cron07:35:002
2845135992822,2cyclictest0-21swapper/211:40:522
2845135992820,5cyclictest2870756-21kworker/u8:3+xprtiod09:00:012
2845135992818,7cyclictest2980068-21unixbench_singl11:11:042
2845135992818,7cyclictest2980068-21unixbench_singl11:11:042
2845135992810,10cyclictest2850489-21sendmail-mta07:25:522
2845135992810,10cyclictest2850489-21sendmail-mta07:25:522
2845134992821,4cyclictest2845866-21missed_timers07:10:491
2845134992820,5cyclictest0-21swapper/107:35:001
2845134992818,7cyclictest2878165-21sendmail09:00:011
2845133992818,7cyclictest2869200-21systemd-run08:30:010
2845135992719,5cyclictest1055-21mta-sts-daemon08:40:422
2845135992718,6cyclictest2876611-21unixbench_singl08:51:042
2845134992722,3cyclictest2848163-21kworker/u8:6-nfsiod08:19:221
2845134992721,4cyclictest1055-21mta-sts-daemon07:25:551
2845134992721,4cyclictest1055-21mta-sts-daemon07:25:551
284513399278,4cyclictest151ktimers/007:10:020
284513399276,5cyclictest14-21ksoftirqd/008:40:530
284513399273,3cyclictest191rcuc/007:30:550
2845133992718,4cyclictest0-21swapper/007:45:570
2845135992617,6cyclictest2853770-21mailstats07:36:022
2845134992619,5cyclictest2864664-21/usr/sbin/munin08:15:011
2845134992619,4cyclictest2857068-21unixbench_multi07:46:061
2845134992617,6cyclictest2847374-21missed_timers07:15:501
284513399265,3cyclictest171rcu_preempt09:00:590
284513599252,6cyclictest171rcu_preempt08:00:522
2845135992518,4cyclictest2875053-21unixbench_multi08:46:042
2845135992518,4cyclictest2867917-21acpi08:25:432
2845135992417,5cyclictest2848466-21basename07:20:432
2845134992416,4cyclictest0-21swapper/108:05:531
2845135992316,5cyclictest261-21systemd-journal07:50:012
2845134992315,5cyclictest261-21systemd-journal07:50:031
284513699183,8cyclictest0-21swapper/309:25:573
284513699183,8cyclictest0-21swapper/309:25:573
284513699183,7cyclictest0-21swapper/311:55:543
284513699183,7cyclictest0-21swapper/311:55:543
2845135991814,2cyclictest0-21swapper/207:10:022
284513699173,8cyclictest0-21swapper/309:35:553
284513699172,9cyclictest0-21swapper/309:15:573
284513699172,8cyclictest0-21swapper/310:41:003
284513699172,6cyclictest0-21swapper/310:45:533
284513699171,7cyclictest0-21swapper/310:20:553
284513699163,7cyclictest0-21swapper/312:10:523
284513699162,8cyclictest0-21swapper/310:05:543
284513699162,8cyclictest0-21swapper/309:55:563
284513699162,8cyclictest0-21swapper/309:40:523
284513699162,7cyclictest0-21swapper/310:10:533
284513699162,7cyclictest0-21swapper/309:50:533
284513699161,8cyclictest0-21swapper/310:30:543
284513699161,8cyclictest0-21swapper/307:45:583
284513699161,7cyclictest0-21swapper/312:35:563
284513699161,7cyclictest0-21swapper/307:30:553
284513699161,6cyclictest0-21swapper/311:45:563
284513699154,6cyclictest0-21swapper/310:00:593
284513699153,7cyclictest0-21swapper/311:30:563
284513699153,6cyclictest0-21swapper/310:55:543
284513699152,8cyclictest0-21swapper/312:25:573
284513699152,8cyclictest0-21swapper/309:30:533
284513699152,8cyclictest0-21swapper/309:30:533
284513699152,8cyclictest0-21swapper/309:20:543
284513699151,8cyclictest0-21swapper/312:00:523
284513699151,8cyclictest0-21swapper/310:15:563
284513699151,8cyclictest0-21swapper/310:15:563
284513699151,7cyclictest0-21swapper/310:25:533
284513699151,7cyclictest0-21swapper/309:00:553
284513699151,7cyclictest0-21swapper/307:15:523
284513699143,6cyclictest0-21swapper/311:20:013
284513699143,6cyclictest0-21swapper/311:20:013
284513699143,6cyclictest0-21swapper/309:45:543
284513699142,8cyclictest0-21swapper/311:10:573
284513699142,8cyclictest0-21swapper/311:10:573
284513699142,7cyclictest0-21swapper/307:50:553
284513699142,6cyclictest0-21swapper/307:25:583
284513699142,6cyclictest0-21swapper/307:25:583
284513699142,5cyclictest0-21swapper/307:40:553
284513699141,7cyclictest0-21swapper/311:05:523
284513699141,7cyclictest0-21swapper/308:05:553
284513699141,6cyclictest0-21swapper/308:00:553
284513699141,5cyclictest0-21swapper/308:15:523
284513699133,8cyclictest0-21swapper/311:20:543
284513699133,5cyclictest0-21swapper/311:35:533
284513699132,7cyclictest0-21swapper/311:40:533
284513699132,6cyclictest0-21swapper/312:15:523
284513699132,6cyclictest0-21swapper/311:25:523
284513699132,6cyclictest0-21swapper/307:20:543
284513699131,8cyclictest0-21swapper/312:20:563
284513699131,7cyclictest0-21swapper/310:50:443
284513699131,7cyclictest0-21swapper/308:56:033
284513699131,7cyclictest0-21swapper/308:50:573
284513699131,6cyclictest0-21swapper/308:45:573
284513699131,6cyclictest0-21swapper/308:20:513
284513699131,6cyclictest0-21swapper/308:20:513
284513699131,5cyclictest0-21swapper/308:40:513
284513699122,7cyclictest0-21swapper/311:50:473
284513699122,6cyclictest0-21swapper/309:15:013
284513699122,5cyclictest0-21swapper/311:00:463
284513699121,7cyclictest0-21swapper/312:30:563
284513699121,7cyclictest0-21swapper/310:35:533
284513699121,6cyclictest0-21swapper/308:10:523
284513699121,5cyclictest0-21swapper/307:35:533
284513699112,6cyclictest0-21swapper/312:05:523
284513699111,5cyclictest0-21swapper/309:10:023
284513699102,5cyclictest0-21swapper/307:55:483
28451369992,4cyclictest0-21swapper/308:25:473
28451369992,4cyclictest0-21swapper/307:10:463
28451369991,5cyclictest0-21swapper/308:35:473
28451369991,4cyclictest0-21swapper/308:35:023
28451369981,4cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional