You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-31 - 21:30

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7s.osadl.org (updated Mon Aug 31, 2026 12:42:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2701385995041,5cyclictest2735903-21systemd-run09:05:032
2701383995042,5cyclictest151ktimers/008:31:120
2701383995036,7cyclictest151ktimers/007:46:090
2701384994737,7cyclictest2755757-21sendmail09:40:001
2701383994733,11cyclictest2841019-21tr12:00:000
2701383994733,11cyclictest2841019-21tr12:00:000
2701385994539,2cyclictest0-21swapper/212:01:122
2701385994536,7cyclictest2725385-21sh08:30:012
2701384994537,5cyclictest2815234-21timerandwakeup11:16:181
2701385994437,4cyclictest1002-21mta-sts-daemon07:31:092
2701383994435,6cyclictest1-21systemd11:30:010
2701383994416,22cyclictest2753893-21ssh09:36:040
2701385994335,5cyclictest2856890-21aten_r2power_vo12:25:562
2701385994322,16cyclictest1002-21mta-sts-daemon10:21:122
2701383994335,5cyclictest1-21systemd11:45:020
2701384994234,5cyclictest1874569-21cups-browsed09:28:301
2701383994228,9cyclictest2715396-21fschecks_time07:56:010
2701385994133,5cyclictest2790288-21wc10:36:002
2701385994132,5cyclictest2777056-21sh10:15:012
2701383994131,7cyclictest2723883-21idleruntime-cro08:25:000
2701383994131,7cyclictest2723883-21idleruntime-cro08:25:000
2701384994034,3cyclictest0-21swapper/109:10:011
2701384994032,5cyclictest2786268-21ssh10:30:021
2701384994031,5cyclictest2800225-21grep10:51:181
2701384994031,5cyclictest2800225-21grep10:51:181
2701384994031,5cyclictest1-21systemd10:45:021
2701384994027,9cyclictest1-21systemd11:30:551
270138399405,27cyclictest623-21sshd08:41:110
2701383994033,5cyclictest2865336-21ls12:40:010
2701383994033,5cyclictest2865336-21ls12:40:010
2701383994033,4cyclictest2825816-21systemd-run11:35:010
2701383994033,3cyclictest0-21swapper/010:00:010
270138399403,31cyclictest2741685-21ssh09:16:050
2701383994032,6cyclictest2848898-21grep12:11:180
2701383994023,12cyclictest567-21gdbus11:51:120
2701383994023,12cyclictest567-21gdbus11:51:120
2701385993929,6cyclictest2722364-21munin-plugin-st08:20:002
2701385993928,8cyclictest2762377-21aten_r2power_cu09:50:532
2701384993932,4cyclictest2780120-21idleruntime-cro10:20:011
2701384993931,5cyclictest0-21swapper/111:25:011
2701384993930,5cyclictest2768006-21latency_hist10:00:021
2701384993929,7cyclictest0-21swapper/111:50:011
2701383993931,4cyclictest0-21swapper/012:26:040
2701385993830,5cyclictest2812297-21unixbench-2d11:11:172
270138599382,31cyclictest2769256-21sh10:01:072
270138599382,27cyclictest2802626-21ssh10:56:052
2701384993831,4cyclictest2840986-21ls12:00:001
2701384993831,4cyclictest2840986-21ls12:00:001
2701384993830,5cyclictest2771804-21fschecks_count10:05:561
2701383993830,5cyclictest421-21dbus-daemon08:30:010
2701383993828,3cyclictest171rcu_preempt11:05:550
270138399382,6cyclictest2817898-21ssh11:21:060
270138399381,22cyclictest2814805-21ssh11:16:070
2701385993730,4cyclictest2865353-21sendmail12:40:012
2701385993730,4cyclictest2865353-21sendmail12:40:012
2701385993730,4cyclictest2741273-21gltestperf09:15:572
2701385993730,4cyclictest1003-21mta-sts-daemon09:46:042
2701385993730,3cyclictest2707390-21kworker/u8:4-xprtiod08:00:012
2701385993728,6cyclictest421-21dbus-daemon09:40:002
2701384993729,5cyclictest2717891-21systemd-run08:05:011
2701383993729,5cyclictest2833717-21head11:46:160
2701383993718,13cyclictest12650irq/32-eth010:11:100
2701385993629,4cyclictest2766707-21grep09:56:172
2701385993628,5cyclictest2820356-21aten_r2power_po11:25:562
2701385993627,6cyclictest2701395-21munin-run07:10:012
2701384993629,4cyclictest2837940-21idleruntime-cro11:55:001
2701384993629,4cyclictest2837940-21idleruntime-cro11:55:001
2701384993627,6cyclictest2760543-21unixbench-2d09:46:171
270138499362,29cyclictest2820941-21ssh11:26:071
2701383993620,10cyclictest12650irq/32-eth009:26:090
2701385993528,4cyclictest57-21kcompactd011:06:082
2701385993527,5cyclictest421-21dbus-daemon11:50:182
2701385993527,5cyclictest421-21dbus-daemon11:50:182
2701384993527,4cyclictest2752694-21cron09:35:001
2701384993527,4cyclictest2752694-21cron09:35:001
2701384993523,8cyclictest2763704-21unixbench_multi09:51:311
270138499352,28cyclictest2854503-21ssh12:21:101
2701383993527,5cyclictest2786743-21/usr/sbin/munin10:30:590
2701383993527,5cyclictest2708887-21systemd-run07:35:020
2701383993526,6cyclictest2734391-21sh09:00:010
2701383993525,6cyclictest12650irq/32-eth010:41:060
2701383993520,12cyclictest2762688-21gltestperf09:50:570
270138599344,22cyclictest2784336-21ssh10:26:022
2701385993428,3cyclictest0-21swapper/212:00:002
2701385993428,3cyclictest0-21swapper/212:00:002
2701385993426,5cyclictest2726867-21munin-plugin-st08:35:002
2701385993426,4cyclictest2801428-21grep10:55:012
2701385993426,4cyclictest2801428-21grep10:55:012
2701385993426,4cyclictest2780107-21idleruntime-cro10:20:012
2701384993427,4cyclictest2862285-21cron12:35:011
2701384993427,4cyclictest2844196-21latency_hist12:05:031
2701384993426,5cyclictest2722900-21fschecks_count08:20:571
2701384993426,5cyclictest2722900-21fschecks_count08:20:571
2701384993425,6cyclictest2812386-21unixbench_singl11:11:181
2701384993425,6cyclictest2705752-21tail07:21:161
2701384993425,6cyclictest2705752-21tail07:21:161
2701383993424,7cyclictest421-21dbus-daemon10:53:080
2701383993424,7cyclictest421-21dbus-daemon10:53:080
2701383993424,5cyclictest2822094-21kworker/u8:0-rpciod11:38:210
2701383993423,8cyclictest2710960-21kworker/u8:5-rpciod08:05:010
2701383993415,15cyclictest2718766-21ntpq08:06:080
270138599334,24cyclictest2826956-21ssh11:36:032
2701385993325,5cyclictest2830055-21ntp_kernel_pll_11:41:062
2701385993324,6cyclictest421-21dbus-daemon12:32:002
2701385993321,7cyclictest2805858-21ssh11:01:122
2701384993326,4cyclictest2710377-21sendmail07:40:011
2701384993325,5cyclictest2848088-21latency12:11:011
2701384993325,5cyclictest1874569-21cups-browsed12:25:561
270138399338,8cyclictest2850455-21ssh12:15:220
270138399333,25cyclictest2760093-21ssh09:46:080
2701383993326,4cyclictest2784400-21ntp_kernel_pll_10:26:050
2701383993325,4cyclictest0-21swapper/011:01:150
2701383993324,6cyclictest1874569-21cups-browsed10:17:070
2701383993323,6cyclictest2811254-21cpu11:10:550
270138399332,24cyclictest665-21snmpd09:13:360
2701383993313,5cyclictest14-21ksoftirqd/010:21:050
270138599325,17cyclictest2848391-21ssh12:11:102
270138599324,13cyclictest2737372-21diskmemload12:21:562
2701385993226,3cyclictest0-21swapper/209:06:132
2701385993223,6cyclictest2723895-21systemd-run08:25:012
2701385993223,6cyclictest2723895-21systemd-run08:25:012
2701384993226,3cyclictest0-21swapper/109:05:011
2701384993225,3cyclictest0-21swapper/112:15:581
2701384993224,6cyclictest2807755-21latency_hist11:05:021
2701384993223,7cyclictest2737372-21diskmemload12:36:161
2701384993223,7cyclictest2737372-21diskmemload12:36:161
2701384993222,8cyclictest2758840-21systemd-run09:45:011
270138399329,6cyclictest14-21ksoftirqd/010:01:070
270138399329,5cyclictest14-21ksoftirqd/012:21:080
2701383993225,4cyclictest0-21swapper/010:55:550
2701383993224,5cyclictest2737029-21grep09:06:160
2701383993224,4cyclictest0-21swapper/009:33:400
2701383993224,4cyclictest0-21swapper/009:33:400
2701383993222,4cyclictest0-21swapper/010:05:550
2701383993215,11cyclictest12650irq/32-eth010:36:050
2701385993125,3cyclictest0-21swapper/211:46:162
2701385993124,4cyclictest2719357-21kworker/u8:0-rpciod08:51:112
2701385993123,5cyclictest2788054-21unixbench_singl10:31:202
2701385993123,5cyclictest2772783-21unixbench_multi10:06:152
2701385993123,5cyclictest0-21swapper/211:35:032
2701385993122,6cyclictest1874569-21cups-browsed09:10:522
2701384993126,2cyclictest0-21swapper/107:15:011
2701384993126,2cyclictest0-21swapper/107:15:011
2701384993124,4cyclictest2802953-21wc10:56:151
2701384993123,5cyclictest567-21gdbus08:50:041
2701384993123,5cyclictest2831901-21munin-plugin-st11:45:011
2701384993123,5cyclictest2728399-21systemd-run08:40:021
270138499312,23cyclictest2781314-21ssh10:21:061
270138399318,15cyclictest2796638-21ssh10:46:080
270138399317,17cyclictest0-21swapper/007:21:100
270138399317,17cyclictest0-21swapper/007:21:100
2701383993122,6cyclictest2706621-21missed_timers07:26:020
270138399312,24cyclictest2727851-21sshd08:36:090
2701383993121,8cyclictest2709682-21needreboot07:36:030
2701383993116,9cyclictest2720278-21ntpq08:11:070
2701383993115,14cyclictest151ktimers/009:01:070
2701383993115,10cyclictest2730822-21ntpq08:46:070
2701383993111,5cyclictest14-21ksoftirqd/008:55:010
2701385993022,5cyclictest420-21avahi-daemon09:41:542
2701385993022,5cyclictest1874569-21cups-browsed09:29:362
2701385993022,5cyclictest0-21swapper/209:31:012
2701385993022,5cyclictest0-21swapper/209:31:012
2701384993023,3cyclictest0-21swapper/112:06:071
2701384993022,5cyclictest567-21gdbus07:26:441
2701384993022,5cyclictest2742201-21unixbench_multi09:16:161
270138399302,20cyclictest12650irq/32-eth012:31:030
2701383993019,5cyclictest2702219-21ntpq07:11:010
2701383993019,5cyclictest2702219-21ntpq07:11:010
2701383993010,4cyclictest14-21ksoftirqd/012:06:050
2701385992924,2cyclictest0-21swapper/208:40:012
2701385992923,3cyclictest2789255-21kworker/u8:10-xprtiod10:40:572
2701385992922,4cyclictest2701382-21cyclictest07:21:072
2701385992922,4cyclictest2701382-21cyclictest07:21:072
2701385992922,4cyclictest2692067-21kworker/u8:0+xprtiod07:40:022
2701385992920,6cyclictest2711896-21true07:45:012
2701385992918,8cyclictest1874569-21cups-browsed11:23:402
2701385992917,9cyclictest2733702-21ntp_kernel_err08:56:032
2701384992924,2cyclictest0-21swapper/111:40:011
2701384992923,3cyclictest1001-21mta-sts-daemon10:46:211
2701384992921,5cyclictest2722361-21ls08:20:001
2701384992920,6cyclictest567-21gdbus08:33:121
2701384992920,5cyclictest2810732-21latency_hist11:10:021
2701384992910,14cyclictest420-21avahi-daemon07:16:061
2701383992923,3cyclictest0-21swapper/009:41:150
270138399290,26cyclictest0-21swapper/009:21:060
270138599284,19cyclictest2845252-21ssh12:06:062
2701385992822,3cyclictest0-21swapper/209:21:142
2701385992821,3cyclictest0-21swapper/210:46:022
2701385992810,11cyclictest351ktimers/207:10:562
2701385992810,11cyclictest351ktimers/207:10:562
2701384992822,3cyclictest0-21swapper/109:21:141
2701384992822,3cyclictest0-21swapper/109:15:011
2701384992822,3cyclictest0-21swapper/107:10:011
2701384992821,4cyclictest2791059-21unixbench_multi10:36:171
2701384992821,3cyclictest0-21swapper/110:31:181
2701384992821,3cyclictest0-21swapper/109:00:011
2701384992820,5cyclictest2731390-21systemd-run08:50:021
2701384992818,7cyclictest2769526-21switchtime10:01:151
2701384992817,4cyclictest2737372-21diskmemload10:15:011
270138399285,9cyclictest2722159-21sed08:16:160
2701383992822,3cyclictest0-21swapper/007:15:550
270138399282,21cyclictest1874566-21cupsd12:01:080
2701385992722,2cyclictest0-21swapper/211:20:012
270138599272,21cyclictest2851354-21ssh12:16:072
2701385992721,3cyclictest0-21swapper/208:40:582
2701385992719,5cyclictest2731365-21cron08:50:012
2701385992719,5cyclictest2716906-21fschecks_time08:00:582
2701385992718,6cyclictest1000-21mta-sts-daemon07:46:012
2701384992721,3cyclictest2707456-21kworker/u8:10-xprtiod07:50:011
270138399272,20cyclictest665-21snmpd07:41:050
2701383992710,4cyclictest14-21ksoftirqd/007:10:010
2701385992615,8cyclictest421-21dbus-daemon07:15:542
2701384992621,3cyclictest0-21swapper/108:45:011
2701384992621,3cyclictest0-21swapper/108:15:011
2701384992620,4cyclictest2715224-21aten_r2power_po07:55:561
2701384992618,5cyclictest2711751-21tail07:41:161
270138399262,19cyclictest665-21snmpd07:51:080
2701385992518,4cyclictest2700169-21kworker/u8:7-xprtiod07:51:052
2701384992517,4cyclictest2725368-21latency_hist08:30:001
2701385992418,4cyclictest2701382-21cyclictest08:11:022
2701384992417,4cyclictest2713632-21aten_r2power_cu07:50:551
2701384992416,5cyclictest2708820-21unixbench_singl07:31:161
2701385992317,3cyclictest1001-21mta-sts-daemon07:25:582
270138599213,13cyclictest2718896-21ps08:06:132
2701384992116,3cyclictest2714993-21kworker/u8:14-xprtiod08:06:051
270138699192,11cyclictest0-21swapper/310:16:043
270138699185,8cyclictest0-21swapper/311:01:103
270138699184,8cyclictest0-21swapper/311:26:173
270138699184,7cyclictest0-21swapper/309:46:043
270138699183,9cyclictest0-21swapper/312:26:073
270138699183,7cyclictest0-21swapper/312:36:123
270138699183,7cyclictest0-21swapper/312:36:123
270138699174,8cyclictest0-21swapper/311:11:043
270138699173,9cyclictest0-21swapper/311:36:073
270138699173,8cyclictest0-21swapper/311:56:063
270138699173,8cyclictest0-21swapper/311:56:063
270138699173,8cyclictest0-21swapper/310:01:053
270138699166,6cyclictest0-21swapper/309:16:103
270138699164,8cyclictest0-21swapper/310:26:033
270138699164,7cyclictest0-21swapper/312:01:143
270138699163,8cyclictest0-21swapper/310:46:063
270138699163,8cyclictest0-21swapper/310:21:053
270138699163,6cyclictest0-21swapper/309:26:033
270138699162,8cyclictest0-21swapper/312:21:063
270138699162,8cyclictest0-21swapper/310:56:053
270138699162,8cyclictest0-21swapper/309:56:083
270138699162,8cyclictest0-21swapper/307:16:053
270138699162,7cyclictest0-21swapper/309:51:043
270138699162,7cyclictest0-21swapper/307:36:083
270138699162,10cyclictest0-21swapper/310:31:093
270138699161,7cyclictest0-21swapper/307:21:073
270138699161,7cyclictest0-21swapper/307:21:073
270138699154,8cyclictest0-21swapper/310:41:043
270138699153,8cyclictest0-21swapper/312:06:053
270138699153,8cyclictest0-21swapper/311:16:083
270138699153,8cyclictest0-21swapper/310:51:063
270138699153,8cyclictest0-21swapper/310:51:063
270138699153,4cyclictest0-21swapper/308:11:043
270138699152,9cyclictest0-21swapper/308:50:023
270138699152,8cyclictest0-21swapper/312:31:003
270138699152,8cyclictest0-21swapper/307:26:113
270138699152,7cyclictest0-21swapper/309:01:113
270138699152,7cyclictest0-21swapper/307:56:063
270138699152,6cyclictest0-21swapper/311:30:543
270138699152,6cyclictest0-21swapper/309:06:033
270138699151,7cyclictest0-21swapper/307:31:063
270138699145,6cyclictest0-21swapper/307:51:113
270138699144,6cyclictest0-21swapper/310:11:113
270138699143,7cyclictest0-21swapper/312:16:063
270138699143,7cyclictest0-21swapper/311:21:033
270138699142,9cyclictest0-21swapper/309:36:043
270138699142,8cyclictest0-21swapper/311:06:043
270138699142,8cyclictest0-21swapper/309:30:583
270138699142,8cyclictest0-21swapper/309:30:583
270138699142,7cyclictest0-21swapper/311:51:003
270138699142,7cyclictest0-21swapper/311:51:003
270138699142,7cyclictest0-21swapper/310:36:083
270138699142,7cyclictest0-21swapper/310:06:023
270138699141,8cyclictest0-21swapper/308:01:063
270138699141,7cyclictest0-21swapper/308:41:043
270138699141,6cyclictest0-21swapper/308:56:093
270138699134,6cyclictest0-21swapper/308:31:053
270138699132,8cyclictest0-21swapper/309:21:023
270138699132,7cyclictest0-21swapper/308:51:033
270138699132,6cyclictest0-21swapper/312:10:563
270138699132,6cyclictest0-21swapper/308:26:043
270138699131,8cyclictest0-21swapper/311:46:093
270138699131,8cyclictest0-21swapper/311:41:043
270138699131,8cyclictest0-21swapper/308:20:013
270138699131,7cyclictest0-21swapper/309:41:023
270138699131,6cyclictest0-21swapper/309:10:203
270138699131,6cyclictest0-21swapper/307:46:033
270138699125,5cyclictest0-21swapper/307:11:033
270138699125,5cyclictest0-21swapper/307:11:033
270138699121,7cyclictest0-21swapper/307:41:053
270138699121,6cyclictest0-21swapper/308:21:053
270138699121,6cyclictest0-21swapper/308:21:053
270138699113,4cyclictest0-21swapper/308:06:073
270138699111,6cyclictest0-21swapper/308:36:053
27013869992,4cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional