You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-05 - 02:47

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Fri Jun 05, 2026 00:42:54)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2822228996451,10cyclictest2890538-21wc22:00:560
2822229995831,24cyclictest2858215-21grep21:10:011
282222899516,37cyclictest151ktimers/020:06:030
2822229995034,13cyclictest0-21swapper/123:40:011
2822228995042,5cyclictest2858215-21grep21:10:010
2822229994919,27cyclictest2935865-21unixbench_multi23:11:151
282222999471,19cyclictest2982739-21ssh00:26:071
282222999471,19cyclictest2982739-21ssh00:26:071
2822229994622,21cyclictest261-21systemd-journal19:20:591
2822229994620,23cyclictest2847711-21grep20:35:011
2822229994617,25cyclictest0-21swapper/123:46:051
2822228994640,4cyclictest2930961-21ssh23:05:010
2822229994534,7cyclictest2947631-21if_wlan023:30:571
2822229994520,21cyclictest2900919-21timerandwakeup22:16:131
282222999450,17cyclictest2887874-21ntpq21:56:041
282222899454,32cyclictest2973150-21ssh00:11:010
2822228994535,7cyclictest421-21dbus-daemon20:46:030
2822230994435,6cyclictest421-21dbus-daemon23:40:012
2822230994433,5cyclictest0-21swapper/200:06:102
2822229994421,2cyclictest0-21swapper/123:01:071
2822229994412,6cyclictest2913521-21switchtime22:36:141
2822229994412,6cyclictest2913521-21switchtime22:36:141
2822230994335,5cyclictest2934123-21latency_hist23:10:012
2822229994335,5cyclictest1-21systemd20:50:011
282222999432,21cyclictest2944838-21ssh23:26:061
2822229994321,18cyclictest0-21swapper/122:30:011
282222999431,18cyclictest2919872-21sort22:46:141
282222999431,18cyclictest2919872-21sort22:46:141
2822228994322,14cyclictest151ktimers/021:46:030
2822230994233,6cyclictest516-21in:imuxsock00:21:102
2822230994233,6cyclictest516-21in:imuxsock00:21:102
2822230994232,7cyclictest2843211-21munin-run20:20:002
2822230994232,7cyclictest2843211-21munin-run20:20:002
2822229994223,15cyclictest0-21swapper/100:16:111
282222999422,16cyclictest2925823-21ssh22:56:051
2822229994214,7cyclictest2825014-21timerwakeupswit19:16:141
282222999421,17cyclictest2931969-21sed23:05:581
282222999421,16cyclictest2910327-21sensors_volt22:31:141
2822228994231,8cyclictest2953015-21/usr/sbin/munin23:40:010
2822230994131,6cyclictest420-21avahi-daemon00:26:092
2822230994131,6cyclictest420-21avahi-daemon00:26:092
2822230994128,9cyclictest2907185-21switchtime22:26:142
2822229994133,5cyclictest2837186-21cron20:00:001
282222999413,15cyclictest428-21mta-sts-daemon21:41:041
2822229994123,15cyclictest2860003-21tail21:11:141
282222999412,13cyclictest2894285-21ssh22:06:061
2822228994125,11cyclictest0-21swapper/021:31:160
282223099405,25cyclictest2868984-21ssh21:26:062
2822230994030,7cyclictest2856722-21munin-plugin-st21:05:012
282222999404,25cyclictest2875208-21ssh21:36:051
282222999404,13cyclictest2884143-21aten_r2power_po21:50:531
2822229994032,5cyclictest2916922-21ssh22:41:141
2822229994032,5cyclictest2916922-21ssh22:41:141
2822229994032,5cyclictest2850790-21latency_hist20:45:021
282222999403,12cyclictest428-21mta-sts-daemon22:21:051
282222999401,18cyclictest2872084-21ssh21:31:041
282222999401,17cyclictest2891081-21ssh22:01:061
2822229994010,8cyclictest2898782-21sh22:13:311
2822228994021,12cyclictest12350irq/32-eth000:31:010
2822228994021,12cyclictest12350irq/32-eth000:31:010
2822230993932,4cyclictest2968821-21cron00:05:012
2822230993932,4cyclictest2902469-21munin-run22:20:012
282222999394,10cyclictest2941742-21ssh23:21:071
282222999392,28cyclictest2881669-21ssh21:46:081
2822229993918,2cyclictest0-21swapper/123:56:111
2822228993926,7cyclictest12350irq/32-eth000:21:020
2822228993926,7cyclictest12350irq/32-eth000:21:020
2822228993920,10cyclictest151ktimers/020:11:000
2822228993920,10cyclictest151ktimers/020:11:000
2822230993831,4cyclictest2861380-21munin-plugin-st21:15:012
2822230993830,5cyclictest2915099-21ssh22:40:012
2822230993830,5cyclictest2915099-21ssh22:40:012
2822230993828,6cyclictest2829783-21latency_hist19:35:022
2822230993826,8cyclictest261-21systemd-journal23:41:092
282223099382,26cyclictest2878537-21ssh21:41:102
2822229993830,3cyclictest0-21swapper/119:51:081
282222999381,15cyclictest2985912-21ssh00:31:051
282222999381,15cyclictest2985912-21ssh00:31:051
2822228993831,3cyclictest0-21swapper/000:20:010
2822228993829,6cyclictest516-21in:imuxsock19:36:080
2822228993815,16cyclictest12350irq/32-eth021:36:060
2822230993729,5cyclictest2877160-21munin-plugin-st21:40:012
2822230993729,5cyclictest2837207-21awk20:00:012
282223099371,11cyclictest0-21swapper/221:21:072
2822229993729,5cyclictest2837732-21fschecks_count20:00:551
282222999372,15cyclictest2938544-21ntpq23:16:061
2822228993730,4cyclictest2829734-21systemd-run19:35:010
2822228993729,5cyclictest420-21avahi-daemon22:52:400
2822228993729,5cyclictest420-21avahi-daemon22:52:400
2822228993727,7cyclictest2967449-21unixbench-2d00:01:160
282223099363,27cyclictest659-21snmpd22:41:032
282223099363,27cyclictest659-21snmpd22:41:032
2822230993631,2cyclictest0-21swapper/222:55:002
2822230993631,2cyclictest0-21swapper/222:55:002
2822230993628,3cyclictest0-21swapper/222:46:072
2822230993628,3cyclictest0-21swapper/222:46:072
2822230993626,6cyclictest2904332-21head22:21:142
2822230993625,8cyclictest2947390-21aten_r2power_vo23:30:532
282223099361,28cyclictest2881770-21sh21:46:112
2822229993630,3cyclictest0-21swapper/120:30:001
2822229993628,5cyclictest0-21swapper/120:20:001
2822229993628,5cyclictest0-21swapper/120:20:001
282222999361,18cyclictest2966969-21ntpq00:01:071
2822228993621,12cyclictest2823782-21latency_hist19:15:020
2822228993615,17cyclictest0-21swapper/021:16:080
2822230993529,3cyclictest0-21swapper/219:40:012
2822230993528,4cyclictest0-21swapper/200:20:012
2822230993527,5cyclictest2977269-21kworker/u8:1-xprtiod00:31:002
2822230993527,5cyclictest2977269-21kworker/u8:1-xprtiod00:31:002
2822230993527,5cyclictest2897691-21switchtime22:11:132
2822230993527,4cyclictest2887881-21modprobe21:56:042
2822230993526,6cyclictest1086143-21cups-browsed22:05:542
282223099352,27cyclictest1086143-21cups-browsed20:36:092
2822229993529,3cyclictest0-21swapper/122:55:011
2822229993529,3cyclictest0-21swapper/122:55:011
2822229993529,3cyclictest0-21swapper/121:20:011
2822229993527,5cyclictest2841594-21unixbench_multi20:11:121
2822229993527,5cyclictest2841594-21unixbench_multi20:11:121
282222999352,14cyclictest1086139-21cupsd21:20:441
2822229993510,7cyclictest2858211-21diskmemload00:24:451
2822229993510,7cyclictest2858211-21diskmemload00:24:451
2822228993527,6cyclictest2846229-21sh20:30:010
282223099344,22cyclictest2847192-21sshd20:31:082
2822230993428,4cyclictest2822227-21cyclictest23:56:022
2822230993426,5cyclictest2858130-21unixbench_singl21:06:152
2822230993425,6cyclictest548-21gdbus20:11:082
2822230993425,6cyclictest548-21gdbus20:11:082
2822228993428,4cyclictest1056-21mta-sts-daemon21:57:440
2822228993425,5cyclictest0-21swapper/022:56:050
2822228993422,7cyclictest2965651-21latency_hist00:00:030
2822230993326,5cyclictest0-21swapper/221:20:012
2822230993325,5cyclictest2855195-21munin-run21:00:012
2822230993325,4cyclictest0-21swapper/221:35:022
2822230993323,7cyclictest2956866-21aten_r2power_vo23:45:552
282222999332,26cyclictest2830700-21sshd19:36:081
2822229993319,11cyclictest2955459-21sh23:42:581
2822228993326,4cyclictest2904180-21ssh22:21:130
2822228993325,4cyclictest0-21swapper/023:06:060
2822228993324,7cyclictest2880328-21sh21:45:010
282223099328,19cyclictest2944906-21ssh23:26:092
2822230993226,3cyclictest0-21swapper/223:11:082
2822230993226,3cyclictest0-21swapper/220:10:012
2822230993223,7cyclictest261-21systemd-journal22:01:092
2822229993227,2cyclictest0-21swapper/120:40:011
2822229993225,4cyclictest2837177-21kworker/u8:5-xprtiod21:05:011
282222899327,18cyclictest12350irq/32-eth000:36:080
2822228993227,2cyclictest0-21swapper/021:25:550
2822228993226,4cyclictest2837193-21idleruntime-cro20:00:010
2822228993224,5cyclictest0-21swapper/022:41:100
2822228993224,5cyclictest0-21swapper/022:41:100
282223099313,22cyclictest2960676-21ntpq23:51:102
2822230993125,3cyclictest0-21swapper/219:15:012
2822230993124,4cyclictest2989562-21grep00:36:182
2822230993123,5cyclictest2940400-21munin-plugin-st23:20:012
2822230993123,5cyclictest2834440-21acpi19:50:512
2822229993125,3cyclictest0-21swapper/121:26:111
2822228993125,4cyclictest2860019-21unixbench_multi21:11:140
282222899312,21cyclictest421-21dbus-daemon20:21:070
282222899312,21cyclictest2969645-21ssh00:05:580
2822228993121,4cyclictest2908828-21latency_hist22:30:010
2822228993115,10cyclictest12350irq/32-eth023:41:110
2822230993023,5cyclictest2965636-21kworker/u8:2+xprtiod00:15:012
2822230993023,4cyclictest2827377-21kernelversion19:25:582
2822230993012,12cyclictest421-21dbus-daemon20:46:042
282222999302,22cyclictest2960614-21ssh23:51:071
2822228993023,4cyclictest439-21systemd-logind21:00:010
2822228993022,6cyclictest0-21swapper/022:10:020
2822228993021,6cyclictest2884833-21sshd21:51:080
2822228993016,8cyclictest12350irq/32-eth023:31:090
282222899300,14cyclictest2856047-21netstat21:01:000
282223099293,22cyclictest2941642-21ssh23:21:032
2822230992924,2cyclictest0-21swapper/223:01:002
2822230992922,3cyclictest0-21swapper/222:56:092
2822230992922,3cyclictest0-21swapper/221:50:552
2822230992921,5cyclictest2853693-21cron20:55:012
2822230992921,4cyclictest0-21swapper/222:35:002
2822229992918,8cyclictest2855192-21munin-plugin-st21:00:021
282222899294,20cyclictest2982751-21ssh00:26:070
282222899294,20cyclictest2982751-21ssh00:26:070
2822228992923,4cyclictest2915117-21munin-run22:40:010
2822228992923,4cyclictest2915117-21munin-run22:40:010
2822228992923,3cyclictest0-21swapper/022:35:010
2822228992920,7cyclictest12350irq/32-eth021:21:090
2822230992821,5cyclictest2837443-21kworker/u8:7+xprtiod20:01:002
2822230992821,4cyclictest2826654-21unixbench_singl19:21:182
282222999282,19cyclictest2988964-21ssh00:36:031
2822229992819,6cyclictest421-21dbus-daemon19:35:011
2822228992823,3cyclictest0-21swapper/022:20:010
2822228992822,3cyclictest2937244-21munin-plugin-st23:15:000
282222899282,21cyclictest2960686-21ssh23:51:100
282222899281,8cyclictest2941212-21df_inode23:20:550
2822228992818,5cyclictest420-21avahi-daemon20:20:010
2822228992818,5cyclictest420-21avahi-daemon20:20:010
2822228992817,9cyclictest261-21systemd-journal19:25:010
282223199272,6cyclictest0-21swapper/323:31:023
2822230992721,4cyclictest0-21swapper/219:41:092
2822230992721,3cyclictest0-21swapper/219:20:012
2822230992718,6cyclictest1-21systemd20:45:022
282222999274,4cyclictest171rcu_preempt00:11:161
2822229992716,8cyclictest2839705-21sshd20:06:091
2822228992721,3cyclictest0-21swapper/023:45:580
2822228992720,3cyclictest0-21swapper/019:41:090
2822228992716,4cyclictest2918871-21telnet22:45:530
2822228992716,4cyclictest2918871-21telnet22:45:530
2822228992716,4cyclictest0-21swapper/022:13:080
2822230992620,3cyclictest0-21swapper/220:21:142
282222999264,16cyclictest547-21gmain19:11:081
282222899269,12cyclictest12350irq/32-eth023:16:030
2822228992617,5cyclictest12350irq/32-eth020:01:090
282222999252,15cyclictest2970079-21ssh00:06:051
2822229992519,3cyclictest0-21swapper/119:45:011
2822228992519,3cyclictest0-21swapper/020:51:080
2822228992519,3cyclictest0-21swapper/019:20:010
282222899251,6cyclictest2849202-21munin-run20:40:010
2822228992516,6cyclictest2945257-21timerwakeupswit23:26:160
2822228992513,2cyclictest0-21swapper/019:46:110
2822230992414,8cyclictest2846211-21sh20:30:002
2822229992418,3cyclictest2811104-21kworker/u8:4-xprtiod19:50:011
2822229992416,5cyclictest2826994-21tail19:25:521
282223199231,7cyclictest0-21swapper/300:11:033
282223199231,6cyclictest0-21swapper/321:46:083
282223199231,5cyclictest0-21swapper/321:21:093
282223199231,4cyclictest0-21swapper/321:31:063
282223199231,4cyclictest0-21swapper/319:41:083
282223199231,4cyclictest0-21swapper/319:26:063
282223199231,4cyclictest0-21swapper/300:21:083
282223199231,4cyclictest0-21swapper/300:21:083
282223199231,4cyclictest0-21swapper/300:01:053
282223199231,3cyclictest0-21swapper/322:46:023
282223199231,3cyclictest0-21swapper/322:46:023
282223199231,3cyclictest0-21swapper/322:41:033
282223199231,3cyclictest0-21swapper/322:41:033
282223199231,3cyclictest0-21swapper/320:11:053
282223199231,3cyclictest0-21swapper/320:11:053
282223199230,4cyclictest0-21swapper/300:26:113
282223199230,4cyclictest0-21swapper/300:26:113
2822230992317,3cyclictest0-21swapper/219:45:542
282222899232,15cyclictest2847065-21ntpq20:31:020
282223199221,6cyclictest0-21swapper/319:16:073
282223199221,5cyclictest0-21swapper/319:11:023
282223199221,4cyclictest0-21swapper/323:51:073
282223199221,4cyclictest0-21swapper/323:40:013
282223199221,4cyclictest0-21swapper/322:31:073
282223199221,4cyclictest0-21swapper/322:21:053
282223199221,4cyclictest0-21swapper/322:11:063
282223199221,4cyclictest0-21swapper/321:11:073
282223199221,4cyclictest0-21swapper/320:51:043
282223199221,4cyclictest0-21swapper/320:46:033
282223199221,4cyclictest0-21swapper/320:31:063
282223199221,4cyclictest0-21swapper/320:21:063
282223199221,4cyclictest0-21swapper/319:46:033
282223199221,4cyclictest0-21swapper/300:36:083
282223199221,3cyclictest0-21swapper/322:36:083
282223199221,3cyclictest0-21swapper/322:36:083
282223199221,3cyclictest0-21swapper/322:26:073
282223199221,3cyclictest0-21swapper/321:41:043
282223199220,4cyclictest0-21swapper/323:41:053
282223199220,3cyclictest0-21swapper/320:01:093
2822228992216,4cyclictest151ktimers/019:26:040
282222899221,17cyclictest2835359-21grep19:51:130
282223199212,6cyclictest0-21swapper/323:11:023
282223199212,4cyclictest0-21swapper/320:36:043
282223199211,5cyclictest0-21swapper/321:01:063
282223199211,5cyclictest0-21swapper/319:51:013
282223199211,4cyclictest0-21swapper/323:46:043
282223199211,4cyclictest0-21swapper/323:01:043
282223199211,4cyclictest0-21swapper/322:01:073
282223199211,4cyclictest0-21swapper/321:36:073
282223199211,4cyclictest0-21swapper/320:41:023
282223199211,4cyclictest0-21swapper/320:26:073
282223199211,4cyclictest0-21swapper/320:00:003
282223199211,3cyclictest0-21swapper/323:56:073
282223199211,3cyclictest0-21swapper/323:21:073
282223199211,3cyclictest0-21swapper/322:55:593
282223199211,3cyclictest0-21swapper/322:51:063
282223199211,3cyclictest0-21swapper/322:51:063
282223199211,3cyclictest0-21swapper/322:16:073
282223199211,3cyclictest0-21swapper/322:06:043
282223199211,3cyclictest0-21swapper/321:56:083
282223199211,3cyclictest0-21swapper/321:26:043
282223199211,3cyclictest0-21swapper/321:16:043
282223199211,3cyclictest0-21swapper/320:16:043
282223199211,3cyclictest0-21swapper/320:16:043
282223199211,3cyclictest0-21swapper/319:36:043
282223199211,3cyclictest0-21swapper/319:31:033
282223199211,3cyclictest0-21swapper/319:21:043
282223199211,3cyclictest0-21swapper/300:16:083
282223199211,3cyclictest0-21swapper/300:06:063
282223199211,16cyclictest0-21swapper/321:06:043
282223199210,4cyclictest0-21swapper/321:51:013
282223199210,4cyclictest0-21swapper/320:06:073
2822229992112,6cyclictest2843875-21kernelversion20:20:581
282222999211,15cyclictest2853134-21ntpq20:51:051
282222899211,17cyclictest2850206-21ntpq20:41:100
282223199201,4cyclictest0-21swapper/300:31:013
282223199201,4cyclictest0-21swapper/300:31:013
282223199201,3cyclictest0-21swapper/323:15:583
282223199201,3cyclictest0-21swapper/323:06:013
282223199201,3cyclictest0-21swapper/320:56:033
282223199200,4cyclictest0-21swapper/323:26:003
2822228991914,2cyclictest0-21swapper/019:10:010
2822229991812,3cyclictest1054-21mta-sts-daemon19:10:031
282223199171,3cyclictest0-21swapper/319:10:013
282223099155,7cyclictest421-21dbus-daemon19:10:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional