You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-29 - 23:52

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Sat Aug 29, 2026 12:42:54)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2965856995738,15cyclictest3040536-21sshd10:11:481
2965856995127,21cyclictest2980720-21unixbench_multi07:56:151
2965857995040,5cyclictest3084253-21latency_hist11:25:022
2965855995042,5cyclictest3022057-21unixbench_singl09:41:150
2965855995042,5cyclictest3022057-21unixbench_singl09:41:150
2965856994939,6cyclictest2164266-21cups-browsed12:03:161
2965856994830,15cyclictest3063676-21ssh10:50:581
2965856994830,15cyclictest3063676-21ssh10:50:581
2965856994828,17cyclictest1-21systemd10:25:021
2965856994724,20cyclictest0-21swapper/111:10:011
2965855994733,12cyclictest3090244-21ssh11:35:010
296585799452,37cyclictest3097607-21ssh11:46:062
2965856994526,15cyclictest3099396-21munin-run11:50:011
296585699451,15cyclictest2971306-21ntpq07:26:101
2965857994438,3cyclictest0-21swapper/207:40:022
296585699443,9cyclictest2989258-21ntpq08:26:061
296585699442,13cyclictest2973846-21diskstats07:35:561
2965855994437,4cyclictest2982345-21cron08:05:010
296585699434,9cyclictest3021480-21ssh09:41:041
296585699434,9cyclictest3021480-21ssh09:41:041
296585699434,9cyclictest2999823-21ntpq09:01:121
2965856994335,5cyclictest421-21dbus-daemon12:30:021
2965856994326,12cyclictest0-21swapper/109:46:021
2965856994326,12cyclictest0-21swapper/109:46:021
2965856994324,14cyclictest0-21swapper/110:56:061
296585699432,19cyclictest3079321-21ssh11:16:071
296585699431,18cyclictest3036804-21ntpq10:06:111
296585699431,12cyclictest2966707-21ntpq07:11:021
296585699431,12cyclictest2966707-21ntpq07:11:021
2965855994336,4cyclictest3042301-21cpuspeed10:15:560
2965855994334,5cyclictest3045449-21fschecks_time10:20:580
296585699424,11cyclictest2968298-21ntpq07:16:211
2965856994233,5cyclictest3101211-21unixbench_singl11:51:151
296585699423,21cyclictest3033670-21ssh10:01:061
2965856994221,18cyclictest2997707-21aten_r2power_po08:55:541
296585699421,29cyclictest3055031-21ssh10:36:101
296585699421,14cyclictest2987779-21ntpq08:21:051
2965855994226,13cyclictest3084183-21munin-plugin-st11:25:010
2965857994128,9cyclictest3076663-21timerandwakeup11:11:162
296585799411,4cyclictest171rcu_preempt12:26:072
296585699414,8cyclictest3091419-21ntpq11:36:031
2965856994133,5cyclictest3124408-21aten_r2power_vo12:30:551
2965856994123,15cyclictest3030189-21fschecks_time09:55:581
296585699412,14cyclictest3051908-21ntpq10:31:031
2965856994120,18cyclictest3044607-21latency_hist10:20:021
2965856994116,4cyclictest3114635-21kworker/u8:16-rpciod12:25:011
2965856994111,3cyclictest0-21swapper/109:21:041
2965857994032,5cyclictest3129755-21munin-run12:40:012
2965857994032,5cyclictest3056779-21ls10:40:012
2965857994031,6cyclictest3124641-21if_wlan012:30:592
2965857994031,5cyclictest3029384-21ssh09:55:002
2965857994031,5cyclictest3029384-21ssh09:55:002
296585699403,12cyclictest2996717-21ntpq08:51:041
2965856994030,6cyclictest1003-21mta-sts-daemon07:31:051
296585699402,17cyclictest3003082-21ssh09:11:061
2965856994013,2cyclictest0-21swapper/111:56:071
296585699401,13cyclictest3082352-21awk11:21:041
2965855994032,5cyclictest151ktimers/008:16:050
2965855994025,12cyclictest3012707-21timerwakeupswit09:26:140
296585699393,9cyclictest3061121-21ssh10:46:111
2965856993931,5cyclictest2992826-21cron08:40:001
2965856993929,6cyclictest0-21swapper/109:26:011
2965856993916,19cyclictest0-21swapper/111:05:021
2965856993915,2cyclictest0-21swapper/111:31:081
2965856993914,3cyclictest0-21swapper/111:11:121
2965856993914,2cyclictest0-21swapper/110:41:041
2965855993933,4cyclictest3068956-21idleruntime-cro11:00:000
2965855993932,4cyclictest2983865-21kworker/u8:6-rpciod09:20:010
2965857993829,5cyclictest3106288-21uniq12:00:552
2965857993828,6cyclictest439-21systemd-logind08:05:022
2965857993828,6cyclictest3069462-21/usr/sbin/munin11:00:552
296585699382,9cyclictest3015414-21ntpq09:31:071
2965856993826,9cyclictest3110072-21switchtime12:06:141
2965856993826,9cyclictest3110072-21switchtime12:06:141
296585699382,14cyclictest3128566-21/usr/sbin/munin12:36:181
2965856993818,16cyclictest0-21swapper/111:40:561
2965856993814,5cyclictest3017720-21aten_r2power_cu09:35:541
296585699381,14cyclictest3027521-21ssh09:51:041
296585699381,14cyclictest3027521-21ssh09:51:041
296585699381,13cyclictest3112384-21uniq12:10:571
296585699381,13cyclictest3085387-21ssh11:26:041
296585699381,12cyclictest3115538-21irqstats12:16:001
296585699381,12cyclictest3005842-21awk09:16:001
2965855993830,5cyclictest0-21swapper/010:26:050
296585799373,26cyclictest3079376-21ssh11:16:092
2965857993729,5cyclictest3095067-21unixbench_multi11:41:152
2965857993728,5cyclictest0-21swapper/211:31:072
2965857993728,4cyclictest0-21swapper/208:36:052
2965857993727,6cyclictest2164266-21cups-browsed09:16:392
296585799371,28cyclictest3112832-21ssh12:11:072
2965856993727,5cyclictest1002-21mta-sts-daemon08:31:081
2965856993716,18cyclictest3049567-21sshd10:26:261
2965856993710,4cyclictest2998821-21kworker/u8:2-xprtiod09:06:181
296585799363,26cyclictest3118946-21ssh12:21:072
2965857993629,4cyclictest0-21swapper/209:21:152
2965857993628,4cyclictest0-21swapper/210:06:142
2965857993627,5cyclictest3092008-21tail11:36:172
2965857993627,4cyclictest0-21swapper/209:11:042
2965857993625,8cyclictest2977853-21cut07:50:022
2965856993617,16cyclictest0-21swapper/107:40:571
2965856993615,3cyclictest0-21swapper/107:21:161
2965856993615,3cyclictest0-21swapper/107:21:161
2965856993612,4cyclictest2993083-21/usr/sbin/munin08:41:161
2965856993612,2cyclictest0-21swapper/108:15:011
2965856993611,5cyclictest0-21swapper/107:55:011
296585599365,28cyclictest3058134-21ssh10:41:090
2965855993629,4cyclictest1-21systemd09:10:020
2965855993629,4cyclictest0-21swapper/009:46:030
2965855993629,4cyclictest0-21swapper/009:46:030
2965855993628,4cyclictest0-21swapper/011:36:110
2965855993627,5cyclictest3117819-21latency_hist12:20:030
2965855993625,8cyclictest3085220-21missed_timers11:25:590
2965855993615,17cyclictest3064063-21ssh10:51:050
2965855993615,17cyclictest3064063-21ssh10:51:050
2965857993528,4cyclictest3056905-21kworker/u8:12-xprtiod10:41:072
2965857993527,5cyclictest3049425-21tail10:26:162
2965857993526,5cyclictest1000-21mta-sts-daemon11:25:482
296585699355,22cyclictest627-21lldpd08:46:031
2965855993528,4cyclictest3056783-21munin-run10:40:010
2965855993526,6cyclictest3008479-21/usr/sbin/munin09:21:130
2965855993525,7cyclictest3060946-21missed_timers10:46:040
2965855993514,11cyclictest57-21kcompactd009:05:020
2965855993511,20cyclictest3039661-21ntpq10:11:000
2965857993428,3cyclictest0-21swapper/210:35:012
2965857993426,5cyclictest2986881-21systemd-run08:20:012
2965857993425,6cyclictest2970377-21systemd-run07:25:012
2965857993425,6cyclictest2970377-21systemd-run07:25:012
296585799341,27cyclictest3103646-21diskmemload11:56:042
2965856993425,6cyclictest2164266-21cups-browsed08:07:521
296585699341,9cyclictest265-21systemd-journal08:16:081
2965856993411,5cyclictest2977736-21unixbench_multi07:46:141
2965855993427,4cyclictest3044221-21kworker/u8:11-rpciod11:16:080
2965855993426,5cyclictest421-21dbus-daemon11:11:120
2965855993425,6cyclictest420-21avahi-daemon09:55:430
2965855993425,6cyclictest3020241-21munin-plugin-st09:40:010
296585599342,26cyclictest3097710-21ssh11:46:100
2965855993421,10cyclictest3029370-21munin-plugin-st09:55:010
2965855993421,10cyclictest3029370-21munin-plugin-st09:55:010
296585599341,28cyclictest3036910-21awk10:06:130
296585599340,3cyclictest171rcu_preempt07:41:050
2965857993327,4cyclictest3032456-21kworker/u8:0-rpciod10:11:492
2965857993327,3cyclictest0-21swapper/209:36:072
2965857993324,6cyclictest2991367-21munin-run08:35:012
2965857993323,7cyclictest2987672-21needreboot08:21:012
2965857993316,10cyclictest3001854-21diskmemload09:46:082
2965857993316,10cyclictest3001854-21diskmemload09:46:082
296585599338,3cyclictest171rcu_preempt08:51:050
296585599337,4cyclictest171rcu_preempt12:21:100
296585599333,26cyclictest3109757-21sh12:06:040
296585599333,26cyclictest3109757-21sh12:06:040
2965855993327,3cyclictest3072000-21cron11:05:000
2965855993325,5cyclictest3050727-21kworker/u8:2-xprtiod10:34:590
296585599332,25cyclictest2990872-21sshd08:31:100
296585799327,18cyclictest1002-21mta-sts-daemon07:16:212
2965857993226,3cyclictest0-21swapper/210:46:212
2965857993224,5cyclictest2994345-21cron08:45:012
2965857993222,7cyclictest485-21in:imuxsock09:30:012
2965856993224,5cyclictest2982274-21head08:01:151
2965855993228,2cyclictest0-21swapper/008:41:030
2965855993225,4cyclictest3127717-21kernelversion12:36:000
2965855993217,11cyclictest151ktimers/008:06:040
2965857993124,5cyclictest57-21kcompactd007:31:082
2965855993126,3cyclictest0-21swapper/008:00:010
2965855993124,5cyclictest2985261-21sed08:11:150
2965855993123,4cyclictest0-21swapper/012:32:470
2965855993119,10cyclictest0-21swapper/010:01:160
2965857993023,4cyclictest3067013-21needreboot10:56:002
2965857993023,4cyclictest3034167-21unixbench-2d10:01:162
2965857993023,3cyclictest0-21swapper/210:51:112
2965857993023,3cyclictest0-21swapper/210:51:112
2965857993021,4cyclictest0-21swapper/212:06:092
2965857993021,4cyclictest0-21swapper/212:06:092
2965855993025,3cyclictest2979248-21unixbench_multi07:51:140
2965855993023,4cyclictest3104239-21unixbench_singl11:56:160
2965855993022,5cyclictest1000-21mta-sts-daemon11:08:300
2965855993017,6cyclictest2992927-21latency_hist08:40:020
2965857992923,3cyclictest0-21swapper/212:16:172
2965857992921,5cyclictest1003-21mta-sts-daemon10:21:092
296585799291,22cyclictest3073257-21ssh11:06:052
296585599297,17cyclictest428-21mta-sts-daemon07:21:110
296585599297,17cyclictest428-21mta-sts-daemon07:21:110
2965855992925,2cyclictest0-21swapper/007:20:010
2965855992923,4cyclictest3095094-21tail11:41:150
2965855992921,4cyclictest0-21swapper/012:00:530
2965855992916,2cyclictest171rcu_preempt11:51:010
296585599291,23cyclictest3015349-21sh09:31:040
2965857992822,3cyclictest0-21swapper/209:40:562
2965857992822,3cyclictest0-21swapper/209:40:562
2965857992822,3cyclictest0-21swapper/209:06:182
2965857992821,5cyclictest0-21swapper/208:50:012
2965857992821,4cyclictest2988693-21aten_r2power_fr08:25:552
2965857992821,4cyclictest1001-21mta-sts-daemon09:04:452
2965857992821,4cyclictest1000-21mta-sts-daemon09:56:012
2965857992818,8cyclictest2966584-21missed_timers07:10:592
2965857992818,8cyclictest2966584-21missed_timers07:10:592
2965855992819,7cyclictest2971509-21switchtime07:26:180
2965855992818,8cyclictest12650irq/32-eth009:11:040
2965855992818,7cyclictest0-21swapper/012:11:020
2965855992816,10cyclictest1002-21mta-sts-daemon08:56:090
296585899273,7cyclictest0-21swapper/310:46:103
296585799274,15cyclictest421-21dbus-daemon07:26:112
296585799273,18cyclictest3100607-21ssh11:51:042
2965857992720,5cyclictest1000-21mta-sts-daemon10:15:052
2965857992719,5cyclictest2997363-21systemd-run08:55:012
2965857992718,6cyclictest567-21gdbus07:51:102
2965855992721,4cyclictest2988249-21sed08:21:140
2965855992719,5cyclictest265-21systemd-journal08:30:010
2965855992713,3cyclictest0-21swapper/007:10:580
2965855992713,3cyclictest0-21swapper/007:10:580
296585899263,6cyclictest0-21swapper/309:41:013
296585899263,6cyclictest0-21swapper/309:41:013
296585899262,7cyclictest0-21swapper/309:46:023
296585899262,7cyclictest0-21swapper/309:46:023
296585899261,8cyclictest0-21swapper/311:31:043
296585899261,6cyclictest0-21swapper/309:15:543
296585799266,10cyclictest3015313-21ssh09:31:032
2965855992621,3cyclictest0-21swapper/012:26:190
2965855992620,4cyclictest2976702-21aten_r2power_po07:45:530
296585899253,4cyclictest0-21swapper/308:21:053
296585899244,4cyclictest0-21swapper/310:31:053
2965857992419,3cyclictest1003-21mta-sts-daemon08:06:492
2965857992418,3cyclictest0-21swapper/207:45:012
2965857992410,4cyclictest171rcu_preempt08:56:052
296585599246,11cyclictest14-21ksoftirqd/007:36:050
2965855992414,5cyclictest1002-21mta-sts-daemon07:30:580
296585899233,4cyclictest0-21swapper/307:11:043
296585899233,4cyclictest0-21swapper/307:11:043
2965857992317,2cyclictest0-21swapper/207:55:572
2965855992315,2cyclictest171rcu_preempt08:46:030
296585899223,6cyclictest0-21swapper/310:55:553
296585899223,4cyclictest0-21swapper/309:56:063
296585799224,12cyclictest428-21mta-sts-daemon08:11:042
296585899213,5cyclictest0-21swapper/308:41:113
2965856992113,4cyclictest0-21swapper/107:10:031
296585899192,11cyclictest0-21swapper/311:51:103
2965857991913,4cyclictest265-21systemd-journal07:10:022
2965855991915,2cyclictest0-21swapper/007:10:030
296585899182,6cyclictest0-21swapper/307:46:033
296585899181,9cyclictest0-21swapper/312:26:113
296585899181,8cyclictest0-21swapper/307:51:023
296585899173,6cyclictest0-21swapper/312:21:093
296585899173,6cyclictest0-21swapper/310:41:083
296585899172,9cyclictest0-21swapper/310:11:463
296585899172,7cyclictest0-21swapper/311:11:053
296585899172,7cyclictest0-21swapper/311:06:013
296585899171,7cyclictest0-21swapper/309:11:033
296585899164,6cyclictest0-21swapper/310:36:133
296585899163,7cyclictest0-21swapper/311:16:133
296585899163,6cyclictest0-21swapper/310:16:083
296585899162,9cyclictest0-21swapper/307:36:063
296585899162,7cyclictest0-21swapper/312:20:023
296585899162,7cyclictest0-21swapper/310:06:043
296585899161,7cyclictest0-21swapper/308:46:073
296585899152,8cyclictest0-21swapper/309:30:593
296585899152,7cyclictest0-21swapper/311:26:033
296585899152,7cyclictest0-21swapper/311:21:053
296585899152,7cyclictest0-21swapper/309:36:073
296585899151,8cyclictest0-21swapper/309:26:063
296585899151,7cyclictest0-21swapper/311:46:033
296585899151,6cyclictest0-21swapper/308:31:043
296585899142,7cyclictest0-21swapper/311:36:073
296585899142,7cyclictest0-21swapper/307:55:583
296585899142,6cyclictest0-21swapper/310:25:573
296585899142,5cyclictest0-21swapper/308:36:083
296585899141,8cyclictest0-21swapper/310:21:073
296585899134,6cyclictest0-21swapper/310:01:033
296585899133,6cyclictest0-21swapper/309:51:173
296585899133,6cyclictest0-21swapper/309:51:173
296585899133,6cyclictest0-21swapper/308:10:023
296585899132,7cyclictest0-21swapper/311:01:043
296585899132,7cyclictest0-21swapper/308:15:013
296585899132,6cyclictest0-21swapper/309:20:573
296585899131,7cyclictest0-21swapper/311:56:063
296585899131,7cyclictest0-21swapper/307:21:053
296585899131,7cyclictest0-21swapper/307:21:053
296585899131,6cyclictest0-21swapper/312:36:053
296585899131,5cyclictest0-21swapper/309:01:063
296585899125,4cyclictest0-21swapper/307:16:233
296585899123,6cyclictest0-21swapper/307:41:023
296585899122,6cyclictest0-21swapper/312:01:013
296585899122,6cyclictest0-21swapper/310:51:013
296585899122,6cyclictest0-21swapper/310:51:013
296585899122,6cyclictest0-21swapper/308:01:033
296585899121,6cyclictest0-21swapper/311:45:023
296585899121,5cyclictest0-21swapper/309:05:553
296585899113,5cyclictest0-21swapper/312:30:593
296585899112,6cyclictest0-21swapper/312:10:553
296585899112,6cyclictest0-21swapper/312:06:033
296585899112,6cyclictest0-21swapper/312:06:033
296585899112,5cyclictest0-21swapper/308:16:023
296585899111,6cyclictest0-21swapper/307:30:543
296585899103,5cyclictest0-21swapper/308:56:023
296585899101,6cyclictest0-21swapper/307:10:033
296585899101,5cyclictest0-21swapper/308:50:583
296585899101,4cyclictest0-21swapper/307:30:023
29658589993,4cyclictest0-21swapper/308:26:043
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional