You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-26 - 12:00

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack2slot7s.osadl.org (updated Wed Nov 26, 2025 00:39:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2041027995731,8cyclictest2197394-21aten2.4-expect23:59:032
2041025995742,11cyclictest0-21swapper/023:40:010
2041025994736,7cyclictest422-21dbus-daemon21:20:020
2041027994636,7cyclictest422-21dbus-daemon23:45:012
2041025994632,11cyclictest2213446-21ssh00:20:010
2041027994534,7cyclictest956-21mta-sts-daemon21:29:142
2041025994531,10cyclictest421-21avahi-daemon21:55:360
2041025994516,22cyclictest2162367-21ssh23:09:150
2041026994436,5cyclictest2206477-21ssh00:10:011
2041025994434,7cyclictest422-21dbus-daemon23:04:270
2041026994335,5cyclictest453-21systemd-logind23:20:011
2041025994333,6cyclictest421-21avahi-daemon23:20:340
2041025994332,8cyclictest2199219-21latency_hist00:00:030
2041025994315,23cyclictest2126511-21sh22:19:130
204102699422,33cyclictest2144502-21ssh22:44:261
2041025994234,5cyclictest2071025-21systemd-run20:50:010
2041025994234,5cyclictest2071025-21systemd-run20:50:010
2041025994233,6cyclictest421-21avahi-daemon23:10:450
2041027994133,5cyclictest2137050-21missed_timers22:34:102
2041027994131,8cyclictest2050006-21munin-run19:40:012
2041026994134,5cyclictest2059016-21munin-plugin-st20:10:011
2041026994133,5cyclictest422-21dbus-daemon19:45:011
2041025994132,4cyclictest0-21swapper/021:54:180
2041025994125,13cyclictest2032753-21kworker/u8:6+events_unbound19:39:150
2041027994032,5cyclictest2213443-21cron00:20:012
2041027994027,9cyclictest453-21systemd-logind20:25:012
2041026994032,5cyclictest422-21dbus-daemon23:50:021
2041026994032,5cyclictest2090610-21fschecks_count21:29:061
2041025994026,8cyclictest12450irq/32-eth022:34:150
2041027993931,5cyclictest2216577-21unixbench_multi00:24:292
2041027993931,5cyclictest2178031-21kworker/u8:9+xprtiod23:50:012
2041027993930,6cyclictest2067994-21cron20:40:012
2041026993932,4cyclictest2074023-21sendmail21:00:011
2041026993932,4cyclictest2062055-21systemd20:20:021
2041026993931,5cyclictest2127464-21grep22:20:011
2041026993930,6cyclictest2079976-21missed_timers21:14:111
2041025993931,5cyclictest2188197-21systemd-run23:45:010
2041025993931,5cyclictest2120062-21grep22:09:280
2041025993931,5cyclictest2120062-21grep22:09:280
2041025993931,5cyclictest2044044-21sendmail-msp19:20:010
2041025993927,9cyclictest1-21systemd23:35:010
2041025993925,12cyclictest2060579-21latency_hist20:15:020
2041027993830,5cyclictest2134538-21systemd-run22:30:012
2041027993830,5cyclictest2097867-21kernelversion21:39:162
2041027993830,5cyclictest2097867-21kernelversion21:39:162
2041027993830,5cyclictest2046979-21unixbench_singl19:29:272
2041026993830,5cyclictest1-21systemd00:25:021
2041025993831,4cyclictest2099000-21grep21:40:010
2041025993831,4cyclictest2099000-21grep21:40:010
2041025993831,4cyclictest2056023-21idleruntime-cro20:00:000
2041025993830,5cyclictest2156162-21ssh23:00:010
2041025993829,6cyclictest2047039-21systemd-run19:30:020
2041025993818,12cyclictest151ktimers/020:44:120
2041027993729,5cyclictest2156216-21ssh23:00:012
2041027993727,7cyclictest2041033-21sh19:10:012
2041027993726,7cyclictest421-21avahi-daemon00:09:042
2041026993730,4cyclictest2057554-21latency_hist20:05:011
2041026993730,3cyclictest0-21swapper/100:15:011
2041026993729,5cyclictest2095270-21sed21:34:321
2041026993728,6cyclictest2054529-21systemd-run19:55:011
2041026993728,6cyclictest2054529-21systemd-run19:55:011
2041026993728,5cyclictest0-21swapper/122:40:011
204102699372,27cyclictest2194316-21ssh23:54:111
204102699371,26cyclictest2148008-21rm22:49:141
2041025993727,6cyclictest2123892-21munin-plugin-st22:15:010
2041025993727,6cyclictest2123892-21munin-plugin-st22:15:010
2041027993628,5cyclictest2226612-21latency_hist00:40:022
2041027993628,5cyclictest2123895-21munin-plugin-st22:15:002
2041027993628,5cyclictest2123895-21munin-plugin-st22:15:002
2041027993626,7cyclictest1-21systemd22:50:022
2041027993626,5cyclictest0-21swapper/223:29:192
2041026993629,4cyclictest955-21mta-sts-daemon00:25:531
2041026993628,5cyclictest2198996-21ssh00:00:011
2041026993627,6cyclictest2186925-21fschecks_time23:44:141
2041026993624,8cyclictest2108590-21missed_timers21:54:121
204102599362,26cyclictest2205490-21sh00:09:140
2041025993613,15cyclictest12450irq/32-eth000:04:250
2041027993529,3cyclictest0-21swapper/223:35:012
2041027993527,5cyclictest2201453-21fschecks_count00:04:132
2041027993527,5cyclictest2115439-21fschecks_time22:04:082
2041027993527,5cyclictest2115439-21fschecks_time22:04:082
2041027993526,6cyclictest421-21avahi-daemon21:16:242
2041027993525,6cyclictest2077041-21systemd-run21:10:022
2041026993529,3cyclictest2038488-21kworker/u8:3-xprtiod19:20:011
2041026993527,5cyclictest2184306-21unixbench_multi23:39:271
2041026993527,5cyclictest2123585-21grep22:14:291
2041026993527,5cyclictest2123585-21grep22:14:291
2041026993527,5cyclictest2060503-21cron20:15:011
2041026993526,6cyclictest2225138-21aten_r2power_fr00:39:051
204102699352,28cyclictest2212578-21ssh00:19:161
2041025993526,6cyclictest186163-21cups-browsed22:23:000
2041025993525,7cyclictest0-21swapper/021:44:090
2041025993524,4cyclictest2105401-21proc_pri21:49:220
2041025993521,8cyclictest12450irq/32-eth021:14:180
204102799345,23cyclictest2222588-21ssh00:34:212
2041027993427,5cyclictest0-21swapper/223:20:012
2041027993426,5cyclictest2045470-21grep19:24:252
2041026993425,7cyclictest2180223-21ntp_kernel_pll_23:34:151
2041026993425,6cyclictest2130977-21cron22:25:011
2041026993425,6cyclictest2047019-21munin-plugin-st19:30:011
2041025993428,4cyclictest2215641-21fschecks_time00:24:100
204102599342,26cyclictest2065034-21kworker/u8:14+rpciod22:29:200
204102799335,18cyclictest437-21mta-sts-daemon22:54:112
2041027993326,4cyclictest2145221-21sh22:45:012
2041027993324,5cyclictest2069538-21systemd-run20:45:012
2041027993323,6cyclictest186163-21cups-browsed22:05:152
2041027993323,6cyclictest186163-21cups-browsed22:05:152
2041026993325,5cyclictest2105851-21unixbench_singl21:49:281
2041026993325,5cyclictest186161-21cupsd22:29:111
2041026993323,7cyclictest2177201-21timerwakeupswit23:29:281
2041025993327,4cyclictest2144362-21ntp_kernel_pll_22:44:200
204102599332,25cyclictest2222587-21sh00:34:200
2041025993318,12cyclictest2219768-21unixbench_multi00:29:270
2041025993314,14cyclictest2051538-21kworker/u8:13+xprtiod20:04:210
204102599331,26cyclictest2115929-21ssh22:04:180
204102599331,26cyclictest2115929-21ssh22:04:180
2041027993226,3cyclictest0-21swapper/222:20:012
2041027993224,5cyclictest422-21dbus-daemon21:04:182
2041027993224,5cyclictest2209941-21munin-run00:15:012
2041027993224,5cyclictest186163-21cups-browsed22:35:372
2041027993224,4cyclictest2101018-21aten_r2power_fr21:44:052
2041027993223,6cyclictest491-21gdbus19:49:222
204102799321,25cyclictest2077014-21diskmemload23:24:232
2041026993227,2cyclictest0-21swapper/123:00:011
2041026993224,5cyclictest2172709-21fschecks_time23:24:091
2041026993224,5cyclictest2050041-21systemd-run19:40:011
2041026993223,6cyclictest421-21avahi-daemon21:04:231
2041026993222,6cyclictest186163-21cups-browsed21:15:401
2041025993223,6cyclictest2064940-21tail20:29:250
2041025993221,8cyclictest12450irq/32-eth021:24:140
204102799316,14cyclictest437-21mta-sts-daemon20:09:182
2041027993125,3cyclictest0-21swapper/221:34:202
2041027993123,5cyclictest266-21systemd-journal23:40:012
2041027993122,6cyclictest421-21avahi-daemon23:14:052
2041027993122,6cyclictest2161984-21fschecks_count23:09:092
2041027993118,10cyclictest2106140-21/usr/sbin/munin21:50:022
2041026993124,4cyclictest2138100-21munin-run22:35:011
2041026993124,4cyclictest2113167-21munin-run22:00:011
2041026993124,4cyclictest2063674-21latency_hist20:25:031
2041026993123,5cyclictest2202735-21cron00:05:011
2041026993123,5cyclictest2064566-21proc_pri20:29:201
204102599319,6cyclictest171rcu_preempt20:19:190
2041025993123,3cyclictest0-21swapper/021:29:140
2041025993117,3cyclictest171rcu_preempt19:24:140
2041025993115,8cyclictest12450irq/32-eth021:34:240
2041027993025,2cyclictest0-21swapper/222:25:012
2041027993024,3cyclictest0-21swapper/219:55:012
2041027993024,3cyclictest0-21swapper/219:55:012
2041027993022,5cyclictest955-21mta-sts-daemon20:33:182
2041027993022,5cyclictest2061999-21cron20:20:012
2041027993022,3cyclictest0-21swapper/223:52:582
2041027993018,8cyclictest0-21swapper/221:54:152
2041026993025,3cyclictest0-21swapper/120:00:011
2041026993022,5cyclictest491-21gdbus22:09:241
2041026993022,5cyclictest491-21gdbus22:09:241
2041026993022,5cyclictest2066536-21systemd20:35:021
2041026993022,5cyclictest186163-21cups-browsed21:21:231
2041026993022,4cyclictest2053166-21latency_hist19:50:021
2041026993019,8cyclictest2162487-21ntp_offset23:09:201
204102599302,6cyclictest2209171-21ssh00:14:200
2041025993024,4cyclictest956-21mta-sts-daemon22:49:050
2041025993024,3cyclictest0-21swapper/022:39:070
2041025993018,6cyclictest12450irq/32-eth023:54:150
2041025993012,10cyclictest12450irq/32-eth023:19:210
2041027992924,2cyclictest0-21swapper/219:20:012
2041027992920,6cyclictest186163-21cups-browsed21:55:362
2041027992915,9cyclictest2219275-21sh00:29:182
204102799291,24cyclictest2080191-21sh21:14:172
2041026992924,2cyclictest0-21swapper/120:40:011
204102699292,20cyclictest2222530-21ssh00:34:181
2041026992921,5cyclictest491-21gdbus21:39:581
2041026992921,5cyclictest491-21gdbus21:39:581
2041026992920,6cyclictest2045473-21unixbench_singl19:24:251
2041025992922,5cyclictest2072361-21unixbench-2d20:54:250
2041025992914,12cyclictest0-21swapper/000:40:010
2041027992821,3cyclictest0-21swapper/223:04:152
2041026992821,4cyclictest956-21mta-sts-daemon21:41:081
2041026992821,4cyclictest955-21mta-sts-daemon19:14:261
2041026992819,6cyclictest186163-21cups-browsed19:34:051
2041026992819,3cyclictest955-21mta-sts-daemon21:05:481
204102599283,19cyclictest2151708-21ssh22:54:150
2041025992822,4cyclictest2191742-21systemd-run23:50:010
2041025992822,3cyclictest0-21swapper/020:24:070
2041025992820,4cyclictest0-21swapper/021:04:180
2041025992818,6cyclictest0-21swapper/020:40:010
2041025992816,8cyclictest12450irq/32-eth021:10:010
204102599281,5cyclictest0-21swapper/023:29:190
2041025992812,13cyclictest2074051-21latency_hist21:00:020
2041027992721,3cyclictest2044257-21kworker/u8:8-xprtiod19:44:102
2041027992719,5cyclictest2042511-21munin-plugin-st19:15:012
2041026992721,3cyclictest0-21swapper/122:54:231
2041026992717,8cyclictest2159385-21unixbench-2d23:04:281
2041025992717,6cyclictest0-21swapper/019:54:150
2041025992717,6cyclictest0-21swapper/019:54:150
2041025992716,8cyclictest2050940-21ntp_kernel_pll_19:44:150
2041027992620,3cyclictest0-21swapper/221:24:232
204102799261,8cyclictest2048638-21kworker/u8:7+xprtiod20:05:002
2041027992617,7cyclictest2073440-21awk20:59:152
2041026992620,3cyclictest0-21swapper/120:50:011
2041026992620,3cyclictest0-21swapper/120:50:011
2041026992619,4cyclictest0-21swapper/122:05:011
2041026992619,4cyclictest0-21swapper/122:05:011
204102599262,4cyclictest171rcu_preempt20:09:180
204102799259,5cyclictest59950lldpd20:13:212
2041025992518,5cyclictest2048533-21unin-run19:35:020
2041027992417,3cyclictest0-21swapper/219:59:122
204102699245,4cyclictest2077014-21diskmemload23:10:211
2041026992417,4cyclictest2072508-21idleruntime-cro20:55:001
204102599241,20cyclictest2066396-21unixbench_multi20:34:250
2041025992411,10cyclictest2041543-21fschecks_count19:14:080
2041027992318,2cyclictest0-21swapper/220:49:042
2041027992318,2cyclictest0-21swapper/220:49:042
2041026992310,10cyclictest957-21mta-sts-daemon20:44:161
2041025992215,4cyclictest0-21swapper/019:49:030
204102799213,12cyclictest959-21mta-sts-daemon19:34:222
204102799212,13cyclictest2064512-21ntp_states20:29:192
2041027992012,5cyclictest2071888-21ntpq20:54:132
204102899192,8cyclictest0-21swapper/319:19:153
204102899192,10cyclictest0-21swapper/323:44:263
2041026991910,6cyclictest2040975-21latency_hist19:10:011
204102899183,9cyclictest0-21swapper/300:29:133
204102899182,9cyclictest0-21swapper/323:34:163
204102899182,8cyclictest0-21swapper/322:19:143
204102899182,7cyclictest0-21swapper/320:59:163
204102899182,11cyclictest0-21swapper/300:04:193
204102899181,10cyclictest0-21swapper/300:24:153
204102599181,14cyclictest2041108-21latency_hist19:10:020
204102899174,7cyclictest0-21swapper/322:34:183
204102899173,8cyclictest0-21swapper/322:24:193
204102899173,8cyclictest0-21swapper/321:44:193
204102899173,7cyclictest0-21swapper/323:39:133
204102899173,7cyclictest0-21swapper/323:09:163
204102899172,9cyclictest0-21swapper/322:14:073
204102899172,9cyclictest0-21swapper/322:14:073
204102899172,9cyclictest0-21swapper/321:54:153
204102899172,8cyclictest0-21swapper/320:24:123
204102899172,8cyclictest0-21swapper/319:59:123
204102899171,10cyclictest0-21swapper/323:24:203
204102899171,10cyclictest0-21swapper/321:24:103
204102899171,10cyclictest0-21swapper/300:09:133
204102899166,7cyclictest0-21swapper/321:34:263
204102899163,8cyclictest0-21swapper/322:44:253
204102899163,8cyclictest0-21swapper/321:49:163
204102899163,8cyclictest0-21swapper/300:34:133
204102899162,9cyclictest0-21swapper/323:59:143
204102899162,9cyclictest0-21swapper/323:19:173
204102899162,9cyclictest0-21swapper/323:04:153
204102899162,9cyclictest0-21swapper/322:59:143
204102899162,9cyclictest0-21swapper/322:39:133
204102899162,8cyclictest0-21swapper/322:29:143
204102899162,8cyclictest0-21swapper/300:19:163
204102899162,10cyclictest0-21swapper/322:49:163
204102899161,9cyclictest0-21swapper/321:39:263
204102899161,9cyclictest0-21swapper/321:39:263
204102899161,7cyclictest0-21swapper/320:54:163
204102899154,8cyclictest0-21swapper/321:14:173
204102899154,6cyclictest0-21swapper/323:49:143
204102899153,8cyclictest0-21swapper/321:19:193
204102899152,8cyclictest0-21swapper/323:29:163
204102899152,7cyclictest0-21swapper/322:50:333
204102899152,6cyclictest0-21swapper/319:44:123
204102899151,9cyclictest0-21swapper/321:59:123
204102899151,8cyclictest0-21swapper/322:09:183
204102899151,8cyclictest0-21swapper/322:09:183
204102899151,8cyclictest0-21swapper/321:29:143
204102899151,7cyclictest0-21swapper/320:39:173
204102899151,7cyclictest0-21swapper/319:34:203
204102899151,7cyclictest0-21swapper/319:14:203
204102899151,7cyclictest0-21swapper/300:39:053
204102899151,6cyclictest0-21swapper/319:54:133
204102899151,6cyclictest0-21swapper/319:54:133
204102899143,7cyclictest0-21swapper/323:54:123
204102899143,6cyclictest0-21swapper/319:29:183
204102899142,7cyclictest0-21swapper/323:14:033
204102899142,7cyclictest0-21swapper/300:14:093
204102899141,9cyclictest0-21swapper/322:02:343
204102899141,9cyclictest0-21swapper/322:02:343
204102899141,7cyclictest0-21swapper/320:29:123
204102899141,7cyclictest0-21swapper/320:09:183
204102899134,6cyclictest0-21swapper/320:19:163
204102899133,6cyclictest0-21swapper/320:50:013
204102899133,6cyclictest0-21swapper/320:50:013
204102899133,4cyclictest0-21swapper/319:24:123
204102899132,7cyclictest0-21swapper/319:49:133
204102899132,6cyclictest0-21swapper/320:05:003
204102899131,6cyclictest0-21swapper/321:04:133
204102899123,5cyclictest0-21swapper/321:09:163
204102899121,6cyclictest0-21swapper/320:34:133
204102899121,6cyclictest0-21swapper/320:10:323
204102899111,7cyclictest0-21swapper/319:38:323
204102899111,6cyclictest0-21swapper/320:44:133
204102899111,4cyclictest0-21swapper/319:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional