You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-08 - 17:00

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Mon Dec 08, 2025 12:39:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2013364995347,4cyclictest2175794-21systemd-run12:20:010
2013364995343,8cyclictest2190513-21cron12:40:000
2013366995134,13cyclictest2061964-21sh09:29:282
2013365995039,8cyclictest2108523-21sendmail_mailqu10:39:311
2013364994741,4cyclictest0-21swapper/008:00:010
2013366994637,6cyclictest1-21systemd10:20:012
2013365994637,6cyclictest2167988-21proc_pri12:09:341
2013366994537,5cyclictest2028336-21munin-plugin-st08:00:012
2013364994525,4cyclictest171rcu_preempt12:34:220
2013364994525,4cyclictest171rcu_preempt12:34:220
2013366994436,5cyclictest2056522-21munin-plugin-st09:20:012
2013366994436,5cyclictest2035870-21systemd-run08:25:022
2013365994433,7cyclictest1-21systemd08:50:021
2013364994437,4cyclictest2135104-21tail11:19:350
2013364994435,6cyclictest1196112-21cups-browsed12:09:250
2013364994335,5cyclictest421-21avahi-daemon10:33:060
2013364994335,5cyclictest2161747-21ssh11:59:370
2013364994335,5cyclictest2161747-21ssh11:59:370
2013364994334,5cyclictest151ktimers/012:24:270
2013364994334,5cyclictest151ktimers/012:24:270
2013365994236,3cyclictest0-21swapper/111:20:011
2013365994233,6cyclictest1-21systemd10:45:011
2013364994236,4cyclictest2115100-21needreboot10:49:220
2013365994132,6cyclictest491-21gdbus09:20:011
2013365994131,7cyclictest2175794-21systemd-run12:20:011
2013364994135,4cyclictest2145172-21head11:34:340
2013364994132,5cyclictest2077464-21tail09:54:110
2013364994125,10cyclictest2082232-21sendmail10:00:010
2013366994033,4cyclictest2105719-21latency_hist10:35:022
2013366994032,5cyclictest422-21dbus-daemon10:45:012
2013366994029,8cyclictest2101071-21kernelversion10:29:222
2013365994033,4cyclictest2155629-21kworker/u8:1-rpciod11:59:381
2013365994033,4cyclictest2155629-21kworker/u8:1-rpciod11:59:381
2013365994032,5cyclictest1-21systemd10:25:021
2013365994031,6cyclictest2022346-21idleruntime-cro07:40:011
2013364994033,4cyclictest12550irq/33-eth011:49:250
2013364994033,4cyclictest12550irq/33-eth011:49:250
2013364994032,5cyclictest421-21avahi-daemon10:18:090
2013364994028,9cyclictest0-21swapper/010:55:010
2013364994011,22cyclictest2068470-21ssh09:39:230
2013366993932,4cyclictest0-21swapper/212:19:322
2013366993932,4cyclictest0-21swapper/212:00:012
2013366993932,4cyclictest0-21swapper/212:00:012
2013366993931,5cyclictest2016346-21idleruntime-cro07:20:012
2013366993931,5cyclictest2016346-21idleruntime-cro07:20:012
2013366993929,6cyclictest2029873-21systemd-run08:05:012
2013366993929,6cyclictest2029873-21systemd-run08:05:012
2013365993932,3cyclictest0-21swapper/112:15:011
201336599392,26cyclictest2078158-21ssh09:54:251
2013364993931,5cyclictest2126013-21ssh11:05:010
2013366993826,9cyclictest2052232-21kernelversion09:14:172
2013365993831,4cyclictest2122721-21systemd11:00:021
2013365993830,5cyclictest2102165-21ssh10:30:011
2013365993829,6cyclictest2065187-21ntp_kernel_pll_09:34:321
2013364993832,4cyclictest2016354-21cut07:20:010
2013364993832,4cyclictest2016354-21cut07:20:010
2013364993831,5cyclictest0-21swapper/009:15:020
2013366993731,3cyclictest0-21swapper/209:50:012
2013366993730,2cyclictest0-21swapper/212:29:152
2013366993730,2cyclictest0-21swapper/212:29:152
2013366993729,5cyclictest2082253-21sendmail10:00:012
2013366993728,5cyclictest2128997-21systemd-run11:10:012
2013365993728,6cyclictest266-21systemd-journal08:45:011
2013365993727,6cyclictest2074602-21ntp_kernel_err09:49:211
2013365993725,8cyclictest2035183-21needreboot08:24:211
2013365993720,9cyclictest2049392-21diskmemload10:19:281
2013364993731,3cyclictest957-21mta-sts-daemon11:29:190
2013364993729,5cyclictest2092418-21systemd-run10:15:010
2013364993728,7cyclictest2128970-21latency_hist11:10:020
2013366993630,3cyclictest0-21swapper/210:54:252
2013366993629,3cyclictest0-21swapper/210:40:012
2013365993628,5cyclictest2082239-21idleruntime-cro10:00:011
2013365993628,5cyclictest2013370-21true07:10:011
2013365993628,4cyclictest0-21swapper/111:44:251
2013365993628,4cyclictest0-21swapper/111:44:251
2013364993630,3cyclictest0-21swapper/011:40:020
2013364993630,3cyclictest0-21swapper/011:40:020
2013364993629,4cyclictest957-21mta-sts-daemon09:29:460
2013364993626,7cyclictest2029647-21timerwakeupswit08:04:320
2013364993626,7cyclictest2029647-21timerwakeupswit08:04:320
2013364993626,2cyclictest0-21swapper/012:04:280
2013364993624,8cyclictest1196112-21cups-browsed09:34:510
2013364993623,10cyclictest0-21swapper/010:40:010
2013366993524,8cyclictest2040386-21munin-plugin-st08:40:012
201336599354,21cyclictest2128294-21ssh11:09:271
2013365993528,3cyclictest0-21swapper/112:39:251
2013365993527,5cyclictest2071907-21unixbench_singl09:44:351
2013365993527,5cyclictest2014998-21latency_hist07:15:021
2013365993527,5cyclictest2014998-21latency_hist07:15:021
2013365993527,5cyclictest0-21swapper/108:40:011
2013365993526,6cyclictest2085667-21latency_hist10:05:021
201336599352,26cyclictest2068496-21ssh09:39:241
2013364993529,2cyclictest0-21swapper/009:10:010
2013364993527,5cyclictest1196112-21cups-browsed10:09:140
2013364993527,5cyclictest1196112-21cups-browsed09:20:460
2013366993428,3cyclictest2172131-21kworker/u8:2-xprtiod12:22:102
2013366993428,3cyclictest2172131-21kworker/u8:2-xprtiod12:22:102
2013366993427,4cyclictest2168329-21unixbench-2d12:09:392
2013366993426,5cyclictest2138154-21mailstats11:24:332
2013366993426,5cyclictest2031357-21systemd-run08:10:012
2013366993425,7cyclictest2065089-21needreboot09:34:292
2013366993425,6cyclictest1196112-21cups-browsed10:23:082
2013366993425,5cyclictest2026836-21sh07:55:012
2013365993426,5cyclictest2119380-21idleruntime-cro10:55:011
2013365993426,3cyclictest0-21swapper/107:49:271
2013365993425,6cyclictest2183057-21munin-plugin-st12:30:001
2013365993425,6cyclictest2183057-21munin-plugin-st12:30:001
2013365993423,8cyclictest2148391-21switchtime11:39:331
2013365993423,8cyclictest2148391-21switchtime11:39:331
201336499347,21cyclictest2137865-21ssh11:24:220
2013364993428,3cyclictest956-21mta-sts-daemon10:44:220
2013364993425,7cyclictest2030761-21ntp_kernel_pll_08:09:230
2013366993325,5cyclictest2152012-21unixbench_singl11:44:362
2013366993325,5cyclictest2152012-21unixbench_singl11:44:362
2013366993325,5cyclictest2049392-21diskmemload11:40:022
2013366993325,5cyclictest2049392-21diskmemload11:40:022
2013366993324,6cyclictest453-21systemd-logind12:15:012
2013366993324,6cyclictest2190298-21unixbench_singl12:39:362
2013366993322,7cyclictest421-21avahi-daemon10:56:062
201336699332,23cyclictest2088204-21ssh10:09:242
201336699331,26cyclictest2071263-21ssh09:44:222
2013365993327,3cyclictest0-21swapper/108:10:011
2013365993325,5cyclictest2154050-21aten_r2power_cu11:49:111
2013365993325,5cyclictest2154050-21aten_r2power_cu11:49:111
2013365993325,5cyclictest2034349-21systemctl08:20:011
2013365993321,6cyclictest1196112-21cups-browsed09:04:271
201336499334,16cyclictest2055723-21ssh09:19:220
2013364993326,4cyclictest2183044-21cron12:30:000
2013364993326,4cyclictest2183044-21cron12:30:000
2013364993326,4cyclictest2098357-21cat10:24:350
2013364993324,7cyclictest2131897-21unixbench_singl11:14:330
2013364993324,7cyclictest2042809-21ntp_kernel_pll_08:49:240
2013364993323,7cyclictest2018718-21ntp_kernel_err07:29:200
2013364993320,3cyclictest171rcu_preempt08:15:020
2013366993226,2cyclictest0-21swapper/209:10:022
2013366993225,4cyclictest2032842-21munin-plugin-st08:15:012
2013365993227,3cyclictest0-21swapper/109:30:011
2013365993226,3cyclictest0-21swapper/109:10:021
2013365993225,4cyclictest2044845-21unixbench_singl08:54:341
2013365993225,4cyclictest0-21swapper/112:34:241
2013365993225,4cyclictest0-21swapper/112:34:241
2013365993224,5cyclictest2105302-21/usr/sbin/munin10:34:351
2013365993224,4cyclictest957-21mta-sts-daemon09:13:551
2013364993227,2cyclictest0-21swapper/007:35:010
2013364993225,5cyclictest266-21systemd-journal08:45:010
2013364993225,4cyclictest2085650-21systemd-run10:05:010
2013364993218,11cyclictest0-21swapper/009:44:180
201336499320,4cyclictest171rcu_preempt11:54:260
201336499320,4cyclictest171rcu_preempt11:54:260
2013366993125,3cyclictest0-21swapper/211:04:322
2013366993124,4cyclictest957-21mta-sts-daemon07:10:012
2013366993124,4cyclictest2115703-21unixbench-2d10:49:362
2013366993124,4cyclictest0-21swapper/210:14:242
2013366993124,3cyclictest0-21swapper/212:33:072
2013366993124,3cyclictest0-21swapper/212:33:072
2013366993124,3cyclictest0-21swapper/208:54:282
201336599313,22cyclictest2058969-21ssh09:24:241
2013365993125,4cyclictest2013363-21cyclictest11:04:321
2013365993124,4cyclictest956-21mta-sts-daemon11:50:461
2013365993124,4cyclictest956-21mta-sts-daemon11:50:461
2013365993123,5cyclictest421-21avahi-daemon10:09:141
201336499318,18cyclictest2121859-21ssh10:59:210
201336499317,5cyclictest0-21swapper/007:44:200
2013364993125,3cyclictest0-21swapper/009:05:010
2013364993124,4cyclictest957-21mta-sts-daemon10:26:420
2013366993024,3cyclictest0-21swapper/211:30:012
2013366993024,3cyclictest0-21swapper/208:20:012
2013366993023,4cyclictest2037398-21munin-run08:30:012
2013366993023,3cyclictest0-21swapper/209:54:272
2013366993022,5cyclictest956-21mta-sts-daemon10:04:292
2013366993022,5cyclictest2038427-21proc_pri08:34:292
2013365993024,3cyclictest0-21swapper/107:35:011
2013365993022,5cyclictest2164100-21ssh12:04:141
2013365993022,5cyclictest2091913-21switchtime10:14:351
2013365993020,7cyclictest698-21cron12:25:001
2013365993020,7cyclictest698-21cron12:25:001
2013364993024,3cyclictest2075589-21latency_hist09:50:020
2013364993024,3cyclictest0-21swapper/012:11:490
2013364993021,6cyclictest2036434-21sort08:29:120
2013364993019,8cyclictest2038878-21cron08:35:010
2013364993019,8cyclictest2014731-21unixbench_multi07:14:340
2013364993019,8cyclictest2014731-21unixbench_multi07:14:340
2013364993017,10cyclictest0-21swapper/008:54:330
201336499301,24cyclictest2151482-21sh11:44:280
201336499301,24cyclictest2151482-21sh11:44:280
201336699293,21cyclictest2134526-21ssh11:19:232
2013366992921,6cyclictest2068295-21missed_timers09:39:192
2013366992921,5cyclictest421-21avahi-daemon07:44:302
2013366992921,5cyclictest2165353-21systemd-run12:05:012
2013366992921,5cyclictest2022040-21switchtime07:39:332
2013366992920,6cyclictest2043385-21sh08:50:012
2013366992917,9cyclictest2017192-21needreboot07:24:182
2013365992921,5cyclictest2027397-21gltestperf07:59:171
2013365992919,8cyclictest2029866-21sh08:05:011
2013365992919,8cyclictest2029866-21sh08:05:011
201336599291,22cyclictest2141242-21ssh11:29:221
2013364992923,3cyclictest0-21swapper/008:20:010
2013364992922,4cyclictest2017867-21munin-plugin-st07:25:010
2013366992821,4cyclictest2131314-21ntpq11:14:222
2013366992821,3cyclictest0-21swapper/211:49:132
2013366992821,3cyclictest0-21swapper/211:49:132
2013366992820,5cyclictest2047893-21cron09:05:012
2013366992820,5cyclictest1196112-21cups-browsed07:47:042
2013365992821,4cyclictest956-21mta-sts-daemon07:24:231
2013365992819,6cyclictest2026830-21cron07:55:011
2013365992819,5cyclictest2130934-21gltestperf11:14:151
2013364992821,5cyclictest1196112-21cups-browsed08:24:260
2013366992722,2cyclictest0-21swapper/209:00:012
2013366992720,3cyclictest0-21swapper/211:34:202
2013366992717,7cyclictest2041889-21cron08:45:012
2013365992718,7cyclictest2016144-21timerwakeupswit07:19:301
2013365992718,7cyclictest2016144-21timerwakeupswit07:19:301
2013365992718,6cyclictest959-21mta-sts-daemon10:47:221
2013365992717,7cyclictest1196088-21cupsd11:34:411
201336499274,4cyclictest171rcu_preempt08:39:290
201336499272,21cyclictest1196112-21cups-browsed07:49:280
2013364992715,7cyclictest266-21systemd-journal07:40:010
2013366992620,3cyclictest0-21swapper/211:54:122
2013366992620,3cyclictest0-21swapper/211:54:122
2013366992620,3cyclictest0-21swapper/209:23:022
201336599267,9cyclictest171rcu_preempt11:24:311
2013365992620,3cyclictest0-21swapper/108:29:301
2013365992617,6cyclictest2032779-21sed08:14:331
201336499262,4cyclictest171rcu_preempt08:59:260
2013365992520,2cyclictest0-21swapper/109:00:021
2013365992517,5cyclictest2018145-21tail07:29:111
201336499258,7cyclictest453-21systemd-logind07:10:010
2013364992516,6cyclictest2025883-21fschecks_time07:54:170
201336599247,4cyclictest171rcu_preempt08:34:281
2013366992317,3cyclictest957-21mta-sts-daemon07:14:182
2013366992317,3cyclictest957-21mta-sts-daemon07:14:182
2013366992317,3cyclictest0-21swapper/207:30:012
2013366992216,3cyclictest0-21swapper/207:34:232
201336599202,13cyclictest2023282-21ntpq07:44:221
201336799193,9cyclictest0-21swapper/308:04:193
201336799193,9cyclictest0-21swapper/308:04:193
201336799192,10cyclictest0-21swapper/312:09:253
201336799181,9cyclictest0-21swapper/309:44:283
201336799181,10cyclictest0-21swapper/310:49:303
201336799174,9cyclictest0-21swapper/310:34:223
201336799172,9cyclictest0-21swapper/310:09:233
201336799172,9cyclictest0-21swapper/309:14:233
201336799172,8cyclictest0-21swapper/309:19:303
201336799172,11cyclictest0-21swapper/307:14:253
201336799172,11cyclictest0-21swapper/307:14:253
201336799171,9cyclictest0-21swapper/309:34:323
201336799171,8cyclictest0-21swapper/312:04:273
201336799171,11cyclictest0-21swapper/311:19:233
201336799163,7cyclictest0-21swapper/309:24:293
201336799162,8cyclictest0-21swapper/311:34:223
201336799162,8cyclictest0-21swapper/311:04:263
201336799162,8cyclictest0-21swapper/310:29:273
201336799162,8cyclictest0-21swapper/310:19:263
201336799162,7cyclictest0-21swapper/312:19:203
201336799162,6cyclictest0-21swapper/308:14:203
201336799161,8cyclictest0-21swapper/310:54:253
201336799153,7cyclictest0-21swapper/311:39:213
201336799153,7cyclictest0-21swapper/311:39:213
201336799152,9cyclictest0-21swapper/312:34:223
201336799152,9cyclictest0-21swapper/312:34:223
201336799152,9cyclictest0-21swapper/311:44:233
201336799152,9cyclictest0-21swapper/311:44:233
201336799152,8cyclictest0-21swapper/310:44:223
201336799152,7cyclictest0-21swapper/312:24:253
201336799152,7cyclictest0-21swapper/312:24:253
201336799152,7cyclictest0-21swapper/311:24:213
201336799152,7cyclictest0-21swapper/309:29:233
201336799151,9cyclictest0-21swapper/312:29:223
201336799151,9cyclictest0-21swapper/312:29:223
201336799151,8cyclictest0-21swapper/312:14:263
201336799151,8cyclictest0-21swapper/310:59:243
201336799151,7cyclictest0-21swapper/310:04:233
201336799151,7cyclictest0-21swapper/309:59:213
201336799151,7cyclictest0-21swapper/308:29:253
201336799151,6cyclictest0-21swapper/309:09:223
201336799144,5cyclictest0-21swapper/308:49:273
201336799142,8cyclictest0-21swapper/312:35:293
201336799142,8cyclictest0-21swapper/309:39:213
201336799142,7cyclictest0-21swapper/311:54:283
201336799142,7cyclictest0-21swapper/311:54:283
201336799142,7cyclictest0-21swapper/310:24:223
201336799142,6cyclictest0-21swapper/308:34:223
201336799141,8cyclictest0-21swapper/310:14:173
201336799141,7cyclictest0-21swapper/311:29:253
201336799141,7cyclictest0-21swapper/311:09:213
201336799141,7cyclictest0-21swapper/308:19:223
201336799141,7cyclictest0-21swapper/307:54:283
201336799141,6cyclictest0-21swapper/309:04:223
201336799141,6cyclictest0-21swapper/307:44:203
201336799141,6cyclictest0-21swapper/307:29:233
201336799132,6cyclictest0-21swapper/311:49:183
201336799132,6cyclictest0-21swapper/311:49:183
201336799132,5cyclictest0-21swapper/307:39:233
201336799131,7cyclictest0-21swapper/311:59:243
201336799131,7cyclictest0-21swapper/311:59:243
201336799131,7cyclictest0-21swapper/311:14:203
201336799131,7cyclictest0-21swapper/310:39:233
201336799131,7cyclictest0-21swapper/309:49:213
201336799131,7cyclictest0-21swapper/308:54:243
201336799131,7cyclictest0-21swapper/307:24:223
201336799131,6cyclictest0-21swapper/308:44:223
201336799130,7cyclictest0-21swapper/307:49:263
201336799122,5cyclictest0-21swapper/308:24:273
201336799121,7cyclictest0-21swapper/309:54:103
201336799121,6cyclictest0-21swapper/308:39:293
201336799121,6cyclictest0-21swapper/307:59:223
201336799113,5cyclictest0-21swapper/308:59:203
201336799111,6cyclictest0-21swapper/307:19:213
201336799111,6cyclictest0-21swapper/307:19:213
201336799111,5cyclictest0-21swapper/307:34:123
201336799102,7cyclictest0-21swapper/307:10:013
201336799100,6cyclictest0-21swapper/308:09:283
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional