You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-21 - 16:54

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot5s.osadl.org (updated Fri Nov 21, 2025 12:44:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1920486995443,5cyclictest2026013-21kworker/u8:2+events_unbound12:25:221
1920481995344,6cyclictest2003704-21kworker/u8:1+events_unbound11:50:150
1920494995244,5cyclictest1923041-21fschecks_count07:15:043
1920494995033,13cyclictest391ktimers/309:05:163
1920486995043,4cyclictest1922342-21apt-config07:15:021
1920494994940,5cyclictest2012477-21munin-plugin-st11:35:013
1920481994943,4cyclictest2013973-21kworker/u8:0+events_unbound11:45:150
1920490994735,8cyclictest1924656-21sendmail07:20:012
1920490994633,10cyclictest311ktimers/211:35:132
1920494994526,14cyclictest0-21swapper/308:55:263
1920494994522,19cyclictest0-21swapper/309:50:253
1920494994522,19cyclictest0-21swapper/309:50:253
1920494994519,22cyclictest0-21swapper/311:40:253
1920481994534,8cyclictest121ktimers/008:20:140
1920486994430,9cyclictest2005165-21grep11:10:241
192048199449,5cyclictest131rcu_preempt08:05:100
192049499431,9cyclictest2023240-21/usr/sbin/munin12:05:163
192049499431,10cyclictest2016118-21/usr/sbin/munin11:45:123
192049499431,10cyclictest1990825-21/usr/sbin/munin10:30:193
1920481994334,4cyclictest0-21swapper/009:00:000
192049499423,9cyclictest2018580-21/usr/sbin/munin11:50:243
192049499421,8cyclictest2000427-21cron10:59:003
192049499421,13cyclictest2020567-21head11:55:233
192049499421,11cyclictest2030416-21meminfo12:25:183
192049499421,11cyclictest1975812-21head09:45:243
192049499421,11cyclictest1935078-21cron07:50:013
1920490994234,5cyclictest1974533-21cron09:45:002
1920486994235,5cyclictest1931103-21kworker/u8:1+events_unbound07:59:251
1920486994234,6cyclictest2020815-21kworker/u8:0+events_unbound12:10:191
1920486994231,8cyclictest2014189-21sendmail11:40:001
192049499418,6cyclictest1945224-21seq08:19:073
192049499411,9cyclictest1959740-21/usr/sbin/munin09:00:203
192049499411,8cyclictest2003231-21/usr/sbin/munin11:05:243
192049499411,8cyclictest1937714-21/usr/sbin/munin07:55:223
192049499411,7cyclictest1987623-21/usr/sbin/munin10:20:243
192049499403,6cyclictest2033148-21munin-run12:35:003
192049499401,9cyclictest1997471-21/usr/sbin/munin10:50:163
192049499401,9cyclictest1992233-21munin-node10:35:113
192049499401,9cyclictest1985479-21if_err_eth110:15:173
192049499401,9cyclictest1928800-21/usr/sbin/munin07:30:193
192049499401,8cyclictest1942476-21/usr/sbin/munin08:10:173
192049499401,7cyclictest1946635-21/usr/sbin/munin08:20:273
192049499401,12cyclictest1988950-21df_inode10:25:143
192049499401,11cyclictest1979387-21grep09:55:253
192049499401,11cyclictest1933000-21users07:40:293
192049499401,11cyclictest1931466-21idleruntime-cro07:40:013
192049499400,9cyclictest1966249-21apt-get09:20:123
1920490994029,8cyclictest311ktimers/212:00:132
1920486994026,8cyclictest1994992-21grep10:40:281
1920481994032,5cyclictest1922873-21apt-get07:15:030
192049499392,4cyclictest1940195-21cron08:05:013
192049499391,9cyclictest2025393-21awk12:10:203
192049499391,9cyclictest2004709-21/usr/sbin/munin11:10:193
192049499391,8cyclictest2028906-21wc12:20:243
192049499391,8cyclictest2006250-21/usr/sbin/munin11:15:163
192049499391,8cyclictest1941503-21sed08:05:233
192049499391,6cyclictest1953375-21tail08:40:263
192049499391,4cyclictest2002064-21cron11:03:003
192049499391,4cyclictest2002064-21cron11:03:003
192049499391,12cyclictest1973518-21meminfo09:40:203
1920494993911,25cyclictest1920822-21cut07:10:183
1920494993911,24cyclictest0-21swapper/308:30:173
1920494993911,24cyclictest0-21swapper/308:30:163
192049499391,11cyclictest2027118-21ntp_states12:15:223
192049499391,10cyclictest1971801-21/usr/sbin/munin09:35:203
1920490993930,6cyclictest1998393-21unixbench_singl10:50:272
1920486993933,4cyclictest1969091-21kworker/u8:1+events_unbound09:49:501
1920486993932,5cyclictest1969091-21kworker/u8:1+events_unbound09:39:461
1920486993930,6cyclictest1941904-21sh08:10:011
192048699391,5cyclictest131rcu_preempt08:15:141
192048199397,4cyclictest131rcu_preempt11:20:140
1920481993926,10cyclictest121ktimers/007:50:140
192049499381,9cyclictest1996105-21meminfo10:45:193
192049499381,9cyclictest1970722-21head09:30:293
192049499381,9cyclictest1950666-21/usr/sbin/munin08:35:123
192049499381,8cyclictest2034202-21/usr/sbin/munin12:35:213
192049499381,8cyclictest2009299-21/usr/sbin/munin11:25:203
192049499381,8cyclictest1963184-21ntp_states09:10:203
192049499381,8cyclictest1954982-21sed08:45:263
192049499381,6cyclictest1968060-21df_inode09:25:133
192049499381,11cyclictest1935893-21ntp_states07:50:243
1920481993831,4cyclictest1971108-21cron09:35:000
192049499372,7cyclictest391ktimers/308:25:133
192049499372,5cyclictest215-21dbus-daemon07:29:183
192049499371,9cyclictest1980213-21df_inode10:00:143
192049499371,9cyclictest1926362-21awk07:25:013
192049499371,7cyclictest1955945-21fschecks_count08:50:163
1920494993713,22cyclictest2007600-21/usr/sbin/munin11:20:193
192049499371,12cyclictest1965572-21sed09:15:263
192049499371,10cyclictest1982474-21/usr/sbin/munin10:05:253
1920481993729,4cyclictest0-21swapper/012:00:000
1920481993726,6cyclictest0-21swapper/011:10:240
192049499365,8cyclictest215-21dbus-daemon11:35:123
192049499362,5cyclictest2021444-21/usr/sbin/munin12:00:153
1920494993617,16cyclictest1983661-21fschecks_count10:10:153
192049499361,5cyclictest1994486-21grep10:40:233
1920490993627,5cyclictest0-21swapper/210:10:152
1920490993627,4cyclictest0-21swapper/212:20:002
1920486993626,6cyclictest0-21swapper/110:40:001
1920486993626,6cyclictest0-21swapper/109:10:011
1920486993625,7cyclictest1922797-21switchtime07:15:031
1920486993623,7cyclictest2003704-21kworker/u8:1+events_unbound11:15:021
1920486993623,6cyclictest1929375-21kworker/u8:2+events_unbound07:53:431
1920481993629,4cyclictest0-21swapper/008:50:010
1920481993629,3cyclictest0-21swapper/010:20:010
1920490993528,4cyclictest1952140-21munin-plugin-st08:40:002
1920490993526,6cyclictest0-21swapper/211:54:592
1920486993528,5cyclictest1983026-21kworker/u8:2+events_unbound10:12:251
1920486993527,5cyclictest1947040-21idleruntime-cro08:25:001
1920486993527,4cyclictest0-21swapper/112:15:161
1920486993521,6cyclictest1931103-21kworker/u8:1+events_unbound07:42:191
192048699350,5cyclictest131rcu_preempt08:10:111
1920481993526,5cyclictest1952142-21munin-plugin-st08:40:000
1920481993525,6cyclictest0-21swapper/010:30:180
1920490993427,4cyclictest0-21swapper/207:15:022
1920490993424,6cyclictest311ktimers/209:55:112
1920486993428,4cyclictest1981278-21kworker/u8:0+events_unbound10:09:531
1920486993423,7cyclictest0-21swapper/109:55:251
1920486993419,5cyclictest1993392-21kworker/u8:2+events_unbound11:04:571
1920486993419,5cyclictest1993392-21kworker/u8:2+events_unbound11:04:571
1920481993427,3cyclictest0-21swapper/007:15:020
1920490993325,5cyclictest0-21swapper/211:00:332
1920490993325,5cyclictest0-21swapper/211:00:322
1920490993325,4cyclictest0-21swapper/210:25:012
1920490993325,4cyclictest0-21swapper/209:10:132
1920490993324,6cyclictest0-21swapper/207:50:332
1920490993322,7cyclictest1944863-21grep08:15:272
1920486993325,4cyclictest0-21swapper/108:25:261
1920486993324,4cyclictest0-21swapper/109:30:011
192048199339,6cyclictest131rcu_preempt09:20:110
1920481993310,4cyclictest131rcu_preempt09:00:120
1920486993224,4cyclictest0-21swapper/109:05:011
1920486993222,6cyclictest1971143-21cat09:35:011
1920486993218,6cyclictest1926572-21kworker/u8:0+events_unbound07:27:511
192048699321,5cyclictest131rcu_preempt08:30:141
192048699321,5cyclictest131rcu_preempt08:30:141
1920481993223,4cyclictest0-21swapper/011:25:250
1920481993223,4cyclictest0-21swapper/010:00:240
1920481993222,8cyclictest2014954-21ntp_kernel_pll_11:40:200
1920481993222,6cyclictest0-21swapper/010:40:280
1920481993221,6cyclictest0-21swapper/009:10:230
1920490993124,4cyclictest1996144-21cat10:45:202
1920490993123,4cyclictest0-21swapper/212:30:222
1920490993123,4cyclictest0-21swapper/209:25:012
1920490993121,7cyclictest1960681-21idleruntime-cro09:05:012
1920490993121,7cyclictest1955148-21grep08:45:272
1920490993120,7cyclictest0-21swapper/209:55:002
1920490993120,7cyclictest0-21swapper/209:55:002
1920486993123,4cyclictest0-21swapper/110:55:161
1920486993122,6cyclictest0-21swapper/110:25:011
1920481993121,6cyclictest0-21swapper/010:10:250
1920490993022,4cyclictest0-21swapper/212:30:002
1920490993022,4cyclictest0-21swapper/208:55:262
1920490993022,4cyclictest0-21swapper/208:00:272
1920490993016,11cyclictest1920475-21cyclictest11:30:112
1920486993020,6cyclictest0-21swapper/109:40:171
1920486993019,7cyclictest1952131-21sh08:40:001
1920486993017,4cyclictest2003704-21kworker/u8:1+events_unbound11:23:321
1920486993014,6cyclictest1931103-21kworker/u8:1+events_unbound08:43:251
1920481993024,3cyclictest2024619-21kthreadd12:10:010
1920481993022,6cyclictest0-21swapper/012:20:010
1920481993022,4cyclictest0-21swapper/012:35:230
1920481993021,5cyclictest0-21swapper/009:05:290
192048199301,4cyclictest131rcu_preempt09:35:130
1920490992922,4cyclictest0-21swapper/209:30:012
1920490992922,3cyclictest0-21swapper/209:30:292
1920490992921,5cyclictest1931484-21sendmail-msp07:40:012
1920490992921,4cyclictest0-21swapper/210:25:252
1920490992921,4cyclictest0-21swapper/208:29:352
1920490992918,8cyclictest1933376-21cron07:45:012
1920490992917,9cyclictest311ktimers/210:35:132
192048699297,5cyclictest131rcu_preempt11:45:131
1920486992924,3cyclictest2035082-21idleruntime-cro12:40:001
1920486992922,3cyclictest0-21swapper/111:50:251
1920486992921,4cyclictest0-21swapper/109:14:391
1920486992915,4cyclictest1993392-21kworker/u8:2+events_unbound10:50:471
192048699290,4cyclictest131rcu_preempt10:30:141
192048199298,3cyclictest131rcu_preempt09:40:140
192048199296,6cyclictest131rcu_preempt12:25:100
192048199295,3cyclictest131rcu_preempt07:30:110
1920481992921,6cyclictest1981278-21kworker/u8:0+events_unbound10:09:550
1920481992919,7cyclictest2007349-21munin-plugin-st11:20:010
192048199291,15cyclictest0-21swapper/008:29:590
192049099288,3cyclictest19730302cut09:40:002
1920490992821,4cyclictest0-21swapper/210:00:242
1920490992820,6cyclictest2001986-21kworker/u8:0+events_unbound11:05:102
1920490992819,6cyclictest2026312-21sh12:15:012
1920490992819,6cyclictest0-21swapper/212:05:182
1920490992819,5cyclictest1995431-21munin-plugin-st10:45:002
1920486992822,4cyclictest2019103-21kworker/u8:2+events_unbound12:00:041
1920486992821,5cyclictest2020577-21unixbench_singl11:55:231
192048699282,15cyclictest0-21swapper/110:04:411
1920486992820,5cyclictest2006971-21kworker/u8:2+events_unbound11:25:241
1920486992817,7cyclictest1964739-21cut09:15:161
1920486992814,8cyclictest131rcu_preempt08:05:021
192048699281,15cyclictest0-21swapper/108:47:491
192048199289,3cyclictest131rcu_preempt12:30:140
192048199288,3cyclictest131rcu_preempt08:00:130
192048199287,9cyclictest11-21ksoftirqd/012:20:190
192048199287,13cyclictest0-21swapper/007:25:140
192048199282,16cyclictest0-21swapper/007:46:540
192048199281,9cyclictest11-21ksoftirqd/011:05:130
1920481992819,6cyclictest0-21swapper/008:00:010
192048199281,17cyclictest2001112-21/usr/sbin/munin11:00:190
192048199281,17cyclictest2001112-21/usr/sbin/munin11:00:190
192048199281,16cyclictest0-21swapper/011:33:110
192048199281,16cyclictest0-21swapper/009:27:240
1920481992810,6cyclictest11-21ksoftirqd/009:55:170
1920490992720,4cyclictest0-21swapper/212:20:172
1920490992720,4cyclictest0-21swapper/211:10:212
192048699277,5cyclictest131rcu_preempt07:46:591
1920486992721,4cyclictest2001986-21kworker/u8:0+events_unbound11:05:251
1920486992718,5cyclictest0-21swapper/110:15:161
1920486992717,6cyclictest1926359-21munin-run07:25:011
192048699271,15cyclictest0-21swapper/109:23:071
1920481992720,5cyclictest1951833-21kworker/u8:2+events_unbound08:50:120
192048199271,15cyclictest0-21swapper/010:47:060
192048199271,12cyclictest0-21swapper/010:36:570
1920490992619,5cyclictest1923421-21kworker/u8:3+events_unbound07:20:072
1920490992619,3cyclictest0-21swapper/207:50:002
1920490992617,4cyclictest0-21swapper/211:40:132
1920490992614,9cyclictest1982371-21ntp_states10:05:232
1920490992612,10cyclictest1-21systemd11:15:112
1920486992618,5cyclictest2022765-21kworker/u8:1+events_unbound12:06:101
1920486992618,4cyclictest0-21swapper/108:55:261
1920486992611,4cyclictest221rcuc/107:40:011
1920486992611,12cyclictest221rcuc/112:25:011
1920486992610,6cyclictest0-21swapper/111:30:131
192048199268,4cyclictest131rcu_preempt12:10:170
1920481992619,3cyclictest0-21swapper/009:45:150
1920481992618,4cyclictest440-21cron07:45:010
1920481992615,8cyclictest2014177-21run-parts11:40:000
192048199261,4cyclictest0-21swapper/008:40:290
192048199261,14cyclictest0-21swapper/010:20:480
1920490992518,4cyclictest0-21swapper/208:20:272
1920490992517,4cyclictest0-21swapper/208:30:262
1920490992517,4cyclictest0-21swapper/208:30:262
1920490992512,9cyclictest1-21systemd10:55:072
1920486992519,4cyclictest1988204-21kworker/u8:2+events_unbound10:25:171
192048699251,8cyclictest24-21ksoftirqd/112:30:541
1920486992518,5cyclictest1986438-21kworker/u8:1+events_unbound10:45:191
1920486992514,6cyclictest0-21swapper/108:50:261
192048199258,6cyclictest11-21ksoftirqd/007:35:160
192048199257,10cyclictest11-21ksoftirqd/009:15:210
192048199255,9cyclictest131rcu_preempt10:55:200
192048199253,8cyclictest11-21ksoftirqd/007:20:110
1920481992520,2cyclictest0-21swapper/010:50:190
1920481992518,5cyclictest2019103-21kworker/u8:2+events_unbound12:00:590
1920481992518,5cyclictest1986438-21kworker/u8:1+events_unbound10:28:420
1920490992418,3cyclictest0-21swapper/208:50:242
1920490992418,3cyclictest0-21swapper/207:30:152
1920490992416,5cyclictest1941897-21cron08:10:012
1920486992415,6cyclictest1977253-21ntp_states09:50:221
1920486992415,6cyclictest1977253-21ntp_states09:50:221
1920481992415,5cyclictest1943204-21grep08:10:240
1920481992415,5cyclictest0-21swapper/008:33:280
1920481992415,5cyclictest0-21swapper/008:33:280
1920490992314,4cyclictest0-21swapper/209:45:152
192048699231,14cyclictest0-21swapper/107:32:311
1920490992216,3cyclictest0-21swapper/211:55:232
1920490992216,3cyclictest0-21swapper/211:20:192
1920490992216,3cyclictest0-21swapper/209:05:202
1920490992213,6cyclictest0-21swapper/212:40:002
1920490992115,3cyclictest1928159-21latency_hist07:30:012
1920490992115,3cyclictest0-21swapper/210:30:252
1920490992112,6cyclictest1985762-21/usr/sbin/munin10:15:212
1920490992011,7cyclictest1942531-21sed08:10:172
1920490992011,6cyclictest0-21swapper/211:50:012
192049099199,7cyclictest2010134-21cat11:25:232
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional