You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-03 - 17:10

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot5s.osadl.org (updated Fri Apr 03, 2026 12:44:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1958797995041,5cyclictest1983625-21sh08:20:001
1958797995021,9cyclictest1961253-21/usr/sbin/munin07:15:221
1958801994937,8cyclictest2014113-21grep09:45:242
1958801994835,9cyclictest311ktimers/208:55:112
1958797994720,9cyclictest2064432-21/usr/sbin/munin12:15:211
1958792994736,8cyclictest121ktimers/011:10:110
1958792994727,16cyclictest2053995-21munin-plugin-st11:45:000
1958806994634,9cyclictest1983629-21munin-plugin-st08:20:013
1958797994618,10cyclictest2018117-21latency_hist10:00:001
1958797994618,10cyclictest2013638-21ntp_states09:45:211
1958797994617,11cyclictest2006445-21latency09:25:171
1958797994535,6cyclictest1959138-21fschecks_time07:10:161
1958797994518,10cyclictest2052753-21taskset11:40:191
195879299452,7cyclictest1960623-21sed07:15:020
1958797994419,9cyclictest2050415-21chrt11:33:211
1958806994336,5cyclictest1972718-21kworker/u8:1+events_unbound08:14:073
1958806994333,6cyclictest2010922-21sh09:40:013
1958797994315,9cyclictest2005781-21taskset09:24:451
195879299431,9cyclictest2051520-21/usr/sbin/munin11:35:220
195879299431,6cyclictest1987240-21cat08:30:020
195879299431,10cyclictest2048448-21sed11:25:270
195879299431,10cyclictest1982691-21ntpq08:15:190
1958806994234,6cyclictest2015990-21kworker/u8:1+events_unbound10:24:303
1958801994234,4cyclictest0-21swapper/211:20:002
1958801994233,5cyclictest1969567-21sendmail07:40:002
1958797994228,9cyclictest2028524-21/usr/sbin/munin10:30:121
1958792994235,4cyclictest2010938-21sendmail09:40:010
1958792994232,7cyclictest2020927-21grep10:05:260
195879299421,4cyclictest2038435-21basename11:00:010
1958792994213,25cyclictest0-21swapper/008:45:010
195879299421,10cyclictest2068908-21head12:25:250
1958797994134,3cyclictest1985572-21apt-get08:25:131
1958797994129,7cyclictest1992846-21/usr/sbin/munin08:45:201
1958797994129,7cyclictest1992846-21/usr/sbin/munin08:45:191
1958797994112,10cyclictest1999032-21cat09:05:001
195879299419,8cyclictest2045999-21cat11:20:170
1958792994134,4cyclictest1993867-21latency_hist08:50:000
1958792994134,4cyclictest1993867-21latency_hist08:50:000
195879299412,6cyclictest2054759-21grep11:45:190
195879299411,9cyclictest2045180-21cron11:17:010
195879299411,12cyclictest2060793-21cron12:05:000
195879299411,12cyclictest2060793-21cron12:04:590
195879299411,11cyclictest1997256-21cron09:00:000
195879299411,10cyclictest1978909-21/usr/sbin/munin08:05:190
1958806994033,5cyclictest2024657-21kworker/u8:0+events_unbound10:34:323
1958806994031,6cyclictest1960010-21unixbench_singl07:10:253
195880699401,9cyclictest0-21swapper/312:20:133
1958806994015,5cyclictest0-21swapper/310:25:113
1958801994031,5cyclictest2021759-21fschecks_count10:10:182
1958801994027,9cyclictest1973241-21apt-config07:50:012
1958797994033,3cyclictest2040570-21apt-get11:05:111
1958797994030,7cyclictest2010287-21/usr/sbin/munin09:35:241
1958797994029,7cyclictest2062734-21apt-get12:10:131
1958797994027,8cyclictest2057615-21apt-get11:55:101
1958797994027,10cyclictest2044097-21/usr/sbin/munin11:15:141
1958797994023,13cyclictest2003458-21/usr/sbin/munin09:15:241
195879299405,11cyclictest2056944-21grep11:50:250
195879299401,9cyclictest2039873-21grep11:00:260
195879299401,7cyclictest1971261-21cron07:45:010
195879299401,14cyclictest2059184-21cut12:00:010
195879299401,13cyclictest0-21swapper/011:05:210
195879299401,12cyclictest2063779-21head12:10:250
1958792994011,26cyclictest0-21swapper/009:05:260
195879299401,11cyclictest2049214-21diskstats11:30:140
195879299400,12cyclictest0-21swapper/009:30:140
1958806993927,7cyclictest0-21swapper/307:20:023
1958801993929,5cyclictest0-21swapper/209:40:162
1958801993929,5cyclictest0-21swapper/209:40:162
1958797993932,4cyclictest2038870-21apt-get11:00:121
1958797993929,8cyclictest231ktimers/110:40:111
1958797993929,7cyclictest2066158-21apt-get12:20:111
1958797993929,7cyclictest2046820-21cron11:21:011
1958797993928,7cyclictest1994843-21/usr/sbin/munin08:50:241
1958797993928,7cyclictest1968689-21/usr/sbin/munin07:35:231
1958797993927,9cyclictest1973411-21apt-get07:50:101
1958797993926,10cyclictest2023967-21/usr/sbin/munin10:15:231
1958797993926,10cyclictest1967668-21cron07:33:011
1958797993925,10cyclictest2014892-21/usr/sbin/munin09:50:161
195879299398,5cyclictest2031104-21grep10:35:250
195879299395,8cyclictest438-21cron10:30:010
195879299393,10cyclictest2007518-21/usr/sbin/munin09:30:010
195879299392,7cyclictest2071468-21fschecks_time12:35:140
195879299392,7cyclictest1989968-21unixbench_multi08:35:250
195879299391,9cyclictest1968423-21/usr/sbin/munin07:35:190
195879299391,9cyclictest1964026-21grep07:20:250
195879299391,5cyclictest1967314-21timerwakeupswit07:30:220
1958792993915,20cyclictest1965671-21grep07:25:240
195879299391,11cyclictest2035803-21ntp_states10:50:210
195879299391,11cyclictest2023866-21ntp_states10:15:200
1958792993910,6cyclictest2033245-21chrt10:44:260
1958801993827,7cyclictest2041565-21grep11:05:242
1958797993830,4cyclictest1964678-21apt-get07:25:121
1958797993829,6cyclictest0-21swapper/107:45:001
1958797993828,7cyclictest2027290-21/usr/sbin/munin10:25:201
1958797993827,7cyclictest2067859-21apt-get12:25:111
1958797993827,7cyclictest2025685-21/usr/sbin/munin10:20:231
1958797993826,9cyclictest2050797-21apt-get11:35:111
1958797993826,7cyclictest2034215-21/usr/sbin/munin10:45:241
1958797993826,7cyclictest1978984-21/usr/sbin/munin08:05:201
1958797993825,9cyclictest2030567-21/usr/sbin/munin10:35:191
1958797993825,10cyclictest1977961-21cron08:01:011
1958797993823,11cyclictest2054476-21/usr/sbin/munin11:45:151
1958797993819,13cyclictest1988970-21/usr/sbin/munin08:35:101
1958797993817,15cyclictest2070108-21ntpq12:30:191
195879299382,9cyclictest2026020-21unixbench_multi10:20:250
195879299381,9cyclictest2011994-21ps09:40:220
195879299381,9cyclictest2011994-21ps09:40:220
195879299381,9cyclictest2005386-21sed09:20:240
195879299381,9cyclictest1973964-21/usr/sbin/munin07:50:190
195879299381,7cyclictest1972973-21apt07:50:010
1958792993811,9cyclictest204-21dbus-daemon08:24:080
195879299381,16cyclictest1988754-21idleruntime-cro08:35:010
195879299381,10cyclictest2022661-21head10:10:260
195879299381,10cyclictest1976593-21sendmail08:00:000
1958806993717,11cyclictest131rcu_preempt07:50:113
1958797993731,3cyclictest1962978-21apt-get07:20:111
1958797993730,4cyclictest2007742-21apt-get09:30:121
1958797993730,4cyclictest1987269-21apt-get08:30:121
1958797993727,7cyclictest2061338-21/usr/sbin/munin12:05:171
1958797993727,7cyclictest2042396-21/usr/sbin/munin11:10:161
1958797993727,7cyclictest1985075-21cron08:22:011
1958797993727,6cyclictest1971501-21/usr/sbin/munin07:45:101
1958797993726,7cyclictest2020882-21grep10:05:261
1958797993726,7cyclictest1975323-21/usr/sbin/munin07:55:161
1958797993725,9cyclictest2035641-21/usr/sbin/munin10:50:181
1958797993722,11cyclictest0-21swapper/111:50:121
1958797993721,13cyclictest1996870-21/usr/sbin/munin08:55:251
1958797993720,13cyclictest1980349-21/usr/sbin/munin08:10:151
195879299373,8cyclictest0-21swapper/012:05:150
195879299372,7cyclictest0-21swapper/007:15:200
195879299372,11cyclictest2035003-21cron10:50:010
195879299371,9cyclictest1977507-21/usr/sbin/munin08:00:210
195879299371,7cyclictest2065063-21wc12:15:230
1958792993716,18cyclictest0-21swapper/009:55:150
195879299371,13cyclictest2003640-21unixbench_multi09:15:240
195879299371,12cyclictest0-21swapper/010:00:190
195879299371,11cyclictest2013594-21awk09:45:190
1958806993630,4cyclictest2045418-21kworker/u8:3+events_unbound12:06:393
1958806993628,6cyclictest2019643-21kworker/u8:2+events_unbound10:18:413
1958806993627,6cyclictest391ktimers/307:20:113
1958806993615,8cyclictest131rcu_preempt11:30:113
1958806993613,13cyclictest1997221-21kworker/u8:0-flush-179:9609:05:113
1958801993625,8cyclictest1967849-21auditctl07:35:012
1958797993626,7cyclictest2059887-21/usr/sbin/munin12:00:211
1958797993626,7cyclictest2059887-21/usr/sbin/munin12:00:211
1958797993626,7cyclictest2000973-21/usr/sbin/munin09:10:131
1958797993626,7cyclictest2000152-21/usr/sbin/munin09:05:241
1958797993626,6cyclictest2037050-21/usr/sbin/munin10:55:131
1958797993622,10cyclictest2011492-21/usr/sbin/munin09:40:151
1958797993622,10cyclictest2011492-21/usr/sbin/munin09:40:141
1958797993618,9cyclictest0-21swapper/110:10:141
195879299362,6cyclictest164-21systemd-journal12:24:450
195879299361,8cyclictest2030071-21grep10:35:020
195879299361,7cyclictest2002380-21cron09:15:000
195879299361,13cyclictest1998523-21grep09:00:260
195879299361,11cyclictest1980864-21wc08:10:230
1958806993528,5cyclictest2024657-21kworker/u8:0+events_unbound11:08:413
1958806993528,5cyclictest1985237-21kworker/u8:2+events_unbound08:24:103
1958801993523,7cyclictest0-21swapper/207:50:252
1958797993528,4cyclictest2018192-21apt-get10:00:111
1958797993524,7cyclictest1990810-21diskstats08:40:141
1958797993519,13cyclictest2048470-21/usr/sbin/munin11:25:261
195879299354,7cyclictest2069798-21basename12:30:140
195879299351,13cyclictest0-21swapper/008:50:230
195880699349,11cyclictest40-21ksoftirqd/308:45:263
195880699349,11cyclictest40-21ksoftirqd/308:45:253
195880699345,7cyclictest40-21ksoftirqd/308:00:243
1958806993427,4cyclictest2055685-21sh11:49:593
1958806993415,5cyclictest131rcu_preempt09:25:003
1958806993415,11cyclictest131rcu_preempt09:25:243
1958806993415,11cyclictest131rcu_preempt08:30:253
1958801993425,5cyclictest1962755-21sendmail07:20:012
1958801993424,7cyclictest2043740-21auditctl11:15:012
1958801993423,7cyclictest0-21swapper/212:00:262
1958801993423,7cyclictest0-21swapper/212:00:262
1958801993419,11cyclictest311ktimers/212:10:102
1958797993426,4cyclictest2071255-21apt-get12:35:111
195879299347,4cyclictest2015395-21proc_pri09:50:240
1958806993322,7cyclictest0-21swapper/310:35:253
1958806993322,5cyclictest131rcu_preempt09:15:023
1958806993316,5cyclictest131rcu_preempt07:35:223
1958806993314,12cyclictest131rcu_preempt12:20:023
1958801993321,9cyclictest438-21cron10:45:012
1958806993226,4cyclictest2043530-21kworker/u8:1+events_unbound11:19:343
1958806993221,7cyclictest1972460-21timerwakeupswit07:45:253
1958806993214,3cyclictest131rcu_preempt07:25:133
1958806993214,11cyclictest131rcu_preempt10:45:193
1958806993213,5cyclictest131rcu_preempt12:35:223
1958801993225,4cyclictest0-21swapper/207:15:012
1958801993224,4cyclictest0-21swapper/211:55:152
1958801993224,4cyclictest0-21swapper/207:25:152
195880699318,12cyclictest40-21ksoftirqd/308:40:013
1958806993125,4cyclictest2045418-21kworker/u8:3+events_unbound11:20:413
1958806993125,4cyclictest2022707-21kworker/u8:3+events_unbound10:14:293
1958806993122,5cyclictest0-21swapper/307:55:243
1958806993122,4cyclictest131rcu_preempt11:10:223
1958806993113,6cyclictest131rcu_preempt09:15:123
1958806993112,6cyclictest0-21swapper/310:54:233
1958806993111,5cyclictest131rcu_preempt12:13:013
1958806993111,12cyclictest131rcu_preempt12:00:013
1958801993122,6cyclictest2071024-21auditctl12:35:012
195880699309,13cyclictest40-21ksoftirqd/311:00:173
195880699309,12cyclictest40-21ksoftirqd/309:50:143
1958806993024,4cyclictest1985237-21kworker/u8:2+events_unbound08:28:413
195880699302,16cyclictest0-21swapper/312:25:243
1958806993020,8cyclictest381rcuc/311:40:143
1958806993019,3cyclictest131rcu_preempt07:40:183
1958806993018,5cyclictest131rcu_preempt09:55:263
1958806993014,4cyclictest131rcu_preempt09:40:033
1958806993014,4cyclictest131rcu_preempt09:40:023
1958806993013,8cyclictest131rcu_preempt08:44:423
1958806993012,7cyclictest40-21ksoftirqd/308:55:223
1958806993012,11cyclictest131rcu_preempt08:50:233
1958806993011,4cyclictest0-21swapper/311:50:243
1958806993010,12cyclictest40-21ksoftirqd/309:45:233
1958801993022,5cyclictest1987429-21grep08:30:152
1958801993022,4cyclictest0-21swapper/210:55:232
1958801993022,4cyclictest0-21swapper/207:20:152
1958801993021,4cyclictest0-21swapper/209:40:012
1958801993019,8cyclictest2031592-21cron10:40:012
195880699298,12cyclictest131rcu_preempt12:05:013
195880699298,12cyclictest131rcu_preempt12:05:003
1958806992916,5cyclictest131rcu_preempt11:35:183
1958806992916,5cyclictest131rcu_preempt09:00:253
1958806992913,3cyclictest131rcu_preempt11:28:523
1958806992911,11cyclictest40-21ksoftirqd/310:00:243
1958801992922,5cyclictest2024657-21kworker/u8:0+events_unbound10:51:142
1958801992921,4cyclictest0-21swapper/211:40:282
1958801992920,4cyclictest0-21swapper/209:10:162
1958801992916,9cyclictest1984872-21grep08:20:252
195880699288,7cyclictest40-21ksoftirqd/310:40:123
195880699283,12cyclictest40-21ksoftirqd/308:05:263
1958801992821,4cyclictest0-21swapper/208:35:242
1958801992820,4cyclictest0-21swapper/209:59:592
1958801992817,7cyclictest0-21swapper/211:00:272
1958801992816,7cyclictest1995085-21grep08:50:252
195880699279,12cyclictest40-21ksoftirqd/312:30:143
1958806992711,6cyclictest40-21ksoftirqd/310:55:153
1958806992710,11cyclictest40-21ksoftirqd/309:30:213
1958801992720,3cyclictest0-21swapper/208:00:152
1958801992716,7cyclictest2046385-21fgrep11:20:222
1958801992715,8cyclictest1970860-21grep07:40:262
1958806992610,5cyclictest131rcu_preempt07:30:053
195880199266,4cyclictest20041372wc09:20:012
1958801992619,4cyclictest2026013-21tail10:20:242
1958801992619,3cyclictest0-21swapper/209:30:242
1958801992618,4cyclictest0-21swapper/212:35:232
1958801992618,4cyclictest0-21swapper/208:05:262
1958801992618,3cyclictest0-21swapper/209:54:592
1958801992616,6cyclictest0-21swapper/208:40:262
195880699257,3cyclictest131rcu_preempt10:05:193
1958801992519,3cyclictest0-21swapper/212:25:002
1958801992515,6cyclictest2062508-21idleruntime-cro12:10:002
1958801992514,6cyclictest2007097-21grep09:25:232
1958801992416,4cyclictest0-21swapper/210:45:162
1958801992415,6cyclictest2057481-21date11:55:002
1958801992317,4cyclictest145-21jbd2/mmcblk1p2-811:45:142
1958801992317,3cyclictest0-21swapper/209:20:222
1958801992317,3cyclictest0-21swapper/209:00:232
1958801992316,5cyclictest311ktimers/210:30:102
1958801992316,5cyclictest2024657-21kworker/u8:0+events_unbound11:29:342
1958801992315,4cyclictest0-21swapper/211:40:012
1958801992315,4cyclictest0-21swapper/210:04:262
1958801992314,5cyclictest0-21swapper/211:30:252
1958801992312,8cyclictest1993873-21basename08:50:012
1958801992312,8cyclictest1993873-21basename08:50:012
1958801992214,5cyclictest1981731-21cat08:15:012
1958801992210,8cyclictest1987035-21cron08:30:002
1958801992116,3cyclictest0-21swapper/209:05:172
1958801992113,5cyclictest0-21swapper/208:15:252
1958801992112,6cyclictest2027257-21ntpq10:25:192
1958801992112,6cyclictest2024262-21cat10:15:252
1958801992012,5cyclictest2065953-21ls12:20:002
195880199192,13cyclictest2021365-21latency_hist10:10:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional