You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-18 - 17:25

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TI
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot6s.osadl.org (updated Mon May 18, 2026 12:44:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2131669997611,19cyclictest2180881-21/usr/sbin/munin11:55:191
2131666997230,35cyclictest2171683-21irqstats11:00:230
2131666997031,33cyclictest2157038-21/usr/sbin/munin09:35:200
2131666997016,47cyclictest2151783-21latency_hist09:05:010
2131666996830,4cyclictest2179273-21diskstats11:45:170
2131666996829,3cyclictest2167525-21/usr/sbin/munin10:35:260
2131666996725,35cyclictest0-21swapper/010:00:280
2131666996618,44cyclictest0-21swapper/009:40:270
2131666996618,44cyclictest0-21swapper/009:40:270
2131666996436,16cyclictest0-21swapper/007:55:140
2131666996436,16cyclictest0-21swapper/007:55:140
213167099639,25cyclictest30-21rcuc/211:20:022
2131666996235,25cyclictest2152149-21/usr/sbin/munin09:05:260
2131666996220,36cyclictest0-21swapper/008:35:250
2131666996132,3cyclictest2133026-21diskstats07:15:190
2131666996122,4cyclictest2162302-21irqstats10:05:250
2131666996050,5cyclictest167350irq/118-8000000.ethernet-tx009:25:270
2131666996021,3cyclictest2154877-21/usr/sbin/munin09:20:260
2131666995928,28cyclictest2134745-21apt07:25:120
2131666995923,33cyclictest2178608-21if_eth111:40:200
2131666995923,33cyclictest2178608-21if_eth111:40:200
2131666995921,35cyclictest2182164-21ptp4l-jitter12:00:260
2131669995823,25cyclictest22-21rcuc/111:05:011
2131666995836,16cyclictest0-21swapper/012:05:150
2131666995831,3cyclictest2142775-21irqstats08:10:260
2131666995830,3cyclictest2139121-21diskstats07:50:180
2131666995830,25cyclictest2160592-21/usr/sbin/munin09:55:210
2131666995829,3cyclictest2163079-21perl10:10:220
2131666995822,32cyclictest2188299-21latency_hist12:40:010
2131666995822,2cyclictest2138512-21memory07:45:250
2131666995822,2cyclictest2138512-21memory07:45:250
213167099577,19cyclictest2175646-21/usr/sbin/munin11:25:182
213167099577,19cyclictest2175646-21/usr/sbin/munin11:25:182
2131666995740,13cyclictest0-21swapper/011:05:150
2131666995730,25cyclictest2170831-21irqstats10:55:280
2131666995729,3cyclictest2177716-21memory11:35:260
2131666995729,3cyclictest2176746-21irqstats11:30:230
2131666995728,3cyclictest2135881-21meminfo07:30:250
2131666995723,10cyclictest2143411-21/usr/sbin/munin08:15:150
2131666995628,24cyclictest2145690-21sh08:29:590
2131666995621,32cyclictest2175146-21phc2sys-jitter11:20:240
213166699553,48cyclictest0-21swapper/008:35:010
2131666995528,24cyclictest2169939-21meminfo10:50:230
2131666995522,7cyclictest2174579-21cron11:20:010
2131666995517,3cyclictest2183490-21diskstats12:10:190
2131666995517,32cyclictest0-21swapper/008:40:160
2131670995417,10cyclictest2169638-21/usr/sbin/munin10:50:182
2131669995416,17cyclictest2183418-21/usr/sbin/munin12:10:191
2131666995439,12cyclictest2186923-21/usr/sbin/munin12:30:200
2131666995412,27cyclictest0-21swapper/008:20:010
2131666995412,27cyclictest0-21swapper/008:20:010
2131666995340,10cyclictest2168066-21/usr/sbin/munin10:40:190
2131666995326,3cyclictest2168872-21diskstats10:45:180
2131666995324,3cyclictest2150417-21memory08:55:260
2131666995324,23cyclictest2132467-21latency_hist07:15:010
2131666995324,23cyclictest2132467-21latency_hist07:15:010
2131666995239,9cyclictest2185420-21/usr/sbin/munin12:20:270
2131666995239,9cyclictest2185420-21/usr/sbin/munin12:20:270
2131666995238,10cyclictest2173509-21/usr/sbin/munin11:10:260
2131666995219,31cyclictest0-21swapper/009:15:180
2131669995140,8cyclictest2133621-21apt-get07:20:001
2131666995144,4cyclictest2156194-21/usr/sbin/munin09:30:170
2131666995140,9cyclictest2180667-21sshd11:52:550
2131666995140,6cyclictest2166564-21memory10:30:250
2131666995134,14cyclictest2165764-21/usr/sbin/munin10:25:260
2131666995134,12cyclictest2181442-21/usr/sbin/munin11:55:280
2131666995124,3cyclictest2140905-21df_inode08:00:180
2131666995121,27cyclictest2159564-21df09:50:160
2131666995120,8cyclictest0-21swapper/012:30:010
2131666995042,4cyclictest2184330-21/usr/sbin/munin12:15:180
2131666995037,10cyclictest2164907-21/usr/sbin/munin10:20:260
2131666995029,18cyclictest1660-21snmpd08:48:350
2131666995026,4cyclictest2153340-21apt-config09:15:000
2131666995018,28cyclictest2149798-21unixbench_multi08:50:260
2131666995017,31cyclictest0-21swapper/011:25:260
2131666995017,31cyclictest0-21swapper/011:25:260
2131666994936,9cyclictest2137932-21/usr/sbin/munin07:40:280
2131666994935,11cyclictest0-21swapper/010:15:130
2131666994932,14cyclictest2134340-21/usr/sbin/munin07:20:280
2131669994841,4cyclictest2150756-21sendmail08:59:591
2131666994838,5cyclictest2158663-21apt-get09:45:130
2131666994833,12cyclictest2141758-21/usr/sbin/munin08:05:190
2131666994828,16cyclictest0-21swapper/007:35:140
213166999476,35cyclictest0-21swapper/109:15:051
2131669994724,9cyclictest24-21ksoftirqd/110:45:011
2131669994638,4cyclictest0-21swapper/111:30:291
2131669994634,7cyclictest0-21swapper/108:35:301
2131669994631,8cyclictest0-21swapper/108:19:591
2131669994631,10cyclictest0-21swapper/109:15:021
2131669994531,10cyclictest2140536-21munin-plugin-st08:00:001
2131669994531,10cyclictest2140536-21munin-plugin-st08:00:001
213167099441,4cyclictest0-21swapper/210:25:012
213167099441,36cyclictest0-21swapper/207:19:592
2131670994336,4cyclictest2178059-21munin-plugin-st11:39:592
2131669994331,8cyclictest0-21swapper/108:45:281
2131666994327,12cyclictest2131718-21latency_hist07:10:000
213167099423,32cyclictest0-21swapper/211:05:262
2131670994228,9cyclictest0-21swapper/210:55:242
2131669994234,5cyclictest0-21swapper/110:35:281
2131669994234,4cyclictest0-21swapper/112:05:001
2131669994231,7cyclictest0-21swapper/109:05:001
2131669994231,7cyclictest0-21swapper/108:00:281
2131669994228,10cyclictest2178062-21sendmail11:40:001
2131669994228,10cyclictest0-21swapper/110:25:271
2131669994225,10cyclictest0-21swapper/108:20:291
2131669994225,10cyclictest0-21swapper/108:20:281
2131670994118,12cyclictest0-21swapper/209:55:272
213167099411,33cyclictest0-21swapper/210:40:292
2131669994128,9cyclictest0-21swapper/111:40:261
2131669994128,9cyclictest0-21swapper/111:40:261
213167099406,11cyclictest2134744-21/usr/sbin/munin07:25:182
2131670994028,7cyclictest2145597-21grep08:25:282
2131670994027,8cyclictest2181423-21grep11:55:292
2131669994023,9cyclictest0-21swapper/111:15:261
2131669994023,9cyclictest0-21swapper/110:20:211
213167099394,29cyclictest14-21rcu_preempt07:10:252
213167099394,29cyclictest14-21rcu_preempt07:10:242
2131670993923,10cyclictest0-21swapper/212:15:262
2131669993927,6cyclictest2188167-21grep12:35:291
2131669993925,9cyclictest0-21swapper/107:35:291
2131669993925,10cyclictest0-21swapper/111:30:021
2131669993925,10cyclictest0-21swapper/111:30:021
2131669993923,8cyclictest0-21swapper/112:30:271
2131669993915,14cyclictest2131663-21cyclictest12:05:141
2131669993825,8cyclictest0-21swapper/107:50:221
213166999381,10cyclictest0-21swapper/112:30:001
2131670993729,4cyclictest167550irq/134-8000000.ethernet-tx211:30:292
2131670993729,4cyclictest167550irq/134-8000000.ethernet-tx208:45:282
2131670993728,4cyclictest0-21swapper/211:00:282
2131670993727,6cyclictest0-21swapper/207:40:012
2131670993726,6cyclictest0-21swapper/207:30:282
2131670993723,12cyclictest311ktimers/210:30:132
213167099371,29cyclictest2187635-21/usr/sbin/munin12:35:172
2131669993725,8cyclictest0-21swapper/109:25:011
2131670993629,4cyclictest167550irq/134-8000000.ethernet-tx211:20:222
2131669993628,5cyclictest231ktimers/107:40:271
2131669993626,7cyclictest231ktimers/111:45:111
2131669993626,6cyclictest2143040-21grep08:10:301
2131669993626,5cyclictest0-21swapper/109:50:281
2131669993625,7cyclictest0-21swapper/109:45:221
2131669993625,6cyclictest0-21swapper/110:55:341
2131669993621,8cyclictest2158424-21grep09:40:281
2131669993621,8cyclictest2158424-21grep09:40:281
2131669993620,9cyclictest0-21swapper/109:05:261
213167099355,2cyclictest0-21swapper/209:20:022
2131670993528,4cyclictest167550irq/134-8000000.ethernet-tx207:45:282
2131670993528,4cyclictest167550irq/134-8000000.ethernet-tx207:45:282
2131670993525,6cyclictest2169359-21grep10:45:302
2131670993523,8cyclictest0-21swapper/211:50:292
213167099351,31cyclictest0-21swapper/208:50:202
2131669993528,4cyclictest231ktimers/107:10:301
2131669993528,4cyclictest231ktimers/107:10:301
2131669993528,4cyclictest1261-21rs:main1
2131669993527,5cyclictest231ktimers/112:15:151
2131669993525,6cyclictest0-21swapper/112:20:191
2131669993525,6cyclictest0-21swapper/112:20:191
2131669993522,6cyclictest0-21swapper/109:25:271
2131669993521,10cyclictest0-21swapper/107:30:281
2131669993520,8cyclictest0-21swapper/108:50:251
2131669993519,8cyclictest0-21swapper/107:25:281
2131669993514,15cyclictest231ktimers/110:30:201
213166999351,28cyclictest0-21swapper/109:30:171
2131670993427,5cyclictest2163512-21cron10:15:002
2131670993426,4cyclictest0-21swapper/208:40:272
2131670993422,8cyclictest0-21swapper/212:10:292
213167099341,4cyclictest0-21swapper/210:30:012
2131669993424,7cyclictest231ktimers/110:10:191
2131669993421,7cyclictest0-21swapper/111:00:291
2131669993420,8cyclictest231ktimers/110:05:291
2131669993419,9cyclictest0-21swapper/111:10:271
2131670993326,4cyclictest167550irq/134-8000000.ethernet-tx210:00:232
2131670993325,5cyclictest0-21swapper/209:39:592
2131670993322,6cyclictest0-21swapper/209:30:272
2131670993322,5cyclictest0-21swapper/208:20:292
2131670993322,5cyclictest0-21swapper/208:20:292
2131669993324,6cyclictest231ktimers/108:25:291
2131669993324,4cyclictest2168446-21mailstats10:40:281
2131669993322,7cyclictest0-21swapper/108:05:281
2131669993321,8cyclictest0-21swapper/110:15:291
2131669993317,11cyclictest2170372-21cat10:55:021
213166999331,4cyclictest0-21swapper/111:25:011
2131669993310,12cyclictest2147594-21/usr/sbin/munin08:40:191
2131670993224,4cyclictest0-21swapper/208:15:282
2131670993222,6cyclictest2154966-21grep09:20:262
2131670993221,8cyclictest0-21swapper/209:45:012
2131670993221,8cyclictest0-21swapper/209:45:012
2131670993221,7cyclictest0-21swapper/209:10:012
213167099321,29cyclictest0-21swapper/210:35:292
213167099321,29cyclictest0-21swapper/210:05:242
2131670993210,5cyclictest832-21systemd-network12:20:162
2131670993210,5cyclictest832-21systemd-network12:20:162
2131669993218,10cyclictest0-21swapper/111:50:281
2131670993124,3cyclictest0-21swapper/209:50:012
213167099312,25cyclictest0-21swapper/210:15:182
2131670993122,4cyclictest0-21swapper/209:15:012
2131670993121,8cyclictest2179845-21cut11:50:002
2131670993121,7cyclictest0-21swapper/209:50:202
2131670993120,7cyclictest0-21swapper/212:25:282
2131670993118,9cyclictest0-21swapper/207:50:212
213167099311,27cyclictest2155554-21kernelversion09:25:222
2131669993123,5cyclictest231ktimers/109:55:201
2131669993116,8cyclictest231ktimers/107:20:051
213166999311,24cyclictest0-21swapper/109:35:171
2131670993022,5cyclictest14-21rcu_preempt07:55:102
2131670993022,5cyclictest14-21rcu_preempt07:55:102
2131670993022,4cyclictest0-21swapper/212:05:182
2131670993021,5cyclictest0-21swapper/208:55:302
2131670993015,11cyclictest0-21swapper/208:30:212
2131669993023,4cyclictest14-21rcu_preempt10:05:011
2131669993018,9cyclictest0-21swapper/107:45:261
2131669993018,9cyclictest0-21swapper/107:45:251
2131670992919,5cyclictest274-21systemd-journal12:05:002
213167099291,2cyclictest0-21swapper/208:40:012
213167099291,26cyclictest0-21swapper/207:20:162
2131670992821,4cyclictest0-21swapper/211:40:132
2131670992821,4cyclictest0-21swapper/211:40:132
2131670992819,5cyclictest0-21swapper/207:40:212
2131670992718,6cyclictest0-21swapper/208:05:282
2131670992718,5cyclictest0-21swapper/208:10:312
2131670992718,4cyclictest0-21swapper/208:00:162
2131669992619,5cyclictest231ktimers/107:10:011
2131670992517,5cyclictest0-21swapper/212:30:282
2131670992416,5cyclictest311ktimers/209:00:202
2131670992414,7cyclictest30-21rcuc/211:10:122
2131670992115,2cyclictest0-21swapper/207:10:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional