You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-05-31 - 02:30

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  

x86 Intel Atom E3825 @1333 MHz, Linux 4.9.47-rt37 (Profile)

Latency plot of shadow in rack #9, slot #4
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack9slot4s.osadl.org (updated Tue May 30, 2023 12:59:01)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
213289911491122,23cyclictest23199-21/usr/sbin/munin07:30:011
201732735687,32sleep10-21swapper/107:20:281
209712563517,31sleep00-21swapper/007:21:520
2132799231205,19cyclictest23323-21munin-node07:30:010
34232900,6sleep02132799cyclictest09:25:280
2132899621,17cyclictest549-21/usr/sbin/munin12:10:291
2132899541,15cyclictest2531-21users07:55:411
2132899521,19cyclictest16546-21entropy10:00:231
2132899511,15cyclictest9185-21find12:30:351
2132899502,14cyclictest25767-21sh11:50:271
2132899501,18cyclictest17736-21taskset08:35:421
2132899501,18cyclictest11461-21chrt12:36:191
2132899501,17cyclictest4351-21chrt10:53:381
2132899501,17cyclictest1072-21df_inode10:45:211
2132899501,16cyclictest8899-21chrt08:15:181
2132899501,16cyclictest30053-21grep10:35:261
2132899501,15cyclictest5627-21expr10:55:371
2132899501,15cyclictest27954-21fschecks_count10:30:261
2132899501,15cyclictest12134-21cat11:15:001
2132899493,15cyclictest0-21swapper/107:50:461
2132899491,17cyclictest7687-21chrt12:26:121
2132899491,17cyclictest6098-21chrt12:24:161
2132899491,17cyclictest31364-21chrt07:47:201
2132899491,17cyclictest21626-21chrt08:46:411
2132899491,15cyclictest32436-21chrt12:05:501
2132899491,15cyclictest16358-21if_err_eth012:50:211
2132899491,14cyclictest20082-21sh11:35:241
2132899491,13cyclictest23157-21chrt11:42:331
2132899482,15cyclictest15991-21sendmail-mta11:25:041
2132899482,14cyclictest8981-21df_inode09:40:221
2132899481,24cyclictest24313-21diskstats08:55:241
2132899481,17cyclictest27958-21cpu09:05:201
2132899481,16cyclictest8351-21chrt09:38:361
2132899481,15cyclictest2769-21sh12:15:201
2132899481,12cyclictest26529-21ntp_states10:25:341
182942480,5sleep021325-21cyclictest08:40:000
2132899471,15cyclictest27776-21chrt07:39:311
2132899471,15cyclictest26536-21iostat_ios09:00:361
2132899471,15cyclictest17896-21/usr/sbin/munin11:30:381
2132899471,15cyclictest15721-21chrt11:23:541
2132899471,14cyclictest15182-21sh09:55:411
2132899461,17cyclictest30101-21awk12:00:371
2132899461,15cyclictest28553-21smartctl07:40:331
2132899451,23cyclictest32626-21users10:40:381
2132899451,15cyclictest12216-21/usr/sbin/munin11:15:251
2132899441,9cyclictest3289-21runrttasks09:27:461
2132899441,18cyclictest5732-21iostat_ios08:05:271
2132899441,15cyclictest7523-21find11:00:361
2132899441,15cyclictest18944-21iostat08:40:251
2132899441,15cyclictest10979-21cat08:20:211
2132899441,13cyclictest559-21chrt09:17:201
2132899441,13cyclictest559-21chrt09:17:191
2132899439,14cyclictest0-21swapper/109:30:171
2132899431,16cyclictest3751-21http_loadtime08:00:251
2132899431,16cyclictest24029-21/usr/sbin/munin07:30:341
2132899431,16cyclictest12681-21/usr/sbin/munin08:25:351
2132899431,14cyclictest23806-21/usr/sbin/munin10:20:251
2132899422,28cyclictest0-21swapper/109:48:501
2132899422,14cyclictest0-21swapper/109:50:341
2132899421,15cyclictest2779-21snmpd12:49:061
2132899421,15cyclictest2779-21snmpd12:49:051
2132899421,14cyclictest2779-21snmpd12:41:351
2132899412,28cyclictest0-21swapper/108:53:391
2132899412,27cyclictest0-21swapper/111:59:151
2132899412,14cyclictest0-21swapper/108:33:341
2132899411,21cyclictest1013-21/usr/sbin/munin09:20:361
2132899411,17cyclictest18411-21cstates10:05:261
2132899411,15cyclictest7008-21apt-get08:10:141
2132899401,17cyclictest24731-21sed11:45:311
2132899401,17cyclictest22139-21cstates10:15:201
2132899401,11cyclictest21243-21awk10:10:411
2132899391,18cyclictest30058-21fschecks_count09:10:261
2132899381,14cyclictest8457-21apt-get11:05:221
21327993616,5cyclictest12219-21apt-get11:15:170
2132799354,10cyclictest23568-21apt-get11:45:120
65622340,5sleep06563-21cpuspeed_turbos12:25:210
51362341,29sleep05172-21aten_r4power_en08:05:190
2132799342,19cyclictest25076-21users10:20:370
309512331,4sleep030974-21munin-run12:05:000
2132799337,13cyclictest81rcu_preempt08:45:180
2132799332,16cyclictest28011-21mandb07:40:040
2132799331,3cyclictest81rcu_preempt09:00:020
2132799325,14cyclictest32767-21taskset10:41:580
2132799321,4cyclictest81rcu_preempt11:43:160
2132799321,4cyclictest81rcu_preempt10:58:330
2132799321,4cyclictest81rcu_preempt09:48:460
2132799321,4cyclictest81rcu_preempt08:33:180
21327993210,6cyclictest13524-21runrttasks08:25:350
2132799312,25cyclictest1225-21apt-get07:55:180
2132799312,16cyclictest16613-21/usr/sbin/munin12:50:270
2132799312,14cyclictest18350-21apt-get08:40:160
21327993115,11cyclictest13648-21grep09:50:340
2132799311,4cyclictest81rcu_preempt12:31:360
2132799311,4cyclictest81rcu_preempt12:18:350
2132799311,4cyclictest81rcu_preempt10:30:080
2132799311,4cyclictest81rcu_preempt09:16:150
2132799311,4cyclictest81rcu_preempt09:16:140
2132799311,4cyclictest81rcu_preempt08:59:270
2132799311,4cyclictest81rcu_preempt07:33:410
21327993114,5cyclictest17075-21ntp_states10:00:340
2132799311,26cyclictest19601-21cron11:34:590
21327993110,10cyclictest0-21swapper/012:35:230
2132799305,21cyclictest0-21swapper/010:35:300
2132799304,6cyclictest153002find09:55:440
2132799301,4cyclictest81rcu_preempt07:45:270
2132799301,17cyclictest21630-21latency_hist11:40:010
2132799300,4cyclictest81rcu_preempt11:26:500
21562291,9sleep00-21swapper/010:46:260
2132799299,7cyclictest27159-21unixbench_multi07:35:350
2132799292,4cyclictest81rcu_preempt09:20:380
2132799292,23cyclictest29072-21munin-run12:00:010
21327992918,7cyclictest26317-21/usr/sbin/munin10:25:310
2132799291,4cyclictest81rcu_preempt11:20:250
2132799291,4cyclictest81rcu_preempt08:15:210
21327992913,4cyclictest31133-21apt-get12:05:170
21327992912,6cyclictest11096-21df_inode08:20:240
21327992911,6cyclictest22974-21runrttasks08:50:400
2132799291,15cyclictest7845-21ntp_states08:10:310
2132799291,15cyclictest30477-21netstat09:10:330
2132799291,14cyclictest3730-21fgrep10:50:330
2132799291,13cyclictest23514-21chrt10:19:010
67722281,5sleep00-21swapper/009:35:100
2132799284,9cyclictest81rcu_preempt10:05:220
2132799282,13cyclictest8541-21/usr/sbin/munin11:05:250
2132799282,13cyclictest2226-21dbus-daemon12:43:480
2132799282,13cyclictest10632-21df11:10:270
2132799282,13cyclictest0-21swapper/008:00:180
2132799282,12cyclictest3637-21packagekitd11:03:170
2132799281,8cyclictest0-21swapper/012:20:310
21327992811,14cyclictest4923-21basename09:30:190
2132799280,3cyclictest81rcu_preempt09:08:590
2132799275,11cyclictest549-21/usr/sbin/munin12:10:180
21327992711,5cyclictest0-21swapper/010:10:240
2132799271,13cyclictest8763-21basename09:40:180
154702271,5sleep00-21swapper/012:48:230
154702271,5sleep00-21swapper/012:48:230
2132799261,14cyclictest25503-21wget11:50:240
2132799261,13cyclictest31783-21chrt07:50:060
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional