You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-03-23 - 06:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  

x86 Intel Atom E3825 @1333 MHz, Linux 4.9.47-rt37 (Profile)

Latency plot of shadow in rack #9, slot #4
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack9slot4s.osadl.org (updated Thu Mar 23, 2023 00:59:01)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
122499912121180,28cyclictest14120-21idleruntime-cro19:30:001
119152483437,31sleep00-21swapper/019:22:020
104822459371,29sleep10-21swapper/119:20:131
2967121580,2sleep129670-21apt23:00:171
12248997958,15cyclictest14119-21cron19:30:000
4532780,7sleep10-21swapper/120:15:311
120612770,4sleep041ktimersoftd/020:45:410
188982690,4sleep0111rcuc/021:05:220
12249996413,16cyclictest5351-21/usr/sbin/munin20:30:151
12249996210,13cyclictest1390-21taskset00:35:251
12249996111,15cyclictest10705-21taskset20:44:261
12249996012,16cyclictest13395-21cat20:50:211
12249996011,15cyclictest8717-21aten_r4power_cu23:30:291
12249996011,15cyclictest8717-21aten_r4power_cu23:30:281
12249996011,15cyclictest17051-21grep22:25:251
12249996011,15cyclictest11674-21timerandwakeup23:35:391
12249996010,17cyclictest17294-21taskset19:35:201
1224999599,17cyclictest5367-21df21:55:191
12249995911,16cyclictest3144-21cpuspeed_turbos23:15:201
12249995911,15cyclictest31056-21chrt00:28:571
12249995910,17cyclictest25652-21taskset00:15:061
12249995910,15cyclictest21943-21chrt19:46:171
1224999586,18cyclictest4701-21chrt20:25:591
12249995810,16cyclictest7909-21runrttasks00:50:341
12249995810,16cyclictest23629-21chrt21:15:561
12249995810,16cyclictest20579-21cat21:10:151
12249995810,15cyclictest20024-21/usr/sbin/munin00:00:301
1224899582,19cyclictest11610-21ntp_states22:10:300
1224999579,16cyclictest12995-21wget23:40:271
1224999578,15cyclictest8621-21runrttasks20:37:331
12249995711,16cyclictest19078-21cat21:05:231
12249995711,15cyclictest11380-21iostat_ios22:10:261
1224999569,17cyclictest15712-21runrttasks23:46:231
1224999569,15cyclictest2635-21seq21:46:531
1224999569,15cyclictest16025-21sshd20:55:401
12249995614,18cyclictest3289-21runrttasks19:40:201
12249995611,16cyclictest4884-21dpkg23:20:141
12249995610,15cyclictest19549-21seq23:57:061
1224999558,14cyclictest12953-21sh22:15:201
1224999557,17cyclictest25185-21chrt00:11:491
12249995511,15cyclictest2779-21snmpd23:07:411
12249995511,15cyclictest24723-21which21:20:251
12249995510,15cyclictest5575-21open_files00:45:321
1224999546,17cyclictest29568-21chrt21:33:431
12249995411,16cyclictest24354-21dpkg19:55:111
12249995310,15cyclictest28134-21/usr/sbin/munin20:05:161
12249995310,14cyclictest28051-21df22:55:191
1224999529,15cyclictest1693-21df_inode20:20:271
1224999528,15cyclictest4076-21uptime00:40:371
12249995210,19cyclictest18334-21/usr/sbin/munin22:30:271
12249995210,18cyclictest8889-21/usr/sbin/munin22:05:271
1224999518,27cyclictest7517-21fw_packets22:00:301
1224999509,15cyclictest32621-21awk21:40:351
1224999508,15cyclictest7082-21forks23:25:221
12249995033,9cyclictest22473-21apt-get19:50:131
12249995015,14cyclictest26354-21fschecks_time22:50:241
12249995014,13cyclictest16669-21cut21:00:161
1224999497,13cyclictest0-21swapper/100:06:241
1224999496,14cyclictest0-21swapper/122:36:391
12249994910,14cyclictest2265-21/usr/sbin/munin23:10:401
1224999486,15cyclictest4562-21runrttasks21:51:271
12249994813,14cyclictest3289-21runrttasks00:32:121
12249994813,13cyclictest3289-21runrttasks22:45:191
12249994810,16cyclictest3289-21runrttasks20:48:381
1224999477,14cyclictest2779-21snmpd20:11:071
12249994710,15cyclictest3289-21runrttasks21:39:391
313242460,5sleep00-21swapper/000:30:010
143862460,7sleep01224899cyclictest23:45:140
1224999468,13cyclictest14568-21/usr/sbin/munin22:20:221
12249994613,13cyclictest17601-21sshd23:51:261
12249994612,14cyclictest26546-21sh20:00:171
12249994612,14cyclictest26546-21sh20:00:161
1224999457,10cyclictest15517-21iostat19:30:221
1224999448,11cyclictest22720-21iostat_ios22:40:241
1224999446,16cyclictest27585-21/usr/sbin/munin00:20:181
12249994414,6cyclictest41250irq/92-i91521:25:281
1224899414,11cyclictest81rcu_preempt22:18:150
1224899384,10cyclictest81rcu_preempt21:37:350
300232361,31sleep030018-21apt-get20:10:140
1224899362,12cyclictest81rcu_preempt22:46:320
1224899361,14cyclictest0-21swapper/021:46:240
1224899361,12cyclictest0-21swapper/023:56:350
1224899353,5cyclictest81rcu_preempt00:37:090
1224899351,14cyclictest28987-21awk21:30:400
33372341,4sleep03403-21iostat00:40:260
214282341,4sleep021432-21ntp_states19:45:290
1224899344,4cyclictest81rcu_preempt19:55:280
1224899341,20cyclictest20494-21cut21:10:140
1224899341,12cyclictest0-21swapper/023:50:010
1224899338,12cyclictest8476-21cron23:30:000
1224899331,4cyclictest81rcu_preempt00:19:480
218632321,5sleep00-21swapper/022:39:240
1224899326,22cyclictest4900-21cron21:55:000
1224899323,15cyclictest13869-21python20:50:310
1224899321,4cyclictest81rcu_preempt22:55:130
1224899321,4cyclictest81rcu_preempt22:26:240
1224899321,4cyclictest81rcu_preempt00:53:020
1224899321,19cyclictest4993-21taskset20:29:010
1224899321,19cyclictest28019-21cat20:05:010
1224899321,19cyclictest28019-21cat20:05:000
1224899317,13cyclictest4495-21pidof23:18:000
1224899311,4cyclictest81rcu_preempt23:41:540
1224899311,4cyclictest81rcu_preempt22:41:110
1224899311,4cyclictest81rcu_preempt21:42:180
1224899311,4cyclictest81rcu_preempt21:17:260
1224899311,4cyclictest81rcu_preempt00:08:570
1224899311,4cyclictest81rcu_preempt00:03:410
12248993114,5cyclictest5331-21irqstats00:45:270
1224899311,3cyclictest81rcu_preempt22:51:080
1224899311,18cyclictest6803-21apt-get22:00:140
30542301,5sleep00-21swapper/020:23:340
1224899302,14cyclictest31900-21apt-get20:15:130
1224899301,4cyclictest81rcu_preempt23:05:500
1224899301,4cyclictest81rcu_preempt22:08:030
1224899301,4cyclictest81rcu_preempt21:55:380
1224899301,4cyclictest81rcu_preempt00:26:050
12248993011,6cyclictest18451-21grep22:30:200
12248993011,5cyclictest27666-21taskset21:28:300
1224899300,4cyclictest81rcu_preempt19:43:290
161712291,5sleep00-21swapper/020:57:100
1224899296,10cyclictest3637-21packagekitd19:50:190
1224899295,5cyclictest81rcu_preempt00:20:200
1224899293,7cyclictest974-21apt-get23:10:180
1224899292,14cyclictest5179-21df_inode23:20:200
12248992919,6cyclictest5662-21fschecks_count20:30:220
1224899291,4cyclictest81rcu_preempt19:31:410
1224899291,14cyclictest24221-21apt-get21:20:170
179342281,5sleep00-21swapper/021:00:430
160022281,5sleep00-21swapper/022:22:090
12248992817,7cyclictest11327-21needreboot23:35:320
1224899281,5cyclictest0-21swapper/020:35:080
1224899281,15cyclictest29264-21taskset20:05:320
252202271,5sleep00-21swapper/000:12:090
185012271,5sleep00-21swapper/019:39:550
1224899271,17cyclictest0-21swapper/023:00:270
100242271,5sleep00-21swapper/023:31:140
100242271,5sleep00-21swapper/023:31:130
1224899261,13cyclictest9972-21processes20:40:290
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional