You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-03-20 - 17:21

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  

x86 Intel Atom E3825 @1333 MHz, Linux 4.9.47-rt37 (Profile)

Latency plot of shadow in rack #9, slot #4
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the alll highest latencies:
System rack9slot4s.osadl.org (updated Mon Mar 20, 2023 12:59:01)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
315162564518,31sleep10-21swapper/107:20:341
316242562517,30sleep00-21swapper/007:21:410
3199799232210,16cyclictest1412-21munin-run07:30:000
78922640,8sleep03199799cyclictest10:35:110
115122461,33sleep011577-21cut12:10:180
31997994224,6cyclictest3-21ksoftirqd/010:00:020
31997993923,6cyclictest3-21ksoftirqd/009:38:480
31997993923,6cyclictest3-21ksoftirqd/008:20:100
31997993822,5cyclictest3-21ksoftirqd/011:31:130
31997993821,6cyclictest3-21ksoftirqd/008:48:190
31997993821,6cyclictest3-21ksoftirqd/007:35:120
31997993721,6cyclictest3-21ksoftirqd/012:24:260
31997993721,5cyclictest3-21ksoftirqd/012:40:050
31997993721,5cyclictest3-21ksoftirqd/009:58:050
31997993620,6cyclictest3-21ksoftirqd/009:32:080
31997993620,6cyclictest3-21ksoftirqd/008:54:270
31997993620,6cyclictest3-21ksoftirqd/008:54:260
31997993620,5cyclictest3-21ksoftirqd/010:06:230
31997993619,6cyclictest3-21ksoftirqd/008:38:160
31998993524,8cyclictest4604-21fw_packets10:25:211
31997993520,5cyclictest3-21ksoftirqd/008:15:370
31997993520,4cyclictest3-21ksoftirqd/012:34:500
31997993520,4cyclictest3-21ksoftirqd/008:00:430
31997993519,6cyclictest3-21ksoftirqd/011:02:520
31997993519,6cyclictest3-21ksoftirqd/010:30:130
31997993519,6cyclictest3-21ksoftirqd/010:19:550
31997993519,5cyclictest3-21ksoftirqd/011:19:430
31997993519,5cyclictest3-21ksoftirqd/007:57:280
31997993518,6cyclictest3-21ksoftirqd/007:43:300
31997993518,5cyclictest3-21ksoftirqd/009:48:240
3199799351,21cyclictest12527-21/usr/sbin/munin10:45:330
87532341,4sleep18757-21ntp_states10:35:381
68332341,29sleep16837-21ntp_states10:30:311
3199899343,27cyclictest2780-21smartctl07:30:201
31997993419,5cyclictest3-21ksoftirqd/012:54:400
31997993419,5cyclictest3-21ksoftirqd/012:08:350
31997993419,5cyclictest3-21ksoftirqd/010:44:000
31997993419,5cyclictest3-21ksoftirqd/008:30:590
31997993418,6cyclictest3-21ksoftirqd/008:44:490
3199799341,5cyclictest0-21swapper/009:22:020
117492340,5sleep0111rcuc/007:55:010
3199899334,4cyclictest81rcu_preempt11:25:051
3199799336,16cyclictest105-21usb-storage10:20:170
31997993318,5cyclictest3-21ksoftirqd/012:46:470
31997993318,5cyclictest3-21ksoftirqd/012:15:420
31997993318,4cyclictest3-21ksoftirqd/009:42:470
31997993317,6cyclictest3-21ksoftirqd/009:10:510
56402321,5sleep10-21swapper/107:36:231
321462321,27sleep132142-21apt-get07:25:131
3199899321,4cyclictest81rcu_preempt09:52:011
31997993218,5cyclictest3-21ksoftirqd/007:31:310
31997993217,5cyclictest3-21ksoftirqd/011:54:310
31997993210,7cyclictest3-21ksoftirqd/011:25:050
31997993210,12cyclictest31995-21cyclictest10:10:560
262292321,4sleep126309-21python08:30:301
219242321,27sleep122007-21memory09:45:261
68262311,26sleep16877-21memory09:05:291
3199899318,4cyclictest81rcu_preempt12:17:061
3199899315,4cyclictest81rcu_preempt09:23:321
3199899314,10cyclictest0-21swapper/110:13:171
31998993118,7cyclictest21252-21diskstats12:35:251
3199799317,8cyclictest3-21ksoftirqd/009:52:010
31997993115,6cyclictest3-21ksoftirqd/012:29:070
31997993115,5cyclictest3-21ksoftirqd/011:09:050
31997993115,5cyclictest3-21ksoftirqd/010:28:530
31997993114,4cyclictest4144-21sendmail-msp08:59:590
31997993112,7cyclictest21207-21wget11:10:270
3199799311,14cyclictest17692-21latency_hist08:10:000
3199799311,11cyclictest19460-21apt08:15:000
242972311,5sleep10-21swapper/112:42:221
137592311,3sleep113763-21apt-get09:25:121
3199899305,4cyclictest81rcu_preempt11:16:101
3199899305,4cyclictest81rcu_preempt07:42:451
3199899303,4cyclictest81rcu_preempt09:12:431
3199899302,22cyclictest7535-21apt-get12:00:171
3199799306,8cyclictest25700-21grep11:20:270
3199799302,17cyclictest0-21swapper/011:58:180
31997993015,4cyclictest3-21ksoftirqd/009:15:470
31997993014,5cyclictest1308-21taskset11:40:340
234032301,3sleep023407-21apt-get08:25:130
229582301,5sleep10-21swapper/108:22:071
196072301,5sleep10-21swapper/108:15:081
36222291,5sleep10-21swapper/110:21:461
33562291,5sleep00-21swapper/011:45:570
326432291,5sleep10-21swapper/108:49:441
3199899295,3cyclictest81rcu_preempt10:05:591
3199899294,4cyclictest81rcu_preempt11:37:251
3199899294,4cyclictest81rcu_preempt11:12:351
3199899294,4cyclictest81rcu_preempt09:31:241
3199899293,4cyclictest81rcu_preempt12:21:091
3199899293,4cyclictest81rcu_preempt11:04:431
3199899291,4cyclictest81rcu_preempt11:54:481
3199799294,8cyclictest6723-21idleruntime09:05:250
318902291,5sleep00-21swapper/011:35:470
250342291,5sleep10-21swapper/108:29:091
151772291,5sleep00-21swapper/010:53:570
137032291,5sleep10-21swapper/107:59:121
95332281,5sleep00-21swapper/007:47:300
3199899284,4cyclictest81rcu_preempt10:47:071
3199899283,4cyclictest81rcu_preempt12:11:051
3199899283,4cyclictest81rcu_preempt08:08:301
3199899282,12cyclictest0-21swapper/110:55:241
31998992819,6cyclictest22-21ksoftirqd/111:06:261
3199899281,7cyclictest10296-21fschecks_time10:40:211
3199899281,4cyclictest81rcu_preempt08:56:491
3199899281,13cyclictest9968-21apt-get07:50:181
3199799283,17cyclictest13763-21apt-get09:25:120
3199799281,8cyclictest9251-21runrttasks12:02:580
31997992811,13cyclictest16248-21netstat10:55:290
262342281,5sleep10-21swapper/111:20:411
260502281,5sleep10-21swapper/112:45:411
18172281,5sleep10-21swapper/110:19:141
154502281,5sleep10-21swapper/108:02:341
115562281,5sleep10-21swapper/109:18:391
56562271,5sleep10-21swapper/109:01:211
34082271,5sleep10-21swapper/111:46:291
3199899276,4cyclictest41250irq/92-i91509:40:191
3199899273,18cyclictest29374-21diskstats08:40:161
3199899273,18cyclictest1079-21memory08:50:241
3199899273,18cyclictest1079-21memory08:50:241
3199899273,10cyclictest26658-21apt-get12:50:161
3199899272,5cyclictest0-21swapper/110:03:271
31998992718,5cyclictest22-21ksoftirqd/109:37:341
31998992716,7cyclictest22-21ksoftirqd/111:42:251
3199799274,17cyclictest4593-21diskstats09:00:180
3199799274,16cyclictest22221-21users12:35:420
284662271,5sleep10-21swapper/108:35:401
264542271,5sleep10-21swapper/109:56:331
205792271,5sleep10-21swapper/112:33:031
194042271,5sleep10-21swapper/108:14:351
72542261,5sleep10-21swapper/111:57:161
3199899264,18cyclictest0-21swapper/112:29:401
3199899262,20cyclictest29557-21ntp_states11:30:371
31998992617,5cyclictest22-21ksoftirqd/107:49:441
31998992616,6cyclictest10304-21sort12:05:311
3199899256,9cyclictest0-21swapper/110:50:341
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional