You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2021-10-21 - 20:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  

Intel(R) Atom(TM) CPU E3825 @ 1.33GHz, Linux 4.9.47-rt37 (Profile)

Latency plot of shadow in rack #9, slot #4
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the alll highest latencies:
System rack9slot4s.osadl.org (updated Thu Oct 21, 2021 12:59:01)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
308409913071241,46cyclictest432-21date07:30:010
308419911821160,16cyclictest32712-21idleruntime-cro07:30:011
305022739641,30sleep10-21swapper/107:22:011
299532558510,33sleep00-21swapper/007:20:390
9482590,9sleep13084199cyclictest07:30:051
3084099382,16cyclictest8830-21apt-get07:50:140
3084099359,13cyclictest20462-21if_err_eth011:10:210
3084099358,13cyclictest14012-21grep08:00:350
3084099353,18cyclictest0-21swapper/010:17:190
30840993513,7cyclictest1921-21ntp_states07:30:280
3084099351,22cyclictest23171-21taskset11:16:380
3084099351,10cyclictest0-21swapper/010:05:290
147422351,29sleep114746-21diskstats09:30:171
126252350,4sleep1201rcuc/110:50:151
3084199344,13cyclictest992-21diskstats11:45:191
3084099348,11cyclictest1309-21cut10:20:220
3084099342,20cyclictest22420-21ntp_states12:40:300
3084099341,20cyclictest6325-21munin-run12:00:010
224212341,28sleep122425-21ntp_states12:40:311
186102341,4sleep018736-21df08:15:180
50162331,3sleep05011-21apt-get09:05:140
3084199334,12cyclictest16612-21memory12:25:281
30841993318,12cyclictest22-21ksoftirqd/111:34:591
3084099332,4cyclictest81rcu_preempt10:56:270
150172331,28sleep114232-21/usr/sbin/munin10:55:341
34182321,27sleep13425-21diskstats07:35:191
3084199323,25cyclictest22392-21sh11:15:261
30841993223,6cyclictest3298-21runrttasks10:01:541
3084099323,15cyclictest22005-21apt-get09:50:160
182812320,4sleep0111rcuc/011:05:160
3084199316,4cyclictest81rcu_preempt08:19:551
3084199315,4cyclictest81rcu_preempt07:53:061
30841993121,6cyclictest7385-21memory10:35:291
3084099312,4cyclictest81rcu_preempt11:24:250
3084099311,4cyclictest81rcu_preempt11:46:450
64332301,6sleep00-21swapper/007:41:490
42252301,5sleep00-21swapper/010:25:460
311602301,5sleep00-21swapper/010:13:370
3084199309,6cyclictest14803-21netstat12:20:251
3084199305,4cyclictest81rcu_preempt12:02:181
3084199305,4cyclictest81rcu_preempt11:01:061
30841993016,8cyclictest20817-21memory09:45:291
3084199301,5cyclictest3298-21runrttasks12:46:461
30841993015,12cyclictest22-21ksoftirqd/109:01:391
30841993012,5cyclictest8971-21latency_hist07:50:011
3084099302,18cyclictest2205-21dbus-daemon08:21:010
3084099302,15cyclictest0-21swapper/009:44:530
3084099301,4cyclictest81rcu_preempt12:38:530
3084099301,4cyclictest81rcu_preempt12:05:570
3084099301,4cyclictest81rcu_preempt08:34:120
3084099301,4cyclictest81rcu_preempt08:27:480
30840993011,6cyclictest12908-21tune2fs09:25:210
3084099300,4cyclictest81rcu_preempt12:53:230
269552301,5sleep10-21swapper/112:53:371
171422301,25sleep017176-21ntp_states09:35:340
155342301,5sleep00-21swapper/012:22:480
90502291,5sleep19085-21cat12:05:261
80052291,5sleep00-21swapper/010:35:570
7092291,5sleep00-21swapper/008:52:460
3084199295,4cyclictest81rcu_preempt08:30:071
3084199294,4cyclictest81rcu_preempt08:28:111
3084199294,18cyclictest18102-21diskstats12:30:181
3084199293,19cyclictest29786-21apt-get08:45:141
3084199292,14cyclictest5707-21memory09:05:291
3084199291,4cyclictest81rcu_preempt09:38:071
3084199291,4cyclictest81rcu_preempt07:41:341
3084099299,5cyclictest47242chrt09:03:510
3084099293,14cyclictest28397-21grep08:40:220
3084099292,4cyclictest81rcu_preempt12:48:240
3084099292,15cyclictest4151-21packagekitd11:36:530
3084099292,15cyclictest11428-21/usr/sbin/munin07:55:230
3084099292,13cyclictest16613-21memory12:25:270
30840992911,6cyclictest26427-21ntpdc11:25:350
3084099291,15cyclictest3538-21/usr/sbin/munin11:50:290
288942291,5sleep10-21swapper/110:05:501
255262291,5sleep00-21swapper/009:59:030
250812291,5sleep10-21swapper/111:21:431
201032291,5sleep00-21swapper/009:45:050
194832291,5sleep10-21swapper/111:07:261
310262281,5sleep10-21swapper/111:39:301
3084199286,4cyclictest81rcu_preempt12:37:571
3084199286,4cyclictest81rcu_preempt10:22:391
3084199284,3cyclictest81rcu_preempt11:55:191
3084199281,4cyclictest81rcu_preempt10:34:051
3084099289,7cyclictest0-21swapper/010:52:050
3084099284,18cyclictest10551-21diskstats12:10:200
3084099284,17cyclictest29786-21apt-get08:45:140
3084099283,13cyclictest0-21swapper/010:00:070
3084099282,14cyclictest0-21swapper/009:10:070
3084099282,13cyclictest31880-21iostat_ios11:40:280
30840992817,6cyclictest3424-21diskstats07:35:200
3084099281,23cyclictest12663-21grep12:15:240
3084099281,13cyclictest11551-21open_inodes09:20:300
275332281,5sleep10-21swapper/108:38:121
27132281,5sleep00-21swapper/008:58:020
253742281,5sleep10-21swapper/109:57:301
138232281,5sleep10-21swapper/112:19:501
122432281,5sleep10-21swapper/109:23:271
118802281,5sleep10-21swapper/110:46:531
103512281,5sleep10-21swapper/109:18:351
99372271,5sleep10-21swapper/110:41:151
43592271,5sleep10-21swapper/110:27:101
310312271,5sleep10-21swapper/110:12:151
3084199277,4cyclictest81rcu_preempt08:08:241
3084199274,3cyclictest81rcu_preempt08:54:361
30841992720,4cyclictest223042sleep109:50:261
30841992717,6cyclictest3653-21wc11:50:321
3084199271,4cyclictest81rcu_preempt07:58:301
30841992710,5cyclictest203592grep11:10:191
3084099272,20cyclictest6865-21tune2fs12:00:210
3084099272,13cyclictest27783-21latency_hist08:40:000
3084099271,17cyclictest0-21swapper/007:45:320
3084099271,14cyclictest28160-21latency11:30:260
3084099271,13cyclictest16666-21aten_r4power_vo08:10:160
3084099271,12cyclictest10238-21runrttasks10:44:220
270442271,5sleep10-21swapper/111:27:181
218372271,5sleep10-21swapper/108:22:411
165502271,5sleep10-21swapper/108:10:061
153962271,23sleep015468-21smart_sda09:30:300
139262271,5sleep10-21swapper/109:26:201
117832271,5sleep10-21swapper/112:13:101
101782271,5sleep00-21swapper/009:16:470
3402261,4sleep10-21swapper/111:43:111
3084199262,9cyclictest18542-21df_inode09:40:251
30841992616,6cyclictest2372-21sed08:55:341
30841992615,7cyclictest14516-21cron08:05:001
3084099262,12cyclictest15287-21iostat_ios08:05:260
30840992615,7cyclictest11208-21cat10:45:270
3084099261,13cyclictest6364-21runrttasks10:33:260
294742261,5sleep10-21swapper/108:43:361
3084199253,19cyclictest0-21swapper/110:15:241
3084099258,5cyclictest40750irq/92-i91512:30:170
3084099251,15cyclictest0-21swapper/011:02:510
3084199242,8cyclictest7188-21df09:10:181
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional