You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-10-01 - 01:32

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack9slot4s.osadl.org (updated Sat Sep 30, 2023 12:59:07)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
274559911671135,28cyclictest2548-21smartctl07:40:371
270802465421,29sleep00-21swapper/007:21:400
272282356310,31sleep10-21swapper/107:23:131
545721180,8sleep02745499cyclictest10:40:360
903221150,8sleep12745599cyclictest10:50:271
12092990,8sleep02745499cyclictest10:30:220
27454996255,4cyclictest2779-21snmpd07:40:380
27454995714,6cyclictest2779-21snmpd07:30:250
2745499552,19cyclictest0-21swapper/009:06:070
2745499552,17cyclictest0-21swapper/010:07:030
2745499552,17cyclictest0-21swapper/010:07:020
2745499552,14cyclictest2779-21snmpd08:34:450
27454995512,12cyclictest0-21swapper/008:05:330
2745499542,18cyclictest0-21swapper/012:28:150
2745499532,25cyclictest10928-21/usr/sbin/munin09:30:190
2745499532,17cyclictest0-21swapper/010:11:420
2745499532,17cyclictest0-21swapper/007:54:050
2745499518,19cyclictest2397-21lldpd08:55:010
2745499518,11cyclictest2779-21snmpd09:02:420
2745499512,16cyclictest0-21swapper/007:57:280
2745499502,7cyclictest0-21swapper/008:55:220
2745499502,14cyclictest0-21swapper/009:49:390
2745499472,13cyclictest0-21swapper/012:44:220
27454994612,9cyclictest2397-21lldpd12:39:470
171782460,7sleep02745499cyclictest08:20:220
2745499452,13cyclictest0-21swapper/008:43:210
27454994414,12cyclictest0-21swapper/008:11:060
96322412,35sleep12779-21snmpd12:17:261
27454994117,12cyclictest0-21swapper/008:45:400
51682401,6sleep12779-21snmpd09:15:161
135652392,32sleep12779-21snmpd12:29:111
71532381,32sleep07245-21cpuspeed_turbos09:20:210
275182381,4sleep12779-21snmpd10:14:391
2745499389,14cyclictest2226-21dbus-daemon11:54:450
2745499382,32cyclictest0-21swapper/011:34:290
162732381,33sleep12779-21snmpd08:16:121
89712371,32sleep12779-21snmpd07:58:291
290422371,32sleep12779-21snmpd07:27:201
2745499377,15cyclictest2779-21snmpd09:10:350
2745499375,15cyclictest0-21swapper/007:38:190
27454993712,6cyclictest0-21swapper/012:18:230
144842370,8sleep02745499cyclictest09:40:010
5452361,31sleep12397-21lldpd10:27:591
37092361,4sleep13831-21users12:00:421
317472361,31sleep12779-21snmpd07:34:021
313402361,31sleep12779-21snmpd08:56:301
27455993628,6cyclictest22-21ksoftirqd/111:45:201
2745499368,4cyclictest81rcu_preempt12:05:080
2745499367,13cyclictest2397-21lldpd11:22:080
2745499365,15cyclictest2779-21snmpd11:11:430
2745499363,16cyclictest14429-21iostat_ios12:30:340
2745499362,15cyclictest0-21swapper/011:03:180
27454993616,7cyclictest31515-21aten_r4power_cu10:25:190
27454993610,6cyclictest0-21swapper/010:58:290
27454993610,6cyclictest0-21swapper/007:28:590
93222351,29sleep09351-21/usr/sbin/munin08:00:200
85992351,5sleep12779-21snmpd09:23:171
27455993525,6cyclictest2779-21snmpd10:19:541
2745499359,9cyclictest0-21swapper/008:36:110
2745499358,13cyclictest2779-21snmpd11:42:440
2745499354,14cyclictest0-21swapper/009:50:550
2745499353,14cyclictest0-21swapper/010:03:380
2745499352,20cyclictest0-21swapper/009:26:220
2745499352,19cyclictest0-21swapper/011:56:000
23502351,4sleep12397-21lldpd10:31:571
207942351,30sleep120803-21iostat_ios09:55:281
27455993427,4cyclictest41250irq/92-i91511:37:541
27455993420,12cyclictest22-21ksoftirqd/111:25:421
2745499347,12cyclictest2397-21lldpd09:59:420
2745499346,5cyclictest28953-21unixbench_singl10:15:390
2745499346,4cyclictest81rcu_preempt12:45:580
2745499346,13cyclictest2397-21lldpd09:40:530
2745499345,12cyclictest2397-21lldpd10:24:360
2745499343,18cyclictest0-21swapper/010:49:200
2745499342,20cyclictest0-21swapper/010:36:470
2745499342,19cyclictest0-21swapper/011:17:490
2745499342,14cyclictest0-21swapper/012:20:120
2745499342,14cyclictest0-21swapper/011:29:170
2745499342,14cyclictest0-21swapper/007:46:330
22082341,4sleep12397-21lldpd11:59:271
17922341,29sleep12397-21lldpd09:05:261
155462341,3sleep10-21swapper/111:06:141
151632341,3sleep10-21swapper/112:31:171
48122331,28sleep12397-21lldpd09:13:081
297062330,4sleep041ktimersoftd/011:45:270
276122332,7sleep10-21swapper/108:46:421
27455993327,3cyclictest41250irq/92-i91509:40:421
2745499337,12cyclictest2397-21lldpd12:54:460
2745499336,13cyclictest2397-21lldpd11:36:060
2745499332,18cyclictest0-21swapper/012:02:270
2745499332,18cyclictest0-21swapper/011:07:150
2745499332,14cyclictest0-21swapper/008:16:500
27454993312,15cyclictest2779-21snmpd08:25:060
141802331,5sleep10-21swapper/109:37:541
116802331,28sleep12397-21lldpd08:05:241
27455993225,4cyclictest41250irq/92-i91508:03:341
27455993223,5cyclictest22-21ksoftirqd/109:00:391
2745599322,26cyclictest0-21swapper/112:05:391
2745499323,17cyclictest0-21swapper/009:17:350
2745499322,18cyclictest0-21swapper/012:12:510
207912322,26sleep12779-21snmpd12:45:501
203942322,7sleep10-21swapper/108:29:541
174812321,5sleep10-21swapper/111:11:531
2745599315,22cyclictest0-21swapper/110:39:541
27455993125,3cyclictest41250irq/92-i91512:36:291
27455993125,3cyclictest41250irq/92-i91510:58:181
27455993125,3cyclictest41250irq/92-i91510:23:531
27455993123,5cyclictest22-21ksoftirqd/110:45:241
27455993121,6cyclictest25429-21cat08:40:401
27455993119,8cyclictest0-21swapper/109:30:261
27455993118,9cyclictest137950irq/90-eth009:45:301
2745499311,5cyclictest81rcu_preempt10:54:240
251542312,6sleep10-21swapper/111:33:251
234762312,6sleep10-21swapper/110:01:401
215072311,6sleep10-21swapper/111:24:301
208642311,26sleep120873-21df_inode08:30:221
144242311,5sleep10-21swapper/108:11:291
2745599306,21cyclictest0-21swapper/111:44:271
2745599303,24cyclictest0-21swapper/108:50:331
27455993025,3cyclictest22-21ksoftirqd/112:20:371
27455993025,3cyclictest22-21ksoftirqd/112:10:361
27455993025,3cyclictest22-21ksoftirqd/111:15:371
27455993023,4cyclictest22-21ksoftirqd/111:50:111
27455993012,15cyclictest0-21swapper/112:53:201
51242291,4sleep12397-21lldpd07:49:241
2745599299,16cyclictest0-21swapper/108:24:551
2745599297,18cyclictest0-21swapper/111:01:521
2745599293,22cyclictest0-21swapper/112:40:521
2745599293,22cyclictest0-21swapper/107:35:301
27455992922,4cyclictest22-21ksoftirqd/109:25:251
2745599292,24cyclictest0-21swapper/109:50:241
27455992912,13cyclictest22-21ksoftirqd/107:50:191
224422291,4sleep12779-21snmpd08:35:071
254342281,23sleep10-21swapper/110:07:321
254342281,23sleep10-21swapper/110:07:311
2745599276,17cyclictest0-21swapper/110:42:231
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional