You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-03-20 - 22:29

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  

ARM Broadcom BCM2835 @1400 MHz, Linux 4.18.16-rt9 (Profile)

Latency plot of shadow in rack #7, slot #3
Data to construct the above plot have been generated using the RT test utility cyclictest.
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot3s.osadl.org (updated Mon Mar 20, 2023 12:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
787622430,2sleep10-21swapper/107:06:421
787622430,2sleep10-21swapper/107:06:421
14822890,5sleep1909399cyclictest08:17:161
14822890,5sleep1909399cyclictest08:17:161
909299881,43cyclictest0-21swapper/010:36:530
909299881,40cyclictest19853-21cut10:31:430
49922870,9sleep3381ktimersoftd/312:15:093
49922870,9sleep3381ktimersoftd/312:15:093
909299811,42cyclictest11539-21latency_hist09:41:200
173002770,5sleep1909399cyclictest10:27:551
9092997614,5cyclictest0-21swapper/007:21:320
9092997420,38cyclictest5448-21idleruntime-cro08:31:190
909399736,45cyclictest7850irq/56-dwc_otg09:56:201
909299725,43cyclictest15441-21ssh10:26:340
201782710,4sleep1909399cyclictest09:11:261
909299702,7cyclictest14064-21ssh12:26:400
909299691,49cyclictest0-21swapper/010:01:430
909299683,53cyclictest31711-21grep12:06:580
909299682,48cyclictest0-21swapper/007:46:320
909299681,54cyclictest31699-21fschecks_count10:46:440
318542680,2sleep30-21swapper/309:26:093
909299676,11cyclictest24930-21missed_timers09:16:470
278252670,3sleep327827-21rt-features11:21:473
9092996545,5cyclictest8050irq/56-dwc_otg_10:06:360
909299651,54cyclictest0-21swapper/012:40:550
909299651,54cyclictest0-21swapper/012:40:550
909399649,4cyclictest0-21swapper/111:39:511
909399642,11cyclictest0-21swapper/109:17:311
909399641,4cyclictest0-21swapper/112:04:551
909299641,55cyclictest0-21swapper/011:31:580
909399632,11cyclictest8050irq/56-dwc_otg_08:21:491
909299638,42cyclictest0-21swapper/011:21:200
9092996340,5cyclictest7950irq/56-dwc_otg_07:41:200
909299631,48cyclictest18694-21kernelversion12:31:460
909399622,13cyclictest7950irq/56-dwc_otg_11:19:511
9092996239,6cyclictest0-21swapper/011:38:030
909299622,48cyclictest28117-21unixbench-2d11:21:500
909299621,50cyclictest0-21swapper/009:56:360
909299621,42cyclictest4060-21cat10:51:540
116982620,2sleep20-21swapper/210:21:402
909399618,4cyclictest0-21swapper/112:31:241
909399618,41cyclictest7850irq/56-dwc_otg11:11:541
909399614,4cyclictest0-21swapper/109:43:061
909399613,4cyclictest7850irq/56-dwc_otg10:36:411
909399611,5cyclictest9277-21rm10:59:021
9092996115,5cyclictest0-21swapper/011:46:380
909399608,4cyclictest0-21swapper/111:10:571
909399605,9cyclictest8050irq/56-dwc_otg_11:50:001
909399604,10cyclictest8050irq/56-dwc_otg_11:41:431
909399604,10cyclictest8050irq/56-dwc_otg_11:41:431
909399601,4cyclictest8050irq/56-dwc_otg_09:11:211
909299603,8cyclictest27709-21grep12:01:540
9092996038,8cyclictest30926-21aten_rbpower_cu08:11:330
909299602,42cyclictest0-21swapper/009:46:350
909299602,42cyclictest0-21swapper/009:46:350
909299602,37cyclictest0-21swapper/010:16:410
909299601,44cyclictest10553-21aten_rbpower_en11:41:350
909299601,44cyclictest10553-21aten_rbpower_en11:41:350
909399593,45cyclictest8050irq/56-dwc_otg_07:56:501
909299592,6cyclictest0-21swapper/011:01:390
909299592,43cyclictest0-21swapper/007:26:420
909299591,46cyclictest16104-21grep11:06:530
9092995910,31cyclictest6971-21aten_rbpower_po10:56:380
909399587,8cyclictest7850irq/56-dwc_otg12:26:391
909399583,7cyclictest16101-21wget09:46:361
909399583,7cyclictest16101-21wget09:46:361
909399582,4cyclictest7950irq/56-dwc_otg_10:47:131
9092995840,4cyclictest7950irq/56-dwc_otg_07:16:190
909299582,45cyclictest21243-21head07:41:500
909299582,42cyclictest0-21swapper/009:11:230
909399577,9cyclictest8050irq/56-dwc_otg_10:42:011
909399577,9cyclictest8050irq/56-dwc_otg_10:42:011
909399571,53cyclictest7850irq/56-dwc_otg10:10:511
909399571,3cyclictest7850irq/56-dwc_otg08:06:451
909299579,37cyclictest8050irq/56-dwc_otg_09:54:450
909299572,39cyclictest0-21swapper/009:41:520
909299571,44cyclictest9-21ksoftirqd/012:21:450
909399566,4cyclictest7950irq/56-dwc_otg_11:34:121
909399564,4cyclictest7850irq/56-dwc_otg10:31:411
909399561,14cyclictest0-21swapper/107:21:341
909299565,41cyclictest13-21rcuc/011:51:440
9092995633,7cyclictest29153-21tr08:06:340
135622560,2sleep10-21swapper/112:26:101
909399557,41cyclictest19327-21unixbench_singl07:36:501
909399556,4cyclictest7950irq/56-dwc_otg_08:41:391
909399555,7cyclictest0-21swapper/108:51:481
909399553,4cyclictest24127-21ssh12:38:071
909399553,4cyclictest24127-21ssh12:38:071
909399553,4cyclictest0-21swapper/109:26:341
909399551,9cyclictest8050irq/56-dwc_otg_09:33:091
909399551,9cyclictest8050irq/56-dwc_otg_09:33:091
909399551,12cyclictest8050irq/56-dwc_otg_10:16:211
909299552,41cyclictest0-21swapper/012:15:010
909299552,41cyclictest0-21swapper/012:15:010
909299551,44cyclictest410-21latency09:26:420
909299551,44cyclictest0-21swapper/011:11:530
9092995512,32cyclictest0-21swapper/008:06:200
259702550,2sleep025971-21/usr/sbin/munin07:56:420
909499546,34cyclictest8050irq/56-dwc_otg_09:41:382
909399548,7cyclictest8050irq/56-dwc_otg_11:02:371
909399545,9cyclictest8050irq/56-dwc_otg_12:17:301
909399544,4cyclictest0-21swapper/111:22:341
909399544,12cyclictest8050irq/56-dwc_otg_08:46:451
909399541,13cyclictest0-21swapper/107:51:301
909299542,43cyclictest0-21swapper/007:35:030
909299542,41cyclictest0-21swapper/007:51:460
909299541,40cyclictest10910-21/usr/sbin/munin07:16:490
909299541,39cyclictest0-21swapper/011:26:510
9092995412,29cyclictest0-21swapper/008:56:340
9094995311,8cyclictest7950irq/56-dwc_otg_11:46:392
909399539,5cyclictest16181-21latency_hist07:31:201
909399538,4cyclictest0-21swapper/111:56:401
909299532,39cyclictest0-21swapper/009:21:470
909299531,42cyclictest0-21swapper/012:16:490
87522539,35sleep00-21swapper/007:10:340
87522539,35sleep00-21swapper/007:10:340
43912530,2sleep20-21swapper/210:11:492
909399529,4cyclictest29030-21latency_hist08:06:211
909399524,4cyclictest0-21swapper/112:07:201
909399523,10cyclictest8050irq/56-dwc_otg_11:54:521
909399522,8cyclictest8050irq/56-dwc_otg_10:21:361
909399521,9cyclictest10259-21seq09:38:541
909399521,10cyclictest28247-21fschecks_time10:01:421
9093995210,4cyclictest19916-21diskmemload10:00:241
909299522,40cyclictest0-21swapper/008:31:500
909299521,33cyclictest22512-21aten_rbpower_po11:56:360
909599511,12cyclictest0-21swapper/307:41:333
909399511,4cyclictest213862sleep107:43:231
909399511,3cyclictest7850irq/56-dwc_otg10:54:011
909399511,13cyclictest30350-21ssh09:23:151
909299512,39cyclictest0-21swapper/009:34:160
909299512,39cyclictest0-21swapper/009:34:160
909299512,38cyclictest0-21swapper/010:41:320
909299512,38cyclictest0-21swapper/010:41:320
909599505,6cyclictest7950irq/56-dwc_otg_10:56:373
909499504,11cyclictest0-21swapper/207:21:322
909399503,4cyclictest8050irq/56-dwc_otg_09:06:121
909399502,5cyclictest0-21swapper/108:40:481
909399501,9cyclictest0-21swapper/110:21:211
909399501,13cyclictest0-21swapper/108:32:241
909299501,37cyclictest14115-21ssh10:24:290
909299501,34cyclictest4624-21sed10:11:520
9092995011,5cyclictest19915-21munin-run09:11:200
909599498,27cyclictest0-21swapper/311:56:353
909399491,9cyclictest4987-21grep08:26:501
909399491,3cyclictest7850irq/56-dwc_otg07:19:511
909399491,11cyclictest5772-21ssh12:16:211
909399491,11cyclictest5772-21ssh12:16:211
909299491,39cyclictest0-21swapper/008:23:590
909299491,39cyclictest0-21swapper/008:16:430
909299491,39cyclictest0-21swapper/008:16:430
909299491,36cyclictest0-21swapper/008:54:320
909599487,25cyclictest0-21swapper/308:51:333
909399483,6cyclictest0-21swapper/107:16:191
909399482,5cyclictest0-21swapper/107:46:331
909399482,3cyclictest7850irq/56-dwc_otg11:26:491
909299486,32cyclictest9-21ksoftirqd/008:41:390
909299482,30cyclictest0-21swapper/008:48:080
909299481,33cyclictest0-21swapper/009:02:020
909399477,25cyclictest0-21swapper/108:56:341
909599467,25cyclictest0-21swapper/309:46:253
909599467,25cyclictest0-21swapper/309:46:253
909599467,24cyclictest0-21swapper/308:01:403
909499467,25cyclictest0-21swapper/208:56:342
909399467,24cyclictest0-21swapper/108:11:331
909399461,8cyclictest0-21swapper/107:35:551
909599456,26cyclictest39-21ksoftirqd/309:01:323
9095994513,23cyclictest0-21swapper/312:35:493
909299452,34cyclictest0-21swapper/008:38:380
909599447,24cyclictest0-21swapper/308:41:343
135712440,2sleep30-21swapper/309:42:363
909599437,25cyclictest0-21swapper/307:56:353
909599433,26cyclictest0-21swapper/307:16:363
909499437,23cyclictest0-21swapper/208:26:342
909599427,24cyclictest0-21swapper/308:16:343
909599427,24cyclictest0-21swapper/308:16:343
909599427,22cyclictest0-21swapper/309:56:343
909499422,6cyclictest8050irq/56-dwc_otg_11:41:322
909499422,6cyclictest8050irq/56-dwc_otg_11:41:322
9094994213,15cyclictest27234-21latency_hist10:41:212
9095994125,9cyclictest37-21rcuc/312:21:363
909499417,25cyclictest0-21swapper/211:11:402
9094994134,5cyclictest8050irq/56-dwc_otg_12:36:392
9094994134,5cyclictest8050irq/56-dwc_otg_12:36:392
909499402,16cyclictest16426-21ssh09:46:412
909499402,16cyclictest16426-21ssh09:46:412
9094994020,5cyclictest27559-21sh11:21:432
9094994015,6cyclictest9734-21sh11:40:332
909599396,21cyclictest0-21swapper/312:16:363
909599394,11cyclictest0-21swapper/310:41:353
909599394,11cyclictest0-21swapper/310:41:353
9094993920,12cyclictest2084-21ssh10:50:252
9094993917,12cyclictest301ktimersoftd/211:01:402
9094993916,5cyclictest28992-21latency_hist08:06:212
909499391,16cyclictest23911-21ssh09:16:192
909499391,15cyclictest5500-21ssh10:54:232
909599386,7cyclictest8050irq/56-dwc_otg_11:46:203
909599386,7cyclictest8050irq/56-dwc_otg_11:46:203
9094993815,5cyclictest15405-21/usr/sbin/munin10:26:382
9094993813,4cyclictest17674-21unixbench_singl09:01:502
909499381,24cyclictest6300-21ls11:36:212
909599374,17cyclictest8050irq/56-dwc_otg_08:06:483
909599371,23cyclictest28200-21cut10:01:423
909499375,4cyclictest0-21swapper/209:26:412
909499374,23cyclictest0-21swapper/207:51:332
909599369,4cyclictest19227-21grep07:36:503
909599366,7cyclictest7850irq/56-dwc_otg10:26:473
909599361,4cyclictest2945-21ssh10:51:363
9095993610,15cyclictest0-21swapper/311:51:383
9094993612,6cyclictest0-21swapper/210:18:082
909499361,24cyclictest20211-21ls09:51:442
909599357,5cyclictest8050irq/56-dwc_otg_11:16:373
909599356,14cyclictest8050irq/56-dwc_otg_12:01:393
909599356,14cyclictest8050irq/56-dwc_otg_07:21:433
909599352,7cyclictest7850irq/56-dwc_otg11:11:383
9095993516,5cyclictest14049-21apt-get12:26:383
9094993519,9cyclictest17720-21ssh11:09:522
909499351,7cyclictest7950irq/56-dwc_otg_11:56:202
9094993510,6cyclictest31583-21missed_timers11:26:442
9094993510,4cyclictest31-21ksoftirqd/211:16:402
6382351,28sleep30-21swapper/309:26:473
909599348,6cyclictest8050irq/56-dwc_otg_09:11:203
909599348,6cyclictest8050irq/56-dwc_otg_08:56:433
909599347,8cyclictest0-21swapper/309:53:453
909599347,14cyclictest0-21swapper/310:21:363
909599343,7cyclictest7850irq/56-dwc_otg12:06:413
9095993412,6cyclictest37-21rcuc/311:04:333
909599341,18cyclictest11666-21ls09:41:203
9095993411,5cyclictest7850irq/56-dwc_otg10:31:553
909499346,16cyclictest8050irq/56-dwc_otg_12:01:432
909499345,13cyclictest8050irq/56-dwc_otg_07:56:362
909499342,8cyclictest8050irq/56-dwc_otg_11:58:182
9094993418,5cyclictest25536-21sh09:16:542
909499341,6cyclictest7950irq/56-dwc_otg_12:31:202
9094993413,12cyclictest69222chrt08:32:312
9094993412,15cyclictest8050irq/56-dwc_otg_12:11:202
145412341,4sleep314527-21apt-get11:46:403
909599337,15cyclictest0-21swapper/309:16:383
909599331,3cyclictest0-21swapper/312:36:393
909599331,3cyclictest0-21swapper/312:36:393
909599331,25cyclictest7054-21ls11:36:423
909499335,6cyclictest8050irq/56-dwc_otg_07:26:452
909499333,6cyclictest7950irq/56-dwc_otg_10:31:382
9094993316,9cyclictest26555-21sh09:59:252
909499331,4cyclictest0-21swapper/208:51:312
69262330,3sleep26931-21sh12:16:492
909599328,4cyclictest8050irq/56-dwc_otg_10:36:373
909599327,6cyclictest8050irq/56-dwc_otg_10:11:403
909599326,5cyclictest32182-21sed11:26:513
909599322,19cyclictest0-21swapper/311:31:473
909599321,7cyclictest32683-21grep10:46:573
9095993216,4cyclictest5440-21apt08:31:193
909499327,4cyclictest8050irq/56-dwc_otg_07:16:362
909499327,23cyclictest0-21swapper/208:46:372
909499326,14cyclictest0-21swapper/207:46:342
9094993215,5cyclictest5811-21latency_hist12:16:212
9094993215,5cyclictest5811-21latency_hist12:16:212
909499321,16cyclictest0-21swapper/210:10:412
9094993210,5cyclictest19916-21diskmemload10:56:412
9094993210,4cyclictest0-21swapper/209:36:452
909599317,6cyclictest8050irq/56-dwc_otg_08:21:423
909599315,5cyclictest31746-21grep08:11:463
909599313,4cyclictest0-21swapper/310:11:203
909599313,14cyclictest0-21swapper/310:16:393
909599312,5cyclictest7950irq/56-dwc_otg_07:31:373
909599312,3cyclictest7850irq/56-dwc_otg11:06:363
909599312,16cyclictest8050irq/56-dwc_otg_09:11:373
909599311,19cyclictest0-21swapper/309:31:373
909599311,19cyclictest0-21swapper/309:31:373
909499319,3cyclictest0-21swapper/209:06:482
909499318,4cyclictest0-21swapper/209:26:032
909499317,4cyclictest0-21swapper/207:41:212
909499317,10cyclictest0-21swapper/212:33:372
9094993116,9cyclictest5639-21ssh09:32:412
9094993116,9cyclictest5639-21ssh09:32:412
9094993113,6cyclictest29-21rcuc/208:16:422
9094993113,6cyclictest29-21rcuc/208:16:422
9094993112,6cyclictest8050irq/56-dwc_otg_10:01:402
9094993110,5cyclictest10215-21timerandwakeup08:41:472
9094993110,4cyclictest0-21swapper/212:21:342
909599309,4cyclictest8050irq/56-dwc_otg_08:31:353
909599306,6cyclictest7850irq/56-dwc_otg08:46:423
909599302,20cyclictest0-21swapper/307:30:033
909599301,25cyclictest0-21swapper/307:51:383
909599301,13cyclictest7850irq/56-dwc_otg07:46:473
909599291,5cyclictest7850irq/56-dwc_otg08:36:403
909499292,13cyclictest0-21swapper/208:37:212
909499291,6cyclictest8050irq/56-dwc_otg_08:06:402
9094992910,5cyclictest2774-21ntpq08:21:472
85882290,23sleep20-21swapper/207:08:262
85882290,23sleep20-21swapper/207:08:262
909499285,4cyclictest7950irq/56-dwc_otg_10:41:262
909499285,4cyclictest7950irq/56-dwc_otg_10:41:262
909499283,6cyclictest7850irq/56-dwc_otg08:11:342
909499283,6cyclictest7850irq/56-dwc_otg07:11:432
909499283,12cyclictest8050irq/56-dwc_otg_07:31:412
87602273,15sleep30-21swapper/307:10:413
87602273,15sleep30-21swapper/307:10:413
103422271,4sleep30-21swapper/307:12:153
909499261,5cyclictest8050irq/56-dwc_otg_07:45:042
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional