layered soaware architectures and - rainfocus · tax demo tests test run ... adf 16 threads plsql 1...

Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|

LayeredSoAwareArchitecturesandPerformanceintheRealWorld

ToonKoppelaarsConsulKngMemberofTechnicalStaffReal-WorldPerformanceServerTechnologies

ConfidenKal–OracleInternal/Restricted/HighlyRestricted

AndrewHoldsworthVicePresidentReal-WorldPerformanceServerTechnologies

JohnClarkeDirectorReal-WorldPerformanceServerTechnologies


SafeHarborStatementThefollowingisintendedtooutlineourgeneralproductdirecKon.ItisintendedforinformaKonpurposesonly,andmaynotbeincorporatedintoanycontract.Itisnotacommitmenttodeliveranymaterial,code,orfuncKonality,andshouldnotberelieduponinmakingpurchasingdecisions.Thedevelopment,release,andKmingofanyfeaturesorfuncKonalitydescribedforOracle’sproductsremainsatthesolediscreKonofOracle.

ConfidenKal–OracleInternal/Restricted/HighlyRestricted 3


•  SessionID:CON6460•  LayeredSo0wareArchitecturesandPerformanceintheRealWorld

•  Monday13:45@MosconeSouth302

•  SessionID:CON6463•  Real-WorldPerformanceMonitoring:CanYouBelievetheCPUNumbers?

•  Monday16:15@MosconeSouth302

•  SessionID:CON6458•  Real-WorldPerformance:OracleDatabaseIn-MemoryforSAP

•  Monday17:30@ParkCentralConcordia

•  SessionID:CON6465•  Real-WorldPerformanceTeam:SQLMonitorReport

•  Tuesday12:15@MosconeSouth302

4September20,2016

Real-WorldPerformanceOOW2016WheretoFindUs


Real-WorldPerformanceOOW2016ThisYear'sObservaKons

•  Almostobsessivebeliefthatplacormtuning/selecKonwillsolveallperformanceproblems

•  Almostasmuchfocusonotheraspectsthatdeliverincrementalgains

•  Verylidleworkonalgorithmicandarchitecturechangesthatyieldorderofmagnitudegains

5September20,2016


Real-WorldPerformanceOOW2016WheretoGetmoreRealWorldPerformance

•  Seeonlinevideolibrary

•  hdp://www.oracle.com/goto/oll/rwp

6September20,2016


Areyouhappywithincrementalperformanceimprovements?


Doyouhavetheabilitytochooseyourperformance?

Doyoumovedatatoprocessingorprocessingtodata?


ApplicaKonArchitecture

§  Ifyouhadtovalidateasinglerowonawebform,howwouldyoudesignyourapp?

§  Whatifyouhadtovalidatemillionsofrowsthatarrivedasabatch?

§  Doyoudothis?Design(Manyrows)=Design(Onerow)*#Rows

8

Howdoyoudesignapps?


Agenda

SoAwareArchitectures

DemosandPerformanceAnalysis

Conclusion

1

2

3



Agenda



Conclusion

1

2

3



WhatisaLayeredSoAwareArchitecture?§  Mostcommonarchitecture§  "n-Tier"architecture§  StandardformostJavaEEapplicaKons

§  Widelyusedbyarchitects,designers,developers

12


WhatisaLayeredSoAwareArchitecture?

§  Organizedintohorizontallayers

§  Eachlayerperformsspecificrole

§  Mostconsistof4-5majorlayers

§  LayersgenerallyincludepresentaKon,businesslogic,model,persistence,anddatabase

13


WhyaLayeredSoAwareArchitecture?§  Layersaretypicallyclosedandexpose

API'sforinvocaKon

§  EachlayermustgothroughlayerAPI'sdirectlybelowit;enforcesisolaKon

§  IsolaKonenablesseparaKonofconcernsandlayerindependence

§  Makesiteasytore-use,build,and(theore+cally)testandmaintainapplicaKon

14


PresentaKonLayer

BusinessLayer

ModelLayer

PersistenceLayer

DatabaseLayer

ImplicaKonsofaLayeredSoAwareArchitecture§  AllapplicaKonlogicbuiltinlayers

outsidedatabase

§  SQLishiddenfordevelopers

§  BodomlayerstranslateGUIstructurestorelaKonaldata,oAengeneraKngrow-by-rowSQLstatements

§  Database=(dumb)tablestoreor"bitbucket",notaprocessingengine

15

SQL


Agenda



PerformanceImpactandConclusion

1

2

3



DemoandPerformanceAnalysis

17


RealWorldExample

• VATClearing/Matching– CompanyAsellstoCompanyB– CompanyBbuysfromCompanyA– EachreportVATonpurchasedgoodsandsoldgoods;onebusinesscanclaimtheVAT

• BuyandSellrecordsmustbematched• Batchesreceiveddailyinsetsof2:buyrecordsandsellrecords• PossibleexcepKons:– Duplicates– Outofsyncrecords;i.e.,BuyandSellrecordsindifferentbatches

DailyBatchProgramforTaxMatching:Briefing


RealWorldExampleInvolvedObjects

PREMATCH_BUYtable

PREMATCH_SELLtable

DUPLICATE_SELLtable

DUPLICATE_BUYtable

Buy

transacKonsfile

Sell

transacKonsfile

MATCHEDtable


RealWorldExampleDataFlow

PREMATCH_BUYtable

PREMATCH_SELLtable

DUPLICATE_SELLtable

DUPLICATE_BUYtable

Buy

transacKonsfile

Sell

transacKonsfile

MATCHEDtable

Loadrowsdaily

Loadrowsdaily



PREMATCH_BUYtable

PREMATCH_SELLtable

DUPLICATE_SELLtable

DUPLICATE_BUYtable

Buy

transacKonsfile

Sell

transacKonsfile

MATCHEDtable

MatchandinsertrowsMatchandinsertrows



PREMATCH_BUYtable

PREMATCH_SELLtable

DUPLICATE_SELLtable

DUPLICATE_BUYtable

Buy

transacKonsfile

Sell

transacKonsfile

MATCHEDtable

Removematchedrows.Unmatchedrowsremainfornextday

Removematchedrows.Unmatchedrowsremainfornextday


MATCHEDtable

Duringloadverify

forduplicates

RealWorldExampleAddiXonalBusinessLogic:DiscardDuplicates

PREMATCH_BUYtable

PREMATCH_SELLtable

DUPLICATE_SELLtable

DUPLICATE_BUYtable

Buy

transacKonsfile

Sell

transacKonsfile

Duringloadverifyforduplicates

Duringloadverifyforduplicates

BeforeloadingarowverifyforduplicatesParkduplicate

rowsinseparatetable

Parkduplicaterowsinseparate

table


RealWorldExample

Doesabitofeverything:•  Insertsintofivetables•  Performsindexedlookupsforde-duplicaKonandmatching•  DeletesfromPREMATCHtables•  Indexmaintenance•  Somebusinesslogic(if-then-else)tode-dupeandmatch

• WealreadyhaveaWebUIthatdoesVATmatchingforsinglebuy/sellrecord• We'llreuseitsbusiness/model/persistencelayers

LoadProfile


RealWorldExample

Thereareanumberoflayereddevelopmentframeworksavailable

DevelopmentFrameworkDecisions


LayeredArchitecturewithHibernate

26

PresentaKonLayer

BusinessLayer

ModelLayer

PersistenceLayer

DatabaseLayer

Table/DMLclassesValidaKon

HibernateDAO'sDataAccessObjects

BusinessenXtyclassesUsableinpresentaKonlayer

UI

TABLES


WeUsedOracleADF

27

PresentaKonLayer

BusinessLayer

ModelLayer

PersistenceLayer

DatabaseLayer

ADF-BCViewObjects(VO's)

ADF-BCEnKtyObjects(EO's)

ADF-BCApplicaKonModule(AM)

UI

TABLES

WealreadyhaveserviceinAMtomatchonebuy/sellrecord

DevelopedBLinJavatorepeatedlycallthis


TaxDemoTestsTestRun#1:•  ADF1Thread•  ADF4Threads•  ADF16Threads•  PL/SQL1Thread

TestRun#2:•  ADF16Threads•  PL/SQL1ThreadArraysize256

•  PLSQL16ThreadsArraySize1

•  PL/SQL16ThreadsArraySize256

TestRun#3:•  ADF16Threads•  PL/SQL16Threads•  PL/SQL16ThreadsArraySize256

•  SetBasedSQLDoP=8


DEMO


TaxDemo–Run#1

29

Config Threads File1Seconds

File2Seconds

File3Seconds

File4Seconds

File5Seconds

TotalSeconds

ADF 1 1,674 2,081 1,561 1,111 2,131 8,558

ADF 4 460 562 425 302 579 2,328

ADF 16 158 192 145 109 198 802

PL/SQL 1 337 421 315 224 430 1,727

DEMO


TaxDemo–Run#1

30

0

2500

5000

7500

ADF 1 Thread ADF 4 Threads ADF 16 Threads PL/SQL 1 ThreadTest

Seconds

ConfigADF 1 Thread

ADF 4 Threads

ADF 16 Threads

PL/SQL 1 Thread

Test #1 Elapsed Time

DEMO


TaxDemo–Run#1

31

0

1000

2000

3000

4000

ADF 1 Thread ADF 4 Threads ADF 16 Threads PL/SQL 1 ThreadTest

Row

s pe

r S

econ

d

ConfigADF 1 Thread

ADF 4 Threads

ADF 16 Threads

PL/SQL 1 Thread

Test #1 Throughput

DEMO


TaxDemo–Run#1

32

PL/SQL1Thread

ADF1Thread

ADFjobspendslessKmeinthedatabase

DEMO


TaxDemo–Run#1(ConKnued)

33

ADF16threads

With16threads,ourADFjobisonlyacKvein

thedatabase5.7secondspersecond

DEMO


WhydidweshowPL/SQL,anyway?

§  Ourlayered(ADF)applicaKonforcedusdownarow-by-rowpath

§  Wecaneasilydorow-by-rowprocessinginPL/SQL

34

DEMO


TaxDemo–Run#1

35

WillSQLtuninghelp?

DEMO


TaxDemo–Run#1

36

Thisiswhatrow-by-rowprocessinglooks

like

DEMO









DEMO


TaxDemo–Run#2

38

Config Threads ArraySize File1Seconds

File2Seconds

File3Seconds

File4Seconds

File5Seconds

TotalSeconds

ADF 16 1 158 192 145 109 198 802

PL/SQL 1 256 163 210 115 113 210 811

PL/SQL 16 1 37 43 31 23 44 178

PL/SQL 16 256 26 31 24 17 33 131

DEMO


TaxDemo–Run#2

39

0

200

400

600

800

ADF 16 Threads PLSQL 1 Thread Array 256 PLSQL 16 Threads PLSQL 16 Threads Array 256Test

Seconds

Config

ADF 16 Threads

PLSQL 1 Thread Array 256

PLSQL 16 Threads

PLSQL 16 Threads Array 256


DEMO


TaxDemo–Run#2

40

0

10000

20000

30000

ADF 16 Threads PLSQL 1 Thread Array 256 PLSQL 16 Threads PLSQL 16 Threads Array 256Test

Row

s pe

r S

econ

d

Config

ADF 16 Threads


PLSQL 16 Threads

PLSQL 16 Threads Array 256

Test #2 Throughput

DEMO


ADFvs.PL/SQLCPUEfficiency

41

ADF(16Threads)

PL/SQL(16Threads)

TheADFprogramusesmoreCPU

DEMO


ADFvs.PL/SQLCPUEfficiencyWhyisRow-by-RowPL/SQLUsingLessCPU?

Device-driver/Ethernet/IP/TCP-UDP/Sockets

OPI OPI

Two-task

SQL*Net

TNS

Prot.adapter

SQLengine

PLSQLengine

OracleLinux

Systemlibrary

SQL

EmbeddedSQL

Sockets/TCP-UDP/IP/Ethernet/Device-driver

LinuxJVM

Persistencylayers

JDBClayers

Systemlibrary

Domainmodellayers

ApplicaKoncodeSedersandgeders

TransacXonlayers

DEMO


PL/SQLvs.ADF:CPUImpactADFOraclecallstackwithFlameGraph

Shippingdatainandoutofthedatabase;outsideOPI/SQL

Engine

ApplicaKonSQL(insideSQLEngine/OPI)

OSnetwork,IPC,Net/TNS/TTS

Deletes InsertsExecutes&fetches

StateiniKalizaKon,contextprep,find

cursor,unmapcursor

Recursivecalls

DEMO


PL/SQLvs.ADF:CPUImpactPL/SQLOraclecallstackwithFlameGraph

PL/SQLEngine

ApplicaKonSQL(insideSQLEngine/OPI)

Deletes Inserts

Executes&fetches

StateiniKalizaKon,contextprep,find

cursor,unmapcursorRecursive

calls

DEMO


PL/SQLvs.ADF:CPUImpact

opiexe

opiexe

Samecode–whydoesADFusemoreCPU?

DEMO


363395.733099 cpu-clock 363333.886369 task-clock 1,054,507,284,440 cycles 497,951,316,616 instructions 35,271,327,010 bus-cycles 555,928 faults 6,782 cpu-migrations 29,011,223,921 cache-references 262,587,106 cache-misses 4,496,968 context-switches 97,003,543,743 branches 3,132,424,179 branch-misses

954.553486931 seconds time elapsed

163360.334736 cpu-clock 163360.401705 task-clock 485,147,753,205 cycles 224,676,903,673 instructions 16,182,224,526 bus-cycles 749,871 faults 1,802 cpu-migrations 13,863,704,572 cache-references 149,387,122 cache-misses 115,832 context-switches 42,229,605,073 branches 821,626,951 branch-misses

165.182311836 seconds time elapsedPL/SQL

PL/SQLvs.ADF:CPUImpact

ADF

200%moreinstrucKons

Requiring100%moreCPU

Causedbymorebranch&cachemisses

DEMO


WhatDoesthePL/SQLArrayInterfaceDoforYou?

47

ArraySize1 ArraySize256

ArrayprocessinguseslessCPU

DEMO









DEMO


TaxDemo–Run#3

49

Config(Thread)

Threads/DoP ArraySize File1

SecondsFile2

SecondsFile3

SecondsFile4

SecondsFile5

SecondsTotal

Seconds

ADF 16 1 158 192 145 109 198 802

PL/SQL 16 1 37 43 31 23 44 178

PL/SQL 16 256 26 31 24 17 33 131

Set 8 N/A 3 3 2 3 3 14

DEMO


TaxDemo–Run#3

50

DEMO

0

200

400

600

800

ADF 16 Threads PLSQL 16 Thread Array 1 PLSQL 16 Thread Array 256 Set DoP 8Test

Seconds

Config

ADF 16 Threads



Set DoP 8



TaxDemo–Run#3

51

DEMO

0e+00

1e+05

2e+05

ADF 16 Threads PLSQL 16 Thread Array 1 PLSQL 16 Thread Array 256 Set DoP 8Test

Row

s pe

r S

econ

d

Config

ADF 16 Threads



Set DoP 8

Test #3 Throughput


0

5000

10000

ADF 1Thread

ADF 4Threads

ADF 16Threads

PL/SQL 1Thread

PL/SQL 1Thread

Array 256

PL/SQL 16Threads

PL/SQL 16Threads

Array 256

Set-BasedDoP=8

Config

CP

U S

econ

ds

TierApps

DB

CPU Seconds per Test by Tier

CPUUsage

52

BlueisAppsTierCPUseconds

RedisDBTierCPUseconds

DEMO


CPUUsage

53

Config Threads/DoP ArraySize DBCPU AppsTierCPU TotalCPUCPUper1k

RowsProcessed

ADF 1 1 3,583.18 9,464.46 13,047.65 3.45

ADF 4 1 3,476.46 7,330.17 11,454.98 3.03

ADF 16 1 4,608.08 8,039.97 12,699.14 3.35

PL/SQL 1 1 1,775.04 0.00 1,775.04 0.47

Pl/SQL 1 256 863.03 0.00 863.03 0.22

PL/SQL 16 1 2,351.20 0.00 2,351.20 0.62

PL/SQL 16 256 1,755.99 0.00 1,755.99 0.46

Set 16 N/A 101.22 0.00 101.22 0.02

DEMO


CPUUsage

54

DEMO

0

1

2

3

ADF 1Thread

ADF 16Threads

ADF 4Threads

PL/SQL 1Thread

PL/SQL 1Thread

Array 256

PL/SQL 16Threads

PL/SQL 16Threads

Array 256

Set-BasedDoP=8

Test

CP

U p

er 1

K R

ows

Config

ADF 1 Thread

ADF 16 Threads

ADF 4 Threads

PL/SQL 1 Thread

PL/SQL 1 Thread Array 256

PL/SQL 16 Threads

PL/SQL 16 Threads Array 256

Set-Based DoP=8

CPU Seconds per 1k Rows


Set-BasedProcessing

•  Techniquesinclude:– DatabaseParallelism– DDLinsteadofDML– MulK-tableinsertswithCommonTableExpressions– WindowfuncKonsforrowlabeling

•  Faster,useslessCPU,andeasiertocodeandsupport•  Set-basedSQLmovesprocessingtodata–OracleDBisaprocessingengine• WithSet-BasedSQL,wecanchooseourperformancewithDoP

ExploitthepowerofSQL


PerformancewithLayeredArchitectures

Database NetworkApplicaXon

serverrunningJVM

Fromdatabase'sperspecXve…

ElapsedXme=(DBKme)+("Timenotindatabase")



Database NetworkApplicaXon

serverrunningJVM

Fromappsserver'sperspecXve…

ElapsedXme=(JVMKme)+("TimenotinJVM")


OS/Thread/Procstart/wake/other


Database Network ApplicaXonserverrunning

JVM

16ThreadADFExample

Elapsed=13.4minutes

DBT=4.8mins JVM=5mins3.6mins

Whatifyournetworkisslow?





JVM

16ThreadADFExample

Elapsed=13.4minutes


Whenyoumovedatatoprocessing(row-by-row),there'sapenalty





JVM

16ThreadADFExample

Elapsed=13.4minutes


Howmuchcontroldoyouhaveoverperformancewiththepenaltyforeachrow?


PerformancewithLayeredArchitecturesWhereistheLeverage?

Wheredoyoustart?

Inthelandofincrementalgains


PerformancewithLayeredArchitecturesWhereistheLeverage?

Wheredoyoustart?


PerformancewithLayeredArchitecturesThrowingapplicaXonparallelismattheproblem

Over-processed

ContenKon

Isthisthepointyoustartbodom-up

analysis?


PerformancewithLayeredArchitecturesThrowingapplicaXonparallelismattheproblemAdminServer-diagnostic.log:[2016-09-15T14:03:23.210-07:00] [AdminServer] [NOTIFICATION] [DFW-40101] [oracle.dfw.incident] [tid: TaxMatch125] [userId: <anonymous>] [ecid: 47e4c839-a6ff-4dc8-b4f6-7155f95baca7-00000023,0:4:1548] [APP: RwpTax2] An incident has been signalled with the incident facts: [problemKey=DFW-99998 [oracle.jbo.pool.ResourcePoolException][oracle.jbo.pool.ResourcePool.allocateResource][RwpTax2] incidentSource=SYSTEM incidentTime=Thu Sep 15 14:03:23 PDT 2016 errorMessage=DFW-99998 executionContextId=null]AdminServer-diagnostic.log:[2016-09-15T14:03:23.210-07:00] [AdminServer] [WARNING] [DFW-40125] [oracle.dfw.incident] [tid: TaxMatch125] [userId: <anonymous>] [ecid: 47e4c839-a6ff-4dc8-b4f6-7155f95baca7-00000023,0:4:1548] [APP: RwpTax2] incident flood controlled with Problem Key "DFW-99998 [oracle.jbo.pool.ResourcePoolException][oracle.jbo.pool.ResourcePool.allocateResource][RwpTax2]"e21b0a54-30f2-4028-bf18-a1c62871c7d4-00000373,0:2:11:1556284] [errid: 349] [detailLoc: /nfs-shared/wls/home/user_projects/domains/rwp_domain/servers/AdminServer/adr/diag/ofm/rwp_domain/AdminServer/incident/incdir_349] [probKey: DFW-99997 [java.lang.OutOfMemoryError]] [APP: RwpTax2] incident 349 created with problem key "DFW-99997

WhatdoyoudowhenyourunoutofapplicaKonserver

resources?

Canyoureallychooseyourperformance

thisway?


Agenda



Conclusion

1

2

3



Conclusion



Conclusions

Layeredarchitecturesdon'tletyouchooseyourperformance•  Row-by-rowalgorithmsacrossmulKplestacks=moreKme,lessthroughput•  Thisper-rowtaxmulKpliesasdatavolumeincreases•  Youcanonlycontrolasmallpiece–thepieceofcodeorinfrastructureyouown•  IncreasedCPUusage,whichincreasessoAware/hardware/Cloudcosts•  EasytorunintocontenKon/bodlenecksanddifficulttoresolve–usuallyinvolvescomplexapplicaKoncodechange

CanYouChooseYourPerformancewithLayeredArchitectures?


Conclusions

Layeredarchitecturesdrivedatatoprocessing,notprocessingtodata•  DatashippedtoAppsKeratunitoflowestcommondenominator–arow(orcolumn!)

•  Databaseviewedaspersistencelayer,notprocessingengine•  Equatesto:– Lotsofroundtrips,lotsofstacktraversal,thread/processsleep/wakeup/start/stop– …meansmoreCPUandlessefficient,and– …bothperformanceandresourceusagedependenton#ofrows/columns

•  IncreasedCPU=highersoAware/hardware/Cloud

MovingProcessingtoData


Conclusions

Layeredarchitecturesfosteraboiom-upperformanceculture•  Layerownersonlyhavevisibility/responsibilityoversmallpieceofpuzzle• OwnersspendlotsofKme"tuning"theirpiece,chasingpercentagepoints•  Lackofabilitytoinnovate•  "Racetothebodom"mindset–expertineachlayerfocusesontheirlayerandnobodylooksholisKcally

IncrementalPerformanceImprovements


Acknowledgments

•  SoAwareArchitecturePaderns(O'Reilly,MarkRichards)hdp://www.oreilly.com/programming/free/files/soAware-architecture-paderns.pdf•  StevenYang,Real-WorldPerformanceGroup• VladoBarun,Real-WorldPerformanceGroup• BjørnEngsig,Real-WorldPerformanceGroup

layered soaware architectures and - rainfocus · tax demo tests test run ... adf 16 threads plsql 1...

Documents