demystifying soa management

Vision ~ Knowledge ~ Results

SOA Suite, BPM, WebCenter, Java - Lean, Agile, Scrum, Results

Demystifying SOA ManagementCustomer Case: Implementing BTM on SOA Suite10g

Tony van Esch, Oracle Technology Specialist

Whitehorses – The Netherlands

Oracle OpenWorld – October 4th, 2011 – Moscone West

Whitehorses B.V. - Recognised Oracle Integration Specialists: @WhitehorsesNL

About me

● 10+ years Oracle experience

● OCP 10g Database + AS

● Consultant at Whitehorses

(@WhitehorsesNL)

● Pragmatic

● Middleware, High Availability, Scalability


Agenda

CustomerCase

Proof of Concept

FinalImplementation

Future


Customer case

● Who: Dutch government organisation

● What: Gain non-intrusive insight in SOA execution

● Why: Accountability toward partners

● How: “Runtime Governance” needed


Customer Case - Architecture

SOASuite

Siebel

B2B

Search

Legacy

AQ

OWSM

OESB

BPELPartners

PartnersRules

DMS


Customer Case: old situation

ESB

Siebel Db

Business Services

Monitoring SOA execution

UI EM

Rules DMS Search

Console Console ConsoleConsoles


Customer Case: wish

ESB

Business Services

Monitoring SOA execution

Console

Siebel Db Rules DMS Search

observer


Business Transaction Management

Process Engine Service Bus

End-to-End, Transaction Monitoring

Collect. Manage. Report.


producer

Collect: Stealth tracking

● No code changes

● Non invasive

● Minimal overhead

JRockit

Weblogic

ESB

BTMObserver

Proxy service

Business service

consumer


BTM: Collect

● Always On, Real-time tracking

● Across all SOA infrastructure and

applications

● Collects payload content

● Harvest flow metrics

● No modifying or tagging messages


BTM: Manage

• Performance: throughput, latency

• Availability

• Exceptions: technical, business


Oracle BTM: console

baselines andthresholds

Performance MetricsSLA


BTW Reporting

•Business content

• Time range

• Specific faults

•Response times

Online

•BI Publisher 10g or 11g

•BTMCLIPublish


BTM Browser


Customer Case: Two stages

Proof of Concept

• New product

• No knowledge

• High stakes

Finalimplementation

• Lessons Learned


Stage 1: Proof of Concept

● Initial design with BTM 10g (Amberpoint)

● No High Availability

● One simple, medium and complex flow

● Stealth mode observer

● Proxy implementation with OSB 10g


Three challenges

● OC4J: not instrumented

● AQ: not instrumented

● Dependencies: manual correlation between

processes


Challenge 1: OC4J

SOASuite Server 2

BTM Server 2BTM Server 1

Weblogic Server(1) Weblogic Server(2)

OBTM-Services (WL-Cluster)

Sphere Perf

EXM

Monitor 1 Monitor 2 Monitor 3 Monitor 4

SOASuite Server 1


OSB-Domain (WL-Domain)

OSB Console

OSB Server 1 OSB Serve3 2

BTM

OsbserverBTM

Osbserver

AdminServer


Challenge 2: OC4J

● No observer available for OC4J

● Need for instrumented proxy: OSB


JRockit

Weblogic

Challenge 2: AQ

AQ original AQ copy

SOASuite

OSB

Siebel

BTMObserver

DMS

Proxy service

Dummy Business service


● Handling AQ queues without tampering

– Created copy queues

– Messages were replicated to copy queues

– OSB proxy consumes from copy queues

● Extra challenges

– No native AQ support in Weblogic10g

– No container managed connections

– Endpoints BPEL compiled in process

Challenge 2: AQ


Challenge 3: Dependencies

● First auto discovery of services

● Then manual correlation

– Which attribute(s) correlate the messages

– Use xpath

– One on one dependency mapping


Lesson learned POC

● Choosen path is correct: delivers & more

● Need for improvements on BTM10g

– Built-in reporting not flexible

– Limited instrumented platforms

– performance

● Don’t treat BTM as standalone add-on


Stage 2: final implementation

● High availability

● performance

– Switched BTM nodes from win32 > Linux 64bit

– Policy tuning (multiple monitoring groups)

– Sizing

● Stability

– migrate to BTM 11g

– Dedicated separate OSB for AQ


Recap… before

SOASuite Server 2



OBTM-Services (WL-Cluster)

Sphere Perf

EXM

Monitor 1 Monitor 2 Monitor 3 Monitor 4

SOASuite Server 1



OSB Console


BTM

OsbserverBTM

Osbserver

AdminServer


SOASuite Server 2



OBTM-Services (WL-Domain)

Sphere

SLMEXM Monitor 1

Monitor 2 Monitor 3 Monitor 4

SOASuite Server 1


OSB-Domain (WL-Domain + Cluster)

OSB Console


BTM

OsbserverBTM

Osbserver

AdminServer

Weblogic Server(3)


OSB Console

OSB Server 3

BTM

Osbserver

Load Balancer

Monitor 5

Monitor 6

Load Balancer

And, after…


Sizing

● 2,5m messages per month

● #logged messages: 2,5 *6 = 15m

● #logged message props: 15M * 3 = 45m

● Size messages: 15M * 6k = 90G

● Size props: 45m * 1k = 45G

● Total size db with 1 month retention: 135G


Reporting

● New release supports BI publisher

● Basic reporting implemented

– Using BTMCLI to extract data from BTM

– Data is loaded in flat table to facilitate simple

querying.

– Reporting on #processed btx per hour.

● Full reporting planned with OBI EE


Results

Problem

● message flow statistics

● Message tracking

● Problem resolution

● Non invasive

● Soasuite10g

uninstrumented

● AQ

Solution

BTM reporting

BTM console: search

content

BTM console / alerting

Observer

OSB proxy

Upgrade 11g


BTM delivers

● Non invasive monitoring

● Monitor uninstrumented components via

proxy config

● Low performance overhead

● Scalable

● Visibility


Future enhancements….

Performance & scalability– Monitored Components: 10,000

• >6x improvement over 11.0

– Services: 5000 - 6000

– Request Throughput: Essentially unlimited

– Transaction Definitions: 100 to 500

• 25x increase from 11.0

Integration- RUEI- EMGC

Extending Platform support- EJB 3.0- WAS 7- Fusion Apps: ADF BC- Jboss 5- .NET 4- BusinessWorks 5.8


Demos

● Moscone South SR 224; Demo #8064

● Online: bit.ly/demoBTM

Contact me…

● [email protected]

● twitter.com/TonyvanEsch

http://bit.ly/demoBTM

http://bit.ly/demoBTM

mailto:[email protected]?subject=OOW presentation BTM

http://twitter.com/TonyvanEsch

http://twitter.com/TonyvanEsch

demystifying soa management

Documents