esb and soa
DESCRIPTION
Paul's presentation at SOA Workshop,Colombo,Sri Lanka identifies how ESBs fit into a Service Oriented Architecture, discusses when to use an ESB and when not to, looks at ESB patterns and anti-patterns, covers some simple ESB approaches and investigates how ESBs can fit into EDA.TRANSCRIPT
![Page 1: ESB and SOA](https://reader034.vdocument.in/reader034/viewer/2022050804/546ea326b4af9fbe268b46c1/html5/thumbnails/1.jpg)
Enterprise Service Bus (ESB) and SOA
Paul [email protected]
CTO and Co-Founder, WSO2VP, Apache Synapse
![Page 2: ESB and SOA](https://reader034.vdocument.in/reader034/viewer/2022050804/546ea326b4af9fbe268b46c1/html5/thumbnails/2.jpg)
SOA – Enterprise Expectations
![Page 3: ESB and SOA](https://reader034.vdocument.in/reader034/viewer/2022050804/546ea326b4af9fbe268b46c1/html5/thumbnails/3.jpg)
SOA & SOAP Core Axis: WS with Apache Axis2© WSO2 Inc. 2006
3
• A set of extensible and composable standards that work together
• Providing a common, standard, interoperable base to implement
SOA
Standards
![Page 4: ESB and SOA](https://reader034.vdocument.in/reader034/viewer/2022050804/546ea326b4af9fbe268b46c1/html5/thumbnails/4.jpg)
SOA & SOAP Core Axis: WS with Apache Axis2© WSO2 Inc. 2006
4
A Sample SOAP Message
<soap:Envelope
xmlns:soap="http://schemas.xmlsoap.org/soap/envel
ope/">
<soap:Header/>
<soap:Body>
<getProductDetails
xmlns="http://warehouse.example.com/ws">
<productID>827635</productID>
</getProductDetails>
</soap:Body>
</soap:Envelope>
![Page 5: ESB and SOA](https://reader034.vdocument.in/reader034/viewer/2022050804/546ea326b4af9fbe268b46c1/html5/thumbnails/5.jpg)
SOA & SOAP Core Axis: WS with Apache Axis2© WSO2 Inc. 2006
5
Key SOA Standards
SOAP
HTTP, TCP, SMTP, UDP
WS-Addressing
SECURE
WS-Security
(encrypt, sign, auth)
WS-SecureConversation
(~SSL)
WS-Trust
XACML
RELIABLE
WS-ReliableMessaging
(ExactlyOnce,
InOrder delivery, ~JMS)
TRANSACTIONAL
WS-AtomicTransaction
(2 Phase Commit ~XA)
WS-BusinessActivity
(Long running loosely coupled)
Business Process Execution Language (BPEL)
WSD
LW
S-P
olicy
Transport
Messaging
Composable
Quality
of
Service
Orchestration
WS-M
eta
data
Exchange
Wire interaction Metadata
The Web services platform forms a complete framework for open
standards enterprise middleware
![Page 6: ESB and SOA](https://reader034.vdocument.in/reader034/viewer/2022050804/546ea326b4af9fbe268b46c1/html5/thumbnails/6.jpg)
Other key approaches for SOA
• JMS– MQSeries, Tibco, Sonic– ActiveMQ, AMQP, Apache QPid– Loosely coupled, asynchronous reliable message passing
• REST– A style of using HTTP – Representational State Transfer– Most HTTP applications are NOT “RESTful”– Key characteristics of a RESTful app:
• Use HTTP Verbs properly• Use mime-types properly• Use linking everywhere• Permanent URLs
![Page 7: ESB and SOA](https://reader034.vdocument.in/reader034/viewer/2022050804/546ea326b4af9fbe268b46c1/html5/thumbnails/7.jpg)
Bus concept
![Page 8: ESB and SOA](https://reader034.vdocument.in/reader034/viewer/2022050804/546ea326b4af9fbe268b46c1/html5/thumbnails/8.jpg)
Busbar
![Page 9: ESB and SOA](https://reader034.vdocument.in/reader034/viewer/2022050804/546ea326b4af9fbe268b46c1/html5/thumbnails/9.jpg)
A common ESB definition
“Any to any data connectivity and transformation (including Web Services) built on an advanced, proven, reliable middleware infrastructure”
![Page 10: ESB and SOA](https://reader034.vdocument.in/reader034/viewer/2022050804/546ea326b4af9fbe268b46c1/html5/thumbnails/10.jpg)
ESB definition
“Any to any data connectivity and transformation (including Web Services) built on an advanced, proven, reliable middleware infrastructure”
which means• Our existing middleware re-branded as an SOA
platform, with some new web services adapters at the edges
![Page 11: ESB and SOA](https://reader034.vdocument.in/reader034/viewer/2022050804/546ea326b4af9fbe268b46c1/html5/thumbnails/11.jpg)
Jason Bloomberg, Zapthink
“You're a software vendor with a product line chock fullof proprietary, tightly-coupled integration middleware…Your software, however, does not lend itself to SOA bestpractices – loose coupling, composable Services, andflexibility in general are all capabilities that you failed tobuild into your software…What to do? The only option is to slap Web Servicesinterfaces on your stuff, call it an Enterprise Service Bus(ESB), and sell it as SOA middleware. Hopefully yourcustomers won't notice the old wine in new bottles.After all, that's what marketing is for!”
![Page 12: ESB and SOA](https://reader034.vdocument.in/reader034/viewer/2022050804/546ea326b4af9fbe268b46c1/html5/thumbnails/12.jpg)
My definition of an ESB
“What does it do?” not “How does it do it?”• Services are independent of the transport and protocol
used to access them• Monitors and manages services with minimal intrusion• Transforms and mediates messages• Bridges between different message exchange patterns• Helps implement service governance• Provides a common backbone to the SOA
![Page 13: ESB and SOA](https://reader034.vdocument.in/reader034/viewer/2022050804/546ea326b4af9fbe268b46c1/html5/thumbnails/13.jpg)
Do you need an ESB to do SOA?
![Page 14: ESB and SOA](https://reader034.vdocument.in/reader034/viewer/2022050804/546ea326b4af9fbe268b46c1/html5/thumbnails/14.jpg)
Do you need an ESB to do SOA?
![Page 15: ESB and SOA](https://reader034.vdocument.in/reader034/viewer/2022050804/546ea326b4af9fbe268b46c1/html5/thumbnails/15.jpg)
SOA can end up as spaghetti
• Too many point-to-point links• Multiple protocols, different qualities of service• No clear picture of all available services
![Page 16: ESB and SOA](https://reader034.vdocument.in/reader034/viewer/2022050804/546ea326b4af9fbe268b46c1/html5/thumbnails/16.jpg)
An ESB can simplify SOA deployment
IntegratedRegistry/
Repository
Web-based console
VirtualizationPerf Mgmt•Load balance•ThrottleTransport matching
Access controlMessage
transformLogging and auditability
![Page 17: ESB and SOA](https://reader034.vdocument.in/reader034/viewer/2022050804/546ea326b4af9fbe268b46c1/html5/thumbnails/17.jpg)
ESB Patterns and Anti-Patterns
• How are ESBs used effectively • How are they abused
– High-level patterns• How the ESB fits into an organization• How the ESB fits into an Enterprise Architecture
– Low-level patterns• How the ESB fits into a specific message flow or business problem
![Page 18: ESB and SOA](https://reader034.vdocument.in/reader034/viewer/2022050804/546ea326b4af9fbe268b46c1/html5/thumbnails/18.jpg)
The Concentrator Pattern
.NET service
CRMservice
ApacheAxis2
service
C/C++service
Concentrator ESBConsistent access, security, logging, audit, monitoring
But no transformation
Data service
Mashup/Web ApplicationDashboard
![Page 19: ESB and SOA](https://reader034.vdocument.in/reader034/viewer/2022050804/546ea326b4af9fbe268b46c1/html5/thumbnails/19.jpg)
The Federated ESB pattern
Enterprise ESBRouting, Audit
DepartmentESB
DepartmentESB
DepartmentESB
![Page 20: ESB and SOA](https://reader034.vdocument.in/reader034/viewer/2022050804/546ea326b4af9fbe268b46c1/html5/thumbnails/20.jpg)
The mini-ESB pattern
• Use a lightweight ESB• Co-locate on the same hardware/VM as the service• Transformation, polling, protocol translation, etc• Why?
– In the control of the team who own the services– Keeps the SOA model (ownership) with a simple effective
approach to exposing services
• What about the embedded ESB model? – e.g. embed transformation and logging into your Service
Hosting platform
![Page 21: ESB and SOA](https://reader034.vdocument.in/reader034/viewer/2022050804/546ea326b4af9fbe268b46c1/html5/thumbnails/21.jpg)
Active vs Passive
• The concentrator pattern is effectively passive– The ESB reacts to messages/requests from the front-end
• Active ESBs:– Poll file systems/FTP/SFTP for updated work– Actively call remote services based on timers– Integrate passive services
![Page 22: ESB and SOA](https://reader034.vdocument.in/reader034/viewer/2022050804/546ea326b4af9fbe268b46c1/html5/thumbnails/22.jpg)
Scenario – Financial Security blocking
Database
legacyflat file
NEW YORK
Existing System
WSO2 ESBPoll
Record->XMLXML->XML
Send
LONDON
WSO2 ESBSplit/Iterate
DBLookup/FilterTransform to MQ
Send
Existing System
XML/JMS
![Page 23: ESB and SOA](https://reader034.vdocument.in/reader034/viewer/2022050804/546ea326b4af9fbe268b46c1/html5/thumbnails/23.jpg)
Push-Me Pull-You
![Page 24: ESB and SOA](https://reader034.vdocument.in/reader034/viewer/2022050804/546ea326b4af9fbe268b46c1/html5/thumbnails/24.jpg)
Push-me Pull-you
![Page 25: ESB and SOA](https://reader034.vdocument.in/reader034/viewer/2022050804/546ea326b4af9fbe268b46c1/html5/thumbnails/25.jpg)
Anti-Patterns
• Anti-Pattern #1 – Implement all your business logic in the ESB– Why not?
• Mixing Infrastructure logic and Business Logic• Maintainability• Tooling and Skills
• Anti-Pattern #2– Apply waterfall and application deployment approaches to
the ESB• Long project cycles• No iterative approach
– Why not?• Lose all flexibility and agility• Once the ESB becomes a static, code-driven system then you would
be better off updating your applications
![Page 26: ESB and SOA](https://reader034.vdocument.in/reader034/viewer/2022050804/546ea326b4af9fbe268b46c1/html5/thumbnails/26.jpg)
Anti-Pattern #3
• “Big Brother”– The ESB is hosted, managed and controlled by a central IT
team– Because of organizational issues using the ESB is complex:
• e.g. – It takes months of meetings to get access– The Central IT team is trying to recoup the investment and internally
charges $000/year to use the ESB– The central IT deployment model holds up users
– Departments and divisions actually sneak behind the ESB• Set up peer-to-peer communications• Avoid the ESB at all costs
![Page 27: ESB and SOA](https://reader034.vdocument.in/reader034/viewer/2022050804/546ea326b4af9fbe268b46c1/html5/thumbnails/27.jpg)
The biggest Anti-Pattern of all
• Use an ESB because:– You heard it was a good idea– The salesman told you that you need one (over a nice
dinner)– You need a new TLA on your resume/CV– Its an excuse to spend several months learning and going to
conferences
![Page 28: ESB and SOA](https://reader034.vdocument.in/reader034/viewer/2022050804/546ea326b4af9fbe268b46c1/html5/thumbnails/28.jpg)
ESB Market
• Proprietary– IBM WebSphere– Oracle/BEA– Tibco
• Open Source– Fuse/ServiceMix– MuleSource/Mule– WSO2 ESB/Synapse
![Page 29: ESB and SOA](https://reader034.vdocument.in/reader034/viewer/2022050804/546ea326b4af9fbe268b46c1/html5/thumbnails/29.jpg)
![Page 30: ESB and SOA](https://reader034.vdocument.in/reader034/viewer/2022050804/546ea326b4af9fbe268b46c1/html5/thumbnails/30.jpg)
Core model of the ESB
• Two main approaches– “Proxy” approach
• Messages come into a proxy– Proxy = { inSequence, targetEndpoint, outSequence, faultSequence}– Sequence = { ordered list of mediators }– Mediator = Unit of function
– Rule/Policy based approach• All messages come to a central sequence
– Sequence categorizes and routes requests based on the message
• Known in Synapse as the “main” sequence
![Page 31: ESB and SOA](https://reader034.vdocument.in/reader034/viewer/2022050804/546ea326b4af9fbe268b46c1/html5/thumbnails/31.jpg)
Sequence concept
![Page 32: ESB and SOA](https://reader034.vdocument.in/reader034/viewer/2022050804/546ea326b4af9fbe268b46c1/html5/thumbnails/32.jpg)
Built-in Mediators
– Drop (end)– Sequence (call another
sequence)– Clone – Callout (call a WS)– Filter (if-then-else)– Switch– Iterate– Aggregate– Send– Router– Smooks (transform library)– Rule (use a Rules engine)– Entitlement (validate access
against a XACML server)
– Property– Header– Validate– DBReport– DBLookup– Class mediator– Command Mediator– Script– Spring– Throttle– Cache– XSLT– XQuery
![Page 33: ESB and SOA](https://reader034.vdocument.in/reader034/viewer/2022050804/546ea326b4af9fbe268b46c1/html5/thumbnails/33.jpg)
Understanding performance
• Performance of an ESB can be critical• Key Measurements are
– Throughput (can the ESB cope with the required load)– Latency (does the ESB add unacceptable time)– Concurrency (does the ESB run out of threads)
• Two key technologies– Streaming and Streaming XML
• Ability to operate in constant memory and handle XML without building a full tree
– Non-blocking IO• Ability to manage large numbers of connections with constant
thread pool
![Page 34: ESB and SOA](https://reader034.vdocument.in/reader034/viewer/2022050804/546ea326b4af9fbe268b46c1/html5/thumbnails/34.jpg)
Case Study
• Mobile phone ringtone/media provider• Every request goes via the ESB
– Performance and Streaming are critical– Streaming and non-blocking are key
• Continuous Availability – Ability to upgrade the system live– Without losing transactions– Under load
![Page 35: ESB and SOA](https://reader034.vdocument.in/reader034/viewer/2022050804/546ea326b4af9fbe268b46c1/html5/thumbnails/35.jpg)
Governance and ESBs
• Governance is a key issue for SOA• Governance is fundamentally about ensuring standards
around Enterprise IT– Policies– People – Processes
• ESBs can be very important for this– Policy Enforcement Point– Monitoring Point– Central Access Point for enterprise services
![Page 36: ESB and SOA](https://reader034.vdocument.in/reader034/viewer/2022050804/546ea326b4af9fbe268b46c1/html5/thumbnails/36.jpg)
Summary
• We have– Identified how ESBs fit into a Service Oriented Architecture– Discussed when to use an ESB and when not to– Looked at ESB patterns and anti-patterns– Covered some simple ESB approaches– Investigated how ESBs can fit into EDA
![Page 37: ESB and SOA](https://reader034.vdocument.in/reader034/viewer/2022050804/546ea326b4af9fbe268b46c1/html5/thumbnails/37.jpg)
Questions