High Volume Web API Managementwith WSO2 ESB

Paul Fremantle, CTO and Co-FounderHiranya Jayathilaka, Associate Technical Lead

© WSO2 2011. Not for redistribution.

lean enterprise middleware

WSO2 Offerings• WSO2 Carbon– On premise product platform– Consistent products sharing the same OSGi core

• WSO2 Stratos– A “Cloud-Enabled Application Platform” (CEAP)– Multi-tenant, Elastic, Metered/Billed deployment– An extension of the Carbon platform

• Same SVN code tree

• WSO2 StratosLive– http://cloud.wso2.com– A running Platform-as-a-Service (PaaS)– An instance of Stratos

© WSO2 2011

ForresterWaveTM ESB Q2 2011 - Leader

© WSO2 2011

WSO2 ESB 4.0

WSO2 ESB Features• The WSO2 Enterprise Service Bus offers:– Easy configuration through an intuitive graphical interface – Lean, high-performance design conserves resources through

maximum efficiency.– Built on the WSO2 Carbon OSGi platform: adding new capabilities

as you need them is a snap.– Integrates with your existing systems, with support for EDI, CSV,

Files, FIX, HL7, SAP, JMS, etc• Asynchronous architecture supports very high loads with

fixed resources• Routing, mediation and transformation are modular,

extensible but provide most use-cases simply by sequencing built-in mediators

• Full support for SOAP/WS-* and REST

© WSO2 2011

Key Features for High Volume Web API

• Load Balancing and Failover built-in– Pluggable models can include affinity based LB

• Full REST Support (XML and JSON)– Especially used in Mobile environments– Can bridge into SOAP or other enterprise systems

• Message Relay Mode and Relay Transport– Excellent performance for 100% streaming

• Priority Execution, Cache and Throttling– Manage workloads within the ESB as well as throttle

backend– Cache content locally if appropriate

• Service Chaining– Asynchronous lightweight orchestration

© WSO2 2011

eBay Case Study – High Volume• Requirements:

– “The solutions we were using no longer met our needs, so we considered whether to build a new internal system or to adopt third-party technology”, Abhinav Kumar, eBay

– Needed enhanced service mediation and orchestration capabilities – Scalability and performance to sustain increasing loads

• Evaluation:– eBay evaluated several industry leading hardware and software ESB solutions, looking at

open source, commercially licensed ESBs and hardware– Test production during 2009 holiday season proved that our system was rock solid– 1m messages/day with zero downtime, zero memory growth

• Results– Shared-nothing architecture on RedHat Linux on x86 hardware– Currently running at more than 1 billion messages / day – Low resource utilization and flat memory usage

• The benefits– “Using the WSO2 ESBs, we've been able to provide customers and partners with the

quality experience they expect on eBay, even as our global customer base has grown,” Abhinav Kumar

© WSO2 2011

Memory Usage at eBay

© WSO2 2011

Deployment

© WSO2 2011

Configuring for High Volume

• Operating System • JVM• ESB• Management

© WSO2 2011

Tuning the OS

Optimize core network settings for high throughput network activity /etc/sysctl.conf

Increase the number of open file descriptors allowed by the OS /etc/security/limits.conf

Tuning the JVM

Allocate sufficient memory for the heap -Xms256m -Xmx2048m -XX:MaxPermSize=256m

Consider reducing the new ratio -XX:NewRatio=n

Consider using the concurrent mark and sweep collector

-XX:+UseConcMarkSweepGC

Read more on JVM and GC tuning http://wso2.org/library/articles/2010/11/taming-java-garbage-

collector

Tuning the ESB

Configure transport thread pools Configured through nhttp.properties file IO dispatcher threads carry out network IO at the

wire level Recommended to have one IO dispatcher per CPU core

Server workers and client workers mediate the messages More the merrier (But keep memory usage and load in

mind)

Message Relay Mode

• WSO2 ESB uses Apache AXIOM and the StAX API for processing XML• XML payloads are streamed through the ESB• Pull parsing model

• But for pure routing, load-balancing and header-only mediation we can avoid even this step

• Message relay mode works around this glitch and enables 100% pure streaming of messages• Works regardless of size and format of messages

Enabling Message Relay

• Enable the binary relay builder (axis2.xml)• Instructs the ESB to stream the incoming messages through

without touching their payloads• Enable the expanding message formatter (axis2.xml)

• Allows the ESB to send messages that were received through the binary relay builder

<messageBuilder contentType="application/xml" class="org.wso2.carbon.relay.BinaryRelayBuilder"/>

<messageFormatter contentType="application/xml" class="org.wso2.carbon.relay.ExpandingMessageFormatter"/>

• Above configuration tells the ESB to process all application/xml messages in message relay mode.

HTTP Relay Transport

• Brand new in WSO2 ESB 4.0 release• Non blocking HTTP transport implementation

specially designed for streaming messages• Doesn’t require the binary relay builder and

expanding formatter• To enable, simply uncomment the relevant

entries in the axis2.xml file

Error Handling

In a large scale, high throughput deployment, errors are not unusual – Expect the unexpected

Configure endpoints to gracefully handle the common errors Connection timeouts Connection close/reset Connection refused HTTP protocol violations!!!!

More on Error Handling

Configure sequences to clearly log errors and if needed notify system administrators

In case the back end server fails to respond, send detailed fault responses to clients – Makes your application more appealing to the customers

Pay attention to HTTP error codes

Monitoring

In general keep an eye on: CPU usage Memory usage Thread counts Fault counts Latency/Response time Active connections

Monitoring Tools

Utilities Provided by OS top netstat

Mediation statistics JMX Clients

Jconsole

Connections

© WSO2 2011

Endpoints

© WSO2 2011

Latency

© WSO2 2011

Listeners

© WSO2 2011

Memory and Garbage Collection

© WSO2 2011

S2S Latency

© WSO2 2011

Threads

© WSO2 2011

Questions?

28

http://www.flickr.com/photos/oberazzi/

Resources

• http://wso2.com/landing/ebay-the-esb-casestudy/

• WSO2 ESB 4.0 Download and Library– http://wso2.org/library/esb

• Message Relay Mode– http://wso2.org/project/esb/java/4.0.0/docs/

message_relay.html

• Twitter: @pzfreo

• Email: paul@wso2.com hiranya@wso2.com © WSO2 2011