1

Scalable messaging with JBoss A-MQ and Camel

Bhavani Ananth@bhavani_anin.linkedin.com/in/bhavaniananth/

2

Naveen Raj Balasubramaniam@Naveenrajbalain.linkedin.com/in/naveenraj/

Messaging And Integration

JBoss A-MQ

JBoss A-MQ – Protocols

JBoss A-MQ – Message Stores

Vertical and Horizontal Scalability

3

Camel in brief!

Agenda

Messaging And Integration

4

Scalable Solution

5

Large Retail Industry

Billing Go Online Vendor

Inventory Warehouse Logistics

Integration Infrastructure

MessagingMessaging

Messaging

Messaging

MessagingMessaging

Data Is Growing Every Second –

Data Deluge

More Data Due to Emergence of

Internet Of Things

Need For A Scalable Solution

What do we have!

Red Hat JBoss A-MQ

7

JBoss A-MQ

8

What is JBoss A-MQ

9

JMS 1.1-compliant messaging system based on Apache Active MQ.

Consists of a broker and client-side libraries enabling remote communication among distributed client applications.

Architecture and Features

10

JBoss-AMQ

Apache Karaf + Fuse Fabric

Monitoring and

Management

Apache Active MQ

STOMP MQTT

Open Wire AMQP NMS

JMS

JBoss A-MQ – Protocols

11

Protocols Supported

12

Client API’s

JMS

NMS

CMS

Rest/Ajax

Wire Protocols

MQTT

AMQP

STOMP

Open Wire

Transports

TCP/SSL

HTTP/HTTPS

WS/WSS

MQTTMQTT is a machine-to-machine (M2M)/"Internet of Things" connectivity protocol. URI- mqtt://Host:[Port]?transportOptions

13

Facebook Messenger Smart Santander Features

Open

Simple

Works well in constrained networks

AMQP

Features

Multi-Vendor interoperability

Highly reliable

Feature rich

AMQP (Advanced Message Queueing Protocol) is an openly published wire specification for asynchronous messaging

URI: “amqp:// Host:[Port]?transportOptions

14

STOMPSTOMP is the Simple (or Streaming) Text Orientated Messaging Protocol.

URI: “stomp://localhost:61613”

15

Broker

Producer

STOMP

Consumer(Ruby,PHP)

TCP

Features

Simple

Used in scripting languages

Openwire

Features

Default wire format used by ActiveMQ.

Efficient

Supports auto reconnect

Supports client failover

OpenWire is the cross language Wire Protocol to allow native access to ActiveMQ from a number of different languages and platforms

16

JBoss A-MQ – Message Stores

17

Message Stores

KahaDB message store

Distributed KahaDB message store

LevelDB message store – the best in terms of scalability and performance

Journaled JDBC adapter

Non-journaled JDBC adapter

18

Message Broker

Message CursorsMessage Store and Message cursors in an Retail

Infrastructure

19

Vendor Systems

System A

System B

Inventory System

Message Store

App Queue 2

Dispatch queue Pending Cursor

App Queue

1

App Queue 3

Queue impl details

Message Broker

Message Cursors – Increase Scalability of Message Stores

Store Based cursors for fast Consumers (Retail Scenario)

20

Dispatch Queue

Pending Cursor

Message Store

Inbound Messages

Vendor Systems

System A

System B

Store Based cursors for slow Consumers

Message Cursors – Increase Scalability of Message Stores Contd…

21

Message Broker

Dispatch Queue

Pending Cursor

Message Store

Inbound Messages

VendorSystems

System A

System B

VMCursors

Message Cursors – Increase Scalability of Message Stores Contd…

<policyEntry queue=">" producerFlowControl="true" memoryLimit="1mb"><pendingQueuePolicy> <vmQueueCursor/> </pendingQueuePolicy></policyEntry>

22

Message Broker

Dispatch Queue

Pending Cursor

Message Store

Inbound Messages

VendorSystems

System A

System B

File Based Cursors

• Does not scale well if the consumers are behind by a large margin

• Used for non persistent messages

Message Cursors – Increase Scalability of Message Stores Contd…

23

Message Broker

Dispatch Queue

Pending Cursor

Message Store

Inbound Messages

File Based Cursors

Temporary Files

VendorSystems

System A

System B

Vertical and Horizontal Scalability

24

Vertical Scalability

Capacity of a single broker to support large numbers of connections from consumers and producers.

To Consider Thread Count CPU Usage Memory Usage

25

Vertical Scalability – Reducing Thread Count

NIO protocol

<transportConnectors><transportConnector name="nio" uri="nio://0.0.0.0:61616"/></<transportConnectors>

26

JBoss A-MQ – Vertical Scalability – Reducing Thread Count

Disable Dedicated Task Runner

ACTIVEMQ_OPTS="-org.apache.activemq.UseDedicatedTaskRunner=false“

Use optimized dispatch for queues<destinationPolicy>

<policyMap><policyEntries>

<policyEntry topic=">" optimizedDispatch="true">

...</policyEntry>

</policyEntries></policyMap>

</destinationPolicy>

27

Vertical Scalability – CPU

Enable tightEncodingEnabled

uri = "failover://(tcp://localhost:61616wireFormat.tightEncodingEnabled=false)";

Enabling compression

tcp://hostA:61617?jms.useCompression=true

28

Vertical Scalability – Memory

Allocate more memory to broker • pass the –Xmx option to the JVM.

ACTIVEMQ_OPTS="-Xmx2048M -Dorg.apache.activemq.UseDedicatedTaskRunner=false“

Configure big enough memory usage

<systemUsage><memoryUsage>

<memoryUsage limit="1024 mb" /></memoryUsage>

</systemUsage>

29

Horizontal Scalability – Network of Brokers

Concepts

Network Connector

Transport Connector

Store and Forward

30

Broker-1

Broker-2

Network Connector

Producer

Transport Connector

Consumer

Transport Connector

Retail Example – Hub and Spoke Topology

31

Store and Forward- Configuration

Store Store

RetailOutlet

Warehouse

Firewall

<networkConnectors> <networkConnector uri="static:(tcp://warehouse:61617)"

userName="admin" password="admin“duplex=“true”conduitSubscriptions=“true”decreaseNetworkConsumerPriority=“false” />

</networkConnectors>

When there are a lot of producers

Brokers, A, B, and C, accept connections from message producers.

Brokers, X and Y, accept connections from message consumers.

Concentrator Topology - Producer Load Balancing

33

A B C

YX

C1 C2 C3 C4

Some Other Topologies

34

Tree Topology

Graph Topology

Mesh Topology

Retail Example-High Availability

Shared Store

Retail Master Broker

Retail Slave Broker

Warehouse Slave Broker

Warehouse

Master Broker

Shared Store

Network Connection on Failure

Netw

ork C

onnection

<networkConnectors> <networkConnector uri="static:(tcp://retail-master:61617, tcp://retail-slave:61617)" </networkConnectors>

<services> <masterConnector remoteURI=“tcp://retail-slave:62001” userName=“admin” password=“admin”/></services>

Other Master Slave Topologies

Master Broker

Slave Broker

Store Store

Message

Shared Nothing Master Slave

Reliable

Only one slave possible

Downtime when master fails

Client

36

Shared Storage Master-Slave

Master Broker with

file lockSlave Broker

Shared File System or

SAN

Shared File system Master Slave

Recommended if SAN is available

Easy to configure

No restriction on the slaves

Client

Slave Broker

Slave polls for lock

37

Hybrid Scaling

38

Client

Acc Payable Master

Acc Payable

Slave

Accounts Payable Information

Acc Receivable Master

Acc Receivable Slave

Accounts Receivable Information

Client Complexity

Less broker cross

talk

Apache CamelA ride to solve your integration problems

39

Camel-What is it

Open source integration framework

Based on Enterprise Integration Patterns(EIPs)

Supports more than 120+ components

Supports DSL programming with Spring, Java, Scala and groovy

Flexible deployment

Can run standalone with maven

40

CBR Wire Tap TransformationEndpoint Translator

Camel-Architecture

41

Camel Coding Structure

import org.apache.camel.builder.RouteBuilder; public class MyRouteBuilder extends RouteBuilder { public void configure() { from("file:SourceFolder?noop=true")

.to("file:TargetFolder"); } }

<camelContext xmlns="http://camel.apache.org/schema/spring"> <route id=“FileToFile"> <from uri="file:SourceFolder?noop=true""/> <to uri="mock:inside"/> </route></camelContext>

Java DSL

Spring DSL

42

Read a file from SourceFolder and write it to TargetFolder

43

<route>     <from uri="activemq:JUDCon?destination.consumer.exclusive=true&amp;destination.consumer.prefetchSize=50"/>    <to uri="mock:2014"/>  </route>

Routes

Camel and Messaging

Configuring the Destination Options at the Endpoint in the Routes

Camel and Messaging Contd…

<bean id="jmsConnectionFactory"    class="org.apache.activemq.ActiveMQConnectionFactory">   <property name="brokerURL" value="tcp://localhost:61616" /></bean> 

Beans

44

 <bean id="activemq"     class="org.apache.activemq.camel.component.ActiveMQComponent">    <property name="configuration" ref="jmsConfig"/>    <property name="transacted" value="true"/>    <property name="cacheLevelName" value="CACHE_CONSUMER" /> </bean>  

 <bean id="pooledConnectionFactory"    class="org.apache.activemq.pool.PooledConnectionFactory" init-method="start" destroy-method="stop">   <property name= "maxConnections" value="8" />   <property name="connectionFactory" ref="jmsConnectionFactory" /></bean>  

 <bean id="jmsConfig"    class="org.apache.camel.component.jms.JmsConfiguration">   <property name="connectionFactory" ref="pooledConnectionFactory"/>   <property name="concurrentConsumers" value="10“ /></bean>

Camel – Scalability Techniques

45

ThrottleLoad

Balancing

from("activemq:queueA").throttle(10).to("activemq:queueB")

<from uri=" activemq:queueA " /><throttle maximumRequestsPerPeriod="3“>  <to uri=" activemq:queueB " />

from(" activemq:queueA ").loadBalance().roundRobin().to(" activemq:queueB ", " activemq:queueC", " activemq:queueD");

<from uri=" activemq:queueA "/><loadBalance>                <roundRobin/>        <to uri=" activemq:queueB"/>                <to uri=" activemq:queueC "/>               <to uri=" activemq:queueD "/>                     </loadBalance>

46

Embedding Camel into Your Broker

•Reduces serialization cost•Reduced network Why?•Using Interceptors to your brokerWhen?

•Add the camel route to activemq.xmlHow?

Summary

• Messaging and Integration• Significance of Asynchronous messaging• How scalability is important to solve large flow of data• Scalability – Vertical and Horizontal• What's camel and Integration

47

Useful Links

References• http://camel.apache.org/architecture.html • http://camel.apache.org/jms.html• http://activemq.apache.org/broker-camel-component.html

48

Thank You

Scale Your Enterprise ECO System

49

@bhavani_an @Naveenrajbala