be jug 090611_apacheservicemix

Copyright © 2010 Progress So3ware Corpora6on and/or its subsidiaries or affiliates. All rights reserved. 1 A Progress So3ware Company

A Progress So3ware Company

Integra6on with

Apache ServiceMix and Camel

BeJUG, Louvain-‐La-‐Neuve, 9th June 2011

Charles Moulliard Sr. Principal Solu6on Architect Apache CommiOer


Speaker background

Charles Moulliard • Engineer in Agronomy & Master in Zoology

• 17 years of experience in IT world development (J2EE, .NET) • Specialised in new technologies web/Internet & integra6on • Project manager in bank, financial, telco, insurance and transport

world

• Solu6on Architect at FuseSource • CommiOer on projects :

– Apache ServiceMix, – Apache Karaf (PMC) &

– Apache Camel

• Author of book : "ServiceMix and Karaf in Ac6on – Mannings"


Goal

« Introduce some projects of the Apache founda6on Integra6on oriented like Camel, ServiceMix

and

Present target Architectures »


Agenda

Integra6on with Camel and ServiceMix • Demys6fy ESB

• ServiceMix -‐ mul6-‐containers plaform • Presenta6on of projects -‐ Camel, CXF, Karaf et Ac6veMq

• Descrip6on of architectures – Messaging

– Osgi

– Messaging + osgi

– web

• High availability, scalibility and clustering – Ac6veMQ, ServiceMix

– Ac6veMQ and ServiceMix

• Cloud and provisioning


« Demis6fy the ESB and present Apache ServiceMix plaform »


ESB -‐ ServiceMix

Demys6fy the concept -‐ Enterprise Service Bus

→ This is not → Nor a

But an exchanging plaform

of messages, objects moving within a transport layer


ESB -‐ ServiceMix (1)

Allow to separate the business layer from those which will transport the informa6on and transform it

Provide connectors (=endpoints) to collect informa6on and send it on the bus as messages

Messages are transformed using marshallers


ServiceMix -‐ Container ship

ServiceMix – Project of the Apache Founda6on

Started in 2005 to implement the Java Business Integra6on specifica6on

Decoupling of the components (business, technical) by using XML messages

Define a model to package/deploy applica6ons (SU/SA) Layer transport is called Normalised Messaging Router


ServiceMix -‐ Container ship (2)

Message is ACK or ERR

Routing is defined into the SU

Normalized = XML exchanges


ServiceMix -‐ Container ship (3)

JBI approach is interes6ng but restric6ve • XML format

• Encapsula6on of the rou6ng • Components dvlpt (25 vs more than 100 for camel)

• Few support from IT actors (IBM, Oracle, …) to improve exis6ng spec 1.0 2.0

• We can make the parellism with EJB story ….

Decision has been taken 3 years ago by members of project to • Become independent of JBI spec (we support it but op6onal)

• Adopt Camel as the new rou6ng bus

• Propose a new kernel (= meaning a new Architecture design)

• Extend the idea of the project, plaform to become a mul6-‐container or container ship


SMX -‐ Container ship (4)

ServiceMix 4

Based on an OSGI kernel • Apache Felix • Eclipse Equinox

Why → offers modularity, jars management and versioning

ServiceMix becomes an Integra6on Plaform for opensource projects • Camel

• CXF • Ac6veMQ,

• Aries (JPA, Transac6on, JNDI) • ….


Karaf – Heart of ServiceMix

Birthdate -‐ 16th of June 2010 Run6me using an OSGI server :

• Apache Felix • Eclipse Equinox

Provide a lightweight container where Applica6ons Composants

Routes ...

can be deployed


Karaf – Heart of ServiceMix (1)

<<jvm>> :ServiceMix4

Java Boot (Register as service - Wrapper)

Lightweight container (Apache Karaf)

OSGI Kernel (Apache Felix / Eclipse Equinox)

Container Container Container Container Container Container


Karaf – heart of ServiceMix (2)

Caracteris6cs :

Administra6on console (locale, remote, ssh, web, jmx)

Provisioning system (features) Hot deployment and configura6on management

Instances management

Security integra6on (JAAS → ldap, jdbc, file) Logging management (log4j, logger, commons logging, ….)

… let’s have a look with a demo …


Which container (s)

The plaform is « modular » because we can select the librairies, containers that we would like to use

Integration

Web Services

Web

IoC

Middelware Java

Camel, JBI Jetty, Pax Web

CXF

ActiveMQ

Spring, Blueprint


« Camel the Integra6on Framework of the Enterprise PaOerns -‐ EIP rou6ng messages through the bus »


Camel

Birthdate -‐ March 2007

OpenSource Integra6on Framework implemen6ng EIP paOerns (book published in 2005 by Gregor Hohpe)

Why EIP To provide word defini6on, grammar and language when designing integra6on projects

Camel uses a Domain Specific Language transposing the EIP paOerns into Java, XML

It supports: • In memory bus Alterna6ve to JBI using NMR

• Route objects can be XML, File, Stream, Bytes,…

• Transac6onal architecture,

• Sync/Async exchanges, • Threads Management,

• Error and excep6on handling


Camel (1)

More than 50 paOerns are implemented

hOp://camel.apache.org/enterprise-‐integra6on-‐paOerns.html


Camel (2)

More than 100 components

hOp://camel.apache.org/components.html


Camel (3)

18 data formaters to simplify data exchange between layers, applica6ons connected

Example : JAXB, Bindy, SOAP

Uses marshaler (xml object) and unmarsharler (object xml)

Will be typically use in IN, OUT flows

hOp://camel.apache.org/data-‐format.html


Camel (4)

The model –> Content Based Router

and its DSL language


Camel (5)


Camel (6)


Camel (7)


Camel (8)

URI = « component name » and proper6es


Camel

Java DSL


Camel

Route Camel + Java DSL


Camel

Endpoints and predicate can be provided into the camel route directly


Camel

Integra6on with IntelliJ, Eclipse, NetBeans


Camel (9)

Instead of Java, use Spring / XML DSL


Camel (10)

IDE assistance


« CXF – The tool to generate Web Services »


CXF

Merge of 2 projects(Cel6c and Xfire)

Simplify the crea6on and deployment of webservices 2 approaches : java to wsdl or wsdl to java.

Support the following standards : • JAX-‐WS : Web Services (XML/SOAP)

• JAX-‐RS : RESTfull service (JSON)

• SOAP 1.1, 1.2, WSDL 1.1

• WS-‐Security : securise the connexion client/server or vice versa

• WS-‐Addressing : standardize the data exchange in the SOAP Header for the rou6ng

• WS-‐RM : message delivery guaranty


CXF (1)

<plugin> <groupId>org.apache.cxf</groupId> <artifactId>cxf-codegen-plugin</artifactId> <version>${cxf-version}</version> <executions> <execution> <id>generate-sources</id> <phase>generate-sources</phase> <configuration> <sourceRoot> ${basedir}/target/generated/src/main/java </sourceRoot> <wsdlOptions> <wsdlOption> <wsdl> ${basedir}/src/main/resources/report.wsdl </wsdl> </wsdlOption> </wsdlOptions> </configuration> <goals> <goal>wsdl2java</goal> </goals> </execution> </executions> </plugin>

To use it, simply add cxf maven plugin into the pom of the project


CXF (2)

… and configure a spring file

<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:cxf="http://camel.apache.org/schema/cxf" xsi:schemaLocation=" http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd http://camel.apache.org/schema/cxf http://camel.apache.org/schema/cxf/camel-cxf.xsd">

<import resource="classpath:META-INF/cxf/cxf.xml"/> <import resource="classpath:META-INF/cxf/cxf-extension-soap.xml"/> <import resource="classpath:META-INF/cxf/cxf-extension-http-jetty.xml"/>

….

<cxf:cxfEndpoint id="reportIncident" address="http://localhost:9080/incident" wsdlURL="etc/report_incident.wsdl" serviceClass="org.apache.camel.example.ReportIncidentEndpoint"> </cxf:cxfEndpoint>

… </beans>


« Ac6veMQ – broker and messaging factory »


Ac6veMQ -‐ MoM layer

A high performance, reliable messaging fabric, suppor6ng JMS, C, .Net, and other frameworks

Offers persistance for messages and bridge between ServiceMix instances

Clients connect to the broker using connectors with simple URIs

File & JDBC-based message stores supported.

… all based on a flexible Spring-based core.

Topics and queues created dynamically.

Network connectors control how the broker interacts with other brokers for discovery, replication, failover, clustering and store & forward.


FUSE Message Broker (Apache Ac6veMQ)

Point to Point • Sender/producer sends messages to a queue.

• Receiver/consumer listens for messages from the queue • Messages are stored un6l read (or expired)

• Messages can be persisted on disk

• Messages are read only once.

c1:Consumer p:Producer

<<jvm>> frodo:ActiveMQ

Foo:File m

1: Producer sends a message to the broker

m

2: Consumer reads the message


FUSE Message Broker (Apache Ac6veMQ)

Publish to subscribe • A JMS client sends message to the topic.

• The JMS broker sends message to all subscribers that are currently alive.

– Alterna6vely, durable subscrip-ons can be used so that all subscribers will receive message even if not alive at 6me of sending

• Messages are read several 6mes (rela6on 1 to many)

s1:Subscriber

p:Producer


Foo:Topic m

m s2:Subscriber

m


How the Apache products relate to each other…

ServiceMix

ActiveMQ

Camel

CXF

<<supports jbi endpoints>> <<deploy into>>

<<supports cxf endpoints>>

<<supports activemq endpoints>> <<deploy into>>

<<deploy into>>

<<supports transport>> <<supports transport>> <<implements flow with>>


« What are the Architectures that we can design with ServiceMix, Camel, Ac>veMq »


<<jvm>> ServiceMix4

Architecture -‐ Messaging

FTP Web Service

Bean<<MyBean>>

from(FTP) .beanRef(« myBean »);

from(« cxf:bean:myWS ») .beanRef(« myBean ») ;

The bus operates the exchanges between the endpoints using Camel, NMR, JBI

BUS


<<jvm>> ServiceMix4

Architecture -‐ Java

The OSGI plaform proposes a Service Registry (comparable to JNDI) where "interfaces" are registered

Clients access to the service using lookup mechanism

« Interface » DAO

A

OSGI registry B

<bean id="incidentServiceTarget" class="org.apache.camel.service.impl.IncidentServiceImpl"> <property name="incidentDAO"> <osgi:reference interface="org.apache.camel.dao.IncidentDAO"/> </property> </bean>

Service PoJo

C

Hibernate/JPA


<<jvm>> ServiceMix4

OSGI Registry

Service PoJo

« Interface » DAO Hibernate/JPA

Architecture – Messaging + Java

Decoupling of the "services" from integra6on layer

from(« cxf:bean:myWS ») .to(« bean:validate ») .beanRef(«org.bejug.jpaService»);

C

Web Service

B

Bean Validate

A


<<jvm>> ServiceMix4

« Interface » DAO

A

OSGI registry B

Service PoJo

C

Hibernate/JPA

Architecture – Java + Web

ServiceMix is a web plaform, it uses JeOy Web Applica6on Server, Pax Web – a tooling for war and OSGI HTTP Service to register Servlet, …

Web Applica-on

OSGI Run-me

Bundle

Web Service

B

Bean Validate

A


Only one ServiceMix instance

Deployment unit bundle (= jar with addi6onal data in MANIFEST.mf file

Several camel routes can be deployed in different bundles, cross-‐communica6on via NMR bus (name is the same but different from JBI)

NMR


What can I do when x instances are required

When we need asymetric deployment because CPU, memory is not sufficient then we use Ac6veMQ to interconnect the instances and persist messages


« High availability, scalability and clustering »


High-‐availability -‐ Ac6veMQ

2 mechanisms : pure Master/Slave et Shared message store

<<jvm>> samwise:ActiveMQ

Master state replicated with the slave


<<jvm>> samwise:ActiveMQ

The brokers compete to acquire lock on the shared system (file or DB)



High Availability -‐ ServiceMix

Only one mechanism is available – Lock based (File, DB)

Bundles can be deployed in ac6ve or passive mode


High-‐Availability – Amq and Smx

Combine Ac6veMQ (shared store) anf ServiceMix (shared lock)

Network of brokers

Slave

Headquarter

queue:order BUS

Camel - FTP

from(amq:queue:order) .bean://order

BUS

bean://order

queue:order

Master

queue:order

BUS

SMX4 Actif (Master)

queue:order

from(ftp) .to(amq:queue:order)

Regional office

Camel - WS

BUS queue:order

from(ftp) .to(amq:queue:order)

from(amq:queue:order) .bean://order

bean://order

SMX4 Passive (Slave)

NetWork of Broker


Cloud, Provisioning

FuseSource Fabric based on Apache Zookeeper

Distributed coordina6on service highly available using a repository for cluster configs and the node (run6me)

<<Zookeeper:toto>> <<Zookeeper:nestor>> Sync

ACLs Data exchange


Cloud, Provisioning (2)


Provisioning – local and remote

Solu6on to deploy locally or remotelly

Idea : we have a profile defining what we would like to deploy and configure and link it to an agent = instance of ServiceMix

karaf@root> fabric:create-profile --parents default webserver

karaf@root> zk:create -r /fabric/configs/versions/base/profiles/webserver/org.fusesource.fabric.agent/repository.karaf mvn:org.apache.karaf.assemblies.features/standard/2.2.1-SNAPSHOT/xml/features karaf@root> zk:create -r /fabric/configs/versions/base/profiles/webserver/org.fusesource.fabric.agent/feature.war war

karaf@root> fabric:create-agent --profile webserver --parent root test


Cloud

Fabric allows to register camel HTTP endpoints with a logical name (we don't need to know Ip/Hostname)

Provide a new strategy for Web/HOp/ WebServices architecture with loadbalancing between the instances of the cloud

<<Zookeeper:2>>

<from uri="fabric:BeJug: jetty:http://10.0.0.1:9090/"/>

<<Zookeeper:3>>

<from uri="fabric:BeJug: jetty:http://10.0.0.2:9090/"/> <<Zookeeper:1>>

<from uri="direct:A/> <to uri="fabric:BeJug"/>

Discovery mechanism


« End of the first part »


Demo

From theory to « pra6que » Transposi6on of the EIP into camel DSL

Development of the Camel routes Design of the mul6-‐6ers architecture (service, DAO, web services, …)

Implementa6on of the web services, web site Packaging and deployment


Context

hOp://camel.apache.org/tutorial-‐osgi-‐camel-‐part2.html


Code Source

Demo 1 – Integrate camel, web services, web technologies in a Java project with Service, DAO and persistence layer

hOps://github.com/cmoulliard/camel-‐osgi-‐servicemix-‐tutorial

Demo 2 – Use FuseSource Fabric to deploy a project in a cloud architecture and use camel-‐fabric to loadbalance requests between nodes of the cloud space

hOps://github.com/fusesource/fabric/tree/master/fabric-‐examples/fabric-‐camel-‐cluster-‐loadbalancing


Links

Integra6on -‐ Camel : hOp://camel.apache.org

ESB -‐ ServiceMix : hOp://servicemix.apache.org Run6me -‐ Karaf : hOp://karaf.apache.org

WebService -‐ CXF : hOp://cxf.apache.org Broker -‐ Ac6veMQ : hOp://ac6vemq.apache.org

OSGI EE -‐ Aries : hOp://aries.apache.org FuseSource : hOp://fusesource.com/documenta6on/ EIP : hOp://www.enterpriseintegra6onpaOerns.com

Fabric : hOp://fabric.fusesource.org/


Questions ?

More info http://fusesource.com/products/

Follow me on

Twitter : http://twitter.com/cmoulliard

LinkedIn : http://www.linkedin.com/in/charlesmoulliard

My blog : http://cmoulliard.blogspot.com

be jug 090611_apacheservicemix

Technology