easy enterprise application integration with rabbitmq and amqp
DESCRIPTION
VMware vFabric RabbitMQ Technical Webinar December 2010 by VMware engineer Emile Joubert. Covers common integration patterns, and how RabbitMQ makes these easily implemented, using AMQP as a communications mechanism. You can view a recording of this presentation on YouTube: http://www.youtube.com/user/SpringSourceDev#p/c/5956C6D9EC319817/0/ABGMjX4K0D8TRANSCRIPT
© 2010 VMware Inc. All rights reserved
Emile Joubert
December 2010
The Advanced Message Queuing Protocol (AMQP) is an application layer
protocol for messaging. This talk introduces RabbitMQ, itsprominent multi-lingual open-source implementation.
© 2010 VMware Inc. All rights reserved
Disclaimer
This session may contain product features that are currently under development.
This session/overview of the new technology represents no commitment from VMware to deliver these features in any generally available product. Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.
Technical feasibility and market demand will affect final delivery. Pricing and packaging for any new technologies or features discussed or presented have not been determined.
"THESE FEATURES ARE REPRESENTATIVE OF FEATURE AREAS UNDER DEVELOPMENT. FEATURE COMMITMENTS ARE SUBJECT TO CHANGE, AND MUST NOT BE INCLUDED IN CONTRACTS, PURCHASE ORDERS, OR SALES AGREEMENTS OF ANY KIND. TECHNICAL FEASIBILITY AND MARKET DEMAND WILL AFFECT FINAL."
© 2010 VMware Inc. All rights reserved
About me...
Emile studied computer science and mathematics before following a career that included electronic publishing; joining LShift in 2008 and SpringSource in 2010.
© 2010 VMware Inc. All rights reserved
Outline
• Messaging• AMQP• Publishing walk-through• RabbitMQ and the benefits of Erlang• Large scale sample uses
© 2010 VMware Inc. All rights reserved
Messaging Use Cases
Decoupling
© 2010 VMware Inc. All rights reserved
Messaging Use Cases
Bidirectional Decouplinge.g. Remote Procedure Call
© 2010 VMware Inc. All rights reserved
Messaging Use Cases
Pipelining and Decoupling
© 2010 VMware Inc. All rights reserved
Messaging Use Cases
Work-distribution and Decouplinge.g. duplication and round-robin
© 2010 VMware Inc. All rights reserved
Messaging Use Cases
Work aggregation, distribution and decoupling
© 2010 VMware Inc. All rights reserved
Why Messaging?
SQL : : Databases : : FilesystemAMQP : : Messaging : : Network
• Logical decoupling• Physical decoupling• Temporal decoupling
© 2010 VMware Inc. All rights reserved
AMQP
• An open, wire-level binary protocol for
Message Orientated Middleware• Created by users and technologists
working together, to satisfy real needs• A straight-forward and complete
solution for business messaging: SMTP for business
© 2010 VMware Inc. All rights reserved
Transport
Framing
AMQP Wire Format
AMQP Model
Network-friendly
Network-optimized
Multi-vendorinteroperability
Flexible routing/queuing parameters
AMQP Model
© 2010 VMware Inc. All rights reserved
AMQP Specification Sample
<method name="publish" content="1" index="40"> <chassis name="server" implement="MUST"/> <field name="reserved-1" type="short“ /> <field name="exchange" domain="exchange-name"/> <field name="routing-key" domain="shortstr"/> <field name="mandatory" domain="bit"/> <field name="immediate" domain="bit"/> </method>
© 2010 VMware Inc. All rights reserved
AMQP Model
• Producers• Exchanges• Bindings• Queues• Consumers
© 2010 VMware Inc. All rights reserved
Publishing: Setup
Exchanges, queues & bindings are created dynamically by producers & consumers
© 2010 VMware Inc. All rights reserved
Publishing
© 2010 VMware Inc. All rights reserved
Publishing
© 2010 VMware Inc. All rights reserved
Publishing
Publishing a second message
© 2010 VMware Inc. All rights reserved
Consuming
© 2010 VMware Inc. All rights reserved
Consuming
Consuming second message
© 2010 VMware Inc. All rights reserved
Publishing Multiple Messages
© 2010 VMware Inc. All rights reserved
Publishing Multiple Messages
...to several consumers on the same queue
© 2010 VMware Inc. All rights reserved
Publishing to Multiple Queues
© 2010 VMware Inc. All rights reserved
Publishing to Multiple Queues
Sending
© 2010 VMware Inc. All rights reserved
Publishing to Multiple Queues
Messages go to every queue bond to a fanout exchange
© 2010 VMware Inc. All rights reserved
Publishing to Multiple Queues
Queues can be consumed from at different rates
© 2010 VMware Inc. All rights reserved
Publishing: Topic Routing
* matches 1 word
# matches 0 or more words
© 2010 VMware Inc. All rights reserved
Publishing: Topic Routing
© 2010 VMware Inc. All rights reserved
AMQP Model Elements
Exchange Properties•passive•durable•auto-delete
Exchange Types•direct•fanout•topic•headers
Queue Properties•passive
•durable
•exclusive
•auto-delete
Message Properties•routing key
•immediate
•delivery-mode
•priority
•expiration
© 2010 VMware Inc. All rights reserved
Prominent AMQP Implementations
RabbitMQ (Erlang)
OpenAMQ (C++)
Apache Qpid (Java and C++)
RedHat MRG (Messaging, Realtime & Grid) (C++)
© 2010 VMware Inc. All rights reserved
Why Erlang?
• Concise (binary matching)• Sane concurrency (no shared state)• Built-in distributed database (Mnesia)• Multi-platform (Windows, Unix)
© 2010 VMware Inc. All rights reserved
Erlang binary matching
decode_method('basic.publish', <<F0:16/unsigned, F1Len:8/unsigned, F1:F1Len/binary, F2Len:8/unsigned, F2:F2Len/binary, F3Bits:8>>)
->F3 = (F3Bits band 1) /= 0,F4 = (F3Bits band 2) /= 0, #'basic.publish'{ticket = F0, exchange = F1, routing_key = F2, mandatory = F3, immediate = F4};
© 2010 VMware Inc. All rights reserved
What is RabbitMQ?
RabbitMQ consists of the broker (server) and several clients (Java, .net and others)
They speak the AMQP protocol to each other
Anyone can write other clients which also speak AMQP and work with RabbitMQe.g. Ruby, Java, Spring, Python, C, C#, Perl,
Erlang, Lisp, Haskell, PHP, <your language here>
It's in many popular Linux distributions and can easily be installed on OS X
Released under MPL
© 2010 VMware Inc. All rights reserved
What is RabbitMQ?
17k lines of Erlang code in the brokerSupports clustering for increased
scalabilityCan work with Pacemaker and
associated tools to provide High Availability
Supports several extension points via plug-ins, e.g. exchange types and backing queue storage
© 2010 VMware Inc. All rights reserved
AMQP compared to JMS
AMQP has a wire-level binary protocol, instead of an API
AMQP offers multi-language interoperability
AMQP message payloads are opaqueUse Spring-AMQP to migrate
© 2010 VMware Inc. All rights reserved
Protocols
AMQP (SSL)STOMPXMPP (Rabbiter)0MQREST
© 2010 VMware Inc. All rights reserved
Real-world features
•Unroutable messages•Client acknowledgements•Persistence•Transactions•Flow control•Security
© 2010 VMware Inc. All rights reserved
Management & Monitoring
RabbitMQ plugin
• Inspect and manage resources
• Load & save configuration
• HTTP API
• Commandline tool
© 2010 VMware Inc. All rights reserved
Management & Monitoring
© 2010 VMware Inc. All rights reserved
Management & Monitoring
© 2010 VMware Inc. All rights reserved
Cloud Ready
• Messaging service hosted in cloud• Cloud infrastructure services
• e.g. Nasa Nebula• Providing a service in the cloud
• e.g. SQS on Amazon
© 2010 VMware Inc. All rights reserved
Nebula Architecture
© 2010 VMware Inc. All rights reserved
UIDAI
© 2010 VMware Inc. All rights reserved
UIDAI
Unique Identification Authority of India (UIDAI) will issue a12-digit for all residents
•One of the largest IT projects ever•Largest online identity DB ever•200 million messages per day peak•600 million residents over 4 years
• Spring• Tomcat• Hibernate• MySQL• Hadoop• Mule• Apache httpd• RabbitMQ
© 2010 VMware Inc. All rights reserved
The future
• AMQP 1.0• Connect-a-thon
• <Your suggestion here>• Get in touch!• http://www.rabbitmq.com/
This presentation will be available at
http://www.springsource.com/webinars
from Monday 13 Deccember 2010