rabbitmq: what’s new & changing after 10 years of application messaging?
TRANSCRIPT
background image: 960x540 pixels - send to back of slide and set to 80% transparency
RabbitMQ: What’s New & Changing after 10 Years of Application messaging
background image: 960x540 pixels - send to back of slide and set to 80% transparency
What is RabbitMQ? RabbitMQ is an open source message broker and queueing server that can be used to let disparate applications share data via a common protocol, or to simply queue jobs for processing by distributed workers.
background image: 960x540 pixels - send to back of slide and set to 80% transparency
Agenda
1. Brief history of RabbitMQ 2. Common use cases for RabbitMQ 3. RabbitMQ in Microservices Architectures 4. Roadmap
background image: 960x540 pixels - send to back of slide and set to 80% transparency
Agenda
1. Brief history of RabbitMQ 2. Common use cases for RabbitMQ 3. RabbitMQ in Microservices Architectures 4. Roadmap
AMQP spec and Erlang language are key to RabbitMQ’s DNA RabbitMQ Was First Released 10 Years Ago
2004 AMQP standard defined
2006 First version RabbitMQ
2007 Rabbit Technologies founded
2010 Acquisition by VMWare
2013 Becomes part of Pivotal Software
2014 First release on Cloud Foundry
2017 10th Anniversary of first OSS release
RabbitMQ is the most widely used & deployed open source message queue 10 Years of Serving Developers
Frequently searched for... Frequently visited...
Decoupling applications through messaging is a growing need RabbitMQ Adoption Continues to Climb
Consistently increasing job market...
76% YOY increase in downloads
Established user community...
background image: 960x540 pixels - send to back of slide and set to 80% transparency
Agenda
1. Brief history of RabbitMQ 2. Common use cases for RabbitMQ 3. RabbitMQ in Microservices Architectures 4. Roadmap
Typical application messaging patterns deployed by developers Common Use Cases for RabbitMQ
● Decouple apps via async services ○ Use queues as buffers ○ Scale app components individually ○ Enable microservices - RMQ is
small/light enough ● Routing and distributing ● Load balancing ● Cross language barriers using
AMQP standard protocol
Application messaging focused for developers What Makes RabbitMQ Unique
● Open standard protocol ● Open source license ● Language independent ● Authentication (e.g. LDAP) ● Authorisation ● Massive client ecosystem ● Get started quickly ● Grow organically ● Suited to cloud deployment
All major operating systems, clouds, and programming languages are supported Common Ways to Consume RabbitMQ
More than 100 clients, integrations, & plugins from the RabbitMQ community
background image: 960x540 pixels - send to back of slide and set to 80% transparency
Agenda
1. Brief history of RabbitMQ 2. Common use cases for RabbitMQ 3. RabbitMQ in Microservices Architectures 4. Roadmap
RabbitMQ Domain-Specific Messaging Fits Cloud-Native Approach Decoupled Messaging Becomes MSA Requirement
Designed for easy adoption by developers ● For asynchronous enterprise
integration patterns. ● When service buses are too heavy
weight. ● Decouples related applications
Fits microservice architecture needs ● Highly-distributed nature eschews
centralized integration. ● Integration and communication
channels must be decoupled. ● Developers can choose the best
messaging ● RabbitMQ ease of adoption shines for
many (but not all) patterns.
background image: 960x540 pixels - send to back of slide and set to 80% transparency
Agenda
1. Brief history of RabbitMQ 2. Common use cases for RabbitMQ 3. RabbitMQ in Microservices Architectures 4. Roadmap
2016 Review Improvements in Internal Processes
● Ongoing Staff investments ● CI - Pivotal Tracker, Concourse ● Erlang Common Test ● EUnit ● Wider distribution
○ GitHub, Bintray, Package Cloud ○ More ways to consume RMQ
● Wider automated testing ○ Ubuntu, Debian, CentOS,
Fedora ○ More client libs for protocols
● Client version decoupling ● Ongoing community participation
Three main themes in vision RabbitMQ Development Direction
● Operability ○ Admin ease of use (non-developer) ○ Safety Policies ○ Security and compliance
● Scalability ○ RabbitMQ as a service ○ Large clusters ○ Predictable behavior and failure modes
● Expansion and support of programming languages and tools ○ Deeper into core languages and Spring ○ Quality support for community clients
Enablement and empowerment of administrators Operability
In 3.5.x, 3.6
● Lazy Queues ● Tunable Flow Control ● Event Exchange (devops)
In 3.7
● New config format (non-Erlang) ● New CLI tools (Elixir) ● Clustering
○ Pluggable peer discovery ○ Easy restarts
● Simplified/flexible logging
In 3.7 (continued) ● Operator policies & limits
○ Connections, channels, queues ● TLS/x509 Cert Trust Store ● OAuth 2 and OpenID Connect ● Proxy protocol support ● Autocluster improvements
3.8 and beyond
● Toxic message handling ● Pure IPv6 support ● Mixed version clusters (downtime)
Stability and performance in extreme environments Scalability
In 3.5.x, 3.6
● Faster Sync of mirrors - batched ● Reduced RAM usage ● LDAP connection pooling ● Sharding
In 3.7
● Mgmt Plugin refactor ○ Distributed ○ Parallelized
● Increased automated testing (internal) ● Long-running test environments (internal) ● Refactor vs new feature balance weighting
3.8 and beyond
● Message Store: write-ahead ● StatsDB: disk storage ● RAFT-based consensus for queue
mirrors ● Reduce dependency on Erlang Mnesia
DB
Expansion and support of programming languages and tools Client, protocol, and plugin ecosystem
In 3.5.x, 3.6, 3.7
● WebSTOMP ● WebMQTT ● JMS Client
○ Donated by Pivotal ○ Integrated into RMQ core (JMS
Topic Exch) ● Major Java client update
○ Logging and metrics ○ Spring - cross development
■ Reactor - NIO support ■ Spring Boot - client metrics ■ Spring AMQP - new config
options
● Major .NET client update ○ .NET Core support ○ Async / await
● New iOS and MacOS client 3.8 and beyond
● AMQP 1.0 improvements ○ New AMQP 1.0 client ○ Cross-protocol Shovel ○ Cross-protocol Federation
● MQTT improvements
Two avenues to obtain RabbitMQ from Pivotal Commercial Support for RabbitMQ from Pivotal
Pivotal App Suite
● Sold via processor core equivalent
● Includes option to use OSS or Pivotal-branded versions
● Pivotal Premier Support (24x7) ● Full OSS support
● Developer support also available
RabbitMQ for Pivotal Cloud Foundry
● Sold via service instance (cluster) ● Provided with PCF Services Suite ● Shared cluster today, dedicated instances
coming ● Includes support to run outside PCF as well
is within PCF as an automated service