![Page 1: INTEGRATION TO SERVERLESS AN EVENTFUL TOUR FROM … · 2018-05-30 · Enterprise integration patterns for microservices Originally designed for building integrated solutions out of](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece5b4373171a196779bb8c/html5/thumbnails/1.jpg)
AN EVENTFUL TOUR FROM ENTERPRISE INTEGRATION TO SERVERLESS
Marius Bogoevici (@mariusbogoevici)Christian Posta (@christianposta)9 May, 2018
![Page 2: INTEGRATION TO SERVERLESS AN EVENTFUL TOUR FROM … · 2018-05-30 · Enterprise integration patterns for microservices Originally designed for building integrated solutions out of](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece5b4373171a196779bb8c/html5/thumbnails/2.jpg)
About Us
Christian Posta@christianpostaChief Architect - Red Hat
Marius Bogoevici@mariusbogoeviciChief Architect - Red Hat
![Page 3: INTEGRATION TO SERVERLESS AN EVENTFUL TOUR FROM … · 2018-05-30 · Enterprise integration patterns for microservices Originally designed for building integrated solutions out of](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece5b4373171a196779bb8c/html5/thumbnails/3.jpg)
What we’re going to talk about
● Event-driven architectures have been around for a bit; what are they, why are they powerful, and why are they back “en-vogue”
● Messaging is often used as a backbone for event-based distributed systems; what options do we have for cloud-native event-driven architectures?
● Integration is a necessary capability for any organization; how does streaming, cloud-native architectures and microservices fit in?
● Is Functions the next utopian architecture? Where does functions fit in a world of microservices?
![Page 4: INTEGRATION TO SERVERLESS AN EVENTFUL TOUR FROM … · 2018-05-30 · Enterprise integration patterns for microservices Originally designed for building integrated solutions out of](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece5b4373171a196779bb8c/html5/thumbnails/4.jpg)
Unfortunately, we cannot predict the future. As an organization, we must be able to observe and experiment in our environments and react accordingly. We need to be agile.
![Page 5: INTEGRATION TO SERVERLESS AN EVENTFUL TOUR FROM … · 2018-05-30 · Enterprise integration patterns for microservices Originally designed for building integrated solutions out of](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece5b4373171a196779bb8c/html5/thumbnails/5.jpg)
On the other hand we must be mindful of our resources; we want to eliminate waste, reduce time to experiment, and make it cheap so we can increase our returns
![Page 6: INTEGRATION TO SERVERLESS AN EVENTFUL TOUR FROM … · 2018-05-30 · Enterprise integration patterns for microservices Originally designed for building integrated solutions out of](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece5b4373171a196779bb8c/html5/thumbnails/6.jpg)
We cannot build complex systems from complex parts. We must keep our components as simple and understandable as possible.
![Page 7: INTEGRATION TO SERVERLESS AN EVENTFUL TOUR FROM … · 2018-05-30 · Enterprise integration patterns for microservices Originally designed for building integrated solutions out of](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece5b4373171a196779bb8c/html5/thumbnails/7.jpg)
We live in an event-driven world
![Page 8: INTEGRATION TO SERVERLESS AN EVENTFUL TOUR FROM … · 2018-05-30 · Enterprise integration patterns for microservices Originally designed for building integrated solutions out of](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece5b4373171a196779bb8c/html5/thumbnails/8.jpg)
Pop quiz!
Universal panacea Solution to ● technical challenges ● business complexity● bring agility, utilization,
clarity into balance
Event-Driven Architecture is a ...
![Page 9: INTEGRATION TO SERVERLESS AN EVENTFUL TOUR FROM … · 2018-05-30 · Enterprise integration patterns for microservices Originally designed for building integrated solutions out of](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece5b4373171a196779bb8c/html5/thumbnails/9.jpg)
Request-reply and event-driven interaction
Interaction: ephemeral and synchronousHighly coupledLow composabilitySimplified modelLow tolerance to failure
Interaction: persistent and asynchronousDecoupledHighly composableComplex modelHigh tolerance to failure
![Page 10: INTEGRATION TO SERVERLESS AN EVENTFUL TOUR FROM … · 2018-05-30 · Enterprise integration patterns for microservices Originally designed for building integrated solutions out of](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece5b4373171a196779bb8c/html5/thumbnails/10.jpg)
What is an event?
● Action or occurrence, something that happened in the past○ ‘Order created’, ‘user logged in’, ‘
● Event characteristics:○ Immutable○ Optionally persistent○ Shareable
● Event types: [1]○ Notification○ State Transfer (Command)○ Event-Sourcing/CQRS
[1] https://martinfowler.com/articles/201701-event-driven.html
![Page 11: INTEGRATION TO SERVERLESS AN EVENTFUL TOUR FROM … · 2018-05-30 · Enterprise integration patterns for microservices Originally designed for building integrated solutions out of](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece5b4373171a196779bb8c/html5/thumbnails/11.jpg)
Designing systems with events● EDA: event-centric approach in system design
○ Treating events as part of your domain model○ Designing components as event handlers and
emitters● EDA is aligned with the goals of domain-driven design
○ Enforce isolation and decoupling between bounded contexts
○ Properly designed events can create an expressive ubiquitous language
● EDA creates highly observable and extensible systems● Event storming: events-first design
![Page 12: INTEGRATION TO SERVERLESS AN EVENTFUL TOUR FROM … · 2018-05-30 · Enterprise integration patterns for microservices Originally designed for building integrated solutions out of](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece5b4373171a196779bb8c/html5/thumbnails/12.jpg)
Events in the digital business● We live in an event-driven world (literally), and that impacts how we do business● Next-generation digital business is about agility and experimentation
○ Shifting focus from analyzing the status quo to understanding the change in progress
○ Blurring the distinction between events and data○ Architectural focus shifting from data-centric to event-driven
● Increased importance of bottom-up approaches in business event design○ Complex event processing driven by experimentation, analytics, machine
learning○ Emphasis on readiness to observe and collect events before ascribing them
a business meaning
![Page 13: INTEGRATION TO SERVERLESS AN EVENTFUL TOUR FROM … · 2018-05-30 · Enterprise integration patterns for microservices Originally designed for building integrated solutions out of](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece5b4373171a196779bb8c/html5/thumbnails/13.jpg)
Event-driven architectures reduce friction
● From a technical standpoint:○ Building robust and resilient distributed architectures
● From a development process standpoint○ High composability encourage agility and experimentation
● From a business standpoint:○ Aligning digital business with the real world
![Page 14: INTEGRATION TO SERVERLESS AN EVENTFUL TOUR FROM … · 2018-05-30 · Enterprise integration patterns for microservices Originally designed for building integrated solutions out of](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece5b4373171a196779bb8c/html5/thumbnails/14.jpg)
Delivering events through infrastructure
![Page 15: INTEGRATION TO SERVERLESS AN EVENTFUL TOUR FROM … · 2018-05-30 · Enterprise integration patterns for microservices Originally designed for building integrated solutions out of](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece5b4373171a196779bb8c/html5/thumbnails/15.jpg)
Event distribution infrastructure with message brokers
● Publish subscribe semantics (vs queuing)● Subscribers receive events at their own pace● High utilization of consumers, regardless of event publish● Persistent vs non-persistent● Example: ActiveMQ, RabbitMQ, etc
![Page 16: INTEGRATION TO SERVERLESS AN EVENTFUL TOUR FROM … · 2018-05-30 · Enterprise integration patterns for microservices Originally designed for building integrated solutions out of](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece5b4373171a196779bb8c/html5/thumbnails/16.jpg)
Event distribution infrastructure with message brokers
![Page 17: INTEGRATION TO SERVERLESS AN EVENTFUL TOUR FROM … · 2018-05-30 · Enterprise integration patterns for microservices Originally designed for building integrated solutions out of](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece5b4373171a196779bb8c/html5/thumbnails/17.jpg)
Types of events
● User activity tracking● Infrastructure monitoring● Business activity events● Domain events
![Page 18: INTEGRATION TO SERVERLESS AN EVENTFUL TOUR FROM … · 2018-05-30 · Enterprise integration patterns for microservices Originally designed for building integrated solutions out of](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece5b4373171a196779bb8c/html5/thumbnails/18.jpg)
Event distribution infrastructure with message brokers
![Page 19: INTEGRATION TO SERVERLESS AN EVENTFUL TOUR FROM … · 2018-05-30 · Enterprise integration patterns for microservices Originally designed for building integrated solutions out of](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece5b4373171a196779bb8c/html5/thumbnails/19.jpg)
Handling large explosion in event sources, requires optimization in broker technology
● Decentralized processing● Move indexing and bookkeeping to consumers● Make fundamental data structure first class citizen (log data
structure)● Replication and failover part of the protocol● Example: Apache Kafka, Kinesis, etc
![Page 20: INTEGRATION TO SERVERLESS AN EVENTFUL TOUR FROM … · 2018-05-30 · Enterprise integration patterns for microservices Originally designed for building integrated solutions out of](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece5b4373171a196779bb8c/html5/thumbnails/20.jpg)
Event distribution infrastructure with message brokers
![Page 21: INTEGRATION TO SERVERLESS AN EVENTFUL TOUR FROM … · 2018-05-30 · Enterprise integration patterns for microservices Originally designed for building integrated solutions out of](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece5b4373171a196779bb8c/html5/thumbnails/21.jpg)
Using events for integration
![Page 22: INTEGRATION TO SERVERLESS AN EVENTFUL TOUR FROM … · 2018-05-30 · Enterprise integration patterns for microservices Originally designed for building integrated solutions out of](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece5b4373171a196779bb8c/html5/thumbnails/22.jpg)
From ESBs to agile integration
Optimized for utilizationCentralized, tightly coupledMixing logic with infrastructure
Optimized for agilityDecentralized, decoupledSeparate messaging middleware from logic
![Page 23: INTEGRATION TO SERVERLESS AN EVENTFUL TOUR FROM … · 2018-05-30 · Enterprise integration patterns for microservices Originally designed for building integrated solutions out of](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece5b4373171a196779bb8c/html5/thumbnails/23.jpg)
Modern enterprise integration: agile, decentralized, cloud-native
Optimized for agilityDecentralized, decoupledSeparate messaging middleware from logic
Preserves benefits of agility while optimizing resource utilizationClear separation of concerns between compute and data infrastructure and application logic
![Page 24: INTEGRATION TO SERVERLESS AN EVENTFUL TOUR FROM … · 2018-05-30 · Enterprise integration patterns for microservices Originally designed for building integrated solutions out of](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece5b4373171a196779bb8c/html5/thumbnails/24.jpg)
Enterprise integration patterns for microservices
● Originally designed for building integrated solutions out of siloed enterprise systems
● Applicable to general-purpose event-driven interaction● Very well suited for building event-oriented distributed systems
(aka event-driven microservices) - e.g. with Apache Camel
![Page 25: INTEGRATION TO SERVERLESS AN EVENTFUL TOUR FROM … · 2018-05-30 · Enterprise integration patterns for microservices Originally designed for building integrated solutions out of](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece5b4373171a196779bb8c/html5/thumbnails/25.jpg)
Enterprise Integration and Streaming
● Perpetual data and event “streams” as a first class citizen● Data in aggregate vs individual messages● Small services working together to interpret large numbers of
streams● Data in perpetual motion● Eventual consistency as data synchronization pattern● Examples: Apache Camel, Kafka Streams, stream-processing
frameworks
![Page 26: INTEGRATION TO SERVERLESS AN EVENTFUL TOUR FROM … · 2018-05-30 · Enterprise integration patterns for microservices Originally designed for building integrated solutions out of](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece5b4373171a196779bb8c/html5/thumbnails/26.jpg)
Modern enterprise integration: agile, decentralized, cloud-native
![Page 27: INTEGRATION TO SERVERLESS AN EVENTFUL TOUR FROM … · 2018-05-30 · Enterprise integration patterns for microservices Originally designed for building integrated solutions out of](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece5b4373171a196779bb8c/html5/thumbnails/27.jpg)
Microservices and Functions
![Page 28: INTEGRATION TO SERVERLESS AN EVENTFUL TOUR FROM … · 2018-05-30 · Enterprise integration patterns for microservices Originally designed for building integrated solutions out of](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece5b4373171a196779bb8c/html5/thumbnails/28.jpg)
● Reduce the responsibility of a service to a specific business functionality
● Allow parallelization of work ● Independently deployable (infra) and independently releasable
(business)● Can optimize for increased utilization (separating out parts of the
code base that exhibit different i/o, throughput, latency needs)● Re-use services where applicable
Microservices
![Page 29: INTEGRATION TO SERVERLESS AN EVENTFUL TOUR FROM … · 2018-05-30 · Enterprise integration patterns for microservices Originally designed for building integrated solutions out of](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece5b4373171a196779bb8c/html5/thumbnails/29.jpg)
Microservices
![Page 30: INTEGRATION TO SERVERLESS AN EVENTFUL TOUR FROM … · 2018-05-30 · Enterprise integration patterns for microservices Originally designed for building integrated solutions out of](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece5b4373171a196779bb8c/html5/thumbnails/30.jpg)
● Reduce overhead in running services● Higher density/utilization gains● Portable across deployment platforms● Rich ecosystem (see Kubernetes!)
Containerization
![Page 31: INTEGRATION TO SERVERLESS AN EVENTFUL TOUR FROM … · 2018-05-30 · Enterprise integration patterns for microservices Originally designed for building integrated solutions out of](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece5b4373171a196779bb8c/html5/thumbnails/31.jpg)
Containers and microservices
![Page 32: INTEGRATION TO SERVERLESS AN EVENTFUL TOUR FROM … · 2018-05-30 · Enterprise integration patterns for microservices Originally designed for building integrated solutions out of](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece5b4373171a196779bb8c/html5/thumbnails/32.jpg)
● Microservices great at enabling agility from existing systems● Well understood business, well understood boundaries● Don’t optimize for microservices unless you have problems with
your application architecture● Don’t complicate experimentation / value discovery with complex
architecture
What’s your usecase?
![Page 33: INTEGRATION TO SERVERLESS AN EVENTFUL TOUR FROM … · 2018-05-30 · Enterprise integration patterns for microservices Originally designed for building integrated solutions out of](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece5b4373171a196779bb8c/html5/thumbnails/33.jpg)
● Usually not well understood● MVPs are throwaway● Usage patterns unknown● Adoption unpredictable
Exploratory use cases
![Page 34: INTEGRATION TO SERVERLESS AN EVENTFUL TOUR FROM … · 2018-05-30 · Enterprise integration patterns for microservices Originally designed for building integrated solutions out of](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece5b4373171a196779bb8c/html5/thumbnails/34.jpg)
● Low number of hours/minutes of use ● Event-driven, spikey utilization● Lots of compute for very short period of time
Under-utilization use cases
![Page 35: INTEGRATION TO SERVERLESS AN EVENTFUL TOUR FROM … · 2018-05-30 · Enterprise integration patterns for microservices Originally designed for building integrated solutions out of](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece5b4373171a196779bb8c/html5/thumbnails/35.jpg)
● Webhook callbacks● Scheduled tasks ● File processing● Reacting to database changes● Limited stream processing
Limited integration use cases
![Page 36: INTEGRATION TO SERVERLESS AN EVENTFUL TOUR FROM … · 2018-05-30 · Enterprise integration patterns for microservices Originally designed for building integrated solutions out of](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece5b4373171a196779bb8c/html5/thumbnails/36.jpg)
● Pay only for usage without regard for topology (Serverless)
● Event driven by nature● On demand● Write only code, heavy lifting is
handled for you● High parallelization● High utilization
Functions as a Service (FaaS)
![Page 37: INTEGRATION TO SERVERLESS AN EVENTFUL TOUR FROM … · 2018-05-30 · Enterprise integration patterns for microservices Originally designed for building integrated solutions out of](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece5b4373171a196779bb8c/html5/thumbnails/37.jpg)
Functions as a Service (FaaS)
![Page 38: INTEGRATION TO SERVERLESS AN EVENTFUL TOUR FROM … · 2018-05-30 · Enterprise integration patterns for microservices Originally designed for building integrated solutions out of](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece5b4373171a196779bb8c/html5/thumbnails/38.jpg)
● Event-driven microservices● Containers (on prem, in the cloud)● Functions ● Serverless (databases, message queues, caches, etc)
As you move to cloud native, you have options.
![Page 39: INTEGRATION TO SERVERLESS AN EVENTFUL TOUR FROM … · 2018-05-30 · Enterprise integration patterns for microservices Originally designed for building integrated solutions out of](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece5b4373171a196779bb8c/html5/thumbnails/39.jpg)
Thanks for coming to our talk!
Christian Posta@christianpostaChief Architect - Red Hat
Marius Bogoevici@mariusbogoeviciPlatform Architect - Red Hat
![Page 40: INTEGRATION TO SERVERLESS AN EVENTFUL TOUR FROM … · 2018-05-30 · Enterprise integration patterns for microservices Originally designed for building integrated solutions out of](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece5b4373171a196779bb8c/html5/thumbnails/40.jpg)
plus.google.com/+RedHat
linkedin.com/company/red-hat
youtube.com/user/RedHatVideos
facebook.com/redhatinc
twitter.com/RedHat
THANK YOU