@crichardson
Decompose that WAR! A pattern language for
microservicesChris Richardson
Author of POJOs in Action Founder of the original CloudFoundry.com
@crichardson [email protected] http://plainoldobjects.com http://microservices.io
@crichardson
Presentation goal
Why patterns and pattern languages?
A pattern language for microservices
@crichardson
About Chris
Founder of a startup that’s creating a platform for developing
event-driven microservices
@crichardson
For more information
https://github.com/cer/event-sourcing-examples
http://microservices.io
http://plainoldobjects.com/
https://twitter.com/crichardson
@crichardson
Agenda
Why a pattern language for microservices?
Core patterns
Deployment patterns
Communication patterns
@crichardson
Yet almost 30 years later developers are still
passionately arguing over “silver bullets”
@crichardson
Suck/Rock Dichotomy
Spring vs. Java EE
JavaScript vs. Java
Functional programming vs. Object-oriented
http://nealford.com/memeagora/2009/08/05/suck-rock-dichotomy.html
Containers vs. Virtual Machines
@crichardson
Gartner Hype Cycle
http://upload.wikimedia.org/wikipedia/commons/b/bf/Hype-Cycle-General.png
It’s awesome
It’s not awesome
Trade-offs understood
@crichardson
How we make decisions
Decide using
emotions
Rationalize with our intellect
http://en.wikipedia.org/wiki/Mahout
@crichardson
The structure of a pattern
=
Great framework for discussing and thinking about technology
@crichardson
The structure of a pattern
Resulting context
aka the situation
Name
Context
Problem
Related patterns
(conflicting) issues etc to address Forces
Solution
@crichardson
Related patterns
Alternative solutions
Solutions to problems introduced by this pattern
Pattern languageA collection of related patterns that solve problems in a particular domain
Relationships
Pattern A results in a context that has a problem solved by Pattern B
Patterns A and B solve the same problem
Pattern A is a specialization of pattern B
http://en.wikipedia.org/wiki/A_Pattern_Language
Access to Water
Promenade
Local townhall
Intimacy gradient
Light on two sides
@crichardson
Meta-pattern
Problem: How to talk/reason about technology?
Solution: Use the pattern format
Benefit: More objective
Drawback: Less exciting
Context: Emotional software development culture
Related patterns: It’s awesome!
@crichardson
Monolithic architecture
Microservice architecture
API gateway
Client-side discovery Server-side discovery
Service registry
Self registration 3rd party registration
Multiple Services per host
Single Service per Host
Service-per-Container
Deployment
Discovery
Core
Communication
Service-per-VM
Partitioning
Messaging Remote ProcedureInvocation
Style
MotivatingPattern
SolutionPattern
Solution A Solution B
General Specific
Work in
progress
@crichardson
Agenda
Why a pattern language for microservices?
Core patterns
Deployment patterns
Communication patterns
@crichardson
Let’s imagine you are building an online store
Browser/Client
SQL Database
Review Service
Product Info Service
Recommendation Service
StoreFrontUI
Order Service
HTML
REST/JSON
@crichardson
Forces
There is a team of developers that must be productive
The application must be easy to understand and modify
Do continuous deployment
Run multiple instances for scalability and availability
Use emerging technologies (frameworks, programming languages, etc)
@crichardson
Tomcat
Pattern: Monolithic architecture
Browser/Client
WAR/EAR
MySQL Database
Review Service
Product Info Service
Recommendation Service
StoreFrontUI
Order Service
HTMLREST/JSON
develop test
deploy
Simple to
scale
@crichardson
Obstacle to frequent deployments
Need to redeploy everything to change one component
Interrupts long running background (e.g. Quartz) jobs
Increases risk of failure
Fear of change
Updates will happen less often - really long QA cycles
e.g. Makes A/B testing UI really difficult
Eggs in one basket
@crichardson
Lots of coordination and communication required
Obstacle to scaling development
I want to update the UI
But the backend is not working
yet!
@crichardson
Apply functional decomposition
X axis - horizontal duplication
Z axis
- data
partit
ioning
Y axis - functional
decomposition
Scale b
y split
ting s
imilar
thing
s
Scale by splitting
different things
@crichardson
Product Info
Microservice architectureProduct Info
Service
Recommendation Service
Review Service
Order Service
Browse Products UI
Checkout UI
Order management UI
Account management UI
Apply X-axis and Z-axis scaling to each service independently
@crichardson
Examples
http://highscalability.com/amazon-architecture
http://techblog.netflix.com/
http://www.addsimplicity.com/downloads/eBaySDForum2006-11-29.pdf
http://queue.acm.org/detail.cfm?id=1394128
~600 services
100-150 services to build a page
@crichardson
BenefitsSmaller, simpler apps
Easier to understand and develop
Less jar/classpath hell - who needs OSGI?
Faster to build and deploy
Scales development: develop, deploy and scale each service independently
Improves fault isolation
Eliminates long-term commitment to a single technology stack
System level architecture vs. service level architecture
Easily and safely experiment with new technologies
@crichardson
DrawbacksComplexity of developing a distributed system
Implementing inter-process communication
Handling partial failures
Implementing business transactions that span multiple databases (without 2PC)
Complexity of testing a distributed system
Complexity of deploying and operating a distributed system
Managing the development and deployment of features that span multiple services
Fortunately solutions exists
@crichardson
Issues to address
How to deploy the services?
How do the services communicate?
How do clients of the application communicate with the services?
How to partition the system into services?
….
@crichardson
Agenda
Why a pattern language for microservices?
Core patterns
Deployment patterns
Communication patterns
@crichardson
ForcesServices are written using a variety of languages, frameworks, and framework versions
Each service consists of multiple service instances for throughput and availability
Building and deploying a service must be fast
Service must be deployed and scaled independently
Service instances need to be isolated
Resources consumed by a service must be constrained
Deployment must be cost-effective
@crichardson
Pattern: Multiple service instances per host
Host (Physical or VM)
Service-A Instance-1
Service-B Instance-2
Service-C Instance-2
Process WAR
OSGI bundle
Benefits and drawbacksBenefits
Efficient resource utilization
Fast deployment
Drawbacks
Poor/Terrible isolation
Poor visibility (with WAR/OSGI deployment)
Difficult to limit resource utilization
Risk of dependency version conflicts
Poor encapsulation of implementation technology
@crichardson
Pattern: Service per VM host
Service VM image
VM
Service
VM
Service
VM
Service
packaged as
deployed as
Benefits and drawbacksBenefits
Great isolation
Great manageability
VM encapsulates implementation technology
Leverage AWS infrastructure for Autoscaling/Load balancing
Drawbacks
Less efficient resource utilization
Slow deployment
@crichardson
VM
VM
Pattern: Service per Container host
Service Container image
Container
Service
Container
Service
Container
Service
packaged as
deployed as
Benefits and drawbacksBenefits
Great isolation
Great manageability
Container encapsulates implementation technology
Efficient resource utilization
Fast deployment
Drawbacks
Immature infrastructure for deploying containers
@crichardson
Agenda
Why a pattern language for microservices?
Core patterns
Deployment patterns
Communication patterns
@crichardson
Communication issues
System Client
Service A
Service B
Service C
The SystemHow do clients of the
system interact with the services?
How do services within the system
interact?
@crichardson
Forces
Mismatch between the fine-grained microservices and needs of the clients
Different clients need different data
LAN vs. WAN vs. Mobile network performance
The number of service instances and their locations (host+port) is changes dynamically
Partitioning into services can change over time and should be hidden from clients
@crichardson
Directly connecting the front-end to the backend
Model
View ControllerProduct Info
service
Recommendation Service
Review service
REST
REST
Thrift
Model
View Controller
Browser/Native App
Traditional server-side web application
Chatty API
Web unfriendly protocols
What’s the host/port??
@crichardson
Pattern: API gateway
Model
View ControllerProduct Info
service
Recommendation Service
Review service
REST
REST
Thrift
API Gateway
Model
View Controller
Browser/Native App
Single entry point
Client specific APIs
Protocol translation
Traditional server-side web application
@crichardson
Example: Netflix API
http://techblog.netflix.com/2013/01/optimizing-netflix-api.html
Device specific end points
@crichardson
Benefits of the API gatewayInsulates the clients from the partitioning
Insulates the clients from the problem of discovery
Provides the optimal API for each client
Reduces the number of requests/roundtrips
Simplifies the client by moving logic for calling multiple services from the client to API gateway
Gateway translates between web-unfriendly protocols and HTTP/WebSockets
@crichardson
Drawbacks of the API gateway
Increased complexity - the API gateway is yet another highly available component that must be developed, deployed and managed
Increased response time due to the additional network hop through the API gateway
Benefits and drawbacks of messaging
Benefits
Decouples client from services
Message broker buffers messages
Supports a variety of communication patterns
Drawbacks
Additional complexity of message broker
Request/reply-style communication is more complex
Client needs to discover location of message broker
Benefits and drawbacks of RPC
Benefits
Simple and familiar
Request/reply is easy
No intermediate broker
Drawbacks
Only supports request/reply
Service must be available
Client needs to discover locations of service instances
@crichardson
The problem of discovery
Service Client
Service Instance A
Service Instance B
Service Instance C
?10.4.3.1:8756
10.4.3.99:4545
10.4.3.20:333
Client or API gateway
Dynamically changing
Service Instance A
Service Instance B
Service Instance C
10.4.3.1:8756
10.4.3.99:4545
10.4.3.20:333
How to load balance?
Dynamically assigned
@crichardson
Pattern: Client-side discovery
Service Client
Registry-aware HTTP Client
Service Registry
Service Instance A
Service Instance B
Service Instance C
10.4.3.1:8756
10.4.3.99:4545
10.4.3.20:333
registerquery
request
load balance
@crichardson
Example: Netflix Eureka and Ribbon
https://github.com/Netflix/eureka/wiki/Eureka-at-a-glancehttps://github.com/Netflix/ribbon
Benefits and drawbacksBenefits
Flexible, application-specific load balancing
Fewer network hops and moving parts compared to Server-side discovery
Drawbacks
Couples the client to the Service Registry
Need implement client-side discovery and load balancing logic in multiple languages/frameworks
Service Registry is yet another moving part to setup and operate - highly available
@crichardson
Pattern: Server-side discovery
Service Client
Router
Service Registry
Service Instance A
Service Instance B
Service Instance C
10.4.3.1:8756
10.4.3.99:4545
10.4.3.20:333
registerquery
requestrequest
load balance
@crichardson
Example
http://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/vpc-loadbalancer-types.html
Internal ELB Internal
ELB
Public ELB
Benefits and drawbacksBenefits
Simpler client code
Built-in to some cloud/container environments, e.g. AWS ELB, Kubernetes, Marathon
Drawbacks
Limited to (generic) load balancing algorithms provided by router
More network hops
Router and Service Registry is yet another moving part to setup and operate - highly available
@crichardson
The problem of registration
Service Registry
Service Instance A
Service Instance B
Service Instance C
10.4.3.1:8756
10.4.3.99:4545
10.4.3.20:333
register
How does registration happen?
@crichardson
Forces
Service instances must be
registered with the service registry on startup
unregistered on shutdown
Service instances that crash must be unregistered from the service registry
Service instances that are running but incapable of handling requests must be unregistered from the service registry
@crichardson
Pattern: Self registration
Service Registry
Service Instance A
10.4.3.1:8756
registerheart-beatunregister
@crichardson
ExamplesNetflix Eureka Java client
API for registering/unregistering with Eureka server
Specify state: STARTING, UP, DOWN, OUT_OF_SERVICE
Register a periodically invoked health check callback
Zookeeper-based Java applications
Service = znode
Service instance = ephemeral child of znode
Benefits and drawbacksBenefits
Simple - no extra components
Service can implement a more sophisticated state model
Drawbacks
Couples the service to service registry
Must implement registration logic in multiple languages/frameworks
Service might lack the self-awareness to unregister itself
@crichardson
Pattern: 3rd party registration
Service Registry
Service Instance A
10.4.3.1:8756
registerheart-beatunregister
Registrarhealthcheck
@crichardson
ExamplesAWS Autoscaling groups
Automatically register/unregister EC2 instance with ELB
Registrator
Registers and unregisters Docker containers with service registry
Kubernetes/Marathon
Automatically register/unregister services
Netflix Eureka Prana
Sidecar application for non-Java clients
Registers/unregisters service with Eureka
Performs health checks
Benefits and drawbacksBenefits
Simpler service
Registrar can perform health checks
Some cloud/container environments provide a Registrar
Drawbacks
Registrar is yet another component that must be setup/operated. Must be highly available
@crichardson
Summary: Patterns and pattern languages are a great way to …
Think about technology
Discuss technology
Apply technology
@crichardson
Summary: The Microservices pattern language is a great way to …
Think about microservices
Discuss microservices
Apply microservices (or not)