microservices application simplicity infrastructure complexity

MICROSERVICES

TRADING INFRASTRUCTURE COMPLEXITY

FOR APPLICATION SIMPLICITY

INTRODUCTION

ABOUT ME

▸ Senior Architect w/Centric Consulting

▸ National Service Offering Lead for Cloud Computing

▸ @williamklos

▸ [email protected]

mailto:[email protected]

INTRODUCTION

AGENDA

▸ A Bit of History

▸ Definition of Microservices

▸ Application Simplicity

▸ Application Complexity

▸ Infrastructure Complexity

▸ Summary

▸ Resources

HISTORY

“ANY ORGANIZATION THAT DESIGNS A SYSTEM (DEFINED BROADLY) - WILL PRODUCE A DESIGN WHOSE STRUCTURE IS A COPY OF THE ORGANIZATION’S COMMUNICATION STRUCTURE.”

Conway’s Law (1967)

HISTORY

HISTORY

MY STORY

▸ First Set of Applications in 1988

▸ Each PC was a “service”

▸ Running DOS, dBASEII, and support scripts - modem

▸ 5.25” floppy-based Sneaker-net

▸ Next Major Set of Applications in 1994

▸ 1 AS/400, 1 database, tons of applications

▸ REST Web Services since 200x

HISTORY

CLASSIC MONOLITH

▸ One cabinet, maybe more if you’re rich.

▸ One database, shared by all applications.

▸ One set of shared libraries.

▸ No need for networking between application modules because they are all right there in the same address space.

▸ May have some logical partitions for other environments.

▸ Probably a monoglot situation.

finance

inventory

report

capacity

ORDER-WOPR

19x9shop

pickship

return

cartbill

queryDB

HISTORY

MODERN MONOLITH

▸ Web services based. Maybe.

▸ Tightly coupled data, tightly coupled network resources, tightly coupled business logic, tightly coupled code.

▸ The monolith is in the shape of a virtual machine.

▸ To put it in the cloud would require a “lift-and-shift”.

▸ But, we are starting to see external resources be a big part of the overall flow.

ship

pick

shop

financereturn

inventory

query

report

notify

bill

review

cart

capacity

ORDER-WOPR

2008DB

DB

DB

HISTORY

WHAT’S CHANGED?

▸ Technology Constraints Removed

▸ Single machines running single core processors,

▸ Slow Networks and Lack of Pervasive Connectivity

▸ Expensive Storage & Expensive RAM

▸ Organizations structured as monoliths

▸ New Technologies

▸ Virtualized Machines, most notably Containers

▸ Wide Variety of Messaging and other Data Movement Options

▸ and of course, the Cloud.

HISTORY

THE PROGRESSION TOWARDS THE SMALL

MONOLITH CLIENT SERVER

WEB & INTERNET

CLOUD & WEB SERVICES

MICRO SERVICES 2020 - 2030

DEFINITION

KETCHUP BOTTLES, JAM TUBS, & SWEETENER PACKETS

DEFINITION

OPEN FIEFDOM

▸ Self-contained, focused purpose society - with a healthy importing and exporting capability.

▸ The ability to open franchises rivals Starbucks and the ability to close them rivals Sears.

▸ Automation is such that the feudal lord can run everything from his throne without the need of pesky serfs.

▸ If the world goes sideways, they can pull up the drawbridge and watch everything outside burn.

WHAT MAKES IT “MICRO”?

IT’S NOT A FUNCTION OF LINES OF CODE

IT’S A FUNCTION OF OPERATIONAL SCOPE

DEFINITION

ONE MICROSERVICE (MINILITH)

shop shop

shop

1 MICROSERVICE 1 MICROSERVICE1 CONTAINER

1 MICROSERVICE1 CONTAINER

1 VM

DEFINITION

RECALL THE LEGACY ORDER MANAGEMENT SYSTEMS

CLASSIC MONOLITH MODERN MONOLITH

finance

inventory

report

capacity

ORDER-WOPR

19x9shop

pickship

return

cartbill

queryDB

ship

pick

shop

financereturn

inventory

query

report

notify

bill

review

cart

capacityORDER-WOPR

2008DB

DB

DB

DEFINITION

A MICROSERVICE-BASED ORDER MANAGEMENT SYSTEM

rest

ship

pick

query

billcart

return

pubsubpubsub

pubsub

rest

rest

rest

reportpubsubpubsub

pubsub

finance

pubsub

shop

rest

review notify

pubsub pubsubpubsub

pubsub

inventory

rest

rest

rest

rest

rest

APPLICATION SIMPLICITY

CHARACTERISTICS

CHARACTERISTICS

SINGLE RESPONSIBILITY

▸ Goal is for it to do one thing, and to do it well. It is responsible for its own behavior and its input contracts.

▸ Has inputs and outputs, and if a data source or state is necessary, it has its own.

▸ Not aware of the outside environment aside from some shared resources in an API Layer or whatever its input data tells it.

▸ All other tenets of microservices spin-off from this core principle.

pick&ship

shippick

CHARACTERISTICS

UNIX’S “SINGLE RESPONSIBILITY PRINCIPLE”

docker ps | grep my_container | wc -l

IN THIS CASE, IT YIELDS A SIMPLE INTEGER

output becomes input to next step

CHARACTERISTICS

ISOLATION

▸ Goal is for it to be a completely self-contained entity with minimal outside dependencies.

▸ This helps minimize impact on other parts of the system.

▸ Has inputs and outputs, and if a data source is necessary, it has its own.

▸ Keeps track of its own state and will only reveal it of asked.

▸ Independently deployable unit.

rest

ship

pick

query

billcart

return

pubsubpubsub

pubsub

rest

rest

rest

reportpubsubpubsub

pubsub

finance

pubsub

shop

rest

review notify

pubsub pubsubpubsub

pubsub

inventory

rest

rest

rest

rest

CHARACTERISTICS

ISOLATED ANATOMY - SIMPLIFIED

▸ At its base level, it only knows what its specific business logic tells it - plus…

▸ …whatever the inputs bring to it.

microservice

CHARACTERISTICS

ISOLATED ANATOMY - COMPLEX (ILL-ADVISED)

▸ The micro service can include all of the extras to be a good application citizen.

▸ Handles tasks that tell it how to operate, scale, communicate, control access, & report its behavior.

▸ Should probably be handled via an API or mediation layer (discussed later).

bill

microservice

security

loggingback pressure

scaling

config versioning

monitoringdiscovery

cachingtransform

CHARACTERISTICS

ISOLATION EXAMPLE

▸ Upgrading the Billing service forces these things to occur:

▸ New Orders and Returns are collected in holding queues

▸ Existing Orders are processed until they reach 0 and the Billing service is stopped.

▸ When new Billing service launches the Order and Returns queues are depleted then processing returns to normal.

▸ No downstream services are affected.

▸ “Safety of Change”

rest

ship

pick

query

billcart

return

pubsubpubsub

pubsub

rest

rest

rest

reportpubsubpubsub

pubsub

finance

pubsub

shop

rest

review notify

pubsub pubsubpubsub

pubsub

inventory

rest

rest

rest

rest

CHARACTERISTICS

COMPARTMENTALIZATION

▸ A structure to help limit the spread of “badness” across a system.

▸ Because microservices should be designed to handle their own issues and have insurance against malcontents, cascading behavior should be minimized.

CHARACTERISTICS

AUTONOMY

▸ Isolation is a pre-requisite for autonomy. It’s hard to automate something that has tendrils of dependency.

▸ When a service can guarantee its own behavior and is not dependent on outside interactions - then it has all it needs to solve its own problems.

▸ Autonomous services allow for more flexibility in dealing with service orchestration, choreography, management, scaling, etc.

shippick reportnotify ship

ship ship

ship ship

ship ship

CHARACTERISTICS

CAN BE STATELESS OR STATEFUL

▸ Stateless allows for auto-scaling without worry since no copy cares about any other.

▸ Strive for immutability

▸ Separate stateful services into their own containers.

▸ A microservice is responsible for its own state and the persistence of that state. There should be no external changes to it.

▸ Don’t share state keeping mechanisms with other microservices.

?

CHARACTERISTICS

ASYNCHRONOUS MESSAGE PASSING

▸ Synchronous operations inside the web service, async outside of it.

▸ Aids in de-coupling, isolation, compartmentalization, resilience, etc.

▸ Helps change mindset from a rigid procedural perspective to a more event-driven one.

▸ Fire and forget. Sometimes.

sync

async

CHARACTERISTICS

ASYNCHRONOUS MESSAGE PASSING EXAMPLE

cart

shop

review

inventorygetProductSummary()

x10@100mseach

getReviewStars()x10@150mseach

getProductList()x10

getCartCount()x1@100mseach

Sync=1000msAsync=100ms



TOTALSync=2600msAsync=350ms

CHARACTERISTICS

INTERNAL INTEGRATION - THE INTERSTITIAL

▸ Essentially the spaces in between the microservices.

▸ Part of the negotiated contract between left-side and right-side.

▸ Can be async or sync mechanisms.

▸ Don’t all have to be the same. What makes sense for each interaction?

pick shippubsub

amqp

rest

interstitial

CHARACTERISTICS

EXTERNAL INTEGRATION

▸ Predicting how external systems are going to misbehave is futile. Design for self-defense.

▸ Construct asynchronous calls where possible.

▸ Negotiate expected traffic rates and volume, be able to handle unforeseen events.

▸ Not really that different over internal service integration.

myms

yourms

ExternalAPI

FORMATCSV?JSON?XML?FORM?

METHODREST?SOAP?PUBSUB?QUEUE?

RESPONSES200?201?404?401?50x?

REACTIONSPROCESSQUEUERE-SUBMITTHROTTLESLEEP

CHARACTERISTICS

MOBILITY

▸ Where possible, a micro service should be able to operate anywhere in the enterprise’s operation.

▸ While a microservice should carry what it needs to execute, certain environmental dependencies are more pragmatically shared.

▸ Should be able to run on AWS, Azure, Joe’s Hosting Shack, or in the corporate data center.

▸ The service should also always be addressable across the locational space.

shop

query

query

CHARACTERISTICS

SHARED CAPABILITY > SHARED DATA OR SHARED INSTANCES

▸ While there is less of a DRY (Don’t Repeat Yourself) push in microservices, it’s not a fully WET (Write Everything Twice) architecture either.

▸ Sharing runtime capabilities like databases and security is perfectly acceptable given those capabilities are sufficiently segmented so as to minimize impact of other applications.

▸ i.e. Don’t negatively impact “change safety”.

CHARACTERISTICS

TIME AWARENESS OF DATA

▸ Data in-flight is either in “the past” (has happened) or in “the hopeful future” (will happen)

▸ There is no single notion of “the present”. Only the “local” now.

▸ Inside our microservice we can control everything within our boundary.

▸ Outside the boundary, not our problem.

thelocalnow

thepast

thefuture

API MANAGEMENT

bill

microservice

security


scaling

config versioning

monitoringdiscovery

cachingtransform

API MANAGEMENT

OVERVIEW

▸ Similar to an orchestration layer like SOA designs but still need to abstract direct visibility to endpoints.

▸ Allows you to change granularity level without impacting users.

▸ Includes common functionality around routing, security, load-balancing and registration services.

ship

pick

query

billcart returnreport finance

shop review notifyinventory

API LAYER

Web App Mobile App

PartnerApps

ClientApps

EmployeeApps

SECURITYROUTINGDISCOVERYLOGGINGLOAD BALANCEREGISTRATIONAUTHENTICATETRANSFORM

API MANAGEMENT

SERVICE DISCOVERY

▸ Upon startup of a micro-service, it registers itself with a controlling entity.

▸ Access to service can be direct if all operating copies are known or by redirection.

▸ Assures all nodes get traffic when autoscaling.

▸ Essentially a load balancer.

billing

billing 172.16.6.22:3000 2016/07/31 13:31:04

billing 172.16.6.22:3001 2016/07/31 13:45:04

API Layer

50ms

40ms

MODULE IP ADDR:PORT TIME STARTED PING

registerService()

POST https://apilayer/order/:ord_id/bill

POST https://172.16.6.22:3001/order/:ord_id/bill

GET https://apilayer/billings

{“hosts”:[“172.16.6.22:3000”,”172.16.6.22:3001”],”count”:2}

Get List of Active Billing Services

Auto Redirect Service

API MANAGEMENT

SECURITY

▸ Primary security would typically be handled by the API Layer.

▸ Individual services have enough security to know when they are being called directly and redirect to API Layer or reject the request with a 401.

▸ Individual services will also need to provide authorization for services that call it (certs, whitelisted Services, etc.).

shipbillcart

return

shop

bill

microservice


config

transform

APPLICATION COMPLEXITY

DETERMINE GRANULARITY


DETERMINING BOUNDARIES

▸ Define a model based upon a specific business perspective.

▸ But it doesn’t need, and maybe it shouldn’t, be based on your standard organizational makeup.

▸ Distributed > Centralized

▸ Different perspectives of the same topic can yield different models

▸ e.g. Uber driver vs. Uber passenger

▸ Domain-Driven Design may help here.

Back-Office Order ContextCustomer Order Context

rest

ship

pick

query

billcart

return

pubsubpubsub

pubsub

rest

rest

rest reportpubsubpubsub

pubsub

finance

pubsub

shop

rest review

notify

pubsub pubsubpubsub

pubsub

inventory

rest

rest

rest

rest

rest

ORDER PROCESSING DOMAIN


DETERMINE GRANULARITY

▸ Need to do one thing very well, but how do you define “one thing”?

▸ Signs of being too small include services being “chatty”, too much dependence on other services, or poor-performance.

▸ Signs of being too big include code changes impacting pseudo-related functionality, compromises in data design, or too much “global cruft”.

shop

shopbill

cart

orderentry

bill

remove

add2cart

viewcart

savecart

emptycart


DETERMINE GRANULARITYshop

findProduct()listProducts()viewProduct()

cart addToCart()removeFromCart()viewCart()emptyCart()saveCart()

bill gatherBillInfo()gatherShipInfo()billOrder()

notify pick report

query ship finance

return review

notifyBilled()notifyRejected()notifyShipped()notifyReturned()

inventory

cancelOrder()makePickList()pickOrder()notifyBO()

logTransaction()

updateStatus()getStatus()

shipOrder()attachTracking()

allocateInv()pickInv()queryInv()replaceInv()relocateInv()returnInv()

generateRA()receiveRA()

createReview()showReviews()blockReviewer()thumbsUp()thumbsDown()

updateLedger()

DATA COUPLING


DATA COUPLING

▸ Data consistency requires coordination.

▸ Coordination can inhibit scalability, throughput, and availability.

▸ There is no global single source of the truth. Truth is local, within the context of the service.

bill

cart

shop

review

inventory

DB

billcart

shop review

inventory

LOW

HIGH

pick

billcartpubsub

rest

shop

rest

inventory

rest

rest

rest

rest


DATA COUPLING - EVENTUAL CONSISTENCY EXAMPLE

1000 0 1000

MODULEINVENTORY (ON HAND)

INVENTORY (ALLOCATED)

INVENTORY (CURRENT)

shop

1000 5 995

995 0 995pick

bill

1000

SHOP INVENTORY (CURRENT)

995

1000

1000 0 1000pick

995

EVENTUAL CONSISTENCY != BATCH

HANDLING FAILURES

CHALLENGES

HANDLING FAILURES - BACKPRESSURE

▸ When the next phase can’t be completed right now, defer it yourself or give it back to the caller.

▸ A way of governing discrepancy between slow and fast systems.

▸ Process it when the issue is cleared.

POST http://apilayer/order/bill billingcartrest

200

publishBillOrder() > 0

backPressureBillQ billPubSubChannel

NO

YES

50x

Relieve Back Pressure When Possible

Apply Back Pressure When Necessary

OK

billQDepth() > max

billQFillRate() > max

CHALLENGES

HANDLING FAILURES - AVAILABILITY

▸ When a service is down, it can be handled in a similar way to back-pressure.

▸ Can also look for additional services of the same type (API Layer should handle).

▸ If severe enough, then stoppages can be cascaded up in a controlled fashion to make sure overall system integrity is maintained.

POST http://apilayer/order/billcart

rest

200

SERVICE IS DOWN

overflowBillQ billPubSubChannel

NO

YES

50x

Pull from Queue When Service Resumes

OK

billing

CHALLENGES

HANDLING FAILURES - CIRCUIT BREAKER

▸ As long as the circuit stays closed, all traffic passes as normal.

▸ If the service is experiencing a backlog or under stress, it can partially “trip” the circuit effectively throttling input.

▸ If the circuit is open, then the service is down and all calling functions need to step back behavior until service resumes.

cartNO

billing

ADD RETRY DELAY

backPressureBillQ

TRANSACTIONS


DISTRIBUTED TRANSACTIONS

▸ Traditionally upholds the illusion that you are the only one using the data you need.

▸ Distributed transactions can cause scalability issues and introduce “brittleness”.

▸ A recommended alternative to distributed transaction is the Saga pattern.

▸ Essentially having an “undo” action along the way executed in reverse in case of failure.

cartshop inventorybill

addToCart() billOrder()removeFromCart()

allocateInv()

compensate()

pick

compensate()

deductInv()commit()

commit()commit() commit()


MORE DISTRIBUTED TRANSACTIONS

▸ Additional ways of handling would include:

▸ Previously discussed methods of applying “back-pressure”, deferred processing, or retries.

▸ Back off of any dogmatic approach to distributed effort in favor of a centralized approach for that particular piece.

▸ e.g. Inventory in our Order Management model.

INFRASTRUCTURE COMPLEXITY

INDIVIDUAL MICROSERVICES ARE RELATIVELY EASY TO DESIGN AND DEPLOY.

THE COMPLEXITY COMES WHEN YOU HAVE MORE THAN A HANDFUL THAT NEED TO BE MANAGED.


YOU NEED HOW MANY SERVERS?

▸ Our example has 12 microservices consisting of 36 actual applications services.

▸ If we need a robust production environment, we’ll want 3x servers (36).

▸ If we want additional environments, then we will need more even still.

▸ Or…we could use containers.

rest

ship

pick

query

billcart

return

pubsubpubsub

pubsub

rest

rest

rest

reportpubsubpubsub

pubsub

finance

pubsub

shop

rest

review notify

pubsub pubsubpubsub

pubsub

inventory

rest

rest

rest

rest


HOW ABOUT CONTAINERS?

▸ The base unit of measure for a micro service is a container.

▸ Containers are like micro-VMs.

▸ Lightweight virtual machine that has all of the dependencies necessary to execute the contained application.

▸ And no more.

MICROSERVICE

pick

shop shop

cart

VM

CONTAINER

CAPACITY


AUTOMATION IS YOUR FRIEND

pick

shop shop

cart cart

review

ship

query

notify

shop ship shop

bill

bill

return

report▸ Service discovery allows containers to be brought into the world and removed from it in an automated fashion.

▸ Products like Kubernetes and Docker Swarm aid in this ability and make the management of you containers fleet much easier to deal with.


DATABASE PROLIFERATION

▸ Self-contained means a private datasource.

▸ Allows for the “right-tool for the right job”.

▸ Adds management and monitoring complexity.

▸ Potentially adds licensing and support costs.


TECHNOLOGY PROLIFERATION

▸ It’s not just databases that are proliferating. Languages, protocols, connectors - all are up for consideration.

▸ Allows for the “right-tool for the right job”.

▸ Builds silos of knowledge, but they’re small.

▸ Of course, you can “monoglot” the crap out of your architecture if you want.


VARIOUS TECH STACKS FOR EACH MICROSERVICEshop

DB: RDBSTACK: .NETVER: 2.50

cart DB: RDBSTACK: JAVAVER: 3.22

bill DB: DocDBSTACK: GOVER: 1.5

notify pick report

query ship finance

return review

DB: RAMSTACK: GOVER: 1.73

inventory

DB: DocDBSTACK: GOVER: 1.01

DB: KeyValueSTACK: ELIXIRVER: 1.7

DB: DocDBSTACK: RUBYVER: 1.21

DB: DocDBSTACK: GOVER: 4.2

DB: RDBSTACK: JAVAVER: 6.3

DB: DocDBSTACK: RUBYVER: 2.00

DB: RDBSTACK: .NETVER: 0.9B

DB: RDBSTACK: JAVAVER: 10.5

THE DEV/OPS PLAY

CHALLENGES

THE DEV/OPS PLAY

▸ Dev/Ops has a huge impact on a successful micro services implementation.

▸ Monitoring, scaling, load-balancing, routing, instantiation, and overall robustness require better levels of automation.

bill

microservice

security


scaling

config versioning

monitoringdiscovery

cachingtransform

SUMMARY

SUMMARY

OPERATING PRINCIPLE EXAMPLES

▸ UNIX

▸ Do One Thing Well

▸ Build Afresh

▸ Expect Output to Become Input

▸ Reduce Human Input

▸ Try/Fail Early

▸ Don’t Hesitate to Throw it Away

▸ Become Toolmakers

▸ Netflix

▸ Antifragility

▸ Immutability

▸ Separation of Concerns

SUMMARY

THINGS TO REMEMBER

▸ Very small scope leads to much easier “micro-applications” that provide small pieces of value. The trade-off is overall environmental management.

▸ A solid DevOps practice is vital in managing the seemingly chaotic environment of a micro services implementation.

▸ You decide the appropriate granularity.

▸ Yes, it’s like SOA, but different. Lighter weight and more nimble.

▸ Once you can automate the management of your microservice fleet, having 5 or 50 or 500 microservices, the amount of pain in administering them should be logarithmic rather than linear.

SUMMARY

MORE THINGS TO REMEMBER

▸ The ironic thing about systems that deal in 1s and 0s is that the rules that govern how to implement them is never 1s and 0s.

▸ There is no one true “right way” to do any of this. Everything is a tradeoff between the needs of the business and the realities of your situation.

▸ That being said, strive towards:

▸ services doing “one thing only”

▸ minimizing coupling (data and functionality)

▸ automation

▸ The function, behavior, and citizenship of your microservice is on you.

SUMMARY

THINGS TO TRY

▸ “Antifragility” testing (Netflix does this) will introduce failures into the system to determine amount of drama created. Try removing nodes or introducing bad keys into your process to see how the system reacts. Seriously!

▸ Treat your servers and processes like “cattle” rather than “pets”. Everything is disposable outside of data.

▸ Assign services to teams. Instead of having people assigned to support entire applications, assign smaller teams to individual services where they own the entire stack including design and support.

“ANY ORGANIZATION THAT DESIGNS A SYSTEM (DEFINED BROADLY) - WILL PRODUCE A DESIGN WHOSE STRUCTURE IS A COPY OF THE ORGANIZATION’S COMMUNICATION STRUCTURE.”

Conway’s Law (1967)

HISTORY

RESOURCES

RESOURCES

MORE DETAILED INFORMATION

▸ http://www.reactivemanifesto.org/

▸ http://thenewstack.io/ten-commandments-microservices/

▸ http://goo.gl/4qLLbG (Data Outside vs. Data Inside)

▸ http://goo.gl/zMnHtU (Cattle vs. Pets)

▸ https://msdn.microsoft.com/en-us/library/jj591569.aspx

▸ http://martinfowler.com/bliki/CircuitBreaker.html

▸ Microservice Architecture, Nadareishvili, Mitra, et al.

▸ Microservices vs. Service-Oriented Architecture, Mark Richards

▸ Reactive Microservices Architecture, Jonas Bonér

http://www.reactivemanifesto.org/

http://thenewstack.io/ten-commandments-microservices/

http://goo.gl/4qLLbG

http://goo.gl/zMnHtU

https://msdn.microsoft.com/en-us/library/jj591569.aspx

http://martinfowler.com/bliki/CircuitBreaker.html