microservices on aws: divide & conquer for agility and scalability

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Level 200 (Introductory)

Jim Tran, AWS Enterprise Solutions ArchitectJuly 28, 2016

Architecting Microservices on AWSDivide and conquer for agility and scalability:

ENTERPRISE APPS

DEVELOPMENT & OPERATIONSMOBILE SERVICESAPP SERVICESANALYTICS

DataWarehousing

Hadoop/Spark

Streaming Data Collection

Machine Learning

Elastic Search

Virtual Desktops

Sharing & Collaboration

Corporate Email

Backup

Queuing & Notifications

Workflow

Search

Email

Transcoding

One-click App Deployment

Identity

Sync

Single Integrated Console

PushNotifications

DevOps Resource Management

Application Lifecycle Management

Containers

Triggers

Resource Templates

TECHNICAL & BUSINESS SUPPORT

Account Management

Support

Professional Services

Training & Certification

Security & Pricing Reports

Partner Ecosystem

Solutions Architects

MARKETPLACE

Business Apps

Business Intelligence

DatabasesDevOps Tools

NetworkingSecurity Storage

RegionsAvailability Zones

Points of Presence

INFRASTRUCTURE

CORE SERVICES

ComputeVMs, Auto-scaling, & Load Balancing

StorageObject, Blocks, Archival, Import/Export

DatabasesRelational, NoSQL, Caching, Migration

NetworkingVPC, DX, DNS

CDN

Access Control

Identity Management

Key Management & Storage

Monitoring & Logs

Assessment and reporting

Resource & Usage Auditing

SECURITY & COMPLIANCE

Configuration Compliance

Web application firewall

HYBRIDARCHITECTURE

Data Backups

Integrated App Deployments

DirectConnect

IdentityFederation

IntegratedResource Management

Integrated Networking

API Gateway

IoT

Rules Engine

Device Shadows

Device SDKs

Registry

Device Gateway

Streaming Data Analysis

Business Intelligence

MobileAnalytics

* As of 1 Feb 2016

2009

48

280

722

82

2011 2013 2015

“The Monolith”

Challenges with monolithic software

Long Build/Test/Release Cycles(who broke the build?)

Operationsis a nightmare(module X is failing, who’s the owner?)

Difficult to scale

New releasestake months

Long time to addnew features

Architecture is hard to maintain and evolve

Lack of innovation

Frustrated customers

Lack of agility

“20080219BonMorningDSC_0022B” by Sunphol Sorakul . No alterations other than cropping. https://www.flickr.com/photos/83424882@N00/3483881705/Image used with permissions under Creative Commons license 2.0, Attribution Generic License (https://creativecommons.org/licenses/by/2.0/)

Monolith development lifecycle

releasetestbuild

delivery pipeline

app(aka the“monolith”)developers

Photo by Sage Ross. No alterations other than cropping. https://www.flickr.com/photos/ragesoss/2931770125/Image used with permissions under Creative Commons license 2.0, Attribution Generic License (https://creativecommons.org/licenses/by/2.0/)

Too much software coupling


Shared libraries


Shared libraries

Shared data

Evolving towards microservices

“IMG_1760” by Robert Couse-Baker. No alterations other than cropping. https://www.flickr.com/photos/29233640@N07/14859431605/Image used with permissions under Creative Commons license 2.0, Attribution Generic License (https://creativecommons.org/licenses/by/2.0/)

“IMG_1760” by Robert Couse-Baker. No alterations other than cropping. https://www.flickr.com/photos/29233640@N07/14859431605/Image used with permissions under Creative Commons license 2.0, Attribution Generic License (https://creativecommons.org/licenses/by/2.0/)

“service-oriented

architecture

composed of

loosely coupled

elements

that have

bounded contexts”

Adrian Cockcroft (former Cloud Architect at Netflix,now Technology Fellow at Battery Ventures)

“service-oriented

architecture

composed of

loosely coupled

elements

that have

bounded contexts”


Services communicate with each other over the network

“service-oriented

architecture

composed of

loosely coupled

elements

that have

bounded contexts”


You can update the services independently; updating one service doesn’t require changing any other services.

“service-oriented

architecture

composed of

loosely coupled

elements

that have

bounded contexts”


Self-contained; you can update the code without knowing anything about the internals of other microservices

“Do one thing, and do it well”

“Swiss Army” by by Jim Pennucci. No alterations other than cropping. https://www.flickr.com/photos/pennuja/5363518281/Image used with permissions under Creative Commons license 2.0, Attribution Generic License (https://creativecommons.org/licenses/by/2.0/)

“Tools” by Tony Walmsley: No alterations other than cropping. https://www.flickr.com/photos/twalmsley/6825340663/Image used with permissions under Creative Commons license 2.0, Attribution Generic License (https://creativecommons.org/licenses/by/2.0/)

“Do one thing, and do it well”

Anatomy of a Micro-service

Data Store(eg, RDS, DynamoDB

ElastiCache, ElasticSearch)


Application/Logic(code, libraries, etc)




Public API

POST /restaurantsGET /restaurants

Application/Logic(code, libraries, etc)




Avoid Software Coupling

Driversmicro-services

Paymentsmicro-service Location

micro-services

Orderingmicro-services

Restaurantmicro-service

Ecosystem of micro-services

= 50 million deployments a year

Thousands of teams

× Microservice architecture

× Continuous delivery

× Multiple environments

(5708 per hour, or every 0.63 second)

Gilt: Luxury designer brands at members-only prices

... Sale every day at noon EST

Principle 1

Micro-services only rely on each other’s public API

“Contracts” by NobMouse. No alterations other than cropping.https://www.flickr.com/photos/nobmouse/4052848608/

Image used with permissions under Creative Commons license 2.0, Attribution Generic License (https://creativecommons.org/licenses/by/2.0/)

Micro-service A Micro-service B

public API public API

Principle 1: Micro-services only rely on each other’s public API


Principle 1: Micro-services only rely on each other’s public API(Hide Your Data)



Nope!



Principle 1: Micro-services only rely on each other’s public API(Evolve API in backward-compatible way…and document!)

storeRestaurant (id, name, cuisine)

Version 1.0.0

public API

Micro-service A



storeRestaurant (id, name, cuisine)storeRestaurant (id, name, arbitrary_metadata)addReview (restaurantId, rating, comments)

Version 1.0.0

Version 1.1.0

public API

Micro-service A



storeRestaurant (id, name, cuisine)storeRestaurant (id, name, arbitrary_metadata)addReview (restaurantId, rating, comments)

storeRestaurant (id, name, arbitrary_metadata)addReview (restaurantId, rating, comments)

Version 1.0.0

Version 1.1.0

Version 2.0.0

public API

Micro-service A

Principle 2

Use the right tool for the job

“Tools #2” by Juan Pablo Olmo. No alterations other than cropping.https://www.flickr.com/photos/juanpol/1562101472/

Image used with permissions under Creative Commons license 2.0, Attribution Generic License (https://creativecommons.org/licenses/by/2.0/)


Principle 2: Use the right tool for the job(Embrace polyglot persistence)

DynamoDB




DynamoDB


AmazonElasticsearchService



RDSAurora




Principle 2: Use the right tool for the job(Embrace polyglot programming frameworks)

RDSAurora



Let’s builda micro-service!

Restaurant Micro-service

GET /restaurants

POST /restaurants


Approach #1

EC2


EC2


EC2EC2 EC2 EC2


EC2EC2 EC2 EC2

Elastic Load Balancer


Approach #2

ContainersUsing ECS


EC2EC2 EC2 EC2



EC2EC2 EC2 EC2


AmazonEC2 ContainerService (ECS)

to managecontainers

• Prototype in less than 2 months

• Deployment time: hours minutes

• Each team can now develop itsrespective applications independently

Coursera13 million users from 190 countries1,000 courses from 119 institutions


Approach #3

API Gateway+ Lambda

DynamoDB


DynamoDB

Lambda


DynamoDB

Lambda


API Gateway

AWS Lambda

lets you run code

without managing servers

Upload your code(Java, JavaScript,

Python)


Python)

Set up your code to trigger from other AWS

services, webservicecalls, or app activity

Lambda automatically

scales


Python)



Lambda automatically

scales


Python)

Pay for only the compute time

you use(sub-second metering)



AWS API Gateway

is the easiest way to deploy micro-services

Create a unified API frontend for

multiplemicro-services



Authenticate and authorize requests




Handles DDoSprotection and API throttling



…as well as monitoring,

logging, rollbacks,

client SDK generation…


Handles DDoSprotection and API throttling

DynamoDB


DynamoDB

Lambdato retrieverestaurants


Lambdato store

restaurants

DynamoDB



API Gateway

POST GET

Lambdato store

restaurants

DynamoDB



API Gateway

POST GET

Web UI

Lambdato store

restaurants

Highly Scalable• Inherently scalable

Secure• API Gateway acts as “front door”• Can add authN/authZ; or throttle API if needed• S3 bucket policies• IAM Roles for Lambda invocations

Cost-efficient• Only pay for actual micro-service usage



From a few…

Driversmicro-services

Paymentsmicro-service Location

micro-services

Orderingmicro-services

Restaurantmicro-service

…to an ecosystem

Principle 3

Secure Your Services

“security” by Dave Bleasdale. No alterations other than cropping.https://www.flickr.com/photos/sidelong/3878741556/

Image used with permissions under Creative Commons license 2.0,Attribution Generic License (https://creativecommons.org/licenses/by/2.0/)

Principle 3: Secure Your Services

• Defense-in-depth• Network level (e.g. VPC, Security Groups, TLS)• Server/container-level• App-level• IAM policies

• Gateway (“Front door”)

• API Throttling

• Authentication & Authorization• Client-to-service, as well as service-to-service• API Gateway: custom Lambda authorizers• IAM-based Authentication• Token-based auth (JWT tokens, OAuth 2.0)

• Secrets management• S3 bucket policies + KMS + IAM• Open-source tools (e.g. Vault, Keywhiz)

API Gateway

Principle 4

Be a good citizenwithin the ecosystem

“Lamington National Park, rainforest” by Jussarian. No alterations other than cropping.https://www.flickr.com/photos/kerr_at_large/87771074/


Hey Sally, we need to call your micro-service to fetch

restaurants details.

Sure Paul. Which APIs you need to call? Once I know

better your use cases I’ll give you permission to register

your service as a client on our service’s directory entry.



Principle 4: Be a good citizen within the ecosystem

Principle 4: Be a good citizen within the ecosystem(Have clear SLAs)

RestaurantMicro-service

15 TPS100 TPS5 TPS20 TPS

Before we let you call our micro-service we

need to understand your use case, expected load

(TPS) and accepted latency

…and many,many others!

Distributed monitoring and tracing• “Is the service meeting its SLA?”• “Which services were involved in a request?”• “How did downstream dependencies perform?”

Shared metrics• e.g. request time, time to first byte

Distributed tracing• e.g. Zipkin, OpenTracing

User-experience metrics

Principle 4: Be a good citizen within the ecosystem(Distributed monitoring, logging and tracing)

Principle 5

More than justtechnology transformation

“rowing on the river in Bedford” by Matthew Hunt. No alterations other than cropping.https://www.flickr.com/photos/mattphotos/19189529/


“Any organization that designs a system will inevitably produce a design whose structure is a copy of the organization’scommunication structure.”

Melvin E. Conway, 1967

Conway’s Law

Decentralize governance and data management

Image from Martin Fowler’s article on microservices, athttp://martinfowler.com/articles/microservices.html

No alterations other than cropping.Permission to reproduce: http://martinfowler.com/faq.html

Silo’d functional teams silo’d application architectures



Cross functional teams self-contained services



Non-pizza image from Martin Fowler’s article on microservices, athttp://martinfowler.com/articles/microservices.html


Cross functional teams self-contained services(“Two-pizza teams” at Amazon)

Full ownership

Full accountability

Aligned incentives

“DevOps”

Non-pizza image from Martin Fowler’s article on microservices, athttp://martinfowler.com/articles/microservices.html


Cross functional teams self-contained services(“Two-pizza teams” at Amazon)

Principle 6

Automate Everything

“Robot” by Robin Zebrowski. No alterations other than cropping.https://www.flickr.com/photos/firepile/438134733/


releasetestbuild

Focused agile teams

2-pizza team delivery pipeline service

releasetestbuild

releasetestbuild

Focused agile teams


releasetestbuild

releasetestbuild

releasetestbuild

Focused agile teams


releasetestbuild

releasetestbuild

releasetestbuild

releasetestbuild

Focused agile teams


releasetestbuild

releasetestbuild

releasetestbuild

releasetestbuild

releasetestbuild

Focused agile teams


releasetestbuild

releasetestbuild

releasetestbuild

releasetestbuild

releasetestbuild

releasetestbuild

Focused agile teams


Principle 6: Automate everything

AWS

CodeCommit

AWS

CodePipeline

AWS

CodeDeploy

EC2 ELBAuto

ScalingLambdaECS

DynamoDBRDS ElastiCache SQS SWF

SES SNS

API GatewayCloudWatch Cloud Trail

KinesisElasticBeanstalk

It’s a journey…

Expect challenges along the way…

• Understanding of business domains

• Coordinating txns across multiple services

• Eventual Consistency

• Service discovery

• Lots of moving parts requires increased coordination

• Complexity of testing / deploying / operating a distributed system

• Cultural transformation

Principles of Microservices

1. Rely only on the public API Hide your data Document your APIs Define a versioning strategy

2. Use the right tool for the job Polygot persistence (data layer) Polyglot frameworks (app layer)

3. Secure your services Defense-in-depth Authentication/authorization

6. Automate everything Adopt DevOps

4. Be a good citizen within the ecosystem Have SLAs Distributed monitoring, logging, tracing

5. More than just technology transformation Embrace organizational change Favor small focused dev teams

Benefits of microservices

Rapid Build/Test/Release Cycles

Clear ownership andaccountability

Easier to scaleeach individual micro-service

New releasestake minutes

Short time to addnew features

Easier to maintain and evolve

Increase innovation

Delighted customers

Increased agility

Benefits of microservices

Rapid Build/Test/Release Cycles

Clear ownership andaccountability

Easier to scaleeach individual micro-service

New releasestake minutes

Short time to addnew features

Easier to maintain and evolve system

Faster innovation

Delighted customers

Increased agility

Additional AWS resources:• Zombie Microservices Workshop:

https://github.com/awslabs/aws-lambda-zombie-workshop• Serverless Webapp - Reference Architecture:

https://github.com/awslabs/lambda-refarch-webapp• Microservices with ECS:

https://aws.amazon.com/blogs/compute/using-amazon-api-gateway-with-microservices-deployed-on-amazon-ecs/

• Serverless Service Discovery:https://aws.amazon.com/blogs/developer/serverless-service-discovery-part-1-get-started/

• ECS Service Discovery:https://aws.amazon.com/blogs/compute/service-discovery-an-amazon-ecs-reference-architecture/

• Microservices without the Servershttps://aws.amazon.com/blogs/compute/microservices-without-the-servers

Popular open-source tools:• Serverless – http://serverless.com• Apex - http://apex.run/

https://aws.amazon.com/devops/

Additional resources

https://github.com/awslabs/aws-lambda-zombie-workshop

https://github.com/awslabs/lambda-refarch-webapp

https://aws.amazon.com/blogs/compute/using-amazon-api-gateway-with-microservices-deployed-on-amazon-ecs/

https://aws.amazon.com/blogs/developer/serverless-service-discovery-part-1-get-started/

https://aws.amazon.com/blogs/compute/service-discovery-an-amazon-ecs-reference-architecture/

https://aws.amazon.com/blogs/compute/microservices-without-the-servers/

http://serverless.com/

https://aws.amazon.com/devops/