devops on aws: deep dive on continuous delivery and the aws developer tools

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

James Saull, Solutions Architect – Amazon Web Services

Emerson Loureiro, Senior Software Engineer - Gilt Groupe

July 2016

DevOps on AWS:Deep Dive on Continuous Delivery

and the AWS Developer Tools

https://secure.flickr.com/photos/mgifford/4525333972

Why are we

here today?

Software moves faster today

Software creation and distribution is

easier and faster than ever:

• Startups can now take on giants with little to

no funding ahead of time

• Getting your software into the hands of

millions is a download away

• Your ability to move fast is paramount to your

ability to fight off disruption

What tools do you need to move fast?

Releasing software in this new software driven world

requires a number of things:

• Tools to manage the flow of your software development

release process

• Tools to properly test and inspect your code for defects

and potential issues

• Tools to deploy your applications

First, we need to

understand a little bit about

software release processes

https://www.flickr.com/photos/jurvetson/5201796697/

Release processes levels

Source Build Test Production

Continuous integration

Continuous delivery

Continuous deployment

Continuous Delivery Benefits

Improve developer

productivity

Find and address

bugs quickly

Deliver updates fasterAutomate the software

release process

A look back at

development at

Amazon..

https://secure.flickr.com/photos/pixelthing/15806918992/

2001

Development transformation at Amazon: 2001-2009

2009

monolithic

application + teams

microservices + 2 pizza teams

In 2009, we

ran a study to

find out where

inefficiencies

might still exist

We were just waiting.

WaitWrite Code WaitBuild

Code WaitDeploy to Test

Deploy to

Prod

Weeks

Mins Days Mins Days Mins Days Mins

Automated actions and transitions; from check-in to production

Development benefits:

• Faster

• Safer

• Simplification & standardization

• Visualization of the process

Pipelines

This has continued to work out really well:

In 2014:

• Thousands of service teams across Amazon

• Building microservices

• Practicing continuous delivery

• Many environments (staging, beta, production)

50 million deploys

Continuous delivery service for fast and

reliable application updates

Model and visualize your software release

process

Builds, tests, and deploys your code every time

there is a code change

Integrates with 3rd party tools and AWS

AWS CodePipeline

Source

Source

GitHub

Build

JenkinsOnEC2

Jenkins

Deploy

JavaApp

Elastic Beanstalk

PipelineStage

Action

Transition

CodePipeline

MyApplication

Source

Source

GitHub

Build

JenkinsOnEC2

Jenkins

Deploy

JavaApp

Elastic Beanstalk

NotifyDevelopers

Lambda

CodePipeline

MyApplication

Parallel actions

Source

Source

GitHub

Build

JenkinsOnEC2

Jenkins

Deploy

JavaApp

Elastic Beanstalk

NotifyDevelopers

Lambda

TestAPI

Runscope

CodePipeline

MyApplication

Sequential actions

8. Retrieve build artifact

EC2 instance

CodePipeline

Source

Source

GitHub

Build

JenkinsOnEC2

Jenkins

Deploy

JavaApp

Elastic Beanstalk

Source Artifact

S3

Build Artifact

S3

5. Get source artifact

1. Get Changes

6. Store build artifact

3. Poll for Job

4. Acknowledge Job

7. Put Success

9. Deploy build artifact

Elastic Beanstalk

Web container

Java App

MyApplication

We have a strong partner list, and it’s growing

Source Build Test Deploy

AWS service integrations

Source Invoke Logic Deploy

AWS Elastic BeanstalkAmazon S3

AWS CodeDeploy

AWS Lambda

AWS CodeCommit

DEMO

https://www.flickr.com/photos/seattlemunicipalarchives/12504672623/

Deploying your

applications

https://secure.flickr.com/photos/simononly/15386966677

Automates code deployments to any instance

Handles the complexity of updating your

applications

Avoid downtime during application deployment

Deploy to Amazon EC2 or on-premises

servers, in any language and on any operating

system

Integrates with 3rd party tools and AWS

AWS CodeDeploy

appspec.yml Exampleversion: 0.0os: linuxfiles:

- source: /destination: /var/www/html

permissions:- object: /var/www/html

pattern: “*.html”owner: rootgroup: rootmode: 755

hooks:ApplicationStop:

- location: scripts/deregister_from_elb.shBeforeInstall:

- location: scripts/install_dependencies.shApplicationStart:

- location: scripts/start_httpd.shValidateService:

- location: scripts/test_site.sh- location: scripts/register_with_elb.sh

appspec.yml Exampleversion: 0.0os: linuxfiles:

- source: /destination: /var/www/html

permissions:- object: /var/www/html

pattern: “*.html”owner: rootgroup: rootmode: 755

hooks:ApplicationStop:

- location: scripts/deregister_from_elb.shBeforeInstall:

- location: scripts/install_dependencies.shApplicationStart:

- location: scripts/start_httpd.shValidateService:

- location: scripts/test_site.sh- location: scripts/register_with_elb.sh

• Remove/add instance to ELB

• Install dependency packages

• Start Apache

• Confirm successful deploy

• More!

• Send application files to one

directory and configuration

files to another

• Set specific permissions on

specific directories & files

v2 v2 v2 v2 v2 v2

one at a time

half at a time

all at once

v2 v2 v2 v1 v1 v1

v2 v1 v1 v1 v1 v1 Agent Agent

Dev Deployment group

ORProd Deployment group

Agent

AgentAgent

Agent Agent

Agent

Choose Deployment Speed and Group

https://www.flickr.com/photos/spacex/16510243060/

Launching to

Production

Launching to Production

After you’ve built and tested your code and

hopefully gone through a few pre-production

deploys, its time for the real thing!

You’ll want think about:

• Impact to customers

• Impact to infrastructure

• Impact to business

How can we track these and communicate

deploys?

Extend AWS CodePipeline Using Custom Actions

Update tickets Provision resources

Update dashboards Send notifications Security scan

Mobile testing

Source

Source

GitHub

Build

JenkinsOnEC2

Jenkins

Deploy

Action

Custom Action

JavaApp

Elastic Beanstalk

MyApplication

CodePipeline

Deploy

JavaApp

Elastic Beanstalk Job Worker3. Perform Job

1. Poll for Job

2. Acknowledge Job

4. Put Success

Amazon

EC2

With custom actions,

the job worker drives the interaction

between AWS CodePipeline

and other applications or services

Source

Source

GitHub

Build

JenkinsOnEC2

Jenkins

Deploy

MyAction

AWS Lambda

JavaApp

Elastic Beanstalk

2. Perform Job

1. Invoke Lambda function

3. PutJobSuccessResult

MyApplication

CodePipeline With AWS Lambda-based actions,

AWS CodePipeline

drives the integration with Lambda,

which then connects with other

applications or services

AWS

Lambda

Micro-services

Deployment at

Gilt with AWS

Emerson Loureiro,

Senior Software

Engineer, Gilt

Gilt...

● Sales go live everyday at

noon

Micro-services

Move fast

Phased rollout

Rollback just as fast

Deployment tooling is important for us...

Can we do it with AWS?

1st Generation (DiscoDeploy)

● SBT pluginawsAccountName: gilt-xxxxxxx

instanceType: m3.xlarge

servicePort: 9000

instanceCount: 2

jvm:

- -Dnewrelic.config.file=/opt/gilt/service/conf/newrelic.yml

- -Dnewrelic.config.agent_enabled=true

- -Dlogger.resource=logback.production.xml

- -Dconfig.resource=application.production.conf

- -Xms12G

- -Xmx12G

- -XX:NewSize=6G

- -XX:MaxNewSize=6G

...

1st Generation (DiscoDeploy)

Docker

Registry

ASG

EC2 EC2 EC2 EC2

ELBsbt deploy-setup

sbt release-docker

sbt deploy-service

sbt deploy-scale

● Good to begin with

1st Generation (DiscoDeploy)

Docker

Registry

ASG

EC2 EC2 EC2 EC2

ELB

2nd Generation (Ionblaster)

● Based on CloudFormation

[

{

"service": "SERVICE",

"domain": "SERVICE.gilt.com",

"instances": { "min": 3, "max": 3 },

"type": "t2.small"

"run": {

"image": "gilt/SERVICE",

"cmd": ["-Dhttp.port=9000"],

"args": ["-p", "9000:9000"]

},

"health_path": "_internal_/healthcheck"

...

}

]

● Based on CloudFormation

sbt docker:publish

ionblaster new SERVICE 0.0.1

2nd Generation (Ionblaster)

0.0.1

DNS

2nd Generation (Ionblaster)

● Based on CloudFormation

sbt docker:publish

ionblaster new SERVICE 0.0.2

ionblaster traffic SERVICE 0.0.2 10

0.0.1

DNS

0.0.2

2nd Generation (Ionblaster)

● Based on CloudFormation

ionblaster traffic SERVICE 0.0.2 100

0.0.1

DNS

0.0.2

2nd Generation (Ionblaster)

● Based on CloudFormation

ionblaster kill SERVICE 0.0.1DNS

0.0.2

● Traffic migration could be better

● No logical grouping among stacks

2nd Generation (Ionblaster)

DNS

0.0.2

● Based on Elastic Beanstalk

ionroller release SERVICE 0.0.1

3rd Generation (Ionroller)

0.0.1

DNS

Application

● Based on Elastic Beanstalk

ionroller release SERVICE 0.0.2

3rd Generation (Ionroller)

0.0.1

DNS

0.0.2

Application

● Based on Elastic Beanstalk

ionroller release SERVICE 0.0.2

3rd Generation (Ionroller)

0.0.1

DNS

0.0.2

Application

● Based on Elastic Beanstalk

ionroller release SERVICE 0.0.2

3rd Generation (Ionroller)

0.0.1

DNS

0.0.2

Application

● Based on Elastic Beanstalk

ionroller release SERVICE 0.0.2

3rd Generation (Ionroller) DNS

0.0.2

Application

What we didn’t like about those...

“We could solve this now, or, just wait six

months, and Amazon will provide a solution.”

Andrey Kartashov, Distinguished Engineer,

Gilt

● Based on CodeDeploy + CloudFormation + AWS CLI (+ ECR)

make create stack

Development Canary

4th Generation (Mobile Service Tools - MST)

Production

● Based on CodeDeploy + CloudFormation + AWS CLI (+ ECR)

make publish

make deploy-development

make deploy-production-canary

make deploy-production

Development Canary

4th Generation (Mobile Service Tools - MST)

Production

● Based on CodeDeploy + CloudFormation + AWS CLI (+ ECR)

make deploy-production

Development Canary

4th Generation (Mobile Service Tools - MST)

Production

● Based on CodeDeploy + CloudFormation + AWS CLI (+ ECR)

make deploy-production

Development Canary

4th Generation (Mobile Service Tools - MST)

Production

● Mostly AWS tooling

● Quick deployments

Development Canary

4th Generation (Mobile Service Tools - MST)

Production

● Boilerplate scripts

● Docker registry problems

● No cross-account support

● Shared CloudFormation template

Development Canary

4th Generation (Mobile Service Tools - MST)

Production

● Boilerplate scripts

● Docker registry problems

● No cross-account support

● Shared CloudFormation template

4th Generation (Mobile Service Tools - MST)

Base

template

● Based on CodeDeploy + CloudFormation + S3

nova stack create ENVIRONMENT

Development Canary

5th Generation (Nova)

Production

Environment

● Based on CodeDeploy + CloudFormation + S3

nova deploy ENVIRONMENT development

nova deploy ENVIRONMENT canary

nova deploy ENVIRONMENT production

Development Canary

5th Generation (Nova)

Production

Environment

● Versioned CloudFormation templates

5th Generation (Nova)

V2V1

Happiness

FIN, ACK

We’ve seen a quick run through today of the benefits of

continuous delivery on our software release process:

• Continuous integration (build/test) helps shrink our

feedback loop greatly

• We can get our software out in front of our users much

more rapidly

• By moving faster we can actually ensure better quality

• CodePipeline allows for integration with almost any

service or tool you can think of!

• Plus visualization of what’s going on!

Resources to learn more:• DevOps: https://aws.amazon.com/devops/

• Continuous integration: https://aws.amazon.com/devops/continuous-integration/

• Continuous delivery: https://aws.amazon.com/devops/continuous-delivery/

• CodePipeline

• https://aws.amazon.com/codepipeline/

• https://aws.amazon.com/documentation/codepipeline/

• CodeDeploy

• https://aws.amazon.com/codedeploy/

• https://aws.amazon.com/documentation/codedeploy/

• https://github.com/awslabs/aws-codedeploy-samples

• Code Services Starter Kit: http://bit.ly/AWSCodeStarterKit

But wait, there’s more!

Please remember to rate this

session under My Agenda on

awssummit.london

Thank you!