TurboCharge Your Continuous Delivery Pipeline with ContainersYaniv Donenfeld, Solutions ArchitectAmazon Web Services

What to expect from the session

• Best practices for containers in continuous delivery solutions

• Toolset to implement such solutions• Demos

Why use containers?

• Process isolation• Portable• Fast• Efficient

Why use containers for continuous delivery?

• Roll out features as quickly as possible• Predictable and reproducible environment• Fast feedback

Demo application architecture

Nginx Proxy Ruby on Rails web app

PostgreSQL on RDS

Amazon EC2 Container Service

• Highly scalable container management service• Easily manage clusters for any scale• Flexible container placement• Integrated with other AWS services• Extensible

• Amazon ECS concepts• Cluster and container instances• Task definition and task

Development and deployment workflow

Orchestration layer

Code repository Build environment

Test environment

Deployment environment

Source

Stage 1 - Source

Development environment

Code repository

Source

AWS CodeCommit

• Private Git repository• Fully managed• Secure• Highly available and scalable

• Alternatives• GitHub• Bitbucket

Docker and Docker Toolbox

• Docker (Linux > 3.10) or Docker Toolbox (OS X, Windows)

• Define app environment with Dockerfile

Dockerfile

FROM ruby:2.2.2RUN apt-get update -qq && apt-get install -y build-essential libpq-devRUN mkdir -p /opt/webWORKDIR /tmpADD Gemfile /tmp/ADD Gemfile.lock /tmp/RUN bundle installADD . /opt/webWORKDIR /opt/web

Docker Compose

Define and run multi-container applications:1. Define app environment with Dockerfile2. Define services that make up your app in

docker-compose.yml3. Run docker-compose up to start and run

entire app

docker-compose.yml

proxy: build: ./proxy ports: - "80:80" links: - webweb: build: ./web command: bundle exec rails server -b 0.0.0.0 environment: - SECRET_KEY_BASE=secretkey expose: - "3000"

Amazon ECS CLI

• Easily create Amazon ECS clusters & supporting resources such as EC2 instances

• Run Docker Compose configuration files on Amazon ECS

New

Amazon ECS CLI

> ecs-cli configure> ecs-cli compose build> ecs-cli compose up --local

New

It’s Dem-o-clock!

Stage 2 - Build

Build environment

Build environment

Partners

Jenkins

• Extensible• Flexible builds

• Ant or Maven based projects• Docker images

• Optionally runs in Docker container

CloudBees Docker Build and Publish plugin

Amazon EC2 Container Registry

• Private Docker Repository• v2 Docker Registry• AWS Identity and Access Management (IAM)

and AWS Auth integration• Low latency push, pulls, and inspection

• Alternatives: • DockerHub• Docker Trusted Registry

New

Stage 3 - Test

Test environment

Test environment

rspec and capybara-webkit

require 'rails_helper.rb'

feature 'Signing in' do scenario 'can sign in' do visit '/users/sign_in' within("#new_user") do fill_in 'Email', :with => 'test@test.com' fill_in 'Password', :with => 'password' end click_button 'Log in' expect(page).to have_content('Signed in successfully.') endend

Jenkins

• Run tests directly via Docker run• Run tests in a Docker slave on Amazon ECS

CloudBees Jenkins ECS plugin

Jenkins slave Dockerfile

FROM jenkinsci/jnlp-slaveUSER rootRUN apt-get update -qq && \ apt-get install -y -qq git curl wget build-essential […]RUN apt-get install -y qt5-default libqt5webkit5-devRUN apt-get install -y xvfb x11-xkb-utils xfonts-100dpi xfonts-75dpi xfonts-scalable xfonts-cyrillic x11-appsENV RUBY_VERSION 2.2.2RUN echo 'gem: --no-document' >> /usr/local/etc/gemrc &&\ mkdir /src && cd /src && git clone https://github.com/sstephenson/ruby-build.git &&\ cd /src/ruby-build && ./install.sh &&\ cd / && rm -rf /src/ruby-build && ruby-build $RUBY_VERSION /usr/local

Jenkins slave Dockerfile

RUN gem update --system && gem install bundler

# Install GemsWORKDIR /tmpADD Gemfile /tmp/ADD Gemfile.lock /tmp/RUN bundle install

USER jenkins

It’s Dem-o-clock!

Stage 4 - Deploy

Deployment environment

Deployment environment

Amazon ECS CLI

> ecs-cli up > ecs-cli compose up> ecs-cli ps

AWS Elastic Beanstalk

• Deploy and manage applications without worrying about the infrastructure

• AWS Elastic Beanstalk manages your database, Elastic Load Balancing (ELB), Amazon ECS cluster, monitoring, and logging

• Docker support• Single container (on Amazon EC2)• Multi container (on Amazon ECS)

Putting it all together

Putting it all together

Orchestration layer

AWS CodePipeline

Model and automate your software release processes• Rapid delivery• Configurable workflow• Customizable• Highly integrated

It’s Dem-o-clock!

Takeaways

• Use Amazon ECS CLI to run application• Run Jenkins jobs in containers• Let AWS CodePipeline orchestrate your pipeline

Yaniv DonenfeldTurboCharge Your Continuous Delivery Pipeline with Containers

donenfel@amazon.com