aws re:invent 2016: chalk talk: succeeding at infrastructure-as-code (gpsct312)

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Chalk Talk: GPSCT312

Succeeding at Infrastructure-as-Code

Chris Munns, Andres Michel, Ronald Sunarno, Aaron Richmond, Imran Mahmood

Amazon Web Services

November 2016

How do you define

Infrastructure as Code?

Infrastructure as Code is a practice in

which infrastructure is provisioned and

managed using code and software

development techniques, such as

version control and continuous

integration and delivery.

Why Infrastructure as Code?

• Faster development cycles

• Increasing need for “Immutable” architectures

• Predictable and reproducible environments are a norm

• Managing instances can become cumbersome and

requires convoluted processes in enterprises

• Automation is a must have element in any application

• Microservices are becoming the new standard for

application architectures, driving a number of trends

“It’s all software”

AWS Resources

Operating System and Host Configuration

Application Configuration

AWS ResourcesOperating System and

Host Configuration

Application

Configuration


Host Configuration

Application

Configuration

Infrastructure Resource Management


Host Configuration

Application

Configuration


Host Configuration Management


Host Configuration

Application

Configuration


Host Configuration Management

Application Deployment


Host Configuration

Application

Configuration

AWS CloudFormation

AWS OpsWorks

AWS CodeDeploy


Host Configuration

Application

Configuration

Amazon VPC

Amazon EC2

IAM

Amazon RDS

Amazon S3

AWS CodePipeline

…

Windows Registry

Linux Networking

OpenSSH

LDAP

AD Domain Registration

Centralized logging

System metrics

Deployment agents

Host monitoring

…

Application dependencies

Application configuration

Service registration

Management scripts

Database credentials

…

AWS CloudFormation

AWS OpsWorks

AWS CodeDeploy

Demo – Infrastructure as Code

Demo Architecture

How do you currently

provision infrastructure for

your customers?

Create templates of your infrastructure

CloudFormation provisions AWS resources

based on dependency needs

Version control/replicate/update templates like

code

Integrates with development, CI/CD,

management tools

Launched in 2010

AWS

CloudFormation

Template CloudFormation Stack

JSON formatted file

Parameter definition

Resource creation

Configuration actions

Configured AWS resources

Comprehensive service support

Service event aware

Customizable

Framework

Stack creation

Stack updates

Error detection and rollback

CloudFormation – Components & Technology

Template Anatomy

{ "Description" : "Create an EC2 instance.”,"Parameters" : {

"UserKeyName" : { "Description" : "The EC2 Key Pair to allow SSH access to the instance", "Type" : "String"

} },"Resources" : {

"Ec2Instance" : { "Type" : "AWS::EC2::Instance", "Properties" : {

"KeyName" : { “Ref” : “UserKeyName”}, "ImageId" : "ami-75g0061f”,“InstanceType” : “m1.medium”

} }

}}

How do you bootstrap your

instances?

Option 1: Use EC2 UserData, which is available as a property of AWS::EC2::Instance

resources

Bootstrapping Applications & Handling Updates

"Resources" : {"Ec2Instance" : {"Type" : "AWS::EC2::Instance","Properties" : {

"KeyName" : { "Ref" : "KeyName" },"SecurityGroups" : [ { "Ref" : "InstanceSecurityGroup" } ],"ImageId" : { "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "AMI" ]},"UserData" : { "Fn::Base64" : { "Fn::Join" : ["",[

"#!/bin/bash -ex","\n","yum -y install gcc-c++ make","\n","yum -y install mysql-devel sqlite-devel","\n","yum -y install ruby-rdoc rubygems ruby-mysql ruby-devel","\n","gem install --no-ri --no-rdoc rails","\n","gem install --no-ri --no-rdoc mysql","\n","gem install --no-ri --no-rdoc sqlite3","\n","rails new myapp","\n","cd myapp","\n","rails server -d","\n"]]}}

}}

Option 2: AWS CloudFormation provides helper scripts for deployment within your EC2

instances

Metadata Key — AWS::CloudFormation::Init

Cfn-init reads this metadata key and installs the packages listed in this key (e.g.,

httpd, mysql, and php). Cfn-init also retrieves and expands files listed as sources.

cfn-hup

cfn-signal

cfn-get-

metadata

Amazon EC2AWS CloudFormation

cfn-init

Bootstrapping Applications & Handling Updates

How do you design a

deployment pipeline?

CloudFormation in a DevOps pipeline

AWS CloudFormationIssue Tracker

App Developers

DevOps Engineers,Infrastructure Developers,

Systems Engineers

Dev Env Code Repo

App Pkgs, CloudFormation Templates, Etc.

CI Server

Test

Staging

ProdCode Review

"Infra-as-Code"

App Code & Templates

How about multiple

environments?

Template File

Defining Stack

The entire infrastructure can be represented in an AWS

CloudFormation template.

Many Stacks & Environments from One Template

Template File

Defining Stack



Use the version control system of your choice to store and track changes to this template


Git

Perforce

SVN

…

Template File

Defining Stack

Git

Perforce

SVN

…

Dev

Test

Prod



Use the version control system of your choice to store and track changes to this template

Build out multiple environments, such as for Development, Test, Production and even DR using the same template


How do I deploy my

application?

AWS CodeDeploy

Automates code deployments to any instance

Handles the complexity of updating your applications

Avoid downtime during application deployment

Deploy to Amazon EC2 or on-premises servers, in

any language and on any operating system

Amazon EC2

How CodeDeploy works

3

Amazon S3

1 Deployable Assets and

AppSpec fileAWS CodeDeploy

Deployment Logic and

Metadata2

4

How do I manage the

application once provisioned?

Ongoing Management

• Updates/patches?

• New software?

• New configurations?

• New code deploys?

• Pool specific changes?

• Environment specific changes?

• Run commands across all hosts?

• Be on top of all running resources?

Could we do this with

CloudFormation?

Could we do this with AWS CloudFormation?

Sure! But potentially tricky to do at scale:• Try changing a vhost configuration on every web

server across multiple environments (dev, stage, prod)

• Install a package on certain hosts, but not others to test out newer versions

• Need to change LDAP config on every running Amazon EC2 Linux host, but they are across 25 different AWS CloudFormation templates?

Why do you need OpsWorks?

Model and group your applications

Manage the lifecycle of your instances

Control Access Management

Monitor the health of your resources

Analyze logging information

Mitigate operational problems

Configure your instances using AWS OpsWorks

Uses Chef to configure the software on the instances

Chef provides a Ruby DSL abstraction for common OS operations

Associates predefined scripts (i.e., Chef cookbooks) with your instances

Applies cookbooks configuration changes using lifecycle events

Model your applications using

AWS OpsWorks

Resource organization

A stack represents

the cloud

infrastructure and

applications that

you want to

manage together.

A layer defines

how to set up and

configure a set of

instances and

related resources.

Then deploy your

app to specific

instances and

customize the

deployment with

Chef recipes.

Decide how to

scale: manually,

with 24/7

instances, or

automatically, with

load-based or

time-based

instances.

AWS OpsWorks

AWS OpsWorks Instance Lifecycle

Setup Configure Deploy Undeploy Shutdown

Agent on each instance understands a set

of commands that are triggered by

OpsWorks. The agent then runs Chef.

Thank you!

Remember to complete

your evaluations!