aws - managing your cloud assets 2013
Post on 08-May-2015
2.008 Views
Preview:
DESCRIPTION
TRANSCRIPT
Journey through the cloud: Managing your cloud assets
Ryan Shuttleworth – Technical Evangelist @ryanAWS
Common use cases & stepping stones into the AWS cloud Learning from customer journeys
Best practices to bootstrap your projects
Journey through the cloud
Getting to grips with system sprawl in an unconstrained environment Leverage management frameworks to take control
Choose systems that suite your technology delivery style
Managing cloud assets
Setting the scene A shrink wrapped approach – Elastic Beanstalk Fine grained control – CloudFormation A middle way - OpsWorks
Agenda
Cloud assets are growing quickly
Defined architectural patterns More applications
More data More services
aws.amazon.com/architecture
Greater agility = faster moving environments
How do we maintain control over what is deployed in the cloud?
Two ends of the spectrum…
Controlled chaos
Meticulous management
Controlled chaos
Meticulous management
Prototype an app Deploy to an instance
Test and develop cycles Go to production
Deploy more instances Deploy RDS
Add load balancers Add autoscaling
Audit what you have?
Quickly becomes DevOps nightmare
Controlled chaos
Meticulous management
Prototype an app Deploy to an instance
Test and develop cycles Go to production
Deploy more instances Deploy RDS
Add load balancers Add autoscaling
Audit what you have?
Quickly becomes DevOps nightmare
Prototype an app Plan infrastructure
Build standards Deploy environment
Manage through change control Formal processes
Formal tools Embed configuration in
chef/puppet
Tightly controlled, but lots of effort to achieve
Controlled chaos
Meticulous management
People Processes Controls
Tools
Prototype an app Deploy to an instance
Test and develop cycles Go to production
Deploy more instances Deploy RDS
Add load balancers Add autoscaling
Audit what you have?
Quickly becomes DevOps nightmare
Prototype an app Plan infrastructure
Build standards Deploy environment
Manage through change control Formal processes
Formal tools Embed configuration in
chef/puppet
Tightly controlled, but lots of effort to achieve
High level, shrink
wrapped
Fine grained, declarative
control
Tools
“I just want my app to run, and I only want
access to machines if I really need to”
“I want to source control my entire datacenter
deployment like I would my apache config”
High level, shrink
wrapped
Fine grained, declarative
control
Elastic Beanstalk
CloudFormation
Tools
“I just want my app to run, and I only want
access to machines if I really need to”
“I want to source control my entire datacenter
deployment like I would my apache config”
High level, shrink
wrapped
Fine grained, declarative
control
Elastic Beanstalk
CloudFormation
Middle ground
AWS OpsWorks
“I just want my app to run, and I only want
access to machines if I really need to”
“I want to source control my entire datacenter
deployment like I would my apache config”
High level, shrink
wrapped
Fine grained, declarative
control
Elastic Beanstalk
CloudFormation
Middle ground
AWS OpsWorks
“I just want my app to run, and I only want
access to machines if I really need to”
“I want to source control my entire datacenter
deployment like I would my apache config”
Quickly deploy and manage apps in AWS…
Elastic Beanstalk
CloudFormation
Tools
…into a range of containers
Elastic Beanstalk
CloudFormation
Tools
.Net PHP
Java Python
Ruby Node.js
…into a range of containers
Elastic Beanstalk
CloudFormation
Tools
http://aws.typepad.com
Keep updated on new containers:
Application package is deployed into Beanstalk
Elastic Beanstalk
CloudFormation
Tools
User Application
Application Service
HTTP Service
Language Interpreter
Operating System
Host
Which creates the container in EC2
Elastic Beanstalk
CloudFormation
Tools
Beanstalk takes care of the environment…
Elastic Beanstalk
CloudFormation
Tools
…adding Elastic Load Balancer…
Elastic Beanstalk
CloudFormation
Tools
…Auto Scaling groups
Elastic Beanstalk
CloudFormation
Tools
…and launching instances
Elastic Beanstalk
CloudFormation
Tools
…all wired into the Beanstalk framework
Elastic Beanstalk
CloudFormation
Tools
…and published under a CNAME
Elastic Beanstalk
CloudFormation
Tools
…with logs and app versions held in S3
Elastic Beanstalk
CloudFormation
Tools
User Application
Application Service
Apache
Python
Operating System
Host
mod_wsgi
…Python mod-wsgi & apache
Elastic Beanstalk
User Application
Application Service
Nginx
Ruby
Operating System
Host
Passenger & Nginx…
Elastic Beanstalk
passenger
What about the data?
Rails
Django
Git Beanstalk
rake
syncdb
RDS
How do you create an app for AWS ElasticBeanstalk?
Like any other app
No dependencies on AWS inherent for Beanstalk deployment
Elastic Beanstalk
CloudFormation
Tools
Source uploaded and executed in AWS
Source downloaded
and executed locally
Package up as normal…
Elastic Beanstalk
CloudFormation
Tools
Java .war file
Microsoft Web
Deploy package
PHP .zip file
Python .zip file
Git integration IDE plugins
Package up as normal…
Elastic Beanstalk
CloudFormation
Tools
Java .war file
Microsoft Web
Deploy package
PHP .zip file
Python .zip file
Git integration IDE plugins
Console deployments and version updates
Elastic Beanstalk
CloudFormation
Tools
Command line tools and wizards
Elastic Beanstalk
CloudFormation
Tools
CheckDNSAvailability elastic-beanstalk-check-dns-availability
CreateApplication elastic-beanstalk-create-application
CreateApplicationVersion
elastic-beanstalk-create-application-version
CreateEnvironment
elastic-beanstalk-create-environment
eb init wizard to initialize an application
eb start/stop start/stop an application
eb update
update application version
eb status
get status of a running application
Eclipse integration
Elastic Beanstalk
CloudFormation
Tools
Eclipse integration
Elastic Beanstalk
CloudFormation
Tools
Eclipse integration
Elastic Beanstalk
CloudFormation
Tools
Eclipse integration
Elastic Beanstalk
CloudFormation
Tools
Visual Studio integration
Elastic Beanstalk
CloudFormation
Tools
Visual Studio integration
Elastic Beanstalk
CloudFormation
Tools
Visual Studio integration
Elastic Beanstalk
CloudFormation
Tools
Visual Studio integration
Elastic Beanstalk
CloudFormation
Tools
Visual Studio integration
Elastic Beanstalk
CloudFormation
Tools
Visual Studio integration
Elastic Beanstalk
CloudFormation
Tools
Visual Studio integration
Elastic Beanstalk
CloudFormation
Tools
Git integration
Elastic Beanstalk
CloudFormation
Tools
Container configuration
Elastic Beanstalk
CloudFormation
Tools
Container configuration
Elastic Beanstalk
CloudFormation
Tools
Container configuration
Elastic Beanstalk
CloudFormation
Tools
Container configuration
Elastic Beanstalk
CloudFormation
Tools
Container configuration
Elastic Beanstalk
CloudFormation
Tools
Full admin access on underlying resources EC2 instances, AMIs, AutoScaling groups…
High level, shrink
wrapped
Fine grained, declarative
control
Elastic Beanstalk
CloudFormation
Middle ground
AWS OpsWorks
“I just want my app to run, and I only want
access to machines if I really need to”
“I want to source control my entire datacenter
deployment like I would my apache config”
What happens under the covers of Beanstalk?
Elastic Beanstalk
CloudFormation
Tools
Elastic Beanstalk
AWS
Create security group
Create auto-scaling policy
Create elastic load balancer
Add instance
Install software
Consolidate logs
What happens under the covers of Beanstalk?
Elastic Beanstalk
CloudFormation
Tools
AWS
Create security group
Create auto-scaling policy
Create elastic load balancer
Add instance
Install software
Consolidate logs
A series of calls into AWS to marshal resource
creation/configuration
CloudFormation – underlying automation framework
Elastic Beanstalk
CloudFormation
Tools
Security Scaling
Database
Networking Monitoring
Messaging
Workflow
DNS
Load Balancing
Backup CDN
Compute
Storage
Elastic Beanstalk
CloudFormation
Tools
CloudFormation
A consistent scripting interface for creating/configuring AWS
resources
A framework for lifecycle management of resources
created using scripts
Security Scaling
Database
Networking Monitoring
Messaging
Workflow
DNS
Load Balancing
Backup CDN
Compute
Storage
CloudFormation – underlying automation framework
CloudFormation components & terminology
Elastic Beanstalk
CloudFormation
Tools
Template CloudFormation
Stack
JSON formatted file
Parameter definition
Resource creation
Configuration actions
Configured AWS services
Comprehensive service support
Service event aware
Customisable
Framework
Stack creation
Stack updates
Error detection and rollback
Template examples
Elastic Beanstalk
CloudFormation
Tools
Template
JSON formatted file
Parameter definition
Resource creation
Configuration actions
{
"Parameters" : {
"KeyName" : {
"Description" : "Name of an existing EC2 KeyPair to enable SSH
access to the instance",
"Type" : "String"
}
},
}
Snippet: Taking user defined parameters
Template examples
Elastic Beanstalk
CloudFormation
Tools
Template
JSON formatted file
Parameter definition
Resource creation
Configuration actions
{
"AWSTemplateFormatVersion" : "2010-09-09",
"Description" : "Create RDS with fixed username and password",
"Resources" : {
"MyDB" : {
"Type" : "AWS::RDS::DBInstance",
"Properties" : {
"AllocatedStorage" : "5",
"DBInstanceClass" : "db.m1.small",
"Engine" : "MySQL",
"EngineVersion" : "5.5",
"MasterUsername" : "MyName",
"MasterUserPassword" : "MyPassword"
}
}
}
}
Snippet: Creating RDS instance
Template examples
Elastic Beanstalk
CloudFormation
Tools
Template
JSON formatted file
Parameter definition
Resource creation
Configuration actions
"AWS::CloudFormation::Init" : {
"config" : {
"packages" : {
"yum" : {
"mysql" : [],
"mysql-server" : [],
"httpd" : [],
"php" : [],
"php-mysql" : []
}
},
"sources" : {
"/var/www/html" : "https://s3.amazonaws.com/
cloudformation-examples/
CloudFormationPHPSample.zip"
},
Snippet: Installing software
The benefit of templated environments
Elastic Beanstalk
CloudFormation
Tools
Template
Procedural definition Create it programmatically
The benefit of templated environments
Elastic Beanstalk
CloudFormation
Tools
Template
Procedural definition Create it programmatically
Parameter driven Dynamic and user-driven
templates
The benefit of templated environments
Elastic Beanstalk
CloudFormation
Tools
Template
Procedural definition Create it programmatically
Parameter driven Dynamic and user-driven
templates
Collaboration Share templates with ease as just files
The benefit of templated environments
Elastic Beanstalk
CloudFormation
Tools
Template
Procedural definition Create it programmatically
Known configuration Store stack configuration in source control
Parameter driven Dynamic and user-driven
templates
Collaboration Share templates with ease as just files
NASA TV & Curiosity Route 53 DNS Multi-region & AZ
Elastic Loadbalancing Amazon Live Streaming
Cloud Front Cloud Formation
NASA TV & Curiosity Route 53 DNS Multi-region & AZ
Elastic Loadbalancing Amazon Live Streaming
Cloud Front Cloud Formation
Template ELBs to front secondary cache
~100 Nginx secondary cache servers
2-3 Nginx mid-tier cache servers
Stack
CloudFormation template defined video caching
infrastructure
NASA TV & Curiosity Route 53 DNS Multi-region & AZ
Elastic Loadbalancing Amazon Live Streaming
Cloud Front Cloud Formation
Availability Zone
As demand increased new stacks added to an availability
zone using template
One-click complex
rollout
Where to get more information?
Elastic Beanstalk
CloudFormation
Tools
http://aws.amazon.com/cloudformation/aws-cloudformation-templates/
Server templates High availability configurations
RDS configurations Open source stacks
Snippets
+ Cloud Former (prototype)
Where to get more information?
Elastic Beanstalk
CloudFormation
Tools
http://aws.amazon.com/cloudformation/aws-cloudformation-templates/
Server templates High availability configurations
RDS configurations Open source stacks
Snippets
+ Cloud Former (prototype)
Comprehensive service support
Elastic Beanstalk
CloudFormation
Tools
Amazon EC2 Instances (On-Demand Instances, Spot Instances, and Reserved Instances) Amazon Elastic Block Store (EBS) Volumes Elastic Load Balancers Elastic IP Addresses Amazon EC2 Security Groups Auto Scaling Groups Amazon Relational Database Service (RDS) Instances Amazon RDS Security Groups Amazon ElastiCache Clusters Amazon ElastiCache Security Groups Amazon ElastiCache Parameter Groups
Amazon CloudWatch Alarms Amazon Simple Storage Service (S3) Buckets
Amazon Simple DB Domains Amazon Simple Queue Service (SQS) Queues
Amazon Simple Notification Service (SNS) Topics Amazon SNS Subscriptions
Amazon Route 53 DNS Records Amazon CloudFront Distributions and Multi-Origin support
AWS Identity and Access Management users and groups AWS Identify and Access Management policies
Amazon Virtual Private Cloud VPCs, Subnets, Gateways Amazon DynamoDB Tables
Comprehensive service support
Elastic Beanstalk
CloudFormation
Tools
Amazon EC2 Instances (On-Demand Instances, Spot Instances, and Reserved Instances) Amazon Elastic Block Store (EBS) Volumes Elastic Load Balancers Elastic IP Addresses Amazon EC2 Security Groups Auto Scaling Groups Amazon Relational Database Service (RDS) Instances Amazon RDS Security Groups Amazon ElastiCache Clusters Amazon ElastiCache Security Groups Amazon ElastiCache Parameter Groups
Amazon CloudWatch Alarms Amazon Simple Storage Service (S3) Buckets
Amazon Simple DB Domains Amazon Simple Queue Service (SQS) Queues
Amazon Simple Notification Service (SNS) Topics Amazon SNS Subscriptions
Amazon Route 53 DNS Records Amazon CloudFront Distributions and Multi-Origin support
AWS Identity and Access Management users and groups AWS Identify and Access Management policies
Amazon Virtual Private Cloud VPCs, Subnets, Gateways Amazon DynamoDB Tables
+ AWS Elastic Beanstalk
High level, shrink
wrapped
Fine grained, declarative
control
Elastic Beanstalk
CloudFormation
Tools
“I just want my app to run, and I only want
access to machines if I really need to”
“I want to source control my entire datacenter
deployment like I would my apache config”
High level, shrink
wrapped
Fine grained, declarative
control
Elastic Beanstalk
CloudFormation
+
+
High level, shrink
wrapped
Fine grained, declarative
control
Elastic Beanstalk
CloudFormation
Developer sandboxes with production ready
infrastructure
+
+
Comprehensive control of all aspects of infrastructure
High level, shrink
wrapped
Fine grained, declarative
control
Elastic Beanstalk
CloudFormation
Middle ground
AWS OpsWorks
“I just want my app to run, and I only want
access to machines if I really need to”
“I want to source control my entire datacenter
deployment like I would my apache config”
Powerful management framework with Chef support
Elastic Beanstalk
CloudFormation
Stack Layers Management
Managed
environment
Definition of environment such as production or test
Management
services
Scaling, cloning, user access, self healing
Collection of
resources
Blueprint for a collection of resources (instances, EBS, EIPs
etc)
OpsWorks
Apps
Your application
assets
Resources to deploy and run in layers
Simple stack creation through layers
Elastic Beanstalk
CloudFormation
OpsWorks
Layers
Load balancing
HA Proxy installation in an availability zone
Application container
Static, Node.js, Rails, PHP
Database layer
MySQL
Simple stack creation through layers
Elastic Beanstalk
CloudFormation
OpsWorks
Layers Ganglia
MemCached
Custom layers
+
Scaling and auto-healing
Elastic Beanstalk
CloudFormation
OpsWorks
Auto healing
Replacement of an ec2 instance regardless of
scaling thresholds
i.e. always maintain 3 nodes across 3 AZs
Scaling and auto-healing
Elastic Beanstalk
CloudFormation
OpsWorks
Auto healing
Replacement of an ec2 instance regardless of
scaling thresholds
i.e. always maintain 3 nodes across 3 AZs
Time based scaling
Run an ec2 instance on a given schedule
i.e. run these instances only during weekend peaks
Load based scaling
Add and remove instances from a layer based upon
load rules
i.e. run these instances only when CPU goes over 80%
Scaling and auto-healing
Elastic Beanstalk
CloudFormation
OpsWorks
Scaling and auto-healing
Elastic Beanstalk
CloudFormation
OpsWorks Lo
ad b
ased
Ti
me
bas
ed
App deployments
Elastic Beanstalk
CloudFormation
OpsWorks
App deployments
Elastic Beanstalk
CloudFormation
OpsWorks
App deployments
Elastic Beanstalk
CloudFormation
OpsWorks
Customization with chef recipes
Elastic Beanstalk
CloudFormation
OpsWorks
Get started fast with Beanstalk (control the chaos)
Take fine grained control with CloudFormation
(auditable environment specs)
Manage with layers and recipes with OpsWorks
(control with chef customization)
http://aws.amazon.com/elasticbeanstalk/
http://aws.amazon.com/cloudformation/
Find out more…
http://aws.amazon.com/opsworks/
aws.amazon.com
get started with the free tier
top related