aws 101 december 2014
TRANSCRIPT
Thanks to our sponsors
Before we start…
…we’d love your feedback!
background
Consumer Business
Tens of millions of active
customer accounts
13 countries: US, UK, Germany, Japan, France, Canada, China,
Italy, Brazil, Mexico, India, Spain, Australia
Seller Business
Sell on Amazon websites
Use Amazon technology for your own retail website
Leverage Amazon’s
massive fulfillment centre network
IT Infrastructure Business
Cloud computing
infrastructure for hosting web-scale solutions
Over 1 million registered customers in over 190
countries
How did Amazon…
… get into Cloud Computing?
AWS Mission
Enable businesses and developers to use web
services* to build scalable, sophisticated applications.
*What people now call “the cloud”
Not excess capacity!
Not excess capacity!
Startups on AWS
Find out more at : aws.amazon.com/solutions/case-studies
Powering the Most Popular Internet Businesses
Find out more at : aws.amazon.com/solutions/case-studies
Enterprises on AWS
Find out more at : aws.amazon.com/solutions/case-studies
Each day AWS adds the equivalent server capacity to power Amazon when it was a
global, $7B enterprise
utility computing
On demand Pay as you go
Uniform Available
Utility computing
Utility computing
On demand Pay as you go
Available Uniform
Utility computing
Utility computing
Compute
Storage
Security Scaling
Database
Networking Monitoring
Messaging
Workflow
DNS Load Balancing
Backup CDN
On demand Pay as you go
Uniform Available
On a global footprint
Region
US-WEST (N. California) EU-WEST (Ireland)
ASIA PAC (Tokyo)
ASIA PAC (Singapore)
US-WEST (Oregon)
SOUTH AMERICA (Sao Paulo)
US-EAST (Virginia)
GOV CLOUD
ASIA PAC (Sydney)
CHINA (Beijing) EU-CENTRAL (Frankfurt)
At the end of a web service aws ec2 run-instances --image-id ami-a813fadf --count 3 --placement AvailabilityZone=eu-west-1a --instance-type m3.medium
aws ec2 run-instances --image-id ami-a813fadf --count 5 --placement AvailabilityZone=eu-west-1c --instance-type m3.large
and a rich Management Console
elasticity
Traditional IT capacity
Elastic capacity
Capacity
Time Your IT needs
On and Off Fast Growth
Variable peaks Predictable peaks
Elastic capacity
Elastic capacity
On and Off Fast Growth
Predictable peaks Variable peaks
WASTE
CUSTOMER DISSATISFACTION
Elastic capacity
Fast Growth On and Off
Predictable peaks Variable peaks
From one instance…
…to thousands
and back…
exploiting elasticity
Sunday Monday Tuesday Wednesday Thursday Friday Saturday
Typical weekly traffic to Amazon.com
November traffic to Amazon.com
November
November traffic to Amazon.com Provisioned capacity
November
November traffic to Amazon.com 76%
24%
Provisioned capacity
November
November 10th 2010 Turned off last physical web server of
Amazon.com
November 10th 2010 Turned off last physical web server of
Amazon.com
October 31st 2011 Turned off last web servers supporting
European business
November traffic to Amazon.com
November
Building a Top500 HPC Cluster on AWS
the toolbox
Compute Storage
AWS Global Infrastructure
Database
App Services
Deployment & Administration
Networking
Reference Model security
Compute Storage
AWS Global Infrastructure
Database
App Services
Deployment & Administration
Networking
Global infrastructure
Regions An independent collection of AWS resources in a defined geography A solid foundation for meeting location-dependent privacy and compliance requirements
Global infrastructure
Availability Zones Designed as independent failure zones Physically separated within a typical metropolitan region
Compute Storage
AWS Global Infrastructure
Database
App Services
Deployment & Administration
Networking
Global infrastructure
Edge Locations To deliver content to end users with lower latency A global network of edge locations Supports global DNS infrastructure (Route53) and Cloud Front CDN
Dallas(2)
St.Louis
Miami
Jacksonville Los Angeles (2)
Palo Alto
Seattle
Ashburn(3)
Newark New York (3)
Dublin
London(2)
Amsterdam (2)
Stockholm
Frankfurt(2) Paris(2)
Singapore(2)
Hong Kong (2)
Tokyo (2)
Sao Paulo
South Bend
San Jose
Osaka Milan
Sydney
Madrid
Seoul
Mumbai
Chennai
Compute Storage
AWS Global Infrastructure
Database
App Services
Deployment & Administration
Networking
Compute Storage
AWS Global Infrastructure
Database
App Services
Deployment & Administration
Networking
Networking
Direct Connect Dedicated connection to AWS
VPN Connection Secure internet connection to AWS
Virtual Private Cloud Private, isolated section of the AWS Cloud
Route 53 Highly available and scalable Domain Name Service
Compute Storage
AWS Global Infrastructure
Database
App Services
Deployment & Administration
Networking
Compute
Vertical Scaling From $0.02/hr
Elastic Compute Cloud (EC2) Basic unit of compute capacity Range of CPU, memory & local disk options Many Instance types available, from micro to cluster compute C4 cluster compute with custom Intel CPUs coming soon
Feature Details
Flexible Run windows or linux distributions
Scalable Wide range of instance types from micro to cluster compute
Machine Images Configurations can be saved as machine images (AMIs) from which new instances can be created
Full control Full root or administrator rights
Secure Full firewall control via Security Groups
Monitoring Publishes metrics to Cloud Watch
Inexpensive On-demand, Reserved and Spot instance types
VM Import/Export Import and export VM images to transfer configurations in and out of EC2
Compute
Auto-scaling Automatic provisioning of compute resources based upon demand, configuration or schedule
Trigger auto-scaling policy
Feature Details
Control Define minimum and maximum instance pool sizes and when scaling and cool down occurs
Integrated to CloudWatch Use metrics gathered by CloudWatch to drive scaling
Instance types Run auto scaling for on-demand instances and spot. Compatible with VPC
aws autoscaling create-auto-scaling-group --auto-scaling-group-name MyGroup --launch-configuration-name MyConfig --availability-zones eu-west-1a --min-size 4 --max-size 200
Compute Storage
AWS Global Infrastructure
Database
App Services
Deployment & Administration
Networking
Compute
Elastic Load Balancing Create highly scalable applications Distribute load across EC2 instances in multiple availability zones
Feature Details
Auto-scaling Automatically scales to handle request volume
Available Load balance across instances in multiple availability zones
Health checks Automatically checks health of instances and takes them in or out of service
Session stickiness Route requests to the same instance
Secure sockets layer Supports SSL offload from web and application servers with flexible cipher support
Monitoring Publishes metrics to Cloud Watch
Compute Storage
AWS Global Infrastructure
Database
App Services
Deployment & Administration
Networking
Compute Storage
AWS Global Infrastructure
Database
App Services
Deployment & Administration
Networking
Storage
S3 - Durable storage, any object 99.999999999% durability of objects Unlimited storage of objects of any type Up to 5TB size per object
Feature Details
Flexible object store Buckets act like drives, folder structures within
Access control Granular control over object permissions
Server-side encryption 256bit AES encryption of objects
Multi-part uploads Improved throughput & control
Object versioning Archive old objects and version new ones
Object expiry Automatically remove old objects
Access logging Full audit log of bucket/object actions
Web content hosting Serve content as web site with built in page handling
Notifications Receive notifications on key events
Import/Export Physical device import/export service
Compute Storage
AWS Global Infrastructure
Database
App Services
Deployment & Administration
Networking
Storage
Elastic Block Store High performance block storage device 1GB to 1TB in size (16TB volumes coming soon) Mount as drives to instances
Feature Details High performance file
system Mount EBS as drives and format as required
Flexible size Volumes from 1GB to 1TB in size
Flexible performance Magnetic, EBS General Purpose (SSD) and EBS Provisioned IOPS (SSD) volume types
Secure Private to your instances
Available Replicated within an Availability Zone
Backups Volumes can be snapshotted for point in time restore
Monitoring Detailed metrics captured via Cloud Watch
Compute Storage
AWS Global Infrastructure
Database
App Services
Deployment & Administration
Networking
Database Relational Database Service Database-as-a-Service No need to install or manage database instances Scalable and fault tolerant configurations
Feature Details
Platform support Create MySQL, PostgreSQL, Microsoft SQL Server and Oracle RDBMS
Preconfigured Get started instantly with sensible default settings
Automated patching Keep your database platform up to date automatically
Backups Automatic backups and point in time recovery and full DB backups
Backups Volumes can be snapshotted for point in time restore
Failover Automated failover to slave hosts in event of a failure
Replication Easily create read-replicas of your data and seamlessly replicate data across availability zones
Compute Storage
AWS Global Infrastructure
Database
App Services
Deployment & Administration
Networking
Database
Amazon Relational Database Service (Amazon RDS) databases stores forum threads, site content, and project configuration data. High availability Multi-AZ database deployment to handle live game metadata and user-generated content. Enterprise-grade fault tolerance for protecting customer data. By managing time-consuming database administration tasks, Amazon RDS allows SEGA to focus on business critical applications.
Compute Storage
AWS Global Infrastructure
Database
App Services
Deployment & Administration
Networking
Database
DynamoDB Provisioned throughput NoSQL database Fast, predictable performance Fully distributed, fault tolerant architecture
Feature Details
Provisioned throughput Dial up or down provisioned read/write capacity
Predictable performance Average single digit millisecond latencies from SSD backed infrastructure
Strong consistency Be sure you are reading the most up to date values
Fault tolerant Data replicated across availability zones
Monitoring Integrated to Cloud Watch
Secure Integrates with AWS Identity and Access Management (IAM)
Elastic MapReduce Integrates with Elastic MapReduce for complex analytics on large datasets
Compute Storage
AWS Global Infrastructure
Database
App Services
Deployment & Administration
Networking
Database
Redshift Managed Massively Parallel Petabyte Scale Data Warehouse Streaming Backup/Restore to S3 Extensive Security 2 TB -> 1.6 PB
RDS Dynamo DB
Redshift
Compute Storage
AWS Global Infrastructure
Database
App Services
Deployment & Administration
Networking
Application Services
CloudFront World-wide content distribution network Easily distribute content to end users with low latency, high data transfer speeds, and no commitments.
Feature Details
Fast Multiple world-wide edge locations to serve content as close to your users as possible
Integrated with other services Works seamlessly with S3 and EC2 origin servers
Dynamic content Supports static and dynamic content from origin servers
Streaming Supports rtmp from S3 and includes support for live streaming from Adobe FMS and Microsoft Media Server
London
Paris
NY
Served from S3 /images/*
3
Served from EC2 *.php
2
Single CNAME www.mysite.com
1
Application Services
Amazon SQS
Processing task/processing trigger
Processing results Amazon SQS Reliable, highly scalable, queue service for storing messages as they travel between instances
Feature Details
Reliable Messages stored redundantly across multiple availability zones
Simple Simple APIs to send and receive messages
Scalable Unlimited number of messages
Secure Authentication of queues to ensure controlled access
Compute Storage
AWS Global Infrastructure
Database
App Services
Deployment & Administration
Networking
Compute Storage
AWS Global Infrastructure
Database
App Services
Deployment & Administration
Networking
Deployment & Admin
Elastic Beanstalk One-click deployment from Eclipse, Visual Studio and Git Rapid deployment of applications All AWS resources automatically created
Feature Details
Platform support Containers for Java, .net and PHP
Resource creation Creates load balancer, instances, autoscaling and monitoring automatically
Monitoring & Logs Integrated with Cloud Watch and consolidates server logs
Versioning Manage versions of applications and easily rollback deployments
Notifications Receive alerts on key events
Full resource access Access all underlying AWS resources as necessary
Deployment & Admin
OpsWorks DevOps focused managed application stacks Underlying Chef recipes allow for complete customisation
Feature Details
Platform support Chef recipes allows for community expansion for platform components such as Solr, NgniX etc
Resource creation Customizable deployments, rollback, partial deployments, patch management, automatic instance scaling, and auto healing
Layered Manage logical application layers and combine into stacks.
Compute Storage
AWS Global Infrastructure
Database
App Services
Deployment & Administration
Networking
Cloud Formation Automate creation of ‘stacks’ in a repeatable way Scripting framework for AWS resource creation
Feature Details
Platform support Support for AWS resources from EC2 to IAM
Resource creation Creates AWS resources behind the scenes and reports on progress
Declarative Specify stacks in JSON format and source control your environments
Customizable Drive stack creation with parameters
Deployment & Admin
Compute Storage
AWS Global Infrastructure
Database
App Services
Deployment & Administration
Networking
Deployment & Admin
Identity & Access Management Granular control of user rights with AWS Automated granting of EC2 service rights
Software Developer Kits Comprehensive support of programming models for using AWS services
Compute Storage
AWS Global Infrastructure
Database
App Services
Deployment & Administration
Networking
+ others WorkSpaces Cloud Search
Simple Email Service Simple Workflow Service
Simple Notification Service ElastiCache (Memcache & Redis)
Elastic MapReduce CloudWatch
…and more to come!
security & compliance
Foundation Services
Compute Storage Database Networking
AWS Global Infrastructure Regions
Availability Zones Edge Locations A
maz
on
Shared responsibility
Foundation Services
Compute Storage Database Networking
Client-side Data Encryption & Data Integrity Authentication
Server-side Encryption (File System and/or Data)
Network Traffic Protection (Encryption/Integrity/Identity)
Platform, Applications, Identity & Access Management
Operating System, Network & Firewall Configuration
Customer Data A
maz
on
Shared responsibility
You
AWS Global Infrastructure Regions
Availability Zones Edge Locations
Certifications
SOC 1 Type 2 (formerly SAS-70)
ISO 27001
PCI DSS for EC2, S3, EBS, VPC, RDS, ELB, IAM
FISMA Moderate Compliant Controls
HIPAA & ITAR Compliant Architecture
Physical Security
Datacenters in nondescript facilities
Physical access strictly controlled
Must pass two-factor authentication at least twice for
floor access
Physical access logged and audited
HW, SW, Network
Systematic change management
Phased updates deployment
Safe storage decommission
Automated monitoring and self-audit
Advanced network protection
Security standards
http://aws.amazon.com/security
So what are we going to build
today?
A dynamic web application featuring:
A highly available, fault tolerant architecture Elasticity to deal with spikes in load
A fully managed database Static content served through a CDN
Break
Thanks to our sponsors
Shaun Pearce AWS Solutions Architect
A dynamic web application featuring:
A highly available, fault tolerant architecture Elasticity to deal with spikes in load
A fully managed database Static content served through a CDN
Region
Instance
Availability Zone Availability Zone
Region
Instance
S3
Availability Zone Availability Zone
Region
Instance
Cloud Front
S3
Availability Zone Availability Zone
Region
Instance
Cloud Front
S3
RDS Availability Zone Availability Zone
Region
Instance
Elastic Load Balancer
Cloud Front
S3 Auto scaling Group
RDS Availability Zone Availability Zone
Instance
But before we start….
Region
Cloud Front
RDS Availability Zone Availability Zone
Region
Instance
Availability Zone Availability Zone
bootstrapping passing data to an instance
Instance request
User data
Instance request
User data
Instance
Meta-data service
Script executed on launch: <script> ipconfig /all > c:\ipconfig.txt netstat > c:\netstat.txt
</script>
Script executed on launch: #!/bin/sh yum -y install httpd php php-mysql chkconfig httpd on /etc/init.d/httpd start
security groups instance firewalling
Security Group
instance
Port 80 (HTTP)
Port 22 (SSH)
Name Description Protocol Port range IP Address, range, or another security group
Region
Instance
Availability Zone Availability Zone
index.php Reads instance meta-data
Some php code that gets the data <?php
// get the instance id $url = "http://169.254.169.254/latest/meta-data/instance-id"; $instance_id = file_get_contents($url); // get the AZ where the instance is running $url = "http://169.254.169.254/latest/meta-data/placement/availability-zone";
$zone = file_get_contents($url);
// get the security group it is in $url = "http://169.254.169.254/latest/meta-data/security-groups";
$group = file_get_contents($url); // get the public DNS name $url = "http://169.254.169.254/latest/meta-data/public-hostname";
$hostname = file_get_contents($url); ?>
Some php code that gets the data <?php
// get the instance id $url = "http://169.254.169.254/latest/meta-data/instance-id"; $instance_id = file_get_contents($url); // get the AZ where the instance is running $url = "http://169.254.169.254/latest/meta-data/placement/availability-zone";
$zone = file_get_contents($url);
// get the security group it is in $url = "http://169.254.169.254/latest/meta-data/security-groups";
$group = file_get_contents($url); // get the public DNS name $url = "http://169.254.169.254/latest/meta-data/public-hostname";
$hostname = file_get_contents($url); ?>
And displays it
Instance ID: <?php echo $instance_id; ?> Availability Zone: <?php echo $zone; ?> Security Group: <?php echo $group; ?>
Region
Instance
Availability Zone Availability Zone
S3 For Static Website Hosting
Region
Instance
S3
Availability Zone Availability Zone
CloudFront Global CDN
Region
Instance
Cloud Front
S3
Availability Zone Availability Zone
Amazon RDS Managed Relational DB
Region
Instance
Cloud Front
S3
RDS Availability Zone Availability Zone
auto-scaling with elastic load balancing
Region
Instance
Elastic Load Balancer
Cloud Front
S3 Auto scaling Group
RDS Availability Zone Availability Zone
Instance
Availability Zone Availability Zone
Region
Availability Zone
Instance Instance Instance
Elastic Load Balancer Auto scaling Group
Availability Zone Availability Zone
Region
Availability Zone
Instance Instance Instance Instance Instance Instance
Elastic Load Balancer Auto scaling Group
Availability Zone Availability Zone
Region
Availability Zone
Instance Instance Instance
Elastic Load Balancer Auto scaling Group
Availability Zone Availability Zone
Region
Availability Zone
Elastic Load Balancer Auto scaling Group
Instance Instance
Availability Zone Availability Zone
Region
Availability Zone
Instance Instance
Elastic Load Balancer Auto scaling Group
Instance
Describes what Auto Scaling will create when adding
Instances
- AMI - - Instance Type - - Security Group -
- Instance Key Pair -
Only one active launch configuration at a time
Auto Scaling will terminate instances with old launch
configuration first rolling update
Auto Scaling managed grouping of EC2 instances
Automatic health check to maintain pool size
Automatically scale the number
of instances by policy – Min, Max, Desired
Automatic Integration with ELB
Automatic distribution & balancing across AZs
Parameters for performing an Auto Scaling action
Scale Up/Down and by how much
ChangeInCapacity (+/- #) ExactCapacity (#)
ChangeInPercent (+/- %)
Cool Down (seconds)
Policy can be triggered by CloudWatch events
Launch Configuration Auto-Scaling Group Auto-Scaling Policy
Region
Instance
Elastic Load Balancer
Cloud Front
S3 Auto scaling Group
RDS Availability Zone Availability Zone
Instance
add some load from another EC2 instance
questions?
Next Steps
We typically see customers start by trying our services
Get started now at : aws.amazon.com/getting-started
You can get started for free…
Get started now at : aws.amazon.com/free
Design your application for the AWS Cloud
More details on the AWS Architecture Center at : aws.amazon.com/architecture
AWS Training & Certification Certification
aws.amazon.com/certification
Demonstrate your skills, knowledge, and expertise
with the AWS platform
Self-Paced Labs
aws.amazon.com/training/ self-paced-labs
Try products, gain new skills, and get hands-on
practice working with AWS technologies
aws.amazon.com/training
Training
Skill up and gain confidence to design, develop, deploy and
manage your applications on AWS
We’d love your feedback!
Thanks to our sponsors
@AWS_UKI for local AWS events & news @AWScloud for Global AWS News and Announcements
©Amazon.com, Inc. and its affiliates. All rights reserved.
Thank You – AWS 101 Ian Massingham - Technical Evangelist
8 December 2014
@IanMmmm