Download - Deep Dive into EC2
Deep Dive into EC2Hiroshi Wada
Jul 24, 2013NICTA Software Dev Tools Series #10
Many many building blocks ...
AWS resources/services are relatively low level
Pick right resources and build your app
Steep learning curveSimilar services with different natureNon-obvious restrictionsDependency among services/resources
Easy to understand them by going *behind* the API
Region andAvailability Zone
Region: location where a set of services are providedRegions are completely isolated from each otherDifferent API endpoints, no replication across, ...Regions may provide different services
Availability Zone: a collection of data centersA region has ~4 AZsIsolated to prevent outage from spreadingResources may or may not be replicated
aws.amazon.com/about-aws/globalinfrastructure/
Region-bound and AZ-bound resources
Resources bound to an accountDNS, Access Management, ...
Resources bound to a RegionS3, machine images, DynamoDB, Elastic IP, LB, ...Replicated across AZs - highly available/durable
Resources bound to an AZInstance (VM), EBS (disk volume), ...Building fault tolerance is your job
Architecture in AZ (guess!)
RegionAZ-"a"
Rack serversoperating VMs(IP network)
No persistent diskfor users
(only ephemeral disks)
Storage AreaNetwork
provisioningEBS
(redundancy, RAID, ...)
FibreChannel
Internet
Rack serversoperating resources shared among AZs
(S3, LB, etc)
Communicationand replicationacross AZs(internal IP network)
Internet
AZ-"b"
Architecture in AZ (guess!)
RegionAZ-"a"
Rack serversoperating VMs(IP network)
No persistent diskfor users
(only ephemeral disks)
Storage AreaNetwork
provisioningEBS
(redundancy, RAID, ...)
FibreChannel
Internet
Rack serversoperating resources shared among AZs
(S3, LB, etc)
Communicationand replicationacross AZs(internal IP network)
Internet
AZ-"b"
Architecture in AZ (guess!)
RegionAZ-"a"
Rack serversoperating VMs(IP network)
No persistent diskfor users
(only ephemeral disks)
Storage AreaNetwork
provisioningEBS
(redundancy, RAID, ...)
FibreChannel
Internet
Rack serversoperating resources shared among AZs
(S3, LB, etc)
Communicationand replicationacross AZs(internal IP network)
Internet
AZ-"b"
Architecture in AZ (guess!)
RegionAZ-"a"
Rack serversoperating VMs(IP network)
No persistent diskfor users
(only ephemeral disks)
Storage AreaNetwork
provisioningEBS
(redundancy, RAID, ...)
FibreChannel
Internet
Rack serversoperating resources shared among AZs
(S3, LB, etc)
Communicationand replicationacross AZs(internal IP network)
Internet
AZ-"b"
Two types of instances: EBS backed
Root disk is in SAN and directly attached to machineStop/restart to change underlying machine in an AZ
Life cycle: start <-> stop -> terminateCharge for the use of EBS (capacity and I/O)
attach directly
EBS volume(virtual disk)
SAN
VM
servers and hypervisors
Two types of instances:instance store
Root disk is loaded from S3 to an ephemeral diskNo persistent storage (only ephemeral)
Life cycle: start -> terminateCustomization
Re-create image or use cloud-init on boot
Image
S3 load into ephemeral disk
VM
servers and hypervisors
Lifecycle of disks/images
A machine image is stored in S3=> you can launch instances in any AZ in the region
An instance is running on a physical machine=> bound to an AZ. Not highly available.
Image
S3
replicated among AZs(not visible to users)
load into an EBS volume
attach directly
EBS volume(virtual disk)
SANload intoephemeral disk
VM
AZ
Lifecycle of disks/images
An EBS volume is in SANs=> bound to an AZ
Only attachable to an instance in same AZNot durable compared to S3
Image
S3
replicated among AZs(not visible to users)
load into an EBS volume
attach directly
EBS volume(virtual disk)
SANload intoephemeral disk
VM
AZ
Lifecycle of disks/images
Snapshot (or machine image) of an EBS volume=> stored in S3. Bound to region. High durability
Creating instance-store images is not trivial ...
Snapshot,image
S3
EBS volume(virtual disk)
SAN
AZ
Create an image from VMEBS VM
Take a snapshot of a volume
Storage options - two flavors
"Regular" diskPros: you're familiar with it. random access.Cons: less available/durable (AZ bound)Services: EBS, ephemeral disks, RDS, ...
Key-value, NoSQLPros: high durability via distributionCons: proprietary interfaceServices: S3, Dynamo DB, SQS, ...