amazon web services introduction
DESCRIPTION
TRANSCRIPT
Amazon Web Services
Overview and introduction
About myself
• Head of Development Estonia @ Insparx GmbH• 24 years programming, 10 years leading teams• Startup experience – running on AWS / Azure clouds
• Linked - http://ee.linkedin.com/in/konstantinroot/ • Twitter - @konstantin_root
Overview
• What is AWS?
• What people usually mean when talking about AWS and why they are wrong
• Difference between AWS and VPS?• Mix AWS with other providers
When to use (or not)
• When to use?– Startups– For quick tests / experiments
• Requires deployment automation!
– Your business requires compliance with secure regulations from specific industry
• When not to use?– You have predictable load and want to have best
$$$/resources
Competition
• Azure• Rackspace• VPS – Linode, DigitalOcean, Softlayer, Hertzner, etc.
Understanding AWS locations
• 8 regions, 44 edge locations
Availability zones
• What are they?• Why they are
required?• How they are
connected?
Services availability
• Availability zone could go down– Usually related to EBS, ELB, EC2
problems
• Whole region could go down• If you require 24/7 with tight SLAs
– use multiple regions
Pricing structure
• Free tier and Startup plan• Each region have slightly different prices• US East is biggest, oldest and cheapest
region• Expect to pay for every
transaction/resource usage– Good predictability of expenses– Better service vs “all you can eat” providers
AWS overview
• Compute & Networking – EC2, EMR (Hadoop), Elastic Load Balancer & Auto Scaling, VPC & Direct Connect, Route 53
• Storage & CDN – EBS, S3, Glacier, CloudFront• Databases – RDS (MySQL, Oracle, SQL Server), DynamoDB,
SimpleDB, Redshift, ElastiCache (Memcached, Redis)• Application services – SQS (queue), SES (email), SNS (push
notifications), FPS (payment), CloudSearch, SWF (workflow), Elastic Transcoder
• Management & Deployment – IAM, CloudWatch, Elastic Beanstalk, OpsWorks (Chef), CloudFormation, CloudHSM (Hardware Security Module)
Services control
• Web Console• CLI– Azure minute of shame
• API• 3rd party services and
products
Payment
• Estonia credit cards are OK
• Expenses report• Understand and use free
tier
Support
• Forums - free– Azure minute of shame ;)
• Emails• Commercial support
Where to start?
Understand IAM
• Access to AWS resources• Root account vs user vs services• Groups, Users, Roles• What is role?– Permissions to access resource API calls
• 2 way authorization– Free application for mobile phones
@Live demo
Access from application
• Default option – use Access Key ID / Secret Access Key combo
• EC2 IAM Role– Avoid hardcoding credentials into
application configuration– Retrieve temporary token
programmatically
Amazon Elastic Compute Cloud
What is EC2
• Xen virtualization• Limited resources per server– Elastic Compute Unit (ECU) ~=
1-1.2Ghz 2007 Opteron/Xeon
• You share resources with others on server– Noisy neighborhood problem– Local disk usage problems
Instance types
• Details @ http://aws.amazon.com/ec2/instance-types/
• Micro instance – tricky one• EBS traffic counts in network limit
– that’s what EBS optimized instances for
Purchase options
• Understanding On-Demand vs Reserved vs Spot
• Different Reserved tiers• Reserved instances marketplace and ability to
change your purchases• What is Spot instance and when to use it?– Protect against price fluctuation– Don’t forget to bid on higher end instances with
same price
Security groups
• Understanding them• Best practices– Azure minute of
shame ;)
@Live demo
EC2 image options
• AWS Marketplace• All major OS are presented by vendors– Ubuntu selector -
http://cloud-images.ubuntu.com/locator/ec2/
• Create your own• EBS vs S3 for AMI
@Live demo
Storage options
• Local storage vs EBS• Local storage– SATA or SSD– No persistence on reboot
• EBS ~100 IOPS– Provisioned IOPS up to 4000 (16KB
block)– You could RAID EBS volumes– Choose EBS optimized instances when
required
Permanent IP address
• Elastic IP• Allocate and (re)assign
to servers• Low cost ~$3.6• Default limit is 5
Elastic scaling• When to use it?• You could use your own solution• Azure year of shame
When not to use EC2
• Have CPU intensive tasks not tied closely with AWS? EC2 is bad $$$ investment vs other providers
• High memory instances are expensive• Email server hosting
File storage options:EBS vs S3 vs Glacier
EBS
• Speed is limited, but could increase it with $$$• Pay for everything
– Pay for provisioned storage: 1GB = $0.10 ($0.125 for provisioned IOPS)– 1 million I/O requests = $0.10
• Could create on the fly snapshots to S3– First one is full then incremental ones
• Part of recent outages was related to it
S3
• Unlimited amount of objects• High durability/availability• High speed within region – 5-15MB/sec• Pay per operation and size only
– Modifications: 1K requests = $0.005– Read: 1K requests = $0.004– Used storage: 1GB = $0.095– 1GB traffic OUT = $0.12 (region traffic free)
• No folders (they are emulated)• 3rd party tool for accessing data - CloudBerry
Glacier
• Use case – archive lots of data, but read them rarely
• Integration with S3 – move files to Glacier automatically
• Pricing– Storage: 1GB = $0.01– Upload / retrieve: 1k operations = $0.050– Traffic: 1GB OUT = $0.120 (region traffic free)– Read 5% of data free then pay for it
Other services
Database options
• Relational databases – MySQL, Oracle, SQL Server– No PostgreSQL
• NoSQL– DynamoDB – fast and great, but $$$– SimpleDB – don’t use unless you
understand perfectly your reasons
• Caching– Memcached– Redis
• Redshift – warehousing solution for TB/PB scale
DynamoDB in details• Great product with some limitations• High durability and quick operations (<10ms)
– http://www.dynaspeed.net/ • Unique pricing:
– Pay per provisioned capacity (1 unit = 1KB)• 10 write units = $0.0065/hour• 50 read units = $0.0065/hour
– Indexed storage after 100MB: $0.25/GB– Traffic OUT: $0.12/GB (region traffic free)– Great saving if purchased upfront in 5k read/write units– Free tier – 5 writes, 10 reads
Elastic Load Balancer
• Integrates with EC2 health checks• DNS failover integration with Route 53• Auto Scaling integration• SSL termination, session stickiness• $0.025/hour, $0.008/GB for traffic• Not as flexible as HAProxy, but OK in
most cases
CloudFront
• AWS version of CDN• Many edge locations in all regions• For static files uses S3 folder as the source
– Could invalidate static objects with some delay
• Could cache dynamic content (think of Varnish)
• Low pricing – you pay only for actual usage
Message queue
• Limitations:– Max 256KB size of message– Long poll, but with 20sec limit – Limited routing options
• Expensive for large amounts:– 1 million requests = $0.50– 1 request unit = 64KB– Traffic $0.12/GB
• Alternatives:– Hosted RabbitMQ, ActiveMQ, Redis, ZeroMQ– 3rd party solution – IronMQ (http://www.iron.io/mq)
Route 53
• DNS service• Scalable and with good API• Advanced capabilities– DNS failover using health checks– Zone apex mapping to ELB– Weighted Round Robin– Latency based routing
Blogs to follow
AWS team http://aws.typepad.com/aws/Azure team http://weblogs.asp.net/scottgu/default.aspx