Rails in the Cloud Lessons learned from building a platform on EC2

Jonathan Weiss 04.07.2010

Who am I?

Working for Peritor in Berlin, Germany

Written, maintain, or involved in

  Webistrano

  Capistrano

  SimplyStored

  Happening

  The great fire of London

http://github.com/jweiss

@jweiss

2

Scalarium

EC2 Cluster Management

  Auto-Config

  Self-Healing

  Auto-Scaling

  One-click-deployment

www.scalarium.com

3

4

Amazon Elastic Compute Cloud

Virtual Server via API call

Pay per hour

Different instance types in different regions

5

EC2 Features

Spot instances

Elastic IP

Elastic Load Balancing

CloudWatch

Auto Scaling

Virtual Private Cloud 

Elastic Block Storage

6

7

Running on EC2 Setting up a Rails app

Running on EC2

Steps to complete

  Choose region and availability zone

  Define architecture

  Choose instance types and AMI

  Boot & configure instances

  Deploy application

  …

  Profit

8

Region and Availability Zone

Regions   Individual EC2 installations

  Multiple datacenters a.k.a availability zones

  Small differences in pricing

  US-east gets features first

Availability Zone

  Datacenter of instances & volumes

  Differently named for different accounts

  Traffic within the same zone is free

  Different Instance type availability

9

Existing Availability Zones

US-EAST-1

  us-east-1a

  us-east-1b

  us-east-1c

  us-east-1d

10

US-WEST-1

  us-west-1a

  us-west-1b

EU-WEST-1

  eu-west-1a

  eu-west-1b

AP-SOUTHEAST-1

  ap-southeast-1a

  ap-southeast-1b

Single AZ Deployment

Most common and simple setup

Not very different from typical Rails setup

11

Multi AZ Deployment

12

Architecture

13

Architecture

14

Architecture

15

Elastic IP

  Static, public IP in contrast to instance IP

 Associate to any instance (2-5min)

  Paying when not in use

 Use for DNS of load balancer

16

Elastic Block Store EBS

 Network based block device: 1GB – 1TB

  Persistent & redundant

  Can be used by one instance at a time

  Incremental Snapshots

  RAID for better performance

17

Elastic Load Balancer ELB

 Managed Load Balancer

 Auto-Scales via Round Robin DNS

  Can start/stop instances

  Slower and not as flexible as instance + HAProxy

The only way to scale past 1 Gbit/s!

18

Relational Data Store RDS

 Managed MySQL instance

  Choose instance size,

pay ~ 30% premium

 Hot-Failover replication available

 Define a 4h maintenance window

  Better performance than single EBS-

backed instance

19

Instance Types

20

RAM Cores x ECU Arch Hourly Cost (US-East)

Monthly Cost (US-East)

Standard

Small 1.7 GB 1 x 1 32bit $0.085 $62

Large 7.5 GB 2 x 2 64bit $0.34 $248

Extra Large 15 GB 4 x 2 64bit $0.68 $496

High-Memory

Extra Large 17.1 GB 2 x 3.25 64bit $0.50 $366

Double Extra Large 34.2 GB 4 x 3.25 64bit $1.20 $878

Quadruple Extra Large 68.4 GB 8 x 3.25 64bit $2.40 $1756

High-CPU

Medium 1.7 GB 2 x 2.5 32bit $0.17 $124

Extra Large 7 GB 8 x 2.5 64bit $0.68 $496

1 ECU = 1.0 – 1.2 GHz Opteron/Xeon from 2007

Recommendation

Start with   Small for load balancer

  Medium for App server

  Large for DB server

Grow to

  XL-HighCPU for load balancer

  XL-HighCPU for App server

  XL-HighMemory for DB server

21

Reserved Instances

Pay amount upfront and you get:

Hourly price ~ 33%

Guaranteed availability

24/7 instance cost: ~ 55%

Recommended!

22

Running on EC2

Steps to complete

 Choose region and availability zone

 Define architecture

 Choose instance types

  Boot & configure instances

  Deploy application

  …

  Profit

23

24

Instance Configuration From blank Linux to your Rails app server

Instance Bootstrap

Images

  One image for every server type

  All software pre-installed

  Few moving parts

  Fast boot

25

Scripted

  One image to rule them all

  Smallest common denominator

  Acquires configuration on start

  Installs & configures during boot process to fit role

  All parts moving

  Slower boot

2 Approaches

Instance Bootstrap

Images

  One image for every server type

  All software pre-installed

  Few moving parts

  Fast boot

26

Scripted

  One image to rule them all

  Smallest common denominator

  Acquires configuration on start

  Installs & configures during boot process to fit role

  All parts moving

  Slow boot

2 Approaches

Chef

Open Source Provisioning Tool

  Configures your servers

  Cookbooks & recipes

  Ruby DSL

27

Chef

28

Chef Solo

29

MemcachedSolo

30

Chef Solo Run

31

Configuration JSON

Run Chef Solo

Deploying

Deploy resource   Compatible with Capistrano layout

  Supports Capistrano hooks via deploy/hook_name.rb

  Async!

32

Chef Experiences

Great so far, but

  Not idempotent

  Unintuitive 2 phases

  Documentation could be better

  Example Cookbooks often so so

33

Running on EC2

Steps to complete

 Choose region and availability zone

 Define architecture

 Choose instance types

 Boot & configure instances

 Deploy application

  …

  Profit

34

© Peritor GmbH - Alle Rechte vorbehalten

Peritor GmbH Blücherstr. 22, Hof III Aufgang 6 10961 Berlin

Tel.: +49 (0)30 69 20 09 84 0 Fax: +49 (0)30 69 20 09 84 9

Internet: www.peritor.com E-Mail: info@peritor.com

Q&A