scalable database options on aws

AWS Database Options and Decision Factors

Best Practice Tips and Techniques

• Optimizing for Manageability and Scale Edmodo

• Optimizing for App Velocity and Scale Obama for America

• Leveraging YesSQL and NoSQL BrandVerity

Q & A

Easily and rapidly analyze

petabytes of data

1/10 the cost of traditional

data warehouses

Automated deployment &

administration

Compatible with popular

BI tools – MicroStrategy,

Jaspersoft

1. Zero to App in ____ Minutes

2. Zero to Millions of users in ____ Days

3. Zero to “Hero” in ____ Months

Focus on your App

Application tier

Load balancer

Database tier

Application tier

Load balancer

Database tier

Security, Innovation, Scale, Transactions, Performance, Durability, Availability, Skills..

Security, Innovation, Scale, Performance, Availability…

Security, Scale, Availability…

Not available

on AWS

SQL NoSQL

Low Cost High Cost

Do-it Yourself Fully

Managed

SQL NoSQL

Do-it Yourself Fully

Managed

MySQL

Oracle

SQL Server

SQL NoSQL

Do-it Yourself Fully

Managed

MySQL

Oracle

SQL Server

MariaDB

Postgres

…

SQL NoSQL

Do-it Yourself Fully

Managed

MongoDB

Cassandra

Redis

Memcache

DynamoDB

ElastiCache

SimpleDB

Should I use SQL

or NoSQL?

Should I use

MySQL on EC2 or

RDS?

Should I use

MongoDB,

Cassandra, or

DynamoDB?

Should I use Redis,

Memcache, or

ElastiCache?

?

What are my scale

and latency needs?

What are my

transactional and

consistency needs?

What are my

read/write, storage

and IOPS needs?

What are my time to

market and server

control needs?

?

Factors SQL NoSQL

Application • App with complex business logic? • Web app with lots of users?

Transactions • Complex txns, joins, updates? • Simple data model, updates, queries?

Scale • Developer managed • Automatic, on-demand scaling

Performance • Developer architected • Consistent, high performance at scale

Availability • Architected for fail-over • Seamless and transparent

Core Skills • SQL + Java/Ruby/Python/PhP • NoSQL + Java/Ruby/Python/PhP

Best of both worlds: Possible to Use SQL and NoSQL models in one App

Amazon RDS is a fully managed SQL database service.

Simple to deploy and scale

Without any operational burden.

Reliable and cost effective

Choice of Database engines

Schema design

Frequent server upgrades

Storage upgrades

Backup and recovery

Software upgrades

Patching

Hardware crash

Query construction

Query optimization

Configuration

Migration

Off load the “administration”

Focus on the “innovation”

Multiple databases per instance

Use MySQL tools & drivers

Quickly set up Read Replicas

High availability Multi-AZ option

Ability to promote Read replicas, Rename as Master

Diagnostics

Native MySQL replication

SSL for encryption over the wire

Monitor metrics

Shell, super user or direct file system access (Think security!)

MySQL Manual for Read Replica

OR Amazon RDS console

AZ location for both Primary and Stand-by are now visible

You can place 3rd replica in an AZ different from Primary and Stand-by

Ability to promote Read replicas

Rename as Master after promotion

ElastiCache is a fully managed Memcached

caching service.

Easy to set up and operate

Scale cache clusters with push button ease

Without any operational burden.

Ultra fast response time for read scaling

Amazon DynamoDB is a fully managed NoSQL

database service.

Store and retrieve any amount of data

Scale throughput to millions of IO

Without any operational burden.

Single digit millisecond latencies

CreateTable

UpdateTable

DeleteTable

DescribeTable

ListTables

PutItem

GetItem

UpdateItem

DeleteItem

BatchGetItem

BatchWriteItem

Query

Scan

Manage tables

Query specific items OR

scan the full table

“Select”, “insert”, “update”

items

Bulk select or update

(max 1MB)

So, what are the tips and techniques for

successful deployments?

Educates millions of students

Reaches millions of citizens

Analyzes billions of Ads

Kimo

Rosenbaum

Where learning happens.

Kimo Rosenbaum

kimo@edmodo.com

AWS re: Invent 2012

• Largest, fastest growing social platform for education

• Secure learning network for teachers and students

• Browser, iOS, Android

• Free for teachers and students

Learning 101

• 100,000 schools

• 14 million users

• 7 million new users in the last year

• 1 million visits daily

Stats 101

RDS DB Instance

Read Replica

RDS DB Instance

Read Replica

RDS DB Instance

Read Replica

Instance

Cache

Instance

Cache

Instance

Web

Auto scaling Group

Availability Zone

MySQL DB Instance RDS DB Instance

Read Replica MySQL DB Instance RDS DB Instance

Read Replica MySQL DB Instance RDS DB Instance

Read Replica

Elastic Load

Balancer Amazon Route 53

Amazon S3 Amazon Cloudfront

Amazon CloudWatch

Instances

• Restore from snapshot

• Replica creation

• Parameter tuning

• Metrics collection

• Know your app/data

DBA 101

Educates millions of students

Reaches millions of citizens

Analyzes billions of Ads

Jay

Edwards

Me.

• Twitter: First dedicated DBA

• OFA: Lead Database Engineer

• PalominoDB: CTO & VP/Operations

Obama for America.

• Technically sophisticated for a campaign

• Not “web-scale”

• Hockey-stick++ growth

• Downtime hurts. A lot…really, really, really a lot.

Hockey-stick++

OFA Architecture RDS Read Replica

ELB

ElastiCache

DynamoDB

RDS with

Multi-AZ

Problems!

• You always need more databases

• OFA had 24+ schemas & 100+ RDS instances

• You never have enough DBAs

• OFA had 1 – 2 x 0.5 fulltime MySQL DBAs

Why RDS?

• Makes operational issues very easy

• Need more replicas? BAM!

• Upsize hardware? KAPOW!

• Point in time restore? BIF!

Why not RDS?

• Hardware cap (vertical v. horizontal)

• Sophisticated use-cases

• Frequent topology changes

• Multi-region replication (on their roadmap)

• DBAs need busy work

Educates millions of students

Reaches millions of citizens

Analyzes billions of Ads Andy

Skalet

Managed Services Bias

New Products/Markets – YesSQL!

Big Data? Cast your problem

AWS Options

Case Study: Crawl history

andy@brandverity.com

• Managed services let you focus on creating value

• Amazon S3 - Very robust, handles large items, but you filter

• Amazon DynamoDB - Extremely fast, scalable, good value • Must cast your problem as kvs or key + range

• Amazon RDS - MySQL, without the headaches

• Amazon ElastiCache - As memcached, fast kvs for small data

• Multi column queries on big data? • Looking forward to the AWS solution

Thank you

aws.amazon.com/dynamodb

aws.amazon.com/rds

Free

Tier

aws.amazon.com/elasticache