data & analytics - session 4 - understanding storage options

James Brown, Business Development Manager

Choosing the right data

storage options with AWS

Object Storage

Block Storage

Connecting AWS Storage to On-premise

What are we going to cover?

We are sincerely eager to hear your feedback on this

presentation and on re:Invent.

Please fill out an evaluation form when you have a chance. We are constantly producing more data

We are sincerely eager to hear your feedback on this

presentation and on re:Invent.

Please fill out an evaluation form when you have a chance.

From all types of industries

#1

Object Storage

●○○

Amazon S3 Simple Storage Service

99.999999999%

Durability

Over 2 Trillion

Unique Customer Objects

Over 1.1 million Peak Transactions Per Second

Spotify adds over

20,000 tracks a

day.

“Amazon S3 gives

us confidence in

our ability to

expand storage

quickly while also

providing high

data durability.”

Emil Fredriksson,

operations director

Amazon S3 Store objects from 1 byte to 5 terabytes

Uses standards based REST and SOAP protocols

Highly durable, fully managed

Designed to make web-scale computing easier

Amazon S3 - Unlimited Storage

Amazon S3 Websites Easily handle peak loads at low cost

Pay as you go pricing

Inexpensive and Highly Scalable web hosting

Static Website Hosting

Real Clear Politics

served up to 20x

normal demand

with S3 Websites

CloudFront Pay as you go pricing with no long term

commitments

Caches content closer to your users to

lower latency and better performance

Integrated low latency content distribution network

Deliver content with lower latency with CloudFront

Storage costs as low as 5½ cents per GB

USE AMAZON S3

WHEN YOU NEED:

High Durability

Unlimited Storage Capacity

High Scale

High Volume Static Websites

Delivery via CloudFront

Amazon Glacier Low-Cost Archiving Service

1¢

Per GB / Month

$125

Per TB / Year

99.999999999%

Durability

3-5 Hours

Data Retrieval

Storage Costs

vs

Retrieval Costs

Lifecycle Policies Create rules on object prefix

Create rules to also delete expired object

Automatically tier S3 data in Glacier

Using Lifecycle policies to move data

Riverbed Whitewater Eliminates the burden of tape

Improve DR

Additional layer of security

Reduce storage requirements by 10 to 30x

A physical device can manage integration with Glacier

3rd Party Solution integration with Glacier

USE AMAZON GLACIER

WHEN YOU NEED:

Long term storage of infrequently accessed data

Reduce costs of existing storage

High durability

From theory into

practice with

Stuart Wright –

Director IT

Francis Hart –

Systems Architect

SEGA

AWS Summit 2013

Stuart Wright - Director IT & Networks

Francis Hart – Systems Architect

• SEGA overview

• Publishing & Development

• Games & Data

• Infrastructure past and present

• What we use

• Infrastructure close up

• S3

• Glacier

• CloudFront

Agenda

SEGA overview

• Who – Games development and publishing

• What - Games – Platforms

• Where - Publishing headquarters

Publishing & Development

• Game publishing, Game hosting

• QA, Logistics, Marketing, Sales, PR, etc

• Game development

• SEGA Europe

FM , Total War, Company of Heroes, Sonic mobile

Games & Data

• Changing games

• Online components

• Analytics

• Game evolution

• Live systems

Infrastructure Past and Present

• SEGA Datacentres

• Physical Hardware

• Flexibility and reliability

• Contracts

• Cloud IAAS

What we use

• AWS storage options

• Application development and delivery

• Backup, Archive, Disaster recovery

Infrastructure close up

• Shift to Virtualization and Cloud Services

• Consumable computing resource

• Consumption and production of data

• Retention of data

SEGA S3

• Shared Storage for EC2

• Static content

• Configuration

• Interface and Integration with other

storage options

SEGA S3

• Example:

EC2 Cloud Formation Cluster

SEGA Glacier

• Backups

• Data Retention

• Legal Retention

• RAW Analytic Data Storage

SEGA Glacier

• Example:

Log Retention (S3 Glacier)

SEGA CloudFront

• Simple

• Cost Effective

• Reliable

SEGA CloudFront

• Static Web Content

• Game Patching

• Media Content

#2

Block Storage

○●○

Amazon EBS Elastic Block Store

Ephemeral storage is on every EC2 instance

Amazon EBS Particularly suited for applications that require

a database, file system, or access to raw block

level storage.

Can be attached to a running instance

Network attached persistent block level storage volumes

EBS Drives can be moved between instances

How do our customers use EBS?

Enterprises

Enterprise workloads are built

on block storage

Oracle, SAP, Microsoft

Gaming customers

Very high performance

databases.

500,000 IOPS for a new game launch

Social Network / Mobile Apps

Very high performance and consistent I/O for

NoSQL and relational DBs

Marketing / Analytics

High performance I/O databases

Multiple drives on one Amazon EC2 instance

10GB 1TB

Provisioned IOPS Amazon EBS

Provisioned IOPS P-IOPS is designed to run transactional

applications that require high and

consistent I/O

Workloads on Provisioned IOPs

Use Cases

Relational Databases

NoSQL Databases (eg MongoDB)

High Performance File Systems

Enterprise Workloads (eg CRM, MS Exchange, ERP)

Provisioned IOPS - For steady I/O use Provisioned IOPS

- For bursty I/O, run the numbers

Optimized for price/performance

EBS Snapshots Amazon EBS

EBS Snapshots - Snapshots from EBS are stored in Amazon S3

- Rollback to a previous version

- Can be copied between regions

EBS Snapshots – Stored in Amazon S3

USE AMAZON EBS

WHEN YOU NEED:

Filesystem for an instance NTFS, ExtFS, RAID, LVM…

Long-term persistent storage

Data changes frequently

Access to raw, unformatted

block-level storage

#3 Connecting AWS Storage to

On-Premise Environments ○○●

AWS Storage Gateway Connecting with on-premise

AWS Storage Gateway - Low latency access to frequently accessed data

- Minimise the need to scale out on-premise

storage

- Snapshots allow access to old data

Up to 150TBs supported from a single Storage Gateway

Extend your on-premise storage

USE AWS STORAGE GATEWAY

WHEN YOU NEED:

Synchronize data for disaster

recovery

Departmental fileshare

Backup your data

Oracle Secure Backup Module

Oracle Secure Backup (OSB) Cloud

Module allows customers to backup

Oracle Databases directly to Amazon S3

using the Oracle Recovery Manager

(RMAN)

Backup Oracle into Amazon S3

Restore times

reduced from 15

to 2½ hours

Took less that 1

hour to configure

per database

Offered significant

improvements in

durability of

backups

IT’S ALL ABOUT

CHOICE PERFORMANCE-ORIENTED

COST-ORIENTED