Red Hat Storage Server for AWS Craig Carl

Solution Architect, Amazon Web Services Tushar Katarki

Principal Product Manager, Red Hat

Veda Shankar Principal Technical Marketing Manager, Red Hat

GlusterFS Community Project l  Community driven Open Source @gluster.org l  Distributed file system on top of local file system l  No metadata server with elastic hashing algorithm l  Spreads files across local file system directories called bricks l  Stores metadata in extended attributes of local file system l  Modular stackable architecture based on user-space FUSE

What is Red Hat Storage l  Open, Software Defined Storage Solution for On-Premise,

Virtualized and Cloud environments l  Enterprise implementation of GlusterFS technology l  RHEL + XFS + GlusterFS l  Support Subscription l  24x7 Premium Support

RED HAT STORAGE CONCEPTS

VOLUME

A namespace presented as a

POSIX mount point and is comprised of bricks.

BRICK

The basic unit of storage, represented by an export

directory on a server

SERVER/NODES

Contain the bricks

Elastic Hash Algorithm

• No central metadata • No Performance Bottleneck • Eliminates risk scenarios • Location hashed intelligently on filename • Unique identifiers, similar to md5sum • The “Elastic” Part • Files assigned to virtual volumes • Virtual volumes assigned to multiple bricks

Why run RHS on AWS?

You need shared access to a file system GlusterFS, NFS and CIFS You need a really large file system I’ve created multiple 2PB clusters

Things to think about -

Run at least replica 2 in two availability zone Single AZ = no durability commitments Dual AZ = 99.95% available Durability

Backing store Performance

Network interface EBS interface EBS performance

Backing stores

EBS between 0.1% – 0.5% AFR per volume

Ephemeral

hs1.8xlarge 48 terabytes of storage across 24 hard disk drives

i2.8xlarge ~5.7 terabytes of storage across 8 SSDs

Running RHS on EC2

Use Reserved Instances! RHS is a perfect use case for reserved instances. Discounts of up to 65% Purely a billing construct, no technical implications. m3.xlarge is perfect for RHS @ $.25/hr (1yr heavy RI)

VOLUME CONFIGURATION

l Global namespace l Bricks are XFS filesystems l Volumes are made of bricks l Distributed l Replicated l Distributed and replicated l Geo-replication

UNIFIED FILE AND OBJECT Object  storage  • Objects  • Containers  • Accounts  Standards  based  • OpenStack  Swi5  interface  • GlusterFS  underneath  • RESTful  • APIs  NAS  protocol  support  • NFS  /  CIFS  /  GlusterFS  • No  translaCon  required    

Clients  talk  directly  to  necessary  nodes  

DATA FLOW: Gluster native client

Red Hat Storage User Perspective (distributed volumes)

server1:/exp1 server2:/exp1

DISTRIBUTED VOLUME

FILE 1 FILE 2 FILE 3

BRICK BRICK

MOUNT POINT server1:/exp1 server2:/exp1

DISTRIBUTED VOLUME

BRICK BRICK

Red Hat Storage user perspective (distributed replicated volumes)

MOUNT POINT

Replicated Volume 0

DISTRIBUTED VOLUME

FILE 1 FILE 2

BRICK (exp1)

Replicated Volume 1

BRICK (exp2)

server1 server2

BRICK (exp3)

BRICK (exp4)

server3 server4

Red Hat Storage Server for Public cloud

RED HAT STORAGE FOR PUBLIC CLOUD

EBS

Scale out performance, capacity, and availability

Scal

e up

ca

paci

ty

• GlusterFS Amazon Machine Images (AMIs) • The only way to achieve high availability of Elastic Block Storage (EBS) • Multiple EBS devices pooled • POSIX compatible (no application to rewrite required to run on Amazon EC2) • Scale out capacity and performance as needed

SINGLE GLOBAL NAMESPACE

...

...

EC2

... ... ... ... ... ...

Red Hat Storage Use Cases in AWS EC2

Data Availability on Amazon Web Services

DNS Gateway

EBS

Red Hat Storage

Web and App Tier Web and App Tier

EBS

Red Hat Storage

Load-balancers Load-balancers

Amazon East Zone A

Amazon East Zone B

EBS

Red Hat Storage

Amazon Region West

Synchronous Replication

Master Slave

Replication

Web and App Tier

Load-balancers

Web Servers

Content

Processing

Load Balancers

Geo Rep

MetaData Store

Distributed, Replicated RHS Vols

Content Delivery Network

Content Delivery Network

Content is available In RHS Volumes

EC2 West Region EC2 East Region

Content Uploads

Content Downloads

Global Access

Scale with Growth

High Availability

Disaster Recovery

Open Source

Multiple Access

Methods

Object Access

Posix

CONTENT SHARING WITH RHS

App Servers

Map Reduce Processing With RHS Hadoop Plugin

Load Balancers

Ad Profile DB

Distributed, Replicated RHS Vols

Impression Log Info

Click Thru Requests

RHS Hadoop Plugin

Scale with Growth

High Availability

Open Source

Map Reduce With RHS

AD SERVING WITH RHS

Click-Thru Logs

Compute Storage

Co-residency

Ad Serving

Demo

Demo Architecture

Check Out Other Red Hat Storage Activities at The Summit

• Enter the raffle to win tickets for a $500 gift card or trip to LegoLand! • Entry cards available in all storage sessions - the more you attend, the more chances you

have to win!

• Talk to Storage Experts: • Red Hat Booth (# 211)

•  Infrastructure •  Infrastructure-as-a-Service

• Storage Partner Solutions Booth (# 605)

• Upstream Gluster projects • Developer Lounge

Follow us on Twitter, Facebook: @RedHatStorage