red hat storage - introduction to glusterfs
TRANSCRIPT
![Page 1: Red Hat Storage - Introduction to GlusterFS](https://reader033.vdocument.in/reader033/viewer/2022060110/555a4cb5d8b42a47748b4661/html5/thumbnails/1.jpg)
1
Red Hat Storage Introduction To GlusterFS
October 2011
![Page 2: Red Hat Storage - Introduction to GlusterFS](https://reader033.vdocument.in/reader033/viewer/2022060110/555a4cb5d8b42a47748b4661/html5/thumbnails/2.jpg)
2
Today’s Speakers
Heather Wellington Program Marketing Manager
Storage Initiative
Tom Trainer Storage Product Marketing
Manager
![Page 3: Red Hat Storage - Introduction to GlusterFS](https://reader033.vdocument.in/reader033/viewer/2022060110/555a4cb5d8b42a47748b4661/html5/thumbnails/3.jpg)
3
Red Hat Acquires Gluster – What it Means for You
● Proven GlusterFS architecture
● Stability and long term viability
● Integration
● New features and functions
● Global reach
● Scalable, affordable, and flexible storage
![Page 4: Red Hat Storage - Introduction to GlusterFS](https://reader033.vdocument.in/reader033/viewer/2022060110/555a4cb5d8b42a47748b4661/html5/thumbnails/4.jpg)
4
What is the Gluster File System?
● Scale-out file storage software for ● Network Attached Storage (NAS)
● Object
● Big Data
● GlusterFS provides • Scalability to Petabytes & beyond
• Affordability
• Use of commodity hardware
• Flexibility
• Deploy in ANY environment
• Linearly scalable performance
• High availability
• Unified files and objects
• File system for Apache Hadoop
• Superior storage economics
![Page 5: Red Hat Storage - Introduction to GlusterFS](https://reader033.vdocument.in/reader033/viewer/2022060110/555a4cb5d8b42a47748b4661/html5/thumbnails/5.jpg)
5
File System in User Space (FUSE)
User Space
GlusterFS
Server (CPU/Mem)
1 TB
1 TB
1 TB
1 TB
1 TB
1 TB
1 TB
1 TB
1 TB
1 TB
1 TB 1 TB
Kernel
1 TB 1 TB
• Not tied to kernel
• No reassemblies
• Independence
• Flexibility
![Page 6: Red Hat Storage - Introduction to GlusterFS](https://reader033.vdocument.in/reader033/viewer/2022060110/555a4cb5d8b42a47748b4661/html5/thumbnails/6.jpg)
6
Many Enterprises Rely on GlusterFS
![Page 7: Red Hat Storage - Introduction to GlusterFS](https://reader033.vdocument.in/reader033/viewer/2022060110/555a4cb5d8b42a47748b4661/html5/thumbnails/7.jpg)
7
GlusterFS Architecture Design Goals
Capacity
Perf
orm
ance
● Innovation ● Eliminate metadata ● Dramatically improve performance ● Unify files and objects
● Elasticity ● Flexibility adapt to growth/reduction ● Add, delete volumes & users ● Without disruption
● Scale linearly ● Multiple dimensions ● Performance ● Capacity ● Aggregated resources
● Simplicity ● Ease of management ● No complex Kernel patches ● Run in user space
![Page 8: Red Hat Storage - Introduction to GlusterFS](https://reader033.vdocument.in/reader033/viewer/2022060110/555a4cb5d8b42a47748b4661/html5/thumbnails/8.jpg)
8
● Scalable
● No metadata server – faster file system
● Enables linear scaling of performance via elastic hashing
● Affordable
● Deploy on commodity hardware
● Flexible
● Software only
● Deploy on the infrastructure of choice
● Simultaneous files and objects
● Apache Hadoop Distributed File System (HDFS) alternative
● Modular, stackable storage OS architecture
● Data stored in native file formats
Key Differentiators
Capacity
Per
form
ance
![Page 9: Red Hat Storage - Introduction to GlusterFS](https://reader033.vdocument.in/reader033/viewer/2022060110/555a4cb5d8b42a47748b4661/html5/thumbnails/9.jpg)
9
What is GlusterFS Elastic Hashing?
● No metadata server
● An algorithmic approach ● Unique hash tag for each file stored
● Tags stored within the file system
● Rapid file read – low latency
Innovative Elastic Approach
![Page 10: Red Hat Storage - Introduction to GlusterFS](https://reader033.vdocument.in/reader033/viewer/2022060110/555a4cb5d8b42a47748b4661/html5/thumbnails/10.jpg)
10
Software Only – Future Proofing Storage
● Superior storage economics & flexibility
● Data center / private cloud use commodity hardware
● Public cloud – i.e. AWS, RightSacle, GoGrid, Nimbula – pay for only what
you need
● No hardware lock-in
● You choose hardware vendors - at purchase time or in the future
● Any Cloud – Public, private, and hybrid
● Performance, capacity, or availability levels
● GlusterFS – not proprietary, files are stored in native formats (i.e. EXT4)
![Page 11: Red Hat Storage - Introduction to GlusterFS](https://reader033.vdocument.in/reader033/viewer/2022060110/555a4cb5d8b42a47748b4661/html5/thumbnails/11.jpg)
11
A Strong Open Source Foundation
Global Adoption ● 200,000+ downloads ● ~16,000 /month
● 550+ registered deployments ● 45 countries
● 2,500+ registered users ● Mailing lists, Forums, etc.
● Active community ● Diverse testing environments
● Bugs identification and fixes
● Code contributions
● Member of broader ecosystem ● OpenStack
● Linux Foundation
● Open Virtualization Alliance
![Page 12: Red Hat Storage - Introduction to GlusterFS](https://reader033.vdocument.in/reader033/viewer/2022060110/555a4cb5d8b42a47748b4661/html5/thumbnails/12.jpg)
12
Anatomy of a GlusterFS Deployment
Gluster Global Namespace (HTTP, NFS, CIFS, Gluster Native) Application Data Objects
Clients/Apps Clients/Apps Clients/Apps
IP Network
VMs
VMDK VMDK
virtual storage pool
ApacheTM HadoopTM
● Standard clients running
standard apps
● Over any standard IP
network
● Access to data, as files
and folders and or objects,
in global namespace,
using a variety of standard
protocols
● Stored in a commoditized,
virtualized, scale-out,
centrally managed pool of
DAS, NAS
![Page 13: Red Hat Storage - Introduction to GlusterFS](https://reader033.vdocument.in/reader033/viewer/2022060110/555a4cb5d8b42a47748b4661/html5/thumbnails/13.jpg)
13
Client/Apps Client/Apps
Unifying Private and Public Clouds
Private Cloud Public Cloud
Replication
GlusterFS Global Namespace
Client/Apps Client/Apps Client/Apps
Client/Apps Client/Apps Client/Apps
Client/Apps
IP Network
Red Hat Storage Software Appliance
Server (CPU/Mem)
Server (CPU/Mem)
Server (CPU/Mem)
1 TB
1 TB
1 TB
1 TB
1 TB
1 TB
1 TB
1 TB
1 TB
1 TB
1 TB 1 TB
1 TB
1 TB
1 TB
1 TB
1 TB
1 TB
1 TB
1 TB
1 TB
1 TB
1 TB
1 TB
1 TB 1 TB
1 TB
1 TB
1 TB
1 TB
1 TB
1 TB
1 TB
1 TB
1 TB
1 TB
1 TB
1 TB
1 TB 1 TB
Scale Out Performance, Capacity, and Availability
Sc
ale
Up
Cap
ac
ity
RHEL
Red Hat Storage Software Appliance
RHEL
Red Hat Storage Software Appliance
RHEL
![Page 14: Red Hat Storage - Introduction to GlusterFS](https://reader033.vdocument.in/reader033/viewer/2022060110/555a4cb5d8b42a47748b4661/html5/thumbnails/14.jpg)
14
CIC Electronic Signature Solutions
● Problem
● Must leverage economics of the cloud
● Storage performance in the cloud too slow
● Need to meet demanding client SLA‟s
● Solution ● Red Hat Storage Software Appliance
● Amazon EC2 and Elastic Block Storage (EBS)
● RightScale Cloud Management
● Benefits
● Enabled faster development and delivery of new
products to clients
● All SLA‟s are met with headroom to spare
● Accelerating move to the cloud
● Scale-out architecture allows for constantly changing
resources to be added and accessed
● Data is highly available – allowing 24/7 client access to
data
Hybrid Cloud: Electronic Signature Solutions
• Reduced time-to-
market for new
products
• Meeting all client SLAs
• Accelerating move to
the cloud
![Page 15: Red Hat Storage - Introduction to GlusterFS](https://reader033.vdocument.in/reader033/viewer/2022060110/555a4cb5d8b42a47748b4661/html5/thumbnails/15.jpg)
15
Common Solutions Built on GlusterFS
● Media serving (CDN) ● Large scale file storage ● Tier 2 & 3 archive ● File sharing ● High Performance Computing (HPC)
storage ● IaaS storage layer ● Disaster recovery ● Backup & restore ● Private cloud
FS
![Page 16: Red Hat Storage - Introduction to GlusterFS](https://reader033.vdocument.in/reader033/viewer/2022060110/555a4cb5d8b42a47748b4661/html5/thumbnails/16.jpg)
16
Pandora Internet Radio
• Problem • Explosive user & title growth
• As many as 12 file formats for each song
• „Hot‟ content and long tail
• Solution • Three data centers, each with a six-node
GlusterFS cluster
• Replication for high availability
• 250+ TB total capacity
• Benefits • Easily scale capacity
• Centralized management; one administrator
to manage day-to-day operations
• No changes to application
• Higher reliability
• 1.2 PB of audio served
per week
• 13 million files
• Over 50 GB/sec peak
traffic
Private Cloud: Media Serving
![Page 17: Red Hat Storage - Introduction to GlusterFS](https://reader033.vdocument.in/reader033/viewer/2022060110/555a4cb5d8b42a47748b4661/html5/thumbnails/17.jpg)
17
Brightcove
• Problem • Cloud-based online video platform
• Explosive customer & title growth
• Massive video in multiple locations
• Costs rising, esp. with HD formats
• Solution • Complete scale-out based on commodity
DAS/JBOD and GlusterFS
• Replication for high availability
• 1PB total capacity
• Benefits • Easily scale capacity
• Centralized management; one administrator
to manage day-to-day operations
• Higher reliability
• Path to multi-site
• Over 1 PB currently in
Gluster
• Separate 4 PB project
in the works
Private Cloud: Media Serving
![Page 18: Red Hat Storage - Introduction to GlusterFS](https://reader033.vdocument.in/reader033/viewer/2022060110/555a4cb5d8b42a47748b4661/html5/thumbnails/18.jpg)
18
Partners Healthcare
• Problem • Capacity growth from 144TB to 1+PB
• Multiple distributed users/departments
• Multi OS access - Windows, Linux and Unix
• Solution • GlusterFS Cluster
• Red Hat Enterprise Linux (RHEL)
• Native CIFS/ NFS access
• Benefits • Capacity on demand / pay as you grow
• Centralized management
• Higher reliability
• OPEX decreased by 10X
• Over 500 TB
• 9 Sun “Thumper”
systems in cluster
Private Cloud: Centralized Storage as a Service
![Page 19: Red Hat Storage - Introduction to GlusterFS](https://reader033.vdocument.in/reader033/viewer/2022060110/555a4cb5d8b42a47748b4661/html5/thumbnails/19.jpg)
19
Simultaneous File and Object Storage (SFO)
● SFO Defined ● As part of GlusterFS, it is the first file system that enables
you to store and access data as an object and as a file
● Flexible and powerful ● Simplifies access and management of data.
• Eases migration of legacy, file-based applications to object
storage for use in the cloud
● Public beta available since 2011 ● Broad community testing and participation
● Selected enterprise customer engagements
A breakthrough from the traditional hardware approach…
![Page 20: Red Hat Storage - Introduction to GlusterFS](https://reader033.vdocument.in/reader033/viewer/2022060110/555a4cb5d8b42a47748b4661/html5/thumbnails/20.jpg)
20
Traditional Hardware Approach
● Proprietary
● Bolt-together disparate
technology ● Combined hardware raises costs
● Higher TCO
● Paying for what you many not need
● Increased risk ● Common hardware elements can fail
● Power supplies
● Fans
● Cabling…lots of cabling
Files Objects DB’s
Carved Up Storage Pool
NAS Object
Traditional Monolithic Hardware
Bolt-on Approach (i.e. EMC VNX)
Block “VNX reminds me of my old VHS, DVD and cable box….
….one thing fails and I’m blown out of the water.”
Beta Customer , 2011
![Page 21: Red Hat Storage - Introduction to GlusterFS](https://reader033.vdocument.in/reader033/viewer/2022060110/555a4cb5d8b42a47748b4661/html5/thumbnails/21.jpg)
21
Software Approach to File and Object Storage
• Network Attached Storage (NAS)
• NFS / CIFS / GlusterFS
• POSIX compliant
• Access files within objects
• Window Access
• Improves Windows performance
• Uses HTTP, not slower CIFS
• We will still support SAMBA
• Object Storage
• API
• Internet Protocol (IP)
• ResTFul
• Get/Put
• Buckets
• Objects seen as files
• Standards based
• Amazon Web Services S3 ReSTFul
interface compatible
• Access data as objects and a NAS
interface to access files (NFS, CIFS,
GlusterFS)
• Backup to AWS
• Public and private clouds
Store discrete video files and move numbers of them as objects and vice versa…
GlusterFS simplifies file and object storage
![Page 22: Red Hat Storage - Introduction to GlusterFS](https://reader033.vdocument.in/reader033/viewer/2022060110/555a4cb5d8b42a47748b4661/html5/thumbnails/22.jpg)
22
Introducing GlusterFS Compatibility for Apache Hadoop
Metadata Server NameNode
● Coexist, or alternative to HDFS
● NameNode metadata server eliminated
● Faster access times – faster file system
● All the features and benefits of GlusterFS
MapReduce MapReduce MapReduce MapReduce MapReduce
GlusterFS GlusterFS GlusterFS GlusterFS GlusterFS
![Page 23: Red Hat Storage - Introduction to GlusterFS](https://reader033.vdocument.in/reader033/viewer/2022060110/555a4cb5d8b42a47748b4661/html5/thumbnails/23.jpg)
23
Why It’s Different
● No metadata server ● No performance bottleneck on data lookups - fast file access
● Reduces requirement for replicated files from 3 to 2 ● 33% capacity savings
● Built in replication ● Synchronous for inter-node replication
● Asynchronous for geo-replication
● No single point of failure
● No block size restrictions ● Ideal for small and large files
● POSIX compliant file system ● Out of the box NFS, CIFS and Gluster native access
● Expanded data access options ● File and object access to data
● Access files from your object interface and access data within objects as files
● File based applications can access data without modification
![Page 24: Red Hat Storage - Introduction to GlusterFS](https://reader033.vdocument.in/reader033/viewer/2022060110/555a4cb5d8b42a47748b4661/html5/thumbnails/24.jpg)
24
Major Retailer – Analytics Group
● Problem ● Performance trails off as file quantity soars
● NameNode server errors degrade availability
● Must scale beyond current performance limitations
● Solution ● Red Hat Storage
● GlusterFS alternative to HDFS
● Elimination of NameNode server
● Benefits ● Accelerating overall performance
● Scale-out architecture allows for constantly changing
resources to be added and accessed
● Files are highly available – allowing 24/7access to data
● NFS and Object access to files
● Higher overall capacity utilization
● Reduced storage spend
Leverages Hadoop and GlusterFS
• Higher performance
• Greater availability
• Lower overall costs
Big Data Analytics
![Page 25: Red Hat Storage - Introduction to GlusterFS](https://reader033.vdocument.in/reader033/viewer/2022060110/555a4cb5d8b42a47748b4661/html5/thumbnails/25.jpg)
25
The Gluster Connector for OpenStack – July 2011
SWIFT
• Enables GlusterFS to be the underlying file system
• Connects GlusterFS to Xen and KVM hypervisor
• Unified File and Object storage
• Highly-available, scale-out NAS
• Alternative to SWIFT
OpenStack Imaging Services
Simultaneous File & Object Storage
… Compute
API Layer
Mobile Apps. Web Clients. Enterprise Software Ecosystem
OpenStack Prior to Gluster
OpenStack with Gluster
![Page 26: Red Hat Storage - Introduction to GlusterFS](https://reader033.vdocument.in/reader033/viewer/2022060110/555a4cb5d8b42a47748b4661/html5/thumbnails/26.jpg)
26
The Gluster Connector for OpenStack - July 2011
• Connector enables GlusterFS to be chosen as the file system
• Provides:
• Unified File and Object storage
• Highly scalable NAS
• High Availability – synchronous and asynchronous replication
• Preferred, scalable alternative to SWIFT
• Virtual motion of virtual machines (a.k.a. vmotion)
GlusterFS
Server (CPU/Mem)
Hypervisor
VM
GlusterFS
Server (CPU/Mem)
Hypervisor
VM
VM
VM
VM
Virtual storage pool
VM
VM
VM
![Page 27: Red Hat Storage - Introduction to GlusterFS](https://reader033.vdocument.in/reader033/viewer/2022060110/555a4cb5d8b42a47748b4661/html5/thumbnails/27.jpg)
27
Red Hat Storage Deployment Options
• Red Hat Storage Software Appliance • Deploy on bare metal
• Any hardware on Red Hat Hardware HCL
• Amazon Web Services (AWS) • Runs within Amazon Machine Image (AMI)
• GoGrid Cloud • Gluster Server Image (GSI) for scale-out NAS on GoGrid cloud
On-premise/datacenter
Public cloud
![Page 28: Red Hat Storage - Introduction to GlusterFS](https://reader033.vdocument.in/reader033/viewer/2022060110/555a4cb5d8b42a47748b4661/html5/thumbnails/28.jpg)
28
Summary
• GlusterFS is scale-out storage • NAS
• Object
• Big Data
• Scalable, affordable, and flexible
• Open Source
• Innovative architecture provides a better way to do storage
![Page 29: Red Hat Storage - Introduction to GlusterFS](https://reader033.vdocument.in/reader033/viewer/2022060110/555a4cb5d8b42a47748b4661/html5/thumbnails/29.jpg)
29
Questions & Answers
Your turn - ask our experts
• Register to try GlusterFS here: http://www.gluster.com/trybuy/
• Follow us on twitter: @RedHatStorage
• Additional resources here: http://www.gluster.com/products/resources/
• Join the community: http://www.gluster.org/
• Read our blog: http://blog.gluster.com/
Contact us at: [email protected] or 1-800-805-5215