riak cs build your own cloud storage
DESCRIPTION
Riak CS Build Your Own Cloud Storage by Pavan VenkateshTRANSCRIPT
![Page 1: Riak CS Build Your Own Cloud Storage](https://reader035.vdocument.in/reader035/viewer/2022081400/554bc045b4c9053a298b5094/html5/thumbnails/1.jpg)
Build Your Own Cloud Storage
![Page 3: Riak CS Build Your Own Cloud Storage](https://reader035.vdocument.in/reader035/viewer/2022081400/554bc045b4c9053a298b5094/html5/thumbnails/3.jpg)
About Basho Our Mission is to Be The Leader in Distributed Systems
• Founded January 2008
• 115+ employees
• Headquartered in Cambridge, with regional offices in San Francisco, Washington DC, London and Tokyo
• Makers of Riak & Riak CS
• Thousands of Users Worldwide including over 20% of the Fortune 50
• 30,000+ downloads per month now up from 19,500 in Dec 2011
• Strategic Partners include Citrix, IDC Frontier, Yahoo! Japan, and Microsoft
![Page 4: Riak CS Build Your Own Cloud Storage](https://reader035.vdocument.in/reader035/viewer/2022081400/554bc045b4c9053a298b5094/html5/thumbnails/4.jpg)
Product Suite Basho’s distributed systems technologies
EnterpriseDS
Open Source Distributed Database
Commercial Distributed Database
Distributed Cloud Storage Pla7orm
• Always-‐available, scalable, ops-‐friendly NoSQL database
• Over 35,000 downloads per month
• Thousands of users worldwide
• Version 1.0 unveiled September 2011
• Adds mulK-‐data center replicaKon & Basho 24x7 support
• Enables Secondary Search and MapReduce Clusters
• Requires commercial contract and secure download
• Version 1.4 in May 2013
• Expands with mulK-‐tenancy, large object support, metering and Amazon S3 API
• Enables public and private cloud storage
• SoXware released to open source on March 20th
![Page 5: Riak CS Build Your Own Cloud Storage](https://reader035.vdocument.in/reader035/viewer/2022081400/554bc045b4c9053a298b5094/html5/thumbnails/5.jpg)
Overview
![Page 6: Riak CS Build Your Own Cloud Storage](https://reader035.vdocument.in/reader035/viewer/2022081400/554bc045b4c9053a298b5094/html5/thumbnails/6.jpg)
Riak CS is...
enterprise cloud storage
Riak S3-compatibility
multi-tenancy
per user reporting
built on top of
large object storage
![Page 7: Riak CS Build Your Own Cloud Storage](https://reader035.vdocument.in/reader035/viewer/2022081400/554bc045b4c9053a298b5094/html5/thumbnails/7.jpg)
Enabling you to host your own
PUBLIC PRIVATE
& CLOUDS
or…. Reliable Storage Behind Apps
![Page 8: Riak CS Build Your Own Cloud Storage](https://reader035.vdocument.in/reader035/viewer/2022081400/554bc045b4c9053a298b5094/html5/thumbnails/8.jpg)
Riak CS Use Cases
Public Cloud Storage
S3 Without AWS Cloud Drive (General Content Storage)
Backup-as-a-Service
Archival and Preservation
Storage for Internal
Clients / Depts.
![Page 9: Riak CS Build Your Own Cloud Storage](https://reader035.vdocument.in/reader035/viewer/2022081400/554bc045b4c9053a298b5094/html5/thumbnails/9.jpg)
Riak CS
Riak Stanchion
![Page 10: Riak CS Build Your Own Cloud Storage](https://reader035.vdocument.in/reader035/viewer/2022081400/554bc045b4c9053a298b5094/html5/thumbnails/10.jpg)
Riak CS node for every
node of Riak 1
![Page 11: Riak CS Build Your Own Cloud Storage](https://reader035.vdocument.in/reader035/viewer/2022081400/554bc045b4c9053a298b5094/html5/thumbnails/11.jpg)
How It Works
![Page 12: Riak CS Build Your Own Cloud Storage](https://reader035.vdocument.in/reader035/viewer/2022081400/554bc045b4c9053a298b5094/html5/thumbnails/12.jpg)
RiakNode
RiakNode
RiakNode
RiakNode
RiakNode
Large Object
Riak CS
S3 API
Reporting API
Riak CS
S3 API
Reporting API
Riak CS
S3 API
Reporting API
Riak CS
S3 API
Reporting API
Riak CS
S3 API
Reporting API
1. User uploads an object
1 MB
2. Riak CSbreaks object
into 1 MB chunks
1 MB 1 MB 1 MB 1 MB 1 MB 1 MB 1 MB 1 MB 1 MB 1 MB 1 MB 1 MB 1 MB 1 MB 1 MB 1 MB 1 MB
3. Riak CSstreams chunks to Riak nodes
4. Riak replicates and stores chunks
![Page 13: Riak CS Build Your Own Cloud Storage](https://reader035.vdocument.in/reader035/viewer/2022081400/554bc045b4c9053a298b5094/html5/thumbnails/13.jpg)
Under The Hood
![Page 14: Riak CS Build Your Own Cloud Storage](https://reader035.vdocument.in/reader035/viewer/2022081400/554bc045b4c9053a298b5094/html5/thumbnails/14.jpg)
Riak Riak is a distributed NoSQL key-value store
Store anything: text, images
More Flexible
![Page 15: Riak CS Build Your Own Cloud Storage](https://reader035.vdocument.in/reader035/viewer/2022081400/554bc045b4c9053a298b5094/html5/thumbnails/15.jpg)
Riak Remains flexible in the long run, because of design
![Page 16: Riak CS Build Your Own Cloud Storage](https://reader035.vdocument.in/reader035/viewer/2022081400/554bc045b4c9053a298b5094/html5/thumbnails/16.jpg)
Riak High availability
Low-latency
Horizontal scalability
Fault-tolerance
Ops friendliness
![Page 17: Riak CS Build Your Own Cloud Storage](https://reader035.vdocument.in/reader035/viewer/2022081400/554bc045b4c9053a298b5094/html5/thumbnails/17.jpg)
Riak Masterless • No master/slave or different roles • All nodes are equal • Write availability and scalability • All nodes can accept/route requests
![Page 18: Riak CS Build Your Own Cloud Storage](https://reader035.vdocument.in/reader035/viewer/2022081400/554bc045b4c9053a298b5094/html5/thumbnails/18.jpg)
Riak No Sharding • Consistent hashing • Prevents “hot spots” • Lowers operational burden of scale • Data rebalanced automatically
![Page 19: Riak CS Build Your Own Cloud Storage](https://reader035.vdocument.in/reader035/viewer/2022081400/554bc045b4c9053a298b5094/html5/thumbnails/19.jpg)
Riak Availability and Fault-Tolerance • Automatically replicates data • Read and write data during
hardware failure and network partition
• Hinted handoff
![Page 20: Riak CS Build Your Own Cloud Storage](https://reader035.vdocument.in/reader035/viewer/2022081400/554bc045b4c9053a298b5094/html5/thumbnails/20.jpg)
USERS multi-tenancy:
Riak CS will track individual usage/stats
access_key secret_key users identified by users authenticated by
![Page 21: Riak CS Build Your Own Cloud Storage](https://reader035.vdocument.in/reader035/viewer/2022081400/554bc045b4c9053a298b5094/html5/thumbnails/21.jpg)
BUCKETS users create buckets.
buckets are like folders. store objects in buckets.
names are globally unique.
![Page 22: Riak CS Build Your Own Cloud Storage](https://reader035.vdocument.in/reader035/viewer/2022081400/554bc045b4c9053a298b5094/html5/thumbnails/22.jpg)
OBJECTS stored in buckets.
store any file type.
![Page 23: Riak CS Build Your Own Cloud Storage](https://reader035.vdocument.in/reader035/viewer/2022081400/554bc045b4c9053a298b5094/html5/thumbnails/23.jpg)
Basic Operations S3 API Commands���
s3cmd mb s3://pavan-test
s3cmd ls s3://pavan-test
s3cmd put README.txt
s3://pavan-test s3cmd get s3://pavan-test/README.txt
![Page 24: Riak CS Build Your Own Cloud Storage](https://reader035.vdocument.in/reader035/viewer/2022081400/554bc045b4c9053a298b5094/html5/thumbnails/24.jpg)
Features
![Page 25: Riak CS Build Your Own Cloud Storage](https://reader035.vdocument.in/reader035/viewer/2022081400/554bc045b4c9053a298b5094/html5/thumbnails/25.jpg)
Riak CS Large Object Support • Started with 5GB / object • Now have multipart upload • Content agnostic
![Page 26: Riak CS Build Your Own Cloud Storage](https://reader035.vdocument.in/reader035/viewer/2022081400/554bc045b4c9053a298b5094/html5/thumbnails/26.jpg)
Riak CS S3-Compatible API • Use existing S3 libraries and tools • RESTful operations • Multipart upload • S3-style ACLs for object/bucket
permissions • S3 authentication scheme
![Page 27: Riak CS Build Your Own Cloud Storage](https://reader035.vdocument.in/reader035/viewer/2022081400/554bc045b4c9053a298b5094/html5/thumbnails/27.jpg)
Riak CS Administration and Users • Interface for user creation,
deletion, and credentials • Configure so only admins can
create users
![Page 28: Riak CS Build Your Own Cloud Storage](https://reader035.vdocument.in/reader035/viewer/2022081400/554bc045b4c9053a298b5094/html5/thumbnails/28.jpg)
Riak CS New Stuff in Riak 1.3 • Multipart upload: parts between
5MB and 5GB • Restrict access to buckets based on
source IP
![Page 29: Riak CS Build Your Own Cloud Storage](https://reader035.vdocument.in/reader035/viewer/2022081400/554bc045b4c9053a298b5094/html5/thumbnails/29.jpg)
Multi-Datacenter Replication • For active backups, availability
zones, disaster recovery, global traffic
• Real-time or full-sync • 24/7 support • Per-node or storage-based pricing
“[deploying Riak CS] reduces the risk of using AWS and allows customers to store their data in their own data centers, on their
own terms.” - Alex Williams, TechCrunch
![Page 30: Riak CS Build Your Own Cloud Storage](https://reader035.vdocument.in/reader035/viewer/2022081400/554bc045b4c9053a298b5094/html5/thumbnails/30.jpg)
Riak CS
![Page 31: Riak CS Build Your Own Cloud Storage](https://reader035.vdocument.in/reader035/viewer/2022081400/554bc045b4c9053a298b5094/html5/thumbnails/31.jpg)
Operations
![Page 32: Riak CS Build Your Own Cloud Storage](https://reader035.vdocument.in/reader035/viewer/2022081400/554bc045b4c9053a298b5094/html5/thumbnails/32.jpg)
OPERATIONAL STATS exposed via HTTP resource: /riak-cs/stats
block bucket object GET, PUT, DELETE LIST KEYS, CREATE,
DELETE, GET/PUT ACL GET, PUT, DELETE
HEAD, GET/PUT ACL
HISTOGRAMS & COUNTERS
![Page 33: Riak CS Build Your Own Cloud Storage](https://reader035.vdocument.in/reader035/viewer/2022081400/554bc045b4c9053a298b5094/html5/thumbnails/33.jpg)
TRACK INDIVIDUAL USER’S
THE “USAGE” BUCKET
ACCESS STORAGE
![Page 34: Riak CS Build Your Own Cloud Storage](https://reader035.vdocument.in/reader035/viewer/2022081400/554bc045b4c9053a298b5094/html5/thumbnails/34.jpg)
QUERY USAGE STATS
Storage and access statistics tracked on per-user basis, as rollups for slices of time • Operations, Count, BytesIn,
BytesOut, + system and user error • Objects, Bytes
![Page 35: Riak CS Build Your Own Cloud Storage](https://reader035.vdocument.in/reader035/viewer/2022081400/554bc045b4c9053a298b5094/html5/thumbnails/35.jpg)
Partnerships
![Page 36: Riak CS Build Your Own Cloud Storage](https://reader035.vdocument.in/reader035/viewer/2022081400/554bc045b4c9053a298b5094/html5/thumbnails/36.jpg)
Citrix CloudStack
• Combined platform for compute and storage
• Summer ‘13: integrated authentication support
• Videos on our blog
![Page 37: Riak CS Build Your Own Cloud Storage](https://reader035.vdocument.in/reader035/viewer/2022081400/554bc045b4c9053a298b5094/html5/thumbnails/37.jpg)
Roadmap
![Page 38: Riak CS Build Your Own Cloud Storage](https://reader035.vdocument.in/reader035/viewer/2022081400/554bc045b4c9053a298b5094/html5/thumbnails/38.jpg)
• Open stack integration • API • Keystone authentication
• Expand S3 API support
• Object versioning support • Cloudstack integration work
![Page 39: Riak CS Build Your Own Cloud Storage](https://reader035.vdocument.in/reader035/viewer/2022081400/554bc045b4c9053a298b5094/html5/thumbnails/39.jpg)
Q&A http://docs.basho.com/riakcs/latest/
http://docs.basho.com/riakcs/latest/riakcs-
tutorials/fast-track/
riak-users [email protected]