put knowledge bucket brain key riak

PUT Knowledge BUCKETBrain KEY Riak

An intro to Riak

by Phil

github.com/sideshowcoder

twitter.com/ischi

CodeThe slides contain code, this is how it works

a_string = "I am a string"a_string

Ok so Riak, eh?client = Riak::Client.newbucket = client.bucket("brain")object = bucket.get_or_new("riak")object.content_type = "application/json"object.data = { message: "Know stuff on riak!", level: "awesome" }object.storeobject.reload.data

What is Riak?Key / Value Store

Distributed

Dynamo based

Bucketsclient.bucket("brain").name

Keysbucket.keys

Valuesbucket.get("riak").data

DistributedKeyspace is spread across the members of a cluster via Consistent

Hashing

HashingKey => Hash(key) => Result => 2ˆn -1 possible outcomes

Digest.hexencode Digest::MD5.digest("my-key")

Consistent Hashing

Source littleriakbook

Replication

Source littleriakbook

CAP TheoremConsistency

Availability

Partition tolerance

Trade-offsRiak chooses adjustable AP, which means it never reaches full

Consistency but can be tuned to the needs

N/R/W ValuesN: Replication count

R: Min number of nodes to read from

W: Min number of nodes to write to

More on the basics behindthis?

Littleriakbook by Eric Redmond

dynamo paper

Ring members$ riak-admin member-status=========================== Membership ============================Status Ring Pending Node-------------------------------------------------------------------valid 100.0% -- 'riak@127.0.0.1'-------------------------------------------------------------------Valid:1 / Leaving:0 / Exiting:0 / Joining:0 / Down:0

Ring Status$ riak-admin ring-status============================ Claimant =============================Claimant: 'riak@127.0.0.1'Status: upRing Ready: true

======================== Ownership Handoff ========================No pending changes.

======================== Unreachable Nodes ========================All nodes are up and reachable

Just Key / Value store? ...mehh!

Tags

Map and Reduce

S3...

2i aka Secondary Indexesaka Tags

object = bucket.get("riak")object.indexes["level_bin"] = ["awesome"]object.storebucket.get_index("level_bin", "awesome")

Map and Reducejs_func = "function(v){ return [Riak.mapValuesJson(v)[0].message]; }"Riak::MapReduce.new(client).add("brain", "riak").map(js_func, :keep => true).run

Let's talk S3Wait what?

Run your own S3 (compatible) backed by Riak

FeaturesSame API as S3 (use aws-sdk in ruby)

Great reporting facilities

It's your own S3!

SetupInstall Riak

Install RiakCS

Install Stanchion

It's detailed here

... or just setup via Vagrant!$ vagrant plugin install vagrant-berkshelf$ vagrant plugin install vagrant-omnibus$ git clone https://github.com/basho/vagrant-riak-cs-cluster$ cd vagrant-riak-cs-cluster$ RIAK_CS_CREATE_ADMIN_USER=1 vagrant up

Virtual setup using vagrant

Create a usercurl -H 'Content-Type: application/json' -X POST \http://127.0.0.1:8080/riak-cs/user \--data '{"email":"admin@admin.com", "name":"admin"}'

OK let's try it!AWS::S3.new({ :access_key_id => "8AA5UQR3EACDS_TIAXLV", :secret_access_key => "M5eG0q1pN9y6hlc4-JbiCHXLt1YXCY2n7Sb5Yg==", :use_ssl => false, :proxy_uri => "http://localhost:8080"})

Create a buckets3bucket = s3.buckets.create("my-bucket", :acl => :public_read)s3.buckets.map(&:name)

Write something!obj = s3bucket.objects["riak.jpg"]obj.write File.read('./images/riak.jpg')obj.acl = :public_reads3bucket.objects.map(&:key)

OK lets read it back

This is loaded from local Riak CS

RecapDistributes Key / Value Store

Easy to interface with

Clients are build of HTTP

Use RiakCS to replace or augment S3

Questions?

ResourcesLittleriakbook by Eric Redmond

Basho docs on Racks