couchbase_uk_2013_getting_started_with_couchbase_app_dev

Getting Started with Couchbase App Development

Robin Johnson

Developer Advocate

Agenda

• Learn what it takes to get started and begin to build an application against Couchbase.

• Lean how you work with a Document Database and what is different. Learn how you work with documents in Couchbase itself.

• Gain an understanding of how fundamentally different approaches in Couchbase allow for your application to scale and consistently perform well.

Getting Started

Where to Obtain Couchbase

Download from couchbase.com/download

Install via .rpm, .deb, .exe, or .app

EASY!

www.couchbase.com/develop

PythonRuby

Go Clojure

Official SDKs

Community SDKs

• Current Release: 1.1.4 (supports both 1.8.1 and 2.0.x)

• Download from the Website (JARs) or use Maven

• Maven: add our Repository: http://files.couchbase.com/maven2/

• Getting Started Guide: http://www.couchbase.com/docs/couchbase-sdk-java-1.1/getting-started.html

Quickstart Java

• Current Release: 1.2.3 (supports both 1.8.1 and 2.0.x)

• Package published to NuGet

• Install-Package CouchbaseNetClient

• Binaries published to http://www.couchbase.com/develop/net/current

• Getting Started Guide: http://www.couchbase.com/docs/couchbase-sdk-net-1.2/getting-started.html

Quickstart .NET

• Based on the information given, the Client tries to establish an initial connection.

• Once that’s done, it connects to a streaming API (HTTP chunked).

• Cluster updates are fetched from that connection.

• Failover/Add/Remove scenarios all update the clients in near realtime – no application restarts required!

• Key/Value access is done directly against the nodes.

• For View access one of the nodes is picked out which aggregates the results.

Client Architecture Overview

Couchbase ClientApp Server

make connection

receive topologyCouchbase TopologyReceived

Client Setup: Getting Cluster Configuration

Client Setup: Getting Cluster Configuration

Couchbase Client

New Node Added to Cluster and Coming OnlineCouchbase

TopologyUpdate

Developing with Couchbase

• Couchbase is structured as a Key-Value store: every Document has a Key and a Value.

• Keys can be any UTF-8 string up to 255 characters long.

• Keys are unique, within a database (bucket), there can only be one document with a associated key.

• Keys are completely in the control of the application developer, there is no internal mechanism for key generation.

• Values can be JSON, strings, numbers, binary blobs, or a special positive atomic counter (unsigned integer).

• Values can be up to 20MB in size.

Fundamentals

Operations in Couchbase

Basics: Retrieve

• get (key) Retrieve a document

• gets(key) Retrieve a document and the CAS value associated with the object

(more on this in a bit)

Basics: Create, Update, Delete

• set (key, value) Store a document, overwrites if exists

• add (key, value) Store a document, error/exception if it already exists

• replace (key, value) Store a document, error/exception if doesn’t exist

• delete(key) Delete the document from the system

Also: new feature to allow applications to express durability requirements: ReplicateTo.TWO, PersistTo.TWO

Atomic Integers

Atomic Counters are a special structure in Couchbase, they are executed in order and are Positive Integer Values

•set (key, value) Use set to initialize the counter

• cb.set(“my_counter”, 1)

•incr (key) Increase an atomic counter value, default by 1

• cb.incr(“my_counter”) # now it’s 2

•decr (key) Decrease an atomic counter value, default by 1

• cb.decr(“my_counter”) # now it’s 1

Simple Example in Ruby# user.rbrequire “rubygems”require “couchbase”

class User attr_accessor :name, :email, :title, :twitter

def initialize(attr = {}) attr.each do |name, value| setter = "#{name}=” next unless respond_to?(setter) send(setter, value) end end

def save client = Couchbase.bucket client.set(@email.downcase, self.to_json) endend

# example.rbrequire “./user.rb”

u1 = User.new({ :email => “robin@couchbase.com”,:name => “Robin Johnson”,:title => “Developer Advocate”,:twitter => “@rbin”

})

u1.save

# user.rbrequire “rubygems”require “couchbase”

class Userattr_accessor :name, :email, :title, :twitter

def initialize(attr = {}) ... end

def savec = Couchbase.bucketc.set(@email.downcase, self.to_json)

end

def self.find_by_email(email)c = Couchbase.bucketdoc = c.get(email.downcase)return doc ? User.new(doc) : nil

endend

# example.rbrequire “./user.rb”

u1 = User.new({ :email => “robin@couchbase.com”,:name => “Robin Johnson”,:title => “Developer Advocate”,:twitter => “@rbin”

})

u1.save

u1 = User.find_by_email(“robin@couchbase.com”)

if u1puts “User Found!”puts u1.inspectelseputs “User Not Registered!”end

Add Lookup Class Method

# user.rbrequire “rubygems”require “couchbase”

class Userattr_accessor :name, :email, :title, :twitterattr_accessor :fb_id, :fb_token

def initialize(attr = {}) ... end

def save ... end

def self.find_by_email(email)c = Couchbase.bucketdoc = c.get(email.downcase)return doc ? User.new(doc) : nil

endend

# example.rbrequire “./user.rb”

u1 = User.find_by_email(“robin@couchbase.com”)

if u1u1.fb_id = “682829292”u1.fb_token = “f02jdjd020d8373730djd02”u1.saveelse# create userend

Agile Model Development

Compare and SwapOptimistic Concurrency in a Distributed System

Actor 1 Actor 2

Couchbase Server

CAS mismatch!Success

# actors.rb

c = Couchbase.bucketc.set(“mydoc”, { :myvalue => nil }

doc1, flags, cas = c.get(“mydoc”, :extended => true)

#c.set (“mydoc”, { “myvalue”: true }, :cas => cas)

# will fail because cas has changedc.set (“mydoc”, { “myvalue”: true }, :cas => cas)

Demo

Programming Model

The Relational Approach to Storing Data

http://martinfowler.com/bliki/AggregateOrientedDatabase.html

Relational databases were not designed with clusters in mind, which is why people have cast around for an alternative. Storing aggregates as fundamental units makes a lot of sense for running on a cluster.

When you need to retrieve data from RDBMS, you are "aggregating" or "denormalizing" the data for your application through queries with joins, where clauses and order by clauses.

In Document Databases, instead of breaking data into tables and foreign keys, you store the aggregate data together in JSON document(s).

What is an aggregate?

Document Database by Comparison

o::1001{uid: “ji22jd”,customer: “Ann”,line_items: [

{ sku: 0321293533, quan: 3, unit_price: 48.0 },{ sku: 0321601912, quan: 1, unit_price: 39.0 },{ sku: 0131495054, quan: 1, unit_price: 51.0 }

],payment: { type: “Amex”, expiry: “04/2001”,

last5: 12345}

• Easy to distribute data• Makes sense to application programmers

Relational vs Document Model

27

Relational data modelHighly-structured table organization with rigidly-defined data formats and record structure.

C1 C2 C3 C4

Document data modelCollection of complex documents witharbitrary, nested data formats andvarying “record” format.

JSONJSON

JSON

{

}

SQL Normalized Tables

28

Addresses

1 DEN 30303CO

2 MV 94040CA

3 CHI 60609IL

Users

KEY First ZIP_IDLast

4 NY 10010NY

1 Jasdeep 2Jaitla

2 Joe 2Smith

3 Ali 2Dodson

4 John 3Doe

ZIP_ID CITY ZIPSTATE

To get information about specific user, you perform a join across two tables

foreign key

SELECT * FROM Users u INNER JOIN Addresses a ON u.zip_id = a.zip_id WHERE key=1

Documents are Aggregates

29

+

Addresses

1 DEN 30303CO

2

3 CHI 60609IL

4 NY 10010NY

ZIP_ID CITY ZIPSTATE

Users

KEY First ZIP_IDLast

2

2 Joe 2Smith

3 Ali 2Dodson

4 John 3Doe

All data in a single document

  { “ID”: 1, “First”: “Jasdeep”, “Last”: “Jaitla”, “ZIP”: “94103”, “CITY”: “SF”, “STATE”: “CA” }

JSON

=

couchbase.get(“user::1”)

Document Data is an Aggregate

1 Jasdeep Jaitla

94103CASF

1 Jasdeep Jaitla

Multiple Documents

Sample Model

• User profile

• Blog Post Object• Contains the Content (ID, Title, Body Text, Author)

• Blog Comment• Comments from other users

• Option 1 - Keep it with Blog Document• Option 2 - Separate it Out

{ “id”: “scalabl3_Hello_World”,“author”: “scalabl3”, “type”: “post”“title”: “Hello World”,“format”: “markdown”, “body”: “Hello from [Couchbase](http://couchbase.com).”, “html”: “<p>Hello from <a href=\“http: …,

“comments”: [ {“format”: “markdown”, “body”:”Awesome post!”}, {“format”: “markdown”, “body”:”Like it.” }]}

Blog Post Object

string id

string author

string type

string title

string format

string body

string html

array comments

Single Document

{ “id”: “scalabl3_Hello_World”,“author”: “scalabl3”, “type”: “post”“title”: “Hello World”,“format”: “markdown”, “body”: “Hello from [Couchbase](http://couchbase.com).”, “html”: “<p>Hello from <a href=\“http: …,“num_comments”: 2}

Blog Post Object

string id

string author

string type

string title

string format

string body

string html

int comments

Comment Object

string format

string body

int id

{“id”: “scalabl3_Hello_World::1”,“format”: “markdown”,“body”: “Awesome post!”}

{“id”: “scalabl3_Hello_World::2”,“format”: “markdown”,“body”: “Love It!”}

Multiple Documents

Differences of Note

Mental Adjustments

• In SQL we tend to want to avoid hitting the database as much as possible

• Even with caching and indexing tricks, and massive improvements over the years, SQL still gets bogged down by complex joins and huge indexes, so we avoid making database calls

• In Couchbase, get’s and set’s are so fast they are trivial, not bottlenecks, this is hard for many people to accept at first; Multiple get statements are commonplace, don’t avoid it!

Consistently Has High Performance

36

Couchbase Ensures…• Responses are always fast• Even if a response can’t be serviced

right away

Shift your Expectations:

• No longer a need to avoid round-trips

• No longer a need to avoid database requests for application speed

A Managed Cache

Scales Easily, Transparently

• As an application developer, you think about interacting with the cluster No longer a need to think about shards No longer have to deploy and manage internal or external proxies to

understand where the data is placed

Add Nodes to Cluster

• Two servers addedOne-click operation

• Docs automatically rebalanced across cluster

Even distribution of docsMinimum doc movement

• Cluster map updated

• App database calls now distributed over larger number of servers

REPLICA

ACTIVE

Doc 5

Doc 2

Doc

Doc

Doc 4

Doc 1

Doc

Doc

SERVER 1

REPLICA

ACTIVE

Doc 4

Doc 7

Doc

Doc

Doc 6

Doc 3

Doc

Doc

SERVER 2

REPLICA

ACTIVE

Doc 1

Doc 2

Doc

Doc

Doc 7

Doc 9

Doc

Doc

SERVER 3 SERVER 4 SERVER 5

REPLICA

ACTIVE

REPLICA

ACTIVE

Doc

Doc 8 Doc

Doc 9 Doc

Doc 2 Doc

Doc 8 Doc

Doc 5 Doc

Doc 6

READ/WRITE/UPDATE READ/WRITE/UPDATE

APP SERVER 1

COUCHBASE Client Library

CLUSTER MAP

COUCHBASE Client Library

CLUSTER MAP

APP SERVER 2

COUCHBASE SERVER CLUSTER

User Configured Replica Count = 1

Document Database, Simple ApproachWork with aggregates, usually JSON. Use simple operations to work with your data. Couchbase understands and is designed for distributed systems.

Deployment of Applications Allow for Scale, PerformanceNew approaches to storing and retrieving data in Couchbase give developers the ability to easily scale up and make their applications perform.

Couchbase in Summary

Easy to Get Started, Straightforward to UseDownload, install server. Binaries are available. Provisioning is fast through the browser and flexible with a REST interface. Install client using tools such as Java’s maven, Ruby’s gem, etc. Interact with simple verbs.

Q&A

Thanks!@rbin on twitter!