couchbase_uk_2013_getting_started_with_couchbase_app_dev
TRANSCRIPT
Getting Started with Couchbase App Development
Robin Johnson
Developer Advocate
Agenda
• Learn what it takes to get started and begin to build an application against Couchbase.
• Lean how you work with a Document Database and what is different. Learn how you work with documents in Couchbase itself.
• Gain an understanding of how fundamentally different approaches in Couchbase allow for your application to scale and consistently perform well.
Getting Started
Where to Obtain Couchbase
Download from couchbase.com/download
Install via .rpm, .deb, .exe, or .app
EASY!
www.couchbase.com/develop
PythonRuby
Go Clojure
Official SDKs
Community SDKs
• Current Release: 1.1.4 (supports both 1.8.1 and 2.0.x)
• Download from the Website (JARs) or use Maven
• Maven: add our Repository: http://files.couchbase.com/maven2/
• Getting Started Guide: http://www.couchbase.com/docs/couchbase-sdk-java-1.1/getting-started.html
Quickstart Java
• Current Release: 1.2.3 (supports both 1.8.1 and 2.0.x)
• Package published to NuGet
• Install-Package CouchbaseNetClient
• Binaries published to http://www.couchbase.com/develop/net/current
• Getting Started Guide: http://www.couchbase.com/docs/couchbase-sdk-net-1.2/getting-started.html
Quickstart .NET
• Based on the information given, the Client tries to establish an initial connection.
• Once that’s done, it connects to a streaming API (HTTP chunked).
• Cluster updates are fetched from that connection.
• Failover/Add/Remove scenarios all update the clients in near realtime – no application restarts required!
• Key/Value access is done directly against the nodes.
• For View access one of the nodes is picked out which aggregates the results.
Client Architecture Overview
Couchbase ClientApp Server
make connection
receive topologyCouchbase TopologyReceived
Client Setup: Getting Cluster Configuration
Client Setup: Getting Cluster Configuration
Couchbase Client
New Node Added to Cluster and Coming OnlineCouchbase
TopologyUpdate
Developing with Couchbase
• Couchbase is structured as a Key-Value store: every Document has a Key and a Value.
• Keys can be any UTF-8 string up to 255 characters long.
• Keys are unique, within a database (bucket), there can only be one document with a associated key.
• Keys are completely in the control of the application developer, there is no internal mechanism for key generation.
• Values can be JSON, strings, numbers, binary blobs, or a special positive atomic counter (unsigned integer).
• Values can be up to 20MB in size.
Fundamentals
Operations in Couchbase
Basics: Retrieve
• get (key) Retrieve a document
• gets(key) Retrieve a document and the CAS value associated with the object
(more on this in a bit)
Basics: Create, Update, Delete
• set (key, value) Store a document, overwrites if exists
• add (key, value) Store a document, error/exception if it already exists
• replace (key, value) Store a document, error/exception if doesn’t exist
• delete(key) Delete the document from the system
Also: new feature to allow applications to express durability requirements: ReplicateTo.TWO, PersistTo.TWO
Atomic Integers
Atomic Counters are a special structure in Couchbase, they are executed in order and are Positive Integer Values
•set (key, value) Use set to initialize the counter
• cb.set(“my_counter”, 1)
•incr (key) Increase an atomic counter value, default by 1
• cb.incr(“my_counter”) # now it’s 2
•decr (key) Decrease an atomic counter value, default by 1
• cb.decr(“my_counter”) # now it’s 1
Simple Example in Ruby# user.rbrequire “rubygems”require “couchbase”
class User attr_accessor :name, :email, :title, :twitter
def initialize(attr = {}) attr.each do |name, value| setter = "#{name}=” next unless respond_to?(setter) send(setter, value) end end
def save client = Couchbase.bucket client.set(@email.downcase, self.to_json) endend
# example.rbrequire “./user.rb”
u1 = User.new({ :email => “[email protected]”,:name => “Robin Johnson”,:title => “Developer Advocate”,:twitter => “@rbin”
})
u1.save
# user.rbrequire “rubygems”require “couchbase”
class Userattr_accessor :name, :email, :title, :twitter
def initialize(attr = {}) ... end
def savec = Couchbase.bucketc.set(@email.downcase, self.to_json)
end
def self.find_by_email(email)c = Couchbase.bucketdoc = c.get(email.downcase)return doc ? User.new(doc) : nil
endend
# example.rbrequire “./user.rb”
u1 = User.new({ :email => “[email protected]”,:name => “Robin Johnson”,:title => “Developer Advocate”,:twitter => “@rbin”
})
u1.save
u1 = User.find_by_email(“[email protected]”)
if u1puts “User Found!”puts u1.inspectelseputs “User Not Registered!”end
Add Lookup Class Method
# user.rbrequire “rubygems”require “couchbase”
class Userattr_accessor :name, :email, :title, :twitterattr_accessor :fb_id, :fb_token
def initialize(attr = {}) ... end
def save ... end
def self.find_by_email(email)c = Couchbase.bucketdoc = c.get(email.downcase)return doc ? User.new(doc) : nil
endend
# example.rbrequire “./user.rb”
u1 = User.find_by_email(“[email protected]”)
if u1u1.fb_id = “682829292”u1.fb_token = “f02jdjd020d8373730djd02”u1.saveelse# create userend
Agile Model Development
Compare and SwapOptimistic Concurrency in a Distributed System
Actor 1 Actor 2
Couchbase Server
CAS mismatch!Success
# actors.rb
c = Couchbase.bucketc.set(“mydoc”, { :myvalue => nil }
doc1, flags, cas = c.get(“mydoc”, :extended => true)
#c.set (“mydoc”, { “myvalue”: true }, :cas => cas)
# will fail because cas has changedc.set (“mydoc”, { “myvalue”: true }, :cas => cas)
Demo
Programming Model
The Relational Approach to Storing Data
http://martinfowler.com/bliki/AggregateOrientedDatabase.html
Relational databases were not designed with clusters in mind, which is why people have cast around for an alternative. Storing aggregates as fundamental units makes a lot of sense for running on a cluster.
When you need to retrieve data from RDBMS, you are "aggregating" or "denormalizing" the data for your application through queries with joins, where clauses and order by clauses.
In Document Databases, instead of breaking data into tables and foreign keys, you store the aggregate data together in JSON document(s).
What is an aggregate?
Document Database by Comparison
o::1001{uid: “ji22jd”,customer: “Ann”,line_items: [
{ sku: 0321293533, quan: 3, unit_price: 48.0 },{ sku: 0321601912, quan: 1, unit_price: 39.0 },{ sku: 0131495054, quan: 1, unit_price: 51.0 }
],payment: { type: “Amex”, expiry: “04/2001”,
last5: 12345}
• Easy to distribute data• Makes sense to application programmers
Relational vs Document Model
27
Relational data modelHighly-structured table organization with rigidly-defined data formats and record structure.
C1 C2 C3 C4
Document data modelCollection of complex documents witharbitrary, nested data formats andvarying “record” format.
JSONJSON
JSON
{
}
SQL Normalized Tables
28
Addresses
1 DEN 30303CO
2 MV 94040CA
3 CHI 60609IL
Users
KEY First ZIP_IDLast
4 NY 10010NY
1 Jasdeep 2Jaitla
2 Joe 2Smith
3 Ali 2Dodson
4 John 3Doe
ZIP_ID CITY ZIPSTATE
To get information about specific user, you perform a join across two tables
foreign key
SELECT * FROM Users u INNER JOIN Addresses a ON u.zip_id = a.zip_id WHERE key=1
Documents are Aggregates
29
+
Addresses
1 DEN 30303CO
2
3 CHI 60609IL
4 NY 10010NY
ZIP_ID CITY ZIPSTATE
Users
KEY First ZIP_IDLast
2
2 Joe 2Smith
3 Ali 2Dodson
4 John 3Doe
All data in a single document
{ “ID”: 1, “First”: “Jasdeep”, “Last”: “Jaitla”, “ZIP”: “94103”, “CITY”: “SF”, “STATE”: “CA” }
JSON
=
couchbase.get(“user::1”)
Document Data is an Aggregate
1 Jasdeep Jaitla
94103CASF
1 Jasdeep Jaitla
Multiple Documents
Sample Model
• User profile
• Blog Post Object• Contains the Content (ID, Title, Body Text, Author)
• Blog Comment• Comments from other users
• Option 1 - Keep it with Blog Document• Option 2 - Separate it Out
{ “id”: “scalabl3_Hello_World”,“author”: “scalabl3”, “type”: “post”“title”: “Hello World”,“format”: “markdown”, “body”: “Hello from [Couchbase](http://couchbase.com).”, “html”: “<p>Hello from <a href=\“http: …,
“comments”: [ {“format”: “markdown”, “body”:”Awesome post!”}, {“format”: “markdown”, “body”:”Like it.” }]}
Blog Post Object
string id
string author
string type
string title
string format
string body
string html
array comments
Single Document
{ “id”: “scalabl3_Hello_World”,“author”: “scalabl3”, “type”: “post”“title”: “Hello World”,“format”: “markdown”, “body”: “Hello from [Couchbase](http://couchbase.com).”, “html”: “<p>Hello from <a href=\“http: …,“num_comments”: 2}
Blog Post Object
string id
string author
string type
string title
string format
string body
string html
int comments
Comment Object
string format
string body
int id
{“id”: “scalabl3_Hello_World::1”,“format”: “markdown”,“body”: “Awesome post!”}
{“id”: “scalabl3_Hello_World::2”,“format”: “markdown”,“body”: “Love It!”}
Multiple Documents
Differences of Note
Mental Adjustments
• In SQL we tend to want to avoid hitting the database as much as possible
• Even with caching and indexing tricks, and massive improvements over the years, SQL still gets bogged down by complex joins and huge indexes, so we avoid making database calls
• In Couchbase, get’s and set’s are so fast they are trivial, not bottlenecks, this is hard for many people to accept at first; Multiple get statements are commonplace, don’t avoid it!
Consistently Has High Performance
36
Couchbase Ensures…• Responses are always fast• Even if a response can’t be serviced
right away
Shift your Expectations:
• No longer a need to avoid round-trips
• No longer a need to avoid database requests for application speed
A Managed Cache
Scales Easily, Transparently
• As an application developer, you think about interacting with the cluster No longer a need to think about shards No longer have to deploy and manage internal or external proxies to
understand where the data is placed
Add Nodes to Cluster
• Two servers addedOne-click operation
• Docs automatically rebalanced across cluster
Even distribution of docsMinimum doc movement
• Cluster map updated
• App database calls now distributed over larger number of servers
REPLICA
ACTIVE
Doc 5
Doc 2
Doc
Doc
Doc 4
Doc 1
Doc
Doc
SERVER 1
REPLICA
ACTIVE
Doc 4
Doc 7
Doc
Doc
Doc 6
Doc 3
Doc
Doc
SERVER 2
REPLICA
ACTIVE
Doc 1
Doc 2
Doc
Doc
Doc 7
Doc 9
Doc
Doc
SERVER 3 SERVER 4 SERVER 5
REPLICA
ACTIVE
REPLICA
ACTIVE
Doc
Doc 8 Doc
Doc 9 Doc
Doc 2 Doc
Doc 8 Doc
Doc 5 Doc
Doc 6
READ/WRITE/UPDATE READ/WRITE/UPDATE
APP SERVER 1
COUCHBASE Client Library
CLUSTER MAP
COUCHBASE Client Library
CLUSTER MAP
APP SERVER 2
COUCHBASE SERVER CLUSTER
User Configured Replica Count = 1
Document Database, Simple ApproachWork with aggregates, usually JSON. Use simple operations to work with your data. Couchbase understands and is designed for distributed systems.
Deployment of Applications Allow for Scale, PerformanceNew approaches to storing and retrieving data in Couchbase give developers the ability to easily scale up and make their applications perform.
Couchbase in Summary
Easy to Get Started, Straightforward to UseDownload, install server. Binaries are available. Provisioning is fast through the browser and flexible with a REST interface. Install client using tools such as Java’s maven, Ruby’s gem, etc. Interact with simple verbs.
Q&A
Thanks!@rbin on twitter!