persistence smoothie

Persistence SmoothieBlending SQL and NoSQL

Michael BleighIntridea, Inc.

photo by Nikki L. via Flickr

Thursday, March 11, 2010

present.ly

tweetstream hashieacts-as-taggable-on

subdomain-fu seed-fumustache_json

github.com/intridea

@mbleigh

You’ve (probably)heard a lot about

NoSQL is a new way to think about

persistence

AtomicityConsistency

IsolationDurability

DenormalizationEventual Consistency

Schema-FreeHorizontal Scale

NoSQL tries to scale (more) simply

NoSQL is going mainstream

New York TimesBusiness InsiderBBC ShopWikiGitHub Meebo

Disqus SourceForgeSony Digg

...but not THAT mainstream.

A word of caution...

NoSQL can divide by zero

NoSQL doesn’t

sleep, it waits

NoSQL counted to infinity, twiceThursday, March 11, 2010

NoSQL is a (growing) collection of tools, not

a new way of life

Key-Value StoresDocument Databases

Column StoresGraph Databases

Key-Value Stores

• Key-value store + datatypes

• Lists, (Scored) Sets, Hashes

• Cache-like functions (expiration)

• (Mostly) In-Memory

• Combo key-value store and document database

• HTTP REST interface

• “Link walking”

• Map-Reduce

Map/Reduce

• Massively parallel way to process large datasets

• First you scour data and “map” a new set of data

• Then you “reduce” the data down to a salient result

map = function() { this.tags.forEach(function(tag) { emit(tag, {count: 1}); });}

reduce = function(key, values) { var total = 0; for (var i = 0; i < values.length; i++) { total += values[i].count; return {count: total};}

Tokyo CabinetDynomite

MemcachedDBVoldemort

Document Databases

MongoDB

• Document store that speaks BSON (Binary JSON)

• Indexing, simple query syntax

• GridFS

• Deliberate MapReduce

CouchDB

• JSON Document Store

• HTTP REST Interface

• Incremental MapReduce

• Intelligent Replication

Column-Oriented Datastores

Cassandra

• Built by Facebook, used by Twitter

• Pure horizontal scalability

• Schemaless

Graph Databases

When should I use this stuff?

Complex, slow joins for “activity stream”

Denormalize, use Key-Value Store

Variable schema,vertical interaction

Document Databaseor Column Store

Modeling multi-step relationships

Graph Database

NoSQL solves real scalability and data

design issues

Ben Scofieldbit.ly/state-of-nosql

Ready to go?

Just one problem...

Your data is already in a SQL database

We CAN all just get along.

Three Ways

The Hard(ish) Way

The Easy Way

A Better Way?

The Hard Way:Do it by hand.

class Post include MongoMapper::Document key :title, String key :body, String key :tags, Array key :user_id, Integer def user User.find_by_id(self.user_id) end def user=(some_user) self.user_id = some_user.id endend

class User < ActiveRecord::Base def posts(options = {}) Post.all({:conditions => {:user_id => self.id}}.merge(options)) endend

Pros & Cons

• Simple, maps to your domain

• Works for small, simple ORM intersections

• MUCH simpler in Rails 3

• Complex relationships are a mess

• Makes your models fat

• As DRY as the ocean

The Easy Way:DataMapper

DataMapper

• Generic, relational ORM

• Speaks pretty much everything you’ve ever heard of

• Implements Identity Map

• Module-based inclusion

DataMapper.setup(:default, "mysql://localhost")DataMapper.setup(:mongodb, "mongo://localhost/posts")

class Post include DataMapper::Resource def self.default_repository_name; :mongodb; end property :title, String property :body, String property :tags, Array belongs_to :userend

class User include DataMapper::Resource property :email, String property :name, String has n, :postsend

Pros & Cons

• The ultimate Polyglot ORM

• Simple relationships between persistence engines are easy

• Jack of all trades, master of none

• Perpetuates (sometimes) false assumptions

• Legacy stuff is in ActiveRecord anyway

Is there a better way?

Maybe.

Gloo: Cross-ORM Relationship Mapper

github.com/intridea/gloo

0.0.0.prealpha.1

Can’t we just sit down and talk to

each other?

class Post include MongoMapper::Resource key :title, String key :body, String key :tags, Array gloo :active_record do belongs_to :user endend

class User < ActiveRecord::Base gloo :mongo_mapper do many :posts endend

Goals/Status

• Be able to define relationships on the terms of any ORM from any class, ORM or not

• Right Now: Partially working ActiveRecord relationships

• Doing it wrong? Maybe

Code Time: Schema4Less

Social Storefront

• Dummy application of a store that lets others “follow” your purchases (a less creepy Blippy?)

• Four requirements:

• users

• purchasing

• listings

• social graph

• I already have an authentication system

• I’m happy with it

• It’s Devise and ActiveRecord

• Stick with SQL

Purchasing

• Users need to be able to purchase items from my storefront

• I can’t lose their transactions

• I need full ACID

• I’ll use MySQL

Social Graph

• I want activity streams and one and two way relationships

• I need speed

• I don’t need consistency

• I’ll use Redis

Product Listings

• I am selling both movies and books

• They have very different properties

• Products are relatively non-relational

• I’ll use MongoDB

Demo and Walkthrough

Wrapping Up

These systems can (and should) live and

work together

Most important step is to actually think about data design

When you have a whole bag of tools, things stop looking

like nails

Questions?

persistence smoothie

Technology

persistence smoothie

berry smoothie - scene7

5daydetox smoothie

multi smoothie recipe

innovation smoothie preview

superfood smoothie sampler

berry delicious smoothie

smoothie - loopnet · smoothie king a consumer favorite...

smoothie delivery

summertime smoothie recipes

smoothie roo - salads

1. berry burst smoothie 4. green reviver smoothie · 1....

ledo smoothie detox

data smoothie

innocent smoothie

case study juice/smoothie bar -...

operation smoothie

fruity smoothie

green smoothie powerpoint

yoplait smoothie recipes