non-relational databases & key/value stores
Post on 11-May-2015
13.856 Views
Preview:
DESCRIPTION
TRANSCRIPT
NON-RELATIONAL DATABASES
Saturday, October 31, 2009
@JPERRAS - JOEL PERRASCanadian Geek
Blog: http://nerderati.com
GitHub: http://github.com/jperras
CakePHP Core since Early 2009, PHP dev. since 2001
McGill University, Montréal, Canada - Physics, Mathematics & Computer Science
Employer: Plank Design (http://plankdesign.com) (Twitter: @plankdesign)
Saturday, October 31, 2009
RELATIONAL DATABASES
Many different vendors: MySQL, PostgreSQL, SQLite, Oracle, ...
Same basic implementation:
B(+)-Trees for pages
B(+)-Trees or hash tables for secondary indexes
Possibly R-Trees for spatial indexes
Saturday, October 31, 2009
WHAT THEY’RE GOOD AT
Saturday, October 31, 2009
Schemas (relational models)
Familiar BCNF structure
Strong consistency
Transactions
Very “mature” & well tested (mostly)
Easy adoption/integration
Saturday, October 31, 2009
FriendFeed
Wikipedia
Google AdWords
RDBMS’ES ARE NOT GOING ANYWHERE
Saturday, October 31, 2009
Most small to medium size applications willnever need to go beyond a single database server.
Saturday, October 31, 2009
Always try and follow the Golden Web ApplicationDevelopment Rule:
Saturday, October 31, 2009
DON’T TRY TO SOLVE A PROBLEM YOU DON’T
HAVE
Saturday, October 31, 2009
The web has created new problem domains in data storage and querying.
Saturday, October 31, 2009
Often use variable schemas
Optional fields: contact lists, addresses, favourite movies/books, etc.
NULL-itis: null values should not be permitted in BCNF, but are everywhere in web applications.
MODERN WEB APPS
Saturday, October 31, 2009
‘Social’ apps => high write/read ratios
Complex Many-to-Many relationships
Joins become a problem in federated architectures
Eventual consistency is usually acceptable
Downtime unacceptable
MODERN WEB APPS
Saturday, October 31, 2009
OTHER CONCERNS
Saturday, October 31, 2009
RULES OF APP AGINGhttp://push.cx/2009/rules-of-database-app-aging
1. All fields become optional2. All relationships become many-to-many3. Chatter (comments explaining hacks) grows with time.
Saturday, October 31, 2009
SOME GOOD PROBLEMS TO HAVE
Even if they are “Hard” ones to solve.
Saturday, October 31, 2009
Load Balancing(you can only live with one machine for so long)
Saturday, October 31, 2009
High Availability(because disks fail, and replication fails)
Saturday, October 31, 2009
What’s a web application developer to do?
Saturday, October 31, 2009
Alternative Data Storage Solutions
Saturday, October 31, 2009
Not a silver bullet.
These can solve some problems, but cause others and have their own limitations.
It’s up to you to weigh the cost/benefit of your chosen solution.
Saturday, October 31, 2009
THE LANDSCAPE
Key/Value Stores/Distributed Hash Tables (DHT)
Document-oriented databases
Column-oriented databases
Saturday, October 31, 2009
KEY/VALUE STORES
Voldemort
Scalaris
Tokyo Cabinet
Redis
MemcacheDB
Saturday, October 31, 2009
DOCUMENT ORIENTED DATA STORES
CouchDB <- (my favourite!)
MongoDB
SimpleDB (Amazon)
Saturday, October 31, 2009
COLUMN-ORIENTED STORES
BigTable (Google)
HBase (Hadoop Database)
Hypertable (BigTable Open Source clone)
Cassandra (Facebook)
Saturday, October 31, 2009
How do we use these technologiesalongside CakePHP ?
Saturday, October 31, 2009
This year’s magical word:
DataSources
Saturday, October 31, 2009
CASE STUDY - COUCHDB
http://github.com/jperras/divan(I will make zip/tar available when more stable - stay tuned)
Saturday, October 31, 2009
CASE STUDY - TOKYO CABINET/TYRANT
http://github.com/jperras/tyrannical(I will make zip/tar available when more stable - stay tuned)
Saturday, October 31, 2009
Non-relational stores are not relational.
Saturday, October 31, 2009
So don’t try to force the interface tobe relational.
Saturday, October 31, 2009
DESIGNING A NON-RELATIONAL DATASOURCE
Favour simplicity over transparency
Don’t try to implement everything that the MySQL driver implements
Use the strengths of the alternative store
Saturday, October 31, 2009
Example Use Cases
Saturday, October 31, 2009
KEY/VALUE STORES
Most have atomic increment/decrement operations
Great for API rate limiters (e.g. 300 API reqs/hour/account)
Counts & sums of normalized data
Most popular items, votes, ratings, some statistics
And more.
Saturday, October 31, 2009
DOCUMENT STORES
Filesystem objects (pdfs, images, excel sheets etc.) - stored as document attachments (size limited).
Allows you to reduce reliance on shared filesystems (NFS)
Address book
Volatile schema situations
CouchDB has a very interesting feature set
Saturday, October 31, 2009
There are many, many use cases.
Saturday, October 31, 2009
Thanks to the DataSource adapter implementation in CakePHP, creating a model-based interface is simple.
Saturday, October 31, 2009
Thank you!
@jperrashttp://nerderati.com
http://github.com/jperrasSaturday, October 31, 2009
CODE
Divan - CouchDB datasource
Yantra - State Machine component for application control flow
CakPHP TextMate Bundle
CakeMate - TextMate/Vim Plugin
Tyrannical - Tokyo Tyrant datasource
Originally by Martin Samson (pyrolian@gmail.com)
Working to improve code - commits coming soon.
Currently working on a framework-agnostic, distributed, plugin/library server.
Saturday, October 31, 2009
top related