alternative database technology in the cloud

Community StackerBret Piatt

LinuxCon 2010Alternative Database Technologies for the Cloud

&

OPEN SOURCE ISDRIVING THE CLOUD

LINUX

XEN

KVM

APACHE

MEMCACHED

VARNISH

WORDPRESS

DRUPAL

MAGENTO

THE HARDEST PARTIS STILL THE DATABASE

TIMES HAVE CHANGED

OTHER THINGSHAVE ALSO CHANGED

THE WEB

WEB APPS ARE BUILT DIFFERENTLY

SERVER-SIDE PROCESSING

BURSTY LOADS, UNPREDICTABILITY

DISTRIBUTED USERS

SIMPLIFIED PROBLEM SET

SIMPLIFIED OPERATING ENVIRONMENT

WEB SCALE REQUIREMENTS

TERABYTES OF DATA, INCLUDING STRUCTURED DATA

MILLIONS TO BILLIONS OF WRITES EACH DAY

GLOBAL REPLICATION AND REDUNDANCY–EVENTUAL CONSISTANCY

COMMODITY HARDWARE–SCALE OUT NOT UP

CACHE EVERYTHING YOU READ

MORE CORES, MORE MEMORY, MORE MACHINES

RIAK

JSON AND REST–WEB GENERATION INTERFACE

MAP/REDUCE TERM MATCHING QUERIES

SCALE HORIZONTALLY WITH MULTIPLE MASTERS

EVENTUAL CONSISTENCY WITH VECTOR CLOCKS

WRITTEN IN ERLANG–CONCURRENT FUNCTIONAL PROGRAMMING

MORE CORES, MORE MEMORY, MORE MACHINES

HBASE

REST-FUL WEB SERVICE GATEWAY

DISTRIBUTED COLUMN STORE MODELED AFTER BIGTABLE

SCALE HORIZONTALLY WITH MULTIPLE MASTERS

ROLLING RESTART FOR CONFIG CHANGES AND MINOR UPGRADES

HIGH PERFORMANCE THRIFT GATEWAY

CASCADING, HIVE, AND PIG SOURCE AND SINK MODULES

MORE CORES, MORE MEMORY, MORE MACHINES

REDIS

API LIBRARIES AVAILABLE IN MANY LANGUAGES

MORE THAN JUST A PLAIN KEY-VALUE STORE

THE DATABASE IS IN MEMORY–RAM IS CHEAP

ASYNCHRONOUS FLUSH TO DISK

REPLICATION IS MASTER->SLAVE

MORE CORES, MORE MEMORY, MORE MACHINES

MONGODB

BSON INTERFACE DOCUMENT DATABASE

FULL INDEX SUPPORT

REPLICATION & HIGH AVAILABLITY ACROSS WANS

AUTO-SHARDING FOR HORIZONTAL SCALE

GRIDFS TO STORE LARGE FILES

MAP/REDUCE BATCH MANIPULATION

MORE CORES, MORE MEMORY, MORE MACHINES

Sources: http://www.businessinsider.com/how-we-use-mongodb-2009-11 -- andhttp://blog.boxedice.com/2010/02/28/notes-from-a-production-mongodb-deployment/

CASSANDRA

FULLY DISTRIBUTED

TRUE HORIZONTAL SCALEABILITY

REPLICATION–INCLUDING WIDE-AREA REPLICATION

PREDICTABLE, BLAZING WRITE PERFORMANCE

CONTROLLABLE READ/WRITE CONSISTENCY

MORE CORES, MORE MEMORY, MORE MACHINES

DRIZZLE

RELEASE EARLY, RELEASE OFTEN–NO 2-YEAR CYCLES

UTF8-THE WEB IS “WORLDWIDE” AFTER ALL

STRONG SQL INTERFACES MEAN STABILITY & INNOVATION

MODULAR ARCHITECTURE ALLOWS EXTENSION…

…OR PERFORMANCE BY REMOVING WHAT YOU DON’T NEED

MORE CORES, MORE MEMORY, MORE MACHINES

WHY OPEN MATTERS

LOCK-IN IS A REAL CONCERN IN THE CLOUD

OPEN SOURCE MAKES IT ACCESSIBLE TO YOU

TECHNOLOGY IS NOT A SUSTAINABLE, LONG-TERM ADVANTAGE

GAIN THE KNOWLEDGE OF A COMMUNITY

SHARE YOUR KNOWLEDGE WITH EVERYONE

TURNS COMPETITION INTO COOPERATION

THERE IS NOSILVER BULLET

MASSIVE SCALESPECIFIC PROBLEMS

REQUIRE OPTIMIZEDSPECIFIC SOLUTIONS

HOW SHOULDI CHOOSE?

Source: http://blog.mozilla.com/data/2010/05/18/riak-and-cassandra-and-hbase-oh-my/

SCALABILITY

ELASTICITY

RELIABILITY

STORAGE

DATA ANALYSIS

COST

SECURITY

EXTENSIBILITY

DR / MIGRATION

ANALYZE THEN DECIDE

QUESTION & ANSWERS

Bret Piatt

Twitter: @bpiatt

Email: bret@openstack.org

THANK YOU