why nosql? why riak? - goto conferencegotocon.com/dl/jaoo-brisbane-2010/slides/justinsheehy... ·...
Post on 04-Oct-2018
232 Views
Preview:
TRANSCRIPT
Why NoSQL?
Justin Sheehyjustin@basho.com
Why Riak?
1
What's all of this NoSQL nonsense?
2
MongoDB
CouchDB
Cassandra
Voldemort
Neo4j
MembaseRedis
HBaseRiak
(and the list goes on...)
What went wrong with SQL databases?
3
Nothing! They are great tools.
What went wrong with SQL databases?
4
Nothing! They are great tools.
NoSQL came before SQL.
5
IBM IMS MUMPS
Honeywell IDS
Cincom TOTAL
dbm
VAX DBMS
(and the list goes on...)
PICK
But that's not really NoSQL, is it?
6
IBM IMS MUMPS
Honeywell IDS
Cincom TOTAL
dbm
VAX DBMS
(and the list goes on...)
PICK
The start of today's NoSQL
7
Amazon: Dynamo (2007)
Google: Bigtable (2006)
Technology is not the important part!
8
Amazon sells books.(esp. 3+ years ago)
Why not just use Oracle/MySQL/etc?
Technology is not the important part!most
The start of today's NoSQL
9
Amazon sells books.(esp. 3+ years ago)
Why not just use Oracle/MySQL/etc?
Business needs (re)created demand foralternative database technologies.
The start of today's NoSQL
What's all of this NoSQL nonsense?
10
MongoDB
CouchDB
Cassandra
Voldemort
Neo4j
MembaseRedis
HBaseRiak
Do we really need all of these?
It's still about business needs.
11
Redis
Riak
use cas
es drive
decisio
ns!
It's still about business needs.
12
Redis
Riak
what solves
your re
al problem
choose
13
what solves your real problemhow do you know ?
two places to start:
➡ innate/exposed data model➡ distribution/operational model
14
data model differences
requirements usually from app developers
Native Data Structures
Semi-Structured Documents
Bigtable Column FamilyTabular/Relational
HierarchicalGraph Traversal Key/Value
Indexed Query
15
data model differencesnot much useful ordering
Bigtable Column Family
Graph Traversal
which one is
more powerfu
l?
16
ColFamGraphK/V etc etc
data model differences
Cassandra
Voldemort
Neo4j
17
distribution model differencesrequirements usually from biz or operations
Locally Embedded
Single Server
Distributed System
Server Replication
18
distribution model differencesrequirements usually from biz or operations
Locally Embedded
Single Server
Distributed System
Server Replication
19
starting to decide
Locally Embedded
Single Server
Distributed System
Server Replication
ColFamGraphK/V etc etc
20
starting to decide
Locally Embedded
Single Server
Distributed System
Server Replication
ColFamGraphK/V etc etc
We need a K/Vor ColFam store.
21
starting to decide
Locally Embedded
Single Server
Distributed System
Server Replication
ColFamGraphK/V etc etc
We need a K/Vor ColFam store.
We need to recoverquickly from server failure.
22
Distributed System
Server Replication
ColFamK/V
now we're getting somewhere!
23
Distributed System
Server Replication
ColFamK/V
we can make a shorter list...Riak, Voldemort, CouchDB, Cassandra, HBase...
24
Distributed System
Server Replication
ColFamK/V
we can make a shorter list...Riak, Voldemort, CouchDB, Cassandra, HBase...
then we can narrow it more...protocols, licensing, benchmarking, simplicity...
25
Distributed System
Server Replication
ColFamK/V
we can make a shorter list...Riak, Voldemort, CouchDB, Cassandra, HBase...
then we can narrow it more...protocols, licensing, benchmarking, simplicity...
and make a choice!
26
Distributed System
Server Replication
ColFamK/V
we can make a shorter list...Riak, Voldemort, CouchDB, Cassandra, HBase...
then we can narrow it more...protocols, licensing, benchmarking, simplicity...
and make a choice!
How about MySQL?
I believe we could, Bob.
27
How about MySQL?
I believe we could, Bob.
wait, WHAT?
28
How about MySQL?
I believe we could, Bob.
wait, WHAT?
Great tools are still great tools.
29
Great tools are still great tools.
Understand your needsbefore you choose.
30
So what's so special about
?
31
Locally Embedded
Single Server
Distributed System
Server Replication
ColFamGraphK/V etc etc
Riak KV
32
Locally Embedded
Single Server
Distributed System
Server Replication
ColFamGraphK/V etc etc
Riak KVRiak Search
33
Locally Embedded
Single Server
Distributed System
Server Replication
ColFamGraphK/V etc etc
Riak KVRiak Search
Riak Core
34
Locally Embedded
Single Server
Distributed System
Server Replication
ColFamGraphK/V etc ?
improvements flow upward
Riak Core can flow sideways
35
Locally Embedded
Single Server
Distributed System
Server Replication
ColFamGraphK/V etc ?
Will the market need this?
36
Locally Embedded
Single Server
Distributed System
Server Replication
ColFamGraphK/V etc ?
Will the market need this?
I know how to get there.
37
client application
protobuf http
riak_client
dynamo model FSMs
riak core
vnode master
k/v vnode
storage engine
The Riak key/value stack:
38
client application
protobuf http
riak_client
dynamo model FSMs
vnode master
k/v vnode
storage engine
client application
protobuf http
riak_client
dynamo model FSMs
the cluster nodes are united by riak core via gossip, consistent hashing, etc
vnode master
k/v vnode
storage engine
client application
protobuf http
riak_client
dynamo model FSMs
vnode master
k/v vnode
storage engine
39
client application
protobuf http
riak_client
dynamo model FSMs
riak core
vnode master
k/v vnode
storage engine
just a local k/v store:
40
client application
protobuf http
riak_client
dynamo model FSMs
riak core
vnode master
k/v vnode
storage engine
just an abstract k/v store:
41
client application
protobuf http
riak_client
dynamo model FSMs
riak core
vnode master
k/v vnode
storage engine
a distributed system at heart:
42
client application
protobuf http
riak_client
dynamo model FSMs
riak core
vnode master
k/v vnode
storage engine
a distributed system at heart:
virtual nodes
gossip
failure detection
vector clocks
sloppy quorums
remote dispatchdynamic membership
43
client application
protobuf http
riak_client
dynamo model FSMs
riak core
vnode master
k/v vnode
storage engine
carefully managed complexity...
44
client application
protobuf http
riak_client
dynamo model FSMs
riak core
vnode master
k/v vnode
storage engine
allows simplicity at the edges
45
client application
protobuf http
riak_client
dynamo model FSMs
riak core
vnode master
k/v vnode
k/v storage engine
let's make a dist. search system!
46
client application
protobuf http
riak_client
dynamo model FSMs
riak core
vnode master
k/v vnode
k/v storage engine
let's make a dist. search system!
adapted FSMs
47
client application
protobuf http
riak_client
dynamo model FSMs
riak core
vnode master
k/v vnode
k/v storage engine
let's make a dist. search system!
adapted FSMs
search storage
search vnode
48
protobuf http
riak_client
dynamo model FSMs
riak core
vnode master
k/v vnode
k/v storage engine
adapted FSMs
search storage
search vnode
search client
solr lucene
done!
49
protobuf http
riak_client
dynamo model FSMs
riak core
vnode master
k/v vnode
k/v storage engine
adapted FSMs
search storage
search vnode
search client
solr lucene
the sum
is greater than
the parts
50
I don't know what's next
51
protobuf http
riak_client
dynamo model FSMs
riak core
vnode master
k/v vnode
k/v storage engine
adapted FSMs
search storage
search vnode
search client
solr lucene
I don't know what's next,
but I know how to build it.
more FSMs
X client
another interface
X storage
X vnode
I don't know what's next
52
riak core
I don't know what's next,
but I know how to build it.
I don't know what's next
scalabilityfault-tolerance
ease of operations
interoperability
pluggable protocolsflexible storage
predictable availability
I know how to build it
53
scalabilityfault-tolerance
ease of operations
interoperability
pluggable protocolsflexible storage
predictable availability
I know how to build itI know how to build it,
but you don't have to.
54
our earlier evaluation criteria:
➡ innate/exposed data model➡ distribution/operational model
55
➡ innate/exposed data model➡ distribution/operational model
fully distributed system
56
➡ innate/exposed data model➡ distribution/operational model
fully distributed system
flexible and growing
57
Justin Sheehyjustin@basho.com
http://www.basho.com/
top related