bigtable: a distributed storage system for structured data...

Databases: Generalities. . . . . . . ... . . .. . . .. . . . . .

Bigtable. . .. . ... .. .

Other NoSQL Thoughts. . .Conclusion

Bigtable: A Distributed Storage System forStructured Data

Course: CS655

Rabiet Louis

Colorado State University

Thursday 17 October 2013

1 / 51

Bigtable. . .. . ... .. .

1 Databases: GeneralitiesRelational databaseOther SQL database modelsWhy is this not enough ?

2 BigtableDescriptionGoogle FS: underlying FSChubby: Failure resiliencePaxosSome optimizations

3 Other NoSQLAn old problemExtensible Record StoresDocument storesKey-value storesRAM databases

4 Thoughts

5 Conclusion

2 / 51

Bigtable. . .. . ... .. .

4 Thoughts

5 Conclusion

3 / 51

Bigtable. . .. . ... .. .

Relational database

Reminders on relational database model

SQL family

First order predicate

Based on two values: true and false.

Use tuples and relation

Tuples: (Louis, CS Student, French) - (Louis, GRA)Relation = Set of attributes and sets of tuples.Attr. = (Id, Major, Nationality).

A query = a formula∀students,Nationality = French && Position = GRA

4 / 51

Bigtable. . .. . ... .. .

Relational database

SQL family

4 / 51

Bigtable. . .. . ... .. .

Relational database

SQL family

4 / 51

Bigtable. . .. . ... .. .

Relational database

SQL family

4 / 51

Bigtable. . .. . ... .. .

Relational database

SQL family

Tuples: (Louis, CS Student, French) - (Louis, GRA)

Relation = Set of attributes and sets of tuples.Attr. = (Id, Major, Nationality).

4 / 51

Bigtable. . .. . ... .. .

Relational database

SQL family

Tuples: (Louis, CS Student, French) - (Louis, GRA)Relation = Set of attributes and sets of tuples.

Attr. = (Id, Major, Nationality).

4 / 51

Bigtable. . .. . ... .. .

Relational database

SQL family

4 / 51

Bigtable. . .. . ... .. .

Relational database

SQL family

A query = a formula

∀students,Nationality = French && Position = GRA

4 / 51

Bigtable. . .. . ... .. .

Relational database

SQL family

4 / 51

Bigtable. . .. . ... .. .

Other SQL database models

Hierarchical Databases

7 Tree structure.

3 Efficient implementations.

Object Databases

3 Close to programming langages.

7 Overkill if the data has simple relations.

5 / 51

Bigtable. . .. . ... .. .

7 Tree structure.

Object Databases

5 / 51

Bigtable. . .. . ... .. .

7 Tree structure.

Object Databases

5 / 51

Bigtable. . .. . ... .. .

7 Tree structure.

Object Databases

5 / 51

Bigtable. . .. . ... .. .

7 Tree structure.

Object Databases

5 / 51

Bigtable. . .. . ... .. .

7 Tree structure.

Object Databases

5 / 51

Bigtable. . .. . ... .. .

Why is this not enough ?

.How to express query on a text ?

You have to reconstruct some kind of relation more or lessmanually even if some solutions exists [JDG08].

.How do you do if a single server is not enough ?

Partioning → write expensivesmall scheme showing why it requires two-phase commit

Replications → write expensive

6 / 51

Bigtable. . .. . ... .. .

6 / 51

Bigtable. . .. . ... .. .

6 / 51

Bigtable. . .. . ... .. .

Partioning → write expensive

small scheme showing why it requires two-phase commit

6 / 51

Bigtable. . .. . ... .. .

6 / 51

Bigtable. . .. . ... .. .

6 / 51

Bigtable. . .. . ... .. .

4 Thoughts

5 Conclusion

7 / 51

Bigtable. . .. . ... .. .

Description

BigData

Usage of the architecture

Locations

Data Personalized: settings, search

Some ideas

Goal is to let users handle data storage structure

Locality is important

A data = an uninterpreted string

Goes nicely with Map Reduce.

8 / 51

Bigtable. . .. . ... .. .

Description

BigData

Locations

Some ideas

8 / 51

Bigtable. . .. . ... .. .

Description

BigData

Locations

Some ideas

8 / 51

Bigtable. . .. . ... .. .

Description

BigData

Locations

Some ideas

8 / 51

Bigtable. . .. . ... .. .

Description

BigData

Locations

Some ideas

8 / 51

Bigtable. . .. . ... .. .

Description

BigData

Locations

Some ideas

8 / 51

Bigtable. . .. . ... .. .

Description

BigData

Locations

Some ideas

8 / 51

Bigtable. . .. . ... .. .

Description

BigData

Locations

Some ideas

8 / 51

Bigtable. . .. . ... .. .

Description

Challenges

Unstructured Data.

Scale.

Continuous Update (crawling).

Read should be allowed at any time.

Very high rates of accesses → load-balancing.

Failure resilient (adding and removing servers at any time).

9 / 51

Bigtable. . .. . ... .. .

Description

Challenges

Unstructured Data.

Scale.

9 / 51

Bigtable. . .. . ... .. .

Description

Challenges

Unstructured Data.

Scale.

9 / 51

Bigtable. . .. . ... .. .

Description

Challenges

Unstructured Data.

Scale.

9 / 51

Bigtable. . .. . ... .. .

Description

Challenges

Unstructured Data.

Scale.

9 / 51

Bigtable. . .. . ... .. .

Description

Challenges

Unstructured Data.

Scale.

9 / 51

Bigtable. . .. . ... .. .

Description

Data Model

10 / 51

Bigtable. . .. . ... .. .

Description

Column-oriented database

Properties

Row writes are atomic.

Row ranges (tablet) are grouped together dynamically

Sorted using lexicographic order

close number → physically close.

Column family is other group.

Several versions: usage of timestamp.

last n versions.fresh enough (age limit).

11 / 51

Bigtable. . .. . ... .. .

Description

Properties

11 / 51

Bigtable. . .. . ... .. .

Description

Properties

11 / 51

Bigtable. . .. . ... .. .

Description

Properties

11 / 51

Bigtable. . .. . ... .. .

Description

Properties

11 / 51

Bigtable. . .. . ... .. .

Description

Properties

11 / 51

Bigtable. . .. . ... .. .

Description

Properties

11 / 51

Bigtable. . .. . ... .. .

Description

Properties

last n versions.

fresh enough (age limit).

11 / 51

Bigtable. . .. . ... .. .

Description

Properties

11 / 51

Bigtable. . .. . ... .. .

Description

Architecture

Chubby

Master

GFS Master.

Scheduler

. ..TS1

. Client

12 / 51

Bigtable. . .. . ... .. .

Description

Architecture

Chubby

Master

GFS Master.

Scheduler

. ..TS1

. Client

12 / 51

Bigtable. . .. . ... .. .

Description

Architecture

Chubby

Master

GFS Master.

Scheduler

. ..TS1

. Client

12 / 51

Bigtable. . .. . ... .. .

Description

Architecture

Chubby

Master

GFS Master.

Scheduler

. ..TS1

. Client

12 / 51

Bigtable. . .. . ... .. .

Description

Architecture

Chubby

Master

GFS Master.

Scheduler

. ..TS1

. Client

12 / 51

Bigtable. . .. . ... .. .

Description

Architecture

Chubby

Master

GFS Master.

Scheduler

. ..TS1

. Client

12 / 51

Bigtable. . .. . ... .. .

Description

Architecture

Chubby

Master

GFS Master.

Scheduler

. ..TS1

. Client

12 / 51

Bigtable. . .. . ... .. .

Description

Architecture

Chubby

Master

GFS Master.

Scheduler

. ..TS1

. Client

12 / 51

Bigtable. . .. . ... .. .

Description

Architecture

Chubby

Master

GFS Master.

Scheduler

. ..TS1

. Client

12 / 51

Bigtable. . .. . ... .. .

Description

Architecture

Chubby

Master

GFS Master.

Scheduler

. ..TS1

. Client

12 / 51

Bigtable. . .. . ... .. .

Description

Internal Storage

Use a key:value storage format (SStable: Sorted String Table).

SStable are immutables.

Every time a write is done: a new table is created.3 Concurrency is easier: No synchronisation, No need to control

the concurrent accesses.3 Can help for serialization, for snapshot.7 Increase the number of SSTables.7 Persistence can be achieved by differential instead of using a

full copy.

13 / 51

Bigtable. . .. . ... .. .

Description

Internal Storage

full copy.

13 / 51

Bigtable. . .. . ... .. .

Description

Internal Storage

full copy.

13 / 51

Bigtable. . .. . ... .. .

Description

Internal Storage

Every time a write is done: a new table is created.

3 Concurrency is easier: No synchronisation, No need to controlthe concurrent accesses.

3 Can help for serialization, for snapshot.7 Increase the number of SSTables.7 Persistence can be achieved by differential instead of using a

full copy.

13 / 51

Bigtable. . .. . ... .. .

Description

Internal Storage

the concurrent accesses.

3 Can help for serialization, for snapshot.7 Increase the number of SSTables.7 Persistence can be achieved by differential instead of using a

full copy.

13 / 51

Bigtable. . .. . ... .. .

Description

Internal Storage

the concurrent accesses.3 Can help for serialization, for snapshot.

7 Increase the number of SSTables.7 Persistence can be achieved by differential instead of using a

full copy.

13 / 51

Bigtable. . .. . ... .. .

Description

Internal Storage

the concurrent accesses.3 Can help for serialization, for snapshot.7 Increase the number of SSTables.

7 Persistence can be achieved by differential instead of using afull copy.

13 / 51

Bigtable. . .. . ... .. .

Description

Internal Storage

full copy.13 / 51

Bigtable. . .. . ... .. .

Description

Tablets Location

B-Tree+ to store tablet location.

14 / 51

Bigtable. . .. . ... .. .

Description

Tablets Assignment and ServingNo duplication of tablets.

Master keeps track of assignements to reallocate tablets ifneeded.A TS acquires a lock in Chubby per tablet.Authorization is given by the writers list in Chubby.

15 / 51

Bigtable. . .. . ... .. .

Description

Tablets Assignment and ServingNo duplication of tablets.Master keeps track of assignements to reallocate tablets ifneeded.

A TS acquires a lock in Chubby per tablet.Authorization is given by the writers list in Chubby.

15 / 51

Bigtable. . .. . ... .. .

Description

Tablets Assignment and ServingNo duplication of tablets.Master keeps track of assignements to reallocate tablets ifneeded.A TS acquires a lock in Chubby per tablet.

Authorization is given by the writers list in Chubby.

15 / 51

Bigtable. . .. . ... .. .

Description

Tablets Assignment and ServingNo duplication of tablets.Master keeps track of assignements to reallocate tablets ifneeded.A TS acquires a lock in Chubby per tablet.Authorization is given by the writers list in Chubby.

15 / 51

Bigtable. . .. . ... .. .

Description

Tablets Assignment and ServingNo duplication of tablets.Master keeps track of assignements to reallocate tablets ifneeded.A TS acquires a lock in Chubby per tablet.Authorization is given by the writers list in Chubby.

15 / 51

Bigtable. . .. . ... .. .

Google FS: underlying FS

Reminder on GFS

Moderate number of Huge Files

A lot of failures

Appending ̸= random write

Concurrent write must be supported

16 / 51

Bigtable. . .. . ... .. .

Reminder on GFS

A lot of failures

16 / 51

Bigtable. . .. . ... .. .

Reminder on GFS

A lot of failures

16 / 51

Bigtable. . .. . ... .. .

Reminder on GFS

A lot of failures

16 / 51

Bigtable. . .. . ... .. .

Reminder on GFS

A lot of failures

16 / 51

Bigtable. . .. . ... .. .

Chubby: Failure resilience

Chubby in BigTable

Used for:

Electing a unique master.

Discover tablets.

Relocate tablets if needed.

Column family lists.

17 / 51

Bigtable. . .. . ... .. .

Chubby in BigTable

Used for:

Discover tablets.

17 / 51

Bigtable. . .. . ... .. .

Chubby in BigTable

Used for:

Discover tablets.

17 / 51

Bigtable. . .. . ... .. .

Chubby in BigTable

Used for:

Discover tablets.

17 / 51

Bigtable. . .. . ... .. .

Chubby in BigTable

Used for:

Discover tablets.

17 / 51

Bigtable. . .. . ... .. .

Chubby in BigTable

Used for:

Discover tablets.

17 / 51

Bigtable. . .. . ... .. .

Some details

5 replicas; one will be the Chubby master.

Give Locks to clients.

Based on the Paxos algorithm.

18 / 51

Bigtable. . .. . ... .. .

Some details

18 / 51

Bigtable. . .. . ... .. .

Some details

18 / 51

Bigtable. . .. . ... .. .

Some details

18 / 51

Bigtable. . .. . ... .. .

Some details

18 / 51

Bigtable. . .. . ... .. .

Some details

SessionA client will need to maintain a session with the Chubby service.

KeepAlive packets are sent by the client to the master.RPC are blocked by the master until the lease is closed to expire.

7 slow failure detection by the client.

3 few redondant packets

7 why clients should fail ?

19 / 51

Bigtable. . .. . ... .. .

Some details

SessionA client will need to maintain a session with the Chubby service.KeepAlive packets are sent by the client to the master.

RPC are blocked by the master until the lease is closed to expire.

19 / 51

Bigtable. . .. . ... .. .

Some details

SessionA client will need to maintain a session with the Chubby service.KeepAlive packets are sent by the client to the master.RPC are blocked by the master until the lease is closed to expire.

19 / 51

Bigtable. . .. . ... .. .

Some details

19 / 51

Bigtable. . .. . ... .. .

Some details

19 / 51

Bigtable. . .. . ... .. .

Some details

19 / 51

Bigtable. . .. . ... .. .

Some details

FailoversIf a master fails a new master need to take over the system.

7 Complicated.

20 / 51

Bigtable. . .. . ... .. .

Some details

7 Complicated.

20 / 51

Bigtable. . .. . ... .. .

Some details

7 Complicated.

20 / 51

Bigtable. . .. . ... .. .

Paxos algorithm

Originaly for parliament decree.

Properties wanted

At most one decree decided (safety)

At least one decree (liveness)

21 / 51

Bigtable. . .. . ... .. .

Paxos algorithm

Properties wanted

21 / 51

Bigtable. . .. . ... .. .

Paxos algorithm

Properties wanted

21 / 51

Bigtable. . .. . ... .. .

Paxos algorithm

Properties wanted

21 / 51

Bigtable. . .. . ... .. .

Properties on ballots

Make liveness possible and ensure consistency (safety property)

Each ballot has a unique number.

Every quorum has at least one priest in common.

For every ballot B, if a priest in the quorum has voted in aearlier ballot then the decree in B is equal to the lastest ballotwhere the priest voted.

22 / 51

Bigtable. . .. . ... .. .

22 / 51

Bigtable. . .. . ... .. .

22 / 51

Bigtable. . .. . ... .. .

22 / 51

Bigtable. . .. . ... .. .

Back to Paxos in Chubby

Consensus

Several coordinators will propose a value.It works because:

Each coordinator will generate an unique number with theirpropositions. (ex: s mod n ≡ id)Promise (ack from replicas that will ignore older coordinator)will include the latest value proposed.

Since we want to have a consensus on several values, Paxoswill be repeated: Need a catch-up mechanism for slowmachines.

23 / 51

Bigtable. . .. . ... .. .

Consensus

Several coordinators will propose a value.

It works because:

23 / 51

Bigtable. . .. . ... .. .

Consensus

23 / 51

Bigtable. . .. . ... .. .

Consensus

Each coordinator will generate an unique number with theirpropositions. (ex: s mod n ≡ id)

Promise (ack from replicas that will ignore older coordinator)will include the latest value proposed.

23 / 51

Bigtable. . .. . ... .. .

Consensus

23 / 51

Bigtable. . .. . ... .. .

Consensus

23 / 51

Bigtable. . .. . ... .. .

Consensus

23 / 51

Bigtable. . .. . ... .. .

Properties of Paxos/Chubby

3 Handle disk corruption by being a non voting member for awhole cycle.

7 Logs can become largeSnapshots can helpStill problematic

7 Implementation difficult

7 Read = a Paxos instance → expensivemaster lease can help.But problem of unstability if network is unstable.

24 / 51

Bigtable. . .. . ... .. .

3 Handle disk corruption

by being a non voting member for awhole cycle.

24 / 51

Bigtable. . .. . ... .. .

3 Handle disk corruption

by being a non voting member for awhole cycle.

24 / 51

Bigtable. . .. . ... .. .

24 / 51

Bigtable. . .. . ... .. .

7 Logs can become large

Snapshots can helpStill problematic

24 / 51

Bigtable. . .. . ... .. .

7 Logs can become large

Snapshots can helpStill problematic

24 / 51

Bigtable. . .. . ... .. .

7 Logs can become largeSnapshots can help

Still problematic

24 / 51

Bigtable. . .. . ... .. .

24 / 51

Bigtable. . .. . ... .. .

7 Read = a Paxos instance

→ expensivemaster lease can help.But problem of unstability if network is unstable.

24 / 51

Bigtable. . .. . ... .. .

24 / 51

Bigtable. . .. . ... .. .

24 / 51

Bigtable. . .. . ... .. .

7 Read = a Paxos instance → expensive

master lease can help.But problem of unstability if network is unstable.

24 / 51

Bigtable. . .. . ... .. .

7 Read = a Paxos instance → expensivemaster lease can help.

But problem of unstability if network is unstable.

24 / 51

Bigtable. . .. . ... .. .

24 / 51

Bigtable. . .. . ... .. .

24 / 51

Bigtable. . .. . ... .. .

Some optimizations

Compaction or Garbage Collection

Concept

The Memtable (logs recent changes) will become big.

The goal of compaction is to reduce the size of this table.

1 minor compaction: after reaching a threshold: freeze andcreate a new memtable. the old table is transform into aSSTable.

3 reduce memory usage.3 increase recovery speed.3 read and write can be done concurrently.7 number of SSTables is still increasing.

2 merging compaction: reduce the number of SSTables.

3 major compaction: merging compaction using all theSSTables.

25 / 51

Bigtable. . .. . ... .. .

Some optimizations

Concept

The Memtable (logs recent changes) will become big.The goal of compaction is to reduce the size of this table.

25 / 51

Bigtable. . .. . ... .. .

Some optimizations

Concept

25 / 51

Bigtable. . .. . ... .. .

Some optimizations

Concept

3 reduce memory usage.

3 increase recovery speed.3 read and write can be done concurrently.7 number of SSTables is still increasing.

25 / 51

Bigtable. . .. . ... .. .

Some optimizations

Concept

3 reduce memory usage.3 increase recovery speed.

3 read and write can be done concurrently.7 number of SSTables is still increasing.

25 / 51

Bigtable. . .. . ... .. .

Some optimizations

Concept

3 reduce memory usage.3 increase recovery speed.3 read and write can be done concurrently.

7 number of SSTables is still increasing.

25 / 51

Bigtable. . .. . ... .. .

Some optimizations

Concept

25 / 51

Bigtable. . .. . ... .. .

Some optimizations

Concept

25 / 51

Bigtable. . .. . ... .. .

Some optimizations

Concept

25 / 51

Bigtable. . .. . ... .. .

Some optimizations

Locality

Grouping column families into a locality group.

Each locality group will have an SSTable.

3 Will provide performance improvement if columns that are notaccessed together are on seperate SSTables.

7 Actually difficult to know how to do it dynamically.

26 / 51

Bigtable. . .. . ... .. .

Some optimizations

Locality

Grouping column families into a locality group.Each locality group will have an SSTable.

26 / 51

Bigtable. . .. . ... .. .

Some optimizations

Locality

26 / 51

Bigtable. . .. . ... .. .

Some optimizations

Locality

26 / 51

Bigtable. . .. . ... .. .

Some optimizations

Compression and caching

Clients can decide what compresion scheme to use forSSTables (portion of it, different per SSTable).

Typical compression:

First alogrithm will look for similarities over a large window.Second algorithm will look for common string in a smallwindow 16KB.

Tablet server will use cache to improve latency.

27 / 51

Bigtable. . .. . ... .. .

Some optimizations

Clients can decide what compresion scheme to use forSSTables (portion of it, different per SSTable).Typical compression:

27 / 51

Bigtable. . .. . ... .. .

Some optimizations

First alogrithm will look for similarities over a large window.

Second algorithm will look for common string in a smallwindow 16KB.

27 / 51

Bigtable. . .. . ... .. .

Some optimizations

27 / 51

Bigtable. . .. . ... .. .

Some optimizations

27 / 51

Bigtable. . .. . ... .. .

Some optimizations

Logging

.How would you store the logs about the tablets ?

A seperate log for each tablet ? GFS is used for accessing amoderate number of files...

One huge log ? What about failure recovery time ?

Two logs are used (only one is active) and they are sortedusing table id, row name and sequence number

28 / 51

Bigtable. . .. . ... .. .

Some optimizations

Logging

A seperate log for each tablet ?

GFS is used for accessing amoderate number of files...

28 / 51

Bigtable. . .. . ... .. .

Some optimizations

Logging

A seperate log for each tablet ?

GFS is used for accessing amoderate number of files...

28 / 51

Bigtable. . .. . ... .. .

Some optimizations

Logging

One huge log ?

What about failure recovery time ?

28 / 51

Bigtable. . .. . ... .. .

Some optimizations

Logging

One huge log ?

What about failure recovery time ?

28 / 51

Bigtable. . .. . ... .. .

Some optimizations

Logging

28 / 51

Bigtable. . .. . ... .. .

Some optimizations

Logging

28 / 51

Bigtable. . .. . ... .. .

Some optimizations

Bloom filters

ProblemRead requires acessing all SSTables inside a tablet.

Stressful for the disk.

A blooming filter can be used to help locate the data whenknowing the column and the row.

A blooming filter is an improved hashing method.

29 / 51

Bigtable. . .. . ... .. .

Some optimizations

Bloom filters

ProblemRead requires acessing all SSTables inside a tablet.Stressful for the disk.

29 / 51

Bigtable. . .. . ... .. .

Some optimizations

Bloom filters

29 / 51

Bigtable. . .. . ... .. .

Some optimizations

Bloom filters

29 / 51

Bigtable. . .. . ... .. .

Some optimizations

Bloom filters

30 / 51

Bigtable. . .. . ... .. .

4 Thoughts

5 Conclusion

31 / 51

Bigtable. . .. . ... .. .

An old problem

Lore[MAG+97] and [Abi97]

Semistructured database is an old problem.

Data model: Object Exchange Model

32 / 51

Bigtable. . .. . ... .. .

An old problem

Semistructured database is an old problem.Data model: Object Exchange Model

32 / 51

Bigtable. . .. . ... .. .

An old problem

Semistructured database is an old problem.Data model: Object Exchange Model

32 / 51

Bigtable. . .. . ... .. .

An old problem

Properties about Lore

Very general: graph with label.

(˜tree)

Hide irregularities in the structure when doing queries.

Pattern match possible.

Merging new data.

33 / 51

Bigtable. . .. . ... .. .

An old problem

Very general: graph with label.

(˜tree)

Merging new data.

33 / 51

Bigtable. . .. . ... .. .

An old problem

Very general: graph with label. (˜tree)

Merging new data.

33 / 51

Bigtable. . .. . ... .. .

An old problem

Merging new data.

33 / 51

Bigtable. . .. . ... .. .

An old problem

Merging new data.

33 / 51

Bigtable. . .. . ... .. .

An old problem

Some Comparison

3 Query will be optimized. (˜compilation)

3 More general than Column-key model.

3 Join support.

3 Virtualization of data placement At the time of the paper

3 Dataguides: visualization of database.

3 Code length (60 000 lines of C++ vs 550 000 for MongoDB)

7 Performance Issue. (˜traversal of graphs)

34 / 51

Bigtable. . .. . ... .. .

An old problem

Some Comparison

3 Join support.

3 Virtualization of data placement

At the time of the paper

34 / 51

Bigtable. . .. . ... .. .

An old problem

Some Comparison

3 Join support.

34 / 51

Bigtable. . .. . ... .. .

An old problem

Some Comparison

3 Join support.

34 / 51

Bigtable. . .. . ... .. .

An old problem

Some Comparison

3 Join support.

34 / 51

Bigtable. . .. . ... .. .

An old problem

Some Comparison

3 Join support.

34 / 51

Bigtable. . .. . ... .. .

An old problem

Some Comparison

3 Join support.

34 / 51

Bigtable. . .. . ... .. .

An old problem

Some Comparison

3 Join support.

34 / 51

Bigtable. . .. . ... .. .

An old problem

Some Comparison

3 Join support.

34 / 51

Bigtable. . .. . ... .. .

Extensible Record Stores

Cassandra [LM10]

Very similar to Bigtable (column-oriented).

Some features

Dynamic partionning of data.

Consistent hashing for distributing the data.

Replication is done by data replication on N nodes.

Global Knowledge of the network (hashing)

Failure detection.

Efficient anti-entropy gossiping protocol

35 / 51

Bigtable. . .. . ... .. .

Cassandra [LM10]

Some features

Failure detection.

35 / 51

Bigtable. . .. . ... .. .

Cassandra [LM10]

Some features

Failure detection.

35 / 51

Bigtable. . .. . ... .. .

Cassandra [LM10]

Some features

Failure detection.

35 / 51

Bigtable. . .. . ... .. .

Cassandra [LM10]

Some features

Failure detection.

35 / 51

Bigtable. . .. . ... .. .

Cassandra [LM10]

Some features

Failure detection.

35 / 51

Bigtable. . .. . ... .. .

Cassandra [LM10]

Some features

Failure detection.

35 / 51

Bigtable. . .. . ... .. .

Consistency

Strong Consistency

#Writers + #Readers > NbReplication

Consistency level

Read and write can have differents level of consistency (1 node torespond, majority, all).

36 / 51

Bigtable. . .. . ... .. .

Consistency

Strong Consistency

#Writers + #Readers > NbReplication

Consistency level

Read and write can have differents level of consistency (1 node torespond, majority, all).

36 / 51

Bigtable. . .. . ... .. .

Some Comparison

3 P2P structure

3 Super Family.

3 Load balancing (move lightly loaded nodes in the “ring”)

3 Some locality knowledge in replication (rack aware, datacenteraware)

3 Consistent hashing (reduce cost if changed)

7 (Eventual) Consistency

37 / 51

Bigtable. . .. . ... .. .

Some Comparison

3 P2P structure

3 Super Family.

37 / 51

Bigtable. . .. . ... .. .

Some Comparison

3 P2P structure

3 Super Family.

37 / 51

Bigtable. . .. . ... .. .

Some Comparison

3 P2P structure

3 Super Family.

37 / 51

Bigtable. . .. . ... .. .

Some Comparison

3 P2P structure

3 Super Family.

37 / 51

Bigtable. . .. . ... .. .

Some Comparison

3 P2P structure

3 Super Family.

37 / 51

Bigtable. . .. . ... .. .

Some Comparison

3 P2P structure

3 Super Family.

37 / 51

Bigtable. . .. . ... .. .

Document stores

MongoDB (No precise article)

Scability by sharding

Document oriented

38 / 51

Bigtable. . .. . ... .. .

Document stores

MongoDB (No precise article)

Scability by sharding

Document oriented

38 / 51

Bigtable. . .. . ... .. .

Key-value stores

Amazon Dynamo[DHJ+07]

Weak Consistency.

Consistent hashing.

Object versionning

Decentralized

Replication using quorum

Failure detection

Merkle tree for Eventual Consensus (Used in Cassandra)

39 / 51

Bigtable. . .. . ... .. .

Key-value stores

Weak Consistency.

Consistent hashing.

Object versionning

Decentralized

Failure detection

39 / 51

Bigtable. . .. . ... .. .

Key-value stores

Weak Consistency.

Consistent hashing.

Object versionning

Decentralized

Failure detection

39 / 51

Bigtable. . .. . ... .. .

Key-value stores

Weak Consistency.

Consistent hashing.

Object versionning

Decentralized

Failure detection

39 / 51

Bigtable. . .. . ... .. .

Key-value stores

Weak Consistency.

Consistent hashing.

Object versionning

Decentralized

Failure detection

39 / 51

Bigtable. . .. . ... .. .

Key-value stores

Weak Consistency.

Consistent hashing.

Object versionning

Decentralized

Failure detection

39 / 51

Bigtable. . .. . ... .. .

Key-value stores

Weak Consistency.

Consistent hashing.

Object versionning

Decentralized

Failure detection

39 / 51

Bigtable. . .. . ... .. .

Key-value stores

Some Comparison

7 Caching ?

7 Snapshot ?

7 (original article) key/value schema will affect speed if valueare huge (to write into the data you need to read it).

40 / 51

Bigtable. . .. . ... .. .

Key-value stores

Some Comparison

7 Caching ?

7 Snapshot ?

40 / 51

Bigtable. . .. . ... .. .

Key-value stores

Some Comparison

7 Caching ?

7 Snapshot ?

40 / 51

Bigtable. . .. . ... .. .

Key-value stores

Some Comparison

7 Caching ?

7 Snapshot ?

40 / 51

Bigtable. . .. . ... .. .

RAM databases

VoltDB http://voltdb.com

Based on H-Store[SMA+07]

Idea is to use main memory as storage.

41 / 51

Bigtable. . .. . ... .. .

RAM databases

VoltDB http://voltdb.com

Based on H-Store[SMA+07]

Idea is to use main memory as storage.

41 / 51

Bigtable. . .. . ... .. .

RAM databases

7 Cost ?

7 Size ?

7 RAM is not persistent.

42 / 51

Bigtable. . .. . ... .. .

RAM databases

7 Cost ?

7 Size ?

42 / 51

Bigtable. . .. . ... .. .

RAM databases

7 Cost ?

7 Size ?

42 / 51

Bigtable. . .. . ... .. .

RAM databases

7 Cost ?

7 Size ?

42 / 51

Bigtable. . .. . ... .. .

4 Thoughts

5 Conclusion

43 / 51

Bigtable. . .. . ... .. .

Answers

Mapreduce

Using HBase.

One map-reduce job per tablets.

44 / 51

Bigtable. . .. . ... .. .

Answers

Mapreduce

Using HBase.One map-reduce job per tablets.

44 / 51

Bigtable. . .. . ... .. .

Answers

Mapreduce

Using HBase.One map-reduce job per tablets.

44 / 51

Bigtable. . .. . ... .. .

Answers

Compaction

Size-TieredCompaction is done if the number of sstables hits a threshold.

7 Need a lot of space to do the copy (and the size of sstablesincreases).

7 Columns of a row could be spread across the different sstables

→ Irregular performance.

Leveled compaction

SSTable are smaller and grouped by levels.Inside a level, sstables will not overlap.

7 More I/O

How to choose from this two policies ?

45 / 51

Bigtable. . .. . ... .. .

Answers

Compaction

Leveled compaction

7 More I/O

45 / 51

Bigtable. . .. . ... .. .

Answers

Compaction

Leveled compaction

7 More I/O

45 / 51

Bigtable. . .. . ... .. .

Answers

Compaction

7 Columns of a row could be spread across the different sstables→ Irregular performance.

Leveled compaction

SSTable are smaller and grouped by levels.

Inside a level, sstables will not overlap.

7 More I/O

45 / 51

Bigtable. . .. . ... .. .

Answers

Compaction

Leveled compaction

7 More I/O

45 / 51

Bigtable. . .. . ... .. .

Answers

Compaction

Leveled compaction

7 More I/O

45 / 51

Bigtable. . .. . ... .. .

Answers

Compaction

Leveled compaction

7 More I/O

45 / 51

Bigtable. . .. . ... .. .

Answers

Compaction

Leveled compaction

7 More I/O

How to choose from this two policies ?45 / 51

Bigtable. . .. . ... .. .

Answers

SSTable assignment

The master is capable of maintining the list:Tl [TSid ] = all tablets handle by TSid .

Creations, Merges and Deletions are handle by the master.

Split are initiated by the tablet server but it notifies themaster.

46 / 51

Bigtable. . .. . ... .. .

Answers

SSTable assignment

46 / 51

Bigtable. . .. . ... .. .

Answers

SSTable assignment

46 / 51

Bigtable. . .. . ... .. .

Limitations

7 No Join

7 ACID (Atomicity, Consistency, Isolation, Durability): eventualconsistency

Problem that arise with semi structured data (texts):

no schema possible ? (relationnal data. use null value)dynamic schema

3 Nice for Map reduce.

Load balancing: an way of scaling SQL server.

7 Paxos

7 Why use a tree for locating tables ?

7 No real solution for the number of tablets.

47 / 51

Bigtable. . .. . ... .. .

Limitations

7 No Join

7 Paxos

47 / 51

Bigtable. . .. . ... .. .

Limitations

7 No Join

7 Paxos

47 / 51

Bigtable. . .. . ... .. .

Limitations

7 No Join

7 Paxos

47 / 51

Bigtable. . .. . ... .. .

Limitations

7 No Join

Problem that arise with semi structured data (texts):no schema possible ? (relationnal data. use null value)

dynamic schema

7 Paxos

47 / 51

Bigtable. . .. . ... .. .

Limitations

7 No Join

Problem that arise with semi structured data (texts):no schema possible ? (relationnal data. use null value)dynamic schema

7 Paxos

47 / 51

Bigtable. . .. . ... .. .

Limitations

7 No Join

7 Paxos

47 / 51

Bigtable. . .. . ... .. .

Limitations

7 No Join

7 Paxos

47 / 51

Bigtable. . .. . ... .. .

Limitations

7 No Join

7 Paxos

47 / 51

Bigtable. . .. . ... .. .

Limitations

7 No Join

7 Paxos

47 / 51

Bigtable. . .. . ... .. .

4 Thoughts

5 Conclusion

48 / 51

Bigtable. . .. . ... .. .

Conclusion

NoSQL ̸= SQL.

Old problem but new data pattern.

49 / 51

Bigtable. . .. . ... .. .

Conclusion

NoSQL ̸= SQL.

49 / 51

Bigtable. . .. . ... .. .

Conclusion

NoSQL ̸= SQL.

49 / 51

Bigtable. . .. . ... .. .

Conclusion

Thank You for your attention !

50 / 51

Bibliography

Serge Abiteboul.

Querying semi-structured data.In Foto N. Afrati and Phokion G. Kolaitis, editors, Database Theory - ICDT ’97, 6th InternationalConference, Delphi, Greece, January 8-10, 1997, Proceedings, volume 1186 of Lecture Notes in ComputerScience, pages 1–18. Springer, 1997.

Giuseppe DeCandia, Deniz Hastorun, Madan Jampani, Gunavardhan Kakulapati, Avinash Lakshman, Alex

Pilchin, Swaminathan Sivasubramanian, Peter Vosshall, and Werner Vogels.Dynamo: amazon’s highly available key-value store.SIGOPS Oper. Syst. Rev., 41(6):205–220, October 2007.

Alpa Jain, AnHai Doan, and Luis Gravano.

Optimizing sql queries over text databases.In Proceedings of the 2008 IEEE 24th International Conference on Data Engineering, ICDE ’08, pages636–645, Washington, DC, USA, 2008. IEEE Computer Society.

Avinash Lakshman and Prashant Malik.

Cassandra: a decentralized structured storage system.SIGOPS Oper. Syst. Rev., 44(2):35–40, April 2010.

Jason McHugh, Serge Abiteboul, Roy Goldman, Dallan Quass, and Jennifer Widom.

Lore: A database management system for semistructured data.SIGMOD Record, 26:54–66, 1997.

Michael Stonebraker, Samuel Madden, Daniel J. Abadi, Stavros Harizopoulos, Nabil Hachem, and Pat

Helland.The end of an architectural era: (it’s time for a complete rewrite).In Proceedings of the 33rd international conference on Very large data bases, VLDB ’07, pages 1150–1160.VLDB Endowment, 2007.

bigtable: a distributed storage system for structured data...

Documents