primal ladis
TRANSCRIPT
-
7/31/2019 Primal Ladis
1/30
DISTRIBUTED INDEXING AND LOCKING :IN SEARCH OF SCALABLE CONSISTENCY
Mahdi Tayarani Najaran, Primal Wijesekera, AndrewWarfield and Norman C. Hutchinson
Networks, Systems,andSecurity Lab (NSS)
-
7/31/2019 Primal Ladis
2/30
Why isnt there just one instance ofgiant SQL Database ?
2
-
7/31/2019 Primal Ladis
3/30
SQL
3
-
7/31/2019 Primal Ladis
4/30
Alternative - 01
Data Partitioning
4
-
7/31/2019 Primal Ladis
5/30
Alternative - 02
NOSQLKV StorageSystems
5
-
7/31/2019 Primal Ladis
6/30
What about SQL
Highly Optimized Sound mathematical base
Highly adopted across domains Very rich query model
6
-
7/31/2019 Primal Ladis
7/30
What is wrong with SQL
Why we think that we can do better
Our Solution
Evaluation
What is next
7
-
7/31/2019 Primal Ladis
8/30
We think
SQL is just great but
Developers arent
8
-
7/31/2019 Primal Ladis
9/30
We have a great platform
SinfoniaEx: A Distributed Transactional Memory Service An open source implementation of Sinfonia
9
-
7/31/2019 Primal Ladis
10/30
Minitransaction
A given minitransaction has A read set A compare set A conditional write set
All updates occur atomically only if all comparisonssucceed
Locks are held only for a very small time
10
-
7/31/2019 Primal Ladis
11/30
What is wrong with SQL
Why we think that we can do better
Our Solution
Evaluation
What is next
11
-
7/31/2019 Primal Ladis
12/30
Replacethe single-nodestoragelayer of an off-the-shelf DBMS engine(eg. MySQL) with a distributed storage
layer
12
-
7/31/2019 Primal Ladis
13/30
The Design
13
-
7/31/2019 Primal Ladis
14/30
The Design contd.
14
-
7/31/2019 Primal Ladis
15/30
Why it is hard to do so
Distributed Indexing With high expressibility
Distributed Locking With complex range locking
( Distributed Storage )
Fundamental Requirements Distributed Scalability even in the presence of write heavy workloads Fault tolerance Strict consistency guarantees
15
-
7/31/2019 Primal Ladis
16/30
Getting Expressibility
B-Tree Single-attribute range queries Suffers for write-intense workloads Balanced tree
Spatial Partitioning Tree Multi-attribute range queries Scales for all types of workloads Un-balanced tree
We picked Spatial Partitioning Tree (SPT) over B-Tree
Store data only in leaf nodes to avoid contention SPTs increase partitioning granularity by re-partitioningleaves when required
16
-
7/31/2019 Primal Ladis
17/30
3-D Spatial Partitioning Tree
src : wikipedia.org
17
18
-
7/31/2019 Primal Ladis
18/30
SPT for Indexing
ALTER TABLE orders
ADD CONSTRAINTpkey_ordersPRIMARY KEY(o_w_id, o_d_id, o_id);
18
19
-
7/31/2019 Primal Ladis
19/30
SPT for Locking Manager
Modified SPT from Index Manager Key is a range. Range insert => lock acquiring Range removal => lock releasing
Supports both records locking and complex rangelocking.
All other locking semantics are intact.
Read and write locks.
19
20
-
7/31/2019 Primal Ladis
20/30
SinExTree
Is the SPT implementation on top of SinfoniaEx Gives a transactionaly consistent and scalable approach Fulfills all the fundamental requirements
20
21
-
7/31/2019 Primal Ladis
21/30
Minitransactions in SPT
Writing a value to Node L
In this instance, Compare set = {A, C, E, H, J, L} Conditional write set = {L}
21
22
-
7/31/2019 Primal Ladis
22/30
Final Architectural Setup
22
23
-
7/31/2019 Primal Ladis
23/30
What is wrong with SQL
Why we think that we can do better
Our Solution
Evaluation
What is next
23
24
-
7/31/2019 Primal Ladis
24/30
Evaluation Setup
The main objective is to measure the throughput of the indexunder maximum stress at different scales
Log and replay Logged the index operations of TPC-C benchmark Replayed logs on the distributed index
Used Amazon EC2 using 72 High CPU Extra Large instances
Varied the number of index hosts from 1 to 64 with 8 fixed
memnodes
24
25
-
7/31/2019 Primal Ladis
25/30
Evaluation ResultsThroughput
25
26
-
7/31/2019 Primal Ladis
26/30
Evaluation ResultsThroughput
26
-
7/31/2019 Primal Ladis
27/30
28
-
7/31/2019 Primal Ladis
28/30
What is wrong with SQL
Why we think that we can do better
Our Solution
Evaluation
What is next
28
29
-
7/31/2019 Primal Ladis
29/30
Next
Future Work Integrate all the components into one system Use this setup for a DBMS as a Service
Future Challenges Making sure the correctness is preserved through out all layers Implementing Security measures to guarantee privacy Define a pricing model with a emphasize on fluctuating workloads
29
30
-
7/31/2019 Primal Ladis
30/30
Thank You !!!
30