Download - c onsistency without borders
consistency without borders
Peter Alvaro, Peter Bailis, Neil Conway, Joseph M. HellersteinUC Berkeley
The transaction concept DEBIT_CREDIT: BEGIN_TRANSACTION; GET MESSAGE; EXTRACT ACCOUT_NUMBER, DELTA, TELLER, BRANCH FROM MESSAGE; FIND ACCOUNT(ACCOUT_NUMBER) IN DATA BASE; IF NOT_FOUND | ACCOUNT_BALANCE + DELTA < 0 THEN PUT NEGATIVE RESPONSE; ELSE DO; ACCOUNT_BALANCE = ACCOUNT_BALANCE + DELTA; POST HISTORY RECORD ON ACCOUNT (DELTA); CASH_DRAWER(TELLER) = CASH_DRAWER(TELLER) + DELTA; BRANCH_BALANCE(BRANCH) = BRANCH_BALANCE(BRANCH) + DELTA; PUT MESSAGE ('NEW BALANCE =' ACCOUNT_BALANCE); END; COMMIT;
An application-level contract
Write Read
Application
Opaquestore
Transactions
Pervasive distribution
CAP
partial failure
asynchrony
Research on consistency
Write Read
Application
Opaquestore
Consistency models
Assert: balance > 0
causal?PRAM?delta?fork/join?red/blue?release?SC?
(translation)
R1(X=1) R2(X=1) W1(X=2) W2(X=0)W1(X=1) W1(Y=2) R2(Y=2) R2(X=0)
Meanwhile, in industry…
Application
Opaquestore
Custom solutionsAssert: balance > 0
(prayer)
Write Read
Distributed consistency: staying relevant
• Is this an important problem?• Is academia disconnected from
reality?• OK, what now?
Goal: help programmers write correct applications.Today: some promising approaches
Case study: a graph
Partitioned, for scalability
Replicated, for availability
Problem: deadlock detection
Task: Identify strongly-connected components
Waits-for graph
Problem: garbage collection
Task: Identify nodes not reachable from Root.
Root
Refers-to graph
Correctness
Deadlock detection• Safety: No false positives-• Liveness: Identify all deadlocksGarbage collection• Safety: Never GC live memory!• Liveness: GC all orphaned memory
Partition
Root
Consistency at the extremes
Linearizable key-value store?
Custom solutions?
Efficient Correct
Consistency across the stack
Object-level consistencyCapture semantics of data structures that• allow greater concurrency • maintain guarantees (e.g. convergence)
Insert Read
Convergentdata structure(e.g., Set CRDT)
Object-level consistency
Insert Read
CommutativityAssociativityIdempotence
ReorderingBatchingRetry/duplication
Tolerant to
Application
Convergentdata structures
Object-level consistency
? ?
GC Assert: No live nodes are reclaimed
Assert: Graph replicasconverge
Flow-level consistency
Flow-level consistencyCapture semantics of data in motion• Asynchronous dataflow model• component properties system-wide guarantees
Flow-level consistencyOrder-insensitivity (confluence)
output set = f(input set)
{ }
{ }=
Flow-level consistencyConfluence is compositional
output set = f g(input set)
Graph queries as dataflowConfluent
Coordinate here
Language-level consistencyDSLs for distributed programming?• Capture consistency concerns in the
type system
Language-level consistency
CALM Theorem:
Monotonic confluent
Conservative, syntactic test for confluence
Language-level consistencyDeadlock detector
Garbage collector
nonmonotonic
correctreusable
efficientintuitive
Where we’ve been;where we’re headed
Remember• Consistency is an application-level
property
• Correctness and performance are compatible
• Meet programmers on their home turf
• Build bridges!
Queries?