minimizing coordination in replicated systems
TRANSCRIPT
Minimizing Coordination in Replicated systems
Cheng Li
Max Planck Institute for Software Systems
Joint work with João Leitão, Allen Clement, Nuno Preguiça, and Rodrigo Rodrigues
MPI-SWS/NOVALINCS
Fast Internet services
Build replicated systems that are fast and correct
4/13/2015 Cheng Li @ PaPoC'15 2
[Making Geo-Replicated Systems Fast as Possible, Consistent when necessary, OSDI’12]
MPI-SWS/NOVALINCS
Fast Internet services
Build replicated systems that are fast and correct
4/13/2015 Cheng Li @ PaPoC'15
Blue ops: no coordination, weak consistency
3
[Making Geo-Replicated Systems Fast as Possible, Consistent when necessary, OSDI’12]
MPI-SWS/NOVALINCS
Fast Internet services
Build replicated systems that are fast and correct
4/13/2015 Cheng Li @ PaPoC'15
State convergence
Invariant preservation
Blue ops: no coordination, weak consistency
4
[Making Geo-Replicated Systems Fast as Possible, Consistent when necessary, OSDI’12]
MPI-SWS/NOVALINCS
Fast Internet services
Build replicated systems that are fast and correct
4/13/2015 Cheng Li @ PaPoC'15
Blue ops: no coordination, weak consistency
Red ops: coordination, stronger consistency
5
State convergence
Invariant preservation
[Making Geo-Replicated Systems Fast as Possible, Consistent when necessary, OSDI’12]
MPI-SWS/NOVALINCS
How to coordinate red ops?
• Totally order all of them– simple but sometimes too conservative/costly
• Observation: a red op only needs to be ordered w.r.t other relevant red ops.
• E.g., auction service– Bid + close auction operations
– Invariant: highest accepted bid is declared winner
4/13/2015 Cheng Li @ PaPoC'15 6
MPI-SWS/NOVALINCS
bidder price
Replicated auction service
7Cheng Li @ PaPoC'154/13/2015
UKUS
bid($10)
winner bidder price
Bob 10
winner bidder price
MPI-SWS/NOVALINCS
bidder price
Replicated auction service
8Cheng Li @ PaPoC'154/13/2015
UKUS
winner bidder price
Bob 10
winner bidder price
bid($15)
bidder price
Alice 15
MPI-SWS/NOVALINCS
bidder price
Replicated auction service
9Cheng Li @ PaPoC'154/13/2015
UKUS
winner bidder price
Bob 10
winner bidder pricebidder price
Alice 15
close()
winner
Bob
MPI-SWS/NOVALINCS
Replicated auction service
10Cheng Li @ PaPoC'154/13/2015
UKUS
Bob won even with a lower bid than Alice.
bidder price
Bob 10
winner bidder price
Alice 15
winner
Bob
bidder price
Alice 15
Bob 10
bidder price
Alice 15
Bob 10
winner
Bob
MPI-SWS/NOVALINCS
Less coordination is better!
4/13/2015 Cheng Li @ PaPoC'15 11
bid
bid
close
bid
bid
bid
close
bid
bid
MPI-SWS/NOVALINCS
Less coordination is better!
4/13/2015 Cheng Li @ PaPoC'15 12
bid
bid
close
bid
bid
bid
close
bid
bid
Visibility restriction
A partial order enforcing visibility
restrictions
Pros: fine-grained tuning in consistency semantics (the amount of coordination) using restrictions
MPI-SWS/NOVALINCS
Outline
• Background
• Partial order-Restriction (PoR) consistency– Visibility restrictions
– Consistency model
• Restriction inference
• Conclusion
4/13/2015 Cheng Li @ PaPoC'15 13
MPI-SWS/NOVALINCS
Outline
• Background
• Partial order-Restriction (PoR) consistency– Visibility restrictions
– Consistency model
• Restriction inference
• Conclusion
4/13/2015 Cheng Li @ PaPoC'15 14
MPI-SWS/NOVALINCS
Visibility restrictions
• A restriction between two operations implies that one must see effects introduced by the other.
• For operation 𝑎, 𝑏, the restriction 𝑟 𝑎, 𝑏 implies that 𝑎 ≺ 𝑏 𝑏 ≺ 𝑎 w.r.t any partial order ≺.
4/13/2015 Cheng Li @ PaPoC'15 15
a0
b0
a1
a2
b1
b2
If 𝑎 ≺ 𝑏 𝑏 ≺ 𝑎,
then 𝑟 𝑎, 𝑏 is met in ≺.
MPI-SWS/NOVALINCS
Partial order-restrictions (PoR) Consistency
• A replicated system 𝑆 is associated with a set of restrictions 𝑅𝑠.
– e.g., 𝑅𝑠 = {𝑟(𝑎, 𝑏), 𝑟(𝑐, 𝑑)}
• 𝑆 is PoR Consistent if, for any its executions, there exists an admissible partial order, where all restrictions in 𝑅𝑠 are met.
4/13/2015 Cheng Li @ PaPoC'15 16
MPI-SWS/NOVALINCS
Outline
• Background
• Partial order-Restriction (PoR) consistency– Visibility restrictions
– Consistency model
• Restriction inference
• Conclusion
4/13/2015 Cheng Li @ PaPoC'15 17
MPI-SWS/NOVALINCS
Challenges
• Which restrictions are sufficient?– They must ensure relevant properties.
• Does the set of restrictions comprise a minimal coordination plan?– i.e., we cannot remove any restriction from the set.
4/13/2015 Cheng Li @ PaPoC'15 18
MPI-SWS/NOVALINCS
Outline
• Background
• Partial order-Restriction (PoR) consistency
• Restriction inference– State convergence
– Invariant preservation
• Conclusion
4/13/2015 Cheng Li @ PaPoC'15 19
MPI-SWS/NOVALINCS
State convergence
• Definition: all replicas reach the same final state when the system becomes quiescent.
• Insight: any pair of operations must be ordered if they don’t commute.
4/13/2015 Cheng Li @ PaPoC'15 20
MPI-SWS/NOVALINCS
Invariant preservation
• Insight: for any violation, add restrictions among concurrent operations
4/13/2015 Cheng Li @ PaPoC'15 21
bid(“Bob”, 10)
bid(“Alice”, 15)
close()
winner bidder price
winner
Bob
bidder price
Alice 15
Bob 10
MPI-SWS/NOVALINCS
Invariant preservation (cont.)
• Insight: for any violation, add restrictions among concurrent operations
4/13/2015 Cheng Li @ PaPoC'15 22
bid(“Bob”, 10)
bid(“Alice”, 15)
close()
winner bidder price
winner
Bob
bidder price
Alice 15
Bob 10
Q: Can we remove any concurrentop while maintaining the violation?
MPI-SWS/NOVALINCS
Invariant preservation (cont.)
• Insight: for any violation, add restrictions among concurrent operations
4/13/2015 Cheng Li @ PaPoC'15 23
bid(“Bob”, 10)
bid(“Alice”, 15)
winner bidder price
winner bidder price
Alice 15
Bob 10
Q: Can we remove any concurrentop while maintaining the violation?A: close must be there.
MPI-SWS/NOVALINCS
Invariant preservation (cont.)
• Insight: for any violation, add restrictions among concurrent operations
4/13/2015 Cheng Li @ PaPoC'15 24
bid(“Bob”, 10)
bid(“Alice”, 15)
close()
winner bidder price
winner
Bob
bidder price
Alice 15
Bob 10
Q: Can we remove any concurrentop while maintaining the violation?
MPI-SWS/NOVALINCS
Invariant preservation (cont.)
• Insight: for any violation, add restrictions among concurrent operations
4/13/2015 Cheng Li @ PaPoC'15 25
bid(“Alice”, 15)
close()
winner bidder price
Bob 10
winner
Bob
bidder price
Alice 15
Bob 10
Q: Can we remove any concurrentop while maintaining the violation?A: Yes, bid(“Bob”, 10).
MPI-SWS/NOVALINCS
Invariant preservation (cont.)
• Insight: for any violation, add restrictions among concurrent operations
4/13/2015 Cheng Li @ PaPoC'15 26
bid(“Alice”, 15)
close()
winner bidder price
Bob 10
winner
Bob
bidder price
Alice 15
Bob 10
This is a minimal violating execution.Solution: 𝑟(𝑏𝑖𝑑, 𝑐𝑙𝑜𝑠𝑒)
Analysis:Step 1: For any such an execution, we add a restriction over a pair of its maximaloperations. Step 2: Iterate all these executions
MPI-SWS/NOVALINCS
Take-home points
• Consistency semantics can be captured by visibility restrictions over operations.
• Invariant and commutativity analysis finds the minimal set of restrictions to preserve convergence and invariants.
• PoR Consistency enforces all restrictions throughout all executions of a replicated system.
4/13/2015 Cheng Li @ PaPoC'15 27
Minimizing Coordination in Replicated systems
Thanks for your attention!