there is more consensus in egalitarian parliaments€¦ · iulian moraru, david andersen, michael...
TRANSCRIPT
![Page 1: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/1.jpg)
There Is More Consensus in Egalitarian Parliaments
Iulian Moraru, David Andersen, Michael Kaminsky
Carnegie Mellon University Intel Labs
Friday, November 8, 13
![Page 2: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/2.jpg)
Faulttolerance Redundancy⇒
Friday, November 8, 13
![Page 3: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/3.jpg)
State Machine Replication
3
Friday, November 8, 13
![Page 4: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/4.jpg)
State Machine Replication
3
Execute the same commands in the same order
Friday, November 8, 13
![Page 5: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/5.jpg)
State Machine Replication
3
Execute the same commands in the same order
Friday, November 8, 13
![Page 6: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/6.jpg)
State Machine Replication
3
Paxos
Execute the same commands in the same order
Friday, November 8, 13
![Page 7: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/7.jpg)
State Machine Replication
3
Paxos
• No external failure detector required
Execute the same commands in the same order
Friday, November 8, 13
![Page 8: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/8.jpg)
State Machine Replication
3
Paxos
• No external failure detector required
• Fast fail-over (high availability)
Execute the same commands in the same order
Friday, November 8, 13
![Page 9: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/9.jpg)
Paxos is important in clustersChubby, Boxwood, SMARTER, ZooKeeper
• Synchronization• Resource discovery• Data replication
4
High throughputHigh availability
Friday, November 8, 13
![Page 10: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/10.jpg)
Paxos is important in the wide-areaSpanner, Megastore
• Bring data closer to clients
• Tolerate datacenter outages
5
Low latency
Friday, November 8, 13
![Page 11: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/11.jpg)
Paxos is important in the wide-areaSpanner, Megastore
• Bring data closer to clients
• Tolerate datacenter outages
5
Low latency
Friday, November 8, 13
![Page 12: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/12.jpg)
Paxos is important in the wide-areaSpanner, Megastore
• Bring data closer to clients
• Tolerate datacenter outages
5
Low latency
85ms130ms
Friday, November 8, 13
![Page 13: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/13.jpg)
Paxos overview
6
Friday, November 8, 13
![Page 14: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/14.jpg)
Paxos overview• Agreement protocol
6
Friday, November 8, 13
![Page 15: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/15.jpg)
Paxos overview• Agreement protocol
• Tolerates F failures with 2F+1 replicas (optimal)
• No external failure detector required
6
Friday, November 8, 13
![Page 16: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/16.jpg)
Paxos overview• Agreement protocol
• Tolerates F failures with 2F+1 replicas (optimal)
• No external failure detector required
• Replicas can fail by crashing (non-Byzantine)
6
Friday, November 8, 13
![Page 17: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/17.jpg)
Paxos overview• Agreement protocol
• Tolerates F failures with 2F+1 replicas (optimal)
• No external failure detector required
• Replicas can fail by crashing (non-Byzantine)
• Asynchronous communication
6
Friday, November 8, 13
![Page 18: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/18.jpg)
1 2 3 4 ...
Using Paxos to order commands
7
Friday, November 8, 13
![Page 19: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/19.jpg)
1 2 3 4 ...
Using Paxos to order commands
7
A BC
Friday, November 8, 13
![Page 20: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/20.jpg)
1 2 3 4 ...
Using Paxos to order commands
7
A BC
Friday, November 8, 13
![Page 21: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/21.jpg)
1 2 3 4 ...
Using Paxos to order commands
7
A BCvote C
vote B
vote A
Friday, November 8, 13
![Page 22: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/22.jpg)
1 2 3 4 ...
Using Paxos to order commands
7
A BCvote C
vote B
vote A
ack B ack B
Friday, November 8, 13
![Page 23: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/23.jpg)
1 2 3 4 ...
Using Paxos to order commands
7
A
B
C
Friday, November 8, 13
![Page 24: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/24.jpg)
1 2 3 4 ...
Using Paxos to order commands
7
A
B
C
Friday, November 8, 13
![Page 25: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/25.jpg)
1 2 3 4 ...
Using Paxos to order commands
7
AB
C
Friday, November 8, 13
![Page 26: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/26.jpg)
1 2 3 4 ...
DUsing Paxos to order commands
7
AB
C
Friday, November 8, 13
![Page 27: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/27.jpg)
1 2 3 4 ...
DUsing Paxos to order commands
7
AB
C
Friday, November 8, 13
![Page 28: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/28.jpg)
1 2 3 4 ...D
Using Paxos to order commands
7
AB C
Friday, November 8, 13
![Page 29: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/29.jpg)
1 2 3 4 ...D
Using Paxos to order commands
7
AB C
• Choose commands independently for each slot
Friday, November 8, 13
![Page 30: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/30.jpg)
1 2 3 4 ...D
Using Paxos to order commands
7
AB C
• Choose commands independently for each slot
• At least 2 RTTs per slot:
Friday, November 8, 13
![Page 31: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/31.jpg)
1 2 3 4 ...D
Using Paxos to order commands
7
AB C
• Choose commands independently for each slot
• At least 2 RTTs per slot:1. Take ownership of a slot
Friday, November 8, 13
![Page 32: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/32.jpg)
1 2 3 4 ...D
Using Paxos to order commands
7
AB C
• Choose commands independently for each slot
• At least 2 RTTs per slot:1. Take ownership of a slot
2. Propose command
Friday, November 8, 13
![Page 33: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/33.jpg)
1 2 3 4 ...
Multi-Paxos
8
Friday, November 8, 13
![Page 34: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/34.jpg)
1 2 3 4 ...
Multi-Paxos
8
Friday, November 8, 13
![Page 35: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/35.jpg)
1 2 3 4 ...
DMulti-Paxos
8
ABC
Friday, November 8, 13
![Page 36: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/36.jpg)
1 2 3 4 ...
DMulti-Paxos
8
A
BC
Friday, November 8, 13
![Page 37: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/37.jpg)
1 2 3 4 ...
DMulti-Paxos
8
A B
C
Friday, November 8, 13
![Page 38: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/38.jpg)
1 2 3 4 ...
DMulti-Paxos
8
A B C
Friday, November 8, 13
![Page 39: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/39.jpg)
1 2 3 4 ...D
Multi-Paxos
8
A B C
Friday, November 8, 13
![Page 40: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/40.jpg)
1 2 3 4 ...D
Multi-Paxos
8
A B C
• 1 RTT to commit
Friday, November 8, 13
![Page 41: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/41.jpg)
1 2 3 4 ...D
Multi-Paxos
8
A B C
• 1 RTT to commit
• Bottleneck for performance and availabilty
Friday, November 8, 13
![Page 42: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/42.jpg)
Can we have it all?
9
Friday, November 8, 13
![Page 43: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/43.jpg)
Can we have it all?
9
• High throughput, low latency
Friday, November 8, 13
![Page 44: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/44.jpg)
Can we have it all?
9
• High throughput, low latency
• Constant availability
Friday, November 8, 13
![Page 45: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/45.jpg)
Can we have it all?
9
• High throughput, low latency
• Constant availability
• Distribute load evenly across all replicas
Friday, November 8, 13
![Page 46: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/46.jpg)
Can we have it all?
9
• High throughput, low latency
• Constant availability
• Distribute load evenly across all replicas
• Use fastest replicas
Friday, November 8, 13
![Page 47: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/47.jpg)
Can we have it all?
9
• High throughput, low latency
• Constant availability
• Distribute load evenly across all replicas
• Use fastest replicas
• Use closest (lowest latency) replicas
Friday, November 8, 13
![Page 48: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/48.jpg)
Can we have it all?
9
Paxos• High throughput, low latency
• Constant availability
• Distribute load evenly across all replicas
• Use fastest replicas
• Use closest (lowest latency) replicas
Friday, November 8, 13
![Page 49: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/49.jpg)
Can we have it all?
9
PaxosMulti-• High throughput, low latency
• Constant availability
• Distribute load evenly across all replicas
• Use fastest replicas
• Use closest (lowest latency) replicas
Friday, November 8, 13
![Page 50: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/50.jpg)
Can we have it all?
9
PaxosMulti-
Egalitarian Paxos(EPaxos)
• High throughput, low latency
• Constant availability
• Distribute load evenly across all replicas
• Use fastest replicas
• Use closest (lowest latency) replicas
Friday, November 8, 13
![Page 51: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/51.jpg)
EPaxos is all about ordering
10
Friday, November 8, 13
![Page 52: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/52.jpg)
EPaxos is all about orderingPrevious strategies:
10
Friday, November 8, 13
![Page 53: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/53.jpg)
EPaxos is all about orderingPrevious strategies:
10
...
Contend for slots
Friday, November 8, 13
![Page 54: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/54.jpg)
EPaxos is all about orderingPrevious strategies:
10
...
Contend for slots Paxos
Friday, November 8, 13
![Page 55: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/55.jpg)
EPaxos is all about orderingPrevious strategies:
10
...
Contend for slots
...
One replicadecides
Paxos
Friday, November 8, 13
![Page 56: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/56.jpg)
EPaxos is all about orderingPrevious strategies:
10
...
Contend for slots
...
One replicadecides
Paxos
Multi-Paxos, Fast Paxos,Generalized Paxos
Friday, November 8, 13
![Page 57: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/57.jpg)
EPaxos is all about orderingPrevious strategies:
10
...
Contend for slots
...
One replicadecides
Paxos
Multi-Paxos, Fast Paxos,Generalized Paxos
...
Take turnsround-robin
Friday, November 8, 13
![Page 58: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/58.jpg)
EPaxos is all about orderingPrevious strategies:
10
...
Contend for slots
...
One replicadecides
Paxos
Multi-Paxos, Fast Paxos,Generalized Paxos
Mencius
...
Take turnsround-robin
Friday, November 8, 13
![Page 59: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/59.jpg)
EPaxos intuition
11
Friday, November 8, 13
![Page 60: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/60.jpg)
EPaxos intuition
1 2 3 4 ...
11
Friday, November 8, 13
![Page 61: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/61.jpg)
EPaxos intuition
1 2 3 4 ...
A
11
Friday, November 8, 13
![Page 62: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/62.jpg)
EPaxos intuition
1 2 3 4 ...A
C B
11
Friday, November 8, 13
![Page 63: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/63.jpg)
EPaxos intuition
1 2 3 4 ...ACB
11
D
Friday, November 8, 13
![Page 64: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/64.jpg)
EPaxos intuition
1 2 3 4 ...ACB
11
D
Friday, November 8, 13
![Page 65: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/65.jpg)
EPaxos intuition
1 2 3 4 ...ACB
11
D
Friday, November 8, 13
![Page 66: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/66.jpg)
EPaxos intuition
1 2 3 4 ...ACB
11
D
Friday, November 8, 13
![Page 67: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/67.jpg)
EPaxos intuition
1 2 3 4 ...ACB
11
D
Friday, November 8, 13
![Page 68: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/68.jpg)
EPaxos intuition
1 2 3 4 ...ACB
11
D
After commit @ each replica
Friday, November 8, 13
![Page 69: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/69.jpg)
EPaxos intuition
1 2 3 4 ...ACB
11
D
A B C DAfter commit @ each replica
Friday, November 8, 13
![Page 70: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/70.jpg)
EPaxos intuition
1 2 3 4 ...ACB
11
• Load balance (every replica is a leader)
D
A B C DAfter commit @ each replica
Friday, November 8, 13
![Page 71: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/71.jpg)
EPaxos intuition
1 2 3 4 ...ACB
11
• Load balance (every replica is a leader)
• EPaxos can choose any quorum for each command
D
A B C DAfter commit @ each replica
Friday, November 8, 13
![Page 72: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/72.jpg)
EPaxos commit protocol
12
R1
R2
R3
R4
R5
Friday, November 8, 13
![Page 73: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/73.jpg)
EPaxos commit protocol
12
R1
R2
R3
R4
R5
PreAccept(A)
A ➝ ∅
Friday, November 8, 13
![Page 74: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/74.jpg)
EPaxos commit protocol
12
R1
R2
R3
R4
R5
PreAccept(A)
A ➝ ∅ A ➝ ∅
Friday, November 8, 13
![Page 75: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/75.jpg)
EPaxos commit protocol
12
R1
R2
R3
R4
R5
PreAccept(A)
A ➝ ∅ A ➝ ∅
Commit A ➝ ∅
Friday, November 8, 13
![Page 76: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/76.jpg)
EPaxos commit protocol
12
R1
R2
R3
R4
R5
PreAccept(A)
A ➝ ∅ A ➝ ∅
Commit A ➝ ∅
Friday, November 8, 13
![Page 77: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/77.jpg)
EPaxos commit protocol
12
R1
R2
R3
R4
R5
PreAccept(A)
A ➝ ∅ A ➝ ∅
PreAccept(B)
B ➝ ∅
Commit A ➝ ∅
Friday, November 8, 13
![Page 78: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/78.jpg)
EPaxos commit protocol
12
R1
R2
R3
R4
R5
PreAccept(A)
A ➝ ∅ A ➝ ∅
PreAccept(B)
B ➝ ∅ B ➝ ∅
Commit A ➝ ∅
Friday, November 8, 13
![Page 79: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/79.jpg)
EPaxos commit protocol
12
R1
R2
R3
R4
R5
PreAccept(A)
A ➝ ∅ A ➝ ∅
PreAccept(B)
B ➝ ∅
B ➝ {A}
B ➝ ∅
Commit A ➝ ∅
Friday, November 8, 13
![Page 80: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/80.jpg)
EPaxos commit protocol
12
R1
R2
R3
R4
R5
PreAccept(A)
A ➝ ∅ A ➝ ∅
PreAccept(B)
B ➝ ∅
B ➝ {A}
Accept(B)
B ➝ {A}B ➝ ∅
Commit A ➝ ∅
Friday, November 8, 13
![Page 81: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/81.jpg)
EPaxos commit protocol
12
R1
R2
R3
R4
R5
PreAccept(A)
A ➝ ∅ A ➝ ∅
PreAccept(B)
B ➝ ∅
B ➝ {A}
Accept(B)
B ➝ {A}B ➝ ∅ ACK
Commit A ➝ ∅
Friday, November 8, 13
![Page 82: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/82.jpg)
EPaxos commit protocol
12
R1
R2
R3
R4
R5
PreAccept(A)
A ➝ ∅ A ➝ ∅
PreAccept(B)
B ➝ ∅
B ➝ {A}
Accept(B)
B ➝ {A}B ➝ ∅ ACK
Commit A ➝ ∅
Commit B ➝ {A}
Friday, November 8, 13
![Page 83: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/83.jpg)
EPaxos commit protocol
12
R1
R2
R3
R4
R5
PreAccept(A)
A ➝ ∅ A ➝ ∅
PreAccept(B)
B ➝ ∅
B ➝ {A}
Accept(B)
B ➝ {A}B ➝ ∅ ACK
Commit A ➝ ∅
Commit B ➝ {A}
Friday, November 8, 13
![Page 84: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/84.jpg)
EPaxos commit protocol
12
R1
R2
R3
R4
R5
PreAccept(A)
A ➝ ∅ A ➝ ∅
PreAccept(B)
B ➝ ∅
B ➝ {A}
Accept(B)
B ➝ {A}B ➝ ∅ ACK
Commit A ➝ ∅
Commit B ➝ {A}
PreAccept(C)
C ➝ {A}
Friday, November 8, 13
![Page 85: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/85.jpg)
EPaxos commit protocol
12
R1
R2
R3
R4
R5
PreAccept(A)
A ➝ ∅ A ➝ ∅
PreAccept(B)
B ➝ ∅
B ➝ {A}
Accept(B)
B ➝ {A}B ➝ ∅ ACK
Commit A ➝ ∅
Commit B ➝ {A}
PreAccept(C)
C ➝ {A} C ➝ {A, B}
Friday, November 8, 13
![Page 86: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/86.jpg)
EPaxos commit protocol
12
R1
R2
R3
R4
R5
PreAccept(A)
A ➝ ∅ A ➝ ∅
PreAccept(B)
B ➝ ∅
B ➝ {A}
Accept(B)
B ➝ {A}B ➝ ∅ ACK
Commit A ➝ ∅
Commit B ➝ {A}
PreAccept(C)
C ➝ {A} C ➝ {A, B}
Commit C ➝ {A, B}
Friday, November 8, 13
![Page 87: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/87.jpg)
EPaxos commit protocol
12
R1
R2
R3
R4
R5
PreAccept(A)
A ➝ ∅ A ➝ ∅
PreAccept(B)
B ➝ ∅
B ➝ {A}
Accept(B)
B ➝ {A}B ➝ ∅ ACK
Commit A ➝ ∅
Commit B ➝ {A}
PreAccept(C)
C ➝ {A} C ➝ {A, B}
Commit C ➝ {A, B}
Friday, November 8, 13
![Page 88: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/88.jpg)
Order only interfering commands• 1 RTT
• Non-concurrent commands
• OR non-interfering commands
• 2 RTTs
• Concurrent AND interfering
13
Friday, November 8, 13
![Page 89: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/89.jpg)
Interference is application-specific
14
Friday, November 8, 13
![Page 90: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/90.jpg)
Interference is application-specificKV store
• Infer from operation key
14
Friday, November 8, 13
![Page 91: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/91.jpg)
Interference is application-specificKV store
• Infer from operation key
Google App Engine
• Programmer-specified
14
Friday, November 8, 13
![Page 92: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/92.jpg)
Interference is application-specificKV store
• Infer from operation key
Google App Engine
• Programmer-specified
Relational databases
• Most transactions are simple, can be analyzed
• Few remaining transactions interfere w/ everything
14
Friday, November 8, 13
![Page 93: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/93.jpg)
Execution
15
A
B
D
C
E
Friday, November 8, 13
![Page 94: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/94.jpg)
Execution
15
A
B
D
C
E
strongly-connected component
Friday, November 8, 13
![Page 95: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/95.jpg)
Execution
15
A
B
D
C
E
strongly-connected component
Friday, November 8, 13
![Page 96: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/96.jpg)
Execution
15
A
B
D
C
E
strongly-connected component
Friday, November 8, 13
![Page 97: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/97.jpg)
Execution
15
A
B
D
C
E
E
D
B, C, A
1
2
3
strongly-connected component
Friday, November 8, 13
![Page 98: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/98.jpg)
Execution
15
A
B
D
C
E
E
D
B, C, A
1
2
3
strongly-connected component
Approximate sequence # order(Lamport clock)
Friday, November 8, 13
![Page 99: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/99.jpg)
EPaxos properties
16
Friday, November 8, 13
![Page 100: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/100.jpg)
EPaxos propertiesLinearizability: If A~B, and A committed before B proposed
then A will be executed before B.
16
Friday, November 8, 13
![Page 101: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/101.jpg)
EPaxos propertiesLinearizability: If A~B, and A committed before B proposed
then A will be executed before B.
16
Fast-path quorum: F +⎡F / 2⎤• Optimal for 3 and 5 replicas
• Better than Fast / Generalized Paxos by 1
Friday, November 8, 13
![Page 102: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/102.jpg)
EPaxos propertiesLinearizability: If A~B, and A committed before B proposed
then A will be executed before B.
16
Fast-path quorum: F +⎡F / 2⎤• Optimal for 3 and 5 replicas
• Better than Fast / Generalized Paxos by 1
Friday, November 8, 13
![Page 103: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/103.jpg)
EPaxos propertiesLinearizability: If A~B, and A committed before B proposed
then A will be executed before B.
16
Fast-path quorum: F +⎡F / 2⎤• Optimal for 3 and 5 replicas
• Better than Fast / Generalized Paxos by 1
Friday, November 8, 13
![Page 104: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/104.jpg)
EPaxos propertiesLinearizability: If A~B, and A committed before B proposed
then A will be executed before B.
16
Fast-path quorum: F +⎡F / 2⎤• Optimal for 3 and 5 replicas
• Better than Fast / Generalized Paxos by 1
Friday, November 8, 13
![Page 105: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/105.jpg)
EPaxos propertiesLinearizability: If A~B, and A committed before B proposed
then A will be executed before B.
16
Fast-path quorum: F +⎡F / 2⎤• Optimal for 3 and 5 replicas
• Better than Fast / Generalized Paxos by 1
Friday, November 8, 13
![Page 106: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/106.jpg)
Results
17
Friday, November 8, 13
![Page 107: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/107.jpg)
Optimal wide-area commit latency
18
Friday, November 8, 13
![Page 108: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/108.jpg)
Optimal wide-area commit latency
18
Friday, November 8, 13
![Page 109: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/109.jpg)
Optimal wide-area commit latency
18
Friday, November 8, 13
![Page 110: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/110.jpg)
Optimal wide-area commit latency
18
Friday, November 8, 13
![Page 111: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/111.jpg)
Optimal wide-area commit latency
18
Friday, November 8, 13
![Page 112: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/112.jpg)
Optimal wide-area commit latency
18
Friday, November 8, 13
![Page 113: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/113.jpg)
85ms90ms130ms
300ms150ms
Optimal wide-area commit latency
18
Friday, November 8, 13
![Page 114: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/114.jpg)
EPaxos
Mencius
Generalized Paxos
Multi-Paxos (CA leader)
0 50 100 150 200 250
Median Commit Latency [ms]
85ms90ms130ms
300ms150ms
Optimal wide-area commit latency
18
Friday, November 8, 13
![Page 115: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/115.jpg)
EPaxos
Mencius
Generalized Paxos
Multi-Paxos (CA leader)
0 50 100 150 200 250
Median Commit Latency [ms]
CA
85ms90ms130ms
300ms150ms
Optimal wide-area commit latency
18
Friday, November 8, 13
![Page 116: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/116.jpg)
VA
EPaxos
Mencius
Generalized Paxos
Multi-Paxos (CA leader)
0 50 100 150 200 250
Median Commit Latency [ms]
CA
85ms90ms130ms
300ms150ms
Optimal wide-area commit latency
18
Friday, November 8, 13
![Page 117: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/117.jpg)
VA OR
EPaxos
Mencius
Generalized Paxos
Multi-Paxos (CA leader)
0 50 100 150 200 250
Median Commit Latency [ms]
CA
85ms90ms130ms
300ms150ms
Optimal wide-area commit latency
18
Friday, November 8, 13
![Page 118: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/118.jpg)
VA OR
EPaxos
Mencius
Generalized Paxos
Multi-Paxos (CA leader)
0 50 100 150 200 250
Median Commit Latency [ms]
JPCA
85ms90ms130ms
300ms150ms
Optimal wide-area commit latency
18
Friday, November 8, 13
![Page 119: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/119.jpg)
VA OR
EPaxos
Mencius
Generalized Paxos
Multi-Paxos (CA leader)
0 50 100 150 200 250
Median Commit Latency [ms]
EUJPCA
85ms90ms130ms
300ms150ms
Optimal wide-area commit latency
18
Friday, November 8, 13
![Page 120: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/120.jpg)
VA OR
EPaxos
Mencius
Generalized Paxos
Multi-Paxos (CA leader)
0 50 100 150 200 250
Median Commit Latency [ms]
EUJPCA
85ms90ms130ms
300ms150ms
Optimal wide-area commit latency
18
EPaxos: Optimal commit latency in wide-areafor 3 and 5 replicas
Friday, November 8, 13
![Page 121: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/121.jpg)
Operations / secOperations / sec
Multi-PaxosMencius
EPaxos 0%EPaxos 2%
EPaxos 100%
0 10000 20000 30000 40000 50000 60000
Higher + more stable throughput
19
0 10000 20000 30000 40000 50000 60000
5 replicas 3 replicas
Friday, November 8, 13
![Page 122: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/122.jpg)
Operations / secOperations / sec
Multi-PaxosMencius
EPaxos 0%EPaxos 100%
0 10000 20000 30000 40000 50000 60000
Multi-PaxosMencius
EPaxos 0%EPaxos 2%
EPaxos 100%
0 10000 20000 30000 40000 50000 60000
Higher + more stable throughput
19
0 10000 20000 30000 40000 50000 60000
5 replicas 3 replicas
0 10000 20000 30000 40000 50000 60000
When one replica is slow
Operations / secOperations / sec
Friday, November 8, 13
![Page 123: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/123.jpg)
EPaxos: higher throughput w/ batching
20
Latency[ms]
log scale
5 ms batching, local area
Multi-PaxosEPaxos 0%EPaxos 100%
Throughput (ops / sec)
Friday, November 8, 13
![Page 124: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/124.jpg)
EPaxos: higher throughput w/ batching
20
Latency[ms]
log scale
5 ms batching, local area
0 100000 200000 300000 400000 5000001
10
100
1000
10000Multi-PaxosEPaxos 0%EPaxos 100%
Throughput (ops / sec)
Friday, November 8, 13
![Page 125: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/125.jpg)
EPaxos: higher throughput w/ batching
20
Latency[ms]
log scale
5 ms batching, local area
0 100000 200000 300000 400000 5000001
10
100
1000
10000Multi-PaxosEPaxos 0%EPaxos 100%
Throughput (ops / sec)
Friday, November 8, 13
![Page 126: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/126.jpg)
EPaxos: higher throughput w/ batching
20
Latency[ms]
log scale
5 ms batching, local area
0 100000 200000 300000 400000 5000001
10
100
1000
10000Multi-PaxosEPaxos 0%EPaxos 100%
Throughput (ops / sec)
Friday, November 8, 13
![Page 127: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/127.jpg)
EPaxos: higher throughput w/ batching
20
Latency[ms]
log scale
5 ms batching, local area
0 100000 200000 300000 400000 5000001
10
100
1000
10000Multi-PaxosEPaxos 0%EPaxos 100%
Throughput (ops / sec)
Friday, November 8, 13
![Page 128: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/128.jpg)
Time [sec]
0 10000 20000 30000
0 10000 20000 30000
0
10000
20000
30000
0 5 10 15 20 25 30 35
Constant availability
21
Committhroughput[ops / sec]
leader failure
replica failure
Multi-Paxos
Mencius
EPaxosreplica failure
delayed commits
Friday, November 8, 13
![Page 129: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/129.jpg)
Time [sec]
0 10000 20000 30000
0 10000 20000 30000
0
10000
20000
30000
0 5 10 15 20 25 30 35
Constant availability
21
Committhroughput[ops / sec]
leader failure
replica failure
Multi-Paxos
Mencius
EPaxosreplica failure
delayed commits
Friday, November 8, 13
![Page 130: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/130.jpg)
Time [sec]
0 10000 20000 30000
0 10000 20000 30000
0
10000
20000
30000
0 5 10 15 20 25 30 35
Constant availability
21
Committhroughput[ops / sec]
leader failure
replica failure
Multi-Paxos
Mencius
EPaxosreplica failure
delayed commits
Friday, November 8, 13
![Page 131: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/131.jpg)
EPaxos insights
22
Friday, November 8, 13
![Page 132: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/132.jpg)
EPaxos insights
22
Order commands explicitly
Friday, November 8, 13
![Page 133: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/133.jpg)
EPaxos insights
22
Order commands explicitly
High throughput
Friday, November 8, 13
![Page 134: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/134.jpg)
EPaxos insights
22
Order commands explicitly
High throughput Stability
Friday, November 8, 13
![Page 135: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/135.jpg)
EPaxos insights
22
Order commands explicitly
High throughput Stability Low latency
Friday, November 8, 13
![Page 136: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/136.jpg)
EPaxos insights
22
Order commands explicitly
High throughput Stability Low latency
Optimize only delays that matter(clients co-located w/ closest replica)
Friday, November 8, 13
![Page 137: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/137.jpg)
EPaxos insights
22
Order commands explicitly
High throughput Stability Low latency
Optimize only delays that matter(clients co-located w/ closest replica)
Smaller quorums
Friday, November 8, 13
![Page 138: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/138.jpg)
EPaxos insights
22
Order commands explicitly
High throughput Stability Low latency
Optimize only delays that matter(clients co-located w/ closest replica)
Smaller quorums
Friday, November 8, 13
![Page 139: There Is More Consensus in Egalitarian Parliaments€¦ · Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Friday, November 8, 13. Fault tolerance](https://reader036.vdocument.in/reader036/viewer/2022071402/60efd4bb2f1bce5a07591984/html5/thumbnails/139.jpg)
23
TLA+ Spec
http://cs.cmu.edu/~imoraru/epaxos/tr.pdf
Formal Proof
Open Source Release
http://github.com/efficient/epaxos
Friday, November 8, 13