eiger : stronger semantics for low- latency geo -replicated storage
DESCRIPTION
Eiger : Stronger Semantics for Low- Latency Geo -Replicated Storage. Wyatt Lloyd * Michael J. Freedman * Michael Kaminsky † David G. Andersen ‡ * Princeton, † Intel Labs, ‡ CMU. Geo-Replicated Storage. is the backend of massive websites. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Eiger : Stronger Semantics for Low- Latency Geo -Replicated Storage](https://reader036.vdocument.in/reader036/viewer/2022062315/568161e7550346895dd20ff5/html5/thumbnails/1.jpg)
Eiger: Stronger Semantics for
Low-LatencyGeo-Replicated
Storage Wyatt Lloyd*
Michael J. Freedman*
Michael Kaminsky†
David G. Andersen‡
*Princeton, †Intel Labs, ‡CMU
![Page 2: Eiger : Stronger Semantics for Low- Latency Geo -Replicated Storage](https://reader036.vdocument.in/reader036/viewer/2022062315/568161e7550346895dd20ff5/html5/thumbnails/2.jpg)
Geo-Replicated Storage
2
is the backend of massive websites
FriendOf
FriendOf
Like“Halting is
Undecidable”
Status
![Page 3: Eiger : Stronger Semantics for Low- Latency Geo -Replicated Storage](https://reader036.vdocument.in/reader036/viewer/2022062315/568161e7550346895dd20ff5/html5/thumbnails/3.jpg)
3
Storage Dimensions
A-F
G-L
M-R
S-Z
Shard Data Across Many Nodes
Like
“Halting is Undecidable”
Status
FriendOf FriendOf
![Page 4: Eiger : Stronger Semantics for Low- Latency Geo -Replicated Storage](https://reader036.vdocument.in/reader036/viewer/2022062315/568161e7550346895dd20ff5/html5/thumbnails/4.jpg)
4
Storage Dimensions
A-F
G-L
M-R
S-Z
Shard Data Across Many Nodes
A-F
G-L
M-R
S-Z
A-F
G-L
M-R
S-Z
Data Geo-Replicated In Multiple
Datacenters
![Page 5: Eiger : Stronger Semantics for Low- Latency Geo -Replicated Storage](https://reader036.vdocument.in/reader036/viewer/2022062315/568161e7550346895dd20ff5/html5/thumbnails/5.jpg)
5
Sharded, Geo-Replicated Storage
A-F
G-L
M-R
S-Z
A-F
G-L
M-R
S-Z
A-F
G-L
M-R
S-Z
![Page 6: Eiger : Stronger Semantics for Low- Latency Geo -Replicated Storage](https://reader036.vdocument.in/reader036/viewer/2022062315/568161e7550346895dd20ff5/html5/thumbnails/6.jpg)
Low Latency– Improves user experience– Correlates with revenue
Strong Consistency– Obey user expectations– Easier for programmers
Fundamentally in Conflict[LiptonSandberg88,
AttiyaWelch94]
Strong Consistency or Low Latency
![Page 7: Eiger : Stronger Semantics for Low- Latency Geo -Replicated Storage](https://reader036.vdocument.in/reader036/viewer/2022062315/568161e7550346895dd20ff5/html5/thumbnails/7.jpg)
Strong Consistency or Low Latency
Megastore [SIGMOD ‘08]
Spanner [OSDI ‘12] .
Gemini [OSDI ‘12] .
Walter [SOSP ’11] .
Dynamo [SOSP ’07]
COPS [SOSP ’11]
Eiger Obey user expectations Easier for programmersCausal+ Consistency
Rich Data ModelRead-only TxnsWrite-only Txns
![Page 8: Eiger : Stronger Semantics for Low- Latency Geo -Replicated Storage](https://reader036.vdocument.in/reader036/viewer/2022062315/568161e7550346895dd20ff5/html5/thumbnails/8.jpg)
8
Eiger Ensures Low Latency
A-F
G-L
M-R
S-Z
A-F
G-L
M-R
S-Z
A-F
G-L
M-R
S-Z
Keep All Ops Local
![Page 9: Eiger : Stronger Semantics for Low- Latency Geo -Replicated Storage](https://reader036.vdocument.in/reader036/viewer/2022062315/568161e7550346895dd20ff5/html5/thumbnails/9.jpg)
Causal+ Consistency Across DCs• If A happens before B– Everyone sees A before B
• Obeys user expectations
• Simplifies programming
New Job!
FriendsBossThen
Then
Then
![Page 10: Eiger : Stronger Semantics for Low- Latency Geo -Replicated Storage](https://reader036.vdocument.in/reader036/viewer/2022062315/568161e7550346895dd20ff5/html5/thumbnails/10.jpg)
Causal For Column Families
• Operations update/read many columns• Range query columns concurrent w/ deletes• Counter columns• See paper for details
Lovelace
Turing
ChurchLovelaceTuring
- - 1/1/54
9/1/36 1/1/54 -
Friends
631
457
Friends CountProfile
Age Town
197 London
100 Princeton
Key1
Key2
Val
Val
![Page 11: Eiger : Stronger Semantics for Low- Latency Geo -Replicated Storage](https://reader036.vdocument.in/reader036/viewer/2022062315/568161e7550346895dd20ff5/html5/thumbnails/11.jpg)
Viewing Data Consistently Is HardAsynchronous requests + distributed data = ?????
A
B
C
156
Update A
2
Update B
3
Update C
4???
![Page 12: Eiger : Stronger Semantics for Low- Latency Geo -Replicated Storage](https://reader036.vdocument.in/reader036/viewer/2022062315/568161e7550346895dd20ff5/html5/thumbnails/12.jpg)
Read-Only Transactions• Logical time gives a global view of data store
– Clocks on all nodes, carried with all messages
• Insight: Store is consistent at all logical times
Logical Time
A 0 2
B 0 3
C 0 5
A1 A2
B1 B2
C1 C2
![Page 13: Eiger : Stronger Semantics for Low- Latency Geo -Replicated Storage](https://reader036.vdocument.in/reader036/viewer/2022062315/568161e7550346895dd20ff5/html5/thumbnails/13.jpg)
13
Read-Only Transactions• Extract consistent up-to-date view of data– Across many servers
• Challenges– Scalability
• Decentralized algorithm– Guaranteed low latency
• At most 2 parallel rounds of local reads• No locks, no blocking
– High performance• Normal case: 1 round of reads
![Page 14: Eiger : Stronger Semantics for Low- Latency Geo -Replicated Storage](https://reader036.vdocument.in/reader036/viewer/2022062315/568161e7550346895dd20ff5/html5/thumbnails/14.jpg)
Read-Only Transactions• Round 1: Optimistic parallel reads• Calculate effective time
• Round 2: Parallel read_at_times
Logical Time
A 0 2
B 0 3
C 0 5
A1 A2
B1 B2
C1 C2
Distributed StorageClient0 1A1
4 6C2
3 B25A2
![Page 15: Eiger : Stronger Semantics for Low- Latency Geo -Replicated Storage](https://reader036.vdocument.in/reader036/viewer/2022062315/568161e7550346895dd20ff5/html5/thumbnails/15.jpg)
Logical Time
A 2
B 3
C 5
A2
B2
C2
Transaction Intuition• Read-only transactions
– Read from a single logical time
• Write-only transactions– Appear at a single logical time
Bonus:Works for
Linearizability
A3
B3
C3
![Page 16: Eiger : Stronger Semantics for Low- Latency Geo -Replicated Storage](https://reader036.vdocument.in/reader036/viewer/2022062315/568161e7550346895dd20ff5/html5/thumbnails/16.jpg)
16
Eiger Provides√ Low latency√ Rich data model√ Causal+ consistency√ Read-only transactions√ Write-only transactions But what does all this cost? Does it scale?
![Page 17: Eiger : Stronger Semantics for Low- Latency Geo -Replicated Storage](https://reader036.vdocument.in/reader036/viewer/2022062315/568161e7550346895dd20ff5/html5/thumbnails/17.jpg)
Eiger Implementation• Fork of open-source Cassandra
• +5K lines of Java to Cassandra’s 75K
• Code Available:– https://github.com/wlloyd/eiger
![Page 18: Eiger : Stronger Semantics for Low- Latency Geo -Replicated Storage](https://reader036.vdocument.in/reader036/viewer/2022062315/568161e7550346895dd20ff5/html5/thumbnails/18.jpg)
Evaluation• Cost of stronger consistency &
semantics – Vs. eventually-consistent Cassandra– Overhead for real (Facebook) workload– Overhead for state-space of workloads
• Scalability
![Page 19: Eiger : Stronger Semantics for Low- Latency Geo -Replicated Storage](https://reader036.vdocument.in/reader036/viewer/2022062315/568161e7550346895dd20ff5/html5/thumbnails/19.jpg)
19
A-F
G-L
M-R
S-Z
A-F
G-L
M-R
S-Z
Replication
Experimental SetupLocal Datacenter (Stanford)
Remote DC (UW)
8 8
8
![Page 20: Eiger : Stronger Semantics for Low- Latency Geo -Replicated Storage](https://reader036.vdocument.in/reader036/viewer/2022062315/568161e7550346895dd20ff5/html5/thumbnails/20.jpg)
20
Facebook Workload Results
6.6% Overhead
![Page 21: Eiger : Stronger Semantics for Low- Latency Geo -Replicated Storage](https://reader036.vdocument.in/reader036/viewer/2022062315/568161e7550346895dd20ff5/html5/thumbnails/21.jpg)
21
Eiger ScalesFacebook Workload
384 Machines!
Scales out
![Page 22: Eiger : Stronger Semantics for Low- Latency Geo -Replicated Storage](https://reader036.vdocument.in/reader036/viewer/2022062315/568161e7550346895dd20ff5/html5/thumbnails/22.jpg)
Improving Low-Latency Storage COPS
EigerData model Key-Value Column-
Family
Read-only Txns Causal stores All stores
Write-only Txns None Yes
Performance Good
Great
DC Failure Throughput
Resilient degradation
![Page 23: Eiger : Stronger Semantics for Low- Latency Geo -Replicated Storage](https://reader036.vdocument.in/reader036/viewer/2022062315/568161e7550346895dd20ff5/html5/thumbnails/23.jpg)
Eiger• Low-latency geo-replicated storage– Causal+ for column families– Read-only transactions–Write-only transactions
• Demonstrated in working system– Competitive with eventual– Scales to large clusters– https://github.com/wlloyd/eiger
![Page 24: Eiger : Stronger Semantics for Low- Latency Geo -Replicated Storage](https://reader036.vdocument.in/reader036/viewer/2022062315/568161e7550346895dd20ff5/html5/thumbnails/24.jpg)
Eiger: Stronger Semantics for
Low-LatencyGeo-Replicated
Storage Wyatt Lloyd*
Michael J. Freedman*
Michael Kaminsky†
David G. Andersen‡
*Princeton, †Intel Labs, ‡CMU