murex: a mutable replica control scheme for structured peer-to-peer storage systems
DESCRIPTION
MUREX: A Mutable Replica Control Scheme for Structured Peer-to-Peer Storage Systems. Presented by Jehn-Ruey Jiang National Central University Taiwan, R. O. C. Outline. P2P Storage Systems The Problems MUREX Analysis and Simulation Conclusion. Outline. P2P Storage Systems - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: MUREX: A Mutable Replica Control Scheme for Structured Peer-to-Peer Storage Systems](https://reader035.vdocument.in/reader035/viewer/2022062722/56813b21550346895da3d696/html5/thumbnails/1.jpg)
MUREX: A Mutable Replica Control Scheme for StructuredPeer-to-Peer Storage Systems
Presented by
Jehn-Ruey JiangNational Central UniversityTaiwan, R. O. C.
![Page 2: MUREX: A Mutable Replica Control Scheme for Structured Peer-to-Peer Storage Systems](https://reader035.vdocument.in/reader035/viewer/2022062722/56813b21550346895da3d696/html5/thumbnails/2.jpg)
2/40
Outline
P2P Storage Systems The Problems MUREX Analysis and Simulation Conclusion
![Page 3: MUREX: A Mutable Replica Control Scheme for Structured Peer-to-Peer Storage Systems](https://reader035.vdocument.in/reader035/viewer/2022062722/56813b21550346895da3d696/html5/thumbnails/3.jpg)
3/40
Outline
P2P Storage Systems The Problems MUREX Analysis and Simulation Conclusion
![Page 4: MUREX: A Mutable Replica Control Scheme for Structured Peer-to-Peer Storage Systems](https://reader035.vdocument.in/reader035/viewer/2022062722/56813b21550346895da3d696/html5/thumbnails/4.jpg)
4/40
P2P Storage Systems
To aggregate idle storage across the Internet to be a huge storage space
Towards Global Storage SystemsMassive NodesMassive Capacity
![Page 5: MUREX: A Mutable Replica Control Scheme for Structured Peer-to-Peer Storage Systems](https://reader035.vdocument.in/reader035/viewer/2022062722/56813b21550346895da3d696/html5/thumbnails/5.jpg)
5/40
Unstructured vs. Structured
Unstructured:No restriction on the interconnection of nodesEasy to build but not scalable
Structured:Based on DHTs (Distributed Hash Tables)More scalable
Our Focus!!
![Page 6: MUREX: A Mutable Replica Control Scheme for Structured Peer-to-Peer Storage Systems](https://reader035.vdocument.in/reader035/viewer/2022062722/56813b21550346895da3d696/html5/thumbnails/6.jpg)
6/40
Non-Mutable vs. Mutable
Non-Mutable (Read-only):CFSPASTCharles
Mutable: IvyEliotOasisOm
Our Focus!!
![Page 7: MUREX: A Mutable Replica Control Scheme for Structured Peer-to-Peer Storage Systems](https://reader035.vdocument.in/reader035/viewer/2022062722/56813b21550346895da3d696/html5/thumbnails/7.jpg)
7/40
Replication
Data objects are replicated for the purpose of fault-tolerance
Some DHTs have provided replication utilities, which are usually used to replicate routing states
The proposed protocol replicates data objects in the application layer so that it can be built on top of any DHT
high data availability
![Page 8: MUREX: A Mutable Replica Control Scheme for Structured Peer-to-Peer Storage Systems](https://reader035.vdocument.in/reader035/viewer/2022062722/56813b21550346895da3d696/html5/thumbnails/8.jpg)
8/40
One-Copy Equivalence
Data consistency Criterion The set of replicas must behave as if there were
only a single copy Conditions:
1. no pair of write operations can proceed at the same time,
2. no pair of a read operation and a write operation can proceed at the same time,
3. a read operation always returns the replica that the last write operation writes.
![Page 9: MUREX: A Mutable Replica Control Scheme for Structured Peer-to-Peer Storage Systems](https://reader035.vdocument.in/reader035/viewer/2022062722/56813b21550346895da3d696/html5/thumbnails/9.jpg)
9/40
Synchronous vs. Asynchronous Synchronous Replication
Each write operation should finish updating all replicas before the next write operation proceeds.
Strict data consistency Long operation latency
Asynchronous Replication A write operation is written to the local replica; data object is then
asynchronously written to other replicas. May violate data consistency Shorter latency Log-based mechanisms to roll back the system
Our Focus
![Page 10: MUREX: A Mutable Replica Control Scheme for Structured Peer-to-Peer Storage Systems](https://reader035.vdocument.in/reader035/viewer/2022062722/56813b21550346895da3d696/html5/thumbnails/10.jpg)
10/40
Fault Models
Fail-StopNodes just stop functioning when they fail
Crash-RecoveryFailures are detectableNodes can recover and rejoin the system after
state synchronization Byzantine
Nodes may act arbitrary
![Page 11: MUREX: A Mutable Replica Control Scheme for Structured Peer-to-Peer Storage Systems](https://reader035.vdocument.in/reader035/viewer/2022062722/56813b21550346895da3d696/html5/thumbnails/11.jpg)
11/40
Outline
P2P Storage Systems The Problems MUREX Analysis and Simulation Conclusion
![Page 12: MUREX: A Mutable Replica Control Scheme for Structured Peer-to-Peer Storage Systems](https://reader035.vdocument.in/reader035/viewer/2022062722/56813b21550346895da3d696/html5/thumbnails/12.jpg)
12/40
Three Problems
Replica migration
Replica acquisition
State synchronization
![Page 13: MUREX: A Mutable Replica Control Scheme for Structured Peer-to-Peer Storage Systems](https://reader035.vdocument.in/reader035/viewer/2022062722/56813b21550346895da3d696/html5/thumbnails/13.jpg)
13/40
DHT – Node Joining
0 2128-1
PeerNodes
Hash Function
Hashed Key Space
node joining
vu
ks
ssData Object
Replica Migration
![Page 14: MUREX: A Mutable Replica Control Scheme for Structured Peer-to-Peer Storage Systems](https://reader035.vdocument.in/reader035/viewer/2022062722/56813b21550346895da3d696/html5/thumbnails/14.jpg)
14/40
DHT – Node Leaving
0 2128-1
PeerNodes
Hash Function
Hashed Key Space
node leaving
qp
Data Object
kr
rr
Data Objectrrrr
State Synchronization
Replica Acquisition
![Page 15: MUREX: A Mutable Replica Control Scheme for Structured Peer-to-Peer Storage Systems](https://reader035.vdocument.in/reader035/viewer/2022062722/56813b21550346895da3d696/html5/thumbnails/15.jpg)
15/40
Outline
P2P Storage Systems The Problems MUREX Analysis and Simulation Conclusion
![Page 16: MUREX: A Mutable Replica Control Scheme for Structured Peer-to-Peer Storage Systems](https://reader035.vdocument.in/reader035/viewer/2022062722/56813b21550346895da3d696/html5/thumbnails/16.jpg)
16/40
The Solution - MUREX
A mutable replica control scheme
Keeping one-copy equivalence for synchronous P2P storage replication under the crash-recovery fault model
Based on Multi-column read/write quorums
![Page 17: MUREX: A Mutable Replica Control Scheme for Structured Peer-to-Peer Storage Systems](https://reader035.vdocument.in/reader035/viewer/2022062722/56813b21550346895da3d696/html5/thumbnails/17.jpg)
17/40
Operations
Publish(CON, DON)CON: Standing for CONtentDON: Standing for Data Object Name
Read(DON) Write(CON, DON)
![Page 18: MUREX: A Mutable Replica Control Scheme for Structured Peer-to-Peer Storage Systems](https://reader035.vdocument.in/reader035/viewer/2022062722/56813b21550346895da3d696/html5/thumbnails/18.jpg)
18/40
Synchronous Replication
n replicas for each data objectK1=HASH1(Data Object Name), …, Kn=HASH
n(Data Object Name)
Using read/write quorums to maintain data consistency (one-copy equivalence)
![Page 19: MUREX: A Mutable Replica Control Scheme for Structured Peer-to-Peer Storage Systems](https://reader035.vdocument.in/reader035/viewer/2022062722/56813b21550346895da3d696/html5/thumbnails/19.jpg)
19/40
0 2128-1
PeerNodes
…Hash Function 1 Hash Function 2 Hash Function n
Data Objectreplica nreplica 1
Hashed Key Space
replica 2
k1k2 kn
rrrr
Data Replication
![Page 20: MUREX: A Mutable Replica Control Scheme for Structured Peer-to-Peer Storage Systems](https://reader035.vdocument.in/reader035/viewer/2022062722/56813b21550346895da3d696/html5/thumbnails/20.jpg)
20/40
Quorum-Based Schemes (1/2)
High data availability and low communication cost
n replicas with version numbers Read operation
Read-lock and access a read quorumObtaining a largest-version-number replica
Write operationWrite-lock and access a write quorumUpdating all replicas with the new version number
the largest+ 1
![Page 21: MUREX: A Mutable Replica Control Scheme for Structured Peer-to-Peer Storage Systems](https://reader035.vdocument.in/reader035/viewer/2022062722/56813b21550346895da3d696/html5/thumbnails/21.jpg)
21/40
Quorum-Based Schemes (2/2)
One-copy equivalence is guaranteed If we restrictWrite-write and write-read lock exclusionIntersection Property
A non-empty intersection in any pair ofA read quorum and a write quorumTwo write quorums
![Page 22: MUREX: A Mutable Replica Control Scheme for Structured Peer-to-Peer Storage Systems](https://reader035.vdocument.in/reader035/viewer/2022062722/56813b21550346895da3d696/html5/thumbnails/22.jpg)
22/40
Multi-Column Quorums
Smallest quorums: constant-sized quorums in the best caseSmaller quorums imply lower communication
cost May achieve the highest data availability
![Page 23: MUREX: A Mutable Replica Control Scheme for Structured Peer-to-Peer Storage Systems](https://reader035.vdocument.in/reader035/viewer/2022062722/56813b21550346895da3d696/html5/thumbnails/23.jpg)
23/40
Messages
LOCK (WLOCK/RLOCK) OK WAIT MISS UNLOCK
![Page 24: MUREX: A Mutable Replica Control Scheme for Structured Peer-to-Peer Storage Systems](https://reader035.vdocument.in/reader035/viewer/2022062722/56813b21550346895da3d696/html5/thumbnails/24.jpg)
24/40
Algorithms for Quorum Construction
![Page 25: MUREX: A Mutable Replica Control Scheme for Structured Peer-to-Peer Storage Systems](https://reader035.vdocument.in/reader035/viewer/2022062722/56813b21550346895da3d696/html5/thumbnails/25.jpg)
25/40
Three Mechanisms
Replica pointer
On-demand replica regeneration
Leased lock
![Page 26: MUREX: A Mutable Replica Control Scheme for Structured Peer-to-Peer Storage Systems](https://reader035.vdocument.in/reader035/viewer/2022062722/56813b21550346895da3d696/html5/thumbnails/26.jpg)
26/40
Replica pointer
A lightweight mechanism to emigrate replicas
A five-tuple:(hashed key, data object name, version number, lock state, actual storing location)
It is produced when a replica is first generated.
It is moved between nodes instead of the actual data object,
![Page 27: MUREX: A Mutable Replica Control Scheme for Structured Peer-to-Peer Storage Systems](https://reader035.vdocument.in/reader035/viewer/2022062722/56813b21550346895da3d696/html5/thumbnails/27.jpg)
27/40
On-demand replica regeneration (1/2)
When node p receives LOCK from node u, it sends a MISS if itdoes not have the replica pointerhas the replica pointer which indicates that v
stores the replica, but v is not alive After executing the desired read/write
operation, node u will send the newest replica obtained/generated to node p
![Page 28: MUREX: A Mutable Replica Control Scheme for Structured Peer-to-Peer Storage Systems](https://reader035.vdocument.in/reader035/viewer/2022062722/56813b21550346895da3d696/html5/thumbnails/28.jpg)
28/40
On-demand replica regeneration (2/2)
Acquiring replicas only when they are requested
Dummy read operationPerformed periodically for rarely-accessed
data objectTo check if replicas of data object are still
aliveTo re-disseminate replicas to proper nodes to
keep data persistency
![Page 29: MUREX: A Mutable Replica Control Scheme for Structured Peer-to-Peer Storage Systems](https://reader035.vdocument.in/reader035/viewer/2022062722/56813b21550346895da3d696/html5/thumbnails/29.jpg)
29/40
Leased lock (1/2) A lock expires after a lease period of L A node should release all locks if it is not in CS
and H>L-C-D holds. H: The holding time of the lock D: The propagation delay C: time to be in CS
![Page 30: MUREX: A Mutable Replica Control Scheme for Structured Peer-to-Peer Storage Systems](https://reader035.vdocument.in/reader035/viewer/2022062722/56813b21550346895da3d696/html5/thumbnails/30.jpg)
30/40
Leased lock (2/2) When releasing all locks, a node starts over
to request locks after a random backoff time If a node starts to substitute another node
at time T, a newly acquired replica can start to reply to LOCK requests at time T+L
![Page 31: MUREX: A Mutable Replica Control Scheme for Structured Peer-to-Peer Storage Systems](https://reader035.vdocument.in/reader035/viewer/2022062722/56813b21550346895da3d696/html5/thumbnails/31.jpg)
31/40
Correctness
Theorem 1. (Safety Property) MUREX ensures the one-copy equivalence co
nsistency criterion Theorem 2. (Liveness Property)
There is neither deadlock nor starvation in MUREX
![Page 32: MUREX: A Mutable Replica Control Scheme for Structured Peer-to-Peer Storage Systems](https://reader035.vdocument.in/reader035/viewer/2022062722/56813b21550346895da3d696/html5/thumbnails/32.jpg)
32/40
Outline
P2P Storage Systems The Problems MUREX Analysis and Simulation Conclusion
![Page 33: MUREX: A Mutable Replica Control Scheme for Structured Peer-to-Peer Storage Systems](https://reader035.vdocument.in/reader035/viewer/2022062722/56813b21550346895da3d696/html5/thumbnails/33.jpg)
33/40
Communication Cost
If no contention In the best case: 3s messages
One LOCK One OK One UNLOCK
When failures occur Communication cost increases gradually
In the worst case: O(n) messages A node sends LOCK message to all n replicas
(there are related UNLOCK, OK, WAIT messages)
s: the size of the last column of multi-column quor
ums
![Page 34: MUREX: A Mutable Replica Control Scheme for Structured Peer-to-Peer Storage Systems](https://reader035.vdocument.in/reader035/viewer/2022062722/56813b21550346895da3d696/html5/thumbnails/34.jpg)
34/40
Simulation Environment
The underlying DHT is TornadoFor quorums under four multi-column structures
MC(5, 3), MC(4, 3), MC(5, 2) and MC(4, 2)For MC(m, s), the leased period is assumed to
be m*(turn-around time)2000 nodes in the systemSimulation for 3000 seconds10000 operations are requested
Half for reading and half for writing Each request is assumed to be destined for a rando
m file (data object)
![Page 35: MUREX: A Mutable Replica Control Scheme for Structured Peer-to-Peer Storage Systems](https://reader035.vdocument.in/reader035/viewer/2022062722/56813b21550346895da3d696/html5/thumbnails/35.jpg)
35/40
Simulation Result 1
1st experiment: no node join or leave
The probability that a node succeeds to perform the desired operation before the leased lock expires
Degree of Contention
![Page 36: MUREX: A Mutable Replica Control Scheme for Structured Peer-to-Peer Storage Systems](https://reader035.vdocument.in/reader035/viewer/2022062722/56813b21550346895da3d696/html5/thumbnails/36.jpg)
36/40
Simulation Result 2
2nd experiment: 200 out of 2000 nodes may join/leave at will
![Page 37: MUREX: A Mutable Replica Control Scheme for Structured Peer-to-Peer Storage Systems](https://reader035.vdocument.in/reader035/viewer/2022062722/56813b21550346895da3d696/html5/thumbnails/37.jpg)
37/40
Simulation Result 3
3rd experiment: 0, 50, 100 or 200 out of 2000 nodes may leave
![Page 38: MUREX: A Mutable Replica Control Scheme for Structured Peer-to-Peer Storage Systems](https://reader035.vdocument.in/reader035/viewer/2022062722/56813b21550346895da3d696/html5/thumbnails/38.jpg)
38/40
Outline
P2P Storage Systems The Problems MUREX Analysis and Simulation Conclusion
![Page 39: MUREX: A Mutable Replica Control Scheme for Structured Peer-to-Peer Storage Systems](https://reader035.vdocument.in/reader035/viewer/2022062722/56813b21550346895da3d696/html5/thumbnails/39.jpg)
39/40
Conclusion Identify three problems for synchronous repli
cation in P2P mutable storage systems Replica migrationReplica acquisitionState synchronization
Propose MUREX to solve the problems byMulti-column read/write quorumsReplica pointerOn-demand replica regenerationLeased lock
![Page 40: MUREX: A Mutable Replica Control Scheme for Structured Peer-to-Peer Storage Systems](https://reader035.vdocument.in/reader035/viewer/2022062722/56813b21550346895da3d696/html5/thumbnails/40.jpg)
40/40
Thanks!!