sylvia ratnasamy (uc berkley dissertation 2002) paul francis mark handley richard karp scott shenker...
TRANSCRIPT
![Page 1: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/1.jpg)
Sylvia Ratnasamy (UC Berkley Dissertation 2002)
Paul Francis
Mark Handley
Richard Karp
Scott Shenker
A Scalable, Content Addressable Network
Slides byAuthors above
with additions/modificatons byMichael Isaacs (UCSB)Tony Hannan (Georgia Tech)
![Page 2: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/2.jpg)
Outline
• Introduction• Design• Design Improvements• Evalution• Related Work• Conclusion
![Page 3: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/3.jpg)
• Hash tables– essential building block in software systems
• Internet-scale distributed hash tables– equally valuable to large-scale distributed systems
• peer-to-peer systems– Napster, Gnutella, Groove, FreeNet, MojoNation…
• large-scale storage management systems– Publius, OceanStore, PAST, Farsite, CFS ...
• mirroring on the Web
Internet-scale hash tables
![Page 4: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/4.jpg)
Content-Addressable Network(CAN)
• CAN: Internet-scale hash table
• Interface– insert (key, value)– value = retrieve (key)
• Properties– scalable– operationally simple– good performance
• Related systems: Chord, Plaxton/Tapestry ...
![Page 5: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/5.jpg)
Problem Scope
• provide the hashtable interface• scalability• robustness• performance
![Page 6: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/6.jpg)
Not in scope
• security• anonymity• higher level primitives
– keyword searching– mutable content
![Page 7: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/7.jpg)
Outline
• Introduction• Design• Design Improvements• Evalution• Related Work• Conclusion
![Page 8: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/8.jpg)
K V
CAN: basic idea
K V
K V
K V
K V
K V
K V
K V
K V
K V
K V
![Page 9: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/9.jpg)
CAN: basic idea
insert(K1,V1)
K V
K V
K V
K V
K V
K V
K V
K V
K V
K V
K V
![Page 10: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/10.jpg)
CAN: basic idea
insert(K1,V1)
K V
K V
K V
K V
K V
K V
K V
K V
K V
K V
K V
![Page 11: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/11.jpg)
CAN: basic idea
(K1,V1)
K V
K V
K V
K V
K V
K V
K V
K V
K V
K V
K V
![Page 12: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/12.jpg)
CAN: basic idea
retrieve (K1)
K V
K V
K V
K V
K V
K V
K V
K V
K V
K V
K V
![Page 13: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/13.jpg)
CAN: solution
• Virtual Cartesian coordinate space
• Entire space is partitioned amongst all the nodes – every node “owns” a zone in the overall space
• Abstraction– can store data at “points” in the space – can route from one “point” to another
• Point = node that owns the enclosing zone
![Page 14: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/14.jpg)
CAN: simple example
1
![Page 15: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/15.jpg)
CAN: simple example
1 2
![Page 16: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/16.jpg)
CAN: simple example
1
2
3
![Page 17: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/17.jpg)
CAN: simple example
1
2
3
4
![Page 18: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/18.jpg)
CAN: simple example
![Page 19: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/19.jpg)
CAN: simple example
I
![Page 20: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/20.jpg)
CAN: simple example
node I::insert(K,V)
I
![Page 21: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/21.jpg)
(1) a = hx(K)
CAN: simple example
x = a
node I::insert(K,V)
I
![Page 22: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/22.jpg)
(1) a = hx(K)
b = hy(K)
CAN: simple example
x = a
y = b
node I::insert(K,V)
I
![Page 23: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/23.jpg)
(1) a = hx(K)
b = hy(K)
CAN: simple example
(2) route(K,V) -> (a,b)
node I::insert(K,V)
I
![Page 24: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/24.jpg)
CAN: simple example
(2) route(K,V) -> (a,b)
(3) (a,b) stores (K,V)
(K,V)
node I::insert(K,V)
I(1) a = hx(K)
b = hy(K)
![Page 25: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/25.jpg)
CAN: simple example
(2) route “retrieve(K)” to (a,b)
(K,V)
(1) a = hx(K)
b = hy(K)
node J ::retrieve(K)
J
![Page 26: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/26.jpg)
Data stored in the CAN is addressed by name (i.e. key), not location (i.e. IP address)
CAN
![Page 27: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/27.jpg)
CAN: routing table
![Page 28: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/28.jpg)
CAN: routing
(a,b)
(x,y)
![Page 29: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/29.jpg)
A node only maintains state for its immediate neighboring nodes
CAN: routing
![Page 30: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/30.jpg)
CAN: node insertion
I
new node1) Discover some node “I” already in CAN
![Page 31: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/31.jpg)
CAN: node insertion
2) pick random point in space
I
(p,q)
new node
![Page 32: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/32.jpg)
CAN: node insertion
(p,q)
3) I routes to (p,q), discovers node J
I
J
new node
![Page 33: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/33.jpg)
CAN: node insertion
newJ
4) split J’s zone in half… new owns one half
![Page 34: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/34.jpg)
Inserting a new node affects only a single other node and its immediate neighbors
CAN: node insertion
![Page 35: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/35.jpg)
CAN: node failures
• Need to repair the space
– recover database• state updates• use replication, rebuild database from replicas
– repair routing • takeover algorithm
![Page 36: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/36.jpg)
CAN: takeover algorithm
• Nodes periodically send update message to neigbors
• An absent update message indicates a failure
– Each node send a recover message to the takeover node
– The takeover node takes over the failed node's zone
![Page 37: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/37.jpg)
CAN: takeover node
• The takeover node is the node whose VID is numerically closest to the failed nodes VID.
• The VID is a binary string indicating the path from the root to the zone in the binary partition tree.
0
11
100 1010 1
10
100 101
11
![Page 38: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/38.jpg)
CAN: VID predecessor / successor
• To handle multiple neighbors failing simultaneously, each node maintains its predecessor and successor in the VID ordering (idea borrowed from Chord).
• This guarantees that the takeover node can always be found.
![Page 39: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/39.jpg)
Only the failed node’s immediate neighbors are required for recovery
CAN: node failures
![Page 40: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/40.jpg)
Basic Design recap
• Basic CAN– completely distributed– self-organizing– nodes only maintain state for their
immediate neighbors
![Page 41: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/41.jpg)
Outline
• Introduction• Design• Design Improvements• Evalution• Related Work• Conclusion
![Page 42: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/42.jpg)
Design Improvement Objectives
• Reduce latency of CAN routing– Reduce CAN path length– Reduce per-hop latency (IP
length/latency per CAN hop)• Improve CAN robustness
– data availability– routing
![Page 43: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/43.jpg)
Design Improvement Tradeoffs
Routing performance and system robustnes
» vs.
Increased per-node state and system complexity
![Page 44: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/44.jpg)
Design Improvement Techniques
1. Increase number of dimensions in coordinate space
2. Multiple coordinate spaces (realities)
3. Better CAN routing metrics (measure & consider time)
4. Overloading coordinate zones (peer nodes)
5. Multiple hash functions
6. Geographic-sensitive construction of CAN overlay network (distributed binning)
7. More uniform partitioning (when new nodes enter)
8. Caching & Replication techniques for “hot-spots”
9. Background load balancing algorithm
![Page 45: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/45.jpg)
1. Increase dimensions
![Page 46: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/46.jpg)
2. Multiple realities
• Multiple (r) coordinate spaces
• Each node maintains a different zone in each reality
• Contents of hash table replicated on each reality
• Routing chooses neighbor closest in any reality (can make large jumps towards target)
• Advantages:– Greater data availability– Fewer hops
![Page 47: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/47.jpg)
Multiple realities
![Page 48: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/48.jpg)
Multiple dimensions vs realities
![Page 49: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/49.jpg)
3. Time-sensitive routing
• Each node measures round-trip time (RTT) to each of its neighbors
• Routing message is forwarded to the neighbor with the maximum ratio of progress to RTT
• Advantage:– reduces per-hop latency (25-40%
depending on # dimensions)
![Page 50: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/50.jpg)
4. Zone overloading
• Node maintains list of “peers” in same zone
• Zone not split until MAXPEERS are there
• Only least RTT neighbor maintained
• Replication or splitting of data across peers• Advantages
– Reduced per hop latency
– Reduced path length (because sharing zones has same effect as reducing total number of nodes)
– Improved fault tolerance (a zone is vacant only when all nodes fail)
![Page 51: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/51.jpg)
Zone overload
Avg of 28 to 218 nodes
![Page 52: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/52.jpg)
5. Multiple hash
• (key, value) mapped to “k” multiple points (and nodes)
• Parallel query execution on “k” paths• Or choose closest one in space• Advantages:
– improve data availability– reduce path length
![Page 53: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/53.jpg)
Multiple hash functions
![Page 54: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/54.jpg)
6. Geographically distributed binning
• Distributed binning of CAN nodes based on their relative distances from m IP landmarks (such as root DNS servers)
• m! partitions of coordinate space
• New node joins partition associated with its landmark ordering
• Topological close nodes will be assigned to same partition
• Nodes in same partition divide the space up randomly
• Disadvantage: coordinate space is no longer uniformily populated
– background load balancing could alleviate this problem
![Page 55: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/55.jpg)
Distributed binning
![Page 56: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/56.jpg)
7. More uniform partitioning
• When adding a node to a zone, choose neighbor with largest volume to split
• Advantage– 85% of nodes have the average load as
opposed to 40% without, and largest zone volume dropped from 8x to 2x the average
![Page 57: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/57.jpg)
8. Caching & Replication
• A node can cache values of keys it has recently requested.
• An node can push out “hot” keys to neighbors.
![Page 58: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/58.jpg)
9. Load balancing
• Merge and split zones to balance loads
![Page 59: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/59.jpg)
Outline
• Introduction• Design• Design Improvements• Evalution• Related Work• Conclusion
![Page 60: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/60.jpg)
Evaluation Metrics
1.Path length2.Neighbor state3.Latency4.Volume5.Routing fault tolerance6.Data availability
![Page 61: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/61.jpg)
CAN: scalability
• For a uniformly partitioned space with n nodes and d dimensions – per node, number of neighbors is 2d– average routing path is (dn1/d) hops
• Simulations show that the above results hold in practice (Transit-Stub topology: GT-ITM gen)
• Can scale the network without increasing per-node state
![Page 62: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/62.jpg)
CAN: latency
• latency stretch = (CAN routing delay) (IP routing delay)
• To reduce latency, apply design improvements:– increase dimensions (d)
– multiple nodes per zone (peer nodes) (p)
– multiple “realities” ie coordinate spaces (r)
– multiple hash functions (k)
– use RTT weighted routing
– distributed Binning
– uniform Partitioning
– caching & replication (for “hot spots”)
![Page 63: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/63.jpg)
Overall improvement
![Page 64: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/64.jpg)
CAN: Robustness
• Completely distributed – no single point of failure
• Not exploring database recovery
• Resilience of routing– can route around trouble
![Page 65: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/65.jpg)
Routing resilience
destination
source
![Page 66: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/66.jpg)
Routing resilience
![Page 67: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/67.jpg)
Routing resilience
destination
![Page 68: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/68.jpg)
Routing resilience
![Page 69: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/69.jpg)
• Node X::route(D)
If (X cannot make progress to D) – check if any neighbor of X can make
progress– if yes, forward message to one such nbr
Routing resilience
![Page 70: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/70.jpg)
Routing resilience
![Page 71: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/71.jpg)
Outline
• Introduction• Design• Design Improvements• Evalution• Related Work• Conclusion
![Page 72: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/72.jpg)
Related Algorithms
• Distance Vector (DV) and Link State (LS) in IP routing– require widespread dissemination of
topology– not well suited to dynamic topology like file-
sharing
• Hierarchical routing algorithms– too centralized, single points of stress
• Other DHTs– Plaxton, Tapestry, Pastry, Chord
![Page 73: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/73.jpg)
Related Systems
• DNS
• OceanStore (uses Tapestry)
• Publius: Web-publishing system with high anonymity
• P2P File sharing systems– Napster, Gnutella, KaZaA, Freenet
![Page 74: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/74.jpg)
Outline
• Introduction• Design• Design Improvements• Evalution• Related Work• Conclusion
![Page 75: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/75.jpg)
Consclusion: Purpose
• CAN– an Internet-scale hash table
– potential building block in Internet applications
![Page 76: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/76.jpg)
Conclusion: Scalability
• O(d) per-node state
• O(d * n^(1/d)) path length
For d = (log n)/2, path lengh and per-node state is O(log n) like other DHTs
![Page 77: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/77.jpg)
Conclusion: Latency
• With the main design improvements latency is less than twice the IP path latency
![Page 78: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/78.jpg)
Conclusion: Robustness
• Decentralized, can route around trouble
• With certain design improvements (replication), high data availability
![Page 79: Sylvia Ratnasamy (UC Berkley Dissertation 2002) Paul Francis Mark Handley Richard Karp Scott Shenker A Scalable, Content Addressable Network Slides by](https://reader034.vdocument.in/reader034/viewer/2022042822/56649e9d5503460f94b9db83/html5/thumbnails/79.jpg)
Weaknesses / Future Work
• Security– Denial of service attacks
• hard to combat because a malicious node can act as a malicious server as well as a malicious client
• Mutable content
• Search