routing on flat labels
DESCRIPTION
Routing on Flat Labels. Matthew Caesar, Tyson Condie, Jayanthkumar Kannan, Karthik Lakshminarayanan, Ion Stoica, Scott Shenker Appeared in Sigcomm 2006 Presented by: Jackson Pang CS 217B, Spring 2009. What’s wrong with Internet addressing today?. - PowerPoint PPT PresentationTRANSCRIPT
Routing on Flat Labels 1
Routing on Flat Labels
Matthew Caesar, Tyson Condie, Jayanthkumar Kannan, Karthik Lakshminarayanan, Ion Stoica, Scott Shenker
Appeared in Sigcomm 2006
Presented by:Jackson Pang
CS 217B, Spring 2009
Routing on Flat Labels 2
What’s wrong with Internet addressing today?
Hierarchical addressing allows excellent scaling properties
But, forces addressing to conform to network topology
Since topology is not static, addresses can’t persistently identify hosts
Routing on Flat Labels 3
Topology-Based Addressing
Disadvantages: complicates Access control Topology changes Multi-homing Mobility
Advantage Scalability Scalability Scalability …
Area 1
Area 2
Area 4
Area 3
B
J
S
K
Q
F
V
X
A
1.11.2
2.12.2
4.2
4.1
3.33.2
3.1
Routing on Flat Labels 4
What’s wrong with Internet addressing today?
The concept of location vs. identity in IP is mixed Most network applications today require persistent
identity It’s hard to provide persistent identity in presence of
hierarchical addressing Need to decouple identity from addressing Drastically complicates network configuration,
mobility, address assignment Is hierarchy the only way to scale routing?
Routing on Flat Labels 5
Motivation for Flat Identifiers
Stable references Shouldn’t have to change when object moves
Object replication Store object at many different locations
Avoid fighting over names Avoid cyber squatting, typo squatting, …
<A HREF=http://isp.com/dog.jpg>my friend’s dog</A>
<A HREF=http://isp.com/dog.jpg>my friend’s dog</A>
<A HREF=http://f0120123112/ >my friend’s dog</A>
<A HREF=http://f0120123112/ >my friend’s dog</A>
Current Proposed
Routing on Flat Labels 6
Is there an alternative?
Why not route on “flat” host identifiers? Assign addresses independently of network topology
Benefits: No separate name resolution system required Simpler network config/allocation/mobility Simpler network-layer access controls
Routing on Flat Labels 7
Basic idea behind ROFL
Scalable routing on flat identifiers
Goal #1: Scale to Internet topologies How do you fetch a web page by typing google.com?
Goal #2: Support for BGP policies How do you preserve the customer – provider, peering
relationships?
Highly challenging problem, but solution would give a number of benefits
Routing on Flat Labels 8
Basic mechanisms behind ROFL
Goal #1: Scale to Internet topologies Mechanism: DHT-style routing, maintain source-routes to
successors/fingers Provides: Scalable network routing without aggregation
Goal #2: Support for BGP policies Mechanism: Intelligently choose successors/fingers to
conform to ISP relationships Provides: Support for policies, operational model of BGP
Routing on Flat Labels 9
How ROFL works
ISPISP
0xFA2910x3B57E(joining host)
0x3F6C0
0x3BAC80x3B57E
0x3F6C0
Successor list: 0x3F6C0
Pointer list: 0x3F6C0 0x3BAC8
Pointer cache: 0x3B57E
2. hosting routers participate in ROFL on behalf of hosts
3. hosting routers maintain pointers with source-routes to attached hosts’ successors/fingers
4. intermediate routers may cache pointers
5. external pointers provide reachability across domains
1. hosts are assigned topology-independent “flat” identifiers
0x3BAC8
Routing on Flat Labels 10
Background Ideas
Distributed Hash Tables Hashing, Consistent Hashing, DHT goals
Chord Join, leave, index fingers, caching Stoica I., et al., “Chord: A Scalable Peer-to-peer Lookup Protocol
for Internet Applications”, IEEE/ACM Transactions on networking, 2003
Canon Support for hierarchy so that we can do BGP-style routing Ganesan, P., et al., “Canon in G Major: Designing DHTs with
Hierarchical Structure”, ACM Distributed Computing Systems, 2004.
Routing on Flat Labels 11
DHT Background: Hash Table
Name-value pairs (or key-value pairs) E.g., www.cnn.com/foo.html (key) and the Web page
(value)
Hash table Data structure that associates keys with values
lookup(key) valuekey value
Routing on Flat Labels 12
DHT Background: Hash Functions
Hashing Transform the key into a number And use the number to index an array
Example hash function Hash(x) = x mod 101, mapping to 0, 1, …, 100 More sophisticated ones: MD5, SHA1, SHA256
Challenges (collisions, joins and leaves) What if there are more than 101 nodes? Fewer? Which nodes correspond to each hash value? What if nodes come and go over time? But ROFL assumes that we’ve got the hashing issue
covered.
Routing on Flat Labels 13
DHT Background: Consistent Hashing
Large, sparse identifier space (e.g., 128 bits) Hash a set of keys x uniformly to large id space Hash nodes to the id space as well
0 1
Hash(name) object_idHash(IP_address) node_id
Id space represented
as a ring.
2128-1
Routing on Flat Labels 14
DHT (Distributed Hash Table):
Hash table spread over many nodes Distributed over a wide area
Main design goals Decentralization: no central coordinator Scalability: efficient even with large # of nodes Fault tolerance: tolerate nodes joining/leaving
Two key design decisions How do we map names on to nodes? How do we route a request to that node?
Routing on Flat Labels 15
Chord Background: What is Chord? What does it do?
Supports just one operation: given a key, it maps the key onto a node
In short: a peer-to-peer lookup service
Solves problem of locating a data item in a collection of distributed nodes, considering frequent node arrivals and departures
Core operation in most p2p systems is efficient location of data items
Routing on Flat Labels 16
Chord Characteristics (O(LogN))
Simplicity, provable correctness, and provable performance
Each Chord node needs routing information about only a few other nodes
Resolves lookups via messages to other nodes (iteratively or recursively)
Maintains routing information as nodes join and leave the system
Routing on Flat Labels 17
Chord Characteristics
Simplicity, provable correctness, and provable performance
Each Chord node needs routing information about only a few other nodes
Resolves lookups via messages to other nodes (iteratively or recursively)
Maintains routing information as nodes join and leave the system
Routing on Flat Labels 18
DNS vs. Chord
DNS
provides a host name to IP address mapping
relies on a set of special root servers
names reflect administrative boundaries
is specialized to finding named hosts or services
Chord
can provide same service: Name = key, value = IP
requires no special servers
imposes no naming structure
can also be used to find data objects that are not tied to certain machines
But that doesn’t mean we replace DNS in ROFL
Routing on Flat Labels 19
The Base Chord Protocol
Specifies how to find the locations of keys
How new nodes join the system
How to recover from the failure or planned departure of existing nodes
Routing on Flat Labels 20
Chord: Recall Consistent Hashing
Hash function assigns each node and key an m-bit identifier using a base hash function such as SHA-1 ID(node) = hash(IP, Port)
ID(key) = hash(key)
Practically, we hash the node’s public key to get the hash key
Leverage on the benefits of consistent hashing
Routing on Flat Labels 21
Chord Definitions: Successor Nodes
6
1
2
6
0
4
26
5
1
3
7
2identifier
circle
identifier
node
X key
successor(1) = 1
successor(2) = 3successor(6) = 0
Shows which node holds which keys
Routing on Flat Labels 22
Node Joins and Departures
6
1
2
0
4
26
5
1
3
7successor(6) = 7
6
1
successor(1) = 3
Routing on Flat Labels 23
Scalable Key Location
A very small amount of routing information suffices to implement consistent hashing in a distributed environment
Each node need only be aware of its successor node on the circle
Queries for a given identifier can be passed around the circle via these successor pointers
Resolution scheme correct, BUT inefficient: it may require traversing all N nodes!
Routing on Flat Labels 24
Acceleration of Lookups
Lookups are accelerated by maintaining additional routing information
Each node maintains a routing table with (at most) m entries (where N=2m) called the finger table
ith entry in the table at node n contains the identity of the first node, s, that succeeds n by at least 2i-1 on the identifier circle (clarification on next slide)
s = successor(n + 2i-1) (all arithmetic mod 2)
s is called the ith finger of node n, denoted by n.finger(i).node
Routing on Flat Labels 25
Finger Tables (of successors)
0
4
26
5
1
3
7
124
[1,2)[2,4)[4,0)
130
finger tablestart int. succ.
keys1
235
[2,3)[3,5)[5,1)
330
finger tablestart int. succ.
keys2
457
[4,5)[5,7)[7,3)
000
finger tablestart int. succ.
keys6
Routing on Flat Labels 26
Finger Tables - Node Joins
0
4
26
5
1
3
7
124
[1,2)[2,4)[4,0)
130
finger tablestart int. succ.
keys1
235
[2,3)[3,5)[5,1)
330
finger tablestart int. succ.
keys2
457
[4,5)[5,7)[7,3)
000
finger tablestart int. succ.
keys
finger tablestart int. succ.
keys
702
[7,0)[0,2)[2,6)
003
6
6
66
6
Routing on Flat Labels 27
Finger Tables - Node Departure
0
4
26
5
1
3
7
124
[1,2)[2,4)[4,0)
130
finger tablestart int. succ.
keys1
235
[2,3)[3,5)[5,1)
330
finger tablestart int. succ.
keys2
457
[4,5)[5,7)[7,3)
660
finger tablestart int. succ.
keys
finger tablestart int. succ.
keys
702
[7,0)[0,2)[2,6)
003
6
6
6
0
3
Routing on Flat Labels 28
Source of Inconsistencies:Concurrent Operations and Failures
Basic “stabilization” protocol is used to keep nodes’ successor pointers up to date, which is sufficient to guarantee correctness of lookups
Those successor pointers can then be used to verify the finger table entries
Every node runs stabilize periodically to find newly joined nodes
Routing on Flat Labels 29
Stabilization after Join
n joins predecessor = nil n acquires ns as successor via some n’
n notifies ns being the new predecessor
ns acquires n as its predecessor
np runs stabilize
np asks ns for its predecessor (now n)
np acquires n as its successor
np notifies n
n will acquire np as its predecessor
all predecessor and successor pointers are now correct
fingers still need to be fixed, but old fingers will still work
np
su
cc(n
p)
= n
s
ns
n
pre
d(n
s)
= n
p
nil
pre
d(n
s)
= n
su
cc(n
p)
= n
Routing on Flat Labels 30
Failure Recovery
Key step in failure recovery is maintaining correct successor pointers
To help achieve this, each node maintains a successor-list of its r nearest successors on the ring
If node n notices that its successor has failed, it replaces it with the first live entry in the list
stabilize will correct finger table entries and successor-list entries pointing to failed node
Performance is sensitive to the frequency of node joins and leaves versus the frequency at which the stabilization protocol is invoked
Routing on Flat Labels 31
Hierarchical DHT: Support for Inter-domain Routing and BGP-ness
What is a Hierarchical DHT?
UCLA MIT
USA
CS EE
Path Locality
Efficiency, Security,
Fault isolation
Path Convergence
Caching, Bandwidth
Optimization
Local DHTs
Access Control
Routing on Flat Labels 32
Crescendo:Convert flat DHTs to hierarchical (Canon-ization)
Key idea: Recursive structure Construct bottom-up; merge smaller DHTs Lowest level: Chord
CS EE
Routing on Flat Labels 33
Crescendo: Merging two Chord rings
0
12
10
52
13
8
3
2
13
8
3Black node x: Connect to y iff
• y closer than any other black node
• y = succ(x + 2^i)
Routing on Flat Labels 34
Now Back to the ROFL…. How ROFL works
ISPISP
0xFA2910x3B57E(joining host)
0x3F6C0
0x3BAC80x3B57E
0x3F6C0
Successor list: 0x3F6C0
Pointer list: 0x3F6C0 0x3BAC8
Pointer cache: 0x3B57E
2. hosting routers participate in ROFL on behalf of hosts
3. hosting routers maintain pointers with source-routes to attached hosts’ successors/fingers
4. intermediate routers may cache pointers
5. external pointers provide reachability across domains
1. hosts are assigned topology-independent “flat” identifiers
0x3BAC8
Routing on Flat Labels 35
Identifiers
Identity tied to public/private key pair Everyone can know the public key Only authorized parties know the private key
Self-certifying identifier: hash of public key Host associates with a hosting router
Proves it knows private key, to prevent spoofing Router joins the ring on the host’s behalf
Anycast Multiple nodes have the same identifier
Routing on Flat Labels 36
AS and BGP Policies (A review) Economic relationships
Peering Provider/customer
Isolation routing contained within hierarchy
Routing on Flat Labels 37
Isolation in ROFL (Canon)
• Traffic between two hosts traverses no higher than their lowest common provider in the AS hierarchy• Accomplished by carefully merging the rings
Joininghost
InternalSuccessor
ExternalSuccessor
ExternalSuccessor
Source Destination
Routing on Flat Labels 38
Policy support in ROFL (BGP support)
B CPeering link
Mechanism:Convert peeringrelationships to Virtual ASes
Source Destination
A
Source Destination
VirtAS
A
CB
Goal: prefer peer route over
provider route
Routing on Flat Labels 39
Scalability in ROFL
Two extensions to improve locality: Maintain proximity-based fingers in a policy-safe fashion Pointer caching strategies: prefer nearby, popular pointers
0x3B57
0x3BAC0x3B57
0xA153
0xF64B
0x3BAC
0xA153
0xF64B
pc: 0xA153 pc: 0x3BAC 0xF64B
Routing on Flat Labels 40
Evaluation
Intra-domain Trace based on “Rocketfuel” over 4 large ISPs with 2-3
millions of hosts Each host w/ 128-bit ID Routers have 9Mbits of memory for pointer cache and
finger tables
Inter-domain Using Routeviews, inter-AS topology graph are derived Ran simulations with 30 thousand nodes and extrapolated
results to to 600 million hosts
Routing on Flat Labels 41
Metric ROFL BGP+DNS
Join overhead
450 per-host
“lightweight joins”: 14 per-host
typically 0 per host
40,000 per prefix
Latency Baseline: 135ms
With pointer caches: 70ms
With lookup: 137ms
No lookup: 54ms
State 2 million pointer cache entries in core,
~100 entries at edge
150 thousand entries at core and edge
77 million DNS entries at root servers
Failure Cached pointers equivalent of backup paths, failures only affect successors and fingers
Undergoes convergence process, failures have global impact
Routing on Flat Labels 42
Metric ROFL BGP+DNS
Join overhead
450 per-host
“lightweight joins”: 14 per-host
typically 0 per host
40,000 per prefix
Latency Baseline: 135ms
With pointer caches: 70ms
With lookup: 137ms
No lookup: 54ms
State 2 million pointer cache entries in core,
~100 entries at edge
150 thousand entries at core and edge
77 million DNS entries at root servers
Failure Cached pointers equivalent of backup paths, failures only affect successors and fingers
Undergoes convergence process, failures have global impact
Routing on Flat Labels 43
Metric ROFL BGP+DNS
Join overhead
450 per-host
“lightweight joins”: 14 per-host
typically 0 per host
40,000 per prefix
Latency Baseline: 135ms
With pointer caches: 70ms
With lookup: 137ms
No lookup: 54ms
State 2 million pointer cache entries in core,
~100 entries at edge
150 thousand entries at core and edge
77 million DNS entries at root servers
Failure Cached pointers equivalent of backup paths, failures only affect successors and fingers
Undergoes convergence process, failures have global impact
Routing on Flat Labels 44
Metric ROFL BGP+DNS
Join overhead
450 per-host
“lightweight joins”: 14 per-host
typically 0 per host
40,000 per prefix
Latency Baseline: 135ms
With pointer caches: 70ms
With lookup: 137ms
No lookup: 54ms
State 2 million pointer cache entries in core,
~100 entries at edge
150 thousand entries at core and edge
77 million DNS entries at root servers
Failure Cached pointers equivalent of backup paths, failures only affect successors and fingers
Undergoes convergence process, failures have global impact
Routing on Flat Labels 45
Metric ROFL BGP+DNS
Join overhead
450 per-host
“lightweight joins”: 14 per-host
typically 0 per host
40,000 per prefix
Latency Baseline: 135ms
With pointer caches: 70ms
With lookup: 137ms
No lookup: 54ms
State 2 million pointer cache entries in core,
~100 entries at edge
150 thousand entries at core and edge
77 million DNS entries at root servers
Failure Cached pointers equivalent of backup paths, failures only affect successors and fingers
Undergoes convergence process, failures have global impact
Routing on Flat Labels 46
Contributions: Clean-slate design with novel ideas
A world without RIR’s to issue IP addresses! The design includes both inter-domain and intra-domain routing
issues. Did not side-step essential the features. ROFL is very attractive for new Internet usage patterns such as:
multicast, mobility, multi-homing. Secret Sauces:
Brings up design issues we will face in a clean-slate approach Put up a good fight to challenge the traditional IP and DNS-based
infrastructure Extension of already powerful ideas such as: DHT, Chord, Hierarchical
DHT (combined 1000+ citations). Honesty about “conservation of dirt” – didn’t bother to sweep them
elsewhere Fundamental computer science ideas: hashing, caching, recursion,
doubly linked-lists
Routing on Flat Labels 47
ROFL Weaknesses:
Identity tied to public / private key pair PKI notion in Internet: who are the trusted authorities?
Unlikely but possible hash collision Identity -> hash translation
Per-AS link-state information assumed to be current and precise. Assumes existence of OSPF-like link-state information Accomplished via the “control messages”
Needs large pointer cache, but ok with current technology? Inter-AS routing needs large bloom filter for each neighboring
AS Issues with quickly updating bloom filters on joins and leaves?
Routing on Flat Labels 48
References
Stoica I., et al., “Chord: A Scalable Peer-to-peer Lookup Protocol for Internet Applications”, IEEE/ACM Transactions on networking, 2003
Ganesan, P., et al., “Canon in G Major: Designing DHTs with Hierarchical Structure”, Distributed Computing Systems, 2004. Proceedings.
Special Thanks for the presentation material: Jennifer Rexford, Princeton Markus Böhning, UC Berkeley