TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
Topology Management for UnstructuredOverlay Networks
Joao Leitao
September 5th, 2012
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
Roadmap
1 Introduction
2 Overview
3 CellFarm
4 X-BOT
5 Thicket
6 OpenFire
7 Conclusions
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
IntroductionP2P Systems
Peer-to-Peer model:
Promise to overcome the limitations of the client-servermodel.
Fault-Tolerance: There is no single point of failure.
Inherent Scalability: All nodes contribute with theirresources.
Popular Examples:
File Sharing: Napster, Emule, Gnutella, Bittorrent.
VOIP: Skype.
IPTV: PPLive.
Internet Anonymity: TOR.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
IntroductionP2P Systems
Peer-to-Peer model:
Promise to overcome the limitations of the client-servermodel.
Fault-Tolerance: There is no single point of failure.
Inherent Scalability: All nodes contribute with theirresources.
Popular Examples:
File Sharing: Napster, Emule, Gnutella, Bittorrent.
VOIP: Skype.
IPTV: PPLive.
Internet Anonymity: TOR.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
IntroductionP2P Global Membeship
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
IntroductionP2P Global Membeship
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
IntroductionP2P Global Membership
Global Membership:
Limits scalability.High maintenance cost under high membership changes(e.g., churn).
Alternative is to rely on a (distributed) membershipservice:
Offers a partial view to each participant.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
IntroductionP2P Global Membership
Global Membership:
Limits scalability.High maintenance cost under high membership changes(e.g., churn).
Alternative is to rely on a (distributed) membershipservice:
Offers a partial view to each participant.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
IntroductionPartial Views
Partial Views:
Encodes neighboring relations across participants.
Their closure establishes an overlay network.
Topology properties can affect P2P services performance:
Efficiency.
Fault-tolerance.
Convergence.
Latency.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
IntroductionPartial Views
Partial Views:
Encodes neighboring relations across participants.
Their closure establishes an overlay network.
Topology properties can affect P2P services performance:
Efficiency.
Fault-tolerance.
Convergence.
Latency.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
IntroductionOverlay Networks
Two types of overlay networks:
Structured Overlay Networks (e.g., DHTs)
Unstructured Overlay Networks (i.e., random overlays).
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
IntroductionStructured Overlay Networks
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
IntroductionStructured Overlay Networks
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
IntroductionUnstructured Overlay Networks
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
IntroductionOverlay Networks
Structured Overlay Networks
Can (easily) offer additional functionality to services.
Complex construction and maintenance.
More susceptible to (high) membership dynamics.
Unstructured Overlay Networks
Lower construction and maintenance costs.
More robust to (high) membership dynamics.
Natural redundancy.
Topology cannot be easily leveraged by services.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
IntroductionOverlay Networks
Structured Overlay Networks
Can (easily) offer additional functionality to services.
Complex construction and maintenance.
More susceptible to (high) membership dynamics.
Unstructured Overlay Networks
Lower construction and maintenance costs.
More robust to (high) membership dynamics.
Natural redundancy.
Topology cannot be easily leveraged by services.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
IntroductionOverlay Networks
Structured Overlay Networks
Can (easily) offer additional functionality to services.
Complex construction and maintenance.
More susceptible to (high) membership dynamics.
Unstructured Overlay Networks
Lower construction and maintenance costs.
More robust to (high) membership dynamics.
Natural redundancy.
Topology cannot be easily leveraged by services.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
IntroductionOverlay Networks
Structured Overlay Networks
Can (easily) offer additional functionality to services.
Complex construction and maintenance.
More susceptible to (high) membership dynamics.
Unstructured Overlay Networks
Lower construction and maintenance costs.
More robust to (high) membership dynamics.
Natural redundancy.
Topology cannot be easily leveraged by services.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
IntroductionProblem Statement
Motivation:
Get the best of both worlds by imbuing some form of relaxedstructure over unstructured overlay networks.
Retain the simplicity, flexibility, and robustness ofunstructured overlays.
Offer some topology properties than can benefit and beleveraged by P2P services and applications.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
IntroductionProblem Statement
Motivation:
Get the best of both worlds by imbuing some form of relaxedstructure over unstructured overlay networks.
Retain the simplicity, flexibility, and robustness ofunstructured overlays.
Offer some topology properties than can benefit and beleveraged by P2P services and applications.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
IntroductionProblem Statement
Motivation:
Get the best of both worlds by imbuing some form of relaxedstructure over unstructured overlay networks.
Retain the simplicity, flexibility, and robustness ofunstructured overlays.
Offer some topology properties than can benefit and beleveraged by P2P services and applications.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
Roadmap
1 Introduction
2 Overview
3 CellFarm
4 X-BOT
5 Thicket
6 OpenFire
7 Conclusions
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
OverviewP2P Typical Architecture
Applica'on*
Peer.to.Peer*Service*
Overlay*Network*
Network*
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
OverviewControl Technique
Applica'on*
Peer.to.Peer*Service*
Overlay*Network****
Network*
Control%
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
OverviewControl Technique
Soft-constraints over neighboring relations.
Enforce topological properties.During the construction of the overlay.
Maintain high level of randomness:
Overlay is flexible to deal with churn.Keep maintenance overhead low.
Example:
Generate dynamic cliques of nodes that can be exploited tosupport replication and load balancing.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
OverviewControl Technique
Soft-constraints over neighboring relations.
Enforce topological properties.During the construction of the overlay.
Maintain high level of randomness:
Overlay is flexible to deal with churn.Keep maintenance overhead low.
Example:
Generate dynamic cliques of nodes that can be exploited tosupport replication and load balancing.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
OverviewControl Technique
Soft-constraints over neighboring relations.
Enforce topological properties.During the construction of the overlay.
Maintain high level of randomness:
Overlay is flexible to deal with churn.Keep maintenance overhead low.
Example:
Generate dynamic cliques of nodes that can be exploited tosupport replication and load balancing.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
OverviewBias Technique
Applica'on*
Peer.to.Peer*Service*
Overlay*Network****
Network*
Bias(Control%
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
OverviewBias Technique
Build an random overlay network.
Iteratively:
Swap some existing overlay links by other links that arebetter.Consider a particular performance criteria (e.g., latency).
Resulting overlay:
Random in nature.Optimized to benefit the operation of a P2P service.
Example:
Promote low-latency overlay links to improve the latency of anapplication-level broadcast service.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
OverviewBias Technique
Build an random overlay network.
Iteratively:
Swap some existing overlay links by other links that arebetter.Consider a particular performance criteria (e.g., latency).
Resulting overlay:
Random in nature.Optimized to benefit the operation of a P2P service.
Example:
Promote low-latency overlay links to improve the latency of anapplication-level broadcast service.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
OverviewBias Technique
Build an random overlay network.
Iteratively:
Swap some existing overlay links by other links that arebetter.Consider a particular performance criteria (e.g., latency).
Resulting overlay:
Random in nature.Optimized to benefit the operation of a P2P service.
Example:
Promote low-latency overlay links to improve the latency of anapplication-level broadcast service.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
OverviewEmbed Technique
Applica'on*
Peer.to.Peer*Service****
Overlay*Network****
Network*
Embed%
Bias(Control%
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
OverviewEmbed Technique
Considering feedback from a P2P service.
Embed a secondary topology over a pure unstructuredoverlay network:
Secondary topology can be leveraged by the service.
Remaining overlay links can be used to transmit controlinformation:
Recover secondary topology in face of failures.Fallback if secondary topology becomes compromised.
Example
Embed spanning trees over an unstructured overlay to supportmultimedia streaming.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
OverviewEmbed Technique
Considering feedback from a P2P service.
Embed a secondary topology over a pure unstructuredoverlay network:
Secondary topology can be leveraged by the service.
Remaining overlay links can be used to transmit controlinformation:
Recover secondary topology in face of failures.Fallback if secondary topology becomes compromised.
Example
Embed spanning trees over an unstructured overlay to supportmultimedia streaming.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
OverviewEmbed Technique
Considering feedback from a P2P service.
Embed a secondary topology over a pure unstructuredoverlay network:
Secondary topology can be leveraged by the service.
Remaining overlay links can be used to transmit controlinformation:
Recover secondary topology in face of failures.Fallback if secondary topology becomes compromised.
Example
Embed spanning trees over an unstructured overlay to supportmultimedia streaming.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
OverviewEnrich Technique
Applica'on*
Peer.to.Peer*Service****
Overlay*Network****
Network*
Enrich%Embed%
Bias(Control%
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
OverviewEnrich Technique
Considering feedback from a P2P service.
Create temporary overlay links at the P2P service layer:
Temporary links are outside the logic of the unstructuredoverlay.Enable the service to contact peers that could not beavailable through the overlay.
Example
Create temporary overlay links that enable one to circumventFirewalls and NAT boxes.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
Overview
Applica'on*
Peer.to.Peer*Service****
Overlay*Network****
Network*
Enrich%Embed%
Bias(Control%
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
OverviewCellFarm
Applica'on*
Peer.to.Peer*Service****
Overlay*Network****
Network*
Enrich%Embed%
Bias(Control:(CellFarm%
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
OverviewX-BOT
Applica'on*
Peer.to.Peer*Service****
Overlay*Network****
Network*
Enrich%Embed%
Bias:(X/BOT%
Control:(CellFarm%
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
OverviewThicket
Applica'on*
Peer.to.Peer*Service****
Overlay*Network****
Network*
Enrich(Embed:(Thicket%
Bias:(X/BOT%
Control:(CellFarm%
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
OverviewOpenFire
Applica'on*
Peer.to.Peer*Service****
Overlay*Network****
Network*
Enrich:(OpenFire(
Embed:(Thicket(
Bias:(X/BOT(
Control:(CellFarm(
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
Overview
Very briefly motivate the problem and solution.
Present some relevant results.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
Overview
Very briefly motivate the problem and solution.
Present some relevant results.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
Control the Topology
Applica'on*
Peer.to.Peer*Service****
Overlay*Network****
Network*
Enrich:(OpenFire(
Embed:(Thicket(
Bias:(X/BOT(
Control:(CellFarm(
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
CellFarm
Goal: Offer support to replication and load balancing:
Process groups.
This can be achieved by manipulating the topology:
Nodes self-organize in (fully connected) cliques.Cliques are used as a replication and load balancing unit.Cliques are highly connected among them.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
CellFarm
Goal: Offer support to replication and load balancing:
Process groups.
This can be achieved by manipulating the topology:
Nodes self-organize in (fully connected) cliques.Cliques are used as a replication and load balancing unit.Cliques are highly connected among them.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
CellFarm
Employed technique: Control.
Flexibility is essential:
Enable the overlay to cope with churn.Clique size governed by three parameters: target size;minimum size; maximum size;
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
CellFarm
Employed technique: Control.
Flexibility is essential:
Enable the overlay to cope with churn.Clique size governed by three parameters: target size;minimum size; maximum size;
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
CellFarmMain Results: Cell Size Distribution
0"
1"
2"
3"
4"
5"
6"
8" 9" 10" 11" 12" 13" 14" 15" 16"
Num
ber'o
f'Cells'
Cell'Size''
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
CellFarmMain Results: Robustness to Churn
0"
10"
20"
30"
40"
50"
60"
70"
80"
90"
100"
0.1" 0.2" 0.3" 0.4" 0.5" 1" 2" 3" 4" 5"
Num
ber"o
f"nod
es"in"largest"con
nected
"compo
enen
t"(%)"
Churn"rate"(%)"
CellFarm(
Unstrctured(
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
CellFarmSummary
CellFarm, based on the control technique operating at theoverlay network layer.
Control technique allows to build and maintain robust andflexible overlays with pre-determined topologicalproperties.
Benefit the operation of P2P resource location systemsbased on unstructured overlays and one-hop replication.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
CellFarmSummary
CellFarm, based on the control technique operating at theoverlay network layer.
Control technique allows to build and maintain robust andflexible overlays with pre-determined topologicalproperties.
Benefit the operation of P2P resource location systemsbased on unstructured overlays and one-hop replication.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
Overview
Applica'on*
Peer.to.Peer*Service****
Overlay*Network****
Network*
Enrich:(OpenFire(
Embed:(Thicket(
Bias:(X/BOT(
Control:(CellFarm(
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
Bias the Topology
Applica'on*
Peer.to.Peer*Service****
Overlay*Network****
Network*
Enrich:(OpenFire(
Embed:(Thicket(
Bias:(X/BOT(
Control:(CellFarm(
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
X-BOT
Goal: Deal with the topology mismatch problem:
Random topology means that many links may besub-optimal.
This can be achieved by manipulating the topology:
Cannot employ control technique.New nodes may offer new opportunities to improve theoverlay topology.Allow nodes to replace sub-optimal links by better links.Approach can be used to improve the overlay for differentcriteria.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
X-BOT
Goal: Deal with the topology mismatch problem:
Random topology means that many links may besub-optimal.
This can be achieved by manipulating the topology:
Cannot employ control technique.New nodes may offer new opportunities to improve theoverlay topology.Allow nodes to replace sub-optimal links by better links.Approach can be used to improve the overlay for differentcriteria.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
X-BOT
Employed technique: Bias.
Generic approach: Companion oracle tags overlay linkswith costs.
X-BOT strives to minimize the overlay cost.
Some minimal coordination is essential:
Protect relevant properties of an unstructured overlay(e.g., connectivity).Rely on 4-node coordination technique to bias the overlay.X-BOT also includes other mechanisms to protect relevantoverlay properties.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
X-BOT
Employed technique: Bias.
Generic approach: Companion oracle tags overlay linkswith costs.
X-BOT strives to minimize the overlay cost.
Some minimal coordination is essential:
Protect relevant properties of an unstructured overlay(e.g., connectivity).Rely on 4-node coordination technique to bias the overlay.X-BOT also includes other mechanisms to protect relevantoverlay properties.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
X-BOT
Employed technique: Bias.
Generic approach: Companion oracle tags overlay linkswith costs.
X-BOT strives to minimize the overlay cost.
Some minimal coordination is essential:
Protect relevant properties of an unstructured overlay(e.g., connectivity).Rely on 4-node coordination technique to bias the overlay.X-BOT also includes other mechanisms to protect relevantoverlay properties.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
X-BOTMain Results: Overlay Cost
0 50 100 150 200 250cycle
0
5x107
1x108
1.5x108
2x108ov
erla
y co
stT-ManAraneolaGoCastX-BOT
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
X-BOTMain Results: Broadcast Latency & Reliability
Gossip-based BroadcastLatency (ms) Reliability (%)
Araneola 3517.0 100.00000GoCast 2108.00 99, 99996T-Man 2545.2 13.80600X-BOT 1879.8 100.00000
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
X-BOTMain Results: Broadcast Latency
Inet-3.0 ScenarioLatency (ms) Reliability (%)
Araneola 3517.0 100.00000GoCast 2108.00 99, 99996T-Man 2545.2 13.80600X-BOT 1879.8 100.00000
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
X-BOTSummary
X-BOT, based on the bias technique operating at theoverlay network layer.
Bias technique allows to achieve improved overlaytopologies that can better match the requirements ofservices executing on top of them.
If a minimal amount of coordination exists among nodes,biasing can be achieved without compromising relevantoverlay properties.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
X-BOTSummary
X-BOT, based on the bias technique operating at theoverlay network layer.
Bias technique allows to achieve improved overlaytopologies that can better match the requirements ofservices executing on top of them.
If a minimal amount of coordination exists among nodes,biasing can be achieved without compromising relevantoverlay properties.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
Bias the Topology
Applica'on*
Peer.to.Peer*Service****
Overlay*Network****
Network*
Enrich:(OpenFire(
Embed:(Thicket(
Bias:(X/BOT(
Control:(CellFarm(
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
Embed the Topology
Applica'on*
Peer.to.Peer*Service****
Overlay*Network****
Network*
Enrich:(OpenFire(
Embed:(Thicket(
Bias:(X/BOT(
Control:(CellFarm(
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
Thicket
Goal: Support efficient and robust data streaming:
Efficient: Rely on spanning tree.Robust: Multiple spanning trees.Load distribution: Multiple interior-node disjoint trees.
Efficient spanning tree topologies should consider feedbackof streaming service:
Cannot employ a technique at the overlay layer.Building independent trees is not an option.Use a single unstructured overlay networks as a base.Embed several spanning trees over that overlay in acoordinated fashion.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
Thicket
Goal: Support efficient and robust data streaming:
Efficient: Rely on spanning tree.Robust: Multiple spanning trees.Load distribution: Multiple interior-node disjoint trees.
Efficient spanning tree topologies should consider feedbackof streaming service:
Cannot employ a technique at the overlay layer.Building independent trees is not an option.Use a single unstructured overlay networks as a base.Embed several spanning trees over that overlay in acoordinated fashion.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
Thicket
Employed technique: Embed.
Coordination is essential.
Overlay links used to embed a tree should consider theexistence of other trees.Trees should be able to recover from partitions efficiently.The maximum forwarding load imposed over nodes shouldbe limited.
Multiples trees allow to transmit redundant data:
e.g., Network coding techniques.Sustain the data stream even if one tree becomestemporarily partitioned.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
Thicket
Employed technique: Embed.
Coordination is essential.
Overlay links used to embed a tree should consider theexistence of other trees.Trees should be able to recover from partitions efficiently.The maximum forwarding load imposed over nodes shouldbe limited.
Multiples trees allow to transmit redundant data:
e.g., Network coding techniques.Sustain the data stream even if one tree becomestemporarily partitioned.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
Thicket
Employed technique: Embed.
Coordination is essential.
Overlay links used to embed a tree should consider theexistence of other trees.Trees should be able to recover from partitions efficiently.The maximum forwarding load imposed over nodes shouldbe limited.
Multiples trees allow to transmit redundant data:
e.g., Network coding techniques.Sustain the data stream even if one tree becomestemporarily partitioned.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
ThicketMain Results: K-interior node distribution (PlanetLab)
0
20
40
60
80
100
0 1 2 3 4 5
num
ber o
f int
erio
r nod
es (%
)
number of trees
Thicket
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
ThicketMain Results: Streaming service robustness in face of targeted node failures
60
65
70
75
80
85
90
95
100
105
0 50 100 150 200
relia
bilit
y (%
)
simulation cycle
NUTSBOLTSThicket
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
ThicketSummary
Thicket, based on the embed technique operating at thepeer-to-peer service layer.
Embed technique allows to efficiently embed interior-nodedisjoint trees over a single unstructured overlay.
Combines the best of gossip-based and tree-baseddissemination approaches.
Improved the usage of available resources when comparedwith existing solutions.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
ThicketSummary
Thicket, based on the embed technique operating at thepeer-to-peer service layer.
Embed technique allows to efficiently embed interior-nodedisjoint trees over a single unstructured overlay.
Combines the best of gossip-based and tree-baseddissemination approaches.
Improved the usage of available resources when comparedwith existing solutions.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
Embed the Topology
Applica'on*
Peer.to.Peer*Service****
Overlay*Network****
Network*
Enrich:(OpenFire(
Embed:(Thicket(
Bias:(X/BOT(
Control:(CellFarm(
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
Enrich the Topology
Applica'on*
Peer.to.Peer*Service****
Overlay*Network****
Network*
Enrich:(OpenFire(
Embed:(Thicket(
Bias:(X/BOT(
Control:(CellFarm(
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
OpenFire
Goal: Deal with the presence of Firewalls and NAT boxes.
Firewalls and NAT boxes make nodes unreachable.This can produce an unbalanced behavior in P2P services(e.g, rumor mongering protocols).Most nodes interact only with public nodes in the Internet.Those nodes can easily became resource exhausted.
Circumvent Firewalls and NAT boxes:
Taking into consideration communication patters at theP2P service layer.Keep connections established by confined nodes open to beused them later.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
OpenFire
Goal: Deal with the presence of Firewalls and NAT boxes.
Firewalls and NAT boxes make nodes unreachable.This can produce an unbalanced behavior in P2P services(e.g, rumor mongering protocols).Most nodes interact only with public nodes in the Internet.Those nodes can easily became resource exhausted.
Circumvent Firewalls and NAT boxes:
Taking into consideration communication patters at theP2P service layer.Keep connections established by confined nodes open to beused them later.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
OpenFire
Employed technique: Enrich.
Use a single temporary overlay link.
Whenever a node receives a request from a peer it updatesits temporary overlay link.Nodes only process a (rumor mongering) request for eachrequest they sent themselves.Additional received requests are forwarded across thetemporary connection.
This solution is interesting:
Can easily ensure balanced processing of messages.Nodes are not required to be aware of Firewalls and NATsexplicitly.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
OpenFire
Employed technique: Enrich.
Use a single temporary overlay link.
Whenever a node receives a request from a peer it updatesits temporary overlay link.Nodes only process a (rumor mongering) request for eachrequest they sent themselves.Additional received requests are forwarded across thetemporary connection.
This solution is interesting:
Can easily ensure balanced processing of messages.Nodes are not required to be aware of Firewalls and NATsexplicitly.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
OpenFire
Employed technique: Enrich.
Use a single temporary overlay link.
Whenever a node receives a request from a peer it updatesits temporary overlay link.Nodes only process a (rumor mongering) request for eachrequest they sent themselves.Additional received requests are forwarded across thetemporary connection.
This solution is interesting:
Can easily ensure balanced processing of messages.Nodes are not required to be aware of Firewalls and NATsexplicitly.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
OpenFireMain Results: Maximum gossip exchanges per node
!" !#$$" %!$$" &#$$" #!$$" '#$$" (!$$" !$#$$" !)!$$"
!"#$%&'()'*(++,-'%./01!*%+'
!"#$%&'()'/(!2!%#%!3'4(#1,!+'
**+","!"
**+",")"
**+","-"
**+","!$"
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
OpenFireMain Results: Maximum latency
!"
#!"
$!"
%!"
&!"
'!!"
'#!"
'$!"
'%!"
'&!"
'" '%!!" ('!!" $%!!" %'!!" )%!!" *'!!" '!%!!" '#'!!"
!"#$%&'()*+$(,%-#./(
%,+0$1(23(&2%4%$+$%#(52+"-%.(
++,"-"'"
++,"-"#"
++,"-"."
++,"-"'!"
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
OpenFireSummary
OpenFire, based on the enrich technique operating at thepeer-to-peer service layer.
Enrich technique allows to overcome some of thechallenges introduced by Firewalls and NAT boxes.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
OpenFireSummary
OpenFire, based on the enrich technique operating at thepeer-to-peer service layer.
Enrich technique allows to overcome some of thechallenges introduced by Firewalls and NAT boxes.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
Roadmap
1 Introduction
2 Overview
3 CellFarm
4 X-BOT
5 Thicket
6 OpenFire
7 Conclusions
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
Conclusions & Future Research DirectionsConclusions
4 techniques to manage the topology of unstructuredoverlay networks.
Imbuing some relaxed form of structure.
Protecting the relevant properties of unstructuredoverlays:
Low overhead.Robustness to churn scenarios.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
Conclusions & Future Research DirectionsConclusions
4 techniques to manage the topology of unstructuredoverlay networks.
Imbuing some relaxed form of structure.
Protecting the relevant properties of unstructuredoverlays:
Low overhead.Robustness to churn scenarios.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
Conclusions & Future Research DirectionsConclusions
At the overlay network layer:
Control (CellFarm).Bias (X-BOT).
At the peer-to-peer service layer:
Embed (Thicket).Enrich (OpenFire).
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
Conclusions & Future Research DirectionsConclusions
In the thesis:
Discusses the design of each solution in detail.Validates all proposed solutions.Experimentally evaluate their impact over particular casestudies.
Summary: The thesis has show and studied how to addsome form structure to unstructured overlay benefiting theoperation of peer-to-peer services.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
Conclusions & Future Research DirectionsConclusions
In the thesis:
Discusses the design of each solution in detail.Validates all proposed solutions.Experimentally evaluate their impact over particular casestudies.
Summary: The thesis has show and studied how to addsome form structure to unstructured overlay benefiting theoperation of peer-to-peer services.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
Conclusions & Future Research DirectionsFuture Research Directions
Improving volunteer parallel computing platforms
New decentralized volunteer computing infrastructures:
For instance based in solutions similar to CellFarm andX-BOT.Offer the opportunity to support more complex parallelprograming models (e.g, MapReduce).
Devise a new class of robust and flexible DHT overlaynetworks based on the X-BOT design.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
Conclusions & Future Research DirectionsFuture Research Directions
Improving volunteer parallel computing platforms
New decentralized volunteer computing infrastructures:
For instance based in solutions similar to CellFarm andX-BOT.Offer the opportunity to support more complex parallelprograming models (e.g, MapReduce).
Devise a new class of robust and flexible DHT overlaynetworks based on the X-BOT design.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
Conclusions & Future Research DirectionsFuture Research Directions
User-centric platform for social applications
Offer a decentralized platform to support a persistent anddynamic social network.
Providing the mechanisms for designing new collaborativeapplications over the social network that can operate in amore decentralized fashion.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
Conclusions & Future Research DirectionsFuture Research Directions
User-centric platform for social applications
Offer a decentralized platform to support a persistent anddynamic social network.
Providing the mechanisms for designing new collaborativeapplications over the social network that can operate in amore decentralized fashion.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
Publications
On the Structure of Unstructured Overlay Networks (fastabstract). J. Leitao, J. Pereira and L. Rodrigues. In Supplement ofthe 38th Annual IEEE/IFIP International Conference on DependableSystems and Networks, Anchorage, Alaska, USA, June, 2008.
Overnesia: a Robust Overlay Network for Virtual Super-Peers.Joao Leitao and Luıs Rodrigues. Technical Report 36/2009,INESC-ID, July 2009 (Available in:http: // www. inesc-id. pt/ ficheiros/ publicacoes/ 5510. pdf ).
X-BOT: A Protocol for Resilient Optimization of UnstructuredOverlays. J. Leitao, J. P. Marques, J. Pereira and L. Rodrigues.Proceedings of the 28th IEEE International Symposium on ReliableDistributed Systems, Niagara Falls, New York, U.S.A., Sep, 2009. pp.236–245.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
Publications
On Adding Structure to Unstructured Overlay Networks. J. Leitao,N. Carvalho, J. Pereira, R. Oliveira, and L. Rodrigues. In Handbookof Peer-to-Peer Networking, X. Shen, H. Yu, J. Buford, M. Akon(Eds.), Springer 2010. pp. 327-365. ISBN: 978-0-387-09750-3.
Balancing Gossip Exchanges in Networks with Firewalls. J. Leitao,R. van Renesse and L. Rodrigues. Proceedings of the 9thInternational Workshop on Peer-to-Peer Systems (IPTPS ’10), SanJose, CA, USA, 27 April, 2010.
Thicket: A Protocol for Building and Maintaining Multiple Trees ina P2P Overlay. M. Ferreira, J. Leitao, and L. Rodrigues.Proceedings of the 29th IEEE Symposium on Reliable DistributedSystems (SRDS), New Delhi, India, 31 October-3 November 2010.
TopologyManagement
forUnstructured
OverlayNetworks
Joao Leitao
Introduction
Overview
CellFarm
X-BOT
Thicket
OpenFire
Conclusions
Publications
Publications
X-BOT: A Protocol for Resilient Optimization of UnstructuredOverlay Networks. J. Leitao, J. P. Marques, J. Pereira, and L.Rodrigues. IEEE Transactions on Parallel and Distributed Systems(Published online January 2012).
Gossip-based Interior-Node-Disjoint Trees. J. Leitao, M. Ferreira, J.Pereira, and L. Rodrigues. IEEE Transactions on Parallel andDistributed Systems (Submitted).