gradient topology: a generalized super-peer topology

Gradient Topology: A Generalized Super-Peer Topology

Gossiping in Distributed Systems

“Gossiping is the endless process of randomly choosing

two members and subsequently letting these two

exchange Information”

[Kermarrec/Van Steen, Gossiping in distributed systems]

Gossiping algorithms have been mostly developed for

random overlay networks.

Peer Sampling, topology construction, computation, monitoring

Random Networks and Gossiping

Efficient and robust information propagation

Low diameter networks, redundant paths.

Symmetry of random networks makes it easier to analyse

systems using mathematical tools.

“By symmetry we mean the existence of different viewpoints from

which the system appears the same.“

P. W. Anderson [More is Different]

Symmetry of random networks means we can only use

message-passing to share information between nodes.

Scale-Free Networks

New nodes preferentially create links to those nodes

with a higher number of links (positive feedback).

Symmetry breaking from a random network.

Nodes now can use information encoded in the topology to

send search requests to hubs.

Random Graph Barabasi’s Scale-Free Graph

PreferentialAttachmentAlgorithm

Ant Foraging – from Random to Ordered*

*Foraging patterns break both spatial and temporal symmetry

Symmetry Breaking

Symmetry breaking is about going from a more disordered

state to a more ordered state.

Self-organization: from a higher to a lower entropy state.

More formally, symmetry breaking describes

a phenomenon where small fluctuations acting on

a system crossing a critical point determine which branch

of a bifurcation is taken.

Mechanisms of Self-Organization*

External events supplied to system

Positive feedback to cascade external events

Negative feedback to limit cascading effects

Decay/exploration to regenerate the self-

organized structure

Temporal symmetry

*Patterns of S.O. In Biology [Denouberg et al]

Gradient Overview

A gossip-generated P2P overlay network that sorts peers into a overlapping redundant trees, where all trees have the same root.

Peers are sorted by a local utility value

Layered over a PSS to prevent partitioning

Efficient search to find high utility peers

Gradient ascent

Gradient descent

Gradient TopologyApp-specific utility function at

every peer.

Highest utility peers are clustered in the centre, while peers with decreasing utility are found at increasing distance from the centre.

Can be implemented as a ranking function using T-Man

Gradient Overlay Network

Greedy Preference Function

Preference function for keeping neighbours. Peer p prefers neighbour a over neighbour b if

and only if

or

where Up(a) and Up(b) are the p's cached utility values for neighbours a and b.

pU<bUpU>aU pp

pUbU<pUaU pp

Soft-Max Preference Function

Select neighbour a over neighbour b with higher probability:

where Pp(a) and Pp(b) are the probabilities of selecting neighbours a and b, respectively.

Probabilities are normalizedover all neighbours.

n

ip

pp

iU

aUaP

1

n

ip

pp

iU

bUbP

1

11

n

ip iP

Who should a peer gossip with?

Again, you can use

Greedy Policy

Softmax Policy

A neighbour to gossip with can be selected

from Gradient neighbours or random

neighbours (from Cyclon)

Discovering High Utility Peers

Gradient structure allows an efficient search heuristic called gradient search.

Next hop can be either greedily chosen as highest

utility neighbour or probabilistically chosen.

Boltzman exploration reduces traffic on popular paths.

Improved performance over Random Walk.

Gvod: Layered Gossip Architecture

P2P Video on

Demand

Gradient Overlay

Network

Peer Sampling

Service (Cyclon)

Gvod Protocol

Utility function returns download point in the video.

VoD layer samples nodes in the Gradient Layer to build:

1. BitTorrent set: neighbours at similar download positions

2. Upper set: neighbours at slightly higher download positions

In contrast to BitTorrent, nodes don’t need to exchange

messages to know whether a neighbour has a piece of

interest or not.

P2P Live Streaming: GradienTv

Approximate auction algorithm uses node

upload bandwidths to allocate places in

streaming overlay trees.

GradienTv: Bandwidth Levels Utility is upload bandwidth capacity.

Utility levels are ranges of upload

bandwidth capacity.

Long range links added to the

similar set to utilise resources of

higher bandwidth peers in centre.

Modified preferential neighbour

selection algorithm in Gradient to

explore within a utility level.

The Project:Decentralized Resource Allocation

Random Overlay Network Approach

Requires lots of message passing to find ’good’

peers

Gradient Overlay Network Approach?

Bounded time to find free resources

Short-range links to reduce time

Bounded (but high?) gossiping overhead

Gossiping/Gradient References

Kermerrac and Van Steen, “Gossiping in Distributed Systems”, ACM SIGOPS OS Review 2007. Jan Sacha, Bartosz Biskupski, Dominik Dahlem, Raymond Cunningham, René Meier,Jim Dowling, and Mads Haahr,"Decentralising a Service-Oriented Architecture", In the Peer-to-Peer Networking and Applications Journal (PPNA), ISSN 1936-6442, Springer, Oct, 2009

Sacha et al, "Using Aggregation for Adaptive Superpeer Discovery on the Gradient

Topology", In Proceedings of the 2nd International Workshop on Self-Managing

Systems, LNCS 3996, pps 73-86, 2006.

Sacha et al, "Discovery of Stable Peers in a Self-Organising Peer-to-Peer Gradient

Topology", In the International Conference on Distributed Applications and

Interoperable Systems (DAIS), LNCS 4025, pages 70-83, 2006.

Live Streaming/VoD References

Amir Payberah, Jim Dowling, Fatemeh Rahimian and Seif Haridi.  gradienTv: Market-

based P2P Live Media Streaming on the Gradient Overlay, Dais 2010.

Gautier Berthou, P2P VoD using the Self-Organizing Gradient Overlay Network, SOAR

2010.