evolution and enhancement of bittorrent network topologies

Introduction Experiments Results Enhancement Results Discussion

Cameron Dale Evolution and Enhancement of BitTorrent Network Topologies

Evolution and Enhancement of BitTorrent Network Topologies

authors:

Cameron Dale, Jiangchuan Liu, Joseph Peters, Bo Li

presented by:

Cameron DaleSimon Fraser University

Burnaby, BC, Canadacamerond@cs.sfu.ca

IWQoS, June 2nd, 2008, University of Twente, Enschede, The Netherlands

Introduction Experiments Results Enhancement Results Discussion

Cameron Dale Evolution and Enhancement of BitTorrent Network Topologies

Overview

● Network topology characteristics

● BitTorrent and its networks

● Experiments and Results

● Enhancement and Results

● Discussion

Introduction Experiments Results Enhancement Results Discussion

Cameron Dale Evolution and Enhancement of BitTorrent Network Topologies

Scale-Free Networks

• node degree k has a power law distribution

• characterized by:– evolving networks– preferential attachment

• tolerant of random node failures

kkP )(

Introduction Experiments Results Enhancement Results Discussion

Cameron Dale Evolution and Enhancement of BitTorrent Network Topologies

Small-World Networks

● characteristics of both random and regular networks

● large amounts of clustering of nodes– Clustering Coefficient (CC): the fraction of all

possible edges that exist between the neighbors of a node

● short average distance between nodes– Characteristic Path Length (CPL)

● known to be effective for the exchange and dissemination of information

Introduction Experiments Results Enhancement Results Discussion

Cameron Dale Evolution and Enhancement of BitTorrent Network Topologies

Motivation

● scale-free networks are resilient

● small-world networks are efficient

● both are desirable for file sharing, and have been found in other P2P networks

● recent papers have found clustering in some of the BitTorrent networks

● all previous results have examined only the initial stage of BitTorrent networks

Introduction Experiments Results Enhancement Results Discussion

Cameron Dale Evolution and Enhancement of BitTorrent Network Topologies

BitTorrent

● peer-to-peer file sharing program

● peer connects to a tracker to find other peers– tracker returns random list of 50 peers

● peer connects to other peers– number of connections is usually limited– default value for most clients is 80

Introduction Experiments Results Enhancement Results Discussion

Cameron Dale Evolution and Enhancement of BitTorrent Network Topologies

BitTorrent Networks

● Connection: the neighbors of a peer, to which it is connected by TCP (undirected)

● Interest: a peer is interested in other peers that have pieces of the download they need

● Unchoked: a peer's indication of a willingness to upload to another peer (incentive mechanism)

● Download: a peer is downloading from other peers

Introduction Experiments Results Enhancement Results Discussion

Cameron Dale Evolution and Enhancement of BitTorrent Network Topologies

Related Work

● Al-Hamra et. al. examined the diameter and found clustering in the Connection network

● Legout et. al. found clustering in the Unchoked network

● both used qualitative methods to demonstrate the clustering

● both examined only the very early stage of a BitTorrent download

Introduction Experiments Results Enhancement Results Discussion

Cameron Dale Evolution and Enhancement of BitTorrent Network Topologies

Basic Experimental Design

● modified BitTornado client

● run on more than 400 PlanetLab nodes

● download a 780 MB sample file

● node will connect, download, stop, delete, and reconnect multiple times

● experiments run for over 100 hours

Introduction Experiments Results Enhancement Results Discussion

Cameron Dale Evolution and Enhancement of BitTorrent Network Topologies

Other Experiments

● remove the limit on the number of connections a peer can make (was 80)– no effect on most of the results– connectivity matrix is random throughout the

experiment

● peers join and leave in groups, causing periods of increased and decreased churn– only a small effect on the Clustering Coefficient

Introduction Experiments Results Enhancement Results Discussion

Cameron Dale Evolution and Enhancement of BitTorrent Network Topologies

Distribution of Peer Characteristics

Nodes

45% 44 hours 6 kB/s 6 kB/s 2

25% 20 hours 12 kB/s 24 kB/s 3

15% 12 hours 25 kB/s 75 kB/s 4

10% 6 hours 50 kB/s 150 kB/s 5

5% 4 hours 100 kB/s 500 kB/s 6

Restarts After

Upload Speed

Download Speed

Unchoke Slots

Introduction Experiments Results Enhancement Results Discussion

Cameron Dale Evolution and Enhancement of BitTorrent Network Topologies

Peer Population

Introduction Experiments Results Enhancement Results Discussion

Cameron Dale Evolution and Enhancement of BitTorrent Network Topologies

Scale-Free Analysis

● plot the degree of each node versus its rank on a log-log scale

● linear fit gives the power law exponent γ and an R2 goodness of fit value

● only the Unchoked network had high R2 values of 0.9 (others were less than 0.7)

● previous real-world experiments show power law exponents between 2 and 3

Introduction Experiments Results Enhancement Results Discussion

Cameron Dale Evolution and Enhancement of BitTorrent Network Topologies

Sample Node Degree Fit

Introduction Experiments Results Enhancement Results Discussion

Cameron Dale Evolution and Enhancement of BitTorrent Network Topologies

Power Law Exponent

Introduction Experiments Results Enhancement Results Discussion

Cameron Dale Evolution and Enhancement of BitTorrent Network Topologies

Small-World Analysis

● calculate the Characteristic Path Length and Clustering Coefficient of each network at periodic intervals

● also generate a similar-sized random graph for comparison

Introduction Experiments Results Enhancement Results Discussion

Cameron Dale Evolution and Enhancement of BitTorrent Network Topologies

Characteristic Path Length

Introduction Experiments Results Enhancement Results Discussion

Cameron Dale Evolution and Enhancement of BitTorrent Network Topologies

Characteristic Path Length

● varies only during the initial stage

● short due to the density of the graph– 440 nodes with average degree 65

● very close to the random graph's CPL

● Unchoked graph is slightly larger– may be due to scale-free nature of the graph

Introduction Experiments Results Enhancement Results Discussion

Cameron Dale Evolution and Enhancement of BitTorrent Network Topologies

Clustering Coefficient

Introduction Experiments Results Enhancement Results Discussion

Cameron Dale Evolution and Enhancement of BitTorrent Network Topologies

CC Compared to Random Graph

Introduction Experiments Results Enhancement Results Discussion

Cameron Dale Evolution and Enhancement of BitTorrent Network Topologies

Clustering Coefficient

● most of the variation is in the initial stage, and none is seen after the transient stage

● Unchoked network shows some clustering in the initial stage

● other than that, there is very little clustering, and none after the transient stage

● lack of clustering means no small-world– most networks are essentially random

Introduction Experiments Results Enhancement Results Discussion

Cameron Dale Evolution and Enhancement of BitTorrent Network Topologies

Connectivity Matrix

● scatter plot of peer connections– point at (x, y) indicates that peer x is connected

to peer y

● peers are assigned an index based on the time they joined the system

● early peers maximize their number of connections quickly

● used by Al-Hamra et. al. to indicate clustering in the Connection network

Introduction Experiments Results Enhancement Results Discussion

Cameron Dale Evolution and Enhancement of BitTorrent Network Topologies

Connectivity Matrix Comparison

Our Results (Hour 4) Al-Hamra et. al.

Introduction Experiments Results Enhancement Results Discussion

Cameron Dale Evolution and Enhancement of BitTorrent Network Topologies

Connectivity Matrix Evolution

Hour 4

Hour 8

Hour 16

Hour 32

Introduction Experiments Results Enhancement Results Discussion

Cameron Dale Evolution and Enhancement of BitTorrent Network Topologies

Connectivity Matrix

● results for initial stage are very similar to previous results

● later results show increased randomness

● in the end, the connectivity matrix is completely random

● the experiment with no limit on the number of connections a peer can make showed a completely random connectivity matrix throughout the experiment

Introduction Experiments Results Enhancement Results Discussion

Cameron Dale Evolution and Enhancement of BitTorrent Network Topologies

Adding Small-World to BitTorrent

● design a tracker that creates a small-world BitTorrent network of peers– only realistic modification possible for BitTorrent

● try similar strategy to Watts and Strogatz– start with a regular graph with a maximized

Clustering Coefficient– add some randomness

Introduction Experiments Results Enhancement Results Discussion

Cameron Dale Evolution and Enhancement of BitTorrent Network Topologies

A Cycle of n-cliques

• connect n-cliques in a cycle• gives a Clustering Coefficient of

• independent of the total number of nodes• for BitTorrent, n = 80, so CC = 0.9986• increases the CPL to twice that of a random

graph (3.8)

)1(

61

nnCC

Introduction Experiments Results Enhancement Results Discussion

Cameron Dale Evolution and Enhancement of BitTorrent Network Topologies

Finding the Small-World

● add some randomness to the regular graph

● replace an n-clique edge with a random edge that connects together n-cliques

● vary the amount of randomness (number of replaced edges)

● graphs usually become small-world with a randomness of 1% to 3%

Introduction Experiments Results Enhancement Results Discussion

Cameron Dale Evolution and Enhancement of BitTorrent Network Topologies

Adding Randomness to n-cliques

Introduction Experiments Results Enhancement Results Discussion

Cameron Dale Evolution and Enhancement of BitTorrent Network Topologies

Small-World Tracker

● assign each new peer to an unfilled n-clique– if all are full, create a new n-clique

● choose the list of peers to return:– a small fixed number from other n-cliques– the rest (the majority) from the peer's n-clique

● two configuration parameters– random-peers: the small fixed number chosen

from other n-cliques– clique-size: the maximum size of an n-clique

Introduction Experiments Results Enhancement Results Discussion

Cameron Dale Evolution and Enhancement of BitTorrent Network Topologies

Simulation (clique-size = 80)

Introduction Experiments Results Enhancement Results Discussion

Cameron Dale Evolution and Enhancement of BitTorrent Network Topologies

Simulation (N = 400)

Introduction Experiments Results Enhancement Results Discussion

Cameron Dale Evolution and Enhancement of BitTorrent Network Topologies

Small-World Tracker Experiment

● experiment is mostly unmodified from the previous one

● only change is the Small-World tracker– random-peers: set to 1– clique-size: set to 40

● same results for peer population, Unchoked network is still scale-free

Introduction Experiments Results Enhancement Results Discussion

Cameron Dale Evolution and Enhancement of BitTorrent Network Topologies

Small-World Tracker CPL

Introduction Experiments Results Enhancement Results Discussion

Cameron Dale Evolution and Enhancement of BitTorrent Network Topologies

Original CC (different scale)

Introduction Experiments Results Enhancement Results Discussion

Cameron Dale Evolution and Enhancement of BitTorrent Network Topologies

Small-World Tracker CC

Introduction Experiments Results Enhancement Results Discussion

Cameron Dale Evolution and Enhancement of BitTorrent Network Topologies

Original CC Compared to Random

Introduction Experiments Results Enhancement Results Discussion

Cameron Dale Evolution and Enhancement of BitTorrent Network Topologies

Small-World Tracker CC to Random

Introduction Experiments Results Enhancement Results Discussion

Cameron Dale Evolution and Enhancement of BitTorrent Network Topologies

Small-World Connectivity Matrix

Introduction Experiments Results Enhancement Results Discussion

Cameron Dale Evolution and Enhancement of BitTorrent Network Topologies

Small-World Tracker Results

● small-world tracker has substantially increased the clustering of peers– all networks were affected, though our changes

were only to the Connected network

● the Characteristic Path Length also increased by 10% to 20%

● the BitTorrent peers now form a small-world network

Introduction Experiments Results Enhancement Results Discussion

Cameron Dale Evolution and Enhancement of BitTorrent Network Topologies

Initial Stage Not Typical

● long-term experiments are needed

● caused by peer dynamics over time– peers become seeds and drop connections to

other seeds– leaving peers free up connection slots to others– new peers are requested periodically from the

tracker

Introduction Experiments Results Enhancement Results Discussion

Cameron Dale Evolution and Enhancement of BitTorrent Network Topologies

Scale-Free Unchoked Network

● power law distribution of nodes with exponent of 2

● unaffected by churn and other changes

● may explain some of BitTorrent's efficiency– scale-free graphs are resistant to churn

Introduction Experiments Results Enhancement Results Discussion

Cameron Dale Evolution and Enhancement of BitTorrent Network Topologies

Lack of Clustering

● almost no clustering in any of the networks

● almost all graphs are completely random– not surprising due to the random peer lists

returned by the tracker

● therefore, no small-world networks are possible

Introduction Experiments Results Enhancement Results Discussion

Cameron Dale Evolution and Enhancement of BitTorrent Network Topologies

Introducing the Small-World

● theoretical framework for creating dense small-world graphs

● simple implementation in a BitTorrent tracker

● dramatically increases the clustering of peers

● all networks show small-world properties

Introduction Experiments Results Enhancement Results Discussion

Cameron Dale Evolution and Enhancement of BitTorrent Network Topologies

Future Work

● experimentally determine the efficiency of small-world BitTorrent networks

● improvements to the Small-World Tracker:– consider effect of small n-cliques on peers– what happens when many peers leave

(consolidating n-cliques)

● further examination of the effect of churn on the clustering of BitTorrent peers

Introduction Experiments Results Enhancement Results Discussion

Cameron Dale Evolution and Enhancement of BitTorrent Network Topologies

Evolution and Enhancement of BitTorrent Network Topologies

authors:

Cameron Dale, Jiangchuan Liu, Joseph Peters, Bo Li

presented by:

Cameron DaleSimon Fraser University

Burnaby, BC, Canadacamerond@cs.sfu.ca

IWQoS, June 2nd, 2008, University of Twente, Enschede, The Netherlands