medusa – new model of internet topology using k-shell decomposition shai carmi shlomo havlin...

MEDUSA – New Model of Internet Topology Using

k-shell DecompositionShai Carmi

Shlomo Havlin

Bloomington 05/24/2005

Who we are

Talk prepared by Shai Carmi. Graduate student in the

Department of Physics, Bar-Ilan University, Israel.

Supervised by Prof. Shlomo Havlin, who gives the talk.

Who we are

Collaborators – Prof. Scott Kirkpatrick, Hebrew

University of Jerusalem, Israel. Dr. Yuval Shavitt, Tel-Aviv

University, Israel. Eran Shir, Ph.D. Student, Tel-

Aviv University, Israel.

Scott

Measuring the Internet Previous efforts to measure the Internet have

used : One machine + Traceroute to many

destinations. Many machines, specially deployed to

traceroute to many destinations Sites restricted to academic or gov’t labs, on

network backbone General perception was that Law of

Diminishing Returns has set in.

Measuring the Internet

DIMES : Distributed Internet MEasurement and Simulations (http://www.netdimes.org), seems to have made a breakthrough.

Don’t manage machines, offer a very lightweight, limited purpose client, and collect its measurements centrally.

100 – 1000 clients via word-of-mouth (Sep04 to Apr05). >5000 clients now, achieved via Science article, slashdot. 82 countries represented. 2-3 M measurements per day.

The network we analyze

We consider the Internet at the level of its autonomous systems (ASes), with roughly 20,000 nodes and 70,000 links.

We use data gathered between March and June 2005. In the future, can study network dynamics, using intervals of months, weeks or even days.

k-shell method

Use recursive pruning to peel network layers. To remove the 1-shell, keep removing all nodes

with one link (degree=1) until only nodes with degree 2 or more remain.

To remove the 2-shell, keep removing nodes with 2 links, until all degrees are >= 3.

Keep going until all nodes are removed.

k-shell method - example

Original Graph


Pruning Degree 1


Keep Pruning Degree 1


Pruning Degree 2


Keep Pruning Degree 2


Pruning Degree 3

k-shell method

Definitions : k-Core – union of all

shells with indices >= k.

k-Crust – union of all shells withindices <= k.

Applications

Can use k-shell method to analyze the AS network.

For example, color each node by its shell index to visualize the network.

Next, plot quantities as a function of the shell index.

Gain understanding of the network structure. More useful indicator then the degree.

AS graph colored by shells

Identification of a nucleus

k-shell method enables us to identify the heart, or nucleus of the network as nodes in the last core.

No parameters need to be fixed. (Topology dependent only).

Stable over time. Significant ASes (tier-1) were verified to be in

the nucleus. Most quantities show singular behavior at the

last shell. Some examples -

Number of nodes and degrees in the shells

Slope=

~2.6

Centrality vs. shell

Where links go

Distances (vs. crusts)

Distances measured between all pairs in the largest cluster of the crust

Number of site-distinct paths in the nucleus

At least 41 distinct paths between each pair

41 is the k-shell index of the nucleus

The nucleus is k-connected!

Beyond the nucleus

It is left to understand the role of the other nodes in the network.

We look at the connectivity properties of the crusts.

Incorporate this with observations from previously shown plots.

Clusters in the crusts

Percolation Threshold

Structure of the AS network

Nodes outside of the nucleus can be categorized into –

The fractal part – nodes in the largest cluster of the one-before-last crust – contains ~70% of the nodes in the network.

The rest of the nodes become then the isolated part.

Properties of the fractal part

Connected (by construction), so that routing is possible without traversing and congesting the nucleus.

Connections to the nucleus decrease path lengths significantly.

Show fractal properties and power-laws.


Fractal dimension calculated using the ‘box cover method’ (SHM 2005).

Crossover behavior between non-fractal and completely fractal – at the percolation critical point.

Percolation theory arguments predict

2 lN


Percolation theory

prediction – slope = 2.5

The 6-crust is renormalized with box of

size 4

Properties of the isolated part Contains ~30% of the ASes, not reachable

without the nucleus. Low degree nodes, high clustering. Many small clusters. Contributions found in up to k=10 shell. Many nodes are connected directly to highly

connected nodes in the nucleus.

AS network model We summarize – the AS network is composed of 3

main sub-components :

1. Nucleus – Nodes in last shell.

2. Fractal Part – Nodes in the largest cluster of the one-before-last crust.

3. Isolated Part – Nodes in all-but-largest clusters of the one-before-last crust.

We name this model Medusa because of its jellyfish like structure.

Some similarities to Faloutsous’ Jellyfish model but important differences.

Medusa model of the AS network

Our view of the Internet

The End.

Thank you for your attention.

Comments

Some properties (such as percolation) are found in the “Random-Scale-Free-Model”

Internet might not be so special. To have more insight must investigate

navigation with commercial restrictions –Many properties change.

Clustering coefficient vs. shell

Nearest neighbor degree vs. shell

Derivation of the fractal dimension At the threshold, almost all the high degree nodes

are removed, such that the network becomes similar to a random (Erdos-Renyi) network.

Percolation in random networks is equivalent to percolation in an infinite dimensional lattice, in which we know the fractal dimension of the largest component is 4, .

For infinite dimensional lattices, . Thus we conclude, ,or the ''shortest-path''

fractal dimension is 2.

2lM

2rl 4rM

Faloutsos’ Internet jellyfish model The Jellyfish Model : Identify core of network as maximal clique.

( not a very robust or reproducible approach) Shells around network labeled by hop count

from core (a small world) Find large portion of peripheral sites connect to

core.

Faloutsos’ Internet jellyfish model

Faloutsos’ view of the internet