Watts-Strogatz Model

• Regular graph with degree k connected to nearest neighbors

• Can be also a grid, torus, or any other “geographical” structure which has high clusterisation and high diameter

• With probability p rewire each edge in the network to a random node.– Q: What happens when p=1?

Watts-Strogatz Model (cont.)• When p=1 we have ~Erdos-Renyi network

• There is a range of p values where the network exhibits properties of both: random and regular graphs:

• High clusterisation;

• Short path length.

Demo Small Worlds

• http://ccl.northwestern.edu/netlogo/models/SmallWorlds

Small Worlds and Real Networks

• More realistic than Erdos-Renyi– Low path length

– High clusterisation

• What other properties of real world networks are missing?

• Degree distribution– Watts-Strogatz model might be good for certain

applications (e.g., P2P networks)

– But can not model real world networks where degree distributions are usually power law.

Preferential attachment Model

• Start with Two connected nodes– Add a new node v – Create a link between v and one of the existing nodes with probability

proportional to the degree of the that node• P(u,v) = d(u)/Total_network_degree

• Rich get richer phenomenon!• Exhibits power-law distributions• Can be extended to m links (Barabasi-Albert model)

– Start with connected network of m0 nodes– Each new node connects to m nodes (m m0) with aforedescribed

pref. attachment principle.

• http://ccl.northwestern.edu/netlogo/models/PreferentialAttachment

Recap: Network properties

• Each network is unique “microscopically”, but in large scale networks one can observe macroscopic properties:

– Diameter (six-degrees of separation);

– Clustering coefficient (triangles, friends-of-friends are also friends);

– Degree distribution (are there many “hubs” in the network?);

Expanders

• Expanders are graphs with very strong connectivity properties.– sparse yet very well-connected

• Example– N nodes, E edges N=E– Does this graph havegood connectivity properties?– 2 edges fail -> isolates up to N/2 nodes

Can one isolate large number of nodes by removing small number of edges?

Expansion

• Expansion α:

• How robust are your graphs?

• To isolate k nodes one needs to remove at least α*kedges

Examples

• Which networks do you think have good expansion (random graph, tree, grid)?

Properties of Expander Graphs

• A graph is an expander if the number of edges originating from every subset of vertices is larger than the number of vertices at least by a constant factor (more than 1).• Sparse yet very well-connected (no small cuts, no bottlenecks)

• Small second eigenvalue λ2(we will talk about it later)

• Rapid convergence of random walk

• For all practical reasons, random walk of TTL = O(logN) on an expander graph with fixed node degree gives an uniform random node from the population

Random Walks

• Let G(V,E) be connected graph.

• Consider random walk on G from node v. – We move to a

neighboring node with probability 1/d(v)

– The sequence of random walks is Markov chain

Random WalksTime0 1 0 0 0 0

1 0.00 0.50 0.50 0.00 0.00

2 0.42 0.00 0.00 0.42 0.17

3 0.00 0.35 0.43 0.08 0.14

4 0.32 0.03 0.10 0.39 0.17

5 0.05 0.29 0.37 0.13 0.16

6 0.27 0.07 0.15 0.35 0.17

7 0.08 0.25 0.33 0.17 0.17

8 0.24 0.10 0.18 0.32 0.17

9 0.11 0.22 0.31 0.19 0.17

10 0.22 0.12 0.20 0.30 0.17

11 0.13 0.21 0.29 0.21 0.17

12 0.20 0.13 0.22 0.28 0.17

• The initial node can be fixed, but also can be drawn from some initial distribution P0

Convergence

• For any connected non-bipartite bidirectional graph, and any starting point, the random walk converges

– Converges to unique stationary distribution

– Power Iteration

t 1 0 0 0 0

1 0.00 0.50 0.50 0.00 0.00

2 0.42 0.00 0.00 0.42 0.17

3 0.00 0.35 0.43 0.08 0.14

4 0.32 0.03 0.10 0.39 0.17

5 0.05 0.29 0.37 0.13 0.16

6 0.27 0.07 0.15 0.35 0.17

7 0.08 0.25 0.33 0.17 0.17

8 0.24 0.10 0.18 0.32 0.17

9 0.11 0.22 0.31 0.19 0.17

10 0.22 0.12 0.20 0.30 0.17

11 0.13 0.21 0.29 0.21 0.17

12 0.20 0.13 0.22 0.28 0.17

13 0.14 0.19 0.28 0.22 0.17

14 0.19 0.14 0.23 0.27 0.17

15 0.15 0.19 0.27 0.23 0.17

16 0.18 0.15 0.23 0.27 0.17

17 0.15 0.18 0.26 0.24 0.17

18 0.18 0.16 0.24 0.26 0.17

19 0.16 0.18 0.26 0.24 0.17

20 0.17 0.16 0.24 0.26 0.17

21 0.16 0.17 0.26 0.24 0.17

22 0.17 0.16 0.24 0.26 0.17

23 0.16 0.17 0.25 0.25 0.17

24 0.17 0.16 0.25 0.25 0.17

25 0.16 0.17 0.25 0.25 0.17

26 0.17 0.16 0.25 0.25 0.17

27 0.16 0.17 0.25 0.25 0.17

28 0.17 0.16 0.25 0.25 0.17

29 0.17 0.17 0.25 0.25 0.17

30 0.17 0.17 0.25 0.25 0.17

Stationary Distribution• Which distribution does the random walk converge in our

graph?

• Random walk converges to the stationary distribution:π(v) = d(v)/2m

– d(v) = degree of v, i.e. # of neighbors.– m: |E|, i.e. # of edges.– If Graph d-regular then to uniform distribution

π’(v) = u: (u,v)∊E π(u)1

d(u)

= u: (u,v)∊Ed(u)2m

∙1

d(u)

= u: (u,v)∊E1

2m

= d(v)2m= π(v)

π 0.17 0.17 0.25 0.25 0.17π' 0.17 0.17 0.25 0.25 0.17

Implications

• The stationary distribution π(v) = d(v)/2m

is proportional to the degree of v.

– What’s the intuition?

– The more neighbors you have, the more chance you’ll be visited.

• We’ll talk about it later

Definitions

𝐴 =

0 11 01 0

1 0 00 1 00 1 1

0 10 0

1 0 11 1 0

Adjacency Matrix

M=

0 1/21/2 01/3 0

1/2 0 00 1/2 00 1/3 1/3

0 1/30 0

1/3 0 1/31/2 1/2 0

Transition (random walk) MatrixM=DA

𝐷 =

1/2 00 1/20 0

0 0 00 0 01/3 0 0

0 00 0

0 1/3 00 0 1/2

Diagonal matrix with Di,i = 1/d(i)

Adjacency Matrix

• A is n x n adjacency matrix of G=(V,E)

– Aij is 1 if there is a link between i and j nodes and 0 otherwise

• Gives us all 1-hop paths.

• How to count # of 2-hop paths?

– A2

Matrix manipulations

• A2 gives us # of 2-hop paths• A3 gives us ?

– # of 3-hop paths, etc.– not simple paths! Can refer as walks.

• What about taking a vector v=(1 0 0 0 0) that represents a message at the first node and multiplying it by A?

𝑣𝐴 = 1 0 0 0 0

0 11 01 0

1 0 00 1 00 1 1

0 10 0

1 0 11 1 0

• vA=(0 1 1 0 0)– indicates how many walks of length 1 from node 1 end up in node i.

• vA2 = (2 0 0 2 1)– Indicates how many walks of length 2 from node 1 end up in node i.

• vA3 = (0 4 5 1 2)– Indicates how many walks of length 3 from node 1 end up in node i.

Matrix manipulations (cont.)

• What about multiplying by a Random Walk Matrix?

1 0 0 0 0

0 1/21/2 01/3 0

1/2 0 00 1/2 00 1/3 1/3

0 1/30 0

1/3 0 1/31/2 1/2 0

= 0 1/2 1/2 0 0

M=

0 1/21/2 01/3 0

1/2 0 00 1/2 00 1/3 1/3

0 1/30 0

1/3 0 1/31/2 1/2 0

Transition (random walk) MatrixM=DA

0 1/2 1/2 0 0

0 1/21/2 01/3 0

1/2 0 00 1/2 00 1/3 1/3

0 1/30 0

1/3 0 1/31/2 1/2 0

= 0.42 0 0 0.42 0.17

What about Random Walk Matrix M? (cont.)

• Recap: Random walk on a graph G: we start at a node v0 and at the t-th step we are at a node vt. We move to a neighbor of vt with probability 1/d(vt).– The sequence of random nodes (vt :t=0,1,2…) is a Markov

chain

• We start from the initial state of the system, e.g. P0: [1 0 0 0 0]; – Can also be drawn from some initial distribution

• Pt= P0Mt

– Or can be written as Pt= (MT)tP0 if we represent P as a column vector

Again the same Example

– When Pt+1 =Pt = π, we have reached stationary distribution, i.e. πM= π

– Recall: that v is eigenvector of matrix M and λ its eigenvalue if vM=λv• so π is eigenvector of M with eigenvalue λ=1

1 0 0 0 0

0 1/21/2 01/3 0

1/2 0 00 1/2 00 1/3 1/3

0 1/30 0

1/3 0 1/31/2 1/2 0

= 0 1/2 1/2 0 0

0 1/2 1/2 0 0

0 1/21/2 01/3 0

1/2 0 00 1/2 00 1/3 1/3

0 1/30 0

1/3 0 1/31/2 1/2 0

= 0.42 0 0 0.42 0.17

Pt= P0Mt