spectral graph theory tools for the analysis of …mic/univ700r/benzi_slides.pdfspectral graph...

Spectral Graph Theory Tools

for the Analysis of Complex Networks

Michele Benzi

Department of Mathematics and Computer ScienceEmory University

Atlanta, GA 30322, USA

Michele Benzi Analysis of Complex Networks

Acknowledgments

� Christine Klymko (Emory)

� Ernesto Estrada (Strathclyde, UK)

� Support: NSF DMS-1115692

� Papers and preprints available on web page


Outline

� Complex networks: motivation and background

� Graph spectra: a brief introduction

� Spectral centrality and communicability measures

� Clustering coefficients


Complex networks: motivation and background

� Complex networks provide models for physical, biological,engineered or social systems (e.g., molecular structure, geneand protein interaction, food webs, transportation networks,power grids, social networks,...).

� Graph analysis provides quantitative tools for the study ofcomplex networks.

� Techniques from spectral graph theory, linear and multilinearalgebra, probability, approximation theory, etc. play a majorrole.

Network science today is a vast multidisciplinary field. Importantearly work was done by social scientists: sociologists,anthropologists, experimental psychologists, economists and evenbibliometrists. More recently, physicists, computer scientists andapplied mathematicians have made major contributions.


Basic references

Some classic early references:

� L. Katz, A new status index derived from sociometric analysis,Psychometrika, 18 (1953), pp. 39–43.

� A. Rapoport, Mathematical models of social interaction, inHandbook of Mathematical Psychology, vol. 2, pp. 493–579.Wiley, New York, 1963.

� D. J. de Solla Price, Networks of scientific papers, Science,149 (1965), pp. 510–515.

� S. Milgram, The small world problem, Psychology Today, 2(1967), pp. 60–67.

� J. Travers and S. Milgram, An experimental study of the smallworld problem, Sociometry, 32 (1969), pp. 425–443.


Basic references (cont.)

The field exploded in the late 1990s due to several breakthroughsby physicists, applied mathematicians and computer scientists.This helped make the field more quantitative and mathematicallysophisticated. Landmark papers include

� D. J. Watts and S. H. Strogatz, Collective dynamics of‘small-world’ networks, Nature, 393 (1998), pp. 440–442.

� L.-A. Barabasi and R. Albert, Emergence of scaling in randomnetworks, Science, 386 (1999), pp. 509–512.

� M. E. J. Newman, Models of the small world, J. Stat. Phys.,101 (2000), pp. 819–841.

� J. Kleinberg, Navigation in a small world, Nature, 406 (2000),p. 845.

� R. Albert and L.-A. Barabasi, Statistical mechanics ofcomplex networks, Rev. Modern Phys., 74 (2002), pp. 47–97.



Recent systematic accounts include

� U. Brandes and T. Erlebach (Eds.), Network Analysis.Methodological Foundations, Springer, LNCS 3418, 2005.

� F. Chung and L. Lu, Complex Graphs and Networks,American Mathematical Society, 2006.

� A. Barrat, M. Barthelemy, and A. Vespignani, DynamicalProcesses on Complex Networks, Cambridge University Press,2008.

� M. E. J. Newman, Networks. An Introduction, OxfordUniversity Press, 2010.

� P. Van Mieghem, Graph Spectra for Complex Networks,Cambridge University Press, 2011.

� E. Estrada, The Structure of Complex Networks, OxfordUniversity Press, 2011.



Two new interdisciplinary journals in 2013:

� Journal of Complex Networks (Oxford University Press)

� Network Science (Cambridge University Press)

http://www.oxfordjournals.org/

http://journals.cambridge.org/


Complex networks: what are they?

But what exactly is a complex network?

Unfortunately, no precise definition exists.

It is easier to say which graphs are not complex networks. Regularlattices are not considered complex networks, and neither arecompletely random graphs such as the Gilbert or Erdos–Renyimodels.

Random graphs are, however, considered by many to be useful astoy models for complex networks.


Regular lattice: not a complex network!


Star graph: also not a complex network!


Erdos–Renyi graph: also not a complex network!


Some features of complex networks

Some of the attributes typical of many real-world complexnetworks are:

� Scale-free (power law degree distribution)

� Small-world (small graph diameter)

� Highly clustered (many triangles, hubs...)

� Hierarchical

� Rich in ‘motifs’

� Self-similar

Caveat: there are important examples of real-world complexnetworks lacking one or more of these attributes.


Example of complex network: B–A model


Example of complex network: Golub collaboration graph


Example of complex network: Erdos collaboration graph


Example of complex network: PPI network of

Saccharomyces cerevisiae (beer yeast)


Example of complex network: social network of injecting

drug users in Colorado Springs, CO


Example of complex network: the Internet


Example of (directed) complex network: a food web

Picture credits: http://www.physicalgeography.net/fundamentals/9o.html


Network analysis

Basic questions about network structure include centrality,communicability and community detection issues:

� Which are the most “important” nodes?� Network connectivity (vulnerability)� Lethality in PPI networks� Author centrality in collaboration networks

� How do “disturbances” spread in a network?� Spreading of epidemics, rumors, fads,...� Routing of messages; returnability

� How to identify “community structures” in a network?� Clustering, transitivity� Partitioning


Basic definitions

Notation:

� G = (V ,E ) is a connected simple graph with N = |V | nodesand m = |E | edges (i.e., G is unweighted, undirected and hasno loops)

� A = [aij ] ∈ RN×N is the associated adjacency matrix:� aij = 1 if nodes i and j are adjacent, aij = 0 otherwise� aii = 0, 1 ≤ i ≤ N� A is symmetric

� For a node i , define its degree di :=�N

j=1 aij (number ofimmediate neighbors in G )

� Graph Laplacian: L := D − A where D = diag (d1, . . . , dN)

� Some modifications necessary for digraphs (directed graphs)


Network spectra

For any matrix A ∈ RN×N , the (generally complex) number λ is aneigenvalue of A if there exists a vector x �= 0 such that

Ax = λx .

We say that x is an eigenvector of A, associated with theeigenvalue λ. Note that eigenvectors are defined up to nonzeroscalar multiples.

Eigenvectors are often normalized to have unit length:

�x�2 =�

|x1|2 + |x2|2 + · · · + |xN |2 = 1 .


Network spectra (cont.)

The eigenvalues are the roots of the characteristic equation

det(A− λIN) = 0 ,

a polynomial equation of degree N.

This can be seen from the fact that the homogeneous system oflinear equations

Ax− λx = (A− λIN)x = 0

can have non-zero solutions if and only if the matrix A− λIN issingular.



The Fundamental Theorem of Algebra ensures that any N × Nmatrix A has N eigenvalues, counted with their multiplicities.

The set consisting of the eigenvalues of A:

σ(A) := {λ1 ,λ2 , . . . ,λN}

is called the spectrum of A.

Recall that a matrix A = [aij ] is symmetric if A = AT ; that is,aij = aji for all i , j = 1 : N.

� The eigenvalues of a real symmetric matrix are all real.

� Eigenvectors corresponding to distinct eigenvalues of asymmetric matrix are necessarily orthogonal.



A general (nonsymmetric) matrix may have complex eigenvalues.If A is real, the complex eigenvalues appear in conjugate pairs.Hence, a real matrix of odd order always has at least one realeigenvalue.

An example of a real matrix with no real eigenvalues is given by

A =

�0 1−1 0

�

The adjacency matrix of an undirected network is alwayssymmetric, hence its eigenvalues are all real. The adjacency matrixof a directed network, on the other hand, is typically nonsymmetricand will have complex (non-real) eigenvalues in general.

The spectrum of a network is the spectrum of the correspondingadjacency matrix.



Perron–Frobenius Theorem: Let A ∈ RN×N be a matrix withnonnegative entries. If A is irreducible, the spectral radius of A,defined as

ρ(A) = max {|λ| : λ ∈ σ(A)}

is a simple eigenvalue of A. Moreover, there exists a uniquepositive eigenvector x associated with this eigenvalue:

Ax = ρ(A)x, x = (xi ), xi > 0 ∀i = 1 : N.

This theorem is of fundamental importance, and it provides thebasis for the notion of eigenvector centrality and for Google’scelebrated PageRank algorithm.

It is also probably behind Facebook’s new Graph Search tool(cf. Mark Zuckerberg’s announcement yesterday).



For an undirected network, the spectrum of the graph LaplacianL = D − A also plays an important role. To avoid confusion, wecall σ(L) the Laplacian spectrum of the network.

The eigenvalues of L are all real and nonnegative, and sinceL 1 = 0, where 1 denotes the vector of all ones, L is singular.Hence, 0 ∈ σ(L).

Theorem. The multiplicity of 0 as an eigenvalue of L coincideswith the number of connected components of the network.

Corollary. For a connected network, the null space of the graphLaplacian is 1-dimensional. Thus, rank(L) = N − 1.



The eigenvector of L associated with the smallest nonzeroeigenvalue is called the Fiedler vector of the network. Since thiseigenvector must be orthogonal to the vector of all ones, it mustcontain both positive and negative entries.

There exist elegant graph partitioning algorithms that assign nodesto different subgraphs based on the sign of the entries of theFiedler vector.

These methods, however, tend to work well only in the case offairly regular graphs. In general, partitioning complex graphs(scale-free graphs in particular) is very difficult.


Example: PPI network of Saccharomyces cerevisiae

0 500 1000 1500 2000

0

500

1000

1500

2000

nz = 13218

Adjacency matrix, N = 2224, m = 6609.



0 500 1000 1500 2000

0

500

1000

1500

2000

nz = 13218

Same, reordered with Reverse Cuthill–McKee



0 10 20 30 40 50 60 700

100

200

300

400

500

600

700

Degree distribution (dmin = 1, dmax = 64)



0 500 1000 1500 2000 2500−15

−10

−5

0

5

10

15

20

Eigenvalues


Example: Intravenous drug users network

0 100 200 300 400 500 600

0

100

200

300

400

500

600

nz = 4024

Adjacency matrix, N = 616, m = 2012.



0 100 200 300 400 500 600

0

100

200

300

400

500

600

nz = 4024

Same, reordered with Reverse Cuthill–McKee



0 10 20 30 40 50 600

50

100

150

200

250

Degree distribution (dmin = 1, dmax = 58)



0 100 200 300 400 500 600 700−10

−5

0

5

10

15

20

Eigenvalues


Centrality measures

There are dozens of different definitions of centrality for nodes in agraph. The simplest is degree centrality, which is just the degree di

of node i . This does not take into account the “importance” ofthe nodes a given nodes is connected to—only their number.

A popular notion of centrality is betweenness centrality, defined forany node i ∈ V as

cB(i) :=�

j �=i

�

k �=i

δjk(i) ,

where δjk(i) is the fraction of all shortest paths in the graphbetween nodes j and k which contain node i :

δjk(i) :=# of shortest paths between j , k containing i

# of shortest paths between j , k.


Centrality measures (cont.)

Betweenness centrality assumes that all communication in thenetwork takes place via shortest paths, but this is often not thecase.

This observation has motivated a number of alternative definitionsof centrality, which aim at taking into account the global structureof the network and the fact that all walks between pairs of nodesshould be considered, not just shortest paths.

We mention here that for directed graphs it is sometimes necessaryto consider both hubs and authorities (Ex.: Jon Kleinberg’s HITSalgorithm). I hope Christine Klymko will give a presentation onthis.


Spectral centrality measures

Bonacich’s eigenvector centrality uses the entries of the dominanteigenvector (that is, the eigenvector corresponding to the spectralradius of the adjacency matrix) to rank the nodes in the network inorder of importance: the larger xi is, the more important node i isconsidered to be. By the Perron–Frobenius Theorem, thiseigenvector is unique provided the network is connected.

The underlying idea is that “a node is important if it is linked tomany important nodes.” This circular definition corresponds to thefixed-point iteration

xk+1 = Axk , k = 0, 1, . . .

which converges, under mild conditions, to the dominanteigenvector of A as k →∞. The rate of convergence depends onthe spectral gap γ = λ1 − λ2. The larger γ, the faster theconvergence.


Spectral centrality measures (cont.)

Google’s PageRank algorithm is a variant of eigenvector centrality,applied to the (directed) graph representing web pages(documents), with hyperlinks between pages playing the role ofdirected edges.

Since the WWW graph is not connected, some tweaks are neededto have a unique PageRank eigenvector. The entries of this vectorhave a probabilistic interpretation in terms of random walks on theweb graph, and their sizes determine the ranking of web pages.

The PageRank vector is the unique stationary probabilitydistribution of the Markov chain corresponding to the random walkon the Web graph.


Spectral centrality measures (cont.)

When a user enters a term or phrase into Google’s search engine,links to pages containing those terms are returned in order ofimportance. PageRank is (was?) one of the criteria Google uses torank pages.

Since hardly anybody looks past the few (5-10) top results, it’svery important to get these “top” pages right. Google’sunsurpassed ability to do this gave it an early edge on thecompetition in the late 1990s.

Computation of PageRank is quite expensive due to the enormoussize of the WWW graph.


Subgraph centrality

Subgraph centrality (Estrada & Rodrıguez-Velasquez, 2005)measures the centrality of a node by taking into account thenumber of subgraphs the node “participates” in.

This is done by counting, for all k = 1, 2, . . . the number of closedwalks in G starting and ending at node i , with longer walks beingpenalized (given a smaller weight).


Subgraph centrality

Recall that

� (Ak)ii = # of closed walks of length k based at node i ,

� (Ak)ij = # of walks of length k that connect nodes i and j .

Using 1/k! as weights leads to the notion of subgraph centrality:

SC (i) =

�I + A +

1

2!A2 +

1

3!A3 + · · ·

�

ii

= [eA]ii

Subgraph centrality has been used successfully in various settings,especially proteomics and neuroscience.

Note: the additional term of order k = 0 does not alter theranking of nodes in terms of subgraph centrality.


Katz centrality

Of course different weights can be used, leading to different matrixfunctions, such as the resolvent (Katz, 1953):

(I − cA)−1 = I + cA + c2A2 + · · · , 0 < c < 1/λ1.

Also, in the case of a directed network one can use the solutionvectors of the linear systems

(I − cA)x = 1 and (I − cAT )y = 1

to rank hubs and authorities. These are the row and column sumsof the matrix resolvent (I − cA)−1, respectively.

Similarly, one can use the row and column sums of eA to ranknodes. This can be done without computing the entries of eA, andis much faster than computing the diagonal entries of eA.


Comparing centrality measures

Suppose A is the adjacency matrix of an undirected, connectednetwork. Let λ1 > λ2 ≥ λ3 ≥ . . . ≥ λN be the eigenvalues of A,and let xk be a normalized eigenvector corresponding to λk . Then

eA =N�

k=1

eλkxkxTk

and therefore

SC (i) =N�

k=1

eλk x2ki ,

where xki denotes the ith entry of xk .

Similar spectral formulas exist for other centrality measures basedon matrix functions (e.g., Katz).


Comparing centrality measures (cont.)

It is clear that when the spectral gap is sufficiently large(λ1 � λ2), the centrality ranking is essentially determined by thedominant eigenvector x1, since the contributions to the centralityscores from the remaining eigenvalues/vectors becomes negligible,in relative terms.

Hence, in this case all these spectral centrality measures tend toagree, especially on the top-ranked nodes, and they all reduce toeigenvector centrality. Degree centrality is also known to bestrongly correlated with eigenvector centrality in this case.

In contrast, the various measures can give significantly differentresults when the spectral gap is small. In this case, going beyondthe dominant eigenvector can result in significant improvements.


More matrix functions

Other matrix functions of interest are cosh(A) and sinh(A), whichcorrespond to considering only walks of even and odd length,respectively.

Because in a bipartite graph Tr (sinh(A)) = 0, the quantity

�B(G )� :=Tr (cosh(A))

Tr (eA)

provides a measure of how “close” a graph is to being bipartite.Here Tr(·) is the trace (sum of diagonal entries) of a matrix.

Hyperbolic matrix functions are also used to define the notion ofreturnability in digraphs (Estrada & Hatano, 2009).


Communicability

Communicability measures how “easy” it is to send a messagefrom node i to node j in a graph. It is defined as (Estrada &Hatano, 2008):

C (i , j) = [eA]ij

=

�I + A +

1

2!A2 +

1

3!A3 + · · ·

�

ij

≈ weighted sum of walks joining nodes i and j .

As before, other power series expansions (weights) can be used,leading to different functions of A.

For a large graph, computing all communicabilities C (i , j) (i �= j)is prohibitively expensive. Instead, averages are often used—as instatistical physics.


Communicability

The average communicability of a node is defined as

�C (i)� :=1

N − 1

�

j �=i

C (i , j) =1

N − 1

�

j �=i

[eA]ij .

Communicability functions can be used to study the spreading ofdiseases (or rumors) and to identify bottlenecks in networks. SeeEstrada, Hatano and B. (Phys. Reports, 2012) for a review.

Recently, community detection algorithms based on variouscommunicability measures have been developed (Fortunato, 2010;Estrada, 2011).


Clustering coefficients

A clustering coefficient measures the degree to which the nodes ina network tend to cluster together. For a node i with degree di , itis defined as

CC (i) =2∆i

di (di − 1)

where ∆i is the number of triangles in G having node i as one ofits vertices.

The clustering coefficient of a graph G is defined as the sum of theclustering coefficients over all the nodes of degree ≥ 2.

Many real world small-world networks, and particularly socialnetworks, tend to have high clustering coefficient (relative torandom networks).


Clustering coefficients (cont.)

The number of triangles in G that a node participates in is given by

∆i =1

2[A3]ii ,

while the total number of triangles in G is given by

∆(G ) =1

6Tr (A3).

Computing clustering coefficients for a graph G reduces tocomputing diagonal entries of A3.

We note for many networks, A3 is a rather dense matrix. Forexample, for the PPI network of beer yeast the percentage ofnonzero entries in A3 is about 19%, compared to 0.27% for A.


spectral graph theory tools for the analysis of …mic/univ700r/benzi_slides.pdfspectral graph...

Documents