from local network motifs to global invariantsngns/docs/review_2010... · onr muri: nexgenetsci...

ONR MURI: NexGeNetSci

From Local Network Motifs From Local Network Motifs to Global Invariantsto Global Invariants

Third Year Review, October 29, 2010

Victor M. Preciado and Ali JadbabaieDepartment of Electrical and Systems Engineering

University of Pennsylvania

Theory DataAnalysis

Numerical Experiments

LabExperiments

FieldExercises

Real-WorldOperations

• First principles• Rigorous math• Algorithms• Proofs

• Correct statistics• Only as good

as underlying data

• Simulation• Synthetic,

clean data

• Stylized• Controlled• Clean,

realworld data

• SemiControlled•Messy,

realworld data

• Unpredictable• After action

reports in lieu of data

Preciado

Local Motifs and Global Invariants


• Motivation and context

• The role of local structural information

• Spectral analysis from local structural information

• Bounds on spectral properties via optimization

• Implications in dynamical processes

Outline


Complex Network: Properties

• Generic features:– Large number of nodes

– Sparse connectivity

– Lack of regularity

• Examples:– Comm networks (e.g. Internet)

– Social networks (e.g. Facebook)

– Biological networks

• We assume limited structural information:– Privacy and/or security concerns

– Storage/computing limitations??

??

• Challenges when only local structural information is available:– Estimation: How could we aggregate local measurements to

infer global properties of the network?

– Inference: What could we say about the behavior of a dynamical process in the network from local measurements?

– Actuation: How could we modify the structure of a network to induce a desired global behavior?

Complex Networks: Some Challenges

Estimation

Inference

Actuation


• Overly focused on random graph models and degree distributions, but we can have very different networks with the same degree distribution [Li et al., 2005]:

• Main drawbacks:

1. Degree distributions are a zero-th order approximation of the network structure, by far not enough

2. Random models are difficult, if not impossible, to justify from an engineering perspective

Usual Approach in “Network Science”


• We also find random graph models capturing increasingly richer structural properties [Mahadevan et al., 2006]

• Main drawbacks:

– Visual inspection is clearly not enough to measure similarity

– What structural measurements are relevant in the behavior of dynamical processes in networks?

More Structured Random Models

Average degree Degree distribution Joint Degree Distribution

Distribution of Triangles Original HOT model


• Our framework: We consider the dynamical behavior of networks

• Since the eigenvalues and eigenvectors are closely related with thenetwork dynamical behavior, spectral graph theory is a convenientframework to study network dynamics

• Some relationships between spectra and dynamics are:– Spreading processes � adjacency spectral radius

– Synchronization � (combinatorial) Laplacian eigenratio

– Diffusion/Consensus � (normalized) Laplacian eigenvalues

Networks = Graphs + Dynamics

0 500 1000 1500 2000

0

500

1000

1500

2000

nz = 45572

-40 -20 0 20 40 60 800

200

400

600

800

��)

Aggregation of local structural measurements

Dynamical Implications


Inference from Local Measurements� Our problems:

1. - What measurements are most relevant in the behavior of dynamical processes?

2. - How can we aggregate local measurements to say somethingabout the global dynamical behavior?

• We study those problems in the framework of spectral graph theory and convex optimization, without making any assumption on the global network structure (i.e., no random models)


I. Use algebraic graph theory to relate the frequency of certain small subgraphs, or motifs, with the so-called spectral moments of the network

II. Propose a distributed technique to compute the frequency of subgraphs from the distribution of local network measurements

III. Use convex optimization to extract relevant spectral information from a sequence of spectral moments

IV. Study implications on dynamical processes

Structure of our Approach


• Algebraic graph theory allows us to compute spectral moments from local structural information. We use the following result:

• Low-order moments: For k≤3 we have the following expressions

I. From Subgraphs to Spectral Moments

� �klengthofwalksclosednn

Amn

i

kik #

11)(

1

��

�

k=2

k=3

n

ed

nm

i i

212 ��

nt

nm

i i

�� 6

21

3

i

i


• Higher-order moments: As we increase the order of the moments, a variety of more and more complicated subgraphs come into the picture. For k=4, we have the following types of closed walks:

• In the first expression, we observe that local measurements can be aggregated via distributed consensus to compute spectral moments

From Subgraphs to Moments

i i i

j

i

iiiii

ii dddddqn

m �� )1()1(21

4

� �eQn

m 8481

4 ��

Moments from local structural measurements

Moments from local structural measurements

Moments from subgraphsfrequencies


• Key observation: The spectral moments are linear combinations of subgraphs embedding frequencies [Preciado, Jadbabaie, 2010]. The coefficients for all nonisomorphic connected subgraphs with 4 or less nodes are

• For example, the fifth moment can be computed as:

mk

k=4 2 4 - - - - 8 - -

k=5 - - 30 - - 10 - - -

k=6 2 12 24 12 6 - 48 36 -

k=7 - - 126 - - 84 - 112 -

k=8 2 28 168 72 32 64 264 464 528

Moments from Subgraphs Frequencies

� �en

m 3010301

5 ��


• We propose a distributed technique to compute subgraphfrequencies. Note that each subgraph can be ’discovered’ by a number of its nodes. For example, for 1-hop neighborhoods:

• In general, if each node have access to its r-hops neighborhood, wecan discover all subgraphs involved in moments of order up to 2r+1(and part of the subgraphs involved in moments of higher order)

II. Distributed Computation of Moments

� ��

ii

nn 31 ��

�i iii i d

nnn)2(

11 ��inn

Q???

1


• Subgraph with 2,404 nodes and 22,786 edges obtained from crawling the Facebook graph in a breadth-first search around a particular node

• We can compute the relevant quantities

which allow us to compute moments

100

101

102

103

10-4

10-3

10-2

10-1

k,Degree

P(k

)

Node Degree Distribution

100

101

102

103

104

10-4

10-3

10-2

10-1

t,Triangles

P(t

)

Distribution of Triangles

100

101

102

103

100

101

102

103

104

k,Degree

t,T

riang

les

Degree-Triangle Scatter Plot

Empirical Example


• So far, we have

• We now present an SDP-based approach to extract information fromthe spectral moments that

1. It is agnostic, in the sense that it does not make any assumptionon the global network structure (no random model)

2. It allows to study the effect of arbitrarily complicated structuralmeasurements in the network spectral properties

III. Extracting Spectral Info from Moments

Counting subgraph

frequencies

Computing spectral

moments?


• How can we extract information from spectral moments? The following problem, called the classical moment problem, is closely related to ours:

Given a sequence of moments (m1,…,mk), and Borelmeasurable sets T � �� R, we are interested in computing

where m in M(�), M(�) being the set of positive Borelmeasures supported by �.

• Generalization of Markov and Chebyshev´s inequalities from probability theory, when a sequence of moments is available

The Classical Moment Problem


• Using duality theory, we obtain the following formulation [Bertsimas, 2005]:

This dual problem is a sum-of-squares program (SOSP) and can be formulated as a semidefinite program [Parrilo, 2006].

• We define the spectral distribution of a graph as

and define the r.v. X��G. Hence, using SOS, we can compute optimal bounds on Pr(X � T)=#{�i � T}/n when we haveaccess to a sequence of spectral moments

Moment Problems, SOS and SDP


• From the set of spectral moments, we compute optimal bounds on #{�i � [a,b]}/n, and #{�i � [-c,c]}/n

• Notice that only those intervals [a,b] in region B and [-c,c] in Care able to support the whole set of eigenvalues. Hence,– We have a lower bound on the spectral radius �(A)>�*

– We can also compute a bound on the Laplacian eigenratio fromthe Laplacian spectral moments

Numerical Results

a

bB

1

�* cC


IV Dynamical Implications: Spreading Processes

� We study a stochastic dynamical model of viral dissemination:

� - Each node has two possible states:� 0. Susceptible (blue)� 1. Infected (red)

� - Spreading parameters:� � probability of contagion� � probability of recovering

��

�

� Spectral results [Draief et al., 2008]:

� - �(A)>�/� is a necessary condition for a small infection to infect a significant part of the network

� - The larger �(A), the better a network disseminate a virus/rumor


Spreading Processes: Simulations

0 20 40 60 80 100 120 140 1600

50

100

150

200

250

300

0.05% initial infection�/�=35 < 45 < �(A)=60

0 10 20 30 40 500

50

100

150

200

250

300

0.2% initial infection�/�=65 > �(A)=60

Counting subgraph

frequencies

Computing spectral

moments

Bound on �(A)>45.0

Implications on Spreading

# of

infe

ctio

ns

# of

infe

ctio

ns

tt


• Ongoing work: Design incentives for each individual to modify their local neighborhood in order to achieve a particular global spectral property. Some preliminaries results [Preciado et al., 2010]:

Decentralized Network Design


• Adapt our framework to:1. Evolving networks: Tracking the evolution of

subgraphs frequencies and model their interactions

2. Links with weights and directions: Since the eigenvalues become complex, we have to work with 2D support

3. Nodes with attributes

• Incentive design: How can we drive nodes to take local actions that improve the global dynamical behavior of the network?

Future Work


• Our work is devoted to study local structural properties and dynamical processes in large-scale complex networks

• There is a direct relationship between many dynamical processes in networks and the eigenvalues of the underlying graph

• There is plenty of information about the eigenvalue spectra from the distribution of local network measurements

• Our approach is agnostic, in which we do not assume any global structure (no random graphs)

• Our results can be of interest to analyze and design large-scale networks from a spectral point of view

Conclusions


• F. Chung, L. Lu, and V. Vu, "The Spectra of Random Graphs with Given Expected Degrees," Internet Mathematics, vol. 1, pp. 257-275, 2003.

• M. Draief, A. Ganesh, and L. Massoulié, "Thresholds for Virus Spread on Networks," Annals of Applied Probability, vol. 18, pp. 359-378, 2008.

• L. Li, D. Alderson, J.C. Doyle, and W. Willinger, " Towards a Theory of Scale-Free Graphs,“ InternerMath, vol. 2, pp. 431-523, 2005.

• P. Parrilo, Algebraic Techniques and Semidefinite Optimization, Massachusetts Institute of Technology: MIT OpenCourseWare, Spring 2006.

• L.M. Pecora, and T.L. Carroll, "Master Stability Functions for Synchronized Coupled Systems," Physical Review Letters, vol. 80(10), pp. 2109-2112, 1998.

• I. Popescu and D. Bertsimas, "An SDP Approach to Optimal Moment Bounds for Convex Classes of Distributions," Mathematics of Operation Research, vol. 50, pp. 632-657, 2005.

• V.M. Preciado, and G.C. Verghese, "Synchronization in Generalized Erdös-Rényi Networks of Nonlinear Oscillators," IEEE Conference on Decision and Control, pp. 4628-4633, 2005.

• V.M. Preciado, and A. Jadbabaie, "Spectral Analysis of Virus Spreading in Random Geometric Networks," IEEE Conference on Decision and Control, pp. 4802-4807, 2009.

• V.M. Preciado, M.M. Zavlanos, A. Jadbabaie, and G.J. Pappas, “Distributed Control of the LaplacianSpectral Moments of a Network,” American Control Conference, 2010.

• V.M. Preciado and A. Jadbabaie, " From Local Measurements to Network Spectral Properties: Beyond Degree Distributions, " IEEE Conference on Decision and Control, 2010.

Some References


• QUESTIONS?


• We study a collection of resistively coupled nonlinear oscillators

IV.b Dynamical Implications: Synchronization

� �

� �

� Spectral results [Pecora and Carrol]: L=D-A, and �i are its eigenvalues

� For stability of the synchronous state we need the Laplacianeigenration �n/�2 < � , where � depends on the individual oscillatordynamics

� Network dynamics:

� Question: What values of � do make the network synchronize?

�


• Using our SDP approach, we can also bound the Laplacian eigenratiofrom local structural properties via the Laplacian moments

• In the Laplacian moments, not only the frequencies of subgraphs are important, but also the degrees of the nodes involved. For example, for the4th moment the following substructures are involved

• The Laplacian moments are functions of the frequencies of these structures and the degrees of the nodes involved:

Laplacian Moments

� � � �� 3213213214321 ,,

212

22

1,,,,

321,,,

44 144681

)(kkk

Ekkkkkkkkkk

c kkkkPPkkkPPn

Lm


Synchronization: Simulations

Counting frequencies of substructures

Computing Laplacianmoments

Bound on �n/�2

Implications on Synchronization

• We simulate a network of 200 resistively coupled Rossler oscillators

��

from local network motifs to global invariantsngns/docs/review_2010... · onr muri: nexgenetsci...

Documents