e6885 network science lecture 10: dynamic networks --icylin/course/netsci-10/netsci-fall... ·...

1

© 2010 Columbia University

E6885 Network Science Lecture 10:

Dynamic Networks -- I

E 6885 Topics in Signal Processing -- Network Science

Ching-Yung Lin, Dept. of Electrical Engineering, Columbia University

November 22nd, 2010

© 2010 Columbia University2 E6885 Network Science – Lecture 10: Dynamic Networks -- I

Course Structure

Final Project Presentation 1412/20/10

Social Influence and Info Diffusion in Networks -- II1312/13/10

Social Influence and Info Diffusion in Networks -- I1212/06/10

Dynamic Networks -- II1111/29/10

Dynamic Networks -- I1011/22/10

Network Topology Inference -- II911/15/10

Network Topology Inference -- I811/08/10

Network Models -- II710/25/10

Network Models -- I610/18/10

Network Sampling and Estimation510/11/10

Network Visualization410/04/10

Network Partitioning and Clustering309/27/10

Network Representations and Characteristics209/20/10

Overview – Social, Information, and Cognitive Network Analysis109/13/10

Topics CoveredClass

Number

Class

Date

2


Static Process and Dynamic Process

� Quantities associated with phenomena can usefully be thought of as stochastic processes defined on network graphs.

� Moreover, they can be represented in terms of collections of random variables, say X, indexed at least in part on a network graph G=(V,E), either of the form {Xi},for , or {Xi (t)}, for and .

� The vector form of vertex and edge attributes:

� Sometimes, we model the behavior of the variable Y, conditional on X. Alternatively, in some context, it may be the behavior of X, conditional on Y.

i V∈ t T∈i V∈

( )i i VX ∈=X ( 2)( , )

[ ]ij i j VY

∈=Y


Nearest Neighbor Prediction

� Predicting a static process on a

graph.

� Example – Predicting Type of

Practice Using Collaboration

Among Lawyers

– 36 lawyers

– Concentrating on one of two

types of legal practice – 20

work in ligitation and 16 work

in corporate law.

3


Nearest Neighbor Prediction (cont’d)

� Suppose we have observed the adjacency matrix of collaboration among these lawyers, as well as the practice of all but one of them.

� How useful would it be to know the practice of one of the neighbors of this one lawyer of unknown practice?

� Of the collaborations involving the 18 litigation lawyers, roughly 40% are with other litigation lawyers and 60% are with corporate lawyers.

� Among collaborations involving corporate lawyers, the same two numbers are 50% each.

� in this case, knowing the practice of a single collaborator provides little predictive value.

4343Corporate

4329Litigation

CorporateLitigationEdge counts (115)


Inference from neighbors

� Use the known practice of all of the collaborators of a given lawyer.

� Using neighbors to predict. Take the threshold of 0.5

� 13 out of the 16 corporate lawyers are correct (81%).

� 16 out of the 18 litigation lawyers are correct (89%).

4


On the quality of inferring interests from social neighbors (Wen and Lin, KDD 2010)

� Modeling user interests enables personalized services

– More relevant search/recommendation results

– More targeted advertising

� Data about users are sparse

– Many user profiles are static, incomplete and/or outdated

– <10% employees actively participate social software [Brzozowski2009]

� Inferring user interests from neighbors can be a solution

– Also bring up a concern of exposing user’s private information

How true are

“You are who you know”,

“Birds of a Feather Flocks

Together”?


Challenges in Observing Users

� Diverse types of media

–Public social media (friending, blogs, etc.)

• Data are public but limited (esp. in enterprises)

–Private communication media (email, instant messaging, face-to-

face meetings, etc)

• Much more data

• Privacy is a major issue

5


Example of Diverse Types of Media

Number of people participated in top 3 media in an Enterprise with 400K employees

Number of entries:

• Social bookmarking: 400K

• Electronic communication: 20M

• File sharing: 140K


Our Goals

� How well a user’s interests can be inferred from his/her social

neighbors?

� Can the diverse types of media be combined to improve inferring user

interests from social neighbors?

� Can the quality of the inference be predicted based on features of

social neighbors?

–Only sufficiently accurate inference may help personalized

services

6


Our Approach

� Infer user interests from social neighbors

–Model user interests based on multiple types of

information they accessed

–Construct employee social network from communication

data

–Infer using social influence model

�Study the relationship between inference quality and

network characteristics

–Identify effective factors to ensure high quality results for

applications


Dataset

� 25315 users’ contributed content

– 20M email/chats

– 400K social bookmarks

– 20K shared public files

– Profile information

• Job role, division, news categories of interests, etc

� Infer social network based on email/chats

X’: number of emails

7


User Interests Model – Implicit Interests

� Model users’ interests implicitly indicated by their contributed content

–Extract latent topics from the multiple types of content using LDA

–Select top-N distinct topics as the implicit interests model of a user

The degree the

user is interestedThe similarity of

topics


User Interests Model – Explicit Interests

� 29% users manually specify interests in their profile

–A list of selected terms

• From a static 1120-term taxonomy related to work

� Compare implicit and explicit interests

–Explicit interests models are more limited

• Implicit interests cover 60.4% explicit interests

• Explicit interests cover 2.2% implicit interests

8


Infer Interests Based on Social Influence

� Social influence model

–Network autocorrelation model [Leenders02]

• Social influence represented as a weighted combination of

neighbors’ attributes

The weight is an exponential

function of the social distance


Inference Quality

100%

62.1%

44.9%

59.4%

Max

14.1%29.6%Using email/IM data only

21.7%45.1%Using all three data

7.2%12.7%Using file sharing data only

10.7%19.2%Using social bookmark data only

St. DeviationMeanCondition

� Implicit interests: how close the inferred top-20 topics to the ground truth

– Significant advantage in combining multiple sources

– Large variance can affect practical application, thus need predict when to infer interests

– Much better recall than precision

� Explicit interests: precision and recall of inferred terms

27.6%61.5%Recall

26.9%30.1%Precision

St. DeviationMeanMeasure

9


Can Inference Quality be Predicted?

�Hypothesis: inference quality can be predicted from social network properties

–User activeness: the amount of contribution

–In-degree

–Out-degree

–Betweenness

–User management role

�Use Support Vector Regression to perform prediction

�Evaluate prediction

–Precision/recall of the prediction (10-fold cross validation)

–Use prediction to improve inference

• Only infer when we predict it’s high quality


Quality Prediction Results

� Precision/recall of prediction

Implicit Interests Explicit Interests

10


Quality Prediction Results

� Improve inference

39.3%85.7%Recall

101%60.5%Precision

Improvement (%)Improved toMeasure

Implicit Interests

Explicit Interests


Feature Comparison

� “Leave-one-feature-out" comparisons of prediction results

Most social influences are from

1&2-degree neighbors

You neighbors decide how well

you can be inferred

11


Feature Comparison (cont’d)

� “Leave-one-feature-out" comparisons of prediction results

You neighbors’ network

positions may be even

more important than

how active they are

– Formal organizational properties

• Manager neighbors are more important in inference

i.e., more social influence (about 5% more)


Conclusion of the Wen-Lin 2010 KDD paper

� There’s large variance in the quality of inferring user interests from social

neighbors

� The “recall” of the inference is much better than “precision”

� The inference quality may be predicted from social network properties

12


Modeling of the Nearest-neighbor methods

� Modeling can be formalized and extended through the construction of

appropriate statistical models.

–The probabilistic model

–The regression model


Markov Random Fields

� Let G=(V,E) be a graph and be a collection of discrete

random variables defined on V. We say that X is a Markov random field

(MRF) of G if

1( ,..., )v

T

NX X=X

( ) 0P = >X x for all possible outcomes x,

( ) ( )( | ) ( | )i ii i i i i i N NP X x P X x− −= = = = =X x X x

and,

where,

( ) 1 1 1( ,..., , ,..., )v

T

i i i NX X X X− − +=X

Vector of all i’sneighbors

13


Gibbs random fields

� A key feature facilitating the practical usage of Markov random fields is

their equivalence, under appropriate conditions, with Gibbs random fields,

i.e. random vectors X with distributions of the form:

1( ) ( )exp{ ( )}P U

κ= =X x x

Here U(.) is called the energy function and

exp{ ( )}Uκ =∑x

x

is the partition function. The equivalence of MRFs and Gibbs random fields is based on the Mammersley-Clifford theorem in Besag (1974)


Gibbs random field (cont’d)

� The energy function can be decomposed as a sum over cliques in G, in the form

( ) ( )c

c

U U∈

=∑x xℂ

The set of all cliques of all sizes in G.

Clique potentials

14


Auto-Logistic MRFs

� Besag suggested introducing the additional conditions that (1) only cliques of size

one or two have non-zero potential functions, and (2) the conditional probabilities

have an exponential family form.

� The first condition is sometimes referred to as ‘pairwise-only’ dependence’.

� Under these conditions, the energy function takes the form

� If Xi are binary random variables

{ , }

( ) ( )i i i ij i j

i V i j E

U x H x x xβ∈ ∈

= +∑ ∑x

Auto-model

{ , }

( ) i i ij i j

i V i j E

U x x xα β∈ ∈

= +∑ ∑x

Auto-logistic -model


Auto-logistic MRFs (cont’d)

� The conditional probabilities have the form

exp( )

( 1| )1 exp( )

i

i i

i

i ij j

j N

i N N

i ij j

j N

x

P Xx

α β

α β

∈

∈

+

= = =+ +

∑

∑X x

Indicating logistic regression of xi on its neighboring xj’s.

� Assumptions of homogeneity can further simplify this model:

exp( )

( 1| )1 exp( )

i

i i

i

j

j N

i N N

j

j N

x

P Xx

α β

α β

∈

∈

+

= = =+ +

∑

∑X x

15


Auto-logistic MRFs (cont’d)

� This model can be read as dictating that the logarithm of the conditional

odds that Xi=1 scales linearly in the number of neighbors j of i with the

value Xi=1.

1

( 1| )log

( 0 | )

i i

ii i

i N N

j

j Ni N N

P Xx

P Xα β

∈

= == +

= =∑

X x

X x

� The auto-logistic model can be extended to the case where the Xi take on

values {0,1,…,m} for arbitrary positive integer m � multi-level logistic or

multi-color models.


Inference and Prediction for MRFs

� Inference of the low-dimensional parameter θ:

1( ) ( ) exp{ ( ; )}

( )P Uθ θ

κ θ= =X x x

� Maximize this function:

� The maximum pseudo-likelihood estimate (MPLE) of (α,β) is defined as:

log ( | )i ii i N N

i V

P X xθ∈

= =∑ X x

, 1 11ˆˆ( , ) argmax [ ( ) ( ) log[1 exp( )]

i

MPLE j

j N

M x M x xα βα β α β α β∈

= + − + + ∑

Number of vertices with the attribute value 1.Number of vertices of pair with the attribute value 1.

16


Prediction for MRFs

� Predicting some or all of the process X, Given a value for the parameter θ.

� It is relatively straightforward to simulate from this distribution using the Gibbs

sampler, a type of Markov chain Monte Carlo algorithm.

� The Gibbs sampler is an iterative algorithm that allows us to exploit the fact that

the univariate conditional distributions are generally available in relatively simple

and closed form.

� Under appropriate conditions, the chain has a stationary distribution which is

equal to Pθ.


Questions?

e6885 network science lecture 10: dynamic networks --icylin/course/netsci-10/netsci-fall... ·...

Documents