talk 2: graph mining tools - svd, ranking, proximity christos faloutsos cmu
TRANSCRIPT
![Page 1: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/1.jpg)
Talk 2: Graph Mining Tools - SVD, ranking, proximity
Christos Faloutsos
CMU
![Page 2: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/2.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 2
Outline• Introduction – Motivation• Task 1: Node importance • Task 2: Recommendations• Task 3: Connection sub-graphs• Conclusions
![Page 3: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/3.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 3
Node importance - Motivation:
• Given a graph (eg., web pages containing the desirable query word)
• Q: Which node is the most important?
![Page 4: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/4.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 4
Node importance - Motivation:
• Given a graph (eg., web pages containing the desirable query word)
• Q: Which node is the most important?
• A1: HITS (SVD = Singular Value Decomposition)
• A2: eigenvector (PageRank)
![Page 5: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/5.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 5
Node importance - motivation
• SVD and eigenvector analysis: very closely related
![Page 6: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/6.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 6
SVD - Detailed outline
• Motivation• Definition - properties• Interpretation• Complexity• Case studies
![Page 7: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/7.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 7
SVD - Motivation
• problem #1: text - LSI: find ‘concepts’• problem #2: compression / dim. reduction
![Page 8: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/8.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 8
SVD - Motivation
• problem #1: text - LSI: find ‘concepts’
![Page 9: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/9.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 9
SVD - Motivation
• Customer-product, for recommendation system:
1 1 1 0 0
2 2 2 0 0
1 1 1 0 0
5 5 5 0 0
0 0 0 2 2
0 0 0 3 30 0 0 1 1
bread
lettu
cebe
ef
vegetarians
meat eaters
tom
atos
chick
en
![Page 10: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/10.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 10
SVD - Motivation
• problem #2: compress / reduce dimensionality
![Page 11: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/11.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 11
Problem - specs
• ~10**6 rows; ~10**3 columns; no updates;
• random access to any cell(s) ; small error: OK
![Page 12: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/12.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 12
SVD - Motivation
![Page 13: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/13.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 13
SVD - Motivation
![Page 14: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/14.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 14
SVD - Detailed outline
• Motivation• Definition - properties• Interpretation• Complexity• Case studies• Additional properties
![Page 15: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/15.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 15
SVD - Definition
(reminder: matrix multiplication
1 2
3 45 6
x 1
-1
3 x 2 2 x 1
=
![Page 16: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/16.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 16
SVD - Definition
(reminder: matrix multiplication
1 2
3 45 6
x 1
-1
3 x 2 2 x 1
=
3 x 1
![Page 17: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/17.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 17
SVD - Definition
(reminder: matrix multiplication
1 2
3 45 6
x 1
-1
3 x 2 2 x 1
=
-1
3 x 1
![Page 18: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/18.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 18
SVD - Definition
(reminder: matrix multiplication
1 2
3 45 6
x 1
-1
3 x 2 2 x 1
=
-1
-1
3 x 1
![Page 19: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/19.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 19
SVD - Definition
(reminder: matrix multiplication
1 2
3 45 6
x 1
-1=
-1
-1-1
![Page 20: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/20.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 20
SVD - Definition
A[n x m] = U[n x r] r x r] (V[m x r])T
• A: n x m matrix (eg., n documents, m terms)
• U: n x r matrix (n documents, r concepts)• : r x r diagonal matrix (strength of each
‘concept’) (r : rank of the matrix)• V: m x r matrix (m terms, r concepts)
![Page 21: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/21.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 21
SVD - Definition
• A = U VT - example:
![Page 22: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/22.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 22
SVD - Properties
THEOREM [Press+92]: always possible to decompose matrix A into A = U VT , where
• U, V: unique (*)• U, V: column orthonormal (ie., columns are unit
vectors, orthogonal to each other)– UT U = I; VT V = I (I: identity matrix)
• : singular are positive, and sorted in decreasing order
![Page 23: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/23.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 23
SVD - Example
• A = U VT - example:
1 1 1 0 0
2 2 2 0 0
1 1 1 0 0
5 5 5 0 0
0 0 0 2 2
0 0 0 3 30 0 0 1 1
datainf.
retrieval
brain lung
0.18 0
0.36 0
0.18 0
0.90 0
0 0.53
0 0.800 0.27
=CS
MD
9.64 0
0 5.29x
0.58 0.58 0.58 0 0
0 0 0 0.71 0.71
x
![Page 24: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/24.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 24
SVD - Example
• A = U VT - example:
1 1 1 0 0
2 2 2 0 0
1 1 1 0 0
5 5 5 0 0
0 0 0 2 2
0 0 0 3 30 0 0 1 1
datainf.
retrieval
brain lung
0.18 0
0.36 0
0.18 0
0.90 0
0 0.53
0 0.800 0.27
=CS
MD
9.64 0
0 5.29x
0.58 0.58 0.58 0 0
0 0 0 0.71 0.71
x
CS-conceptMD-concept
![Page 25: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/25.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 25
SVD - Example
• A = U VT - example:
1 1 1 0 0
2 2 2 0 0
1 1 1 0 0
5 5 5 0 0
0 0 0 2 2
0 0 0 3 30 0 0 1 1
datainf.
retrieval
brain lung
0.18 0
0.36 0
0.18 0
0.90 0
0 0.53
0 0.800 0.27
=CS
MD
9.64 0
0 5.29x
0.58 0.58 0.58 0 0
0 0 0 0.71 0.71
x
CS-conceptMD-concept
doc-to-concept similarity matrix
![Page 26: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/26.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 26
SVD - Example
• A = U VT - example:
1 1 1 0 0
2 2 2 0 0
1 1 1 0 0
5 5 5 0 0
0 0 0 2 2
0 0 0 3 30 0 0 1 1
datainf.
retrieval
brain lung
0.18 0
0.36 0
0.18 0
0.90 0
0 0.53
0 0.800 0.27
=CS
MD
9.64 0
0 5.29x
0.58 0.58 0.58 0 0
0 0 0 0.71 0.71
x
‘strength’ of CS-concept
![Page 27: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/27.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 27
SVD - Example
• A = U VT - example:
1 1 1 0 0
2 2 2 0 0
1 1 1 0 0
5 5 5 0 0
0 0 0 2 2
0 0 0 3 30 0 0 1 1
datainf.
retrieval
brain lung
0.18 0
0.36 0
0.18 0
0.90 0
0 0.53
0 0.800 0.27
=CS
MD
9.64 0
0 5.29x
0.58 0.58 0.58 0 0
0 0 0 0.71 0.71
x
term-to-conceptsimilarity matrix
CS-concept
![Page 28: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/28.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 28
SVD - Example
• A = U VT - example:
1 1 1 0 0
2 2 2 0 0
1 1 1 0 0
5 5 5 0 0
0 0 0 2 2
0 0 0 3 30 0 0 1 1
datainf.
retrieval
brain lung
0.18 0
0.36 0
0.18 0
0.90 0
0 0.53
0 0.800 0.27
=CS
MD
9.64 0
0 5.29x
0.58 0.58 0.58 0 0
0 0 0 0.71 0.71
x
term-to-conceptsimilarity matrix
CS-concept
![Page 29: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/29.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 29
SVD - Detailed outline
• Motivation• Definition - properties• Interpretation• Complexity• Case studies• Additional properties
![Page 30: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/30.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 30
SVD - Interpretation #1
‘documents’, ‘terms’ and ‘concepts’:• U: document-to-concept similarity matrix• V: term-to-concept sim. matrix• : its diagonal elements: ‘strength’ of each
concept
![Page 31: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/31.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 31
SVD – Interpretation #1
‘documents’, ‘terms’ and ‘concepts’:Q: if A is the document-to-term matrix, what
is AT A?A:Q: A AT ?A:
![Page 32: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/32.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 32-32
SVD – Interpretation #1
‘documents’, ‘terms’ and ‘concepts’:Q: if A is the document-to-term matrix, what
is AT A?A: term-to-term ([m x m]) similarity matrixQ: A AT ?A: document-to-document ([n x n]) similarity
matrix
![Page 33: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/33.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 33Copyright: Faloutsos, Tong (2009) 2-33
SVD properties
• V are the eigenvectors of the covariance matrix ATA
• U are the eigenvectors of the Gram (inner-product) matrix AAT
Further reading:1. Ian T. Jolliffe, Principal Component Analysis (2nd ed), Springer, 2002.2. Gilbert Strang, Linear Algebra and Its Applications (4th ed), Brooks Cole, 2005.
![Page 34: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/34.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 34
SVD - Interpretation #2
• best axis to project on: (‘best’ = min sum of squares of projection errors)
![Page 35: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/35.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 35
SVD - Motivation
![Page 36: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/36.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 36
SVD - interpretation #2
• minimum RMS error
SVD: givesbest axis to project
v1
first singular
vector
![Page 37: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/37.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 37
SVD - Interpretation #2
![Page 38: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/38.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 38
SVD - Interpretation #2
• A = U VT - example:
1 1 1 0 0
2 2 2 0 0
1 1 1 0 0
5 5 5 0 0
0 0 0 2 2
0 0 0 3 30 0 0 1 1
0.18 0
0.36 0
0.18 0
0.90 0
0 0.53
0 0.800 0.27
=9.64 0
0 5.29x
0.58 0.58 0.58 0 0
0 0 0 0.71 0.71
x
v1
![Page 39: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/39.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 39
SVD - Interpretation #2
• A = U VT - example:
1 1 1 0 0
2 2 2 0 0
1 1 1 0 0
5 5 5 0 0
0 0 0 2 2
0 0 0 3 30 0 0 1 1
0.18 0
0.36 0
0.18 0
0.90 0
0 0.53
0 0.800 0.27
=9.64 0
0 5.29x
0.58 0.58 0.58 0 0
0 0 0 0.71 0.71
x
variance (‘spread’) on the v1 axis
![Page 40: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/40.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 40
SVD - Interpretation #2
• A = U VT - example:– U gives the coordinates of the points in the
projection axis
1 1 1 0 0
2 2 2 0 0
1 1 1 0 0
5 5 5 0 0
0 0 0 2 2
0 0 0 3 30 0 0 1 1
0.18 0
0.36 0
0.18 0
0.90 0
0 0.53
0 0.800 0.27
=9.64 0
0 5.29x
0.58 0.58 0.58 0 0
0 0 0 0.71 0.71
x
![Page 41: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/41.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 41
SVD - Interpretation #2
• More details• Q: how exactly is dim. reduction done?
1 1 1 0 0
2 2 2 0 0
1 1 1 0 0
5 5 5 0 0
0 0 0 2 2
0 0 0 3 30 0 0 1 1
0.18 0
0.36 0
0.18 0
0.90 0
0 0.53
0 0.800 0.27
=9.64 0
0 5.29x
0.58 0.58 0.58 0 0
0 0 0 0.71 0.71
x
![Page 42: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/42.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 42
SVD - Interpretation #2
• More details• Q: how exactly is dim. reduction done?• A: set the smallest singular values to zero:
1 1 1 0 0
2 2 2 0 0
1 1 1 0 0
5 5 5 0 0
0 0 0 2 2
0 0 0 3 30 0 0 1 1
0.18 0
0.36 0
0.18 0
0.90 0
0 0.53
0 0.800 0.27
=9.64 0
0 5.29x
0.58 0.58 0.58 0 0
0 0 0 0.71 0.71
x
![Page 43: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/43.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 43
SVD - Interpretation #2
1 1 1 0 0
2 2 2 0 0
1 1 1 0 0
5 5 5 0 0
0 0 0 2 2
0 0 0 3 30 0 0 1 1
0.18 0
0.36 0
0.18 0
0.90 0
0 0.53
0 0.800 0.27
~9.64 0
0 0x
0.58 0.58 0.58 0 0
0 0 0 0.71 0.71
x
![Page 44: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/44.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 44
SVD - Interpretation #2
1 1 1 0 0
2 2 2 0 0
1 1 1 0 0
5 5 5 0 0
0 0 0 2 2
0 0 0 3 30 0 0 1 1
0.18 0
0.36 0
0.18 0
0.90 0
0 0.53
0 0.800 0.27
~9.64 0
0 0x
0.58 0.58 0.58 0 0
0 0 0 0.71 0.71
x
![Page 45: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/45.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 45
SVD - Interpretation #2
1 1 1 0 0
2 2 2 0 0
1 1 1 0 0
5 5 5 0 0
0 0 0 2 2
0 0 0 3 30 0 0 1 1
0.18
0.36
0.18
0.90
0
00
~9.64
x
0.58 0.58 0.58 0 0
x
![Page 46: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/46.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 46
SVD - Interpretation #2
1 1 1 0 0
2 2 2 0 0
1 1 1 0 0
5 5 5 0 0
0 0 0 2 2
0 0 0 3 30 0 0 1 1
~
1 1 1 0 0
2 2 2 0 0
1 1 1 0 0
5 5 5 0 0
0 0 0 0 0
0 0 0 0 00 0 0 0 0
![Page 47: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/47.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 47
SVD - Interpretation #2
Exactly equivalent:‘spectral decomposition’ of the matrix:
1 1 1 0 0
2 2 2 0 0
1 1 1 0 0
5 5 5 0 0
0 0 0 2 2
0 0 0 3 30 0 0 1 1
0.18 0
0.36 0
0.18 0
0.90 0
0 0.53
0 0.800 0.27
=9.64 0
0 5.29x
0.58 0.58 0.58 0 0
0 0 0 0.71 0.71
x
![Page 48: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/48.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 48
SVD - Interpretation #2
Exactly equivalent:‘spectral decomposition’ of the matrix:
1 1 1 0 0
2 2 2 0 0
1 1 1 0 0
5 5 5 0 0
0 0 0 2 2
0 0 0 3 30 0 0 1 1
= x xu1 u2
1
2
v1
v2
![Page 49: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/49.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 49
SVD - Interpretation #2
Exactly equivalent:‘spectral decomposition’ of the matrix:
1 1 1 0 0
2 2 2 0 0
1 1 1 0 0
5 5 5 0 0
0 0 0 2 2
0 0 0 3 30 0 0 1 1
= u11 vT1 u22 vT
2+ +...n
m
![Page 50: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/50.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 50
SVD - Interpretation #2
Exactly equivalent:‘spectral decomposition’ of the matrix:
1 1 1 0 0
2 2 2 0 0
1 1 1 0 0
5 5 5 0 0
0 0 0 2 2
0 0 0 3 30 0 0 1 1
= u11 vT1 u22 vT
2+ +...n
m
n x 1 1 x m
r terms
![Page 51: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/51.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 51
SVD - Interpretation #2
approximation / dim. reduction:by keeping the first few terms (Q: how many?)
1 1 1 0 0
2 2 2 0 0
1 1 1 0 0
5 5 5 0 0
0 0 0 2 2
0 0 0 3 30 0 0 1 1
= u11 vT1 u22 vT
2+ +...n
m
assume: 1 >= 2 >= ...
![Page 52: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/52.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 52
SVD - Interpretation #2
A (heuristic - [Fukunaga]): keep 80-90% of ‘energy’ (= sum of squares of i ’s)
1 1 1 0 0
2 2 2 0 0
1 1 1 0 0
5 5 5 0 0
0 0 0 2 2
0 0 0 3 30 0 0 1 1
= u11 vT1 u22 vT
2+ +...n
m
assume: 1 >= 2 >= ...
![Page 53: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/53.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 53
SVD - Detailed outline
• Motivation• Definition - properties• Interpretation
– #1: documents/terms/concepts– #2: dim. reduction– #3: picking non-zero, rectangular ‘blobs’
• Complexity• Case studies• Additional properties
![Page 54: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/54.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 54
SVD - Interpretation #3
• finds non-zero ‘blobs’ in a data matrix
1 1 1 0 0
2 2 2 0 0
1 1 1 0 0
5 5 5 0 0
0 0 0 2 2
0 0 0 3 30 0 0 1 1
0.18 0
0.36 0
0.18 0
0.90 0
0 0.53
0 0.800 0.27
=9.64 0
0 5.29x
0.58 0.58 0.58 0 0
0 0 0 0.71 0.71
x
![Page 55: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/55.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 55
SVD - Interpretation #3
• finds non-zero ‘blobs’ in a data matrix
1 1 1 0 0
2 2 2 0 0
1 1 1 0 0
5 5 5 0 0
0 0 0 2 2
0 0 0 3 30 0 0 1 1
0.18 0
0.36 0
0.18 0
0.90 0
0 0.53
0 0.800 0.27
=9.64 0
0 5.29x
0.58 0.58 0.58 0 0
0 0 0 0.71 0.71
x
![Page 56: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/56.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 56
SVD - Interpretation #3
• finds non-zero ‘blobs’ in a data matrix =• ‘communities’ (bi-partite cores, here)
1 1 1 0 0
2 2 2 0 0
1 1 1 0 0
5 5 5 0 0
0 0 0 2 2
0 0 0 3 30 0 0 1 1
Row 1
Row 4
Col 1
Col 3
Col 4Row 5
Row 7
![Page 57: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/57.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 57
SVD - Detailed outline
• Motivation• Definition - properties• Interpretation• Complexity• Case studies• Additional properties
![Page 58: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/58.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 58
SVD - Complexity
• O( n * m * m) or O( n * n * m) (whichever is less)
• less work, if we just want singular values• or if we want first k singular vectors• or if the matrix is sparse [Berry]• Implemented: in any linear algebra package
(LINPACK, matlab, Splus, mathematica ...)
![Page 59: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/59.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 59
SVD - conclusions so far
• SVD: A= U VT : unique (*)• U: document-to-concept similarities• V: term-to-concept similarities• : strength of each concept• dim. reduction: keep the first few strongest
singular values (80-90% of ‘energy’)– SVD: picks up linear correlations
• SVD: picks up non-zero ‘blobs’
![Page 60: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/60.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 60
SVD - Detailed outline
• Motivation
• Definition - properties
• Interpretation
• Complexity
• SVD properties
• Case studies
• Conclusions
![Page 61: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/61.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 61
SVD - Other properties - summary
• can produce orthogonal basis (obvious) (who cares?)
• can solve over- and under-determined linear problems (see C(1) property)
• can compute ‘fixed points’ (= ‘steady state prob. in Markov chains’) (see C(4) property)
![Page 62: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/62.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 62
SVD -outline of properties
• (A): obvious• (B): less obvious• (C): least obvious (and most powerful!)
![Page 63: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/63.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 63
Properties - by defn.:
A(0): A[n x m] = U [ n x r ] [ r x r ] VT [ r x m]
A(1): UT [r x n] U [n x r ] = I [r x r ] (identity matrix)
A(2): VT [r x n] V [n x r ] = I [r x r ]
A(3): k = diag( 1k, 2
k, ... rk ) (k: ANY real
number)A(4): AT = V UT
![Page 64: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/64.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 64
Less obvious properties
A(0): A[n x m] = U [ n x r ] [ r x r ] VT [ r x m]
B(1): A [n x m] (AT) [m x n] = ??
![Page 65: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/65.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 65
Less obvious properties
A(0): A[n x m] = U [ n x r ] [ r x r ] VT [ r x m]
B(1): A [n x m] (AT) [m x n] = U 2 UT
symmetric; Intuition?
![Page 66: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/66.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 66
Less obvious properties
A(0): A[n x m] = U [ n x r ] [ r x r ] VT [ r x m]
B(1): A [n x m] (AT) [m x n] = U 2 UT
symmetric; Intuition?‘document-to-document’ similarity matrix
B(2): symmetrically, for ‘V’ (AT) [m x n] A [n x m] = V L2 VT Intuition?
![Page 67: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/67.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 67
Less obvious properties
A: term-to-term similarity matrix
B(3): ( (AT) [m x n] A [n x m] ) k= V 2k VT
and
B(4): (AT A )
k ~ v1 12k v1
T for k>>1
where
v1: [m x 1] first column (singular-vector) of V
1: strongest singular value
![Page 68: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/68.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 68
Less obvious properties
B(4): (AT A )
k ~ v1 12k v1
T for k>>1
B(5): (AT A )
k v’ ~ (constant) v1
ie., for (almost) any v’, it converges to a vector parallel to v1
Thus, useful to compute first singular vector/value (as well as the next ones, too...)
![Page 69: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/69.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 69
Less obvious properties - repeated:
A(0): A[n x m] = U [ n x r ] [ r x r ] VT [ r x m]
B(1): A [n x m] (AT) [m x n] = U 2 UT
B(2): (AT) [m x n] A [n x m] = V 2 VT
B(3): ( (AT) [m x n] A [n x m] ) k= V 2k VT
B(4): (AT A )
k ~ v1 12k v1
T
B(5): (AT A )
k v’ ~ (constant) v1
![Page 70: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/70.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 70
Least obvious properties - cont’d
A(0): A[n x m] = U [ n x r ] [ r x r ] VT [ r x m]
C(2): A [n x m] v1 [m x 1] = 1 u1 [n x 1]
where v1 , u1 the first (column) vectors of V, U. (v1 == right-singular-vector)
C(3): symmetrically: u1T A = 1 v1
T
u1 == left-singular-vector
Therefore:
![Page 71: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/71.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 71
Least obvious properties - cont’d
A(0): A[n x m] = U [ n x r ] [ r x r ] VT [ r x m]
C(4): AT A v1 = 12 v1
(fixed point - the dfn of eigenvector for a symmetric matrix)
![Page 72: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/72.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 72
Least obvious properties - altogether
A(0): A[n x m] = U [ n x r ] [ r x r ] VT [ r x m]
C(1): A [n x m] x [m x 1] = b [n x 1]
then, x0 = V (-1) UT b: shortest, actual or least-squares solution
C(2): A [n x m] v1 [m x 1] = 1 u1 [n x 1]
C(3): u1T A = 1 v1
T
C(4): AT A v1 = 12 v1
![Page 73: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/73.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 73
Properties - conclusions
A(0): A[n x m] = U [ n x r ] [ r x r ] VT [ r x m]
B(5): (AT A )
k v’ ~ (constant) v1
C(1): A [n x m] x [m x 1] = b [n x 1]
then, x0 = V (-1) UT b: shortest, actual or least-squares solution
C(4): AT A v1 = 12 v1
![Page 74: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/74.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 74
SVD - detailed outline
• ...• SVD properties• case studies
– Kleinberg’s algorithm– Google’s algorithm
• Conclusions
![Page 75: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/75.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 75
Kleinberg’s algo (HITS)
Kleinberg, Jon (1998). Authoritative sources in a hyperlinked environment. Proc. 9th ACM-SIAM Symposium on Discrete Algorithms.
![Page 76: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/76.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 76
Recall: problem dfn
• Given a graph (eg., web pages containing the desirable query word)
• Q: Which node is the most important?
![Page 77: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/77.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 77
Kleinberg’s algorithm• Problem dfn: given the web and a query• find the most ‘authoritative’ web pages for
this query
Step 0: find all pages containing the query terms
Step 1: expand by one move forward and backward
![Page 78: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/78.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 78
Kleinberg’s algorithm
• Step 1: expand by one move forward and backward
![Page 79: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/79.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 79
Kleinberg’s algorithm
• on the resulting graph, give high score (= ‘authorities’) to nodes that many important nodes point to
• give high importance score (‘hubs’) to nodes that point to good ‘authorities’)
hubs authorities
![Page 80: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/80.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 80
Kleinberg’s algorithm
observations
• recursive definition!
• each node (say, ‘i’-th node) has both an authoritativeness score ai and a hubness score hi
![Page 81: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/81.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 81
Kleinberg’s algorithm
Let E be the set of edges and A be the adjacency matrix: the (i,j) is 1 if the edge from i to j exists
Let h and a be [n x 1] vectors with the ‘hubness’ and ‘authoritativiness’ scores.
Then:
![Page 82: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/82.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 82
Kleinberg’s algorithm
Then:
ai = hk + hl + hm
that is
ai = Sum (hj) over all j that (j,i) edge exists
or
a = AT h
k
l
m
i
![Page 83: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/83.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 83
Kleinberg’s algorithm
symmetrically, for the ‘hubness’:
hi = an + ap + aq
that is
hi = Sum (qj) over all j that (i,j) edge exists
or
h = A a
p
n
q
i
![Page 84: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/84.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 84
Kleinberg’s algorithm
In conclusion, we want vectors h and a such that:
h = A a
a = AT hRecall properties:
C(2): A [n x m] v1 [m x 1] = 1 u1 [n x 1]
C(3): u1T A = 1 v1
T
=
![Page 85: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/85.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 85
Kleinberg’s algorithmIn short, the solutions to
h = A a
a = AT h
are the left- and right- singular-vectors of the adjacency matrix A.
Starting from random a’ and iterating, we’ll eventually converge
(Q: to which of all the singular-vectors? why?)
![Page 86: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/86.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 86
Kleinberg’s algorithm
(Q: to which of all the singular-vectors? why?)
A: to the ones of the strongest singular-value, because of property B(5):
B(5): (AT A )
k v’ ~ (constant) v1
![Page 87: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/87.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 87
Kleinberg’s algorithm - results
Eg., for the query ‘java’:
0.328 www.gamelan.com
0.251 java.sun.com
0.190 www.digitalfocus.com (“the java developer”)
![Page 88: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/88.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 88
Kleinberg’s algorithm - discussion
• ‘authority’ score can be used to find ‘similar pages’ (how?)
![Page 89: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/89.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 89
SVD - detailed outline
• ...• Complexity• SVD properties• Case studies
– Kleinberg’s algorithm (HITS)– Google’s algorithm
• Conclusions
![Page 90: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/90.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 90
PageRank (google)
•Brin, Sergey and Lawrence Page (1998). Anatomy of a Large-Scale Hypertextual Web Search Engine. 7th Intl World Wide Web Conf.
LarryPage
SergeyBrin
![Page 91: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/91.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 91
Problem: PageRank
Given a directed graph, find its most interesting/central node
A node is important,if it is connected with important nodes(recursive, but OK!)
![Page 92: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/92.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 92
Problem: PageRank - solution
Given a directed graph, find its most interesting/central node
Proposed solution: Random walk; spot most ‘popular’ node (-> steady state prob. (ssp))
A node has high ssp,if it is connected with high ssp nodes(recursive, but OK!)
![Page 93: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/93.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 93
(Simplified) PageRank algorithm
• Let A be the adjacency matrix;
• let B be the transition matrix: transpose, column-normalized - then
1 2 3
45
p1
p2
p3
p4
p5
p1
p2
p3
p4
p5
=
To From
B1
1 1
1/2 1/2
1/2
1/2
![Page 94: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/94.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 94
(Simplified) PageRank algorithm
• B p = p
p1
p2
p3
p4
p5
p1
p2
p3
p4
p5
=
B p = p
1
1 1
1/2 1/2
1/2
1/2
1 2 3
45
![Page 95: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/95.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 95
Definitions
A Adjacency matrix (from-to)
D Degree matrix = (diag ( d1, d2, …, dn) )
B Transition matrix: to-from, column normalized
B = AT D-1
![Page 96: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/96.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 96
(Simplified) PageRank algorithm
• B p = 1 * p
• thus, p is the eigenvector that corresponds to the highest eigenvalue (=1, since the matrix is
column-normalized)
• Why does such a p exist? – p exists if B is nxn, nonnegative, irreducible
[Perron–Frobenius theorem]
![Page 97: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/97.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 97
(Simplified) PageRank algorithm
• In short: imagine a particle randomly moving along the edges
• compute its steady-state probabilities (ssp)
Full version of algo: with occasional random jumps
Why? To make the matrix irreducible
![Page 98: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/98.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 98
Full Algorithm
• With probability 1-c, fly-out to a random node
• Then, we havep = c B p + (1-c)/n 1 =>
p = (1-c)/n [I - c B] -1 1
![Page 99: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/99.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 99
Alternative notation
M Modified transition matrix
M = c B + (1-c)/n 1 1T
Then
p = M p
That is: the steady state probabilities =
PageRank scores form the first eigenvector of the ‘modified transition matrix’
![Page 100: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/100.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 100
Parenthesis: intuition behind eigenvectors
![Page 101: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/101.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 101
Formal definition
If A is a (n x n) square matrix , x) is an eigenvalue/eigenvector pair of A if A x = x
CLOSELY related to singular values:
![Page 102: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/102.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 102
Property #1: Eigen- vs singular-values
if
B[n x m] = U[n x r] r x r] (V[m x r])T
then A = (BTB) is symmetric and
C(4): BT B vi = i2 vi
ie, v1 , v2 , ...: eigenvectors of A = (BTB)
![Page 103: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/103.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 103
Property #2
• If A[nxn] is a real, symmetric matrix
• Then it has n real eigenvalues
(if A is not symmetric, some eigenvalues may be complex)
![Page 104: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/104.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 104
Property #3
• If A[nxn] is a real, symmetric matrix
• Then it has n real eigenvalues
• And they agree with its n singular values, except possibly for the sign
![Page 105: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/105.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 105
Intuition
• A as vector transformation
2 11 3
A
10
x
21
x’
= x
x’
2
1
1
3
![Page 106: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/106.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 106
Intuition
• By defn., eigenvectors remain parallel to themselves (‘fixed points’)
2 11 3
A0.52
0.85
v1v1
=
0.52
0.853.62 *
1
![Page 107: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/107.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 107
Convergence
• Usually, fast:
![Page 108: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/108.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 108
Convergence
• Usually, fast:
![Page 109: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/109.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 109
Convergence
• Usually, fast:• depends on ratio
1 : 21
2
![Page 110: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/110.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 110
Kleinberg/google - conclusions
SVD helps in graph analysis:
hub/authority scores: strongest left- and right- singular-vectors of the adjacency matrix
random walk on a graph: steady state probabilities are given by the strongest eigenvector of the (modified) transition matrix
![Page 111: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/111.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 111
Conclusions
• SVD: a valuable tool
• given a document-term matrix, it finds ‘concepts’ (LSI)
• ... and can find fixed-points or steady-state probabilities (google/ Kleinberg/ Markov Chains)
![Page 112: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/112.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 112
Conclusions cont’d
(We didn’t discuss/elaborate, but, SVD
• ... can reduce dimensionality (KL)
• ... and can find rules (PCA; RatioRules)
• ... and can solve optimally over- and under-constraint linear systems (least squares / query feedbacks)
![Page 113: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/113.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 113
References
• Berry, Michael: http://www.cs.utk.edu/~lsi/
• Brin, S. and L. Page (1998). Anatomy of a Large-Scale Hypertextual Web Search Engine. 7th Intl World Wide Web Conf.
![Page 114: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/114.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 114
References
• Christos Faloutsos, Searching Multimedia Databases by Content, Springer, 1996. (App. D)
• Fukunaga, K. (1990). Introduction to Statistical Pattern Recognition, Academic Press.
• I.T. Jolliffe Principal Component Analysis Springer, 2002 (2nd ed.)
![Page 115: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/115.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 115
References cont’d
• Kleinberg, J. (1998). Authoritative sources in a hyperlinked environment. Proc. 9th ACM-SIAM Symposium on Discrete Algorithms.
• Press, W. H., S. A. Teukolsky, et al. (1992). Numerical Recipes in C, Cambridge University Press. www.nr.com
![Page 116: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/116.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 116
Outline• Introduction – Motivation• Task 1: Node importance • Task 2: Recommendations & proximity• Task 3: Connection sub-graphs• Conclusions
![Page 117: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/117.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 117
Acknowledgement:
Most of the foils in ‘Task 2’ are by
Hanghang TONGwww.cs.cmu.edu/~htong
![Page 118: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/118.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 118
Detailed outline
• Problem dfn and motivation
• Solution: Random walk with restarts
• Efficient computation
• Case study: image auto-captioning
• Extensions: bi-partite graphs; tracking
• Conclusions
![Page 119: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/119.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 119
A
B i
ii
i
Motivation: Link Prediction
Should we introduceMr. A to Mr. B?
?
![Page 120: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/120.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 120
Motivation - recommendations
customers Products / movies
‘smith’
Terminator 2 ??
![Page 121: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/121.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 121
Answer: proximity
• ‘yes’, if ‘A’ and ‘B’ are ‘close’• ‘yes’, if ‘smith’ and ‘terminator 2’ are
‘close’
QUESTIONS in this part:- How to measure ‘closeness’/proximity?- How to do it quickly?- What else can we do, given proximity
scores?
![Page 122: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/122.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 122
How close is ‘A’ to ‘B’?
A BH1 1
D1 1
E
F
G1 11
I J1
1 1
a.k.a Relevance, Closeness, ‘Similarity’…
![Page 123: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/123.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 123
Why is it useful?
• RecommendationAnd many more• Image captioning [Pan+]• Conn. / CenterPiece subgraphs [Faloutsos+], [Tong+],
[Koren+]and• Link prediction [Liben-Nowell+], [Tong+]• Ranking [Haveliwala], [Chakrabarti+]• Email Management [Minkov+]• Neighborhood Formulation [Sun+]• Pattern matching [Tong+]• Collaborative Filtering [Fouss+]• …
![Page 124: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/124.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 124
Test Image
Sea Sun Sky Wave Cat Forest Tiger Grass
Image
Keyword
Region Automatic Image Captioning
Q: How to assign keywords to the test image?A: Proximity! [Pan+ 2004]
![Page 125: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/125.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 125
Center-Piece Subgraph(CePS)
A C
B
A C
B
Original GraphCePS
Q: How to find hub for the black nodes?A: Proximity! [Tong+ KDD 2006]
CePS guy
Input Output
![Page 126: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/126.jpg)
Detailed outline
• Problem dfn and motivation
• Solution: Random walk with restarts
• Efficient computation
• Case study: image auto-captioning
• Extensions: bi-partite graphs; tracking
• Conclusions
Lipari 2010 (C) 2010, C. Faloutsos 126
![Page 127: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/127.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 127
How close is ‘A’ to ‘B’?
A BH1 1
D1 1
E
F
G1 11
I J1
1 1Should be close, if they have - many, - short- ‘heavy’ paths
![Page 128: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/128.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 128
Why not shortest path?
A: ‘pizza delivery guy’ problem
A BD1 1
A BD1 1
E
F
G1 11
Some ``bad’’ proximities
![Page 129: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/129.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 129
A BD1 1
A BD1 11 E
Why not max. netflow?
A: No penalty for long paths
Some ``bad’’ proximities
![Page 130: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/130.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 130
What is a ``good’’ Proximity?
A BH1 1
D1 1
E
F
G1 11
I J1
1 1
• Multiple Connections
• Quality of connection
•Direct & In-directed Conns
•Length, Degree, Weight…
…
![Page 131: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/131.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 131
1
4
3
2
56
7
910
8
11
12
Random walk with restart
[Haveliwala’02]
![Page 132: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/132.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 132
Random walk with restartNode 4
Node 1Node 2Node 3Node 4Node 5Node 6Node 7Node 8Node 9Node 10Node 11Node 12
0.130.100.130.220.130.050.050.080.040.030.040.02
1
4
3
2
56
7
910
811
120.13
0.10
0.13
0.13
0.05
0.05
0.08
0.04
0.02
0.04
0.03
Ranking vector More red, more relevant
Nearby nodes, higher scores
4r
![Page 133: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/133.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 133
2c 3cQ c ...W 2W 3W
Why RWR is a good score?
all paths from i to j with length 1
all paths from i to j with length 2
all paths from i to j with length 3
W : adjacency matrix. c: damping factor
1( )Q I cW ,( , ) i jQ i j r
i
j
![Page 134: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/134.jpg)
Detailed outline
• Problem dfn and motivation
• Solution: Random walk with restarts– variants
• Efficient computation
• Case study: image auto-captioning
• Extensions: bi-partite graphs; tracking
• Conclusions
Lipari 2010 (C) 2010, C. Faloutsos 134
![Page 135: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/135.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 135
Variant: escape probability
• Define Random Walk (RW) on the graph• Esc_Prob(CMUParis)
– Prob (starting at CMU, reaches Paris before returning to CMU)
CMU Paristhe remaining graph
Esc_Prob = Pr (smile before cry)
![Page 136: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/136.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 136
Other Variants• Other measure by RWs
– Community Time/Hitting Time [Fouss+]– SimRank [Jeh+]
• Equivalence of Random Walks– Electric Networks:
• EC [Doyle+]; SAEC[Faloutsos+]; CFEC[Koren+]
– Spring Systems
• Katz [Katz], [Huang+], [Scholkopf+]• Matrix-Forest-based Alg [Chobotarev+]
![Page 137: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/137.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 137
Other Variants• Other measure by RWs
– Community Time/Hitting Time [Fouss+]– SimRank [Jeh+]
• Equivalence of Random Walks– Electric Networks:
• EC [Doyle+]; SAEC[Faloutsos+]; CFEC[Koren+]
– Spring Systems
• Katz [Katz], [Huang+], [Scholkopf+]• Matrix-Forest-based Alg [Chobotarev+]
All are “related to” or “similar to” random walk with restart!
![Page 138: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/138.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 138
Map of proximity measurements
RWR
Esc_Prob + Sink
Hitting Time/Commute
Time
Effective Conductance
String System
Regularized Un-constrainedQuad Opt.
Harmonic Func. ConstrainedQuad Opt.
Mathematic Tools
X out-degree
“voltage = position”
relax
4 ssp decides 1 esc_prob
KatzNormalize
Physical Models
![Page 139: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/139.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 139
Notice: Asymmetry (even in undirected graphs)
A
B C
DE
C-> A : highA-> C: low
![Page 140: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/140.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 140
Summary of Proximity Definitions• Goal: Summarize multiple relationships
• Solutions– Basic: Random Walk with Restarts
• [Haweliwala’02] [Pan+ 2004][Sun+ 2006][Tong+ 2006]
– Properties: Asymmetry• [Koren+ 2006][Tong+ 2007] [Tong+ 2008]
– Variants: Esc_Prob and many others.• [Faloutsos+ 2004] [Koren+ 2006][Tong+ 2007]
![Page 141: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/141.jpg)
Detailed outline
• Problem dfn and motivation
• Solution: Random walk with restarts
• Efficient computation
• Case study: image auto-captioning
• Extensions: bi-partite graphs; tracking
• Conclusions
Lipari 2010 (C) 2010, C. Faloutsos 141
![Page 142: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/142.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 142
Reminder: PageRank
• With probability 1-c, fly-out to a random node
• Then, we havep = c B p + (1-c)/n 1 =>
p = (1-c)/n [I - c B] -1 1
![Page 143: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/143.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 143
Ranking vector Starting vectorAdjacency matrix
(1 )i i ir cWr c e
Restart p
p = c B p + (1-c)/n 1The onlydifference
![Page 144: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/144.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 144
Computing RWR
1
43
2
5 6
7
9 10
811
12
0.13 0 1/3 1/3 1/3 0 0 0 0 0 0 0 0
0.10 1/3 0 1/3 0 0 0 0 1/4 0 0 0
0.13
0.22
0.13
0.050.9
0.05
0.08
0.04
0.03
0.04
0.02
0
1/3 1/3 0 1/3 0 0 0 0 0 0 0 0
1/3 0 1/3 0 1/4 0 0 0 0 0 0 0
0 0 0 1/3 0 1/2 1/2 1/4 0 0 0 0
0 0 0 0 1/4 0 1/2 0 0 0 0 0
0 0 0 0 1/4 1/2 0 0 0 0 0 0
0 1/3 0 0 1/4 0 0 0 1/2 0 1/3 0
0 0 0 0 0 0 0 1/4 0 1/3 0 0
0 0 0 0 0 0 0 0 1/2 0 1/3 1/2
0 0 0 0 0 0 0 1/4 0 1/3 0 1/2
0 0 0 0 0 0 0 0 0 1/3 1/3 0
0.13 0
0.10 0
0.13 0
0.22
0.13 0
0.05 00.1
0.05 0
0.08 0
0.04 0
0.03 0
0.04 0
2 0
1
0.0
n x n n x 1n x 1
Ranking vector Starting vectorAdjacency matrix
1
(1 )i i ir cWr c e
Restart p
p = c B p + (1-c)/n 1
![Page 145: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/145.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 145
0 1/3 1/3 1/3 0 0 0 0 0 0 0 0
1/3 0 1/3 0 0 0 0 1/4 0 0 0 0
1/3 1/3 0 1/3 0 0 0 0 0 0 0 0
1/3 0 1/3 0 1/4
0.9
0 0 0 0 0 0 0
0 0 0 1/3 0 1/2 1/2 1/4 0 0 0 0
0 0 0 0 1/4 0 1/2 0 0 0 0 0
0 0 0 0 1/4 1/2 0 0 0 0 0 0
0 1/3 0 0 1/4 0 0 0 1/2 0 1/3 0
0 0 0 0 0 0 0 1/4 0 1/3 0 0
0 0 0 0 0 0 0 0 1/2 0 1/3 1/2
0 0 0 0 0
0
0
0
0
00.1
0
0
0
0
0 0 1/4 0 1/3 0 1/2 0
0 0 0 0 0 0 0 0 0 1/3 1/3
1
0 0
Q: Given query i, how to solve it?
??
Adjacency matrix Starting vectorRanking vectorRanking vector
Query
![Page 146: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/146.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 146
1
43
2
5 6
7
9 10
8 11
120.130.10
0.13
0.130.05
0.05
0.08
0.04
0.02
0.04
0.03
OntheFly: 0 1/3 1/3 1/3 0 0 0 0 0 0 0 0
1/3 0 1/3 0 0 0 0 1/4 0 0 0 0
1/3 1/3 0 1/3 0 0 0 0 0 0 0 0
1/3 0 1/3 0 1/4
0.9
0 0 0 0 0 0 0
0 0 0 1/3 0 1/2 1/2 1/4 0 0 0 0
0 0 0 0 1/4 0 1/2 0 0 0 0 0
0 0 0 0 1/4 1/2 0 0 0 0 0 0
0 1/3 0 0 1/4 0 0 0 1/2 0 1/3 0
0 0 0 0 0 0 0 1/4 0 1/3 0 0
0 0 0 0 0 0 0 0 1/2 0 1/3 1/2
0 0 0 0 0
0
0
0
0
00.1
0
0
0
0
0 0 1/4 0 1/3 0 1/2 0
0 0 0 0 0 0 0 0 0 1/3 1/3
1
0 0
0
0
0
1
0
0
0
0
0
0
0
0
0.13
0.10
0.13
0.22
0.13
0.05
0.05
0.08
0.04
0.03
0.04
0.02
1
43
2
5 6
7
9 10
811
12
0.3
0
0.3
0.1
0.3
0
0
0
0
0
0
0
0.12
0.18
0.12
0.35
0.03
0.07
0.07
0.07
0
0
0
0
0.19
0.09
0.19
0.18
0.18
0.04
0.04
0.06
0.02
0
0.02
0
0.14
0.13
0.14
0.26
0.10
0.06
0.06
0.08
0.01
0.01
0.01
0
0.16
0.10
0.16
0.21
0.15
0.05
0.05
0.07
0.02
0.01
0.02
0.01
0.13
0.10
0.13
0.22
0.13
0.05
0.05
0.08
0.04
0.03
0.04
0.02
No pre-computation/ light storage
Slow on-line response O(mE)
ir
ir
![Page 147: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/147.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 147
0.20 0.13 0.14 0.13 0.68 0.56 0.56 0.63 0.44 0.35 0.39 0.34
0.28 0.20 0.13 0.96 0.64 0.53 0.53 0.85 0.60 0.48 0.53 0.45
0.14 0.13 0.20 1.29 0.68 0.56 0.56 0.63 0.44 0.35 0.39 0.33
0.13 0.10 0.13 2.06 0.95 0.78 0.78 0.61 0.43 0.34 0.38 0.32
0.09 0.09 0.09 1.27 2.41 1.97 1.97 1.05 0.73 0.58 0.66 0.56
0.03 0.04 0.04 0.52 0.98 2.06 1.37 0.43 0.30 0.24 0.27 0.22
0.03 0.04 0.04 0.52 0.98 1.37 2.06 0.43 0.30 0.24 0.27 0.22
0.08 0.11 0.04 0.82 1.05 0.86 0.86 2.13 1.49 1.19 1.33 1.13
0.03 0.04 0.03 0.28 0.36 0.30 0.30 0.74 1.78 1.00 0.76 0.79
0.04 0.04 0.04 0.34 0.44 0.36 0.36 0.89 1.50 2.45 1.54 1.80
0.04 0.05 0.04 0.38 0.49 0.40 0.40 1.00 1.14 1.54 2.28 1.72
0.02 0.03 0.02 0.21 0.28 0.22 0.22 0.56 0.79 1.20 1.14 2.05
4
PreCompute
1 2 3 4 5 6 7 8 9 10 11 12r r r r r r r r r r r r
1
43
2
5 6
7
9 10
8 11
120.130.10
0.13
0.130.05
0.05
0.08
0.04
0.02
0.04
0.03
13
2
5 6
7
9 10
811
12
R:
c x Q
Q
![Page 148: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/148.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 148
2.20 1.28 1.43 1.29 0.68 0.56 0.56 0.63 0.44 0.35 0.39 0.34
1.28 2.02 1.28 0.96 0.64 0.53 0.53 0.85 0.60 0.48 0.53 0.45
1.43 1.28 2.20 1.29 0.68 0.56 0.56 0.63 0.44 0.35 0.39 0.33
1.29 0.96 1.29 2.06 0.95 0.78 0.78 0.61 0.43 0.34 0.38 0.32
0.91 0.86 0.91 1.27 2.41 1.97 1.97 1.05 0.73 0.58 0.66 0.56
0.37 0.35 0.37 0.52 0.98 2.06 1.37 0.43 0.30 0.24 0.27 0.22
0.37 0.35 0.37 0.52 0.98 1.37 2.06 0.43 0.30 0.24 0.27 0.22
0.84 1.14 0.84 0.82 1.05 0.86 0.86 2.13 1.49 1.19 1.33 1.13
0.29 0.40 0.29 0.28 0.36 0.30 0.30 0.74 1.78 1.00 0.76 0.79
0.35 0.48 0.35 0.34 0.44 0.36 0.36 0.89 1.50 2.45 1.54 1.80
0.39 0.53 0.39 0.38 0.49 0.40 0.40 1.00 1.14 1.54 2.28 1.72
0.22 0.30 0.22 0.21 0.28 0.22 0.22 0.56 0.79 1.20 1.14 2.05
PreCompute:
1
43
2
5 6
7
9 10
8 11
120.130.10
0.13
0.130.05
0.05
0.08
0.04
0.02
0.04
0.03
1
43
2
5 6
7
9 10
811
12
Fast on-line response
Heavy pre-computation/storage costO(n ) O(n )
0.13
0.10
0.13
0.22
0.13
0.05
0.05
0.08
0.04
0.03
0.04
0.02
3 2
![Page 149: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/149.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 149
Q: How to Balance?
On-line Off-line
![Page 150: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/150.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 150
How to balance?
Idea (‘B-Lin’)
• Break into communities
• Pre-compute all, within a community
• Adjust (with S.M.) for ‘bridge edges’
H. Tong, C. Faloutsos, & J.Y. Pan. Fast Random Walk with Restart and Its Applications. ICDM, 613-622, 2006.
![Page 151: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/151.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 151
Detailed outline
• Problem dfn and motivation
• Solution: Random walk with restarts
• Efficient computation
• Case study: image auto-captioning
• Extensions: bi-partite graphs; tracking
• Conclusions
![Page 152: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/152.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 152
gCaP: Automatic Image Caption• Q
…
Sea Sun Sky Wave{ } { }Cat Forest Grass Tiger
{?, ?, ?,}
A: Proximity! [Pan+ KDD2004]
![Page 153: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/153.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 153
Test Image
Sea Sun Sky Wave Cat Forest Tiger Grass
Image
Keyword
Region
![Page 154: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/154.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 154
Test Image
Sea Sun Sky Wave Cat Forest Tiger Grass
Image
Keyword
Region
{Grass, Forest, Cat, Tiger}
![Page 155: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/155.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 155
C-DEM (Screen-shot)
![Page 156: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/156.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 156
C-DEM: Multi-Modal Query System for DrosophilaEmbryo Databases [Fan+ VLDB 2008]
![Page 157: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/157.jpg)
Detailed outline
• Problem dfn and motivation
• Solution: Random walk with restarts
• Efficient computation
• Case study: image auto-captioning
• Extensions: bi-partite graphs; tracking
• Conclusions
Lipari 2010 (C) 2010, C. Faloutsos 157
![Page 158: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/158.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 158
Problem: update
E’ edges changed
Involves n’ authors, m’ confs.
n authors
m Conferences
![Page 159: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/159.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 159
Solution:
• Use Sherman-Morrison Lemma to quickly update the inverse matrix
![Page 160: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/160.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 160
Fast-Single-Update
176x speedup
40x speedup
log(Time) (Seconds)
Datasets
Our method
Our method
![Page 161: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/161.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 161
pTrack: Philip S. Yu’s Top-5 conferences up to each year
ICDE
ICDCS
SIGMETRICS
PDIS
VLDB
CIKM
ICDCS
ICDE
SIGMETRICS
ICMCS
KDD
SIGMOD
ICDM
CIKM
ICDCS
ICDM
KDD
ICDE
SDM
VLDB
1992 1997 2002 2007
DatabasesPerformanceDistributed Sys.
DatabasesData Mining
DBLP: (Au. x Conf.) - 400k aus, - 3.5k confs - 20 yrs
![Page 162: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/162.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 162
pTrack: Philip S. Yu’s Top-5 conferences up to each year
ICDE
ICDCS
SIGMETRICS
PDIS
VLDB
CIKM
ICDCS
ICDE
SIGMETRICS
ICMCS
KDD
SIGMOD
ICDM
CIKM
ICDCS
ICDM
KDD
ICDE
SDM
VLDB
1992 1997 2002 2007
DatabasesPerformanceDistributed Sys.
DatabasesData Mining
DBLP: (Au. x Conf.) - 400k aus, - 3.5k confs - 20 yrs
![Page 163: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/163.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 163
KDD’s Rank wrt. VLDB over years
Prox.Rank
Year
Data Mining and Databases are getting closer & closer
![Page 164: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/164.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 164
cTrack:10 most influential authors in NIPS community up to each year
Author-paper bipartite graph from NIPS 1987-1999. 3k. 1740 papers, 2037 authors, spreading over 13 years
T. Sejnowski
M. Jordan
![Page 165: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/165.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 165
Conclusions - Take-home messages• Proximity Definitions
– RWR– and a lot of variants
• Computation– Sherman–Morrison Lemma– Fast Incremental Computation
• Applications– Recommendations; auto-captioning; tracking
– Center-piece Subgraphs (next)
– E-mail management; anomaly detection, …
![Page 166: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/166.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 166
References• L. Page, S. Brin, R. Motwani, & T. Winograd. (1998), The
PageRank Citation Ranking: Bringing Order to the Web, Technical report, Stanford Library.
• T.H. Haveliwala. (2002) Topic-Sensitive PageRank. In WWW, 517-526, 2002
• J.Y. Pan, H.J. Yang, C. Faloutsos & P. Duygulu. (2004) Automatic multimedia cross-modal correlation discovery. In KDD, 653-658, 2004.
![Page 167: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/167.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 167
References• C. Faloutsos, K. S. McCurley & A. Tomkins. (2002) Fast
discovery of connection subgraphs. In KDD, 118-127, 2004.
• J. Sun, H. Qu, D. Chakrabarti & C. Faloutsos. (2005) Neighborhood Formation and Anomaly Detection in Bipartite Graphs. In ICDM, 418-425, 2005.
• W. Cohen. (2007) Graph Walks and Graphical Models. Draft.
![Page 168: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/168.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 168
References• P. Doyle & J. Snell. (1984) Random walks and electric
networks, volume 22. Mathematical Association America, New York.
• Y. Koren, S. C. North, and C. Volinsky. (2006) Measuring and extracting proximity in networks. In KDD, 245–255, 2006.
• A. Agarwal, S. Chakrabarti & S. Aggarwal. (2006) Learning to rank networked entities. In KDD, 14-23, 2006.
![Page 169: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/169.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 169
References• S. Chakrabarti. (2007) Dynamic personalized pagerank in
entity-relation graphs. In WWW, 571-580, 2007.
• F. Fouss, A. Pirotte, J.-M. Renders, & M. Saerens. (2007) Random-Walk Computation of Similarities between Nodes of a Graph with Application to Collaborative Recommendation. IEEE Trans. Knowl. Data Eng. 19(3), 355-369 2007.
![Page 170: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/170.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 170
References• H. Tong & C. Faloutsos. (2006) Center-piece subgraphs:
problem definition and fast solutions. In KDD, 404-413, 2006.
• H. Tong, C. Faloutsos, & J.Y. Pan. (2006) Fast Random Walk with Restart and Its Applications. In ICDM, 613-622, 2006.
• H. Tong, Y. Koren, & C. Faloutsos. (2007) Fast direction-aware proximity for graph mining. In KDD, 747-756, 2007.
![Page 171: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/171.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 171
References• H. Tong, B. Gallagher, C. Faloutsos, & T. Eliassi-
Rad. (2007) Fast best-effort pattern matching in large attributed graphs. In KDD, 737-746, 2007.
• H. Tong, S. Papadimitriou, P.S. Yu & C. Faloutsos. (2008) Proximity Tracking on Time-Evolving Bipartite Graphs. SDM 2008.
![Page 172: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/172.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 172
References• B. Gallagher, H. Tong, T. Eliassi-Rad, C. Faloutsos.
Using Ghost Edges for Classification in Sparsely Labeled Networks. KDD 2008
• H. Tong, Y. Sakurai, T. Eliassi-Rad, and C. Faloutsos. Fast Mining of Complex Time-Stamped Events CIKM 08
• H. Tong, H. Qu, and H. Jamjoom. Measuring Proximity on Graphs with Side Information. ICDM 2008
![Page 173: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/173.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 173
Resources
• www.cs.cmu.edu/~htong/soft.htmFor software, papers, and ppt of presentations
• www.cs.cmu.edu/~htong/tut/cikm2008/cikm_tutorial.htmlFor the CIKM’08 tutorial on graphs and proximity
Again, thanks to Hanghang TONGfor permission to use his foils in this part
![Page 174: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/174.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 174
Outline• Introduction – Motivation• Task 1: Node importance • Task 2: Recommendations & proximity• Task 3: Connection sub-graphs• Conclusions
![Page 175: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/175.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 175
Detailed outline
• Problem definition
• Solution
• Results
H. Tong & C. Faloutsos Center-piece subgraphs: problem definition and fast solutions. In KDD, 404-413, 2006.
![Page 176: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/176.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 176
Center-Piece Subgraph(Ceps)• Given Q query nodes• Find Center-piece ( )
• Input of Ceps– Q Query nodes– Budget b– k softAnd number
• App.– Social Network– Law Inforcement– Gene Network– …
b
B
A
C
B
A
C
![Page 177: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/177.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 177
Challenges in Ceps
• Q1: How to measure importance?
• (Q2: How to extract connection subgraph?
• Q3: How to do it efficiently?)
![Page 178: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/178.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 178
Challenges in Ceps
• Q1: How to measure importance?
• A: “proximity” – but how to combine scores?• (Q2: How to extract connection subgraph?• Q3: How to do it efficiently?)
![Page 179: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/179.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 179
AND: Combine Scores
• Q: How to combine scores?
![Page 180: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/180.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 180
AND: Combine Scores
• Q: How to combine scores?
• A: Multiply
• …= prob. 3 random particles coincide on node j
![Page 181: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/181.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 181
Detailed outline
• Problem definition
• Solution
• Results
![Page 182: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/182.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 182
Case Study: AND query
R. Agrawal Jiawei Han
V. Vapnik M. Jordan
H.V. Jagadish
Laks V.S. Lakshmanan
Heikki Mannila
Christos Faloutsos
Padhraic Smyth
Corinna Cortes
15 1013
1 1
6
1 1
4 Daryl Pregibon
10
2
11
3
16
![Page 183: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/183.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 183
Case Study: AND query
R. Agrawal Jiawei Han
V. Vapnik M. Jordan
H.V. Jagadish
Laks V.S. Lakshmanan
Heikki Mannila
Christos Faloutsos
Padhraic Smyth
Corinna Cortes
15 1013
1 1
6
1 1
4 Daryl Pregibon
10
2
11
3
16
![Page 184: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/184.jpg)
Lipari 2010 (C) 2010, C. Faloutsos 184
Conclusions
Proximity (e.g., w/ RWR) helps answer ‘AND’ and ‘k_softAnd’ queries
![Page 185: Talk 2: Graph Mining Tools - SVD, ranking, proximity Christos Faloutsos CMU](https://reader031.vdocument.in/reader031/viewer/2022012919/5697bfe11a28abf838cb39f3/html5/thumbnails/185.jpg)
Overall conclusions
• SVD: a powerful tool– HITS/ pageRank– (dimensionality reduction)
• Proximity: Random Walk with Restarts– Recommendation systems– Auto-captioning– Center-Piece Subgraphs
Lipari 2010 (C) 2010, C. Faloutsos 185