advanced topics in spectral graph theory · 2014-07-25 · advanced topics in spectral graph theory...
TRANSCRIPT
![Page 1: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/1.jpg)
Advanced topics in spectral
graph theory
Edwin Hancock
Department of Computer Science
University of York
Supported by a Royal Society
Wolfson Research Merit Award
![Page 2: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/2.jpg)
Links explored in this talk
• What more can we learn about the structure and
complexity of networks from their graph spectra?
• Structure: Relationships between graph spectra
and structure of networks. How random walks
can be used as probes of structure.
• Complexity: How can we use graph spectra to
compute index of network complexity.
![Page 3: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/3.jpg)
Machine learning perspective
• Learn about structure of networks.
• Entropy from node degree statistics.
• Relationships between graph spectra and tree or cycle
frequencies, and entropy and degree statistics.
• Kernels for measuring network similarity and graph
embedding.
• Description length principles for learning distributions
determining structure of graphs – “generative models”.
![Page 4: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/4.jpg)
Structure
![Page 5: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/5.jpg)
Graph spectra and
random walks
Use spectrum of Laplacian matrix
to compute hitting and commute
times for random walk on a graph
![Page 6: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/6.jpg)
Laplacian Matrix
• Weighted adjacency matrix
• Degree matrix
• Laplacian matrix
otherwise
EvuvuwvuW
0
),(),(),(
Vv
vuWuuD ),(),(
WDL
![Page 7: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/7.jpg)
Laplacian spectrum
• Spectral Decomposition of Laplacian
• Element-wise
)()(),( vuvuL kk
k
k
T
k
k
kk
TL
),....,( ||1 Vdiag )|.....|( ||1 V
||21 ....0 V
![Page 8: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/8.jpg)
Properties of the Laplacian
• Eigenvalues are positive and smallest eigenvalue is zero
• Multiplicity of zero eigenvalue is number connected components of graph.
• Zero eigenvalue is associated with all-ones vector.
• Eigenvector associated with the second smallest eigenvector is Fiedler vector.
||21 .....0 V
![Page 9: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/9.jpg)
Continuous time random walk
![Page 10: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/10.jpg)
Heat Kernels
• Solution of heat equation and measures
information flow across edges of graph
with time:
• Solution found by exponentiating
Laplacian eigensystem
tt Lht
h
TT
kk
k
kt tth ]exp[]exp[
TWDL
![Page 11: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/11.jpg)
Heat kernel and random walk
• State vector of continuous time random
walk satisfies the differential equation
• Solution
tt Lp
t
p
00]exp[ phpLtp tt
![Page 12: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/12.jpg)
Example.
Graph shows spanning tree of heat-kernel. Here weights of graph are
elements of heat kernel. As t increases, then spanning tree evolves from
a tree rooted near centre of graph to a string (with ligatures).
Low t behaviour dominated by Laplacian, high t behaviour dominated by
Fiedler-vector.
![Page 13: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/13.jpg)
Moments of the heat-kernel
trace
….can we characterise graph by
the shape of its heat-kernel trace
function?
![Page 14: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/14.jpg)
Heat Kernel Trace
Time (t)->
Trace
]exp[][ thTri
it
Shape of heat-kernel
distinguishes
graphs…can we
characterise its shape
using moments
![Page 15: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/15.jpg)
Heat Kernel Trace
Time (t)->
Trace
]exp[][ thTri
it
Shape of heat-kernel
distinguishes
graphs…can we
characterise its shape
using moments
dtthTrts s )]([)(0
1
Use moments of heat kernel
trace:
![Page 16: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/16.jpg)
Rosenberg Zeta function
• Definition of zeta function
s
k
k
s
)()(0
![Page 17: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/17.jpg)
Heat-kernel moments
• Mellin transform
• Trace and number of connected components
• Zeta function
dttts
i
ss
i ]exp[)(
1
0
1
dttts s ]exp[)(0
1
]exp[][0
tChTri
it
dtChTrts
s t
ss
i
i
][)(
1)(
0
1
0
C is multiplicity of zero
eigenvalue or number of
connected components in
graph.
Zeta-function is related
to moments of heat-
kernel trace.
![Page 18: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/18.jpg)
Zeta-function behavior
![Page 19: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/19.jpg)
Objects
72 views of each object taken in 5 degree intervals as camera moves
in circle around object.
Feature points extracted using corner detector.
Construct Voronoi tesselation image plane using corner points as
seeds.
Delaunay graph is region adjacency graph for Voronoi regions.
![Page 20: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/20.jpg)
Heat kernel moments
(zeta(s), s=1,2,3,4)
![Page 21: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/21.jpg)
PCA using zeta(s), s=1,2,3,4)
![Page 22: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/22.jpg)
Zeta function derivative
• Zeta function in terms of natural exponential
• Derivative
• Derivative at origin
]lnexp[)()(00
kk
k
s
k ss
]lnexp[ln)('0
k
kk ss
0
0
1lnln)0('
k
k k
k
![Page 23: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/23.jpg)
Meaning
• Number of spanning trees in graph
)](exp[)( ' od
d
G
Vu
u
Vu
u
![Page 24: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/24.jpg)
COIL
![Page 25: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/25.jpg)
Deeper probes of structure
Ihara zeta function
![Page 26: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/26.jpg)
Zeta functions
• Used in number theory to characterise
distribution of prime numbers.
• Can be extended to graphs by replacing
notion of prime number with that of a
prime cycle.
![Page 27: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/27.jpg)
Ihara Zeta function
• Determined by distribution of prime cycles.
• Transform graph to oriented line graph (OLG) with edges as nodes and edges indicating incidence at a common vertex.
• Zeta function is reciprocal of characteristic polynomial for OLG adjacency matrix.
• Coefficients of polynomial determined by eigenvalues of OLG adjacency matrix.
• Powers of OLG adjacency matrix give prime cycle frequencies.
![Page 28: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/28.jpg)
Oriented Line Graph
![Page 29: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/29.jpg)
Ihara Zeta Function
• Ihara Zeta Function for a graph G(V,E)
– Defined over prime cycles of graph
– Rational expression in terms of characteristic polynomial of oriented line-graph
A is adjacency matrix of line digraph
Q =D-I (degree matrix minus identity
![Page 30: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/30.jpg)
Characteristic Polynomials from IZF
• Perron-Frobenius operator is the adjacency matrix TH
of the oriented line graph
• Determinant Expression of IZF
– Each coefficient,i.e. Ihara coefficient, can be derived from the
elementary symmetric polynomials of the eigenvalue set
• Pattern Vector in terms of
![Page 31: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/31.jpg)
Analysis of determinant
• From matrix logs
• Tr[T^k] is symmetric polynomial of
eigenvalues of T
]][exp[]det[
1)(
1 k
sTTr
TsIs
k
k
k
N
N
N
N
TTr
TTr
TTr
.............][
.....
...][
........][
21
2
21
2
1
2
1
1
![Page 32: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/32.jpg)
Distribution of prime cycles
• Frequency distribution for cycles of length l
• Cycle frequencies
l
l
lsNsds
ds )(ln
][)(ln)!1(
10
l
sl
l
l TTrsds
d
lN
![Page 33: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/33.jpg)
Experiments: Edge-weighted Graphs
Feature Distance
& Edit Distance
Three Classes of Randomly
Generated Graphs
![Page 34: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/34.jpg)
Experiments: Hypergraphs
![Page 35: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/35.jpg)
Heat Kernel Signature
• The heat kernel signature has been used to
characterise 3D shapes
• Sun et al (2009)
• Sample self-heat of vertices over time
• Closely related to heat kernel trace
• Characterised individual vertices
• Overall signature from histogram of all HKS
]),,(),,(),,([HKS210
xxHxxHxxH ttt
![Page 36: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/36.jpg)
Quantum Walks
![Page 37: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/37.jpg)
Quantum Walks on Graphs
• Have both discrete time (DQW) and continuous time (CQW) variants.
• Use qubits to represent state of walk.
• State-vector is composed of complex numbers rather than an probabilities. Governed by unitary matrices rather than stochastic matrices.
• Admit interference of different feasible walks.
• Reversible, non ergodic, no limiting distribution.
• Sensitive to symmetry structure of graph (leads to faster hitting and commute times).
![Page 38: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/38.jpg)
Quantum and Classical Walks on Graphs
– Contrast continuous-time quantum walk and classical random
walk on a graph
• Continuous-time quantum walk
– A) state space: set of vertices
– B) the state vector is complex-valued
– C) the evolution is governed by a time-varying unitary matrix
• Classical random walk – A) state space: set of vertices
– B) the state vector is real-valued
– C) the evolution is governed by a double stochastic matrix
– Continuous-time quantum walk is reversible and non-ergodic, and does not have a limiting distribution.
![Page 39: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/39.jpg)
Continuous time walk
• Classical
• Quantum
)()( tLptpdt
d
tt iLdt
d ||
![Page 40: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/40.jpg)
Continuous time walk
• Classical (heat eqution)
• Quantum (complex wave equation)
)()( tLptpdt
d
tt iLdt
d ||
0|]exp[| iLt
)0(]exp[)( pLttp
![Page 41: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/41.jpg)
Walks compared
Classical walk Quantum walk
![Page 42: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/42.jpg)
Quantum vs Classical
• Hitting time behavior of quantum walks is completely different from
its classical counterpart (see Fig. quantum vs classical on a line)
5 10 15 20 25 30 35 40 45 500
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0.09
0.1
time
pro
ba
bili
ty
Quantum Walk
Random Walk
![Page 43: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/43.jpg)
Why interesting
• Classical walk characterised by probabillity
state-vector.
• In quantum case characterised by a complex
wave function.
• In quantum case interference and entanglement
lead to interesting phenomenon, not present in
classical case.
![Page 44: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/44.jpg)
Relations to Symmetry
• Symmetrical substructures
result in destructive and
constructive interference
patterns
• Interference leads to
cancellation of backtracking
paths and faster hitting
times
![Page 45: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/45.jpg)
Uses in ML/PR
• Interference effects endow QW algorithms with properties not
exhibited by classical walks - quantum weirdness.
• Lift co-spectralities – better distinguish trees and strongly regular
graphs – (Emms etc al PR 2009).
• Quantum commute time – shows long range symmetry sensitivity
(Emms, QIC 2010).
• Quantum information theory of walk leads to symmetry detection
(Rossi et al Phys Rev E, 2013) and new family of symmetry
sensitive graph kernels.
![Page 46: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/46.jpg)
Embeddings of cojoined
trees: Classical and
quantum commute times. In
case of quantum commute
time, symmetrically placed
nodes embed to same
location is subspace.
![Page 47: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/47.jpg)
Hitting time
Q(u,v): Expected number of steps
of a random walk before node v is
visited commencing from node u.
![Page 48: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/48.jpg)
Commute time
CT(u,v): Expected time taken for a
random walk to travel from node u
to node v and then return again
CT(u,v)=Q(u,v)+Q(v,u)
![Page 49: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/49.jpg)
Idea: compute edge
attribute that is robust to
modifications in edge-
structure.
Commute time: averages over all
paths connecting a pair of nodes.
Effects of edge deletion reduced.
![Page 50: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/50.jpg)
Green’s function
• Spectral representation
• Meaning: psuedo inverse of Laplacian
)()(),(||
2
1 vuvuG ii
V
i
i
![Page 51: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/51.jpg)
Commute Time
• Commute time
– Hitting time and the Green’s function
– Commute time and Laplacian eigen-spectrum
),(),(),( uuGd
volvvG
d
volvuQ
uv
),(),(),(),(),(),(),( uvGd
volvuG
d
volvvG
d
voluuG
d
voluvQvuQvuCT
uuvu
||
2
2)()(
1),(
V
i
ii
i
vuvolvuCT
![Page 52: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/52.jpg)
Commute Time Embedding
• Embedding that preserves commute time has
co-ordinate matrix (vectors of co-ordinates are
columns):
Tvol
![Page 53: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/53.jpg)
Example embedding
![Page 54: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/54.jpg)
Embedding of a wheel with 4 spokes
![Page 55: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/55.jpg)
![Page 56: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/56.jpg)
![Page 57: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/57.jpg)
Complexity
Information theory, graphs and
kernels.
![Page 58: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/58.jpg)
Protein-Protein Interaction Networks
![Page 59: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/59.jpg)
Characterising graphs
• Topological: e.g. average degree, degree distribution, edge-density, diameter, cycle frequencies etc.
• Spectral or algebraic: use eigenvalues of adjacency matrix or Laplacian, or equivalently the co-efficients of characteristic polynomial.
• Complexity: use information theoretic measures of structure (e.g. Shannon entropy).
![Page 60: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/60.jpg)
Complexity characterisation
• Information theory: entropy measures
• Structural pattern recognition: graph
spectral indices of structure and topology.
• Complex systems: measures of centrality,
separation, searchability.
![Page 61: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/61.jpg)
Information theory
• Entropic measures of complexity:
Shannon , Erdos-Renyi, Von-Neumann.
• Description length: fitting of models to
data, entropy (model cost) tensioned
against log-likelihood (goodness of fit).
• Kernels: Use entropy to computeJensen-
Shannon divergence
![Page 62: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/62.jpg)
Entropy
• Thermodynamics: measure of disorder in a system. Change
in entropy with energy measure temperature of system
DE=TDH.
• Statistical mechanics: Entropy is measure of uncertainty of
microstates of a system H=-k Sipi ln pi – Boltzmann.
• Quantum mechanics: Confusion of states H=-kTr[r ln r ] in
terms of density matrix for states – Von Neumann.
• Information theory: Shannon information H=- Sipi ln pi – in
terms of probability of transmission of a message in an
information channel.
![Page 63: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/63.jpg)
Von Neumann entropy
• System with Hamiltonian (energy)
eigenvalues are li i=1,..N.
• VN entropy S=-Sili ln li
• Quadratic approximation -li ln li =li (1- li)
• Passerini and Severini – Hamiltonian is
normalised Laplacian L=D-1/2(D-A) D-1/2
![Page 64: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/64.jpg)
Von-Neumann Entropy
• Derived from normalised Laplacian
spectrum
• Comes from quantum mechanics and is
entropy associated with density matrix.
2
ˆln
2
ˆ||
1
i
V
i
iVNH
TDADDL ˆˆˆ)(ˆ 2/12/1
![Page 65: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/65.jpg)
Approximation
• Quadratic entropy
• In terms of matrix traces
||
1
2||
1
||
1
ˆ4
1ˆ2
1
2
ˆ1
2
ˆ V
i
i
V
i
ii
V
i
iVNH
]ˆ[4
1]ˆ[
2
1 2LTrLTrHVN
![Page 66: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/66.jpg)
Computing Traces
• Normalised Laplacian
• Normalised Laplacian squared
||]ˆ[ VLTr
Evu vudd
VLTr),(
2
4
1||]ˆ[
![Page 67: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/67.jpg)
Simplified entropy
Evu vu
VNdd
VH),( 4
1||
4
1
Collect terms together, von Neumann
entropy reduces to
![Page 68: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/68.jpg)
Homogeneity index (Estrada)
Evuvuvu
Evu
vu
ddddVVG
ddG
),(
2
),(
2/12/1
211
1||2||
1)(
)()(
Based on degree
statistics
![Page 69: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/69.jpg)
Homogeneity meaning
Evu
vuAvuCTG),(
),(2),(~)(
Limit of large degree
Largest when commute time differs from 2
due to large number of alternative
connecting paths.
![Page 70: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/70.jpg)
Extend to directed graphs
• Use directed Laplacian
• Find approximations for strongly and
weakly directed graphs.
![Page 71: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/71.jpg)
Directed Laplacian
Transition matrix left eigenvector components
Noramised Laplacian
![Page 72: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/72.jpg)
![Page 73: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/73.jpg)
Trace calculations
![Page 74: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/74.jpg)
Von Neumann Entropy
• Partition edge set into union of
undirectional edges (one way only) E1 and
bidirectional edges (both ways) E2.
• E=E1U E2
![Page 75: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/75.jpg)
Directed Graphs
Eji Eji
out
j
out
i
out
i
in
j
in
i
dddd
d
VVH
),( ),(22
2
1
)(2
111
Von Neumann entropy comes from in-degree
and out-degree of vertices connected by edges
Development comes from Laplacian of a directed graph (Chung).
Eji Ejiout
j
out
i
in
j
out
i
out
i
in
i
ddddd
d
VVH
),( ),(2
2
11
2
111
![Page 76: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/76.jpg)
Strongly Directed Graphs
Eji
in
j
out
i ddVVH
),(2
1
2
111
Most of edges are unidirectional, few
bidirectional edges (|E1|>>|E2|)
![Page 77: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/77.jpg)
Weakly Directed Graphs
Ejiin
j
out
i
out
j
in
j
out
i
in
i
dd
dddd
VVH
),(2
//
2
111
Most of edges are bidirectional, few
unidirectional edges (|E1|<<|E2|)
Development comes from Laplacian of a directed graph (Chung).
![Page 78: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/78.jpg)
Links to assortivity
• Weakly directed: proportional to in/out degree
ratio of nodes; inversely proportional to product
of out degree of start node and in degree of end
node (degree flow).
• Strongly directed: inversely proportional to
product of out degree of start node and in
degree of end node.
![Page 79: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/79.jpg)
Evolving graphs
![Page 80: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/80.jpg)
Distinguishing different graphs
![Page 81: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/81.jpg)
Financial Market Data
• Look at time series correlation for set of
leading stocks.
• Create undirected or directed links on
basis of time series correlation.
![Page 82: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/82.jpg)
• Directed von
Neumann entropy as
stock market
network evolves.
• Troughs represent
financial crises,
while the deepest
one corresponds to
Black Monday, 1987. Black
Monday
Friday the 13th
mini-crash
Bankruptcy of
Lehman
Brothers
1997 Asian
financial crisis
![Page 83: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/83.jpg)
• Directed von
Neumann entropy
change during Black
Monday, 1987.
• Entropy witnesses a
sharp drop on Black
Monday and
recovers in a few
trading days’ time.
Black
Monday
![Page 84: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/84.jpg)
• Directed von
Neumann entropy
change during 1997
Asian financial
crisis.
• Entropy decreases
significantly and
does not recover
until after a
relatively long time.
1997 Asian
financial crisis
![Page 85: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/85.jpg)
• Directed von
Neumann entropy
change during
Bankruptcy of
Lehman Brothers.
• Entropy fluctuates
significantly and
cannot recover in a
long time.
Bankruptcy of
Lehman
Brothers
![Page 86: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/86.jpg)
• Comparison of
directed and
undirected von
Neumann entropies as
stock market network
evolves.
• Directed entropy is
more sensible to
slight changes in
network structure than
undirected entropy.
![Page 87: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/87.jpg)
• Directed and
undirected von
Neumann entropies
change during Friday
the 13th mini-crash.
• Directed entropy has a
better performance in
catching minor
changes in the
network structure than
its undirected
analogue.
Friday the 13th
mini-crash
![Page 88: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/88.jpg)
• Connected component
number as stock market
network evolves.
• Network becomes
fragmented during
financial crises.
• Von Neumann entropy is
able to catch such
structure changes.
![Page 89: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/89.jpg)
• Node degree assortativity
coefficients as stock
market network evolves.
• Top-left: out-degree & out-
degree; top-right: in-degree
& in-degree; bottom-left:
out-degree & in-degree;
bottom-right: in-degree &
out-degree.
• The smallest value appears
on Black Monday, 1987.
![Page 90: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/90.jpg)
• Directed/undirected von
Neumann entropies and
node degree assortativity
coefficients as stock
market network evolves.
• The indegree-outdegree
assortativity behaves
slightly different from the
other three.
• Both directed and
undirected entropies are
able to detect the changes.
![Page 91: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/91.jpg)
• Histograms of added
edges in degree space
during financial crisis
(Black Monday).
• Most edges have higher
probability to appear
between two nodes with
low degrees.
![Page 92: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/92.jpg)
• Histograms of removed
edges in degree space
during financial crisis
(Black Monday).
• Edges that connect two
nodes with low degrees
have higher probability
to disappear.
![Page 93: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/93.jpg)
• Histograms of
unchanged edges in
degree space during
financial crisis (Black
Monday).
• Edges that connect two
nodes with higher
degrees are more likely
to remain unchanged.
![Page 94: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/94.jpg)
• Histograms of added/removed/unchanged edges on different von Neumann entropy
components during financial crisis (Black Monday).
• Added/removed edges have the highest probability to happen either when the first
component is equal to one or when the other components have small values.
• The probability of an edge remain unchanged becomes highest when the first
component is equal to one and does not depend on other two components.
![Page 95: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/95.jpg)
Graph Kernels
![Page 96: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/96.jpg)
Kernel Functions
• Graphs are structures and do not reside in a Euclidean or
even a metric space.
– Can we embed them in such spaces?
– Based on their similarities or features characterising their internal
structure.
• What about this data: If we do so, can we be sure the
embedding is useful and will
allow us to easily separate
different classes of graph.
Here data is not separable with a
linear boundary.
![Page 97: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/97.jpg)
Kernel Functions
• Need a transformation for the data which render them
separable.
• Try this non-linear transformation
• Only two of the three dimensions (√2xy ignored)
• Now there is a linear subspace which separates the two classes
2
2
2
y
xy
x
y
x
2
2
2
y
xy
x
y
x
x
y
x2
y2
![Page 98: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/98.jpg)
Kernel Functions
• Consider two points a and b in the original feature space from the previous example
• They transform as follows
• What is the dot-product in the new space?
• Let K(ua,ub)=(ua·ub)2 = va·vb
• K(ua,ub) is an example of a kernel function
– Equal to the dot-product in the new space
– A function of the positions in the original space
2
2
2222
)(
2
ba
baba
babbaababa
yyxx
yyyxyxxx
uu
vv
2
2
2
a
aa
a
a
a
a
a
y
yx
x
y
xvu
2
2
2
b
bb
b
b
b
b
b
y
yx
x
y
xvu
![Page 99: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/99.jpg)
Kernel Embedding
• The kernel function tells us everything because we can find the points in the new space just using the values of the kernel function
• Let Xu be the datamatrix for the original points and Xv be the corresponding datamatrix for the new transformed points,with data as columns
• Form the kernel matrix of the new points (note similarity to kernel matrix in MDS)
• An element of K is given by
• So given the original datamatrix and the kernel function, we can find the kernel matrix of the new points
v
T
v XXK
),( jijiij KK uuvv
![Page 100: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/100.jpg)
Kernel Embedding
• The rest follows exactly our analysis in MDS
• Hence given a kernel function and the original points, we
can find the transformed points
• This is called the kernel embedding
T
v
T
T
vv
T
UX
UU
UUK
XX
21
21
21
![Page 101: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/101.jpg)
Properties of a kernel function
Not every function is a kernel function
Must be a function of original positions, and equal to a dot-product
in some transformed space
Gives conditions:
First two come directly from fact K(.,.) is a dot-product
Final one enables K to embed exactly into new space (kernel
embedding)
criterion Mercer
inequalitySchwarz -CauchyKKK
SymmetryKK
s)eigenvalue positiveor (zero definite-semi positive bemust
),(),(),(
),(),(
2
K
yyxxyx
xyyx
![Page 102: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/102.jpg)
Kernelising an algorithm
• Many methods can be kernelised; this means they should
depend on values of the kernel function rather than point
positions
• Need to re-write to use elements of kernel matrix K
Observation:
– Any direction of interest with respect to our data must lie within
the space filled by the data
• There is no data variation in any other direction
• This is called the span of the points
• Implies any interesting vector can be written
• We can try to find a instead of u
– This often enables us to kernalise an algorithm
aXxxxu nnaaa 2211
![Page 103: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/103.jpg)
Graph Kernels
• Random Walk Kernel (Gartner et al 2003)
– Count the number of matching walks between two graphs
– A is the product graph of G1 and G2
– k is the walk length
– The number of walks becomes very large
– The random walk graph kernel suffers from the problem of tottering
– Reduces expressive power and masks structural differences
Vji k
ij
k
k AGGK),( 0
21 ),(
![Page 104: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/104.jpg)
Backtrackless Random Walk
• A random walk of length k is a sequence of vertices
– Such that
• A backtrackless random walk has the additional condition
– A sequence of oriented edges, excluding backtracking step
121 ,,, kuuu Euue iii ),( 1
1 ii ee
![Page 105: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/105.jpg)
Backtrackless Random Walk Kernel
• The backtrackless random walk kernel is
• Defined on the product graph of the OLGs
• By eliminating the reverse edges in the OLG, we eliminate
backtracking
• Complexity is a problem
– Efficient method to compute given in (Aziz, Wilson, Hancock,
IEEE TNNLS 2013)
Vji k
ij
k
k AGGK),( 0
21 ),(
221121
212121
212121
),(),(
))},(),,{((
}),{(
EvvEuu
vvuuGGE
VVvvGGV
![Page 106: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/106.jpg)
Jensen-Shannon Kernel
• Defined in terms of J-S divergence
• Properties: extensive, positive.
• Don’t need to compare walks (Bai Lu,
Hancock, JMIV 2013).
)()()(),(
),(2ln),(
jijiji
jijiJS
GHGHGGHGGJS
GGJSGGK
![Page 107: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/107.jpg)
Computation
• Construct direct product graph for each
graph pair.
• Compute von-Neumann entropy difference
between product graph and two graphs
individually.
• Construct kernel matrix over all pairs.
![Page 108: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/108.jpg)
![Page 109: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/109.jpg)
Financial Data
![Page 110: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/110.jpg)
Financial Data
![Page 111: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/111.jpg)
Financial Data
![Page 112: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/112.jpg)
![Page 113: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/113.jpg)
Financial Data
![Page 114: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/114.jpg)
Financial Data
![Page 115: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/115.jpg)
![Page 116: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/116.jpg)
![Page 117: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/117.jpg)
Learning
![Page 118: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/118.jpg)
Generative Models
• Structural domain: define probability distribution over
prototype structure. Prototype together with parameters
of distribution minimise description length (Torsello and
Hancock, PAMI 2007) .
• Spectral domain: embed nodes of graphs into vector-
space using spectral decomposition. Construct point
distribution model over embedded positions of nodes
(Bai, Wilson and Hancock, CVIU 2009).
![Page 119: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/119.jpg)
Deep learning
• Deep belief networks (Hinton 2006, Bengio 2007).
• Compositional networks (Amit+Geman 1999, Fergus
2010).
• Markov models (Leonardis 200
• Stochastic image grammars (Zhu, Mumford, Yuille)
• Taxonomy/category learning (Todorovic+Ahuja, 2006-
2008).
![Page 120: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/120.jpg)
Aim
• Combine spectral and structural methods.
• Use description length criterion.
• Apply to graphs rather than trees.
![Page 121: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/121.jpg)
Prior work
• IJCV 2007 (Torsello, Robles-Kelly, Hancock) –shape classes from edit distance using pairwise clustering.
• PAMI 06 and Pattern Recognition 05 (Wilson, Luo and Hancock) – graph clustering using spectral features and polynomials.
• PAMI 07 (Torsello and Hancock) – generative model for variations in tree structure using description length.
• CVIU09 (Xiao, Wilson and Hancock) – generative model from heat-kernel embedding of graphs.
![Page 122: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/122.jpg)
Structural learning
Using description length
![Page 123: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/123.jpg)
Description length
• Wallace+Freeman: minimum message
length.
• Rissanen: minimum description length.
Use log-posterior probability to locate model that is
optimal with respect to code-length.
![Page 124: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/124.jpg)
Similarities/differences
• MDL: selection of model is aim; model
parameters are simply a means to this
end. Parameters usually maximum
likelihood. Prior on parameters is flat.
• MML: Recovery of model parameters is
central. Parameter prior may be more
complex.
![Page 125: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/125.jpg)
Coding scheme
• Usually assumed to follow an exponential
distribution.
• Alternatives are universal codes and predictive
codes.
• MML has two part codes (model+parameters). In
MDL the codes may be one or two-part.
![Page 126: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/126.jpg)
Method
• Model is supergraph (i.e. Graph prototypes) formed by graph union.
• Sample data observation model: Bernoulli distribution over nodes and edges.
• Mode: complexity: Von-Neumann entropy of supergraphs.
• Fitting criterion:
MDL-like-make ML estimates of the Bernoulli parameters
MML-like: two-part code for data-model fit + supergraph complexity.
![Page 127: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/127.jpg)
Model overview
• Description length criterion
code-length=negative + model code-length
log-likelihood (entropy)
Data-set: set of graphs G
Model: G parameters of network model, likelihood P p(g|G)
Updates by expectation maximisation:
Network parameters (M-step)
+ correspondence indicators (E-step).
)()|(),( HGLLGL
![Page 128: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/128.jpg)
Generative model
• Train on graphs with set of predetermined
characteristics.
• Sample using Monte-Carlo.
• Reproduces characteristics of training set,
e.g. Spectral gap, node degree
distribution, etc.
![Page 129: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/129.jpg)
Erdos Renyi
![Page 130: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/130.jpg)
Barabasi Albert (scale free)
![Page 131: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/131.jpg)
Dealunay Graphs
![Page 132: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/132.jpg)
Experiments---generate new samples
![Page 133: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/133.jpg)
Some ongoing work
• Using quantum walks
probe graphs in different ways to classical walks,
responding to symmetric or distant structure more
rapidly.
• Use quantum Jensen-Shannon divergence
directly model similarity of walks using density matrices,
and develop new family of kernels for quantum walks.
• Wavepackets
solve wave equation on graph using edge-based
Laplacian. Similulate wavepackets to obtain new
signature.
![Page 134: Advanced topics in spectral graph theory · 2014-07-25 · Advanced topics in spectral graph theory Edwin Hancock Department of Computer Science University of York Supported by a](https://reader033.vdocument.in/reader033/viewer/2022042405/5f1efd24139396036c4c4911/html5/thumbnails/134.jpg)
Conclusions
• Shown how graph spectra can be used as
characterisations of both structure and
complexity.
• Presented MDL framework which uses
complexity characterisation to learn generative
model of graph structure.
• Future: Deeper measures of structure
(symmetry) and detailed dynamics of network
evolution.