graph spectra,information theory and complex neworks · graph data • problems based on graphs...
TRANSCRIPT
Graph spectra,Information
Theory and Complex Neworks
Edwin Hancock
Department of Computer Science
University of York
Supported by a Royal Society
Wolfson Research Merit Award
Aims
• How to characterise graphs in a compact manner.
• Laplacian matrix and graph spectra.
• Spanning trees and prime cycles.
• Relationship with quantum and random walks, and with
information theory.
Structural Variations
Protein-Protein Interaction Networks
Graph data
• Problems based on graphs arise in areas such as
language processing, proteomics/chemoinformatics,
data mining, computer vision and complex systems.
• Relatively little methodology available, and vectorial
methods from statistical machine learning not easily
applied since there is no canonical ordering of the nodes
in a graph.
• Can make considerable progress if we develop
permutation invariant characterisations of variations in
graph structure.
Graph data
• Problems based on graphs arise in areas such as
language processing, proteomics/chemoinformatics,
data mining, computer vision and complex systems.
• Relatively little methodology available, and vectorial
methods from statistical machine learning not easily
applied since there is no canonical ordering of the nodes
in a graph.
• Can make considerable progress if we develop
permutation invariant characterisations of variations in
graph structure.
Graph data
• Problems based on graphs arise in areas such as
language processing, proteomics/chemoinformatics,
data mining, computer vision and complex systems.
• Relatively little methodology available, and vectorial
methods from statistical machine learning not easily
applied since there is no canonical ordering of the nodes
in a graph.
• Can make considerable progress if we develop
permutation invariant characterisations of variations in
graph structure.
Characterising graphs
• Topological: e.g. average degree, degree distribution, edge-density, diameter, cycle frequencies etc.
• Spectral or algebraic: use eigenvalues of adjacency matrix or Laplacian, or equivalently the co-efficients of characteristic polynomial.
• Complexity: use information theoretic measures of structure (e.g. Shannon entropy).
Complex systems
• Spatial and topological indices: node degree
stats; edge density;
• Communicability: communities, measures of
centrality, separation, etc. (Baribasi, Watts and
Strogatz, Estrada).
• Processes on graphs: Markov process, Ising
models, random walks, searchability (Kleinberg).
Aims
• Show how random walks can be used as
probes of graph structure.
• Explain links with spectral graph theory.
• Show links with complexity
characterisations, and information theory.
Our contributions
• IJCV 2007 (Torsello, Robles-Kelly, Hancock) –shape classes from edit distance using pairwise clustering.
• PAMI 06 and Pattern Recognition 05 (Wilson, Luo and Hancock) – graph clustering using spectral features and polynomials.
• PAMI 07 (Torsello and Hancock) – generative model for variations in tree structure using description length.
• CVIU09 (Xiao, Wilson and Hancock) – generative model from heat-kernel embedding of graphs.
• QIC09 (Emms, Wilson and Hancock) quantum version of commute time.
• PR09a,b,c,(Emms, Wilson and Hancock) graph matching using quantum walks, lifting cospectrality of graphs using quantum walks.
• PR109(Xiao, Wilson and Hancock) graph characteristics from heat kernel trace.
• TNN11 (Ren, Wilson and Hancock) Ihara zeta function as graph characterisation,
• Phy Rev E (Escolano, Hancock, Lozano) Heat Diffusion: The theromdynanmc depth approach to network complexity
Random walks
And links to graph spectra.
Problem studied • How can we find efficient means of characterising
graph structure which does not involve exhaustive
search? Enumerate properties of graph structure without
explicit search, e.g. count cycles, path length frequencies,
etc..
• Can we analyse the structure of sets of graphs
without solving the graph-matching problem? Inexact graph matching is computational bottleneck for
most problems involving graphs.
• Answer: let a random walker do the work.
Graph Spectral Methods Use eigenvalues and eigenvectors of adjacency graph (or
Laplacian matrix) - Biggs, Cvetokovic, Fan Chung
Singular value methods for exact graph-matching and
point-set alignment). (Umeyama, Scott and Longuet-Higgins,
Shapiro and Brady).
Use of eigenvectors for image segmentation (Shi and Malik)
and for perceptual grouping (Freeman and Perona, Sarkar
and Boyer).
Graph-spectral methods for indexing shock-trees (Dickinson
and Shakoufandeh)
Random walks on graphs
• Determined by the Laplacian spectrum
(and in continuous time case by heat-
kernel).
• Can be used to interpret, and analyse,
spectral methods since they can be
understood intuitively as path-based.
Graph spectra and
random walks
Use spectrum of Laplacian matrix
to compute hitting and commute
times for random walk on a graph
Laplacian Matrix
• Weighted adjacency matrix
• Degree matrix
• Laplacian matrix
otherwise
EvuvuwvuW
0
),(),(),(
Vv
vuWuuD ),(),(
WDL
Laplacian spectrum
• Spectral Decomposition of Laplacian
• Element-wise
)()(),( vuvuL kk
k
k
T
k
k
kk
TL
),....,( ||1 Vdiag )|.....|( ||1 V
||21 ....0 V
Properties of the Laplacian
• Eigenvalues are positive and smallest eigenvalue is zero
• Multiplicity of zero eigenvalue is number connected components of graph.
• Zero eigenvalue is associated with all-ones vector.
• Eigenvector associated with the second smallest eigenvector is Fiedler vector.
||21 .....0 V
Discrete-time random walk
• State-vector
• Transition matrix
• Steady state
• Determined by leading eigenvector of
adjacency matrix A, or equivalently the second
(Fiedler) eigenvector of the Laplacian.
01 pTTpp t
tt
ADT 1
ss Tpp
Continuous time random walk
Heat Kernels
• Solution of heat equation and measures
information flow across edges of graph
with time:
• Solution found by exponentiating
Laplacian eigensystem
tt Lht
h
TT
kk
k
kt tth ]exp[]exp[
TWDL
Heat kernel and random walk
• State vector of continuous time random
walk satisfies the differential equation
• Solution
tt Lp
t
p
00]exp[ phpLtp tt
Heat kernel and path lengths
• In terms of number of paths of
length k from node u to node v
!),(]exp[),(
|
1 k
tvuPtvuh
k
k
kt
),( vuPk
)()()1(),( vuvuP ii
k
Vi
ik
Example.
Graph shows spanning tree of heat-kernel. Here weights of graph are
elements of heat kernel. As t increases, then spanning tree evolves from
a tree rooted near centre of graph to a string (with ligatures).
Low t behaviour dominated by Laplacian, high t behaviour dominated by
Fiedler-vector.
Moments of the heat-kernel
trace
….can we characterise graph by
the shape of its heat-kernel trace
function?
Heat Kernel Trace
Time (t)->
Trace
]exp[][ thTri
it
Shape of heat-kernel
distinguishes
graphs…can we
characterise its shape
using moments
Heat Kernel Trace
Time (t)->
Trace
]exp[][ thTri
it
Shape of heat-kernel
distinguishes
graphs…can we
characterise its shape
using moments
dtthTrts s )]([)(0
1
Use moments of heat kernel
trace:
Rosenberg Zeta function
• Definition of zeta function
s
k
k
s
)()(0
Heat-kernel moments
• Mellin transform
• Trace and number of connected components
• Zeta function
dttts
i
ss
i ]exp[)(
1
0
1
dttts s ]exp[)(0
1
]exp[][0
tChTri
it
dtChTrts
s t
ss
i
i
][)(
1)(
0
1
0
C is multiplicity of zero
eigenvalue or number of
connected components in
graph.
Zeta-function is related
to moments of heat-
kernel trace.
Zeta-function behavior
Objects
72 views of each object taken in 5 degree intervals as camera moves
in circle around object.
Feature points extracted using corner detector.
Construct Voronoi tesselation image plane using corner points as
seeds.
Delaunay graph is region adjacency graph for Voronoi regions.
Heat kernel moments
(zeta(s), s=1,2,3,4)
PCA using zeta(s), s=1,2,3,4)
PCA on Laplace spectrum
Ox-Caltech database
Line-patterns
• Use Huet+Hancock representation (TPAMI-99).
• Extract straight line segments from Canny edge-map.
• Weight computed using continuity and proximity.
• Captures arrangement Gestalts.
Zeta function derivative
• Zeta function in terms of natural exponential
• Derivative
• Derivative at origin
]lnexp[)()(00
kk
k
s
k ss
]lnexp[ln)('0
k
kk ss
0
0
1lnln)0('
k
k k
k
Meaning
• Number of spanning trees in graph
)](exp[)( ' od
d
G
Vu
u
Vu
u
COIL
Ox-Cal
Eigenvalue polynomials (COIL)
Trace
Determinant
Eigenvalue Polynomials (Ox-
Cal)
Spectral polynomials (COIL)
Spectral Polynomials (Ox-Cal)
COIL: node and edge frequency
Ox-Cal: node+edge frequency
Performance
• Rand index=correct/(correct+wrong).
Zeta func. 0.92
Sym. Poly.
(evals)
0,90
Sym. Poly.
(matrix)
0.88
Laplacian
spectrum
0.78
Deeper probes of structure
Ihara zeta function
Zeta functions
• Used in number theory to characterise
distribution of prime numbers.
• Can be extended to graphs by replacing
notion of prime number with that of a
prime cycle.
Ihara Zeta function
• Determined by distribution of prime cycles.
• Transform graph to oriented line graph (OLG) with edges as nodes and edges indicating incidence at a common vertex.
• Zeta function is reciprocal of characteristic polynomial for OLG adjacency matrix.
• Coefficients of polynomial determined by eigenvalues of OLG adjacency matrix.
• Powers of OLG adjacency matrix give prime cycle frequencies.
Oriented Line Graph
Ihara Zeta Function
• Ihara Zeta Function for a graph G(V,E)
– Defined over prime cycles of graph
– Rational expression in terms of characteristic polynomial of oriented line-graph
A is adjacency matrix of line digraph
Q =D-I (degree matrix minus identity
Characteristic Polynomials from IZF
• Perron-Frobenius operator is the adjacency matrix TH
of the oriented line graph
• Determinant Expression of IZF
– Each coefficient,i.e. Ihara coefficient, can be derived from the
elementary symmetric polynomials of the eigenvalue set
• Pattern Vector in terms of
Analysis of determinant
• From matrix logs
• Tr[T^k] is symmetric polynomial of
eigenvalues of T
]][exp[]det[
1)(
1 k
sTTr
TsIs
k
k
k
N
N
N
N
TTr
TTr
TTr
.............][
.....
...][
........][
21
2
21
2
1
2
1
1
Distribution of prime cycles
• Frequency distribution for cycles of length l
• Cycle frequencies
l
l
lsNsds
ds )(ln
][)(ln)!1(
10
l
sl
l
l TTrsds
d
lN
Experiments: Edge-weighted Graphs
Feature Distance
& Edit Distance
Three Classes of Randomly
Generated Graphs
Experiments: Hypergraphs
Complexity
Information theory, graphs and
kernels.
Protein-Protein Interaction Networks
Characterising graphs
• Topological: e.g. average degree, degree distribution, edge-density, diameter, cycle frequencies etc.
• Spectral or algebraic: use eigenvalues of adjacency matrix or Laplacian, or equivalently the co-efficients of characteristic polynomial.
• Complexity: use information theoretic measures of structure (e.g. Shannon entropy).
Complexity characterisation
• Information theory: entropy measures
• Structural pattern recognition: graph
spectral indices of structure and topology.
• Complex systems: measures of centrality,
separation, searchability.
Information theory
• Entropic measures of complexity:
Shannon , Erdos-Renyi, Von-Neumann.
• Description length: fitting of models to
data, entropy (model cost) tensioned
against log-likelihood (goodness of fit).
• Kernels: Use entropy to computeJensen-
Shannon divergence
Von-Neumann Entropy
• Derived from normalised Laplacian
spectrum
• Comes from quantum mechanics and is
entropy associated with density matrix.
2
ˆln
2
ˆ||
1
i
V
i
iVNH
TDADDL ˆˆˆ)(ˆ 2/12/1
Approximation
• Quadratic entropy
• In terms of matrix traces
||
1
2||
1
||
1
ˆ4
1ˆ2
1
2
ˆ1
2
ˆ V
i
i
V
i
ii
V
i
iVNH
]ˆ[4
1]ˆ[
2
1 2LTrLTrHVN
Computing Traces
• Normalised Laplacian
• Normalised Laplacian squared
||]ˆ[ VLTr
Evu vudd
VLTr),(
2
4
1||]ˆ[
Simplified entropy
Evu vu
VNdd
VH),( 4
1||
4
1
Collect terms together, von Neumann
entropy reduces to
Homogeneity index
Evuvuvu
Evu
vu
ddddVVG
ddG
),(
2
),(
2/12/1
211
1||2||
1)(
)()(
Based on degree
statistics
Homogeneity meaning
Evu
vuAvuCTG),(
),(2),(~)(
Limit of large degree
Largest when commute time differs from 2
due to large number of alternative
connecting paths.
Uses
• Complexity-based clustering (especially
protein-protein interaction networks).
• Defining information theoretic (Jensen-
Shannon) kernels.
• Controlling complexity of generative
models of graphs.
Protein-Protein Interaction Networks
Experiment
Quantum walks
….have richer structure due to interference and complex representation.
What can a quantum walker discover about a graph that a classical walker can not?
Quantum Walks
• Have both discrete time (DQW) and continuous time (CQW) variants.
• Use qubits to represent state of walk.
• State-vector is composed of complex numbers rather than an probabilities. Governed by unitary matrices rather than stochastic matrices.
• Admit interference of different feasible walks.
• Reversible, non ergodic, no limiting distribution.
Quantum Walks
• Governed by a wave-function (complex-number) rather than a probability vector.
• Collapse wave function to give observable probability state-vector
• Different quantum walks on a graph can interfere, and this gives them interesting additional structure.
*
tttp
New possibilities
Interference in quantum walks used to develop alternatives to classical random walks algorithm.
Consequences of interference
• Quantum walks hit exponentially faster than classical
ones.
• Local symmetry reflected in cancellation of complex
amplitude.
• Can be exploited to lift co-spectrality of strongly regular
graphs.
• Quantum version of commute time embedding can be
used to detect symmetries.
Literature
• Fahri and Gutmann: CQW can penetrate family of trees in polynomial time, whereas classical walk can not.
• Childs: CQW hits exponentially faster than classical walk.
• Kempe: exploits polynomial hitting time to solve 2-SAT problem and suggest solution to routing problem.
• Shenvi, Kempe and Whaley: search of unordered database.
Discrete Quantum walk
Discrete-time Quantum Walk
• State of walk described by a complex wave function which is updated according to
• Update operator U=TC is a unitary rather than stochastic matrix.
• The coin operator C gives the amplitudes for each path.
• The transition operator T determines transitions between nodes.
ttt UTC 1
The unitary operator governing the evolution of the walk can be written in matrix form as
where the basis states are the set of all ordered pairs (i,j) such that
• Based on same graph transformation as Ihara zeta function.
• However, the eigenvalues of U are nnn where the i are the n eigenvalues of the transition matrix for the lassical random walk (the adjacency matrix normalised) and
• Thus the spectrum of U is of no use in distinguishing strongly regular graphs.
Unitary matrix for DQW
Quantum Walk
Ordered pairs of
vertices
Unitary
Classical Walk
Vertices
Stochastic
•States:
•Transition
Matrix
•Probability
of state:
•Evolution
Discrete Walks Compared
Example: Walk on the line
• Probability distribution for the quantum walk on the line (T=200 steps, only even values plotted).
• Very intricate
• Highly peaked around position
• Almost uniform in centre.
Walks compared
Classical walk Quantum walk
Matrix interpretation
• For a classical walk entries of r-th power of
A is number of paths of length r between a
pair of nodes.
• Motivates the study of r-th power of U
which is total amplitude associated with all
quantum walks of length r between a pair
of nodes.
Strongly Regular Graphs
• Two SRGs with parameters (16,6,2,2)
A B
Is the sp+(U3) able to distinguish all
SRGs?
• There is no method
proven to be able to
decide whether two
SRGs are isomorphic in
polynomial time.
• There are large families
of strongly regular
graphs that we can test
the method on
MDS embeddings of the SRGs with parameters
(25,12,5,6)-red, (26,10,3,4)-blue, (29,14,6,7)-black,
(40,12,2,4)-green using the adjacency spectrum
(top) and the spectrum of S+(U^3) (bottom).
Testing
• The algorithm takes time
O(|E|^3).
• We have tested it for all
pairs of graphs from the
table opposite and found
that it successfully
determines whether the
graphs are isomorphic in
all cases.
Conclusions
i) Spectral graph theory and links to random walk.
ii) Characterisations based on graph spectra and random
walks.
iii) Zeta functions provide compact representation, linked to
spanning tree and prime cycle structure of graphs.
iv) Ihara zeta function constructed from oriented line graph
and is linked to discrete quantum walk on a graph.