Graph Matching with Anchor Nodes: A Learning Approach

Nan Hu Raif M. Rustamov Leonidas GuibasStanford UniversityStanford, CA, USA

nanhu@stanford.edu, rustamov@stanford.edu, guibas@cs.stanford.edu

Abstract

In this paper, we consider the weighted graph matchingproblem with partially disclosed correspondences betweena number of anchor nodes. Our construction exploits re-cently introduced node signatures based on graph Lapla-cians, namely the Laplacian family signature (LFS) on thenodes, and the pairwise heat kernel map on the edges. Inthis paper, without assuming an explicit form of parametricdependence nor a distance metric between node signatures,we formulate an optimization problem which incorporatesthe knowledge of anchor nodes. Solving this problem givesus an optimized proximity measure specific to the graphsunder consideration. Using this as a first order compat-ibility term, we then set up an integer quadratic program(IQP) to solve for a near optimal graph matching. Our ex-periments demonstrate the superior performance of our ap-proach on randomly generated graphs and on two widely-used image sequences, when compared with other exist-ing signature and adjacency matrix based graph matchingmethods.

1. IntroductionThe exact and approximate graph matching problem is

of great interest in computer vision due to its numerous ap-plications in areas such as 2D and 3D image registration,object recognition and biomedical identification, and objecttracking in video sequences. An important variant of theproblem is the semi-supervised setting where a small pro-portion of correct node correspondences between the graphsare known. Such correspondences can be based on addi-tional information provided for only a few nodes, humanjudgement or prior knowledge, etc. Algorithms that cantake advantage of this information to infer the correspon-dences for the rest of the graph nodes are highly desirable.

While there has been a recent effort in applying machinelearning concepts to the graph matching problem in com-puter vision [3, 15], these works are based on the assump-tion that a training set consisting of pairs of graphs with

fully correct correspondences given, and that the trainingset is representative enough of testing graphs, so that learn-ing done with training graphs can be usefully transferredto testing graphs. This problem setup, however, is differ-ent from our setting where we only have two graphs withpartially known correspondences; in a sense, these knowncorrespondences constitute our “training data”. In addition,this amounts to a much smaller and more restricted amountof training data, making the problem challenging.

In this work we provide a method for effectively incor-porating known correspondences into the commonly usedinteger quadratic program formulation of graph match-ing. Specifically, our main contribution is to devise a newfirst order compatibility term between two nodes of differ-ent graphs. Our method uses the recently proposed one-parameter family of node signatures called Laplacian Fam-ily Signatures (LFS) [11], which provide a feature vector(signature) for each node based solely on the node’s struc-tural position within the graph. In contrast to [11], we donot assume an explicit form of parametric dependence forgenerating these signatures, but leave it in an unspecifiedgeneric form. Since graph matching is performed usingthe dissimilarity/distance between these signatures, we de-rive the distance between the generic signatures. As a re-sult of this manipulation, we find that the entire processof computing and comparing these generic signatures canbe encoded into a single proximity matrix. We then intro-duce an algorithm to learn this proximity matrix from theknowledge of provided correct correspondences. This isdone by requiring anchor nodes in one graph to correspondnear to their known partners in the other and to be far fromnon-correspondences, which can be set up as a max-marginproblem [28].

This method was chosen due to a number of benefits.First, our max-margin formulation makes an effective use ofthe scarce training data: even a small number of known cor-respondences (two anchor correspondences are used in allof our experiments) leads to a large number of constraintson the proximity matrix. Second, our formulation resultsin learning a proximity matrix that is relatively small (tens

1

by tens in the examples shown) which allows us to reliablylearn it without over-fitting. Third, our max-margin prob-lem, can be solved using column generation [16], whichresults in an efficient algorithm that scales well with the in-creasing size of the graphs and number of constraints.

Notation and Paper Organization Let G = (V,E) andG0

= (V 0, E0) be two undirected weighted graphs. Our

goal is to find an approximate matching between these twographs based solely on their structural properties (e.g. no ex-ternally provided attributes are available for nodes). We as-sume that a partial correspondence between graphs is given.Namely, let U ⇢ V and U 0 ⇢ V 0 be the subsets of nodesthat are known to be in correspondence; we will refer tothese as anchor node sets for G and G0 respectively.

We follow the commonly used integer quadratic program(IQP) formulation for the graph matching problem based ontwo kinds of compatibility terms. The first order compati-bility d(i, a) encodes similarity of a node i 2 V in graph Gto a node a 2 V 0 in graph G0. For pairs of nodes i, j 2 Vand a, b 2 V 0, the second order compatibility d(i, j, a, b)measures the compatibility of matching the node pair (i, j)to node pair (a, b).

Our main goal in this paper is to incorporate the knowl-edge of the anchor correspondences into the first order com-patibility term which will be expressed as follows:

d(i, a) = cBdB(i, a) + capdap(i, a),

where cB and cap are some weights. The first term dB(i, a)is based on our formulation of LFS proximity matrix andis presented in Section 3.1. The construction of the secondterm dap(i, a) which involves the heat kernel is explainedin Section 3.2. The matching scheme incorporating boththe first order and second order compatibility functions ispresented in Section 4. In Section 5, we present experimentsusing our algorithm on three common datasets.

2. Related WorkOur family of signatures are closely related to node-

based signatures on graphs, different forms of which hasalready been considered, e.g. [7, 23, 9, 27, 12]. Recently,Sun et. al [24] proposed the heat kernel signature (HKS)for application of shape matching in geometry processing.Their signature is based on the simulated heat diffusion pro-cess on manifold. Aubry et. al. [1] later proposed a sig-nature of similar structure based on quantum processes ongraph. Both have been shown in [11] to be special instancesof LFS.

Different forms of spectral matrices have already beenconsidered in matching. Among the pioneering work isUmeyama’s [25] weighted graph matching algorithm from adecomposition of adjacency matrices. His method was later

generalized to graphs of different sizes [17, 31]. Robles-Kelly et. al. [20] used the steady state of the Markov tran-sition matrix to order the nodes and match using edit dis-tances.Later in [21], the same authors proposed to use theleading eigenvector of the adjacency matrix to serialize thegraph nodes for matching. Qiu et. al [19] considered us-ing the Fiedler vector together with the proximity to theperimeter of the graph to partition the graph into discon-nected components for a hierarchical matching. Cho et. al.[4] constructed a reweighed random walk similar to person-alized PageRank on the association graph with the additionof an absorbing node, and used the quasi-stationary distri-bution to find a matching. Emms et. al. [6] simulated aquantum walk on the auxiliary graph and used the particleprobability of each auxiliary node as the cost of assignmentfor a bipartite matching.

In a broader sense, other relaxation-based matching al-gorithms are also related to our work. Gold and Rangarajan[8] proposed the well-known Graduated Assignment Algo-rithm. van Wyk et. al. [26] designed a projection ontoconvex set (POCS) based algorithm to solve IQP. Schelle-wald et. al. [22] constructed a semi-definite programmingrelaxation of the IQP. Leordeanu et. al. [13] proposed aspectral method to solve a relaxed IQP by only consideringlinear inequality constraints at discretization. The idea wasfurther extended by Cour et. al. [5], where they added anaffine constraint during relaxation. Zaslavskiy et. al. [29]approached the IQP from the point of a relaxation of theoriginal least-square problem to a convex and concave opti-mization problem on the set of doubly stochastic matrices.Leordeanu et. al. [14] proposed an integer projected fixedpoint (IPFP) algorithm to iteratively search for a fixed pointsolution and then discretize it into the matching domain.

3. Anchor Based CompatibilityIn this section we discuss the construction and compu-

tation of two kinds of first order compatibility terms thattake advantage of known correspondences between anchornodes.

3.1. Generic Node Signatures and Their Compari-son

We start by reviewing the concept of Laplacian FamilySignatures introduced in [11]. Consider one of the graphsto be matched, say G = (V,E). Let w be the weights onedges, i.e. w : E 7! R+. The graph Laplacian is defined asL = D �A, where A is the graph adjacency matrix with

Aij =

(w(i, j) if (i, j) 2 E

0 otherwise

and D is a diagonal matrix of total incident weights, i.e.Dii =

Pj Aij . L has numerous nice properties [2], of

which most relevant to us is the symmetry and positivesemi-definiteness. This makes it possible to consider theeigen-decomposition of L; we denote by {�k,�k}|V |

k=1

theeigenpairs of graph Laplacian matrix L (eigenvalue and as-sociated eigenvector). We use the same notation with theprime symbol added for the corresponding constructs of oursecond graph G0.

The Laplacian Family Signatures (LFS) for a node u 2V is a one-parameter family of structural node descriptorsthat is defined by

su(t) =X

k

h(t;�k)�k(u)2 (1)

where h(t;�k) is a real valued function. Special h(t;�k)of different forms will result in the heat kernel signature(HKS) [24] when h(t;�k) = exp(�t�k), the wave kernelsignature (WKS) [1] when h(t;�k) = exp(� (t�log �k)

2

2�2 ),or the wavelet signature if h(t;�k) admits some special be-havior as described in [10].

These signatures describe a given node’s structural rela-tionship to its neighborhood. For example, HKS has an in-terpretation in terms of a simulated heat diffusion process:for each node, this signature captures the amount of heatleft at the node at various times (here t) assuming that a unitamount is put on the node initially (t = 0). These signaturesare naturally intrinsic, namely if two graphs are isomorphic,then the signatures of corresponding nodes are the same; thesignatures are also stable under small perturbations [11].

The above discussion suggests using these signatures asnode attributes to design first order compatibility terms —for if the signatures of two nodes from the two graphs arevery different, then these vertices are less likely to be incorrespondence. However, such an approach does not takeinto account the given anchor correspondences, because theform of the function h(t;�k) is explicitly provided before-hand.

To overcome this difficulty, in this paper, in contrast to[11], we will not assume an explicit form for the functionh(t;�k), nor will we assume a specific form of dissimilaritymeasure when comparing the LFS of two nodes. Instead,we assume that h(t;�k) is a generic linear combination ofsome real-valued functions {bi(t)}Nb

i=1

, given as

h(t;�k) =

NbX

i=1

akibi(t), (2)

where {aki} are some real coefficients. We assume a similarexpression for the second graph G0 with possibly a differ-ent set of coefficients {a0ki}. Let h·, ·i be an arbitrary innerproduct of real-valued functions. Assuming that LFS com-parison employs this dot product, the dissimilarity betweentwo nodes u 2 V, v 2 V 0 can be expressed as

d2 (su(t), s0v(t)) = hsu(t)� s0v(t), su(t)� s0v(t)i

Substituting (2) to (1), we have

su(t) =

KX

k=1

�2k(u)

NbX

i=1

akibi(t),

s0v(t) =K0X

k=1

�02k(v)NbX

i=1

a0kibi(t).

Denote A = [aki] 2 RK⇥Nb , ✓u =

2

64�21

(u)...�2K(u)

3

75 2 RK ,

A0= [a0ki] 2 RK0⇥Nb , ✓0v =

2

64�02

1

(v)...�02K0(v)

3

75 2 RK0. Let

b(t) =

2

64b1

(t)...bNb(t)

3

75 2 RNb and Cij = hbi(t), bj(t)i. Now

after denoting C = [Cij ] 2 RNb⇥Nb , we obtain

d2 (su(t), s0v(t))

= h✓>u Ab(t)� ✓0>v A0b(t), ✓>u Ab(t)� ✓0>v A

0b(t)i

=

✓u✓0v

�> A�A0

�C

A�A0

�>

| {z }B2R(K+K0)⇥(K+K0)

✓u✓0v

�

| {z }wuv2R(K+K0)

= w>uvBwuv

This formulation holds for any inner-product based dis-similarity metric. The number of basis functions, althoughassumed finite above can be easily extended to infinite, i.e.the formulation is still valid as Nb ! 1 and the basis perse is also arbitrary. The only restriction, as a result of thepositive semi-definiteness of C, is B ⌫ 0.

The above discussion gives the general expression thatwe will use as a part of our first-order compatibility mea-sure. Namely, we set dB(i, a) =

pw>

iaBwia for any twonodes i 2 V, a 2 V 0. This representation is especially use-ful since it avoids determining the intermediate matrices C,A and A0 explicitly, but allows us to learn directly the prox-imity matrix B which is a small matrix (tens by tens in ourexperiments).

3.1.1 Learning the Proximity Matrix

Here we explain how to learn the proximity matrix B fromthe knowledge of anchor nodes. A good proximity ma-trix B should move closer node pairs that correspond, andmove away nodes that are non-matches. If we let U ⇢ Vbe the set of anchor nodes in graph G, and U 0 ⇢ V 0 betheir known correspondences in graph G0, then we want

d2B(i, a) = w>iaBwia to be small if i and a are in corre-

spondence, and to be large otherwise. One way of achievingthis is to formulate the problem as a max-margin problemsimilar to SVM.

max �s.t. w>

ibBwib � w>iaBwia � �, 8i, 8b 6= a

w>jaBwja � w>

iaBwia � �, 8a, 8j 6= iB ⌫ 0

kBkF 1 .

This can be easily verified to be equivalent to

min

1

2

kBk2Fs.t. tr((wibw

>ib � wiaw

>ia)B) � 1, 8i, 8b 6= a

tr((wjaw>ja � wiaw

>ia)B) � 1, 8a, 8j 6= i

B ⌫ 0 .

For large graphs, however, the problem could be verypossibly infeasible. Therefore, we allow some violation inthe training set and introduce slack variables.

min

1

2

kBk2F + C 1

n

Pni=1

⇠is.t. tr((wikw

>ik � wijw

>ij)B) � 1� ⇠i, 8i, 8k 6= j

tr((wljw>lj � wijw

>ij)B) � 1� ⇠i, 8j, 8l 6= i

B ⌫ 0

⇠i � 0 ,

where n = |U | is the number of anchor nodes.One drawback of this formulation is that we put uniform

weights on slack variables. However, intuitively for a vi-olation of the margin constraint, we would rather to havethe violated nodes to be near to the correct matches withinthe graph, namely we want to put a non-uniform scale onthe slack variables to penalize more severely for nodes thatare farther from the correct matches. Therefore, we intro-duce a loss-function ⌦(k, j) to re-scale the slack variables.In our graph matching setting, ⌦(k, j) could be the shortestdistance over the graph, or the heat kernel as described inSection 3.2 (we used heat kernel in our experiments as ithas been shown to be more robust than the adjacency ma-trix [11] and, hence, shortest distance). Now the problembecomes

min

1

2

kBk2F + C 1

n

Pni=1

⇠is.t. tr((wibw

>ib � wiaw

>ia)B) � 1� ⇠i

⌦

0(b,a) , 8i, 8b 6= a

tr((wjaw>ja � wiaw

>ia)B) � 1� ⇠i

⌦(j,i) , 8a, 8j 6= i

B ⌫ 0

⇠i � 0 .

Let (·)vec be the vector form of a matrix, and b = Bvecand ik = (wikw

>ik�wijw

>ij)vec. The above problem could

be solved by first relaxing the semi-definite constraint andthen projecting the solution to the semi-definite cone. Therelaxed problem is a quadratic programming problem

min

1

2

kbk2 + C 1

n

Pni=1

⇠is.t. >

ibb � 1� ⇠i⌦

0(b,a) , 8i, 8b 6= a

>jab � 1� ⇠i

⌦(j,i) , 8a, 8j 6= i

⇠i � 0 .

The dual of it is

max � 1

2

Pib

Pja ↵ib↵ja

>ib ja +

P↵ib

s.t. ↵ib � 0

Pi

⇣↵ib

⌦

0(b,a) +

↵ja

⌦(j,i)

⌘ C

n .

As the number of constraints in this problem is ofO(|U |(|V | + |V 0|)), it becomes impossible to solve whenthe size of the graphs is very large. One technique that couldbe used to lower the computational cost is column genera-tion [16]. The key idea of this iterative algorithm is thatalthough the number of constraints is large, only a smallportion of them will be nonzero at the solution. Therefore,only this small subset of constraints are necessary to the so-lution. To find this subset, the algorithm iteratively adds oneconstraint per training sample that violated the constraintthe most until all constraints are satisfied. In addition, aftereach iteration, we need to project B back to semi-definitecone to restrain B ⌫ 0. The pseudocode of the algorithm isomitted here for the sake of saving space.

3.2. Heat Kernel with Anchor NodesIn this subsection we introduce the second term appear-

ing in our first order compatibility measure. This term isbased on the heat diffusion process on graph G. Specifi-cally, consider the graph heat kernel kt(u, v), which mea-sures the amount of heat transferred from node u to node vafter time t, assuming a unit amount was placed at u in thebeginning (t = 0). The heat kernel has the following repre-sentation in terms of the eigen-decomposition of the graphLaplacian:

kt(u, v) =X

k

exp(�t�k)�k(u)�k(v).

We use the heat kernel value of anchor nodes at a given nodeas another first order constraint. Namely, for any node v ofgraph G define the heat kernel distance to anchor nodes as

dHap(v) =X

u2U

kt(u, v),

where U is the set of anchor nodes of G. The same con-struction using the anchor nodes U 0 of our second graph G0

provides the quantity d0Hap (·) for each node of G0.

For two nodes i and a in graphs G and G0, we define thesecond portion of our first order compatibility measure by

dap(i, a) =���dHap(i)� d

0Hap (a)

��� .

This quantity is a plausible measure of dissimilarity be-tween nodes because the anchor nodes U and U 0 are knownto be in correspondence. Moreover, the heat kernel is nat-urally intrinsic (if two graphs are isomorphic, their corre-sponding heat kernels are the same) and it is stable undersmall perturbations [11].

4. Matching SchemeHere we formulate the graph matching as an integer

quadratic program (IQP). Let G = (V,E) and G0=

(V 0, E0) be the two graphs, U and U 0 be the anchor node

set for G and G0 respectively. For any two nodes i 2 V \Uand a 2 V 0\U 0, let d2B(i, a) = w>

iaBwia be the learnedproximity, and let dap(i, a) =

���dHap(i)� d0Hap (a)

���. Our firstorder compatibility term is

d(i, a) = cBdB(i, a) + capdap(i, a).

Now we need a second order compatibility term, forwhich we will use the heat kernel as done in [11]. For anytwo nodes i, j 2 V and a, b 2 V 0, the pairwise heat kerneldistance is defined as

dk(i, j, a, b) = |kt(i, j)� k0t(a, b)| ,

and this gives a measure of how compatible matching nodesi and a is with matching j and b. As has been discussed in[11], the heat kernel can be thought of as a noise tolerantapproximation of the adjacency matrix, and is stable undersmall perturbations. Thus, our second order proximity termcan be thought as a generalization of the commonly usedadjacency-based second order term.

Combining all this information, weconstruct a compatibility matrix W 2R(|V |�|U |)(|V 0|�|U 0|)⇥(|V |�|U |)(|V 0|�|U 0|),

Wia,jb =

(dk(i, j, a, b), i 6= j, a 6= b

d(i, a), i = j, a = b

Let X 2 {0, 1}(|V |�|U |)(|V 0|�|U 0|) be the one-to-one map-ping matrix, and x 2 {0, 1}(|V |�|U |)(|V 0|�|U 0|) be the vectorform of X. The IQP can be written as

x

⇤= argmax(x

>Wx)

s.t. x 2 {0, 1}(|V |�|U |)(|V 0|�|U 0|)

8iX

a2V 0\U 0

xia 1, 8aX

i2V \Uxia 1 .

As is well-known this problem is NP-complete and thereis rich literature on approximation algorithms for this prob-lem. Comparison of the performance of different IQP ap-proximation solvers is outside the scope of our paper. Inour experiments, we selected a recently proposed algorithm,reweighed random walk matching (RRWM) [4]. The mainreason we chose this algorithm is its superior performancewhen compared with other state-of-the-art approximationsolvers, including SM [13], SMAC [5], HGM [30], IPFP[14], GAGM [8], SPGM [26]. In consideration of space,we omit the introduction of their algorithm here and leavethe details to the original paper [4].

5. ExperimentsWe tested our approach on three different datasets: 1)

synthetically generated random graphs; 2) CMU Hotel se-quence for large baseline matching; and 3) pose house se-quence from [18] for large rotation angle matching.

5.1. Synthetic Random GraphsIn this section, following the experimental protocol of

[4], we synthetically generated random graphs and per-formed a comparative study. For a pair of graph G

1

andG

2

, they share nin common nodes and n(1)

out and n(2)

out outliernodes. Edges are constructed with a density ⇢ and weightsare randomly distributed in [0, 1]. Perturbation is done withadded random Gaussian noise N (0,�2

).In this experiment, we test the matching performance

for W constructed using i) only the adjacency matrix [4],ii) only pairwise heat kernel distance dk(i, j, a, b), iii)dk(i, j, a, b) with WKS [11], iv) dk(i, j, a, b) with dap(i, a)(cB = 0,cap = 1), v) dk(i, j, a, b) with dB(i, a) (cB =

1,cap = 0), and vi) dk(i, j, a, b) with dap(i, a) and dB(i, a)(cB = 8,cap = 3), on three different settings: 1) differentlevels of deformation noise �; 2) different numbers of out-liers; 3) different edge densities ⇢. Fig. 1 shows the averagematching accuracy. In the experiment, the number of an-chor nodes |U | = 2. The Red solid curve for RRWM isthe baseline approach using the adjacency matrices. FromFig. 1 (a), it can be seen that with the help of learned prox-imity matrix B and the term dap(i, a), the matching resultsare more robust to noise. In Fig. 1 (c), the matching ac-curacy is much improved at different edge densities for arelatively large deformation noise (� = 0.5). Not only thematching accuracy is improved, their corresponding com-putational time is also decreased as shown in Fig. 1 (f).

5.2. CMU Hotel SequenceIn this experiment, we test our descriptors on the CMU

Hotel sequence, which is widely used in performance eval-uation of graph matching algorithms as a wide baselinedataset. It consists of 101 frames, and there are 30 fea-ture points labeled consistently across all frames. We build

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.40.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Deformation noise σ

Acc

urac

y

# of inliers nin = 20

# of outliers nout = 0

Edge density ρ = 1

RRWMdk RRWM

WKS+dk RRWM

dap+dk RRWM

dB+dk RRWM

dB+dap+dk RRWM

(a)

0 1 2 3 4 5 6 7 8 9 100.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

# of outliers nout

Acc

urac

y

# of inliers nin = 20

Deformation noise σ = 0.25

Edge density ρ = 1

RRWMdk RRWM

WKS+dk RRWM

dap+dk RRWM

dB+dk RRWM

dB+dap+dk RRWM

(b)

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0.1

0.2

0.3

0.4

0.5

0.6

0.7

Edge density ρ

Acc

urac

y

# of inliers nin = 20

# of outliers nout = 0

Deformation noise σ = 0.5

RRWMdk RRWM

WKS+dk RRWM

dap+dk RRWM

dB+dk RRWM

dB+dap+dk RRWM

(c)

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.40.01

0.02

0.03

0.04

0.05

0.06

0.07

Deformation noise σ

Tim

e

# of inliers nin = 20

# of outliers nout = 0

Edge density ρ = 1

RRWMdk RRWM

WKS+dk RRWM

dap+dk RRWM

dB+dk RRWM

dB+dap+dk RRWM

(d)

0 1 2 3 4 5 6 7 8 9 10

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

0.2

0.22

# of outliers nout

Tim

e

# of inliers nin = 20

Deformation noise σ = 0.25

Edge density ρ = 1

RRWMdk RRWM

WKS+dk RRWM

dap+dk RRWM

dB+dk RRWM

dB+dap+dk RRWM

(e)

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10.02

0.03

0.04

0.05

0.06

0.07

Edge density ρ

Tim

e

# of inliers nin = 20

# of outliers nout = 0

Deformation noise σ = 0.5

RRWMdk RRWM

WKS+dk RRWM

dap+dk RRWM

dB+dk RRWM

dB+dap+dk RRWM

(f)

Figure 1: Matching accuracy and computation time of IQP with different compatibility functions.

fully connected graphs purely based on the geometry of thefeature points, taking the Euclidean distance as the weightsbetween pair of feature points. Affinity matrix W were setup similarly as in Section 5.1. |U | = 2 nodes were ran-domly selected as the anchor nodes. We compute the aver-age matching accuracy of each frame to the rest of framesin the sequence. Fig. 2 (a) showed the performance of thematching. As can be seen the matching performance wasimproved when heat kernel is used in lieu of the adjacencymatrix, because the noise tolerance property of the formersmoothes out the effect of deformation noise. With the add-on effect of proximity matrix B and dap(i, a), furthermore,the matching performance was much improved. Fig. 2 (b,c)showes an example of the matching between the 20th andthe 90th frame of the sequence (yellow lines are correctmatches and red lines are wrong matches). Matching givesall correct matches, hence is only shown once.

In the second part of this experiment, we select to testthe influence of the number of anchor nodes on the averagematching rate. We intentionally drop off the term dap to re-duce the side effect, i.e. the matching scheme will be basedon dB + dk. We increase the number of anchor nodes andcompare the matching performance in otherwise the samesetting.

As shown in Fig. 3, with the increase of the number ofanchor nodes, the overall matching performance increased.However, the marginal performance gain seems to have adrop-off with the increase of the number of anchor nodes,

since the matching rate gap between |U | = 10 and |U | = 5

is much smaller than that between |U | = 5 and |U | = 2.

5.3. Pose House Sequence

In this experiment, we test our descriptor on the posehouse sequence used in [18]. The dataset consists of 70frames with 51 labeled feature points across the sequence.The house undergoes large pan and tilt angle change (0 �45

� for pan angle and0�30

� for tilt). The compatibilitymatrix W is built the same way as in previous experiments.|U | = 2 nodes were randomly selected as the anchor nodes.Fig. 4 showed the matching results. In Fig. 4 (a), eachlump from left to right represent a tilt angle from 0 to 30

�

with a 5

� step and within a lump is the pan angle changefrom left to right for 0 to 45

� with a 5

�step. It can beenseen that extreme pan angles gave poor results while midrange pan angles yield matches with much higher accuracy— while matching accuracy was not much influenced by thedifference in tilt angles. With the addition of our learnedproximity matrix B, the gap between different pan anglesdecreases and the overall matching accuracy is superior toothers. Fig. 4 (b,c) shows the matching results and the firstand last frame of the sequence, which represent the largestpan and tilt angles. Even in this extreme case, it can be seenthat our dB + dap + dk matching gives useful results, and ismuch better than the adjacency matrix based matching.

10 20 30 40 50 60 70 80 90 10055

60

65

70

75

80

85

90

95

100

Image Index

Ave

rage

Acc

urac

yAverage accuracy of an image to the rest of the sequence

RRWMdk RRWM

WKS+dk RRWM

dap+dk RRWM

dB+dk RRWM

dB+dap+dk RRWM

(a) Matching performance

(b) RRWM

(c) dB + dap + dk RRWM

Figure 2: Matching on Hotel sequence. Yellow lines de-pict the correct matches, while red lines show the wrongmatches.

6. ConclusionIn this paper, we have considered the problem of graph

matching where some correspondences are known. We havedesigned a learning algorithm which uses the anchor cor-respondences as training samples. The matching problemis set up as an IQP, where we use the learned proximityand heat kernel distance to anchor nodes as first order com-patibility, and pairwise heat kernel distance difference as asecond order compatibility. With a very small number ofanchor nodes (|U | = 2 in all of the experiments) we haveobtained superior performance as compared to the state-of-the-art techniques based on adjacency matrices.

Acknowledgment: The authors would like to ac-knowledge NSF grants IIS 1016324, CCF 1161480, DMS1228304, AFOSR FA9550-12-1-0372, the Max Planck

10 20 30 40 50 60 70 80 90 10060

65

70

75

80

85

90

95

100

Image Index

Ave

rage

Acc

urac

y

Average accuracy of an image to the rest of the sequence

|U|=2|U|=5|U|=10

Figure 3: Matching on Hotel sequence with different num-ber of anchor nodes. (Matching scheme is dB + dk.)

Center for Visual Computing and Communications, and aGoogle research award.

References[1] M. Aubry, U. Schlickewei, and D. Cremers. The wave kernel

signature: A quantum mechanical approach to shape analy-sis. In ICCV Workshop 4DMOD, 2011. 2, 3.1

[2] T. Biyikoglu, J. Leydold, and P. F. Stadler. Laplacian eigen-vectors of graphs: Perron-Frobenius and Faber-Krahn TypeTheorems. Springer, 2007. 3.1

[3] T. Caetano, J. McAuley, C. L., Q. Le, and A. Smola. Learn-ing graph matching. IEEE Trans. PAMI, 31(6):1048–1058,2009. 1

[4] M. Cho, J. Lee, and K. M. Lee. Reweighted random walksfor graph matching. In Proceedings of the 11th Europeanconference on Computer vision: Part V, ECCV’10, pages492–505, Berlin, Heidelberg, 2010. Springer-Verlag. 2, 4,5.1, 5.1

[5] T. Cour, P. Srinivasan, and J. Shi. Balanced graph matching.In NIPS’06, pages 313–320, 2006. 2, 4

[6] D. Emms, R. C. Wilson, and E. R. Hancock. Graph matchingusing the interference of continuous-time quantum walks.Pattern Recogn., 42(5):985–1002, May 2009. 2

[7] M. Eshera and K. Fu. A graph distance measure for imageanalysis. IEEE Trans. Syst. Man Cybern., page 398Ð408,1984. 2

[8] S. Gold and A. Rangarajan. A graduated assignment algo-rithm for graph matching. IEEE Trans. Patt. Anal. Mach.Intell., 18, 1996. 2, 4

[9] M. Gori, M. Maggini, and L. Sarti. Exact and approximategraph matching using random walks. IEEE Trans. PatternAnal. Mach. Intell., 27(7):1100–1111, July 2005. 2

[10] D. K. Hammond, P. Vandergheynst, and R. Gribonval.Wavelets on graphs via spectral graph theory. Applied andComputational Harmonic Analysis, 30(2):129–150, Mar.2011. 3.1

10 20 30 40 50 60 70

55

60

65

70

75

80

85

90

95

Image Index

Ave

rage

Acc

urac

yAverage accuracy of an image to the rest of the sequence

RRWMdk RRWM

WKS+dk RRWM

dap+dk RRWM

dB+dk RRWM

dB+dap+dk RRWM

(a) Matching performance

(b) RRWM

(c) dap + dk RRWM

(d) dB + dap + dk RRWM

Figure 4: Matching on pose house sequence. Yellow linesdepict the correct matches, while red lines show the wrongmatches.

[11] N. Hu and L. Guibas. Spectral Descriptors for Graph Match-ing. ArXiv e-prints, Apr. 2013. arXiv:1304.1572[cs.CV]. 1,2, 3.1, 3.1, 3.1.1, 3.2, 4, 5.1

[12] S. Jouili and S. Tabbone. Graph matching based on nodesignatures. In Proceedings of GbRPR ’09, pages 154–163,Berlin, Heidelberg, 2009. Springer-Verlag. 2

[13] M. Leordeanu and M. Hebert. A spectral technique for cor-respondence problems using pairwise constraints. In ICCV’05, pages 1482–1489, Washington, DC, USA, 2005. IEEEComputer Society. 2, 4

[14] M. Leordeanu, M. Hebert, and R. Sukthankar. An integerprojected fixed point method for graph matching and map

inference. In NIPS. Springer, December 2009. 2, 4[15] M. Leordeanu, R. Sukthankar, and M. Hebert. Unsuper-

vised learning for graph matching. Int. J. Comput. Vision,96(1):28–45, Jan. 2012. 1

[16] M. E. Lübbecke and J. Desrosiers. Selected topics in col-umn generation. Oper. Res., 53(6):1007–1023, Nov. 2005.1, 3.1.1

[17] B. Luo and E. R. Hancock. Structural graph matching usingthe em algorithm and singular value decomposition. IEEETrans. Pattern Anal. Mach. Intell., 23(10):1120–1136, Oct.2001. 2

[18] J. J. McAuley, T. de Campos, and T. S. Caetano. Unifiedgraph matching in euclidean spaces. IEEE Conf. CVPR,pages 1871–1878, 2010. 5, 5.3

[19] H. Qiu and E. R. Hancock. Graph matching and clusteringusing spectral partitions. Pattern Recogn., 39(1):22–34, Jan.2006. 2

[20] A. Robles-Kelly and E. R. Hancock. String edit distance,random walks and graph matching. In Proceedings of theJoint IAPR International Workshop on Structural, Syntactic,and Statistical Pattern Recognition, pages 104–112, London,UK, UK, 2002. Springer-Verlag. 2

[21] A. Robles-Kelly and E. R. Hancock. Graph edit distancefrom spectral seriation. IEEE Trans. Pattern Anal. Mach.Intell., 27(3):365–378, Mar. 2005. 2

[22] C. Schellewald and C. Schnörr. Probabilistic subgraphmatching based on convex relaxation. In EMMCVPR’05,pages 171–186, Berlin, Heidelberg, 2005. Springer-Verlag.2

[23] A. Shokoufandeh and S. J. Dickinson. A unified frameworkfor indexing and matching hierarchical shape structures. InIWVF-4, pages 67–84, London, UK, UK, 2001. Springer-Verlag. 2

[24] J. Sun, M. Ovsjanikov, and L. Guibas. A concise and prov-ably informative multi-scale signature based on heat diffu-sion. In Eurographics Symposium on Geometry Processing(SGP), 2009. 2, 3.1

[25] S. Umeyama. An eigendecomposition approach to weightedgraph matching problems. IEEE Trans. Pattern Anal. Mach.Intell., 10(5):695–703, 1988. 2

[26] B. J. van Wyk and M. A. van Wyk. A pocs-based graphmatching algorithm. IEEE Trans. Pattern Anal. Mach. Intell.,26(11):1526–1530, Nov. 2004. 2, 4

[27] P. C. Wong, H. Foote, G. Chin, P. Mackey, and K. Perrine.Graph signatures for visual analytics. IEEE trans. on visual-ization and computer graphics, 12(6):1399–413, 2006. 2

[28] E. P. Xing, A. Y. Ng, M. I. Jordan, and S. J. Russell. Dis-tance metric learning with application to clustering withside-information. In NIPS, pages 505–512, 2002. 1

[29] M. Zaslavskiy, F. Bach, and J.-P. Vert. A path following al-gorithm for the graph matching problem. IEEE Transactionson PAMI, 31(12):2227–2242, 2009. 2

[30] R. Zass and A. Shashua. Probabilistic graph and hypergraphmatching. CVPR, 2008. 4

[31] G. Zhao, B. Luo, J. Tang, and J. Ma. Usingeigen-decomposition method for weighted graph matching.ICIC’07, pages 1283–1294, 2007. 2