IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE. VOL. IO. NO. 5. SEPTEMBER 1988 695

An Eigendecomposition Approach to W e ighted Graph Ma tching Problems

SHINJI UMEYAMA

Absfruct-This paper discusses an approximate solution to the weighted graph matching prohlem (WGMP) for both undirected and directed graphs. The W G M P is the problem of f inding the opt imum matching between two weighted graphs, which are graphs with weights at each arc. The proposed method employs an analytic, instead of a combinatorial or iterative, approach to the opt imum matching prob- lem of such graphs. By using the eigendecomposit ions of the adjacency matrices (in the case of the undirected graph matching problem) or some Hermitian matrices derived from the adjacency matrices (in the case of the directed graph matching problem), a matching close to the opt imum one can be found efficiently when the graphs are sufficiently close to each other. Simulation experiments are also given to evaluate the performance of the proposed method.

Index Terms-Eigendecomposit ion, inexact matching, structural de- scription, structural pattern recognition, weighted graph matching.

I. INTRODUCTION

S TRUCTURAL description, i.e., the description of a certain object in terms of its parts, their properties,

and their mutual relations. is one of the most general methods for represent ing the real world. Properties and relations may be qualitative (symbolic values) or quanti- tative (numerical values). For example, in the interpre- tation of visual images, a “man” can be descr ibed in terms of his parts (head, body, arms, etc.), their proper- ties (shape, color, etc.), and their relations (head is above body, etc.). Another simple example is a point pattern in 2-D or 3-D space. A point pattern can be represented by using a structural description if we employ distances be- tween points as their mutual relations and point labels (if they exist) as their properties.

There are many problems in handl ing structural de- scriptions, for example, construction of a description from the given data, classification of the given descriptions, etc. Above all, one of the most difficult but interesting problems is opt imum matching between structural de- scriptions, i.e., the problem of finding the correspon- dence between their parts in order to make the corre- sponding propert ies and relations as consistent as possible [ 11. This problem is fundamental in various fields, espe- cially in the fields of image analysis and computer vision. For example, matching feature point descriptions. de-

Manuscript received March 31. 1986: revised January IS. 1988. The author is with the Mathematical Engineering Section. Computer

Science Division, Electrotechnical Laboratory. l-l-4 Umezono. Tsukuba- shi. Ibaraki 305. Japan.

IEEE Log Number 8823194.

r ived from a sensing system, with known models stored in a database is considered to be an essential part of a computer vision system [2].

A great deal of material deal ing with this problem has been publ ished. You [3] employed a combinatorial (tree search) approach and appl ied the branch-and-bound al- gorithm to this problem. The algorithm always gives the true opt imum matching; however, it may be impractical in analyzing highly complex structures because of its in- herent combinatorial property. Tsai and Fu [l] also de- scribed a tree search technique for finding isomorphisms between graphs which bear both symbolic and numerical labels. Kitchen [4], [5] solved this problem in both qual- itative and quantitative cases by using a relaxation method. The method is tolerant to noise. and it can take advantage of hardware parallelism. However, the relax- ation method is inherently a local optimization method, and it is sometimes difficult to determine a suitable up- dating rule (the local compatibility function).

A structural description can be represented by an attrib- uted or labeled (hyper-) graph. A weighted graph, i.e., a graph with weights at each arc, is a simple example of a quantitative structural description. In this paper. we treat weighted graphs with the same number of nodes and em- ploy the analytic, instead of combinatorial or iterative, approach to the opt imum matching problem of such graphs. By using the eigendecomposi t ions of the adja- cency matrices (in the case of the undirected graph match- ing problem) or some Hermitian matrices der ived from the adjacency matrices (in the case of the directed graph matching problem), a matching close to the opt imum one can be found efficiently when the graphs are sufficiently close to each other.

An ordinary graph in graph theory can be considered to be a weighted graph with a binary weighting function. Thus, the weighted graph matching problem (WGMP) in- c ludes the graph isomorphism problem, which is proved neither to be NP-complete nor to have an efficient algo- rithm [6]. It is possible to apply the same method pro- posed here to the graph isomorphism problem.

II. WEIGHTED GRAPH MATCHING PROBLEM (WGMP)

A. Preliminaq A weighted graph G is an ordered pair ( V, \v) where I/

is a set of nodes of the graph and w is a weighting function which gives a real nonnegat ive value w( z~;, 2~;) to each

0162~8828/88/0900-0695$01.00 0 1988 IEEE

696 IEEE TRANSACTIONS ON PATTERN ANALYSlS AND MACHlNE 1NTELLlGENCE. VOL. 10. NO. 5. SEPTEMBER 1988

pair of nodes ( viVj) (arc of the graph), Zli E V, Vj E I’, and vi # Vj. W e assume 1 I/ 1 = IZ. The nodes of a graph are supposed to be labeled by names such as v,, v2, * . * , u,, to be dist inguished from one another. The weighted graph is called undirected when its weighting function is sym- metric, i.e., W(V;, Vj) = W(Vj, 2’;) for all Vi, Z;, V; # Vj.

Otherwise, it is called directed. Obviously, an ordinary graph is a weighted graph with a binary weighting func- tion. When a weighting function takes a value between 0 and 1, such a graph is sometimes called a fuzzy graph [7]. However, here we do not place such a restriction on the range of the weighting function; i.e., it can take any non- negat ive real value.

The adjacency matrix of a weighted graph G = ( I/, w) is an IZ x n matrix A, def ined as follows:

i

av = W(V;, Vj)

AC = [%I a-- = o. i*j

(1) II When G is undirected, A, becomes a symmetric matrix. Fig. 1 shows an example of a weighted undirected graph G with four nodes and its adjacency matrix A,.

B. Definitions of the WGMP Let G = (Vi, w,), H = (V,, w2) be weighted graphs

with n nodes. The WGMP is the problem of finding a one- to-one cor respondence 9 between I/, = { v,, v2, * * * , v, } and V2 = { vi, vi, * . * , VA > which minimizes a “dif- ference” between G and H. In this paper, we use the fol- lowing criterion for a measure of difference:

J(G) = ii, j$, (WI(Vir vj) - w*(+(vi)3 a(vj)))2. C2)

If G and H are weighted graphs and AG and AH are their adjacency matrices, respectively, J(a) can be reformu- lated as follows by using a permutat ion matrix P:

J(P) = llPAGPT - A,,/I’ (3)

where the permutat ion matrix P represents the node cor- respondence + and ]I . I] is the Eucl idean norm ( II A II = (Cy, i Cy= i ( a;j 12)‘12). Thus, the WGMP is reduced to the problem of finding the permutat ion matrix P which min- imizes J(P).

Two ordinary graphs are called isomorphic if there ex- ists a one-to-one cor respondence between their node sets, which preserves adjacency. Similarly, we call two weighted graphs G and H isomorphic if there exists a one- to-one cor respondence @ which makes .7( +) equal to 0. Thus, in this case we have

PAGPT = AH. (4)

Because ordinary graphs are weighted graphs with a bi- nary weighting function, the WGMP includes the graph isomorphism problem.

In general, it is not easy to find the exact solution of the WGMP since the WGMP is a purely combinatorial problem. Thus, we need to develop an efficient method which gives a “nearly” opt imum solution, i.e., a per-

Fig. 1. A weighted undirected graph G and its adjacency matrix A,

mutation matrix P’ whose criterion value J( P’ ) is very close to the opt imum value. Of course, since we cannot know the opt imum criterion value in advance, what we can do is to give an algorithm which determines a per- mutation matrix of a small criterion value. The purpose of this paper is to propose a fast matching algorithm giv- ing such permutat ion matrices.

Note that the WGMP of graphs with weights at each node as well as at each arc can be formulated and solved in a similar way by defining a;; = w (i, i ) in (1) where w (i, i ) is a weight at node i.

III. WEIGHTED UNDIRECTED GRAPH MATCHING ALGORITHM

In this section, we show an algorithm that gives a nearly opt imum solution to the weighted undirected graph matching problem.

Let G and H be weighted undirected graphs and AG and A, be their adjacency matrices, respectively. The opti- mum matching between G and H is a permutat ion matrix P which minimizes J(P) in (3). It is, in general, difficult to find this matrix P directly. However, if we extend the domain of J to the set of orthogonal matrices, the orthog- onal matrices (Q) which minimize J(Q) can be obtained in c losed forms by using the eigendecomposit ion of the adjacency matrices AG and AH. This extension of the do- main is very natural because a permutat ion matrix is a kind of orthogonal matrix. A nearly opt imum permutat ion matrix is determined by using these orthogonal matrices as clues.

Here, we assume that both adjacency matrices AG and A, have n distinct eigenvalues, respectively. This is not a strong assumption for real data. Besides that, even if they have multiple roots, this can be overcome by per- turbing them. Small perturbations will not effect the result of matching.

A. Determination of Nearly Opt imum Permutation Matrix

First, we give Theorem 1 [8]. Theorem 1: If A and B are Hermitian matrices (A =

A* and B = B*) with eigenvalues oI 2 cy2 2 * . * 2 CY,,

UMEYAMA: WEIGHTED GRAPH MATCHING PROBLEMS 691

and 0, 2 fi2 1 * * * 1 p,, respectively, then Thus,

From Theorem 1, we have the following theorem. Theorem 2: Let A and B be n X n (real) symmetric

matrices with IZ distinct eigenvalues oI > cy2 > * . * > CY,, and /3, > fi2 > * . * > fi,,, respectively, and their eigendecompositions be given by

A = U,AJJ,T, (6)

B = U&U,7 (7)

where U, and U, are orthogonal matrices and A, = diag ((Y;) and A, = diag (pi). Then, the minimum of I( QAQT - B 112, where Q ranges over the set of all orthogonal mat- rices, is attained for the following Q ’s:

Q = U,SU,T,

SE S, = (diag (s,, s2, * * * , s,,)(s; = 1 or -1). (8)

The minimum value achieved is Cr=, ((Y; - pi)‘. Proof From Theorem 1, (9) holds for any orthogo-

nal matrix R since eigenvalues of RARr are the same as those of A.

11 RART - Bj12 L ;$, (a; - /3;)2. (9)

On the other hand, if we use Q in (8), we have

11 QAQ’ - BI12

( 10)

where we used the equations that II UX 11 = 11 XUTll = )I X 11 for any orthogonal matrix U and SAAS = S2A, = AA since S and A, are both diagonal matrices and S2 = I. Thus, the minimum of II QAQT - B [I2 is attained for Q ’s given in (8). 0

The orthogonal matrices which minimize J(Q) in (3) are given from Theorem 2 by

Q = U&J;, s E s,, (11)

assuming eigendecompositions of AG and AH as

AG = U,A,U;, (12)

AH = U,A,U;. (13)

Now we assume that G and H are isomorphic. In this case, we have from (4), (12), and (13) that

PUGACU$PT = lJHAHU;. (14)

PU, = U”S, SE s, (15)

since the eigenvectors of a matrix are uniquely deter- mined, except for their positive and negative directions when all eigenvalues are distinct (when x is an eigenvec- tor of a matrix A, -x is also an eigenvector of A). Then,

P = u,su~. (16) This means that there exists some S E S, which exactly makes Q a permutation matrix when G and H are iso- morphic. We write this diagonal matrix as S, and P = U,SU~. Let U, = [hV], U, = [ Sij], S = diag (s;), and r(i) p=i Then we have

Obviously, the following holds:

2 Skhk&r(i).k s I*=, sk iiFiri,k~ i: h.. k=l

n

5 ,c, 1 %hikg?rci ).k 1

Thus,

( 17)

(18)

(19)

(20) where uH and U, are matrices which have as each ele- ment the absolute value of each element of U, and UJ, respectively. Since the length of each row vector of U, and U, is equal to 1 and the values of its elements are nonnegative, the following holds for each g-element xi, (scalar product of the ith row of uH and the jth row of -- 77,) of u&:

0 I xjj I 1. (21)

Thus, we have

tr (PTCIHUg) 5 n (22)

for any permutation matrix P. On the other hand, the fol- lowing holds, obviously:

tr (FTUHfZIG) = tr (Ij’P)

= n. (23)

698 IEEE ‘TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE. VOL. IO. NO. 5, SEPTEMBkR 1988

Thus, from (20), (22), and (23), we have

tr (P“UHUg) I jz. (24)

This means that p maximizes tr (PT??,ng) since tr (PT~/~~) I n for any permutat ion matrix P. There- fore, when G and H are isomorphic, the opt imum per- mutation matrix can be obtained as a permutat ion matrix P which maximizes tr (P’u,UL). This problem is an instance of the assignment (bipartite maximum weighted matching) problem, and can be solved by the Hungar ian method, running in 0( n”) time [9].

Next, we consider the case when G and H are not iso- morphic. From the argument above, even if G and H are not isomorphic, it can be expected that the pennutat ion matrix P which maximizes tr ( PTuHuL) will be very close to the opt imum permutat ion matrix when G and H are nearly isomorphic. Thus, this permutat ion matrix P can be used as a candidate for the opt imum one. This may be valid only when G and H are nearly isomorphic. How- ever, even if G and H are not sufficiently close to each other, the global cor respondence between G and H should be reflected in P. Thus, this permutat ion matrix P gives a good initial cor respondence when we improve it, for example, by hill-climbing or relaxation methods.

B. Exumple for Weighted Undirected Graph Matchirlg W e give an artificial example of our method to clarify

the algorithm. Fig. 2 shows the weighted undirected graphs G and H with four nodes. The adjacency matrices of G and H are AG and AH given in (2.5) and (26):

/ 0.0 5.0 8.0 6.0

\ 5.0 0.0 5.0 1.0

A, = 8.0 5.0 ’

0.0 2.0 (25)

\6.0 1.0 2.0 O.O/

0.0 1.0 8.0 4.0

1.0 0.0 5.0 2.0 A, =

’ 8.0 5.0 0.0 5.0 (26)

4.0 2.0 5.0 0.0

Eigendecomposit ions of Ac; and AH are given as follows:

A, = &A&J;

Ao = diag (14.25, -0.28, -4.83, -9.14),

/ 0 .614 0.141 -0.182 -0.755

(27)

0.434 -0.528 0.726 0.079 UG =

0.548 -0.270 -0.582 0.536

\ 0.366 0.793 0.317 0.369

A, = U,A,U;

H

Fig. 2. An example of the

“1 4 G

ixl 5

2 8

“2’ 5 “i weighted undirected graph matching problem.

AH = diag (13.26, -0.77, -3.43, -9.05),

From this. we have

/ 0.614 0.141 0.182 0.755

\

CI, 0 .434 0.528 0.726 0.079

= 0.548 0.270

’ 0.582 0.536

(29)

0.366 0.793 0.317 0.369

iO.538 0.436 0.425 0.583\

72, 0 .344 0.880 0.018 0.327

= ’ 0.624 0.026 0.250 0.740

(30)

Thus,

iO.450 0.187 0.870 0.079/

/0.909 0.818 0.973 0.893\ --’ u,&

i 0.585 0.653 0.612 0.950 = ’ 0.991 0.524 0.892 0.601

(31)

0.520 0.931 0.846 0.618

The permutat ion matrix P obtained by applying the Hun- garian method to (31) is

0 0 1 0

0001 P= c ) 1000.

(32)

0100 (28) \

UMEYAMA: WEIGHTED GRAPH MATCHING PROBLEMS 699

Hence, zll in G corresponds to vi in H, u2 corresponds to Substituting (37) and (38) into (41), we have vi, vs corresponds to vi, and v4 corresponds to vi. This is the opt imum matching. J’(P) = IIP(A, + iA,)PT - (A, + iA,)112

= II(PAGsPT - AHS) + i(PAGNPT - A~~)II’

= 11 PAGsPT - A& + 11 PAGNPT - A,[]‘. IV. WEIGHTED DIRECTED GRAPH MATCHING

ALGORITHM

W e give the algorithm for the weighted directed graph matching problem in this section. Let G and H be weighted directed graphs and AG and AH be their adja- cency matrices, respectively.

A nearly opt imum solution for the weighted undirected graph matching problem was given in the previous section by using the eigendecomposi t ions of adjacency matrices. This method cannot be appl ied directly to the case of the directed graphs since adjacency matrices are no longer symmetric in this case. However, we can also give a nearly opt imum solution to the weighted directed graph matching problem in a way similar to the case of the un- directed graphs if we use some Hermitian matrices de- rived from the adjacency matrices.

A. Determination of Nearly Opt imum Permutation Matrix

(42) Therefore, a permutat ion matrix P which minimizes J’(P) is the permutat ion matrix which minimizes the differences of the symmetric and the skew-symmetric components of adjacency matrices at the same time.

This matrix P is not, in general, the opt imum permu- tation matrix which minimizes J(P) in (3). However, we adopt here J’ (P) as a criterion instead of J(P) because of its simplicity. J’ (P) is usually equivalent to J( P) when G and H are nearly isomorphic, and they are exactly equivalent when G and H are isomorphic or if G and H are not isomorphic, but undirected.

From Theorem 1, we have the following theorem. Theorem 3: Let C and D be n X n Hermitian matrices

with II distinct e igenvalues yI > y2 > * * . > y,, and 6, > 62 > * *. > 6,,, respectively, and their e igendecom- posit ions be given by

Any matrix can be decomposed uniquely into a sum of c = w,r,w;, (43) a symmetric and a skew-symmetric matrix; thus, adja- cency matrices AG and AH can be decomposed as follows: D = W,r, W; (44)

A GS = (AG + &1/T (33) where Wc and W, are unitary matrices and rc = diag (ri) and rD =

A GN = (AG - A~)/% (34) D112, diag (Si). Then, the minimum of II TCT* -

where T ranges over the set of all unitary matrices,

A HS = (AH + A;)/29 (35) is attained for the following T’s:

A HN = (A, - 422 (36)

where AGS and AHS are symmetric components of AG and AH, and A,, and A, are skew-symmetric components of AG and AH, respectively. Complex matrices EG and EH are def ined from these matrices as follows:

T = W,SW;,

S E S2 = { diag (s,, s2, * * . , s,,)I

Si is any complex number satisfying

ISiJ = 1). (45) EG = AGS + iAGN,

EH = AHS + iAH,,,.

(37) The minimum value achieved is Cl=, ( yi - 6i)2. (38) The proof of this theorem can be obtained in a way sim-

tlar to that of Theorem 2. Obviously, EG and EH are Hermitian matrices.

Ez = A& - iA&.,,

= AGS + iA,

W e assume here that both EG and EH have n distinct e igenvalues for the same reason as in the weighted undi- rected matching problem and give their e igendecomposi- tions as follows:

= E,, (39) EG = WGl?GWrj, (46)

Eg = A& - iA&,, EH = W,r,W,$. (47)

= A, + iAHN

= EH.

Then, from Theorem 3 and an argument similar to that of

(40) the previous section (in this case, the domain of J’ is ex- tended to the set of unitary matrices), a nearly opt imum

Here, we introduce a new criterion .I’ using EG and EH. permutat ion matrix can be given as the permutat ion ma-

J’(P) = ~IPE,P~ - ~~11~. trix P which maximizes tr (P’w,wg). This is obtained

(41) --

by applying the Hungar ian method to W, Wg.

700 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. IO, NO. 5, SEPTEMBER 1988

B. Example of Weighted Directed Graph Matching We give again an artificial example of our method to

clarify the algorithm. Fig. 3 shows the weighted directed graphs G and H with four nodes. The adjacency matrices of G and H are AG and AH, given in (48) and (49):

0.0 3.0 4.0 2.0

0.0 0.0 1.0 2.0

A, = ’ 1.0 0.0 0.0 1.0

0.0 0.0 1.0 0.0

0.0 4.0 2.0 4.0

0.0 0.0 1.0 0.0

A, = . 0.0 2.0 0.0 2.0

0.0 1.0 2.0 0.0

EG and EH are as follows:

/ 0.0 1.5 0.0 1.5 2.5 0.5 1.0 1.0 EG =

2.5 0.5 0.0 1.0

0.0 0.5 1.0

’ - 1.5 -0.5 0.0 0.0

-1.0 -1.0 0.0 0.0

0.0 2.0 1.0 2.0

2.0 0.0 1.5 0.5 EH =

1.0 1.5 0.0 2.0

2.0 0.5 2.0 0.0

(48) VI’ 4 V4’

4 2 2

EJ

2

“2’ 4’ 2

(49) Fig. 3. An example of the weighted directed graph matching problem.

+i

0.25 -0.12 -0.11

0.18 -0.61 -0.43 0.40

0.50 0.32 0.31

0.00 0.00 0.00

(52)

EH = W&W;,

I?” = diag (5.69, -0.35, -1.39, -3.95),

(50) 0.45 -0.25 0.39 -0.43

0.45 -0.70 -0.19 0.49 wfj =

0.45 0.27 -0.71 -0.20

\ 0.50 0.58 0.38 0.52 /

(51) (53)

From this, we have

Eigendecompositions of EG and EH are given as follows:

EG = W&W;,

lYc = diag (4.76, 0.11, -1.34, -3.54),

UMEYAMA: WEIGHTED GRAPH MATCHING PROBLEMS 701

Thus,

/0.957 0.895 0.956 0.753\

--

w,w; 0.886

i

0.947 0.966 0.751 = ’

0.629 0.891 0.834 0.977 (56)

0.905 0.984 0.998 0.840

The permutat ion matrix P obtained by applying the Hun- garian method to (56) is

0010 P= ( 1000

. 0001 ) (57)

0 1 0 0

This is the opt imum matching.

V. PERFORMANCE EVALUATION OF THE WEIGHTED

GRAPH MATCHING ALGORITHM

Simulation experiments are presented here to illustrate the performance of the proposed weighted graph matching algorithm. Pairs of weighted graphs are generated sto- chastically and matched with each other by using several matching algorithms.

Weights of a graph G were produced by a pseudoran- dom number generator, which assigns a real number uni- formly distributed in the range of O-l .O to each arc. A graph H was created by modifying the graph G, i.e., by adding uniformly distributed noise in the range of -e to ft to each weight of G and shuffling the order of nodes, or by producing a new H in the same way as G (we in- dicate this case by writing E = “random”).

Graph pairs of different sizes and different noise levels (E) were produced as input to the algorithms. Sizes rang- ing from 5 to 10 and noise levels ranging from 0 to 0.2 or E = random have been tested. For each parameter, 50 pairs of weighted (undirected and directed) graphs were generated and matched with each other by the following four algorithms:

a) the proposed weighted graph matching algorithm; b) a tree search algorithm (branch-and-bound method)

proposed by You [3], which was used to obtain real op- t imum matchings;

c) the hill-climbing method, i.e., improving a ran- domly generated initial cor respondence by exchanging node correspondences locally so that the value of the cri- terion [eq. (3)] decreases (see Fig. 4); searching stops if the criterion cannot be improved any more; and

d) the hill-climbing method with an initial correspon- dence obtained by the proposed weighted graph matching algorithm.

The tree search algorithm was not tested on graphs with sizes greater than 7 because of the CPU time limit. The sample means and standard deviations of the criterion val-

Fig. 4. Localexchangeofnode correspondences

ues and the total CPU time e lapsed during the matching process were recorded for each parameter. J(P) in (3) was used to compute the criterion value for the cases of both undirected and directed graph matchings, a l though J’ (P) in (42) is used in the directed graph matching al- gorithm. The number of real opt imum matchings obtained by each method was also recorded. The experiments were done on a VAX-l l/780.

Tables I and II show the sample means and standard deviations of the criterion values and the numbers of real opt imum matchings (k) obtained by each method for sev- eral noise levels (E = 0.0, 0.05, 0.10, 0.15, 0.20, and random) where the sizes of the generated graphs are n = 7 and 10. Table I is for the case of undirected graphs, and Table II is for the case of directed graphs. The number of real opt imum matchings obtained is not shown in the table when the size of the graphs is y1 = 10 since method b) (the tree search algorithm) was not tested, and the real opt imum matchings are not known in this case. The last column of each table shows the expected values of the criterion. Supposing that the opt imum matching is not af- fected by adding noise, the expected value E( n, E) of the criterion is given as follows:

E(n, e) = 2n(n - 1)/3 (58) since the var iance of the noise uniformly distributed in the range of --E to + E is equal to e*/3.

Tables III and IV give the sample means and standard deviations of the total CPU time in mill iseconds, e lapsed during the matching process for several graph sizes ( IZ = 5, 6, 7, 8, 9, and 10) when the noise levels were E = 0.1 and = 0.2. Table III is for the case of undirected graphs, and Table IV is for the case of directed graphs.

From these tables, the following is obvious. A combi- natorial method [method b)] seems impractical for large problems (matchings between large graphs) because of its time. The hill-climbing method [method c)] may be ap- plicable to large problems since it is not too slow. How- ever, this method often gives local opt ima because it is a kind of local optimization method. The proposed match- ing method [method a)] is the fastest method among the methods in these experiments and almost always gives the real opt imum matching when a pair of graphs are nearly isomorphic (e I 0.1). However, if a pair of graphs are

TABLE I TABLE III PERFORMANCE EVALUATION OF THE WEIGHTED UNDIRECTED GRAPH CPU TIME ELAPSED (MILLISECONDS) DURIVG THF UNDIRECTED GRAPH

MATCHING ALGORITHMS (50 TRIALS PER DATUM)” MATCHING PROCESS (50 TRIALS PER DATUM)” -

n=i E=olo

\lethod (a) Al&hod (b) Ilethod (c) hlethod (d) \lf~rll0<1 ie) \li~tllilil (I,) \l~~tlloil (c] \I,~ll,O<l /<I) E lIeaniS.D.) k Kzan(S.D.) k hlean(S.D.) k .\lean(S D.) k E(n.:) n \lcanlS D ) Nean(S D ) \Ican(S.D ) \lrarliS D 1

0.00 0.001 0 00) 50 O.OO( 0.00) 50 I 32( 1.29) 18 O.OO( 0.00) 50 0 3 :i6( i) I-I’?( 21) 33( 1Fi iiii 0 OS 0 40( 145) 46 0.04( 0 01) 50 1.30( 1 21) 19 0.04( 0 01) so 0 035 6 .)I( 6) 4911 IO!)) 106( 30) 8li li) 0 10 O.iG( 1.25) 37 0.14( 0.03) 50 137( 1.16) 13 O.?l( 0.34) 48 0 140 i 721 J) liii;( 3iG) Y13( 5li) 13jl 2-l) 0.15 l.iS( l.Sl) 23 0.32( 0.06) 50 1 5i( 1.09) 16 0.43( 0 12) 46 0.31.5 s W( ii -(-I 3li( i;) 131( II) 0.20 ?.SY( 2 IG) 12 0.57( 0.10) SO 1 !IO( 1.18) I.5 0 i?( 0.53) 4-l 0.560 0 1X( IO) -t-i il!l( 12;) %I( i4)

rando7n i.22i I i5) 0 2.40( 0.S:) i0 2.91( 0 SS) 12 ?.i6( 1.01) 18 10 li’2i 16) -i-l Sil( ZTG) 35( 13s)

” = 10 c=O?O

.\lerhod [a) hlethod (I,) \lethod (c) Slethod (d) E \lean(S D ) k IleaniS D.) k \lean(S.D , k .\lean(S D ) k E(n,:] ~lerhorl (.lj .\lF-tllO<l (I,) \lrthoii (r) \lrtlKxi (4)

0 00 0 001 0 001 50 2.19)

( ) j

4.011 2.611 11 0 .iO 0 0.05 0 3i( ( 3.ij( 2.11)

001 0.001 n \leall(S D ) \leaniS D ) \lean(S D ) \lean(S D ) 0 O,( O.dl) - 0 0% i .J6( Y) 193( 40) 531 Ii) 63( 13)

0.10 1 Bl( 3 10) ( ) 4 32( 2 SO) 0 29( 0.04) 0 300 ii 5 I( i) Sl!I( 181) lO.j( 291 lO’( 29) 0 1.5 5 93( 4 98) (- )- 4 9i( 2 62) 109( 146) 0 615 i iC,i i) "Yllf 429) 2091 65) Ii31 41) 0 20 O.Yi( 3 20) i i2( 2 41) 2.2 I( 1 $1) I 200 8 104( 9) -(-I 3Si( 90) ?J?( Oi)

rnallo”L 14 Iii 2 59) 6 671 1 011 G 6ii 1 11) 9 l-10( 11) -i-i .S.j7( 139) 4li[ 142) 10 IYli 111 -i-l YlO( 250) 66Z( 241)

“n and t are the size of graphs and the noise level, respectively. Mean and S.D. give the sample means and the standard deviations of the criterion “n and t are the size of graphs and the noise level, respectively. Mean values J(P). and k is the number of real optimum matchings obtained by and S.D. are the sample means and standard deviations of the total CPU each method. The expected value of the criterion is also given as E(n, E). time.

TABLE I1 TABLE IV PERFORMANCE EVALUATION OF THE WEIGHTED DIRECTED GRAPH CPU TIME ELAPSED (MILLISECONDS) DURING THE DIRECTED GRAPH

MATCHING ALGORITHM (50 TRIALS PER DATUM)U MATCHING PROCESS (50 TRIALS PER DATUM)<’

II=7

Nethod (a) hlethod (b) hlethod (c) !&thod Cd) cc \lean(S.D.j I; Nean(S.D.j I; Ilean(S.D.j k hlean(S.D.j 1; ~(n.z)

0.00 0.001 0.04(

0.001 50 0.001 0.00) SO 24 0.05 o.ooj 50 0.04( o.ooj

1.W so 2.05i

1.961 i.96j

O.OO( o.o.ii

0.00) o.ooj

SO 0 23 50 O.OR.5

0.10 0.37( 1.00) 4i 0.14( 0.02) 50 2.4.2( 1.91) 19 0.14( 0.02) 50 0.140 0.15 1.15( 1.7s) 39 0.3?( 0.04) 50 2.73( 1.86) 17 0.3?( 0.04) 50 0.315 0.20 2.6YC 2.52) 25 0.5ii O.OSi 50 2.i6C l.Yl) 19 0.7ii 0.621 ‘ii 0.560

madorn 6.6i( i.2ij 0 3.38i 0.d so 3x$ 0.65) 5 3.82i o.ssj 13 .

z=O?O

c Kw,[S.D.j k hLw(S.D.j k \lrn,,(S.l).j k .\lw(S.D.j i( E(n.:) 0.00 O.OO( 0 00) SO 0

\Icrho<i ji,l \lethod (I,/ \lethod fci llrthod idi

0.05 0.0X( 0.01) - 0.0X T1 ~ir~lll~~ D I ~~~~~~~~ b j \lean(s.Lj j \leanis b.i

0.10 I .OG( 2.11) - 0.300 .5 Gl( G) lii( 23) 531 2,) 93( 1.5)

0.13 4.Sl( .I,;?) - O.fli5 G Y!% G) iC2i 100) 1X( -1s) 16i( 43)

0.20 ri:IJ( R.2 I) - 1 .?OO 7 131( 12) 19 I’l( rno) 319( 99) ?63( i6)

inniiorri l.l:ll( 1.79) - 8 1X0( I?) -i-l CO?( 1%) 3!13( 1Oi) 9 210( I") -i-l 9361 261) 602( 266)

10 31G( 19) ‘n and t are the size of graphs and the noise level, respectively. Mean

( ) 14X( 418) lOli( 417)

and S.D. give the sample means and the standard deviations of the criterion ‘n and E are the size of graphs and the noise level. respectively. Mean values J(P), and k is the number of real optimum matchings obtained by and S.D. are the sample means and standard deviations of the total CPU each method. The expected value of the criterion is also given as E( n, E). time.

not sufficiently close to each other (E > 0. 1 ), this method VI. CONCLUSION

usually fails to give good matchings. Method d), i.e., the The approximate solution for the WGMP has been given hill-climbing method with an initial correspondence ob- in both the undirected and directed cases. The proposed tained by the proposed matching method, shows the best method employs an analytic approach based on the eigen- performance among the four methods. This method al- decomposition of the adjacency matrix of a graph and al- most always succeeds in giving the real optimum match- most always gives the optimum matching when a pair of ings, even when the noise level is fairly high. This is be- graphs are nearly isomorphic. If graphs are not suffi- cause the initial correspondence obtained by the proposed ciently close to each other, the proposed method some- method is considered to reflect the global correspondence times fails to give the optimum matching. However, the between graphs. hill-climbing method can improve the obtained matching

702 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE. VOL. IO. NO. 5, SEPTEMBER 19X8

E = 0.10

\Iethod (a, Ilethod (b) Llethod (c) Nethod (d) n hlean(S.D ) Slean(S.D.) Ilean(S.D.) ilean(S.D.) .5 i9( 6) I.%( i) YS( 30) 8YC 12) 6 91( 6) 4Glj 12) 173( 31) i-t2( 2oj 7 121( 9) 1135( 19) 31i( 9-4) 204( li) 8 li3( !I) 5Y9( 14i) 3Oi( 66) 9 '233( I?) SW 326) 4201 661

10 BOO( 16) ( ) li56( 4ii) 53Y( 9ij

UMEYAMA: WEtGHTED GRAPH MATCHING PROBLEMS 703

even in this case since the obtained matching reflects the global correspondence between graphs. This method is resistant to the combinatorial explosion in execution time compared to the purely combinatorial approach. It also has the virtue of global optimization, unlike the local op- timization method.

When the weighted graphs are far different from the iso- morphic cases, the method may not work as well as in the nearly isomorphic cases. However, it is usually meaning- less to find the optimum matching between such graphs. Thus, the proposed method is considered to be satisfac- tory in most cases.

The proposed matching method requires a pair of graphs of the same size. This is an important restriction in prac- tice since it means, for example, in a computer vision sys- tem, that the whole object of interest is in the field of view and that the segmentation and feature extraction process will always yield the correct results. It is difficult to sat- isfy these requirements in general. However, in the con- trolled world of industrial robot vision, they are some- times satisfied [2]. Thus, the proposed matching method should be applied to such fields. Of course, the WGMP for the graphs with different numbers of nodes (the subgraph isomorphism problem in graph theory) is a more practical and important problem. However, this is left open for future work.

REFERENCES

[I] W. H. Tsai and K. S. Fu, “Error-correcting isomorphisms of attrib- uted relational graphs for pattern analysis,” IEEE Truns. SW., Man, Cyherne:.. vol. SMC-9, pp. 151-768. Dec. 1979.

121 H. S. Baird, Mode/-Bmrd lrnrrgc Marchincq Using Locution. Cam- bridge, MA: M.I.T. Press, 1985.

[3] M. You and A. K. C. Wang, “An algorithm for graph optimal iso- morphism,” in Proc. 1984 ICPR, pp. 316-319, 1984.

[4] L. Kitchen and A. Rosenfeld, “Discrete relaxation for matching rela- tional structures,” IEEE Truns. Svst., Man. Cybernet., vol. SMC-9, pp. 869-874, Dec. 1979.

]5] L. Kitchen, “Relaxation applied to matching quantitative relational structures.” IEEE Truns. Syst., Man, Cybemet.. vol. SMC-IO, pp. 96-101, Feb. 1980.

[6] M. Gamy and D. S. Johnson, Computers and Intructabilif~: A Guide to the Theor! cj~NP-Conrpieteness. San Francisco, CA: W. H. Free- man, 1979.

[7] D. Dubois and H. Prade, Fuzzy Sets ond Systems: Theory and Appli- cations. New York: Academic, 1980.

[8] A. J. Hoffman and H. W. Wielandt, “The variation of the spectrum of a normal matrix,” Duke Math. J., vol. 20, pp. 37-39, 1953.

[9] C. H. Papadimitriou and K. Steiglitz, Combinatorial Optimization: Al- gorirhtn und Complexity. Englewood Cliffs, NJ: Prentice-Hall, 1982.

Shinji Umeyama was born on August 25, 1954 in Tokushima, Japan. He received the B.S. and M.S. degrees in electrical engineering from Kyoto University in 1977 and 1979, respectively.

Since 1979, he has been with the Electrotech- nical Laboratory, Ibaraki, Japan. From 1986 to 1987 he was a Visiting Research Scientist in the Division of Electrical Engineering of the National Research Council of Canada. His current research interests include graph theory, structural pattern recognition, and computer vision.