paper adriano

8/3/2019 Paper Adriano

1/51

ARTIFICIAL INTELLIGENCE 263

I n c r e a s i n g T r e e S e a r c h E f f i c i e n c y f o rC o n s t r a i n t S a t i s f a c t i o n P r o b l e m sR o b e r t M . H a r a l i c k a n d G o r d o n L . E ll io t tD e p a r t m e n t o f E l e c t r ic a l E n g i n e e r in g a n d D e p a r t m e n t o fC o m p u t e r S c i e n c e , V i r g i n i a P o l y t e c h n i c I n s t i t u te a n d S t a t eU n i v er s i t y , B l a c k s b u r g , V I 2 4 0 6 1 , U . S . A .

Recommended by Nils Nilsson

ABSTRACTIn this paper we e xplore the num ber of tree search operations required to solve binary constraintsatisfaction problems. W e show analytically and experimentally th at the two principles of first tryingthe places most l ikely to fail and remembering wh at has b een done to avoid repeating the samemistak e twice improve the standard backtracking search. W e experimentally show that a lookaheadprocedure called forward checking (to anticipate the futu re) which emp loys the most likely to fai lprinciple performs b ette r than standard backtracking, Ullm an's, W altz's, M ackw orth's, andH ara lick 's discrete relaxation in all cases tested, and b etter than Gaschn ig's backma rking in thelarger problems.

1. Introduct ionAssociated with search procedures are heuristics. In this paper we provide atheory which explains why two heuristics used in constraint satisfaction sear-ches work. The heuristics we discuss can be given a variety of one linedescriptions such as:--Lookahead and anticipate the future in order to succeed in the present.--To succeed, try first where you are most likely to fail.--Remember what you have done to avoid repeating the same mistake.--Lookahead to the future in order not to worry about the past.

We will attempt to show that for a suitably defined random constraintsatisfaction problem, the average number of tree search operations whichemploys these principles will be smaller than that required by the standardbacktracking tree search.

To begin our discussion, we need a precise description of the constraintsatisfaction problem we are attempting to solve by a search procedure. We

Artificial Intelligence 14 (1980) 263-313Copyright by North-Holland Publishing Company


2/51

264 R .M . H A R A L I C K A N D G . L . E L L I O T Fa s su m e t h a t t h e r e a r e N u n i t s ( so m e a u t h o r s c a l l t h e se v a r i a b l e s i n s t e a d o fun i t s ) . Each un i t has a se t o f M poss ib l e va lues o r l abe l s . The cons t ra in ts a t i s f a c t i o n p r o b l e m we c o n s i d e r i s t o d e t e r m i n e a l l p o s s i b l e a s s i g n m e n t s f o fl a b e ls t o u n i t s su c h t h a t f o r e v e r y p a i r o f u ni ts , t h e c o r r e sp o n d i n g l a b e la ss ignment s sa t i s fy the cons t ra in t s . More fo rma l ly , i f U i s t he se t o f un i t s and Lis t h e s e t o f la b e l s , th e n t h e b i n a r y c o n s t r a i n t R c a n b e r e p r e se n t e d a s a b i n a r yre la t ion on U x L : R C_ (U x L) x (U x L ) . I f a pa i r of uni t - lab els (ul , 1~,uz, 12)R , then labels 11 and 12 are sa id to be cons i s t en t o r compa t ib l e fo r un i t s u~ andu2. A label ing f of a l l the uni t s sa t i sf ies the const ra ints i f for every pai r u~, u2 ofuni ts (ul , f (ul) , u2,f(u2))E R. Hara l i ck e t a l . [8 ] ca l l such a l abe l ing a cons i s t en tlabel ing.

T h e p r o b l e m o f d e t e r m i n i n g c o n s i s t e n t l ab e l in g s is a g e n e r a l f o r m o f m a n yprob lems re l a t ed to a r t i f i c i a l i n t e l l i gence . For example , scene l abe l ing andmatch ing [1 , 14] , l i ne in t e rp re t a t ion [16] , edge l abe l ing [5 ] , g raph homomor-ph i sms and i somorph i sms [15] , g raph co lo r ing [9 ] , boo lean sa t i s f i ab i l i t y [8 ] , andp r o p o s i t i o n t h e o r e m p r o v i n g [ 1 0 ] a r e a l l sp e c i a l c a se s o f t h e g e n e r a l c o n s i s t e n tl abe l ing p rob lem. Ul lman [15] , Wal t z [16] , Rosenfe ld e t a l . [14] , Gaschn ig [2 , 3 ,4 ] , a n d M c G r e g o r [ 1 2 ] a t t e m p t t o f i n d e f f i c i e n t m e t h o d s t o so l v e t h e c o n s i s t e n tl a b e l i n g p r o b l e m . K n u t h [ 1 7 ] a l so a n a l y ze s t h e b a c k t r a c k i n g t r e e s e a r c h ,wh i c h i s t h e b a s i s o f m o s t m e t h o d s u se d t o so l v e t h e c o n s i s t e n t l a b e l i n gp r o b l e m .

F o r t h e p u r p o se o f i ll u s tr a ti n g t h e s e a r c h r e q u i r e d t o so l v e t h is p r o b l e m , wec h o o s e th e N - q u e e n s p r o b l e m , h o w t o p la c e N - q u e e n s o n a n N x N c h e c k e r -b o a r d s o t h a t n o q u e e n c a n t a k e a n o t h e r . H e r e , t h e u n i t s e t c o r r e s p o n d s t o t h er o w c o o r d i n a t e s o n a c h e c k e r b o a r d a n d we d e n o t e t h e m b y p o s i t iv e in t e g e rs .T h e l a be l se t co r r e s p o n d s t o t h e c o l u m n c o o r d i n a t e s o n a c h e c k e r b o a r d a n d w ed e n o t e t h e m b y a l p h a b e t i c c h a r ac t e r s . H e n c e , t h e u n i t -l a b e l p a i r (1 , A , 2 , D )sa ti s fi e s t h e c o n s t r a in t R , [( 1, A , 2 , D ) ~ R ] , s i n ce a q u e e n o n r o w 1 c o l u m n Ac a n n o t t a k e a q u e e n o n r o w 2 c o l u m n D . B u t , t h e u n i t - l a b e l p a i r ( 1 , A , 3 , C )d o e s n o t s a t is f y t h e c o n s t r a i n t R b e c a u se q u e e n s c a n t a k e e a c h o t h e r d i a g o n -a l ly (see Fig . ! ) -

Us ing the number l e t t e r conven t ion fo r un i t - l abe l pa i r s , F ig . 2 i l l us t ra t e s ap o r t i o n o f a b a c k t r a c k i n g t r e e t r a c e f o r t h e 6 - q u e e n s p r o b l e m . N o t i c e h o w t h eu n i t 5 l a b e ls A , C , E , a n d F o c c u r t w i c e in t h e t r a c e , e a c h t i m e b e i n g t e s t e d a n df a il in g f o r t h e s a m e r e a so n : i n c o m p a t i b i l it y w i t h u n i ts 1 o r 2 . T h e se r e d u n d a n tt e s ts c a n b e e l i m i n a t e d i f t h e f a c t t h e y f a il e d c a n b e r e m e m b e r e d o r i f u n i ts 1 o r2 c o u l d l o o k a h e a d a n d p r e v e n t 5 f r o m t a k i n g t h e l a b e l s A , C , E , o r F . T h er e m e m b e r i n g d o n e b y G a s c h n ig ' s b a c k m a r k i n g [2 ] a n d t h e f o r w a r d c h e c k i n ga p p r o a c h d e sc r i b e d i n t h i s p a p e r h e l p e l i m i n a t e t h e se p r o b l e m s . N o t i c e t h a tonce un i t 3 t akes l abe l E (F ig . l ( a ) ) t he on ly l abe l s l e f t fo r un i t s 4 and 6 a rei n c o m p a t i b l e . T h e f o r wa r d c h e c k i n g a l g o r i t h m wi l l n o t d i s c o v e r t h i s f u t u r eincompa t ib i l i t y . However , t he f i r s t t ime l abe l B i s a ssoc ia t ed wi th un i t 4 , t he re


3/51

INCREASING TREE SEARCH EFFICIENCY 265B C D E F

r2 i

= I l . . ' ~ \ \

2 " , ' ": " , ~ , , I , , " ,, , I \ 1 \5 i~ i ,v iK ,f ~ I / 1 1 \ , II l / ILI j I \I / I I ~ \ \

Ib IA B C D E F~ ,

2 '3 ' / b ' X

I , . = I

r / \I I / ~ 11 \5 I ,4 ~I . j I i \i I L \

I o lFIG. 1. (a) illustrates how the labeling A, C, E for units 1, 2, 3 implies that the only labels for units4 and 6 are incompatible in the 6 queens problem. (b) illustrates how the labeling A, C, E, B forunits 1, 2, 3, 4 implies that there is no label for unit 6 in the 6 queens problem.

1 A22

A ,BC3 A ,B ,C ,D3 E

4 A4 B

55

A , B , CD6 A , B , C , D , E , FE ,F4 C ,D ,E ,F

3 F4 A4 B

5 A ,B ,C ,D ,E ,F4 C ,D ,E ,F

FIG. 2. A segment of a tree trace that the standard backtracking algorithm produces for a 6 queensproblem. No solutions are found in this segment. The entry 2 A,B, for example, indicates that labels Aand B were unsuccessful at level 2, but 2 C succeeds when checked with past units, and the tree searchcontinues with the next level.


4/51

266 R.M. HARALICK AND G.L. ELLIO'IWis absolutely no label possible for unit 6. Hence, the search through the labelsfor 5 and 6 are entirely superfluous and forward checking will discover this(Fig. l(b)). The lookahead procedures (discrete relaxation) of [11, 13, 14, 15,16] help alleviate the problem illustrated in Fig. l(a) as well as in Fig. l(b).

Section 2 gives a description of the full and partial looking ahead, forwardchecking, backchecking, and backmarking procedures. In Section 3 we com-pare the complexity of these algorithms as they solve the N-queens problemand problems generated randomly. We measure complexity in terms of numberof table lookups and number of consistency checks. These results show thatstandard backtracking is least efficient in most cases and bit parallel forwardchecking is most efficient for the cases tried.

In Section 4, we give a statistical analysis of constraint satisfaction searchesand demonstrate the statistical reason why forward checking requires fewerexpected consistency checks than standard backtracking. In Section 5 weexplore other applications of the fail first or prune early tree search strategiesand show that such particular strategies as choosing the next unit to be thatunit having fewest labels left and testing first against units whose labels areleast likely to succeed reduce the expected number of consistency testsrequired to do the tree search. Finally, by changing the unit search orderdynamically in every tree branch so that the next unit is always the one withfewest labels left, we show experimentally that performance improves for eachprocedure and that forward checking even increases its computational ad-vantage over the other algorithms.

2 . S o m e P r o c e d u r e s f o r T r e e S e a r c h R e d u c i n gIn this section we give brief descriptions of five procedures, and a variation ofdata structure in one, which can be used within the standard backtrackingframework to reduce tree search operations. They are called full and partiallooking ahead, forward checking, backchecking, and backmarking. Each ofthese procedures invests resources in additional consistency tests or datastructures at each point in the tree search in order to save (hopefully) moreconsistency tests at some point later in the tree search.For ease in explaining these procedures, we call those units already havinglabels assigned to them the past units. We call the unit currently being assigneda label the current unit and we call units not yet assigned labels the futureunits. We assume the existence of a unit-label table which at each level in thetree search indicates which labels are still possible for which units. Past unitswill of course have only one label associated with each of them. Future unitswill have more than one. The tree search reducing procedures invest early togain later. Hence, the result of applying any of them in the tree search will beto decrease the number of possible labels for any future unit or reduce thenumber of tests against past units.


5/51

INCREASING TR EE SEARCH EFFICIENCY 2672.1 . Looking aheadW a l t z f i l t e r i n g [ 1 6 ] , a p r o c e d u r e b y U l l m a n [ 1 5 ] , d i s c r e t e r e l a x a t i o n [ 1 4 ] , a n dt h e g r o p e r a t o r o f H a r a l i c k e t a l . [8 ] a r e a ll e x a m p l e s o f a l g o r i t h m s t h a t l o o ka h e a d t o m a k e s u r e t h a t ( 1 ) e a c h f u t u r e u n i t h a s a t l e a s t o n e l a b e l w h i c h i sc o m p a t i b l e w i t h t h e l a b e ls c u r r e n t l y h e l d b y t h e p a s t a n d p r e s e n t u n i t s a n d ( 2)e a c h f u t u r e u n i t h a s a t l e a s t o n e l a b e l w h i c h i s c o m p a t i b l e w i t h o n e o f t h ep o s s i b l e l a b e l s f o r e a c h o t h e r f u t u r e u n i t . L o o k i n g a h e a d p r e v e n t s t h e t r e es e a r c h f r o m r e p e a t e d l y g o i n g f o r w a r d a n d t h e n b a c k t r a c k i n g b e t w e e n u n i t s ua n d v , v < u , o n l y t o u l t i m a t e l y d i s c o v e r t h a t t h e l a b e ls h e l d b y u n i ts 1 t h r o u g hv c a u s e i n c o m p a t i b i l it y o f al l l a b e ls b e t w e e n s o m e u n i t w , w > u , a n d s o m ep a s t , c u r r e n t , o r f u t u r e u n i t .

B e c a u s e l o o k i n g a h e a d i n t h i s m a n n e r c a n n o t r e m e m b e r a n d s a v e m o s t o ft h e r e s u l ts o f t es t s p e r f o r m e d i n t h e l o o k a h e a d o f f u t u r e u n i ts w i t h f u t u r e u n i t sf o r u s e i n f u t u r e l o o k a h e a d s , t h e f u l l s a v i n g s o f l o o k i n g a h e a d a r e n o t r e a l i z e df o r m a n y p r o b l e m s . A p a r t ia l l o o k a h e a d t h a t d o e s n o t d o a ll th e c h e c k s o f f u lll o o k a h e a d w ill p e r f o r m b e t t e r , a n d o n e t h a t c h e c k s o n l y fu t u r e w i t h p r e s e n tu n i t s ( n e g l e c t s f u t u r e w i t h f u t u r e s ) w i l l d o m u c h b e t t e r b e c a u s e a l l t e s t s i tp e r f o r m s c a n b e u s e fu l ly r e m e m b e r e d .

T h e p r o c e d u r e L _ A _ T R E E S E A R C H a n d its a s so c ia te d su b r ou t in e sC H E C K _ . F O R W A R D a n d L O O K _ _ F U T U R E ( F ig . 3 (a ), ( b) a n d ( c) ) is a f o r m a ld e s c r i p t i o n o f t h e f u l l l o o k i n g a h e a d a l g o r it h m , w h i c h c a n e a s i ly b e t r a n s la t e di n t o a n y s t ru c t u r e d r e c u r s i v e l a n g u a g e . U is a n i n t e g e r r e p r e s e n t i n g t h e u n i t,a n d w i ll i n c r e m e n t a t e a c h l e v e l o f t h e t r e e s e a r c h . I t ta k e s o n t h e v a l u e 1 a t th ei n it ia l c a ll . F is a o n e d i m e n s i o n a r r a y i n d e x e d b y u n i t , w h e r e e n t r y F(u) f o ru n i t u i s t h e l a b e l a s s i g n e d to u . T a n d N E W _ T a r e t a b le s , w h i c h c a n b et h o u g h t o f a s a n a r r a y o f l i s t s , T(u) i s a l i s t o f l a b e l s w h i c h h a v e n o t y e tb e e n d e t e r m i n e d t o b e n o t p o s s ib l e fo r u n i t u. (W e i m p l e m e n t e d T a s a 2d i m e n s i o n a r r a y , w i t h t h e n u m b e r o f e n t r i e s i n e a c h l i s t ( o r r o w ) s t o r e d i n t h ef i r s t p o s i t i o n o f t h e r o w . T h i s i m p l e m e n t a t i o n u s e s a p p r o x i m a t e l y( N U M B E R _ O F _ U N I T S ) 2 x ( N U M B E R _ O F _ L A B E L S ) w o rd s o f m e m o r y f ort a bl e s to r a g e sin c e t h e r e c an b e N U M B E R _ O F _ U N I T S l ev e ls o f r e c u rs i o n .)T h e t r e e s e a r c h i s i n it ia l ly c a ll e d w i t h T c o n t a i n i n g a l l l a b e l s f o r e a c h u n i t . A l lo t h er v ar ia b le s c an b e i n te ge rs . E M P T Y _ T A B L E a n d N U M B E R _ O F _ U N I T Sa n d N U M B E R _ O F _ L A B E L S h a ve o b v i o u s m e an in gs .

T h e f u n c ti o n R E L A T I O N ( u b 11, u2, 12) r e t u r n s TRUE i f ( u l , !1, U2,/2) E R ,o t h e r w i s e i t r e t u r n s FALSE. C H E C K _ F O R W A R D c he ck s t h a t e ac h f u t u reu n i t - l a b e l p a i r i s c o n s i s t e n t w i t h t h e p r e s e n t l a b e l F(u) f o r u n i t u a s i t c o p i e st h e t ab le T i n to th e n e x t l e ve l t ab le N E W _ T , L O O K _ F U T U R E t h e n ch e ck st h a t e a c h f u t u r e u n i t - la b e l p a i r in N E W _ T is c o n s i s t e n t w i t h a t le a s t o n e l a b elf o r e v e r y o t h e r u n i t , a n d d e l e t e s t h o s e t h a t a r e n o t .

I n th i s i m p l em e n ta t i on C H E C K _ F O R W A R D a nd L O O K _ F U T U R E r e t u r na fla g, E M P T Y _ R O W _ F L A G , if a u n it is f o u n d w i th n o p o ss ib le c o n s i st e n t


6/51

268 R . M . H A R A L I C K A N D G . L . E L L I O ' I " F1 . R E C U R S I V E P R O C E D U R E L_A_TREE_SEARC H(U, F , T) ;2. FOR F(U) = ea ch e leme nt o f T (U ) B E G I N3. IF U < NUMBE R_OF_UNITS T H E N B E G I N4. N E W T = CHECK_FORWARD(U, F (U ) , T ) ;5. CALL LOOK_FUTURE(U, NEW_T) ;6 . IF NEW_T i s no t EMPTY_ROW_FLAG THEN7. CALL L_A_TREE_SEARC H(U + 1, F , NEW _T) ;8. END;9 . . E L S E10. Outpu t the l abe l ing F ;11. E N D ;12. E N D L_A_TREE_SEARCH;

(a )1 . P R O C E D U R E CHECK_FOR WAR D(U, L, T) ;2 . NEW_T = emp ty tab le ;3 . F O R U2 = U + 1 TO NUMBER_OF_UNITS B E G I N4. FOR L2 = each e lem ent o f T (U2)5. IF RELATION(U, L, U2, L2) THEN6. Enter L2 in to the l is t NEW _T(U2);7. IF NEW_T(U2) is em pty THEN8 . R E T U R N ( E M P T Y _ R O W _ F L A G ) ; /* N o c o n s is t e n t l a b e ls * /9. END;10. R E T U R N (NEW_T) ;

11. END CHECK_FORWARD;(b )

1 . P R O C E D U R E LOOK_FUTURE(U, NEW_T) ;2. IF U + 1 _-> NUM BE R_O F_U NITS T H E N R E T U R N ;3 . FOR U1 = U + 1 TO NUMBER_OF_UNITS B E G I N4. FOR L1 = each e lem ent o f NEW_T(U1)5. FOR U2 = U + 1 T O NUMBER_OF_UNITS excep t sk i pp ing U1 B E G I N6. FOR L2 = each e leme nt o f NEW _T(U2)7. IF RELATION(U1, L1, U2, L2) THEN8 . B R E A K f o r L2 l oop ; / * cons i s ten t l abe l found * /9. I F n o cons i s ten t l abe l wa s found fo r U2 T H E N B E G I N

10. De lete L1 f rom l is t NEW _T(U1);11. B R E A K f o r U2 l oo p ; / * un i t U2 has no l abe l cons i s ten t w i th U l , L1 * /12. E N D ;13. E N D f o r U2 l oop ;14. E N D for L1 loop;15. IF NEW_T(U1 ) i s emp ty T H E N B E G I N1 6. N E W T = E M P T Y _R O W _ F LA G ;1 7 . R E T U R N ;18. E N D ;19. E N D for U1 loop;20. R E T U R N ;21. E N D LOOK_FUTURE;

(c )


7/51

I N C R E A S I N G T R E E S E A R C H E F F I C IE N C YFro. 3 (continued).

269

1 . P R O C E D U R E PARTIAL_LOOK_FUTURE(U, NEW_T) ;2. IF U + 1 >_-NUMBER_OF_UNITS T H E N R E T U R N ;3 . F O R U 1 = U + 1 TO N U M B E R _ O F _ U N I T S - 1 B E G I N4. FOR L1 = each e leme nt o f NEW_T(U1 )5. FOR U2 = U1 + 1 T O NUMBER_OF_UNITS B E G I N6 . F O R L 2 = e a c h e l e m e n t o f NEW_T(U2)7. IF RELATION(U1, L1, U2, U2) THEN8 . B R E A K f o r L2 l o o p ; / * cons i s ten t l abe l found * /9. IF no cons i s ten t labe l was found fo r U2 T H E N B E G I N

10. Delete L1 f rom l is t NEW _T(U1);1 1 . B R E A K f o r U2 l oo p ; / * un it U2 has no l abe l cons i s ten t w i th U1, L1 * /12. E N D ;13. E N D f o r U2 l oop ;14. E N D for L1 loop;15. IF NEW _T(U1 ) is em pty T H E N B E G I N16. NEW_T = EMPTY_ROW_FLAG;17. R E T U R N ;18. E N D ;19. E N D for U1 loop;20. R E T U R N ;21. E N D PARTIAL_LOOK_FUTURE;

(d)FIG. 3. (a), (b) , and (c) expre ss the full look ing ahead algorithm . Rep lace th e call toL O O K _ F U T U R E a t l in e 5 in ( a ) w i th a n i d e n ti ca l c a ll to P A R T I A L _ L O O K _ F U T U R E ( d) a n d t h epar t ia l looking ahead algor i thm is obtained. Forw ard checking consists of (a) and (b) wi th l ine 5 of( a) , t h e c a ll to L O O K _ F U T U R E , d e l e te d s o t h at o n l y C H E C K _ F O R W A R D is c al le d . F o r w a rdchecking does no checks of future uni ts wi th future units. (c) i s the L OO K._FU TU RE procedu re,which deletes future labels which are not consistent wi th at least one label for every uni t o ther thanthe lab el ' s own uni t. (d) i s the PA RT IAL _LO OK _.FU TU RE proc edure. I t d if fers f romLO OK _F UT UR E (c) only at l ines 1 , 3 , and 5 . Each future uni t - label pai r i s checked only wi th uni tsin i ts own future.

l a b e l s . T h u s t h e n e x t l e v e l o f t h e t r e e s e a r c h w i ll n o t b e c a l l e d , o t h e r w i s e e a c he n t r y in N E W _ T is c o n s i s t e n t w i t h u , F ( u ) , a n d t h e r e f o r e , a l l t h e p a s tu n i t - l a b e l p a i r s .2 .2 . Pa r t ia l lo o k in g a h e a dP a r t i a l l o o k i n g a h e a d is a v a r i a t i o n o f l o o k i n g a h e a d w h i c h d o e s a p p r o x i m a t e l yh a l f o f th e c o n s i s t e n c y c h e c k s t h a t f u l l l o o k i n g a h e a d d o e s w h i l e c h e c k i n gf u t u r e w i t h fu t u r e u n i ts . E a c h f u t u r e u n i t - l a b e l p a i r i s c h e c k e d o n l y w i t h u n i t si n i ts o w n f u t u r e , r a t h e r t h a n a l l o t h e r f u t u r e u n i t s . T h u s p a r t i a l l o o k i n g a h e a dis l e ss p o w e r f u l t h a n f u ll l o o k i n g a h e a d i n t h e s e n s e t h a t i t w i l l n o t d e l e t e a sm a n y u n i t - l a b e l p a i r s f r o m t h e l i s ts o f p o t e n t i a l f u t u r e l a b e l s . W e w i l l , h o w e v e r ,s e e t h a t p a r t i a l l o o k i n g a h e a d d o e s f e w e r t o t a l c o n s i s t e n c y c h e c k s t h a n f u lll o o k i n g a h e a d i n a l l c a s e s t e s t e d .


8/51

270 R.M. HARALICKAND G.L. ELLIOTTThe checks of future with future units do not discover inconsistencies often

enough to justify the large number of tests required, and these results cannotbe usefully remembered. Since partial looking ahead does fewer of these lessuseful tests, it is more efficient. A look ahead that checks only future withcurrent or past units can have better performance since these more powerfultests can also be usefully remembered.

The formal algorithm for partial looking ahead is L_A_TREE_SEARCH(Fig. 3(a)), with the call to LOOK_FUTURE on line 5 replaced with anidentical call to P ARTIAL_L OOK_FUTURE (Fig. 3(d)).

2 .3 . Fo rw a rd check ingForward checking is a partial lookahead of future units with past and presentunits, in which all consistency checks can be remembered for a while. Thismethod is similar to looking ahead, except that future units are not checkedwith future units, and the checks of future units with past units are remem-bered from checks done at past levels in the tree search. Forward checkingbegins with a state of affairs in which there is no future unit having any of itslabels inconsistent with any past unit-label pairs. This is certainly true at theroot of the tree search, since there are no past units with which to beinconsistent. Because of this state of affairs, to get the next label for the currentunit, forward checking just selects the next label from the unit table for thecurrent unit. That label is guaranteed to be consistent with all past unit-label pairs. Forward checking tries to make a failure occur as soon as possiblein the tree search by determining if there is any future unit having no labelwhich is consistent with the current unit-label pair. If each future unit hasconsistent labels, it remembers by copying all consistent future unit-label pairsto the next level's unit-label table. If every future unit has some label in theunit label table which is consistent with the current unit-label pair, then thetree search can move forward to the next unit with a state of affairs similar tohow it started. If there is some future unit having no label in the unit labeltable which is consistent with the current unit-label pair, then the tree searchremains at the current level with the current unit and continues by selecting thenext label from the table. If there is no label, then it backtracks to the previousunit and the previous label table.

The formal algorithm for forward checking is the ProcedureL_A_TREE_SEARCH (Fig. 3(a)) with line 5, the call to LOOK_FUTURE,removed. Forward checking is just looking ahead, omitting the future withfuture checks.An improvement in efficiency can be gained in the lookahead type ofalgorithms by using a data structure for the unit-label tables that is suggestedby McGregor [12]. McGregor simultaneously developed a weaker form of theforward checking algorithm and compared them on subgraph isomorphism


9/51

INCREASING TR EE SEARC H EFFICIENCY 271p ro ble m s. In L _ A _ T R E E S E A R C H a nd C H E C K _ F O R W A R D t h e lis ts T ( u )o r N E W _ T ( u ) c a n b e s t o r e d a s b i t v e c t o r s , w i t h o n e b i t i n t h e m a c h i n e w o r dfor each poss ib l e l abe l ( th i s i s e ssen t i a l ly a se t r epresen ta t ion) . L ines 4 , 5 and 6in C H E C K _ F O R W A R D (F ig. 3 ( b) ) c a n th e n b e r e p la c e d w i th t h e f o ll ow i n gs t a t e m e n t :

N E W _ T ( U 2 )= AN D ( T ( U 2 ) , R E L A T I O N _ B I T _ V E C T O R ( U , L , U 2 )) .

T h i s s i n g l e s t a t e m e n t r e p l a c e s a l o o p , t a k i n g a d v a n t a g e o f t h e p a r a l l e l b i th a n dl in g c a pa b il it ie s o f m o s t c o m p u t e rs . R E L A T I O N _ B I T _ V E C T O R ( u , l, u2)r e t u r n s a b i t v e c t o r w i t h a b i t o n i n e a c h p o s i t i o n c o r r e sp o n d i n g t o a l a b e l 1 2f o r w h i c h R E L A T I O N ( u , l, u 2 , / 2 ) w o u l d h a v e b e e n t r u e . T h is i s e s s en t ia l ly th es e t { 1 2 1 (u , , u 2 , 1 2 ) ~ R } . T h u s N E W _ T ( u 2 ) b e c o m e s t h e s e t { 12 ET ( u 2 ) l ( u , ! , u 2 , / 2 ) E R } , p r e c i s e l y wh a t l i n es 4 , 5 , a n d 6 d o . I f t h e n u m b e r o fl a b e l s i s l e s s t h a n t h e wo r d l e n g t h o f t h e c o m p u t e r u se d , t h e n t h e r e l a t i o n c a nb e d i r e c t l y s t o r e d i n a n a r r a y o f s i z e ( N U M B E R _ O F _ U N I T S )2 ( N U M B E R _ O F _ L A B E L S ) , an d th e ta b le s T a n d N E W _ T w ill t ak e ap -p r o x i m a t e l y ( N U M B E R _ O F _ U N I T S )2 w o r d s o f s to r a g e . R e d u c e d f o r m s o f t h er e l a t i o n e x i s t f o r s o m e p r o b l e m s , s u c h a s t h e N - q u e e n s p r o b l e m o r t h es u b g r a p h i s o m o r p h i s m p r o b l e m , i n w h i c h o n l y t w o d i m e n s i o n a l t a b l e s n e e d b es t o r e d a n d a q u i c k c a l c u l a t i o n w i l l g e n e r a t e t h e n e e d e d l i s t . T h e s a m e t e c h -n i q u e c a n b e a p p l i e d t o t h e f u l l a n d p a r t i a l l o o k i n g a h e a d a l g o r i t h m s , b u t t h e ywi ll n o t b e c o m p a r e d h e r e s i n c e th e t h r e e a l g o r it h m s w i ll h a v e a p p r o x i m a t e l yt h e s a m e r e l a t i o n sh i p s o f ef f ic i en c y , t o e a c h o t h e r , w i t h o r w i t h o u t t h e i m -p r o v e d d a t a s t r u c t u r e .

2.4 . BackcheckingB a c k c h e c k i n g is s i m i la r t o f o r wa r d c h e c k i n g i n t h e wa y i t r e m e m b e r s u n i t- l ab e lp a i r s wh i c h a r e k n o wn t o b e i n c o n s i s t e n t w i t h t h e c u r r e n t o r a n y p r e v i o u s u n i tl a b e l. H o w e v e r , it k e e p s t r a c k o f t h e m b y t e st i n g t h e c u r r e n t u n i t l a b e l o n l ywi t h p a s t u n i t - l a b e l p a i r s a n d n o t f u t u r e o n e s . So i f , f o r i n s t a n c e , l a b e l s A , B ,a n d C f o r u n i t 5 we r e t e s t e d a n d f o u n d i n c o m p a t i b l e w i t h l a b e l B f o r u n i t 2 ,t h e n t h e n e x t t i m e u n i t 5 m u s t c h o o se a l a b e l , i t s h o u l d n e v e r h a v e A , B , o r Cas label possibi l i t ies as long as uni t 2 s t i l l has the label B.

E a c h t e s t t h a t b a c k c h e c k i n g p e r f o r m s w h i l e l o o k i n g b a c k f r o m t h e c u r r e n tu n i t u t o so m e p a s t u n i t v , f o r wa r d c h e c k i n g w i l l h a v e p e r f o r m e d a t t h e t i m eu n i t v wa s t h e c u r r e n t u n i t . O f c o u r se , a t t h a t t i m e , f o r wa r d c h e c k i n g w i l l a l soh a v e c h e c k e d a l l f u t u r e u n i t s b e y o n d u n i t u . H e n c e , b a c k c h e c k i n g p e r f o r m sf e we r c o n s i s t e n c y t e s t s , a n a d v a n t a g e . B u t b a e k c h e c k i n g p a y s t h e p r i c e o fh a v i n g m o r e b a c k t r a c k i n g a n d a t l e a s t a s l a r g e a t r e e a s f o r wa r d c h e c k i n g .B a c k c h e c k i n g i t s e l f i s n o t a s g o o d a s f o r wa r d c h e c k i n g .


10/51

272 R.M. HARA LICK AND G.L. ELLI OTF

2 . 5 . BackmarkingBackmarking (defined in Gaschnig [2] and also discussed in Gaschnig [3]) isbackchecking with an added feature. Backmarking eliminates performing someconsistency checks that were previously done, had not succeeded, and if doneagain would again not succeed. Backmarking also eliminates performing someconsistency checks that were previously done, had succeeded, and if done againwould again succeed. To understand how backmarking works, recall that thetree search by its very nature goes forward, then backtracks, and goes forwardagain. We focus our attention on the current unit u. We let v be the lowestordered unit to which we have backtracked (has changed its label) since the lastvisit to the current unit u. Backmarking remembers v. If v = u, then back-marking proceeds as backchecking. If v < u, then since all the labels for unit uhad been tested in the last visit to unit u, any label now needing testing, needsonly to be tested against the labels for units v to u - 1, which are the oneswhose labels have changed since the last visit to unit u. That is, the tests donepreviously against the labels for units 1 through v - 1 were successful and ifdone again would again be successful because labels for units 1 through v - 1have not changed and the only labels permitted for the current unit u are thosewhich have passed the earlier tests (see Fig. 4).

U N I T N U M B E R

-1 -u

I11

t zb dm~ rZ

iIII

A BFIG. 4. A segment of a tree trace. Because backmarking remembers from the first visit to unit uwhich labels for u were c ompatibl e with the labels for units ! t hroug h v - 1 (segment A) do n othave to be perfo rmed . Only those for units v through u - l (seg ment B) have to be perf ormed .


11/51

I N C R E A S I N G T R E E S E A R C H E F F IC I E N C Y 2 7 3

T h e f o r m a l a l g o r i t h m f o r B a c k m a r k i n g a p p e a r s i n F i g . 5 . I t i s e s s e n t i a l l yG a s c h n i g ' s a lg o r i th m [ 2] , b u t m o d i f i e d t o f in d a ll s o l u t i o n s . T h e v a r i a b l e U a n da rr ay F a r e t h e sa m e a s i n l o o k i n g a h e a d . L O W U N I T i s a o n e d i m e n s i o n a la rra y o f N U M B E R _ O F _ U N I T S e n tr ie s , a n d L O W U N I T ( i ) w i ll i n d ic a te th el o w e s t l e v e l a t w h i c h a c h a n g e o f l a b e l h a s o c c u r r e d s i n c e t h e l a s t t i m e t h eM A R K arra y is s et . M A R K is d i m e n s io n e d N U M B E R _ O F _ U N I T S b yN U M B E R _ O F _ L A B E L S , a n d M A R K ( u , l ) w i ll i n d ic a te th e l o w e st l ev e l a tw h i c h a c o n s i s t e n c y t e s t f a i l e d w h e n t h e u n i t - la b e l p a i r (u , l ) a t t h e c u r r e n t l e v e lw a s l a s t t e s t e d a g a i n s t t h e p r e v i o u s u n i t - l a b e l p a i r s o n p r e v i o u s l e v e l s . A t a n yp o i n t i f M A R K ( u , l ) i s l e s s t h a n L O W U N I T ( u ) , t h e n t h e a l g o r i t h m k n o w s t h a t(u , l ) h a s a lr e a dy b e e n te s te d a g a in s t th e un i t - la b e l pairs a t l e ve l s b e lo w th ev a l u e i n L O W U N I T ( u ) a n d w i ll f a il a t l e v e l M A R K ( u , l ) , s o t h e r e is n o n e e d t or e p e a t th e t e st s. I f M A R K ( u , l ) i s g r ea t er o r eq u a l t o L O W U N I T ( u ) , t h e n a llt e st s w il l s u c c e e d b e l o w a n d l e v e l L O W U N I T ( u ) a n d o n l y t e st s a ga in s t u n it s a tL O W U N I T ( u ) t o t h e c u r r e n t u n i t n e e d b e t e s t e d .

B e f o r e t h e i n it ia l c a ll t o B A C K M A R K , a ll e n t r ie s i n L O W U N I T a n dM A R K a r e i n it ia l iz e d t o 1 , a n d B A C K M A R K i s c a l le d w i t h th e i n it ia l u = 1 .S i n c e th e s a m e M A R K a n d L O W U N I T a rr ay s a r e u s e d a t a l l l e v e ls o fr e cu r si on o f t h e tr e e se a rc h , a p p ro x im a t e ly ( N U M B E R _ O F _ U N I T S ) ( N U M B E R _ O F _ L A B E L S ) w o r d s o f t a bl e s to r ag e are n e e d e d .

1 . R EC U R S I VE PR O C ED U R E B A C K M A R K ( U , F , M A R K , L O W U N I T ) ;2 . FOR F ( U ) = 1 TO N U M B E R _ O F _ L A B E L S B E G I N3. IF M A R K ( U , F ( U ) ) _-> LO W U N I T ( U ) T H E N B E G I N4 . T E S T F L A G = T R U E;5 . I = L O W U N I T ( U ) ;6. WHILE ( I < U) B E G I N / * F i n d l o w e s t f a i l u r e * /7 . T E S T F L A G = R E L A T I O N ( I , F ( I) , U , F ( U ) ) ;8 . I F N O T T E S T F L A G T H EN BR EAK w h i l e l o o p ;9 . I = 1 + 1 ;10. END w h i l e l o o p ;

1 1 . M A R K ( U , F ( U ) ) = I ; / * M a r k l a b e l w i th l o w e s t f a il u r e * /12 . IF T E S T F L A G THEN13 . I F U < N U M B E R _ O F _ U N I T S THEN1 4 . CALL B A C K M A R K ( U + 1 , F , M A R K , L O W U N I T ) ;15 . ELSE16. O u t p u t t h e l a b e l i n g F ;17 . E N D ;18 . E N D f o r F l o o p ;1 9 . L O W U N I T ( U ) = U - 1 ; / * P r e v i o u s l e v e l w i l l n o w c h a n g e * /2 0 . FO R I = U + 1 TO N U M B E R _ O F _ U N I T S ;2 1 . L O W U N I T ( I) = M I N ( L O W U N I T ( I ) , U - 1 ;2 2 . R E T U R N ;2 3 . E N D B A C K M A R K ;FIG. 5. G a s e h n i g ' s b a c k m a r k i n g p r o c e d u r e a s it w a s m o d i f i e d t o f in d a ll s o lu t i o n s t o c o n s t r a i n ts a t i s f a c t i o n p r o b l e m s ( s e e [2]).


12/51

274 R.M. HARALICKAND G.L. ELLIOT]"3. Experimental Results

In this section we compare the six procedures, partial and full looking ahead,backtracking, backchecking, forward checking, and backmarking, on the N-queens problem for 4


13/51

I N C R E A S I N G T R E E S E A R C H E F F IC I E NC Y 2 7 5

I0"

IIAC~KING FUIL LOOKING AHEAD o

BACKCFECK NG PARTIAL LOOKING AHEAD D

FORWARDCHECKING BACIC',ARKING

BIT PARALLELF ~ CHECKING (BIT VECTOR OPERATIONS)

( / )I . - .0 3h it ' - -> -( JZI.IJI - -U ~Z0( j 'M .0

, . = , ~= E: 3z

1 0 2 ,

I I I | I I4 5 6 7 8 9 I0N U M B E R O F Q U E E N S

FIG. 6 . T h i s f ig u r e c o m p a r e s t h e e f f ic ie n c y o f t h e s t a n d a r d b a c k t r a c k i n g p r o c e d u r e w i t h b a c k -c h e c k i n g , l o o k i n g a h e a d , f o r w a r d c h e c k i n g , a n d b a c k m a r k i n g , fo r t h e N - q u e e n s p r o b l e m i n th en a t u r a l u n i t o r d e r . T h e n u m b e r o f b i t v e c t o r o p e r a t i o n s f o r b i t p a r a l l e l f o r w a r d c h e c k i n g i s a l s os h o w n .


14/51

2 7 6 R.M. HAR AL IC K AND G.L. ELLIO'VI"

1 0 3

t/3zo5oqLt .o 1 0 213EIL l130~EZ

1 0

/ ' ~ _ I I I I6 7 8 9 10N

F I G . 7 . T h e n u m b e r o f s o lu t i on s f o r t h e N - q u e e n s p r o b l e m .

I l11 12 13

f o r f o r w a r d c h e c k i n g d o n e i n t h e b i t v e c t o r d a t a s t r u c t u r e . T h o u g h b a c k m a r k -i n g a p p e a r s t o d o s l i g h t l y f e w e r c o n s i s t e n c y c h e c k s t h a n f o r w a r d c h e c k i n g o nt h e N - q u e e n s p r o b l e m , t h e u s e o f m a c h i n e p a r a l l e l i s m g i v e s b i t p a r a l l e lf o r w a r d c h e c k i n g a c l e a r a d v a n t a g e o v e r a l l t h e o t h e r a l g o r i t h m s .

R a n d o m c o n s t r a i n t p r o b l e m s w i t h f i x e d p r o b a b i l i t y 0 . 6 5 o f c o n s i s t e n c y c h e c ks u c c e s s p r o b a b i l i t y o f 0 . 6 5 a r e t e s t e d i n F i g s . 8 a n d 9 . T h e n u m b e r o fc o n s i s t e n c y t e s t s a ppe a r s t o g r o w e x po n e n t i a l l y i n F i g . 8 , u n t i l a s u f f i c i e n t l yl a r ge p r o b l e m s i z e is r e a c h e d . A t t h is p o i n t t h e n u m b e r o f s o l u t i o n s d r o p s , a s i si n d i c a t e d i n F i g . 9 , a n d t h e n u m b e r o f c o n s i s t e n c y t e s t s a p p e a r s t o g r o w m o r es l o w l y . F i g . 9 e x p l a i n s t h e u n e v e n e s s o f t h e c u r v e s i n F i g . 8 . T o o f e w r a n d o m


15/51

I N C R E A S I N G T R E E S E A R C H E F F I C I E N C Y 277

i0 ~

Io4

t ~b J

>-(_>ZILl~-.

O0Z0C.>

o 10 3nrUJm

z

Io 2

I I I f I I

FULL LOOKING AHEADPARTIAL LOOKING AHEADBAClq~IARKINGFORWARD CHECKING

PARALLEL FORWARD CHECKING(bit vector operations)

I , l I I I , i I I i I I4 5 6 7 8 9 I 0 I IN U M B E R O F U N I T S = N U M B E R O F L A B E L S = N

I ~ G . 8 . T h e n u m b e r o f c o n s i s t e n c y t e s t s i n th e a v e r a g e o f 5 r u n s o f t h e i n d i c a t e d p r o g r a m s .R e l a t i o n s a r e r a n d o m w i t h c o n s i s t e n c y c h e c k p r o b a b i l i t y p = 0 . 6 5 a n d n u m b e r o f u n i t s = n u m b e r o fl a b e l s = N . E a c h r a n d o m r e l a t i o n i s te s t e d o n a l l 6 m e t h o d s , u s i n g t h e s a m e 5 d i f f e re n t r e l a t io n sg e n e r a t e d f o r e a c h N . T h e n u m b e r o f b i t - v e c to r o p e r a t i o n s i n b i t p a r a l le l f o r w a r d c h e c k i n g i s a l s os h o w n f o r t h e s a m e r e l a t i o n s .


16/51

278 R . M . H A R A L I C K A N D G . L . E L L I O T F

u~zo. - - I

ola_otYlaJmz

100908 07C6C

5C4C

30

I I I 1 1 l 1 I

!/

I I I t I I I I4 5 6 7 8 9 10

N = NUMBER OF UNITS --NUMBER OF LABELSF I G . 9 . T h e a v e r a g e n u m b e r o f s o l u t i o n s f o r t h e p r o b a b i l is t i c r e l a t io n s i n F i g . 8 . T h i s i s t h e a v e r a g e o f 5e x p e r i m e n t s f o r e a c h p r o b l e m s i ze , a n d r e l a ti o n s a r e r a n d o m w i t h c o n s is t e n c y c h e c k s u c c e s sp r o b a b i l i t y p = 0 . 6 5 . T h e d o t t e d c u r v e i s t h e e x p e c t e d n u m b e r o f s o l u ti o n s .

r e l a t i o n s w e r e t e s t e d f o r t h e m e a n s t o s e t t l e c l o s e l y t o t h e e x p e c t e d v a l u e s f o rt h is t yp e o f p r o b l e m , a n d t h e a v e r a g e n u m b e r o f s o l u t i o n s v a ri e s er a t ic a ll y h i g ha n d l o w .

I n t h e r a n d o m p r o b l e m s , f o r w a r d c h e c k i n g d o e s s l i g h t l y f e w e r c o n s i s t e n c yc h e c k s t h a n b a c k m a r k i n g i n t h e l a r g e r p r o b l e m s i z e s , a n d o n c e a g a i n m a c h i n epa r a l l e l i s m g ive s b i t pa r a l l e l fo r w a r d c h e c k in g a c le a r a dva n ta g e .

T h e n u m b e r o f t a b l e l o o k u p s f o r t h e N - q u e e n s a n d r a n d o m r e l a t i o n p r o b -l e m s a r e c o m p a r e d i n F i g s. 1 0 a n d 1 1 . O n l y t h e l o o k u p s i n t h e M A R K a r ra y i nb a c k m a r k i n g a n d b ac k c h e ck i n g , a n d t h e T o r N E W _ T t a bl es i n t h e l o o k a h e a d


17/51

I N C R E A S I N G T R E E S E A R C H E F F I C IE N C Y 279

106 i i i i

I FULL LOOKING AHEADPARTIAL LOOKIN~ AHEADrBACKCHECKINGf.C ; K INGFORWARDCHECKING

BIT PARALLEL FORNARD CH~CKING(bit vector lookups)

, o 'i

0 'o , I,., ,o_Jo 0

IO I I | I I I I | I I I I4 5 6 7 8 9 I0 NUMBER OF QUEENS

F x o . 1 0 . T h e n u m b e r o f t a b l e I o o k u p s u s e d t o f i n d a l l s o l u t i o n s t o t h e N - q u e e n s p r o b l e m f o rv a r y i n g N , w i t h t h e n a t u r a l u n i t o r d e r .


18/51

280 R.M. HARA LICK AND G.L. ELLIOTT

I O 6 ~ , ~ ] , [ ~ ~ i

" 4ow1O 0I . . -

I0

j * BACKCHECKINGBAClq'IARKING~ ~ F U LL O O KIN GHEADPAR TIAL O OKINGHEADFORWARDCHECKING

BIT PARALLEL FORWARDCHECKING-(bit vector lookups)

I L L I I I

N = N U M B E R F U N IT S = U M B E R F L A B E L SFIG. 11. The num ber of table l ookups for the average of 5 rand om relation s at each N, with numbe r ofunits = num ber of labels = N, and p robability of consisten cy check success p = 0.65. The se rand omproblems are the same as shown in Figs. 8 and 9.


19/51

I N C R E A S I N G T R E E S E A R C H E F F I C IE N C Y 281type algorithms are considered. These table lookups occur at line 3 in Fig. 5,backmarking, line 2 in Fig. 3(a), line 4 in Fig. 3(b), and lines 4 and 6 in each ofFigs. 3(c) and 3(d), the lookahead type algorithms. The entering of values intothe tables are not considered, since they always follow at least one consistencycheck, and never happen more often than consistency checks.Backtracking is not shown, since it does no table lookups of the typeconsidered. Partial and full looking ahead always do more table Iookups thanforward checking in these cases, and forward checking does better thanbackmarking in the larger problem sizes. Even full looking ahead does fewertable lookups than backmarking in the larger random problems. The number oftable lookups into bit vectors is smaller than the number of table lookups inother algorithms, when the bit parallel data structure is used in forwardchecking.

Fig. 12 demonstrates that full looking ahead visits the fewest nodes in thetree search, since it eliminates the most potential nodes during its examinationof future unit-label pairs. Fig. 13 indicates that the number of nodes visited inthe tree search is largest for the middle levels in the tree search, with the fulllooking ahead procedure having the fewest nodes at each level.

Figs. 14, 15, and 16 show segments of the trace of nodes visited by the fulland partial looking ahead, and forward checking algorithms for the 6-queensproblem. Backmarking and backchecking will have the same tree trace asbacktracking (see Fig. 2). More detailed trace of the action of backmarking canbe found in [3, 4].

Fig. 17 shows the number of consistent labelings at each depth of the treesearch for the 8-queens problem, and Fig. 18 shows the average number ofconsistent labelings for random problems. This is the number of nodes at eachlevel which have not yet been found to be inconsistent. Backmarking andbackchecking will have the same search tree as backtracking, and consequentlyhas the same number of nodes and consistent labelings at each depth (see Figs.13, 17, and 18). Their efficiencies are gained by reducing the amount of workspent at each node, checking against past units. However, the lookaheadalgorithms perform extra work at each node to reduce the number of nodes,and as Figs. 19 and 20 show, the relation checks and table lookups for theiookahead type algorithms are concentrated more at the shallow depths of thetreesearch. As Figs. 6, 8, 10, and 11 show, full and partial looking ahead do toomuch work at each node for the problems shown, and forward checking andbackmarking do better.

The percentage of nodes at each depth in the tree search that fail becausesome inconsistency is discovered are shown in Figs. 21 and 22. In the casesshown, in the backtracking, backchecking, and backmarking algorithms, over95% of the nodes (instantiated labels) fail at the deepest level of the treesearch,because they are inconsistent with some past unit-label pair.

The lookahead type algorithms reduce the number of nodes in the treesearch


20/51


21/51


22/51

28 4 R.M. HARA LICK AND G.L. ELLIOTI"1 A

2 C2 D

3 B , F2 E , F1 B2 D

3 A3 F

4 A5 C

6 E2 E

3 A , C2 F

FIG. 14. A segment of a tree trace made by the full looking ahead procedure in a 6-queens problem.One consistent labeling, 1B 2D 3F 4A 5C 6E, appears in this portion of the trace. 1A 2C fails to spawnany further nodes because the LOOK _FU TUR E algorithm will, after deleting several potential labels,discover that one future unit has no possible labels.

1 A2 C

3 E , F2 D

3 B , F2 E

3 B2 F

1 B2 D

3 A4 C

3 F4 h ,

5 C6 E2 E

3 A4 F

3 C2 F

FIG. 15. A segment of a tree trace showing the nodes of the tree search in the partial looking aheadprocedure in a 6-queens problem. One consistent labeling, 1B 2D 3F 4A 5C 6E, appears in this portionof the trace. 1A 2C 3A,B,C, and D do not appear because CHECK_F ORW ARD removes them fromthe table at nodes 1A and 2C. However 1A 2C 3E fails to have successors because the only labels leftfor future units 4 and 6 are incompatible and are removed by LOOK... FUTURE (see Fig. l(a)).


23/51

INCREASING TREE SEARCH EFFICIENCY 2851 A

2 C3 E

4 B3 F

4 B2 D

3 B4 E

3 F4 C

2 E3 B

4 F5 C

2 F3 B , D1 B

2 D3 A

4 C5 E

3 F4 A

5 C6 E

5 E4 C

FIG. 16. A segment of a tree trace made b y the forward checking procedure in a 6-queens problem.On e con sistent labeling, 1B 2D 3F 4A 5C 6E, is found . Notice that IA 2C 3E 4B fails because theCH ECK _FOR WA RD procedure discovers hat there are no labels remaining for unit 6 at the 4B node(see Fig. l(b)).

c h e c k i n g a l g o r i t h m w i l l f i n d a l l t h e c o n s i s t e n t l a b e l i n g s a t e a c h l e v e l o f t h e t r e es e a r c h t h a t b a c k t r a c k i n g d o e s , b u t a t a h i g h e r c o s t t h a n t h e o r i g i n a l f o r w a r dc h e c k i n g a l g o r i t h m . T h e r e p l a c e m e n t o f t h e s e li n es i n f o r w a r d c h e c k i n g w illr e a l iz e a 1 5 % s a v i n g o f c o n s i s t e n c y c h e c k s a n d t a b l e I o o k u p s in th e 1 0 - q u e e n sp r o b l e m , a n d o v e r 4 0 % s a v i n g s i n t h e 1 0 u n i t s b y 1 0 l a b e ls r a n d o m p r o b l e m .F i gs . 2 3 a n d 2 4 a d d r e s s t h e q u e s t i o n o f w h a t m e a s u r e b e s t d e t e r m i n e s t h ea l g o r i t h m i c t i m e c o m p l e x i t y . A c a r e f u l c h e c k o f a l l t h e a l g o r i t h m s w i l l s h o wt h a t n o s te p is e x e c u t e d m o r e o f t e n t h a n t h e m a x i m u m o f t h e n u m b e r o fc o n s i st e n c y c h e ck s o r t h e n u m b e r o f t a b le l o o k u p s . A s t h e p r o b l e m s iz ei n c r e a s e s in th e I o o k a h e a d t y p e o f a l g o r i t h m s , t h e r a t i o o f t a b l e l o o k u p s t oc o n s i st e n c y c h e c k s s e e m s t o d e c r e a se f r o m a m a x i m u m o f a b o u t 2 t o n o m o r et h a n 1.5 ta b l e l o o k u p s p e r c o n s i s te n c y c h e ck i n b o t h t h e N - q u e e n s a n d r a n d o mp r o b l e m s . T h i s r a t i o is g u a r a n t e e d t o b e g r e a t e r t h a n o r e q u a l t o o n e , b y t h e


24/51

2 8 6 R . M . H A R A L I C K A N D G . L . E L L I O T T

1 0 3

T

o,~,oz(.9Z...1bda n...]ZUdZ0

Nz

t t * , I l i I v , , I

FULL LOOKINGAHEAD

B A C I q ' ~ R K I N G

I I I I I I2 3 4 5 6 ~ B ; I0 I I 12 13

D E P T H IN T R E E S E A R C H

FIG. 17. The num ber of co nsistent labelings as a fun ction of tree depth, for the 8-queen s problem withthe natural unit order.

a l g o r i t h m s s t r u c t u r e , t h u s e i t h e r m a y b e u s e d a s a m e a s u r e o f a l g o r i t h m i c t i m ec o m p l e x i ty f o r th e l o o k a h e a d t y p e o f a l go r it h m s .

B e c a u s e in b o th t h e N - q u e e n s a n d r an d o m p r o b le m s b a c k m a r k i n g s e e m s toh a v e a s t e a d i l y i n c r e a s i n g r a t i o o f t a b l e l o o k u p s t o c o n s i s t e n c y c h e c k s a sp r o b l e m s i z e g r o w s , o n l y ta b l e l o o k u p s ( w h i c h e q u a l s t h e n u m b e r o f n o d e s i nt h e t r e e s e a r c h i n t h i s c a s e ) c a n b e u s e d a s a t r u e m e a s u r e o f a l g o r i t h m i c t i m ec o m p l e x i t y f o r b a c k m a r k i n g . O n l y i n t h e c a s e t h a t a c o m p u t a t i o n o f a r e l a t i o n


25/51

I N C R E A S I N G T R E E S E A R C H E F F I C I E N C Y 28 7

3 EI - -Q .h i

h -"U 3z. J

I,-,-zL Up .(/ 3Z0

n,,.I . IJa n

Z

1 0 3

i 0 2

I0

i i i i m |

F U L L

C . E C K I ~

P A R T I A L

B A C K T R A C K I N GB A C K H E C K I N GB A C K ? ' t ~ K I N G

I I I I I I I I I I, ~ 3 4 5 6 7 8 ~ io ,J 12 ,3D E P T H I N T R E E S E A R C H

FIG. 18. The average number of nodes visited at each depth of the treesearch for solutions to 5random relations with n um ber of units = num ber of labels = 10, and proba bil ity of consistencychec k success of p = 0.65. T hese are the 5 relations shown at the N = 10 case in Figs. 8, 9, and 11.

c h e c k i s s i g n i f i c a n t ly m o r e e x p e n s i v e t h a n t h e c o s t o f a n o d e ' s l o o p c o n t r o l a n da ta b l e l o o k u p w i l l r e l a t i o n c h e c k s b e a u s e f u l p r a c t i c a l m e a s u r e f o r t h e t i m ec o m p l e x i t y o f b a c k m a r k i n g . T h e r e a s o n t h a t i t i s a p r a c t ic a l m e a s u r e i n t h i s c a s e i st h a t t h e n o d e a n d t a b l e l o o k u p s c o s t w il l d o m i n a t e t h e c o st o f e x e c u t i o n o n l y i nv e r y l a r g e p r o b l e m s i z e s , s o l a r g e t h a t t h e p r o b l e m s c a n n o t b e s o l v e d i n ar e a s o n a b l e t i m e , a n d r e l a t io n t e s ts w i l l d o m i n a t e t h e c o s t i n t h e s m a l l e r p r o b l e m sw h i c h c a n b e s o l v e d i n a p r a c t ic a l a m o u n t o f t i m e .


26/51

288 R.M. HAR ALI CK AND G.L. ELLI OTF

i O 6 , , , , , , , i , , ~ ,

i t )I - -(/ 3,o

> -UZu JI , -c OZ0( , . )

, oZ

IO

F U L L L O O K I NG A H EA Du PA R T I A L L O O K I N G A H EA D FORNARD CHECKING

I

II

A I I I I I i2 3 4 5 6 7

o BACKTRACKING BACKCHECKINGZXBA CI q~R KING

Il I I I I8 D E P T H I N T R E E S E A R C H

FIG. 19. This figure comp ares the num ber of consistency tests made at each level in the tree searchfor six different procedures, for the 8-queens problem, in the natural unit order.


27/51

,o4

o FULL LOO KING AHEAD P A RTIA L L O O K ING

~CKING

o BACKCHECK1NGBACI~i~RKI NG

i O ~

x 'v0oLCJ.-J

l i ! I l I " = I I I ! I I I

INCREASING TREE SEARCH EFFICIENCY 289

I I I I 1 I I I I 1 I I 1I 2 3 4 5 6 7 8 9 I 0 I ID E P T H K I N T R E E S E A R C H

FIG. 20. The number o f t a b l e I o o k u p s a t e a c h p o s s i b l e d e p t h k i n t h e t r e e s e a r c h t o f i n d a l ls o l u t i o n s t o t h e 8 - q u e e n s p r o b l e m , f o r t h e v a r i o u s a l g o r i t h m s .

4 . S t a t i st i c a l M o d e l f o r C o n s t r a in t S a t i s f a c t io n S e a r c h e sO u r s t a t i s t i c a l m o d e l f o r r a n d o m c o n s t r a i n t s a t i s f a c t i o n i s s i m p l e . T h e p r o b -a b i l i t y t h a t a g i v e n c o n s i s t e n c y c h e c k s u c c e e d s i s i n d e p e n d e n t o f t h e p a i r o fu n i t s o r l a b e l s i n v o l v e d a n d i s i n d e p e n d e n t o f w h a t e v e r l a b e l s m a y a l r e a d y


28/51

290 R . M . H A R A L I C K A N D G . L . E L L I O T T

I O 0

9 0> -Z

0o

70

z

6 00h-

~ 5 0...1

.. r

~ 3 08z~ aoh iz 1 0 , iLIJ(,3

B A C K T R A C K I N GB A C K C H E C K IN G~ C I 4 ~ R K I N G

FULLLOOKING AHEAD

P A R T I A L ~L O O K I N G C H E C K I N GL AHEAD

0 ~ I 2 5 4 5 6 7 8 9 I 0D E P T H IN T R E E S E A R C H

F IG . 2 ] . T h i s f i g u r e s h o w s t h e p e r c e n t a g e o f n o d e s a t a g i v e n d e p t h i n t h e t r e e s e a r c h w h i c h f a i lb e c a us e s o m e i n c o n s i st e n c y is d e t e c t e d a t t h a t n o d e . R e s u l t s a r e s h o w n f o r t h e v a r i o u s a l g o r i t h m si n t h e 8 - q u e e n s p r o b l e m . B a c k c h e c k i n g a n d b a c k m a r k i n g o f t e n d i s c o v e r t h a t t h e r e i s a n i n -c o n s i s t e n c y b y u s i n g t a b l e I o o k u p s , r a t h e r t h a n p e r f o r m i n g a ll t h e c h e c k s . N o d e s i n t h e l o o k a h e a dt y p e a l g o r i t h m s f a il b e c a u s e a f u t u r e u n i t f a i ls to h a v e a n y r e m a i n i n g l a b e ls a f t e r i n c o n s i s t e n t f u t u r el a b e l s a r e r e m o v e d .


29/51

I N CREA S I N G TREE S EA RCH EF F I CI EN CY 2 9 1

BACKTRACKINGBACKCHECKINGBACIq"*ARK NG>-UZ 8C

70

6 O0I--LI.Ir . , 5 0. . i

~ 4o"1"I--W 3 08Z~ 2oW

FULL LOOKING AHEAD

PARTIAL F O R V~ C H EC KIN G

I 2 3 4 5 6 7 8

DE P TH I N TRE E S E ARCHFIG. 22. This figure shows the percentage of nodes at a given depth in the tree search which failb e c a u s e s o m e i n c o n s i s t e n c y i s detected at that node. Results are shown for the average over 5random relations, with c o n s i s t e n c y c h e c k p r o b a b i l i t y p = 0.65 and num ber of units = numb er oflabe ls = 10.

h a v e b e e n a s s i g n e d t o p a s t u n i t s . H e n c e ,P( (U k+b l k+b U , l ) ~ R I ! 1 . . . . . I k a r e c o n s i s t e n t l a b e l s o f u l . . . . . U k)

= P ( ( U k + b l k+ l , U , ! ) E R ) f o r e v e r y u , l.T h e N - q u e e n s p r o b l e m i s a m o r e d i ff ic u lt p r o b l e m , w i th fe w e r s o l u t i o n s b u t

r e q u i r i n g m o r e c o n s i s t e n c y t e s t s t h a n t h e c o r r e s p o n d i n g r a n d o m c o n s t r a i n tp r o b l e m w i t h t h e sa m e p r o b a b i l it y o f c o n s i s t e n c y c h e c k s u c c e s s. A c o m p a r i s o n


30/51

292 R.M. HARALICK AND G,L. ELLIOl 'T

, , I1 -O l . 6> - jz I .~i , iI -c o 1.4o 1~ 1 . 3I:E~ . 1 . 20 3_ ~ I . I 1 . 0._1~ .9t nI - . 8

n,, .7u JEl= E . 6::DZ

4 5 d 7 8

BAC KMARK I NG

BIT PARALLEL FORWARDCHECKING( b i t v e c t o r t a b l e l o o k u p s p e rb l t v e c t o r c o n s i s t e n c y c h e c ko p e r a t i o n )P A R T I A L L O O K I N G A H E A DF U L L L O O K I N G A H E A DF O R W A R D C H E C K I N G

9 I 0 N U M B E R O F Q U E E N S

* e BACKCIIECKING

FIG. 23. T h i s f i g u r e s h o w s t h a t i n t h e N - q u e e n s p r o b l e m , a l l t h e a l g o r i t h m s t h a t k e e p t a b l e s ,e x c e p t b a c k m a r k i n g , r e f e r e n c e t h o s e t a b l e s n o m o r e t h a n 3 5% m o r e o f t e n t h a n t h e y r e fe r e n c e t h er e l a t io n . H o w e v e r , b a c k m a r k i n g a p p e a r s t o h a v e a c o n t i n u a l l y g r o w i n g r a ti o o f ta b l e I o o k u p s t or e l a t io n t e s t s , a s p r o b l e m s i z e g r o w s .

o f t h e g r a p h s f o r t h e t w o p r o b l e m s i n S e c t i o n 3 w i l l s h o w t h a t w h i l e t h en ume r ic a l va lue s o f th e qua n t i t i e s va r y c o n s ide r a b ly , th e b a s ic c h a r a c te r o f th ea l g o r i t h m s o p e r a t i o n i s s i m i l a r f o r b o t h p r o b l e m s .

I n o ur a n a ly s i s , w e w i l l a s s ume th a t a g ive n pa ir o f un i t s w i th a g ive n pa ir o fl a b e l s i s c o n s i s t e n t w i t h p r o b a b i l i t y p , p b e i n g i n d e p e n d e n t o f w h i c h u n i t s ,w h i c h l a b e l s , o r a n y p a s t p r o c e s s i n g . I f e a c h u n i t h a s t h e s a m e n u m b e r , M , o fpo s s ib le la b e l s , th e n a n y k - tup le o f la b e l s fo r a n y k un i t s h a s pr o b a b i l i typ k ( k - 1 ) / 2 o f b e i n g c o n s i s t e n t s i n c e e a c h l a b e l i n g m u s t s a t i s f y k(k- 1) con-s i s te n c y c h e c ks . S in c e th e r e a r e M k po s s ib le la b e l in g s o f k un i t s , th e e xpe c te dn u m b e r o f c o n s i s t e n t l a b e l i n g s i s

M k p k ( k - l V 2 .T h e e x p e c t e d n u m b e r o f n o d e s p r o c e s s e d a t l e v e l k i n a s t a n d a r d b a c k t r a c k -

i n g s e a r c h w i l l b e M , t h e n u m b e r o f p o s s i b l e l a b e l s , t i m e s t h e n u m b e r o f


31/51

INCREASING TRE E SEARCH EFFICIENCY 29 3

2 2

2 .0

1.8OM.I-' ro 1.6>-r.)zu.li-o')

z0U

ta 1 . 2

"40 1 .00. Jw

, ~ 0 . au_0wm 0 . 6~Ez

BACKHARK I NG

B I T P A R A L L E L F O R W A R D C t t E C K IN GI . . . . ~ . . . , . ~ ( b i t v e c t o r l o o k u p s p e r b i t v e c t o re n c o i o n s )

/ ~ ' - - e ~ . ~ F U L L L O O K IN G A IIE AD~ F O R W A R D C H E C K I N G

I I I I I I4 5 6 7 8 9 I 0 N = N u m b e r o f u n it s=N u m b e r o f I o b e ls~ B A C K C t l E C K I N G

F I G . 2 4 . T h i s f i g u r e d e m o n s t r a t e s t h a t in t h e a v e r a g e o v e r 5 r a n d o m r e l a t i o n s f o r e a c h p r o b l e m s i zet h a t w a s t e s t e d , t h a t t h e r a t i o o f t a b l e I o o k u p s t o c o n s i s t e n c y c h e c k s s e e m s t o a p p r o a c h o n e , e x c e p ti n t h e b a c k m a r k i n g a lg o r i t h m , in w h i c h t h e r a t io s e e m s t o s t e a d i ly i n c r e a s e w i t h p r o b l e m s i z e . T h er a n d o m r e l a t io n s a r e t h e s a m e a s t h o s e s h o w n i n F i g s . 8 , 9 , a n d 1 1 .

c o n s i s t e n t l a b e l i n g s a t t h e p r e v i o u s l e v e l , M k -l p (k-lXk-2)/2.T h u s t h e r e a r eMkp(k-IXk-2y2

t r e e s e a r c h n o d e s a t l e v e l k .W e c a n a l so c o u n t th e e x p e c t e d n u m b e r o f c o n s i s t e n c y c h e c k s p e r fo r m e d b y

b a c k t r a c k i n g . W e e x p e c t M kp (k-IX*-2)12 l e v el k n o d e s a n d a t e a c h n o d e a l a b e lm u s t h e t e s te d f o r c o n s i s t e n c y w i t h t h e la b e l s g iv e n t h e p r e v i o u s k - 1 u n i ts .T h e f i rs t c o n s i s t e n c y c h e c k f a il s w i t h p r o b a b i l i t y 1 - p . I f it f a il s , w e h a v e s p e n t1 t es t . I f it s u c c e e d s w e h a v e s p e n t 1 t e st a n d a r e c o m m i t t e d t o m a k e a n o t h e ro n e w h i c h m i g h t a l s o s u c c e e d w i t h p r o b a b i l i t y p . A l l ( k - 1 ) t e s ts w i ll s u c c e e d


32/51

294 R . M . H A R A L I C K A N D G . L . E L L I O T ' I "with probability p t k - 1 ) . Hence the expected number of consistency checksperformed at each node is

k - I ip'-l(1 - p ) + (k - 1)p k-L.This may be simplified by recognizing the telescopic nature of the sum which isequal to

k - 2

But this is a geometric sum and is equal to1 _pk-I

1--pTherefore the expected number of consistency checks at level k will beMkp ~k- lk-2~, the number of nodes at level k times ( 1 - - p k - l ) / ( 1 - - p ) , theexpected number of consistency checks at a node, making

Mkptk_lX k_2)/2 1 _ pk - I1 - - pconsistency checks at level k. Of course the expected total number of con-sistency checks will be the summation of the expected number of consistencychecks for each level k for k ranging from 1 to N, the number of units.

The computation of the number of labelings for the forward checkingalgorithm is somewhat more complicated because the algorithm stops checkingwhen a future unit has no labels that are consistent with the past and presentunit-label pairs. A consistent labeling to depth k occurs when the tree searchsuccessfully reaches a given label for unit k and forward checking of thatunit-label pair produces no future unit that has no remaining labels. Thus theconsistent labelings to depth k for forward checking meet exactly the followingconditions:

(1 ) 11111, 11212 . . . . 11klk are consistent unit-label pairs.(2) There is no future unit u in levels k + 1 . . . . N for which there is no label! so that 11, l is consistent with 1111~, .., Uklk.The k unit-label pairs are consistent with probability pktk-~}/2, and there areM k possible labelings to depth k (condition (1)). A future unit-label pair isconsistent with the k past and present unit-label pairs with probability pk andthere are M possible labels for a future unit, so the probability that a futureunit has no label that is consistent with the k past and present units is (1 -pk)M.Since there are N units, there are N - k future units, and the probability that allof these has at least one label that is consistent is [1- (1 __pk ) M]N- k (condition(2)). Thus the expected number of consistent labelings to depth k for forward


33/51

I N C R E A S I N G T R E E S E A R C H E F F I C I E N C Y 295c h e c k i n g i s

Mkpktk - ' ) /2[1 - - (1 -- pk)M ]~-k.T h e e x p r e s s i o n f o r t h e e x p e c t e d n u m b e r o f n o d e s i n t h e f o r w a r d c h e c k i n g

t r e e s e a r c h a t l e v e l k i s v e r y s i m i l a r t o t h a t f o r t h e n u m b e r o f c o n s i s t e n tl a b e li n g s to d e p t h k , s i nc e e a c h n o d e w i ll p e r f o r m f o r w a r d c h e c k s t o d e t e r m i n ei f i ts la b e l w i ll b e c o m e a c o n s i s t e n t l a b e li n g . T h e l a b e ls f o r a n o d e m u s t m e e tc o n d i t i o n ( 1 ) a b o v e , b u t t h e f u t u r e u n i t s a r e r e q u i r e d t o h a v e s u c c e e d e d w i t h a tl e a s t o n e l a b e l o n l y f o r c h e c k s w i t h t h e u n i t - l a b e l p a i r s u111 . . . . Uk-~lk-1, s i n c e e a c hn o d e w a s i n th e t a b l e f o r a c o n s i s t e n t l a b e l in g t o d e p t h k - 1 , a n d i f a n y f u t u r e u n i ta s s e e n f r o m l e v el k - 1 f a i l ed t o h a v e a l a b e l , t h e n i t w o u l d n o t h a v e s p a w n e dn o d e s a t t h e n e x t l e ve l . T h u s i n e a c h n o d e t h e f u t u r e u n i t s w il l h a v e a t l e a st o n el a b e l a n d t h e s e c o n d c o n d i t i o n o c c u r s w i t h p r o b a b i l i t y [1 - (1 - p k - l ) M ] N - k T h u st h e e x p e c t e d n u m b e r o f n o d e s a t d e p t h k i n f o rw a r d c h e c k i n g is

Mkpk(k-1)/2[1 - - (1 - pk-1)M ] N - k .A s li g ht o v e r a p p r o x i m a t i o n f o r t h e e x p e c t e d n u m b e r o f c o n si s te n c y c h e c k s at

d e p t h k i n th e t r e e s e a r c h c a n b e f o u n d b y m u l t ip l y i n g t h e e x p e c t e d n u m b e r o fn o d e s a t t h e d e p t h t im e s t h e e x p e c t e d n u m b e r o f la b e ls r e m a i n i n g f o r e a chf u t u r e u n i t t im e s t h e n u m b e r o f f u t u r e u n i t s , N - k . S i n c e e a c h f u t u r e u n i t w i llh a v e a t l e a s t o n e l a b e l , t h i s e x p e c t e d n u m b e r o f l a b e l s w i l l b e

Mp k- I1 - - ( 1 - - p k - I ) u "

T h u s t h e e x p e c t e d n u m b e r o f c o n s is t e n cy ch e c k s in f o r w a r d c h e c k i n g w ill b eMk+ lp (k+2Xk-~v2[1 (1 - pk-1)M ] N - k - I ( N _ k ) .

T h e e x a c t e x p e c t e d v a l u e c a n b e o b t a i n e d b y r e p l a c i n g t h e n u m b e r o f f u t u r eu n i ts t e r m , ( N - k ) , w i th t h e e x p e c t e d n u m b e r o f f u t u r e u n i ts te s t e d , s in c ef o r w a r d c h e c k i n g w i l l s t o p t e s t in g a s s o o n a s a f u t u r e u n i t i s d i s c o v e r e d t o h a v en o p o s s i b l e l a b e l s . E a c h o f t h e s e t e s t s o f a f u t u r e u n i t w i l l s u c c e e d w i t hp r o b a b i l it y 1 - - ( 1 - - p k ) M, a n d r e a s o n i n g s i m i la r t o t h a t f o r t h e n u m b e r o fc o n s i s t e n c y c h e c k s a t e a c h n o d e i n b a c k t r a c k i n g w i l l g i v e

1 [1 - (1 - -pk)UlN -k(1 _ pk )Mf o r t h e e x p e c t e d n u m b e r o f f u t u r e u n i t s t e s t e d . T h u s t h e e x p e c t e d n u m b e r o fc o n s i s t e n c y c h e c k s a t l e v e l k i n f o r w a r d c h e c k i n g w i ll b e

M k + l p ( k + 2 ) ~ k - l ) / 2 [ 1 - - (1 __pR - I )M] IV -k - I . 1 -- [1 -- (1 _ _ p k ) M l N - k(1 __pk )MT h e n u m b e r o f t a b l e l o o k u p s i n f o r w a r d c h e c k i n g i s t h e s u m o f t h e n u m b e r


34/51

296 R.M. HARALICK AND G.L. ELLIOTTo f co n s i s te n c y c h e c k s a n d t h e n u m b e r o f n o d e s . T h u s t h e e x p e c t e d n u m b e r o ft ab le Iookups a t dep th k in the t r ee sea rch wi l l be

r M p k - IM k p k - ' 2 [ 1 - ( 1 - p k - ' p ' l [ 1 + 1 ( 11 - [1 ~ ( ! ; f ; ) U ] N - k ] .T h e e x p e c t e d n u m b e r o f b i t v e c t o r o p e r a t i o n s w i t h t h e b i t p a r a l l e l d a t a

s t r u c t u r e i n f o r wa r d c h e c k i n g c a n e a s i l y b e f o u n d , b y r e m o v i n g t h e t e r m f o r t h en u m b e r o f l a b e l s r e m a i n i n g f o r e a c h f u t u r e u n i t f r o m t h e e x p r e s s i o n f o r t h en u m b e r o f c o n s i s t e n c y t e s t s , s i n c e o n l y o n e o p e r a t i o n w i l l b e p e r f o r m e d f o reach uni t , g iving

M k p k ( k -l ) 2 [ 1 - - (1 _ _ p k - , ) U ] N - k 1 -- [1 -- (1 _ _ p k ) U ] N - k( 1

f o r t h e e x p e c t e d n u m b e r o f b i t v e c t o r o p e r a t i o n s a t l e v e l k i n b i t p a r a l l e lf o r wa r d c h e c k i n g .T h e n u m b e r o f t a b l e l o o k u p s i n b i t v e c t o r s i s s t i l l t h e su m o f t h e n u m b e r o fb i t v e c t o r o p e r a t i o n s a n d t h e n u m b e r o f n o d e s , fo r

M k p ~ ( k - l ) J 2 [ 1 - - ( 1 - - p k - ' ) U ] N - * [ 1 + 1 - - [1 ~ ( ! ;P)~)U]N-k ]t ab le lookups in to b i t vec to r s i n b i t pa ra l l e l fo rward check ing .

T o i l l u s t r a t e t h e g e n e r a l f o r m o f t h e e x p r e s s i o n s we c o m p u t e d f o r t h ee x p e c t e d n u m b e r o f c o n si s te n c y c h e c k s a n d e x p e c t e d n u m b e r o f s o lu t io n s , w ep r e se n t a fe w g r a p hs . F i g . 25 il lu s t ra t e s t h e g r a p h o f th e e x p e c t e d n u m b e r o fc o n s i s t e n c y c h e c k s a s a f u n c t i o n o f t r e e d e p t h f o r a r a n d o m c o n s t r a i n t s a ti s -fac t ion p ro b le m hav ing N = 17 un i t s and l abe l s and a p robab i l i t y p = 0 .70 o f ac o n s t r a i n t b e i n g s a t i s f i e d . N o t i c e t h a t t h e g r e a t e r n u m b e r o f t e s t s f o r wa r dc h e c k i n g d o e s e a r l y i n th e t r e e s e a r c h p a y s o ff l a t e r i n t h e t r e e s e a r c h b o t h i nt e r m s o f n u m b e r o f c o n s i s te n c y t e s ts a n d i n n u m b e r o f su c ce s s f u l i n s t an t i a ti o n sa t each t r ee dep th (F ig . 26) .

F ig . 27 il lu s t ra t es t h e e x p e c t e d n u m b e r o f so lu t i o n s a s a f u n c t i o n o f N a n d pp a r a m e t e r s o f a r a n d o m c o n s t r a i n t s a t i s f a c t i o n p r o b l e m . I n c r e a s i n g N f o r af ix e d p e v e n t u a l l y c a u se s f e we r so l u ti o n s t o e x is t b e c a u se t h e n u m b e r o fcons t ra in t s i s i nc reas ing quadra t i ca l ly .

W e , o f c o u r se , e x p e c t t h e n u m b e r o f c o n s i s t e n c y t e s t s t o i n c r e a se a s Ni n c r e a se s a n d p r e m a i n s f i x e d s i n c e t h e s e a r c h sp a c e i s b e c o m i n g l a r g e v e r yr a p i d l y . T h i s i s sh o wn f o r t h e f o r wa r d c h e c k i n g p r o c e d u r e i n F i g . 2 8 . A l soe x p e c t e d i s f o r t h e a v e r a g e n u m b e r o f c o n s i s t e n c y c h e c k s p e r l a b e l i n g t oinc rease a s N inc reases and p rema ins f ixed (Fig. 29). As N inc reases , t heprob lem of f ind ing the f i r s t so lu t ion a s we l l a s a l l so lu t ions i s becoming moreand more d i f f i cu l t . The re fo re , i t i s no t expec ted fo r t he number o f cons i s t ency


35/51

I N C R E A S I N G T R E E S E A R C H E F F I C I E N C Y 297

107

10 6

1 0 ~Ia

v )~ 1 0 ~I, -) -

wp._1/3g 103t jIJ.otY[,[Jgza 1 0 2i , itl Jnx

I ! I I ! |

B A C K T R A C K I N G

1 0 i J a I I2 4 6 8 10 12 16 18D E P T H I N T R E E S E A R C H

C

I I I I ,~"~,l t ,

F I G . 2 5 . T h e n u m b e r o f c o n s i s t e n c y t e st s a s a f u n c t i o n o f tr e e d e p t h f o r a n N = 1 7 , p = 0 . 7 0r a n d o m c o n s t r a i n t s a t i s f a c t i o n p r o b l e m .

t e s t s p e r s o l u t i o n t o d e c r e a s e a s t h e n u m b e r o f s o l u t i o n s i n c r e a s e s . T h e r e a s o nf o r t h i s i s t h a t a s t h e n u m b e r o f s o l u t i o n s i n c r e a s e s m o r e o f t h e t e s t s r e q u i r e dt o ve r if y a s o l u t i o n b e c o m e s h a r ed b e c a u s e t h e s o l u t i o n s h a v e c o m m o ns e g m e n t s . T h i s i s i l l u s t r a t e d i n F i g . 3 0 .

A n e x p e r i m e n t a l c h e c k o f t h e t h e o r e ti c a l e q u a t i o n s f o r t h e n u m b e r o f


36/51

2 9 8 R .M . H A R A L I C K A N D G . L . E L L IO T I "106_

1 0 5

ZE ~o4t t l

Z

~ 10 3

z

~ 10 2u

bL0

I [ 1 0ZQWuaXw 1

I I I I I I ! ,

BACKT RACKIN5

FORWARDCHECKING

i i i i i i i i2 4 6 8 10 12 14 16 1~

D EPTH I N TR EE SEA R C HFIG. 26. The num ber of con sistent labelings as a fun ction of tree d epth for an N = 17, p = 0.70random const raint sat isfact ion problem .s o l u t i o n s a n d n u m b e r o f s o l u t i o n s a t a g i v e n d e p t h i n t h e t r e e s e a r c h w i t hr a n d o m r e l a t i o n s is g i v e n in F ig s . 3 1 a n d 3 2. A l t h o u g h t h e a v e r a g e n u m b e r o fs o l u t i o n s i s c l o s e to t h e t h e o r e t i c a l r e s u l t , t h e i n d i v i d u a l r e l a t i o n s v a r y w i d e l y .T h e I b a r s m a r k a d i s t a n c e o f o n e s t a n d a r d d e v i a t i o n o f t h e m e a n a b o v e a n db e l o w t h e a v e r a g e o f t h e t r i a ls .

F i g . 3 3 d e m o n s t r a t e s t h e a c c u r a c y o f t h e t h e o r e t i c a l e x p r e s s i o n f o r t h e


37/51


38/51

300 R.M. HARALICK AND G.L. ELLIOT]"

1 0 6

1 0 5

~ I 0 4

~ 1031 0IL lO .XIJJ

1 0

I I , ! I ! ! , l -

~ --.60_ _ ~ = . 5 5 F O R W A R D H E C K IN G

I 1 I | | I I I | I4 5 6 7 8 9 1 0 11 1 2 1 3 1 4 1 5N

FIG. 28. This figure shows how the num ber of consistency tests increases as N increases, and p isheld constant.

expected number of consistency tests with random relations in the forwardchecking algorithm. The total expected number of consistency tests shown iscalculated from the sum of the expected number of tests for each level in thevarious problem sizes. For this expression to be correct, the expression for theexpected number of nodes at each level in the forward checking algorithm mustalso be correct.


39/51

I N C R E A S I N G T R E E S E A R C H E F F I C I EN C YL i 1 i

1 0 6 F O R W A R D C H E C K I N G

| I

P = . 6 6

301

10 =-P - - , 6 0 P = . 6 3

z.JILlm

~ 1 0 4LtJ0 ..Y jLL IT

~ z l O 3LIJ

ZO(..)LLO, Y 1 0 2h i(1::E:DZaI.iJI- - I1013_xw

P=.55

P = . 5 7

!

4 5 6 7 B 9 10 11 12 13 14 15N

FIG. 29 . Thi s f i g u r e s h o w s h o w t h e n u m b e r o f c o n s i s t e n c y t e s t s p e r s o l u t i o n i n c r e a s e s a s Ni n c r e a s e s a n d p i s h e l d c o n s t a n t , i n f o r w a r d c h e c k i n g .

5 . T h e F a i l F i r s t P r i n c i p l eO n e o f t h e s t r a t e g i e s w h i c h h e l p s t r e e s e a r c h i n g f o r c o n s t r a i n t s a t is f a c t io np r o b l e m s i s t h e f a i l f i r s t o r p r u n e e a r l y s t r a t e g y o f t h e l o o k i n g a h e a d a n df o r w a r d c h e c k i n g p r o c e d u r e s . T h e r e a r e o t h e r w a y s t h a t w e c a n a p p l y t h eg e n e r a l pr i n c i p l e o f t r y i n g t o f a i l f i r s t ( a n d o f c o u r s e r e m e m b e r t h a t f a c t s o t h a tt h e r e a r e n o u n n e c e s s a r i l y r e p e a t e d m i s t a k e s ) . I n t h i s s e c t i o n w e d i s c u s s t w o


40/51

302 R.M. HARALICK AND G.L. ELLIOTT

Q z l O.J~Jnn. Jfit:uJn(/ )L)h i'1"(Jbzh i~ 2Z0u,.0w

zawI--0.x I0

N=]I

N- 9N=8

N=

II 0 I 0 0

E X P E C T E D N U M B E R O F L A B E L I N G S

FIG. 30. T h i s f ig u r e i ll u s t ra t e s t h a t as t h e e x p e c t e d n u m b e r o f s o l u t i o n s i n c r e a s e s , t h e e x p e c t e dnumber o f c o n s i s t e n c y c h e c k s p e r s o l u t i o n d e c r e a s e s , f o r v a r y i n g p r o b a b i l i t y p a n d f i x e d p r o b l e msize N = num ber o f u n i t s = n u m b e r o f l a b e l s , i n th e f o r w a r d c h e c k i n g a l g o r i t h m .

o t h e r a p p l i c a t i o n s o f t h i s s t r a t e g y . T h e f i r s t i s b y o p t i m i z i n g t h e o r d e r i n w h i c hw e d o c o n s i s t e n c y t e s t s . T h e f a il fi rs t p r i n c i p l e s t a t e s t h a t w e s h o u l d f i rs t t r yt h o s e t e s t s i n t h e g i v e n s e t o f t e s t s t h a t a r e m o s t l i k e l y t o f a i l s i n c e i f t h e y d of a il w e d o n o t h a v e t o d o t h e r e m a i n d e r o f t h e t e s ts i n t h e se t .

T h e s e c o n d a p p l i c a t i o n i s i n d y n a m i c a l l y c h o o s i n g t h e o p t i m a l o r d e r i n w h i c ht o p r o c e s s u n i t s i n e a c h b r a n c h o f t h e t r e e s e a r c h . O p t i m a l u n i t o r d e r c h o o s i n g ,e v e n o n a l o c a l b a si s, w il l n o t o n l y l o w e r t h e n u m b e r o f e x p e c t e d c o n s i s t e n c yt e s t s p e r p r o b l e m a s c o m p a r e d w i t h a r a n d o m o r d e r i n g , b u t i t a l s o l o w e r s t h ev a r i a n c e o f t h i s a v e r a g e . F o r t h e u n i t o r d e r c h o i c e , t h e f a i l f i r s t p r i n c i p l e s t a t e s


41/51


O ),.9Z

m._1i -zu.iI. -O 30 3

014 .o

Z

IOC9 0 i J8 0 '7 06 0

5 0

4 0

3 0

2 0

I0

I ' ' I I I I

I I I I I 1 14 5 6 7 8 9 I0N = N U M B E R O F U N IT S= NU M BE R O F L A B E L S

FIG. 31. The expected number of consistent labelings for random relations with number ofunits = number of labels = N, and consistency check success probability p = 0.65. Dots representthe average number of consistent labelings for 25 random relations tested at each problem size, andthe I bars indicate one standard deviation of the mean above and below the experimental mean.

that the next unit to choose should be that o ne with the fewest possible labelsleft.5 . 1 . O p t i m i z i n g t h e c o n s i s t e n c y c h e c k o r d e r i n t r e e s e a r c h i n gSuppose we are solving a constrai nt satisfaction prob lem and suppose units1 . . . . K have alread y been assigned labels !1 . . . . lr and we are trying to find alabel lr+l for unit K + 1. The label lr+~ must come from some set Sr+~ of labelsand i t must be consis tent with each of the previous labels l~ . . . . i t , that is, wemust have (k, lk, K + 1,/rl) ~ R for k = 1 . . . . K. To det erm ine the label /r+x,


42/51

304 R.M. HARA LICK AND G.L. ELLI OTr

3 0 0

10 09 08 070

II.~ 3 0_ JU _ou.Im 2cIz

I 1

1 0 ~1 2 3 4 5 6 7 8 9 10T = DEPTH IN TREE SEARCH

FIG. 32 . The number of cons is tent l abe l ings to depth k in the t ree sea rch for the average of 25random cons t ra in t sa t i s fac t ion problems wi th probabi l i ty of cons is tency check success of 0 .65 andnum be r o f un i ts = numbe r o f l a bel s = I0 . The d o t t e d c u rve i s t he t he o re t i c a l e xpe c t e d num be r o flabe l ings for such a problem.w e s e q u e n t i a l l y g o t h r o u g h a l l t h e l a b e l s i n S r + l a n d p e r f o r m t h e K c o n s i s t e n c yc h e c k s : ( k , lk , K + 1 , i t l ) E R . I f o n e c h e c k f a il s , t h e n w e t r y t h e n e x t l a b e l i nSr,+~. I f a l l c h e c k s s u c c e e d , t h e n w e c a n c o n t i n u e t h e d e p t h f ir s t s e a r c h w i t h t h en e x t u n i t .


43/51


44/51

306 R.M. HARALICKAND G.L. ELLIOTTchecks are independent events so that the probability of the tests succeeding onunits 1 through K is Ilkr=lP(k).

For each order of testing, these probabilities determine the expected numberof tests in the following way. Let k~ . . . . kr be a permutat ion of 1 .. . ,Kdesignating the order in which the consistency checks will be performed. Thetest (k l , l k l , K + 1, ix+ l) E R will succeed with probability P ( k l ) and fail withprobability 1 - P(k~) . If it fails, we incur a cost of one consistency check and wetry the next label. If it succeeds, we will have incurred a cost of one consistencycheck and we are committed to try the next test ( k 2 , l k 2 , K + 1, ix+ x)E R . Thistest succeeds with probability P(k2) and fails with probability 1 -P (k2) . At thispoint, we have incurred a cost of two tests and may be committed to makemore tests if this one succeeded.Fig. 34 shows the tree of K + 1 possible outcomes. Since the tests areassumed independent, the probability for each outcome can be computed bymultiplying probabilities. For example, the probability of failing on consistencycheck with unit k3 is P(k~)P(k2)(1 -P(k3)) . Also associated with each outcomeis the number of tests performed to get there. For example, failure on the testwith unit k3 incurs a cost of 3 tests.

The expected number of tests C performed is computed byK i - I KC = ~ i [1 - P(k,)] j_~P(k j )+ K i~ : P(k , ) .

Upon rearranging and simplifying this expression we obtainK - I i

Now by the proposition at the end of Section 5, this is minimized by havingk l . . . . . k x be any permutation of 1,...,K satisfying P ( k l ) 6 P ( k 2 ) ~ ' " < -P(kr). Hence, to minimize expected numbers of tests, we must choose theorder so that the tests with units most likely to fail are done first.To illustrate the advantage of using optimum consistency test order, weconsider the 10-queens problem when the units are naturally ordered from 1 toN and the current unit is K, then the fail first principle states tha t tests withpast units must be done in the order of decreasing constraints. Since the rowprevious to row k has the strongest constraint on row k, the test order shouldbe first unit K - l , then K - 2 , up to unit 1, in the N-queens problem.Backtracking requires 1,297,488 tests when done in the wrong order (unit1, 2 . . . . . K - 1) and 1,091,856 tests when done in the right order. It is interes-ting to note that Gaschnig's backjumping procedure [3] when done with theconsistency tests in the wrong order (1,131,942 tests) performs worse thanstandard backtracking with consistency tests in the right order. Furthermore,for the N-queens problem, backjumping with consistency tests in the rightorder for the N-queens problem is equivalent to standard backtracking with


45/51


P(k I )Succeed

l - P(kl)Fail

l test

P(k 2 )

Succeed

P(k 3) i a i l2 tests1 - P(k3)Succeed "FailtestsSucceed

P ( k )

Succeed

FIG. 34. The K + 1 outcomes of K tests.

, ) FaiK tests

consistency tests in the right order because backjumping backtracks to thehighest level at which a failure is detected, and there is always at least one labelat a given level which fails when checked with the immediately preceding level.5 . 2 . O p t i m i z i n g t r ee s e a r c h o r d e rEvery tree search must assume some order for the units to be searched in. Theorder may be uniform throughout the tree or may vary from branch to branch.


46/51

3 0 8 R . M . H A R A L I C K A N D G . L . E L L I O T / "I t i s c l e a r f r o m e x p e r i m e n t a l r e s u l t s t h a t c h a n g i n g t h e s e a r c h o r d e r c a ni n f l u e n c e th e a v e r a g e e f f ic i e n c y o f t h e s e a r c h . I n t h i s s e c ti o n w e a d o p t t h ee f fi c ie n c y c r i te r i o n o f b r a n c h d e p t h a n d w e s h o w h o w b y a l w a y s c h o o s in g t h en e x t u n i t h a v i n g s m a l l e s t n u m b e r o f l a b e l c h o i c e s w e c a n m i n i m i z e t h ee x p e c t e d b r a n c h d e p t h .

S u p p o s e u n i t s 1 . . . . N a r e u n i t s w h i c h a r e y e t t o b e a s s ig n e d l a b e l s . L e tn ( m ) b e t h e n u m b e r o f p o s si b le o r a v a i la b l e l a b el s f o r u n it m . W e a s s u m e t h a te a c h o f t h e n ( m ) l a b e ls p o s s ib l e f o r u n i t m h a s t h e s a m e p r o b a b i l i t y q o fs u c c e e d i n g a n d t h a t s u c c e s s o r f a i l u r e o f o n e o f t h e l a b e l s i s a n i n d e p e n d e n te v e n t f r o m s u c c e s s o r f a i l u r e f o r a n y o f t h e o t h e r l a b e l s . T h u s , t h e p r o b a b i l i t yt h a t a u n i t m w i ll n o t h a v e a n y l a b e l t h a t s u c c e e d s is (1 - q ) ~e ,) . T h e p r o b a b i l i t yt h a t s o m e l a b e l f o r u n i t m s u c c e e d s i s , t h e r e f o r e , P ( m ) = 1 - ( 1 _ q)~


47/51

INCREASING TRE E SEARCH EFFICIENCY 309B y t h e p r o p o s i t i o n a t t h e e n d o f S e c t i o n 5 , t h i s i s m i n i m i z e d w h e n t h e u n i t

c h o s e n a t e a c h l e v e l i s t h a t u n i t w h o s e s u c c e s s p r o b a b i l i t y i s s m a l l e s t . T h u s a tl e v e l j w e c h o o s e u n i t k j, w h e r e

Pj (k j )


48/51

3 1 0 R .M . H A R A L I C K A N D G .L . E L L I O T TI I i ! i I i I i I

~ B A C K M A R K I N GF O R W A R D C H E C K I N G

B A C K M A R K I N G O P T I M A LORDER

F O R W A R D C H E C K I N GO P T I M A L O R D E R

U}V~WI-,->-UZW

103Z00L L0~ J

Z

10

I l | I I I I I I I5 6 7 8 9 10 11 12 13 14

N = N U M B E R O F U N I T S = N U M B E R O F L A B E L SF IG . 35 . T h e i m p r o v e m e n t i n e ff ic i en c y f o r t h e a v e r a g e o f 5 r a n d o m r e l a t io n p r o b l e m s w i t hp r o b a b i l i ty p = 0 . 6 5 a n d n u m b e r o f u n i t s = n u m b e r o f l a b e l s = N , w h e n l o c a l l y o p t i m i z e d u n i t o r d e ri s u s e d . T h e r e l a t i o n s a r e t h e s a m e a s t h o s e s h o w n i n F i g s. 9 a n d 1 0 .

/b y l o o k i n g a t t h e p r o d u c t s . T h e r e a r e t h r e e c a s e

paper adriano

Documents