anytime and/or best-first search for optimization in ...dechter/publications/r201.pdf · anytime...

Anytime AND/OR Best-First Search for Optimization in Graphical

Models

Natalia Flerova [email protected]

University of California Irvine

Radu Marinescu [email protected]

IBM Research, Dublin, Ireland

Rina Dechter [email protected]

University of California Irvine

Abstract

Depth-first search schemes areknown to be more cost-effective forsolving graphical models tasks thanBest-First Search schemes. In thispaper we show however that any-time Best-First algorithms recentlydeveloped for path-finding prob-lems, can fare well when appliedto graphical models. Specifically,we augment best-first schemesdesigned for graphical modelswith such anytime capabilities anddemonstrate their potential whencompared against one of the mostcompetitive depth-first branch andbound scheme. Though Best-Firstsearch using weighted heuristics issuccessfully used in many domains,the crucial question of weightparameter choice has not beensystematically studied and presentsan interesting machine learningproblem.

1. Introduction

In recent years we have seen a variety extensionsof best-first search algorithms into flexible anytimescheme making these algorithms more practicalapproximate algorithms. The most popular algo-rithms are based on Weighted A* (WA*) (Pohl,

Preliminary work. Under review by the International Con-ference on Machine Learning (ICML). Do not distribute.

1970). The idea is to inflate the heuristic values bya constant factor of w ≥ 1, making the heuristicinadmissible to a degree controlled by w, but stillguaranteeing a solution cost within a factor of wfrom the optimal one while typically yielding fastersearch. If the approximate solution is found quicklyand additional time is available, the search for abetter solution with smaller weight resumes. Severalanytime weighted best-first search schemes were in-vestigated in the past decade (Hansen and Zhou,2007; Likhachev, Gordon, and Thrun, 2003;van den Berg et al., 2011; Richter, Thayer, and Ruml,2010) and shown to be quite effective.

All these developments were conducted primarily inthe context of path-finding problems (e.g., plan-ning problems) for which best-first strategies arethe method of choice. In contrast, for combi-natorial optimization over graphical models, suchas MAP/MPE or WCSP, depth-first branch andbound are the preferred schemes. These algo-rithms were extensively studied for finding both exactand approximate solutions (Kask and Dechter, 2001;Marinescu and Dechter, 2009b; Otten and Dechter,2011; de Givry et al., 2005). Best-first search al-gorithms, though known to be more time efficient(Dechter and Pearl, 1985), are rarely considered dueto their inherently large memory requirements andlack of anytime behavior. Moreover, since in graphicalmodels all solutions have the same length, the mainbenefit of best-first search of avoiding the explorationof unbounded paths, becomes irrelevant.

Some recent studies (Marinescu and Dechter, 2009b)showed that if given sufficient memory, best-first canbe superior to depth-first search, while otherwise, itwill often fail. Therefore, the recent extensions of Best-

Anytime Best-First Search

first search into anytime methods raise the hope thatthe algorithm’s qualities can be brought to bear, lead-ing to this investigation in their potential as anytimesearch schemes for graphical models as well.

Contributions. We extended several variantsof weighted A* into graphical models by apply-ing the ideas to the AOBF class of algorithms(Marinescu and Dechter, 2009b). The simplest ex-tension we consider, denoted wAOBF, runs AOBFwith the weighted heuristic iteratively, decreasingthe weight at each iteration, until either w =1 or until a time bound is reached. Addition-ally we extend Anytime Repairing A* (ARA*)(Likhachev, Gordon, and Thrun, 2003) and AnytimeAO* (Bonet and Geffner, 2012), yielding wR-AOBFand p-AOBF respectively. We investigated empir-ically the effectiveness of each of the schemes andits guiding parameters as an anytime scheme andcompared the emerging schemes against Breadth-Rotating AND/OR Branch-and-Bound (BRAOBB)(Otten and Dechter, 2011), one of the best anytimeschemes for graphical models.

Our results show that weighted best-first schemes canprovide a useful addition to the class of anytime com-binatorial optimization scheme for graphical models.The scheme was superior to BRAOBB on various in-stances, although BRAOBB was more cost-effectiveon the majority of the instances. In particular, theweighted scheme performed well when the heuristicevaluation function is of weak or medium strength.Our study is only at its initial phase however. Toget more conclusive results we plan to conduct fur-ther investigation into weight policies and aim to iden-tify problem’s features and parameters of the algo-rithms that favor anytime best-first search over any-time depth-first search for graphical models.

For weighted search algorithms, including wAOBFand wR-AOBF, the weight parameter controls thetrade off between the solution accuracy and the timeand space requirements. The questions of how tochoose the starting weight and how to decrease it overtime have yet to receive a thorough study and suchchoices are usually made manually in ad hoc manner.At this point we experimented with several human-constructed weighting policies. However, it is desir-able to learn the starting weight and weight policy thatcould yield the best anytime behaviour based on theproblem parameters. Currently such automatic weightpolicy choice is a work in progress.

2. Anytime Heuristic SearchAlgorithms

In this section, we describe the anytime algorithmswhich we explore later in our empirical evaluation sec-tion. The main focus of the experiments is on the twoanytime Best First schemes: wAOBF and wR-AOBF.

Iterative Weighted AOBF (wAOBF): Since theaccuracy of a solution found by Weighted AOBF isbounded by w, it is possible to formulate a simple any-time scheme, called wAOBF, that executes WeightedAOBF iteratively, starting with an initial weight, anddecreasing the weight at each iteration according tosome predetermined rule. Clearly this approach, sim-ilar to the Restarting Weighted A* by Richter at al.(Richter, Thayer, and Ruml, 2010), results in a seriesof solutions, each with a sub-optimality factor equal tow. We experimented with multiple weight decreasingschedules, discussion of which we defer till the nextsection.

Anytime Repairing AOBF (wR-AOBF): Run-ning each search iteration from scratch, as wAOBFdoes, is wasteful, since the same search sub-space might be explored multiple times. To rem-edy this problem, we propose Anytime RepairingAOBF scheme, wR-AOBF, which is based on anidea of Anytime Repairing A* (ARA*) algorithm(Likhachev, Gordon, and Thrun, 2003). At each it-eration, wR-AOBF keeps track of the partially ex-plored AND/OR graph and, after decreasing w, it per-forms a bottom-up update of all node values startingfrom the leaf nodes (whose h-values are inflated withthe new weight) and continuing upwards towards theroot node. Then, the search continues with the newlyidentified best partial solution tree. Like ARA*, wR-AOBF provides the same guarantees with respect tothe quality of the suboptimal solutions found.

We compare and contrast the performance of our any-time BF algorithms with the following two schemes:

Anytime Stochastic AOBF (p-AOBF): Anotheranytime scheme for AND/OR spaces is the AnytimeAO* algorithm (Bonet and Geffner, 2012) introducedrecently for solving finite-horizon MDPs. The idea isto allow AO* to also expand nodes that otherwise maynot be on any optimal solution. Specifically, at eachstep the algorithm expands a tip node that does notbelong to the current best partial solution graph witha fixed probability (1 − p). The algorithm does notprovide any theoretical guarantees on the quality ofthe intermediate solutions. The extension of the algo-rithm to the context minimal AND/OR search graphfor graphical models is straightforward and is called


p-AOBF. As we show in experimental section, foroptimization problems over graphical models p-AOBFrarely demostrates truly anytime behavior.

Anytime AND/OR Branch and Bound(BRAOBB): Depth first AND/OR Branch-and-Bound (AOBB) (Marinescu and Dechter, 2009a)is a powerful search scheme for graphical models. Thealgorithm, however, lacks a proper anytime behaviorbecause at each AND node all but one independentchild subproblems have to be solved completely,before the last one is even considered. Breadth-Rotating AND/OR Branch-and-Bound (BRAOBB)(Otten and Dechter, 2011) remedies this deficiencyof AOBB by combining depth-first exploration ofthe search space with the notion of rotating throughdifferent subproblems in a breadth-first manner.Empirically, BRAOBB outputs the first subopti-mal solutions significantly faster than plain AOBB(Otten and Dechter, 2011).

3. Experiments

We compare the anytime behavior of all the algo-rithms described above when exploring the same con-text minimal AND/OR graph for each problem in-stance. The search space is determined by a commonvariable ordering. All the algorithms are guided bythe mini-bucket heuristics (Dechter and Rish, 2003),whose strength is controlled by a parameter called i-bound (higher i-bounds typically yield more accurateheuristics). The algorithms output solutions at differ-ent times until the optimal solution is found.

We considered 3 benchmarks: 16 pedigree networks, 17binary grids, 6 SPOT5 Weighted CSPs and 4 driverlogproblems. The task solved was Most Probable Expla-nation (max-prod), so higher costs are better. Eachinstance was solved for 11 different values of i-bound,ranging from 2 to 22.

For wAOBF and wR-AOBF we decided on a large ini-tial weight equal to 64 due to our desire to obtainthe initial solutions quickly and solve hard instances,many of which are infeasible for the algorithms usingnon-weighted heuristic.

Our empirical evaluation consists of two sets of ex-periments, results of which we report separately: 1)the comparison between wAOBF and wR-AOBF withthe simplest weighting policies and the competingBRAOBB and p-AOBF algorithms, 2) the explorationof more sophisticated weighting policies.

3.1. Anytime weighted BFS vs Depth-First BB

At the first stage of the evaluation the anytime BFschemes used the two most straightforward weight-ing policies: 1) substract(0.1) - at each iteration theweight is decreased by a fixed amount 0.1; 2) divide(2)- at each iteration the weight is divided by 2. Prelim-inary experiments, omitted for lack of space, showedthat for wR-AOBF the first policy provides better re-sults, while for wAOBF the second policy was prefer-able.

We compare these two resulting schemes (denoted wR-AOBF and wAOBF-H respectively) with BRAOBBand p-AOBF (p=0.5), reporting the results in twoways: 1) by showing the performance on a set of repre-sentative problems that demonstrate the trends com-mon to the majority of instances; 2) by summarizingstatistics over all the instances in each benchmark.

The time limit for this set of experiments was 1 hour,memory limit was 4 Gb.

3.1.1. Results on a selected set of instances

Figures 1 and 2 show the anytime profiles of wAOBF-H, wR-AOBF, BRAOBB and p-AOBF on select in-stances from all our benchmarks. For each instancethe results for two i-bounds are displayed side by side:the left column corresponds to a weaker heuristic andthe right one to a stronger one.

Comparing the anytime schemes on these representa-tive problems we immediately see that p-AOBF is notcompetitive. On the grid and pedigree instances iteither directly outputs the exact solution (e.g. pedi-gree38, i=12) or a solution close to optimal (e.g. 50-18-5, i=10) after a considerable time, contrary to thedesirable anytime behaviour, namely quickly obtainingthe initial solution and improving it over time. More-over, p-AOBF fails to produce any solution within thetime limit for instances with weak heuristic (e.g. 50-18-5, i=2) or for any of the SPOT5 or driverlogproblems (e.g. 505). As clearly unpromising algorithmp-AOBF is excluded from the subsequent discussion.

The performance of wAOBF, wR-AOBF andBRAOBB greatly depends on the benchmark and theheuristic strength. The following common trends canbe distinguished:

Grids: wAOBF and wR-AOBF are often superior toBRAOBB when the heuristic is weak, i.e. the i-boundis considerably lower than the problem’s induced width(e.g. 50-18-5, i=2). However, when the heuristic isstronger, BRAOBB is the fastest to find the optimalsolution (e.g. 50-19-5, i=20). For easy problems solved


0 500 1000 1500 2000 2500 3000 3500time (sec)

−40

−35

−30

−25

log(

prob

abili

ty)

50-18-5 (n=324 k=3 w=24 h=84), i=2

wAOBF-H(w!=2.0)wR-AOBF(w!=1.9)BRAOBBoptimal (C*)

0 500 1000 1500 2000 2500 3000 3500time (sec)

−29−28−27−26−25−24−23−22

log(

prob

abili

ty)

50-18-5 (n=324 k=3 w=24 h=84), i=10

p-AOBFwAOBF-H(w!=1.0)wR-AOBF(w!=1.0)BRAOBBoptimal (C*)

0 100 200 300 400 500 600 700time (sec)

−25.0

−24.5

−24.0

−23.5

log(

prob

abili

ty)

50-19-5 (n=361 k=3 w=25 h=93), i=16


0 50 100 150 200 250time (sec)

−24.4

−24.2

−24.0

−23.8

−23.6

−23.4

−23.2

log(

prob

abili

ty)

50-19-5 (n=361 k=3 w=25 h=93), i=20


0 500 1000 1500 2000time (sec)

−135

−130

−125

−120

−115

log(

prob

abili

ty)

pedigree7 (n=1068 k=4 w=28 h=140), i=8


0 500 1000 1500 2000time (sec)

−126

−124

−122

−120

−118

−116

−114

log(

prob

abili

ty)

pedigree7 (n=1068 k=4 w=28 h=140), i=16


0 100 200 300 400time (sec)

−98

−96

−94

−92

−90

−88

log(

prob

abili

ty)

pedigree38 (n=724 k=4 w=16 h=52), i=6


0 5 10 15 20 25time (sec)

−100

−95

−90

log(

prob

abili

ty)

pedigree38 (n=724 k=4 w=16 h=52), i=12


Figure 1. Grids and pedigrees, cost as a function of time (wAOBF-H, wR-AOBF, BRAOBB, p-AOBF). Starting weight= 64. C∗ - optimal cost, i - i-bound, n - number of variables, k - domain size, w - induced width, h - pseudo-tree height,w! - weight at termination. Time limit - 1 hour, memory limit - 4 Gb.

optimally in under 30 seconds, which are not presentedhere due to being of little interest, BRAOBB is almostalways superior for all i-bounds.

Comparing the two BF schemes we see that wAOBF-H is inferior to wR-AOBF on most grid instances, dueto both inefficient re-exploring of a large portion of the


0 500 1000 1500 2000 2500 3000time (sec)

−825

−820

−815

−810

−805

−800

−795

log(

prob

abili

ty)

412 (n=300 k=3 w=35 h=105), i=6


0 50 100 150time (sec)

−820

−815

−810

−805

−800

−795

log(

prob

abili

ty)

412 (n=300 k=3 w=35 h=105), i=10


0 10 20 30 40 50 60 70 80time (sec)

−660

−640

−620

−600

log(

prob

abili

ty)

505 (n=240 k=3 w=22 h=71), i=2

wAOBF-H(w!=2.0)wR-AOBF(w!=1.3)BRAOBB

0.0 0.5 1.0 1.5time (sec)

−630

−620

−610

−600

−590

log(

prob

abili

ty)

505 (n=240 k=3 w=22 h=71), i=6wAOBF-H(w!=2.0)wR-AOBF(w!=1.2)BRAOBB

0 500 1000 1500 2000 2500 3000 3500time (sec)

−16700

−16650

−16600

−16550

−16500

log(

prob

abili

ty)

driverlog02ac (n=301 k=2 w=45 h=132), i=2


0.0 0.2 0.4 0.6 0.8 1.0time (sec)

−16650

−16600

−16550

−16500

log(

prob

abili

ty)

driverlog02ac (n=301 k=2 w=45 h=132), i=4wAOBF-H(w!=2.0)wR-AOBF(w!=1.9)BRAOBB

0 100 200 300 400 500 600 700 800time (sec)

−25780−25760−25740−25720−25700−25680−25660−25640

log(

prob

abili

ty)

driverlog05ac (n=351 k=2 w=66 h=202), i=2


0 50 100 150 200 250 300 350time (sec)

−80

−70

−60

−50

−40

−30

−20

−10

log(

prob

abili

ty)

−2.566e4driverlog05ac (n=351 k=2 w=66 h=202), i=4


Figure 2. SPOT5 and driverlog WCSPs, cost as a function of time (wAOBF-H, wR-AOBF, BRAOBB, p-AOBF). Startingweight = 64, C∗ - optimal cost, i - i-bound, n - number of variables, k - domain size, w - induced width, h - pseudo-treeheight, w! - weight at termination. Time limit - 1 hour, memory limit - 4 Gb.

search space and fast decreasing of the weight, whichleads to fast increase in memory requirements. How-ever, there are some exceptions, such as 50-18-5, i=2,

where wAOBF-H achieves better results.

Pedigrees: these instances have many similarities to


0 900 1800 2700time (sec)

0.002.865.718.5711.4314.2917.1420.0022.8625.7128.57

% best solutions

Grids: medium heuristicwAOBF-HwR-AOBFBRAOBB

0 900 1800 2700time (sec)

0.003.136.259.3812.5015.6318.7521.8825.0028.1331.25

% best solutions

Pedigrees: medium heuristicwAOBF-HwR-AOBFBRAOBB

0 900 1800 2700time (sec)

0.006.6713.3320.0026.6733.3340.0046.6753.3360.0066.67

% best solutions

SPOT5 WCSP: medium heuristicwAOBF-HwR-AOBFBRAOBB

0 900 1800 2700time (sec)

0481216202428323640

% best solutions

Driverlogs: weak heuristicwAOBF-HwR-AOBFBRAOBB

Figure 3. % of instances, for which algorithm finds bestsolution at a given time for a fixed heuristic strength.

the grids in terms of induced width, domain size andamount of determinism, so the behaviour of the al-gorithms on these two benchmarks are quite similar.Note that on largest problems, such as pedigree7 andpedigree38 shown here, BRAOBB is often inferior toboth BF schemes even for heuristics of considerablestrength (e.g. pedigree38, i=12), while wAOBF-H al-most always performs worse than wR-AOBF.

SPOT5 WCSPs: these problems are more challeng-ing for all our schemes. On some of the problemsBRAOBB reaches the optimal solution quickly, butfails to prove it’s optimality withing the time limit(e.g. 412, i=6), on others it outputs inferior bounds(e.g. 505, i=6). Both BF schemes tend to run outof memory before finding solutions of good accuracy(e.g. 412, i=10). For the hardest instances, such as505, none of the algorithms managed to produce exactsolution.

Drivelogs: the hardest of our benchmarks presentdifficulties for all 3 algorithms. The cost of the opti-mal solutions for these instances are unknown, thoughthe behaviour of BRAOBB on certain instances (e.g.driverlog05ac, i=2) suggests that it did find the exactsolutions without proving their optimality within allo-cated time. Unlike the previous benchmarks, wAOBF-H is better than wR-AOBF, consistently outputtingmore accurate solutions. Its success on these memory-intensive instances can be attributed to more modestmemory requirements, since wR-AOBF needs to keeptrack of the partially explored AND/OR graph duringeach iteration.

3.1.2. Aggregated results

Figure 3 displays the fraction of instances, for whicha particular algorithm found the best solution at aspecific time point. The p-AOBF is discarded fromconsideration as obviously inferior. For space reasonswe only display results for medium heuristic strength,except for driverlogs, for which only weak heuristic isfeasible. We do not count the ties between algorithmsand thus the results not necessarily sum to 100% ateach time point.

For grids and driverlogs BRAOBB is the best schemein about 30% and 40% of instances respectively for allthe time points considered. For pedigrees wR-AOBFfinds the most accurate solution for about 12% of in-stances and even wAOBF-H is superior in around 6%of cases for a particular time point. On SPOT5 WC-SPs wR-AOBF demonstrates clear superiority, obtain-ing the best solutions in up to 2/3 of instances.


0 200 400 600 800 1000 1200time (sec)

−9.0−8.5−8.0−7.5−7.0−6.5−6.0−5.5

log(

prob

abili

ty)

wAOBF: 90-20-5 (n=400 k=3 w=27 h=99), i=10

substract(0.1) (w!=64)divide(2) (w!=4)inverse (w!=1.18519)piecewise(1.05,8) (w!=1.16026)sqrt(1.0) (w!=1.13879)optimal (C*)

102030405060weight

−11

−10

−9

−8

−7

−6

log(

prob

abili

ty)

wAOBF: 90-20-5 (n=400 k=3 w=27 h=99), i=10substract(0.1) (w!=64)divide(2) (w!=4)inverse (w!=1.18519)piecewise(1.05,8) (w!=1.16026)sqrt(1.0) (w!=1.13879)optimal (C*)1.5xC*2xC*

0 500 1000 1500 2000 2500 3000time (sec)

−145

−140

−135

−130

log(

prob

abili

ty)

wAOBF: pedigree31 (n=1183 k=5 w=29 h=115), i=18

substract(0.1) (w!=61.1)divide(2) (w!=2)inverse (w!=2.28571)piecewise(1.05,8) (w!=2.08366)sqrt(1.0) (w!=1.13879)optimal (C*)

102030405060weight

−260

−240

−220

−200

−180

−160

−140

−120

log(

prob

abili

ty)

wAOBF: pedigree31 (n=1183 k=5 w=29 h=115), i=18

substract(0.1) (w!=61.1)divide(2) (w!=2)inverse (w!=2.28571)piecewise(1.05,8) (w!=2.08366)sqrt(1.0) (w!=1.13879)optimal (C*)1.5xC*2xC*

0 500 1000 1500time (sec)

−2.70

−2.68

−2.66

−2.64

−2.62

−2.60

log(

prob

abili

ty)

wAOBF: 505 (n=240 k=3 w=22 h=71), i=10

substract(0.1) (w!=64)divide(2) (w!=2)inverse (w!=1.93939)piecewise(1.05,8) (w!=1.98444)sqrt(1.0) (w!=64)

102030405060weight

−2.5

−2.0

−1.5

−1.0

−0.5

0.0

log(

prob

abili

ty)

wAOBF: 505 (n=240 k=3 w=22 h=71), i=10

substract(0.1) (w!=64)divide(2) (w!=2)inverse (w!=1.93939)piecewise(1.05,8) (w!=1.98444)sqrt(1.0) (w!=64)estimate (C')

Figure 4. wAOBF: solution cost vs time (left) and weight (right), starting weight = 64. C∗ - optimal cost, ”-i” - i-bound,n - number of variables, k - domain size, w - induced width, h - pseudo-tree height, w! - weight at termination, C′ = w! ·C- optimal solution estimation.

3.2. The influence of the weight on anytimeBF schemes.

Aiming to further improve the performance of anytimeBF schemes and to gain a better understanding of theimpact of weight on their performance we devised moresophisticated weight policies than those discussed inthe previous subsection. We experimented with 5 dif-ferent strategies for decreasing the weight, includingthe substract and divide strategies mentioned above.

The notation is as follows: wi is the weight used at theith iteration, w1 = 64, k and d are real-valued policyparameters. We tried the following policies, each forseveral values of parameters:

• substract(k): wi = wi−1 − k

• divide(k): wi = wi−1/k

• inverse: wi = w1/i

• piecewise(k, d): if wi ≥ d then wi = w1/i elsewi = wi−1/k

• sqrt(k): wi =√wi−1/k

The substract and divide policies lay on the oppo-site ends of the strategies spectrum. The first methodchanges the weight very gradually and consistently,leading to a slow improvement of the solution. Thesecond approach yields less smooth anytime behaviour,


0 100 200 300 400 500 600 700time (sec)

−9.0

−8.5

−8.0

−7.5

−7.0

−6.5

−6.0

−5.5

log(

prob

abili

ty)

wR-AOBF: 90-20-5 (n=400 k=3 w=27 h=99), i=10

substract(0.1) (w!=1.2)divide(2) (w!=2)inverse (w!=1)piecewise(1.05,8) (w!=1)sqrt(1.0) (w!=1.13879)optimal (C*)

102030405060weight

−11

−10

−9

−8

−7

−6

log(

prob

abili

ty)

wR-AOBF: 90-20-5 (n=400 k=3 w=27 h=99), i=10

substract(0.1) (w!=1.2)divide(2) (w!=2)inverse (w!=1)piecewise(1.05,8) (w!=1)sqrt(1.0) (w!=1.13879)optimal (C*)1.5xC*2xC*

0 500 1000 1500 2000 2500time (sec)

−145

−140

−135

−130

log(

prob

abili

ty)

wR-AOBF: pedigree31 (n=1183 k=5 w=29 h=115), i=18

substract(0.1) (w!=1.1)divide(2) (w!=2)inverse (w!=1.10345)piecewise(1.05,8) (w!=1.10501)sqrt(1.0) (w!=1.06714)optimal (C*)

102030405060weight

−260

−240

−220

−200

−180

−160

−140

−120

log(

prob

abili

ty)

wR-AOBF: pedigree31 (n=1183 k=5 w=29 h=115), i=18

substract(0.1) (w!=1.1)divide(2) (w!=2)inverse (w!=1.10345)piecewise(1.05,8) (w!=1.10501)sqrt(1.0) (w!=1.06714)optimal (C*)1.5xC*2xC*

0 100 200 300 400 500 600 700 800time (sec)

−2.70

−2.68

−2.66

−2.64

−2.62

−2.60

log(

prob

abili

ty)

wR-AOBF: 505 (n=240 k=3 w=22 h=71), i=10

substract(0.1) (w!=1.9)divide(2) (w!=2)inverse (w!=1.93939)piecewise(1.05,8) (w!=1.98444)sqrt(1.0) (w!=64)

102030405060weight

−2.6−2.4−2.2−2.0−1.8−1.6−1.4−1.2

log(

prob

abili

ty)

wR-AOBF: 505 (n=240 k=3 w=22 h=71), i=10

substract(0.1) (w!=1.9)divide(2) (w!=2)inverse (w!=1.93939)piecewise(1.05,8) (w!=1.98444)sqrt(1.0) (w!=64)estimate (C')

Figure 5. wR-AOBF: solution cost vs time (left) and weight (right), starting weight = 64. C∗ - optimal cost, ”-i” -

i-bound, n - number of variables, k - domain size, w - induced width, h - pseudo-tree height, w! - weight at termination,C

′ = w! · C - optimal solution estimation. Time limit - 1 hour, memory limit - 2 Gb.

since the weight rapidly approaches 1 and much fewerintermediate solutions are found. This could poten-tially allow the schemes to produce the exact solutionfast, but on hard instances presents a danger of leap-ing directly to a prohibitively small weight and thusfailing, without exploring feasible weights that couldpotentially produce good solutions. The other poli-cies were constructed manually based on a followingintuition: we wanted to improve the solution rapidlyby decreasing the weight fast initially and then ”fine-tune” the solution as much as the memory limit allows,by decreasing the weight slowly as it approaches 1.

The performance of our anytime BF schemes em-ploying these weight policies on the representative in-

stances for each benchmark are shown in Figures 4(wAOBF) and 5 (wR-AOBF). The left column dis-plays the solution cost as a function of time, the rightone - as a function of the weight. This set of exper-iments was conducted on a weaker machine that thefirst one, thus the memory limit was 2 Gb, with timelimit of 1 hour. Several values of numerical parame-ters for each policies were tried, the ones that yieldedthe best performance are included here. The startingweight equals 64, w! denotes the weight at the time ofalgorithm termination.

Consider the left columns of Figures 4 and 5. The su-perior policy is benchmark dependent. For most valuesof i-bound both wR-AOBF and wAOBF algorithms


achieve the best results for hard grids and pedigreesusing the sqrt policy, as shown here on the example ofinstances 90-20-5 and pedigree31 respectively. How-ever, for the SPOT5 WCSPs (e.g. 505) the resultsare more varied. For higher values of i-bounds (e.g.i=10) wAOBF achieves the best results using dividepolicy, while for wR-AOBF piecewise is superior withinverse coming close second. Weaker heuristics yieldproblems that are too large for either of algorithm tosolve for any weight smaller than initial values of 64and thus are not discussed.

The right columns of Figures 4 and 5 demonstratethat, unsurprisingly, there is little variance betweendifferent policies in terms of the dependency of the so-lution cost on the weight. For the grid and pedigreeinstances we also show the theoretical bounds, corre-sponding to weights 1.5 and 2 (i.e. 1.5 ·C∗ and 2 ·C∗).In practice the solutions found by both wAOBF andwR-AOBF are better than suggested by theory. How-ever, when the termination weights w! are close to 1(e.g. for grid 50-20-5, i=10, sqrt(1.0) w! = 1.13879),it allows us to obtain reasonably accurate theoreticalbounds, especially useful if the optimal solution is notavailable, as is the case for problem 505. For this in-stance we plot an estimation C′ equal to the best avail-able bound w! · C. The true optimal solution wouldlay between the lines corresponding to the algorithms’outputs and C′.

Overall, it is clear that the new weighting policies, suchas sqrt allow to achieve better results than the simplersubstract and divide ones that we considered initially.A thorough empirical evaluation of the superior poli-cies using larger memory limit is currently a work inprogress.

3.3. Summary of the experimental results

The main conclusions that can be drawn from our ex-periments are:

1. Both wAOBF and wR-AOBF are inferior toBRAOBB in cases of: a) easy instances b) harder in-stances with strong heuristics c) small memory limit;The anytime BF schemes are competitive or superiorlarge problems and weak to medium heuristics. 2. Theperformance of wAOBF and wR-AOBF is significantlysensitive to the weight policies.

4. Conclusion

In this paper we conducted an extensive empiricalevaluation of several anytime best-first schemes forgraphical models, and showed that they have poten-tial for providing an alternative to anytime branch-

and-bound search for some benchmarks. In particu-lar, we found that even our simplest scheme wAOBF,that iteratively solves the problem from scratch, grad-ually decreasing the weight, can be reasonably effec-tive for problems with high induced width, such assome SPOT5 WCSPs, while the more sophisticatedwR-AOBF proved successful for many problems ofmoderate difficulty, such as large grids and pedigrees.We predict further improvement of the performance iflarger amount of memory is available.

The advantage of the weighted anytime schemes is thatthey output the weighting parameter which providesan upper-bound guarantee. This bound can be infor-mative when the anytime scheme is employing smallweights and optimal solution is unknown.

We plan to further improve the performance of ouranytime best-first schemes by using sophisticatedweight policies and by investigating the connection be-tween the weight, problem’s induced width and the al-gorithm’s performance in order to automatically learnthe most effective weighting schedule instead of choos-ing parameters ad-hoc. We are also exploring possiblenon-parametric approaches.

Acknowledgement

This work was supported by NSF grant IIS-1065618.

References

Bonet, B., and Geffner, H. 2012. Action selection forMDPs: Anytime AO* vs. UCT. In AAAI.

de Givry, S.; Heras, F.; Zytnicki, M.; and Larrosa,J. 2005. Existential arc consistency: Getting closerto full arc consistency in weighted csps. In IJCAI,84–89.

Dechter, R., and Pearl, J. 1985. Generalized best-firstsearch strategies and the optimality of A*. Journalof the ACM 32:506–536.

Dechter, R., and Rish, I. 2003. Mini-buckets: A gen-eral scheme for bounded inference. Journal of theACM 50(2):107–153.

Hansen, E., and Zhou, R. 2007. Anytime heuristicsearch. Journal of Artificial Intelligence Research28(1):267–297.

Kask, K., and Dechter, R. 2001. A general schemefor automatic search heuristics from specification de-pendencies. Artificial Intelligence 129(1–2):91–131.

Likhachev, M.; Gordon, G.; and Thrun, S. 2003.


ARA*: Anytime A* with provable bounds on sub-optimality. NIPS 16.

Marinescu, R., and Dechter, R. 2009a. AND/ORBranch-and-Bound search for combinatorial opti-mization in graphical models. Artificial Intelligence173(16-17):1457–1491.

Marinescu, R., and Dechter, R. 2009b. Memory inten-sive AND/OR search for combinatorial optimizationin graphical models. Artificial Intelligence 173(16-17):1492–1524.

Otten, L., and Dechter, R. 2011. Anytime AND/ORdepth first search for combinatorial optimization. InSOCS.

Pohl, I. 1970. Heuristic search viewed as path findingin a graph. Artif. Intell. 1(3-4):193–204.

Richter, S.; Thayer, J.; and Ruml, W. 2010. The joyof forgetting: Faster anytime search via restarting.In ICAPS, 137–144.

van den Berg, J.; Shah, R.; Huang, A.; and Goldberg,K. 2011. Anytime nonparametric A*. In AAAI.

anytime and/or best-first search for optimization in ...dechter/publications/r201.pdf · anytime...

Documents