efficient simulation budget allocation for selecting …...informs journal on computing vol. 20, no....

This article was downloaded by: [128.112.66.66] On: 05 February 2016, At: 19:07Publisher: Institute for Operations Research and the Management Sciences (INFORMS)INFORMS is located in Maryland, USA

INFORMS Journal on Computing

Publication details, including instructions for authors and subscription information:http://pubsonline.informs.org

Efficient Simulation Budget Allocation for Selecting anOptimal SubsetChun-Hung Chen, Donghai He, Michael Fu, Loo Hay Lee,

To cite this article:Chun-Hung Chen, Donghai He, Michael Fu, Loo Hay Lee, (2008) Efficient Simulation Budget Allocation for Selecting anOptimal Subset. INFORMS Journal on Computing 20(4):579-595. http://dx.doi.org/10.1287/ijoc.1080.0268

Full terms and conditions of use: http://pubsonline.informs.org/page/terms-and-conditions

This article may be used only for the purposes of research, teaching, and/or private study. Commercial useor systematic downloading (by robots or other automatic processes) is prohibited without explicit Publisherapproval, unless otherwise noted. For more information, contact [email protected].

The Publisher does not warrant or guarantee the article’s accuracy, completeness, merchantability, fitnessfor a particular purpose, or non-infringement. Descriptions of, or references to, products or publications, orinclusion of an advertisement in this article, neither constitutes nor implies a guarantee, endorsement, orsupport of claims made of that product, publication, or service.

Copyright © 2008, INFORMS

Please scroll down for article—it is on subsequent pages

INFORMS is the largest professional society in the world for professionals in the fields of operations research, managementscience, and analytics.For more information on INFORMS, its publications, membership, or meetings visit http://www.informs.org

http://pubsonline.informs.org

http://dx.doi.org/10.1287/ijoc.1080.0268

http://pubsonline.informs.org/page/terms-and-conditions

http://www.informs.org

INFORMS Journal on ComputingVol. 20, No. 4, Fall 2008, pp. 579–595issn 1091-9856 �eissn 1526-5528 �08 �2004 �0579

informs ®

doi 10.1287/ijoc.1080.0268© 2008 INFORMS

Efficient Simulation Budget Allocation forSelecting an Optimal Subset

Chun-Hung Chen, Donghai HeDepartment of Systems Engineering and Operations Research, George Mason University, Fairfax, Virginia 22030

{[email protected], [email protected]}

Michael FuRobert H. Smith School of Business and Institute for Systems Research, University of Maryland,

College Park, Maryland 20742, [email protected]

Loo Hay LeeDepartment of Industrial and Systems Engineering, The National University of Singapore,

Kent Ridge, 119260, Singapore, [email protected]

We consider a class of the subset selection problem in ranking and selection. The objective is to identify thetop m out of k designs based on simulated output. Traditional procedures are conservative and inefficient.

Using the optimal computing budget allocation framework, we formulate the problem as that of maximizingthe probability of correctly selecting all of the top-m designs subject to a constraint on the total number ofsamples available. For an approximation of this correct selection probability, we derive an asymptotically optimalallocation and propose an easy-to-implement heuristic sequential allocation procedure. Numerical experimentsindicate that the resulting allocations are superior to other methods in the literature that we tested, and therelative efficiency increases for larger problems. In addition, preliminary numerical results indicate that theproposed new procedure has the potential to enhance computational efficiency for simulation optimization.

Key words : simulation optimization; computing budget allocation; ranking and selectionHistory : Accepted by Marvin Nakayama, Area Editor for Simulation; received August 2006; revised

August 2007, October 2007, and November 2007; accepted December 2007. Published online in Articles inAdvance May 30, 2008.

1. IntroductionWe consider the problem of selecting the top m outof k designs, where the performance of each designis estimated with noise (uncertainty). The primarycontext is simulation, where the goal is to determinethe best allocation of simulation replications amongthe various designs to maximize the probability ofselecting all top-m designs. This problem setting fallsunder the well-established branch of statistics knownas ranking and selection or multiple comparison pro-cedures (see Bechhofer et al. 1995). In the contextof simulation, Goldsman and Nelson (1998) providean overview of this field; see also Andradóttir et al.(2005) and Swisher et al. (2003).

Most of the ranking-and-selection research hasfocused on identifying the best design. Typical ofthese are two-stage or sequential procedures that ulti-mately return a single choice as the estimated opti-mum, e.g., Dudewicz and Dalal (1975) and Rinott(1978). Even the traditional “subset selection” proce-dures aim at identifying a subset that contains thebest design, dating back to Gupta (1965), who pre-sented a single-stage procedure for producing a sub-

set (of random size) containing the best design with aspecified probability. Extensions of this work relevantto the simulation setting include Sullivan and Wilson(1989), who derive a two-stage subset selection pro-cedure that determines a subset of maximum size mthat, with a specified probability, contains at least onedesign whose mean response is within a prespeci-fied distance from the optimal mean response. Thisindifference zone procedure approach also results ina subset of random size, and the designs are assumedto follow a normal distribution, with independencebetween designs assumed and unknown and unequalmoments. The primary motivation for such proce-dures is screening, whereby the selected subset can bescrutinized further to find the single optimum.

To reiterate, instead of selecting the very best de-sign from a given set or finding a subset that is highlylikely to contain the best design, the objective in thispaper is to find all top-m designs. The only substan-tive work we are aware of addressing this problemis Koenig and Law (1985), who along the lines ofthe procedure in Dudewicz and Dalal (1975), developa two-stage procedure for selecting all the m best

579

Dow

nloa

ded

from

info

rms.

org

by [

128.

112.

66.6

6] o

n 05

Feb

ruar

y 20

16, a

t 19:

07 .

For

pers

onal

use

onl

y, a

ll ri

ghts

res

erve

d.

Chen et al.: Efficient Simulation Budget Allocation for Selecting an Optimal Subset580 INFORMS Journal on Computing 20(4), pp. 579–595, © 2008 INFORMS

designs (see also §10.4 of Law and Kelton 2000 foran extensive presentation of the problem and proce-dure). The number of additional simulation replica-tions for the second stage is computed based on aleast favorable configuration, resulting in very conser-vative allocations, so that the required computationalcost is much higher than actually needed.

In this paper, we develop an efficient approachfor such a class of ranking-and-selection problems.Unlike traditional frequentist approaches constrainedwith least favorable configuration, our procedure isdeveloped using a Bayesian model. The rationalefor the adoption of the Bayesian model is the easeof derivation of the solution approach. Conservativeleast-favorable configuration is no longer requiredand so the efficiency can be enhanced. Further com-parison of the Bayesian model with the frequentistmodel can be found in Inoue and Chick (1998) andInoue et al. (1999).

To improve efficiency for ranking and selection,several approaches have been explored for problemsof selecting a single best design. Intuitively, to ensurea high probability of correct selection, a larger por-tion of the computing budget should be allocatedto those designs that are critical in the process ofidentifying the best design. A key consequence isthe use of both the means and variances in the allo-cation procedures, rather than just the variances, asin Dudewicz and Dalal (1975) and Rinott (1978).Among examples of such approaches, the optimalcomputing budget allocation (OCBA) approach byChen et al. (1997, 2000) maximizes a simple heuris-tic approximation of the correct selection probabil-ity; extensions of the OCBA approach include Leeet al. (2004), who consider multiple objective func-tions; Trailovic and Pao (2004), who consider theobjective of minimizing variance; and Fu et al. (2007),who consider correlated sampling. The approach byChick and Inoue (2001a, b) estimates the correct selec-tion probability with Bayesian posterior distributions,and allocates further samples using decision-theorytools to maximize the expected value of informa-tion in those samples. The procedure by Kim andNelson (2006) allocates samples to provide a guaran-teed lower bound for the frequentist probability ofcorrect selection integrated with ideas of early screen-ing. More recently, Branke et al. (2007) provide anice overview and extensive comparison for some ofthe aforementioned selection procedures. These pro-cedures are developed to remedy the drawbacks ofinefficiency for traditional two-stage procedures byallocating simulation samples in a more efficient man-ner. However, all of this work has focused on selectingthe single best, whereas no such research results existfor efficiently selecting the top-m designs since theKoenig and Law (1985) paper appeared. This paper

aims to fill this gap by providing an efficient alloca-tion procedure for selecting the m best designs.

Development of such an efficient procedure forselecting the m best designs is also beneficial tosome recent developments in global optimizationthat, when applied to the simulation setting, requirethe selection of an “elite” subset of good candidatesolutions in each iteration of the algorithm. Exam-ples of these include the cross-entropy method (CE;see Rubinstein and Kroese 2004), the population-based incremental learning method (PBIL; see Rudlofand Köppen 1996), the model reference adaptivesearch method (Hu et al. 2007a, b), genetic algo-rithms (Holland 1975, Chambers 1995), and moregenerally, evolutionary population-based algorithmsthat require the selection of an “elite” population inthe evolutionary process (see Fu et al. 2006). Insteadof trying to find a subset that contains the singlebest among a currently generated set of candidatesolutions, the objective is to find an optimal subsetsuch that all members are among the best perform-ers in that candidate set. The reason for this require-ment is that this entire subset is used to update thesubsequent population or sampling distribution thatdrives the search for additional candidates. A sub-set with poor performing solutions will result in anupdate that leads the search in a possibly mislead-ing direction. The overall efficiency of these types ofsimulation optimization algorithms depends on howefficiently we simulate the candidates and correctlyselect the top-m designs. The algorithm developedherein is generic enough so that it can be integratedwith any such evolutionary population-based searchmethods. Note that among the selected m designs,there is no further ranking done within the set.Again, this is consistent with the requirements of theCE and PBIL methods, as well as other evolution-ary population-based methods that require an “elite”population of some type.

The contribution of this paper is threefold. From aranking-and-selection perspective, we offer a heuris-tic for selecting all top-m designs out of k, whereour empirical studies suggest that it can be moreefficient than existing methods. From the computingbudget allocation perspective, our heuristic illustrateshow the previous OCBA method for identifying a sin-gle best design can be modified to instead select anoptimal subset. From a simulation optimization per-spective, we illustrate one possible way of efficientlyallocating simulation replications for those evolution-ary population-based search methods that require anelite set to guide the search; however, further researchis required to fully realize the benefits of such integra-tion, which is clearly highly dependent on the searchmethod adopted. This paper is organized as follows.

Dow

nloa

ded

from

info

rms.

org

by [

128.

112.

66.6

6] o

n 05

Feb

ruar

y 20

16, a

t 19:

07 .

For

pers

onal

use

onl

y, a

ll ri

ghts

res

erve

d.

Chen et al.: Efficient Simulation Budget Allocation for Selecting an Optimal SubsetINFORMS Journal on Computing 20(4), pp. 579–595, © 2008 INFORMS 581

In the next section, we formulate the optimal com-puting budget allocation problem for selecting thetop-m designs. Section 3 derives an allocation basedon approximating the correct selection probability andan asymptotic analysis and, based on the analysis,proposes a heuristic sequential allocation scheme. Theperformance of the resulting allocations is illustratedwith a series of numerical examples in §4, includ-ing numerical results that indicate that the proposednew procedure has the potential to enhance computa-tional efficiency for simulation optimization. Section 5concludes the paper.

2. Problem StatementWe introduce the following notation:

T = total number of simulation replications(budget),

k= total number of designs,m= number of top designs to be selected in the

optimal subset,Sm = set of m (distinct) indices indicating designs

in selected subset,Ni = number of simulation replications allocated to

design i,Xij = jth simulation replication for design i,J̄i = 1/Ni�

∑Nij=1Xij , sample mean for design i,

Ji = mean for design i, 2i = variance for design i,

�x�= 1/√

2��e−x2/2, standard normal probabilitydensity function,

�x�= ∫ x

−� �t� dt, standard normal cumulative dis-tribution function.

The objective is to find a simulation budget allo-cation that maximizes the probability of selectingthe optimal subset, defined as the set of m (<k) bestdesigns, for m a fixed number. Note that rank orderwithin the subset is not part of the objective. In thispaper, we will take Sm to be the m designs withthe smallest sample means. Let J̄ir be the rth smallest(order statistic) of {J̄1� J̄2� � � � � J̄k}, i.e., J̄i1 ≤ J̄i2 ≤ · · · ≤ J̄ik .Then, the selected subset is given by

Sm ≡ �i1� i2� � � � � im��

Without loss of generality, we will take the m bestdesigns as those designs with the m smallest means,so that in terms of our notation, the correct selection(CS) event is defined by Sm containing all of the msmallest mean designs:

CSm ≡{⋂i∈Sm

⋂j�Sm

Ji ≤ Jj �

}={maxi∈Sm

Ji ≤mini�Sm

Ji

}� (1)

The OCBA problem is given by

maxN1��Nk

P�CSm�

s.t. N1 +N2 + · · ·+Nk = T �(2)

Here, N1 + N2 + · · · + Nk denotes the total computa-tional cost assuming the simulation execution timesfor different designs are roughly the same. This for-mulation implicitly assumes that the computationalcost of each replication is constant across designs. Thesimulation budget allocation problems given in Chenet al. (2000) are actually special cases of (2) with m= 1.For notational simplification, we will drop the “m” inP {CSm} in the remaining discussion.

We assume that the simulation output samples {Xij }are normally distributed and independent from repli-cation to replication (with mean Ji and variance 2

i ),as well as independent across designs. The normalityassumption is typically satisfied in simulation becausethe output is obtained from an average performanceor batch means, so that central limit theorem effectsusually hold.

3. Approximate AsymptoticallyOptimal Allocation Scheme

Our approach is developed based on Bayesian set-ting. We estimate P {CS} using the Bayesian modelpresented in Chen (1996) and He et al. (2007). Themean of the simulation output for each design, Ji,is assumed unknown and treated as a random vari-able. After the simulation is performed, a posteriordistribution for the unknown mean Ji, pJi � Xij� j =1� � � � �Ni), is constructed based on two pieces of infor-mation: (i) prior knowledge of the system’s perfor-mance, and (ii) current simulation output. Thus, in theBayesian framework, the probability of correct selec-tion defined by (1) is given by

P�CS�= P�J̃i ≤ J̃j for all i ∈ Sm and j � Sm�� (3)

where J̃i, i = 1� � � � � k, denotes the random variablewhose probability distribution is the posterior distri-bution of design i. As in Chen (1996), we assumethat the unknown mean Ji has a conjugate normalprior distribution and consider noninformative priordistributions, which implies that no prior knowledgeis available about the performance of any designbefore conducting the simulations, in which case theposterior distribution of Ji is (see DeGroot 1970)

J̃i ∼N

(J̄i�

2i

Ni

)�

After the simulation is performed, J̄i can be calcu-lated, 2

i can be approximated by the sample vari-ance, and the P {CS} given by Equation (3) can thenbe estimated using Monte Carlo simulation. However,because estimating P {CS} via Monte Carlo simula-tion is time-consuming and the purpose of budgetallocation is to improve simulation efficiency, we

Dow

nloa

ded

from

info

rms.

org

by [

128.

112.

66.6

6] o

n 05

Feb

ruar

y 20

16, a

t 19:

07 .

For

pers

onal

use

onl

y, a

ll ri

ghts

res

erve

d.


adopt an approximation of P {CS} using a lowerbound.

3.1. Approximating the Probability ofCorrect Selection

For a constant c,

P�CS� = P�J̃i ≤ J̃j for all i ∈ Sm and j � Sm�

≥ P�J̃i ≤ c and J̃j ≥ c for all i ∈ Sm and j � Sm�

= ∏i∈Sm

P�J̃i ≤ c�∏i�Sm

P�J̃i ≥ c�≡APCSm� (4)

where the last line is due to independence across de-signs. We refer to this lower bound for P {CS}, whichcan be computed easily and eliminates the need forextra Monte Carlo simulation, as the approximate prob-ability of correct selection for m best (APCSm). The valuefor c will be between J̄im and J̄im+1

, and the rationalefor this will be explained in §3.3. Using the approxi-mation given by Equation (4), the OCBA problem (2)becomes

maxN1��Nk

∏i∈Sm

P�J̃i ≤ c�∏i�Sm

P�J̃i ≥ c�

s.t. N1 +N2 + · · ·+Nk = T �

(5)

Now we solve OCBA problem (5), assuming the vari-ables {Ni} are continuous.

3.2. Asymptotically Optimal SolutionFor notation simplification, we define the variable�i = J̄i − c, i= 1�2� � � � � k.

For i ∈ Sm,

PJ̃i ≤ c� =∫ 0

−�1√

2� i/√Ni�

e−x−�i�2/2 2

i /Ni� dx

=∫ �

�i/ i/√Ni�

1√2�

e−t2/2 dt =�

( −�i i/

√Ni

)�

and for i � Sm,

PJ̃i ≥ c� =∫ �

0

1√2� i/

√Ni�

e−x−�i�2/2 2

i /Ni� dx

= �

(�i

i/√Ni

)�

Now let F be the Lagrangian relaxation of (5), withLagrange multiplier :

F = ∏i∈Sm

P�J̃i≤c�· ∏i�Sm

P�J̃i≥c�−

( k∑i=1

Ni−T

)

= ∏i∈Sm

�

( −�i i/

√Ni

)· ∏i�Sm

�

(�i

i/√Ni

)−

( k∑i=1

Ni−T

)�

Furthermore, the Karush-Kuhn-Tucker (KKT) (Walker1999) conditions of this problem can be stated asfollows:

For i ∈ Sm,

!F

!Ni

= ∏j∈Smj �=i

P �J̃j ≤ c� · ∏j�Sm

P�J̃j ≥ c�

· −12�

(�i

i/√Ni

)�i iN−1/2i − = 0� (6)

For i � Sm,

!F

!Ni

= ∏j∈Sm

P�J̃j ≤ c� · ∏j�Smj �=i

P �J̃j ≥ c�

· 12�

(�i

i/√Ni

)�i iN−1/2i − = 0� (7)

Also, !F /! = 0 returns the budget constraint∑ki=1Ni − T = 0.To examine the relationship between Ni and Nj for

i �= j , we consider three cases:Case 1. i ∈ Sm and j � Sm. Equating the expressions

in Equations (6) and (7),

∏r∈Smr �=i

P �J̃r≤c�· ∏r�Sm

P�J̃r≥c�· −12�

(�i

i/√Ni

)�i iN−1/2i −

= ∏r∈Sm

P�J̃r≤c�· ∏r�Smr �=j

P �J̃r≥c�· 12�

(�j

j/√Nj

)�j

jN−1/2j − �

Simplifying,

P�J̃j ≥ c� · e−�2i /2

2i /Ni�

−�i i

N−1/2i

= P�J̃i ≤ c� · e−�2j /2

2j /Nj �

�j

jN−1/2j �

Taking the log on both sides,

logP�J̃j ≥ c��− �2i Ni

2 2i

+ log(−�i i

)− 1

2logNi�

= logP�J̃i ≤ c��− �2j Nj

2 2j

+ log(�j

j

)− 1

2logNj��

Now, we consider the asymptotic limit T → � withNi = "iT ,

∑ki=1"i = 1. Substituting for Ni and dividing

by T yields

1T

logP�J̃j≥c��− �2i

2 2i

"i+1T

log(−�i i

)− 1

2Tlog"iT �

= 1T

logP�J̃i≤c��− �2j

2 2j

"j+1T

log(�j

j

)− 1

2Tlog"jT ��

Dow

nloa

ded

from

info

rms.

org

by [

128.

112.

66.6

6] o

n 05

Feb

ruar

y 20

16, a

t 19:

07 .

For

pers

onal

use

onl

y, a

ll ri

ghts

res

erve

d.


and then taking T →� yields

�2i

2i

"i =�2j

2j

"j �

Therefore, we obtain the ratio between "i and "j orbetween Ni and Nj as

Ni

Nj

= "i

"j

=( i/�i j/�j

)2

� (8)

Because �i = J̄i − c, i = 1�2� � � � � k, it is clear that theoptimal allocation depends on the value of c (see §3.3for determining a good value), and thus the m = 1case does not in general reduce to the original OCBAallocation for selecting the best design.Case 2. Both i� j ∈ Sm and i �= j . From Equation (6),

!F /!Ni = !F /!Nj = 0 yields∏r∈Smr �=i

P �J̃r ≤ c� · ∏r�Sm

P�J̃r ≥ c�

· −12�

(�i

i/√Ni

)�i

2i /Ni�

i

N 3/2i

−

= ∏r∈Smr �=j

P �J̃r ≤ c� · ∏r�Sm

P�J̃r ≥ c�

· −12�

(�j

j/√Nj

)�j

2j /Nj�

j

N 3/2j

− �

Then,

P�J̃j ≤ c� · e−�2i /2

2i /Ni�

−�i i

N−1/2i

= P�J̃i ≤ c� · e−�2j /2

2j /Nj �

−�j j

N−1/2j �

Following the analogous derivation that led to Equa-tion (8) yields the same result.Case 3. i� j � Sm, and i �= j . Again, following the

same derivation procedures as in the previous twocases leads to Equation (8) again, so it holds for anyi, j ∈ �1�2� � � � � k�, and i �= j .

In conclusion, if a solution satisfies Equation (8),then the KKT sufficient conditions must hold asymp-totically, so that the corresponding solution is a locallyoptimal solution to the Lagrangian relaxation of theOCBA problem (5). We therefore have the followingresult.

Theorem 1. The allocation given by 8� is asymptoti-cally as T →�� a locally optimal solution for the OCBAproblem 5�, where �i = J̄i − c for c a constant, and thevariances 2

1 � 22 � � � � �

2k are finite; i.e., APCSm is asymp-

totically maximized by the allocation given by 8�.

3.3. Determination of c ValueThe parameter c impacts the quality of the approxi-mation APCSm to P {CS}. Because APCSm is a lowerbound of P {CS}, choosing c to make APCSm as large

~Ji1

Ji1

Jim

Jim

~ ~Jim+1

Jim+1

Jik

Jik

~

cc′ c′′

Figure 1 An Example of Probability Density Functions for J̃i , i =1�2� � � � � k, c′ < J̄im < c < J̄im+1

< c′′

as possible is likely to provide a better approxima-tion of APCSm to P {CS}. Figure 1 is provided to helpexplain our choice of c by giving an example of prob-ability density functions for J̃i, i= 1�2� � � � � k.

Note that APCSm is a product of P�J̃i ≤ c� for i ∈ Smand P�J̃i ≥ c� for i � Sm. Consider the equal variancecase VarJ̃i1� = VarJ̃i2� = · · · = VarJ̃ik �, where for anyvalue of c, P�J̃i1 ≤ c� > P�J̃i2 ≤ c� > · · ·> P�J̃im ≤ c�, andP�J̃im+1

≥ c� < P�J̃im+2≥ c� < · · · < P�J̃ik ≥ c�. To prevent

APCSm from being small, we want to choose c toavoid any of the product terms being too small, espe-cially P�J̃im ≤ c� and P�J̃im+1

≥ c�, because one of thesetwo terms will be the smallest in the product, depend-ing on the value of c. Thus, a good choice of c liesbetween J̄im and J̄im+1

because(i) if c= c′ < J̄im , then P�J̃im < c′� < 0�5, and this term

decreases with decreasing c′, resulting in a negativeimpact on APCSm;

(ii) similarly, if c= c′′ > J̄im+1, then P�J̃im+1

> c′′� < 0�5,and this term decreases with increasing c′′.

With these considerations, one would like to max-imize both P�J̃im ≤ c� and P�J̃im+1

≥ c�. In this paper,we choose to maximize the product of P�J̃im ≤ c� andP�J̃im+1

≥ c�. Define � i ≡ i/√Ni. Then,

P�J̃im ≤ c�P�J̃im+1≥ c�=�

(c− J̄im� im

)�

(J̄im+1

− c

� im+1

)�

Following the same approach as used to establishTheorem 1, this quantity is asymptotically maximizedwhen

c= � im+1J̄im + � im J̄im+1

� im + � im+1

� (9)

and we use this value of c in our implementation,which in numerical testing results in good perfor-mance while requiring negligible computation cost.

3.4. Heuristic Sequential Allocation SchemeThe allocation given by (8) assumes known variancesand independence of estimated sample means acrossdesigns. In practice, a sequential algorithm is used toestimate these quantities using the updated samplevariances. Furthermore, the “constant” c and samplemeans are also updated during each iteration. Each

Dow

nloa

ded

from

info

rms.

org

by [

128.

112.

66.6

6] o

n 05

Feb

ruar

y 20

16, a

t 19:

07 .

For

pers

onal

use

onl

y, a

ll ri

ghts

res

erve

d.


design is initially simulated with n0 replications in thefirst stage, and additional replications are allocatedincrementally with % replications to be allocated ineach iteration. By utilizing the allocation given in The-orem 1, we present the following heuristic sequentialalgorithm.

OCBA-m Allocation Procedure

INPUT k�m�T �%�n0 (T − kn0 a multiple of % andn0 ≥ 5);

INITIALIZE l← 0;Perform n0 simulation replicationsfor all designs; N l

1 =N l2 =···=N l

k=n0.LOOP WHILE

∑ki=1N

li < T DO

UPDATE Calculate sample means J̄i = 1/N li � ·∑Nl

ij=1Xij , and sample standard

deviation si=√1/N l

i −1��∑Nl

ij=1Xij− J̄i�

2,i= 1� � � � � k, using the new simulationoutput; compute � i = si/

√N li ,

i= 1� � � � � k, and c= � im+1J̄im + � im J̄im+1

�/

� im + � im+1�; update �i = J̄i − c,

i= 1� � � � � k.ALLOCATE Increase the computing budget

by % and calculate the new budgetallocation, N l+1

1 , N l+12 � � � � �N l+1

k ,according to

N l+11

s1/�1�2= N l+1

2

s2/�2�2= · · · = N l+1

k

sk/�k�2� 10�

SIMULATE Perform additional max(N l+1i −N l

i �0�simulations for design i,i= 1� � � � � k; l← l+ 1.

END OF LOOP

The approximations made in this OCBA-m proce-dure are further discussed below.

3.4.1. Variances. The allocation given in Theo-rem 1 assumes known variances. The above sequen-tial algorithm estimates these quantities using theupdated sample variances. As more simulation repli-cations are iteratively allocated to each design, thevariance estimation improves. To avoid poor estima-tion at the beginning, n0 should not be too small(we suggest n0 ≥ 5). Also, it is wise to avoid large% (we suggest % < 100) to prevent a poor alloca-tion before a correction can be made in the next iter-ation, which is particularly important in the earlystages. Our numerical testing indicates that the per-formance of the OCBA-m procedure is not sensitiveto the choice of n0 and % if these guidelines are fol-lowed, and the impact of approximating variance bysample variance is not significant.

3.4.2. Sequential Allocation. The OCBA-m proce-dure is a sequential algorithm. The sequential nature

of the estimation introduces dependence in the sam-ple estimates themselves so that the independenceacross designs assumed in (4) does not hold. How-ever, the sequential approach is an effective way toestimate the unknown variance, and several stud-ies have demonstrated significant efficiency gains inusing sequential allocation versus one-time or two-stage allocation (cf. Inoue et al. 1999, Chen et al. 2006).

3.4.3. Asymptotically Large Computing Budget.Although the allocation given by Equation (8) in The-orem 1 is derived by taking T → �, the numericalresults presented in the next section indicate that thecorresponding allocation in the OCBA-m procedureworks very efficiently for small T , as well.

3.4.4. Continuous Ni. The resulting Ni in theALLOCATE step based on Equation (8) is a continu-ous number that must be rounded to an integer. Inthe numerical experiments in the next section, Ni isrounded to the nearest integer such that the sum-mation of additional simulation replications for alldesigns equals %. This is simply for ease of computingbudget management in numerical testing, providinga fair comparison with other allocation procedures.We have found numerically that the OCBA-m perfor-mance is not sensitive to how we round Ni, probablydue to the robustness of a sequential procedure.

3.4.5. Determination of c Value. The parameter cimpacts the quality of the approximation APCSm toP {CS}. Section 3.3 provides a simple approach todetermine c by maximizing the product of P�J̃im ≤ c�

and P�J̃im+1≥ c�, which are the critical terms for most

cases and provide a proxy for maximizing APCSm.For the purpose of determining the computing bud-get allocation, our numerical testing shows that sucha simple approach performs very well. However, forthe purpose of estimating the probability of correctselection, one would want to choose c by consideringmore than these two terms or even all the terms inAPCSm.

Note that this OCBA-m procedure is designed toselect all of the top-m designs when m ≥ 2. For them = 1 case, the original OCBA procedure given inChen et al. (2000) is different from the OCBA-m dueto different approximations made. In this case, theoriginal OCBA procedure is slightly preferred, eventhough this OCBA-m procedure still works effectively.

4. Numerical Testing and Comparisonwith Other Allocation Procedures

In this section, we test the OCBA-m algorithmby comparing it on several numerical experimentswith different allocation procedures: equal allocation,which simulates all design alternatives equally; theKoenig and Law (1985) procedure denoted by KL;

Dow

nloa

ded

from

info

rms.

org

by [

128.

112.

66.6

6] o

n 05

Feb

ruar

y 20

16, a

t 19:

07 .

For

pers

onal

use

onl

y, a

ll ri

ghts

res

erve

d.


proportional to variance (PTV), which is a mod-ification of KL that allocates replications propor-tional to the estimated variances; and the originalOCBA allocation algorithm for selecting only the bestdesign (Chen et al. 2000). For notational simplic-ity, we assume that J(1) < J(2) < · · · < J(k), so design[1] is the best, and correct selection would be Sm =�(1)� (2)� � � � � (m)� (but this is unknown a priori).

We also test the OCBA-m algorithm under the simu-lation optimization setting, in which OCBA-m is inte-grated with three different optimization methods. Theperformance of each of the three methods integratingthe OCBA-m allocation is compared with the perfor-mance of the same method using equal allocation.

4.1. Computing Budget Allocation Procedures

4.1.1. Equal Allocation. The simulation budget isallocated equally to all designs; i.e., Ni = T /k foreach i. The performance of equal allocation will serveas a benchmark for comparison.

4.1.2. KL (Koenig and Law 1985). The two-stageprocedure of Koenig and Law (1985) selects a sub-set of specified size m, with probability at least P ∗,so that the selected subset is exactly the actual sub-set with the best (smallest) expected values, providedthat J(m+1)− J(m) is no less than an indifference zone, d.As in our setting, the ordering within the selectedsubset does not matter.

In the first stage, all designs are simulated forn0 replications. Based on the sample variance esti-mate (s2i ) obtained from the first stage and given thedesired correct selection probability P ∗, the number ofadditional simulation replications for each design inthe second stage is determined by

Ni =maxn0 + 1� �h23s

2i /d

2�� for i= 1�2� � � � � k� (11)

where �·� is the integer “round-up” function, and h3is a constant that depends on k, m, P ∗, and n0.

4.1.3. Proportional to Variance (PTV). This is asequential modified version of the KL procedure,based on the observation that (11) implies that Ni

is proportional to the estimated sample variance s2i .Thus, the PTV procedure sequentially determines {Ni}based on the newly updated sample variances byreplacing Equation (10) in the ALLOCATE step of theOCBA-m algorithm by

N l+11

s21= N l+1

2

s22= · · · = N l+1

k

s2k�

Thus, the number of replications for each designgrows in proportion to the sample variance. Note thatthe indifference-zone parameter has been removed inthis modification to make it comparable to the otherprocedures.

4.1.4. OCBA (Chen et al. 2000). The originalsequential OCBA procedure of Chen et al. (2000) allo-cates the computing budget with the objective ofselecting only the best design, i.e., m = 1, for whichextensive numerical testing has demonstrated its effi-ciency. Although it is not designed for m > 1, wetest this procedure here for benchmarking purposes,and denote it by OCBA. Specifically, the budget allo-cation in Equation (10) of the OCBA-m algorithm isreplaced by

(1) N l+1i /N l+1

j = siJ̄b − J̄j �/sj J̄b − J̄i��2 for all i �=

j �= b,

(2) N l+1b = sb

√∑ki=1�i �=bN

l+1i /si�

2,where b= argmini J̄i.

4.2. Numerical Results for DifferentAllocation Procedures

To compare the performance of the procedures, wecarried out numerical experiments for several typi-cal selection problems. In comparing the procedures,the measurement of effectiveness used is the P {CS}estimated by the fraction of times the procedure suc-cessfully finds all the true m-best designs out of100,000 independent experiments. Because this penal-izes incorrect selections equally—e.g., a subset con-taining the top-1, top-2, � � � , and top-(m − 1) designsand missing only the top-m design is treated no dif-ferently than a subset containing not a single one ofthe top-m designs—in our numerical experiments, wealso include a second measure of selection quality, theso-called expected opportunity cost E[OC], where

OC≡m∑j=1

Jij − J(j)��

This measure penalizes particularly bad choicesmore than mildly bad choices. For example, whenm= 3, a selection of {top-1, top-2, top-4} is betterthan {top-1, top-2, top-5}, and both are better than{top-1, top-3, top-5}. Note that OC returns a minimumvalue of zero when all the top-m designs are cor-rectly selected. The estimated E[OC] is the averageof the OC estimates over the 100,000 independentexperiments.

Each of the procedures simulates each of thek designs for n0 = 20 replications initially (followingrecommendations in Koenig and Law 1985, Law andKelton 2000). KL allocates additional replications in asecond stage (so the total number is not fixed a pri-ori), whereas the other procedures allocate replica-tions incrementally by %= 50 each time until the totalbudget, T , is consumed. For each level of computingbudget, we estimate the achieved P {CS} and E[OC].

Because KL is a two-stage indifference-zone proce-dure, we must specify the values for the desired prob-ability of correct selection, P ∗, and the indifference

Dow

nloa

ded

from

info

rms.

org

by [

128.

112.

66.6

6] o

n 05

Feb

ruar

y 20

16, a

t 19:

07 .

For

pers

onal

use

onl

y, a

ll ri

ghts

res

erve

d.


zone d to satisfy the condition that J(m+1)− J(m) ≥ d,where a smaller d implies a higher required compu-tation cost based on Equation (10). In practice, thevalue of J(m+1) or J(m) is unknown beforehand, but forbenchmarking purposes, we set d= J(m+1)− J(m), whichleads to the minimum computational requirement (ormaximum efficiency) for the procedure. As is done forthe other procedures, the resulting P {CS} and E[OC]can be estimated over the 100,000 independent exper-iments. Because the required computation cost alsovaries from one experiment to another, we will indi-cate the average number of total replications based onthe 100,000 independent experiments.Example 1 (Equal Variance). There are 10 alter-

native designs, with distribution Ni�62� for designi = 1�2� � � � �10. The goal is to identify the top-3designs via simulation samples, i.e., m = 3 in thisexample.

To characterize the performance of different pro-cedures as a function of T , we vary T between 200and 8,000 for all of the procedures other than KL, andthe estimated achieved P {CS} and E[OC] as a functionof T are shown in Figures 2(a) and 2(b), respectively.For KL, we test two cases, P ∗ = 0�9 and P ∗ = 0�95, andthe corresponding estimated P {CS} and E[OC] versusthe average total simulation replications are shown astwo single points (the triangle and circle) in Figures2(a) and 2(b), respectively.

We see that all procedures obtain a higher P {CS}and a lower E[OC] as the available computing bud-get increases. However, OCBA-m achieves the high-est P {CS} and the lowest E[OC] for the same amountof computing budget. It is interesting to observethat OCBA, which performs significantly better thanequal allocation and PTV when the objective is tofind the single best design, fares worse in this exam-ple than these two allocations when the objective ischanged to finding all the top-3 designs. Equal alloca-tion performs almost identically to PTV, which makessense because the variance is constant across designs.Specifically, the computation costs to attain P {CS} =0.95 for OCBA-m, OCBA, Equal, and PTV are 800,3,200, 1,950, and 2,000, respectively.

Not surprisingly, the performance of KL is along theperformance curve of PTV because KL basically allo-cates the computing budget based on design variance.However, KL achieves a substantially higher P {CS}than the desired level (e.g., exceeding 0.99 for the tar-get minimum of P ∗ = 0�9) by spending a much highercomputing budget than actually needed, consistentwith the fact that typical two-stage indifference-zoneprocedures are conservative.Example 2 (Variance Increasing in Value of

Mean). This is a variant of Example 1. All settings arepreserved except that the variance is increasing in thedesign index, so good designs have smaller variances.

Specifically, the designs are distributed Ni� i2� fordesign i= 1�2� � � � �10. Again, m= 3.

The test results shown in Figures 3(a) and 3(b) arequalitatively similar to those in Example 1. OCBA-machieves the highest P {CS} for the same amount ofcomputing budget. However, PTV (and KL) performspoorly in this example because good designs receiverelatively less computing budget due to their smallervariances, which tends to slow down the process ofdistinguishing good designs. Specifically, the com-putation costs to attain P {CS} = 0.95 for OCBA-m,OCBA, Equal, and PTV are 350, 750, 700, and 2,250,respectively.Example 3 (Variance Decreasing in Value of

Mean). The third example is another variant of Exam-ples 1 and 2, but this time the variance is decreasing inthe design index; i.e., the distribution is Ni� 11− i�2�for design i = 1�2� � � � �10. Under this setting, gooddesigns have larger variance. Again, m= 3.

The test results shown in Figures 4(a) and 4(b) aresimilar to those in the previous examples, with againOCBA-m performing the best. However, in contrastto Example 2, PTV (and KL) performs relatively wellin this example because good designs receive muchmore computing budget due to their higher variances.On the other hand, OCBA performs poorly because itspends an excessive amount of the computing budgetto distinguish between the very top designs becauseits objective is to find the best. In this example, thecomputation costs to attain P {CS}= 0�95 for OCBA-m,OCBA, Equal, and PTV are 1,400, 7,900, 3,050, and2,200, respectively.Example 4 (s� S� Inventory Problem). The fourth

example is an (s� S) inventory policy problem basedon the example given in §1.5.1 of Law and Kelton(2000). The system involves a single item underperiodic review, full backlogging, and random leadtimes (uniformly distributed between the 0.5 and 1.0period), with costs for ordering (including a fixedsetup cost of $32 per order and an incremental costof $3 per item), on-hand inventory ($1 per item perperiod), and backlogging (fixed shortage cost of $5per item per period). The times between demandsare independent and identically distributed (i.i.d.)exponential random variables with a mean of 0.1period. The sizes of demands are i.i.d. random vari-ables taking values 1, 2, 3, and 4, with probabilities1/6, 1/3, 1/3, and 1/6, respectively. The (s� S) policyspecifies that if the on-hand inventory at the reviewpoint is at or below the level s, then an order isplaced of an amount that would bring the inventoryup to level S. The 10 inventory policies are definedby the parameters s1� s2� � � � � s10� = 20�20�20�40�40�40�60�60�60�80� and S1� S2� � � � � S10� = 30�40�50�50�60�70�70�80�90�90�, respectively. The objec-tive is to find the top-3 (m= 3) policies with minimumexpected average inventory cost over 120 periods.

Dow

nloa

ded

from

info

rms.

org

by [

128.

112.

66.6

6] o

n 05

Feb

ruar

y 20

16, a

t 19:

07 .

For

pers

onal

use

onl

y, a

ll ri

ghts

res

erve

d.


0.90

0.91

0.92

0.93

0.94

0.95

0.96

0.97

0.98

0.99

1.00

200

450

700

950

1,200

1,450

1,700

1,950

2,200

2,450

2,700

2,950

3,200

3,450

3,700

3,950

4,200

4,450

4,700

4,950

5,200

5,450

5,700

5,950

6,200

6,450

6,700

6,950

7,200

7,450

7,700

7,950

T

P{C

S}

EqualPTVOCBAOCBA-m

KLP* = 90%

KLP* = 95%

Figure 2(a) P CS� vs. T Using Four Sequential Allocation Procedures and KL (Triangle for P ∗ = 90% and Circle for P ∗ = 95%) for Example 1

200

450

700

950

1,200

1,450

1,700

1,950

2,200

2,450

2,700

2,950

3,200

3,450

3,700

3,950

4,200

4,450

4,700

4,950

5,200

5,450

5,700

5,950

6,200

6,450

6,700

6,950

7,200

7,450

7,700

7,950

T

KLP* = 90%

KLP* = 95%

0

0.005

0.010

0.015

0.020

0.025

0.030

0.035

0.040

0.045

0.050

E[O

C]

EqualPTVOCBAOCBA-m

Figure 2(b) E[OC] vs. T Using Four Sequential Allocation Procedures and KL (Triangle for P ∗ = 90% and Circle for P ∗ = 95%) for Example 1

Dow

nloa

ded

from

info

rms.

org

by [

128.

112.

66.6

6] o

n 05

Feb

ruar

y 20

16, a

t 19:

07 .

For

pers

onal

use

onl

y, a

ll ri

ghts

res

erve

d.


200

400

600

800

1,000

1,200

1,400

1,600

1,800

2,000

2,200

2,400

2,600

2,800

3,000

3,200

3,400

3,600

3,800

4,000

4,200

4,400

4,600

4,800

5,000

5,200

5,400

5,600

5,800

6,000

6,200

6,400

6,600

6,800

7,000

T

EqualPTVOCBAOCBA-m

KLP* = 90%

KLP* = 95%

0.90

0.91

0.92

0.93

0.94

0.95

0.96

0.97

0.98

0.99

1.00P

{CS}

Figure 3(a) P {CS} vs. T Using Four Sequential Allocation Procedures and KL (Triangle for P ∗ = 90% and Circle for P ∗ = 95%) for Example 2

EqualPTVOCBAOCBA-m

KLP* = 90% KL

P* = 95%

0

0.005

0.010

0.015

0.020

0.025

0.030

0.035

0.040

0.045

0.050

E[O

C]

200

400

600

800

1,000

1,200

1,400

1,600

1,800

2,000

2,200

2,400

2,600

2,800

3,000

3,200

3,400

3,600

3,800

4,000

4,200

4,400

4,600

4,800

5,000

5,200

5,400

5,600

5,800

6,000

6,200

6,400

6,600

6,800

7,000

T


Dow

nloa

ded

from

info

rms.

org

by [

128.

112.

66.6

6] o

n 05

Feb

ruar

y 20

16, a

t 19:

07 .

For

pers

onal

use

onl

y, a

ll ri

ghts

res

erve

d.


200

500

800

1,100

1,400

1,700

2,000

2,300

2,600

2,900

3,200

3,500

3,800

4,100

4,400

4,700

5,000

5,300

5,600

5,900

6,200

6,500

6,800

7,100

7,400

7,700

8,000

8,300

8,600

8,900

9,200

9,500

9,800

T

EqualPTVOCBAOCBA-m

KLP* = 90%

KLP* = 95%

0.90

0.91

0.92

0.93

0.94

0.95

0.96

0.97

0.98

0.99

1.00P

{CS}


200

500

800

1,100

1,400

1,700

2,000

2,300

2,600

2,900

3,200

3,500

3,800

4,100

4,400

4,700

5,000

5,300

5,600

5,900

6,200

6,500

6,800

7,100

7,400

7,700

8,000

8,300

8,600

8,900

9,200

9,500

9,800

T

EqualPTVOCBAOCBA-m

KLP* = 90% KL

P* = 95%

0

0.005

0.010

0.015

0.020

0.025

0.030

0.035

0.040

0.045

0.050

E[O

C]


Dow

nloa

ded

from

info

rms.

org

by [

128.

112.

66.6

6] o

n 05

Feb

ruar

y 20

16, a

t 19:

07 .

For

pers

onal

use

onl

y, a

ll ri

ghts

res

erve

d.


200

350

500

650

800

950

1,100

1,250

1,400

1,550

1,700

1,850

2,000

2,150

2,300

2,450

2,600

2,750

2,900

3,050

3,200

3,350

3,500

3,650

3,800

3,950

T

EqualPTVOCBAOCBA-m

KLP* = 90%

KLP* = 95%

0.90

0.91

0.92

0.93

0.94

0.95

0.96

0.97

0.98

0.99

1.00P

{CS}


200

350

500

650

800

950

1,100

1,250

1,400

1,550

1,700

1,850

2,000

2,150

2,300

2,450

2,600

2,750

2,900

3,050

3,200

3,350

3,500

3,650

3,800

3,950

T

EqualPTVOCBAOCBA-m

KLP* = 90% KL

P* = 95%

0

0.005

0.010

0.015

0.020

0.025

0.030

0.035

0.040

0.045

0.050

E[O

C]


Dow

nloa

ded

from

info

rms.

org

by [

128.

112.

66.6

6] o

n 05

Feb

ruar

y 20

16, a

t 19:

07 .

For

pers

onal

use

onl

y, a

ll ri

ghts

res

erve

d.


The test results shown in Figures 5(a) and 5(b)are similar to those in previous examples, in thatOCBA-m is clearly the top performer again; however,this time, OCBA is the runner-up by a slight mar-gin. The computation costs to attain P {CS} = 0�95 forOCBA-m, OCBA, Equal, and PTV are 500, 1,200, 1,650,and 1,350, respectively.Example 5 (Larger-Scale Problem). This is a var-

iant of Example 1 (constant variance), with the num-ber of designs increased to 50. The alternatives havedistribution Ni�102� for design i = 1�2� � � � �50, andm= 5. Because KL’s performance basically followsthat of PTV, but its required computing budget is farbeyond the range we are considering here, we excludeKL from the numerical testing.

Figures 6(a) and 6(b) depict the simulation results.As in earlier examples, OCBA-m achieves the highestP {CS} and the lowest E[OC] with the same amountof computing budget; however, the performance gapbetween OCBA-m and other procedures is substan-tially greater. This is because a larger design spaceallows the OCBA-m algorithm more flexibility in allo-cating the computing budget, resulting in even bet-ter performance. On the other hand, OCBA performspoorly because it spends a lot of computing budget ondistinguishing the very top designs, a tendency that ispenalized even more for larger m. Again, because thevariance is constant across designs, the performanceof Equal and PTV are nearly indistinguishable. In thisexample, the computation costs to attain P {CS}= 0.95for OCBA-m, OCBA, Equal, and PTV are 4,050, 31,050,27,050, and 27,200, respectively.

4.3. Numerical Results for SimulationOptimization

In these numerical examples, we combine theOCBA-m allocation procedure with three iterativeoptimization search algorithms, requiring the selec-tion of an “elite” subset of good candidate solutions ineach iteration. The OCBA-m procedure is integratedinto each of the three optimization algorithms, andthe resulting performance of the algorithm is com-pared with the same algorithm using equal simula-tion of all candidate solutions. The purpose of theseexamples is not to find the best optimization searchalgorithm, but rather to explore whether the OCBA-mprocedure can enhance the efficiency of any simula-tion optimization algorithm. The three optimizationalgorithms considered are the following.

4.3.1. Cross-Entropy Method. The cross-entropy(CE) method (see Rubinstein and Kroese 2004)searches the underlying variable space by adaptivelyupdating a parameterized sampling distribution. Thebasic principle is to minimize the Kullback-Leiblerdivergence between the unknown optimal samplingdistribution and the parameterized distribution. Thealgorithm is summarized as follows.

Step 1. Initialize a sampling distribution P .Step 2. Sample k candidate solutions using P .Step 3. Simulate these sampled k alternatives and

select a subset containing the top-m.Step 4. Update P based on the selected top-m and

the CE principle.Step 5. Go back to Step 2 if the stopping criterion

is not met.

4.3.2. Population-Based Incremental Learning(PBIL). The PBIL algorithm was originally developedfor binary search problems (Baluja 1994). We test oneof its new developments for continuous optimizationproblems due to Rudlof and Köppen (1996). The PBILupdates the sampling distribution P with a probabilis-tic learning technique using the estimated mean ofan “elite” subset of good candidate solutions in eachiteration. This algorithm is almost the same as the CEalgorithm, except in Step 4, the PBIL learning princi-ple is applied.

4.3.3. Neighborhood Random Search (NRS). Wealso test a simple random search method. In eachiteration, k alternative designs are simulated andthen the top-m solutions are selected. For the nextiteration, a large proportion of candidate solutionsare generated by sampling the neighborhood of theelite solutions. The longer the distance, the smallerthe probability a solution is sampled. The remaining(smaller) portion of samples are taken from the entirevariable space to ensure convergence to the globaloptimum. The algorithm is summarized as follows.Step 1. Uniformly sample k candidate solutions

over the design variable space.Step 2. Stop, if the stopping criterion is met.Step 3. Simulate the sampled k alternatives and

select a subset containing the top m.Step 4. Sample 80% of the new k candidate solu-

tions in the neighborhood of the top-m elite solutions.Another 20% is taken uniformly from the entire vari-able space.Step 5. Go back to Step 2.In each of the above three optimization algorithms,

Step 3 is a ranking-and-selection problem in whichthe top-m designs must be identified. The overall effi-ciency of these types of simulation optimization algo-rithms depends on how efficiently we simulate thecandidates and correctly select the top-m designs.Example 6 (Griewank Function). The Griewank

function is a common example in the globaloptimization literature (see Fu et al. 2006), given intwo-dimensional (2-D) form by

f x1�x2�=140x2

1 + x22�− cosx1� cos

(x2√2

)+ 1�

where x1 and x2 are continuous variables and −10 ≤x1 ≤ 10, −10 ≤ x2 ≤ 10. The unique global minimum

Dow

nloa

ded

from

info

rms.

org

by [

128.

112.

66.6

6] o

n 05

Feb

ruar

y 20

16, a

t 19:

07 .

For

pers

onal

use

onl

y, a

ll ri

ghts

res

erve

d.


0.80

0.82

0.84

0.86

0.88

0.90

0.92

0.94

0.96

0.98

1.00

1,00

0

1,85

0

2,70

0

3,55

0

4,40

0

5,25

0

6,10

0

6,95

0

7,80

0

8,65

0

9,50

0

10,3

50

11,2

00

12,0

50

12,9

00

13,7

50

14,6

00

15,4

50

16,3

00

17,1

50

18,0

00

18,8

50

19,7

00

20,5

50

21,4

00

22,2

50

23,1

00

23,9

50

24,8

00

25,6

50

26,5

00

27,3

50

28,2

00

29,0

50

29,9

00

T

P{C

S}

EqualPTVOCBAOCBA-m

Figure 6(a) P {CS} vs. T Using Four Sequential Allocation Procedures for Example 5

1,00

0

1,85

0

2,70

0

3,55

0

4,40

0

5,25

0

6,10

0

6,95

0

7,80

0

8,65

0

9,50

0

10,3

50

11,2

00

12,0

50

12,9

00

13,7

50

14,6

00

15,4

50

16,3

00

17,1

50

18,0

00

18,8

50

19,7

00

20,5

50

21,4

00

22,2

50

23,1

00

23,9

50

24,8

00

25,6

50

26,5

00

27,3

50

28,2

00

29,0

50

29,9

00

T

0

0.005

0.010

0.015

0.020

0.025

0.030

0.035

0.040

0.045

0.050

E[O

C]

EqualPTVOCBAOCBA-m

Figure 6(b) E[OC] vs. T Using Four Sequential Allocation Procedures for Example 5

Dow

nloa

ded

from

info

rms.

org

by [

128.

112.

66.6

6] o

n 05

Feb

ruar

y 20

16, a

t 19:

07 .

For

pers

onal

use

onl

y, a

ll ri

ghts

res

erve

d.


8

6

4

2

010

5

–5 –5

5

–10 –10

100

0

Figure 7 2-D Griewank Function Tested in Example 6

of this function is at x∗1�x∗2�= 0�0� and f x∗1�x

∗2�= 0.

The additive noise incurred in stochastic simulation isN0�12�. Figure 7 gives an illustration of this functionwithout noise.

In numerical implementation, the stopping crite-rion for the ranking-and-selection problem in Step3 is when the posterior APCSm given by Equation(4) is no less than 1 − 0�2 ∗ exp−q/50�, where q isthe iteration number. We set k = 100 and m = 5. Incomparing the procedures, the measurement of effec-tiveness used is the average error between the bestdesign thus far and the true optimal solution over200 independent experiments. The results are shownin Figure 8. The thick lines indicate the performancewith OCBA-m for different optimization algorithms,

0.04

0.06

0.08

0.10

0.12

0.14

0.16

0.18

0.20

0 10,000 20,000 30,000 40,000 50,000 60,000 70,000 80,000 90,000 100,000

T

Ave

rage

err

or o

f E

[f(x

* )]

PBIL EQUAL

PBIL OCBAM

NRS EQUAL

NRS OCBAM

CE EQUAL

CE OCBAMCE EQUAL

PBIL EQUAL

NRS EQUAL

CE OCBAM

PBIL OCBAMNRS OCBAM

Figure 8 Performance Comparison for Three Optimization Search Algorithms With and Without OCBA-m for Example 6

whereas the thin lines show the performances with-out OCBA-m. Lines with different shades/patternsrepresent the use of different optimization searchalgorithms.

We see that the optimality gap decreases for allprocedures as the available computing budget in-creases. In this example, NRS performs better thanPBIL, which does better than CE. However, OCBA-msignificantly enhances the efficiency for all threesearch methods. For example, with integration ofOCBA-m, NRS can achieve an average error of 0.1using a computation cost of 22,800. Without OCBA-m,NRS spends a computation cost of 68,500 to achievethe same level of error. Similarly, the computationcosts for PBIL to reduce the average error to 0.12with and without OCBA-m are 11,500 and 53,700,respectively. The speedup factor of using OCBA-mis even larger if the target level of optimality getshigher.Example 7 (Rosenbrock Function). The Rosen-

brock function is another common example in theglobal optimization literature (e.g., Fu et al. 2006). It isa nonconvex function with a “banana-shaped” valleygiven in 2-D by

f x1�x2�= 100x2 − x21�

2 + x1 − 1�2�

where x1 and x2 are continuous variables and −5 ≤x1 ≤ 5, −5≤ x2 ≤ 5. The global minimum of this func-tion is at x∗1�x

∗2�= 1�1� and f x∗1�x

∗2�= 0. The addi-

tive noise incured in stochastic simulation is N0�102�.

Dow

nloa

ded

from

info

rms.

org

by [

128.

112.

66.6

6] o

n 05

Feb

ruar

y 20

16, a

t 19:

07 .

For

pers

onal

use

onl

y, a

ll ri

ghts

res

erve

d.


10

8

6

4

2

0–5 –5

0 0

5 5

× 104

Figure 9 2-D Rosenbrock Function Tested in Example 7

Figure 9 gives an illustration of this function withoutnoise.

The numerical setting is the same as that in Exam-ple 6. The test results are shown in Figure 10. UnlikeExample 6, the PBIL method has the best performancein this example. Although the order of optimizationmethods are different from that in Example 6, thelevel of efficiency enhancement using OCBA-m is verysimilar.

0.2

0.4

0.6

0.8

1.0

1.2

1.4

NRS EQUAL

PBIL EQUAL

CE EQUAL

PBIL OCBAM

CE OCBAM

NRS OCBAM

0 10,000 20,000 30,000 40,000 50,000 60,000 70,000 80,000 90,000 100,000

T

Ave

rage

err

or o

f E

[f(x

* )]

PBIL EQUAL

PBIL OCBAM

NRS EQUAL

NRS OCBAM

CE EQUAL

CE OCBAM

Figure 10 Performance Comparison for Three Optimization Search Algorithms With and Without OCBA-m for Example 7

5. ConclusionsWe present an efficient allocation procedure for aclass of ranking-and-selection problems in whichthe objective is to identify the top-m designs outof k (simulated) competing designs. The goal is tomaximize the simulation efficiency, expressed as theprobability of correct selection within a given com-puting budget. We propose a heuristic to approxi-mate the associated correct selection probability, andthen derive an asymptotically optimal allocation pro-cedure for an approximation to the approximate prob-ability. Numerical testing indicates that the allocationprocedure is significantly more efficient and robustthan other methods in the literature, with the relativeefficiency increasing in problem size. Furthermore,although the procedure was derived based on anasymptotic derivation, the numerical results indicatethat the procedure is effective even when the comput-ing budget is small. The numerical results illustratethat the allocation specified by the original OCBAalgorithm (Chen et al. 2000), designed for selectingthe single best design, does not perform well in select-ing the top-m designs, providing another motivationfor the need of a new methodology when the objec-tive is extended beyond selecting just the best design.Our allocation method has the potential to improvethe efficiency of population-based global optimiza-

Dow

nloa

ded

from

info

rms.

org

by [

128.

112.

66.6

6] o

n 05

Feb

ruar

y 20

16, a

t 19:

07 .

For

pers

onal

use

onl

y, a

ll ri

ghts

res

erve

d.


tion methods that require the selection of an “elite”subset of good candidate solutions in each iterationof the algorithm. While different optimization algo-rithms perform differently in our two preliminarynumerical examples, the new OCBA-m allocation sig-nificantly enhances the computational efficiency foreach individual optimization algorithm.

AcknowledgmentsThis work was supported in part by the National Sci-ence Foundation Grants IIS-0325074, DMI-0540312, andDMI-0323220, by NASA Ames Research Center GrantNNA05CV26G, by the Federal Aviation AdministrationGrant 00-G-016, and by Air Force Office of ScientificResearch Grants FA9550-04-1-0210 and FA9550-07-1-0366.

ReferencesAndradóttir, S., D. Goldsman, B. W. Schmeiser, L. W. Schruben,

E. Yücesan. 2005. Analysis methodology: Are we done?Proc. 2005 Winter Simulation Conf., Association for ComputingMachinery, New York, 790–796.

Baluja, S. 1994. Population-based incremental learning: A methodfor integrating genetic search based function optimization andcompetitive learning. Technical Report CMU-CS-94-163, Schoolof Computer Science, Carnegie Mellon University, Pittsburgh.

Bechhofer, R. E., T. J. Santner, D. M. Goldsman. 1995. Design andAnalysis of Experiments for Statistical Selection, Screening, andMultiple Comparisons. John Wiley & Sons, New York.

Branke, J., S. E. Chick, C. Schmidt. 2007. Selecting a selection pro-cedure. Management Sci. 53 1916–1932.

Chambers, L. 1995. Practical Handbook of Genetic Algorithms. CRCPress, Boca Raton, FL.

Chen, C. H. 1996. A lower bound for the correct subset-selectionprobability and its application to discrete event system simu-lations. IEEE Trans. Automatic Control 41 1227–1231.

Chen, C. H., D. He, M. Fu. 2006. Efficient dynamic simulation allo-cation in ordinal optimization. IEEE Trans. Automatic Control 512005–2009.

Chen, C. H., J. Lin, E. Yücesan, S. E. Chick. 2000. Simulation bud-get allocation for further enhancing the efficiency of ordinaloptimization. J. Discrete Event Dynam. Systems: Theory Appl. 10251–270.

Chen, H. C., C. H. Chen, L. Dai, E. Yücesan. 1997. New develop-ment of optimal computing budget allocation for discrete eventsimulation. Proc. 1997 Winter Simulation Conf., Association forComputing Machinery, New York, 334–341.

Chick, S., K. Inoue. 2001a. New two-stage and sequential proce-dures for selecting the best simulated system. Oper. Res. 491609–1624.

Chick, S., K. Inoue. 2001b. New procedures to select the best sim-ulated system using common random numbers. ManagementSci. 47 1133–1149.

DeGroot, M. H. 1970. Optimal Statistical Decisions. McGraw-Hill,New York.

Dudewicz, E. J., S. R. Dalal. 1975. Allocation of observations in rank-ing and selection with unequal variances. Sankhya 37 28–78.

Fu, M. C., J. Hu, S. I. Marcus. 2006. Model-based randomized meth-ods for global optimization. Proc. 17th Internat. Sympos. Math.Theory Networks Systems, Kyoto, Japan, 355–363.

Fu, M. C., J. Q. Hu, C. H. Chen, X. Xiong. 2007. Simulation alloca-tion for determining the best design in the presence of corre-lated sampling. INFORMS J. Comput. 19 101–111.

Goldsman, D., B. L. Nelson. 1998. Comparing systems via simu-lation. J. Banks, ed. Handbook of Simulation: Principles, Method-ology, Advances, Applications, and Practice. John Wiley & Sons,New York, 273–306.

Gupta, S. S. 1965. On some multiple decision (selection and rank-ing) rules. Technometrics 7 225–245.

He, D., S. E. Chick, C. H. Chen. 2007. Opportunity cost and OCBAselection procedures in ordinal optimization for a fixed numberof alternative systems. IEEE Trans. Systems, Man, Cybernetics—Part C 37 951–961.

Holland, J. H. 1975. Adaptation in Natural and Artificial Systems. TheUniversity of Michigan Press, Ann Arbor.

Hu, J., M. C. Fu, S. I. Marcus. 2007a. A model reference adap-tive search algorithm for global optimization. Oper. Res. 55549–568.

Hu, J., M. C. Fu, S. I. Marcus. 2007b. A model reference adaptivesearch algorithm for stochastic optimization with applicationsto Markov decision processes. Proc. 46th IEEE Conf. DecisionControl. IEEE Press, Washington, D.C., 975–980.

Inoue, K., S. E. Chick. 1998. Comparison of Bayesian and frequen-tist assessments of uncertainty for selecting the best system.Proc. 1998 Winter Simulation Conf., Institute of Electrical andElectronic Engineers, Piscataway, NJ, 727–734.

Inoue, K., S. E. Chick, C. H. Chen. 1999. An empirical evaluationof several methods to select the best system. ACM TransitionsModel. Comput. Simulation 9 381–407.

Kim, S.-H., B. L. Nelson. 2006. Selecting the best system. S.G. Henderson, B. L. Nelson, eds. Handbooks in OperationsResearch and Management Science: Simulation, Chapter 18. Else-vier, New York.

Koenig, L. W., A. M. Law. 1985. A procedure for selecting a sub-set of size m containing the l best of k independent normalpopulations. Comm. Statist.—Simulation Comm. 14 719–734.

Law, A. M., W. D. Kelton. 2000. Simulation Modeling Analysis.McGraw-Hill, New York.

Lee, L. H., E. P. Chew, S. Y. Teng, D. Goldsman. 2004. Opti-mal computing budget allocation for multi-objective simula-tion models. Proc. 2004 Winter Simulation Conf., Association forComputing Machinery, New York, 586–594.

Rinott, Y. 1978. On two-stage selection procedures and related prob-ability inequalities. Comm. Statist. 7 799–811.

Rubinstein, R. Y., D. P. Kroese. 2004. The Cross-Entropy Method:A Unified Approach to Combinatorial Optimization, Monte-CarloSimulation, and Machine Learning. Springer, New York.

Rudlof, S., M. Köppen. 1996. Stochastic hill climbing with learn-ing by vectors of normal distributions. Proc. First OnlineWorkshop on Soft Comput., Nagoya University, Nagoya, Japan,60–70.

Sullivan, D. W., J. R. Wilson. 1989. Restricted subset selection pro-cedures for simulation. Oper. Res. 37 52–71.

Swisher, J. R., S. H. Jacobson, E. Yücesan. 2003. Discrete-eventsimulation optimization using ranking, selection, and multiplecomparison procedures: A survey. ACM Trans. Model. Comput.Simulation 13 134–154.

Trailovic, L., L. Y. Pao. 2004. Computing budget allocation for effi-cient ranking and selection of variances with application totarget tracking algorithms. IEEE Trans. Automatic Control 4958–67.

Walker, R. C. 1999. Introduction to Mathematical Programming. Pren-tice Hall, Upper Saddle River, NJ.

Dow

nloa

ded

from

info

rms.

org

by [

128.

112.

66.6

6] o

n 05

Feb

ruar

y 20

16, a

t 19:

07 .

For

pers

onal

use

onl

y, a

ll ri

ghts

res

erve

d.

efficient simulation budget allocation for selecting …...informs journal on computing vol. 20, no....

Documents