13 May 1983, Volume 220, Number 4598

Optimization bySimulated Annealing

S. Kirkpatrick, C. D. Gelatt, Jr., M. P. Vecchi

In this article we briefly review thecentral constructs in combinatorial opti-mization and in statistical mechanics andthen develop the similarities between thetwo fields. We show how the Metropolisalgorithm for approximate numericalsimulation of the behavior of a many-body system at a finite temperature pro-vides a natural tool for bringing the tech-niques of statistical mechanics to bear onoptimization.

involving a few hundred cities or less.The traveling salesman belongs to the’.:l,,“‘,large class of NP-complete (nondeter- ‘_‘:Iministic polynomial t ime complet&$problems, which has received extensive <Cl :study in the past 10 years (3). No method ,“,?:,for exact solution with a computing ef- ,‘,f.,fort bounded by a power of N has been 1”found for any of these problems, but if ‘,:such a solution were found, it could bemapped into a procedure for solving allmembers of the class. It is not known

sure of the “goodness” of some complex what features of the individual problemssystem. The cost function depends on in the NP-complete class are the cause ofthe detailed configuration of the many their difficulty.parts of that system. We are most famil- Since the NP-complete class of prob-iar with optimization problems occurring lems contains many situations of practi-in the physical design of computers, so cal interest, heuristic methods have beenexamples used below are drawn from developed with computational require-

We have applied this point of view to anumber of problems arising in optimaldecign of computers. Applications topartitioning, component placement, andwiring of electronic systems are de-scribed in this article. In each context.we introduce the problem and discussthe improvements available from optimi-zation.

Of classic optimization problems, thetraveling salesman problem has receivedthe most intensive study. To test t h epower of simulated annealing. we usedthe algorithm on traveling s a l e s m a nproblems with as many as several thou-sand cities. This work is described in afinal section, followed by our cvnclu-sions.

Combinatorial Optimization

The subject of combinatorial optimiza-tion (I) consists of a set of problems thatare central to the disciplines of computerscience and engineering. Kesearch in thisarea aims at developing efficient tech-niques for finding minimum or maximumvalues of a function of very many inde-pendent variables (2). This function, usu-ally called the cost function or objectivefunction, represents a quantitative mea-13 MAY 1983

with N, so that in practice exact solu- ”

Summary. There is a deep and useful connection between statistical mechanics(the behavior of systems with many degrees of freedom in thermal equilibrium at afinite temperature) and multivariate or combinatorial optimization (finding the mini-mum of a given function depending on many parameters). A detailed analogy withannealing tn solids provides a framework for optimization of the properties of verylarge and complex systems. This connection to statistical mechanics exposes newinformation and provides an unfamiliar perspective on traditional optimization prob-lems and methods.

that context. The number of variablesinvolved may range up into the tens ofthousands.

The classic example, because it IS s osimply stated. of a combinatorial optimi-zation problem is the traveling salesmanproblem. Given a list of .V cities and ameans of calculating the cost of travelingbetueen any two citie\. one must planthe salesman’s route. which will passthrough each city once and return finallyto the starting point, minimizing the totalcost. Problems with this flavor arise inall areas of scheduling and design. T w osubsidiary problems are of general inter-est: predicting the expected cost of thesalesman’s optimal route. averaged oversome class of typical arrangements ofcities, and estimating or obtainingbounds for the computing effort neces-sary to determine that route.

All exact methods known for deter-mining an optimal route require a com-puting effort that increases exponentially

ments proportional to small powers of N.Heuristics are rather problem-specific:there is no guarantee that a heuristicprocedure for finding near-optimal solu-tions f9r one NP-complete problem willbe effective for another.

There are two basic strategies forheuristics: “divide-and-conquer” and it-erative improvement. In the first, onedivides the problem into subproblems ofmanageable size. then solves the sub-problems. The solutions to the subprob-lems must then be patched back togeth-er. For this method to produce very goodsolutions, the subproblems must be natu-rally disjoint, and the division made mustbe an appropriate one, so that errorsmade in patching do not offset the gains

S. KirkpatrIck and C. D. Gelatt, Jr, are researchstaff member\ and M. P. Vecchi was a visttingxientist at IBM Thomas J. Watson Research Cen-ter, Yorktown Heights. New York 10598. M. P.Vecchi’\ present address i5 Institute Venerolano deInvesttgaclones Cientificas. Ca racas IOIOA. Vene-LUCki.

671