Jeffrey Hyman’s Summer Research at UCLA

J. D. Hyman∗

August 18, 2006

Abstract

We examine three Monte Carlo sampling methods on idealized examples. Im-

plementing the Metropolis algorithm on created functions, we compare the single

temperature to four temperatures running in parallel. Using simulated annealing

and parallel tempering on the traveling salesman problem, we examine what each

of the methods is capable of exploring. We focus specifically on parallel tempering’s

ability to find local minima. Presenting a basic algorithm for temperature set selec-

tion in parallel tempering, I conclude with ideas for further research and a personal

critique of my summer research at UCLA.

∗St. Olaf College. Research partially supported by the REU program of the UCLA RTG, NSF grant

DMS-0601395

1

1 Introduction

The methods known as Monte Carlo, gatherrough estimates through sampling, and werenamed Monte Carlo by scientist at Los AlamosNational Laboratory in the 1940’s, becauseof their reliance on probability and resem-blance to many games of chance. The exis-tence of similar methods can be seen as farback as Buffon’s needle experiment that ar-rived at an estimate of π through randomsampling. The power of Monte Carlo comesfrom its ability to sample large regions withlittle computational force.1.

Monte Carlo methods have been usedconsistently since their development and oneparticular method, the Metropolis Algorithm2, has become a standard. Many other MonteCarlo methods are based upon the Metropo-lis Algorithm, including the two which wefocus on, simulated annealing and paralleltempering. Our goal is to focus on these twoparticular methods and investigate what theyare capable of doing.

We begin sampling on idealized exam-ples to work on our methods. Then we ap-ply the methods to a well known problem,the traveling salesman problem(TSP).

Our preliminary results show that thetwo methods are capable exploring differentparts of a problem. Simulated annealing isa tool of optimization and can aid in findingthe global minimum of a tour. While, moreinterestingly, parallel tempering is able toexpose the energy landscape of the tour.

During our study we realized the impor-tance of temperature sets in parallel tem-pering, and in order to expedite our workcreated a simple algorithm for the selectionof temperatures. The method is feedbackoriented and relies on educated guesses be-

1Kalos and Whitlock2Outlined in detail : Section 2

fore attempting a temperature set.

2 Metropolis

The Metropolis Algorithm for advanced sam-pling was first presented in a paper by Metropo-lis, Rosenbluth, Rosenbluth, Teller, and Tellerin 1953.3 The algorithm is often refereedto as the Metropolis algorithm, due to hisname being first on the paper.

The method is a Monte Carlo sweep methodthat randomly samples moves and acceptsall moves that have a lower energy state,but also accepts a move to an increased en-ergy state with a certain probability. Thatprobability is

P (x → y) = e−∆E(y−x)/T . (1)

Where ∆E(y−x) is the difference in en-ergy between the original point and the lo-cation of the new move. T is the temper-ature is used to control how big of uphillmoves are accepted. Lower temperaturesaccept smaller moves, and larger tempera-tures accept greater ones. The power of thisalgorithm is found in its acceptance of somemoves to an increased lever of energy. Byaccepting these uphill moves, the samplingmay escape from local minima.

An extensive discussion of the Metropo-lis algorithm can be found in Monte CarloMethods, Volume I: Basics By Kalos andWhitlock.

3 Simulated Annealing

The process of simulated annealing was cre-ated by S. Kirkpatrick, C. D. Gelatt, and

3Equation of State Calculations by Fast Com-puting Machines, Jour. Chem. Phys. June 1953

2

M.P. Vecchi4. Their process is based onthe Metropolis algorithm where the temper-ature is slowly decreased to allow smallerand smaller moves. Figure 1 shows an aMetropolis sample being annealed. Noticehow the width sampled is slowly decreasedas the number of iterations increased. Thischange is the result of the temperature de-creasing which allow for smaller uphill moves.

Kirkpatrick and colleagues found thatsimulated annealing could be used as a op-timization method where,due to the size ofthe problem, standard methods failed. Theprocess is able to sample large regions atthe beginning of a run, and slowly movesinto minimum as the temperature decreases.It hopes to retain a large enough tempera-ture to exit local minimum but by the timethe global minimum arises it cannot escape.The downfall of the method is its inabilityto confirm that it has found the optimal so-lution. It can only confirm that its has thebest local minimum of the minima sampled.

4 Parallel Tempering

Parallel Tempering might be thought of asrunning multiple Metropolis samples simul-taneously, and every so often a swap of twotemperatures is attempted. This swap be-tween two temperatures, exchanges part ofthe information between the two plots. Theadvantage of this swapping between tem-peratures comes in two different fashions.First, if a lower temperature in the set is ina local minimum for which it cannot escape,it may swap positions with a higher tem-perature and be placed in another area toexplore. That high temperature can escapethe local minimum and continue its sam-

4Optimization by Simulated AnnealingScience, May 13 1983

pling. Another advantage gained by paral-lel tempering is its lower computation timethan simulated annealing. Simulated an-nealing has to check all the temperatures in-between its highest to lowest temperature,while parallel tempering only has the num-ber of temperatures assigned.

The swaps between temperatures occurwith a probability that looks similar to theMetropolis method’s acceptance of uphill moves.

Let tn and tm be neighboring tempera-tures in a parallel tempering swapping set,where tn < tm. Given that a swap betweenthe temperatures is attempted the accep-tance of the swap is defined by the prob-ability

P (accepts swap) = min{1, e(∆B∗∆E)} (2)

Where ∆B = (1/tm − 1/tn) and ∆E =E(tkm) − E(tkn). Where E(tk) is the energyof the temperature at time k.

Figure 2 shows two temperatures swap-ping, and just below 500 iterations they moveback and forth between two tour distancesuntil one samples out of its minima.

Earl and Deem discuss the parallel tem-pering origins and possible future applica-tions with great vigor 5. The full potentialof parallel tempering has yet to be tapped,along with its wide range of possible appli-cations including work with protein folding,and polymers.

5 Implementation on Ide-

alized examples

To begin our analysis we make a function toimplement our methods on. The function

5Earl, D.J., Deem, M.W. Parallel Temper-ing:Theory, Applications, and New PerspectivesJan 4, 2006, Physical Chemistry Chemical Physics

3

we created has multiple minima, so that wecould experiment with uphill acceptance ofmoves and swapping between temperaturesin parallel tempering.

5.1 Parallel Tempering and En-ergy Functions

We created a function and ran both a singletemperature Metropolis method and a fourtemperature parallel tempering method. Thefunction f(x) is defined as

g(x) =

1 + sin(2π ∗ x):−2 ≤ x < −1.25

2(1 + sin(2π ∗ x)):−1.25 ≤ x < −.253(1 + sin(2π ∗ x)):−.25 ≤ x < .754(1 + sin(2π ∗ x)):.75 ≤ x < 1.755(1 + sin(2π ∗ x)):.75 ≤ x ≤ 2

f(x) = −g(x) + 9 (3)

Figure 3 shows f(x) graphed over −2 <x < 2.

We run a single temperature Metropolissample on f(x) and figure 4 shows a his-togram of the sampling taking place. Someof the minima are exposed, but other partsof the graph are completely unexplored.

Next we apply parallel tempering withfour temperatures swapping. Figure 5 is ahistogram of the same temperature as theMetropolis but with parallel tempering alsotaking place. The peeks of the histogramcorrespond to the the valleys of the graph,and the temperature was able to exploremultiple minimum and exposes the energylandscape of the function.

Parallel tempering not only shows theglobal minimum but also shows the locationof the local minimum in a energy system.

6 TSP

A salesman wants to find the shortestdistance for a trip he has to take to N

different cities. He wishes to visit each cityonly once and end where he began. Howdoes he go about finding the optimal tour

distance for this traveling salesman?

The traveling salesman problem(TSP) hasbecome a classic in the world of computa-tional mathematics and has been studiedextensively. Due to this wealth of informa-tion concerning the problem, it is a perfectsubject to test new methods on. Compar-ing the solutions of the new methods to theexisting data allows for the accuracy of themethod to be gauged.

We choose to apply simulated anneal-ing and parallel tempering on the TSP be-cause it is a problem for which Monte Carlosweeps and the Metropolis algorithm are ap-plicable.

We created a tour of 100 cities with Mat-lab’s pseudo-random generator, where wepaired up i and i+1 as x and y coordinatesto plot the cities on a unit square. AllonPercus6 found the optimal tour lengthn forour particular city set to be 7.064.

6.1 Nearest Neighbor Heuris-tic Method

The Nearest Neighbor Heuristic Method isa construction heuristic method that helpsto get a tour within 20 percent of its op-timal tour length. The method is basedon a greedy algorithm where only down hillmoves are accepted. We start with selectingone city and calculate the distance from it

6Dr. Percus was our resident expert on the TSPand should be credited with greatly increasing thepace of the project.

4

to all other cities. Then select the shortestdistance as the next city in the tour. Theprocess repeats itself which each new cityin the tour, but only checks the distance tounaffiliated cities. The method results ina tour distance with many good moves atthe beginning and a few terrible distancesat the end, due to limited cities available.Figure 6 shows our cities with the randomtour, and Figure 7 shows our cities withthe nearest neighbor construction and peri-odic boundary conditions implemented. Be-fore sampling with either simulated anneal-ing or parallel tempering, we use the nearestneighbor heuristic to rearrange our city set,to be closer to the optimal tour.

6.2 Simulated Annealing

Applying simulated annealing to the TSPas a optimization method was done by Kirt-patrick, and has been a common use of themethod ever sense. Our examination of sim-ulated annealing on the TSP was short livedand over-shadowed by work with paralleltempering, due to the interesting new re-sults parallel tempering exposed.

6.3 Parallel Tempering

We choose to run parallel tempering on theTSP with certain variables fixed and certainones as what we would experiment chang-ing. Fixed Variables: Number of cities =100, Number of cites swapped per move 1pair. Our variables were tmax, tmin, Spac-ing of temperatures(linear, geometric) andthe number of iterations between attemptedtemperature swaps. After running the near-est neighbor heuristic construction we letthe temperatures attempt any move for thefirst 100 iterations so that the tours wouldbe different.

In our version of the TSP a move con-sisted of randomly selecting two cities in thecity set and swapping their place in the or-der of the set.

The first three non-fixed parameters arediscussed in section 6.3.1; in our trials chang-ing the number of moves between swaps wetested: every iteration, every ten iterations,and every 1000 iterations. However, oneshould note that just because a swap is at-tempted it does not mean that one is com-pleted. Figure 8, 9, 10 are all geometri-cally spaced samples running for 1000 itera-tions, with temperatures in the key. Figure8 found a best tour distance of 8.0329. Fig-ure 9 found a best tour distance of 8.0144.Figure 10 found a best tour distance of 8.015.

From these figures one sees how attempt-ing swaps often can pull lower temperatureout of minima before they are able to ex-plore. Attempting swaps not often enoughcauses the system to lose the power of par-allel tempering.

A major issue involved progress in theproblem is finding a temperature set thatoptimizes exploration and is discussed in sec-tion 6.3.1.Often times a low temperature getsstuck in a minimum and is swapped outof the location, however, that low temper-ature leaves a trail that it was stuck for awhile7. The swapping temperatures movedownhill gradually, and thus the higher tem-peratures are able to get close enough tostuck low ones that a swap will be exceptedand progress may continue. Starting with anearest neighbor construction we were ableto make progress towards the optimal tourdistance. Our best local minimum of 7.6952was found using a set of five temperaturesof .01,.012,.015,.017,.02 run over 50,000 it-

7This is explored further in section 6.3.2

5

erations.

6.3.1 Comparing Temperature Sets

A major influence on the performance ofparallel tempering on the TSP comes fromthe selection of the temperature sets. We settmax = .02 and tmin = .01 and attempt tem-perature swap every 10 iterations. We chosetwo ways to selected the temperature, firstgeometrically and the second linearly. Fig-ure 11 shows the geometrically selected tem-peratures, which find a minimum of 7.9946after 10,000 iterations. Figure 12 shows thelinearly selected set, which found a mini-mum of 8.0534 over 10,000 iterations.

In selection of temperature sets, focusshould primarily be on tmin and tmax. tmax

should be large enough to escape from anylocal minimum, but not so large that it un-does the tour distance gained by the near-est neighbor construction. We set an up-per bound on tmax at .5101. Our reason forthis selection was given periodic boundaryconditions the maximum distance betweenany two cities is .7071 and the minimum is0, therefore the average distance is .3536.Solving equation 1 for probability of accept-ing a move to equal .5 where ∆E was .3536,we received .5101 to be the largest temper-ature we would need.

On the other hand tmin needs to be smallenough so that it could not escape most lo-cal minimum but would still be able to ex-plore them. Which leads us to the next sec-tion, Local Minima exploration.

6.3.2 Local Minima Exploration

Similar to our idealized examples, paralleltempering on the TSP is able to find localminima. Creating a histogram of the lowertemperatures we are able to see the loca-tions of the local minima. Figure 13 and

figure 14 are corresponding graphs to show-ing three distinct minima. Figure 14 showsthree distinct clusters of samples at ≈ 7.75,≈ 7.9 and ≈ 8.08 for tour distance. Theseclusters correspond to Three local minimashown in figure 13 that the lowest tempera-ture get stuck in.

7 Temperature Selection

Algorithm

A major problem that arose during imple-mentation of parallel tempering on the TSPwas being able to control how the tempera-tures interacted with one another. In partic-ular, the probability that given an attemptfor a swap occurs, that the swap will occur.This probability relies upon the tempera-ture set. An algorithm that, given certaininformation about a problem, would selectother temperatures was created. It is a verystraight forward approach, solving equation2 for tn. It is described in terms of the TSP,but may be applied to other problems aswell.

Every set of cities has a particular tem-perature set that optimizes sampling. Onceone has a run a few random temperaturesand found a range that does not get im-mediately stuck, nor climbs away, they mayinput certain parameters into the algorithmand gain temperature sets. These param-eters are: the maximum distance betweentemperatures for which a swap of temper-atures wishes t occur, the probability thata swap occurs, and a temperature at a tailends of the set, either tmin or tmax.

One should make sure that the compu-tation of the sampling only attempts a swapwith one pair of temperatures for a given at-tempt of swapping, else the algorithm doesnot work. The failure occurs because if t1

6

and t2 swap and then an attempt of swap-ping t2 and t3 occurs, the difference of dis-tance between t2 and t3 is no longer theoriginal distance; rather, it is the distancebetween t1 and t3 and the probability of ac-cepting the swap is decreased.

1 Choose tmin, (call this t1) and find av-erage distance for t1 call this (dt1).

2 Choose a ∆D that is the largest differ-ence in distances between dt1 and dt2

for which you wish to accept a swapof temperatures. dt2 is the average dis-tance for t2, Note dt2 = dt1 + ∆D.

3 Define ∆B = 1/t2 − 1/t1 and ∆E =(dt1 + ∆D)− dt1 .

4 Let S1 be the event that the opportu-nity for a swap between t1 and t2 oc-curs. Let R1 be the event that a swapof temperature t1 and t2 occurs. De-fine P1 as the probability that a swapoccurs between t1 and t2 given an at-tempt occurs.

P1 = P (R1|S1)

Pick a value for P1 which is the prob-ability you wish to accept the givenswap.

5

P1 =e(∆B∗∆E)

P1 =e(1/t2−1/t1)∗((dt1+∆D)−dt1 )

1/t2 =ln(P1)/∆D + 1/t1

Note that the right side of the equa-tion has a numeric value given our pa-rameter choices. Solve this equation

for t2 and we have our second temper-ature.

6 Once we have t2, implement these tem-peratures in parallel tempering. Findthe average distance for t2 when it isswapping with t1, then restart the al-gorithm to find the rest of your tem-peratures.

This algorithm may also be used startingat a given Tmax and working backwards tolower temps. Note that for the larger tempsone will want a larger ∆D because the aver-age distance between temperatures is larger.

8 Summary and Further

Research

From my work this summer I found that thismost influential parameter in parallel tem-pering is the probability that given a swapis attempted, the event will occur. It is fairto say this parameter controls parallel tem-pering. If the probability of swapping out ofa energy location is very high, the tempera-tures will never have time to explore. Con-versely if the probability of swapping tem-peratures is too low, the power of paralleltempering is lost because swapping rarelyoccurs. Further research ought be done tofind the optimal probability of swapping.

Simulated annealing was used, but ourfocus was primarily on parallel tempering,and thus no new information was found con-cerning simulated annealing. However, if agood temperature set was found for paral-lel tempering, one could create an annealingprocess that mimics parallel tempering andcompare the two as optimization methods.

Using parallel tempering to explore thedistribution of the local minima is clearly

7

the next step that this research should take.Support for the effectiveness of this methodcomes from both our trials on functions andthe TSP. The ability to expose energy land-scapes carries great potential and ought beexplored.

However, the process of parallel temper-ing is dependent upon finding a optimal tem-perature for any set of cities. In taking astep back and working on a good algorithmfor temperature selection, one finds a prob-lem that has many more applications thanthe TSP. To my knowledge there is no uni-versal system for temperature set selectionin parallel tempering. The lack of such asystem could be due to the differences ofthe problems that parallel tempering can beapplied.

Another direction that this research couldlead would be using simulated annealing witha fast cooling rate to freeze the system ina local minimum. This fast paced anneal-ing, might also expose the energy landscapeof the tour. An idea I had which I barelyput into practice, was a combination par-allel tempering simulated annealing processwhere the temperature set in parallel tem-pering would also be annealed. The valueof this would be the temperatures would al-ways be moving down and yet keep theirconfigurations with one another, and stillhave the power of swapping. Clearly theentire temperature set would have to starthigher, but it would be able to explore morevalleys fuller and still be able to get out ofthe minima.

9 Personal Critique

I am including a section called personal cri-tique both for my own good as part of reflec-tion and to acknowledge those who aided in

this summer’s research. I arrived at UCLAnot sure as to what my project would be.I knew that I had some preparation in thesubject matter from course work at St. OlafCollege8, but was unsure of the direction orthe circumstances of the research. I hadbeen sent a couple of papers, and a title.I was fortunate to have seemingly unlim-ited resources in both of my mentors, RussCaflisch and Richard Wang, and to haveconnected with Allon Percus who aided myproject in in-numerous ways during the timemy mentors were away on travel.

I am unsure of how much other under-graduate research typically covers in one sum-mer but I feel like our project covered a lotof ground. This expedient work was the re-sult of pre-arrival preparation, luck in find-ing resources, and an open direction of theproject. I arrived every morning having aidea to try out and leaving in the afternoonwith twenty more to try tomorrow. Butmore importantly I had a wonderful groupthat encouraged me to continue exploringmy ideas. Occasionally I would create a tan-gent for myself, but my mentors would checkin and reset me on course. This research hashelped me personally to understand what acareer in mathematics might be like. Mydeepest thanks to Russ Caflisch, RichardWang, Allon Percus, Andrew Bernoff, JonAzose, James M. Hyman, Matt Richey IPAM,NSF, and UCLA for their aid in preparationand execution of this summer REU. Cheers!

List of Figures

1 A Metropolis algorithm withsimulated annealing . . . . . . 9

2 Two Metropolis samples withswapping occurring . . . . . . 9

8Matt Richey in particular

8

3 f(x) . . . . . . . . . . . . . . . 94 A histogram of a single tem-

perature sampling f(x) . . . . 95 A histogram of a single tem-

perature sampling f(x), whileswapping with 3 other tem-peratures running in parallel . 10

6 A tour of 100 cities withoutthe nearest Neighbor Heuris-tic method . . . . . . . . . . . 10

7 A tour of 100 cities with thenearest Neighbor Heuristic Methodand Periodic boundary con-ditions . . . . . . . . . . . . . 10

8 Parallel tempering on the TSPattempting swaps every iter-ation . . . . . . . . . . . . . . 10

9 Parallel tempering on the TSPattempting swaps every 10 it-erations . . . . . . . . . . . . 11

10 Parallel tempering on the TSPattempting swaps every 100iterations . . . . . . . . . . . 11

11 Parallel Tempering with fivetemperatures selected Geomet-rically run for 10,000 Iterations 11

12 Parallel Tempering with fivetemperatures selected linearlyrun for 10,000 Iterations . . . 11

13 Parallel Tempering with fivetemperatures used to show howparallel tempering can explorelocal minima . . . . . . . . . . 12

14 A histogram of the tempera-ture .0075 from figure ?? ex-posing local minima . . . . . . 12

0 500 1000 1500 2000 2500 3000 3500 4000 4500 50001

2

3

4

5

6

7

8

9

10

11

Iterations

Ener

gy

Cooling Rate 0.97

Figure 1: A Metropolis algorithm with sim-ulated annealing

0 100 200 300 400 500 600 700 800 900 10007.5

8

8.5

9

9.5

10

10.5

Temp1Temp2

Figure 2: Two Metropolis samples withswapping occurring

9

0 50 100 150 200 250 300 350 4001

2

3

4

5

6

7

8

9

Figure 3: f(x)

!2 !1.5 !1 !0.5 0 0.5 1 1.5 20

20

40

60

80

100

120

140

160

180

Figure 4: A histogram of a single tempera-ture sampling f(x)

!2 !1.5 !1 !0.5 0 0.5 1 1.5 20

10

20

30

40

50

60

70

Figure 5: A histogram of a single temper-ature sampling f(x), while swapping with 3other temperatures running in parallel

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1The roundtrip length for 100 cities is 34.956393 units

Figure 6: A tour of 100 cities without thenearest Neighbor Heuristic method

10

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1The roundtrip length for 100 cities is 8.032884 units

Figure 7: A tour of 100 cities with the near-est Neighbor Heuristic Method and Periodicboundary conditions

0 100 200 300 400 500 600 700 800 900 10008

8.5

9

9.5

10

10.5

Iterations

Tour

Dist

ance

0.010.01250.0150.01750.02

Figure 8: Parallel tempering on the TSPattempting swaps every iteration

0 100 200 300 400 500 600 700 800 900 10008

8.5

9

9.5

10

10.5

Iterations

Tour

Dist

ance

0.010.01250.0150.01750.02

Figure 9: Parallel tempering on the TSPattempting swaps every 10 iterations

0 100 200 300 400 500 600 700 800 900 10008

8.5

9

9.5

10

10.5

Iterations

Tour

Dist

ance

0.010.01250.0150.01750.02

Figure 10: Parallel tempering on the TSPattempting swaps every 100 iterations

11

0 1000 2000 3000 4000 5000 6000 7000 8000 9000 100007.5

8

8.5

9

9.5

10

10.5

11

Iterations

Tour

Dist

ance

0.010.01120.01250.0150.02

Figure 11: Parallel Tempering with fivetemperatures selected Geometrically run for10,000 Iterations

0 1000 2000 3000 4000 5000 6000 7000 8000 9000 100008

8.5

9

9.5

10

10.5

11

Iterations

Tour

Dist

ance

0.010.01250.0150.01750.02

Figure 12: Parallel Tempering with fivetemperatures selected linearly run for10,000 Iterations

0 1000 2000 3000 4000 5000 6000 7000 8000 9000 100007.5

8

8.5

9

9.5

10

10.5

Iterations

Tour

Dist

ance

0.010.0120.0150.0170.02

Figure 13: Parallel Tempering with fivetemperatures used to show how paralleltempering can explore local minima

7.5 7.6 7.7 7.8 7.9 8 8.1 8.2 8.3 8.4 8.50

200

400

600

800

1000

1200

1400

1600

Tour Distance

Num

ber o

f Sam

ples

Figure 14: A histogram of the temperature.0075 from figure 13 exposing local minima

12