stat 906: computer intensive methods in finance don mcleish: mc6138 email:[email protected]...

56
STAT 906: Computer Intensive Methods In Finance • Don McLeish: MC6138 email:[email protected] •Text: Monte Carlo Simulation and Finance. Don L. McLeish (Wiley, 2005) Available from me @60% of $119. Early version of book and slides available at www.mcleish.ca (follow links)

Upload: amie-sims

Post on 17-Dec-2015

221 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

STAT 906: Computer Intensive Methods In Finance• Don McLeish: MC6138

email:[email protected]

•Text: Monte Carlo Simulation and Finance. Don L. McLeish (Wiley, 2005) Available from me @60% of $119. Early version of book and slides available at •www.mcleish.ca (follow links)

Page 2: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

Other References

• The Mathematics of Financial Derivatives• (Wilmott, Howison, Dewynne) • Monte Carlo: Concepts, Algorithms and Applications.

George Fishman QA298.F57• Arbitrage Theory in Continuous Time. Tomas Bjork• Mathematics of Financial Markets. R.J. Elliott and P.E.

Kopp (Springer) HG4515.3.E37• Glasserman, P. (2003) Monte Carlo Methods in

Financial Engineering (Applications of Mathematics, 53) Springer, New York

• Jaeckel, P. (2003) Monte Carlo Methods in Finance, Wiley, New York

Page 3: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

Marking

•Marks distribution

– Midterm Tests (2): 35– Project&presentation (15 minutes per

presentation) 50 – Class problem presentations 15

•Tentative Calendar: Week of – Sept 13+15:– Sep 20+22 : – Sep 27+29 : – Oct. 4+6: – Oct 11+13: – Oct 18+20: – Oct 25: Midterm Test 1– Oct 27:– Nov 1 : – Nov 10 :– Nov 17: – Nov 24: – Nov 26: Midterm Test 2– Dec 1+3+5: Projects

Page 4: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

Contents

•A general discussion of simulation.•Example: 2 person game. •Some basic theory of finance (Chapter 2)•How to generate random numbers (Uniform and non-uniform)(Chapter 3)

•Variance Reduction•Quasi Monte Carlo•Simulation and Option valuation•Estimation and Calibration•Sensitivity Analysis•Numerical solutions to PDE’s

Page 5: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

Course Prerequisites

• Knowledge of (or tolerance of) Stochastic Calculus.

• Some finance knowledge/interest

• Computing: Some computer language (e.g. MATLAB, R, C++)

• “Tolerance of” means have taken course, seen before, or willing to read on side.

Page 6: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

Simulation: Monte Carlo Methods

•Simulation: Imitation of a real-world process or system. •Model: set of assumptions concerning the operation of the system•Monte Carlo: simulation of random phenomena

•Parameters: numerical values associated with the model•Why use simulation? •A cheap source of experience. E.g. Airbus pilots• Permits changing model and parameters, examining interactions, changes to system,

Page 7: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

Option

• Gives the holder the right (no obligation) to buy or sell an asset at a prescribed price,time determined by the contract.

• e.g. a (European) call option on IBM stock. exercise price $120. expiry date Oct 20, 2003. If S=stock price on Oct 20 the value of option on expiry is max(S-120,0)

Page 8: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

Uses of financial derivatives

• Speculate on a rise (call option) or fall (put option) in asset price.

• Hedge a portfolio already held-e.g. a promise to deliver IBM stock at point in future. Insurance against disadvantageous moves in asset prices, currency exchange rates, interest rates, credit changes, weather, electricity demand, …...

Page 9: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

Asset prices as random variables

• Random models often applied to complex systems which are essentially causal with many factors influencing result in a complex way. e.g. Brownian motion, toss of coin, currency exchange rates, …

• What are the constants---- a dollar and if so whose…..? Choice of “numeraire”.

Page 10: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

Why use simulation?

•Experience•Permits experimenting with the controllable system parameters to identify optimal settings•Permits examining effect of environmental or exogenous changes.

•Identify which of several systems is most efficient•Determine which variables are most important.•Verify and check robustness of analytic solutions.

Page 11: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

Advantages of Simulation

•investigate effects of changes, new designs, new models, etc. without costly implementation.•Stress testing: test systems under different scenarios (e.g. higher interest rates, different exchange rates)

•“What if” questions.•NON-FINANCE applications

– Identify bottlenecks in systems and rectify

– Increase experience with complex system at lower cost.

Page 12: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

Disadvantages

•Two models for same process may differ. •Simulation output is random so hard to interpret results. •Building models and running simulation is time consuming

•ANALYTIC SOLUTIONS, IF AVAILABLE, SHOULD BE USED! (analytic solutions to a similar (simpler) model can be used to improve a simulation)

Page 13: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

Non Finance Applications

•Manufacturing systems: e.g. material handling, inventory, assembly plants, scheduling, •Public Systems: health care- hospital management, emergency room, Military

•Natural resource management, transportation, traffic systems, airport (e.g. Motorway)•Construction systems: project planning, scheduling, •entertainment: restaurants, movies etc.

Page 14: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

Types of Systems

•State variables: determine the state of the system in order to simulate future events.•Discrete: state variables change only at discrete time points. (discrete event simulation)

•Continuous: state variables change continuously over time •Stochastic, deterministic

Page 15: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

Example (Discrete): Bank queue

•Interested in required number of tellers.•State variables: number of tellers available, number of customers in bank, (possibly also the time in service of customers being served)

Page 16: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

EXAMPLE: QUEUE

•Calling population•queue (waiting line)•server(s)

Page 17: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

One-server queue

Page 18: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

Examples of queueing systems

•Repair facility(garage, mechanics, etc.)•hospital, airport,production line, job shop, computer, telephone, ticket office, grocery store, taxi, mass transit, warehouse

•Identify the customers and the server(s) in each.

Page 19: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

Example (continuous-time model):TSE300 index.

•X(t) = TSE300 index at time t. Changes (almost) continuously in time•On what present values does the future depend? (state variables) : Current value of index

•possibly other factors depending on model (e.g. interest rates, exchange rates, balance of trade, agricultural prices, forest product prices, kitchen sink …)

Page 20: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

Steps in contructing a simulation

• Formulate Problem (why do a simulation?)

• Set objectives• suggest candidate

models.• Collect real data-

identify most appropriate model

• Computer

implementation of model (code or program)

• Verify model (i.e. debug)

• validate model (i.e. accurately represents system?)

Page 21: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

Designing Simulation

•Design the simulation- what alternatives are to be simulated? How long a run- what data is collected etc.•Run simulation and analyse output.

•More runs? •Change model? •Change parameters? •Document•report on results Conclusion

Page 22: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

Prisoner’s dilemna• A non-zero-sum game called the "Prisoner's Dilemma"• The two players choose between two moves, either "cooperate" or

"defect". • each player gains when both cooperate, but if only one of them cooperates,

the other one, who defects, will gain more. If both defect, both lose (or gain very little) but not as much as the "cheated" cooperator whose cooperation is not returned. The whole game situation and its different outcomes can be summarized below. Points=payoff to A

• Mathematical model of “plea-bargain”. The dilemma resides in the fact that each prisoner has a choice between only two options, but cannot make an optimal decision without knowing what the other one will do.

Page 23: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

Example: Simulate the Prisoner’s Dilemma.

•Payoff matrix•Prisoners Alf and Butch either confess or not. Payoffs taken from table: A chooses row (don’t confess/ confess) •B chooses column (don’t confess/confess)

•Payoff (to Alf)•cooperate: 5 -10•Defect 10 0•(to B)•cooperate defect• 5 10•-10 0

Page 24: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

Summary of game& simulation results.

A = 5 -10 10 0

B = 5 10 -10 0

>> [p,q,val]=nonzerosum(A,B,10000)p = 0 1q = 0 1val = 0 0“Defect”=second strategy dominates for both players

Page 25: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

TWO PERSON GAME-MATLAB CODE

• [p,q]=nonzerosum(A,B,nsim)• n=size(A); p=ones(1,n(1)); % q=ones(n(2),1);

• for i=1:nsim• [m,s]=max(A*q); p(s)=p(s)+1;• [m,s]=max(p*B); q(s)=q(s)+1;• end• p=p-ones(1,n(1)); p=p/sum(p);• q=q-ones(n(2),1); q=q/sum(q);

Page 26: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

Another two-person game.

•Payoff matrix•Two companies A and B each choose bid . Payoffs taken from table:

•Payoff to A•A’s Bid Payoffs•12 3 2 -2•13 1 -4 4•14 0 -5 6•Payoff to B•B’s bid B1 B2 B3• -1 -2 3• 0 4 -6• 0 5 -5

Page 27: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

OUTPUT

• [p,q]=nonzerosum(A,B,50000)

• p = 0.6667 0 0.3333

• q = 0 0.53334 0.46666

• and the value of the game

• 0 to A and 0.3333 to B.

Page 28: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

Generating random bid

• Generate a uniform [0,1] random variable.

• If U<2/3, then bid $12,

• otherwise bid $14.

Page 29: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

Basic Theory of Finance: The No Arbitrage Principle

• Can I Make money with positive probability with net investment of $0??( this is called an arbitrage)

Page 30: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

One stock, One bond, one period

• Assume stock presently worth $s will be worth su or sd after one month where d<1<u.

• A riskless bond, presently worth $1 is valued $(1+r) next month

• Buy x bonds and y shares of stock.• Value in 1 month =x(1+r)+ysu

or x(1+r)+ysd depending on whether stock goes up or down.

Page 31: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

Possible stock movements.

Page 32: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

Find an arbitrage

• Can you find values of (x,y) so that the net cost is $0 (e.g. sell short bonds and buy stocks) so that you are certain you will not lose money and you make a positive amount with positive prob. i.e. both x(1+r)+ysu>=0 and

• x(1+r)+ysd >=0 with either or both is >0.

• For net investment =0, need sy=-x.

Page 33: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

No point on 0-investment line in yellow region…no arbitrage.

Page 34: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

Condition for no arbitrage

• Arbitrage IS possible UNLESS

• d<1+r<u

• i.e. $1 in stock either returns more (u) or less (d) than bond. In this case, the bond return is a CONVEX linear combination of the stock returns.

• THAT IS 1+r=qu+(1-q)d for some 0<q<1.

Page 35: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

RISK NEUTRAL MEASURE

• Solve for q:

• q=(1+r-d)/(u-d)

• The distribution which assigns probability q to the stock rising to su and probability

1-q to the stock falling to sd is the risk neutral measure, denoted Q.

Page 36: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

Expected value of stock price under Q

If there is to be no arbitrage, there exists a probability measure Q such that the expected price of future value of the stock S1 discounted to the present using the return from a risk-free bond is exactly the present value of the stock.

Denote by Q the probability distribution which puts probabilities q and 1 − q on these points su, sd.

Page 37: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

Risk neutral measure

• consensus investors' attitude to risk avoidance. It is not necessarily true that the expected value of S1 under the actual probability distribution P describing the future probabilities of the stock is equal to s, only the expected value under Q.

Page 38: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

DerivativesFinancial Derivatives: investments which derive their value from that of a corresponding asset, such as a stock. A European call option is an option which permits you (but does not compel you) to purchase the stock at a fixed future date ( the maturity date) or for a given predetermined price, the exercise price of the option). Example: a call option with exercise price $10 on a stock whose future value is denoted S1, is worth on expiry S1 − 10 if S1 > 10, nothing if S1 < 10. The difference S1 − 10 between the value of the stock on expiry and the exercise price of the option is your profit if you exercises the option, purchasing the stock for $10 and sell it on the open market at $ S1. However, if S1 < 10, there is no point in exercising your option (you are not compelled to do so) .. return = $0.

Page 39: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

Replicating a Contingent Claim

A function of the stock price V (S1) which may represent the return from aportfolio of stocks and derivatives is called a contingent claim. V (S1) represents the payoff to an investor from a certain financial instrument or derivative when the stock price at the end of the period is S1.In a complete market, there is an investment solely in the stock and bond which reproduces these values V (su) and V (sd) exactly.

Solve for x and y•x(1+r)+ysu=V(su)•x(1+r)+ysd=V(sd)

r

suysuVx

sdsu

sdVsuVy

1

)(

)()(

obtain weSolving

**

*

Page 40: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

Investment in stock

• Replicating portfolio requires that investment in stock is slope of value function between (sd,su)

• A complete market is one in which every contingent claim (every function V(S1) of a stock price) can be replicated with using (marketable) stocks and risk-free bonds.

Page 41: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

Discounted Expected value of V(S1) under Q

Page 42: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

Recap

• Under the Q measure, the expected discounted-to-present value of any marketed asset (stock or contingent claim) is its present price.

• In a complete market, there is only one measure Q measure with this ``martingale’’ property.

Page 43: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

Multiperiod Models

• When an asset price evolves over time, decisions about an investment are made at various periods during the life of investment.

• Decisions made with the benefit of information including the price of the asset and any related assets at all previous time periods, beginning at some time t=0 when we started observing the process.

Page 44: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

The “History” sequence.

• When an asset price evolves over time, the investor normally makes decisions about the investment at various periods during its life. Such decisions are made with the benefit of current information, including the price of the asset and any related assets at all previous time periods, beginning at some time t = 0.

• Denote this information available for use at time t as Ht. • Ht is what is sigma-field generated by the past. • two fundamental properties of Ht:

1. Ht increases in t. (our information about this and related processes increases over time).

2. Ht includes the value of the asset price Su , for all u less than or equal to t. (St is adapted to or measurable with respect to Ht).

Page 45: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

In general, under Q, the discounted stock price is a

martingale

Page 46: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

CAPM AND PORTFOLIO SELECTION

1. toadd that sfor weight )','(

mean)deviation, (standard

of valuespossible all ofset heConsider t

')'(

')'(Then

matrix). a ( cov(R)

and vector)(a )(

thatassume We)'.,...,(

weightsof vector a )',...,(

returns (random) oftor column vec aConsider

1

1

www

wwRwVar

wRwE

RE

www

RRR

n

n

Page 47: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

Forms the interior of a region like this:

Page 48: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

The boundary of this region is the “efficient frontier”. The point with smallest standard deviation is the

most conservative investment

Page 49: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

The conservative portfolio

1

11

1

matrix) covariance inverse of sums row tonal(proportio

w

are weightsngcorrepondi and 1'1

1

is thissatisfying of valueMinimum

] []' [

where2 satisfies :Solution

mean) (fixed ' and

1' subject to

variance)portfolio(min ' min

1gg

1g

1- 1-

11122

2222

2

A

AAA

w

w

ww

Page 50: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

Add a risk-free bond paying r

(p, p)

Everything below this line L is now feasible

Any point on L can be achieved with a weighted average of the market (point p) and the risk-free bond

Page 51: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

The Market portfolio

i

i

1-

1-

return excess edstandardiz

return excess edstandardiz R

toalproportion sdweight

r1)-(

toalproportion

weightshas

r

wm

Page 52: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

Entropy

• Consider ascertaining the value of a random variable X with distribution

On average it will take 1+1(.5)=1.5 queries with yes/no answers to determine its value

For a discrete distribution, such that P[X = x] = p(x), the entropy may be defined to be

Page 53: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

Entropy depends on the probabilities p(x), not the values x.

So for example if g(.) is a one-one function, X and g(X) have the same entropy. The maximum entropy principle asserts that given whatever constraints on a distribution are present, distributions tend to maximize entropy. Examples:

•Distribution on [a,b] with maximum entropy is uniform.

•Distribution on [0, 1) with maximum entropy subject to E(X)=1 is exponential(1)

•Distribution on (-1,1) with maximum entropy subject to constraints E(X)=, var(X)=2 is Normal( , 2)

Page 54: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

Discrete analogue to NormalThe Lagrangian becomes

Settting the derivative with respect to p(x) equal to zero and solving we obtain a normal p.d.f.

Page 55: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

Maximum entropy dice

• Maximum entropy distribution on integers {2,3,…,12} subject to having mean 7,

variance 2 £ 35/12 is given below

Page 56: STAT 906: Computer Intensive Methods In Finance Don McLeish: MC6138 email:dlmcleis@uwaterloo.ca Text: Monte Carlo Simulation and Finance. Don L. McLeish

Sum of values on two dice compared with max entropy

distribution