computational physics adaptive, adaptive, multi...

Computational Physics

Feb 21, 2019

http://hadron.physics.fsu.edu/~eugenio/comphy/

[email protected]

Adaptive, Adaptive, Multi-Dimensional, & Multi-Dimensional, &

Monte Carlo IntegrationMonte Carlo Integration

Computational Physics

Feb 21, 2019

http://hadron.physics.fsu.edu/~eugenio/comphy/

[email protected]

Adaptive, Adaptive, Multi-Dimensional, & Multi-Dimensional, &

Monte Carlo IntegrationMonte Carlo Integration

Series Integration

f(x)

xxa xb

w=x b−xa/N

xa xb

f(x)

x

4 ∑k=0

N −1

f xaw k1/2

I N =w6 f xa f x b2 ∑

k=1

N −1

f x ak w

∫x a

x b

f x dx = I N O w4∫xa

xb

f x dx = I N O w2

2 =115∣I N 2

− I N 1∣

error on 2nd integration

2 =13∣I N 2

− I N 1∣


I N = w2∗ f x a f xb2∑

k=1

N −1

f xak w

trapezoidal rule Simpson's rulereview from last lecture

Series Integration

f(x)

xxa xb

w=x b−xa/N

xa xb

f(x)

x

4 ∑k=0

N −1

f xaw k1/2


k=1

N −1

f x ak w

∫x a

x b

f x dx = I N O w4∫xa

xb

f x dx = I N O w2

2 =115∣I N 2

− I N 1∣


2 =13∣I N 2

− I N 1∣


I N = w2∗ f x a f xb2∑

k=1

N −1

f xak w

trapezoidal rule Simpson's rulereview from last lecture

Adaptive IntegrationIntegration with just enough steps, N, to achieve the accuracy we want

Decide on the integration accuracy Evaluate the integral with a small number of steps N1

Then double the number N2 = 2N1, & evaluate integral again

Calculate the error on I2

use

If error is to within accuracy then we are finished,

otherwise repeat doubling of integration steps until desired accuracy is reached

ϵ2 = 13∣I 2−I 1∣trapezodial or ϵ2 = 1

15∣I 2−I 1∣Simpson

Adaptive IntegrationIntegration with just enough steps, N, to achieve the accuracy we want

Decide on the integration accuracy Evaluate the integral with a small number of steps N1

Then double the number N2 = 2N1, & evaluate integral again

Calculate the error on I2

use

If error is to within accuracy then we are finished,

otherwise repeat doubling of integration steps until desired accuracy is reached

ϵ2 = 13∣I 2−I 1∣trapezodial or ϵ2 = 1

15∣I 2−I 1∣Simpson

Adaptive IntegrationDoubling the number of integration steps with half the calculations

½ 1 1 1 ½

½ 1 1 1 1 1 1 1 ½

doubling effectively adds an additional set of points halfway between the previous points

I N = w 12

f xa12

f xb∑k=1

N −1

[ f x ak w ]


½ 1 1 1 ½

½ 1 1 1 1 1 1 1 ½


I N = w 12

f xa12

f xb∑k=1

N −1

[ f x ak w ]


½ 1 1 1 ½

½ 1 1 1 1 1 1 1 ½


adaptive trapezoidal rule integration(extra points are the odd points)

I N = w 12

f xa12

f xb∑k=1

N −1

[ f x ak w ]

I 2=12

I 1+ w2∗ ∑k odd

1... N 2−1

f ( xa+ k w2)

ϵ2 = 13∣I 2− I 1∣N 2=2N1 w2=

w1

2


½ 1 1 1 ½

½ 1 1 1 1 1 1 1 ½


adaptive trapezoidal rule integration(extra points are the odd points)

I N = w 12

f xa12

f xb∑k=1

N −1

[ f x ak w ]

I 2=12

I 1+ w2∗ ∑k odd

1... N 2−1

f ( xa+ k w2)

ϵ2 = 13∣I 2− I 1∣N 2=2N1 w2=

w1

2


½ 1 1 1 ½

½ 1 1 1 1 1 1 1 ½


I 2=12

I 1+ w2∗ ∑k odd

1... N 2−1

f ( xa+ k w2)

I 2 = 12

I 1 −13

w2 ∑k oddk=1

N 2−1

f (xa+ w2 k ) adaptive trapezoidal rule integration

(extra points are the odd points)

adaptive Simpson's rule integration (extra points are the new mid points minus odd endpoint point values)


k=1

N−1

f x ak w4 ∑k=0

N−1

f x awk1 /2

I N = w 12

f xa12

f xb∑k=1

N −1

[ f x ak w ]

ϵ2 = 13∣I 2− I 1∣ ϵ2 = 1

15∣I 2− I 1∣

+ 23

w2 ∑k=0

N 2−1

f (xa+ w2(k+ 1/2))

N 2=2N1 w2=w1

2N 2=2N1 w2=

w1

2


½ 1 1 1 ½

½ 1 1 1 1 1 1 1 ½


I 2=12

I 1+ w2∗ ∑k odd

1... N 2−1

f ( xa+ k w2)

I 2 = 12

I 1 −13

w2 ∑k oddk=1

N 2−1

f (xa+ w2 k ) adaptive trapezoidal rule integration

(extra points are the odd points)

adaptive Simpson's rule integration (extra points are the new mid points minus odd endpoint point values)


k=1

N−1

f x ak w4 ∑k=0

N−1

f x awk1 /2

I N = w 12

f xa12

f xb∑k=1

N −1

[ f x ak w ]

ϵ2 = 13∣I 2− I 1∣ ϵ2 = 1

15∣I 2− I 1∣

+ 23

w2 ∑k=0

N 2−1

f (xa+ w2(k+ 1/2))

N 2=2N1 w2=w1

2N 2=2N1 w2=

w1

2

Integrals over infinite ranges

∫0

∞

f x dx

Solve by changing variables: z= x1x

x= z1−z

& dx= dz1−z 2

∫0

∞

f x dx = ∫0

1 11−z 2 f z

1−z dz


∫0

∞

f x dx

Solve by changing variables: z= x1x

x= z1−z

& dx= dz1−z 2

∫0

∞

f x dx = ∫0

1 11−z 2 f z

1−z dz


∫0

∞

f x dx

solution is to change variables: z= x1x

x= z1−z

& dx= dz1−z 2

∫0

∞

f x dx = ∫0

1 11−z 2 f z

1−z dz

We can make two changes of variables: y = x - a & z = y/(1+y)

to calculate: ∫a

∞

f xdx = ∫0

1 11−z 2 f z

1−z a dz


∫0

∞

f x dx

solution is to change variables: z= x1x

x= z1−z

& dx= dz1−z 2

∫0

∞

f x dx = ∫0

1 11−z 2 f z

1−z dz

We can make two changes of variables: y = x - a & z = y/(1+y)

to calculate: ∫a

∞

f xdx = ∫0

1 11−z 2 f z

1−z a dz

Multi-Dimensional Integration

∫xa

xb

∫ya

yb

f x , ydy dx

∫xa

xb

f xdx

f x=∫ya

yb

f x , ydy

For N points in each integral calculation there are N2 calculations

Solve by Series Integration

“Divide and Conquer”

TrapezoidalSimpson's Rule


∫xa

xb

∫ya

yb

f x , ydy dx

∫xa

xb

f xdx

f x=∫ya

yb

f x , ydy

For N points in each integral calculation there are N2 calculations

Solve by Series Integration

“Divide and Conquer”

TrapezoidalSimpson's Rule


I=∫0

1

dx1∫0

1

dx2 ...∫0

1

dx12 f x1, x2, ... x12

For 100 steps in each integration there are 10012 =1024 calculations

Example: Atomic Physics

3 Dimension/electron * 4 electrons = 12 Dimensions

Assuming 1 Giga evaluations/sec It would take over 107 years!!!!

4Be


I=∫0

1

dx1∫0

1

dx2 ...∫0

1

dx12 f x1, x2, ... x12

For 100 steps in each integration there are 10012 =1024 calculations



Assuming 1 Giga evaluations/sec It would take over 107 years!!!!

4Be

Monte Carlo Integration

Monte Carlo methods provide an alternative method of calculating an integral.

Using Random Numbers to Solve Integrals


Monte Carlo methods provide an alternative method of calculating an integral.

Using Random Numbers to Solve Integrals

Random NumbersPseudo-Random Numbers

The numbers are pseudo-random in the sense that they are generated deterministically from a seed number, but are distributed in what has statistical similarities to random fashion.





Python modules for generating random numbers: ● import random:

● a smaller set of functions for random numbers● import numpy.random:

● a more complete set of utilities with many generating functions along with array manipulation capabilities











Numpy.random and random modules uses a Mersenne Twister algorithm to generate pseudorandom numbers which has become the generator of choice for serious physics calculations.

Let's always use the NumPy random module






Numpy.random and random modules uses a Mersenne Twister algorithm to generate pseudorandom numbers which has become the generator of choice for serious physics calculations.

Let's always use the NumPy random module

NumPy's random module

import numpy as np

>>> np.random.seed(136)The seed is an integer value. Any program that starts with the same seed will generate exactly the same sequence of random numbers each time it is run. This is useful for debugging programs but otherwise not needed as the seed is uniquely set each time the program executes.

Note: one could also import via:import numpy.random as rnd>>> rnd.seed(136)

NumPy's random module

import numpy as np

>>> np.random.seed(136)The seed is an integer value. Any program that starts with the same seed will generate exactly the same sequence of random numbers each time it is run. This is useful for debugging programs but otherwise not needed as the seed is uniquely set each time the program executes.

Note: one could also import via:import numpy.random as rnd>>> rnd.seed(136)

NumPy's random moduleGenerate random integers in the range [min, max)

>>> np.random.randint(5,10)8

Generates a single random number in [0.0, 1.0)>>> np.random.random()0.70110427435769551

Generate an array of random numbers in the interval [0.0, 1.0) >>> np.random.rand(1)array([ 0.73549029])

>>> np.random.rand(5)array([ 0.6652181 , 0.58861746, 0.8514131 , 0.68607923, 0.8785746 ])

>>> np.random.rand(2,3)array([[ 0.81698429, 0.632073 , 0.10512043], [ 0.88226248, 0.47654622, 0.45082853]])

NumPy's random moduleGenerate random integers in the range [min, max)

>>> np.random.randint(5,10)8

Generates a single random number in [0.0, 1.0)>>> np.random.random()0.70110427435769551

Generate an array of random numbers in the interval [0.0, 1.0) >>> np.random.rand(1)array([ 0.73549029])

>>> np.random.rand(5)array([ 0.6652181 , 0.58861746, 0.8514131 , 0.68607923, 0.8785746 ])

>>> np.random.rand(2,3)array([[ 0.81698429, 0.632073 , 0.10512043], [ 0.88226248, 0.47654622, 0.45082853]])

NumPy's random modulehpc-login% pydoc numpy.random ... ...DESCRIPTION ======================== Random Number Generation ======================== ... ...

==================== ========================================================= Univariate distributions ============================================================================== beta Beta distribution over ``[0, 1]``. binomial Binomial distribution. chisquare :math:`\chi^2` distribution. exponential Exponential distribution. f F (Fisher-Snedecor) distribution. gamma Gamma distribution. geometric Geometric distribution. gumbel Gumbel distribution. hypergeometric Hypergeometric distribution. laplace Laplace distribution. logistic Logistic distribution. lognormal Log-normal distribution. logseries Logarithmic series distribution. negative_binomial Negative binomial distribution. noncentral_chisquare Non-central chi-square distribution. noncentral_f Non-central F distribution. normal Normal / Gaussian distribution. pareto Pareto distribution. poisson Poisson distribution.

...

...

NumPy's random modulehpc-login% pydoc numpy.random ... ...DESCRIPTION ======================== Random Number Generation ======================== ... ...

==================== ========================================================= Univariate distributions ============================================================================== beta Beta distribution over ``[0, 1]``. binomial Binomial distribution. chisquare :math:`\chi^2` distribution. exponential Exponential distribution. f F (Fisher-Snedecor) distribution. gamma Gamma distribution. geometric Geometric distribution. gumbel Gumbel distribution. hypergeometric Hypergeometric distribution. laplace Laplace distribution. logistic Logistic distribution. lognormal Log-normal distribution. logseries Logarithmic series distribution. negative_binomial Negative binomial distribution. noncentral_chisquare Non-central chi-square distribution. noncentral_f Non-central F distribution. normal Normal / Gaussian distribution. pareto Pareto distribution. poisson Poisson distribution.

...

...

Generating random distributions

>>> import numpy as np>>> import matplotlib.pyplot as plt>>>>>> plt.hist(numpy.random.normal(size=10000), bins=numpy.linspace(-3,3,100))>>> plt.show()

hpc-login-24 % pydoc numpy.random.normal

Generating random distributions

>>> import numpy as np>>> import matplotlib.pyplot as plt>>>>>> plt.hist(numpy.random.normal(size=10000), bins=numpy.linspace(-3,3,100))>>> plt.show()

hpc-login-24 % pydoc numpy.random.normal


Two of the most common ways to employ Monte Carlo techniques are

Monte Carlo Sampling method

Monte Carlo Mean-Value method


Two of the most common ways to employ Monte Carlo techniques are

Monte Carlo Sampling method

Monte Carlo Mean-Value method

Monte Carlo Sampling Method

Box off the region of integrationcalculate the area of the box

Randomly place points in the boxCount # of points in the box vs # under the

functionAreaf(x) =Areabox * Nf(x)/Nbox

f(x) = -xπ + π xfmax

xmax x

f(x)

Nf(x) = #

Nbox = # + #

“random marks”

Monte Carlo Sampling Method

Box off the region of integrationcalculate the area of the box

Randomly place points in the boxCount # of points in the box vs # under the

functionAreaf(x) =Areabox * Nf(x)/Nbox

f(x) = -xπ + π xfmax

xmax x

f(x)

Nf(x) = #

Nbox = # + #

“random marks”

Monte Carlo Mean-Value Method

Integration using the Monte Carlo method is done by averaging the value of the function at randomly selected points within the integration interval

I=∫a

b

dx f x=b−a⟨ f ⟩

⟨ f ⟩≃1N ∑

i=0

N

f x i

Statistical Error f= f / Nstandard deviation of mean

I~ f

“The Work Horse”

f(x)

x

<f>

Monte Carlo Mean-Value Method

Integration using the Monte Carlo method is done by averaging the value of the function at randomly selected points within the integration interval

I=∫a

b

dx f x=b−a⟨ f ⟩

⟨ f ⟩≃1N ∑

i=0

N

f x i

Statistical Error f= f / Nstandard deviation of mean

I~ f

“The Work Horse”

f(x)

x

<f>

fmax

xmax x

f(x)import numpy.random as rndnUnder = 0 for _ in range(nPoints): x = xMin + (xMax - xMin)*rnd.random() fRand = fMin + (fMax - fMin)*rnd.random() if fRand < f( x ): nUnder += 1 print(“Integration by Samples = ”, \ fMax*(xMax - xMin) * nUnder/nPoints)

import numpy.random as rndsum = 0for _ in range(nPoints): x = xMin + (xMax - xMin)*rnd.random() sum += f( x ) print(“Integration by Mean-Value = ”, \

(xMax – xMin) * sum/nPoints )

Example: MC integration

f(x) = -xπ + πx

<f>

xmax x

f(x)

xmin

xmin

∫0

eln (π)(π−1)

π x−xπ

fmax

xmax x

f(x)import numpy.random as rndnUnder = 0 for _ in range(nPoints): x = xMin + (xMax - xMin)*rnd.random() fRand = fMin + (fMax - fMin)*rnd.random() if fRand < f( x ): nUnder += 1 print(“Integration by Samples = ”, \ fMax*(xMax - xMin) * nUnder/nPoints)

import numpy.random as rndsum = 0for _ in range(nPoints): x = xMin + (xMax - xMin)*rnd.random() sum += f( x ) print(“Integration by Mean-Value = ”, \

(xMax – xMin) * sum/nPoints )

Example: MC integration

f(x) = -xπ + πx

<f>

xmax x

f(x)

xmin

xmin

∫0

eln (π)(π−1)

π x−xπ

Multi-Dimensional Monte Carlo

I=∫a

b

dx∫c

d

dy f x , y≃b−ad−c∗ 1N ∑

i

N

f x i , y i

Easy to generalize mean-value integration to many dimensionsMulti-Dimensional Monte Carlo

I=∫a

b

dx∫c

d


i

N

f x i , y i

Easy to generalize mean-value integration to many dimensions


I=∫a

b

dx∫c

d


i

N

f xi , y i


import numpy.random as rnddef integrateMC(func, dim, limit, N=100): I,sum = 1/N,0 for n in range(dim): I *= (limit[n][1] - limit[n][0]) for k in range(N): x = [] for n in range(dim): x += [limit[n][0] + (limit[n][1] - limit[n][0])*rnd.random()] sum += func(x) return I*sum

def f(x): return np.sin(x[0] * x[1]**2)

dim,limit = 2,[[0,np.pi],[0,np.pi]]print(integrateMC(f, dim, limit))

main part of program

integrand function

multi-dimensional Monte Carlointegration implementation

f (x , y )=sin(xy2)


I=∫a

b

dx∫c

d


i

N

f xi , y i


import numpy.random as rnddef integrateMC(func, dim, limit, N=100): I,sum = 1/N,0 for n in range(dim): I *= (limit[n][1] - limit[n][0]) for k in range(N): x = [] for n in range(dim): x += [limit[n][0] + (limit[n][1] - limit[n][0])*rnd.random()] sum += func(x) return I*sum

def f(x): return np.sin(x[0] * x[1]**2)

dim,limit = 2,[[0,np.pi],[0,np.pi]]print(integrateMC(f, dim, limit))

main part of program

integrand function

multi-dimensional Monte Carlointegration implementation

f (x , y )=sin(xy2)

Multi-Dimensional Integrationvia MC mean value

I=∫0

1

dx1∫0

1

dx2 ...∫0

1

dx12 f x1, x2, ... x12

For N=106 random points in the MC integration there are ~106 calculations



Assuming 1 Giga evaluations/sec It would take ~ 10-3 sec

4Be

≃1−0 12∗1N ∑

i

N

f x1,i x2,

i x12i

compare to 107 yrs

Multi-Dimensional Integrationvia MC mean value

I=∫0

1

dx1∫0

1

dx2 ...∫0

1

dx12 f x1, x2, ... x12

For N=106 random points in the MC integration there are ~106 calculations



Assuming 1 Giga evaluations/sec It would take ~ 10-3 sec

4Be

≃1−0 12∗1N ∑

i

N

f x1,i x2,

i x12i

compare to 107 yrs

Monte Carlo Error

Monte Carlo error is StatisticalError decreases as 1/ N

Mean-Value Integration is an ordinary statistical mean

x=1N ∑

i=1

N

xi

The error on the mean is the standard deviation of the mean (SDOM)

x= x

NWith the standard deviation σξ defined as usual

x= 1N −1∑ xi−xb

2

Note: ∑ [x i−x 2]=[∑ x i2]−N x2 is very useful in computing SDOM!

Monte Carlo Error

Monte Carlo error is StatisticalError decreases as 1/ N

Mean-Value Integration is an ordinary statistical mean

x=1N ∑

i=1

N

xi

The error on the mean is the standard deviation of the mean (SDOM)

x= x

NWith the standard deviation σξ defined as usual

x= 1N −1∑ xi−xb

2

Note: ∑ [x i−x 2]=[∑ x i2]−N x2 is very useful in computing SDOM!

Monte Carlo Error∫0

∫0

sin x∗y2dx dy=1.9051±0.0006

Mean-value integration error decreases linearly

with 1/√N

SDOMAVG

Averaged over 20 samples for each value of number of

Monte Carlo samples

nSamples, n = 10000, 20 While nSamples < MaxSamples vSum, vSumSq = 0.0, 0.0 for _ in range(n): v = integrateMC(func, nDim, limits, nSamples) vSum += v vSumSq += v*v avg = vSum/n SDOM = sqrt(1/n) * sqrt( 1/(n - 1) * (vSumSq - n*avg*avg) ) nSamples *= 2

Monte Carlo Error∫0

∫0

sin x∗y2dx dy=1.9051±0.0006

Mean-value integration error decreases linearly

with 1/√N

SDOMAVG

Averaged over 20 samples for each value of number of

Monte Carlo samples

nSamples, n = 10000, 20 While nSamples < MaxSamples vSum, vSumSq = 0.0, 0.0 for _ in range(n): v = integrateMC(func, nDim, limits, nSamples) vSum += v vSumSq += v*v avg = vSum/n SDOM = sqrt(1/n) * sqrt( 1/(n - 1) * (vSumSq - n*avg*avg) ) nSamples *= 2

See python code at http://hadron.physics.fsu.edu/~eugenio/comphy/examples/mcint.pySee python code at http://hadron.physics.fsu.edu/~eugenio/comphy/examples/mcint.py

http://hadron.physics.fsu.edu/~eugenio/comphy/examples/mcint.py

Let's get working on #5

MONTE CARLO

Let's get working on #5

MONTE CARLO

computational physics adaptive, adaptive, multi...

Documents