chapter 3: random number generators - lund observatory · 2017-11-30 · astm21 chapter 3: random...

ASTM21 Chapter 3: Random number generators p.

Chapter 3: Random number generators

Random number generators

Generating arbitrary distributions

The transformation method

The rejection method

MATLAB functions for common distributions

1


Random number generators

Generating (pseudo-) random numbers X ~ U(0,1) is basic to all experimental statistics, simulation, experiment design, and data analysis.Computer languages often contain a system-supplied random number generator, sayx = rand() or x = rand(seed)

where seed is an integer used to "seed" the generator.Designing good random number generators is an art that has only recently matured. Do not try to design one yourself, or use code that is more than 10 years old. Some quotes from Numerical Recipes (3rd ed., Ch. 7):

Be cautious about any source earlier than about 1995, since the field has progressed enormously in the following decade. [...] The greatest lurking danger for a user today is that many out-of-date and inferior methods remain in general use. [...] If all scientific papers whose results are in doubt because of [bad random number generators] were to disappear from library shelves, there would be a gap on each shelf about as big as your fist.

Numerical Recipes contains examples of good, portable random number generators. But the simplest is to use generators that are part of a well-reputed software package. The so-called Mersenne Twister (Matsumoto & Nishimura 1997) used by MATLAB [mt19937ar] since about 2008 is considered to be of high quality - it passes a number of stringent tests for randomness, including the ‘Diehard’ test suite (Marsaglia 1998).

2


Random number generators: Use of seed

When MATLAB is started, and you ask for (say) three random numbers using rand(), you get:>> rand(3,1)ans = 0.814723686393179 0.905791937075619 0.126986816293506

The next one is:>> randans = 0.913375856139019

If you quit MATLAB and start again, you get for example:>> rand(2,2)ans = 0.814723686393179 0.126986816293506 0.905791937075619 0.913375856139019

Each time MATLAB is started, the default random stream is initialized with the same seed (= 0).

3


Random number generators: Use of seed

To check how the default random stream was initialized, use:>> RandStream.getGlobalStreamans = mt19937ar random stream (current global stream) Seed: 0 NormalTransform: Ziggurat

In simulation experiments it is often desirable to start each experiment (or batch of experiments) from a different, but well-defined state, so that a certain experiment can be reproduced exactly. (Why?) This is done by initializing the default random stream with different seeds for each batch. >> stream = RandStream.create('mt19937ar','seed',100)stream = mt19937ar random stream Seed: 100 NormalTransform: Ziggurat>> RandStream.setGlobalStream(stream)>> rand(3,1)ans = 0.543404941790965 0.278369385093796 0.424517590749133

4


Random number generators: Some caveats

• Do not re-initialise unnecessarily: for example to repeat an experiment 1000 times in sequence, do not re-initialise in between, unless each experiment takes a really long time. (Why?)

• If your investigation depends very critically on the quality of the random number generator, try another one and check that the results are (statistically) the same.

• Warning: In MATLAB, never use rand('seed',1), rand('state',2), randn('seed',3), etc. to set the seed. These are obsolete forms which cause MATLAB to switch to a very old random number generator, as can be seen by checking the default stream:>> rand('seed',100)>> RandStream.getGlobalStreamans = legacy random stream (current global stream) RAND algorithm: V4 (Congruential) RANDN algorithm: V5 (Ziggurat)

The congruential RAND algorithm from V4 (1992) is definitely not a good one!

5


Generating arbitrary distributions: The transformation methodTo generate a univariate (pseudo-) random variable y with given pdf p(y), there are a few basic techniques that can be used, and some nice tricks for special distributions (like the Gaussian). They all start with the generation of one or several uniform variates x ~ U(0,1).

The transformation method requires that you can compute (without too much difficulty) the inverse cdf of p(y), F–1(x).

6


The transformation method: Three examplesExample 1: The exponential distribution

To generate a random variable y with the exponential pdf with parameter λ (see L2:24),

we use the analytical cdf

with the inverse

Thus, if x ~ U(0,1) we find that y = −(ln x)/λ has the desired distribution.

Note that 1 − x ~ U(0,1) so we can save one subtraction by using x instead of 1 − x.

7


The transformation method: Three examplesExample 2: The Cauchy distribution

The standard form of the Cauchy distribution is

which is symmetric about x = 0 and has a FWHM (Full Width at Half Maximum) of 2 units.As discussed in L1:19 this distribution has undefined mean value and infinite variance. The analytical cdf is

with the inverse

Thus, if x ~ U(0,1) we find that y = tan[(x − 0.5)π] has the desired distribution.

8


The transformation method: Three examplesExample 3: The Box-Muller transformation for the normal distributionIt is not convenient to use the transformation method directly on the one-dimensional normal distribution N(0,1), because the cdf

and its inverse cannot be expressed in terms of elementary functions. However, if x ~ N(0,1) and y ~ N(0,1) are independent normal variates, then their joint pdf is

Transforming to polar coordinates by means of x = r cos !, y = r sin !, we find

which shows that r and ! are independent, that ! ~ U(0, 2π), and that the cdf of r is F(r) = 1 − exp(−r2/2), with inverse r = F–1(u) = [−2 ln(1 − u)]1/2. Thus, given two uniform variates u1 ~ U(0,1) and u2 ~ U(0,1) we obtain two independent normal variates as

which is the Box-Muller transformation. However, more efficient algorithms exist (p. 13).9


Generating arbitrary distributions: The rejection methodThe rejection method does not require the inverse cdf, or even the cdf, but only that you can compute p(x) for any given x. Moreover, you need some other function f (x) such that

• p(x) ≤ f (x) everywhere• the integral of f is finite (say A)• the inverse cumulative function of f can be computed (F–1(a).)

10


Generating arbitrary distributions: The rejection methodUsing the rejection method, the algorithm to generate x0 ~ p(x) is:

$ 1. Generate a number a ~ U(0, A)$ 2. Apply transformation x0 = F–1(a)$ 3. Compute f (x0) and p(x0)$ 4. Generate another random number b = U(0, f (x0))$ 5. If b ≤ p(x0) accept x0, otherwise goto 1

11


The rejection method: Two examplesExample 1: Generate x ~ Beta(2,5)

This is the beta distribution (see L1:18) with parameters α = 2 and β = 5:

The transformation method is not useful, since the cdf is a polynomial of degree 6. But we have p(x) < 2.5 everywhere (see diagram), so we can use f (x) = 2.5 (0 ≤ x ≤ 1) with integral A = 2.5. The cdf of f (x) is y = F(x) = Ax for 0 ≤ x ≤ 1, and the inverse is x = F–1(y) = y/A. Thus the procedure in this case is: 1. Generate a number a ~ U(0, A) 2. Apply transformation x = F–1(a) = a/A [this is simply x ~ U(0,1)] 3. Compute f (x) = A and p(x) 4. Generate another random number y = U(0, A) 5. If y ≤ p(x) accept x, otherwise goto 1

It can be seen that this is equivalent to placing random points (x, y) in the rectangle outlined by f (x) and accepting the x value if the y value is below p(x).The efficiency of the method depends on the ratio ofareas below the two curves, i.e., A = 2.5 in this case.On average, A pairs (x, y) are needed to generate one x.

12

−0.2 0 0.2 0.4 0.6 0.8 1 1.20

0.5

1

1.5

2

2.5

3

x

p(x)

f (x)


The rejection method: Two examplesExample 2: The ziggurat algorithm for x ~ N(0,1)

The ziggurat algorithm (by Marsaglia) is the most commonly used method to generate Gaussian numbers (e.g., randn in MATLAB) because it is very efficient.It is essentially the rejection method applied to segments of the Gaussian curve (see diagram). First one of the rectangles is selected at random (as they have equal area), then a second random number is used to decide if the x value is to the left of the dotted line, otherwise the rejection method is applied to decide if it is below the Gaussian curve.

13

Artist’s drawing of a Guto-Sumerian ziggurat (step pyramid) Source: www.iranian.com0 1 2 3 4

0

0.1

0.2

0.3

0.4

0.5


MATLAB functions for common distributions

14

Distribution pdf or pmf cdfp = F(x)

inverse cdfx = F−1(p)

random generator

standard(0, 1)

Uniform unifpdf unifcdf unifinv unifrnd rand

Beta betapdf betacdf betainv betarnd

Normal normpdf normcdf norminv normrnd randn

Chi-squared chi2pdf chi2cdf chi2inv chi2rnd

Exponential exppdf expcdf expinv exprnd

Gamma gampdf gamcdf gaminv gamrnd

Binomial binopdf binocdf binoinv binornd

Poisson poisspdf poisscdf poissinv poissrnd

Initialization of random number renerator: rng(seed), rng(’shuffle’)

chapter 3: random number generators - lund observatory · 2017-11-30 · astm21 chapter 3: random...

Documents