computer applications for engineers - 4.2 - random... · 2019-11-06 · an interesting number 2...

Asst. Prof. Dr. Prapun [email protected]

Generating Random Numbers

1

Computer Applications for EngineersET 601

Office Hours: (BKD 3601-7)Wednesday 9:30-11:30Wednesday 16:00-17:00Thursday 14:40-16:00

An interesting number

2

Here is an interesting number:

0.814723686393179

This is the first number produced by the MATLAB random number generator with its default settings.

Start up a fresh MATLAB, set format long, type rand, and it’s the number you get. Verified in MATLAB 2013a

It may seem perverse to use a computer, that most precise and deterministic ofall machines conceived by the human mind, to produce “random” numbers. Morethan perverse, it may seem to be a conceptual impossibility. Any program, after all,will produce output that is entirely predictable, hence not truly “random.”

[Numerical Recipes, Ch 7]

Pseudorandom Number

3

Random numbers were originally either manually or mechanically generated, by using such techniques as spinning wheels, or dice rolling, or card shuffling.

The modern approach is to use a computer to successively generate pseudorandom numbers. Although they are deterministically generated, they approximate

independent uniform (0, 1) random variables. So, “random” numbers in MATLAB are not unpredictable. They are

generated by a deterministic algorithm. The algorithm is designed to be sufficiently complicated so that its output appears to

be random to someone who does not know the algorithm, and can pass various statistical tests of randomness.

Our assumption Assume that we have a good pseudorandom number generators. Example: the rand command in MATLAB.

(Truly) random digits from physical mechanisms

6

Employ physical mechanisms rather than formal algorithms to provide random digits.

Hotbits project: www.fourmilab.ch/hotbits Use radioactive decay. Run by Autodesk founder John Walker If you connect to this site, you can listen to the Geiger counter ticks.

Random.org site: www.random.org Sample atmospheric noise by using a radio tuned between stations. The site also provides a general discussion of random numbers.

Lavarnd site: www.lavarnd.org Take digital photographs of a webcam with its lens cap on. Rely on thermal “noise”

Lavarand site: lavarand.sgi.com Take digital photographs of the patterns

made by colored lava lamps.

rng

7

The sequence of numbers produced by rand is determined by the internal settings of the uniform random number generator that underlies rand, randi,and randn.

You can control that shared random number generator using rng. This can be useful for controlling the repeatability of your

results.

http://www.mathworks.com/support/2013b/matlab/8.2/demos/controlling-random-number-generation.html

( random number generator)

rng default/shuffle

12

Every time you start MATLAB, the generator resets itself to the same state.

You can reset the generator to the startup state at any time in a MATLAB session (without closing and restarting MATLAB) by rng('default') rng default

To avoid repeating the same results when MATLAB restarts: Execute the command rng('shuffle') rng shuffle It reseeds the generator using a different seed

based on the current time.

rng: Save and Restore the Generator Settings

13

Save the current generator settings in s

Restore the saved generator settings

The first call to randichanged the state of the generator, so the second result is different.

References

14

Chapter 3 (Random Numbers) in Sheldon M. Ross. “Simulation.” Academic Press, 2012, 5th Edition

Chapter 9 (p. 245-252) in Peter Olofsson, “Probabilities The Little Numbers That Rule Our Lives”, Wiley, 2006

Chapter 9 (Random Numbers) in Cleve Moler. “Numerical Computing with MATLAB.” SIAM, 2004

Chapter 7 (Random Numbers) (Sections 7.0-7.1.1) in W. H. Press, S. A. Teukolsky, W. T. Vetterling, B. P. Flannery. “Numerical Recipes: The Art of Scientific Computing.” Cambridge, 2007, 3rd Edition.

References

15

Park, S.K., and K.W. Miller. “Random Number Generators: Good Ones Are Hard to Find.” Communications of the ACM, 31(10):1192–1201. 1998.

C. Moler, Random thoughts, “10^435 years is a very long time”, MATLAB News and Notes, Fall, 1995

Asst. Prof. Dr. Prapun [email protected]

Generating Random Variables

16

Computer Applications for EngineersET 601

Office Hours: (BKD 3601-7)Wednesday 9:30-11:30Wednesday 16:00-17:00Thursday 14:40-16:00

Generation of arbitrary RVs

17

There are many built-in families of RV in MATLAB that can be generated by MATLAB’s random command.

Popular families are provided. Binomial, Exponential, Geometric, Normal, Poisson, Uniform Beta, Birnbaum-Saunders, Burr Type XII, Chi-Square, Extreme

Value, F, Gamma, Generalized Extreme Value, Generalized Pareto, Hypergeometric, Inverse Gaussian, Logistic, Loglogistic, Lognormal, Nakagami, Negative Binomial, Noncentral F, Noncentral t, Noncentral Chi-Square, Rayleigh, Rician, Student's t, t Location-Scale, Weibull

doc random How to generate RVs whose families are not built-in?

Bernoulli Trials

18

We want to generalize the old technique.

rand(1,120) < 0.5 randi([0 1],1,120)

1/ 2, 0,1/ 2, 1,0, otherwise.

X

xp x x

To generate a RV X whose pmf is given by

First use rand create a uniform RV U.

Then, set 1, 1/ 2,0, 1/ 2.

UX

U


First use rand create a uniform RV U.

Then, set

, 1,

1 , 0,0, otherwise.

X

p xp x p x

1, ,0, .

U pX

U p

clear all; close all; S_X = [3 4]; p_X = [1/3 2/3]; n = 1e6;

U = rand(1,n); V = [U < 2/3]; X = S_X(V+1); rf = hist(X,S_X)/n; stem(S_X,rf,'rx') hold on stem(S_X,p_X,'bo') xlim([min(S_X)-1,max(S_X)+1]) legend('Rel. freq. from sim.'... ,'pmf p_X(x)') xlabel('x') grid on

Example

19


First use rand to create a uniform RV U.

Then, set

Finally, map to .

Of course, we can combine all the steps into just one line:

1/ 3, 3,2 / 3, 4,0, otherwise,

X

xp x x

1, 2 / 3,0, 1/ 3.

UV

U

2 2.5 3 3.5 4 4.5 50

0.1

0.2

0.3

0.4

0.5

0.6

0.7

x

Rel. freq. from sim.pmf pX(x)

X = S_X([rand(1,n) < 2/3]+1);

Example

20


First use rand to create a uniform RV U.

Then, set

Finally, map to .

1/ 6, 3,1/ 3, 4,1/ 2, 8,0, otherwise.

X

xx

p xx

1, 0 1/ 6,2, 1/ 6 1/ 2,3, 1/ 6 1,

UV U

U

clear all; close all; S_X = [3 4 8]; p_X = [1/6 1/3 1/2]; n = 1e6; F_X = cumsum(p_X); U = rand(1,n); [dum,V] = histc(U,[0 F_X/F_X(end)]); X = S_X(V); rf = hist(X,S_X)/n; stem(S_X,rf,'rx') hold on stem(S_X,p_X,'bo') xlim([min(S_X)-1,max(S_X)+1]) legend('Rel. freq. from sim.'... ,'pmf p_X(x)') xlabel('x') grid on

2 3 4 5 6 7 8 90

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

x


histc vs hist

21

N = hist(U,centers) Bins’ centers are defined by the vector centers.

The first bin includes data between -inf and the first center and the last bin includes data between the last bin and inf.

N(k) count the number of entries of vector U whose values falls inside the kth bin.

N = histc(U,edges) Bins’ edges are defined by the vector edges. N(k) count the value U(i) if edges(k) ≤ U(i) < edges(k+1).

The last (additional) bin will count any values of U that match edges(end).

Values outside the values in edges are not counted. May use -inf and inf in edges.

[N,BIN_IND] = histc(U,EDGES) also returns vector BIN_IND indicating the bin index that each entry in U sorts into.

Example: histc

22

>> p_X = [1/6 1/3 1/2];>> F_X = cumsum(p_X)

F_X =

0.1667 0.5000 1.0000

>> U = rand(1,5)

U =

0.2426 0.9179 0.9409 0.1026 0.8897

>> [dum,V] = histc(U,[0 F_X])

dum =

1 1 3 0

V =

2 3 3 1 3

Loops

23

Loops are MATLAB constructs that permit us to execute a sequence of statements more than once.

There are two basic forms of loop constructs: while loops and for loops.

The major difference between these two types of loops is in how the repetition is controlled. The code in a while loop is repeated an indefinite number of

times until some user-specified condition is satisfied. The code in a for loop is repeated a specified number of times,

and the number of repetitions is known before the loops starts.

while loop

24

General form:

The statements in the body are repeatedly executed as long as the expression expr remains true.

while exprbody

end

clear all; close all; S_X = [3 4 8]; p_X = [1/6 1/3 1/2]; n = 1e6; U = rand(1,n); V = zeros(size(U)); %Preallocation for k = 1:n F = p_X(1); m = 1; while (U(k) > F) m = m+1; F = F+p_X(m); end V(k) = m; end X = S_X(V); hist(X) rf = hist(X,S_X)/n; stem(S_X,rf,'rx') hold on stem(S_X,p_X,'bo') xlim([min(S_X)-1,max(S_X)+1]) legend('Rel. freq. from sim.'... ,'pmf p_X(x)') xlabel('x') grid on

Example: Geometric1 RV

25

clear all; close all; p = 1/3; n = 1e6; U = rand(1,n); V = zeros(size(U)); %Preallocation for k = 1:n F = p; pk = p; m = 1; while (U(k) > F) m = m+1; pk = pk*(1-p); F = F + pk; end V(k) = m; end X = V; X_Range = 1:10; p_X = p*(1-p).^(X_Range-1); rf = hist(X,X_Range)/n; stem(X_Range,rf,'rx') hold on stem(X_Range,p_X,'bo') xlim([min(X_Range)-1,max(X_Range)+1]) legend('Rel. freq. from sim.'... ,'pmf p_X(x)') xlabel('x') grid on

1

1

2

1 , 1,2,3,

1

11

1 1

kX

X

kX

kX

p k p p k

p p

p k p pp

p k p p

0 1 2 3 4 5 6 7 8 9 10 110

0.05

0.1

0.15

0.2

0.25

0.3

0.35

x


Alternatively, can put the formula for pX(k) here.

User-defined functions in MATLAB

26

Functions are files that can accept input arguments and return output arguments.

We have already seen (and used) MATLAB script files

A script file consists of a sequence of MATLAB statements that you could also type directly into the Command Window.

MATLAB built-in functions The actual contents of the built-in functions are not generally known or seen by the user.

Variables in a function file are local by default. Functions operate on variables within their own workspace, separate from the

workspace you access at the MATLAB command prompt. The names of the file and of the function should be the same (to avoid

confusion). The first line of the function declares the function name, input arguments, and

output arguments; without this line the file would be a script file.

Examples

27

function y = sqr(x) y = x.^2; end sqr.m

This is not necessary in many cases in current versions of MATLAB, but it is considered good style anyway.

function z = multiply(x,y) z = x.*y end

function [m s] = scorestat(x) m = mean(x); s = std(x); end

Anonymous Functions

28

Defined within a single MATLAB statement.

Consist of a single MATLAB expression and any number of input and output arguments.

Can be defined right at the MATLAB command line, or within a function or script.

Give you a quick means of creating simple functions without having to create a file for them each time.

Syntax: f = @(arglist) expression

datasample

29

clear all; close all; S_X = [3 4 8]; p_X = [1/6 1/3 1/2]; n = 1e6; X = datasample(S_X,n,'Weights',p_X); hist(X) rf = hist(X,S_X)/n; stem(S_X,rf,'rx') hold on stem(S_X,p_X,'bo') xlim([min(S_X)-1,max(S_X)+1]) legend('Rel. freq. from sim.'... ,'pmf p_X(x)') xlabel('x') grid on

Inverse Transform Method

30

Let U be a uniform random variable on the interval (0,1).

To generate a continuous random variable X with cumulative distribution function FX, set .

Whether it is feasible to implement depends on whether the inverse function of the integral of pdf is itself feasible to compute, either analytically or numerically.

Inverse function

31

-3 -2 -1 0 1 2 3

-3

-2

-1

0

1

2

3

x

g(x) = x3

g-1(x)-3

-2

-1

0

1

2

3

-3

-2

-1

0

1

2

3

x

g(x) = x 3g -1(x)

close all; clear all; x =linspace(-1.5,1.5,100); y = x.^3; plot(x,y,'k','LineWidth',1.5); grid on hold on; xlabel('x') plot(y,x,'LineWidth',1.5) legend('g(x) = x^3', 'g^{-1}(x)') mx = max([max(y) max(x)]); mn = min([min(y) min(x)]); xlim([mn,mx]); ylim([mn,mx]); x = linspace(mn,mx,100); plot(x,x,'--r')

Example: Expo. RV

32

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 50

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

x

F X(x)

Empirical CDF

Theoretical FXEmpirical

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 50

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

x

f X(x)

Theoretical fXEmpirical

close all; clear all; n = 1e6; lambda = 2; U = rand(1,n); X = -log(U)/lambda; minx = 0; maxx = 5; x = linspace(minx,maxx,50); % Verification via the cdf plots F_X = 1-exp(-lambda*x); plot(x,F_X,'LineWidth',1.5) hold on h = cdfplot(X); set(h,'LineStyle','--',... 'Color','r','LineWidth',1.5) xlim([minx,maxx]) ylabel('F_X(x)') legend('Theoretical F_X', 'Empirical') % Verification via the pdf figure f_X = lambda*exp(-lambda*x); plot(x,f_X,'LineWidth',1.5) hold on dx = x(2)-x(1); pdfest = histc(X,x)/(n*dx); plot(x+dx/2,pdfest,'r*') legend('Theoretical f_X', 'Empirical') xlim([minx,maxx]) grid on xlabel('x') ylabel('f_X(x)')

Rejection Method

33

Acceptance-Rejection Technique

Powerful and general

Assume known and computable distribution (pdf or pmf).

Does not require that the cdf and its inverse be readily computable Better than the transformation method discussed earlier.

Assume availability of RV whose support is the same as the

Motivational Example 1

34

It is easy to generate 2-D points uniformly over a finite area.

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

n = 1e3; U = rand(1,n); Y = rand(1,n); plot(Y,U,'.'); figure c = 2; U = rand(1,n); V = c*U; Y = rand(1,n); plot(Y,V,'.');


35

Suppose we want to generate realizations of a continuous random variable X whose pdf is given by

First, we generate 2-D points over the area that cover the pdf above.

Then, we keep only the points that are below .

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

2 , 0 1,0, otherwise.X

x xf x

figure % Define the pdf a = 2; f_X = @(x) a*x.^(a-1); x = linspace(0,1,100); plot(x,f_X(x),'r--') hold on % Keep only the points that % are below the pdf I = [V < f_X(Y)]; plot(Y(I),V(I),'.'); % Use only the x-coordinates X = Y(I);


36

Conclusion: If we had some way of choosing a random point in two dimensions, with uniform probability in the area under the pdf curve, then the x coordinate of that random point would have the desired pdf.

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

10

20

30

40

50

60

70

80

90

100

x

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.5

1

1.5

2

2.5

3

x

f X(x)


Start with n = 1,000.


37

Conclusion: If we had some way of choosing a random point in two dimensions, with uniform probability in the area under the pdf curve, then the x coordinate of that random point would have the desired pdf.

Start with n = 100,000.

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

1000

2000

3000

4000

5000

6000

7000

8000

9000

10000

x

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

x

f X(x)



38

Note: It is not possible to directly distribute 2-D points over infinite area.

So, if we want to generate X whose support is infinite, we will need to start with area that is finite.

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 50

0.5

1

1.5

2

2.5

3

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 50

1

2

3

4

5

6


39

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 50

0.5

1

1.5

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 50

0.5

1

1.5

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 50

0.5

1

1.5

0 0.5 1 1.5 2 2.5 3 3.50

50

100

150

x0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

x

f X(x)


References

40

Chapter 4 (Generating Discrete Random Variables)Chapter 5 (Generating Continuous Random Variables) In Sheldon M. Ross. “Simulation.”

Academic Press, 2012, 5th Edition

Chapter 7 (Random Numbers) (Sections 7.3) in W. H. Press, S. A. Teukolsky, W. T. Vetterling, B. P. Flannery. “Numerical Recipes: The Art of Scientific Computing.” Cambridge, 2007, 3rd Edition.

computer applications for engineers - 4.2 - random... · 2019-11-06 · an interesting number 2...

Documents