random number generation fall 2012
DESCRIPTION
Random Number Generation Fall 2012. By Yaohang Li, Ph.D. Review. Last Class Variance Reduction This Class Random Number Generation Uniform Distribution Non-uniform Distribution Random Number Generation Assignment 3 Next Class Quasi-Monte Carlo. Random Numbers. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Random Number Generation Fall 2012](https://reader035.vdocument.in/reader035/viewer/2022070403/56813a26550346895da207bc/html5/thumbnails/1.jpg)
Random Number Generation
Fall 2012
By Yaohang Li, Ph.D.
![Page 2: Random Number Generation Fall 2012](https://reader035.vdocument.in/reader035/viewer/2022070403/56813a26550346895da207bc/html5/thumbnails/2.jpg)
Review• Last Class
– Variance Reduction• This Class
– Random Number Generation– Uniform Distribution– Non-uniform Distribution Random Number Generation– Assignment 3
• Next Class– Quasi-Monte Carlo
![Page 3: Random Number Generation Fall 2012](https://reader035.vdocument.in/reader035/viewer/2022070403/56813a26550346895da207bc/html5/thumbnails/3.jpg)
Random Numbers• Application of Random Numbers
– Simulation• Simulate natural phenomena
– Sampling• It is often impractical to examine all possible cases, but a
random sample will provide insight into what constitutes typical behavior
– Numerical analysis– Computer programming– Decision making
• “Many executives make their decisions by flipping a coin…”
– Recreation
![Page 4: Random Number Generation Fall 2012](https://reader035.vdocument.in/reader035/viewer/2022070403/56813a26550346895da207bc/html5/thumbnails/4.jpg)
Natural Random Number
• Natural Random Numbers– No two snowflakes are the same
– Sources
• White Noise
• Water Molecule Distribution
• etc.
– Generation
• Measurement
• Irreproducible
• Errors
![Page 5: Random Number Generation Fall 2012](https://reader035.vdocument.in/reader035/viewer/2022070403/56813a26550346895da207bc/html5/thumbnails/5.jpg)
Pseudorandom Number Generators
• Pseudorandom Numbers– Using a Mathematical Formula
– Deterministic
– Behave like real random numbers
• Comments– There is no “perfect” pseudorandom number generator
– We should never completely trust results from a single pseudorandom number generator
– Good random number generators are hard to find
![Page 6: Random Number Generation Fall 2012](https://reader035.vdocument.in/reader035/viewer/2022070403/56813a26550346895da207bc/html5/thumbnails/6.jpg)
Middle-square method
• Developed by von Neumann• Procedure
– 4 digit starting value is created
– Square and produce 8-digit number
– Get the middle 4 digits as the result and seed for next number
• Problems– What if the middle 4 digits are 0
– Forsythe found that the sequence may stuck in 6100, 2100, 4100, 8100, 6100, …
• Not a good generator
![Page 7: Random Number Generation Fall 2012](https://reader035.vdocument.in/reader035/viewer/2022070403/56813a26550346895da207bc/html5/thumbnails/7.jpg)
Quality of Pseudorandom Numbers
• Uniformity• Randomness• Independence• Reproducibility• Portability• Efficiency• A sufficiently long period
![Page 8: Random Number Generation Fall 2012](https://reader035.vdocument.in/reader035/viewer/2022070403/56813a26550346895da207bc/html5/thumbnails/8.jpg)
Generating Uniform Random Numbers
• Generating Uniform Random Numbers– Uniform distribution on [0,1)
– Generation
• Un=Xn/m– Xn: Random number Integer
– m: Max(Xn)+1: Usually the word size of a computer
– Un: Uniform real random number at [0,1)
![Page 9: Random Number Generation Fall 2012](https://reader035.vdocument.in/reader035/viewer/2022070403/56813a26550346895da207bc/html5/thumbnails/9.jpg)
Linear Congruential Method
• Linear Congruential Method– Most commonly used generator for pseudorandom numbers
• m: modulus• a: multiplier• b: additive constant
– Period• m constrains the period
– max period: 2m-1– m is usually chosen to be either prime of a power-of-two
)(mod1 mbaxx nn
![Page 10: Random Number Generation Fall 2012](https://reader035.vdocument.in/reader035/viewer/2022070403/56813a26550346895da207bc/html5/thumbnails/10.jpg)
Shift-Register Generators (SRG)
• Shift-Register Generators– based on the following recursion
– ai and xi are either 0 or 1
– Comments
• The recursion produces only bits
• Incorporate these bits into integers
),2(mod1
0
k
iinikn xax
![Page 11: Random Number Generation Fall 2012](https://reader035.vdocument.in/reader035/viewer/2022070403/56813a26550346895da207bc/html5/thumbnails/11.jpg)
Lagged-Fibonacci Generators
• Lagged-Fibonacci Generators (LFG)– Additive Lagged-Fibonacci Generators
– Multiplicative Lagged-Fibonacci Generators
• Comments– LFG has a much longer period than LCGs
– (2k-1)2m-1
kjxxx mknjnn ),2(mod
kjxxx mknjnn ),2(mod
![Page 12: Random Number Generation Fall 2012](https://reader035.vdocument.in/reader035/viewer/2022070403/56813a26550346895da207bc/html5/thumbnails/12.jpg)
Spectral Tests
![Page 13: Random Number Generation Fall 2012](https://reader035.vdocument.in/reader035/viewer/2022070403/56813a26550346895da207bc/html5/thumbnails/13.jpg)
Spectral Tests
![Page 14: Random Number Generation Fall 2012](https://reader035.vdocument.in/reader035/viewer/2022070403/56813a26550346895da207bc/html5/thumbnails/14.jpg)
Inversive Congruential Generators
• Inversive Congruential Generators (ICGs)– Recursive ICGs
– Explicit ICGs
– Advantage of ICGs
• ICGs do not fall in hyperplanes
),(mod1 mbxax nn
)(modmbanxn
)(mod1 mcc
![Page 15: Random Number Generation Fall 2012](https://reader035.vdocument.in/reader035/viewer/2022070403/56813a26550346895da207bc/html5/thumbnails/15.jpg)
Combined Generators
• Combined Generators– Combining different recurrences can increase the period length
– Improve the structural properties of pseudorandom generators
– Construct a new random sequence
• exclusive-or operator
• addition modulo
• addition of floating-point random numbers modulo 1
– x, y
• Different random number sources
,nnn yxz
![Page 16: Random Number Generation Fall 2012](https://reader035.vdocument.in/reader035/viewer/2022070403/56813a26550346895da207bc/html5/thumbnails/16.jpg)
Parallel Random Number Generators
• Requirements of Parallel Random Numbers– Every random number sequence generated on each processor
should satisfy the requirements of a good sequential generator.
– The parallel generator must be reproducible both on different machines and on the same machine with a different partitioning of the processing resources.
– The parallelly generated random streams must be uncorrelated and must not overlap.
– The parallel generator should work for an arbitrary, but perhaps bounded, number of processors.
![Page 17: Random Number Generation Fall 2012](https://reader035.vdocument.in/reader035/viewer/2022070403/56813a26550346895da207bc/html5/thumbnails/17.jpg)
Parallel Random Numbers Generations (Leapfrog)
• Leapfrog
![Page 18: Random Number Generation Fall 2012](https://reader035.vdocument.in/reader035/viewer/2022070403/56813a26550346895da207bc/html5/thumbnails/18.jpg)
Parallel Random Numbers Generations (Sequence Splitting)
• Sequence Splitting
![Page 19: Random Number Generation Fall 2012](https://reader035.vdocument.in/reader035/viewer/2022070403/56813a26550346895da207bc/html5/thumbnails/19.jpg)
Parallel Random Numbers Generations (Sequence Splitting)
• Random Tree Method– Also called
parameterization method
![Page 20: Random Number Generation Fall 2012](https://reader035.vdocument.in/reader035/viewer/2022070403/56813a26550346895da207bc/html5/thumbnails/20.jpg)
SPRNG
• http://sprng.cs.fsu.edu
![Page 21: Random Number Generation Fall 2012](https://reader035.vdocument.in/reader035/viewer/2022070403/56813a26550346895da207bc/html5/thumbnails/21.jpg)
Random Choices from a Finite Set
•A random integer X between 0 and k-1
– U is a random number uniformly distributed in [0,1)
•A more general case
kUX
1...,
,
0,
21
2112
11
Uppifpx
ppUifpx
pUifx
X
kk
![Page 22: Random Number Generation Fall 2012](https://reader035.vdocument.in/reader035/viewer/2022070403/56813a26550346895da207bc/html5/thumbnails/22.jpg)
Inverse Function Method
• Cumulative Distribution Function– Most real-valued distribution may be expressed in terms of its
distribution function F(x)
• Inverse Function Method– X=F-1(U)
– Now the problem reduces to how to evaluate the inverse function F-1()
![Page 23: Random Number Generation Fall 2012](https://reader035.vdocument.in/reader035/viewer/2022070403/56813a26550346895da207bc/html5/thumbnails/23.jpg)
Interesting Trick
• Generating the random samples of F(x)=x2
• Inverse Function Method– X=U-1/2
• A short cut method– If X1 is a random variable having the distribution F1(x) and if
X2 is a random variable having the distribution F2(x)• max(X1, X2) has the distribution F1(x)F2(x)• min(X1, X2) has the distribution F1(x)+F2(x)-F1(x)F2(x)
– Then X=max(U1, U2) has the distribution of F(x)=x2
– Hard to believe that max(U1, U2) and U-1/2 have the same distribution
![Page 24: Random Number Generation Fall 2012](https://reader035.vdocument.in/reader035/viewer/2022070403/56813a26550346895da207bc/html5/thumbnails/24.jpg)
Normal Distribution• Polar Method
1. Generate two independent random variables, U1 and U2
2. Set V1=2U1-1, V2=2U2-1
3. Set S=V1*V1+V2*V2
4. If S>=1, return to Step 1
5. Set X1 and X2 according to the following two equations
S
SVX
S
SVX
ln2
ln2
22
11
![Page 25: Random Number Generation Fall 2012](https://reader035.vdocument.in/reader035/viewer/2022070403/56813a26550346895da207bc/html5/thumbnails/25.jpg)
Acceptance-Rejection Method
• Desired pdf– Suppose we bound the desired
probability distribution function to sample from a box
• Algorithm1. Generate a random variable x from
U(0,1)
2. Generate another random variable y from U(0,1)
3. If x<f(y)/fmax then return y
4. else repeat from step 1
![Page 26: Random Number Generation Fall 2012](https://reader035.vdocument.in/reader035/viewer/2022070403/56813a26550346895da207bc/html5/thumbnails/26.jpg)
Acceptance-Rejection Method Example
• Determine an algorithm for generating random variates for a random variable that take values 1, 2, …, 10 with probabilities 0.11, 0.12, 0.09, 0.08, 0.12, 0.10, 0.09, 0.09. 0.10, 0.10 respectively
• Acceptance-Rejection Method– u1=U(0,1), u2=U(0,1), c=max(p())=0.12– Y=floor(10*u1+1)– while (u2>p(Y)/c)
• u1=U(0,1), u2=U(0,1)• Y=floor(10*u1+1)
– output Y
![Page 27: Random Number Generation Fall 2012](https://reader035.vdocument.in/reader035/viewer/2022070403/56813a26550346895da207bc/html5/thumbnails/27.jpg)
Analysis of Acceptance-Rejection Method
• Advantage– Acceptance-Rejection Method can fit in different pdfs
– popularly used in complicated probability geometry
• Disadvantage– Inefficient if the volume of the region of interest is small
relative to that of the box
• most of the darts will miss the target
![Page 28: Random Number Generation Fall 2012](https://reader035.vdocument.in/reader035/viewer/2022070403/56813a26550346895da207bc/html5/thumbnails/28.jpg)
Exponential Distribution
• F(x)=1-e-x/
– Logarithm method (inverse function method)
– X=-lnU
![Page 29: Random Number Generation Fall 2012](https://reader035.vdocument.in/reader035/viewer/2022070403/56813a26550346895da207bc/html5/thumbnails/29.jpg)
Shuffling Algorithm
• Let X1, X2, …, Xt be a set of t numbers to be shuffled1. j=t
2. Generate U
3. Set k=floor (jU)+1
4. Exchange Xk with Xj
5. Decrease j by 1. If j>1, return to step 2
![Page 30: Random Number Generation Fall 2012](https://reader035.vdocument.in/reader035/viewer/2022070403/56813a26550346895da207bc/html5/thumbnails/30.jpg)
Summary• Random Numbers– Uniform Random Numbers
• Generation of Uniform Random Numbers– Natural Random Number Generators– Pseudorandom Number Generators
• Pseudorandom Number Generators– Requirement of Pseudorandom Number Generators– LCG– LFG– SRG– ICG– Combined Random Number Generators
• Parallel Random Number Generators– Requirement of Parallel Random Number Generators– Techniques for Parallel Random Number Generators
• Leapfrog• Sequence Splitting• Random Tree
![Page 31: Random Number Generation Fall 2012](https://reader035.vdocument.in/reader035/viewer/2022070403/56813a26550346895da207bc/html5/thumbnails/31.jpg)
Summary• Numerical Distribution
– Random Choices from a finite set– General methods for continuous distributions
• inverse function method• acceptance-rejection method
– Distributions• Normal distribution
– Polar method• Exponential distribution
– Shuffling
![Page 32: Random Number Generation Fall 2012](https://reader035.vdocument.in/reader035/viewer/2022070403/56813a26550346895da207bc/html5/thumbnails/32.jpg)
What I want you to do?
• Review Slides• Review basic probability/statistics concepts