sampling from pdfs - donald bren school of information and … · 2017. 4. 18. · •sampling from...

Sampling from PDFs II

CS295, Spring 2017

Shuang Zhao

Computer Science Department

University of California, Irvine

CS295, Spring 2017 Shuang Zhao 1

Announcements

• Additional readings on the course website

• Homework 1 due this Thursday (Apr 20)

• Programming Assignment 1 will be out on the same day


Last Lecture

• Monte Carlo integration II• Convergence properties

• Integrals over higher-dimensional domains

• Sampling from PDFs• Inversion method


Today’s Lecture

• Sampling from PDFs II• General methods

• Rejection sampling

• Metropolis-Hasting algorithm

• Alias method

• Sampling specific distributions• Exponential distribution

• Normal distribution

• Distributions of directions (i.e., unit vectors)


Recap: Inversion Method

• Given a 1D distribution with CDF F, then

follows this given distribution

• Problem: some distrb.(e.g., normal) have noclosed-form CDFs


Rejection Sampling

• Consider a distribution over with PDF f

• Assume f is bounded so that

• Basic rejection sampling:1. Draw x from U(Ω)

2. Draw y from U(0, M]

3. If , return x

4. Otherwise, go to 1.

• Why valid?• Accepted samples (x, y)

distribute uniformly overthe subgraph of f(x)


Rejection Sampling

• To improve acceptance rate, use an envelop distribution (that can be easily sampled)

• Given , let M be a constant with . The sampling algorithm then becomes:

1. Draw x from


3. If , return x


• can be made piecewisefor better performance


Rejection Sampling

• Generalize naturally to higher dimensions

• Assume pdf and envelop over sample space :

1. Draw x from


3. If , return x



Rejection Sampling

• Pros: only need the PDF f• No CDF or its inverse

• Cons: not all generated samples are accepted• Lower performance, especially for high-dimensional

problems (aka “curse of dimensionality”)

• In practice, only use rejection sampling when there is no better option


Rejection Sampling

• The idea of rejecting generated samples can be used in many other ways

• Example:• To sample a conditional density , one can

generate samples according to and reject those not satisfying the condition y

• Metropolis-Hasting Algorithm


Metropolis-Hasting Algorithm

• A Markov-Chain Monte Carlo (MCMC) method

• Given a non-negative function f, generate a chain of (correlated) samples X1, X2, X3, … that follow a probability density proportional to f

• Main advantage: f does not have to be a PDF (i.e., unnormalized)


Metropolis-Hasting Algorithm

• Input • Non-negative function f

• Probability density suggesting a candidate for the next sample value x, given the previous sample value y

• The algorithm: given current sample Xi

1. Sample X’ from

2. Let and draw

3. If , set Xi+1 to X’; otherwise, set Xi+1 to Xi

• Start with arbitrary initial state X0


Metropolis-Hasting: Example


d = np.random.normal(scale=sigma)ang = np.pi*np.random.rand()X1 = X + np.array([d*np.cos(ang), d*np.sin(ang)])

a = f(X1)/f(X) # a only depends on f as g is symmetricif np.random.rand() < a:

X = X1

Metropolis-Hasting: Example


Metropolis-Hasting: Issues

• The samples are correlated• To obtain independent samples, have to take every

n-th sample: Xn, X2n, X3n, … for some n (determined by examining autocorrelation of adjacent samples)

• Initial samples may follow a different distribution

• Use a “burn-in” period by discarding the first few samples (e.g., first 1000)


Metropolis Light Transport

• Applying the Metropolis-Hasting algorithm to physically-based rendering

• To be discussed later in this course!


[Veach & Guibas 1997] [Kelemen et al. 2002] [Jakob & Marschner 2012]

Alias Method

• Efficient way to sample finite discrete distributions with finite sample spaces

• Assuming • O(N) preprocessing

• O(1) sampling

• Inversion method• O(N) preprocessing (for creating the CMF)

• O(logN) sampling


Alias Method

• Observation: sampling a uniform distribution with N outcomes is easy:

• Can we “convert” a non-uniform one into a uniform one?

• Yes!


Alias Method

• All bins have the probability 1/N to be chosen

• Each bin involves at most two outcomes

• Construction process: O(N)• Repeatedly choose a bin with a probability < 1/N and “steal”

the missing portion from another bin with probability > 1/N to form a bin with exactly two outcomes and probability 1/N


Bin 1 Bin 3Bin 2

Today’s Lecture

• Sampling from PDFs II• General methods

• Rejection sampling

• Metropolis-Hasting algorithm

• Alias method

• Sampling specific distributions• Exponential distribution

• Normal distribution

• Distributions of directions (i.e., unit vectors)


Exponential Distribution

• Sample space:

• PDF: for some λ>0

• CDF:

• Sampling method (inversion)

• More on this distribution later!


Normal Distribution

• Sample space:

• PDF:

• CDF:

• Inversion sampling is possible but requires numerically inverting the error function erf


Normal Distribution

• Box-Muller Transform• Let be drawn independently from U(0, 1]

• Then,

both have standard normal distribution and are independent


Distributions of Directions

• Uniform position on a 2D disc

• Uniform position on the surface of a unit sphere in 3D (i.e., )

• Uniform position on the surface of a unit sphere in (n+1)-dimensional space (i.e., )


Uniform Distribution on 2D DiscSolution 1

• Sample space:

• Using the idea of rejecting samples:

uniformly sample points in the bounding square and reject those not in the disc

1. Draw from U(0, 1)

2.

3. If , return x



1


• Sample space: (polar coordinates)

• Due to symmetry, .Next we focus on sampling r

• Observation: the probabilityfor a sampled point to locatewithin a disc with radius a is

for all


1

a


• Thus, r can be sampled with inversion method:

• Putting everything together:1. Draw from U[0, 1)

2. (polar coordinates)

3. (Cartesian coordinates)


Uniform Distribution onSolution 1

• Sample space:

• Using the idea of rejecting samples:

uniformly sample points in the bounding cube and reject those not in the disc

1. Draw from U(0,1)

2.

3. If , return




• Sample space:(spherical coordinates)

• Due to symmetry, .Next we focus on sampling θ

• Observation: the probability fora sampled point to locate within aspherical cap with angle θ’ is


Surface area:

2πr2(1 – cosθ)


• Thus, θ can be sampled with inversion method:

• Putting everything together:1. Draw from U[0, 1)

2. (spherical coord.)

3. (Cartesian coord.)


Uniform Distribution on

• Sample space:

• General approach1. Draw from N(0, 1) (standard normal)

2.

3. Return

• Why is this valid?• x has the multivariate normal distribution with identity

covariance matrix

• This distribution is rotationally symmetric around the origin


Next Lecture

• The rendering equation

• Monte Carlo path tracing I


sampling from pdfs - donald bren school of information and … · 2017. 4. 18. · •sampling from...

Documents