fourier analysis of stochastic sampling for assessing bias and variance in integration kartic subr,...
TRANSCRIPT
Fourier Analysis of Stochastic Sampling For Assessing Bias and Variance in Integration
Kartic Subr, Jan Kautz University College London
great sampling papers
Spectral analysis of sampling must be
IMPORTANT!
BUT WHY?
numerical integration, you must try
assessing quality: eg. rendering
Shiny ball, out of focusShiny ball in motion
…pixel
multi-dim integral
variance and bias
High variance High bias
bias and variance
High variance High bias
predict as a function of sampling strategy and
integrand
variance-bias trade-off
High variance High bias
analysis is non-trivial
Abstracting away the application…
0
numerical integration implies sampling
0sampled integrand
(N samples)
numerical integration implies sampling
0sampled integrand
the sampling function
integrand
sampling functionsampled integrand
multiply
sampling func. decides integration quality
integrandsampled function
multiplysampling function
strategies to improve estimators
1. modify weights
eg. quadrature rules
strategies to improve estimators
1. modify weights
eg. importance sampling
2. modify locations
eg. quadrature rules
abstract away strategy: use Fourier domain
1. modify weights 2. modify locations
eg. quadrature rulesanalyse sampling function in Fourier domain
abstract away strategy: use Fourier domain
1. modify weights
a. Distribution eg. importance sampling)
2. modify locations
eg. quadrature rules
sampling function in the Fourier domain
frequency
amplitude (sampling spectrum)
phase (sampling spectrum)
stochastic sampling & instances of spectra
Sampler (Strategy 1)
Fouriertransform
draw
Instances of sampling functions Instances of sampling spectra
assessing estimators using sampling spectra
Sampler (Strategy 1)
Sampler(Strategy 2)
Instances of sampling functions Instances of sampling spectra
Which strategy is better? Metric?
accuracy (bias) and precision (variance)
estimated value (bins)
freq
uenc
yreference
Estimator 2
Estimator 1
Estimator 2 has lower bias but higher variance
overview
related work
signal processing
assessing sampling patterns
spectral analysis of integration
Monte Carlo sampling
Monte Carlo rendering
stochastic jitter: undesirable but unavoidable
signal processing
Jitter [Balakrishnan1962] Point processes [Bartlett 1964] Impulse processes [Leneman 1966]
Shot noise [Bremaud et al. 2003]
we assess based on estimator bias and variance
assessing sampling patterns
Point statistics [Ripley 1977] Frequency analysis [Dippe&Wold 85, Cook 86, Mitchell 91] Discrepancy [Shirley 91]
Statistical hypotheses [Subr&Arvo 2007]
Others [Wei&Wang 11,Oztireli&Gross 12]
recent and most relevant
spectral analysis of integration
numerical integration schemes [Luchini 1994; Durand 2011] errors in visibility integration [Ramamoorthi et al. 12]
recent and most relevant
spectral analysis of integration
numerical integration schemes [Luchini 1994; Durand 2011] errors in visibility integration [Ramamoorthi et al. 12]
1. we derive estimator bias and variance in closed form2. we consider sampling spectrum’s phase
Intuition(now)
Formalism(paper)
sampling function = sum of Dirac deltas
+
+
+
Review: in the Fourier domain …
primal Fourier
Dirac deltaFourier transform
Frequency
Real
Imaginary
Complex plane
amplitudephase
Review: in the Fourier domain …
primal Fourier
Dirac deltaFourier transform
Frequency
Real
Imaginary
Complex plane
Real
Imaginary
Complex plane
amplitude spectrum is not flat
=
+
+
+
primal Fourier
=
+
+
+
Fourier transform
sample contributions at a given frequency
Real
Imaginary
Complex plane
5
1 2 3 4 5
At a given frequency
3
2
4
1
sampling function
the sampling spectrum at a given frequency
sampling spectrum
Complex plane
53
2
4
1
centroid
given frequency
the sampling spectrum at a given frequency
sampling spectrum instances
expected centroid centroid variancegiven frequency
expected sampling spectrum and variance
expected amplitude of sampling spectrum variance of sampling spectrum
frequency
DC
intuition: sampling spectrum’s phase is key
• without it, expected amplitude = 1!– for unweighted samples, regardless of distribution
• cannot expect to know integrand’s phase– amplitude + phase implies we know integrand!
Theoretical results
Result 1: estimator bias
bias
reference
inner product
frequency variable
S f
sampling spectrum integrand’s spectrum
Implications
1. S non zero only at 0 freq. (pure DC) => unbiased estimator
2. <S> complementary to f keeps bias low
3. What about phase?
expanded expression for bias
bias
expanded expression for bias
reference
bias
phase
amplitude
Sf fS
omitting phase for conservative bias prediction
reference
bias
phase
amplitude
Sf fS
new measure: ampl of expected sampling spectrum
ours periodogram
Result 2: estimator variance
variance
frequency variableinner product
S || f ||2
sampling spectrum integrand’s power spectrum
the equations say …
• Keep energy low at frequencies in sampling spectrum– Where integrand has high energy
case study: Gaussian jittered sampling
1D Gaussian jitter
samples
jitter using iidGaussian distributed 1D random variables
1D Gaussian jitter in the Fourier domain
real
Imaginary Complex planeFourier transformed samples at an arbitraryfrequency
Jitter in position manifests as phase jitter
centroid
derived Gaussian jitter properties
• any starting configuration
• does not introduce bias
• variance-bias tradeoff
Testing integration using Gaussian jitter
random points
binary function p/w constant function p/w linear function
bias-variance trade-off using Gaussian jitter
bias
varia
nce
Gaussian jitter
random
grid
Poisson disklow-discrepancy Box jitter
Gaussian jitter converges rapidly
Log-number of primary estimates
log-
varia
nce
Gaussian jitter
Random: Slope = -1O(1/N)
Poisson disklow-discrepancyBox jitter
Conclusion: Studied sampling spectrum
sampling spectrum
integrand spectrum
integrand
sampling function
Conclusion: bias
sampling spectrum
integrand spectrum
integrand
sampling function
bias depends on E( ) .
Conclusion: variance
sampling spectrum
integrand spectrum
integrand
sampling function
bias depends on E( ) .
variance is V( ) .2
Acknowledgements
Take-home messages
53
2
4
1
relative phase is key Ideal sampling spectrum
No energy in sampling spectrumat frequencies where integrand has high energy
Questions?
http://www.wordle.net/show/wrdl/6890169/FMCSIG13
Sorry, what? Handling finite domain?
• Integrand = integrand * box
conclusion