variance reduction and brownian simulation methods yossi shamai raz kupferman the hebrew university
Post on 19-Dec-2015
223 views
TRANSCRIPT
Variance reduction and Brownian Simulation Methods
Yossi Shamai
Raz Kupferman
The Hebrew University
All (incompressible) fluids are governed by mass-momentum conservation equations
u(x,t) = velocity
(x,t) = polymeric stress
QuickTime™ and a decompressor
are needed to see this picture.
QuickTime™ and a decompressor
are needed to see this picture.
QuickTime™ and a decompressor
are needed to see this picture.
Dumbbell models
Dumbbell models
(q,x,t) = pdf.
The polymers are modeled by two beads connected by a spring (dumbbell) . The conformation is modeled by an end-to-end vector q.
less affect
more affect
q
The (random) conformations are distributed according to a density function (q,x,t), which satisfies an evolution equation
advection deformation diffusion
The stress is an ensemble average of polymeric conformations,
QuickTime™ and a decompressor
are needed to see this picture.
q
g(q) = qF(q)
The stress
Conservation laws (macroscopic dynamics)
Polymeric density distribution (microscopic dynamics)
• Problem: high dimensionality
QuickTime™ and a decompressor
are needed to see this picture.
QuickTime™ and a decompressor
are needed to see this picture.
• Assumption: 1-D
Closable systemsIn certain cases, a PDE for (x,t) can be derived, yielding a closed-form system for u(x,t), (x,t).
QuickTime™ and a decompressor
are needed to see this picture.
• Closable systems can be solved by standard methods.• Brownian simulations can be used for non-closable systems.
Example (Semi-linear systems): if g(q) = q2, and b(q,u) = b(u) q then (x,t) satisfies the PDE
QuickTime™ and a decompressor
are needed to see this picture.
Outline
1. Brownian simulation methods
2. Some mathematical preliminaries on spatial correlations
3. A variance reduction mechanism in Brownian simulations
4. Examples
QuickTime™ and a decompressor
are needed to see this picture.
Brownian simulationsThe average stress (x,t) is an expectation with respect to a stochastic process q(x,t) with PDF (q,x,t)
q(x,t) is simulated by a collection of realizations qi(x,t).
The stress is approximated by an empirical mean QuickTime™ and a
decompressorare needed to see this picture.
PDE SPDE
A reminder: real-valued Brownian motion
1. B(t) is a random function of time.
2. Almost surely continues.
3. Independent increments.
4. B(t)-B(s) ~ N(0,t-s).L2-valued Brownian motion
1. B(x,t) is a random function of time and space.
2. For fixed x, B(x,t) is a real-valued Brownian motion.
3. Finite normQuickTime™ and a
decompressorare needed to see this picture.
Spatial correlationsB(x,t) is characterized by the spatial correlation function
QuickTime™ and a decompressor
are needed to see this picture.
1. Symmetry: c(x,y) = c(y,x).
2. c(x,x) = 1.
3. L2 - function:
QuickTime™ and a decompressor
are needed to see this picture.
Spatial correlations (cont.)An L2 - function is a correlation function iff
a. c(x,x) = 1.
b. It has a “square root” in L2
QuickTime™ and a decompressor
are needed to see this picture.
Oscillatory
Discretization QuickTime™ and a decompressor
are needed to see this picture.
UniformQuickTime™ and a
decompressorare needed to see this picture.
Piecewise constant uncorrelated
QuickTime™ and a decompressor
are needed to see this picture.
QuickTime™ and a decompressor
are needed to see this picture.
• No spatially uncorrelated L2-valued Brownian motion.
• Spatially uncorrelated noise has meaning only in a discrete setting. It is a sequence of piecewise constant standard Brownian motions, uncorrelated at any two distinct steps, that converges to 0.
QuickTime™ and a decompressor
are needed to see this picture.
Spatial correlations (cont.)
Spatial correlations (cont.)Spatial correlations can be alternatively
described by Correlation operators
QuickTime™ and a decompressor
are needed to see this picture.
QuickTime™ and a decompressorare needed to see this picture. QuickTime™ and a decompressorare needed to see this picture.
• C is nonnegative, symmetric and trace class.
QuickTime™ and a decompressor
are needed to see this picture.
• For any f,g in L2
• No Id-correlated Brownian motion (trace Id = ∞ ).
QuickTime™ and a decompressor
are needed to see this picture.
SDEs versus SPDEs
QuickTime™ and a decompressor
are needed to see this picture.
SDEs (Stochastic Differential Equations)
QuickTime™ and a decompressor
are needed to see this picture.
QuickTime™ and a decompressor
are needed to see this picture.
• F,G are operators
SPDEs (Stochastic Partial Differential Equations)
Ito’s integral
Ito’s integral
QuickTime™ and a decompressor
are needed to see this picture.
QuickTime™ and a decompressor
are needed to see this picture.QuickTime™ and a decompressorare needed to see this picture.
• q(x,t) has spatial correlation.
PDE (Fokker-Plank)
SDE
SDEs
QuickTime™ and a decompressor
are needed to see this picture.
QuickTime™ and a decompressor
are needed to see this picture.
PDE (Fokker-Plank)
SPDE
SPDEsQuickTime™ and a
decompressorare needed to see this picture.
SDEs versus SPDEs
Brownian simulationsunifying approach
• Equivalence class insensitive to spatial correlations.
€
QuickTime™ and a decompressor
are needed to see this picture.
• Consistency: for every x, q(x,0) ~ (q,x,0).
QuickTime™ and a decompressor
are needed to see this picture.
Lemma: Let (u,,q) be a solution for the stochastic system on some time interval [0,T]. Let (q,x,t) be the PDF corresponding to q(x,t). Then (u,,) is a solution for the deterministic system on [0,T].
QuickTime™ and a decompressor
are needed to see this picture.
Brownian simulation methodsThe stochastic process q is simulated by n “realizations” driven by i.i.d Brownian motions. Expectation is approximated by an empirical mean with respect to the realizations: QuickTime™ and a
decompressorare needed to see this picture.
QuickTime™ and a decompressor
are needed to see this picture.
Advantages: 1. No Fokker-plank equation.
2. Easy to simulate.
Disadvantages:1. No error
analysis.2. Variance is
O(n-1).
QuickTime™ and a decompressor
are needed to see this picture.
Brownian simulation methodsThe approximation The system
CONNFFESSIT (Calculations of Non Newtonian Fluids Finite Elements and Stochastic Simulation Techniques) - Piecewise constant uncorrelated noise (Ottinger et al. 1993)
BCF - Spatially uniform noise (Hulsen et al. 1997)
QuickTime™ and a decompressor
are needed to see this picture.
Correlation affects approximation but not the exact solution
Error reduction ?
1. Prove that e(n,t)0.
2. Reduce the error by choosing the spatial correlation of the Brownian noise:
Step 1. Express e(n,t) as a function F(c).
Step 2. Minimize F(c).
The error of the Brownian simulations is
Goals
QuickTime™ and a decompressor
are needed to see this picture.
The idea of adapting correlation to minimize variance first proposed by Jourdain et al. (2004) in the context of shear flow with a specific FEM scheme.
An “integral-type” system
QuickTime™ and a decompressor
are needed to see this picture.
The Brownian simulation is
QuickTime™ and a decompressor
are needed to see this picture.
Example
Results:
Brownian simulation
The stress
n = 2000 with spatially uniform noise ( c(x,y) = 1 ).
The (normalized) error as a function of time
Large error (1.47)
QuickTime™ and a decompressor
are needed to see this picture.
QuickTime™ and a decompressor
are needed to see this picture.
“smooth” simulation
The Brownian simulation at t=20
(dotted curve)
Brownian simulation
Stress
Results:
The Brownian simulation at t=20
(dotted curve)
“noisy” simulations
n = 2000 with piecewise constant uncorrelated noise.
The (normalized) error as a function of time
reduced error (1.06)
QuickTime™ and a decompressor
are needed to see this picture.
QuickTime™ and a decompressor
are needed to see this picture.
Error analysisWe want to analyze the error of the Brownian simulations
QuickTime™ and a decompressor
are needed to see this picture.
Lets demonstrate the analysis for semi-linear system…
Closable systemsIn certain cases, a PDE for (x,t) can be derived, yielding a closed-form system for u(x,t), (x,t).
QuickTime™ and a decompressor
are needed to see this picture.
• Closable systems can be solved by standard methods.• Brownian simulations can be used for non-closable systems.
Example (Semi-linear systems): if g(q) = q2, and b(q,u) = b(u) q then (x,t) satisfies the PDE
QuickTime™ and a decompressor
are needed to see this picture.
Error analysis for Semi-linear systems
• Linearize (properly)
In semi-linear systems, the stress field (x,t) satisfies a PDE
We want to estimate the error of the Brownian simulations QuickTime™ and a
decompressorare needed to see this picture.
QuickTime™ and a decompressor
are needed to see this picture.
An analogous evolution equation for T(x,t) is derived
QuickTime™ and a decompressor
are needed to see this picture.
QuickTime™ and a decompressor
are needed to see this picture.
QuickTime™ and a decompressor
are needed to see this picture.
Linearized system
QuickTime™ and a decompressor
are needed to see this picture.
andQuickTime™ and a
decompressorare needed to see this picture.
Theorem 1. To leading order:
QuickTime™ and a decompressor
are needed to see this picture.
and k is a kernel function determined by the parameters.
QuickTime™ and a decompressor
are needed to see this picture.
where
QuickTime™ and a decompressor
are needed to see this picture.
The function F can be also expressed in terms of the correlation operator C,
• F is convex
• In principle, the analysis is the same
• Proof is restricted to closable systems
Theorem 1. To leading order in n,
QuickTime™ and a decompressor
are needed to see this picture.
Error analysis for Closable systems
The optimization problem
Minimize F(c) over the domain:
S = {c(x,y) : c has a root in L2, c(x,x) = 1}
Difficulties:
A. Infinite dimensional optimization problem.
B. S is not compact.
• In general, there is no minimizer
Find a sequence of correlations cn S such that F(cn)
converge toQuickTime™ and a
decompressorare needed to see this picture.
Finite dimensional approximations
1. Set a natural k.
2. Discretize the problem to a k-point mesh
QuickTime™ and a decompressor
are needed to see this picture.
QuickTime™ and a decompressor
are needed to see this picture.
Theorem 2. The sequence of errors
converges (as k∞ ) to the optimal error
QuickTime™ and a decompressor
are needed to see this picture.
QuickTime™ and a decompressor
are needed to see this picture.
The F-D optimization problem
The F-D optimization problem is:
Minimize F(A), A is k-by-k symmetric PSD
Subject to Aii = 1, i=1,…,k
We want to minimize F(ck) over Sk.
• ck(x,y) is indexed by k2 mesh points (xi ,xj) (matrix).
• Symmetric Positive-Semi-Definite.
• ck (xi ,xi) = 1.
F is convex SDP algorithms (Semi-Definite Programming)
So what did we do?
Developed a unifying approach for a variance reduction mechanism in Brownian simulations.
Formulated an optimization problem (in infinite dimensions).
Showed that it is amenable to a standard algorithm (SDP).
Example 1
A linear advection-dissipation equation in [0,1].
QuickTime™ and a decompressor
are needed to see this picture.
QuickTime™ and a decompressor
are needed to see this picture.
QuickTime™ and a decompressor
are needed to see this picture.
In stochastic formulation,
QuickTime™ and a decompressor
are needed to see this picture.
The Brownian simulation is
QuickTime™ and a decompressor
are needed to see this picture.
The error is
Variance independent of correlations (no reduction)
Insights: the dynamics (advection and dissipation) do not mix different points in space. Thus, the error only ‘sees’ diagonal elements of the correlations, which are fixed by the constraints.
QuickTime™ and a decompressor
are needed to see this picture.
An “integral-type” system (x[0,1]):
QuickTime™ and a decompressor
are needed to see this picture.
Closable:
Example 2
QuickTime™ and a decompressor
are needed to see this picture.
Results:
Brownian simulation
The stress
n = 2000 with spatially uniform noise ( c(x,y) = 1 ).(BCF)
The (normalized) error as a function of time
Large error (1.47)
QuickTime™ and a decompressor
are needed to see this picture.
QuickTime™ and a decompressor
are needed to see this picture.
“smooth” simulation
The Brownian simulation at t=20
(dotted curve)
Brownian simulation
Stress
Results:
The Brownian simulation at t=20
(dotted curve)
“noisy” simulations
n = 2000 with piecewise constant uncorrelated noise (CONNFFESSIT).
The (normalized) error as a function of time
optimal error (1.06)
QuickTime™ and a decompressor
are needed to see this picture.
QuickTime™ and a decompressor
are needed to see this picture.
Why?…
the optimal error is obtained by taking c 0 (CONNFESSIT).
QuickTime™ and a decompressor
are needed to see this picture.
• g(x,y,t) is singular on the diagonal (x=y), and a smooth positive function off the diagonal.
• c(x,x) = 1
The error is
Example 3: 1-D planar Shear flow model. (Jourdain et al. 2004)
The system:
QuickTime™ and a decompressor
are needed to see this picture.
QuickTime™ and a decompressor
are needed to see this picture.
QuickTime™ and a decompressor
are needed to see this picture.
Closable: set QuickTime™ and a
decompressorare needed to see this picture.
QuickTime™ and a decompressor
are needed to see this picture.
To leading order, the error of the Brownian simulations is
• C - the spatial correlation operator.
QuickTime™ and a decompressor
are needed to see this picture.
• K(t) - a nonnegative bounded operator.
QuickTime™ and a decompressor
are needed to see this picture.
• Any sequence ck 0 yields the optimal error (e.g, spatially constant uncorrelated)
QuickTime™ and a decompressor
are needed to see this picture.
So is CONFFESSIT always optimal?
• No!
We can construct a problem for which e(n,t) = n-1(const + Tr[K(t)C]) for K(t) bounded and not PSD.
Theorem. If the semi-groups are Hilbert-Schmidt (they have L2-kernels) then CONNFFESSIT is optimal.
Some further thoughts…
• The spatial correlation of the initial data q(x,0) may also be considered.
• Non-closable systems?
• Gain insights about the optimal correlation by understand relations between type of equation and optimal correlation.