Particle Filters in high dimensions

Peter Jan van Leeuwen and Mel AdesData-Assimilation Research Centre DARC

University of Reading

Lorentz Center 2011

Data assimilation: general formulation

Solution is pdf!

NO INVERSION !!!

Bayes theorem:

Parameter estimation:

with

Again, no inversion but a direct point-wise multiplication.

Nonlinear filtering: Particle filter

Use ensemble

with the weights.

What are these weights?• The weight is the normalised value of the

pdf of the observations given model state .• For Gaussian distributed variables is is given

by:

• One can just calculate this value• That is all !!!

Standard Particle filter

Not very efficient !

A closer look at the weights I

Probability space in large-dimensional systems is ‘empty’: the curse of dimensionality

u(x1)

u(x2) T(x3)

A closer look at the weights IIAssume particle 1 is at 0.1 standard deviations s of M independent observations.Assume particle 2 is at 0.2 s of the M observations.

The weight of particle 1 will be

and particle 2 gives

A closer look at the weights III

The ratio of the weights is

Take M=1000 to find

Conclusion: the number of independent observations isresponsible for the degeneracy in particle filters.

Increased efficiency: proposal density

Instead of drawing samples from p(x) we draw samples froma proposal pdf q(x).

Use ensemble

with weights

Particle filter with proposal transition

density

Barotropic vorticity equation

256 X 256 grid points600 time stepsTypically q=1-3Decorrelation time scale= = 25 time steps

Observations Every 4th gridpointEvery 50th time step

24 particlessigma_model=0.03 sigma_obs=0.01

Vorticity field standard particle filter

Ensemble mean PFTruth

Vorticity field new particle filter

Ensemble meanTruth

Posterior weights

Rank histogram:How the truth ranks in the

ensemble

Equivalent Weights Particle Filter

Recall:

Assume

Find the minimum for each particle by perturbing each observation, gives

Equivalent Weights Particle filter

Calculate the full weight for each of these

Determine a target weight C that 80% of the particles can reach and determine in

such that

This is a line search, so doable.

Equivalent Weights Particle Filter

• This leads to 80% of the particle having an almost equal weight, so no degeneracy by construction!• Example …

Gaussian pdf in high dimensions

Assume variables are identically independently distributed:

Along each if the axes it looks like a standard Gaussian:

However, the probability mass as function of the distance to thecentre is given by:

d=100 d=400 d=900

r in standard deviation

The so-calledImportant Ring

Why?

In distribution

Fisher has shown

So we find

Importance Ring

r

d

Experimental evidence, sums of d squared random numbers:

Given this what do theseefficient particles represent???

Conclusions

Particle filters with proposal transition density:

• solve for fully nonlinear posterior pdf• very flexible, much freedom• scalable => high-dimensional problems• extremely efficient

• But what do they represent?

We need more people !

• In Reading only we expect to have 7 new PDRA positions available in the this year

• We also have PhD vacancies• And we still have room in the

Data Assimilation and Inverse Methods in Geosciences MSc program