1

On the Statistical Analysis of Dirty PicturesJulian Besag

2

Image Processing

Required in a very wide range of practical problemsComputer visionComputer tomographyAgricultureMany more…

Picture acquisition techniques are noisy

3

Problem Statement

Given a noisy picture And 2 source of information (assumptions)

A multivariate record for each pixelPixels close together tend to be alike

Reconstruct the true scene

4

Notation

S – 2D region, partitioned into pixels numbered 1…n

x = (x1, x2, …, xn) – a coloring of S x* (realization of X) – true coloring of S y = (y1, y2, …, yn) (realization of Y) –

observed pixel color

5

Assumption #1

Given a scene x, the random variables Y1, Y2, …, Yn are conditionally independent and each Yi has the same known conditional density function f(yi|xi), dependent only on xi.

Probability of correct acquisition

n

iii xyfxyl

1

)|()|(

6

Assumption #2

The true coloring x* is a realization of a locally dependant Markov random field with specified distribution {p(x)}

7

Locally Dependent M.r.f.s

Generally, the conditional distribution of pixel i depends on all other pixels, {S\i}

We are only concerned with local dependencies

iSi

xxpxxP iiiiSi

\

)|()|( \

8

Previous Methodology

Maximum Probability Estimation Classification by Maximum Marginal

Probabilities

9

Maximum Probability Estimation

Chose an estimate x such that it will have the maximum probability given a record vector y.

In Bayesian framework x is MAP estimate In decision theory – 0-1 loss function

)()|()|( xpxylyxP

n

iii xyfxyl

1

)|()|(

10

Maximum Probability Estimation

Iterate over each pixel Chose color xi at pixel i from probability

Slowly decreasing T will guarantee convergence

TiiiiiiSiiT xxpxyfxyxP

1

\ )|()|(),|(

11

Classification by Maximum Marginal Probabilities Maximize the proportion of correctly

classified pixels

Note that P(xi | y) depends on all records Another proposal: use a small

neighborhood for maximizationStill computationally hard because P is not

available in closed form

iSx

i xpxylyxP\

)()|()|(

12

Problems

Large scale effectsFavors scenes of single color

Computationally expensive

13

Estimation by Iterated Conditional Modes The previously discussed methods have

enormous computational demands, and undesirable large-scale properties.

We want a faster method with good large-scale properties.

14

Iterated Conditional Modes

)|( maximizingjust by given is ˆ of estimate Initial

. pixel of neighbors

8 gsurroundin given the color theofy probabilit the)ˆ|(

color given the record observedan ofy probabilit the)|(

)ˆ|()|()ˆ,|( argmax

ninformatio available allgiven pixeleach at ˆ update :Goal

* xscene true theof estimate lprovisiona a ˆ

\

ii

iiii

iiii

iiiiiiSix

i

xyfx

i

xxxp

xyxyf

xxpxyfxyxP

ix

x

i

When applied to each pixel in turn, this procedure defines a single cycle of an iterative algorithm for estimating x*

15

Examples of ICM

Each example involves: c unordered colors Neighborhood is 8 surrounding pixels A known scene x* At each pixel i, a record yi is generated from a

Gaussian distribution with mean and variance κ.

)( *ix

16

The hillclimbing update step

penalty. smoothness a is termsecond the

values,fitted and observed matching involves first term The

color theofmean the)(

pixelat color observed the

used.often most value theis 5.1

color. like have toneighbors gencouraginparameter fixed a

.color having of neighbors ofnumber current the)(ˆ

)(ˆ)(2

1 argmin

:requires ICM of cycle singleA

2

ii

i

ii

iiix

xx

iy

xixu

xuxyi

17

Extremes of β

β = 0 gives the maximum likelihood classifier, with which ICM is initialized

β = ∞, xi is determined by a majority vote of its neighbors, with yi records only used to break ties.

18

Example 1

6 cycles of ICM were applied, with β = 1.5

19

Example 2

Hand-drawn to display a wide array of features yi records were generated by superimposing

independent Gaussian noise, √κ = .6 8 cycles, β increasing from .5 to 1.5 over the 1st 6

20

Models for the true scene

Most of the material here is speculative, a topic for future research

There are many kinds of images possessing special structures in the true scene.

What we have seen so far in the examples are discrete ordered colors.

21

Examples of special types of images Unordered colors

These are generally codes for some other attribute, such as crop identities

Excluded adjacencies It may be known that certain colors cannot

appear on neighboring pixels in the true scene.

22

More special cases…

Grey-level scenesColors may have a natural ordering, such as

intensity. The authors did not have the computing equipment to process, display, and experiment with 256 grey levels.

Continuous intensities{p(x)} is a Gaussian M.r.f. with zero mean

23

More special cases…

Special features, such as thin linesAuthor had some success reproducing

hedges and roads in radar images. Pixel overlap

24

Parameter Estimation

This may be computationally expensive This is often unnecessary We may need to estimate θ in l(y|x; θ)

Learn how records result from true scenes. And we may need to estimate Φ in p(x;Φ)

Learn probabilities of true scenes.

25

Parameter Estimation, cont.

Estimation from training data Estimation during ICM

.ˆ of econvergenc untilor cycles ofnumber fixed afor 2 Return to 5.

.ˆ newobtain toˆ and ,ˆ ,ˆcurrent on based ICM, of cycle single aout Carry 4.

)ˆmaxarg

:ˆ new aobtain toˆcurrent on likelihood-pseudo maximumby Estimate 3.

)ˆ maximizes which ˆ valueby the Estimate 2.

. and guessing scene, true theof ˆ estimate initialObtain .1

ˆ

x

xx

;|x(xp

x

;θxl(y|θ

x

iiSi

26

Example of Parameter Estimation

Records produced with Gaussian noise, κ = .36 Correct value of κ , gradually increasing β gives 1.2% error Estimating β = 1.83 and κ = .366 gives 1.2% error κ known but β = 1.8 estimated gives 1.1% error

27

Block reconstruction

pixels. ofblock small a represents where,ˆˆ maximize

toˆ choose ,ˆˆ maximize toˆ choosing of Instead

B)x|y,xP(

x)x|y,xP(x

S\BB

BS\iii

Suppose the Bs form 2x2 blocks of four, with overlap between blocks

At each stage, the block in question must be assigned one of c4 colorings, based on 4 records, and 26 direct and diagonal adjacencies:

28

Block reconstruction example

Univariate Gaussian records with κ = .9105 Basic ICM with β = 1.5 gives 9% error rate ICM with β = ∞ estimated gives 5.7% error

29

Conclusion

We began by adopting a strict probabilistic formulation with regard to the true scene and generated records.

We then abandoned these in favor of ICM, on grounds of computation and to avoid unwelcome large-scale effects.

There is a vast number of problems in image processing and pattern recognition to which statisticians might usefully contribute.