image restoration chapter05 handouts 2 fine

1

Digital Image Processing, 2nd ed.Digital Image Processing, 2nd ed.www.imageprocessingbook.com

© 2002 R. C. Gonzalez & R. E. Woods Somkiat Wangsiripitak

Chapter 5Image Restoration

Chapter 5Image Restoration

The ultimate goal of restoration techniques is to improve an image in some predefined sense.

- image enhancement is largely a subjective process- image restoration is for the most part an objective process

Restoration attempts to reconstruct or recover an image that has been degraded by using a priori knowledge of the degradation phenomenon.- modeling the degradation- applying the inverse process in order to recover the original image

Spatial processing is applicable when the only degradation is additive noise.

Degradations such as image blur are approach in the frequency domain, with filters based on various criteria of optimality.



5.1 A Model of the Image Degradation/Restoration Process


f(x,y) : an input imageg(x,y) : a degraded imageh(x,y) : the degradation functionη(x,y) : the additive noisef(x,y) : an estimate of the original image

The more we know about h and η, the closer f(x,y) will be to f(x,y).

2





If H is a linear, position-invariant process, then

g(x,y) = h(x,y) * f(x,y) + η(x,y) (5.1-1)

G(u,v) = H(u,v) F(u,v) + N(u,v) (5.1-2)

In section 5.2 – 5.4, assume that H is the identity operator, and deal only with degradations due to noise.



5.2 Noise Models5.2 Noise Models

The principal sources of noise in digital images arise during

- image acquisition (digitization) and/or

- transmission.

3



5.2.1 Spatial and Frequency Properties of Noise5.2.1 Spatial and Frequency Properties of Noise

With the exception of spatially periodic noise (Section 5.2.3), assume that noise is - independent of spatial coordinates, and- uncorrelated with respect to the image itself(there is no correlation between pixel values and the values of noise components)

These assumptions are invalid in some applications(quantum-limited imaging, such as in X-ray and nuclear-medicine imaging)

The complexities of dealing with spatially dependent and correlated noise are beyond the scope of our discussion.



5.2.2 Some Important Noise Probability Density Functions


The following are among the most common PDFs found in image processing applications.

Gaussian noise (also called normal) (Fig. 5.2(a))- used frequently in practice.

The PDF of a Gaussian random variable, z, is given by

(5.2-1)

z : gray levelµ : the mean of average value of zσ : its standard deviationσ2 : variance of z

22 2/)(

21)( σµ

σπ−−= zezp

4





Rayleigh noise (Fig. 5.2(b))The PDF of a Rayleigh noise is given by

for z ≥ a (5.2-2)

for z < a

the mean and variance of this density are (5.2-3)

(5.2-4)

The basic shape of this density is skewed to the right.

The Rayleigh density can be quite useful for approximating skewed histograms.

−=

−−

0

)(2)(

/)( 2 bazeazbzp

4)4(

4/

2 πσ

πµ−

=

+=b

ba





Erlang (Gamma) noise (Fig. 5.2(c))The PDF of a Erlang noise is given by

for z ≥ 0 (5.2-5)

for z < 0where a > 0, b : positive integer, ! : factorial

the mean and variance of this density are(5.2-6)

(5.2-7)

Although Eq.(5.2-5) often is referred to as the gamma density, strictly speaking this is correct only when the denominator is the gamma function, Γ(b).

−=−

−

0)!1()(

1az

bb

eb

zazp

22

abab

=

=

σ

µ

5





Exponential noise (Fig. 5.2(d))The PDF of a exponential noise is given by

for z ≥ 0 (5.2-8)

for z < 0where a > 0

the mean and variance of this density function are

(5.2-9)

(5.2-10)

This PDF is a special case of the Erlang PDF, with b=1.

=−

0)(

azaezp

22 1

1

a

a

=

=

σ

µ





Uniform noise (Fig. 5.2(e))The PDF of a uniform noise is given by

if a ≤ z ≤ b (5.2-11)

otherwise

the mean and variance of this density are given by

(5.2-12)

(5.2-13)

−=0

1)( abzp

12)(

22

2 ab

ba

−=

+=

σ

µ

6





Impulse (salt-and-pepper) noise (Fig. 5.2(f))The PDF of (bipolar) impulse noise is given by

for z = a

for z = b (5.2-11)

otherwise

If b > a, gray-level b will appear as a light dot in the image.gray-level a will appear like a dark dot.

If Pa = Pb = 0, the impulse noise is called unipolar.

If Pa = Pb (≠ 0), impulse noise value will resemble salt-and-pepper (or shot-and-spike) granules randomly distributed over the image.

=

0)( b

a

PP

zp





Noise impulses can be negative or positive.scaling usually is part of image digitizing process.

Impulse corruption usually is large compared with the image signal.

pure black (pepper) or white (salt)equal to the minimum and maximum allowed values in the digitized images.

7





The preceding PDFs provide useful tools for modeling a broad range of noise corruption situations found in practice.

Gaussian noise : electronic circuit noise and sensor noise due to poor illumination and/or high temperature

Rayleigh density : characterizing noise phenomena in range imaging.

Exponential and gamma density : application in laser imaging

Impulse noise : quick transients, such as faulty switching, take place during imaging.

Uniform density : useful as the basis for numerous random number generators that are used in simulations



Fig.5.3 : a test pattern for illustrating the noise model



8





The parameters of the noise were chosen in each case so that the histogram corresponding to the three gray levels in the test pattern would start to merge.





See a close correspondence in comparing the histograms in Fig.5.4 with the PDFs in Fig.5.2.

It is difficult to differentiate visually between the first five images in Fig.5.4, even though their histograms are significantly different.

9



5.2.3 Periodic Noise5.2.3 Periodic Noise

Periodic noise in an image arises typically from electrical or electromechanical interference during image acquisition.

spatially dependent noise

Periodic noise can be reduced significantly via frequency domain filtering.



5.2.3 Periodic Noise5.2.3 Periodic Noise

Fig.5.5(a) : corrupted by (spatial) sinusoidal noiseof various frequencies

The Fourier transform of a pure sinusoid is a pair of conjugate impulseslocated at the conjugate frequencies of the sine wave (Table 4.1 (p.211)).

if the amplitude of a sine wave in the spatial domain is strong enough,

expect to see in the spectrum of the image a pair of impulses for each sine wave in the image.

Fig.5.5(b)

10



5.2.4 Estimation of Noise Parameters5.2.4 Estimation of Noise Parameters

The parameters of periodic noise typically are estimated by inspection of the Fourier spectrum of the image.

Visual analysisAutomated analysis is possible in situations in which the noise

spikes are either exceptionally pronounced, or when some knowledge is available

If the imaging system is available,one simple way to study the characteristics of system noise is to capture a set of images of “flat” environments. The resulting images typically are good indicators of system noise.

When only images already generated by the sensor are available,frequently it is possible to estimate the parameters of the PDFfrom small patches of reasonably constant gray level.




Ex. The vertical strips (of 150 x 20 pixels) shown in Fig.5.6 were cropped from the Gaussian, Rayleigh, and uniform images in Fig.5.4.

The shape of these histograms correspond quite closely to the histograms in Fig.5.4.

11




The simplest way to use the data from the image strips is for calculating the mean and variance of the gray levels.

If S is a strip (subimage),

(5.2-15)

(5.2-16)

Zi : the gray-level values of the pixels in Sp(zi) : the corresponding normalized histogram values

The shape of the histogram identifies the closest PDF match.

If the shape is approximately Gaussian, then mean and variance is all we need because the Gaussian PDF is completely specified by these two parameters.

∑

∑

∈

∈

−=

=

Szii

Szii

i

i

zpz

zpz

)()(

)(

22 µσ

µ



5.3 Restoration in the Presence of NoiseOnly-Spatial Filtering

5.3 Restoration in the Presence of NoiseOnly-Spatial Filtering

When the only degradation present in an image is noise

g(x,y) = f(x,y) + η(x,y) (5.3-1)G(u,v) = F(u,v) + N(u,v) (5.3-2)

The noise terms are unknown, so subtracting then is not a realisticoption.

In the case of periodic noise, it usually is possible to estimate N(u,v)from the spectrum of G(u,v), as noted in Section 5.2.3.

N(u,v) can be subtracted from G(u,v) to obtain an estimate of F(u,v)

Spatial filtering is the method of choice in situations when only additive noise is present.

12



Arithmetic mean filter

Sxy : the set of coordinates in a rectangular subimage window of size m x n, centered at point (x,y)

The arithmetic mean filtering process computes the average value of the corrupted image g(x,y) in the area defined by Sxy.

(5.3-3)

Noise is reduced as a result of blurring

5.3.1 Mean Filters5.3.1 Mean Filters

∑∈

=xySts

tsgmn

yxf),(

),(1),(



Geometric mean filter

(5.3-4)

A geometric mean filter achieves smoothing comparable to the arithmetic mean filter, but it tends to lose less image detail in the process.

Harmonic mean filter

(5.3-5)

Works well for salt noise, but fails for pepper noise.It does well also with other types of noise like Gaussian noise.

mn

Ststsgyxf

xy

1

),(),(),(

Π=

∈

5.3.1 Geometric mean filter5.3.1 Geometric mean filter

∑∈

=

xySts tsg

mnyxf

),( ),(1),(

13



Contraharmonic mean filter

(5.3-6)

Q : the order of the filter

This filter is well suited for reducing or virtually eliminating the effects of salt-and-pepper noise.

Positive values of Q eliminates pepper noiseNegative values of Q eliminates salt noise

cannot do both simultaneously

If Q = 0 the arithmetic mean filterIf Q = -1 the harmonic mean filter


∑

∑

∈

∈

+

=

xy

xy

Sts

QSts

Q

tsg

tsgyxf

),(

),(

1

),(

),(),(




(a) X-ray image

(b) Gaussian noisezero meanvariance = 400

(c) Arithmetic mean filter (size 3x3)

(d) Geometric meanfilter (size 3x3)

The geometric mean filter did not blur the image as much as the arithmetic filter

14




(a) Pepper noise withprobability of 0.1

(b) Salt noise with probability of 0.1

(c) filtering (a) using contraharmonicmean filter (Q=1.5)

(d) filtering (b) using (Q= -1.5)

Positive-order filter did a better job of cleaning the background, at the expense of blurring the dark areas.The opposite was true of the negative-order filter.




The arithmetic and geometric mean filters (particularly the latter) are well suited for random noise like Gaussian or uniform noise.

The contraharmonic filter is well suited for impulse noisemust be known whether the noise is dark or light in order to select the proper sign for Q.

Fig.5.9 : The result of choosing the wrong sign for Q.

15



5.3.2 Order-Statistics Filters5.3.2 Order-Statistics Filters

Order-statistics filters are spatial filters whose response is based on ordering (ranking) the pixels contained in the image area encompassed by the filter.

Median filterreplaces the value of a pixel by the median of the gray levels in the

neighborhood of that pixel

(5.3-7)

For certain types of random noise, they provide excellent noise-reduction capabilities, with considerably less blurring than linear smoothing filters of similar size.

)},({),(),(

tsgmedianyxfxySts ∈

=




Max and min filtermax filter

(5.3-8)

- Useful for finding the brightest points in an image.- Because pepper noise has very low values,

it is reduced by this filter.

min filter

(5.3-9)

- Useful for finding the darkest points in an image.- Reduces salt noise.

)},({max),(),(

tsgyxfxySts ∈

=

)},({min),(),(

tsgyxfxySts ∈

=

16




Midpoint filtercomputes the midpoint between the maximum and minimum values

in the area.

(5.3-10)

This filter combines order statistics and averaging.

This filter works best for randomly distributed noise, like Gaussian or uniform noise.

+=

∈∈)},({min)},({max

21),(

),(),(tsgtsgyxf

xyxy StsSts




Alpha-trimmed mean filter- delete the d/2 lowest and the d/2 highest gray-level values of g(s,t)

in the neighborhood Sxy.- gr(s,t) represent the remaining mn - d pixels.- a filter formed by averaging these remaining pixels is called an

alpha-trimmed mean filter.(5.3-11)

d can range from 0 to mn – 1.d = 0 the arithmetic mean filter.d = (mn-1) a median filter.Other value of d useful in situations involving multiple types of

noise, such as a combination of salt-and-pepper and Gaussian noise

∑∈−

=xySts

r tsgdmn

yxf),(

),(1),(

17




(b) One pass with the median filter.(Several noise points are still visible.)

(c) Second pass with the median filter.

(d) Third pass

Repeated passes of a median filter tend to blur the image.

It is desirable to keep the number of passes as low as possible.




(a) Applying the max filter to the pepper noise image of Fig.5.8(a)- also removed (set to a light gray level) some dark pixels.

(b) Applying the min filter to the salt noise image in Fig.5.8(b)- in this particular case, did a better job than the max filter on

noise removal.- removed some white points around the border of light objects

18




(a) Uniform noise (variance 800, zero mean)(b) Further addition of salt-and-pepper noise

(Pa = Pb = 0.1)(c) Arithmetic mean filter (size 5x5)(d) Geometric mean filter (size 5x5)(e) Median filter (size 5x5)(f) alpha-trimmed mean (size 5x5, with d=5)

The arithmetic and geometric mean filters do not do well because of the presence of impulse noise.

The alpha-trimmed filter giving slightly better noise reduction than median filter.



5.3.3 Adaptive Filters5.3.3 Adaptive Filters

The filters discussed thus far are applied to an image without regard for how image characteristics vary from one point to another.

Adaptive filter whose behavior changes based on statistical characteristics of the image inside the filter region.

- Performance superior to that of the filters discussed thus far- an increase in the filter complexity.

(we still are dealing with the case in which the degraded image is equal to the original image plus noise.)

19




Adaptive, local noise reduction filter

The simplest statistical measures of a random variable are- its mean (gives a measure of average gray level in the region)- its variance (givers a measure of average contrast in that region)

Operate on a local region, Sxy

The response of the filter at any point (x,y) is to be based on four quantities.(a) g(x,y) : the value of the noisy image at (x,y)(b) σ2

η : the variance of the noise corrupting f(x,y) to form g(x,y)(c) mL : the local mean of the pixels in Sxy(d) σ2

L : the local variance of the pixels in Sxy




Operation

(1) if σ2η = 0 the filter should return simply the value of g(x,y)

(zero-noise case : g(x,y) = f(x,y) )

(2) if the local variance is high relative to σ2η

the filter should return a value close to g(x,y)(the high local variance typically is associated with edges, and these should be preserved.)

(3) if the two variances are equalthe filter should return the arithmetic mean value of the pixels in Sxy(the local area has the same properties as the overall image,and the local noise is to be reduced simply by averaging.)

20



An adaptive expression:(5.3-12)

The only quantity that needs to be known or estimated is the variance of the overall noise, ση2 . The other parameters are computed from the pixels in Sxy.

We seldom have exact knowledge of ση2 .it is possible for the condition (ση2 ≤ σL

2) to be violated in practice.a test should be built into an implementation of Eq.(5.3-12) so that the ratio is set to 1 if the condition ση2 > σL

2 occurs.This makes this filter nonlinear, but it prevents negative gray levels.

Another approach is to allow the negative values to occur, and then rescale the gray level values at the end.

a loss of dynamic range in the image


[ ]LL

myxgyxgyxf −−= ),(),(),( 2

2

σση




(a) Gaussian noise (zero mean, variance 1000)

(b) Arithmetic mean filter(size 7 x 7)

(c) Geometric mean filter(size 7 x 7)

(d) Using the adaptive filter (size 7 x 7) with ση2 = 1000.much sharper than (b) and (c)

If the quantity ση2 is not known and an estimates used that is too low, the algorithm will return an image that closely resembles the original because the corrections will be smaller than they should be.

21




Adaptive median filter

The median filter discussed in Section 5.3.2 performs well as long as the spatial density of the impulse noise is not large (Pa and Pb < 0.2).

The adaptive median filtering can handle impulse noise with probabilities even larger than these.- preserve detail while smoothing nonimpulse noise (“traditional” median filter does not do)

- the adaptive median filter changes (increases) the size of Sxy during filter operation, depending on certain conditions.




The key to understanding the mechanics of this algorithm is to keep in mind that it has three main purposes:- to remove salt-and-pepper (impulse) noise,- to provide smoothing of other noise that may not be impulsive,- and to reduce distortion, such as excessive thinning or thickening of object boundaries..

zmin = minimum gray level in Sxyzmax = maximum gray level in Sxyzmed = median of gray levels in Sxyzxy = gray level at coordinates (x,y)Smax = maximum allowed size of Sxy

22




The adaptive median filtering algorithm.

Level A : A1 = zmed - zminA2 = zmed - zmax

If A1 > 0 AND A2 < 0, Go to level BElse increase the window sizeIf window size ≤ Smax repeat level AElse output zxy .

Level B : B1 = zxy - zminB2 = zxy - zmaxIf B1 > 0 AND B2 < 0, output zxyElse output zmed

considered to be “impulselike” noiseconsidered to be “impulselike” noise

determine if zmed is an impulse or not

true (zmin < zmed < zmax) not impulse go to level B

see if zxy is an impulse

true (zmin < zxy < zmax) not impulseoutput zxy (unchanged pixel value)

false (zxy = zmin or zxy = zmax) impulse output zmed (not impulse)

false impulse increases the size of the window and repeats level A.

This looping continues until the algorithm either finds a median value that is not an impulse (and branches to level B), or the maximum window size is reached (returns the value of zxy).




No guarantee that this value is not an impulse

The smaller the noise probabilities Pa and/or Pb are, or the larger Smaxis allowed to be, the less likely it is that a premature exit condition will occur.

As the density of the impulses increases, it stands to reason that we would need a larger window to “clean up” the noise spikes.

23




The adaptive filter preserved sharpness and detail.

The choice of maximum allowed window size depends on the application, but a reasonable starting value can be estimated by experimenting with various size of the standard median filter first.

(a) Salt-and-pepper noise with Pa = Pb = 0.25

(b) 7x7 median filter (loss of detail in the image)

(c) Adaptive median filtering with Smax = 7



5.4 Periodic Noise Reduction by Frequency Domain Filtering

5.4 Periodic Noise Reduction by Frequency Domain Filtering

- Bandreject filter

- Bandpass filter

- Notch filter

Tools for periodic noise reduction or removal.

24



5.4.1 Bandreject Filters5.4.1 Bandreject Filters

An ideal bandreject filter :

(5.4-1)

D(u,v) : the distance from the origin of the centered frequency rectangle.W : the width of the bandD0 : its radial center

+>

+≤≤−

−<

=

2),(1

2),(

20

2),(1

),(

0

00

0

WDvuDif

WDvuDWDif

WDvuDif

vuH




A Butterworth bandreject filter :

(5.4-2)

A Gaussian bandreject filter :

(5.4-3)

n

DvuDWvuD

vuH 2

20

2 ),(),(1

1),(

−

+

=

220

2

),(),(

21

1),(

−−

−= WvuDDvuD

evuH

25




(a) Sinusoidal noise of various frequencies

(b) Spectrum of (a).the pairs of bright dots are the noise components.

(c) Butterworth bandrejectfilter of order 4

(d) Result of filtering.

It would not be possible to get equivalent results by a direct spatial domain filtering approach using small convolution masks.



5.4.2 Bandpass Filters5.4.2 Bandpass Filters

A bandpass filter performs the opposite operation of a band reject filter.Hbp(u,v) = 1 – Hbr(u,v) (5.4-4)

• Bandpass filtering generally remove too much image detail.• Bandpass filtering is quite useful in isolating the effect on an image of

selected frequency bands.

This image was generated by(1) Using Eq.(5.4-4) to obtain the bandpass filter

corresponding to the band reject filter used in the previous example

(2) taking the inverse transform of the band pass-filtered transform.Most image detail was lost, but the information that remains is most useful. noise pattern.

Bandpass filtering helped isolate the noise pattern.

26



5.4.3 Notch Filters5.4.3 Notch Filters

Fig.5.18

Due to the symmetry of the Fourier transform, notch filters must appear in symmetric pairsabout the origin.

The one exception is if the notch filter is located at the origin.




The transfer function of an ideal notch reject filter :

(5.4-5)

Where(5.4-6)(5.4-7)

The transfer function of a Butterworth notch reject filter of order n :

(5.4-8)

A Gaussian notch reject filter :

(5.4-9)

These three filters become highpass filters if u0 = v0 = 0.

otherwiseDvuDorDvuDif

vuH 0201 ),(),(10

),(≤≤

=

[ ][ ] 2/12

02

02

2/120

201

)2/()2/(),(

)2/()2/(),(

vNvuMuvuD

vNvuMuvuD

−−+−−=

−−+−−=

n

vuDvuDD

vuH

+

=

),(),(1

1),(

21

20

−

−=20

21 ),(),(21

1),( DvuDvuD

evuH

27




Obtain notch filters that pass, rather than suppress, the frequencies contained in the notch areas, by

Hnp(u,v) = 1 – Hnr(u,v) (5.4-10)




(a) The nearly horizontal lines of the noise pattern.We expect its contribution in the frequency

domain to be concentrated along the vertical axis.

(b) Spectrum of (a).

(c) Ideal notch pass filter. (used to obtain noise pattern)

(d) Noise pattern. (corresponds closely to the pattern in (a) )

(e) Result of corresponding notch reject filter

28



5.4.4 Optimum Notch Filtering5.4.4 Optimum Notch Filtering

Clearly defined interference patterns are not common.

Fig. 5.20(a) The interference pattern is quit similar to the one shown in Fig.5.16(a), but the former pattern is considerably more subtle and, consequently, harder to detect in the frequency plane.

Fig. 5.20(b) : the Fourier spectrum of the image (a).The starlike components were caused by the interference, and several pairs of componentsare present, indicating that the pattern contained more than just one sinusoidal component.




The optimum notch filtering is optimum, in the sense that it minimizes local variances of the restored estimate f(x,y).

The procedure consists of

- first isolating the principal contributions of the interference pattern

- and then subtracting a variable, weighted portion of the pattern from the corrupted image

29




The procedure :

- Placing a notch pass filter, H(u,v), at the location of each spike.the Fourier transform of the interference noise pattern is

N(u,v) = H(u,v)G(u,v) (5.4-11)

- The corresponding pattern in the spatial domain is obtain from(5.4-12)

- The effect of components not present in the estimate of η(x,y) can be minimized instead by subtracting from g(x,y) a weighted portion of η(x,y) to obtain an estimate of f(x,y)

(5.4-13)

w(x,y) is called a weighting or modulation function. Select this function so that the result is optimized in some meaningful way.

{ }),(),(),( 1 vuGvuHvu −ℑ=η

),(),(),(),(ˆ yxyxwyxgyxf η−=




One approach is to select w(x,y) so that the variance of the estimate f(x,y) is minimized over a specified neighborhood of every point (x,y).

w(x,y) is

(5.4-21)

where ā(x,y) is the average value of a in the neighborhood.(see p.251-252 for the derivation of eq.(5.4-21))

To obtain the restored image f(x,y)- compute w(x,y) from Eq.(5.4-21)- use Eq.(5.4-13)

),(),(

),(),(),(),(),( 22 yxyx

yxyxgyxyxgyxwηη

ηη

−

−=

30




As w(x,y) is assumed to be constant in a neighborhood, computing this function for every value of x and y in the image is unnecessary.

w(x,y) is computed for one point in each nonoverlapping neighborhood (preferably the center point) and then used to process all the image points contained in that neighborhood.

Fig.5.21 – 5.23 a neighborhood of size (2a+1) by (2b+1)a = b = 15 .

Fig.5.21 :Fourier spectrum (without shifting) of the image shown in Fig.5.20(a).




(a) The spectrum of N(u,v), where only the spikes are present.(b) Noise interference pattern η(x,y) obtained by taking the inverse

Fourier transform of N(u,v)

Note the similarity between this pattern and the structure of the noise present in Fig.5.20(a).

31




The processed image obtained by using E1.(5.4-13).



5.5 Linear, Position-Invariant Degradations5.5 Linear, Position-Invariant Degradations

Degradation model:g(x,y) = H[ f(x,y) ] + η(x,y) (5.5-1)

Assume that η(x,y)=0 so that g(x,y) = H[ f(x,y) ].

H is linear ifH[ af1(x,y) + bf2(x,y) ] = aH[f1(x,y) ] + bH[ f2(x,y) ] (5.5-2)

where a and b are scalars.

If a=b=1, Eq.(5.5-2) becomesH[ f1(x,y) + f2(x,y) ] = H[ f1(x,y) ] + H[ f2(x,y) ] (5.5-3)

which is called the property of additivity.

with f2(x,y)=0, Eq.(5.5-2) becomesH[ af1(x,y) ] = aH[ f1(x,y) ] (5.5-4)

which is called the property of homogeneity.

32




A linear operator possesses both the property of additivity and the property of homogeneity.

g(x,y) = H[ f(x,y) ] is said to be position (or space) invariant ifH[ f(x-α, y-β) ] = g(x-α, y-β) (5.5-5)

This definition indicates that the response at any point in the image depends only on the value of the input at that point, not on its position.

¦¦

g(x,y) = h(x,y) * f(x,y) + η(x,y) (5.5-16)

G(u,v) = H(u,v)F(u,v) + N(u,v) (5.5-17)




In summary, a linear, spatially-invariant degradation system with additive noise - can be modeled in the spatial domain as the convolution of the

degradation (point spread) function with an image, followed by η(x,y).- (the convolution theorem), the same process can be expressed in the

frequency domain as the product of the transforms of the image and degradation, followed by the addition of N(u,v).

This chapter focuses on linear, space-invariant restoration techniques,because degradations are modeled as being the result of convolution, and restoration seeks to find filters that apply the process in reverse, the term image deconvolution is used frequently to signify linear image restoration.

The filter used in the restoration often are called deconvolution filters.

33



5.6 Estimating the Degradation Function5.6 Estimating the Degradation Function

Three principal ways to estimate the degradation function use inimage restoration :

(1) Observation(2) Experimentation(3) Mathematical modeling

The process of restoring an image by using a degradation function that has been estimated in some way sometimes is called blind deconvolution.



5.6.1 Estimation by Image Observation5.6.1 Estimation by Image Observation

One way is to gather information from the image itself. Ex.- If the image is blurred, we can look at a small section of the image

containing simple structures, like part of an object and the background.- We would look for areas of strong signal content.- We can construct an unblurred image of the same size and

characteristics as the observed subimage. gs(x,y) : observed subimagefs(x,y) : constructed subimage

(our estimate of the original image in that area) assuming that the effect of noise is negligible because of choice of a strong-signal area.

(5.6-1)assuming position invariance, we can construct a complete function H(u,v) on a larger scale, but having the same shape.

),(ˆ),(

),(vuFvuG

vuHS

Ss =

34



5.6.2 Estimation by Experimentation5.6.2 Estimation by Experimentation

If equipment similar to the equipment used to acquire the degraded image is available, it is possible in principle to obtain an accurate estimate of the degradation.

- set the system until images are degraded as closely as possible tothe image we wish to restore.

- obtain the impulse response of the degradation by imaging an impulse (small dot of light) using the same system settings.

• An impulse is simulated by a bright dot of light, as bright as possible to reduce the effect of noise.

• Fourier transform of an impulse is a constant. From Eq.(5.5-17),

(5.6-2)G(u,v) : the Fourier transform of the observed image.A : a constant describing the strength of the impulse.

AvuGvuH ),(),( =



5.6.2 Estimation by Experimentation5.6.2 Estimation by Experimentation

35



5.6.3 Estimation by Modeling5.6.3 Estimation by Modeling

Ex. The degradation model proposed by Hufnagel and Stanley [1964] is based on the physical characteristics of atmospheric turbulence.

(5.6-3)

k : a constant that depends on the nature of the turbulence

6/522 )(),( vukevuH +−=




Another major approachImage has been blurred by uniform linear motion between the image and the sensor during image acquisition.

x0(t) : the time varying components of motion in the x-directionsy0(t) : the time varying components of motion in the y-directionsT : the duration of the exposure

The blurred image (5.6-4)

¦

(5.6-8)

G(u,v) = H(u,v) F(u,v) (5.6-9)

If the notation variable x0(t) and y0(t) are known, the transfer function H(u,v) can be obtained directly from Eq.(5.6-8).

[ ]∫ −−=T

dttyytxxfyxg0 00 )(),(),(

[ ]∫ +−=T tvytuxj dtevuH0

)()(2 00),( π

36




Suppose that the image in question undergoes uniform linear motion in the x-direction only ( x0(t) = at/T, y0(t) = 0 )

when t = T, the image has been displaced by a total distance a.

(5.6-10)

If y0(t) = bt/T as well,

(5.6-11)

uaj

T Tuatj

T tuxj

euauaT

dte

dtevuH

π

π

π

ππ

−

−

−

=

=

=

∫∫

)sin(

),(

0

/2

0

)(2 0

[ ] )()(sin)(

),( vbuajevbuavbua

TvuH +−++

= πππ




37



5.7 Inverse Filtering5.7 Inverse Filtering

An estimate, F(u,v), of the transform of the original image,

(5.7-1)

Substituting G(u,v) from Eq.(5.5-17) in Eq.(5.7-1),(5.7-2)

Even if we know the degradation function, we cannot recover the undegraded image exactly, because N(u,v) is a random function whose Fourier transform is not known.

If H(u,v) has zero or very small values,

then the ratio N(u,v)/H(u,v) could easily dominate the estimate F(u,v)

),(),(),(ˆ

vuHvuGvuF =

),(),(),(),(ˆ

vuHvuNvuFvuF +=



one approach is to limit the filter frequencies to values near the origin.

• H(0,0) is equal to the average value of h(x,y), and usually the highest value of H(u,v).

• By limiting the analysis to frequencies near the origin, we reduce the probability of encountering zero values.

Ex. Fig.5.25(b) was inverse filtered with Eq.(5.7-1) using the exact inverse of the degradation function that generated that image.

The degradation function used was

with k=0.0025.

A Gaussian-shape function has no zeros. But the degradation values became so small that the result of full inverse filtering is useless.


[ ] 6/522 )2/()2/(),( NvMukevuH −+−−=

38




(a) Full inverse filter.

Butterworth lowpassfunction of order 10.

(b) Result with H cut off outside a radius of 40 (Radii below 70 tended toward blurred images)

(c) Radii near 70 yielded the best visual results

(d) values above 70 started to produce degraded images.



5.8 Minimum Mean Square Error (Wiener) Filtering5.8 Minimum Mean Square Error (Wiener) Filtering

The inverse filtering approach makes no explicit provision for handling noise.

In this section we discuss an approach that incorporates both the degradation function and statistical characteristics of noise into the restoration process.

The objective is to find an estimate f of the uncorrupted image fsuch that the mean square error between then is minimized.

(5.8-1)assume that the noise and the image are uncorrelated.

one or the other has zero meanThe gray levels in the estimate are a linear function of the levels in the degraded image.

{ }22 )ˆ( ffEe −=

39




(5.8-2)

Wiener filterminimum mean square error filter (least square error filter)

The Wiener filter does not have the same problem as the inverse filter with zeros in the degradation function, unless both H(u,v) and Sη(u,v) are zero for the same value(s) of u and v.

),(),(/),(),(

),(),(

1

),(),(/),(),(

),(*

),(),(),(),(

),(*),(ˆ

2

2

2

2),(

vuGvuSvuSvuH

vuHvuH

vuGvuSvuSvuH

vuH

vuGvuSvuHvuS

SvuHvuF

f

f

f

vuf

+=

+=

+=

η

η

η




H(u,v) = degradation functionH*(u,v) = complex conjugate of H(u,v)|H(u,v)|2 = H*(u,v)H(u,v)Sη(u,v) = |N(u,v)|2 = power spectrum of the noiseSf(u,v) = |F(u,v)|2 = power spectrum of the undegraded image.

If the noise is zero, then the noise power spectrum vanishes and the Wiener filter reduces to the inverse filter.

The power spectrum of the undegraded image seldom is known.An approach used frequently is to approximate Eq.(5.8-2) by

(5.8-3)

K : a specified constant

),(),(

),(),(

1),(ˆ2

2

vuGKvuH

vuHvuH

vuF

+=

40




The value of K was chosen interactively to yield the best visual results.




41



5.9 Constrained Least Squares Filtering5.9 Constrained Least Squares Filtering

A constant estimate of the ratio of the power spectra in the Wiener filter is not always a suitable solution.

• “Constrained Least Squares Filtering” requires knowledge of only the mean and variance of the noise.

• These parameters usually can be calculated from a given degraded image.

Wiener filter is based on minimizing a statistical criterion and, as such, it is optimal in an average sense.

Constrained Least Squares Filter yields an optimal result for each image to which it is applied.(these optimality criteria, while satisfying from a theoretical point of view, are not related to the dynamics of visual perception.)




In vector-matrix formg = Hf + η (5.9-1)

Central to the method is the issue of the sensitivity of H to noise.• One way to alleviate the noise sensitivity problem is to base

optimality of restoration on a measure of smoothness, such as the second derivative of an image (the Laplacian)

Find the minimum of a criterion function, C,

(5.9-2)subject to the constraint

||g – Hf ||2 = ||η||2 (5.9-3)

where is the Euclidean vector norm.

[ ]∑∑−

=

−

=

∇=1

0

1

0

22 ),(M

x

N

yyxfC

∑=

=≅n

kk

T w1

22 www

42




The frequency domain solution to this optimization problem is

(5.9-4)

γ : a parameter that must be adjusted so that the constraint in Eq.(5.9-3) is satisfied

P(u,v) : the Fourier transform ot the function

(5.9-5)

p(x,y), recognized as the Laplacian operator, as well as all other relevant spatial domain functions, must be properly padded with zeros prior to computing their Fourier transform for use in Eq.(5.9-4).

If γ is zero, Eq.(5.9-4) reduces to inverse filtering.

),(),(),(

),(*),(ˆ22 vuG

vuPvuHvuHvuF

+=

γ

−−−

−=

010141

010),( yxp




The parameter γ in Eq.(5.9-4) is a scalar, while the value of K in Eq.(5.8-3) is an approximation to the ratio of two unknown frequency domain functions whose ratio seldom is constant.

manually selecting γ would be a more accurate estimation

Result of processing Fig.5.29(a), (d), and (g) with constrained least squares filters. (select γ manually to yield the best visual results)

The constrained least squares filter would outperform the Wiener filter when selecting the parameters manually for better visual results.

43




It is possible to adjust the parameter γ interactively until acceptable results are achieved.

If we are interested in optimality, γ must be adjusted so that the constraint in Eq.(5.9-3) is satisfied.

A procedure for computing γ by iteration is as described in p.268.

Optimum restoration in the sense of constrained least squares does not necessarily imply “best” in the visual sense.

In general, automatically determined restoration filters yield inferior results to manual adjustment of filter parameters.




Fig. 5.25(b),

44



5.10 Geometric Mean Filter5.10 Geometric Mean Filter

The generalization :

(5.10-1)

(α, β : positive, real constants)

• When α=1 this filter reduces to the inverse filter.• With α=0 the filter becomes the so-called parametric Wiener filter,

which reduces to the standard Wiener filter when β=1.• With β=1,

as α decreases below ½, the filter will behave more like inverse filter.as α increases above ½, the filter will behave more like Wiener filter.

• When α=½ and β=1, the filter is commonly referred to as the spectrum equalization filter.

),(

),(),(

),(

),(*),(

),(*),(ˆ

1

22 vuG

vuSvuS

vuH

vuHvuH

vuHvuF

f

α

η

α

β

−

+

=



5.11 Geometric Transformations5.11 Geometric Transformations

Geometric transformations often are called Rubber-sheet transformations.

A geometric transformation consists of

(1) Spatial transformation : defines the “re-arrangement” of pixels on the image plane

(2) Gray-level interpolation : deals with the assignment of gray levels to pixels in the spatially transformed image.

45



5.11.1 Spatial Transformations5.11.1 Spatial Transformations

Suppose that an image f(x,y) undergoes geometric distortion to produce an image g(x´,y´).

x´ = r(x,y) (5.11-1)y´ = s(x,y) (5.11-2)

If r(x,y) = x/2 and s(x,y) = y/2, the “distortion” is simply a shrinking of the size of f(x,y) by one-half in both spatial directions.

Formulate the spatial relocation of pixels by the use of tiepoints, which are a subset of pixels whose location in the input (distorted) and

output (corrected) images is known precisely.



5.11.1 Spatial Transformations5.11.1 Spatial Transformations

Suppose that the geometric distortion process within the quadrilateral region is modeled by a pair of bilinear equations so that

r(x,y) = c1x + c2y + c3xy + c4 (5.11-3)s(x,y) = c5x + c6y + c7xy + c8 (5.11-4)

Then, from Eqs.(5.11-1) and (5.11-2),x´ = c1x + c2y + c3xy + c4 (5.11-5)y´ = c5x + c6y + c7xy + c8 (5.11-6)

Since there are a total of eight known tiepoints, these equations can be solved for the eight coefficients ci, i = 1,2,…,8.

The coefficients constitute the geometric distortion model used to transform all pixels within the quadrilateral region defined by the tiepoints used to obtain the coefficients.

46



5.11.2 Gray-Level Interpolation5.11.2 Gray-Level Interpolation

The simplest scheme for gray-level interpolation is based on a nearest neighbor approach.

zero-order interpolation (Fig.5.33)

(1) The mapping of integer (x,y) coordinates into fractional coordinates (x´,y´) by means of Eqs.(5.11-5) and (5.11-6)

(2) The selection of the closest integer coordinate neighbor to (x´,y´)(3) The assignment of the gray level of this nearest neighbor to the

pixel located at (x,y).




Nearest neighbor interpolation has the drawback of producing undesirable artifacts, such as distortion of straight edges in images of high resolution.

Cubic convolution interpolation : fits a surface of the sin (z) / z type through a much larger number of

neighbors (say, 16) in order to obtain a smooth estimate of the gray level at any desired point.

used in 3D graphics, medical imaging

For general-purpose image processingbilinear interpolation

use the gray levels of the four nearest neighbors

v(x´,y´) = ax´ + by´ + cx´y´ + d (5.11-7)

47




The four coefficients are easily determined from the four equations in four unknowns that can be written using the four known neighbors of (x´,y´).

When these coefficients have been determined, v(x´,y´) is computed and this value is assigned to the location in f(x,y).

The image have such few gray levels in the sharp boundaries that almost any type of geometric distortion will cause significant degradation.




When images have more texture, geometric correction errors end to be less noticeable.

image restoration chapter05 handouts 2 fine

Documents