Introduction to Image Processing

Grass

Sky

TreeTree

? ?

Filtering in the Frequency Domain

Discrete Fourier Transform

Image Enhancement in the Frequency Domain

• The frequency is the number of oscillations per second. The higher the frequency, the shorter the period. In imaging, it means how quickly/slowly the grey level intensity changes.

• Images represent variations of brightness or colour in space. If the variation is spatial and L, the period is a distance, then 1/L is termed the spatial frequency of the variation.

A High Frequency Signal (1-D) A Low Frequency Signal (1-D)

• The frequency domain describes the rate of signal change. We can associate frequencies in the Fourier Transform with patterns of intensity variations in the image. Thus different information about an image can be extracted and manipulated.

• Some tasks would be difficult or impossible to perform directly in the spatial domain, e.g., it is very difficult to do a good job of sharpening a blurred image caused by motion in the spatial domain. This task is generally solved by frequency domain filtering.

• Spatial domain filters are useful for lessening the effects of additive/random noise. However in the frequency domain we can uncover patterned noise (e.g. sinusoidal noise) and remove it using band reject/notch frequency domain filters.

• Applying the equivalent filters in the spatial domain involves convolution which requires a great amount of computation

Why Frequency Domain?

Fourier Theory

• Basic ideas: – A periodic function, however

complex it might appear, can be represented as a weighted sumsum of sines/cosines functions of different frequencies.

– Although there may be little regularity apparent in an image, it can be decomposed into a set of sinusoidal components, each of which has a well defined frequency.

Fourier transformbasis functions

Approximating a square wave as the sum of sine waves

Example 1

time domain frequency domain

time domain frequency domain

time domain frequency domain

time domain frequency domain

Example 2

Time, Spatial Domain Signals

Frequency Domain Signals

Fourier Transform

Inverse Transform

1-D, Continuous Case:

dxexfuF uxj 2)()(Fourier Transform:

dueuFxf uxj 2)()(Inverse Fourier Transform:

Fourier Transform

2 cos 2 sin 2j uxe ux j ux “Euler’s formula”

1-D, Discrete Case:

1

0

/2)(1

)(M

x

MuxjexfM

uF Fourier Transform:

Inverse Fourier Transform:

1

0

/2)()(M

u

MuxjeuFxf

u = 0,…,M-1

x = 0,…,M-1

F(u) can be written as:

)()()( ujeuFuF )()()( ujIuRuF

22 )()()( uIuRuF

where

)(

)(tan)( 1

uR

uIu

Discrete Fourier Transform

Polar coordinate:

magnitude phase

0 M 2M-M

DFT repeats itself every M points (Period = M)

We display only in this range

1

0

/2)(1

)(M

x

MuxjexfM

uF From DFT:

Periodicity of 1-D DFT

u is the number of complete cycles of the sinusoid that fits into the width M of the image. These form the basis functions of the frequency domain representation and the weights for each sine and cosine function are known as Fourier coefficients.

F(u) = F(u+kM)

0 M-1

Time Domain Signal

DFT

f(x) )(uF

0 M-1

Low frequencyarea

High frequencyarea

The graph F(u) is not easy to understand !

Conventional Display for 1-D DFT

M/2

F(u) = F*(-u)

)(uF

0-M/2 M/2-1

)(uF

0 M-1

Shift center of the graph F(u) to 0 to get better display which is easier to understand.

High frequency area

Low frequency area

Better Display for 1-D DFT

M/2

1

0

1

0

)//(2),(1

),(M

x

N

y

NvyMuxjeyxfMN

vuF

2-D IDFT

1

0

1

0

)//(2),(),(M

u

N

v

NvyMuxjevuFyxf

2-D DFT

u = frequency in x direction, u = 0 ,…, M-1v = frequency in y direction, v = 0 ,…, N-1

x = 0 ,…, M-1y = 0 ,…, N-1

For an image of size MxN pixels

2-D Discrete Fourier Transform

F(u,v) can be written as:

),(),(),( vujevuFvuF or),(),(),( vujIvuRvuF

22 ),(),(),( vuIvuRvuF where

),(

),(tan),( 1

vuR

vuIvu

For the purpose of viewing, we usually display only themagnitude part of F(u,v)

2-D Discrete Fourier Transform

magnitude

phase

Fourier Basis

v

Fourier ExamplesRaw Image Fourier Amplitude

Sinusoid,higher frequency

Sinusoid,lower frequency

Sinusoid,tilted

DC term + side lobeswide spacing

DC term+ side lobesclose spacing

Titled spectrum

Fourier basis element

Vector (u,v)• Magnitude gives frequency• Direction gives orientation.

e i2 uxvy

More Fourier Examples

Here u and v are larger than in the previous slide.

More Fourier Examples

And larger still...

More Fourier Examples

Discrete Fourier Transform - Magnitude

• The image contains components of all frequencies, but their magnitude gets smaller for higher frequencies

• Low frequencies contain more image information than the higher ones

• Two dominating directions in the Fourier image, vertical and horizontal. These originate from the regular patterns in the background

Original image Fourier transform Logarithmic operator applied

Discrete Fourier Transform - Phase

• The value of each point determines the phase of the corresponding frequency

• The phase information is crucial to reconstruct the correct image in the spatial domain

Phase image Re-transform using only magnitude

Discrete Fourier Transform

Inverse Transform

If we attempt to reconstruct the image with an inverse Fourier Transform after destroying either the phase information or the amplitude information, then the reconstruction will fail.

Phase Carries More Information

MagnitudeandPhase:

RawImages:

Reconstruct(inverse FFT)mixing themagnitude andphase images

Phase “Wins”

Properties of 2-D DFT• Periodicity

– the 2-D DFT and its inverse are infinitely periodic in the u and v directions

F(u,v) = F(u+k1M,v) = F(u,v+k2N) = F(u+k1M, v+k2N)

• Symmetry– for real image f(x,y), DFT is conjugate symmetric,

i.e.

• Translation

f(x,y)ej2∏(u0x/M+v0y/N) F(u-u0, v-v0)

f(x,y)(-1)x+y F(u-M/2, v-N/2)

*( , ) ( , )F u v F u v ( , ) ( , )F u v F u v

For an image of size MxN pixels, its 2-D DFT repeats itself every M points in the x-direction and every N points in the y-direction.

We display only in this range

1

0

1

0

)//(2),(1

),(M

x

N

y

NvyMuxjeyxfMN

vuF

0 M 2M-M

0

N

2N

-N

2-D DFT:

Periodicity of 2-D DFT

2-D Circular Shift

High frequency area Low frequency area

v

u

u

Better Display for 2-D DFT

v

Original displayof 2-D DFT

0 M 2M-M

0

N

2N

-N

Display of 2-D DFTAfter circular shift

2-D Circular Shift: How it Works

Log EnhancedFourier Transform

)),(1log( vuFc

The Spectrum of DFT

Original Image

Spectrum Shift

• We shift the origin of the transform to the centre. Now the low frequency information is in the centre of the DFT.

Convolution Theorem

Filtering in the spatial/time domain with h(x, y) is equivalent to filtering in the frequency domain with H(u,v), where F and H are the DFT of f and h respectively

Multiplication on the right hand side is component-wise, i.e. |F(u,v)| x |H(u,v)|

multiplication in the frequency domain

convolution in the spatial/time domain

F(u,v) H(u,v)f(x,y) * h(x,y)

From the property of Fourier Transform:

),(),(),(),(),(),( vuGvuHvuFyxhyxfyxg

multiplication in the frequency domainis easier than convolution in the spatialdomain.

Basic Filtering Steps

Frequency Domain Filtering

Fourier Transform

Inverse Fourier Transform

Multiplication

Frequency Domain Filters

• Ideal Low Pass Filter

– where d(u,v) is the distance of (u,v) from the centre of the filter and d0 is a positive number (the radius of the white circle).

– for smoothing and blurring

• Ideal High Pass Filter

– for extracting the details of an image

lowpasshighpass 1 HH

0

0

),( if 0

),( if 1),(

dvud

dvudvuH

Other Ideal Filters

The ideal bandpass filter retains the frequencies inside a given band and eliminates all the other.

• The ideal bandreject filter eliminates the frequencies inside a given band and retains all the other.

Frequency Domain Filtering

FourierAmplitude

Multiply by a filter in thefrequency domain <=> convolve with the fiter inspatial domain.

Ringing and Blurring

Ideal in frequency domain means non-ideal in spatial domain, vice versa.

FT

Examples of ILPF

Transfer function

Where D0 = cut off frequency, n = filter order.

Butterworth Lowpass Filter

nDvuDvuH

20 ]/),([1

1),(

Notes on BLPF• The image to be filtered is an MxN pixeled image• D(u,v) is the distance from the centre to the point (u,v), given by the formula • For a lowpass filter, this means that as D(u,v) increases, H(u,v) decreases.

The effect is to dampen the higher frequencies which are represented as being a distance far from the centre and emphasise the lower frequencies which are represented by points close to the centre

• D0 is known as the “cutoff” frequency. In an ideal lowpass filter, this is the point past which all frequencies would be eliminated.

• Increasing D0 increases the number of frequencies that are “passed” (or lessens the dampening effects of higher frequencies)

• Decreasing D0 means a smaller number of frequencies are allowed to pass (or the dampening effects of higher frequencies are increased). This would result in a more blurred image.

• n is the order of the filter. Increasing n increases and decreases H(u,v) for D(u,v) less than and greater than D0 respectively, which means there is a dramatic passage from those frequencies which are kept and those which are dampened/eliminated. As n increases, H(u,v) nears zero when the value of D(u,v) is high. When n is very high, it is a good approximate of an ideal lowpass filter.

2/122 ])2/()2/[(),( NvMuvuD

There is less ringing effect compared to those of ideal lowpassfilters!

Results of BLPF

Examples of IHPF

Ringing effect can be obviously seen!

Butterworth High Pass Filters

The Butterworth high pass filter is given as:

where n is the order and D0 is the cut off distance as before

nvuDDvuH

20 )],(/[1

1),(

Results of BHPF

( , )( 1)x yf x y

( ( , )( 1) ) ( / 2, / 2)

.

x yf x y F u M v N

where denotes the Fourier Transform

• First we centre the image

• Why?

• What does that mean?

– It means that when we apply the Fourier transform, the lowest frequencies will be in the centre and the highest frequencies will be around the edges.

• Secondly we compute the Fourier Transform as above

General Filtering Method

• Now we multiply by a filter function H(u,v), i.e.

– (or if you’re fussy)

• Note that this is not matrix multiplication. It is pointwise (or point by point) multiplication.

• H(u,v) generally takes real values between 0 and 1

• G(u,v) is complex-valued because F(u,v) is complex-valued

( , ) ( , ) ( , )G u v H u v F u v

General Filtering Method

( - / 2, - / 2)F u M v N

General Filtering Method

• Calculate the Inverse Fourier Transform of G(u,v).

using the Inverse Fourier Transform formula

• We then take the real part of g(x,y) (to remove any small complex residues)

• This is then multiplied by (-1)x+y to realign the image again and produce the final filtered image.

1( ( , ))G u v

-1 -12 ( / / )

0 0

( , ) ( , )M N

i ux M vy N

u v

g x y G u v e

f(x,y)

1-D DFTby row

F(u,y)

1-D DFTby column

F(u,v)

Property of DFT - Separability

1-D DFTby column

F(x,v)1-D DFTby row

Alternative 1

Alternative 2

• A process which aims to improve bad images so they will “look” better

• “Better” visual representation

• Subjective

• No quantitative measures

• A process which aims to invert known or estimated degradation to images

• Remove effects of sensing environment

• Objective

• Mathematical, model dependent quantitative measures

Enhancement vs. Restoration

Inverse Filtering• Simple (noiseless) case:

• Then original image can be obtained by

• Note: H(u,v) may have zero or near zero values in most parts of (u,v) range– at those points the division operation is undefined or

results in meaningless values– for points having very small |H(u,v)|, although the

division can be done, the noise will be amplified to an intolerable extent

– restrict the area to low frequency part or use Gaussian weighting to solve this problem.

),(

),(),(),(),(),(

vuH

vuGvuFvuFvuHvuG

),(

),(),( 1

vuH

vuGFyxf

Acknowlegements

Slides are modified based on the original slide set from Dr Li Bai, The University of Nottingham, Jubilee Campus plus the following sources:

• Digital Image Processing, by Gonzalez and Woods• Digital Image Processing, a practical introduction using

Java by Nick Efford• http://www.comp.dit.ie/bmacnamee/materials/dip/lectures/

ImageProcessing7-FrequencyFiltering.ppt• http://gear.kku.ac.th/~nawapak/178353/Chapter04.ppt• http://cns-alumni.bu.edu/~slehar, An Intuitive Explanation

of Fourier Theory• http://www.cs.unc.edu/Research/vision/comp256/

vision07.ppt• http://vision.soongsil.ac.kr/~hic/LecData2007/grad/5.Image

%20Transforms.ppt