dig image processin
TRANSCRIPT
![Page 1: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/1.jpg)
EC 624 Digital Image Processing (3 0 2 8)
Class I Introduction
Instructor: PK. BoraInstructor: PK. Bora
![Page 2: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/2.jpg)
Digital Image ProcessingDigital Image Processing means processing Digital Image Processing means processing
of of digitaldigital imagesimages on digital hardware usually a on digital hardware usually a computercomputer
![Page 3: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/3.jpg)
What is an analog image?Electrical Signal, for example, the output of Electrical Signal, for example, the output of a video camera, that gives the electric a video camera, that gives the electric voltage at locations in an imagevoltage at locations in an image
![Page 4: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/4.jpg)
What is a digital Image2D array of numbers 2D array of numbers representing the sampled representing the sampled version of an imageversion of an imageThe image defined over a The image defined over a grid, each grid location grid, each grid location being called a pixel.being called a pixel.Represented by a finite Represented by a finite grid and each intensity grid and each intensity data is represented a finite data is represented a finite number of bits.number of bits.A binary image is A binary image is represented by one bit represented by one bit graygray--level image is level image is represented by 8 bits.represented by 8 bits.
Pixel and IntensitiesPixel and Intensities
1616 1818 1919 2020
255255 1919 1818 2020
2121 2121 2222 3030
2121 3333 2222 2323
![Page 5: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/5.jpg)
Mathematically
We can think of an We can think of an image image as a function, as a function, ff,,from Rfrom R22 to R:to R:
ff( ( x, y x, y ) gives the ) gives the intensityintensity at position ( at position ( x, y x, y ) ) Realistically, we expect the image only to be Realistically, we expect the image only to be defined over a rectangle, with a finite range:defined over a rectangle, with a finite range:
ff: [: [aa, , bb]]xx[[cc, , dd] ] [0, 1][0, 1]
![Page 6: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/6.jpg)
What is a Colour Image?Three components:Three components:
R,G, B each usually R,G, B each usually represented by 8 bitsrepresented by 8 bits
We call 24We call 24--bit videobit videoThese three primary are These three primary are mixed in different mixed in different proportions to get proportions to get different different colourscoloursFor different processing For different processing applications other formats applications other formats ((YIQ,YCbCrYIQ,YCbCr, HIS etc) are , HIS etc) are usedused
( , )( , ) ( , )
( , )
r x yf x y g x y
b x y
⎡ ⎤⎢ ⎥= ⎢ ⎥⎢ ⎥⎣ ⎦
A color image is A color image is just a three just a three –– component component function. We can function. We can write this as a write this as a ““vectorvector--valuedvalued”” function:function:
![Page 7: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/7.jpg)
Types of Digital Image
Digital images include Digital images include Digital photosDigital photosImage sequences used for video broadcasting and Image sequences used for video broadcasting and playbackplaybackMultiMulti--sensor data like satellite images in the visible, sensor data like satellite images in the visible, infrared and microwave bandsinfrared and microwave bandsMedical images like ultraMedical images like ultra--sound, Gammasound, Gamma--ray images, ray images, XX--ray images and radioray images and radio--band images like MRI etcband images like MRI etcAstronomical imagesAstronomical imagesElectronElectron--microscope images used to study material microscope images used to study material structurestructure
![Page 8: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/8.jpg)
ExamplesPhotgraphic
Ultrasound Mammogram
![Page 9: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/9.jpg)
Image processing
Digital Image processing deals with manipulation and analysis ofDigital Image processing deals with manipulation and analysis of the the digital image by a digital hardware, usually a computer.digital image by a digital hardware, usually a computer.Emphasizing certain pictorial information for better clarity (huEmphasizing certain pictorial information for better clarity (human man interpretation)interpretation)Automatic machine processing of the scene data.Automatic machine processing of the scene data.Compressing the image data for efficient utilization of storageCompressing the image data for efficient utilization of storage space space and transmission bandwidthand transmission bandwidth
![Page 10: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/10.jpg)
Image ProcessingAn An image processingimage processing operation operation
typically defines a new image typically defines a new image gg in terms in terms of an existing image of an existing image f.f.
We can also transform O the domain of We can also transform O the domain of ff::
![Page 11: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/11.jpg)
Image Processingimage filtering: change image filtering: change rangerange of imageof image
f
x
hf
x
f
x
hf
x
image warping: change image warping: change domaindomain of imageof image
g(xg(x) = ) = h(f(xh(f(x))))
g(xg(x) = ) = f(h(xf(h(x))))
![Page 12: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/12.jpg)
Example Image RestorationImage Restoration
Degraded Image Restored ImageProcessing
![Page 13: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/13.jpg)
Image Processing steps
Acquisition, Sampling/ Quantization/ Acquisition, Sampling/ Quantization/ CompressionCompressionImage enhancement and restorationImage enhancement and restorationFeature ExtractionFeature ExtractionImage SegmentationImage SegmentationObject RecognitionObject RecognitionImage InterpretationImage Interpretation
![Page 14: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/14.jpg)
Image Acquisition
An analog image is An analog image is obtained by scanning the obtained by scanning the sensor output. Some of the sensor output. Some of the modern scanning device modern scanning device such as a CCD camera such as a CCD camera contains an array of photocontains an array of photo--detectors, a set of detectors, a set of electronic switches and electronic switches and control circuitry all in a control circuitry all in a single chipsingle chip
![Page 15: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/15.jpg)
Image Acquisition
Sample and Hold
Sample and Hold
Analog toDigital
Analog toDigital
Digital Image
Takes a measurement and “holds” it for conversion to digital.
Converts a measurement to digital
Image Sensor
![Page 16: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/16.jpg)
Sampling/ Quantization/ Compression
A digital image is obtained by sampling and A digital image is obtained by sampling and quantizing an analog image. quantizing an analog image. The analog image signal is sampled at rate The analog image signal is sampled at rate determined by the application concerneddetermined by the application concernedStill image 512X512, 256X256Still image 512X512, 256X256Video: 720X480, 360X240, 1024x 768 (HDTV)Video: 720X480, 360X240, 1024x 768 (HDTV)The intensity is quantized into a fixed number of The intensity is quantized into a fixed number of levels determined human perceptual limitationlevels determined human perceptual limitation8 bits is sufficient for all but the best applications 8 bits is sufficient for all but the best applications 10 bits 10 bits –– Television production, printingTelevision production, printing1212--16 bits 16 bits –– Medical imageryMedical imagery
![Page 17: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/17.jpg)
Sampling/ Quantization/ Compression (Contd.)
Raw video is very bulky Raw video is very bulky Example:The transmission of highExample:The transmission of high--definition uncompressed digital video at definition uncompressed digital video at 1024x 768, 24 bit/pixel, 25 frames requires 1024x 768, 24 bit/pixel, 25 frames requires 472 Mbps472 MbpsWe have to compress the raw data to store We have to compress the raw data to store and transmitand transmit
![Page 18: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/18.jpg)
Image Enhancement
Improves the qualities of an image byImproves the qualities of an image byenhancing the contrast enhancing the contrast sharpening the edgessharpening the edges
removing noise, etc. removing noise, etc. As an example, let us explain the image As an example, let us explain the image
filtering operation to remove noise. filtering operation to remove noise.
![Page 19: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/19.jpg)
Example: Image Filtering
Original Image Filtered ImageOriginal Image Filtered Image
![Page 20: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/20.jpg)
Histogram Equalization
Enhance the contrast of images by Enhance the contrast of images by transforming the values in an intensity transforming the values in an intensity image to its normalized histogramimage to its normalized histogramthe histogram of the output image is the histogram of the output image is
uniformly distributed.uniformly distributed.Contrast is betterContrast is better
![Page 21: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/21.jpg)
Feature Extraction
Extracting Features like edgesExtracting Features like edgesVery important to detect the boundaries of the objectVery important to detect the boundaries of the objectDone through digital differentiation operationDone through digital differentiation operation
![Page 22: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/22.jpg)
Example: Edge Detection
Original Saturn Image Edge Image
![Page 23: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/23.jpg)
Segmentation
Partitioning of an image into connected Partitioning of an image into connected homogenous regions.homogenous regions.Homogeneity may be defined in terms of:Homogeneity may be defined in terms of:Gray valueGray valueColourColourTextureTextureShapeShapeMotionMotion
![Page 24: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/24.jpg)
Segmented Image
![Page 25: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/25.jpg)
Object Recognition
An object recognition system finds An object recognition system finds objects in the real world from an image objects in the real world from an image of the world, using object models which of the world, using object models which are known a prioriare known a priorilabelling problem based on models of labelling problem based on models of known objectsknown objects
![Page 26: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/26.jpg)
Object Recognition (Contd.)
••
Object or model representationObject or model representationFeature extractionFeature extractionFeatureFeature--model matchingmodel matchingHypotheses formationHypotheses formationObject verificationObject verification
![Page 27: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/27.jpg)
Image Understanding
Inferring about the scene on the basis of the Inferring about the scene on the basis of the recognized objectsrecognized objectsSupervision is requiredSupervision is requiredNormally considered as part of artificial Normally considered as part of artificial intelligenceintelligence
![Page 28: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/28.jpg)
Books
1.1. R. C. Gonzalez and R. E. Woods, R. C. Gonzalez and R. E. Woods, Digital Digital Image ProcessingImage Processing, Pearson Education, , Pearson Education, 2001 (Main Text2001 (Main Text))
2.2. A. K. Jain, A. K. Jain, Fundamentals of Digital Image Fundamentals of Digital Image processingprocessing, Pearson Education, 1989., Pearson Education, 1989.
3. R. C. Gonzalez , R. E. Woods and S. L. 3. R. C. Gonzalez , R. E. Woods and S. L. EddinsEddins, , Digital Image Processing using MATLABDigital Image Processing using MATLAB, , Pearson Education, 2004. (Lab Ref)Pearson Education, 2004. (Lab Ref)
![Page 29: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/29.jpg)
Evaluation Scheme
End Sem 50End Sem 50Mid Sem 25Mid Sem 25Quiz 5Quiz 5Matlab Assignment 10Matlab Assignment 10Mini Project 10Mini Project 10
Total 100 Total 100
![Page 30: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/30.jpg)
1. MINI PROJECT
Matlab Implementation and preparing Report and DemonstratiMatlab Implementation and preparing Report and Demonstration of any on of any advanced topic like:advanced topic like:
Video compressionVideo compressionVideo Video mosaicingmosaicingVideoVideo--based trackingbased trackingMedical Image CompressionMedical Image CompressionVideo WatermarkingVideo WatermarkingMedical Image SegmentationMedical Image SegmentationImage and Video RestorationImage and Video RestorationBiometric recognitionBiometric recognition
![Page 31: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/31.jpg)
2D Discrete Time (Space) Fourier Transform
• Recall DTFT of a 1D sequence, Given is
and
Note that
exists if and only if is absolutely summable, i.e.,
[ ]{ }, .....x n n = − ∝ ∝
( ) [ ]n=-
= x n j nX e ωω∝
−
∝∑
[ ] ( )1x n = X2
j ne dπ
ω
πω ω
π −∫
( ) ( )2 = XX ω π ω+
( )X ω [ ]x n
[ ]n
x n∝
=−∝< ∝∑
![Page 32: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/32.jpg)
Relationship between CTFT and DTFT
• Consider a discrete sequence obtained by sampling an analog signal at a uniform sampling rate , where T is the sampling period.
• We can represent the sampling process by means of the Dirac delta function with the relation
• Now the sampled signal can be represented in continuous domain as,
• Thus, the analog and discrete frequencies are related as
[ ]{ }, ....x n n = − ∝ ∝
( )ax t 1TsF =
[ ] ( ) , n=0, 1......ax n x nT= ±
( ) ( ) ( )
( ) ( )
an=-
an=-
= x
= x
sx t t t nT
nT t nT
δ
δ
∝
∝∝
∝
−
−
∑
∑
.w T= Ω
![Page 33: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/33.jpg)
2D DSFT• Consider the signal defined over the
two-dimensional space.
• Also assume
• Then the two-dimensional discrete-space Fourier transform
(2D DSFT) and its inverse are defined by the following relations:
and
{ [ , ], ,....., , ,....., }f m n m n= −∞ ∞ = −∞ ∞
[ , ] .m n
f m n∞ ∞
=−∞ =−∞
< ∞∑ ∑
)(],[ ),( vnumj
n m
enmfvuF +−∞
−∞=
∞
−∞=∑ ∑=
( )2
1[ , ] ( , )4
j um vnf m n F u v e du dvπ
∞ ∞+ +
−∞ −∞
= ∫ ∫
![Page 34: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/34.jpg)
• Note that is doubly periodic in u and v. • Following properties of are easily verified:
– Linearity– Separability– Shifting theorem:
– Convolution theorem:
– Eigen function– Modulation– Correlation– Inner product– Parseval’s theorem
( , )F u v( , )F u v
( )0 0
2 DSFT
2 DSFT0 0
If [ , ] ( , ), then
[ , ] ( , ),
D
j um vnD
f m n F u v
f m m n n e F u v− +
←⎯⎯⎯→
− − ←⎯⎯⎯→
2 DSFT 2 DSFT1 1 2 2
2 DSFT1 2 1 2
If [ , ] ( , ) and [ , ] ( , ), then
[ , ]) * [ , ] ( , ) ( , )
D D
D
f m n F u v f m n F u v
f m n f m n F u v F u v
←⎯⎯⎯→ ←⎯⎯⎯→
←⎯⎯⎯→
![Page 35: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/35.jpg)
2D DFT
Motivation
• Consider the 1D DTFT which is uniquely defined for
each
• Numerical evaluation of involves very large (infinite) data and is to be done for each
• An easier way is the Discrete Fourier transform (DFT) which is obtained by
sampling at a regular interval .
• Sampling periodically in the frequency domain at a rate means that the data sequence will be periodic with a period
• The relation between the Fourier transform of an analog signal and the DFT of the sampled version is illustrated in the Figure below.
( ) [ ] jwn
nX x n eω
∞−
=−∞
= ∑. [0, 2 ].ω π∈
( )X ω.ω
( )X ω1N
.N(t)ax
1N
![Page 36: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/36.jpg)
2D DFT
1 2 and k k
The 2D DFT of a 2D sequence is defined as
and the inverse 2D DFT is given by
[ ] [ ] 1 22 2-1 -1
1 2 1 20 0
, , , 0,1,..., 1, =0,1,...,N-1 N M j mk nk
M N
n m
F k k f m n e k M kπ π⎛ ⎞− +⎜ ⎟
⎝ ⎠
= =
= = −∑∑
[ ] [ ] 1 2
2 1
2 2N-1 M-1
1 2k =0 k =0
1, k , , 0,1,..., 1, =0,1,...,N-1 j mk nk
M Nf m n F k e m M nMN
π π⎛ ⎞+⎜ ⎟⎝ ⎠= = −∑ ∑
2D DFT is periodic in both
Thus [ ] [ ]1 2 1 2, ,F k k F k M k N= + +
![Page 37: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/37.jpg)
Properties of 2D DFT
0 1 0 2
2 DFT1 2
2 22 DFT
0 0 1 2
If [ , ] [ , ], then
[ , ] [ , ]
D
j m k n kD M N
f m n F k k
f m m n n e F k kπ π⎛ ⎞− +⎜ ⎟
⎝ ⎠
←⎯⎯⎯→
− − ←⎯⎯⎯→
•Shifting property
•Separability propery
Since 1 2 2
2 2 22j mk nk j nkj mkM N NMe e eπ π ππ⎛ ⎞− + −−⎜ ⎟
⎝ ⎠ =
![Page 38: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/38.jpg)
Properties of 2D DFT
1 2 22 2 22j mk nk j nkj mkM N NMe e eπ π ππ⎛ ⎞− + −−⎜ ⎟
⎝ ⎠ =
•Separability propery
Since
We can write
[ ] [ ]
[ ]
[ ]
1 2
21
2
1
2 2-1 -1
1 20 0
22-1 -1
0 0
2-1
1 10
2-1
1 10
, ,
,
[ , ]
w h e r e [ , ] ,
N M j m k n kM N
n m
N M j n kj m kNM
n m
N j n kN
n
M j m kM
m
F k k f m n e
f m n e e
F k n e
F k n f m n e
π π
ππ
π
π
⎛ ⎞− +⎜ ⎟⎝ ⎠
= =
−−
= =
−
=
−
=
=
⎛ ⎞= ⎜ ⎟
⎝ ⎠
=
=
∑ ∑
∑ ∑
∑
∑
Thus the 2D DFT can be computed from a 1D FFT routine
![Page 39: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/39.jpg)
2D Fourier Transform
• Frequency – domain representation of 2D signal ::• Consider a two-dimensional signal • The signal and its two-dimensional Fourier transform
are related by ::
• u and v represent the spatial frequency in radian/length.• F(u,v) represents the component of f(x,y) with frequencies u and v.• A sufficient condition for the existence of F(u,v) is that f(x,y) is absolutely
integrable.
( ), .f x y( ),f x y ( ),F u v
( ) 2DFT
( )
, ( , )
( , ) ( , ) j xu yv
f x y F u v
F u v f x y e dx dy∞ ∞
− +
−∞ −∞
←⎯⎯→
= ∫ ∫( )
2
1( , ) ( , ) 4
j x u y vf x y F u v e d u d vπ
∞ ∞+
−∞ −∞
= ∫ ∫
( , ) f x y dxdy∝ ∝
−∝ −∝< ∝∫ ∫
![Page 40: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/40.jpg)
2D Fourier Transform
• u and v represent the spatial frequency in horizontal and vertical directions inradian/length.
• F(u,v) represents the component of f(x,y) with frequencies u and v.
Illustration of 2D Fourier transform
![Page 41: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/41.jpg)
2D Fourier Transform
• A sufficient condition for the existence of F(u,v) is that f(x,y) is absolutely integrable.
( , ) f x y dxdy∝ ∝
−∝ −∝< ∝∫ ∫
![Page 42: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/42.jpg)
Properties of 2D Fourier Transform1. The 2D Fourier transform is in general a complex function of the
real variables and As such, it can be expressed in terms of the magnitude and the phase .
2. Linearity Property:
3. Shifting Property:
Phase information changes, no change in amplitude.
4. Modulation Property:
.vu( , )F u v ( , )F u v∠
2D FT1 1
2D FT2 2
2D FT1 2 1 2
( , ) ( , )
( , ) ( , )
( , ) ( , ) ( , ) ( , )
f x y F u v
f x y F u v
a f x y bf x y a F u v bF u v
←⎯⎯→
←⎯⎯→
+ ←⎯⎯→ +
),(),( )( 2 vuFeyyxxf vyuxjFTDoo
oo +−⎯⎯ →←−−
),(),( )(oo
yvxuj vvuuFeyxf oo −−⇔++
![Page 43: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/43.jpg)
5. Complex exponentials are the eigen functions of linear shift invariant systems.
The Fourier bases are the Eigen functions of linear systems
For an imaging system, h(x, y) is called the point spread function and H (u, v) is called the optical transfer function
![Page 44: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/44.jpg)
6. Separability property:
Particularly if then .
Suppose , then
1
1
( , ) ( , ) .
( , )
where ( , ) is 1-D Fourier Transform
jux jvy
jvy
F u v f x y e dx e dy
F u v e dy
F u v
∞ ∞− −
−∞ −∞
∞−
−∞
=
=
∫ ∫
∫
)()(),( 21 yfxfyxf =1 2( , ) ( ) ( )F u v F u F v=
( ) x, .a
yf x y rect recta
⎛ ⎞ ⎛ ⎞= ⎜ ⎟ ⎜ ⎟⎝ ⎠ ⎝ ⎠
( ) ( )2
, = asincau sin
= a sincau sincav
F u v a cav
![Page 45: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/45.jpg)
7. 2D Convolution:
Thus the convolution of two functions is equivalent to product of the corresponding Fourier transforms.
2
If ( , ) ( , )* ( , ) ( , ) ( , ) ( , )Similarly if ( , ) ( , ) ( , )
1 ( , ) ( , )* ( , )4
g x y f x y h x yG u v F u v H u v
g x y f x y h x y
G u v F u v H u vπ
==
=
=
![Page 46: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/46.jpg)
8. Preservation of inner product:Recall that the inner product of two functions is defined by
The inner product is preserved through Fourier transform. Thus,
Particularly,
Hence Norm is preserved through 2D Fourier transform.
2
1( , ), ( , ) ( , ), ( , )4
( , ), ( , ) ( , ) ( , )
f x y h x y F u v H u v
where F u v H u v F u v H u v du dv
π∞ ∞
−∞ −∞
=
= ∫ ∫
2
2 22
1( , ), ( , ) ( , ), ( , )4
1( , ) ( , ) 4
f x y f x y F u v F u v
f x y dx dy F u v du dv
π
π
∞ ∞ ∞ ∞
−∞ −∞ −∞ −∞
=
∴ =∫ ∫ ∫ ∫
( ) ( ), and h x,yf x y
( , ), ( , ) ( , ) ( , ) f x y h x y f x y h x y dx dy∞ ∞
−∞ −∞
= ∫ ∫
![Page 47: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/47.jpg)
2D Discrete Time (Space) Fourier Transform
• Recall DTFT of a 1D sequence, Given is
and
Note that
exists if and only if is absolutely summable, i.e.,
[ ]{ }, .....x n n = − ∝ ∝
( ) [ ]n=-
= x n j nX e ωω∝
−
∝∑
[ ] ( )1x n = X2
j ne dπ
ω
πω ω
π −∫
( ) ( )2 = XX ω π ω+
( )X ω [ ]x n
[ ]n
x n∝
=−∝< ∝∑
![Page 48: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/48.jpg)
Relationship between CTFT and DTFT
• Consider a discrete sequence obtained by sampling an analog signal at a uniform sampling rate , where T is the sampling period.
• We can represent the sampling process by means of the Dirac delta function with the relation
• Now the sampled signal can be represented in continuous domain as,
• Thus, the analog and discrete frequencies are related as
[ ]{ }, ....x n n = − ∝ ∝
( )ax t 1TsF =
[ ] ( ) , n=0, 1......ax n x nT= ±
( ) ( ) ( )
( ) ( )
an=-
an=-
= x
= x
sx t t t nT
nT t nT
δ
δ
∝
∝∝
∝
−
−
∑
∑
.w T= Ω
![Page 49: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/49.jpg)
2D DSFT• Consider the signal defined over the
two-dimensional space.
• Also assume
• Then the two-dimensional discrete-space Fourier transform
(2D DSFT) and its inverse are defined by the following relations:
and
{ [ , ], ,....., , ,....., }f m n m n= −∞ ∞ = −∞ ∞
[ , ] .m n
f m n∞ ∞
=−∞ =−∞
< ∞∑ ∑
)(],[ ),( vnumj
n m
enmfvuF +−∞
−∞=
∞
−∞=∑ ∑=
( )2
1[ , ] ( , )4
j um vnf m n F u v e du dvπ
∞ ∞+ +
−∞ −∞
= ∫ ∫
![Page 50: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/50.jpg)
• Note that is doubly periodic in u and v. • Following properties of are easily verified:
– Linearity– Separability– Shifting theorem:
– Convolution theorem:
– Eigen function– Modulation– Correlation– Inner product– Parseval’s theorem
( , )F u v( , )F u v
( )0 0
2 DSFT
2 DSFT0 0
If [ , ] ( , ), then
[ , ] ( , ),
D
j um vnD
f m n F u v
f m m n n e F u v− +
←⎯⎯⎯→
− − ←⎯⎯⎯→
2 DSFT 2 DSFT1 1 2 2
2 DSFT1 2 1 2
If [ , ] ( , ) and [ , ] ( , ), then
[ , ]) * [ , ] ( , ) ( , )
D D
D
f m n F u v f m n F u v
f m n f m n F u v F u v
←⎯⎯⎯→ ←⎯⎯⎯→
←⎯⎯⎯→
![Page 51: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/51.jpg)
• Colour plays an important role in image processing• Colour image processing can be divided into two
major areas• Full-colour processing: Colour sensors such as colour
cameras and colour scanners are used to capture coloured image. Processing involves enhancement and other image processing tasks
• Pseudo-colour processing : Assigning a colour to a particular monochrome intensity range of intensities to enhance visual discrimination.
Colour Image Processing
![Page 52: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/52.jpg)
• Visible spectrum: approx. 400 ~ 700 nm
• The frequency or mix of frequencies of the light determines the colour
• Visible colours: VIBGYOR with UV and IR at the two extremes (excluding)
Colour Fundamentals
![Page 53: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/53.jpg)
• Cones are the sensors in the eye responsible for colour vision • Humans perceive colour using three types of cones• Primary colours: RGB – because the cones of our eyes can
basically absorb these three colours. • The sensation of a certain colour is produced due to the mixed response of these three types of cones in a certain proportion
• Experiments show that 6-7 million cones in the human eye can be divided into red, green and blue vision.
• 65% cones are sensitive to red vision, 33% are for green and only 2% are for blue vision (blue cones are the most sensitive)
HVS review
![Page 54: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/54.jpg)
Experimental curves for colour Sensitivity
Absorption of light by red, green and blue cones in the human eye as a function of wavelength
![Page 55: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/55.jpg)
Colour representations: Primary colours
• According to the CIE (Commission Internationale de l’Eclairage, The International Commission on Illumination) the wavelength of
each primary colour is set as follows: blue=435.8nm, green=546.1nm, and red=700nm.
• However this standard is just an approximate; it has been found experimentally that no single colour may be called red, green, or blue There is no pure red, green or blue colour.
• The primary colours can be added in certain proportions to produce different colours of light.
![Page 56: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/56.jpg)
• The colour produced by mixing RGB is not a natural colour. • A natural colour will have a single wavelength, say λ. • On the other hand, the same colour is artificially produced by combining weighted R, G and B each having different wavelength.
• The idea is that these three colours together will produce the same amount of response as that would have been produced by wavelength λ
alone (proportion of RGB is
taken accordingly), thereby giving the sensation of the colour with wavelength λ
to some extent.
Natural and Artificial Colour
![Page 57: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/57.jpg)
• Mixing two primary colours in equal proportion produces a secondary colour of light: magenta (R+B), cyan (G+B) and yellow (R+G).
• Mixing RGB in equal proportion produces white light.
• The second figure shows primary/secondary colours of pigments.
Colour representations: Secondary colours
![Page 58: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/58.jpg)
There is a difference between the primary colours of light and primary colours of pigments.
Primary colour of a pigment is defined as one that subtracts or absorbs a primary colour of light and reflects or transmits the other two.• Hence, the primary colours of
pigments are magenta, cyan, and yellow.
• Corresponding secondary colours are red, green, and blue.
Colour representations: Secondary colours
![Page 59: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/59.jpg)
• Brightness perceived (subjective brightness) is a logarithmic function of light intensity. In other words it embodies the chromatic notion of intensity.
• Hue is an attribute associated with the dominant wavelength in a mixture of light waves. It represents the dominant colour as perceived by an observer. Thus, when we call an object red, orange, or yellow, we are specifying its hue.
• Saturation refers to the relative purity or the amount of white light mixed with hue. The pure spectrum colours are fully saturated. colour such as pink (red and white) is less saturated. The degree of saturation is inversely proportional to the amount of white light added.
Brightness, Hue, and Saturation
![Page 60: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/60.jpg)
• Red, Green, Blue, Yellow, Orange, etc. are different hues. Red and Pink have the same hue, but different saturation. A faint red and a piercing intense red have different brightness.
• Hue and saturation taken together are called chromaticity. • So, brightness + chromaticity defines any colour.
Brightness, Hue, and Saturation (contd..)
![Page 61: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/61.jpg)
XYZ Colour System
• CIE (Commision Internationale de L’Eclairage), 1931. Spectral RGB primaries (scaled, such that X=Y=Z matches spectrally flat white).
• The entire colour gamut can be produced by the three primaries used in CIE 3-colour system. A particular colour (of wavelength λ) be represented by three components X, Y, and Z. These are called tri-stimulus values.
0.490 0.310 0.2100.177 0.813 0.0110.000 0.010 0.990
denotes corresponding spectral component
RXY GZ B
λ
λ
λ
λ
⎡ ⎤⎡ ⎤ ⎡ ⎤⎢ ⎥⎢ ⎥ ⎢ ⎥= ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎣ ⎦ ⎣ ⎦
![Page 62: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/62.jpg)
• A colour is then specified by its tri-chromatic coefficients, defined asx= X/(X+Y+Z) y= Y/(X+Y+Z)z = Z/(X+Y+Z)so thatx + y +z=1
• For any wavelength of light in the visible spectrum, these values can be obtained directly from curves or tables compiled from experimental results.
Colour composition using XYZ
XYZ Colour System
![Page 63: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/63.jpg)
• Shows colour composition as a function of x and y (only two of x, y and z are independent z = 1 – (x + y) and so not independent of them)
• The triangle in the diagram below shows the colour gamut for a typical RGB system plotted as the XYZ system.
The axes extend from 0 to 1. The origin corresponds to BLUE. The extreme points on the axes correspond to RED and GREEN.The point corresponding to x= y= 1/3 (marked by the white spot) corresponds to WHITE.
x
y
01
1
Chromaticity Diagram
![Page 64: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/64.jpg)
• The positions of various spectrum colours from violet (380nm) to red (700 nm) are indicated around the boundary (100% saturation). These are pure colours.
• Any inside point represents mixture of spectrum colours.
• A straight line joining a spectrum colour point to the equal energy point shows all the different shades of the spectrum colour.
Actual Chromaticity Diagram
![Page 65: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/65.jpg)
• Any color in the interior of the “horse shoe" can be achieved
through the linear combination of two pure spectral colors
• A straight line joining any two points shows all the different
colours that may be produced by mixing the two colours
corresponding to the two points
• The straight line connecting red and blue is referred to as the
line of purples
![Page 66: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/66.jpg)
RGB primaries form a triangular color gamut. The white colour falls in the center of the diagram
W
![Page 67: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/67.jpg)
• Colour models are normally invented for practical reasons, and so a wide variety exist.
• The RGB colour space (model) is a linear colour space that formally uses single wavelength primaries.
• Informally, RGB uses whatever phosphors a monitor has as primaries
• Available colours are usually represented as a unit cube — usually called the RGB cube — whose edges represent the R, G, and B weights.
RGB 24-bit colour cube
Schematic of the RGB colour cube
Colour vision model: RGB colour Model
![Page 68: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/68.jpg)
111
C RM GY B
⎡ ⎤ ⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥ ⎢ ⎥= −⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎣ ⎦ ⎣ ⎦
Cyan, Magenta, Yellow Primary pigment colour
Subtractive color space
Related to RGB by
111
CMY
⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥=⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎣ ⎦
Should produce black
Practical printing devices additional black pigment is needed. This gives the CMYK colour space
CMY and CMYK colour models
![Page 69: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/69.jpg)
Decoupling the intensity from colour components has several advantages:• Human eyes are more sensitive to the intensity than to the hue• We can distribute the bits for encoding in a more effective way. • We can drop the colour part altogether if we want gray-scale images. • In this way, black-and-white TVs can pick up the same signal as color ones.
• We can do image processing on the intensity and color parts separately.
Example:Histogram equalization on the intensity part to contrast enhance the image while leaving the relative colors the same
Decoupling the colour components from intensity
![Page 70: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/70.jpg)
SIH
I
• Hue is the colour corresponding to the dominant wavelength measured in angle with reference to the red axis
• Saturation measures the purity of the colour. In this sense impurity means how much white is present. Saturation is 1 for a pure colour and less than 1 for an impure colour.• Intensity is the chromatic equivalent of brightness also means the grey level component.
HSI Colour system
![Page 71: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/71.jpg)
• HSI model can be obtained from the RGB model.
• The diagonal of the joining Black and White in the
RGB cube is the intensity axis
HSI Colour Model
![Page 72: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/72.jpg)
HSI Model
![Page 73: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/73.jpg)
• HSI colour model based on a
triangle and a circle are shown
• The circle and the triangle are
perpendicular to the intensity axis.
HSI Colour Model
![Page 74: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/74.jpg)
[ ]
[ ]
( ) [ ]
)(31
),,min(31
))(()(
)()(21
cos
if360 if
212
1
BGRI
BGRBGR
S
BGBRGR
BRGR
GBGB
H o
++=
++−=
⎪⎭
⎪⎬
⎫
⎪⎩
⎪⎨
⎧
−−+−
−+−=
⎩⎨⎧
>−≤
=
−θ
θθ
Conversion from RGB space to HSI
The following formulae show how to convert from RGB space to HSI:
![Page 75: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/75.jpg)
oo H 1200 ≤≤oo H 240120 ≤≤
oo H 360240 ≤≤
( )( )BRIG
HHSIR
SIB
+−=
⎥⎦
⎤⎢⎣
⎡−°
+=
−=
360coscos1
)1(
( )( )
120(1 )
cos1cos 60
3
H HR I S
S HG IH
B I R G
= − °= −
⎡ ⎤= +⎢ ⎥°−⎣ ⎦= − +
( )( )BGIR
HHSIB
SIGHH
+−=
⎥⎦
⎤⎢⎣
⎡−°
+=
−=°−=
360coscos1
)1(240
Conversion from RGB space to HSITo convert from HSI to RGB, the process depends on which colour sector H lies in. For the RG sector:For the GB sector:For the BR sector:
![Page 76: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/76.jpg)
• YIQ colour model is the NTSC standard for analog video transmission
Y stands for intensityI is th e in phase component, orange-cyan axisQ is the quadrature component, magenta-green axis
• Y component is decoupled because the signal has to be made compatible for both monochrome and colour television.
• The relationship between the YIQ and RGB model is
0.299 0.581 0.1140.596 0.274 0.3220.211 0.523 0.312
Y RI GQ B
⎡ ⎤ ⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥ ⎢ ⎥= − −⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎢ ⎥−⎣ ⎦ ⎣ ⎦ ⎣ ⎦
YIQ model
![Page 77: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/77.jpg)
• International standard for studio-quality video• This colour model is chosen in such a way that it achieves
maximum amount of decorrelation. • This colour model is obtained by extensive experiments on human observers
0.299 0.587 0.114
b
r
Y R G BC B YC R Y
= + += −
= −
Y-Cb-Cr colour model
![Page 78: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/78.jpg)
Colour balancingRefers to the adjustment of the relative amounts of red, green, and blue primary colors in an image such that neutral colors are reproduced correctly Colour imbalance is a serious problem in Colour Image Processing
• Select a gray level, say white, where RGB components are equal• Examine the RGB values. Keep one component fixed and
match the other components to it, there by defining a
transformation for each of the variable components
• Apply the transformation to all the images
![Page 79: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/79.jpg)
Example: Colour Balanced Image
![Page 80: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/80.jpg)
Histograms of a colour image
1 2 3 4 5 6 7 8S1
S4
S7
0
1
2
3
4
5
6
7
8
Hue
Saturation
-Histogram of Luminance and chrominance components separately
-Colour histograms ( H-S components or normalized R-G components
-Useful way to segment objects like skin non-skin
-Colour based indexing of images
![Page 81: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/81.jpg)
Contrast enhancement by histogram equalisationHistogram equalisation cannot be applied separately for each channel• Convert to HIS space
• Apply histogram equalisation to the I component
• Correct the saturation if needed
• Convert back to RGB valuesDigital Image Processing, 2nd ed.Digital Image Processing, 2nd ed.
www.imageprocessingbook.com
© 2002 R. C. Gonzalez & R. E. Woods
Chapter 6Color Image Processing
Chapter 6Color Image Processing
![Page 82: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/82.jpg)
Colour image smoothing• Vector processing is used
• Averaging in vector is equivalent to averaging separately in each channel
Example- Averaging low pass filter: Averaging a vector is equivalent to averaging all the components
![Page 83: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/83.jpg)
Colour image sharpening
![Page 84: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/84.jpg)
Vector median filter
•We cannot apply the median filtering to the component images separately because that will result in colour distortion
•If each channel is separately median filtered then the net median will be completely different from the values of the pixel in the window
•Vector median filter will minimize the sum of the distances of a vector pixel from the other vector pixels in the window
•The pixel with the minimum distance will give the vector median.
•The set of all vector pixels inside the window is given by
W 1 2{ }N=X x x x, , ....,
![Page 85: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/85.jpg)
Computation of vector median filter
(1) Find the sum of the distances of the vector pixel from all other neighbouring vector pixels in the window given by
th (1 )i i N≤ ≤iδ
1( , )
N
i i jj
dδ=
= ∑ x x
where represents an appropriate distance measure between the ith and jth neighbouring vector pixels
( , ) d i jx x
(2) Arrange s in the ascending order. Assign the vector pixel a rank
equal to that of
Thus, an ordering implies the same ordering of
the corresponding vectors given as
iδ ix
.iδ
(1) (2) ( ).... Nδ δ δ≤ ≤ ≤
(1) (2) ( ).... N≤ ≤ ≤x x x
where are the rank-ordered vector pixels with the number inside the parentheses denoting the corresponding rank.
(1) (2) ( ).... N≤ ≤ ≤x x x
(1) Find the sum of the distances of the vector pixel from all other neighbouring vector pixels in the window given by
th (1 )i i N≤ ≤iδ
1( , )
N
i i jj
dδ=
= ∑ x x
where represents an appropriate distance measure between the ith and jth neighbouring vector pixels
( , ) d i jx x
(2) Arrange s in the ascending order. Assign the vector pixel a rank
equal to that of
Thus, an ordering implies the same ordering of
the corresponding vectors given as
iδ ix
.iδ
(1) (2) ( ).... Nδ δ δ≤ ≤ ≤
(1) (2) ( ).... N≤ ≤ ≤x x x
where are the rank-ordered vector pixels with the number inside the parentheses denoting the corresponding rank.
(1) (2) ( ).... N≤ ≤ ≤x x x
![Page 86: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/86.jpg)
Computation of vector median filter (contd..)
The set of rank ordered vector pixels is given by
R (1) (2) ( ){ , ,..., }N=X x x x
(3) Take the vector median as VMF (1)=x x
The vector median is defined as the vector that corresponds to the minimum SOD to all other vector pixels
![Page 87: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/87.jpg)
Edge Detection and Colour image segmentation
Considering the vector pixels as feature vectors we can apply clustering technique to segment the colour image
![Page 88: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/88.jpg)
EDGE DETECTION•
Edge detection is one of the important and difficult operations in image processing.
•
It is important step in image segmentation the process of partitioning image into constituent objects.
•
Edge indicates a boundary between object(s) and background.
![Page 89: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/89.jpg)
Edge
•
When pixel intensity is plotted along a particular spatial dimension the existence of edge should mean sudden jump or step.
![Page 90: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/90.jpg)
Magnitude of first derivative is maximum
Second derivative crosses zero at the edge point
X0
dfdx
X0
2
2
d fdx
![Page 91: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/91.jpg)
•
All edge detection methods are based on the above two principles.
•
In two dimensional spatial coordinates the intensity function is a two dimensional surface. We have to consider the maximum of the magnitude of the gradient.
![Page 92: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/92.jpg)
•
The gradient magnitude gives the edge location.
•
For simplicity of implementation, the gradient magnitude is approximated by
•
The direction of the normal to the edge is obtained from
•
Second derivative is implemented as a Laplacian given by
![Page 93: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/93.jpg)
•
differentiation is highly prone to high frequency noise. •
An ideal differentiation corresponds to the function being changed in the frequency domain by the addition of a zero at origin. Thus there is an increase of 20dB per decade. This will lead to high frequency noise being amplified.
•
To circumvent this problem, low pass filtering has to be performed.
•
Differentiation is implemented as finite difference operation.
![Page 94: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/94.jpg)
•
Three types of differences generally done are:
•
forward difference = f(x+1) –
f(x)•
backward difference = f(x) –
f(x+1)
•
centre difference = { f(x+1) –
f(x-1) } / 2
•
The most common kernels used for the gradient edge detector are the Roberts, Sobel and Prewitt edge operators.
![Page 95: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/95.jpg)
Roberts Edge Operator
Disadvantage: High sensitivity to noise
![Page 96: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/96.jpg)
Prewitt Edge Operator
•
Does some averaging operation to reduce the effect of noise.
•
May be considered as the forward difference operations in all 2-pixel blocks in a 3 x 3 window.
![Page 97: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/97.jpg)
Sobel Edge Operator
•
Does some averaging operation to reduce the effect of noise, like the Prewitt operator.
•
May be considered as the forward difference operations in all 2 x 2 blocks in a 3 x 3 window.
![Page 98: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/98.jpg)
Gradient Based Edge detection
•
Find fx and fy using a suitable operator.
•
Compute gradient
•
Edge pixels are those for which where T is a suitable threshold
![Page 99: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/99.jpg)
Example
![Page 100: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/100.jpg)
Second derivative Based
•
For the two-dimensional image, we can consider the orientation-free Laplacian operator as the second derivative. The Laplacian of the image f is given by
![Page 101: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/101.jpg)
![Page 102: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/102.jpg)
Laplacian Operator
•
Advantages:No thresholding symmetric operation
•
Disadvantages:Noise is more amplifiedIt does not give information about edge orientation
![Page 103: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/103.jpg)
Model based edge detection
•
Marr studied the literature on mammalian visual systems and summarized these in five major points:–
In natural images, features of interest occur at a variety of scales. No single operator can function at all of these scales, so the result of operators at each of many scales should be combined.
–
A natural scene does not appear to consist of diffraction patterns or other wave-like effects, and so some form of local averaging (smoothing) must take place.
–
The optimal smoothing filter that matches the observed requirements of biological vision (smooth and localized in the spatial domain and smooth and band-limited in the frequency domain) is the Gaussian.
![Page 104: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/104.jpg)
–
When a change in intensity (an edge) occurs there is an extreme value in the first derivative or intensity. This corresponds to a zero crossing in the second derivative.
–
The orientation independent differential operator of lowest order is the Laplacian.
•
Based on the five observations an edge detection algorithm is proposed as follows:
–
Convolve the image with a two dimensional Gaussian function. –
Compute the Laplacian of the convolved image. –
Edge pixels are those for which there is a zero crossing in the
second derivative.
![Page 105: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/105.jpg)
LOG Operation
Convolving the image with Gaussian and Laplacian operator can be combined into convolution with Laplacian of Gaussian (LoG) operator ( Inverted Maxican Hat)
� Continuous function and discrete approximation
![Page 106: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/106.jpg)
Canny Edge detector
• Canny’s criterion:
–
Minimizing the error of detection.–
Localization of edge i.e. edge should be detected where it is present in the image.
–
single response corresponding to one edge.
![Page 107: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/107.jpg)
Canny Algorithm1.
Smooth the image with Gaussian filter •
If one wants more detail of the edge, then the variance of the filter is made large .If less detail is required, then the variance is made small.
•
Noise is smoothed out
2.
Gradient Operation3.
Non Maximal Suppression:
•
Consider the pixels in the neighborhood of the current pixel. If
the gradient magnitude in either of the pixels is greater than the current pixel, mark the current pixel as non edge.
4.
Thresholding with hysterisis: •
mark all pixels with Δf > TH
as the edges. •
mark all pixels with Δf < TL
as non edges. •
a pixel with TL
> Δf < TH
marked as an edge only if it is connected to a strong edge.
![Page 108: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/108.jpg)
Example
Canny
![Page 109: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/109.jpg)
Edge linking
•
After labeling the edges, we have to link the similar edges to get the object boundary.
•
Two neighboring points (x1
,y1
)
and (x2
,y2
) are linked if
![Page 110: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/110.jpg)
Line Detection and Hough transform
• Many edges can be approximated by straight lines.
• For edge pixels, there are possible lines. n 1 2
n(n )−
• To find whether a point is closer to a line we have to perform
comparisons. Thus, a total of comparisons.1 2
n(n )− 30( )n
![Page 111: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/111.jpg)
y mx c= +( , )m c
x
y
m
c
• Hough transform uses parametric representation of a
straight line for line detection.
![Page 112: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/112.jpg)
( , )x y
c mx y= − +
x
y
m
c
P
1l2l
1 1( , )x y
• The points and are mapped to lines and
respectively in space
),( yx 1 1( , )x y1l 2l
m c−• and will intersect at a point P representing the
values of the line joining and2l1l ( , )m c
),( yx 1 1( , ).x y
• The straight line map of another point collinear with these two
points will also intersect at P The intersection of multiple lines in the m- c plane will give the (m,c) values of lines in the edge image plane.
![Page 113: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/113.jpg)
The transformation is implemented by an accumulator array A, each
accumulator corresponding to a quantized value of ( , ).m c
Corresponding to each edge point for each in the range
find
( , ),x yim
1by ),( jiAIncrement
The array A is initialized to zero.
[ ]min max, ,m m
j ic m x y= − +
m
c
maxM m→
maxN c→
![Page 114: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/114.jpg)
Hough transform algorithm
Initialize a 2-D array A of accumulators to zero.
For each edge point find ( , ),x y c mx y= − +
Increment 1by ),( jiA
Threshold the accumulators: the indices of accumulators with
entry greater than a threshold give values of the lines.( , )m c
Group the edges that belong to each line by traversing each line.
![Page 115: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/115.jpg)
Hough transform variation
• and are, in principle, unbounded: cannot handle
all situations.
m c
• Rewrite the line equation as
cos( ) sin( ) x yθ θ ρ+ =
![Page 116: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/116.jpg)
• Instead of we can consider as the parameters
with varying between -90o and 90o and varying from
0 to for an image.
( , ),m c ( , )ρ θα p
2 2M N+ M N×
θρ
![Page 117: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/117.jpg)
Example
![Page 118: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/118.jpg)
Circle detection• Other parametric curves like circle
ellipse etc. can be detected by Hough transform technique.
2 2 2( ) ( ) constanto ox x y y r− + − = =For circles of undetermined radius, use 3-d Houghtransform for parameters
![Page 119: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/119.jpg)
Example
![Page 120: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/120.jpg)
• Today’s world is dependent upon a lot of data either stored in a computer or transmitted through a communication system
• Compression involves reducing the number of bits to represent the data for storing and transmission.
• Particularly, Image compression is the application of compression
on digital
images.
Compression Basics
![Page 121: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/121.jpg)
Storage Requirement
• Example:One second of digital video without compression requires 720X480X24X25~24.8 MB
• Example: One 4-minute song: 44100samples per second X16 bits per sampleX4X60~20 MB
•How to store these data efficiently?
![Page 122: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/122.jpg)
• The large data rate also means a larger bandwidth requirement for transmission
• For an available bandwidth of the maximum allowable bit-rate is 2B.
2B bits/s can be resolved without ambiguity
How to send large amount of data in real-time data through a limited bandwidth channel, say a telephone channel?
We have to compress the raw data to store and transmit.
Band-width requirement
,B
![Page 123: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/123.jpg)
Lossless Vs Lossy Compression
Lossless Lossy• A compressed image can be
restored without any loss of information
• Perfect reconstruction is not possible but visually useful information is retained
• Provides large compressionApplications
Medical imagesDocument imagesGIS
ExamplesVideo broadcastingVideo conferencing
Progressive transmission of imagesDigital libraries and image databases
![Page 124: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/124.jpg)
Encoder and Decoder
A digital compression system requires two algorithms:
• Compression of data at the source (encoding) • Decompression at the destination (decoding)
![Page 125: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/125.jpg)
What are the principles behind compression?
• Compression is possible if the signal data contains redundancy.
• Statistically speaking the data contain highly correlated sample values.
Example : Speech data, Image dataTemperature and Rainfall Data
![Page 126: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/126.jpg)
Types of Redundancy
• Coding Redundancy• Spatial Redundancy• Temporal Redundancy • Perceptual Redundancy
![Page 127: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/127.jpg)
Coding Redundancy
• Some symbols may be used more often than othersIn English text, the letter E is far more common than the letter Z.
• More common symbols are given shorter code-lengths• Less common symbols are given bigger code-lengths
Coding redundancy is exploited in loss-less coding like Huffman coding
![Page 128: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/128.jpg)
( ) ( ) ( ) ( ). , . , . - - -e a q and j− − −
• Morse Code
• Morse noticed that certain letters occurred more frequently than others. In order to reduce the average time required to send a telegraph message, frequent letters were given shortersymbols.
• Example:
Example of Coding Redundancy
![Page 129: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/129.jpg)
Spatial Redundancy
( ) ( )1 , 2 ,...,x n x n− −
• Neighboring data samples are correlated
• Given a sample a part of
can be predicted if the data are correlated
•
( )x n
![Page 130: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/130.jpg)
Spatial Correlation: Example
![Page 131: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/131.jpg)
• In video, same objects may be present in consecutive
frames so that objects may be predicted
Temporal Redundancy
Frame k Frame k+1
![Page 132: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/132.jpg)
Perceptual Redundancy
• Humans are sensible to only limited changes in the amplitude of the signal
• While choosing the levels of quantization, this fact may be considered
• ‘Visually lossless’ means that the degradation is not visible to the human eye
![Page 133: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/133.jpg)
Example: Humans are less sensitive to variation of colour
32 levels64 levels
![Page 134: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/134.jpg)
Principle Behind lossless Compression
Lossless compression methods work by identifying some frequently occurring symbols in the data, and by representing these symbols in an efficient way.
Examples:
– Run-Length Encoding (RLE).
– Huffman Coding.
– Arithmetic coding.
![Page 135: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/135.jpg)
Elements of Information Theory
• Information is a measure of uncertainty• A more uncertain (probable) symbol has more
information
• Information of a symbol is related to probability
![Page 136: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/136.jpg)
Information Theory (Contd.)
1 2
1 2
2
Source is a random variable that takes the symbols , ,...,
with probabilities , ,...,
The self information or information ( ) is defined as
1( ) log
n
n
i
ii
Xx x x
p p pI x
I xp
⎛ ⎞= ⎜ ⎟
⎝ ⎠
![Page 137: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/137.jpg)
• Suppose a symbol x always occurs. Thenp(x) = 1 => I(x) = 0 ( no information)If the base of the logarithm is 2, then the unit of information is called a bit.If p(x) = 1/2 , I(x) = -log2(1/2) = 1 bit.
• Example: Tossing of a fair coin: outcome of this experiment requires one bit to convey the information.
Information Theory (Contd.)
![Page 138: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/138.jpg)
•
Average Information content of the source.
•
Measures the uncertainty associated with a source
and is called entropy
21
1( ) log /n
ii i
H X p bits symbolp=
⎛ ⎞=∑ ⎜ ⎟
⎝ ⎠
Information Theory (Contd.)
![Page 139: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/139.jpg)
Entropy
• Introduced by Ludwig Boltzmann• His only epitaph
lnS k W=
![Page 140: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/140.jpg)
Properties of Entropy
2
2
2
1. 0 ( ) log ( )2. ( ) log ( ) when all symbols are equally likelyIf is a binary soure with symbols 0 and 1 emitted with probability and (1- ) respectively, then
1( ) log ( ) (1 ) log
H X nH X n n
Xp p
H X p pp
≤ ≤=
= + − 21( )
1 p−
![Page 141: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/141.jpg)
Properties of a Code· Codes should be uniquely decodable.· Should be instantaneous ( we can decode the code by
reading from left to right as soon as the code word is received.
· Instantaneous codes satisfy the prefix property (no code word is a prefix to any other code) .
• The average codeword length is given by
1
nL l pi ia v g i
∑==
avgL
![Page 142: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/142.jpg)
Kraft’s Inequality
1 2
i=1
2
There is an instantaneous binary code withcodewords having lengths , ,.... if and only if
2 1
For example, there is an instantaneous binary codewith lengths 1, 2, 3;,3, since
1 12 2
i
In l
l l l
−
⎛ ⎞⎜ ⎟⎝ ⎠
≤
+ +
∑
2 2 3
3 3
1 1 1 1 1.125 12 2 2 2
1 1 12 2
An example of such a code is 0; 10; 110; 111. There is no instantaneous binary codewith lengths 1, 2, 2, 3, since
⎛ ⎞ ⎛ ⎞⎜ ⎟ ⎜ ⎟⎝ ⎠ ⎝ ⎠
⎛ ⎞ ⎛ ⎞ ⎛ ⎞+ + + = >⎜ ⎟ ⎜ ⎟ ⎜ ⎟⎝ ⎠ ⎝ ⎠ ⎝ ⎠
+ =
![Page 143: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/143.jpg)
Shannon’s Noiseless Coding theorem
1 2
Given a discrete memoryless source X with symbols , ,...,
the average code word length for any instantaneous code is given by ( )
More over there exists at least one code such that
n
avg
avg
x x x
L H X
L
≥
≤ ( ) 1 H X +
![Page 144: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/144.jpg)
Shannon’s Noiseless Coding theorem (Contd..)
1 2
Given a discrete memory less source X with symbols , ,..., if we code strings of symbols at a time,
the average code word length for any instantaneous code is given by ( )
n
avg
x x x
L H X→
![Page 145: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/145.jpg)
Example
1
2
3
4
2 2
2
Symbol Probability 0.125 0.125 0.250 0.500( ) 0.125log (1/ 0.125) 0.125log (1/ 0.125)
0.125log (1/ 0.25) 0.
xxxxH X = +
+ + 2125log (1/ 0.5) 1.125 bit/symbol=
![Page 146: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/146.jpg)
Example (Contd..)
1
2
3
4
Symbol Probability code 0.125 000 0.125 001 0.250 01 0.500 1
0.125 3 0.125 3 0.25av
xxxxL X X X= + + 2 0.5 1 1.125 bit/symbol
X+
=
![Page 147: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/147.jpg)
• A prefix code can be represented by a binary tree, each branch being denoted by 0 or 1, emanating from a root node and having n leaf nodes
• A prefix code is obtained by tracing out branches from the root node to each leaf node.
Prefix code and binary tree
![Page 148: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/148.jpg)
• Based on a loss-less statistical method of the 1950s.
0.125 0.125 0.25 0.5
0
0
0
11
1
Huffman coding
• Creates a probability tree and combines the two lowest probabilities to obtain the code
![Page 149: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/149.jpg)
Huffman Coding ( Contd..)
• Most common data value (with the highest frequency)
has the shortest code• Huffman table of data value versus code must be sent • Time of coding and decoding can be long• Typical compression ratios 2:1 – 3:1
![Page 150: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/150.jpg)
ip
Steps in Huffman Coding
• Arrange symbol probabilities in decreasing order while there is more than one node
• Merge the two nodes with the smallest probabilities to form a new node with probabilities equal to their sum
• Arbitrarily assign 1 and 0 to each pair of branches merging in to a node
• Read sequentially from the root node to the leaf node where the symbol is located
![Page 151: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/151.jpg)
40 40 10 10 10 10 10 10 10 10 10 0 0 0 0
40 2 10 9 0 4
Run-length coding
• Looks for sequential pixel valuesExample: 1 row of an image with the new code
below
• Has reduced the size from 18 bytes to 6• Higher compression ratios when predominantly low frequency information
•Typical compression ratios of 4:1 to 10:1•Used in Fax machine•Used for coding the quantized transform coefficients in a lossy coder
![Page 152: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/152.jpg)
Arithmetic coding
• Codes a sequence of symbols rather than a single symbol at a time
• Now, the sequence has to be coded
• A single number lying between 0 and 1 is generated,
corresponding to all the symbols in the sequence, this
number is called tag1 2 3( ) 0.7, ( ) 0.8, ( ) 1F a F a F a= = =
1 2 2 3a a a a
1 2 3 1 2 3{ , , }; ( ) 0.7, ( ) 0.1, ( ) 0.2A a a a p a p a p a= = = =
![Page 153: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/153.jpg)
• Choose the interval corresponding to the first symbol; the tag willlie in the interval
•Go on subdividing the subintervals according to the symbol probability •The code is the AM of the final subinterval
•The tag is sent to decoder which has to know the symbol probabilities
•The decoder will repeat the same procedure to decode the symbols
0 0 0.49 0.539 0.5446
0.5460.5460.560.7
a1a1
a2
a1
a3
a1a1
a3 a3 a3a3
a3a2a2
a2a2
a2 Tag
![Page 154: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/154.jpg)
Decoding algorithm for Arithmetic coding
( )
0 0
1*
1 1
*1
k
In itia lise 0 , 0 , 1R epeat 1
such that ( )
U pdate u ,U ntil k= size o f the sequence
k
k k
k X k x k
k
k l uk k
T A G ltu l
x F x t F x
l
−
− −
−
= = == +
−=
−≤ <
Disadvantage• Assumes data to be stationary, does not consider dynamics of data
•Arithmetic code is used to code the symbols in JPEG2000
![Page 155: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/155.jpg)
• Similar to run-length coding but with some statistical methods similar to Huffman
• Dynamically builds up a dictionary by both encoders and decoders
• Examples:Unix command compressionImage Compression- Graphics Interchange Format (GIF)Portable document format (PDF)
LZW (Lemple, Ziv and Welch) coding
![Page 156: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/156.jpg)
•
Initialize table with single character strings
STRING = first input character WHILE not end of input stream CHARACTER = next input character IF STRING + CHARACTER is in the string table
STRING = STRING + CHARACTER ELSE
output the code for STRING add STRING + CHARACTER
to the string table STRING = CHARACTER END WHILE
output code for STRING
LZW Coding (contd..)
![Page 157: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/157.jpg)
ExampleLet ‘aabbbaa’ be the sequence to be encoded; the dictionary will be
The output for the given sequence is ‘11253, which is ‘aabbbaa’ according to dictionary
![Page 158: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/158.jpg)
• Throws away both non-relevant information and a part of
relevant information to achieve required compression
• Usually, involves a series of algorithm-specific
transformations to the data, possibly from one domain
to another (e.g to frequency domain in Fourier Transform)
without storing all the resulting transformation terms and
thus, loosing some of the information contained
Lossy Compression
![Page 159: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/159.jpg)
• Perceptually unimportant information is discarded.
• The remaining information is represented efficiently to achieve compression
• The reconstructed data contains degradations with respect to the original data
Lossy Compression (contd..)
![Page 160: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/160.jpg)
Example
• Differential Encoding: Stores the difference between consecutive data samples using a limited number of bits.
• Discrete Cosine Transform (DCT): Applied to image data.
• Vector Quantization
• JPEG (Joint Photographic Experts Group)
![Page 161: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/161.jpg)
Fig. Original Lena image, and Reconstructed image from
lossy Compression
![Page 162: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/162.jpg)
Rate distortion theory
•
Rate distortion theory deals with the problem of representing information allowing a distortion
-Less exact representation requires less bits
X Y
Lossy Coder
Average distortion ( ) ( )2 2 ( ) ( / )x y
E X Y x y p x p y x∑∑− = −
![Page 163: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/163.jpg)
Rate Distortion theory
X Ysource Lossy
Coder
I( X,Y ) = H(X) – H(X| Y)
Hence: minimize I( X,Y ) under the constraint D
Minimize the bit rate
Constraint: Average Distortion between X and Y ≤
D
![Page 164: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/164.jpg)
Rate Distortion function for a Gaussian source
If the source X is a Gaussian random variable, the rate distortionfunction is given by
( ) 2 21 log , D<2
0 , otherwise
R D Dσ σ⎛ ⎞= ⎜ ⎟⎝ ⎠
2σ is the variance of Gaussian random variable
σ2
D
R(D)
![Page 165: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/165.jpg)
Rate Distortion
•
Gaussian case presents the worst case of coding
•
For a non Gaussian case, achievable bit error rate is lower than that of Gaussian.
• If we do not know about anything about
the distribution of X, then Gaussian case gives us the pessimistic bound.
•
An increase of 1 bit improves the SNR by about 6 dB.
![Page 166: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/166.jpg)
Lossy Encoder
Fig. A Typical Lossy Signal/Image Encoder
Compressed Data
Entropy CodingQuantization
Quantization tableEntropy-coding table
Input Data Prediction/ Transformation
![Page 167: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/167.jpg)
Differential Encoding• Given a sample a part of
can be predicted if the data are correlated.
• A simple prediction scheme expresses the predicted value as a linear combination of past p samples:
[ ] [ ] [ ]1 , 2 ,..., ,x n x n x n p− − −
[ ] [ ]1
ˆp
ii
x n a x n i=
= −∑
[ ]x n
![Page 168: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/168.jpg)
• The prediction parametersare estimated using correlation among data
• The prediction parameters and the prediction error are transmitted
1 2, ..... pa a a
1 2, ..... pa a aˆ( ) ( )x n x n−
Linear Prediction Coding (LPC)
![Page 169: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/169.jpg)
LPC (contd..)
• Variants of LPC (10) are used for coding speech for mobile communication
• Speech is sampled at 8000 samples per second• Frames of 240 samples ( 30 msec of data) are
considered for LPC• Corresponding to each frame, quantized versions of
10 prediction parameters and approximate prediction errors are transmitted
![Page 170: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/170.jpg)
Transform coding• Transform coding applies an invertible linear coordinate
transformation to the image.
• Most of the energy will be stored in a few transform coefficients
• Example: Discrete Cosine transform (DCT), Discrete wavelet transform (DWT)
TransformCorrelated data
Less correlated data
![Page 171: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/171.jpg)
Transform Merits Demerits
KLT Theoretically optimal •Data dependent•Not fast
DFT Very fast • Assumes periodicity of data• High frequency distortion is more because of Gibb’s phenomenon
DCT • Less high frequencydistortion
•High energy compactionHigh Energy CompactionScalabilty
Blocking Artifacts
Computationally complex
•Also, DCT is theoretically closer to KLT and implementation wise closer to DFT
Transform selection
DWT
![Page 172: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/172.jpg)
• Reversible transform, like Fourier Transform• N samples of signal
[ , ], 0,1, ..., 1, 0,1, ..., 1f m n m N n N= − = −
Discrete Cosine Transform (DCT)
( )( )1 1
0 0
(2 1) 2 1( , ) ( ) ( ) [ , ]cos , 0,1,.. 1, 0,1,.., 1
2
1 1 0 0 ( ) ( )
2 2 1, 2,.., -1 1,2,.., -1
N N
cm n
m u n vF u v u v f m n u N v N
N
u vN N
with u and vu N v N
N N
πα α
α α
− −
= =
⎛ ⎞+ + += = − = −⎜ ⎟⎜ ⎟
⎝ ⎠⎧ ⎧= =⎪ ⎪⎪ ⎪= =⎨ ⎨⎪ ⎪= =⎪ ⎪⎩ ⎩
∑ ∑
DCT is given by
![Page 173: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/173.jpg)
x 50 54 49 55 52 53 54 49DCT 147.07 -0.32 -2.54 1.54 -1.41 1.13 -4.30 -3.0366
Round 147 0 -3 2 -1 1 -4 -3Threshold 147 0 0 0 0 0 -4 0
IDCT 51 54 50 53 53 50 54 51
• We see that only two DCT coefficients contain most information about the original signal
• DCT can be easily extended to 2D
DCT (contd..)
![Page 174: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/174.jpg)
Block DCT• DCT can be efficiently implemented in blocks using FFT and other
fast methods.• FFT based transform is more computationally efficient if applied
in blocks rather than on the entire data.
• For a data length and point FFT, computational
complexity is of order 2logN N
N N
• If the data is divided into sub-blocks of length then
the number of sub-blocks is and the
nNn
computational complexity 2 2log logN n n N nn
= =
![Page 175: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/175.jpg)
How to choose block size?• Smaller block-size gives more computational efficiency• Neighboring blocks will be correlated causing inter block redundancy.• If blocks are coded independently, blocking artifact will appear
Block size
Reconstruction error
2 2× 4 4× 8 8×
• Beyond block size, reduction in error is not significant. 8 8×
![Page 176: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/176.jpg)
4 DCT co-efficients per 8X8 block
16 DCT co-efficients per 8X8 block
8 DCT co-efficients per 8X8 block
![Page 177: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/177.jpg)
• Replaces the transform coefficients with lower-precision approximations which can be coded in a more compact form
• A many-to-one function.
• Precision is limited by the number of bits available.
• X= 147.07 -0.32 -2.54 1.54 -1.41
Quant(X)= 147 0 -3 2 -1
Quantization
![Page 178: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/178.jpg)
Quantization (contd..)
• Information theoretic significance
• More the variance of the co-efficients, more is the information
• Estimate the variance of each transform coefficient from
given image or determine the variance from the assumed model
• In the DCT,
DC co-efficients: Raleigh distribution
AC co-efficients: Generalized Gaussian distribution model
![Page 179: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/179.jpg)
•Two methods for quantization are zonal coding and threshold coding
Zonal coding
• The co-efficients with more information content (more variance)
are retained
Threshold coding• The co-efficients with higher energy are retained, the rest are
assigned zero
• More adaptive
• Computationally exhaustive
![Page 180: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/180.jpg)
Zonal Coding mask and the number of bits allotted for each coefficient
![Page 181: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/181.jpg)
Original image and its DCT
Reconstructed image from truncated DCT
![Page 182: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/182.jpg)
• Joint Photographic Expert Group• A generally used lossy image coding format• Allows tradeoff between compression ratio and image
quality• Can achieve high compression ratio(20+) with almost
invisible difference
JPEG
![Page 183: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/183.jpg)
8x8DCT
HuffmanCoding
quantizationImage
HuffmanTable
QuantizationTable
Coded Image
JPEG (contd..)
![Page 184: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/184.jpg)
Baseline JPEG
•
Divide image into blocks of size 8X8.•
Level shift all 64 pixels values in each
block by subtracting, (where is themaximum number of gray levels).•
Compute 2D DCT of a block.
•
Quantize DCT coefficients using a quantization table.•
Zig-zag
scan the quantized DCT coefficients to
form 1-D sequence.•
Code 1-D sequence (AC and DC) usingJPEG
Huffman variable length codes.
12n− 2n
![Page 185: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/185.jpg)
An image block and DCT
An 8X8 intensity block
![Page 186: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/186.jpg)
QuantizationQuantization table
![Page 187: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/187.jpg)
Zig-zag
scanning
![Page 188: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/188.jpg)
Zigzag scanning
AC Coefficients18 –9 –3 –8 1 –3 1 –2 4 –2 4 0 3 –1 0 1 1 0 –1 –1 0 0 0 –1 (39 zeros)
![Page 189: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/189.jpg)
Wavelet Based Compression
•
Recall that DWT is implemented through Row-wise and column-wise filtering and the down-sampling by 2 after each filtering. The approximate image is further decomposed.
•
First and second stages of decomposition are illustrated in the figure below.
LL1
LH1
HL1
HH 1
First stage
LH1
HL1
HH 1
Second stage
LL2
LH2
HL2
HH2
![Page 190: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/190.jpg)
Embedded Tree Image Coding
Embedded bit stream: A bit stream at a lower rate is contained in a higher rate bit stream (good for progressive transmission)
• Embedded Zero-tree Wavelet (EZW) codingalgorithm, Shapiro [1993]
•Set Partitioning In Hierarchical Trees (SPIHT)- based algorithm, Said and Pearlman [1996]
•EBCOT (Embedded Block Coding with Optimized Truncation) proposed by Taubman
in 2000.
![Page 191: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/191.jpg)
Tree representation of Wavelet Decomposition
![Page 192: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/192.jpg)
EZW
EZW scans wavelet coefficients subband
by subband. Parents are scanned before any of their children, but only after all neighboring parents have been scanned.
![Page 193: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/193.jpg)
EZW coding
Each coefficient is compared against the current threshold T. A coefficient is significant if its amplitude is greater then T;such a coefficient is then encoded as
Positive significant (PS) Negative significant (NS)
Zerotree root (ZTR)
is used to signify a coefficient below T, with all its children also below TIsolated zero (IZ)
signify a coefficient below T, but with
at least one child not below T2 bits are needed to code this information
![Page 194: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/194.jpg)
Successive Approximation quantization
•Sequentially applies a sequence of thresholds T0,···,TN-1 to determine significance•Three-level mid-tread quantizer•Refined using 2-level quantizer
![Page 195: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/195.jpg)
Example
l g l g 522 max 22 2 32o C oThreshold T ⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎣ ⎦= = =
![Page 196: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/196.jpg)
Quantization
+8
-8
![Page 197: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/197.jpg)
2nd
Pass
![Page 198: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/198.jpg)
• Not only better efficiency, but also more functionality
• Superior low bit-rate performance• Lossless and lossy compression• Multiple resolution• Region of interest(ROI)
JPEG 2000
![Page 199: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/199.jpg)
Transform Quantization EntropyCoding
JPEG
J2K
DCT DiscreteCosineTransform
DWT DiscreteWaveletTransform
8x8Quantization
Table
Quantizationfor each
sub-band
HuffmanCoding
ArithmeticCoding
JPEG2000 v.s. JPEG
![Page 200: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/200.jpg)
low bit-rate performance
JPEG2000 v.s. JPEG
![Page 201: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/201.jpg)
• A video sequence consists of a number of pictures, containing a lot of time domain redundancy.
• This is often exploited to reduce data rates of a video sequence leading to video compression.
• Motion-compensated frame differencing can be used very effectively to reduce redundant information in sequences
• Finding corresponding points between frames (i.e., motion estimation) can be difficult because of occlusion, noise, illumination changes, etc
• Motion vectors (x,y-displacements) are sent
Video Compression
![Page 202: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/202.jpg)
Reference frame Current frame
Predicted frame Error frame
Motion-compensated Prediction
![Page 203: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/203.jpg)
Search procedure
Current frame
Reference frame
Current block
Best match Search region
![Page 204: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/204.jpg)
Search Algorithms
• Exhaustive Search• Three-step search• Hierarchical Block Matching
![Page 205: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/205.jpg)
Three step search algorithm
Search window of ±(2N –1) pixels is selected (N=3)Search at location (0,0)Set S= 2N-1 (the step size)Search at eight locations ±S pixels around location (0,0)From the nine locations searched so far, pick the location with smallest Mean Absolute Difference (MAD) and make this the new search origin.Set S=S/2.Repeat stages 4-6 until S=1
![Page 206: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/206.jpg)
Minimum atfirst iteration
Minimum atseconditeration
Minimum atthird iteration
First iteration
Seconditeration
Thirditeration
Three step search algorithm
![Page 207: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/207.jpg)
Video Compression Standards
Two Formal OrganizationsInternational Standardization Organization/ International Electro- technical Commission (ISO/IEC) International Telecommunications Organization (ITU-T)
ITU-T Standard •H.261 (1990) •H.263 (1995)
ISO/IEC standards•MPEG-1 (1993)•MPEG-4 (1998)
ITU-T and ISO/IEC MPEG-2 (1995)H.264/AVC (2003)
![Page 208: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/208.jpg)
Applications And The Bit-rates Supported
Standard Application Target bit-rateH.261 Video conferencing and video
telephonyPx64 kbps
1≤P≤30MPEG-1 CD-ROM video applications 1.5Mbps
MPEG-2/ H.262 HDTV and DVD 15-30 MbpsH.263 Transmissions over PSTN
networksUp to 64kbps
MPEG-4 Multimedia applications 5kbps-50Mbps
H.264 Broad cast over cable, Video on demand, Multimedia streaming services
64kbps to 240Mbps
![Page 209: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/209.jpg)
• Motion Pictures Expert Group• Standards for coding video and the associated
audio• Compression ratio above 100• MPEG 1, MPEG 2, MPEG 4, MPEG 7, MPEG
21….
MPEG Video Standards
![Page 210: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/210.jpg)
MPEG 2 Coder Decoder
![Page 211: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/211.jpg)
• The MPEG system specifies three types of frames within a sequence:
• Intra-coded picture (I-frame): Coded independently from all other frames
• Predictive-coded picture (P-frame): Coded based on a prediction from a past I- or P-frame
• Bidirectionally predictive coded picture (B- frame): Coded based on a prediction from a past and/or future I- or P-frame(s). Uses the least
MPEG 2 Frame Types
![Page 212: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/212.jpg)
MPEG 2 GOP structure
![Page 213: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/213.jpg)
Includes both spatial- and frequency-domain techniques:• Basic gray level transformations• Histogram Modification • Average and Median Filtering • Frequency domain operations • Edge enhancement
Image Enhancement
Aimed at improving the quality of an image for
•Better human perception or
•Better interpretation by machines.
![Page 214: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/214.jpg)
Enhancement technique
Better imageInput image
Image enhancement
•Application specific•No general theory•Can be done in
- Spatial domain: Manipulate pixel intensity directly- Frequency domain: Modify the Fourier transform
![Page 215: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/215.jpg)
Spatial Domain technique
X-Simplest case• depends only on the value of
does not depend on the position of the pixel in the image. • called brightness transform or point processing
[.]g at [ , ];f x y
[ , ] ( [ , ])
( )
g x y T f x yors T r
=
=
![Page 216: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/216.jpg)
Contrast stretching
( )s T r=
![Page 217: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/217.jpg)
Some useful transformations
![Page 218: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/218.jpg)
r
255s r= −
![Page 219: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/219.jpg)
( )s T r=
r Enhanced in the range 100-150 and 150-255
![Page 220: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/220.jpg)
Thresholding [ , ]
[ , ] 255 else[ , ] 0
If I m n ThI m n
I m n
>=
=
Th= 120
![Page 221: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/221.jpg)
log( 1)s c r= +
Log transformation
• Compresses the dynamic range
where c is the scaling factor.
Example : Used to display the 2D Fourier Spectrum
![Page 222: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/222.jpg)
Log transformation (Contd..)
![Page 223: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/223.jpg)
s crγ=
γ c
• Expands dynamic range
Example : Image scaling, same effect as adjusting camera exposure time.
Power law transformation
where and are positive constants
• Often referred to as gamma-correction1,γ =
![Page 224: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/224.jpg)
Example: Image Display in the monitor
Sample Input to Monitor Monitor output
![Page 225: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/225.jpg)
Sample Input
Gamma Corrected Input
Monitor Output
Gamma Correction
![Page 226: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/226.jpg)
Gamma corrected image
Original Image Corrected by 1.5γ =
![Page 227: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/227.jpg)
Gamma correction (Contd..)
![Page 228: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/228.jpg)
r
ss
r
Gray level slicing
![Page 229: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/229.jpg)
Results of slicing in the black and white regions
![Page 230: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/230.jpg)
• Highlights the contribution of specific bits to image intensity
• Analyses relative importance of each bit; aids determining the
number of quantization levels for each pixel
Bit-plane slicing
![Page 231: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/231.jpg)
MSB plane obtained by
thresholding at 128
MSB plane
Original
![Page 232: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/232.jpg)
Original Image and Eight bit-planes
![Page 233: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/233.jpg)
Includes• Histogram• Histogram Equalization• Histogram specification
Histogram Processing
![Page 234: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/234.jpg)
{0,1,.., 1}
( )
Number of pixels with gray level
Total number of pixels
k
kk
k
k
r Lnp rn
nr
n
∈ −
=
→
→
Histogram
histogram:For B-bit image, initialize bins with 0For each pixel x,yIf f(x,y)=i, increment bin with index Iendifendfor
2B
![Page 235: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/235.jpg)
Histogram
![Page 236: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/236.jpg)
Low-contrast image Histogram
![Page 237: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/237.jpg)
Improved-contrast image Histogram
![Page 238: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/238.jpg)
Suppose r represents continuous gray levels0 1r≤ ≤ . Consider a transformation of the form ( )s T r= that satisfies the following conditions (1) ( )s T r= is single valued, monotonically increasing in r . (2) ( )0 1T r≤ ≤ for 0 r 1≤ ≤ [ ] [ ]0,1 0,1T⎯⎯→ Inverse transformation is ( )1T s r− = , 0 1s≤ ≤
Histogram Equalisation
![Page 239: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/239.jpg)
Histogram Equalisation (contd.)
r can be treated as a random variable in [ ]0, 1 with pdf ( )rp r .The pdf of ( )s T r= is
( ) ( )
( )1
rs
r T s
p rp s
dsdr −=
=
( ) ( )
( )0
s=T r ,0 r 1
ds then, dr
r
r
r
Suppose p u du
p r
= ≤ ≤
=
∫
( ) ( )( )
1, 0 1rs
r
p rp s s
p r∴ = = ≤ ≤
![Page 240: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/240.jpg)
Histogram Equalisation
0
{0,1,.., 1}
( )
( )
k
kk
kk i
i
r Ln
p rn
g p r=∑
∈ −
=
=
The resulting needs to be scaled and rounded.
kg
![Page 241: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/241.jpg)
Histogram-equalized image image Histogram
![Page 242: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/242.jpg)
Histogram-equalized Image
![Page 243: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/243.jpg)
Example
The following table shows the process of histogram equalization for a 128X128 pixel 3-bit (8-level) image.
Gray level ( )kr kn kn
n
07
ki
ki
ns round Xn=
⎛ ⎞⎛ ⎞= ⎜ ⎟⎜ ⎟⎝ ⎠⎝ ⎠∑
0 1116 0.0681 0 1 4513 0.2755 2 2 5420 0.3308 5 3 2149 0.1312 6 4 1389 0.0848 6 5 917 0.0560 7 6 654 0.0399 7 7 226 0.0138 7
![Page 244: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/244.jpg)
Histogram specification Given an image with a particular histogram, another image which has a specified histogram can be generated and this process is called histogram specification or histogram matching.
( )( )
r
z
p r original histogram
p z desired histogram
→
→
( )0
r
rs p u du= ∫
( )0
z
zr p w dw= ∫
( ) ( )1 1z G s G T r− −= =
![Page 245: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/245.jpg)
Image filtering
Image filtering involves • Neighbourhood operation • taking a filter mask from point to point in an image and perform operations
on pixels inside the mask.
![Page 246: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/246.jpg)
Linear Filtering
he case of linear filtering, the mask is placed over the pixel; the gray valueof the image are multiplied with the corresponding mask weights and thenadded up to give the new value of the pixel.
Thus the filtered image g[m,n] is given by ', '
' '[ , ] [ ', ']m n
m ng m n w f m m n n= − −∑∑
Where summations are performed over the window. The filtering window is usually symmetric about the origin so that we can write
', '' '
[ , ] [ ', ']m nm n
g m n w f m m n n= + +∑∑
![Page 247: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/247.jpg)
Linear Filtering Illustrated
![Page 248: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/248.jpg)
Averaging Low-pass filter
An example of a linear filter is the averaging low-pass
filter. The output avgI of an averaging filter at any pixel is the average of the neighbouring pixels inside the filter mask. It can be given as ,[ , ] [ , ]avg i j
i jf m n w f m i n j= + +∑∑ ,
where the filter mask is of size m n× and , ,,i j i jf w are the image pixel values and filter weights respectively. Averaging filter can be used for blurring and noise reduction.
• Show that averaging low-pass filter reduces noise. • Large filtering window means more blurring
![Page 249: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/249.jpg)
1/9 1/9 1/9
1/9 1/9 1/9
1/9 1/9 1/9
Original Image Noisy Image
Filtered Image
Averaging filter
![Page 250: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/250.jpg)
Low-pass filter example
• Filtered with 7X7 averaging mask
![Page 251: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/251.jpg)
High-pass filter
0 0 0
0 1 0
0 0 0
1/9 1/9 1/9
1/9 1/9 1/9
1/9 1/9 1/9
A highpass
filtered image can be computed as the difference betweenthe original and a lowpass
filtered version.Highpass
= Original −
Lowpass
-
-1/9 -1/9 -1/9
-1/9 8/9 -1/9
-1/9 -1/9 -1/9
=
![Page 252: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/252.jpg)
−1
-1/9 -1/9 -1/9
-1/9 8/9 -1/9
-1/9 -1/9 -1/9
High-pass filtering
-1/9 -1/9 -1/9
-1/9 8/9 -1/9
-1/9 -1/9 -1/9
![Page 253: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/253.jpg)
[ , ] [ , ] [ , ] 1 ( 1) [ , ] [ , ] [ , ] ( 1) [ , ] [ , ]
s av
av
high
f m n Af m n f m n AA f m n f m n f m n
A f m n f m n
= − >
= − + −
= − +
Unsharp
Masking
![Page 254: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/254.jpg)
Median filtering
• The median filter is a nonlinear filter that outputs
the median of the data inside a moving window of
pre-determined length. This filter is easily
implemented and has some attractive properties
• Useful in eliminating intensity spikes( salt & pepper noise)
• Better at preserving edges
• Works up to 50% of noise corruption
•Verify that median filter is a nonlinear filter.
![Page 255: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/255.jpg)
18 20 20
15 255 17
20 20 20
Sorted data
15 17 18 20 20 20 20 20 255
Median= 20, and 255 will be replaced by 20
Median
![Page 256: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/256.jpg)
Median Filtering
![Page 257: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/257.jpg)
IMAGE TRANSFORMS
![Page 258: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/258.jpg)
Image transform
Signal data are represented as vectors. The transform changes the basis of the signal space. The transform is usually linear but not shift-invariant.
Useful for compact representation of dataseparation noise and salient image featuresefficient compression. .
A transfom may beorthonormal/unitary or non-orthonormalcomplete, overcomplete, or undercomplete.applied to image blocks or the whole image
. .
![Page 259: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/259.jpg)
1D TRANSFORMDATA
![Page 260: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/260.jpg)
UNITARY TRANSFORM
![Page 261: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/261.jpg)
Unitary transform and basis
0,0 0 ,1 0 , 1
1,0 1,1 1, 1
1,0 1,1 1, 1
* * *
0,0 1,0 1,0
* * *
0,1 1,1 1, 1
1
* * *
1,0 1,1 1, 1
......
......:
.....
Then............
:.....
N
N
N N N N
N
N
N N N N
t t tt t t
t t t
t t tt t t
t t t
−
−
− − − −
−
−
−
− − − −
⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥=⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦
⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥=⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦
T
T
![Page 262: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/262.jpg)
Unitary transform and basis (Contd..)
* * *
0,0 1,0 1,0
* * *
0,1 1,1 1, 1
* * *
0, 1 1, 1 1, 1
*
0 ,0
*
0 ,1
*
1,0
Therefore,......[0] [0]......[1] [1]
: : :[ 1] ..... [ 1]
[0] :
N
N
N N N N
N
t t tf Ft t tf F
f N t t t F N
tt
Ft
−
−
− − − −
−
⎡ ⎤⎡ ⎤ ⎡ ⎤⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥=⎢ ⎥⎢ ⎥ ⎢ ⎥− −⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎣ ⎦⎣ ⎦
⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥=⎢⎢⎢⎣ ⎦
* *
1,0 1,0
* *
1,1 1,1
* *
1, 1 1, 1
[1] : ... [1] :
N
N
N N N
t tt t
F Ft t
−
−
− − −
⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥+ +
⎥ ⎢ ⎥ ⎢ ⎥⎥ ⎢ ⎥ ⎢ ⎥⎥ ⎢ ⎥ ⎢ ⎥
⎣ ⎦ ⎣ ⎦
Since the columns of are independent, they form a basis for the N-
dimensional space
1T −
![Page 263: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/263.jpg)
Examples of Unitary Transform
2D DFT
Other Examples : DCT (Discrete Cosine Transform)
DST
(Discrete Sine Transform)
DHT
(Discrete Hardmard
Transform)
KLT
(Karhunen
Loeve
Transform)
![Page 264: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/264.jpg)
1.
The rows of T form an orthogonal basis
for the N-
dimensional complex space.
2.
All the eigen vectors of T have unit magnitude
Tf
= λf
Properties of Unitary Transform
![Page 265: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/265.jpg)
3.
Parseval’s theorem: T is energy preserving transformation, because it is an unitary transform which preserves energy
F*' F = energy in transform domainf*'
f = energy in data domain
F*' F =
[Tf]*'
Tf=
f*'
T*' Tf
=
f*' I f= f*'
f
4.
Unitary transform is a length and distance preserving transform.
5. Energy is conserved, but often will be unevenly distributedamong
coefficients.
![Page 266: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/266.jpg)
Decorrelating propertymakes the data sequence uncorrelated. useful for compression
Let f = [ f[0], … ,f[N-1] ] T be data vector,
= Covariance matrix
= Covariance matrix of transformation
Diagonal elements of CFt– variance, off-diagonal elements of CFt
–
Covariance
Perfect Decorrelation – off-diagonal elements are zero.
![Page 267: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/267.jpg)
2D CASE
![Page 268: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/268.jpg)
Separable Property
![Page 269: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/269.jpg)
Matrix representation
If we consider separable, unitary and symmetric transform.
Thus we can write
2 1
1 1
1 2 1 1 2 20 0[ , ] [ , ] [ , ]N N
n nf n n t k n t n k− −
= =∑ ∑=
![Page 270: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/270.jpg)
2D Separable, unitary and symmetric transforms have following properties:
i.
Energy preservingii.
Distance preservingiii.
Energy compactioniv.
Other properties specific to particular transforms.
.
![Page 271: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/271.jpg)
Karhunen
Loeve
(KL) transform
Let f = [ f[0], … ,f[N-1] ] T be data vector,
= Covariance matrix
Cf can be diagonalised by
where is diagonal matrix formed by eigen valuesU is matrix formed by eigen vectors as columns
Transform matrix of KL transform is
fUCU′ =Λ
Λ
T U ′=
![Page 272: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/272.jpg)
KL transform is
E(FKLT) = 0
KLT KLT fF =T (f -μ )
KL Transform
![Page 273: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/273.jpg)
KL Taransform
Covariance Matrix isKLTFC
![Page 274: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/274.jpg)
For KLT eigen values are arranged in descending order and the transformation matrix is formed by considering the eigen vectorsin order of the eigen values.
Reconstructed value is
Suppose we want to retain only k- transform coefficients we will retain the transformation matrix formed by the k largest eigen vectors.
Principal Component Analysis (PCA) – Linear combination of largest principal eigen vectors.
![Page 275: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/275.jpg)
KLT Illustrated
F1
F2
![Page 276: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/276.jpg)
KL transformMean square error of approximation is
Mean square error is minimum over all affine transforms.
Transform matrix is data dependent so computationally extensive.
1 1
0 0
1
ˆ ˆ( ) ( )N J
i ii i
N
ii j
E Energyof Energyofλ λ
λ
− −
= =
−
=
∑ ∑
∑
′ ′= −
= −
=
f - f f - f f f
![Page 277: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/277.jpg)
1D-DCTLet f = [ f[0], … ,f[N-1] ] T be data vector
1D-DCT of data vector f and its IDCT are
where
![Page 278: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/278.jpg)
The transformation can be written asF = TC f
where TC
is transformation matrix with elements
DCT is a real transform.
TC is orthogonal transformTC
‘
TC = I so TC
-1
= TC‘
![Page 279: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/279.jpg)
Relation with DFTDCT :
Let
DCT of is given by
21 2 2( ) ( )Re ( )0
C
j j knkN N NF k k al f n e en
π πα
⎧ ⎫⎪ ⎪⎪ ⎪⎨ ⎬⎪ ⎪⎪ ⎪⎩ ⎭
− −−= ∑
=
( ) ( ) , 1f n f n n N′ = ≤ −0 , 1 2 1N n N= − < ≤ −
( )f n 22 1 2 2( ) ( )Re ( )0
C
j j knkN N NF k k al f n e en
π πα
⎧ ⎫⎪ ⎪⎪ ⎪⎨ ⎬⎪ ⎪⎪ ⎪⎩ ⎭
− −−′= ∑
=
[ ], for 0,1,..., 10, otherwise
( ) f n n Nf n = −⎧⎨⎩
′ =
![Page 280: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/280.jpg)
Another interpretationExtend the data symmetrically about n=N-1 so that
DFT of is given by2 2 (2 1 )
' 2 2( ) ( ) (2 1 )10
j jnk k N nN Nk f n e f N n e
NFn
π π− − − −⎧ ⎫⎪ ⎪+ − −⎨ ⎬
⎪ ⎪⎩ ⎭
−= ∑=
(21),fNN N=− −<≤−
(2 1)2
22
2 21 2 2 2 2( )0
1 ( )cos( )0
k nN
j nkNe
j nkNe
j j j jnk nk nk nkN N N N Nf n e e e en
N f nn
π
π
π
π π π π⎧ ⎫⎪ ⎪⎪ ⎪⎨ ⎬⎪ ⎪⎪ ⎪⎩ ⎭
= +
− − − −−= +∑
=
−∑=
[ ], for 0,1,..., 1(2 1 ), , 1,...,2 1
( ) f n n Nf N n n N N N
f n = −⎧⎨ − − = + −⎩
′ =
( )f n′
( )
( )
( ) '( )
( ) (2 1)2
(2 1)2
1 ( )cos( )0
1 ( )cos( )0
kc
k
k F k
kk nN
k nN
j jnk nkN Ne e
j nkNe
NF f nn
N f nn
α
α
α
π
ππ π
π=
+
−= +
−
−∴ = ∑=
−∑=
![Page 281: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/281.jpg)
DCT
Interpretation of DCT with DFT
![Page 282: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/282.jpg)
2D-DCT
The 2D –DCT of is
2D – DCT can be implemented from 1D – DCT by
Performing 1D – DCT column wiseThen performing 1D – DCT row wise
1 2( , )f nn
![Page 283: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/283.jpg)
DCT is close to KLTFirstly, DCT of basis vectors will be eigen vectors of triangular matrix
Secondly, a first order markov process with correlation coefficient ρ has a covariance matrix
![Page 284: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/284.jpg)
If ρ is close to 1 then
Therefore for a Markov first order process with ρ close to 1 DCT is close to KLT.
Because of closeness to KLT, energy compaction, data decorrelation and ease of computation DCT is used in many applications.
![Page 285: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/285.jpg)
Many image processing operations are efficiently implemented in terms of matrices
Particularly, many linear transforms are used
Simple example: colur transformation
0.299 0.587 0.114 0.596 -0.275 -0.321 0.212 -0.523 0.311
Y RI GQ B
⎡ ⎤ ⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥ ⎢ ⎥=⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎣ ⎦ ⎣ ⎦
Matrices
![Page 286: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/286.jpg)
Matrix representation of linear operations• Let be data samples.
• These data points can be represented by a vector
• Consider a transformation
• Denoting we get where
[ ] [ ] [ ]0 , 1 ,......, 1x x x N − N
0
1
-1
x
x
x N
⎡ ⎤⎡ ⎤⎣ ⎦⎢ ⎥⎡ ⎤⎢ ⎥⎣ ⎦
⎢ ⎥⎢ ⎥⎢ ⎥⎡ ⎤⎣ ⎦⎣ ⎦
x =
[ ] [ ]1
0
, 0,1,......, 1N
njj
y n a x j n N−
=
= = −∑[ ][ ]
[ ]
0
1...
-1
y
y
y N
⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦
y = y = Ax
0,0 0,1 0,2 0, 1
1,0 1,1 1,2 1, 1
2,0 2,1 2,2 2, 1
1,0 1,1 1,2 1, 1
. . . . . . . . .
.
. . . .
N
N
N
N N N N N
a a a aa a a aa a a a
a a a a
−
−
−
− − − − −
⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥=⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦
A
![Page 287: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/287.jpg)
Example: Discrete Fourier transform
[ ] [ ]21
0 k=0,1,.........N-1
N i nkN
nx k x n e
π− −
=
=∑
[ ][ ]
( )
2 2 ( 1)
2 21 (
0 1 1 . . . 1
1 1 . . . . .. ..
1 . . . [ 1]
j j NN N
j N j NN N
X
X e e
e eX N
π π
π π
− − −
− − − −
⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥ =⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥−⎣ ⎦
( )
[ ][ ]
1) 1
0
1...
[ 1]N
x
x
x N−
⎡ ⎤⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥ −⎣ ⎦ ⎣ ⎦
![Page 288: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/288.jpg)
Example: Rotation operation
x
y
r
r
θ0θ
( ),x y′ ′
( ),x y cos sinx x yθ θ′ = −
sin cosy x yθ θ′ = +
cos sinsin cos
x xy y
θ θθ θ
′ −⎡ ⎤ ⎡ ⎤ ⎡ ⎤=⎢ ⎥ ⎢ ⎥ ⎢ ⎥′⎣ ⎦ ⎣ ⎦ ⎣ ⎦
![Page 289: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/289.jpg)
Matrices - Basic Definitions• Transpose of a matrix:
• Symmetric Matrix: =
• Hermitian Matrix: =
Example:
• Inverse of a Matrix: for a non singular matrix
TA A
TA
*TA A
T1 i 1 i, =
-i 1 -i 1∗⎡ ⎤ ⎡ ⎤
= ⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎣ ⎦
A A
1− =A A I A
![Page 290: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/290.jpg)
• A matrix of complex elements is called unitary if
Example =
A 1 T− ∗=A A
1 1 1 3 3 3
1 1 1 3 1 1 3 2 2 2 23 3 3
1 1 1 3 1 1 3 2 2 2 23 3 3
j j
j j
⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥⎛ ⎞ ⎛ ⎞⎢ ⎥+ − −⎜ ⎟ ⎜ ⎟⎜ ⎟ ⎜ ⎟⎢ ⎥⎝ ⎠ ⎝ ⎠⎢⎢ ⎛ ⎞ ⎛ ⎞
− − − +⎢ ⎜ ⎟ ⎜ ⎟⎜ ⎟ ⎜ ⎟⎢ ⎝ ⎠ ⎝ ⎠⎣ ⎦
⎥⎥⎥⎥
A
Unitary Matrix
![Page 291: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/291.jpg)
•
Orthogonal Matrix
1 T− =A A
cos sinx x yθ θ′ = −
sin cosy x yθ θ′ = +
-sinsin coscosθ θ
θ θ⎡ ⎤
= ⎢ ⎥⎣ ⎦
A1 cos sin
-sin cosTθ θ
θ θ− ⎡ ⎤= =⎢ ⎥⎣ ⎦
A A
Example: Rotation operation
For an orthogonal matrix A
Real-valued unitary matrices are orthogonal
![Page 292: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/292.jpg)
Example
• Is the following matrix orthogonal?
1 1 2 2
1 1 2 2
⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥−⎢ ⎥⎣ ⎦
![Page 293: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/293.jpg)
• A matrix is called Toeplitz if the diagonal contains the same element, each of the sub-diagonals contains the same element and each of the super-diagonals contains the same element.
• The system matrix corresponding to linear convolution of two sequences is a Toeplitz matrix.
• The autocorrelation matrix of a wide-sense stationary process is Toeplitz.
Example
0,0 0,1 0, 1
1,0 0,0 0,1 1, 1
2,0 1,0 0,0 2, 1
1,0 1,1 1,0 0,0
. . . . . . . . .
.
. . .
N
N
N
N N
a a aa a a aa a a a
a a a a
−
−
−
− −
⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥=⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦
A
Toeplitz Matrix
![Page 294: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/294.jpg)
Circulant Matrix
• A matrix is called a Circulant Matrix if each row is obtained by the circular shift of the previous row.
Example
• The system matrix corresponding to circular convolution of two sequences is a circulant matrix.
0,0 0,1 0, 1
0, 1 0,0 0,1 0, 2
0, 2 0, 1 0,0 0, 3
0,1 0,2 0, 1 0,0
. . . . . . . . .
.
. . . ...
N
N N
N N N
N
a a aa a a aa a a a
a a a a
−
− −
− − −
−
⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥=⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦
A
![Page 295: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/295.jpg)
Eigen Values and Eigen Vectors• If is a square matrix and is a scalar such that
then is the eigen value and is the corresponding eigen vector.
• Thus the eigen vectors are invariant in direction under an operation by a matrix.
Example: Rotation operation with then
then No real eigen values and eigen vectors exist.
Now consider Rotation operation with then
so that each vector is invariant under rotation by
A λ λ=Ax xλ x
90θ =0 -11 0
x xy y′⎡ ⎤ ⎡ ⎤ ⎡ ⎤=⎢ ⎥ ⎢ ⎥ ⎢ ⎥′⎣ ⎦ ⎣ ⎦ ⎣ ⎦
0 -11 0⎡ ⎤
= ⎢ ⎥⎣ ⎦
A
180θ = -1 00 -1⎡ ⎤
= ⎢ ⎥⎣ ⎦
A
0180
![Page 296: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/296.jpg)
Basically, a filtering operation. •Set-theoretic operations are used for image processing
applications. Such operations are simple and fast.• In normal image filtering, we have an image and
filtering is done through the convolution mask.• In morphological image filtering, the same is done
through the structuring element
Morphological Image Processing
![Page 297: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/297.jpg)
{ }2( , ) | ( , )x y x y ∈ Z
{ }( , ) | ( , ) 1A x y I x y= =
{ }2( , , ) | ( , ) , ( , )A x y z x y I x y z= ∈ =Z
• Image plane is represented by a set
in Binary Image Morphology
The value of ‘z’ gives the gray value and (x, y) gives the gray point.
• A binary object is represented by
A
Binary Morphology Basics
• In Gray-scale Morphology,{ }( , ) | ( , ) 0cA x y I x y= =Background
![Page 298: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/298.jpg)
Structuring elementA
Entire image
A
Structuring element• It is similar to mask in convolution
• It is used to operate on the object image A
B
![Page 299: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/299.jpg)
{ }ˆ ( , ) | ( , )A x y x y A= − − ∈
(1) Reflection Operation: Reflection of is given by
B̂
B
Reflection and Translation operations
A
Translation of by an amount is given by
( ) { }|x
A a x a A= + ∈
(2) Translation Operation:
xA
x
![Page 300: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/300.jpg)
Binary image morphology operations
a. Dilation operation
b. Erosion operation
c. Closing operation
d. Opening operation
e. Hit or Miss transform
![Page 301: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/301.jpg)
A
( ){ }ˆ|x
A B x A B⊕ = ∩ ≠ ∅
B
Dilation operation
Given a set A and the structuring element B, we define dilation of A with B as
A
B
A
![Page 302: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/302.jpg)
Why dilation?• If there is a very small object, say (hole) inside the object ‘A’, then
this unfilled hole inside the object is filled up.• Small disconnected regions outside the boundary may be
connected by dilation.• Irregular boundary may be smoothened out.
![Page 303: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/303.jpg)
Properties of Dilation
A B B A⊕ = ⊕
( ) ( )A B C A B C⊕ ⊕ = ⊕ ⊕
( )x xA B A B⊕ = ⊕
1.
2.
3. Dilation is translation invariant
Dilation is associative
Dilation is commutative
![Page 304: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/304.jpg)
Erosion operationA
BErosion of A with B is given by,
{ }| xA B x B AΘ = ⊆
• Here, structuring element ‘B’ should be completely inside the
object ‘A’.
![Page 305: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/305.jpg)
Why Erosion?
1. Two nearly connected regions will be separated by erosion operation.
1. Shrinks the size of the objects3. Removes peninsulas and small objects .4. Boundary may be smoothened
![Page 306: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/306.jpg)
Properties of Erosion
1. Erosion is translation invariant ( ) xxA B A BΘ = Θ
( )A B B AΘ ≠ Θ
( ) ( )A B C A B CΘ Θ = Θ ⊕
Dilation and erosion are dual operations in the sense that,
( ) ˆC CA B A BΘ = ⊕
( )ˆ C CA B A B⊕ = Θ
2. Erosion is not commutative
3. Erosion is not associative
![Page 307: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/307.jpg)
Opening and Closing operation
Dilation, and Erosion changes size
By opening and closing operations, irregular boundaries may be smoothened without changing the overall size of the object
Original Dilated
![Page 308: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/308.jpg)
Closing operation
Dilation, followed by Erosion( )A B A B B• = ⊕ Θ
After Dilation operation, the size of the object is increased and it is brought back to the original size by erosion operation
Object AStructuring element B
⇒Closing
By closing operation, irregular boundaries may be smoothened depending upon the structural element
![Page 309: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/309.jpg)
Example
Original Dilated Closed
![Page 310: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/310.jpg)
Opening operation
Erosion, followed by Dilation
( )A B A B B= Θ ⊕o
• Opens the weak links between the nearby objects
• Smoothens the irregular boundary
In all operations, performance depends upon the structuring element and is applied to binary images.
Example: Edge detection
![Page 311: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/311.jpg)
Example
• A={(1,0), (1,1), (1,2),(1,3), (0,3)}
• B={(0.0), (1.0)}
={(1,0), (1,1), (1,2),(1,3), (0,3),(2,0), (2,1), (2,2),(2,3)}A B⊕ =
{(1,3), (0,3)}A BΘ =
We can similarly do other morphological operations
![Page 312: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/312.jpg)
Hit or Miss transform
The main aim of this transform is pattern matching
( ) ( )1 2CA B A B A B∗ = Θ ∩ Θ
2 1B W B= −
( ) ( )1 2ˆ C
A B A B A B∗ = Θ ∩ ⊕
The transform is given by
where
W is the window around the structuring element
![Page 313: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/313.jpg)
Procedure for Hit-or-Miss Transform
Object AStructuring element B1
W
B2=W-B1
After B2 is obtained, find
( ) ( )1 2CA B A B A B∗ = Θ ∩ Θ
CA
![Page 314: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/314.jpg)
ExampleIn a text, how many F’s are present?
Structuring element
or
The structuring element will match both E & F .In such cases, background is also considered for pattern matching
Hit or miss transform will match F only
![Page 315: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/315.jpg)
Example
![Page 316: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/316.jpg)
Applications of binary morphology operations
1. Boundary extraction( ) ( )A A A Bβ = − Θ
![Page 317: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/317.jpg)
Applications of binary morphology operations
2. Region filling
(a)Starts with a pixel inside the unfilled region
(b)Select a proper structuring element B and perform
( )1C
n nX X B A−= ⊕ ∩
Repeat this operation 1n nX X −=
After getting findnA X∪nX to get the filled region
0X
![Page 318: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/318.jpg)
3. Thinning operation
• This gives the image as single pixel width line
Thinningobject
objectskeletonizing
![Page 319: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/319.jpg)
3. Thinning operation
• Thinning and skeletonizing are impact operations
( )A B A A B⊗ = − ∗Thinning operator
![Page 320: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/320.jpg)
Go on matching until exact matching is achievedOriginal image A
1’s 0’s
1B
Now, consider another structuring element 2B
( ) ( )1 21A B A A B A A B B⊗ = − ∗ − − ∗ ∗⎡ ⎤⎣ ⎦
Then, the thinning operation is
In this manner, we can do hit or miss with structuring elements to get ultimately thinned object
![Page 321: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/321.jpg)
The skeleton is given by the operation
0
( ) ( )
( ) ( ) ( )
K
kk
k
S A S A
whereS A A kB A kB B
=
=
= Θ − Θ o
U
Skeletonizing
![Page 322: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/322.jpg)
•.Generalization of binary morphology to gray-level images• Max and Min operations are used in place of OR and AND • Nonlinear operation• The generalization only applies to flat structuring elements. .
Gray-scale morphology
![Page 323: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/323.jpg)
Gray scale morphological operations
( ),f x y
( ),b x y
Object (intensity surface)
Structuring element (will also have an
intensity surface)
Domain of ( ),f x y fD
Domain of ( ),b x y bDDomain of structuring element
Domain of object
![Page 324: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/324.jpg)
Dilation at a point (s,t) of f with b
( )( ) ( ) ( ) ( ){ }, max , , , , , ,f bf b s t f s x t y b x y s x t y D x y D⊕ = − − + − − ∈ ∈
• It can also be written like in convolution ( ) ( ), ,f x y b s x t y+ − −
• Here, mask is rotated by 180 degree and placed over the object;
from overlapping pixels, maximum value is considered.
• Since it is a maxima operation, darker regions become bright
Applications1. Pepper noise can be removed
2. Size of the image is also changed.
![Page 325: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/325.jpg)
Dilation Illustrated
![Page 326: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/326.jpg)
Dilation Result
![Page 327: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/327.jpg)
Erosion operation
( )( ) ( ) ( ) ( ){ }, min , , ,( , ) , ,f bf b s t f s x t y b x y s x t y D x y DΘ = + + − + + ∈ ∈
• It is a minimum operation and hence, bright details
will be reduced.Application: Salt noise is removed
• We can erose and dilate with a flat structuring element
![Page 328: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/328.jpg)
Erosion Result
![Page 329: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/329.jpg)
Closing operation
( )f b f b b• = ⊕ Θ
• Removes pepper noise
• Keeps intensity approximately constant
• Keeps brightness features
![Page 330: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/330.jpg)
Opening operation
( )f b f b b= Θ ⊕o
• Removes salt noise
• Brightness level is maintained
• Dark features are preserved
![Page 331: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/331.jpg)
Opening and Closing illustration
![Page 332: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/332.jpg)
Closing and Opening results
closing opening
Original image
![Page 333: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/333.jpg)
• Gray-scale dilation and erosion are duals with respect to
function complementation and reflection
( ) ( ) ( )( )ˆ, ,c cf b s t f b s tΘ = ⊕
• Gray-scale opening and closing are duals with respect to
function complementation and reflection
( ) ˆc cf b f b• = o
Duality
![Page 334: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/334.jpg)
• Opening followed by closing
• Removes bright and dark artifacts, noise
Smoothing
![Page 335: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/335.jpg)
g f b f b= ⊕ − Θ• Subtract the Eroded image from the dilated image
• Similar to boundary detection in the case of binary image
• Direction Independent
Morphological gradient
![Page 336: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/336.jpg)
Probability and Random processes• Probability and random processes are widely used in image processing. For example
inImage enhancementImage coding Texture Image processingPattern matching
• Two ways of applications:• (a)
The intensity levels can be considered as the values of a discrete random variable with a probability mass function.
• (b) Image intensity as two-dimensional random process
![Page 337: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/337.jpg)
• The following figure shows an image and its histogram.
• We can use this distribution of grey levels to extract meaningful information about the image.
Example: Coding application and segmentation application
0 5 0 1 0 0 1 5 0 2 0 0 2 5 0
0
2 0 0
4 0 0
6 0 0
8 0 0
1 0 0 0
1 2 0 0
1 4 0 0
![Page 338: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/338.jpg)
We may also model the analog image as a continuous random variable
![Page 339: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/339.jpg)
Probability concepts
• Random Experiment: An experiment is a random experiment if its outcome cannot be predicted precisely. One out of a number of outcomes is possible in a random experiment. A single performance of the random experiment is called a trial.
• 2. Sample Space: The sample space is the collection of all possible outcomes of a random experiment. The elements of are called sample points.
• 3. Event: An event A is a subset of the sample space such that probability can be assigned to it.
![Page 340: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/340.jpg)
Probability Definitions
Classical definition of probability)Consider a random experiment with a finite number of outcomes. If all the
outcomes of the experiment are equally likely, the probability of an event is defined by
Where is the number of outcomes favourable to A.Example• A fair die is rolled once. What is the probability of getting a‘6’?• Here and
AN
( ) ANP AN
=
{'1', '2 ', '3', '4 ', '5 ', '6 '}S = { '6 '}.A =
1( ) .6
P A∴ =
![Page 341: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/341.jpg)
Relative Frequency Definition
( ) A
n
nP A Lim
n→∞=
An
Face 1 2 3 4 5 6
Frequency 82 81 88 81 90 78
If an experiment is repeated n times under similar conditions and the event A occurs in times, then
Example: Suppose a die is rolled 500 times. The following table shows the frequency each face.
Then 78 1( )500 6
P A =
![Page 342: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/342.jpg)
Image Intensity as a Random Variable
0p1p 1Lp −
x 0 1 . L-1
.( )XP x
•The probability is estimated from the histogram of the image.
For each gray level ,
[ ] i
ifh iN
=
where is the number of pixels with intensity i.
if
![Page 343: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/343.jpg)
Some Important Properties
( ) ( ) ( ) ( )P AUB P A P B P A B= + − ∩
( ) 0P A B∩ =
Probability that either A or B or both occurs is given by
If A and B are mutually exclusive or disjoint, then
( ) 1P S =
![Page 344: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/344.jpg)
Conditional Probability
• the conditional probability of B given A denoted by is defined byP(B/A)
( )( )
Similarly( )( / )
( )
A B
A
A B
A
NN
NNNN
P A BP A
P A BP A BP B
∩
∩
∩
∩=
P(B/A) =
=
=
![Page 345: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/345.jpg)
Independent events
• Two events are called independent if the probability of occurrence of one event does not affect the probability of occurrence of the other. Thus the events A and B are independent if and only if
• or ( / ) ( )P B A P B=
( / ) ( )and hence
( ) ( ) ( )
P A B P A
P A B P A P B
=
∩ =
![Page 346: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/346.jpg)
A random variable associates the points in the sample space with real numbers. Consider the probability space and function mapping the sample space into the real line.
Random variable
S
Domain of XRange of X
Figure Random variable X
Real line
![Page 347: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/347.jpg)
Probability Distribution FunctionThe probability ({ }) ({ | ( ) , })P X x P s X s x s S≤ = ≤ ∈ is called the probability distribution function (also called the cumulative distribution function abbreviated as CDF) of X and denoted by ( ).XF x Thus ( , ],x−∞ ( ) ({ })XF x P X x= ≤
Properties of Distribution Function • 0 ( ) 1XF x≤ ≤ • )(xFX is a non-decreasing function of .X Thus, 1 2 1 2, then ( ) ( )X Xx x F x F x< < • 0)( =−∞XF
• 1)( =∞XF • 1 2 2 1({ }) ( ) ( )X XP x X x F x F x< ≤ = −
Value of the random variable
Random variable
( )XF x
![Page 348: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/348.jpg)
Example
Suppose { , }S H T= and :X S → is defined by ( ) 1X H = and ( ) 1.X T =−
Therefore, X is a random variable that can take values 1 with
probability 12
and -1 with probability 12
.
SH
T
1
-1
X
o x
( )XF x
x
![Page 349: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/349.jpg)
If )(xFX is differentiable, the probability density function (pdf) of ,X denoted by ( ),Xf x is defined as
)()( xFdxdxf XX =
Properties of the Probability Density Function
• ( ) 0.Xf x ≥
This follows from the fact that ( )XF x is a non-decreasing function
• ( ) ( )x
X XF x f u du−∞
= ∫
• ∫∞
∞−
= 1)( dxxf X
• ∫−
=≤<2
1
)()( 21
x
xX dxxfxXxP
0x
( )Xf x
0 0x x+ Δ x
- 0 0 0 0 0({ }) ( )XP x X x x f x x< ≤ + Δ Δ
Probability Density Function
![Page 350: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/350.jpg)
Example Uniform Random Variable
1
( ) -0
X
a x bf x b a
otherwise
⎧ ≤ ≤⎪= ⎨⎪⎩
Gaussian or Normal Random Variable
2121( )
2
X
X
x
XX
f x eμ
σ
πσ
⎛ ⎞−− ⎜ ⎟
⎝ ⎠=
, x−∞ < < ∞
![Page 351: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/351.jpg)
Functions of a Random variable
Let X be a random variable and (.)g be a function of X . Then ( )Y g X= is a random variable. We are interested to find the pdf of .Y For example, suppose X represents the random voltageinput to a full-wave rectifier. Then the rectifier output Y is given by .Y X= We have to find the probability description of the random variable .Y We consider the casewhen ( )g X is a monotonically increasing or monotonically decreasing function . Here
1 ( )
( ) ( )( )( )
X XY
x g y
f x f xf ydy g xdx
−=
⎤= = ⎥′ ⎦
Example: Probability density function of a linear function of a random variable Suppose , 0.Y aX b a= + >
Then and
( )( )( )X
XY
y b dyx aa dx
y bff x af y dy adx
−= =
−
∴ = =
![Page 352: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/352.jpg)
The expectation operation extracts a few parameters of a random variable and provides asummary description of the random variable in terms of these parameters.
The expected value or mean of a continuous random variable X is defined by
1
( ) for a continuousRV
( ) for a discrete RV
X
X N
i X ii
xf x dxEX
x p xμ
∞
−∞
=
⎧∫⎪⎪= = ⎨
⎪∑⎪⎩
Generally for any function ( )g X of a RV X
( ) ( ) ( )XEY Eg X g x f x dx∞
−∞= = ∫
Particularly,
Mean-square value ∞
−∞= ∫2 2
XE X x f (x )d x
Variance ∞
−∞∫2 2 2
X X X Xσ = E(X - μ ) = (x - μ ) f (x)dx
EXPECTATION AND MOMENTS OF A RANDOM VARIABLE
![Page 353: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/353.jpg)
Multiple Random variablesIn many applications we have to deal with many random variables. For example, the noise affecting the RB channels of colour video may be represented by three random variables. In such situations, it is convento define the vector-valued random variables where each component of the vector is a random variable. Joint CDF of n random variables Consider nrandom variables 1 2 , ,.., nX X X defined on the same sample space. We define the random vectas,
1
2
.
.
n
XX
X
⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥=⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦
X
A particular value of the random vector X is denoted by
1
2= .
n
xx
x
⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦
x
The CDF of the random vector X is defined as follows
( , ,.. ), ,.., 1 21 2 ({ , ,.. })1 1 2 2
F x x x FX X X nnP X x X x X xn n
=
= ≤ ≤ ≤
(x)X
![Page 354: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/354.jpg)
Multiple random variablesIf X is a continuous random vector, that is, ( , ,.. ), ,.. 1 21 2
F x x xX X X nnis continuous in each of
its arguments, then X can be specified by the joint probability density function
( , ,.. ) ( , ,.. ), ,.. 1 2 , ,.. 1 2...1 2 1 21 2
nf f x x x F x x xX X X n X X X nx x xn nn
∂= =
∂ ∂ ∂(x)X
We also define the following important parameters.
The mean vector of ,X denoted by ,Xμ is defined as
1
2
1
2
( )( )
( )
( )
n
n
X
X
X
E XE X
E
E Xμ
μ
μ
⎡ ⎤⎢ ⎥⎢ ⎥= =⎢ ⎥⎢ ⎥⎣ ⎦⎡ ⎤⎢ ⎥⎢ ⎥= ⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦
XμX
![Page 355: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/355.jpg)
Multiple Random Variables
Similarly for each ( , ) 1,2,.., , 1,2,..,i j i n j n= = we can define the covariance
,
( , ) ( )( )
and the ( , )
i j i j
i j
i j
i j i X j X i j X X
i jX X
X X
Cov X X E X X EX X
correlation coefficientCov X X
μ μ μ μ
ρσ σ
= − − = −
=
All the possible covariances and variances can be represented in terms of amatrix called the covariance matrix XC defined by
1 1 2 1
2 1 2 2
1 2
( )( )var( ) cov( , ) cov( , )cov( , ) var( ) . cov( , )
cov( , ) cov( , ) var( )
n
n
n n n
EX X X X XX X X X X
X X X X X
μ μ ′= − −
⎡ ⎤⎢ ⎥⎢ ⎥=⎢ ⎥⎢ ⎥⎣ ⎦
X X XC X X
![Page 356: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/356.jpg)
Multi-dimensional Gaussian
Suppose for any positive integer ,n 1 2, ,....., nX X X represent n jointly random variables.
These random variables are called jointly Gaussian if the random variables 1 2, ,.....,
nX X X
have joint probability density function given by
( )1
1 2
1( ) ' ( )2
, ,....., 1 2( , ,... )2 det( )
X
nnX X X n
ef x x xπ
−− − −
=X XX μ C X μ
XC
where ( )( ) 'E= − −X X X
C X μ X μ is the covariance matrix and
[ ]1 2( ) ( ), ( )...... ( ) 'nE E X E X E X= =Xμ X is the vector formed by the means of the random
variables.
![Page 357: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/357.jpg)
Example 2D Gaussian
Two random variables YX and are called jointly Gaussian if their joint density function is 2 2( ) ( )( ) ( )1
2 2 22 (1 ),2
,
21
, 2 1( , )
x x y yX X Y YXY
X YX Y X Y
X Y X YX Yf x y e
μ μ μ μσ σρ σ σ
ρ
πσ σ ρ
− − − −
−
⎡ ⎤− − +⎢ ⎥
⎣ ⎦
−=
- , - x y∞ < < ∞ ∞ < < ∞
The joint pdf is determined by 5 parameters
• means and X Yμ μ
• variances 2 2
and X Yσ σ
• correlation coefficient , .X Yρ
![Page 358: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/358.jpg)
Karhunen-Loe’ve transform (KLT)
• Let a random vector be given as and characterized by mean and
auto co-variance matrix . • is a positive definite matrix. This matrix can be digitalized as
Where are the eigen values of the matrix and is the matrix formed with the eigen vectors as its columns.
• Consider the transformation . Then, will be a diagonal matrix and the transformation is called the Karhunen
Loe’ve
transform
(KLT).
X
[ ][ ]
[ ]
1
2..
X
X
X N
⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥= ⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦
X xμ
( )( )'x xE μ μ= − −XC X X
XC
1
2
N
0 . .0 . .
= . . . .. 0
T
λλ
λ
⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦
Xφ C φ
1 2, ..... Nλ λ λ XC φ
( )T= − xY φ X μ YC
![Page 359: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/359.jpg)
Recall that a random variable maps each sample point in the sample space to a point in the
real line. A random process maps each sample point to a waveform. A random process is
thus a function of t and s. The random process X(t,s) is usually denoted by X(t).
Discrete time random process X[n].
2( , )X t s 3s
2s 1s
S
1( , )X t s
3( , )X t s
Random Process
![Page 360: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/360.jpg)
The discrete random process is a function of one- dimensional variable n.
• Some important parameters of a random process are:• Mean:
• Variance:
• Auto correlation:
• Auto covariance:
[ ]{ }X n
[ ] [ ] [ ]( ) [ ] [ ]( ),XC n m E X n n X m mμ μ= − −
[ ] [ ] [ ]( ),XR n m E X n X m=
[ ] [ ]2 2( [ ] )n E X n nσ μ= −
[ ] [ ]n EX nμ =
![Page 361: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/361.jpg)
Wide sense stationary (WSS) random process
• For WSS process the Mean
and the autocorrelation function
is a function of the lag only.• We denote the autocorrelation function of a WSS process at lag k by • Autocorrelation function is even symmetric with a maximum at k=0.
[ ] constantXEX n μ= =
[ ], [ ] [ ]XR m n EX m X n=n m−
[ ],X n
[ ]XR k[ ],X n
![Page 362: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/362.jpg)
• Mean vector:
• Autocorrelation matrix:
• Auto covariance matrix:
[ ][ ]
[ ]
1
2..
X
X
X N
⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥= ⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦
X
[ ][ ]
[ ]
[ ][ ]
[ ]
1 1
2 2. .. .
x
x
x
EX
EXE
EX N N
μ
μ
μ
⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥= = =⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎣ ⎦
Xμ X
X E ′=R XX
( )( )′x X XC = E X -μ X - μ
We can represent N samples by an N-dimensional random vector
Check that are symmetric Toeplitz matrixand X XR C
Matrix Representation
![Page 363: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/363.jpg)
Frequency-domain Representation
• A host of tools are available to study a WSS process. • Particularly, we may have the frequency domain representation of a WSS process in
terms of the power spectral density (PSD) given by
• The autocorrelation function is obtained by the inverse transform of as given by
( ) [ ] j kX X
k
S R k e ωω∞
−
=−∞
= ∑( )XS ω
[ ] 1 ( )2
j kX XR k S e d
πω
π
ω ωπ −
= ∫
![Page 364: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/364.jpg)
Gaussian random process
• A random process is called a Gaussian process if for any N, the joint density function is given by,
( )( ) ( )
1( ) ' ( )2
[1], [2],..., [ ] 1 2 31, , ,...,
2 det
where the vecors and the matrices are as interpreted earlier.
X X X N N Nf x x x x eπ
−− −
=-1
X X Xx μ C x μ
xC
[ ]{ }X n
![Page 365: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/365.jpg)
Markov process
• is a random process with discrete state .i.e. can take the one of discrete values with certain probabilities as shown below.State:Probabilities:
• is called first-order Markov if
• Thus for a first-order Markov process, the current state depends on the immediate past.
• Similarly, is called p th-order Markov if
• A first-order Markov process is generally called a Markov process.
{ [ ]}X n [ ]X n L0 , 1 2 1, , . . . . Lx x x x −
0 1 2 1, , ,.... Lx x x x −
0 1 2 1, , ,..., Lp p p p −
{ [ ]}X n[ ] [ ] [ ]{ } [ ] [ ]{ }1 2 1( | 1 , 2 ,... ) ( | 1 )n n n n nP X n x X n x X n x P X n x X n x− − −= − = − = = = − =
{ [ ]}X n
[ ] [ ] [ ]{ }( )[ ] [ ] [ ] [ ]{ }( )
1 2
1 2
| 1 , 2 ,...
| 1 , 2 ,...,
n n n
n n n n p
P X n x X n x X n x
P X n x X n x X n x X n p x
− −
− − −
= − = − =
= = − = − = − =
![Page 366: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/366.jpg)
Random field
• A two dimensional random sequence is called a random field. For a random field we can define the mean and the autocorrelation functions as follows:
• Mean:
• Autocorrelation:
• A random field is called a wide-sense stationary (WSS) or homogeneous random field if
• Thus, for a WSS random field the autocorrelation function can be defined by
{ [ , ]}X m n{ [ , ]},X m n
[ ], [ , ]EX m n m nμ=
[ ], , , [ , ] [ , ]XR m n m n EX m n X m n′ ′ ′ ′=
{ [ , ]}X m n[ ], , , is a function of the lags [ , ].XR m n m n m m n n′ ′ ′ ′− −
{ [ , ]},X m n [ ],XR k l
[ ]
[ ]
, [ , ] [ , ) [ , ] [ , ] ,
X
X
R k l EX m n X m k n lEX m k n l X m nR k l
= + +
= + +
= − −
![Page 367: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/367.jpg)
• A random field is called an isotropic random field if
• A random field is called a separable random field if can be separated as
Two-dimensional power spectral density• We have the frequency domain representation of a WSS random field in terms of the
two-dimensional power spectral density given by
• The autocorrelation function is given by
• A random field is called a Markov random field if the current state at a location depends only on the states of the neighboring locations.
{ [ , ]}X m n
[ ] 2 2, , , is a function of the distance ( ) ( )XR m n m n m m n n′ ′ ′ ′− + −
{ [ , ]}X m n [ ],XR m n[ ] [ ] [ ]
1 2,X X XR m n R m R n=
( )( , ) [ , ] j uk vlX X
l kS u v R k l e
∞ ∞− +
=−∞ =−∞
= ∑ ∑
[ ],fR k l[ ] ( )
2
1, ( , )4
j uk vlX XR k l S u v e dudv
π π
π ππ+
− −
= ∫ ∫
{ [ , ]}f m n
![Page 368: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/368.jpg)
Divide the image into homogenous segments.
Homogeneity may be in terms of(1) Gray values (within a region the gray values don’t vary much )
Ex: gray level of characters is < gray level of background colour (2) Texture : some type of repetitive statistical uniformity(3) Shape(4) Motion (used in video segmentation)
Segmentation
![Page 369: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/369.jpg)
Example
![Page 370: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/370.jpg)
Segmentation based on Colour
Segmentation based on texture
![Page 371: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/371.jpg)
Optical character recognitionIndustrial inspectionRoboticsDetermining the microstructure of biological ,metallurgical specimensRemote sensing Astronomical applicationsMedical image segmentation Object based compression techniques (MPEG 4)Related area in object representation
Application
![Page 372: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/372.jpg)
• Histogram-based segmentation• Region-based segmentation
–Edge detection–Region growing–Region splitting and merging.
• Clustering–K-means–Mean shift
• Motion segmentation
Main approaches
![Page 373: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/373.jpg)
•Problem: we are given an image of a paper, and we would like to extract the text from the image.•Thresholding: define a threshold Th such that
If I(x,y) < ThThen object(character)
Else background (paper)•How do we determine the threshold ?
– Just choose 128 as a threshold (problematic for dark images)
– Use the median/mean (both are not good, as most of the paper is white)
Example: Text and Background
![Page 374: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/374.jpg)
Example
![Page 375: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/375.jpg)
• Assumption– Regions are distinct in terms of gray level range
Gray levelFreq
uen
cy
Histogram-based Threshold
![Page 376: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/376.jpg)
•Compute the gray level histogram of the image.•Find two “clusters”: black and white.•Minimizing the L2 error:
–Select initial estimate T–Segment the image using T.–Compute the average gray level of each segment mb ,mw
–Compute a new threshold value: T = ½ (mb +mw )–Continue until convergence.
•We are already familiar with this algorithm !
Histogram-based threshold
![Page 377: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/377.jpg)
• Noise• Many holes and discontinuities in the segmentation.• Changes in the illumination • We do not use spatial information.Some of the problems can be solved using image processing techniques. For example, we can enhance the result using morphological operations.
Yet – How can we overcome the changes in the illumination ?
Problems with this approach
![Page 378: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/378.jpg)
• Divide the image into sub-images. • Assume that the illumination in each sub-images is constant.• Use a different threshold for each sub-image.• Alternatively – use a running window (and use the threshold of the
window only for the central pixel )
Problems:• Rapid illumination changes.• Regions without text: we can try to recognize that these regions is
unimodal.
Adaptive Thresholding
![Page 379: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/379.jpg)
• We may use probability based approach.• Intensity histogram is a mixture of Gaussian distribution.• Normalized histogram = sum of two Gaussian with different mean and variance.• Identify the Gaussian• Decide the threshold
Optimal Thresholding
![Page 380: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/380.jpg)
We would like to use spatial information.We assume that neighboring pixels tend to belong to the same segment (not always true)
• Edge detection: Looking for the boundaries of the segments.
• Problem: Edges usually do not determine close contours. We can try to do it with edge linking
Region-based segmentation
![Page 381: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/381.jpg)
Basic FormulationLet R represent the entire image region.
Segmentation: Partitioning R into n subgroups Ri s.t:a) b) is a connected regionc)d)e)
P is the partition predicate
Ui
i RR =
iRI φ=ji RR
TrueRP i =)(
FalseRRP ji =)( U
Partition should be such that each region is homogenous as well as connected.
Region-based segmentation
![Page 382: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/382.jpg)
A predicate P has values TRUE or False• The intensity variation within the region is not much. Not
a valid predicate• The intensity difference between two pixels is less than
5.Valid Predicate• The distance between two R, G, B vectors is less than
10.Valid Predicate
Example of a Predicate
![Page 383: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/383.jpg)
• Choose a group of points as initial regions.• Expand the regions to neighboring pixels using a predicate:
– Color distance from the neighbors.– The total error in the region (till a certain threshold):
•Variance•Sum of the differences between neighbors.•Maximal difference from a central pixel.
– In some cases, we can also use structural information: the region size and shape.
• In this way we can handle regions with a smoothly varying gray level or color.
• Question: How do we choose the starting points ? It is less important if we also can merge regions.
Region growing
![Page 384: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/384.jpg)
• One solution of selecting the seed point is to choose the modes of histogram.
• Select pixels corresponding to modes as seed points.
IntensityS1 S2 S3
Frequency
Selection of seed points
![Page 385: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/385.jpg)
FalseRP i =)(TrueRRP ji =)( U
Region merging and splitting
• In region merging, we start with small regions (it can be pixels), and iteratively merge regions which are similar.
• In region splitting, we start with the whole image, and split regions which are not uniform.
• These methods can be combined. Formally:1. Choose a predicate P.2. Split into disjoint regions any region Ri for which3. Merge any adjacent regions Ri and Rj for which 4. Stop when no further merging and splitting is possible.
![Page 386: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/386.jpg)
R1
R3 R4
R21 R22
R23 R24 R1 R3 R4
R21 R22 R24
R2
R23
R
With quadtree, one can use a variation of the split & merge scheme: •Start with splitting regions.•Only at the final stage: merge regions.
QuadTree
R4
![Page 387: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/387.jpg)
• Address the image as a set of points in the n-dimensional space:– Gray level images: p=(x,y,I(x,y)) in R3
– Color images: p =(x,y,R(x,y),G(x,y),B(x,y)) in R5
– Texture: p= (x,y,vector_of_fetures)– Color Histograms: p=(R(x,y),G(x,y),B(x,y)) in R3. we ignore
the spatial information.
• From this stage, we forget the meaning of each coordinate. We deal with arbitrary set of points.
• Therefore, we first need to “normalize” the features (For example - convert a color image to the appropriate linear space representation)
Segmentation as clustering
![Page 388: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/388.jpg)
Given two vectors , we can use measures like• Euclidean distance• Weighted Euclidean distance• Normalized correlation
and i jX X
Similarity Measure
![Page 389: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/389.jpg)
Here, we merge each time the closest neighbors.
Again, we can use splitting & merging
![Page 390: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/390.jpg)
Idea:• Determine the number of clusters• Find the cluster centers and point-cluster correspondences to minimize error
Problem: Exhaustive search is too expensive.Solution: We will use instead an iterative search [Recall the ideal
quantization procedure.]
Algorithm– Fix cluster centers – Allocate points to closest cluster– Fix allocation; compute best cluster centers
K-means
Error function =
1, 2 ,..., kμ μ μ
![Page 391: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/391.jpg)
• Data set (72,180) (65,120) (59,119) (64,150) (65,162) (57,88) (72,175) (44,41) (62,114) (60,110) (56,91) (70,72)
• Intitial Cluster centres (45,50) (75,117) (45,117) (80,180)
• Iteration 1
• (45,50) (44,41) New mean (44,41)
• (75,117) (62,114), (65,120) New mean (63,117)
• (45,117) (57,88),(59,119),(56,91),(60,110)New mean ( 58,102)
• (80,180) (72,180), (64,150),(65,162), (72,175),(70,172)New mean ( 39,170)
Illustration of K-means
![Page 392: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/392.jpg)
Example – clustering with K-means using gray-level and color histograms
![Page 393: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/393.jpg)
• K-means is a powerful and popular method for clustering. However:– It assumes a pre-determined number of clusters– It “likes” compact clusters. Sometimes, we are looking for
long but continues clusters.
• Mean Shift:– Determine a window size (usually small).– For each point p:
• Compute a weighted mean of the shift in the window:
• Set p := p + m• Continue until convergence.
– At the end, use a more standard clustering method.
∑∈
−=windowi
ii ppwm )( rrr ),( ii ppdw =
Mean Shift
![Page 394: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/394.jpg)
This method is based on the assumption that points are more and more dense as we are getting near the
cluster “central mass”.
Mean Shift (contd..)
![Page 395: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/395.jpg)
• Background subtraction:– Assumes the existence of a dominant background.
• Optical flow (use the motion vectors as features)• Multi model motion:
– Divide the image to layers such that in each layer, there exist a parametric motion model.
Motion segmentation
![Page 396: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/396.jpg)
Texture
• Texture may be informally defined as a structure composed of a large number of more or less ordered similar patterns or structures
• Textures provide the idea about the perceived smoothness, coarseness or regularity of the surface.
• Texture has played increasingly important role in diverse application of image processing
– Computer vision– Pattern recognition– Remote sensing– Industrial inspection and – Medical diagnosis.
![Page 397: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/397.jpg)
![Page 398: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/398.jpg)
Texture Image Processing
Texture analysis: how to represent and model texture• Texture synthesis: construct large regions of texture from small example images• Shape from texture: recovering surface orientation orsurface shape from image texture.
![Page 399: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/399.jpg)
• In image processing texture analysis is aimed at two main issues:
– Segmentation of the scene in an image into different homogeneously textured regions without a priori knowing the textures.
– Classification of the textures present in an image into a finite number of known texture classes. A closely related field is the image data retrieval on the basis of texture. Thus a speedy classification can help in browsing images in a database.
• Texture classification methods can be broadly grouped into one of the two approaches: – Non-filtering approach and – Filtering approach.
![Page 400: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/400.jpg)
Co-occurrence Matrix
( , )C i jd
Pd
Cd
Objective: Capture spatial relationsA co-occurrence matrix is a 2D array in which• Both the rows and columns represent a set of possibleimage values• indicates how many times gray value i co-occurs withvalue j in a particular spatial relationship d.• The spatial relationship is specified by a vector d = (dr,dc).From we can compute , the normalized gray-level co-occurrence matrix, where each value is divided by the sum of all the values.
Cd
![Page 401: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/401.jpg)
Example
1 1 0 0 1 1 0 0
1 1 0 0 1 1 0 0
0 0 1 1 0 0 1 1
0 0 1 1 0 0 1 1
1 1 0 0 1 1 0 0
1 1 0 0 1 1 0 0
0 0 1 1 0 0 1 1
0 0 1 1 0 0 1 1
d
d
1 pixel right16 12
C =12 16
16 1256 56P =12 1656 56
=
⎡ ⎤⎢ ⎥⎣ ⎦⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦
d
![Page 402: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/402.jpg)
Measures Extracted from GLCC Matrix
,( , )( )k
i jC i j i j−∑∑ d
,( ( , ))
i jMax C i jd
,
( , ) /( )k
i jC i j i j−∑∑ d
From Co-occurrence matrices extract some quantitative features:
1. the maximum element of C
2. the element difference moment of order k
3. the inverse element difference moment of order k
4. entropy
5. uniformity
,
( , ) log ( , )i j
C i j C i j−∑∑ d d
2 ( , )i j
C i j∑∑ d
![Page 403: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/403.jpg)
Disadvantages
• Computationally expensive• Sensitive to gray scale distortion (co-occurrence matricesdepend on gray values)• May be useful for fine-grain texture. Not suitable forspatially large textures.
![Page 404: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/404.jpg)
Non- Filtering approach
• Structural texture analysis methods that consider texture as a composition of primitive elements arranged according to some placement rule. These primitives are called texels. Extracting the texels from the natural image is a difficult task. Therefore these methods have limited applications.
• Statistical methods that are based on the various joint probabilities of gray values. Co-occurrence matrices estimate the second order statistics by counting the frequencies for all the pairs of gray values and all displacements in the input image. Several texture features can be extracted from the co-occurrence matrices such as uniformity of energy, entropy, maximum probability, contrast, inverse difference moments, and correlation and probability run-lengths.
• Model based methods that include fitting of model like Markov random field, autoregressive, fractal and others. The estimated model parameters are used to segment and classify textures.
![Page 405: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/405.jpg)
Filtering approach
• In the filtering approach, the input image is passed through a linear filter followed by some energy measure.
• Feature vectors are extracted based on these energy outputs. • Texture classification is based on these feature vectors. • The following figure shows the basic filtering approach for texture classification.
• Filtering approach includes Laws mask, ring/wedge filters, dyadic Gabor filter banks, wavelet transforms, quadrature mirror filters, DCT, Eigen filters etc.
![Page 406: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/406.jpg)
Gabor filters
•
Gabor
Filters•
Fourier coefficients depend on the entire image (Global): We losespatial information.•
Objective: Local Spatial Frequency Analysis•
Gabor
kernels: Fourier basis multiplied by a Gaussian–
( )2
00 22
( )1( ) exp ( ) exp22
x xg x i x x iω θσπσ
⎛ ⎞−= − − −⎜ ⎟
⎝ ⎠
The product of a symmetric Gaussian with an oriented sinusoid–
![Page 407: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/407.jpg)
Gabor filter
Gabor filters come in pairs: symmetric and antisymmetric–
Each pair recover symmetric and antisymmetric
components in aparticular direction.–:the spatial frequency to which the filter responds strongly–
σ: the scale of the filter. When σ
= infinity, similar to FT•
We need to apply a number of Gabor filters at different scales,orientations, and spatial frequencies.
![Page 408: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/408.jpg)
Two Dimensional Signals and Systems
• An analog image is modeled as a two-dimensional (2D) signal. • Consider an image plane where a point is denoted by the coordinate
• The intensity at is a two-dimensional function and is denoted by
• The video is modeled as a three-dimensional function
• The digital image is defined over a grid, each grid location being called a pixel.
• We will denote this 2D discrete space signal as
( , ).x y
( , )x y( , ).f x y
( , , ).f x y t
[ , ].f m n
![Page 409: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/409.jpg)
Some of the useful one-dimensional(1 D) functions
1. Dirac Delta or Impulse function ::
• Sifting property
• Scaling property
( ) ( )x u xδ ′=
( ) 0 0
( ) 1
x x
x dx
δ
δ∞
−∞
= ≠
=∫( ) ( ) ( )f x x x dx f xδ
∞
−∞
′ ′ ′− =∫( )( ) xaxa
δδ =
•Dirac Delta function is the derivative of the unit step function
a x -a
12a
![Page 410: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/410.jpg)
2. Kronecker Delta or discrete-time impulse function ::
• Sifting property :
3. Rectangle function ::
0 0[ ]
1 1n
nn
δ≠⎧
= ⎨ =⎩
[ ] [ - ] = [ ]m
mf m n m f nδ
=∞
=−∞∑
11, ( ) 2
0 otherwise
xrect x
⎧ ≤⎪= ⎨⎪⎩
![Page 411: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/411.jpg)
4. Sinc function ::
5. Complex exponential function ::
• These functions are defined in two or multiple dimensions by the seperability property ::
• For example, the complex exponential function is separable.
sinsin ( ) xc xxπ
π=
j xe ω
1 2( , ) ( ) ( )f x y f x f y=( , ) is symmetric if ( , ) ( ) ( )
f x y f x y f x f y=
1 2 1 2( )j x y j x j ye e eω ω ω ω+ =
![Page 412: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/412.jpg)
The two-dimensional delta functions• 2D Dirac delta function ::
• 2D Kronecker Delta function ::
Linear Systems and Shift Invariance
• A system is called linear if :• And the input and output are :
( , ) ( ) ( )x y x yδ δ δ=[ , ] [ ] [ ]m n m nδ δ δ=
1 2 1 2[ [ , ] [ , ]] [ , ] [ , ]T af m n bf m n aTf m n bTf m n+ = +
' '
' '
[ , ] [ ', '] [ ', ']
[ , ] [ , ] [ ', '] [ ', ']m n
m n
f m n f m n m m n n
g m n Tf m n f m n T m m n n
δ
δ
= − −
= = − −
∑∑
∑∑
![Page 413: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/413.jpg)
• Shift invariance ::
•
For a 2-D linear shift invariant system with input and impulse response ,the output is given by
0 0
' '
' '
[ ] [ ][ ] [ ][ , ] [ , ] [ ', '] [ , , ', ']
[ ', '] [ ', ']m n
m n
f n g nf n n g n ng m n Tf m n f m n h m n m n
f m n h m m n n
∑ ∑
∑ ∑
→− → −
= =
= − −
],[ nmf
],[ nmh ],[ nmg
[ , ] [ , ]* [ , ]g m n h m n f m n=
![Page 414: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/414.jpg)
• 2D convolution involves:1. Rotate by to get
2. Shift the origin of to
3. Multiply/overlap elements and sum up.
• 2D convolution can be similarly defined in the continuous domain through
1 1
2 2
Suppose [ , ] is defined for 0,1, . . . . 1 and 0,1, . . . . , 1and [ , ] is defined for 0,1, . . . . , 1 and 0,1, . . . . , 1Then [ , ] is defined for 0,1,
h m n m M n Nf m n m M n N
g m n m
= − = −= − = −= 1 2 1 2. . . . , ( 2) and 0,1, . . . . ,( 2) M M n N N+ − = + −
[ ],h m n′ ′ 180 [ ],h m n′ ′− −
[ ],h m n′ ′− − [ , ]m n
( , )* ( , )
( , ) ( , ) ,
f x y h x y
f x y h x x y y dx dy∞ ∞
−∞ −∞
′ ′ ′ ′ ′ ′= − −∫ ∫
![Page 415: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/415.jpg)
Illustration of convolution of x[m,n] and h[m,n]
![Page 416: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/416.jpg)
![Page 417: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/417.jpg)
Illustration of convolution of x[m,n] and h[m,n] (contd)
[m,n]
![Page 418: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/418.jpg)
Example
[0 0] 0[1 0] 2[2 1] 12
g ,g ,g ,
=== and so on…
![Page 419: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/419.jpg)
Causality
The concept of causality is also extended. Particularly important is the non-symmetrical half-plane (NSHP) model.
For a causal system, present output depends on present and past inputs. Other wise, the system is non-causal.
[m,n]
![Page 420: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/420.jpg)
Lectures on
WAVELET TRANSFORM
![Page 421: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/421.jpg)
OUTLINE
FT, STFT, WS & DWT
Multi-Resolution Analysis (MRA)
Perfect Reconstruction Filter Banks
Filter Bank Implementation of DWT
Extension to 2D Case (Image)
Applications in Denoising, Compression etc.,
![Page 422: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/422.jpg)
Fourier Transform
Fourier analysis -- breaks down a signal into constituent sinusoids of different frequencies. a serious drawback In transforming to the frequency domain, time information is lost. When looking at a Fourier transform of a signal, it is impossible to tell when a particular event took place.
dtetfF tj∫∞
∞−
−= ωω )()(
![Page 423: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/423.jpg)
FT – Sine wave –two frequencies stationary
Peaks corresponding to 5 Hz and 10 Hz
1( ) 0.25sin100 sin 200f t t tπ π= +
![Page 424: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/424.jpg)
FT – Sine wave – two frequencies- nonstationary
Peaks corresponding to 50 Hz and 100 Hz
2sin 200 50
( )0.25sin100 sin 200 otherwise
t tf t
t tπ
π π<⎧
= ⎨ +⎩
![Page 425: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/425.jpg)
The Short Time Fourier Transform Gabor
( , ) ( ) ( ) j tSTFT
tF f t w t e dtωτ ω τ∫
−= −
Time parameter Frequency parameter
Window function centered at τ
![Page 426: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/426.jpg)
FT – Sine wave – two frequencies- nonstationary
![Page 427: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/427.jpg)
Short Time Fourier Transform (STFT)
Take FT of segmented consecutive pieces of a signal.Each FT then provides the spectral content of that time segment onDifficulty is in selecting time window.
NOTELow frequency signal better resolved in frequency domainHigh frequency signal better resolved in time domain
![Page 428: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/428.jpg)
Uncertainty Theorem
Uncertainty Theorem - We cannot calculate frequency and time of a signal with absolute certainty (Similar to Heisenberg’s uncertainty principle involving momentum and velocity of a particle).
In FT we use the basis which has infinite support and infinite energy.
In wavelet transform we have to localize both in time domain (through translation of basis function) and in frequency domain (through scaling).
![Page 429: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/429.jpg)
Example
![Page 430: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/430.jpg)
The Wavelet Transform
Analysis windows of different lengths are used for different frequencies:Analysis of high frequencies Use narrower windows for better time resolutionAnalysis of low frequencies Use wider windows for better frequency resolutionHeisenberg principle still holds!!!The function used to window the signal is called the wavelet
![Page 431: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/431.jpg)
Mother Wavelet
Shannon Wavelet
sin(2 ) sin( )( ) x xxx
π πψπ−
=
In wavelet we have a mother wavelet as the basic unit.
Daubechies Haar
![Page 432: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/432.jpg)
Translation and Scaling
( )f x ( )f x b−
b
Translation( )f ax
1( )
| | | |f ax F
a a
ω↔
⎛ ⎞⎜ ⎟⎝ ⎠
ScalingMother Wavelet
1( ) translated to b and scaled by ,
x bx aa b a a
−Ψ = Ψ →
⎛ ⎞⎜ ⎟⎝ ⎠
![Page 433: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/433.jpg)
Continuous Wavelet Transform
∫∞
∞−Ψ=Ψ= )(),( )( )( ,,, xxfdxxxfW bababa
( ) ∫ ∫∞
∞−
∞
∞−Ψ
Ψ= 2,,.)( 1
adbdaxW
Cxf baba
2( )FC dω
ωω
∞
Ψ − ∞
Ψ= ∫
Where energy is
and ( ) of ( )F FT xωΨ = Ψ
![Page 434: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/434.jpg)
Admissibility Criterion
Requirement is =>
1) term of the mother wavelet must be zero.
2) Ψ(x) should be of finite energy => Ψ(x) should have finite support (asymptotically decaying signal).
3) spectrum should be concentrated in a narrow band.
∞<ΨC
(0) 0F DCΨ = ⇒
2
FΨ ( )dω ω∫ < ∞ ⇒
![Page 435: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/435.jpg)
Wavelet series expansion
,1( )s
ttssττ−⎛ ⎞Ψ = Ψ⎜ ⎟
⎝ ⎠
m mo o owhere scale s s and translation n sτ τ− −= =
If scale and translation take place in discrete steps
/ 2, ( ) ( )m m
m n o o ot s s t nτΨ = Ψ −
Discrete wavelet transform
Dyadic wavelet
/ 2,
If 2 & 1
( ) 2 (2 )o o
m mm n
s
t t n
τ= =
Ψ = Ψ −
![Page 436: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/436.jpg)
can be represented as a wavelet series iff( )f t
22 2,
,( ), ( )m n
m nA f f t w t B f∑≤ ≤
where A and B are positive constants independent of ( )f t
Such a family of discrete wavelet functions is called a frame
When the wavelets form a family of orthogonalbasis functions
,A B=
![Page 437: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/437.jpg)
can be represented as a series combination of wavelets( )f t
, ,( ) ( )m n m nm n
f t w t∑∑= Ψ
, ,
/20
( ), ( )
( ) ( )
m n m n
m mo o
t
W f t w t
s f t s t n dtτ∫
=
= Ψ −
/ 2, ( ) 2 (2 )m m
m n t t nΨ = Ψ −
where
and
Discrete Wavelet transform
![Page 438: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/438.jpg)
Multi resolution analysis (MRA)A scaling function φ(t) is introduced
Eg: Haar scaling function is given by
( ) 1 0 1 0 elsewhere
t tφ = ≤ ≤=
( )tφ
0 1
1
This scaling function is also to be scaled and translated
to generate a family of scaling functions/ 2
, ( ) 2 (2 )j jj k t t kφ φ= −
![Page 439: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/439.jpg)
A function can be generated using the basis set of
translated scaling functions.
( )f t
,( ) ( )k j kk
f t a t kφ∑= −
In the case of Haar basis, comprises of all piecewise
continuous functions.
( )f t
This set of functions is called span of { }, ( ),j k t k Zφ ∈
Let this space be denoted by Vj
![Page 440: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/440.jpg)
Requirements of MRA
Requirement 1
The scaling functions should be orthogonal with respect to their integral translates
, ,
, ,
( ), ( ) 0
( ), ( ) 0 j k j l
j k j lt
t t l k
t t dt l k
φ φ
φ φ∫
= ≠
= ≠
• Haar basis function is orthogonal
![Page 441: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/441.jpg)
Requirement 2
1 1 2... ....
OV V V V−⊂ ⊂ ⊂ ⊂
21
1,0 ( )tφ2
The Haar scaling function at resolution 0
0,0 1,0 1,01 1 1( ) ( ) ( )
22 21 1( ) 2 (2 ) 2 (2 1)2 2
t x t
t t t
φ φ φ
φ φ φ
= + −
∴ = + −
scaling functions at low scale is nested with in the subspace spanned by the higher scale. In general, we can write
basis function at lower scale
basis function at the higher scale
( ) 2 (2 )kk
t h t kφ φ∑= −
Similarly,
( ) 2 (2 )kk
t b t kψ φ∑= −
![Page 442: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/442.jpg)
Requirement 2(contd..)
11 1, 2 2
0 1
o
k
h h
h k
= =
= >
2
1
For example
Haar
wavelet
otherwise
khk
0
...2,1,0 ,22
1
=
==
Triangular wavelet
![Page 443: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/443.jpg)
Requirement 30 01 V ... V V V ... C−∞ ∞⊂ ⊂ ⊂ ⊂ ⊂ ⊂
{ } function zero 0=∞V
Requirement 4All square integrable functions can be represented with arbitrary precision.In particular
V0
V1
V2
001 WVV ⊕= 0W
1W2( )V L R∞ =
2 1 1 0 0 1V V W V W W= ⊕ = ⊕ ⊕and so on. Here denotes the direct sum.⊕
Thus at a scale, the function can be represented by a scale part and a number of wavelet parts.
![Page 444: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/444.jpg)
MRA equation or Dilation equation
( ) 2 (2 )kk
t h t kφ φ∑= −
( ) 2 (2 )kk
t b t kψ φ∑= −
For scaling function,
For the wavelet bases,
![Page 445: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/445.jpg)
MRA and DWT
1
1
0
φ(t)φ1,0 (t)
12
2
( )1,0
1 0, 0, 1 1
1( ) ( ) ( )2
( ) ( ) ( ) ( )k k k kk k
t t t
f t C t d t f t V
φ φ ψ
φ ψ∑ ∑
= +
= + ∈
Scaling part + wavelet part
Low-pass part + high-pass part
Ψ(t)
0
12 1
1
![Page 446: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/446.jpg)
MRA and DWT ( Contd.)
1W
0W
V0
V1
V2
1 12 1, 1, 2 2
0 0 11, 1, 1,
( ) ( ) ( ), ( )
( ) ( ) ( )
k k k kk k
k k k k k kk k k
f t C t d t f t V
C t d t d t
φ ψ
φ ψ ψ
∑ ∑
∑ ∑ ∑
= + ∈
= + +
Similarly,
and so on
How to find those c and d coefficients?
We have to learn a bit of filterbank theory to have an answer.
![Page 447: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/447.jpg)
Perfect Reconstruction bank
[ ]f n
[ ]0
LPFh n
[ ]1
HPFh n
2↓
2↓
[ ]0g n2↑
2↑ [ ]1g n [ ]f̂ n
Analysis filter bank Synthesis filter bank
Note that
2↓
2↑( )X z
1( ) ( ( ) ( ))2
Y z X z X z= + −
![Page 448: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/448.jpg)
On the synthesis side
To avoid aliasing, can be selected by a simple
relationship with
0 1[ ]and [ ]g n g n
[ ] [ ]0 1and .h n h n
0 1 01
0 1 0
( ) ( ), ( ) ( )
[ ] [ ], [ ] ( 1) [ ]o
no
G z H z G z H z
g n h n g n h n+
= = − −
= = −
![Page 449: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/449.jpg)
Orthonormal filters
A class of perfect reconstruction filters needed for the filter
bank implementation of discrete wavelet transform (DWT)
1 0 [ ] ( 1) [ 1 ]nh n h N n≡ − − −
These filters satisfy the relation
where is the tap length required to be evenN
The synthesis filters are given by
[ ] [ ] {0,1}i ig n h n i= − ∈
![Page 450: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/450.jpg)
Orthonormal filter banks
[ ]f n
0[ ]h n
1[ ]h n
2↓
2↓
[ ]0h n−2↑
2↑ [ ]1h n− [ ]f̂ n
Analysis filter bank Synthesis filter bank
is the original signal
is the reconstructed signal
[ ]f n
[ ]f̂ n
![Page 451: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/451.jpg)
Filter bank implementation of DWT
The MRA equation (or) dilation equation for the scaling function is
( ) [ ] 2 (2 )k
t h k t kφφ φ∑= −
By slightly modifying the notations, we have
, , , ,
( ) [ ] 2 (2 )
( ) [ ] 2 (2 )
( ) ( ) ( )
k
k
j k j k j k j kk k
t h k t k
t h k t k
f t c t d t
φφ φ
φ
Ψ
= −
Ψ = Ψ −
= + Ψ
∑
∑
∑ ∑
, ,( ), ( )j k j kt tφ Ψ are orthogonal individually and to each
otherFurther, orthonormality is assumed
![Page 452: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/452.jpg)
,j kcThe co-efficient can be given as the convolution of
1, and [ ]j kc h kφ+ − at alternate points
, 1,[ 2 ] j k j ll
c h l k cφ += −∑
, 1,[ 2 ]j k j kl
d h l k CΨ += −∑Similarly
is an anti-causal filter, but it is not a serious issue
and can be addressed by proper translation of the function.
[ ]h kφ −
Contd…Using the orthogonality
of the scaling and the wavelet bases
[ ]h kφ − 2↓ 1,j kC +
,j kC
![Page 453: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/453.jpg)
How to get the filter co-efficients?Integrating the dilation equation on both sides
( ) 2 [ ] (2 ) 2 -
1 [ ] ( )2
[ ] 2 ------ (1)
k
k
k
t dt h k t k dt t k u
h k u du
h k
φ
φ
φ
φ φ
φ
∞ ∞
−∞ −∞
∞
−∞
= − =
=
⇒ =
∑∫ ∫
∑ ∫
∑
2
2
( ) 1
[ ] 1 (2)k
x dx
h kφ
φ∞
−∞
=
⇒ = − − − − −
∫
∑
Similarly
![Page 454: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/454.jpg)
Due to the orthogonality of the scaling function and its integer
translates, we have
( ) ( ) [ ]x x m dx mφ φ δ∞
−∞
− =∫
[ ] [ 2 ] [ ]k
h k h k m mφ φ δ− =∑ ------- (3)
If dilation equation is applied, we get
Considering the orthogonality of scaling function and wavelet
functions at a particular scale
[ ] ( 1) [ ]kh k h N kφΨ = − −
Hence, form perfect reconstruction
orthonormal
filter banks
[ ] and [ ]h k h kφ Ψ
![Page 455: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/455.jpg)
Filter bank representation for 2-tap filter
2 2
[0] [1] 2
& [0] [1] 1
h h
h hφ φ
φ φ
+ =
+ =
The unique solution for these equations s
0
1
1[0] [1] 2
1[0] ( 1) [1]21[1] ( 1) [0]2
h h
h h
h h
φ φ
φ
φ
Ψ
Ψ
= =
= − =
= − = −
There will be no filter with odd tap length
The equations representing the filter co-efficients are
![Page 456: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/456.jpg)
4-tap wavelet (Daubachies wavelet )3
0
32
0
[ ] 2
[ ] 1
[0] [2] [1] [3] 0
k
k
h k
h k
h h h h
φ
φ
φ φ φ φ
=
=
=
=
+ =
∑
∑
For a 4-tap filter,
With these 3 equations, Daubachies
wavelets can be generated
1 3 3 3[0] [1]4 2 4 2
3 3 1 3[2] [3]4 2 4 2
h h
h h
φ φ
φ φ
+ += =
+ −= =
![Page 457: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/457.jpg)
Where to start and where to stop?
The raw data is considered as a scaled version of data
at infinite resolution
V0
V1
V2
0W
1W
![Page 458: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/458.jpg)
The process of approximation from highest resolution can be
explained as shown by the following figure
Lowest resolution
Detail 2
Highest resolution
Detail 1[ ]f k
[ ]h kφ −
[ ]h kΨ −
2↓
2↓
[ ]h kφ −
[ ]h kΨ −
2↓
2↓
Approximate
![Page 459: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/459.jpg)
Reconstruction
Synthesis filter banks can be applied to get back the original signal
Approximation
Reconstructed
signal
+
+Detail - 1
Detail - 2
2
2
2
Processing
Processing
Processing
[ ]g kφ
[ ]g kψ
[ ]g kψ
[ ]g kφ2
![Page 460: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/460.jpg)
2D Case
For 2 dimensional case, separability property enables the use
of 1D filters
1 2 1 1 2 2( , ) ( ) ( )t t t tΨ = Ψ Ψ
The corresponding filters can be first applied in first dimension
and then in other dimension.
First, the LPF and HPF operations are done row wise and then
column wise. This can be explained with the following figure
![Page 461: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/461.jpg)
[ ],f m n
LP 2↓row
HP 2↓row
LP
HP
2↓
2↓
column
column
LL-scaling co-eff
LH-scaling co-eff
LP
HP
2↓
2↓
column
column
HL-scaling co-eff
HH-scaling co-eff
LL LH
HL HH
Original image
![Page 462: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/462.jpg)
Co-efficient of diagonal detail at level 1
Co-efficient of vertical detail at level 1
Co-efficient of horizontal detail at level 1
Co-efficient of approximation at level 1
![Page 463: Dig Image Processin](https://reader031.vdocument.in/reader031/viewer/2022022215/544e0f3aaf7959f0178b4fdc/html5/thumbnails/463.jpg)
Decomposition at level 1 Decomposition at level 2
Original image