essay on principles of image processing and analysis-s
Post on 06-Apr-2018
214 Views
Preview:
TRANSCRIPT
-
8/3/2019 Essay on Principles of Image Processing and Analysis-s
1/18
PRINCIPLES OF IMAGE PROCESSING AND ANALYSIS IN ROBOTICSBy Vinay KumarThe University of Texas, Austin
Introduction
Image processing is a vast field of science dedicated to the study of Image
representation, manipulation and reconstruction and is applicable to various fields
like Biomedical Imaging, Robotics, Remote sensing, Defense Surveillance and many
more. This essay explores the various important concepts underlying the Image
processing realm and accompanies relevant examples for some concepts. All the
example implementations are performed with MATLAB programming language.
The essay details the concepts of image processing with different chapters. Chapter 1
introduces the concepts related to image formation and representation.
Chapter 2 discusses the various Image processing techniques currently being used
and Chapter 3 discusses some Specific Applications. A final summary and References
are provided at the end of the chapters.
Chapter 1Image Representation
Image processingis defined as methods and techniques employed to prepare an image
for analysis at a later point of time. The techniques applied generally include noise
reduction, enhancement, simplification and filtering.
Image Analysis is defined as the methods and techniques employed to analyze the
saved and processed image in order to extract any relevant information. The examples
various techniques used are facial recognition, pattern recognition.
What is an Image??
An image is a representation of a real scene which can have different attributes
based on the image acquisition tool used.
Any image can be broadly classified in the following categories:
1. Dimension: Based on dimensions images can be 2-dimensional images which lack
the depth information of a scene and are used mostly for feature extraction, navigation
etc or3-dimensional images which do contain the depth information and are majorly
used in motion detection, scene recreation and medical imaging.
2. Appearance: Color or Black and White
Black and White models used normally are Grayscale where multiple gray inks are
used to reproduce image or One Ink where only one color Black is used and by
changing the size of the black dot different gray levels are reproduced.
-
8/3/2019 Essay on Principles of Image Processing and Analysis-s
2/18
Color models used are RGB where the image is reproduced by the mixture of the three
color dots red, green and blue or the CMYK model where the combination of cyan,
magenta, yellow and black colors is used to reproduce the image.
3. Form: Printed form or Digital form.
With the advent of computers the digital form of image has increasingly become
important. It is important to discuss in detail the terms related to the digital images as
they form the basis for the various image processing techniques discussed in the later
part of essay.
Digital Images
Every image either digital or printed are made up of equal sized basic elements which
are calledpicture cells(also sometimes called picture elements) orabbreviated aspixels
.In order to capture an image intensity of each pixel is measured and recorded and
similarly in order to recreate an image the intensity of light at each pixel location is
varied.
Pixels are indexed as (x, y) or column-row (c, r) location from the origin of image and
contain a numerical value which is the basic unit of information within the given
image at a particular resolution and quantization level (to be discussed later).Hence an
image is a collection of the data representing the numerical values of light intensities
of a large number of pixels. A black and white image consists of pixels containing
different intensities of gray color and on the other hand color image consists of pixels
containing different intensities of the three colors Red Green and Blue.
A digital image is a digitized version of a real image and stores the information as a
collection of 0s or 1s which represent the intensity of light at each pixel. Hence a
digital image can be read by computers and thus can be manipulated or rewritten in
different form for the further study. If a system uses a digitization with 8 bits , the
image generated uses 28 =256 distinct light intensities. A good example of pixel details
is given below pictorially.
The picture below describes how the pixels are represented with numbers pertaining
to the different light intensities possible depending on the number of bits used for the
digitization. The header at the beginning of every image file indicates the number of
pixels in each row and column, thus the program knows how many bits correspond to
how many pixels.
As seen below we can see how an image is divided into pixels and how the digitized
version of an image is stored in the computers.
-
8/3/2019 Essay on Principles of Image Processing and Analysis-s
3/18
Figure 1: Image in its original
Figure 2: Image in its Pixellated form
Figure 3: Image
in its further
pixellated form
Figure 4: Image showing the RGB pixel values for a part of the original image
-
8/3/2019 Essay on Principles of Image Processing and Analysis-s
4/18
Digital Image Data types:
Binary images-These image types assign one numerical value from the set (0, 1) to
each pixel in the image. The black pixel corresponds to a zero or off and white
corresponds to a one or on in these images. Thus an image is a stream of 0s and 1s
and normally grouped as 8 bits.
Gray Scale Images- These images assign the numerical value to each pixel which
corresponds to the intensity level for that point. The range of values assigned dependson the bit resolution used in the image.
RGB Images- These images assign three numerical values to each pixel corresponding
to the red, green and blue intensity value respectively. The example in Figures 2 -5
shows how an image is assigned pixel for the RGB type image.
Figure 5: Image further showing the RGB pixel values of a part of the original image.
-
8/3/2019 Essay on Principles of Image Processing and Analysis-s
5/18
Mathematics of Image analysis
In this section we briefly discus about the various mathematical principles which are
used in the image processing field.
Image processing analysis is normally done in 2 different domains
Frequency Domain: In frequency domain the frequency spectrum of whole image is
calculated and further altered or processed to analyze the image. This method does
not break down the image into its pixel constituent for analysis and rather takes the
image as a whole.
Spatial Domain:On the contrary in spatial domain the analysis is done on pixels and
all alterations and processing is done at pixel level.
Although the above mentioned two methods seem different but an alteration in one
also affects the analysis done in the other domain. For example if a spatial filter is
applied to reduce noise then it also affects the frequency spectrum.
Fourier Transformation: As discussed earlier, all digital images are represented by
numerical values depending on the method used, every image can be considered as a
signal which can be analyzed further with techniques used for real signals. One of the
techniques frequently used in signals and systems is the Fourier Transform which
states that Any periodic signal can be decomposed into a collection of Sines and
Cosines of different amplitude and frequencies. Thus mathematically any signal f(t)
can be written as
On the other hand if we add all the Sines and Cosines together which constitute asignal, we can reconstruct the signal back. The above representation of breaking asignal into different sine and cosine frequencies is called the Fourier series and thecollection of all the frequencies present is called the Frequency spectrumof the signal.
The example below demonstrates the reconstruction of a triangular wave by addingSine waves of different frequencies. Also we can see the frequency spectrums of eachwave and the final triangular wave generated has all the frequencies that we used togenerate.
Figure 6: Time domain plot of a Sine wave
-
8/3/2019 Essay on Principles of Image Processing and Analysis-s
6/18
Figure 7: Frequency domain plot of a Sine a wave at f=4
Fi ure 8: Time domain lot of a summation of two Sine waves
Figure 9: Frequency domain plot for summation of two Sine waves
-
8/3/2019 Essay on Principles of Image Processing and Analysis-s
7/18
Figure 10: Time domain plot for summation of three Sine waves
Figure 11: Frequency domain plot for summation of three Sine waves
Figure 12: Time domain plot for summation of ten Sine waves
-
8/3/2019 Essay on Principles of Image Processing and Analysis-s
8/18
An important result to be analyzed from the above example is that the more a signal
has sharp changes in it like square wave, impulse etc the more number of frequencies
required to reconstruct it.
Taking forward the above result the Noise and Edge detection basic principle can be
easily understood. Referring to the Figure 3 below it can be seen that the intensity of
the pixels at the edge of the image is quite different from its surrounding pixels. Thus
there is a sudden change in the signal representing that portion, and applying the
result of Fourier series transform above we can be sure that the frequency spectrum
of the image will have higher frequencies for the edges and noises since they are
rapidly changing signals requiring higher sines and cosines for representation.
Further passing the signal from a low pass filter will severely attenuate the amplitude
of higher frequencies and thus help in reducing the noise but also will blur the imageby attenuating the high frequencies corresponding to the edges.
Image quality measurement metrics
The quality of an image is quantified and defined by two important terms which are
Resolution and Quantization.
Resolution- Resolution measures the size of an image .For still images it is specified
as a spatial resolution and is given as Column(C) byRow(R), where the values C and R
refer to the number of pixels used to cover the space representing the image. The
resolution for analog signal is a function of the sampling rate and for a digital signal isthe function of number of pixels present. Thus the resolution of an image decreases if
it is sampled less frequently.
The resolution for a video is defined as Temporal resolution and is defined as the
number of images captured in a given period of time also called as frames per second
(fps)
Figure 13: Frequency domain plot for summation of ten Sine waves
-
8/3/2019 Essay on Principles of Image Processing and Analysis-s
9/18
Figure 14: Different Resolutions of an image( Image Courtesy: www.cmu.edu)
Original Image 512 x 512 resolution Image with 256 x 256 resolution
Image with128 x128 resolution Image with 64 x 64 resolution
Image with 32 x 32 resolution
Quantization-The concept of Quantization is key to the process of digitization of
images. It is defined as the mapping of a continuous signal representing a scene to a
discrete number corresponding to the intensity of each pixel constituting the image.
-
8/3/2019 Essay on Principles of Image Processing and Analysis-s
10/18
In the process of converting Analog image from a Capture device (camera) to a Digital
signal , the voltage read at the sensor which corresponds to the wavelength of the
light entering the sensor after reflection from the object is divided into different levels
with each level representing the specific intensity level .Depending on the hardware
used these possible levels are given as 2nwhere n is the number of bits . Thus a 8-
bit sensor divides the intensity into 28 =256 levels with 0 representing blackand 255 representing the white.
Quantization and Resolution are independent of each other, as it is possible to have a
high resolution image which can have just two intensity levels of 0 1nd 1 representing
dark or light and can also be quantized into 8 bits yielding a range of 0-255 intensity
levels. The requirement for particular values of Quantization or Resolution depends on
the use of the image for a given application.
An important concept closely related to Quantization is that of Sampling Theorem also
called the Nyquist Sampling Theorem. When a digitized signal is reconstructed the
sampling theorem plays an important role. It is concerned with the number of samples
needed to recreate the original signal. It can be restated in relation to image
processing as An analogue image can be reconstructed exactly from its digitized form
as long as the sampling frequency is at least twice the highest frequency present in the
image. The signal reconstructed with sampling frequency less than the Nyquest
frequency (the optimum frequency) suffers from a phenomenon called Aliasing.
As mentioned earlier we can measure the frequencies present in an image by checking
the frequency spectrum and by adhering to the proper sampling frequency can
reconstruct the image with adequate resolution. It is a common practice to keep the
sampling frequency equal to 4-5 times larger than the highest frequency of the signal.If an image is reconstructed with a low sampling rate it will lack the high frequencies
present in the original image and thus the important details of the image are lost.
Chapter 2Image Processing Techniques
Once the image of interest is stored in the system the by applying quantization and
digitization, they need to be processed before we can use the Image Analysis routines.
The purpose of Image processing techniques is to remove faults, trivial information
and any shortcomings introduced during the image acquisition. Common faults
include blurred image and noisy images. Major image processing techniques include
Histogram analysis, thresholding, edge detection, segmentation and masking. We willbe briefly discussing each technique.
Histogram Analysis: An Image histogram is a plot of the relative frequency of
occurrence of each of the pixel at gray level plotted against the values. In other words
it gives the number of times each gray level occurs in the image.
-
8/3/2019 Essay on Principles of Image Processing and Analysis-s
11/18
Histogram gives a visual revelation of the basic contrast that is present in the image
and we can identify any differences in the pixel distribution of the image foreground
and background.
Histogram analysis is frequently used to improve the contrast of a poorly contrasted
image. Two techniques are commonly employed for correction:
1. Contrast Stretching or Normalization where in a piecewise linear function isused to transform the gray levels to different values by stretching their value
according to the linear function, thus transformed image has a better contrast
than one without Normalization.
2. Histogram Equalization is a procedure where in no image correction is donewithout any user input and the gray pixel level stretching is done.
Add histogram stretching example
Thresholding: Thresholding is one of the important techniques in Image
processing where an image is divided into different levels and assigning each pixel
to those levels by comparing each pixels grayness value to a threshold.
Thresholding can be either performed as a single level by choosing a singlethreshold or multiple thresholding by choosing multiple levels.
A simple example ofsingle thresholdingcan be as below:
Turn a gray scale image to a binary image (black & white) image by choosing a gray
level Lin the original image and turn each pixel to black or white by checking if its
gray value is greater than or less than Li.e.
Figure 15: A low contrast image and its
corresponding Histogram
Figure 16: A well contrasted image and its
corresponding Histogram by applying
Histogram stretching
-
8/3/2019 Essay on Principles of Image Processing and Analysis-s
12/18
White if its gray level > L.
Convert a pixel to Black if its gray level < L.
A simple example ofDouble thresholdingcan be as below:
Choose two values L1 AND L2 for Thresholding
White if its gray level > L1and < L2.
Convert a pixel to Black if its gray level IS not between two thresholds.
Thresholding technique is an important part of the Image Segmentation
technique where it is used to isolate objects from its background and also finds
much application in Robot Vision applications.
THRESHOLD EXAMPLE WILL GO HERE
Convolution mask: Convolution mask is one of the filtering techniques used inImage processing .It is also sometimes called as Neighborhood Processing .As the
name suggests convolution is applied to the image of interest The idea is to move a
mask : a rectangle (odd sized) over the given image. The mask is first placed on
the upper left corner of the image and the summation of the product of value of
each pixel multiplied by the mask values is calculated. Further this summation is
divided by constant normalizing value. If the summation calculates to a zero value
then it is replaced by 1 or by the largest number. A new copy of the image is
generated and the resulting number obtained after the normalization process
discussed above is substituted in the center of the block that was superimposed by
the mask. The whole process is repeated by moving the mask to the right andagain replacing the center value with calculated normalized value. The operation is
continued till all the rows of the image are affected by this operation. Figure 18
depicts the idea of Convolution mask procedure pictorially.
Figure 17: An example for Thresholding applied to an image
-
8/3/2019 Essay on Principles of Image Processing and Analysis-s
13/18
An important point to be noted here is that the first and the last rows and columns
are never affected by this Convolution procedure and are either ignored or replaced
by zeroes.
One of the applications of Convolution mask filtering technique is to us it for
Noise Reduction in the image. Specifically the technique is called Neighborhood
Averaging , where in a mask is used to reduce the gray value of a pixel which is
totally different from its neighbors ( and hence called noise).The mask does not
affect the values which are in gray values equal or near to their neighborhood
pixels. Thus the method in a sense works as a low-pass filter by attenuating the
sharp differences between the neighborhood pixels and by not affecting the pixels
whose intensities are similar.
Since this method introduces new gray levels in the image, thus it affects theHistogram of the image and also reduces the sharpness of the image .An
alternative method to overcome this shortcoming is a method called Median
Filtering which uses the median value of the pixel value to replace the center
value rather than calculating the convolution.
Figure 18: An example for Applying Convolution Courtesy: S.Niku
-
8/3/2019 Essay on Principles of Image Processing and Analysis-s
14/18
Edge Detection:
One of the most important uses of Noise filtering in Image processing is to facilitate
in the Image analysis by helping in Feature Detection and Feature Extraction.
Edge detection is one of the most important techniques used in Image Processing.
Edges can be considered as a discontinuity or gradient in the pixel values which
exceeds a given threshold value. An edge represents an observable difference in the
pixel values .Considering the tables below we can clearly observe that there is a
clear difference in gray values of right hand figure for columns 3 & 4 and
Indicating the presence of Edge.
The techniques used in Edge detection operate in the image and result in a line
drawing of the image. These lines can represent the changes in the values such as
cross-sections of planes, textures, or difference in light intensities between parts
and backgrounds. The principle behind these techniques is to operate on the
difference between the gray levels of pixels or groups of pixels through the use of
Convolution Masks (discussed in previous section) . The final representation after
applying the techniques takes lesser memory and saves in computational and
storage costs apart from helping in Object recognition and Segmentation.
If we plot the gray values we traverse the image from left to right we can get a
profile of the image. The edge profile can be a ramp edgewhere the gray values
change slowly or a step edgewhere the gray values change suddenly.
Referring to Figure 19 we can see that an ideal edge can be thought of a clear
distinction between the pixel vales, which is only possible in Binary images. The
real edge shown in the image is more prevalent in other image types and thus just
a comparison of the pixel values is inadequate for edge detection. Practically, the
first and the second derivative of the graph are employed for edge detection, where
in we can clearly see that the Edge can be detected at the peak of the firstderivative and the zero crossing of the second derivative.
45 46 102 108
44 45 115 112
42 46 120 134
45 46 47 44
44 45 43 42
42 46 44 43
-
8/3/2019 Essay on Principles of Image Processing and Analysis-s
15/18
Derivatives
Suppose we have a plot of the function f(x) for the profile of an image. We candefinitely plot the derivative f (x)of the image. The derivative returns zero for all
the constant sections of the profile, and returns non-zero values only in the part of
image in which there is difference. For an image with two dimensions using partial
derivatives we define the gradient andlaplacianas
respectively. The above gradient points in the direction of greatest increase for the
function f(x, y). The direction of increase is given by
and the magnitude is given by Most edge detection methods find the magnitude of the gradient and then apply
threshold to the result.
First order derivatives
As we know the definition of the Derivative is given as
Figure 19: An example for Edge Detection using Derivatives Courtesy: S.Niku
-
8/3/2019 Essay on Principles of Image Processing and Analysis-s
16/18
Since in an image the smallest value for h is 1 being the difference between the
index values of two adjacent pixels the digital implementation of the above
definition is given as
Other definition of Derivative is given as
and The digital version comes out to be as
and Using the above expressions for derivatives and leaving out the scaling factors
horizontal and vertical filters can be taken as
and The use of above filters finds the vertical and horizontal edges in an image. In
order to provide a smoothing effect a combined filter is used which is given as
Prewitt filter for Vertical Edge Prewitt filter for Horizontal Edge
detection detection
Second order derivatives
As we have used the first order derivatives for edge detection, Second order derivatives
use is also common. As mentioned earlier, the sum of second derivatives in both the
direction is called the Laplacian. Defined by
It is implemented by the filter as
-
8/3/2019 Essay on Principles of Image Processing and Analysis-s
17/18
The second derivative is also called Discrete Laplacian. The laplacian has a property of
Rotation Invariant. That is , if the Laplacian is applied to an image ,and the image isrotated , we obtain same result when we rotate the image and then apply the
Laplacian . The major problems with Laplacian filters are that they are sensitive to
noise.
Add an example of edge detection
Segmentation:
Segmentation is the name given to generic techniques by which an image is
subdivided into its constituent regions or objects. The main purpose of these
techniques is to separate the information contained in the images into mutually
exclusive regions which can be later used for other purposes.
Segmentation occupies a vital role in image processing as it is the first important step
that must be taken before the other tasks such as Feature extraction or classification
are to be performed. All segmentation techniques are based on the following basic
approaches:
1. Edge Methods: This approach detects the edges as a means to identify theboundary between regions by finding the sharp differences in the pixel
intensities.
2.
Region Methods: This approach assigns pixels to different regions based on apredefined selected criterion.
As we have already discussed the Edge Detection method which is also a part of
Segmentation technique ,we will now discuss some more Region methods used in
segmentation.
Region growing and Splitting:Region growing is an approach of segmentation in which
pixels are grouped into larger regions based on predefined similarity criteria. The
process starts by selecting a number of seed pixels (also called Nucleipixels)which
are randomly distributed over the image and appending pixels in the neighborhood
region to the same region if they satisfy the similarity criterion of intensity, color orother properties. The nuclei regions act as nucleus for subsequent growing and
merging. The small regions thus formed are combined into larger regions to create the
final segmented regions.
The region splittingtechnique employs the similar philosophy as the region growing
but it is the reverse approach. The method starts by treating the whole image as a
single region which is then successively broken down into smaller and smaller regions
-
8/3/2019 Essay on Principles of Image Processing and Analysis-s
18/18
until any further subdivision results in difference between adjacent regions falling
below some threshold value. A most used example for region splitting is using the
Split-and-merge technique.
Chapter 3Image Analyzing Techniques
Image analysis is a collection of techniques used to extract information from the stored
images which have been already processed with image processing techniques. The
techniques include object recognition, feature extraction and extraction of depth
information.
Object Recognition: An object may be recognized by its features. The features
normally include Gray levels, Morphological features like area, perimeter, and
moments. Let us discuss some more about the various features.
a. Gray Levels: The different parts or objects in an image can be identified bychecking for average, maximum or minimum gray levels. For example, there
may be three parts in an object and each with different color or texture. If
average, maximum and minimum gray levels of the object are found, the objects
can be recognized by comparison of these values.
b. Morphological Features: The different morphological features are perimeter,area, diameter etc. The perimeter of the object may be found by applying edge
detection routine and then counting number of pixels on the perimeter .Area
can be calculated by region growing techniques discussed earlier in chapter 2 .
c. Aspect ratio: Aspect ratio is defined as the width to length ratio of an enclosingrectangle about t he object.
Summary
In this essay we have discussed the basics of Image processing and Analysis methods
currently being used. Different examples have been used to make the theory much
more clear and the examples have been implemented with MATLAB. The image
processing techniques are an essential part in the Robotics domain and a successful
implementation leads to building of perfect robotics systems.
References
1. MATHWORKS, MATLAB Documentation Manual.2. Introduction to Robotics, Analysis control & Applications by Syed B.Niku 2nd
edition.
3. Fundamentals of Digital Image Processing by C.Solomon & T. Breckon.4. Digital Image processing using MATLAB by Gonzalez ,Woods & Eddins
top related