essay on principles of image processing and analysis-s

8/3/2019 Essay on Principles of Image Processing and Analysis-s

1/18

PRINCIPLES OF IMAGE PROCESSING AND ANALYSIS IN ROBOTICSBy Vinay KumarThe University of Texas, Austin

Introduction

Image processing is a vast field of science dedicated to the study of Image

representation, manipulation and reconstruction and is applicable to various fields

like Biomedical Imaging, Robotics, Remote sensing, Defense Surveillance and many

more. This essay explores the various important concepts underlying the Image

processing realm and accompanies relevant examples for some concepts. All the

example implementations are performed with MATLAB programming language.

The essay details the concepts of image processing with different chapters. Chapter 1

introduces the concepts related to image formation and representation.

Chapter 2 discusses the various Image processing techniques currently being used

and Chapter 3 discusses some Specific Applications. A final summary and References

are provided at the end of the chapters.

Chapter 1Image Representation

Image processingis defined as methods and techniques employed to prepare an image

for analysis at a later point of time. The techniques applied generally include noise

reduction, enhancement, simplification and filtering.

Image Analysis is defined as the methods and techniques employed to analyze the

saved and processed image in order to extract any relevant information. The examples

various techniques used are facial recognition, pattern recognition.

What is an Image??

An image is a representation of a real scene which can have different attributes

based on the image acquisition tool used.

Any image can be broadly classified in the following categories:

1. Dimension: Based on dimensions images can be 2-dimensional images which lack

the depth information of a scene and are used mostly for feature extraction, navigation

etc or3-dimensional images which do contain the depth information and are majorly

used in motion detection, scene recreation and medical imaging.

2. Appearance: Color or Black and White

Black and White models used normally are Grayscale where multiple gray inks are

used to reproduce image or One Ink where only one color Black is used and by

changing the size of the black dot different gray levels are reproduced.


2/18

Color models used are RGB where the image is reproduced by the mixture of the three

color dots red, green and blue or the CMYK model where the combination of cyan,

magenta, yellow and black colors is used to reproduce the image.

3. Form: Printed form or Digital form.

With the advent of computers the digital form of image has increasingly become

important. It is important to discuss in detail the terms related to the digital images as

they form the basis for the various image processing techniques discussed in the later

part of essay.

Digital Images

Every image either digital or printed are made up of equal sized basic elements which

are calledpicture cells(also sometimes called picture elements) orabbreviated aspixels

.In order to capture an image intensity of each pixel is measured and recorded and

similarly in order to recreate an image the intensity of light at each pixel location is

varied.

Pixels are indexed as (x, y) or column-row (c, r) location from the origin of image and

contain a numerical value which is the basic unit of information within the given

image at a particular resolution and quantization level (to be discussed later).Hence an

image is a collection of the data representing the numerical values of light intensities

of a large number of pixels. A black and white image consists of pixels containing

different intensities of gray color and on the other hand color image consists of pixels

containing different intensities of the three colors Red Green and Blue.

A digital image is a digitized version of a real image and stores the information as a

collection of 0s or 1s which represent the intensity of light at each pixel. Hence a

digital image can be read by computers and thus can be manipulated or rewritten in

different form for the further study. If a system uses a digitization with 8 bits , the

image generated uses 28 =256 distinct light intensities. A good example of pixel details

is given below pictorially.

The picture below describes how the pixels are represented with numbers pertaining

to the different light intensities possible depending on the number of bits used for the

digitization. The header at the beginning of every image file indicates the number of

pixels in each row and column, thus the program knows how many bits correspond to

how many pixels.

As seen below we can see how an image is divided into pixels and how the digitized

version of an image is stored in the computers.


3/18

Figure 1: Image in its original

Figure 2: Image in its Pixellated form

Figure 3: Image

in its further

pixellated form

Figure 4: Image showing the RGB pixel values for a part of the original image


4/18

Digital Image Data types:

Binary images-These image types assign one numerical value from the set (0, 1) to

each pixel in the image. The black pixel corresponds to a zero or off and white

corresponds to a one or on in these images. Thus an image is a stream of 0s and 1s

and normally grouped as 8 bits.

Gray Scale Images- These images assign the numerical value to each pixel which

corresponds to the intensity level for that point. The range of values assigned dependson the bit resolution used in the image.

RGB Images- These images assign three numerical values to each pixel corresponding

to the red, green and blue intensity value respectively. The example in Figures 2 -5

shows how an image is assigned pixel for the RGB type image.

Figure 5: Image further showing the RGB pixel values of a part of the original image.


5/18

Mathematics of Image analysis

In this section we briefly discus about the various mathematical principles which are

used in the image processing field.

Image processing analysis is normally done in 2 different domains

Frequency Domain: In frequency domain the frequency spectrum of whole image is

calculated and further altered or processed to analyze the image. This method does

not break down the image into its pixel constituent for analysis and rather takes the

image as a whole.

Spatial Domain:On the contrary in spatial domain the analysis is done on pixels and

all alterations and processing is done at pixel level.

Although the above mentioned two methods seem different but an alteration in one

also affects the analysis done in the other domain. For example if a spatial filter is

applied to reduce noise then it also affects the frequency spectrum.

Fourier Transformation: As discussed earlier, all digital images are represented by

numerical values depending on the method used, every image can be considered as a

signal which can be analyzed further with techniques used for real signals. One of the

techniques frequently used in signals and systems is the Fourier Transform which

states that Any periodic signal can be decomposed into a collection of Sines and

Cosines of different amplitude and frequencies. Thus mathematically any signal f(t)

can be written as

On the other hand if we add all the Sines and Cosines together which constitute asignal, we can reconstruct the signal back. The above representation of breaking asignal into different sine and cosine frequencies is called the Fourier series and thecollection of all the frequencies present is called the Frequency spectrumof the signal.

The example below demonstrates the reconstruction of a triangular wave by addingSine waves of different frequencies. Also we can see the frequency spectrums of eachwave and the final triangular wave generated has all the frequencies that we used togenerate.

Figure 6: Time domain plot of a Sine wave


6/18

Figure 7: Frequency domain plot of a Sine a wave at f=4

Fi ure 8: Time domain lot of a summation of two Sine waves

Figure 9: Frequency domain plot for summation of two Sine waves


7/18

Figure 10: Time domain plot for summation of three Sine waves

Figure 11: Frequency domain plot for summation of three Sine waves

Figure 12: Time domain plot for summation of ten Sine waves


8/18

An important result to be analyzed from the above example is that the more a signal

has sharp changes in it like square wave, impulse etc the more number of frequencies

required to reconstruct it.

Taking forward the above result the Noise and Edge detection basic principle can be

easily understood. Referring to the Figure 3 below it can be seen that the intensity of

the pixels at the edge of the image is quite different from its surrounding pixels. Thus

there is a sudden change in the signal representing that portion, and applying the

result of Fourier series transform above we can be sure that the frequency spectrum

of the image will have higher frequencies for the edges and noises since they are

rapidly changing signals requiring higher sines and cosines for representation.

Further passing the signal from a low pass filter will severely attenuate the amplitude

of higher frequencies and thus help in reducing the noise but also will blur the imageby attenuating the high frequencies corresponding to the edges.

Image quality measurement metrics

The quality of an image is quantified and defined by two important terms which are

Resolution and Quantization.

Resolution- Resolution measures the size of an image .For still images it is specified

as a spatial resolution and is given as Column(C) byRow(R), where the values C and R

refer to the number of pixels used to cover the space representing the image. The

resolution for analog signal is a function of the sampling rate and for a digital signal isthe function of number of pixels present. Thus the resolution of an image decreases if

it is sampled less frequently.

The resolution for a video is defined as Temporal resolution and is defined as the

number of images captured in a given period of time also called as frames per second

(fps)

Figure 13: Frequency domain plot for summation of ten Sine waves


9/18

Figure 14: Different Resolutions of an image( Image Courtesy: www.cmu.edu)

Original Image 512 x 512 resolution Image with 256 x 256 resolution

Image with128 x128 resolution Image with 64 x 64 resolution

Image with 32 x 32 resolution

Quantization-The concept of Quantization is key to the process of digitization of

images. It is defined as the mapping of a continuous signal representing a scene to a

discrete number corresponding to the intensity of each pixel constituting the image.


10/18

In the process of converting Analog image from a Capture device (camera) to a Digital

signal , the voltage read at the sensor which corresponds to the wavelength of the

light entering the sensor after reflection from the object is divided into different levels

with each level representing the specific intensity level .Depending on the hardware

used these possible levels are given as 2nwhere n is the number of bits . Thus a 8-

bit sensor divides the intensity into 28 =256 levels with 0 representing blackand 255 representing the white.

Quantization and Resolution are independent of each other, as it is possible to have a

high resolution image which can have just two intensity levels of 0 1nd 1 representing

dark or light and can also be quantized into 8 bits yielding a range of 0-255 intensity

levels. The requirement for particular values of Quantization or Resolution depends on

the use of the image for a given application.

An important concept closely related to Quantization is that of Sampling Theorem also

called the Nyquist Sampling Theorem. When a digitized signal is reconstructed the

sampling theorem plays an important role. It is concerned with the number of samples

needed to recreate the original signal. It can be restated in relation to image

processing as An analogue image can be reconstructed exactly from its digitized form

as long as the sampling frequency is at least twice the highest frequency present in the

image. The signal reconstructed with sampling frequency less than the Nyquest

frequency (the optimum frequency) suffers from a phenomenon called Aliasing.

As mentioned earlier we can measure the frequencies present in an image by checking

the frequency spectrum and by adhering to the proper sampling frequency can

reconstruct the image with adequate resolution. It is a common practice to keep the

sampling frequency equal to 4-5 times larger than the highest frequency of the signal.If an image is reconstructed with a low sampling rate it will lack the high frequencies

present in the original image and thus the important details of the image are lost.

Chapter 2Image Processing Techniques

Once the image of interest is stored in the system the by applying quantization and

digitization, they need to be processed before we can use the Image Analysis routines.

The purpose of Image processing techniques is to remove faults, trivial information

and any shortcomings introduced during the image acquisition. Common faults

include blurred image and noisy images. Major image processing techniques include

Histogram analysis, thresholding, edge detection, segmentation and masking. We willbe briefly discussing each technique.

Histogram Analysis: An Image histogram is a plot of the relative frequency of

occurrence of each of the pixel at gray level plotted against the values. In other words

it gives the number of times each gray level occurs in the image.


11/18

Histogram gives a visual revelation of the basic contrast that is present in the image

and we can identify any differences in the pixel distribution of the image foreground

and background.

Histogram analysis is frequently used to improve the contrast of a poorly contrasted

image. Two techniques are commonly employed for correction:

1. Contrast Stretching or Normalization where in a piecewise linear function isused to transform the gray levels to different values by stretching their value

according to the linear function, thus transformed image has a better contrast

than one without Normalization.

2. Histogram Equalization is a procedure where in no image correction is donewithout any user input and the gray pixel level stretching is done.

Add histogram stretching example

Thresholding: Thresholding is one of the important techniques in Image

processing where an image is divided into different levels and assigning each pixel

to those levels by comparing each pixels grayness value to a threshold.

Thresholding can be either performed as a single level by choosing a singlethreshold or multiple thresholding by choosing multiple levels.

A simple example ofsingle thresholdingcan be as below:

Turn a gray scale image to a binary image (black & white) image by choosing a gray

level Lin the original image and turn each pixel to black or white by checking if its

gray value is greater than or less than Li.e.

Figure 15: A low contrast image and its

corresponding Histogram

Figure 16: A well contrasted image and its

corresponding Histogram by applying

Histogram stretching


12/18

White if its gray level > L.

Convert a pixel to Black if its gray level < L.

A simple example ofDouble thresholdingcan be as below:

Choose two values L1 AND L2 for Thresholding

White if its gray level > L1and < L2.

Convert a pixel to Black if its gray level IS not between two thresholds.

Thresholding technique is an important part of the Image Segmentation

technique where it is used to isolate objects from its background and also finds

much application in Robot Vision applications.

THRESHOLD EXAMPLE WILL GO HERE

Convolution mask: Convolution mask is one of the filtering techniques used inImage processing .It is also sometimes called as Neighborhood Processing .As the

name suggests convolution is applied to the image of interest The idea is to move a

mask : a rectangle (odd sized) over the given image. The mask is first placed on

the upper left corner of the image and the summation of the product of value of

each pixel multiplied by the mask values is calculated. Further this summation is

divided by constant normalizing value. If the summation calculates to a zero value

then it is replaced by 1 or by the largest number. A new copy of the image is

generated and the resulting number obtained after the normalization process

discussed above is substituted in the center of the block that was superimposed by

the mask. The whole process is repeated by moving the mask to the right andagain replacing the center value with calculated normalized value. The operation is

continued till all the rows of the image are affected by this operation. Figure 18

depicts the idea of Convolution mask procedure pictorially.

Figure 17: An example for Thresholding applied to an image


13/18

An important point to be noted here is that the first and the last rows and columns

are never affected by this Convolution procedure and are either ignored or replaced

by zeroes.

One of the applications of Convolution mask filtering technique is to us it for

Noise Reduction in the image. Specifically the technique is called Neighborhood

Averaging , where in a mask is used to reduce the gray value of a pixel which is

totally different from its neighbors ( and hence called noise).The mask does not

affect the values which are in gray values equal or near to their neighborhood

pixels. Thus the method in a sense works as a low-pass filter by attenuating the

sharp differences between the neighborhood pixels and by not affecting the pixels

whose intensities are similar.

Since this method introduces new gray levels in the image, thus it affects theHistogram of the image and also reduces the sharpness of the image .An

alternative method to overcome this shortcoming is a method called Median

Filtering which uses the median value of the pixel value to replace the center

value rather than calculating the convolution.

Figure 18: An example for Applying Convolution Courtesy: S.Niku


14/18

Edge Detection:

One of the most important uses of Noise filtering in Image processing is to facilitate

in the Image analysis by helping in Feature Detection and Feature Extraction.

Edge detection is one of the most important techniques used in Image Processing.

Edges can be considered as a discontinuity or gradient in the pixel values which

exceeds a given threshold value. An edge represents an observable difference in the

pixel values .Considering the tables below we can clearly observe that there is a

clear difference in gray values of right hand figure for columns 3 & 4 and

Indicating the presence of Edge.

The techniques used in Edge detection operate in the image and result in a line

drawing of the image. These lines can represent the changes in the values such as

cross-sections of planes, textures, or difference in light intensities between parts

and backgrounds. The principle behind these techniques is to operate on the

difference between the gray levels of pixels or groups of pixels through the use of

Convolution Masks (discussed in previous section) . The final representation after

applying the techniques takes lesser memory and saves in computational and

storage costs apart from helping in Object recognition and Segmentation.

If we plot the gray values we traverse the image from left to right we can get a

profile of the image. The edge profile can be a ramp edgewhere the gray values

change slowly or a step edgewhere the gray values change suddenly.

Referring to Figure 19 we can see that an ideal edge can be thought of a clear

distinction between the pixel vales, which is only possible in Binary images. The

real edge shown in the image is more prevalent in other image types and thus just

a comparison of the pixel values is inadequate for edge detection. Practically, the

first and the second derivative of the graph are employed for edge detection, where

in we can clearly see that the Edge can be detected at the peak of the firstderivative and the zero crossing of the second derivative.

45 46 102 108

44 45 115 112

42 46 120 134

45 46 47 44

44 45 43 42

42 46 44 43


15/18

Derivatives

Suppose we have a plot of the function f(x) for the profile of an image. We candefinitely plot the derivative f (x)of the image. The derivative returns zero for all

the constant sections of the profile, and returns non-zero values only in the part of

image in which there is difference. For an image with two dimensions using partial

derivatives we define the gradient andlaplacianas

respectively. The above gradient points in the direction of greatest increase for the

function f(x, y). The direction of increase is given by

and the magnitude is given by Most edge detection methods find the magnitude of the gradient and then apply

threshold to the result.

First order derivatives

As we know the definition of the Derivative is given as

Figure 19: An example for Edge Detection using Derivatives Courtesy: S.Niku


16/18

Since in an image the smallest value for h is 1 being the difference between the

index values of two adjacent pixels the digital implementation of the above

definition is given as

Other definition of Derivative is given as

and The digital version comes out to be as

and Using the above expressions for derivatives and leaving out the scaling factors

horizontal and vertical filters can be taken as

and The use of above filters finds the vertical and horizontal edges in an image. In

order to provide a smoothing effect a combined filter is used which is given as

Prewitt filter for Vertical Edge Prewitt filter for Horizontal Edge

detection detection

Second order derivatives

As we have used the first order derivatives for edge detection, Second order derivatives

use is also common. As mentioned earlier, the sum of second derivatives in both the

direction is called the Laplacian. Defined by

It is implemented by the filter as


17/18

The second derivative is also called Discrete Laplacian. The laplacian has a property of

Rotation Invariant. That is , if the Laplacian is applied to an image ,and the image isrotated , we obtain same result when we rotate the image and then apply the

Laplacian . The major problems with Laplacian filters are that they are sensitive to

noise.

Add an example of edge detection

Segmentation:

Segmentation is the name given to generic techniques by which an image is

subdivided into its constituent regions or objects. The main purpose of these

techniques is to separate the information contained in the images into mutually

exclusive regions which can be later used for other purposes.

Segmentation occupies a vital role in image processing as it is the first important step

that must be taken before the other tasks such as Feature extraction or classification

are to be performed. All segmentation techniques are based on the following basic

approaches:

1. Edge Methods: This approach detects the edges as a means to identify theboundary between regions by finding the sharp differences in the pixel

intensities.

2.

Region Methods: This approach assigns pixels to different regions based on apredefined selected criterion.

As we have already discussed the Edge Detection method which is also a part of

Segmentation technique ,we will now discuss some more Region methods used in

segmentation.

Region growing and Splitting:Region growing is an approach of segmentation in which

pixels are grouped into larger regions based on predefined similarity criteria. The

process starts by selecting a number of seed pixels (also called Nucleipixels)which

are randomly distributed over the image and appending pixels in the neighborhood

region to the same region if they satisfy the similarity criterion of intensity, color orother properties. The nuclei regions act as nucleus for subsequent growing and

merging. The small regions thus formed are combined into larger regions to create the

final segmented regions.

The region splittingtechnique employs the similar philosophy as the region growing

but it is the reverse approach. The method starts by treating the whole image as a

single region which is then successively broken down into smaller and smaller regions


18/18

until any further subdivision results in difference between adjacent regions falling

below some threshold value. A most used example for region splitting is using the

Split-and-merge technique.

Chapter 3Image Analyzing Techniques

Image analysis is a collection of techniques used to extract information from the stored

images which have been already processed with image processing techniques. The

techniques include object recognition, feature extraction and extraction of depth

information.

Object Recognition: An object may be recognized by its features. The features

normally include Gray levels, Morphological features like area, perimeter, and

moments. Let us discuss some more about the various features.

a. Gray Levels: The different parts or objects in an image can be identified bychecking for average, maximum or minimum gray levels. For example, there

may be three parts in an object and each with different color or texture. If

average, maximum and minimum gray levels of the object are found, the objects

can be recognized by comparison of these values.

b. Morphological Features: The different morphological features are perimeter,area, diameter etc. The perimeter of the object may be found by applying edge

detection routine and then counting number of pixels on the perimeter .Area

can be calculated by region growing techniques discussed earlier in chapter 2 .

c. Aspect ratio: Aspect ratio is defined as the width to length ratio of an enclosingrectangle about t he object.

Summary

In this essay we have discussed the basics of Image processing and Analysis methods

currently being used. Different examples have been used to make the theory much

more clear and the examples have been implemented with MATLAB. The image

processing techniques are an essential part in the Robotics domain and a successful

implementation leads to building of perfect robotics systems.

References

1. MATHWORKS, MATLAB Documentation Manual.2. Introduction to Robotics, Analysis control & Applications by Syed B.Niku 2nd

edition.

3. Fundamentals of Digital Image Processing by C.Solomon & T. Breckon.4. Digital Image processing using MATLAB by Gonzalez ,Woods & Eddins

essay on principles of image processing and analysis-s

Documents