automatic face detection - dr. hussein s...
TRANSCRIPT
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 1
PREFACE
This report describes that the project work carried out in the Faculty of Arts, Computing, Engineering and Sciences at Sheffield Hallam University between June and September 2010.
The submission of the report is in accordance with the requirements for the award of the
degree of MSC Telecommunication & Electronics Engineering under the auspices of the
University.
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 2
ACKNOWLEDGEMENT
First of all I would like to thanks Almighty for all His kind blessings without which my work
has been knock-off. Also I would like to give the credit to the people who remember me in
their prayer for my achievements.
I take an opportunity to express my gratitude to Dr. HUSSEIN ABDUL RAHMAN, senior
faculty of Sheffield Hallam University for his constant support, valuable suggestions and
great ideas which enhance my level of performance. He also motivated me by providing apart
from his vast knowledge and sapience.
I would like to appreciate the faculty members management of university for providing me
great support through out my master course of education. Last but not the least, I would
honour to thanks my parents, brother and sisters for their encouragement and confidence on
my ability.
September, 2010 Khaleel Uddin Mohammed. Sheffield, UK M.sc Telecommunication & Electronics, 2010.
Department of Arts, Computing, Engineering and Sciences. Sheffield Hallam University.
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 3
Declaration I declare that, all the material contained in this document is solely my work except where it is indicated through proper citations and references. No part of this work has been submitted to any other university for a degree or any other qualification. September, 2010 Khaleel Uddin Mohammed. Sheffield, UK M.sc Telecommunication & Electronics, 2010. Department of Arts, Computing, Engineering and Sciences. Sheffield Hallam University.
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 4
ABSTRACT
Human face recognition systems have gained a considerable attention during last
decade due to its vast applications in the field of computer and advantages over previous
biometric methods. There are many applications with respect to security, sensitivity and
secrecy. Face detection is the most important and first step of recognition system.
Human face detection suffers from various challenges due to variation regarding
image conditions, size, resolution, poses and rotation. Its accurate and robust detection has
been a great task for the researchers. There exist various numbers of methods and techniques
for face detection but none can guarantee successful in all conditions for all kinds of faces
and images. Some methods are exhibiting good results in certain conditions and others are
good with different kinds of images. Face detection based on skin colour is found to be more
effective technique because of the properties of skin colour which is unique and can be easily
separated from the other objects present in the image and background.
Key Words: Face recognition, Face detection.
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 5
CONTENTS
Preface
Acknowledgement
Declaration
Abstract
Content
List of Figures 1 Introduction 1
1.1 Over view 2
1.2 Aims and Objective 2
1.3 Expected Output 2
1.4 What is Face Detection 2
1.4.1 Challenges 3
1.5 Thesis Structure 3
2 Literature Review 5
2.1 Background 6
2.2 Literature survey 7
2.3 Case studies 7
2.3.1 Case study-1 7
2.3.2 Case study-2 7
2.4 Methods of Face Detection 8
2.4.1 Knowledge Based Method 8
2.4.2 Feature Based Method 9
2.4.3 Template Based Method 10
2.4.4 Appearance Based method 11
2.5 Difference Between Face Detection and Face Recognition 11
2.5.1 Sensor Module 12
2.5.2 Face detection and Extraction of Facial Feature 12
2.5.3 Classification Module 12
2.5.4 System Database Module 12
3 Introduction to Image Processing 14
3.1 What is Image 15
3.2 What is Pixel 15
3.3 Image Processing 15
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 6
3.4 Image Enhancement 15
3.5 Image Restoration 16
3.6 Colour Image Processing 17
3.6.1 Radiance 19
3.6.2 Luminance 19
3.6.3 Brightness 19
3.6.4 Hue 19
3.6.5 Saturation 19
3.7 Image Segmentation 20
3.8 Edge Detection 20
3.9 Histogram Processing 21
3.10 Thresholding 22
4 MATLAB 24
4.1 MATLAB 25
4.2 MATLAB System 25
4.3 Advantages of MATLAB 26
4.4 Disadvantages of MATLAB 27
4.5 MATLAB Windows 27
4.6 Script Files 28
4.7 Arrays 29
4.8 Control Flows 30
4.8.1 For Loop 30
4.8.2 While Loop 31
4.8.3 If-Else-End Construction 31
4.8.4 Switch-Case Constructions 32
4.8.5 Try-Catch Block 32
4.9 Application areas of MATLAB 32
4.10 Key Features 33
4.11 MATLAB for Image Processing 33
4.12 Image Formats and Types Supported by MATLAB 34
4.12.1 Binary Images 34
4.12.2 Indexed Images 34
4.12.3 Grayscale Images 35
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 7
4.12.4 True Colour Images 35
5 Algorithms, Results and Comparisons 36
5.1 Colour Models 37
5.1.1 RGB Colour Space 37
5.1.2 HSI (Hue, Saturation, Intensity) Colour Space 38
5.1.3 YCrCb Colour Space 38
5.2 Algorithms 39
5.2.1 Skin Colour Based Face detection in RGB Colour Space 39
5.2.2 Skin Colour Based Face Detection in YCrCb Colour Space 39
5.2.3 Skin Colour Based Face Detection in HSI Colour Space 41
5.3 Proposed Algorithm For Face Detection 43
5.4 Experimental Results 44
6 Discussion, Conclusion and Future Work 49
6.1 Discussion 50
6.2 Conclusion 50
6.3 Future Work 51
7 References
8 Appendixes
8.1 MATLAB Code
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 8
LIST OF FIGURES
FIGURE NO. NAMES OF THE FIGURES PAGE NO
2.4.3.1 Face template for face localization 10
3.4.1 Eyes 16
3.4.2 X-ray 16
3.5.1 Image restoration 17
3.6.1 Refraction of light 18
3.6.2 Gray-scale 18
3.6.3 RGB image components 19
3.8.1 Edge detection 21
3.9.1 Histogram Parts 22
3.10.1 Thresholding 23
3.4.1 Matlab window 28
4.10.1 Image coordinate systems 33
4.10.2 Spatial coordinate system 34
5.1.1.1 RGB cube 38
5.2.3.1 HSV-cone 43
5.4.1 Input image-1 45
5.4.2 Image-1 segmented 45
5.4.3 Applying RGB to segmented image-1 45
5.4.4 Image-1 after edge detection 46
5.4.5 Binarlization 46
5.4.6 Noise removed by performing Morphological operation 46
5.4.7 Output of image-1(Face detected) 46
5.4.8 Input image-2 47
5.4.9 Image-2 segmented 47
5.4.10 Applying RGB to segmented image-2 47
5.4.11 Image-2 after edge detection 48
5.4.12 Binarlization 48
5.4.13 Noise removed by performing Morphological operations 48
5.4.14 Output of image-2(Face detected) 48
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 9
Chapter-1
Introduction
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 10
1.1 Over view:
Face detection demand is growing with a rapid speed because of its vast application in
the field of computer vision. For the implementation of different security methods as such
face tracking, face recognition and expression recognition and pose estimation all this
methods requires face detection algorithm. Though its got a lots of challenges variation in
poses, illumination, facial expression and occlusion the demand of face detection is still high
because of its non-intrusive nature. Because unlike other bio-metric, face detection method
gives better identification for security and best result than any other bio-metric methods.
Face detection is not only the first step of any automatic face recognition system but also for
human computer interaction system, generic object recognition and detection and many
surveillance systems.
1.2 Aims and Objectives:
Aim of the thesis is to evaluate the face detection process using the all three colour spaces
so that advantages and application of all three colour space can club together into a single
proposed algorithm. The objectives of the thesis is
• First is to understand the different methods of face detection methods
available under which various techniques are used.
• After the analysis of different method a suitable technique is proposed which
is more accurate and have high detection ratio.
• Compare the proposed technique of face detection with the other techniques.
1.3 Expected outcome:
The expected output of the automatic face detection method based on skin colour is
the same image which is given as input to the system but with boundaries across the faces
present in the image showing the exact location and size of the image. This boundaries across
the face may be of any shape like square and circle but in the algorithm we have proposed,
accurate shapes (squares or circles) are not consider but it completely depends on the shape
of the face so that no redundant information comes into the picture.
1.4 What is Face Detection:
Face detection is defined as finding the number of face present in an image and
locating their position and size. Generally the process of face detection is partition in two
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 11
steps in the first stem the whole image is scanned to find the region (features) that can be
identified as face, skin colour is the most common among all. And the next is localisation
which gives more accurate estimation of size and position of the faces. Face detection is the
first step of human identification and recognition system.
1.4.1 Challenges: Face detection method faces are various challenges such as pose, imaging
condition.
Pose: The image of a face vary due to relative camera-face pose and some facial
feature becomes partially or completely occluded.
Image orientation: face images directly vary for different rotation about the camera’s
optical axis. Image orientation directly affects the angle of the face.
Illumination: This problem is mainly due to lighting, makes a larger difference with
the same face as compare to difference within different faces while comparing
(Javidi.Bahram, 2002).
Occlusion: Sometime, faces in the image are occluded with other objects such as
moustaches, beard, optical lenses and other types of object make it very difficult to find
accurate image.
Facial expression: Some persons may have different expression at different times,
this also contributes to challenge in face detection.
Face size: Size of the face also make difficult to automate a system for face detection
and recognition (Muhammed Tariq Mahmood, 2002).
Image condition: This problem includes factors such as intensity, resolution, camera
lighting, background, characteristics of image capturing device and distance between camera
and person, plays an important role in the process of face detection.
1.5 Thesis structure:
Chapter 1: Introduction to face detection covers the aims and objective of face
detection and also the problem faced in achieving effective face detection.
Chapter 2: Briefly describe the methods of face detection and explain the difference
between the face detection and face recognition.
Chapter 3: It gives the clear picture of image processing and the different fields of
image processing.
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 12
Chapter 4: This chapter deals with the MATLAB tools and image formats supported
of MATLAB. And also describe the control flows of MATLAB.
Chapter 5: In this chapter three colour spaces and their algorithms along with the
proposed algorithm are discussed in depth.
Chapter 6: The final chapter if this thesis discussions and future work on the face
detection method is carried out.
Chapter 7: This chapter composes of code of the algorithm proposed on skin colour
based face detection.
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 13
Chapter-2
Literature Review
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 14
2.1 Back ground:
From the advent of cell phones in the electronics history till today there is continuous
development in the technology used in the cell phones. Modern cell phones have so many
features such as integrated cameras, high resolution display, multiple communications
interfaces, and processing power and many more equal to at least, to a mid-1990’s PC. Cell
phones are most widely distributed in terms of range of users, geographical distribution than
any other class of computers. This makes them an attractive development platform for many
image processing and computer vision algorithms. Among this algorithms face detection is
early step in many computer vision systems, including Video communication, Super video
compression, Face recognition, Video surveillance, Responsive user interface, Video-
augmented speech understanding, Intelligent autofocus etc.
Video communication: A high quality image with fixed bandwidth can be obtained
by compressing the detected image at lower compression rate than the background.
Super video compression: Once a face is detected, it can be matched with some pre-
recorded faces of the presenter and then the chosen face index transmitted and displayed,
resulting in an satisfactory video experience using very small bandwidth.
Face recognition: After detecting a face it is compared with a database of known
faces, and then an appropriate response can be made when a face s recognised.
Video surveillance: Detected faces can be tracked in order to find out when a person
has entered an area of interest.
Responsive user interface: A tool may perform some pre-programmed actions when a
face is detected.
Video-augmented speech understanding: Speech understanding can be made easier by
detecting and then tracking face movements to determine which of several similar-sounding
phonemes was actually being pronounced.
Intelligent autofocus: Once a face is detected, a camera’s focus can be used to bring
the face into improved focus (Jon A.Web, 2010).
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 15
2.2 Literature Survey:
2.3 Case studies:
The case studies contain the over view of two methods of face detection using the same
technique (skin colour) with two different approaches one uses the guassian mixture model
and other uses the morphological operations.
2.3.1 Case study 1:
Face detection in colour images: Face detection is divided into three parts human skin
segmentation to describe the probable regions corresponding to face, adaptive shape analysis
to separate human faces from initial segmentation and view based face detection to further
identify the location of each human face. Human colour (skin colour) which is important
feature of human face and produces information which is invariant to face detection. Feature
space in which skin colour cluster tightly and reside remotely to background colour, is used
to effectively exploit skin colour for face detection.
YCbCr colour space is the common for face detection since it is uniform and more
important is that it separate luminance and chrominance. As many studies reveals that
chrominance an luminance must be separated from each other so that there will be no effect
of illumination (insensitive to illumination). Dispersion of skin colour is prototyped with a
Guassian mixture model (WenmiaoLu and ShaohuaSun, 2000).
2.3.2 Case study 2:
Face Detection Using Colour Based Segmentation and Morphological Processing:
This case study describe a algorithm that uses colour based segmentation and some
morphological processing. The colour based segmentation. The colour based segmentation
takes the advantage of patterns developed in only two colour spaces i.e HSI and YCrCb,
instead of three colour spaces, followed by the morphological operations and a template
matching. Different ethnic group have different degrees of the melanin and pigmentation, the
range of colours that human facial skin takes on is clearly a subspace of the total colour
space.
The steps defined for this algorithm is as follows:
1. Select an image as input.
2. Apply HSV and YCbCr colour mode.
3. Calculate skin ranges for H, Cb and Cr.
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 16
4. Obtain colour segmented image.
5. Apply open operation on the segmented image.
6. Check for holes if any. And find width and height of the region.
7. Find correlation of face with template.
8. Region consider as image ( Dr. Arti Khaparde, Sowmya Reddy.Y Swetha
Ravipudi, 2007).
2.4 Methods of face detection:
Automatically detecting human face has played a vital and challenging task in
computer vision research. The consequence of this is its vast application, since it is the first of
towards intelligent vision-based human computer interaction and it the first stage of
automatic face recognition system. Detecting human face had been a difficult and challenging
task since it involves the many problem such as image size, face size, face angle, pose,
features extraction, type and many more. For this there are number of methods and
algorithms are developed each has its own significance. The four categories of face detection
algorithms are as follows
2.4.1 Knowledge based method:
This method relies completely on the simples rules which describe the features of the
face and the relation (for example distance) between them such as there exist a uniform
intensity in the central region of the face and also eyes are symmetric to each other. The
search algorithms guided by the rules are applied to find the target face [Yang, Ming-Hsuan,
2001]. The other features may be chin, nose and eyebrows. A hierarchical approach which
studies the face at various resolution can be used. At top level faces of the people present in
the image are taken out roughly and in the low level facial features are extracted thus
separating the face with the non-face region accordingly depending up on the rules defined on
the facial features.
Yang and Huang introduce a knowledge based method also known as top down
method in 1994. This method is divided in three stages. In the first stage high rules are
defined and in the second stage edge detection techniques is done using histogram and
equalization methods.
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 17
Knowledge based method is good for the frontal image thus finds difficulty in
detecting faces in taken at different poses or head orientation. This is because it is not easy to
translate human knowledge about the rules into definite rules. In practice this method of face
detection is not often used.
2.4.2 Features based method:
The central theme of this method is to find the features of the face that are invariant or
not affected by any factors such as variation in lighting conditions, pose and others. The
different features that can be used are eyes, eyebrows, mouth, nose, face shape, texture and
skin colour which are extracted using the derivatives filters, edge detectors, morphological
operations or thresholding. Statistical analysis is done to get the relationship and verify the
presence of faces.
Skin colour is one of the powerful and effective features to detect the face since it has
the colour thus colour scene segmentation is computationally fast. The other important of
skin colour as facial feature to detect face is it is robust to changes in the view point, complex
background and shading to partial occlusion. Though geographically, people of different
country has different skin colour. This difference lies in their intensity not on their
chrominance. The unique texture of skin colour is used to separate it from different object
present in the image. Each pixel of the image is labelled according to its similarity with the
skin colour and then the regions are made as face if it contains skin colour pixel above a
particular number(threshold) and non-face for the region is it contain less than the threshold.
This colour based method is sensitive to light, occlusion, adjacent faces ant also if the
features are corrupted severely. And it is also find difficulty in detecting face in complex
background.
Feature detection method is completely based on detecting feature of the face. Edge
detection one of the technique used to detect the features and followed by grouping the edges
and other is to use blob and streak instead of edges. For example in the face model eyes, nose
and cheeks bones are represented as two dark blobs and three light blobs. For the facial
features such as outlines of eyebrows and lips, streak model is used. To improve the detection
accuracy multi features methods that combines several features is used. Firstly to detect the
face by using features such as skin colour, texture and shape and lastly to verify these using
features such as nose, hair and eyebrows.
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 18
2.4.3 Template based method:
In this method a large number of templates of faces are stored into the system. The
input image from which the faces is to be detected is processed by making head outline which
is elliptical (roughly) is made using edge detector and filters. After that a contours (line
joining the facial features) of the facial features are drawn which gives the information about
face and features geometry. After all, the correlation between the predefined stored templates
and the features extracted from the input image is computed to know whether there exist a
face in the image or not. As it is a correlation process input images with different poses,
scales and shapes are sensitive parameters. To overcome this sensitive parameters deformable
template method have been proposed which allow to scale, translate and rotate by using
elastic model which model the geometry of the face. Parameters of the model can be not only
the shape but also intensity information of facial features. P.Sinha [1994] utilises a group of
images invariants to describe the face pattern. The different parts of the face have different
brightness for example the brightness of the eyes and nose changes while illumination. This
methods calculated the pair wise ratios of brightness and projected directions. A face is
localised by satisfying all conditions for dark and bright regions. The figure below is the
enhance figure that have 23 defined relation for a face template.
Fig 2.4.3.1 Face template for face localization (P.Sinha, 1994)
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 19
It is difficult to define all templates of the faces thus make it unsuccessful for different
poses and illumination problems.
2.4.4 Appearance-based methods:
Similar to template based method templates are learned by computer system in the
form of training algorithms which classify the regions into two types of classes there are face
or non-face class. The face class shows the images that contain the face images and the non-
face class shows the images other than the images of face class that is anything that is not a
face. For learning face and non-face classes machine learning algorithm is used. All
techniques used under appearance based methods have common steps there are classification
of face and non face classes, pre-processing, learning and post processing of the images.
Statistical analysis and machine learning techniques are used to determine the probability
distribution function of the pixel brightness patterns of images of the two classes. The names
of some famous appearance based methods are hidden Markov, neural networks, Adaboost,
decision tree, eigen faces, LDA and decision tree model.
2.5 Difference between face detection and face recognition:
The immense application of face recognition, its has became a popular area of
research in field of computer vision. Due to its ability of problem solving, it has gained
attention of not only computer researchers but also psychologist ant neuroscientist. The
increase in commercial and law enforcement application commanding personal
authentication for people’s surveillance, transaction security, the name face recognition
becomes famous. The other bio-metrics techniques which are also more reliable such as
fingerprints and iris recognition which are intrusive and they need the assistance of user
unlike face recognition which is non-intrusive as it is grounded completely on images
recorded from a distant by a camera. And the most important thing is that it does not required
user assistant and works even when the user has no idea about the existence of the face
recognition system. Face gives much information about the person in personal identification
system than the finger and iris does.
Face recognition works in two modes there are face identification and face
verification. Face identification works by matching the input image with the images stored in
the computer/database and in face verification the user gives an identity as an input to the
system and the machine accepts or reject depending on matching process performed on input
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 20
image and the images stored in the database with this specific identity (Filareti Tsalakanidou ,
Sotiris Malassiotis 2 and Michael G. Strintzis, 2009).
A face recognition system comprises of four modules there are as follows:
2.5.1 Sensor module:
This module captures face images, depending upon the requirement the image
acquisition device can be black and white camera or the colour camera.
2.5.2 Face detection and extraction of facial features:
The input images are scanned to detect the presence of face and window the location
containing only the face area. After finding the exact location and size of the face, irrelevant
information (neck, hair, shoulders and background) are cast aside which will help in
extracting of facial features used by face classifier to verify the identity of unknown face.
2.5.3 Classification Module:
In this module, comparison of the template or images extracted from the face
detection process and the a large set of stored images or templates in the database is done and
generate matching scores which gives the information about how identical the two images
are. A decision making module then perform either verification or identification on the basis
of matching score. For face verification, the matching score is compared with predefined
threshold and on the basis of comparison the user is accepted or rejected. For the face
identification, a set of matching scores between the extracted templates and the images or
templates of enrolled users is calculated. If the matching score is good, indicates that the user
face (unknown) is more similar to the face in the template stored in the database than any
other templates. To conform that unknown face is actually the one stored in the database,
matching score is compared with the predefined threshold (in the database).
2.5.4 System database module:
Database module is used to stack the templates of the entered (enrolled) users and also
solely responsible for enrolling users in the database of face recognition system. Sensor
module reads the images of his/her face and they are stored in database for training the
classifier, the stored images are termed as gallery images. If feasible these images are stored
in more than one session depending upon the face variations such as hairstyle, beard, make-
up, etc this enhance the performance of face recognition system. The training algorithm
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 21
determines the different values of threshold. The training algorithm is, for instance given
some images as input, then the features are extracted as process defined in the above step and
the template that provides much similar information as that of the input image are generated
and this are called training. This training algorithm depends on the method of face
recognition. The aim of the training is to encode the most discriminative quality of a user
based on the classifier selected, and to find out the values of the different thresholds (Filareti
Tsalakanidou , Sotiris Malassiotis 2 and Michael G. Strintzis, 2009).
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 22
Chapter-3
Introduction to Image Processing
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 23
3.1 What is Image:
A Two dimensional function, f(x, y) with x and y are spatial coordinates is termed as
Image and the amplitude of function f(x, y) at any point of coordinates (x, y) is called as
intensity of the image or the gray level. If the value of x, y and the amplitude of f(x, y) are
discrete than the image is said to be digital image.
3.2 What is Pixel:
Pixel is the basic, smallest discrete component of the image that can be controlled.
Each pixel has particular location and value. They are also called as pels, picture elements,
image elements.
3.3 Image Processing:
The term Digital image processing refers to processing of two dimensional data
(image) and the term digital image refers to an array of real or complex number represented
by finite number of bits.
Digital image processing has broad range of applications, such as remote sensing via
satellites and spacecrafts, medical processing, radar and acoustic image processing and
robotics.
There are various applications of image processing comes out of different fields of it.
The different fields of image processing are as followed.
Image enchancement
Image restoration.
Colour image processing
Image segmentation
Edge detection
Histogram processing
3.4 Image Enhancement:
According to David Lindsay image enhancement is : All that makes the difference
whether one sees darkness through the light or brightness through the shadows.
The main objective of enhancement is to improve the quality of images for human
visualisation and analysis that is making it(images) more suitable for a specific application.
Increasing contrast, removing blurring and noise are some of the operation of enhancement.
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 24
Image enhancement approaches are classified into two categories: spatial domain method and
frequency domain method. Spatial domain techniques are based on the location of pixels in
the image and frequency domain techniques are based on changing the Fourier transform of
an image. There are various techniques of image enhancement some of them are contrast
stretching, Power-law transformation, Gray level slicing, Bit-plane slicing etc. (Rafael C.
Gonzalez and Richard E. Woods, 2001)
Fig 3.4.1 Eyes (Zia-ur Rahman, 2001)
Fig 3.4.2 X-ray (Zia-ur Rahman, 2001)
3.5 Image Restoration:
Immanuel Kant: ‘’Things which we see are not by themselves what we see....It
remains completely unknown to us what the object may be by themselves and apart from the
receptivity of our senses. We know nothing but our manner of perceiving them.’’
The difference between the image enhancement and image restoration is that image
enhancement is subjective process whereas image restoration is largely objective process.
Image restoration is used to fix the color, the contrast, the tone and to repair damages which
is caused due to non-linear filtering produced by sensors, geometry distortion correction,
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 25
noise filtering and environmental limitation. Thus restoration techniques are pointed toward
modelling the degradation and putting on the inverse process in order to retrieve the original
image. the following figures will shows the best result of image restoration.
Fig 3.5.1 Image restoration (ozlouisville)
3.6 Colour Image Processing:
Pablo Picasso: ‘’for a long time I limited myself to one colour-as a form of discipline.
The use of colour in image processing is encouraged by two factors since colour often
help in object identification and extraction from the images is one factor and the another
factor is human eye can distinguish thousand of colour shades and intensities and only about
two dozen shades of gray. Colour image processing is divided into two major parts: full
colour image processing and pseudo colour image processing. In full colour processing the
important thing to be noticed is from where the images are acquired (from the colour TV
camera or colour scanner). And in pseudo colour processing, the problem is of assigning
colour to a particular range of intensities. Since many years full colour image processing is
not used however from a decade, colour sensors and colour processing hardware have
available at reasonable prices resulting in utilization of broad range of application such as
publishing, visualization, and the internet.[ Rafael C. Gonzalez, Richard E. Woods, 2001]
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 26
In colour image processing, the colour spectrum is divided into six broad region:
violet, blue, green, yellow, orange and red which are formed by passing white light through
the optical prism as shown below. The colour that human perceive in an object are
determined by the nature of the light reflected from the object. The object that reflects light
that is poised in all visible wavelengths looks white to the user an the object that favours
reflectance in a confined area of visible spectrum exhibits some shades of colour. For
instance green object reflects light in the range of 500nm to 570nm (wavelength) whilst
observing most of the energy at other wavelength.
Fig 3.6.1 Refraction of light (Ian Flitcroft, 2007)
Characteristics of light plays an important role in science of colour. The lights that
comes out of the black and white television is achromatic light that is white, grey and white
are achromatic colour and the light that comes out from this colour is achromatic light (as
shown below) ranges from 400 to 700nm. Radiance, luminance and brightness are used to
define the quality of achromatic light.
Fig 3.6.2 Gray-scale (Tomás Castelazo, 2006)
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 27
3.6.1 Radiance: The net amount of energy that comes out from the light source.
3.6.2 Luminance: It is the amount of energy an observer perceives from a light source.
3.6.3 Brightness: brightness is a subjective signifier and in practical it is not possible to
measure the brightness.
Red, green and blue are the three primary colours separated from the other on the
basis of the sensing capability of cones present in human eye. Due to the absorption property
of the human eye, colours are seen as variable combination of red, green and blue which are
called as primary colour. All colours resulting from the combination of primary colour are
termed as secondary colours such as magenta, yellow, cyan and red. The characteristic that is
used to separate the colours from each other are brightness, hue and saturation.
3.6.4 Hue:
Hue represent dominant colour as perceived by the observer, it is property associated
with the colour that has dominant wavelength in a mixture of light waves.
3.6.5 Saturation:
It gives the information about the purity of the colour or the amount of white colour
mixed with hue. For instance pink is a combination of red and white colours is less saturated.
That the degree of saturation and the amount of white colour added are inversely proportional
to each other.
The combination of Hue and saturation are termed as chromaticity and hence all
colours are characterized by their brightness and chromaticity.
Fig 3.6.3 RGB image components (Gonzalez and Woods, 1992)
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 28
3.7 Image segmentation:
Segmentation is a process of subdividing an image into constituents regions or objects
so that the minute details of the image are read or analyzed. The objective of segmentation is
to change or modify the representation of the image into meaningful and simplified way (easy
to analyzed). This process comes into picture when the object of interest is to detect or
identified from a set of objects and terminate immediately after the object is isolated from the
rest, there is no point in carrying out the segmentation process past the level of detail required
to identify the object of interest.
Like that of other fields of image processing, image segmentation also got numerous
application some of them are: Diagnosis, study of anatomical structure, object location in
satellite images, military imaging etc. Generally image segmentation algorithms are based on
two properties of intensity values: discontinuity and similarity. In the category of
discontinuity, images are segmented based on abrupt changes in intensity. And in the second
category, images are partition based on the similarity exists with the predefined criteria.
Based upon the two categories of image segmentation there are various of approaches are
analyzed among them the most wanted are edge detection and thresholding.
3.8 Edge Detection:
Its is the most approach for detecting meaningful discontinuities in grey level. The
set of connected pixels that lies on the boundary between two regions are termed as edge or it
is ability of the pixels to measure the gray-level transitions in a meaningful way and the edge
detection is to detect this pixels. Images are blurred due to bad sampling and imperfections of
image acquisition. Thus the edges are more closely modelled as ‘’ ramplike ‘’ profile as
shown below. The slope of the ramp and the blurring in the image are inversely proportional
to each other. Any pixels that lies on the ramp line are said be edge point. The length of the
line which depends on scope of the line is determined by the degree of blurring. Thus, it is
found that sharp edges tends to be thin and the blurred edges tends to be thick.
In the figure shown below first and second derivative of ramp is calculated. The first
derivative is positive at the point of transition into and out of the ramp (from the left to right)
and constant throughout the ramp line. Second derivative is positive only at the instant of
transition associated with the black side, negative only at the instant of transition associated
with the light side and zero throughout the ramp line. Thus first derivative is used to identify
the presence of an edge at a point in an image and the second derivative is used to conclude
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 29
whether an edge pixel lies on dark or light side of an image. Second derivative has two
attributes, for every edge in an image it produces two values and the other is zero crossing
property. The zero crossing, an imaginary line crossing zero at the midpoint of two values of
second derivative, joining the positive and negative values.
Fig 3.8.1 Edge detection (Shehrzad Qureshi, 2009)
3.9 Histogram processing:
Its a most common technique of image enhancement. It is a vertical bar chart that
gives the information about the distribution of set of data. when a table with a large set of
measurement is given, Histogram is used to organised and display the data in more simple
and user friendly format. A Histogram will make it effortless to determine where the bulk of
values falls in a measurement scale, and how much variation there exists between them.
The following shows some of the situation where Histogram is needed:
Summarize large data sets graphically
Compare process results with specification limits
Communicate information graphically.
Help to assist in decision making.
(http://www.scribd.com/doc/6811944/Histogram-VG)
As shown below, Histogram consists of five parts:
1. Title: The title briefly describes the information contained in the Histogram.
2. Horizontal or X-Axis: The horizontal or X-axis shows you the scale of
values into which the measurements fit.
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 30
3. Bars: The bars have two important characteristics—height and width. The
height represents the number of times the values within an interval occurred. The width
represents the length of the interval covered by the bar.
4. Vertical or Y-Axis: The Y-axis is the scale that demonstrates the number
of times the values within an interval occurred which is termed as frequency.
5. Legend: The legend provides additional information for example where the
data came from and how the measurements were gathered.
Fig 3.9.1 Histogram Parts
3.10 Thresholding:
Thresholding enjoys central position in applications of image segmentation because of
its intuitive property and ease of implementation. It tries to identify an object from its
background. Intensity is the property that pixels in a region can share with each other. So, a
normal way to segment such a regions is through separation of light and dark regions which
is termed as thresholding. Its produces binary image from grey-level by bending all pixels
below some threshold to zero and all pixels about that threshold to one.[11]
If g(x,y) is a thresholding version of image f(x, y) at some global threshold T.
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 31
Fig 3.10.1 Thresholding (Lindsay come, 2005)
The problem with the thresholding is that it does not consider the relationship
between the pixels but consider only the intensity property of the image. Thus there is no
guarantee pixels identified by the thresholding process is continuous. No information is given
if an extra pixels are included or disregarded within the given region. There are various
techniques for thresholding among them the simplest is global thresholding where image
histogram is partition by using a single global threshold T. Then segmentation process is
started to scan the image pixel by pixel and labelling each as object or background based on
the gray level of the pixel is lesser or greater than the value of T. In global thresholding only
a single threshold is defined and result with a problem of changes in illumination across the
scene causes some parts to be brighter in light and some parts darker in shadow. There is one
more technique of thresholding based on several variables which is called as multispectral
thresholding.
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 32
Chapter-4
MATLAB
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 33
4.1 MATLAB:
MATLAB (is the abbreviation of MATrix LABoratory) is special purpose computer
program to perform engineering and scientific calculation. It is used for simple mathematical
calculation such as mathematics, computation as well as to solve complex problems such as
for understanding and teaching engineering concepts, data analysis, exploration,
visualisation, prototyping, analysing and simulating of actual electrical and power systems.
MATLAB is a high-level language and interactive environment chosen to perform
computationally intensive tasks faster than with traditional programming languages such as
C, C++, and Fortran ( The MathWorks, Inc., 2010).
MATLAB can be used in wide range of application such as signal processing, image
processing, control design , biological computation, finance modelling and analysis. The
most interesting feature of MATLAB is that integration of MATLAB code with other
languages and applications. In 1970s MATLAB was produced as a technical tool, because of
its huge application it has expanded and now used for car design, aerospace and many
advance researches.
Toolboxes, one of the important feature of MATLAB. Toolbox allow the user to
learn and apply specialized technology and hence preferred as a tool of choice by many
researchers. This feature of MATLAB is a collection of m-files that stretch the MATLAB to
workout for particular families of problem. Toolbox are available for the area: control
system, image processing, fuzzy logic, signal processing and many more (Khairul Anuar
Ishak, 2008).
4.2 MATLAB system:
MATLAB system consist of five main sections:
1. The MATLAB language: MATLAB language is high-level matrix language
that includes control flow statement, data structures, functions and object oriented
programming features. It allows both programming in small which produces output quickly
and also programming in large and complex programs.
2. The MATLAB working environment: these are the set of tolls and facilities
which includes facilities for managing the variables in workspace and importing and
exporting data also developing, debugging and profiling m-files.
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 34
3. Handle graphics: It is graphics system which includes commands for image
processing, animation two-dimensional and three dimensional data visualization also it
includes low level command that helps in fully customizing the appearance of graphics.
4. The MATLAB mathematical function library: it is large collection of simple
algorithms such as sine, cosine and complex arithmetic to more advanced functions such as
Bessel functions, FFT (Fast Fourier Transform), eigen values.
5. The MATLAB Application Program Interface(API): This section of
MATLAB allows to write C and Fortran programs that interact with MATLAB (Khairul
Anuar Ishak, 2008).
4.3 Advantages of MATLAB: MATLAB has many advantages over conventional computer
language, some of them are
1. User friendly: Programs written in MATLAB can be easy modified with the
built-in integrated development environment and corrected with the MATLAB debugger and
hence used for rapid prototyping of new program.
2. Platform independence: MATLAB runs on many computer operating systems
thus providing a high range of platform independence.
3. Predefined Function: MATLAB has a large numbers of functions that are
predefined that gives tested and pre-packaged solutions to many basic technical problems
such as mean, square root, standard deviation etc. Also one more special purpose tool of
MATLAB to solve the complex problems in specific areas are Toolboxes.
4. Device-Independent Plotting: MATLAB has integral plotting and imaging
commands which is not found in other languages of computer. This plots and images can be
visualized on any graphical output device installed on computer on which MATLAB is
present.
5. Graphical User Interface: MATLAB comprises of tool that permit the user to
construct a graphical user interface. With this powerful mechanism of MATLAB, even an
inexperience user can design data analysis program.
6. MATLAB Compiler: Flexibility of MATLAB lies in compiling MATLAB
program into a device-independent p-code after that interpreting the p-code instruction at
runtime which effects the execution of program since the code id interpreted instead of
compiled (Khairul Anuar Ishak, 2008).
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 35
4.4 MATLAB has two disadvantages:
The execution of program takes more time than the compiled language since it is an
interpreted language. This effect can be reduce by proper structuring the program and by the
use of MATLAB compiler to compile the final program before distribution and general
use.The cost of MATLAB is very high which is five to ten times greater than C or Fortran
compiler. But luckily student edition of MATLAB, an inexpensive, same as full edition of
MATLAB is available.
4.5 MATLAB Windows:
Windows used in MATLAB and their basics functions are as shown below.
WINDOWS FUNCTIONS
Command Issue command to MATLAB for processing
Command history Running history of prior command issued in
the command window
Launch pad Tree layout for access to tools, demos and
documentation
Current directory GUI for directory and file manipulation in
MATLAB
Help GUI for finding and viewing on-line
documentation
Workspace GUI for viewing, loading and saving
MATLAB variables
Array editor GUI for modifying the content of MATLAB
variables
Editor/debugger Text editor and debugger for MATLAB text
files.
MATLAB has Command History Window, Command Window and Work Space
Window. When the MATLAB software is open the following screen opens which has
Command Window on the right side, Work Space Window on the top-left and Command
History Window on the bottom left.
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 36
Fig 3.4.1 Matlab window (Robert Evans, 2009)
Solutions of simple problems can be evaluated using the command window which is
fast and efficient. As the number of commands increases or when there is need to change the
value of any variable, typing at MATLAB prompt becomes tiresome. For this MATLAB
provides simple text file, this files are called script files or M-files. It is called as script file
because MATLAB simply reads from the script found in the file and M-file as these file must
end with the extension ’.m’, for instance, example1.m.
4.6 Script M-Files:
MATLAB not only supports many built-in function such as logarithmic,
trigonometry, complex and exponential but also the user-defined functions which worked as
conveniently as built-in functions. Lookfor command is used to find the functions that are
relevant to the particular application. For example exponential gives a list of functions that
deal with exponent.
lookfor exponential
exp - Exponential.
expint - Exponential integral function.
expm - Matrix exponential.
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 37
expmdemo - Matrix Exponentials
expmdemo1 - Matrix exponential via Pade approximation.
expmdemo2 - Matrix exponential via Taylor series.
Note : help command is used to know the working and utilisation of the function or
command.
The built-in functions are store as image file so there function and computational
details are not readily accessible. The functions that are implemented in M-files thus there
computational details and code can be read.(Williams J.Palm, 2009)
4.7 Arrays:
Operation that involve analysis of single number is called scalars, is the basis of
mathematics. But there exists many occasion where it is needed to perform operation on more
than one number at a time (repeated scalar). To overcome this problem, MATLAB defines
arrays operation.
Arrays are create by starting with left bracket followed by the values separated by
space or commas and then close the array with right bracket.
(source bruce little field )
Cell array and structures are the two data types of MATLAB. These are used to group
dissimilar but related arrays into a single variable. This allow much help in data management
as group of related data can be organised and accessed through a cell array or structure. The
elements of structure can be accessed using the command field and is different from cell
arrays which are accessed using standard array indexing operation.
(source :William j palm)
Example 1: A 2 X 2 cell array K containing the location name, date, air temperature
and the water temperature at three different point in the pond at the same time
K (1, 1) = {‘ Swift pond’};
K(1, 2) = {‘July 28, 1987 ’};
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 38
K(2, 1) = {‘[30 40 50]’};
K(2, 2) = {[35, 45, 55; 65,75, 85; 95, 15, 25]};
Example 2: A structure array to create database of student containing the information
about the types Student name, student ID, email address and marks. Each type of data is field.
Structure array of students
Student-1 Student-2
Name: Nigel Name: David
Student ID: 1903956 Student ID: 1903896
Email: [email protected] Email: [email protected]
Marks: 78, 56, 67 Marks: 45, 69,72
(William J. Palm, 2009)
4.8 Control Flow:
For simple problem interactive mode of MATLAB is very useful but for the complex
problem it requires a script file which are termed as program and writing such a file is called
programming. The value of MATLAB is greatly increased by the use of decision making
(Control flow) functions in it programs. Decision making functions are very useful since it
lets past computations influence future operations. MATLAB provides five decision making
or control flow structures there are Loops, While Loops, If-Else-End, Switch-Case
construction and Try-Catch blocks.
4.8.1 For Loop:
It allows a group of commands to be repeated for a fixed number of times. The
general form of For Loops is
For x = array
(command)
End
Example:
For a = 2 : 4 :100
b = k ^2
end
The variable a is assigned the value of 2 (initially), then b is calculated as b = k^2. In
each cycle a is incremented by 4 and calculates b until a exceeds 100.
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 39
4.8.2 While Loop:
As against the For Loop that calculates group of command a fixed number of times,
while loop calculates group of commands indefinite number of times. The general form of
while loop is
While expression
(command)
End
Example:
a = 5;
while a = 0
disp(a)
a = a – 2;
end
The variable a takes the values 3, 1, -1, -3........because the condition x¬= 0 is always
satisfied. Thus the loop never terminates then to terminate this process Ctrl-C is pressed.
4.8.3 If-Else-End Construction:
This decision making statement is used where the sequence of command required to
be conditionally evaluated based on a relational test. The general form of if-else-end
statement is
If expression
(command evaluated if true)
else
(command evaluated if true)
End
Example:
>> sweet = 5;
>> cost = sweet * 10
cost = 50
>> if sweet > 3 % 40% discount
cost = (1-40/100) * cost;
end
>> cost
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 40
cost = 40
4.8.4 Switch-Case Constructions:
These statement is used when a large number of commands are to be evaluated
depending on repeated use of an equality test with single common argument. The general
form is
Switch expression
Case test_expression1
(command)
Case { test_expression2, test_expression3, test_expression4}
(command)
Otherwise
(command3)
End
4.8.5 Try-Catch Block:
It provides user-controlled error-trapping capabilities. Thus it help in capturing error
found by MATLAB which gives the ability to control the error to the user. There are
generally expressed as
Try
(command1)
Catch
(command2)
End
4.9 Application areas of MATLAB:
• Technical computing
• Control system
• FPGA design
• Mechatronics
• Image and video processing
• Embedded systems
• Digital signal processing
• Computational finance
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 41
4.10 Key Features:
� High-level language for technical computing
� Development environment for managing code, files, and data
� Interactive tools for iterative exploration, design, and problem solving
� Mathematical functions for linear algebra, statistics, Fourier analysis, filtering,
optimization, and numerical integration
� 2-D and 3-D graphics functions for visualizing data
� Tools for building custom graphical user interfaces
� Functions for integrating MATLAB based algorithms with external
applications and languages, such as C, C++, Fortran, Java, COM, and Microsoft Excel.
4.11 MATLAB for Image Processing:
MATLAB stores images in the form of two dimensional matrices such that each
element in the matrix represent a single pixel of the image. True colour images uses three
dimension matrices in which first plane represent intensity of red pixels, second plane
represent intensity of green and the third plane represent blue.
Pixel coordinate is used to give information about the location in an image. In pixel
coordinate system image is addressed as framework of discrete elements arranged in top to
bottom and right to left. . Pixel coordinate system are expressed as (r, c).
Fig 4.10.1 Image coordinate systems (Mathworks)
(http://www.mathworks.com/access/helpdesk/help/toolbox/images/brcu_al-1.html)
Spatial coordinate is other type of coordinate system in which locations in a image are
expressed in terms of x and y, spatial coordinate are expressed as (x, y). The spatial
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 42
coordinate system is continuous where as pixel coordinate is discrete.
Fig 4.10.2 Spatial coordinate system (Mathworks)
4.12 Image Formats and Types Supported by MATLAB:
The following image formats are supported by MATLAB:
• BMP (Bitmap)
• HDF (Hubble Deep Field)
• JPEG (Joint Photographic Experts Group)
• PCX (Picture Exchange Format )
• TIFF (Tagged Image File)
• XWB
MATLAB supports four types of images
1. Binary images
2. Indexed Images
3. Grayscale Images
4. Truecolour Images
4.12.1 Binary Images: Its an array (matrices) of logical 0s and 1s reads as black and white
respectively. To test whether a given array is binary (logical), command islogical (k) is used.
If the output is 1 then it is binary otherwise not.
4.12.2 Indexed Images: These are also called as pseudo-colour images, it comprises of array
and colourmap matrix. The pixel values in the array are direct indices into a colour-map.
Array is represented as X and colour map which is an array of m-by-3 with a floating point
in the range of[0 1] is represented as map. The m-rows of colour-map (map) gives the
information about the values of components (red, green and blue) of a single colour. An
indexed image uses direct mapping of pixel values to colourmap values. The matrix map of
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 43
indexed images are m-3 array of class double containing floating point values in the range of
[0, 1].
4.12.3 Grayscale Images: Grayscale images are the data matrix are also called as intensity
ranges from [0, 255] and [0, 65535] for class unit8 and unit16 respectively. Grayscale images
are stored as individual matrix.
4.12.4 True Colour Images: These type of images are also called as RGB images,
represented as m-n-3 array of colour pixel. That is each pixel is triplet corresponding to red,
green and blue component. An true colour images regarded as a tack of three gray scale
images that when fed into the red, green and blue inputs of colour monitor, produces a colour
image on the screen. The components of true colour image are each of 8-bits which implies
true colour images are stored as 24-bits in graphical file format.[9]
Operation: MATLAB
command:
Convert between intensity/indexed/RGB format to binary
format. dither()
Convert between intensity format to indexed format. gray2ind()
Convert between indexed format to intensity format. ind2gray()
Convert between indexed format to RGB format. ind2rgb()
Convert a regular matrix to intensity format by scaling. mat2gray()
Convert between RGB format to intensity format. rgb2gray()
Convert between RGB format to indexed format. rgb2ind()
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 44
Chapter-5
Algorithms, Results and
Comparisons
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 45
5 Algorithms Result and Comparisons:
There are various algorithms developed for skin colour based face detection using
different colour models such as RGB colour model, HSV colour model and YCrCb. The
algorithm we proposed is the combination of all three or one can say part of all three colour
models. For clear understanding of the proposed algorithm, the algorithms of three colour
models and their characteristics must be clear.
5.1 Colour Models:
The purpose of colour model is to ease the specification of colour in some standard. In
general a colour model is a specification of a coordinate system, subspace present in colour
space is used to represent each colour by a single point.
5.1.1 RGB Colour Space:
RGB model is based on Cartesian coordinate system which is cube aid is shown
below. The cube has RGB values at the three corners, colours like cyan, magenta and yellow
and other three corners, black is at the origin and white is at farthest corner from the origin.
The gray scale lies on the line joining black and white. They are termed as ‘’additive
primaries’’ because the colours are added together to produce the desired colour. Due to high
correlation between colour components: red, green and blue also each of the component is
subjected to luminance effect from the light intensity of the environment, thus this suffers
dissatisfaction from many image processing applications. Also practically, this model is not
well suitable for describing colours in terms of human interpretation.(Sanjay Kr. Singh, D. S.
Chauhan, Mayank Vatsa, Richa Singh, 2003).
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 46
Fig 5.1.1.1 RGB cube (Paul Bourke, 1995)
5.1.2 HSI (Hue, Saturation, Intensity) Colour Space:
This colour model is ideal for hardware implementation. Practical human
interpretation is describe in term of Hue Saturation and Intensity (Brightness). HSI model
separates the intensity component from the colour carrying information, thus is an ideal tool
for developing image processing algorithm based on colour description that are natural and
intuitive to human. HSI is based on cylindrical coordinate, hue(H) is represented with an
angle 0, varying from 0 to 360, saturation(S) corresponds to radius varying from 0 to 1 and
finally the intensity(I) varies along the z axis with 0 being black and 1 being white.
When the value of S is 0, colour is a gray value of intensity. When the value of S is 1,
colour is on the boundary of tope cone base. The greater the saturation, the farther the colour
is from white / gray/ black. Chancing the value of colour will change the colour from red
which is at 00 to green which is at 1200, blue at 2400, and black to red at 3600. And when I =
0, the colour is black and therefore H is undefined. By varying the value of I, a colour can be
made darker or lighter. Shaded of the colour comes out when the value of I is adjusted while
maintaining S = 1.
5.1.3 YCrCb Colour Space:
This colour space has been defined to meet the increasing demand of digital
algorithms in handling video information and has become the widely used colour space in
digital videos. It has three components, two of them is of chrominance and one is of
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 47
luminance. This model comes in the family of television transmission of colour space along
with YUC and YIQ which are designed for analog space for PAL and NTSC systems.[14]
YCrCb colour model was developed in order to allow to transmit colour information
onto the television sets keeping in mind that existing black and white television sets continue
to display the images in tones of grey. And got the characteristic of isolating luminance and
colour information, thus used in many applications such as compression.
5.2 Algorithms:
5.2.1 Skin Colour Based Face detection in RGB Colour Space:
Skin colour is the most important facial feature and is unique because of its colour
ingredients. Skin colour pixels can be easily detected using normalized colour histogram
which is future normalized for any change in intensity on dividing of luminance. And thus
RGB vector is converted in to an [r, g ] vector of normalized colour which in turns offers a
fast mean of skin detection. This gives skin colour region which localizes face. As stated
early that RGB suffers from luminance effect but it still able to allows us to remove certain
colours that are clearly out of range of the normal skin colour. After studying and analysing
the various thresholds in RGB space it was found that following rule works well in removing
some redundant pixels which are labelled as non-face.[14]
0.836G – 14 < B <0.836G + 44 => Skin
0.79G – 67 < B < 0.78G + 42 => Skin
5.2.2 Skin Colour Based Face Detection in YCrCb Colour Space:
it has been found that skin colour pixels belonging to region exhibit similar Cb and Cr
values also it is found that skin colour model based on the Cr and Cb values can provide good
coverage of human races.
YCrCb signals which are created from the corresponding gamma-adjusted RGB source
using two defined constants KB and KR.
After the gamma correction and prior to scaling and offsets the YCrCb signals are
termed as Y1,PB and PR which are defined as shown below.
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 48
Here, the prime ′ symbols mean gamma correction is being used, thus R′, G′ and B′
nominally range from 0 to 1, with 0 representing the minimum intensity (e.g., for display of
the color black) and 1 the maximum (e.g., for display of the colour white). From this the
resulting luminance signal (Y) value will then have a nominal range from 0 to 1. And the
chrominance value (PB and PB) values will then have a nominal range from -0.5 to +0.5.
For representing signals in digital form, rounding off and scaling and offsets are
added. For instance scaling and offset applied to the Y′ component per specification results in
the value of 16 for black and the value of 235 for white when using an 8-bit representation.
The standard has 8-bit digitized versions of CB and CR scaled to a different range of 16 to
240. As a result, rescaling by the fraction (235-16)/(240-16) = 219/224 is mandatory when
doing colour matrixing or processing in YCbCr space, leading quantization distortions when
the successive processing is not executed using higher bit depths.
The form of Y′CbCr that was defined for standard-definition television use in
the ITU-R BT.601 (formerly CCIR 601) standard for use with digital component video is
derived from the corresponding RGB space as follows:
KB = 0.114
KR = 0.299
From the above constants and formulas, the following can be derived for ITU-R
BT.601. Analog YPbPr from analogue R'G'B' is derived as follows:
Y/ = 0.299 * R/ + 0.587 * G/ + 0.114 * B/
PB = -0.168739 * R/ - 0.331264 * G/ + 0.5 * B/
PR = 0.5 * R/ - 0.418688 * G/ - 0.81312 * B/
Digital Y′CbCr (8 bits per sample) is derived from analogue R'G'B' as follows:
Y/ = 16 + (65.481 * R/ + 128.553 * G/ + 24.966 * B/ )
PB = 128 + (-37.797 * R/ - 74.203 * G/ + 112.0 * B/
PR = 128 + (112.0 * R/ - 93.786 * G/ - 18.214 * B/
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 49
Note that the resultant signals range from 16 to 235; the values from 0 to 15 are
called footroom, while the values from 236 to 255 are called headroom.
Alternatively, digital Y′CbCr is derived from digital Rd Gd Bd (8 bits per sample)
according to the following equations:
Y/ = 16 + (65.481 * Rd + 128.553 * Gd + 24.966 * Bd ) / 256
PB = 128 + (-37.797 * Rd - 74.203 * Gd + 112.0 * B/d) / 256
PR = 128 + (112.0 * Rd - 93.786 * Gd - 18.214 * Bd ) /256
Y/ = 16 + (0.255 * Rd + 0.5021 * Gd + 0.0975 * Bd )
PB = 128 + (-0.148 * Rd – 0.368 * Gd + 0.439 * B/d)
PR = 128 + (0.439 * Rd – 0.368 * Gd - 0.071 * Bd )
The above equation are used in the coding part of the proposed algorithm. Cr and Cb
are used instead of PB and PR. After studying and experimenting with various thresholds,
finally found that the best results were found by using the following decree and otherwise
assume that it is NOT skin and may be removed from further consideration (Michael Padilla
and Zihong Fan, 2003).
102 < Cb < 128 => Skin
5.2.3 Skin Colour Based Face Detection in HSI Colour Space:
The HSV colour space is more intuitive and provides colour information in a manner
same as the humans think of colours and how artists typically mix colours. Hue along with
saturation provides the useful selective information regarding skin. The range of Hue for non
face region is found to be as shown below and otherwise it is assumed to be skin. HSI colour
model is represented using three
19 < H < 240 => Not Skin
HSI colour space is defined by using the RGB as scale as transformation from RGB
space to and HSI space (Gonzales and woods)
H = 3600 – α / 3600 if b > g
= α / 3600 other wise
Where α = cos-1 2R – G – B / (R2 + G2 + B2 – RG – GB – BR )1/2
S = 1-[min (R, G, B)] / I
I = (R + G + B) / 3
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 50
Colours are defined given an RGB colour cube. The colour cube is oriented such that
the value component is oriented along the gray axis, which runs from black to white.
HSV are defined purely with reference to some RGB space, they are not absolute
colour spaces: to specify a colour precisely requires reporting not only HSV values, but also
the characteristics of the RGB space they are based on, including the gamma correction in
use.
Computer science offers a few poorer cousins to these perceptual spaces that may also
turn up in software interface, such as HSV . They are easy mathematical transformations of
RGB, and they seem to be perceptual systems because they make use of the hue–
lightness/value–saturation terminology. But take a close look; don’t be fooled. Perceptual
colour dimensions are poorly scaled by the colour specifications that are provided in these
and some other systems. For example, saturation and lightness are confounded, so a
saturation scale may also contain a wide range of lightness (for example, it may progress
from white to green which is a combination of both lightness and saturation). Likewise, hue
and lightness are confounded so, for example, a saturated yellow and saturated blue may be
designated as the same ‘lightness’ but have wide differences in perceived lightness. These
flaws make the systems difficult to use to control the look of a colour scheme in a systematic
manner. If much tweaking is required to achieve the desired effect, the system offers little
benefit over grappling with raw specifications in RGB (Cynthia A. Brewer, 1999).
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 51
Fig 5.2.3.1:HSV-cone
5.3 Proposed Algorithm For Face Detection:
In the proposed algorithm, it is assumed that by combining the detected region from
all the three algorithm, skin region is extracted. After combining the three algorithms and
extracting skin colour, following this face is detected by edges using edge detection which
draws boundary across the face region. It is something like if one algorithm detect the skin
colour for an image and others two algorithm fails and gives the false result, even then the
face is extracted by using the combination algorithm. This supposition of combining
algorithm, is based on the basic idea of Venn diagram from set theory. Suppose the result
from RGB colour is ‘’A’’, results from YCbCr colour space is ‘’B’’ and the result from HSI
colour space is ‘’C’’ and if any of the result contains a skin image then the union of the three
will surely be a skin image. This can be understood by the following example: : let, A = {a, b,
c, d}, B = {b, e} and C = {b, f}. If “a” = “skin image” then skin image is present only in A
and the next two are failed in extracting the skin colour then the union of the three U = (A U
B U C) ={a, b, c, d, e, f} hence this algorithm can detect skin colour or else if we take ‘’e’’ or
‘’f’’ as skin image then also the union (U) will have skin image in it also this is applicable if
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 52
all the three (A, B, C) has skin image. Hence in all sense the skin region are going to be
detected (Sanjay Kr. Singh, D. S. Chauhan, Mayank Vatsa, Richa Singh, 2003).
After extracting the skin region, facial features such as eyes, nose and mouth are
extracted. The image obtained after applying skin colour information is given to binarization
i.e., it is transformed to gray-scale image and then to a binary image by applying suitable
threshold. This is done to eradicate the hue and saturation values and consider only the
luminance part. This luminance part is then transformed to binary image with some threshold
because the features we want to consider further for face extraction are darker than the
background colours. Opening and closing operations are performed to filter out noise. The
morphological close operation is nothing but a dilation followed by an erosion, and the
morphological open operation is an erosion followed by dilation, these operations are
performed to remove holes.
5.4 Experimental Results:
In this section, images resulted from the proposed algorithm are shown with two
examples (Input-1 and Input-2). Each example has six images as outputs: segmented image,
applying RGB to segmented image, image after edge detection, binarlization, noise removed
by performing morphological operation, output image (Face detected).
Segmentation: Segmentation is the main part of the algorithm, gives the minute description of
the pixel. So that the suitable operations can be performed. The operation performed in the
proposed algorithm are based on the three colour spaces used.
Applying RGB: Convert image resulted from segmentation to colour image.
Edge detection: This is done in order to trace the edges and separate the regions containing
face with that of non-face.
Binarlization: It converts the image into binary image.
Morphological operation: As explained earlier, it perform open and close operation to remove
noise.
Output image: Shows the outcome of the algorithm.
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 53
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 54
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 55
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 56
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 57
Chapter-6
Discussion, conclusion and Future
Work
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 58
6.1 Discussion:
Face detection using image skin colour methods have shown reasonable advancement
in the area of computer vision and biometric image analysis. The combination with machine
learning techniques has resulted more accurate and efficient.
During current study we revealed the area of face detection. A comprehensive
survey is carried out to understand the problems and opportunities in context of computer
vision and machine learning. The area demands a lot of research to achieve a higher level of
accuracy and to gain computational efficiency. Different approaches and techniques are
discussed. The pros and corn helped to understand the problem and leads to proper method
selection according to the nature of image database.
Skin colour based face detection proposed in this thesis is the most effective than that
of the face detection methods based on other facial features. Any combination of facial
features along with skin colour uplift the performance to high level. As explained there exits
various methods and in each method there exist various techniques, the best method based on
the experiments results is consider as features based method in that the best technique is
using skin colour.
The three different colour space RGB, HSV and YCrCb are briefly discussed and from
the discussion it is concluded that using all three colour spaces will be more advantageous.
Thus the thesis covers all the factors that contribute to the effectiveness of the face detection
process from all three colour space available.
6.2 Conclusion:
From the work carried out on this thesis, it is concluded that it is very difficult and still a
challenging task to build up an automatic face detection method that is works effectively
in all situation whether the image consists of different face poses, bad image condition or
image effected by illumination. Researchers are still struggling hard to find out the
perfect solution to this complicated problem. But their exist, the different face detection
method to overcome different problems of face detection. For instance to overcome the
problem of lighting (illumination) RGB colour space is not used instead HSV or YCrCb
are used. Some methods or techniques shows good result for particular problem or
application and others shows good result for different application. So it may not be
insane to say, that researchers still have lot of work to do under face detection.
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 59
6.3 Future work:
The goal, robust face detection and recognition, is still far. The case study discussed
during the current work is interesting for further research and development activities. Due to
a variations and a huge number of images is perfect platform for experimenting algorithms.
Face detection based on skin colour studied during this thesis work can be
implemented to other biometric systems. For instance systems involved detection and
recognition of iris, ear, palm, signature, and finger prints etc need comprehensive image
analysis. These techniques are very helpful while analysing patterns and classification.
As discussed in chapter two the face detection has been a challenging task due to
number of factors like poses, image condition and illumination. Most of the exiting
methods and algorithms are unable to provide solutions to these problems of variations. A
considerable attention is required to develop efficient methods that can work with and
overcome to such problems. The research in this area is very actively going on and the
current study will help for laying solid foundation for it. Most of the researchers are
performing experiments by using different colour spaces and even different combination of
this colour spaces as this thesis used all three colour spaces in performing experiments.
For more effective and accurate face detection other facial features apart from the skin
colour such as nose, mouth and eyes can be used which really enhance the performance. Any
one of the facial feature or the combination of any two or all can be consider to increase the
effectiveness of the face detection process (K.V Kale, S.C Mehrotra and RR Manza, 1999).
Case study-2 describe in this thesis uses the morphological operations such as opening
and closing which help in reducing the noise after the segmentation is carried out. This
morphological operations are also utilized in the code of this thesis, which also reduces the
noise level up to satisfactory level.
It is found that MATLAB is the only effective tool or the platform of carrying outface
detection process, thus this becomes a problem for many students and young researches as
there is no comprehensive platform for performing experiments. All have to rely on
MATLAB or one has to develop code from other language like C, C++ or JAVA. The
complex mathematics and statistical equations demand hung amount of time and efforts.
There are few C++ libraries provided by some research groups free of cost for further
research in the area of image processing but still they are insufficient for building up a
platform for experiments in the area of face detection and recognition.
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 60
Chapter-7
References
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 61
References:
1. Bryan S.Morse. (2000). Thresholding. Available:
http://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/MORSE/threshold.pdf
2. Cynthia A. Brewer (1999). Color Use Guidelines for Data Representation.
Alexandria: American Statistical Association. p55-56.
3. Dr. Arti Khaparde*, Sowmya Reddy.Y Swetha Ravipudi. (2007). Face
Detection Using Color Based Segmentation and Morphological Processing. Available:
http://www.rimtengg.com/iscet/proceedings/pdfs/misc/147.pdf. Last accessed 21 sep 201
4. Filareti Tsalakanidou , Sotiris Malassiotis 2 and Michael G. Strintzis Read
more: Face Recognition - Face Detection, Global Approaches for, Feature Based Techniques,
Problems and Considerations, Conc. (2009).Face Recognition. Available:
http://encyclopedia.jrank.org/articles/pages/6741/Face-Recognition.html. Last accessed 14
Jul 2010.
5. FMI. (). RGB image components. Available:
http://www.zamg.ac.at/eumetrain/CAL_Modules/CALRGB/rgb1_2a.htm. Last accessed 23
Aug 2010.
6. Ian Flitcroft. (2007). refraction of light [online]. Last accessed at 23 july 2010
at: http://journeybystarlight.blogspot.com/2007/06/what-am-i-confused-life-of-light.html
7. Javidi, Bahram. Image Recognition and Classification : Algorithms, Systems,
and Applications. New York, NY, USA: Marcel Dekker Incorporated, 2002.
8. Jon A. Web. (2010). Why Face Detection?. Available:
http://code.google.com/p/jjil/wiki/FaceDetectionWithJJILIntro. Last accessed 15 july 2010.
9. Khairul Anuar Ishak. (2008). MATLAB - The Language Of Technical
Computing. Available:
http://staff.iiu.edu.my/azami/MATLAB%20tutorial%20of%20fundamental%20programming.
10. Kristian Sandberg. (2005). Introduction to image processing in Matlab
1. Available:
http://amath.colorado.edu/courses/5720/2000Spr/Labs/Worksheets/Matlab_tutorial/matlabim
pr.html
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 62
11. K.V Kale, S.C Mehrotra and RR Manza (1999). Advance in computer vision
and information technology. New Delhi, India: I.K Innternational
12. Lindsay coome. (2005). Thresholding an Image. Available:
http://www.webdesign.org/raster-graphics/paint-shop-pro/thresholding-an-image.3750.html.
Last accessed 30 Aug 2010.
13. Michael Padilla and Zihong Fan. (2003). Automatic Face Detection Using
Color Based Segmentation and Template/Energy Thresholding. Available:
http://www.stanford.edu/class/ee368/Project_03/Project/reports/ee368group16.pdf
14. Muhammad Tariq Mahmood. (2006). Face Detection by Image
Discriminating. Available:
http://www.bth.se/fou/cuppsats.nsf/all/6c509ae86a297ca4c12571d300512cac/$file/DVD009-
MasterThesisReport.pdf. Last accessed 10 Aug 2010.
15. Oz. Image restoration [online] last accessed 2 july2010 at
http://ozlouisville.com/professional-photo-printing/restoration/
16. Paul Bourke. (1995). RGB colour space. Available:
http://local.wasp.uwa.edu.au/~pbourke/texture_colour/colourspace/. Last accessed 13 Aug
2010.
17. P. Sinha.(1994) Object Recognition via Image Invariants: A Case Study.
Investigative Ophthalmology and Visual Science, 35, pp. 1.735-1.740.
18. Rafael C. Gonzalez, Richard E. Woods (2001). Digital Image Processing. 2nd
ed. New Jersey: Prentice-Hall . p75-103.
19. Robert Evans. (2009). MATLAB Desktop. Available:
http://docs.cs.cf.ac.uk/html/606/node5.html. Last accessed 30 Aug.
20. Sanjay Kr. Singh, D. S. Chauhan, Mayank Vatsa, Richa Singh. (2003). A
Robust Skin Color Based Face Detection Algorithm. Tamkang Journal of Science and
Engineering,. 6 (4), p227-234.
21. Shehrzad Qureshi. (2009). edge detection. Available:
http://www.squreshi.com/images.html. Last accessed 23 Aug 2010.
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 63
22. The MathWorks, Inc.. (2010). MATLAB - The Language Of Technical
Computing. Available: http://www.mathworks.com/products/matlab/. Last accessed 28 Jul
2010.
23. Tomás Castelazo. (2006).Gray-level [online]. Last accessed 16 Aug 2010 at:
http://commons.wikimedia.org/wiki/File:Gray_scale.jpg
24. William J.Palm (2009). Introduction to MATLAB for engineers. 3rd ed. New
York: Mc Gram Hill. p113-138.
25. WenmiaoLu and ShaohuaSun. (2000). Face detection in colour
images.Available: http://www.scribd.com/doc/24842801/Face-Detection-in-Color-Images).
Last accessed 8 sep 2010.
26. Yang, Ming-Hsuan, “Face detection and gesture recognition for human-
computer interaction”, Boston, Mass. ; London : Kluwer Academic 2001.
27. Zia-ur Rahman. (2001).Eyes [online]. Last accessed 28 july 2010
at:http://dragon.larc.nasa.gov/medical_imaging.pdf
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 64
Chapter-8
Appendixes
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 65
8.1 MATLAB Code:
% function segment=skin(I);
close all;
clear all;
clc;
[file, pathname] = uigetfile('*.jpg','Load Image');
cd(pathname);
I=imread(file);
% I=imcrop(I);
imwrite(I,'org.jpg');
I=imresize(I,[255 255]);
qq=I;
imshow(I);
% I=imresize(I,[256,256]);
I=double(I);
[hue,s,v]=rgb2hsv(I);
cb = 0.148* I(:,:,1) - 0.291* I(:,:,2) + 0.439 * I(:,:,3) + 128;
cr = 0.439 * I(:,:,1) - 0.368 * I(:,:,2) -0.071 * I(:,:,3) + 128;
[w h]=size(I(:,:,1));
for i=1:w
for j=1:h
if 140<=cr(i,j) & cr(i,j)<=165 & 140<=cb(i,j) & cb(i,j)<=195 & 0.01<=hue(i,j)
& hue(i,j)<=0.1
segment(i,j)=1;
else
segment(i,j)=0;
end
end
end
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 66
imshow(segment);
title('segmented');
im(:,:,1)=I(:,:,1).*segment;
im(:,:,2)=I(:,:,2).*segment;
im(:,:,3)=I(:,:,3).*segment;
figure;imshow(uint8(im));
title('colour-Segmented image');
sk=im;
im=rgb2gray(im);
BWa = edge(im,'sobel');
figure,imshow(BWa);
title('segmentated with edge detection,gray-scale image ');
im=imresize(im,[255 255]);
X = imcomplement(im);
figure,imshow(X);
title('binarlizatiom');
bwa = bwareaopen(X,90);
se = strel('disk',5);
bww = imclose(bwa,se);
bwb = imclose(bww,se);
figure,imshow(bwb);
title('After morphological operations');
BWa = edge(bwb,'sobel');
AUTOMATIC FACE DETECTION
Sheffield Hallam University Page 67
figure,imshow(BWa);
dim = size(BWa)
col = round(dim(2)/2)-90;
row = min(find(BWa(:,col)))
boundary = bwtraceboundary(BWa,[row, col],'N');
hold on;
imshow(qq)
BW_filled = imfill(BWa,'holes');
boundaries = bwboundaries(BW_filled);
[m n]=size(boundaries);
w=(m+n);
for k=1:(w-1)
b = boundaries{k};
plot(b(:,2),b(:,1),'g','LineWidth',1);
title('Face detected');
end
snr1(file)