Download - Prrofing for Doc
-
8/9/2019 Prrofing for Doc
1/169
1
-
8/9/2019 Prrofing for Doc
2/169
ii
ABSTRACT
Face recognition is a wide area which is currently studied by lot of researchers and
vendors but still there are some areas that even researchers have not touched. Partial
face recognition can be considered as one of these areas. Few years back researchers
considered that partial face recognition is an unrealistic approach because there is
less information which can use to recognize a person. But after the research
conducted by researchers like Sato et al (1998), it is proven that partial face region
also contain some amount of information that can use to recognition.
By conducting this project, I tried to investigate how far partial face regions involve
recognizing individual. The artifact allows users to input probe images and then the
system will determine individuals face whose belongs that particular face region.
Even though it appears to be a simple process the system should perform intensive
amount of work to achieve it.
A domain research, which has conducted to identify and study about the problem,
related to face recognition, similar system. Then by analysing it I decided to the area
that needed constrain in system research.
During system research researcher studied about the fundamentals about the image
processing, face detection techniques and face recognition techniques. Then after
analysing them it decided to select appropriate techniques to extract partial face
regions and recognize individuals.
The document contains introduction, domain research, system research, project
development plan, requirement specification and technical investigation, which
contain details about above-mentioned areas.
Finally appropriate image processing techniques, face detection techniques and face
recognition techniques were selected and justified along with selection of the project
developing methodologies and project developing platforms.
-
8/9/2019 Prrofing for Doc
3/169
iii
ACKNOWLEDGEMENT
First I would like to express my appreciation to my supervisor Ms Oshini Jayatunga
and my assessor Mr. Gamindu Hemachandra for their guidance, suggestions and
support that given to me during my project. Without their directions I will not able to
accomplish this project.
Special thanks to my family without their help guidance and love I would not able to
make this far. Then I like to thank my colleagues for their advice and guidance about
the project. Special thanks go to my friend Tharindu Roshantha and Andrew Jebaraj
for being good friends to me and help me to proof read my document and correcting
my mistakes.
Finally, I would like to thank to APIIT Sri Lanka for facilitating me to by providing
me necessary educational help form well qualified lecture staff, library and lobotomy
facilities which need to make this project success. Especially thanking the library, for
facilitating me to get the necessary reading materials which were needed to my
project research.
-
8/9/2019 Prrofing for Doc
4/169
iv
TABLE OF CONTENT5S
ABSTRACT ................................................................................................................. iiACKNOWLEDGEMENT ......................................................................................... iiiTABLE OF CONTENT5S .......................................................................................... ivLIST OF FIGURES .................................................................................................... ixLIST OF TABLES ...................................................................................................... xiLIST OF ABBREVIATION ...................................................................................... xiiCHAPTER 1 ................................................................................................................ 1
1 INTRODUCTION ................................................................................................... 1
1.1Project Background ............................................................................................ 11.2Understanding the Problem ................................................................................ 21.3Sub Problems ...................................................................................................... 31.4Project Objectives ............................................................................................... 41.5Proposed Solution ............................................................................................... 41.6Project Scope ...................................................................................................... 5
1.6.1Functionality of the Artefact ....................................................................... 51.7Constrains ........................................................................................................... 71.8Assumptions ....................................................................................................... 7
CHAPTER 2 ................................................................................................................ 82 Domain Investigation .............................................................................................. 8
2.1Outline of the generic face recognition system .................................................. 82.2Similar systems ................................................................................................... 9
2.2.1Google Picasa .............................................................................................. 92.2.2APPLE IPhoto ........................................................................................... 102.2.3Face.Com automatic face book photo tagger ............................................ 11
2.3Problem of face recognition systems ................................................................ 12
-
8/9/2019 Prrofing for Doc
5/169
v
2.3.1Pose variance ............................................................................................. 122.3.2Lighting/Illumination variance .................................................................. 132.3.3Different facial expression......................................................................... 132.3.4Different facial occlusions ......................................................................... 132.3.5Individual characteristics ........................................................................... 142.3.6Scale Invariance and robustness ................................................................ 14
CHAPTER 3 .............................................................................................................. 153 System research and investigation ........................................................................ 15
3.1Fundamental of Image processing .................................................................... 153.1.1Image classifications.................................................................................. 153.1.2Image segmentation ................................................................................... 163.1.3Edge detection ........................................................................................... 19
3.2Face recognition approaches............................................................................. 243.2.1Three dimensional face recognition .......................................................... 243.2.2Two dimensional face recognition ............................................................ 253.2.3Geometric based approaches ..................................................................... 253.2.4Appearance based approach ...................................................................... 26
3.3Face region extraction approaches.................................................................... 283.3.1Template matching .................................................................................... 283.3.2Haar like features - Viola-Jones face detector ........................................... 323.3.3Facial feature detection using AdaBoost with shape constraints .............. 353.3.4Justification on selected technologies ....................................................... 37
3.4Face recognition methods ................................................................................. 383.4.1Elastic branch graph matching approach ................................................... 383.4.2Appearance based subspace methods ........................................................ 413.4.3Support vector machine kernel correlation feature analysis method ......... 423.4.4Eigenface Approach .................................................................................. 43
-
8/9/2019 Prrofing for Doc
6/169
vi
3.4.5Justification on selected technique ............................................................ 563.5Approach to the solution ................................................................................... 59
3.5.1Approach to face region extraction ........................................................... 593.5.2Approach to face region identification ...................................................... 593.5.3Approach to face region recognition ......................................................... 59
Chapter 4 .................................................................................................................... 604 Requirement Specification .................................................................................... 60
4.1System requirements ......................................................................................... 604.1.1Functional requirements ............................................................................ 604.1.2Non Functional requirements .................................................................... 60
4.2Resource requirements...................................................................................... 614.2.1Hardware Requirements ............................................................................ 614.2.2Software Requirements.............................................................................. 62
CHAPTER 5 .............................................................................................................. 645 Development methodologies ................................................................................. 64
5.1Waterfall ........................................................................................................... 645.2Incremental Model ............................................................................................ 655.3Hybrid model .................................................................................................... 665.4Prototyping ....................................................................................................... 675.5Justification for selection method ..................................................................... 695.6Developing stages ............................................................................................. 69
CHAPTER 6 .............................................................................................................. 726 Technical research and investigation .................................................................... 72
6.1Developing platforms ....................................................................................... 726.1.1Dot.Net ...................................................................................................... 726.1.2Sun Java ..................................................................................................... 766.1.3MATLABMathWorks ........................................................................... 77
-
8/9/2019 Prrofing for Doc
7/169
vii
6.2Image processing APIs ..................................................................................... 786.2.1OpenCV ..................................................................................................... 786.2.2EmguCV .................................................................................................... 786.2.3Aforge.Net ................................................................................................. 78
6.3Math Related Libraries ..................................................................................... 796.4Developing Approaches.................................................................................... 81
6.4.1Approach One ............................................................................................ 816.4.2Approach Two ........................................................................................... 816.4.3Approach Three ......................................................................................... 826.4.4Approach Four ........................................................................................... 82
6.5Justification for selection of programming environment.................................. 82CHAPTER 7 .............................................................................................................. 857 System Design ....................................................................................................... 85
7.1Design Approach .............................................................................................. 867.1.1Programming ApproachObject Oriented Approach .............................. 867.1.2Design architecture .................................................................................... 86
7.2Use Case Diagram ............................................................................................ 877.3Use case descriptions ........................................................................................ 88
7.3.1Add New Record ....................................................................................... 887.3.2View Record .............................................................................................. 897.3.3Train Image Set ......................................................................................... 907.3.4Find Match ................................................................................................. 917.3.5Set Configuration ....................................................................................... 927.3.6Modify Record ........................................................................................... 93
7.4System Overview .............................................................................................. 957.5Activity Diagrams ............................................................................................. 96
7.5.1Add Record ................................................................................................ 96
-
8/9/2019 Prrofing for Doc
8/169
viii
7.5.2View Record .............................................................................................. 977.5.3Training Image Space ................................................................................ 987.5.4Find Match ................................................................................................. 997.5.5Automatic Face Region Verification ....................................................... 1007.5.6Pre-processing/Normalization ................................................................. 100
7.6Class Diagrams ............................................................................................... 1017.7Classes Description ......................................................................................... 102
7.7.1Data management related classes ............................................................ 1027.7.2Application logic related classes ............................................................. 1027.7.3GUI Related Classes ................................................................................ 1037.7.4Other supportive classes .......................................................................... 103
CHAPTER 8 ..............................................................Error! Bookmark not defined.8 TESTING AND EVALUATION ........................................................................ 105
8.1Unit testing...................................................................................................... 1068.2Integration testing ........................................................................................... 1077.3 System Integration testing .............................................................................. 1077.4 Performance testing ..........................................Error! Bookmark not defined.7.5 Accuracy testing ............................................... Error! Bookmark not defined.7.6 Sample Unit Testing Test plans for eye region extraction module ...........Error!
Bookmark not defined.REFERENCE ........................................................................................................... 132APPENDICES .............................................................................................................. iA.Image Processing Techniques ................................................................................. iiB.Eigenface Step by Step (Formulas) ........................................................................ vi
-
8/9/2019 Prrofing for Doc
9/169
ix
LIST OF FIGURESFigure 1.1: Altered faces. ............................................................................................. 3Figure 1.2: Normal face recognition process ............................................................... 3Figure 1.3: Proposed solution overview ...................................................................... 4Figure 2.1: Configuration of generic face recognition ................................................. 8Figure 2.2: Google Picasa ............................................................................................ 9Figure 2.3: APPLE I photo face recognition .............................................................. 10Figure 2.4: Face.com Photo Tagging ......................................................................... 11Figure 2.5: Face Book Photo Tagging ....................................................................... 11Figure 2.6: Different pose Audio Visual Technologies ............................................. 12Figure 2.7: Different illuminations ............................................................................ 13Figure 2.8: Different facial expression....................................................................... 13Figure 2.9: Different facial occlusions ....................................................................... 13Figure 3.1: Image representation in plane, As a matrix ............................................. 15Figure 3.2: Image segmentation approach ................................................................. 16Figure 3.3: Low noise object/background image histogram ...................................... 18Figure 3.4: Canny edge detection process.................................................................. 21Figure 3.5: Masks ....................................................................................................... 22
Figure 3.6: Sober Template ........................................................................................ 22Figure 3.7: Geometrical features ................................................................................ 26Figure 3.8: Template matching strategy..................................................................... 27Figure 3.9: Template matching .................................................................................. 28Figure 3.10: Template ................................................................................................ 29Figure 3.11: Test one Grey-Level Template matching .............................................. 29Figure 3.12: Test two Grey-Level Template matching .............................................. 30Figure 3.13: Test three Grey-Level Template matching ............................................ 30Figure 3.14: Test four Grey-Level Template matching ............................................. 30Figure 3.15: Test Five Grey-Level Template matching ............................................. 31Figure 3.16: Test Five template is not available ........................................................ 31Figure 3.17: Rectangle Features ................................................................................. 33Figure 3.18: The features selected by AdaBoost ....................................................... 33Figure 3.19: Cascade of classifiers............................................................................. 34Figure 3.20: Features selected by AdaBoost .............................................................. 36
http://c/Users/jumbo/Desktop/Mid%20Dox%20Modifyed.doc%23_Toc267629895http://c/Users/jumbo/Desktop/Mid%20Dox%20Modifyed.doc%23_Toc267629895http://c/Users/jumbo/Desktop/Mid%20Dox%20Modifyed.doc%23_Toc267629895http://c/Users/jumbo/Desktop/Mid%20Dox%20Modifyed.doc%23_Toc267629895 -
8/9/2019 Prrofing for Doc
10/169
x
Figure 3.21: Elastic branch graph matching face recognition.................................... 39Figure 3.22: Face image transformation .................................................................... 44Figure 3.23: Distribution of faces in image space...................................................... 44Figure 3.24: Faces in face space ................................................................................ 45Figure 3.25: Overview of Eigen face Approach ........................................................ 46Figure 3.27: Representation of A x =b. ..................................................................... 47Figure 3.26: Vector .................................................................................................... 47Figure 3.28: Representation of A v = v. .................................................................. 48Figure 3.29: Training Images ..................................................................................... 50Figure 3.30: Example images from the ORL database .............................................. 53Figure 3.31: Mean face obtained from the ORL database ......................................... 53Figure 3.32: Eigenfaces .............................................................................................. 54Figure 5.1: SDLC Model ........................................................................................... 65Figure 5.2: Incremental model ................................................................................... 66Figure 5.3: Hybrid of waterfall and incremental model ............................................. 66Figure 5.4: Prototyping model ................................................................................... 67Figure 5.5: Evolutionary Prototyping ........................................................................ 68Figure 6.1 : Dot net Framework ................................................................................. 73Figure 6.2: Dot net Code Execution process .............................................................. 73Figure 6.3: Java Architecture ..................................................................................... 77Figure 7.1: Use Case Diagram ................................................................................... 87Figure 7.2: System Architecture ............................................................................... 95Figure 7.3: Activity DiagramAdd Record .............................................................. 96Figure 7.4: Activity DiagramView Record ............................................................ 97Figure 7.5: Activity DiagramTraining Image Space .............................................. 98Figure 7.6: Activity DiagramFind Match............................................................... 99Figure 7.7: Activity Diagram - Automatic Face Region Verification ..................... 100Figure 7.8: Activity Diagram - Pre-processing/Normalization ................................ 100Figure 7.9: Class Diagram ....................................................................................... 101
http://c/Users/jumbo/Desktop/Mid%20Dox%20Modifyed.doc%23_Toc267629920http://c/Users/jumbo/Desktop/Mid%20Dox%20Modifyed.doc%23_Toc267629920http://c/Users/jumbo/Desktop/Mid%20Dox%20Modifyed.doc%23_Toc267629920http://c/Users/jumbo/Desktop/Mid%20Dox%20Modifyed.doc%23_Toc267629920 -
8/9/2019 Prrofing for Doc
11/169
xi
LIST OF TABLES
Table 3.1: Detection rates for various numbers of false positives on the test set ...... 34Table 3.2: Comparison of face detection approach .................................................... 38Table 3.3: Recognition results between different galleries using EGBM .................. 40Table 3.4: Recognize and Detect Faces ..................................................................... 56Table 6.1: Comparison of Programming languages ................................................... 76Table 6.2: Comparison of matrix libraries ................................................................. 81Table 7.1: Use case description - Add New User ...................................................... 89Table 7.2: Use case descriptionView Record ........................................................ 90Table 7.3: Use case descriptionTrain Image Set .................................................... 91Table 7.4: Use case descriptionFind Match ........................................................... 92Table 7.5: Use case descriptionSet Configuration ................................................. 93Table 7.6: Use case descriptionModify Record ..................................................... 94Table 8.1: Test Case : Eye region detection ...............Error! Bookmark not defined.Table 8.2: Test Name : Eye region Extraction ...........Error! Bookmark not defined.
-
8/9/2019 Prrofing for Doc
12/169
xii
LIST OF ABBREVIATION
1D One Dimensional Vector
2D Two Dimensional
3D Three Dimensional
ADO Activex Data Objects
API Application Programming Interface
ASP Active Server Page
CFA Correlation Feature Analysis
CPU Central Processed Using
EBGM Elastic Brunch Graph Matching
GUI Graphical User Interface
KCFA Kernel Correlation Feature AnalysisLDA Linear Discriminate Analysis
LINQ Language-Integrated Query
MACF Minimize Average Correlation Filter
MSE Mean Square Error
NSTC National Science And Technology Counsel
OOP Object Oriented Programming
PC Personal Computer
PCA Principal Component Analysis
SDLC System Development Life CycleSQL Structured Query Language
SVM Support Vector Machine
VB Visual Basic
-
8/9/2019 Prrofing for Doc
13/169
-
8/9/2019 Prrofing for Doc
14/169
2
researchers and vendors for face recognition. Still there are lot of areas to improve in
automated face recognition.
1.2Understanding the ProblemHowever, when it come to the real world, it might not be possible to capture a full
frontal picture of a face at all the times in uncontrolled environments. Even though
there are many face recognition systems available, most of these work in optimal
conditions. Especially without full frontal face, these systems fail to recognise a face.
As a result of this most of the systems cannot give an accurate face match result.
Because of that, there can be lot of complications in identifying a person in an
image.
As an example in the news article written by Willing (2003) indicates failures of
implementing face recognition systems in airport at Logan USA. Furthermore,
Willing (2003) stated that there were similar project that had to shut down because of
incapability of giving accurate face matches due to different reasons.
One of the reason identified was criminals intentionally alter their appearance using
disguises to defraud law enforcement and the public. Also because of the influences
of different cultures and environment factors sometimes it is not possible to expose
full face. In those situations normally, face recognition approaches fail to give well
accurate result.
There are various reasons for the failings of the current systems. One possible reason
is they cannot identify disguised faces comparing with full faces because current
approaches are not capable of identifying individuals using partial face regions
which have fewer characteristics in small face region comparing with full faces.
-
8/9/2019 Prrofing for Doc
15/169
3
Figure 1.1: Altered faces.
(a) Muslim girl wearing veil [Source: Chopra, 2007]. (b) Women wearing masks
[Source: Tarlow, 2007]. (c) Person wearing sunglass [Source: University of Buenos
Aires ,n.d].
As mentioned before most of face recognition solutions cannot recognise
individuals faces, which are intestinally occlude. Therefore, it is understood that an
affective and robust face recognition system should be capable of identifying
partially occluded faces including other regular features.
1.3Sub ProblemsAutomated face recognition is a sequence of process. As mentioned in
MyHeritage.com (2006) the following diagram shows how face recognition systems
are functioning.
Figure 1.2: Normal face recognition process
By understanding the above, there are lot of sub steps performed within the above
steps. Although there are lot of approaches for full-face recognition, the proposed
solution deviates from the regular face recognition approaches. Therefore, when it
takes the suggested solution the overall process of the face recognition applies but
internal process sequence is different from the regular face recognising process.
Acquire Images
form video or
still camera
Detect and
extract face area
form the image
Match extracted
face against images
in database
-
8/9/2019 Prrofing for Doc
16/169
4
1.4Project ObjectivesThis face recognition system will identify individuals based on characteristics of
separate face segmentations and the objectives of the project as follows.
Investigation of unique face features of eye, nose and mouth regions for
recognises individuals.
When it come to separate face regions there are less unique features
that help to identify individuals. Identifying unique features of the
individuals has being archiving throughout this project.
Improve capabilities of the detecting features of local segmentations of face
It is necessary to find the efficient algorithm to extract features of the
face segmentations.
Implement robust, efferent face recognition system based on facts found in
the research.
A thorough research being carried on face recognition techniques and
available algorithm on partial face recognition and choose a
appreciate method and implement face recognition system based in it.
1.5Proposed Solution
The purposed solution takes a segment of a face (eye region, nose region or mouth
region) at a time and identifies which has submitted region. Based on the input
region it will extract the features that are unique to each region. Then it will extract
particular face region form the faces in database. After that, it will match the
features, which have extracted from both two-face region.
Face Region
Identifier
Face Region
Extractor
Region
Feature
Extractor
Database
face Features
Submitted
Image face
Face
Matcher
Results
Figure 1.3: Proposed solution overview
-
8/9/2019 Prrofing for Doc
17/169
5
1.6Project ScopeThe Scope of this Project is to develop a system that allows recognise individuals
using partial regions of face. Initially the project is based on recognise individuals
using submitted face regions.
The scope is as follows:
This project is only focus on identifying individuals using only eye region
but as external functionality, it will try to implement it to mouth region.
The all images have taken in 0-degree camera angle, which mean all images
are frontal view images, and taken in controlled environment.
The eyes should be open and mouth should close in the faces including both
input image and images in database. All faces should be in neutral mood.
It assumed that input image of the system (The image to be recognised) is
pre scaled and it is not necessary to scale back.
Furthermore, the solution will not work on recognising in following situations
Disguised faces, which cannot use for extract at least one of eye region or
nose region or mouth region.
Different poses of face regions.(taken in different angles)
Low quality images ( less resolutions, etc)
Different facial expressions
1.6.1Functionality of the Artefact1.6.1.1 Core FunctionalityCore Functionality of the artefact can be identified as follows. The main
functionality of the solution is recognize individuals using partial face segments.
During this research and implementation, it focuses on face recognition using eye
region of a person. Because of that which will mention as partial face recognition as
onward here consider as partial face recognition using eye region.
-
8/9/2019 Prrofing for Doc
18/169
6
Furthermore, it can be divide as follows
Input identification
This will identify whether input region is a face region or not and if it is a
face region what is the region.
i.e.
User input an eye region then first at all it will check whether it is a face
region if yes it will identify what is the face region.
Face region detection
This will detect the particular face region of full frontal faces in database and
extract particular face region from the faces
Face match
This will match submitted face region with extracted face regions in database
and provide match result.
1.6.1.2Extra functionality Face recognition using mouth region
Full frontal face recognition
Detect and extract different face regions from submitted full
frontal face and recognize individuals based on partial face
regions.
The proposed artefact will develop a standalone application, which runs on PC and a
project report, which include the details of the project.
-
8/9/2019 Prrofing for Doc
19/169
7
1.7Constrains1. Camera view
2. Inadequate resolution of the images
3. Lighting condition
4. Scale of the image
5. Distance between camera and person
6. Facial expressions
7. Occlusion
8. Ageing
1.8Assumptions1. The input images, which submitted to system and images in database are in
same scale.
2. The input image has extract manually based on given dimensions.
3. Images have taken in controlled environment, which avoids different lighting
conditions, variance of view, variance distance between camera and person.
4. All images have constant resolution.
5. All faces have taken in neutral facial experience.
6. The faces areas taken into recognize have not occluded.
7. It is assume that all images taken are in same age because of that the distance
of each features of the face does not change.
-
8/9/2019 Prrofing for Doc
20/169
8
CHAPTER 2
2 DOMAIN INVESTIGATION2.1Outline of the generic face recognition systemTypical face recognition systems recognize faces based on various factors and
technologies. Although different face recognition systems use different approaches,
most of them perform key main steps, which are common to most face recognition
approaches.
As mentioned by Zhao et al.(2003), a generic face recognition system consist three
main processing steps. Face detection that involve detecting and separation face
region from submitted face image or video (which is an image sequence), feature
extraction which identifies and extract features of the submitted images. Furthermore
Zhao et al.(2003) described that feature can be local features such as lines or
fiducial points, or facial features such as eyes, nose, and mouth. . The next and final
step is recognising faces by matching input image against the faces in database.
Figure 2.1: Configuration of generic face recognition
Face Detection
Feature
Extraction
Face
Reco nition
Input Images
and Video
-
8/9/2019 Prrofing for Doc
21/169
9
2.2 Similar systemsFace recognition systems dates back in history of over 5 decades which starts form
1960. At Present, face recognition has achieve an excellent development and gained
attention from areas such as surveillance, biometric identification etc.
Because of that, there are many systems that have been developed using face
recognition technology but during the research, researchers could not find a system
which is capable of partial face recognition.
2.2.1Google PicasaGoogle Picasa is a photo organizing and editing software, which is a free application
initiated by Google. One of newest feature of Picasa is face recognition feature.
According to Baker (2006) Google Picasa use Face recognition approach called
NevenVison for recognition faces.
Baker (2006) also mentioned that Neven Vison is very advanced techniques and it
covers over 15 patents. Furthermore Neven Vison is not only for face recognition
but also for object recognition.
Figure 2.2: Google Picasa
[Source: Waltercedric, n.d. ]
-
8/9/2019 Prrofing for Doc
22/169
10
2.2.2APPLE IPhotoApple IPhoto can be consided as an alternative to Google Picasa in Mac Os
environment. It also has similar features as Google Picasa. In Apple IPhoto product
page describes face recognition feature as follows.
Apple Inc (2010) stated that iPhoto introduces Faces: a new feature that
automatically detects and even recognizes faces in your photos. iPhoto uses face
detection to identify faces of people in your photos and face recognition to match
faces that look like the same person.
Figure 2.3: APPLE I photo face recognition
[Source: Lee, 2009]
-
8/9/2019 Prrofing for Doc
23/169
11
2.2.3Face.Com automatic face book photo taggerAccording to Bil (2010) Face.com provide auto face tagging ability to facebook
photo albums. It will check all the photos available in users photo collection in
facebook and tag user and friends.
Figure 2.4: Face.com Photo Tagging
[Source: Bil, 2010]
Figure 2.5: Face Book Photo Tagging
During the experimentation, the researcher has identified that this application gives
poor results. Especially when it try to identify images in different lighting condition.
-
8/9/2019 Prrofing for Doc
24/169
12
2.3 Problem of face recognition systemsEven thought face recognition is five decades old. In study filed of face recognition
system there are lot of unsolved problems and biometric identification area. At times
most of the approaches of the face recognition, which is currently in use and in
experimenting stages, can only give solutions for few problems in face recognition.
During the research, the researcher could not find a robust face recognition
algorithm, which gives 100 % accurate results.
Generally, the following situations can categorize as problem in face recognition.
2.3.1Pose variancePose variance mean difference of the camera angel that takes photos (Image).
Figure 2.6: Different pose Audio Visual Technologies
[Source: Audio Visual Technologies Group, n.d.]
-
8/9/2019 Prrofing for Doc
25/169
13
2.3.2Lighting/Illumination varianceDifferent lighting affects to recognition.
Figure 2.7: Different illuminations
[Source: Audio Visual Technologies Group, n.d.]
2.3.3Different facial expression
Figure 2.8: Different facial expression
[Source: Audio Visual Technologies Group, n.d.]
2.3.4Different facial occlusions
Figure 2.9: Different facial occlusions
[Source: Audio Visual Technologies Group, n.d.]
-
8/9/2019 Prrofing for Doc
26/169
14
2.3.5Individual characteristicsFace recognition systems might depend on some face characteristics such as skin
colour. Some time face recognition systems, which are designed for Caucasianmight
not be capable of handling faces of other races.
In addition, gender can also consider as another factor sometime face recognition
system that is use for females lip identification might not work for males.
2.3.6Scale Invariance and robustnessThe face scale depends on the distance between person and camera. Therefore, some
time it is not capable of handling faces taken at different distance.
Apart from that, face recognition requires lot of computational power which should
be considered because although there are high-end computers which are capable of
handling such process, they are reasonably expensive.
-
8/9/2019 Prrofing for Doc
27/169
15
CHAPTER 3
3 SYSTEM RESEARCH AND INVESTIGATION3.1Fundamental of Image processingDigital image processing can be considered as one of the interesting area of the
computer vision. Image processing based on concepts on mathematics. Following
section shows fundamentals of image processing.
3.1.1Image classificationsAccording to Jhne (2005, pp32) there are entirely different ways to display an
image. The most popular way to represent an image is rectangular grid. Gonzalez
(2004, pp 01) further described it as follows; digital images can be represent as a
matrix as it can be represent as 2D function where x and y gives coordinates in
geometric plane.
Figure 3.1: Image representation in plane, As a matrix
[Source: Gonzalez, 2004, pp 01] & [Source: Jhne , 2005 , pp32]
According to (Gonzalez, 2004, pp 01) Images can be categorize as follows,
-
8/9/2019 Prrofing for Doc
28/169
16
3.1.1.1Raster ImageAs defined by Busselle( 2010) Raster images known as bitmap images. Raster
images are made by collection of dots, which called pixels. Pixels are tiny scares that
contain colour information about the particular location of the image. These images
are resolution dependent.
3.1.1.2Vector ImageAs defined by Busselle( 2010) vector images are collection of connected lines and
curves. Each line and curves defined by mathematical formula. Therefore, these
images are resolution independent and it has the ability to control mathematical
formula according to the scale.
3.1.1.3Binary ImageBinary images represent images by using 0 and 1.The pixel of the object represent as
1 and background as 0. By using threshold, a digital colour image could convert
into a binary image. Threshold is a method that uses to segment an image.
3.1.2Image segmentationAccording to Grady & Schwartz (2006), Image segmentation has often been
defined as the problem of localizing regions of an image relative to content.
According to Biswas (2008) Image segmentation approaches can divide as two
different categories as follows.
Image segmentation
Discontinuity based Region based
Figure 3.2: Image segmentation approach
-
8/9/2019 Prrofing for Doc
29/169
17
Furthermore Biswas (2008) and Yang (2004) described discontinuity based image
segmentation is focused on Isolated points, lines and edges detection while region
based / similarity based approach is focus on thresholding.
Spann (n.d) mentioned that, grey level histogram can consider as one of most
popular image segmentation method among other techniques.
3.1.2.1Grey level histogram-based segmentationImage histogram is a graph which shows the size of the area of the image that is
captured for each tonal variation that the camera is capable of recording (Sutton,
n.d).
Spann (n.d) mentioned that Thresholding, Clustering are image segmentation
techniques that are based on grey level histograms.
Noise is unwanted external information that is in image. First the noise should be
filter out form the image or it should consider when it processing the image.
Please refer appendix for Identifying Noise form histogram.
3.1.2.2ThresholdingThe operation known as simple thresholding consists in using zero for all pixels
whose level of grey is below a certain value (called the threshold) and the maximum
value for all the pixels with a higher value (kioskea.net , 2008).
As mentioned previously, this is a most popular method in image segmentation. As
mentioned by kioskea.net(2008) it classify an image based on pixel value of the
image. It checks whether gray values of a particular picture is greater than or less
than Threshold point and convert it to binary image by applying 1 or 0 to every
pixels in image.
-
8/9/2019 Prrofing for Doc
30/169
18
3.1.2.2.1 Grey level thresholdingGrey level thresholding defined whether particular grey pixel belongs to the object or
to the background. By using this method, it is possible to separate an object form the
background.
Figure 3.3: Low noise object/background image histogram
[Source: Spann ,n.d]
Spann (n.d) defined grey level thresholding algorithm as follows.
If the greylevel of pixel p
-
8/9/2019 Prrofing for Doc
31/169
19
3.1.2.2.2 Determine thresholdSpann (n.d) mentioned that to determine threshold there are several approaches but
following approaches shows high interaction.
Interactive threshold
Adaptive threshold
Minimisation method
Please refer appendix for detailed explanation for approaches for determiningthreshold.
3.1.3Edge detectionNeoh and Hazanchuk (2005) mentioned, Edge detection is a fundamental tool used
in most image processing applications to obtain information from the frames as a
precursor step to feature extraction and object segmentation. In addition, they
further explained it as follows This process detects outlines of an object and
boundaries between objects and the background in the image. According to Green
(2002), Edge detecting an image significantly reduces the amount of data and filters
out useless information, while preserving the important structural properties in an
image.
Green(2002a) categorized edge detection approaches into two main category.
Gradient and Laplacian. According to Green(2002) gradient method detects the
edges by looking for the maximum and minimum in the first derivative of the
image and The Laplacian method searches for zero crossings in the second
derivative of the image to find edges .
The relationship between edge detection and threshold is once image is binarized the
next step is applying threshold by doing that it can decide whether edges are
available or not.
If the threshold is lower, more edges can be found and high threshold might cause to
loss of edges.
-
8/9/2019 Prrofing for Doc
32/169
20
3.1.3.1Cannys edge detectionAccording to Fisher et al (2003) Cannys edge detection is algorithm which consider
as optimal edge detector. Furthermore Fisher et al (2003) mentioned, It takes as
input a gray scale image, and produces as output an image showing the positions of
tracked intensity discontinuities.
According to Green (2002b) there are few criteria that should consider.
Edges occurring in images should not be response to non-edges and edges
in images should not be missed identify. The algorithm that takes to
identify the edges must to identify (mark) the all real edges in the image. The edge points must to be well localized. The distance between the edge
pixels must to be well organized and close to the edges in the real image.
Must to well identify the noisy images before using the edge detection
algorithm. Then have to filter the noisy picture first using appropriate
filers.
Green (2002b) applied following steps to detect edges using Cannys edge detection.
Step 1 Filter out any noise in the original image
Green (2002b) used Gaussian filter to remove noise.
Step 2 Find the edge strength
Green (2002b) has achieved by finding the gradient of the image. To do that
he perform the Sobel operator 2-D spatial gradient measurement on an
image.
Step 3 Finding the edge direction
Green (2002b) stated that achieving this is a trivial task.
-
8/9/2019 Prrofing for Doc
33/169
21
Step 4
Once the edge direction is known, the next step is to relate the edge
direction to a direction that can be traced in an image (Green, 2002b).
Step 5
After the edge directions are known, non maximum suppression now has to
be applied (Green ,2002b).
Step 6
Then it use hysteresis to determine final edges. He did it by using suppressing all
edges, which does not connect to selected edge.
Figure 3.4: Canny edge detection process
[Source: szepesvair, 2007, pp19]
3.1.3.2Sobel Edge DetectionAccording to (Green, 2002a ) The Sobel operator performs a 2-D spatial gradient
measurement on an image. Typically, it is used to find the approximate absolute
gradient magnitude at each point in an input greyscale image.
(Green, 2002a) and Biswas(2008) Describe it further as Sobel detector can consider
as algorithm that performs two dimensional gradient measurements on an image.
-
8/9/2019 Prrofing for Doc
34/169
22
This technique achieve by calculating the estimated gradient magnitude at each point
in a gray scale image.
In the article presented by Green (2002a).He mentioned that the Sobel edge detector
used a pair of 3 x 3 convolution masks. One of mask estimate gradient in the x-axis
other mask estimating the gradient in the y-axis. Normally mask is smaller than the
actual image because of that this mask is slid over the image. Because of that, it
manipulates a square of pixel at a time.
The following figure shows an example of Sobel template.
Figure 3.5: Masks
[Source: szepesvair, 2007, pp14]
Figure 3.6: Sober Template
[Source: szepesvair, 2007, pp14]
Following formulas presented by Green (2002a).
The formula to find magnitude of the gradient
-
8/9/2019 Prrofing for Doc
35/169
23
Approximate magnitude of the gradient
It has identified that by using grey-level histogram segmentation and applying edge
detection like canny edge detector, it is possible to extract face regions. According to
view of researcher, this method can be useful in extraction face regions from the
face but it might not give accurate results because histogram thresholding value
might depend on gender and race.
-
8/9/2019 Prrofing for Doc
36/169
24
3.2Face recognition approachesThe first face recognition approach was introduced by Sir Francis Galton in 1888
(Kepenekci, 2001) which was done by measuring four characteristics of French
prisoners. Because of that attempt of Sir Francis Galton to identify persons in more
scientific way, it made foundation of the biometric recognition which cause to
improvement of face recognition.
As mentioned before there are different approaches for face recognition. Those
approaches can divide into main section based on the image type. Those are two-
dimensional and three dimensional face recognition.
3.2.1Three dimensional face recognition3D model based face recognition can consider as latest trend of the face recognition.
As mentioned by Bonsor & Johnson (2001) , this face recognition approach uses
distinctive features of the faces. Which mean features like dept of the face features,
curves of the eyes, nose and chin use to recognise a faces.
Akarun, Gokberk, & Salah (2005) stated that three dimensional face recognition has
higher accuracy rate over two dimensional traditional face recognition but when it
comparing with traditional face recognition approach it has few disadvantages such
as cost of implementation , unfamiliarity of technology and lack of hardware
specially camera equipments. Because of that, still 2D face recognition has higher
demand.
As mentioned by Gaokberk (2006) there are different approaches that use in 3D face
recognition. Point cloud-based approaches, depth image-based approaches, curve-
based approaches, differential geometry-based approaches, facial feature-based
geometrical approaches and shape descriptor-based approaches can consider most
popular approaches in 3D face recognition.
2D (Two-dimensional) face recognition can consider as good alternative to 3D face
recognition that has developed over 50 years. Following section will describe about
the 2D face recognition approach.
-
8/9/2019 Prrofing for Doc
37/169
25
3.2.2Two dimensional face recognitionTwo dimensional face recognition can be considered as the most oldest and popular
method that is currently using in the field. This approach can process 2D images that
have taken in regular cameras (probably cameras in security systems) and identify
faces based on different approaches. As mentioned by NSTC subcommittee on
biometrics (2006) some of algorithms use faces landmarks to identify faces while
other algorithm use normalized face data to identify probe image.
Based on NSTC subcommittee on biometrics (2006) explanations 2D face
recognition approaches can categories as follows.
Geometric based approach in other words feature based approach that
consider face feature such as nose, mouth and eyes to identify faces.
Photometric based approach or view based approach that consider images as
set of pixels and compare characteristics of those pixels in both images.
3.2.3Geometric based approachesGeometric based face recognition techniques involve processing and computation
face land marks in other words it use face features such as relative positions of nose,
mouth and eyes, distance between landmarks, etc to identify individuals.
As stated by Brunelli and Poggio (1993) the idea behind geometric face recognition
is extract relative position and other parameters of distinctive features such as eyes
.After finding the features they will match with the known individuals featuresdetails and find out the closest distance of the matches. According to Kanade (1973)
the research has achieved 45-75% success recognition rate with 20 test cases
.According to Brunelli and Poggio (1993) disadvantage of geometric face
recognition over view based face recognition is extraction of face features.
The following image shows some of face features that can use for recognize faces.
-
8/9/2019 Prrofing for Doc
38/169
26
Figure 3.7: Geometrical features
[Source: Brunelli & Poggio , 1993]
However, with the modern techniques and approaches, face feature detection has
gained lot of improvements comparing with old day. As mentioned by Bagherian,
Rahmat and Udzir (2009) extraction the features can do by different approaches such
as colour segmentation approach, template based approach that use pre-defined
image to detect face features and relative locations.
With development of appearance based face recognition approach geo metric face
recognition approach has deviate from the face recognition area and the techniquesused by geometric face recognition have used to different areas such as facial
expression detection ,etc.
3.2.4Appearance based approachThe second approach of the face recognition is appearance based face recognition,
which is quite popular among researchers and industries relate to machine vision. In
this approach, it takes probe face as a set of pixels into a matrix and compare with
the known individuals. Brunelli and Poggio (1993) described it as follows the
image, which is represented as a bi dimensional array of intensity values, is
compared using a suitable metric with a single template representing the whole
face. The above-mentioned approach is only an overall picture of the appearance
based recognisor approach. There are more sophisticated ways of performing
appearance based face recognition.
-
8/9/2019 Prrofing for Doc
39/169
27
Figure 3.8: Template matching strategy
[Source: Brunelli & Poggio ,1993]
Based on NSTC subcommittee on biometrics (2006) statistical face recognition
approach can be categorise as follows Principal-component analysis (PCA) , Linear
discriminate analysis (LDA) and elastic brunch graph matching .
The different techniques use different approaches, algorithm in recognising process.
Some time as mentioned by Zhao et al (2003) hybrid methods like Modular
eigenfaces, Hybrid LFA, Shape-normalized Flexible appearance models,
Component-based. Therefore, finally the following conclusion can make that facerecognition can have different approaches some time there can be novel approaches
that use combination of different techniques.
During this project, it considers only 2D face recognition because it is not faceable to
find enough resources to do 3D face recognition. In 2D face recognition, it would
constraint on view based face recognition because by using geo metric face
recognition the data (distance between features) which can use to recognize is not
enough.
i.e:
If it takes eye region, it can only take few measures. Like distance between I corners,
width of eyelid, etc. These measures can be verified because of that this method is
not good reliable measure.
-
8/9/2019 Prrofing for Doc
40/169
28
3.3Face region extraction approachesThe author has identified several approaches to extract face features during the
research. Among them following techniques show promising results.
3.3.1Template matchingAccording to Latecki( n.d.) Template matching compares two images (potions of
Images) and find out similarity between them. Nashruddin(2008) elaborated it as
follows Template matching is a technique for finding small parts of an image which
match a template image. It slides the template from the top left to the bottom right of
the image, and compare for the best match with template. The template dimensionshould be equal or smaller than the reference image. He also motioned that there are
many methods for template matching.
Latecki (n.d.) Explained template matching by using following the diagrams.
Figure 3.9: Template matching
[Source : Latecki , n.d.]
According to Latecki (n.d.) in here, the matching processes changes the position of
the template image to all possible positions of the source image. Then it computes a
numerical index that indicates how well the template matches the image in that
position. Match is done on a pixel-by-pixel basis.
Latecki (n.d) described two type of template matching
-
8/9/2019 Prrofing for Doc
41/169
29
3.3.1.1Bi-Level Image template matchingIt use Bi-Level template matching identify whether particular object is available in
particular source image. Letecki(n.d) stated that this methods only suitable for giving
yes or no (only for checking availability of a image).
3.3.1.2Grey-Level Image Template matchingInstead of Bi-level Image template, matching it checks difference level of image is
used. To measure that correlation can use.
Letecki(n.d) has conducted an experiment on five data set and achieved following
correlation maps.
Figure 3.10: Template
[Source: Letecki , n.d.]
Figure 3.11: Test one Grey-Level Template matching
[Source: Letecki , n.d.]
-
8/9/2019 Prrofing for Doc
42/169
30
Figure 3.12: Test two Grey-Level Template matching
[Source: Letecki , n.d.]
Figure 3.13: Test three Grey-Level Template matching
[Source: Letecki , n.d.]
Figure 3.14: Test four Grey-Level Template matching
[Source: Letecki , n.d.]
-
8/9/2019 Prrofing for Doc
43/169
31
Figure 3.15: Test Five Grey-Level Template matching
[Source: Letecki , n.d.]
Figure 3.16: Test Five template is not available
[Source: Letecki , n.d.]
Letecki (n.d.) has used following formula to calculate the correlation
1
0
1
0
22
1
0 )(N
i
N
i
ii
i
N
i i
yyxx
yyxxcor
x is the template gray level image x is the average grey level in thetemplate image
y is the source image section is the average grey level in the sourceimage
N is the number of pixels in the sectionimage
(N= template image size = columns *rows)
-
8/9/2019 Prrofing for Doc
44/169
32
The value cor is between 1 and +1, with larger values representing a stronger
relationship between the two images.
If the correlation value is less than correlation values of template then there will not
be a template image in source.
By using grey level template matching it is possible to match face regions but it
might not get accurate results because intensity values of the faces can be depend of
the race and gender and computational time can be high because template matching
use pixel based calculations. However, by limiting variance it is possible to perform
this approach to face region extraction.
As a alternative to pixel based calculation Violla & Jones (2001) proposed feature
based approach. Following section describe about it.
3.3.2Haar like features - Viola-Jones face detectorViolla & Jones (2001) proposed an approach to detect objects using a boosted
cascade of simple features. They introduced new image representation method called
Integral image according to Violla & Jones (2001) integral images allow
processing the data other than using raw images. Then they have used a learning
algorithm AdaBoost to select visual features from a larger set and yields extremely
efficient classifiers. Then they combined successively more complex classifiers in a
cascade structure, which dramatically increases the speed of the detector by focusing
attention on promising regions of the image.
Violla & Jones (2001) approach they have used features that generated by haar basis
functions. According to Violla & Jones (2001) the reason for using features are that
features can act to encode ad-hoc domain knowledge that is difficult to learn using a
finite quantity of training data. Furthermore, that has used three types of features.
Two rectangle feature , three-rectangle features and four-rectangle features which are
respectively the difference between the sum of the pixels within two rectangular
regions , computes the sum within two outside rectangles subtracted from the sum in
a centre rectangle , computes the difference between diagonal pairs of rectangles.
-
8/9/2019 Prrofing for Doc
45/169
33
Figure 3.17: Rectangle Features
[Source: Violla & Jones , 2001]
According to Violla & Jones (2001) the reason for using above mentioned integral
images are rectangle features can be computed very rapidly using an integral image.
By using number of negative (which does not include false faces) and positive
images (which does include faces) to the AdaBoost has trained for extract the
features.
Figure 3.18: The features selected by AdaBoost
[Source:Violla & Jones 2001]
The two features are shown in the top row and then over layed on a typical training
face in the bottom row. The first feature measures the difference in intensity between
the region of the eyes and a region across the upper cheeks. The feature capitalizes
on the observation that the eye region is often darker than the cheeks. The second
feature compares the intensities in the eye regions to the intensity across the bridge
of the nose (Violla & Jones 2001).
-
8/9/2019 Prrofing for Doc
46/169
34
In the end, they have formed the cascade, which can be, identify as a decision three,
which has classifier network to filter our negative results and provide a well accurate
result. The following diagram shows structure of a cascade.
Figure 3.19: Cascade of classifiers
[Source:Violla & Jones 2001]
The approach, which is proposed, by Violla & Jones (2001) has 38 stages with over
6000 features.
They have trained each classifier of the cascade by 4916 faces and 10,000 non-faces
size of the all images are 24 x 24.
Table 3.1: Detection rates for various numbers of false positives on the test set
[Source:Violla & Jones 2001]
This method can consider as good approach to face region extraction but identifying
face regions can be difficult because face can have lot of representation of
rectangular features therefore Cristinacce & Cootes (2003) proposed approach,
which can use to detect face features.
-
8/9/2019 Prrofing for Doc
47/169
35
3.3.3Facial feature detection using AdaBoost with shape constraintsThis can consider as extend of face detection method proposed by Violla & Jones
(2001).
Cristinacce & Cootes (2003) proposed a method for facial feature extraction using
AdaBoost with shape constraints for locate the eye, nose, mouth corners in frontal
face image.
Their approach can divide into two main sections face detection and face
segmentation detection.
3.3.3.1Face DetectionCristinacce & Cootes (2003) used the face detection method used by Viola & Jones
to detect the faces for feature selection, which has described previously. Cristinacce
& Cootes (2003) described it as follows The output of the face detector is a image
region containing the face, which is then examined to predict the location of the
internal face features.
Cristinacce & Cootes (2003) deviated from Viola and Jones approach when selection
negative and positive images and building AdaBoost template using haar like
feature. Viola & Jones use human faces as positive examples and regions known
not to contain a human face as the negative examples. According to Cristinacce &
Cootes (2003) in this approach, the positive examples are image patches centred on a
particular facial feature and the negative examples are image patches randomly
displaced a small distance from the same facial feature.
3.3.3.2Local feature detectionCristinacce & Cootes (2003) performed same algorithm to locate the local features
detection. The following figure shows few features selected by AdaBoost for right
eye.
-
8/9/2019 Prrofing for Doc
48/169
36
Figure 3.20: Features selected by AdaBoost
[Source: Cristinacce & Cootes,2003]
The other thing that the approach taken by Cristinacce & Cootes(2003) which is
extends approach taken by Viola & Jones(2001) is in this method it use shape
constraints to check candidate feature points. Cristinacce & Cootes (2003) described
it as follows Firstly a shape model is fitted to the set of points and the likelihood of
the shape assessed. Secondly, limits are set on the orientation, scale and position of aset of candidate feature points relative to the orientation, scale and position implied
by the global face detector.
The shape model they have used designed in similar way that proposed by Dryden
and Mardia(1998 cited Cristinacce & Cootes ,2003). Then Cristinacce & Cootes
aligned the points in to a common co-ordinate frame. After taking the distribution
According to Cristinacce & Cootes (2003) they got multi-variant Gaussian type of
distribution. Then they estimated probability of the give shape ps(x).Then they got
threshold T by comparing training dataset. So if probability of given shape is greater
than threshold they consider as a shape.
Cristinacce & Cootes (2003) has analysed to find the range of variation in position
of the features relative to the bounding box found by the full face detection.
Furthermore, the feature detectors what have implemented during this application
returns list of candidate points that probability of given shape is greater than
threshold.
During their research, they have achieved following recognition rates.
According to Cristinacce & Cootes (2003) the maximum time the entire process took
was less than .5 Seconds. They have archived 88.8% detection rate as follows.
-
8/9/2019 Prrofing for Doc
49/169
37
Feature distance is within 5% of the eye separation for 65% of faces, 10% for 85%
of faces and 15% for 90% of faces Cristinacce & Cootes (2003).
This method shows promising results but to implement this method it requires lot of
time and effort because of AdaBoost training but during the technical investigation,
it found a method to overcome this problem. The method for implementation will
describe in later.
3.3.4Justification on selected technologiesIn the research, it discussed about viola-jones face feature detection using Haar-like
features, template matching approach, face detection and facial feature detectionusing AdaBoost approach and shape constraint. In Violla-Jones (Violla &
Jones,2001) approach, they have used features that generated by Haar based
functions. Features are rectangle areas, which represent the binary intensity values of
the faces. This method achieved 76.1% - 96.9% face detection rate. Cristinacce &
Cootes (2003) purposed extended version of Viola-Jones (Violla & Jones 2001)
approach for detect local features of faces. Cristinacce & Cootes (2003) have
achieved excellent results than Violla-Jones approach. Kuo & Hannah (2005).
purposed template-matching approach for eye feature extraction and they have
archived 94 % for iris extraction, 88% eye corners and eyelid extraction.
Both Violla-Jones (Violla & Jones, 2001) and Cristinacce-Cootes (Cristinacce &
Cootes, 2003) approaches used AdaBoost network to train the Haar cascades. In
addition, Kuo & Hannah (2005) approach does not use any neural network approach.
To train the AdaBoost they have used huge number of positive and nonnegative
images, which requires lot of time and approaches but once cascade files created it is
possible to reuse it for different set of data.
Kuo & Hannah (2005) mentioned that there approach is relatively inefficient because
this algorithm does not work in occluded faces and pose variance. In addition, the
algorithm is capable of identifying only eyes.
So above details can summarize as follows.
-
8/9/2019 Prrofing for Doc
50/169
38
Violla-Jones Approach Cristinacce-Cootes
Approach
Template based Approach
High detection rate High detection rate High detection rate
Detect only face Improved to detect face,
eyes , nose and mouth
Eye region only
ANN training requires ANN training requires ANN training does not
requires
Table 3.2: Comparison of face detection approach
Researcher has identified there are trained haar-like feature cascade file which can
use for Violla-Jones Approach and Cristinacce-Cootes Approach. Then it does not
require use of AdaBoost training.
By considering above facts, it decided that to use haar-like features for detect the
face and face regions. Because by comparing above three techniques it shows high
accuracy rate and efficiency. The reason for rejecting template-based approach is the
accuracy of the template matching is depending on the template and testing set.
3.4 Face recognition methods3.4.1Elastic branch graph matching approachEBGM (elastic branch graph matching) approach is simple dynamical link
architecture implementation. As mentioned by NSTC subcommittee on biometrics
(2006) there are non-linear characteristics of face images that does not address by
linear methods like PCA, LDA that will describe next. By using EBGM it is possible
to avoid or minimize impact of nonlinear characteristics like poses elimination,
expressions lighting conditions, etc in face recognition.
Wiskott et al (1997) presented an approach which is based on geometrical local
faced based for face recognition. This approach is known as elastic brunch graph
matching. In this approach, faces are represented as labelled graphs. The total
concept behind face representation is Gabor wavelet transformation purposed by
Daugman (1989 cited by Wiskott et al 1997).
-
8/9/2019 Prrofing for Doc
51/169
39
Furthermore, in the face graph, node represent fiducial points (eyes, nose, etc) and
edges represent distance between each fiducial points. Furthermore those points are
describe by sets of wavelet components (jets).
Figure 3.21: Elastic branch graph matching face recognition
[Source: Wiskott et al ,1997]
According to Wiskott et al (1997) image graph of new faces are extract using elastic
graph matching process that represent the face. According to Wiskott et al (1997) it
is vital to have accurate node positioning for face recognition to get accurate result
therefore it use phase information to get accurate node positioning. Object-adapted
graphs which use fiducial points representation handle rotations of objects (In thiscase faces) in depth.
The face graph is extraction base on bunch graph, which is combined representation
of all model graphs for cover wide variance, rather than one model does. Wiskott et
al (1997) also mentioned that use of individual separate models to cover variations
like such as differently shaped eyes, mouths, or noses, different types of beards,
variations due to sex, age, race, etc.
According to Wiskott et al (1997), the matching process involves few manual
supervising phase to build up a bunch graph. Then individuals can be recognise
based on learned system by comparing image graph and model graphs obtained from
the gallery images taken in the database. Then it takes graph, which has highest
similarity value.$
-
8/9/2019 Prrofing for Doc
52/169
40
The above method purposed by Wiskott et al (1997) is not specific only for human
face recognition they have mentioned that it is possible to use this system for other
object recognition also. Furthermore, above method work fine in lot of variance
caused by size, expression, position and pose changes.
The experiment they have done by using FERET database they have achieved high
recognition rate over frontal - frontal (98 %) face recognition with and without
expression. They could archive 57% - 81 % recognition rate by comparing half
profile left side of the face and half-profiling right side of the face.
The following table shows the summery of their results which achieved in their
approach.
Model Gallery Probe images First Rank First 10 Rank
Noofsu
ccess
Image
success
percentag
e
Noofsu
ccess
Image
success
percentag
e
250 fa 250 fb 245 98 248 99
250 hr 181 hl 103 57 147 81
250 pr 250 pl 210 84 236 94
249 fa + 1 fb 171 hl + 79 hr 44 18 111 44
171 hl + 79 hr 249 fa + 1 fb 42 17 95 38
170 hl + 80 hr 217 pl + 33 pr 22 9 67 27
217 pl + 33 pr 170 hl + 80 hr 31 12 80 32
Table 3.3: Recognition results between different galleries using EGBM
[Source: Wiskott et al ,1997]
f: frontal views a, b: expression h: half-profiles p: profiles l, r: left and right
-
8/9/2019 Prrofing for Doc
53/169
41
It has identified that this method is suitable for multi-view based face recognition but
like geometric based face recognition approach, this cannot apply for partial face
recognition because of lack of information for face recognition.
3.4.2Appearance based subspace methodsAs stated by Delac, Grgic and Liatsis (2005) there are several statistical (appearance
based) methods have been purposed. Among them PCA and LDA perform major
roles. According to Navarrete and Ruiz-del-solar (2001) in subspace method it is
project the input faces onto a reduced dimensional space where the recognition is
carried out, performing a holistic analysis of the faces.
Furthermore, Navarrete and Ruiz-del-solar (2001) stated that PCA and LDA could
consider as above projection methods that reduce high dimensional image space to
low dimensional image space. Heseltine (2005) elaborate it further as follows PCA
,LDA and other methods can be use to image subspace projection in order to
compare face images by calculating image separation in a reduced dimensionality
coordinate space. .He also mentioned that it use a training set of face images in
order to compute a coordinate space in which face images are compressed to fewer
dimensions, whilst maintaining maximum variance across each orthogonal subspace
dimension.(Heseltine, 2005).
3.4.2.1What is SubspaceAccording to Moghaddam(1999) visual data can be represented as points in a high -
dimensional vector space. For example, a m m-by-n n pixel 2D image can be mapped to
x Rmn
a vector, by lexicographic ordering of the pixel elements Which mean animage that has m by n pixels can represent as a matrix / vector which has NxM
dimensions.
According to Yambor (2000) the images projected into a subspace, then it creates
combined all subspace into a one which has all training images subspaces then the
test (probe) image project into subspace. After that, each test image compares with
the training images by a similarity or distance measure. The most similar or closest
images identify as the match image.
-
8/9/2019 Prrofing for Doc
54/169
42
3.4.3Support vector machine kernel correlation feature analysis methodSavvides et al (2006) proposed a method for recognize partial faces and holistic
faces based on kernel correlation feature analysis (KCFA) and support vector
machine.
Here they have used correlation filter to extract features form the face images and
face regions. Minimize average correlation filter which has designed to minimize the
average correlation plane energy resulting from the training images Xie(2005 cited
Savvides et al 2006) has used in this approach to output required correlation peak
values that need to identify correlation peeks that match to face and face regions that
need to extract.
But as mentioned by Savvides et al (2006) stated that it is necessary to have generic
data set to build a correlation filter but up to this stage they had not used any generic
data set. Therefore, they proposed use of novel method call class dependent feature
analysis to overcome limitation of making generic dataset.
According to Savvides et al (2006) Class dependent feature analysis (CFA) is a
method that use to dimensionality reduction and feature extraction, which is similar
to PCA (Principle component analysis).They further stated that their proposed class
dependent feature analysis method achieved higher success rate than traditional
PCA. As mentioned by Savvides et al (2006) PCA address all number of images in
while CFA address number of classes ( here class mean set of image of same
individual).
According to Savvides et al (2006), they have designed one correlation filter which
is a minimize average correlation filter (MACE).First they have trained MACE set
using 12776 images of 222 different classes. Because of that, they have received 222
dimensional correlation feature vector after projection of input images to correlation
filters, which have generated by MACE.
Based on description provided by Savvides et al (2006) because of Non linear
attributes of face images such as lighting condition, pose linear subspace
methods(PCA and LDA) cannot perform well because of that those non linear
-
8/9/2019 Prrofing for Doc
55/169
43
subspace method deviate to map non linear attributes in higher dimensional feature
space. Again, the calculations in higher dimensional feature space cost lot of
computational power. Therefore, kernel trick methods use to perform those
calculations to computer higher dimensional feature mapping.
Savvides et al (2006) stated that the kernel correlation filters could extend using
linear advance correlation filter by performing kernel trick. They extended their CFA
method by kernel trick and performing it over the all images.
According to Savvides et al (2006), they have used support vector machine as a
decision classifier on distance measure on KCFA projection space. They input
KCFA projection coefficients to training the SVM.
By examine distance and similarity threshold values between training image and
probe image (in KCFA projection coefficients) they have identified best match that
mean they recognised the individuals using this method which is similar to PCA.
They gained promising results during this approach. The results as follows the eye-
region yields a verification rate of 83.1% compared to 53% obtained by using themouth region and 50% with the nose region.
3.4.4Eigenface ApproachAccording to Fladsrud (2005),Eigenface approach can consider as one of popular
face recognition approach that purposed by Sirovich & Kirby and the method
purposed by Kirby refined by Matthew Turk and Alex Pentland by adding pre-
processing and face detection procedure.
As mentioned before an image can represent as vector. If the image width is w by
and height is h then image can be represent w x h dimensional vector.
-
8/9/2019 Prrofing for Doc
56/169
44
Figure 3.22: Face image transformation
[Source: Fladsrud,2005]
X=[x1,x2,x3.............................,xn]T
It assumed that n is the total of pixel in image.
The rows of the image place each after other and it form a vector. Above vector
belongs to a image space which is w by h all images whose dimensions is w by h and
by putting row each after each it can be represent as one dimensional vector which
shows in figure 4.4.Following images shows basis of the image space.
When it take faces all the faces look like same because any face has a mouth, a nose,
pair of eyes, etc which are relatively located at approximately same place. As
mentioned by Fladsrud (2005) because of above quality of the face all faces (face
vectors) are located in very small area in the image space. Figure 3.26 represents
distribution of faces (face vectors) in image space.
Figure 3.23: Distribution of faces in image space
[Source: O'Toole et al ,1993]
-
8/9/2019 Prrofing for Doc
57/169
45
By understanding that it is possible to understand that represent the faces in image
space is waste of space. O'Toole et al (1993) purposed use of PCA to find the
specific vectors that are highly sensitive for distribution of face images. These
vectors can call as face space which is subspace of face images. Furthermore
OToole et al (1993) stated that Face space will be a better representation for face
images than image space which is the space which containing all possible images
since there will be increased variation between the faces in face space.
By projecting faces in to face space will provide following distribution of faces in
face space.
Figure 3.24: Faces in face space
[Source: O'Toole et al ,1993]
According to OToole et al (1993) the vectors in face space are known as eigenfaces.
The simplest method to compare two images is compare pixel by pixel but when it
considering 100 pixels by 100 pixels it contains 104 pixels doing comparison for that
amount of pixels is time consuming and inefficient. As a solution for that Kerby &
Sirovich(1990) used karhunen-Love expansion which is popular as principle
component analysis(PCA) .Also Kerby & Sirovich(1991) stated , The main idea
behind applying PCA to a image is find out weights ( vectors) which are responsible
to distribution of face space within the image space.
Because of the application of PCA, it is possible to keep image data in way that is
more compact and because of that, the comparison of two images vector is less time
consuming and efficient than matching image pixel by pixel.
-
8/9/2019 Prrofing for Doc
58/169
46
Start
Training set
(known
faces)
E = Eigenfaces
W=weights of E
training setRecognition Module
De and >e
No
D>e and < e
No
D = Average
Distanse
(W,Wx)
Yes
Yes
Input
Unknown
images E
X is not a face
X is unknown
face
X is a known
faceYes
End
No
E = EigenfacesW=weights of E
training set
Figure 3.25: Overview of Eigen face Approach
Above figure represents overall idea behind the Eigen face algorithm. The diagram is
base on Turk & Pentland (1991). The training set which are known images are
transformed into set of Eigen vectors (Eigen faces)(E).After that it calculate weights
of E training set. Then it compares the weight of the new face and the weight of
training set (W).D is the average distance of between weight vectors .e is the
maximum allowable distance from any face class. is the Euclidian distance between
(L2 Norm) projection. The