prrofing for doc

8/9/2019 Prrofing for Doc

1/169

1


2/169

ii

ABSTRACT

Face recognition is a wide area which is currently studied by lot of researchers and

vendors but still there are some areas that even researchers have not touched. Partial

face recognition can be considered as one of these areas. Few years back researchers

considered that partial face recognition is an unrealistic approach because there is

less information which can use to recognize a person. But after the research

conducted by researchers like Sato et al (1998), it is proven that partial face region

also contain some amount of information that can use to recognition.

By conducting this project, I tried to investigate how far partial face regions involve

recognizing individual. The artifact allows users to input probe images and then the

system will determine individuals face whose belongs that particular face region.

Even though it appears to be a simple process the system should perform intensive

amount of work to achieve it.

A domain research, which has conducted to identify and study about the problem,

related to face recognition, similar system. Then by analysing it I decided to the area

that needed constrain in system research.

During system research researcher studied about the fundamentals about the image

processing, face detection techniques and face recognition techniques. Then after

analysing them it decided to select appropriate techniques to extract partial face

regions and recognize individuals.

The document contains introduction, domain research, system research, project

development plan, requirement specification and technical investigation, which

contain details about above-mentioned areas.

Finally appropriate image processing techniques, face detection techniques and face

recognition techniques were selected and justified along with selection of the project

developing methodologies and project developing platforms.


3/169

iii

ACKNOWLEDGEMENT

First I would like to express my appreciation to my supervisor Ms Oshini Jayatunga

and my assessor Mr. Gamindu Hemachandra for their guidance, suggestions and

support that given to me during my project. Without their directions I will not able to

accomplish this project.

Special thanks to my family without their help guidance and love I would not able to

make this far. Then I like to thank my colleagues for their advice and guidance about

the project. Special thanks go to my friend Tharindu Roshantha and Andrew Jebaraj

for being good friends to me and help me to proof read my document and correcting

my mistakes.

Finally, I would like to thank to APIIT Sri Lanka for facilitating me to by providing

me necessary educational help form well qualified lecture staff, library and lobotomy

facilities which need to make this project success. Especially thanking the library, for

facilitating me to get the necessary reading materials which were needed to my

project research.


4/169

iv

TABLE OF CONTENT5S

ABSTRACT ................................................................................................................. iiACKNOWLEDGEMENT ......................................................................................... iiiTABLE OF CONTENT5S .......................................................................................... ivLIST OF FIGURES .................................................................................................... ixLIST OF TABLES ...................................................................................................... xiLIST OF ABBREVIATION ...................................................................................... xiiCHAPTER 1 ................................................................................................................ 1

1 INTRODUCTION ................................................................................................... 1

1.1Project Background ............................................................................................ 11.2Understanding the Problem ................................................................................ 21.3Sub Problems ...................................................................................................... 31.4Project Objectives ............................................................................................... 41.5Proposed Solution ............................................................................................... 41.6Project Scope ...................................................................................................... 5

1.6.1Functionality of the Artefact ....................................................................... 51.7Constrains ........................................................................................................... 71.8Assumptions ....................................................................................................... 7

CHAPTER 2 ................................................................................................................ 82 Domain Investigation .............................................................................................. 8

2.1Outline of the generic face recognition system .................................................. 82.2Similar systems ................................................................................................... 9

2.2.1Google Picasa .............................................................................................. 92.2.2APPLE IPhoto ........................................................................................... 102.2.3Face.Com automatic face book photo tagger ............................................ 11

2.3Problem of face recognition systems ................................................................ 12


5/169

v

2.3.1Pose variance ............................................................................................. 122.3.2Lighting/Illumination variance .................................................................. 132.3.3Different facial expression......................................................................... 132.3.4Different facial occlusions ......................................................................... 132.3.5Individual characteristics ........................................................................... 142.3.6Scale Invariance and robustness ................................................................ 14

CHAPTER 3 .............................................................................................................. 153 System research and investigation ........................................................................ 15

3.1Fundamental of Image processing .................................................................... 153.1.1Image classifications.................................................................................. 153.1.2Image segmentation ................................................................................... 163.1.3Edge detection ........................................................................................... 19

3.2Face recognition approaches............................................................................. 243.2.1Three dimensional face recognition .......................................................... 243.2.2Two dimensional face recognition ............................................................ 253.2.3Geometric based approaches ..................................................................... 253.2.4Appearance based approach ...................................................................... 26

3.3Face region extraction approaches.................................................................... 283.3.1Template matching .................................................................................... 283.3.2Haar like features - Viola-Jones face detector ........................................... 323.3.3Facial feature detection using AdaBoost with shape constraints .............. 353.3.4Justification on selected technologies ....................................................... 37

3.4Face recognition methods ................................................................................. 383.4.1Elastic branch graph matching approach ................................................... 383.4.2Appearance based subspace methods ........................................................ 413.4.3Support vector machine kernel correlation feature analysis method ......... 423.4.4Eigenface Approach .................................................................................. 43


6/169

vi

3.4.5Justification on selected technique ............................................................ 563.5Approach to the solution ................................................................................... 59

3.5.1Approach to face region extraction ........................................................... 593.5.2Approach to face region identification ...................................................... 593.5.3Approach to face region recognition ......................................................... 59

Chapter 4 .................................................................................................................... 604 Requirement Specification .................................................................................... 60

4.1System requirements ......................................................................................... 604.1.1Functional requirements ............................................................................ 604.1.2Non Functional requirements .................................................................... 60

4.2Resource requirements...................................................................................... 614.2.1Hardware Requirements ............................................................................ 614.2.2Software Requirements.............................................................................. 62

CHAPTER 5 .............................................................................................................. 645 Development methodologies ................................................................................. 64

5.1Waterfall ........................................................................................................... 645.2Incremental Model ............................................................................................ 655.3Hybrid model .................................................................................................... 665.4Prototyping ....................................................................................................... 675.5Justification for selection method ..................................................................... 695.6Developing stages ............................................................................................. 69

CHAPTER 6 .............................................................................................................. 726 Technical research and investigation .................................................................... 72

6.1Developing platforms ....................................................................................... 726.1.1Dot.Net ...................................................................................................... 726.1.2Sun Java ..................................................................................................... 766.1.3MATLABMathWorks ........................................................................... 77


7/169

vii

6.2Image processing APIs ..................................................................................... 786.2.1OpenCV ..................................................................................................... 786.2.2EmguCV .................................................................................................... 786.2.3Aforge.Net ................................................................................................. 78

6.3Math Related Libraries ..................................................................................... 796.4Developing Approaches.................................................................................... 81

6.4.1Approach One ............................................................................................ 816.4.2Approach Two ........................................................................................... 816.4.3Approach Three ......................................................................................... 826.4.4Approach Four ........................................................................................... 82

6.5Justification for selection of programming environment.................................. 82CHAPTER 7 .............................................................................................................. 857 System Design ....................................................................................................... 85

7.1Design Approach .............................................................................................. 867.1.1Programming ApproachObject Oriented Approach .............................. 867.1.2Design architecture .................................................................................... 86

7.2Use Case Diagram ............................................................................................ 877.3Use case descriptions ........................................................................................ 88

7.3.1Add New Record ....................................................................................... 887.3.2View Record .............................................................................................. 897.3.3Train Image Set ......................................................................................... 907.3.4Find Match ................................................................................................. 917.3.5Set Configuration ....................................................................................... 927.3.6Modify Record ........................................................................................... 93

7.4System Overview .............................................................................................. 957.5Activity Diagrams ............................................................................................. 96

7.5.1Add Record ................................................................................................ 96


8/169

viii

7.5.2View Record .............................................................................................. 977.5.3Training Image Space ................................................................................ 987.5.4Find Match ................................................................................................. 997.5.5Automatic Face Region Verification ....................................................... 1007.5.6Pre-processing/Normalization ................................................................. 100

7.6Class Diagrams ............................................................................................... 1017.7Classes Description ......................................................................................... 102

7.7.1Data management related classes ............................................................ 1027.7.2Application logic related classes ............................................................. 1027.7.3GUI Related Classes ................................................................................ 1037.7.4Other supportive classes .......................................................................... 103

CHAPTER 8 ..............................................................Error! Bookmark not defined.8 TESTING AND EVALUATION ........................................................................ 105

8.1Unit testing...................................................................................................... 1068.2Integration testing ........................................................................................... 1077.3 System Integration testing .............................................................................. 1077.4 Performance testing ..........................................Error! Bookmark not defined.7.5 Accuracy testing ............................................... Error! Bookmark not defined.7.6 Sample Unit Testing Test plans for eye region extraction module ...........Error!

Bookmark not defined.REFERENCE ........................................................................................................... 132APPENDICES .............................................................................................................. iA.Image Processing Techniques ................................................................................. iiB.Eigenface Step by Step (Formulas) ........................................................................ vi


9/169

ix

LIST OF FIGURESFigure 1.1: Altered faces. ............................................................................................. 3Figure 1.2: Normal face recognition process ............................................................... 3Figure 1.3: Proposed solution overview ...................................................................... 4Figure 2.1: Configuration of generic face recognition ................................................. 8Figure 2.2: Google Picasa ............................................................................................ 9Figure 2.3: APPLE I photo face recognition .............................................................. 10Figure 2.4: Face.com Photo Tagging ......................................................................... 11Figure 2.5: Face Book Photo Tagging ....................................................................... 11Figure 2.6: Different pose Audio Visual Technologies ............................................. 12Figure 2.7: Different illuminations ............................................................................ 13Figure 2.8: Different facial expression....................................................................... 13Figure 2.9: Different facial occlusions ....................................................................... 13Figure 3.1: Image representation in plane, As a matrix ............................................. 15Figure 3.2: Image segmentation approach ................................................................. 16Figure 3.3: Low noise object/background image histogram ...................................... 18Figure 3.4: Canny edge detection process.................................................................. 21Figure 3.5: Masks ....................................................................................................... 22

Figure 3.6: Sober Template ........................................................................................ 22Figure 3.7: Geometrical features ................................................................................ 26Figure 3.8: Template matching strategy..................................................................... 27Figure 3.9: Template matching .................................................................................. 28Figure 3.10: Template ................................................................................................ 29Figure 3.11: Test one Grey-Level Template matching .............................................. 29Figure 3.12: Test two Grey-Level Template matching .............................................. 30Figure 3.13: Test three Grey-Level Template matching ............................................ 30Figure 3.14: Test four Grey-Level Template matching ............................................. 30Figure 3.15: Test Five Grey-Level Template matching ............................................. 31Figure 3.16: Test Five template is not available ........................................................ 31Figure 3.17: Rectangle Features ................................................................................. 33Figure 3.18: The features selected by AdaBoost ....................................................... 33Figure 3.19: Cascade of classifiers............................................................................. 34Figure 3.20: Features selected by AdaBoost .............................................................. 36
http://c/Users/jumbo/Desktop/Mid%20Dox%20Modifyed.doc%23_Toc267629895http://c/Users/jumbo/Desktop/Mid%20Dox%20Modifyed.doc%23_Toc267629895http://c/Users/jumbo/Desktop/Mid%20Dox%20Modifyed.doc%23_Toc267629895http://c/Users/jumbo/Desktop/Mid%20Dox%20Modifyed.doc%23_Toc267629895


10/169

x

Figure 3.21: Elastic branch graph matching face recognition.................................... 39Figure 3.22: Face image transformation .................................................................... 44Figure 3.23: Distribution of faces in image space...................................................... 44Figure 3.24: Faces in face space ................................................................................ 45Figure 3.25: Overview of Eigen face Approach ........................................................ 46Figure 3.27: Representation of A x =b. ..................................................................... 47Figure 3.26: Vector .................................................................................................... 47Figure 3.28: Representation of A v = v. .................................................................. 48Figure 3.29: Training Images ..................................................................................... 50Figure 3.30: Example images from the ORL database .............................................. 53Figure 3.31: Mean face obtained from the ORL database ......................................... 53Figure 3.32: Eigenfaces .............................................................................................. 54Figure 5.1: SDLC Model ........................................................................................... 65Figure 5.2: Incremental model ................................................................................... 66Figure 5.3: Hybrid of waterfall and incremental model ............................................. 66Figure 5.4: Prototyping model ................................................................................... 67Figure 5.5: Evolutionary Prototyping ........................................................................ 68Figure 6.1 : Dot net Framework ................................................................................. 73Figure 6.2: Dot net Code Execution process .............................................................. 73Figure 6.3: Java Architecture ..................................................................................... 77Figure 7.1: Use Case Diagram ................................................................................... 87Figure 7.2: System Architecture ............................................................................... 95Figure 7.3: Activity DiagramAdd Record .............................................................. 96Figure 7.4: Activity DiagramView Record ............................................................ 97Figure 7.5: Activity DiagramTraining Image Space .............................................. 98Figure 7.6: Activity DiagramFind Match............................................................... 99Figure 7.7: Activity Diagram - Automatic Face Region Verification ..................... 100Figure 7.8: Activity Diagram - Pre-processing/Normalization ................................ 100Figure 7.9: Class Diagram ....................................................................................... 101
http://c/Users/jumbo/Desktop/Mid%20Dox%20Modifyed.doc%23_Toc267629920http://c/Users/jumbo/Desktop/Mid%20Dox%20Modifyed.doc%23_Toc267629920http://c/Users/jumbo/Desktop/Mid%20Dox%20Modifyed.doc%23_Toc267629920http://c/Users/jumbo/Desktop/Mid%20Dox%20Modifyed.doc%23_Toc267629920


11/169

xi

LIST OF TABLES

Table 3.1: Detection rates for various numbers of false positives on the test set ...... 34Table 3.2: Comparison of face detection approach .................................................... 38Table 3.3: Recognition results between different galleries using EGBM .................. 40Table 3.4: Recognize and Detect Faces ..................................................................... 56Table 6.1: Comparison of Programming languages ................................................... 76Table 6.2: Comparison of matrix libraries ................................................................. 81Table 7.1: Use case description - Add New User ...................................................... 89Table 7.2: Use case descriptionView Record ........................................................ 90Table 7.3: Use case descriptionTrain Image Set .................................................... 91Table 7.4: Use case descriptionFind Match ........................................................... 92Table 7.5: Use case descriptionSet Configuration ................................................. 93Table 7.6: Use case descriptionModify Record ..................................................... 94Table 8.1: Test Case : Eye region detection ...............Error! Bookmark not defined.Table 8.2: Test Name : Eye region Extraction ...........Error! Bookmark not defined.


12/169

xii

LIST OF ABBREVIATION

1D One Dimensional Vector

2D Two Dimensional

3D Three Dimensional

ADO Activex Data Objects

API Application Programming Interface

ASP Active Server Page

CFA Correlation Feature Analysis

CPU Central Processed Using

EBGM Elastic Brunch Graph Matching

GUI Graphical User Interface

KCFA Kernel Correlation Feature AnalysisLDA Linear Discriminate Analysis

LINQ Language-Integrated Query

MACF Minimize Average Correlation Filter

MSE Mean Square Error

NSTC National Science And Technology Counsel

OOP Object Oriented Programming

PC Personal Computer

PCA Principal Component Analysis

SDLC System Development Life CycleSQL Structured Query Language

SVM Support Vector Machine

VB Visual Basic


13/169


14/169

2

researchers and vendors for face recognition. Still there are lot of areas to improve in

automated face recognition.

1.2Understanding the ProblemHowever, when it come to the real world, it might not be possible to capture a full

frontal picture of a face at all the times in uncontrolled environments. Even though

there are many face recognition systems available, most of these work in optimal

conditions. Especially without full frontal face, these systems fail to recognise a face.

As a result of this most of the systems cannot give an accurate face match result.

Because of that, there can be lot of complications in identifying a person in an

image.

As an example in the news article written by Willing (2003) indicates failures of

implementing face recognition systems in airport at Logan USA. Furthermore,

Willing (2003) stated that there were similar project that had to shut down because of

incapability of giving accurate face matches due to different reasons.

One of the reason identified was criminals intentionally alter their appearance using

disguises to defraud law enforcement and the public. Also because of the influences

of different cultures and environment factors sometimes it is not possible to expose

full face. In those situations normally, face recognition approaches fail to give well

accurate result.

There are various reasons for the failings of the current systems. One possible reason

is they cannot identify disguised faces comparing with full faces because current

approaches are not capable of identifying individuals using partial face regions

which have fewer characteristics in small face region comparing with full faces.


15/169

3

Figure 1.1: Altered faces.

(a) Muslim girl wearing veil [Source: Chopra, 2007]. (b) Women wearing masks

[Source: Tarlow, 2007]. (c) Person wearing sunglass [Source: University of Buenos

Aires ,n.d].

As mentioned before most of face recognition solutions cannot recognise

individuals faces, which are intestinally occlude. Therefore, it is understood that an

affective and robust face recognition system should be capable of identifying

partially occluded faces including other regular features.

1.3Sub ProblemsAutomated face recognition is a sequence of process. As mentioned in

MyHeritage.com (2006) the following diagram shows how face recognition systems

are functioning.

Figure 1.2: Normal face recognition process

By understanding the above, there are lot of sub steps performed within the above

steps. Although there are lot of approaches for full-face recognition, the proposed

solution deviates from the regular face recognition approaches. Therefore, when it

takes the suggested solution the overall process of the face recognition applies but

internal process sequence is different from the regular face recognising process.

Acquire Images

form video or

still camera

Detect and

extract face area

form the image

Match extracted

face against images

in database


16/169

4

1.4Project ObjectivesThis face recognition system will identify individuals based on characteristics of

separate face segmentations and the objectives of the project as follows.

Investigation of unique face features of eye, nose and mouth regions for

recognises individuals.

When it come to separate face regions there are less unique features

that help to identify individuals. Identifying unique features of the

individuals has being archiving throughout this project.

Improve capabilities of the detecting features of local segmentations of face

It is necessary to find the efficient algorithm to extract features of the

face segmentations.

Implement robust, efferent face recognition system based on facts found in

the research.

A thorough research being carried on face recognition techniques and

available algorithm on partial face recognition and choose a

appreciate method and implement face recognition system based in it.

1.5Proposed Solution

The purposed solution takes a segment of a face (eye region, nose region or mouth

region) at a time and identifies which has submitted region. Based on the input

region it will extract the features that are unique to each region. Then it will extract

particular face region form the faces in database. After that, it will match the

features, which have extracted from both two-face region.

Face Region

Identifier

Face Region

Extractor

Region

Feature

Extractor

Database

face Features

Submitted

Image face

Face

Matcher

Results

Figure 1.3: Proposed solution overview


17/169

5

1.6Project ScopeThe Scope of this Project is to develop a system that allows recognise individuals

using partial regions of face. Initially the project is based on recognise individuals

using submitted face regions.

The scope is as follows:

This project is only focus on identifying individuals using only eye region

but as external functionality, it will try to implement it to mouth region.

The all images have taken in 0-degree camera angle, which mean all images

are frontal view images, and taken in controlled environment.

The eyes should be open and mouth should close in the faces including both

input image and images in database. All faces should be in neutral mood.

It assumed that input image of the system (The image to be recognised) is

pre scaled and it is not necessary to scale back.

Furthermore, the solution will not work on recognising in following situations

Disguised faces, which cannot use for extract at least one of eye region or

nose region or mouth region.

Different poses of face regions.(taken in different angles)

Low quality images ( less resolutions, etc)

Different facial expressions

1.6.1Functionality of the Artefact1.6.1.1 Core FunctionalityCore Functionality of the artefact can be identified as follows. The main

functionality of the solution is recognize individuals using partial face segments.

During this research and implementation, it focuses on face recognition using eye

region of a person. Because of that which will mention as partial face recognition as

onward here consider as partial face recognition using eye region.


18/169

6

Furthermore, it can be divide as follows

Input identification

This will identify whether input region is a face region or not and if it is a

face region what is the region.

i.e.

User input an eye region then first at all it will check whether it is a face

region if yes it will identify what is the face region.

Face region detection

This will detect the particular face region of full frontal faces in database and

extract particular face region from the faces

Face match

This will match submitted face region with extracted face regions in database

and provide match result.

1.6.1.2Extra functionality Face recognition using mouth region

Full frontal face recognition

Detect and extract different face regions from submitted full

frontal face and recognize individuals based on partial face

regions.

The proposed artefact will develop a standalone application, which runs on PC and a

project report, which include the details of the project.


19/169

7

1.7Constrains1. Camera view

2. Inadequate resolution of the images

3. Lighting condition

4. Scale of the image

5. Distance between camera and person

6. Facial expressions

7. Occlusion

8. Ageing

1.8Assumptions1. The input images, which submitted to system and images in database are in

same scale.

2. The input image has extract manually based on given dimensions.

3. Images have taken in controlled environment, which avoids different lighting

conditions, variance of view, variance distance between camera and person.

4. All images have constant resolution.

5. All faces have taken in neutral facial experience.

6. The faces areas taken into recognize have not occluded.

7. It is assume that all images taken are in same age because of that the distance

of each features of the face does not change.


20/169

8

CHAPTER 2

2 DOMAIN INVESTIGATION2.1Outline of the generic face recognition systemTypical face recognition systems recognize faces based on various factors and

technologies. Although different face recognition systems use different approaches,

most of them perform key main steps, which are common to most face recognition

approaches.

As mentioned by Zhao et al.(2003), a generic face recognition system consist three

main processing steps. Face detection that involve detecting and separation face

region from submitted face image or video (which is an image sequence), feature

extraction which identifies and extract features of the submitted images. Furthermore

Zhao et al.(2003) described that feature can be local features such as lines or

fiducial points, or facial features such as eyes, nose, and mouth. . The next and final

step is recognising faces by matching input image against the faces in database.

Figure 2.1: Configuration of generic face recognition

Face Detection

Feature

Extraction

Face

Reco nition

Input Images

and Video


21/169

9

2.2 Similar systemsFace recognition systems dates back in history of over 5 decades which starts form

1960. At Present, face recognition has achieve an excellent development and gained

attention from areas such as surveillance, biometric identification etc.

Because of that, there are many systems that have been developed using face

recognition technology but during the research, researchers could not find a system

which is capable of partial face recognition.

2.2.1Google PicasaGoogle Picasa is a photo organizing and editing software, which is a free application

initiated by Google. One of newest feature of Picasa is face recognition feature.

According to Baker (2006) Google Picasa use Face recognition approach called

NevenVison for recognition faces.

Baker (2006) also mentioned that Neven Vison is very advanced techniques and it

covers over 15 patents. Furthermore Neven Vison is not only for face recognition

but also for object recognition.

Figure 2.2: Google Picasa

[Source: Waltercedric, n.d. ]


22/169

10

2.2.2APPLE IPhotoApple IPhoto can be consided as an alternative to Google Picasa in Mac Os

environment. It also has similar features as Google Picasa. In Apple IPhoto product

page describes face recognition feature as follows.

Apple Inc (2010) stated that iPhoto introduces Faces: a new feature that

automatically detects and even recognizes faces in your photos. iPhoto uses face

detection to identify faces of people in your photos and face recognition to match

faces that look like the same person.

Figure 2.3: APPLE I photo face recognition

[Source: Lee, 2009]


23/169

11

2.2.3Face.Com automatic face book photo taggerAccording to Bil (2010) Face.com provide auto face tagging ability to facebook

photo albums. It will check all the photos available in users photo collection in

facebook and tag user and friends.

Figure 2.4: Face.com Photo Tagging

[Source: Bil, 2010]

Figure 2.5: Face Book Photo Tagging

During the experimentation, the researcher has identified that this application gives

poor results. Especially when it try to identify images in different lighting condition.


24/169

12

2.3 Problem of face recognition systemsEven thought face recognition is five decades old. In study filed of face recognition

system there are lot of unsolved problems and biometric identification area. At times

most of the approaches of the face recognition, which is currently in use and in

experimenting stages, can only give solutions for few problems in face recognition.

During the research, the researcher could not find a robust face recognition

algorithm, which gives 100 % accurate results.

Generally, the following situations can categorize as problem in face recognition.

2.3.1Pose variancePose variance mean difference of the camera angel that takes photos (Image).

Figure 2.6: Different pose Audio Visual Technologies

[Source: Audio Visual Technologies Group, n.d.]


25/169

13

2.3.2Lighting/Illumination varianceDifferent lighting affects to recognition.

Figure 2.7: Different illuminations


2.3.3Different facial expression

Figure 2.8: Different facial expression


2.3.4Different facial occlusions

Figure 2.9: Different facial occlusions



26/169

14

2.3.5Individual characteristicsFace recognition systems might depend on some face characteristics such as skin

colour. Some time face recognition systems, which are designed for Caucasianmight

not be capable of handling faces of other races.

In addition, gender can also consider as another factor sometime face recognition

system that is use for females lip identification might not work for males.

2.3.6Scale Invariance and robustnessThe face scale depends on the distance between person and camera. Therefore, some

time it is not capable of handling faces taken at different distance.

Apart from that, face recognition requires lot of computational power which should

be considered because although there are high-end computers which are capable of

handling such process, they are reasonably expensive.


27/169

15

CHAPTER 3

3 SYSTEM RESEARCH AND INVESTIGATION3.1Fundamental of Image processingDigital image processing can be considered as one of the interesting area of the

computer vision. Image processing based on concepts on mathematics. Following

section shows fundamentals of image processing.

3.1.1Image classificationsAccording to Jhne (2005, pp32) there are entirely different ways to display an

image. The most popular way to represent an image is rectangular grid. Gonzalez

(2004, pp 01) further described it as follows; digital images can be represent as a

matrix as it can be represent as 2D function where x and y gives coordinates in

geometric plane.

Figure 3.1: Image representation in plane, As a matrix

[Source: Gonzalez, 2004, pp 01] & [Source: Jhne , 2005 , pp32]

According to (Gonzalez, 2004, pp 01) Images can be categorize as follows,


28/169

16

3.1.1.1Raster ImageAs defined by Busselle( 2010) Raster images known as bitmap images. Raster

images are made by collection of dots, which called pixels. Pixels are tiny scares that

contain colour information about the particular location of the image. These images

are resolution dependent.

3.1.1.2Vector ImageAs defined by Busselle( 2010) vector images are collection of connected lines and

curves. Each line and curves defined by mathematical formula. Therefore, these

images are resolution independent and it has the ability to control mathematical

formula according to the scale.

3.1.1.3Binary ImageBinary images represent images by using 0 and 1.The pixel of the object represent as

1 and background as 0. By using threshold, a digital colour image could convert

into a binary image. Threshold is a method that uses to segment an image.

3.1.2Image segmentationAccording to Grady & Schwartz (2006), Image segmentation has often been

defined as the problem of localizing regions of an image relative to content.

According to Biswas (2008) Image segmentation approaches can divide as two

different categories as follows.

Image segmentation

Discontinuity based Region based

Figure 3.2: Image segmentation approach


29/169

17

Furthermore Biswas (2008) and Yang (2004) described discontinuity based image

segmentation is focused on Isolated points, lines and edges detection while region

based / similarity based approach is focus on thresholding.

Spann (n.d) mentioned that, grey level histogram can consider as one of most

popular image segmentation method among other techniques.

3.1.2.1Grey level histogram-based segmentationImage histogram is a graph which shows the size of the area of the image that is

captured for each tonal variation that the camera is capable of recording (Sutton,

n.d).

Spann (n.d) mentioned that Thresholding, Clustering are image segmentation

techniques that are based on grey level histograms.

Noise is unwanted external information that is in image. First the noise should be

filter out form the image or it should consider when it processing the image.

Please refer appendix for Identifying Noise form histogram.

3.1.2.2ThresholdingThe operation known as simple thresholding consists in using zero for all pixels

whose level of grey is below a certain value (called the threshold) and the maximum

value for all the pixels with a higher value (kioskea.net , 2008).

As mentioned previously, this is a most popular method in image segmentation. As

mentioned by kioskea.net(2008) it classify an image based on pixel value of the

image. It checks whether gray values of a particular picture is greater than or less

than Threshold point and convert it to binary image by applying 1 or 0 to every

pixels in image.


30/169

18

3.1.2.2.1 Grey level thresholdingGrey level thresholding defined whether particular grey pixel belongs to the object or

to the background. By using this method, it is possible to separate an object form the

background.

Figure 3.3: Low noise object/background image histogram

[Source: Spann ,n.d]

Spann (n.d) defined grey level thresholding algorithm as follows.

If the greylevel of pixel p


31/169

19

3.1.2.2.2 Determine thresholdSpann (n.d) mentioned that to determine threshold there are several approaches but

following approaches shows high interaction.

Interactive threshold

Adaptive threshold

Minimisation method

Please refer appendix for detailed explanation for approaches for determiningthreshold.

3.1.3Edge detectionNeoh and Hazanchuk (2005) mentioned, Edge detection is a fundamental tool used

in most image processing applications to obtain information from the frames as a

precursor step to feature extraction and object segmentation. In addition, they

further explained it as follows This process detects outlines of an object and

boundaries between objects and the background in the image. According to Green

(2002), Edge detecting an image significantly reduces the amount of data and filters

out useless information, while preserving the important structural properties in an

image.

Green(2002a) categorized edge detection approaches into two main category.

Gradient and Laplacian. According to Green(2002) gradient method detects the

edges by looking for the maximum and minimum in the first derivative of the

image and The Laplacian method searches for zero crossings in the second

derivative of the image to find edges .

The relationship between edge detection and threshold is once image is binarized the

next step is applying threshold by doing that it can decide whether edges are

available or not.

If the threshold is lower, more edges can be found and high threshold might cause to

loss of edges.


32/169

20

3.1.3.1Cannys edge detectionAccording to Fisher et al (2003) Cannys edge detection is algorithm which consider

as optimal edge detector. Furthermore Fisher et al (2003) mentioned, It takes as

input a gray scale image, and produces as output an image showing the positions of

tracked intensity discontinuities.

According to Green (2002b) there are few criteria that should consider.

Edges occurring in images should not be response to non-edges and edges

in images should not be missed identify. The algorithm that takes to

identify the edges must to identify (mark) the all real edges in the image. The edge points must to be well localized. The distance between the edge

pixels must to be well organized and close to the edges in the real image.

Must to well identify the noisy images before using the edge detection

algorithm. Then have to filter the noisy picture first using appropriate

filers.

Green (2002b) applied following steps to detect edges using Cannys edge detection.

Step 1 Filter out any noise in the original image

Green (2002b) used Gaussian filter to remove noise.

Step 2 Find the edge strength

Green (2002b) has achieved by finding the gradient of the image. To do that

he perform the Sobel operator 2-D spatial gradient measurement on an

image.

Step 3 Finding the edge direction

Green (2002b) stated that achieving this is a trivial task.


33/169

21

Step 4

Once the edge direction is known, the next step is to relate the edge

direction to a direction that can be traced in an image (Green, 2002b).

Step 5

After the edge directions are known, non maximum suppression now has to

be applied (Green ,2002b).

Step 6

Then it use hysteresis to determine final edges. He did it by using suppressing all

edges, which does not connect to selected edge.

Figure 3.4: Canny edge detection process

[Source: szepesvair, 2007, pp19]

3.1.3.2Sobel Edge DetectionAccording to (Green, 2002a ) The Sobel operator performs a 2-D spatial gradient

measurement on an image. Typically, it is used to find the approximate absolute

gradient magnitude at each point in an input greyscale image.

(Green, 2002a) and Biswas(2008) Describe it further as Sobel detector can consider

as algorithm that performs two dimensional gradient measurements on an image.


34/169

22

This technique achieve by calculating the estimated gradient magnitude at each point

in a gray scale image.

In the article presented by Green (2002a).He mentioned that the Sobel edge detector

used a pair of 3 x 3 convolution masks. One of mask estimate gradient in the x-axis

other mask estimating the gradient in the y-axis. Normally mask is smaller than the

actual image because of that this mask is slid over the image. Because of that, it

manipulates a square of pixel at a time.

The following figure shows an example of Sobel template.

Figure 3.5: Masks


Figure 3.6: Sober Template


Following formulas presented by Green (2002a).

The formula to find magnitude of the gradient


35/169

23

Approximate magnitude of the gradient

It has identified that by using grey-level histogram segmentation and applying edge

detection like canny edge detector, it is possible to extract face regions. According to

view of researcher, this method can be useful in extraction face regions from the

face but it might not give accurate results because histogram thresholding value

might depend on gender and race.


36/169

24

3.2Face recognition approachesThe first face recognition approach was introduced by Sir Francis Galton in 1888

(Kepenekci, 2001) which was done by measuring four characteristics of French

prisoners. Because of that attempt of Sir Francis Galton to identify persons in more

scientific way, it made foundation of the biometric recognition which cause to

improvement of face recognition.

As mentioned before there are different approaches for face recognition. Those

approaches can divide into main section based on the image type. Those are two-

dimensional and three dimensional face recognition.

3.2.1Three dimensional face recognition3D model based face recognition can consider as latest trend of the face recognition.

As mentioned by Bonsor & Johnson (2001) , this face recognition approach uses

distinctive features of the faces. Which mean features like dept of the face features,

curves of the eyes, nose and chin use to recognise a faces.

Akarun, Gokberk, & Salah (2005) stated that three dimensional face recognition has

higher accuracy rate over two dimensional traditional face recognition but when it

comparing with traditional face recognition approach it has few disadvantages such

as cost of implementation , unfamiliarity of technology and lack of hardware

specially camera equipments. Because of that, still 2D face recognition has higher

demand.

As mentioned by Gaokberk (2006) there are different approaches that use in 3D face

recognition. Point cloud-based approaches, depth image-based approaches, curve-

based approaches, differential geometry-based approaches, facial feature-based

geometrical approaches and shape descriptor-based approaches can consider most

popular approaches in 3D face recognition.

2D (Two-dimensional) face recognition can consider as good alternative to 3D face

recognition that has developed over 50 years. Following section will describe about

the 2D face recognition approach.


37/169

25

3.2.2Two dimensional face recognitionTwo dimensional face recognition can be considered as the most oldest and popular

method that is currently using in the field. This approach can process 2D images that

have taken in regular cameras (probably cameras in security systems) and identify

faces based on different approaches. As mentioned by NSTC subcommittee on

biometrics (2006) some of algorithms use faces landmarks to identify faces while

other algorithm use normalized face data to identify probe image.

Based on NSTC subcommittee on biometrics (2006) explanations 2D face

recognition approaches can categories as follows.

Geometric based approach in other words feature based approach that

consider face feature such as nose, mouth and eyes to identify faces.

Photometric based approach or view based approach that consider images as

set of pixels and compare characteristics of those pixels in both images.

3.2.3Geometric based approachesGeometric based face recognition techniques involve processing and computation

face land marks in other words it use face features such as relative positions of nose,

mouth and eyes, distance between landmarks, etc to identify individuals.

As stated by Brunelli and Poggio (1993) the idea behind geometric face recognition

is extract relative position and other parameters of distinctive features such as eyes

.After finding the features they will match with the known individuals featuresdetails and find out the closest distance of the matches. According to Kanade (1973)

the research has achieved 45-75% success recognition rate with 20 test cases

.According to Brunelli and Poggio (1993) disadvantage of geometric face

recognition over view based face recognition is extraction of face features.

The following image shows some of face features that can use for recognize faces.


38/169

26

Figure 3.7: Geometrical features

[Source: Brunelli & Poggio , 1993]

However, with the modern techniques and approaches, face feature detection has

gained lot of improvements comparing with old day. As mentioned by Bagherian,

Rahmat and Udzir (2009) extraction the features can do by different approaches such

as colour segmentation approach, template based approach that use pre-defined

image to detect face features and relative locations.

With development of appearance based face recognition approach geo metric face

recognition approach has deviate from the face recognition area and the techniquesused by geometric face recognition have used to different areas such as facial

expression detection ,etc.

3.2.4Appearance based approachThe second approach of the face recognition is appearance based face recognition,

which is quite popular among researchers and industries relate to machine vision. In

this approach, it takes probe face as a set of pixels into a matrix and compare with

the known individuals. Brunelli and Poggio (1993) described it as follows the

image, which is represented as a bi dimensional array of intensity values, is

compared using a suitable metric with a single template representing the whole

face. The above-mentioned approach is only an overall picture of the appearance

based recognisor approach. There are more sophisticated ways of performing

appearance based face recognition.


39/169

27

Figure 3.8: Template matching strategy

[Source: Brunelli & Poggio ,1993]

Based on NSTC subcommittee on biometrics (2006) statistical face recognition

approach can be categorise as follows Principal-component analysis (PCA) , Linear

discriminate analysis (LDA) and elastic brunch graph matching .

The different techniques use different approaches, algorithm in recognising process.

Some time as mentioned by Zhao et al (2003) hybrid methods like Modular

eigenfaces, Hybrid LFA, Shape-normalized Flexible appearance models,

Component-based. Therefore, finally the following conclusion can make that facerecognition can have different approaches some time there can be novel approaches

that use combination of different techniques.

During this project, it considers only 2D face recognition because it is not faceable to

find enough resources to do 3D face recognition. In 2D face recognition, it would

constraint on view based face recognition because by using geo metric face

recognition the data (distance between features) which can use to recognize is not

enough.

i.e:

If it takes eye region, it can only take few measures. Like distance between I corners,

width of eyelid, etc. These measures can be verified because of that this method is

not good reliable measure.


40/169

28

3.3Face region extraction approachesThe author has identified several approaches to extract face features during the

research. Among them following techniques show promising results.

3.3.1Template matchingAccording to Latecki( n.d.) Template matching compares two images (potions of

Images) and find out similarity between them. Nashruddin(2008) elaborated it as

follows Template matching is a technique for finding small parts of an image which

match a template image. It slides the template from the top left to the bottom right of

the image, and compare for the best match with template. The template dimensionshould be equal or smaller than the reference image. He also motioned that there are

many methods for template matching.

Latecki (n.d.) Explained template matching by using following the diagrams.

Figure 3.9: Template matching

[Source : Latecki , n.d.]

According to Latecki (n.d.) in here, the matching processes changes the position of

the template image to all possible positions of the source image. Then it computes a

numerical index that indicates how well the template matches the image in that

position. Match is done on a pixel-by-pixel basis.

Latecki (n.d) described two type of template matching


41/169

29

3.3.1.1Bi-Level Image template matchingIt use Bi-Level template matching identify whether particular object is available in

particular source image. Letecki(n.d) stated that this methods only suitable for giving

yes or no (only for checking availability of a image).

3.3.1.2Grey-Level Image Template matchingInstead of Bi-level Image template, matching it checks difference level of image is

used. To measure that correlation can use.

Letecki(n.d) has conducted an experiment on five data set and achieved following

correlation maps.

Figure 3.10: Template

[Source: Letecki , n.d.]

Figure 3.11: Test one Grey-Level Template matching



42/169

30

Figure 3.12: Test two Grey-Level Template matching


Figure 3.13: Test three Grey-Level Template matching


Figure 3.14: Test four Grey-Level Template matching



43/169

31

Figure 3.15: Test Five Grey-Level Template matching


Figure 3.16: Test Five template is not available


Letecki (n.d.) has used following formula to calculate the correlation

1

0

1

0

22

1

0 )(N

i

N

i

ii

i

N

i i

yyxx

yyxxcor

x is the template gray level image x is the average grey level in thetemplate image

y is the source image section is the average grey level in the sourceimage

N is the number of pixels in the sectionimage

(N= template image size = columns *rows)


44/169

32

The value cor is between 1 and +1, with larger values representing a stronger

relationship between the two images.

If the correlation value is less than correlation values of template then there will not

be a template image in source.

By using grey level template matching it is possible to match face regions but it

might not get accurate results because intensity values of the faces can be depend of

the race and gender and computational time can be high because template matching

use pixel based calculations. However, by limiting variance it is possible to perform

this approach to face region extraction.

As a alternative to pixel based calculation Violla & Jones (2001) proposed feature

based approach. Following section describe about it.

3.3.2Haar like features - Viola-Jones face detectorViolla & Jones (2001) proposed an approach to detect objects using a boosted

cascade of simple features. They introduced new image representation method called

Integral image according to Violla & Jones (2001) integral images allow

processing the data other than using raw images. Then they have used a learning

algorithm AdaBoost to select visual features from a larger set and yields extremely

efficient classifiers. Then they combined successively more complex classifiers in a

cascade structure, which dramatically increases the speed of the detector by focusing

attention on promising regions of the image.

Violla & Jones (2001) approach they have used features that generated by haar basis

functions. According to Violla & Jones (2001) the reason for using features are that

features can act to encode ad-hoc domain knowledge that is difficult to learn using a

finite quantity of training data. Furthermore, that has used three types of features.

Two rectangle feature , three-rectangle features and four-rectangle features which are

respectively the difference between the sum of the pixels within two rectangular

regions , computes the sum within two outside rectangles subtracted from the sum in

a centre rectangle , computes the difference between diagonal pairs of rectangles.


45/169

33

Figure 3.17: Rectangle Features

[Source: Violla & Jones , 2001]

According to Violla & Jones (2001) the reason for using above mentioned integral

images are rectangle features can be computed very rapidly using an integral image.

By using number of negative (which does not include false faces) and positive

images (which does include faces) to the AdaBoost has trained for extract the

features.

Figure 3.18: The features selected by AdaBoost

[Source:Violla & Jones 2001]

The two features are shown in the top row and then over layed on a typical training

face in the bottom row. The first feature measures the difference in intensity between

the region of the eyes and a region across the upper cheeks. The feature capitalizes

on the observation that the eye region is often darker than the cheeks. The second

feature compares the intensities in the eye regions to the intensity across the bridge

of the nose (Violla & Jones 2001).


46/169

34

In the end, they have formed the cascade, which can be, identify as a decision three,

which has classifier network to filter our negative results and provide a well accurate

result. The following diagram shows structure of a cascade.

Figure 3.19: Cascade of classifiers


The approach, which is proposed, by Violla & Jones (2001) has 38 stages with over

6000 features.

They have trained each classifier of the cascade by 4916 faces and 10,000 non-faces

size of the all images are 24 x 24.

Table 3.1: Detection rates for various numbers of false positives on the test set


This method can consider as good approach to face region extraction but identifying

face regions can be difficult because face can have lot of representation of

rectangular features therefore Cristinacce & Cootes (2003) proposed approach,

which can use to detect face features.


47/169

35

3.3.3Facial feature detection using AdaBoost with shape constraintsThis can consider as extend of face detection method proposed by Violla & Jones

(2001).

Cristinacce & Cootes (2003) proposed a method for facial feature extraction using

AdaBoost with shape constraints for locate the eye, nose, mouth corners in frontal

face image.

Their approach can divide into two main sections face detection and face

segmentation detection.

3.3.3.1Face DetectionCristinacce & Cootes (2003) used the face detection method used by Viola & Jones

to detect the faces for feature selection, which has described previously. Cristinacce

& Cootes (2003) described it as follows The output of the face detector is a image

region containing the face, which is then examined to predict the location of the

internal face features.

Cristinacce & Cootes (2003) deviated from Viola and Jones approach when selection

negative and positive images and building AdaBoost template using haar like

feature. Viola & Jones use human faces as positive examples and regions known

not to contain a human face as the negative examples. According to Cristinacce &

Cootes (2003) in this approach, the positive examples are image patches centred on a

particular facial feature and the negative examples are image patches randomly

displaced a small distance from the same facial feature.

3.3.3.2Local feature detectionCristinacce & Cootes (2003) performed same algorithm to locate the local features

detection. The following figure shows few features selected by AdaBoost for right

eye.


48/169

36

Figure 3.20: Features selected by AdaBoost

[Source: Cristinacce & Cootes,2003]

The other thing that the approach taken by Cristinacce & Cootes(2003) which is

extends approach taken by Viola & Jones(2001) is in this method it use shape

constraints to check candidate feature points. Cristinacce & Cootes (2003) described

it as follows Firstly a shape model is fitted to the set of points and the likelihood of

the shape assessed. Secondly, limits are set on the orientation, scale and position of aset of candidate feature points relative to the orientation, scale and position implied

by the global face detector.

The shape model they have used designed in similar way that proposed by Dryden

and Mardia(1998 cited Cristinacce & Cootes ,2003). Then Cristinacce & Cootes

aligned the points in to a common co-ordinate frame. After taking the distribution

According to Cristinacce & Cootes (2003) they got multi-variant Gaussian type of

distribution. Then they estimated probability of the give shape ps(x).Then they got

threshold T by comparing training dataset. So if probability of given shape is greater

than threshold they consider as a shape.

Cristinacce & Cootes (2003) has analysed to find the range of variation in position

of the features relative to the bounding box found by the full face detection.

Furthermore, the feature detectors what have implemented during this application

returns list of candidate points that probability of given shape is greater than

threshold.

During their research, they have achieved following recognition rates.

According to Cristinacce & Cootes (2003) the maximum time the entire process took

was less than .5 Seconds. They have archived 88.8% detection rate as follows.


49/169

37

Feature distance is within 5% of the eye separation for 65% of faces, 10% for 85%

of faces and 15% for 90% of faces Cristinacce & Cootes (2003).

This method shows promising results but to implement this method it requires lot of

time and effort because of AdaBoost training but during the technical investigation,

it found a method to overcome this problem. The method for implementation will

describe in later.

3.3.4Justification on selected technologiesIn the research, it discussed about viola-jones face feature detection using Haar-like

features, template matching approach, face detection and facial feature detectionusing AdaBoost approach and shape constraint. In Violla-Jones (Violla &

Jones,2001) approach, they have used features that generated by Haar based

functions. Features are rectangle areas, which represent the binary intensity values of

the faces. This method achieved 76.1% - 96.9% face detection rate. Cristinacce &

Cootes (2003) purposed extended version of Viola-Jones (Violla & Jones 2001)

approach for detect local features of faces. Cristinacce & Cootes (2003) have

achieved excellent results than Violla-Jones approach. Kuo & Hannah (2005).

purposed template-matching approach for eye feature extraction and they have

archived 94 % for iris extraction, 88% eye corners and eyelid extraction.

Both Violla-Jones (Violla & Jones, 2001) and Cristinacce-Cootes (Cristinacce &

Cootes, 2003) approaches used AdaBoost network to train the Haar cascades. In

addition, Kuo & Hannah (2005) approach does not use any neural network approach.

To train the AdaBoost they have used huge number of positive and nonnegative

images, which requires lot of time and approaches but once cascade files created it is

possible to reuse it for different set of data.

Kuo & Hannah (2005) mentioned that there approach is relatively inefficient because

this algorithm does not work in occluded faces and pose variance. In addition, the

algorithm is capable of identifying only eyes.

So above details can summarize as follows.


50/169

38

Violla-Jones Approach Cristinacce-Cootes

Approach

Template based Approach

High detection rate High detection rate High detection rate

Detect only face Improved to detect face,

eyes , nose and mouth

Eye region only

ANN training requires ANN training requires ANN training does not

requires

Table 3.2: Comparison of face detection approach

Researcher has identified there are trained haar-like feature cascade file which can

use for Violla-Jones Approach and Cristinacce-Cootes Approach. Then it does not

require use of AdaBoost training.

By considering above facts, it decided that to use haar-like features for detect the

face and face regions. Because by comparing above three techniques it shows high

accuracy rate and efficiency. The reason for rejecting template-based approach is the

accuracy of the template matching is depending on the template and testing set.

3.4 Face recognition methods3.4.1Elastic branch graph matching approachEBGM (elastic branch graph matching) approach is simple dynamical link

architecture implementation. As mentioned by NSTC subcommittee on biometrics

(2006) there are non-linear characteristics of face images that does not address by

linear methods like PCA, LDA that will describe next. By using EBGM it is possible

to avoid or minimize impact of nonlinear characteristics like poses elimination,

expressions lighting conditions, etc in face recognition.

Wiskott et al (1997) presented an approach which is based on geometrical local

faced based for face recognition. This approach is known as elastic brunch graph

matching. In this approach, faces are represented as labelled graphs. The total

concept behind face representation is Gabor wavelet transformation purposed by

Daugman (1989 cited by Wiskott et al 1997).


51/169

39

Furthermore, in the face graph, node represent fiducial points (eyes, nose, etc) and

edges represent distance between each fiducial points. Furthermore those points are

describe by sets of wavelet components (jets).

Figure 3.21: Elastic branch graph matching face recognition

[Source: Wiskott et al ,1997]

According to Wiskott et al (1997) image graph of new faces are extract using elastic

graph matching process that represent the face. According to Wiskott et al (1997) it

is vital to have accurate node positioning for face recognition to get accurate result

therefore it use phase information to get accurate node positioning. Object-adapted

graphs which use fiducial points representation handle rotations of objects (In thiscase faces) in depth.

The face graph is extraction base on bunch graph, which is combined representation

of all model graphs for cover wide variance, rather than one model does. Wiskott et

al (1997) also mentioned that use of individual separate models to cover variations

like such as differently shaped eyes, mouths, or noses, different types of beards,

variations due to sex, age, race, etc.

According to Wiskott et al (1997), the matching process involves few manual

supervising phase to build up a bunch graph. Then individuals can be recognise

based on learned system by comparing image graph and model graphs obtained from

the gallery images taken in the database. Then it takes graph, which has highest

similarity value.$


52/169

40

The above method purposed by Wiskott et al (1997) is not specific only for human

face recognition they have mentioned that it is possible to use this system for other

object recognition also. Furthermore, above method work fine in lot of variance

caused by size, expression, position and pose changes.

The experiment they have done by using FERET database they have achieved high

recognition rate over frontal - frontal (98 %) face recognition with and without

expression. They could archive 57% - 81 % recognition rate by comparing half

profile left side of the face and half-profiling right side of the face.

The following table shows the summery of their results which achieved in their

approach.

Model Gallery Probe images First Rank First 10 Rank

Noofsu

ccess

Image

success

percentag

e

Noofsu

ccess

Image

success

percentag

e

250 fa 250 fb 245 98 248 99

250 hr 181 hl 103 57 147 81

250 pr 250 pl 210 84 236 94

249 fa + 1 fb 171 hl + 79 hr 44 18 111 44

171 hl + 79 hr 249 fa + 1 fb 42 17 95 38

170 hl + 80 hr 217 pl + 33 pr 22 9 67 27

217 pl + 33 pr 170 hl + 80 hr 31 12 80 32

Table 3.3: Recognition results between different galleries using EGBM

[Source: Wiskott et al ,1997]

f: frontal views a, b: expression h: half-profiles p: profiles l, r: left and right


53/169

41

It has identified that this method is suitable for multi-view based face recognition but

like geometric based face recognition approach, this cannot apply for partial face

recognition because of lack of information for face recognition.

3.4.2Appearance based subspace methodsAs stated by Delac, Grgic and Liatsis (2005) there are several statistical (appearance

based) methods have been purposed. Among them PCA and LDA perform major

roles. According to Navarrete and Ruiz-del-solar (2001) in subspace method it is

project the input faces onto a reduced dimensional space where the recognition is

carried out, performing a holistic analysis of the faces.

Furthermore, Navarrete and Ruiz-del-solar (2001) stated that PCA and LDA could

consider as above projection methods that reduce high dimensional image space to

low dimensional image space. Heseltine (2005) elaborate it further as follows PCA

,LDA and other methods can be use to image subspace projection in order to

compare face images by calculating image separation in a reduced dimensionality

coordinate space. .He also mentioned that it use a training set of face images in

order to compute a coordinate space in which face images are compressed to fewer

dimensions, whilst maintaining maximum variance across each orthogonal subspace

dimension.(Heseltine, 2005).

3.4.2.1What is SubspaceAccording to Moghaddam(1999) visual data can be represented as points in a high -

dimensional vector space. For example, a m m-by-n n pixel 2D image can be mapped to

x Rmn

a vector, by lexicographic ordering of the pixel elements Which mean animage that has m by n pixels can represent as a matrix / vector which has NxM

dimensions.

According to Yambor (2000) the images projected into a subspace, then it creates

combined all subspace into a one which has all training images subspaces then the

test (probe) image project into subspace. After that, each test image compares with

the training images by a similarity or distance measure. The most similar or closest

images identify as the match image.


54/169

42

3.4.3Support vector machine kernel correlation feature analysis methodSavvides et al (2006) proposed a method for recognize partial faces and holistic

faces based on kernel correlation feature analysis (KCFA) and support vector

machine.

Here they have used correlation filter to extract features form the face images and

face regions. Minimize average correlation filter which has designed to minimize the

average correlation plane energy resulting from the training images Xie(2005 cited

Savvides et al 2006) has used in this approach to output required correlation peak

values that need to identify correlation peeks that match to face and face regions that

need to extract.

But as mentioned by Savvides et al (2006) stated that it is necessary to have generic

data set to build a correlation filter but up to this stage they had not used any generic

data set. Therefore, they proposed use of novel method call class dependent feature

analysis to overcome limitation of making generic dataset.

According to Savvides et al (2006) Class dependent feature analysis (CFA) is a

method that use to dimensionality reduction and feature extraction, which is similar

to PCA (Principle component analysis).They further stated that their proposed class

dependent feature analysis method achieved higher success rate than traditional

PCA. As mentioned by Savvides et al (2006) PCA address all number of images in

while CFA address number of classes ( here class mean set of image of same

individual).

According to Savvides et al (2006), they have designed one correlation filter which

is a minimize average correlation filter (MACE).First they have trained MACE set

using 12776 images of 222 different classes. Because of that, they have received 222

dimensional correlation feature vector after projection of input images to correlation

filters, which have generated by MACE.

Based on description provided by Savvides et al (2006) because of Non linear

attributes of face images such as lighting condition, pose linear subspace

methods(PCA and LDA) cannot perform well because of that those non linear


55/169

43

subspace method deviate to map non linear attributes in higher dimensional feature

space. Again, the calculations in higher dimensional feature space cost lot of

computational power. Therefore, kernel trick methods use to perform those

calculations to computer higher dimensional feature mapping.

Savvides et al (2006) stated that the kernel correlation filters could extend using

linear advance correlation filter by performing kernel trick. They extended their CFA

method by kernel trick and performing it over the all images.

According to Savvides et al (2006), they have used support vector machine as a

decision classifier on distance measure on KCFA projection space. They input

KCFA projection coefficients to training the SVM.

By examine distance and similarity threshold values between training image and

probe image (in KCFA projection coefficients) they have identified best match that

mean they recognised the individuals using this method which is similar to PCA.

They gained promising results during this approach. The results as follows the eye-

region yields a verification rate of 83.1% compared to 53% obtained by using themouth region and 50% with the nose region.

3.4.4Eigenface ApproachAccording to Fladsrud (2005),Eigenface approach can consider as one of popular

face recognition approach that purposed by Sirovich & Kirby and the method

purposed by Kirby refined by Matthew Turk and Alex Pentland by adding pre-

processing and face detection procedure.

As mentioned before an image can represent as vector. If the image width is w by

and height is h then image can be represent w x h dimensional vector.


56/169

44

Figure 3.22: Face image transformation

[Source: Fladsrud,2005]

X=[x1,x2,x3.............................,xn]T

It assumed that n is the total of pixel in image.

The rows of the image place each after other and it form a vector. Above vector

belongs to a image space which is w by h all images whose dimensions is w by h and

by putting row each after each it can be represent as one dimensional vector which

shows in figure 4.4.Following images shows basis of the image space.

When it take faces all the faces look like same because any face has a mouth, a nose,

pair of eyes, etc which are relatively located at approximately same place. As

mentioned by Fladsrud (2005) because of above quality of the face all faces (face

vectors) are located in very small area in the image space. Figure 3.26 represents

distribution of faces (face vectors) in image space.

Figure 3.23: Distribution of faces in image space

[Source: O'Toole et al ,1993]


57/169

45

By understanding that it is possible to understand that represent the faces in image

space is waste of space. O'Toole et al (1993) purposed use of PCA to find the

specific vectors that are highly sensitive for distribution of face images. These

vectors can call as face space which is subspace of face images. Furthermore

OToole et al (1993) stated that Face space will be a better representation for face

images than image space which is the space which containing all possible images

since there will be increased variation between the faces in face space.

By projecting faces in to face space will provide following distribution of faces in

face space.

Figure 3.24: Faces in face space

[Source: O'Toole et al ,1993]

According to OToole et al (1993) the vectors in face space are known as eigenfaces.

The simplest method to compare two images is compare pixel by pixel but when it

considering 100 pixels by 100 pixels it contains 104 pixels doing comparison for that

amount of pixels is time consuming and inefficient. As a solution for that Kerby &

Sirovich(1990) used karhunen-Love expansion which is popular as principle

component analysis(PCA) .Also Kerby & Sirovich(1991) stated , The main idea

behind applying PCA to a image is find out weights ( vectors) which are responsible

to distribution of face space within the image space.

Because of the application of PCA, it is possible to keep image data in way that is

more compact and because of that, the comparison of two images vector is less time

consuming and efficient than matching image pixel by pixel.


58/169

46

Start

Training set

(known

faces)

E = Eigenfaces

W=weights of E

training setRecognition Module

De and >e

No

D>e and < e

No

D = Average

Distanse

(W,Wx)

Yes

Yes

Input

Unknown

images E

X is not a face

X is unknown

face

X is a known

faceYes

End

No

E = EigenfacesW=weights of E

training set

Figure 3.25: Overview of Eigen face Approach

Above figure represents overall idea behind the Eigen face algorithm. The diagram is

base on Turk & Pentland (1991). The training set which are known images are

transformed into set of Eigen vectors (Eigen faces)(E).After that it calculate weights

of E training set. Then it compares the weight of the new face and the weight of

training set (W).D is the average distance of between weight vectors .e is the

maximum allowable distance from any face class. is the Euclidian distance between

(L2 Norm) projection. The

prrofing for doc

Documents