10/22/18

1

TheAnalysisofFacesinBrainsandMachines

CS332VisualProcessinginComputerandBiologicalVisionSystems

HMAXmodel

PaulaJohnson

ElizabethWarren

Whyisfaceanalysisimportant?

Remember/recognizepeoplewe’veseenbefore

Categorization– e.g.gender,race,age,kinship

Socialcommunication– emotions/mood,intentions,trustworthiness,competenceorintelligence,attractiveness

Sceneunderstanding,e.g.directionofgazesuggestsfocusofattention

10/22/18

2

Whyisfacerecognitionhard?

changingpose changingillumination

changingexpressionclutterocclusion

aging

Jenkins,White,VanMontfort &Burton,Cognition,2011

Howgoodareweatfacerecognition?

10/22/18

3

Facerecognitionperformanceinhumans

chanceperformance

testmybrain.org

Wilmeretal.,2012Duchaine &Nakayama,2006

10/22/18

4

Bruceetal.,1999

Facerecognitionperformanceinhumans

Whichofthe10photosonthebottomdepictsthetargetface?

Viewersare~70%correct

Performancedegradeswithchangesinpose,expression

Onlyslightimprovementwithshortvideoclipoftarget

Importanceoffamiliarvs.unfamiliarfacerecognition!

Howgoodarethebestmachines?Publicdatabasesoffaceimagesserveasbenchmarks:

LabeledFacesintheWild(LFW,http://vis-www.cs.umass.edu/lfw)>13,000imagesofcelebrities,5,749differentidentities

YouTubeFacesDatabase(YTF,http://www.cs.tau.ac.il/~wolf/ytfaces)3,425videos,1,595differentidentities

Privatefaceimagedatasets:

(Facebook)SocialFaceClassificationdataset4.4millionfacephotos,4,030differentidentities

(Google) 100-200millionfaceimages,~8milliondifferentidentities

LFW YTFFacebookDeepFace 97.4% 91.4%GoogleFaceNet 99.6% 95.1%Humanperformance 97.5% 89.7%

10/22/18

5

Machinevisionapplicationsoffacerecognition

surveillance

accesscontrol

security,forensics

Moreapplicationsoffacerecognition

content-basedimageretrieval socialmedia

graphics,HCIhumanoidrobots

10/22/18

6

Aspectsoffaceprocessing

Facedetection– findimageregionsthatcontainfaces

Faceidentification– whoistheperson?

Categorization– gender,age,race

Facialexpression– mood,emotion

Non-verbalsocialperceptionandcommunication

ItallbeganwithTakeoKanade (1973)…PhDthesis,PictureProcessingSystembyComputerComplexand

RecognitionofHumanFaces

• Specialpurposealgorithmstolocateeyes,nose,mouth,boundariesofface

• ~40geometricfeatures,e.g.ratiosofdistancesandanglesbetweenfeatures

10/22/18

7

- FromtalkbyTakeoKanade,CBMMFaceIDChallengeWorkshop

Eigenfaces forrecognition(Turk&Pentland)PrincipalComponentsAnalysis(PCA)

Goal: reducethedimensionalityofthedatawhileretainingasmuchinformationaspossibleintheoriginaldataset

PCAallowsustocomputealineartransformationthatmapsdatafromahighdimensionalspacetoalowerdimensionalsubspace

10/22/18

8

Typicalsampletrainingset…

Oneormoreimagesperperson

Aligned&croppedtocommonpose,size

Simplebackground

SampleimagesfromtheYalefacedatabase

Eigenfaces forrecognition(Turk&Pentland)

1-16

PerformPCA onalargesetoftrainingimages,tocreateasetofeigenfaces,Ei(x,y),thatspanthedataset

Firstcomponentscapturemostofthevariationacrossthedataset,latercomponentscapturesubtlevariations

EachfaceimageF(x,y)canbeexpressedasaweightedcombinationoftheeigenfaces Ei(x,y):

Ψ(x,y):averageface(acrossallfaces)

Ψ(x,y)

http://vismod.media.mit.edu/vismod/demos/facerec/basic.html

F(x,y)=Ψ(x,y)+Σi wi*Ei(x,y)

10/22/18

9

RepresentingindividualfacesEachfaceimageF(x,y)canbeexpressedasaweightedcombinationoftheeigenfaces Ei(x,y):

Recognitionprocess:(1) Computeweightswi

fornovelfaceimage

(2) Findimageminfacedatabasewithmostsimilarweights,e.g.

min (wi −wim

i=1

k

∑ )2

F(x,y)=Ψ(x,y)+Σi wi*Ei(x,y)

Faceseverywhere...