kinship classification b y modeling facial feature heredity
DESCRIPTION
IEEE International Conference on Image Processing 2013. KINSHIP CLASSIFICATION B Y MODELING FACIAL FEATURE HEREDITY. Ruogu Fang 1 , Andrew C. Gallagher 1 Tsuhan Chen 1 , Alexander Loui 2 1 Cornell University 2 Eastman Kodak Company. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: KINSHIP CLASSIFICATION B Y MODELING FACIAL FEATURE HEREDITY](https://reader035.vdocument.in/reader035/viewer/2022081519/5681431f550346895daf855e/html5/thumbnails/1.jpg)
KINSHIP CLASSIFICATION
BY MODELING FACIAL
FEATURE HEREDITY
Ruogu Fang1, Andrew C. Gallagher1
Tsuhan Chen1, Alexander Loui2
1 Cornell University2 Eastman Kodak Company
1
IEEE International Conference on Image Processing 2013
![Page 2: KINSHIP CLASSIFICATION B Y MODELING FACIAL FEATURE HEREDITY](https://reader035.vdocument.in/reader035/viewer/2022081519/5681431f550346895daf855e/html5/thumbnails/2.jpg)
2
KINSHIP CLASSIFICATION BY MODELING FACIAL FEATURE HEREDITY Problem Definition:Recognize the family that a query person belongs to from a set of families. Solution: Reconstruct the query face from a mixture of parts from a set of family members for the recognition.Motivation:Genetic model of reproduction using the mathematical tool of sparsity.
![Page 3: KINSHIP CLASSIFICATION B Y MODELING FACIAL FEATURE HEREDITY](https://reader035.vdocument.in/reader035/viewer/2022081519/5681431f550346895daf855e/html5/thumbnails/3.jpg)
A GENETIC PERSPECTIVE
• Why do we look like the way that we do?– DNA
• How are our appearances affected by ancestors?– Inheritance and mutation
• Facial features are part of the appearance.
3
DNA
The facial feature heredity also follow the model of genetics.
![Page 4: KINSHIP CLASSIFICATION B Y MODELING FACIAL FEATURE HEREDITY](https://reader035.vdocument.in/reader035/viewer/2022081519/5681431f550346895daf855e/html5/thumbnails/4.jpg)
4
MENDEL’S LAWS I• Law of Random Segregation: For every particular
trait, one randomly selected allele from each parent is passed down to the offspring.
B
b
b b
Bb
B b
B
b
BB Bb
bb
B: Brown eyes (dominant) b: blue eyes (recessive)
bb
bb
Bb
Bb
Each facial feature of an individual can be represented by a sparse
combination of the relatives with this feature.
![Page 5: KINSHIP CLASSIFICATION B Y MODELING FACIAL FEATURE HEREDITY](https://reader035.vdocument.in/reader035/viewer/2022081519/5681431f550346895daf855e/html5/thumbnails/5.jpg)
5
FAMILY SPARSITY
?
Few families are selected.
![Page 6: KINSHIP CLASSIFICATION B Y MODELING FACIAL FEATURE HEREDITY](https://reader035.vdocument.in/reader035/viewer/2022081519/5681431f550346895daf855e/html5/thumbnails/6.jpg)
6
ENERGY FUNCTION• For one family, given sufficient training samples
of a family (m = feature length, n = number of training samples)
• A new sample from the same family• Approximately lies in the linear span of the family
member samples associated with this family • For all unrelated families,
L1 norm: Individual sparsity term
(illumination, pose and expression)
L2,1 norm: Family sparsity term
N= # families
![Page 7: KINSHIP CLASSIFICATION B Y MODELING FACIAL FEATURE HEREDITY](https://reader035.vdocument.in/reader035/viewer/2022081519/5681431f550346895daf855e/html5/thumbnails/7.jpg)
7
MENDEL’S LAWS II
• Law of Independent Assortment: Genes of separate traits are passed down independently from parents to offspring.
Credit: Northeast Medical School
The facial features should be analyzed independently.
![Page 8: KINSHIP CLASSIFICATION B Y MODELING FACIAL FEATURE HEREDITY](https://reader035.vdocument.in/reader035/viewer/2022081519/5681431f550346895daf855e/html5/thumbnails/8.jpg)
8
INDEPENDENCE OF FACIAL PARTS
For each part, a part-based dictionary is built.
![Page 9: KINSHIP CLASSIFICATION B Y MODELING FACIAL FEATURE HEREDITY](https://reader035.vdocument.in/reader035/viewer/2022081519/5681431f550346895daf855e/html5/thumbnails/9.jpg)
9
CLASSIFICATION
1. Choose three representative parts with smallest possible residues R.
2. Rank the normalized residues for all families on these three parts.
3. Sum the ranks and use the highest rank.
…# families
…# families
…# families
Error
Error
Error
…Reconstruction error for part p from family j
Remove outliers due to
recessive genes
Byproduct: Find the three
most distinguishabl
e features
![Page 10: KINSHIP CLASSIFICATION B Y MODELING FACIAL FEATURE HEREDITY](https://reader035.vdocument.in/reader035/viewer/2022081519/5681431f550346895daf855e/html5/thumbnails/10.jpg)
10
POTENTIAL APPLICATIONS
Family Photo Album DistillationFamily Image Retrieval
Social Websites: Auto Family Tagging
Tag Your Family Members
From Sara Lee’s family?
From Kelly Ng’s family?
Find Lost Relatives
![Page 11: KINSHIP CLASSIFICATION B Y MODELING FACIAL FEATURE HEREDITY](https://reader035.vdocument.in/reader035/viewer/2022081519/5681431f550346895daf855e/html5/thumbnails/11.jpg)
11
FAMILY 101 DATABASE
• 101 Different Families • 607 Individuals• 14,816 Images
Kennedy
27 (410)
# people
# images
Download: http://tinyurl.com/kinshipclassification
![Page 12: KINSHIP CLASSIFICATION B Y MODELING FACIAL FEATURE HEREDITY](https://reader035.vdocument.in/reader035/viewer/2022081519/5681431f550346895daf855e/html5/thumbnails/12.jpg)
12
FAMILY DATABASE COLLECTIONKennedy Family
27 Individuals
48 Images of Caroline Kennedy
![Page 13: KINSHIP CLASSIFICATION B Y MODELING FACIAL FEATURE HEREDITY](https://reader035.vdocument.in/reader035/viewer/2022081519/5681431f550346895daf855e/html5/thumbnails/13.jpg)
13
Database # of Family
# of People
# of Images
Highlights References
Labeled Faces in the Wild (LFW)
0 5749 13,233 Unconstrained, “natural” variability in pose, lighting, etc.
[Hung et al. 2007]
PubFig 0 200 58,797 Real world, deep and large, celebrities and politicians
[Kumar et al. 2009]
Cornell Kinship Verification
150 300 300 Controlled parent-child pairs [Fang et al. 2010]
UB KinFace 90 180 270 Child with young parent and old parent faces
[Xia et al. 2011]
Family101 101(206 Nuclear)
607 14,816 Real world, family structure of 2-3 genenrations, variations of age, pose, illumination, expression, ethnicity, etc. Political, royal, wealthy and celebrity families.
This Work
RELATED DATABASES
• Facts about Family101 Database– Multiple generations– Every nuclear family has 6 family members on average– Every individual has 24 images on average
![Page 14: KINSHIP CLASSIFICATION B Y MODELING FACIAL FEATURE HEREDITY](https://reader035.vdocument.in/reader035/viewer/2022081519/5681431f550346895daf855e/html5/thumbnails/14.jpg)
14
EXPERIMENT SETUP• Feature: Dense SIFT 16x16• Baseline
– K nearest neighbors (KNN)– Support vector machine (SVM)– Sparse representation based recognition (SRC)
• Unless specified in each scenario: – 3 family members for training, 2 for testing. – 20 families randomly selected for evaluation. – 30 images/person for both training and testing.
• Evaluation metric: Mean per-family accuracy
![Page 15: KINSHIP CLASSIFICATION B Y MODELING FACIAL FEATURE HEREDITY](https://reader035.vdocument.in/reader035/viewer/2022081519/5681431f550346895daf855e/html5/thumbnails/15.jpg)
15
EXP 1: NO. OF FAMILIES 3 family members for training2 family members for testing30 images/person for training/testing
![Page 16: KINSHIP CLASSIFICATION B Y MODELING FACIAL FEATURE HEREDITY](https://reader035.vdocument.in/reader035/viewer/2022081519/5681431f550346895daf855e/html5/thumbnails/16.jpg)
16
EXP 2: NO. OF PEOPLE FOR TRAINING
20 families randomly selected30 images/person for training/testing
![Page 17: KINSHIP CLASSIFICATION B Y MODELING FACIAL FEATURE HEREDITY](https://reader035.vdocument.in/reader035/viewer/2022081519/5681431f550346895daf855e/html5/thumbnails/17.jpg)
17
FACIAL FEATURE MATCHING
• Task: Find the people with similar facial features to the query person.
Training Images
Test Images
Martin SheenHigh
Low
Hair
Eyes
Nose
Mouth
Martin Sheencoefficients
![Page 18: KINSHIP CLASSIFICATION B Y MODELING FACIAL FEATURE HEREDITY](https://reader035.vdocument.in/reader035/viewer/2022081519/5681431f550346895daf855e/html5/thumbnails/18.jpg)
18
CONCLUSION
• Motivation: biological process of inheritance– Mendel’s laws of random segregation
and independent assortment
• A new challenge: kinship classification• A new framework: reconstruct the query
face from a mixture of parts from a set of families
• A new dataset: Family 101
![Page 19: KINSHIP CLASSIFICATION B Y MODELING FACIAL FEATURE HEREDITY](https://reader035.vdocument.in/reader035/viewer/2022081519/5681431f550346895daf855e/html5/thumbnails/19.jpg)
19
FUTURE WORK
• Use family tree structure
• Hallucination– Hallucinate what the appearance of the father
might be, just by looking at the differences between a child and her mother.
![Page 20: KINSHIP CLASSIFICATION B Y MODELING FACIAL FEATURE HEREDITY](https://reader035.vdocument.in/reader035/viewer/2022081519/5681431f550346895daf855e/html5/thumbnails/20.jpg)
Q & A
Thank you!
20
KINSHIP CLASSIFICATION
BY MODELING FACIAL
FEATURE HEREDITY Ruogu Fang Andrew C. Gallagher
Tsuhan Chen Alexander Loui
Project Page and Dataset Download:http://tinyurl.com/kinshipclassification
![Page 21: KINSHIP CLASSIFICATION B Y MODELING FACIAL FEATURE HEREDITY](https://reader035.vdocument.in/reader035/viewer/2022081519/5681431f550346895daf855e/html5/thumbnails/21.jpg)
21
FACE DETECTION & ALIGNMENTActive Shape Model: 82 Facial PointsFace DetectionFace Alignment: 6 Fiducial Points