Download - Topic 5. Human Faces
![Page 1: Topic 5. Human Faces](https://reader036.vdocument.in/reader036/viewer/2022081506/568145af550346895db2ada3/html5/thumbnails/1.jpg)
TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu
Topic 5. Human Faces
Human face is extensively studied in vision. Depending on the applications, there are a long list of tasks [5]:1. Detection and Recognition: Face detection (finding all faces in a picture), facial feature detection (eyes, lips, …), Face localization (detecting a single face in image), Face recognition or identification (from a database, classification) Face authentication (verifying claim, bank id), Age/gender recognition, Face tracking (location and pose over time) Facical expression recognition (affective states), aesthetic study.
2. Modeling and Photorealistic Synthesis: Appearance models, deformable templates, lighting models, facial action units, face hallucination (high resolution from low resolution), pose adjustment, image editing (removing wrinkles, eye glass, red-eye etc.)
3. Artistic rendering Sketch, portrait, caricature, cartoon, painting, …
![Page 2: Topic 5. Human Faces](https://reader036.vdocument.in/reader036/viewer/2022081506/568145af550346895db2ada3/html5/thumbnails/2.jpg)
TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu
Face Image Databases
The CMU Rowley dataset
![Page 3: Topic 5. Human Faces](https://reader036.vdocument.in/reader036/viewer/2022081506/568145af550346895db2ada3/html5/thumbnails/3.jpg)
TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu
Face Image Databases
The CMU Schneidrman and Kanade Dataset
![Page 4: Topic 5. Human Faces](https://reader036.vdocument.in/reader036/viewer/2022081506/568145af550346895db2ada3/html5/thumbnails/4.jpg)
TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu
References.
1. P. Hallinan, G. Gordon, A. Yuille, P. Giblin, and D. Mumford, 2D and 3D Patterns of the Face, A.K. Peters, Ltd. Book chapters 2-4. (handouts).
2. D.H. Ballard, "Generaling the Hough transform to detect arbitrary shapes", (in handbook). 3. P. Viola and M. Jones, "Robust Real Time Object Detection", 4. F. Fleuret and D. Geman, " Coarse-to-fine face detection", IJCV 41(1/2),2001. 5. M.H. Yang, D. Kriegman, N. Ahuja, “Detecting faces in images, a survey”, PAMI
vol.24,no.1, January, 2002.
6 T.F.Cootes, G.J. Edwards and C.J.Taylor. "Active Appearance Models", ECCV 1998 7. C. Liu, S. C. Zhu, and H. Y. Shum, "Learning inhomogeneous Gibbs models of faces by
minimax entropy", ICCV 2001.
8. Y. Tian, T. Kanade, and J. Cohn, "Recognizing action units for facial expression analysis" PAMI, Feb, 2001. 9. H. Chen, Y. Q. Xu, H. Y. Shum, S. C. Zhu, and N. N. Zhen, "Example-based facial sketch
generation with non-parametric sampling", ICCV 2001.
![Page 5: Topic 5. Human Faces](https://reader036.vdocument.in/reader036/viewer/2022081506/568145af550346895db2ada3/html5/thumbnails/5.jpg)
TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu
Outline
We proceed in three steps:
• A survey on face detection and recognition techniques
2. Mathematical models of face images
3. Face synthesis: photorealistic and non-photorealistic.
![Page 6: Topic 5. Human Faces](https://reader036.vdocument.in/reader036/viewer/2022081506/568145af550346895db2ada3/html5/thumbnails/6.jpg)
TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu
Face Detection Methods [5]
![Page 7: Topic 5. Human Faces](https://reader036.vdocument.in/reader036/viewer/2022081506/568145af550346895db2ada3/html5/thumbnails/7.jpg)
TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu
Face vs non-face Clsutering
6 clusters in a 19 x19 space (Sung and Poggio)
![Page 8: Topic 5. Human Faces](https://reader036.vdocument.in/reader036/viewer/2022081506/568145af550346895db2ada3/html5/thumbnails/8.jpg)
TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu
Distance Measure
D1
D2
For each input image, it measures two distances for each cluster center: D1 is the Mahalanobis distance and D2 is the Euclidean distance.
Thus Sung and poggio have 2 x 6 x 2 = 24 features for classification in a multiple layer perceptron.
![Page 9: Topic 5. Human Faces](https://reader036.vdocument.in/reader036/viewer/2022081506/568145af550346895db2ada3/html5/thumbnails/9.jpg)
TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu
Deformable Face Template
Deformable face template by Fishler and Elschlager 1973. M. Fishler and R. Elschlager, “The representation and matching of pictorial structures”,
IEEE Trans. on Computer. Vol.C-22, 67-92, 1973.
![Page 10: Topic 5. Human Faces](https://reader036.vdocument.in/reader036/viewer/2022081506/568145af550346895db2ada3/html5/thumbnails/10.jpg)
TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu
Local Deformation and Global Transform
Geometric variations of faces: (Hallinan, Yuille, Mumford et al)
![Page 11: Topic 5. Human Faces](https://reader036.vdocument.in/reader036/viewer/2022081506/568145af550346895db2ada3/html5/thumbnails/11.jpg)
TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu
Deformable Model of Facial Features
Eye template using parabolic curves by Yuille et al 1989-92. A.L.Yuille, D. Cohen, and P.Hallinan, “Feature extraction from faces using deformable templates”, CVPR 89, IJCV 92.
We can derive meaningful diffusion equations from the energy functionals.
![Page 12: Topic 5. Human Faces](https://reader036.vdocument.in/reader036/viewer/2022081506/568145af550346895db2ada3/html5/thumbnails/12.jpg)
TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu
Upper Face Action Units
![Page 13: Topic 5. Human Faces](https://reader036.vdocument.in/reader036/viewer/2022081506/568145af550346895db2ada3/html5/thumbnails/13.jpg)
TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu
Lower Face Action Units
![Page 14: Topic 5. Human Faces](https://reader036.vdocument.in/reader036/viewer/2022081506/568145af550346895db2ada3/html5/thumbnails/14.jpg)
TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu
Templates for Various States
![Page 15: Topic 5. Human Faces](https://reader036.vdocument.in/reader036/viewer/2022081506/568145af550346895db2ada3/html5/thumbnails/15.jpg)
TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu
Templates for Various States
![Page 16: Topic 5. Human Faces](https://reader036.vdocument.in/reader036/viewer/2022081506/568145af550346895db2ada3/html5/thumbnails/16.jpg)
TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu
Features for Action Unit Recognition
![Page 17: Topic 5. Human Faces](https://reader036.vdocument.in/reader036/viewer/2022081506/568145af550346895db2ada3/html5/thumbnails/17.jpg)
TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu
Classification from Feature Vector
![Page 18: Topic 5. Human Faces](https://reader036.vdocument.in/reader036/viewer/2022081506/568145af550346895db2ada3/html5/thumbnails/18.jpg)
TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu
Recognition Rate
![Page 19: Topic 5. Human Faces](https://reader036.vdocument.in/reader036/viewer/2022081506/568145af550346895db2ada3/html5/thumbnails/19.jpg)
TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu
Apparence Model: Landmarks on a face
400 images each labeled with 122 points.
![Page 20: Topic 5. Human Faces](https://reader036.vdocument.in/reader036/viewer/2022081506/568145af550346895db2ada3/html5/thumbnails/20.jpg)
TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu
Eigen-vectors for Geometry and Photometry
![Page 21: Topic 5. Human Faces](https://reader036.vdocument.in/reader036/viewer/2022081506/568145af550346895db2ada3/html5/thumbnails/21.jpg)
TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu
Apparence Model
![Page 22: Topic 5. Human Faces](https://reader036.vdocument.in/reader036/viewer/2022081506/568145af550346895db2ada3/html5/thumbnails/22.jpg)
TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu
Face Localization and Recognition
![Page 23: Topic 5. Human Faces](https://reader036.vdocument.in/reader036/viewer/2022081506/568145af550346895db2ada3/html5/thumbnails/23.jpg)
TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu
A Linear HMM Model for Face
![Page 24: Topic 5. Human Faces](https://reader036.vdocument.in/reader036/viewer/2022081506/568145af550346895db2ada3/html5/thumbnails/24.jpg)
TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu
Face Detection
![Page 25: Topic 5. Human Faces](https://reader036.vdocument.in/reader036/viewer/2022081506/568145af550346895db2ada3/html5/thumbnails/25.jpg)
TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu
Sample of the 4D space
![Page 26: Topic 5. Human Faces](https://reader036.vdocument.in/reader036/viewer/2022081506/568145af550346895db2ada3/html5/thumbnails/26.jpg)
TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu
Multi-scale Detection
![Page 27: Topic 5. Human Faces](https://reader036.vdocument.in/reader036/viewer/2022081506/568145af550346895db2ada3/html5/thumbnails/27.jpg)
TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu
Edge Features
![Page 28: Topic 5. Human Faces](https://reader036.vdocument.in/reader036/viewer/2022081506/568145af550346895db2ada3/html5/thumbnails/28.jpg)
TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu
Decision Tree
![Page 29: Topic 5. Human Faces](https://reader036.vdocument.in/reader036/viewer/2022081506/568145af550346895db2ada3/html5/thumbnails/29.jpg)
TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu
Examples of Decision Trees
![Page 30: Topic 5. Human Faces](https://reader036.vdocument.in/reader036/viewer/2022081506/568145af550346895db2ada3/html5/thumbnails/30.jpg)
TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu
Bounds Analysis
![Page 31: Topic 5. Human Faces](https://reader036.vdocument.in/reader036/viewer/2022081506/568145af550346895db2ada3/html5/thumbnails/31.jpg)
TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu
Some Examples
![Page 32: Topic 5. Human Faces](https://reader036.vdocument.in/reader036/viewer/2022081506/568145af550346895db2ada3/html5/thumbnails/32.jpg)
TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu
Face Prior Learning: Experimental Details
• 83 key points defined on face
• 720 individuals with all kinds of types
• Dimension reduced to 33 by PCA
• 40000 samples drawn by the inhomogeneous Gibbs sampler in each Monte Carlo integration
• 50 features pursuit
• Total runtime: about 5 days on a PIII 667, 256MB PC
![Page 33: Topic 5. Human Faces](https://reader036.vdocument.in/reader036/viewer/2022081506/568145af550346895db2ada3/html5/thumbnails/33.jpg)
TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu
Obs & Syn Samples (1)
Observed
faces
Synthesized faces without any features
![Page 34: Topic 5. Human Faces](https://reader036.vdocument.in/reader036/viewer/2022081506/568145af550346895db2ada3/html5/thumbnails/34.jpg)
TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu
Synthesis Samples
Synthesized faces with 20 features
Synthesized faces with 10 features
![Page 35: Topic 5. Human Faces](https://reader036.vdocument.in/reader036/viewer/2022081506/568145af550346895db2ada3/html5/thumbnails/35.jpg)
TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu
Synthesis Samples
Synthesized faces with 30 features
Synthesized faces with 50 features
![Page 36: Topic 5. Human Faces](https://reader036.vdocument.in/reader036/viewer/2022081506/568145af550346895db2ada3/html5/thumbnails/36.jpg)
TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu
50 Observed Histograms
![Page 37: Topic 5. Human Faces](https://reader036.vdocument.in/reader036/viewer/2022081506/568145af550346895db2ada3/html5/thumbnails/37.jpg)
TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu
50 Synthesized Histograms