3d-aware expression flow for 2d face...
TRANSCRIPT
![Page 1: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/1.jpg)
3D-Aware Expression Flow for 2D Face Compositing
Fei Yang1 Jue Wang2 Eli Shechtman2
Lubomir Bourdev2 Dimitris Metaxas1
1. Rutgers University 2. Adobe Systems
![Page 2: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/2.jpg)
Hard to capture the momentHard to capture the momentPhotometric flaws Non-desirable expression
![Page 3: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/3.jpg)
GoalGoal
Our resultReference Target
![Page 4: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/4.jpg)
Previous workPrevious work• Photo enhancement
– Joshi et al. [2010]
• Face swapping– Bitouk et al. [2008]
![Page 5: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/5.jpg)
Face editingFace editing
Reference Target
Replace whole face
Replace whole face
![Page 6: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/6.jpg)
Previous workPrevious work• Expression mapping
– 3D approaches• Pighin et al. [1998]
• Blanz et al. [2003]
• Metaxas et al. [2004]
– 2D approaches• Williams [1990]
• Liu et al. [2001]
![Page 7: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/7.jpg)
Previous workPrevious work• Interactive Digital Photomontage
– Agarwala et al. [2004]
![Page 8: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/8.jpg)
Local component transferLocal component transfer
Reference Target
• Copy mouth region
Photomontage(unnatural)
Our result
![Page 9: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/9.jpg)
Outline of our approachOutline of our approach• Overview
• 3D Model Fitting
• Image Compositing
• Results and Evaluation
![Page 10: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/10.jpg)
System overviewSystem overview
Target
Reference
Input images 2D landmarks 3D jointmodel fitting
Target image warped by expression flow
Reference imagereorientation
Final compositing
![Page 11: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/11.jpg)
Outline of our approachOutline of our approach• Overview
• 3D Model Fitting
• Image Compositing
• Results and Evaluation
![Page 12: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/12.jpg)
Training datasetTraining dataset
ExpressionIdentity⋯ ⋯ ⋯ ⋯
• Vlasic et al. [2005]– 16 subjects, 5 expressions, 5 visemes
⋯
⋯
⋯
⋯
⋯
⋯
⋯
![Page 13: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/13.jpg)
Face modelFace model• Linear span
• PCA subspace– Mean shape
– Eigenvectors , , … ,
– Eigenvalues , , … ,
– New shape ∙
∙ ∙ ∙ ⋯ ∙
Blanz et al. [1999]
![Page 14: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/14.jpg)
Face modelFace model• Optimization
– Total energy function:
– Fidelity term:
– Subspace energy term:
∙
|| ||
: Projections of3D landmarks
: Facial features
![Page 15: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/15.jpg)
Shape to image fittingShape to image fitting• Matching features
– Internal landmarks
– Face boundary landmarks
![Page 16: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/16.jpg)
Shape to image fittingShape to image fitting
• Algorithm1. Detect landmarks
Milborrow and Nicolls [ECCV 2008]
![Page 17: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/17.jpg)
Shape to image fittingShape to image fitting
• Algorithm1. Detect landmarks
2. Place 3D mean shape
![Page 18: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/18.jpg)
Shape to image fittingShape to image fitting
• Algorithm1. Detect landmarks
2. Place 3D mean shape
3. Find face boundary
![Page 19: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/19.jpg)
Shape to image fittingShape to image fitting
• Algorithm1. Detect landmarks
2. Place 3D mean shape
3. Find face boundary
4. Find corresponding vertex
![Page 20: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/20.jpg)
Shape to image fittingShape to image fitting
• Algorithm1. Detect landmarks
2. Place 3D mean shape
3. Find face boundary
4. Find corresponding vertex
5. Update 3D shape
![Page 21: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/21.jpg)
Shape to image fittingShape to image fitting
• Algorithm1. Detect landmarks
2. Place 3D mean shape
3. Find face boundary
4. Find corresponding vertex
5. Update 3D shape
After 3 iterations
![Page 22: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/22.jpg)
Fitting shapes jointlyFitting shapes jointlyExpression
IdentityIdentity
a
b
, ∙ ⋯ , ∙
, ∙ ⋯ , ∙
, ∙⋯
, ∙
⋯ , ∙
⋯ , ∙
, ∙⋯⋯
, ∙ ⋯ , ∙ ⋯ , ∙⋯
⋯⋯⋯⋯
, ∙ ⋯ , ∙
, ∙ ⋯ , ∙
, ∙⋯
, ∙
⋯ , ∙
⋯ , ∙
, ∙
⋯⋯
, ∙ ⋯ , ∙ ⋯ , ∙
⋯
⋯⋯⋯⋯
![Page 23: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/23.jpg)
Fitting shapes jointlyFitting shapes jointly
Target Fitting independently
![Page 24: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/24.jpg)
Fitting shapes jointlyFitting shapes jointlyExpression
IdentityIdentity
a
b
, ∙ ⋯ , ∙
, ∙ ⋯ , ∙
, ∙⋯
, ∙
⋯ , ∙
⋯ , ∙
, ∙⋯⋯
, ∙ ⋯ , ∙ ⋯ , ∙⋯
⋯⋯⋯⋯
, ∙ ⋯ , ∙
, ∙ ⋯ , ∙
, ∙⋯
, ∙
⋯ , ∙
⋯ , ∙
, ∙⋯⋯
, ∙ ⋯ , ∙ ⋯ , ∙⋯
⋯⋯⋯⋯
Problem: best fit comes from different identities
![Page 25: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/25.jpg)
Fitting shapes jointlyFitting shapes jointlyExpression
IdentityIdentity
a
b
, ∙ ⋯ , ∙
, ∙ ⋯ , ∙
, ∙⋯
, ∙
⋯ , ∙
⋯ , ∙
, ∙⋯⋯
, ∙ ⋯ , ∙ ⋯ , ∙⋯
⋯⋯⋯⋯
, ∙ ⋯ , ∙
, ∙ ⋯ , ∙
, ∙⋯
, ∙
⋯ , ∙
⋯ , ∙
, ∙
⋯⋯
, ∙ ⋯ , ∙ ⋯ , ∙
⋯
⋯⋯⋯⋯
![Page 26: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/26.jpg)
Fitting shapes jointlyFitting shapes jointly
a
b
, ∙
, ∙
, ∙⋯
⋯
, ∙
, ∙
, ∙⋯
⋯
min
![Page 27: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/27.jpg)
Outline of our approachOutline of our approach• Overview
• 3D Model Fitting
• 2D Compositing
• Results and Evaluation
![Page 28: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/28.jpg)
2D compositing2D compositing• Warping with expression flow
Target Flow Warped Target Difference
![Page 29: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/29.jpg)
2D compositing2D compositing• Automatic crop region generation
– “Graph Cuts” image segmentation [Agarwala et al. 2004]
User click
Crop region
![Page 30: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/30.jpg)
User assistanceUser assistance• Adjust landmarks
![Page 31: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/31.jpg)
User assistanceUser assistance• Adjust crop region
Copy mouth onlyMark fold region Copy mouth and fold
![Page 32: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/32.jpg)
Outline of our approachOutline of our approach• Overview
• Image Fitting
• 2D Compositing
• Results and Evaluation
![Page 33: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/33.jpg)
ExamplesExamples
TargetReference Warped byExpression Flow
Warped by 3D rotation
• Example 1
![Page 34: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/34.jpg)
ExamplesExamples
Our Result2D Method
• Example 1
Mouth distorted
Mouth too close to chin
![Page 35: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/35.jpg)
ExamplesExamples• Example 4
TargetReference Warped by 3D rotation
Warped byExpression Flow
![Page 36: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/36.jpg)
ExamplesExamples• Example 4
Our Result2D Method
Smiling eyes
Lower jaw
![Page 37: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/37.jpg)
ExamplesExamples
TargetReference
• Example 2
Warped byExpression Flow
Warped by 3D rotation
![Page 38: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/38.jpg)
ExamplesExamples
Our Result2D Method
• Example 2
Smiling eyes
Wider cheeks
![Page 39: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/39.jpg)
ExamplesExamples• Example 3
TargetReference Warped by 3D rotation
Warped byExpression Flow
![Page 40: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/40.jpg)
ExamplesExamples• Example 3
Our Result2D Result
Smiling eyes
Lower jaw
![Page 41: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/41.jpg)
User studyUser study
01 02 03
04 05
0607 08
11
aver
age
resp
onse
Our results
Question: Which image appears more realistic?
![Page 42: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/42.jpg)
User studyUser study• With vs. without Expression Flow
With Without
Question: Which image appears more realistic?
![Page 43: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/43.jpg)
ComparisonComparison
Our resultFaceGen1 Whole face replacementTargetReference
• Expression flow vs. other methods
1 FaceGen: http://www.facegen.com/
![Page 44: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/44.jpg)
Expression flow onlyExpression flow onlyReference Target Our Result
![Page 45: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/45.jpg)
From neutral to frownFrom neutral to frownBeginning After a few years…
![Page 46: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/46.jpg)
From neutral to frownFrom neutral to frownReference Target Our Result
![Page 47: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/47.jpg)
Failure casesFailure casesReference Target Our Result
• Large pose change
• Asymmetric expression
![Page 48: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/48.jpg)
ConclusionConclusion
• Local feature compositing
• Expression flow
• Joint 3D fitting
http://www.juew.org/projects/expressionflow.htm
![Page 49: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/49.jpg)
Reference Target Our Result
![Page 50: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/50.jpg)
Reference Target Our Result
![Page 51: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/51.jpg)
Reference Target Our Result
![Page 52: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/52.jpg)
Reference Target Our Result
![Page 53: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/53.jpg)
Reference Target Our Result
![Page 54: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/54.jpg)
Reference Target Our Result
![Page 55: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/55.jpg)
Reference Target Our Result
![Page 56: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/56.jpg)
Reference Target Our Result
![Page 57: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/57.jpg)
Reference Target Our Result
![Page 58: 3D-Aware Expression Flow for 2D Face Compositingresearch.cs.rutgers.edu/~feiyang/paper/Feiyang_Siggraph2011_slides.pdf · 3D-Aware Expression Flow for 2D Face Compositing Fei Yang1](https://reader031.vdocument.in/reader031/viewer/2022041817/5e5b4ef8fb943734047b544e/html5/thumbnails/58.jpg)
Link
linkReference Target Our Result