Matthias Wimmer, Sylvia Pietzsch, Freek Stulp and Bernd Radig
Chair for Image Understanding
Institute for Computer Science
Technische Universität München
Learning Robust Objective Functions with Application to Face Model Fitting
Christoph Mayer
11.06.2007 2/13Technische Universität München
Christoph Mayer
Facial Expression Recognition
Natural Human-Computer Interaction
tactile channel
visual channel
audatory channel
olfactory channel
gustatory channel
auditory channel
visual channel
tactile channel
olfactory channel
gustatory channel
11.06.2007 3/13Technische Universität München
Christoph Mayer
Model-based Image Interpretation
Objective Function Calculates how well a parameterized model matches an image.
The model contains a parameter
vector p hat represents the model’s
configuration.
Fitting Algorithm
Searches for the model that matches the image best
by minimizing the objective function.
11.06.2007 4/13Technische Universität München
Christoph Mayer
Ideal Objective Functions
P1: Correctness Property:
The global minimum corresponds to the best model fit.
P2: Uni-modality Property:
The objective function has no local extrema.
¬ P1 P1
¬P2
P2
11.06.2007 5/13Technische Universität München
Christoph Mayer
Introducing Objective Functions
a) image b) along perpendicular c) edge values d) designed objective function e) ideal objective function f) training samples g) learned objective function
11.06.2007 6/13Technische Universität München
Christoph Mayer
Traditional Approach
Shortcomings: Requires domain knowledge. Based on designer’s intuition. Time-consuming.
Manually design the
objective function
Manually evaluate on test images
designed objective function
good
not good
11.06.2007 7/13Technische Universität München
Christoph Mayer
Learning the Objective Function (Step 1)Manually annotate images with ideal parameterization
Ideal objective function
11.06.2007 8/13Technische Universität München
Christoph Mayer
Learning the Objective Function (Step 2)Manually annotate images with idealparameterization
Ideal objective function
Automatically generate further
image annotations
result = 0 result = 0.2result = 0.3
11.06.2007 9/13Technische Universität München
Christoph Mayer
Learning the Objective Function (Step 3)
Number of features: 6 styles · 3 sizes · 25 locations = 450
Locations
Styles
Sizes
Manually annotate images with idealparameterization
Ideal objective function
Automatically generate further
image annotations
Manually specifya set of
image features
11.06.2007 10/13Technische Universität München
Christoph Mayer
Learning the Objective Function (Step 4+5)
Automatically obtain calculation rules of objective function.Mapping of feature values to the value of the objective function.
Machine learning by Model Trees.Select the most relevant features.
Manually annotate images with idealparameterization
Ideal objective function
Automatically generate further
image annotations
Manually specifya set of
image features
Automatically generate
training data
learned objective function
11.06.2007 11/13Technische Universität München
Christoph Mayer
Evaluation of local objective functions
Designed Learned
Evaluation of displacement and face turning.
Weak global minimum using the designed objective function.
Strong global minimum using the learned objective function.
11.06.2007 12/13Technische Universität München
Christoph Mayer
Evaluation of the global function 95% of models
are located at 0.12 using the learned objective function.
95% of models are located at 0.16 using a state-of-the-art approach.
11.06.2007 13/13Technische Universität München
Christoph Mayer
Conclusion
Evaluation in natural dialog situations.
Application for Daimler-Crysler in car-driving
situations.
Integration of a three-dimensional face model.