Evidential modeling for Evidential modeling for pose estimationpose estimation

Fabio Cuzzolin, Ruggero Frezza

Computer Science Department

UCLA

Myself

Master’s thesis on gesturegesture recognitionrecognition

at the University of Padova Ph.D. thesis on the theory of theory of

evidenceevidence Post-doc in Milan with the Image and

Sound Processing group Post-doc at UCLA in the Vision Lab

Px

Py

P

F (s)x

F (s)y

y

x

y

x

My past work…

geometric approachgeometric approach to the theory of belief functions space of belief functions geometry of Dempster’s rule

.. again ..

algebra of compatible frames linear independence on lattices action recognition and object

tracking metrics on the space of dynamical

models

43

21

… and today’s talk

the pose estimation problemthe pose estimation problem

model-free pose estimationmodel-free pose estimation

evidential modelevidential model

experimental resultsexperimental results

Pose estimation estimating the “posepose” (internal configuration)

of a moving body from the available images

salient image measurements: featuresfeatures

Qtq k ˆt=0

t=T

Model-based estimation if you have an a-priori modela-priori model of the object .. .. you can exploit it to help (or drive) the

estimation

example: kinematic model

Model-free estimation

if you do not have any information about the body..

the only way to do inference is to learn a maplearn a map between features and

poses directly from the data

this can be done in a training stagetraining stage

Collecting training data motion capture system

3D locations of markers = pose

Training data when the object performs some “significant”

movements in front of the camera … … a finite collection of configuration values

are provided by the motion capture system

… while a sequence of features is computed from the image(s)

q q

y y

Q~

1

1

T

T

Learning feature-pose maps

Hidden Markov modelsHidden Markov models provide a way to build feature-pose maps from the training data

a Gaussian density for each state is set up on the feature space -> approximate feature spaceapproximate feature space

mapmap between each region and the set of training poses qk with feature value yk inside it

Evidential model

approximate feature spaces ..

.. and approximate parameter space ..

.. form a family of compatible family of compatible frames: the evidential modelframes: the evidential model

Estimation

these belief functions are projected onto the approximate parameter space ..

.. and combined through Dempster’s rule

a point-wise estimate of the pose is obtained by probabilistic approximation

new features are represented as belief functions ..

Human body tracking

two experiments, two views

four markers on the right arm

six markers on both legs

Feature extraction

three steps: original image, color segmentation, bounding box

185

94

161

38

185

94

161

38

Performances comparison of three models: left view

only, right view only, both views

pose estimation yielded by the overall model

estimate associated with the “right” model

“left” model

ground truth

Estimation errors Euclidean distance between real and

predicted marker position

marker 4

3cm

marker 2

8cm

Visual estimate

Tk

kIkpI..1

)()(ˆˆ compares the actual image

with the weighted sum of the training images

Conclusions pose estimation of unknown objects is a

difficult task a bottom-up model has to be built from the

data in a training session the DS framework allows to formalize the

idea of feature-pose maps in a natural way through the notion of compatible frames

Dempster’s combination provides a method to integrate features to increase robustness