Download - 01/18 Lab meeting
![Page 1: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/1.jpg)
01/18 Lab meeting
UCLA Vision Lab
Department of Computer Science
University of California at Los Angeles
Fabio Cuzzolin
Los Angeles, January 18 2005
![Page 2: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/2.jpg)
… past and present PhD student, University of Padova, Department of Computer
Science (NAVLAB laboratory) with Ruggero Frezza
Visiting student, ESSRL, Washington University in St. Louis
Visiting student, UCLA, Los Angeles (VisionLab)
Post-doc in Padova, Control and Systems Theory group
Young researcher, Image and Sound Processing Group,
Politecnico di Milano
Post-doc, UCLA Vision Lab
![Page 3: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/3.jpg)
3
… the research
research
Computer vision object and body tracking
data association
gesture and action recognition
Discrete mathematics
linear independence on lattices
Belief functions and imprecise probabilities
geometric approach
algebraic analysis
combinatorial analysis
![Page 4: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/4.jpg)
1Upper and lower probabilities
![Page 5: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/5.jpg)
5
Past work
Geometric approach to belief functions (ISIPTA’01, SMC-C-05)
Algebra of families of frames (RSS’00, ISIPTA’01, AMAI’03)
Geometry of Dempster’s rule (FSKD’02, SMC-B-04)
Geometry of upper probabilities (ISIPTA’03, SMC-B-05)
Simplicial complexes of fuzzy sets (IPMU’04)
![Page 6: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/6.jpg)
The theory of belief functions
![Page 7: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/7.jpg)
7
Uncertainty descriptions
A number of theories have been proposed to extend or replace classical probability: possibilities, fuzzy sets, random sets, monotone capacities, etc.
theory of evidence (A. Dempster, G. Shafer)
belief functions Dempster’s rule families of frames
![Page 8: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/8.jpg)
8
Motivations
![Page 9: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/9.jpg)
9
Axioms and superadditivity
probabilities additivity: if then
1)(.2 p0)(.1 p
BA )()()( BpApBAp
belief functions 3. superadditivity
0)(.1 s 1)(.2 s
i ji
nn
jiin AAsAAsAsAAs )...()1(...)()()...( 11
1
![Page 10: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/10.jpg)
10
Example of b.f.
![Page 11: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/11.jpg)
11
belief functions s: 2Θ ->[0,1]
AB
BmAs )( A
Belief functions
B2
B1
1)( B
Bm
..where m is a mass function on 2Θ s.t.
![Page 12: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/12.jpg)
12
Dempster’s rule
b.f. are combined through Dempster’s rule'', ssss
ABBmABel)()(
Ai
Bj
AiBj=A
intersection of focal elements
ji
ji
BAji
ABAji
BmAm
BmAm
Am)()(1
)()(
)(21
21
![Page 13: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/13.jpg)
13
Example of combination
s1: m({a1})=0.7, m({a1 ,a2})=0.3
a1
a2
a3
a4
s2: m()=0.1, m({a2 ,a3 ,a4})=0.9
s1 s2 : m({a1})=0.19, m({a2})=0.73
m({a1 ,a2})=0.08
![Page 14: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/14.jpg)
14
Bayes vs Dempster
Belief functions generalize the Bayesian formalism as:
1- discrete probabilities are a special class of belief functions
2 - Bayes’ rule is a special case of Dempster’s rule
3 - a multi-domain representation of the evidence is contemplated
![Page 15: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/15.jpg)
15
My research
Theory of evidence
algebraic analysis geometric
analysis
categorial?
probabilistic analysis
combinatorial analysis
![Page 16: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/16.jpg)
Algebra of frames
![Page 17: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/17.jpg)
17
Family of frames
example: a function y [0,1] is quantized in three different ways
1
.0 .1
.00 .01 .10 .11
.0 .1 .2 .3 .4
0.00 0.09
0.90 0.99
0.49
0.25 0.750.50
refining
Common refinement
![Page 18: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/18.jpg)
18
Lattice structure
minimal refinement
1F
maximal coarsening
F is a locally Birkhoff (semimodular with finite length) lattice bounded
below
order relation: existence of a refining
![Page 19: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/19.jpg)
Geometric approach to upper and lower probabilisties
![Page 20: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/20.jpg)
20
it has the shape of a simplex
),( APClS A
Belief space
the space of all the belief functions on a given frame
each subset A A-th coordinate s(A) in an Euclidean space
![Page 21: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/21.jpg)
21
Geometry of Dempster’s rule
constant mass loci
foci of conditional subspaces
Dempster’s rule can be studied in the geometric setup too
Px
Py
P
F (s)x
F (s)y
y
x
y
x
![Page 22: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/22.jpg)
22
),,( AACl A
the space of plausibilities is also a simplex
*sP
),(~*
yxs ppP
s
yyP
xxP
P
x
y
p
p,1
0,
x
y
p
p
S
Geometry of upper probs
![Page 23: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/23.jpg)
23
PPs s *
Belief and probabilities
s
*sP
P)(sP
1xP 2xP
nxP
Ps
)(sP
}{ AX
study of the geometric interplay of belief and probability
![Page 24: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/24.jpg)
24
}{
)()(
xA A
AmsP
Consistent probabilities
Each belief function is associated with a set of consistent probabilities, forming a simplex in the probabilistic subspace
the vertices of the simplex are the probabilities assigning the mass of each focal element of s to one of its points
the center of mass of P(s) coincides with Smets’ pignistic function
![Page 25: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/25.jpg)
25
Possibilities in a geometric setup
possibility measures are a class of belief functions
they have the geometry of a simplicial complex
![Page 26: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/26.jpg)
Combinatorial analysis
![Page 27: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/27.jpg)
27
Total belief theorem
a-priori constraint
conditional constraint
generalization of the total probability theorem
![Page 28: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/28.jpg)
28
Existence
candidate solution: linear system nn where the columns of A are the focal
elements of stot
bAx
problem: choosing n columns among m s.t. x has positive components
method: replacing columns through
i j
ji eeeee
![Page 29: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/29.jpg)
29
Solution graphs
all the candidate solutions form a graph
Edges = linear transformations
![Page 30: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/30.jpg)
30
New goals...
algebraic analysis
combinatorial analysis
Theory of evidence
geometric analysis
?
probabilistic analysis
![Page 31: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/31.jpg)
31
Approximations
compositional criterion the approximation behaves like s when
combined through Dempster
tpCs
C dttstss minarg
problem: finding an approximation of s
probabilistic and fuzzy approximations
![Page 32: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/32.jpg)
32
Indipendence and conflict
s1,…, sn are not always combinable
1,…, n are indipendent if
iinn AAA ,)(...)( 11
any s1,…, sn are combinable are defined on independent frames
![Page 33: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/33.jpg)
33
Pseudo Gram-Schmidt
Vector spaces and frames are both semimodular lattices -> admit independence
mn
mn
ssss
FF
',...,',...,
',...,',...,
11
11
pseudo Gram-Schmidt
new set of b.f. surely combinable
![Page 34: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/34.jpg)
34
Canonical decomposition unique decomposition of s into simple b.f.
nees ...1
convex geometry can be used to find it
xx PPs
yy PPs
s
xe
ye
![Page 35: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/35.jpg)
35
Tracking of rigid bodies
m-1m
past and present target association
rigid motion constraints can be written as conditional belief functions total belief needed
Am-1
past targets - model
associations
m-1m
Am-1
= Am-1 m-1
m
Am-1 ()
old estimates
rigid motion constraints
Kalman filters
Am
current targets – model association
Am
new estimates
data association of points belonging to a rigid body
![Page 36: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/36.jpg)
36
Total belief problem and combinatorics
relation with positive linear systems
homology of solution graphs
matroidal interpretation
general proof, number of solutions, symmetries of the graph
![Page 37: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/37.jpg)
2Computer vision
![Page 38: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/38.jpg)
38
Vision problems
HMM and size functions for gesture recognition (BMVC’97)
object tracking and pose estimation (MTNS’98,SPIE’99, MTNS’00, PAMI’04)
composition of HMMs (ASILOMAR’02)
data association with shape info (CDC’02, CDC’04, PAMI’05)
volumetric action recognition (ICIP’04,MMSP’04)
![Page 39: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/39.jpg)
Size functions for gesture recognition
![Page 40: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/40.jpg)
40
Size functions for gesture recognition Combination of HMMs (for dynamics) and
size functions (for pose representation)
![Page 41: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/41.jpg)
41
Size functions “Topological” representation of contours
![Page 42: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/42.jpg)
42
Measuring functions
Functions defined on the contour of the shape of interest
real image
measuring function
family of lines
![Page 43: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/43.jpg)
43
Feature vectors a family of measuring functions is chosen
… the szfc are computed, and their means form the feature vector
![Page 44: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/44.jpg)
44
Hidden Markov models
Finite-state model of gestures as sequences of a small number of poses
![Page 45: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/45.jpg)
45
Four-state HMM
Gesture dynamics -> transition matrix A
Object poses -> state-output matrix C
![Page 46: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/46.jpg)
46
EM algorithm
feature matrices: collection of feature vectors along time
EM A,C
learning the model’s parameters through EM
two instances of the same gesture
![Page 47: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/47.jpg)
Compositional behavior of Hidden Markov models
![Page 48: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/48.jpg)
48
Composition of HMMs Compositional behavior of HMMS: the
model of the action of interest is embedded in the overall model
Example: “fly” gesture in clutter
![Page 49: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/49.jpg)
49
State clustering Effect of clustering on HMM topology
“Cluttered” model for the two overlapping motions
Reduced model for the “fly” gesture extracted through clustering
![Page 50: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/50.jpg)
50
Kullback-Leibler comparison
We used the K-L distance to measure the similarity between models extracted from clutter and in absence of clutter
![Page 51: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/51.jpg)
Model-free object pose estimation
![Page 52: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/52.jpg)
52
Model-free pose estimation Pose estimation: inferring the configuration of a moving object from one or
more image sequences
Most approaches in the literature are model-based: they assume some knowledge about the nature of the body (articulated, deformable, etc) and some sort of model Qtq k ˆT=0 t=T
![Page 53: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/53.jpg)
53
Model-free scenario Scenario: the configuration of an uknown object
is desired, given no a-priori information about the nature itself of the body
The only info available is carried by the images
We need to build a map from image measurements to body poses
This can be done by using a learning technique, based on training data
![Page 54: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/54.jpg)
54
“Evidential” model
The evidential model is built during the training stage, when the feature-pose maps are learned
1
23
approximate feature spaces
training set of sample poses
feature-pose maps (refinings)
![Page 55: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/55.jpg)
55
32
1
Feature extraction
From the blurred image the region with color similar to the region of interest is selected, and the bounding box is detected.
1
2
3
![Page 56: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/56.jpg)
56
Estimates from the combined model
Ground truth versus estimates...
... for two components of the pose
![Page 57: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/57.jpg)
JPDA with shape information for data association
![Page 58: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/58.jpg)
58
JPDA with shape info
YX
Z
XY
Z
robustness: clutter does not meet shape constraints
occlusions: occluded targets can be estimated
JPDA model: independent targets
Shape model: rigid links
Dempster’s fusion
![Page 59: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/59.jpg)
59
Triangle simulation
the clutter affects only the standard JPDA estimates
![Page 60: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/60.jpg)
60
Body tracking
Application: tracking of feature points on a moving human body
![Page 61: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/61.jpg)
Volumetric action recognition
![Page 62: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/62.jpg)
62
Volumetric action recognition
problem: recognizing the action performed by a person viewed by a number of cameras
step 1: modeling the dynamics of the motion
step 2: extracting image features
2D approaches: features are extracted from single views -> viewpoint dependence
volumetric approach: features are extracted from a volumetric reconstruction of the moving body
![Page 63: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/63.jpg)
63
Multiple sequences
synchronized views from different cameras, chromakeying
![Page 64: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/64.jpg)
64
Volumetric intersection
silhouette extraction of the moving object from all views
3D object shape reconstruction through intersection of occlusion cones
more views -> more details
![Page 65: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/65.jpg)
65
3D feature extraction
locations of torso, arms, and legs of the moving person
k-means clustering to
separate bodyparts
![Page 66: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/66.jpg)
66
Feature matrices
two instances of the action “walking”
TORSO COORDINATES
ABDOMEN COORDINATES
RIGHT LEG COORDINATES
LEFT LEG COORDINATES
XY
Z
![Page 67: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/67.jpg)
67
Modeling and recognition
• model of the “walking” action
…
classification: each new feature matrix is fed to all the learnt models, generating a set of likelihoods
HMM 1
HMM 2
HMM n
![Page 68: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/68.jpg)
3Combinatorics
![Page 69: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/69.jpg)
69
Independence on lattices
three distinct independence relations
modularity equivalent formulations
LI3
LI2=LI3
LI1
semimodular lattice modular lattice
LI2
LI1
![Page 70: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/70.jpg)
70
Scheme of the proof
![Page 71: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/71.jpg)
4Conclusions
![Page 72: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/72.jpg)
72
from real to abstract
OBJECT TRACKING DATA ASSOCIATION
MEASUREMENTCONFLICT
POINTWISE ESTIMATE
CONDITIONALCONSTRAINTS
ALGEBRAICANALYSIS
GEOMETRICAPPROACH TOTAL BELIEF
the solution of real problems stimulates new theoretical issues
![Page 73: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/73.jpg)
73
…concluding
the ToE comes from a strong critics to the Bayesian framework
useful for sensor fusion problems under incomplete information
real problem solutions stimulate the extension of the formalism
complex objects mathematically rich Young theory need completion
![Page 74: 01/18 Lab meeting](https://reader036.vdocument.in/reader036/viewer/2022081512/56814657550346895db37325/html5/thumbnails/74.jpg)
74
In the near future..
search for a metric on the space of dynamical systems – stochastic models
sistematic description of the geometric approach to non-additive measures
understand the intricate relations between probability and combinatorics