relational dynamic bayesian networks to improve multi-target tracking. cristina manfredotti and enza...
TRANSCRIPT
Relational Dynamic Bayesian Networks to improve
Multi-Target Tracking.
Cristina Manfredotti and Enza Messina
DISCo, University of Milano-Bicocca
2C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009
Relations to improve tracking
3C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009
Complex activity recognition
Y.Ke, R.Sukthankar, M.Hebert; Event Detection in Crowed Videos
4C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009
Objectives
Goals: 1. To model relations and 2. To maintain beliefs over particular
relations between objects
In order to simultaneously:
• Improve tracking with informed predictions and
• Identify complex activities based on observations and prior knowledge
5C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009
Relational Domain
Relational Domain: set of objects characterized by attributes1 and with relations1 between them
CarIdcolorposition(t)velocity(t)direction(t)DecreasingVelocity(t)
A
SameDirection(t)distance(t)Before(t)
Car BIdcolorposition(t)velocity(t)direction(t)DecreasingVelocity(t)SameDirection(t)distance(t)Before(t)
1Attributes and relations are predicate in FOL.
6C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009
Relational State
The State of a Relational Domain is the set of the predicates that are true in the Domain.
r
a
s
ss
Relational state
State of attributes
State of relations
7C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009
Relational Bayesian Networks:
Uncertainty in a Relational Domain Relational (Dynamic) Bayesian Networks
• Syntax RBN:– a set of nodes, one for each variable
– a directed, acyclic graph – a conditional distribution for each node
given its parents
This distribution must take into account the actual “complexity” of the nodes!
• Syntax RBN:– a set of nodes, one for each predicate
– a directed, graph– a conditional distribution for each node
given its parents
8C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009
Dynamics
The State of a Relational Domain is the set of the predicates that are true in the Domain.
State evolves with time
We extend a RBN to a RDBN as we are used to extend a BN to a DBN.
9C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009
Inference
Markov assumption andConditional independence of data on state.
bel(st) = ® p(zt|st)s p(st|st-1)bel(st-1)dst-1
Bayesian Filter
The problem of computing:
bel(st) = p(st|z1:t)
10C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009
Inference
Relations in the State result in correlating the State of different objects between them
p(xt-1|z1:t-1) p(xt|z1:t-1) p(xt|z1:t)
Bel(xt-1) Bel(xt) Bel(xt)
Transition model
Sensor model
t = t+1
11C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009
Sensor model (1st assumption)
part of the state relative to relations, sr, not directly observable
p(zt|st) = p(zt|sa
t)
observation zt independent by the relations between objects.
Intuitively:
Travelling Together vs Being Close
12C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009
Transition model: a trick
p(st|st-1) = p(sat,sr
t|sat-1, sr
t-1)
Sat-1
Srt-1
Sat
Srt
Intuitive
13C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009
p(sat,sr
t|sat-1,sr
t-1)=
But srt independent by sa
t-1 given srt-1 and sa
t
p(sat,sr
t|sat-1,sr
t-1) = p(sat|sa
t-1,srt-1) p(sr
t|srt-1, sa
t)
bel(st) = p(st|z1:t) = p(sat,sr
t|z1:t)
bel(st)=αp(zt|sat,sr
t)s p(sat,sr
t|sat-1,sr
t-1)bel(st-1)dst-1
p(zt|sat,sr
t) = p(zt|sat)
Relational Inference
p(sat|sa
t-1,srt-1) p(sr
t|sat-1,sr
t-1, sat)
Transition model (2nd assumption)
14C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009
* It is a technique that implements a recursive Bayesian Filter through a Monte Carlo simulation. The key idea is to represent the posterior pdf as a set of samples (particles) paired with weights and to filter the mesurament based on these weights..
Particle Filtering* (general case)
15C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009
Relational Particle Filter
16C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009
RPF: extraction
Xat,(m)
Xrt,(m)
Xat,(m)
~ p(xat,(m)|sa
t-1,srt-1)
Xat,(m)
~ p(xrt,(m)|sa
t = xat,(m),sr
t-1)
Xrt,(m)
17C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009
RPF: weighting
The consistency of the probability function ensures the convergence of the algorithm.
Xat,(m)
Xrt,(m)
Weight ( ) ~p(zt|xat)
The weighting step is done according to the attributes part of each particle only, the relational part follows.
18C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009
Experiments: FOPT
19C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009
Experiments: Transition Model
• If relation true
• If relation false
20C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009
Experiments: Results
21C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009
Further experiments
Data: 15 simulated objects.From each cell, an object can jump to one of the n next cells where n depends by the cell.
Objects can move together. If traveling together,
two (or more) objects will always be in cells from which it is possible for one to reach the
other or vice-versa.
If traveling together, two objects will behave similarly (i.e. if one
turns left, the other will follow).
22C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009
Tracking AND activity Recognition
Xat,(m)
Xrt,(m)
Xat,(m)
Xrt,(m)
Xat,(m)
Xa{t,(m)}Xo{t,(m)}
Xrt,(m)
Xat+1,(m)
1° step of sampling: prediction of the state of attributes
Xat,(m)
Xa{t,(m)}Xo{t,(m)}
Xrt,(m)
Xat+1,(m)
Xa{t,(m)}Xo{t,(m)}
Xrt+1,(m)
2° step of sampling: prediction of the state of relationsOr activity prediction
23C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009
step 12step 24
True Positive Rate
False P
ositive Rate
The worst (time step 24) and the best (time step 12) ROC curve for the relation recognition task.
Further Results
0 1
1
24C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009
PF: 4.6500 2.2333 3.7333 2.7667 …RPF: 4.6000 2.4667 1.3333 2.1000 …
PF : 4.7000 3.6667 5.6667 2.6000 … RPF: 4.6000 3.5333 5.2667 2.5333 …
Further Results (cont.)
Tracking error (distance) for each of the 15 objects.
Comparable behaviour of the errors BUT
for related objects RPF trackes always better than PF.
PF : 4.6667 4.6667 3.8333 1.9333 … RPF: 4.7667 2.7667 3.5333 1.5333 …
PF: 2.0667 5.9000 1.6000RPF: 2.0333 5.8333 2.2333
25C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009
To conclude ...
• Modeling Relations “dynamically”:– To improve multi target tracking– To recognize complex activities
• Inference in Dynamic Relational Domain– In theory complex BUT
– Simplified by
• “smart decomposition” of the transition model
• “non-relational” sensor model
• Showed promising results