target tracking: lecture 6 multiple sensor issues and ... · \distributed fusion architectures,...

$: Target Tracking: Lecture 6 Multiple Sensor Issues and ... · \Distributed Fusion Architectures, Algorithms, and Performance within a Network-Centric Architecture," Ch.17, Handbook$
REGLERTEKNIK

AUTOMATIC CONTROL

Target Tracking: Lecture 6Multiple Sensor Issues and Extended Target Tracking

Emre [email protected]

Division of Automatic ControlDepartment of Electrical Engineering

Linkoping UniversityLinkoping, Sweden

December 17, 2014

E. Ozkan Target Tracking December 17, 2014 1 / 53

Lecture Outline

Multiple Sensor Tracking Architectures

Multiple Sensor Tracking Problems

Extended Target Tracking


Multi Sensor Architectures cont’d

Figures taken from: M.E. Liggins and Kuo-Chu Chang“Distributed Fusion Architectures, Algorithms, and Performance within a Network-Centric Architecture,”

Ch.17, Handbook of Multisensor Data Fusion: Theory and Practice, Taylor & Francis, Second Edition, 2009.


Multi Sensor Architectures cont’d




[email protected]

Multi Sensor Architectures: Centralized




Multi Sensor Architectures: Hierarchical




Multi Sensor Architectures: Hierarchical with Feedback




Multi Sensor Architectures: Decentralized




Multi Sensor Architectures: Pros & Cons

The traditional centralized architecture gives optimal performance but

• Requires high bandwidth communications.• Requires powerful processing resources at the fusion center.• There is a single point of failure and hence reliability is low.

For distributed architectures

• Communications can be reduced significantly by communicating tracksless often.

• Computational resources can be distributed to different nodes• Higher survivability.• It is a necessity for legacy systems e.g. purchased radars might not

supply raw data.


Problems in Multi Sensor TT

Registration: Coordinates (both time and space) of different sensors orfusion agents must be aligned.

Bias: Even if the coordinate axis are aligned, due to the transformations,biases can result. These have to be compensated.

Correlation: Even if the sensors are independently collecting data, processedinformation to be fused can be correlated.

Rumor propagation: The same information can travel in loops in the fusionnetwork to produce fake information making the overall systemoverconfident. This is actually a special case of correlation.

Out of sequence measurements: Due to delayed communications betweenlocal agents, sometimes measurements belonging to a target whose morerecent measurement has already been processed, might arrive to a fusioncenter.


Correlation

Centralized Case

FusionCenter

xk|k

y2k

y1k

Decentralized Case

FusionCenter

xk|k

y2k

y1kTracker-1

Tracker-2

x1k|k

x2k|k

Suppose the fusion center have theprediction xk|k−1 in both cases.

Centralized Case[y1k

y2k

]=

[C1

C2

]xk +

[e1k

e2k

]where e1

k and e2k are independent.

Decentralized Case[x1k|kx2k|k

]=

[II

]xk −

[x1k

x2k

]


Correlation cont’d

Suppose the target follows the dynamics

xk = Axk−1 + wk

and the ith sensor measurement is given as

yik = Cixk + eik

Then with the KF equations

xik|k =Axik−1|k−1 +Kik(y

ik − CiAxik−1|k−1)

=Axik−1|k−1 +KikCi(xk −Axik−1|k−1) +Ki

keik

=Axik−1|k−1 +KikCiA(xk−1 − xik−1|k−1) +Ki

kCiwk +Kikeik



Define xik , xk − xik|k, then

xik =xk −Axik−1|k−1 −KikCiA(xk−1 − xik−1|k−1)−Ki

kCiwk −Kikeik

=Axk−1 + wk −Axik−1|k−1 −KikCiA(xk−1 − xik−1|k−1)

−KikCiwk −Ki

keik

=(I −KikCi)Ax

ik−1 + (I −Ki

kCi)wk −Kikeik

Hence

xik =(I −KikCi)Ax

ik−1 + (I −Ki

kCi)wk −Kikeik

xjk =(I −KjkCj)Ax

jk−1 + (I −Kj

kCj)wk −Kjkejk

We can calculate the correlation matrix Σijk , E(xikx

jTk ) as

Σijk = (I −Ki

kCi)AΣijk−1A

T (I −KjkCj)

T + (I −KikCi)Q(I −Kj

kCj)T



Assuming that Σij0 = 0, we can calculate the correlation between the

estimation errors of the local trackers recursively as

Σijk = (I −Ki

kCi)AΣijk−1A

T (I −KjkCj)

T + (I −KikCi)Q(I −Kj

kCj)T

This necessitates that the fusion center knows the individual Kalmangains Ki

k and Kjk of the local trackers which is not very practical.

Assuming that the errors are independent is not a good idea either.

Neglecting the correlation makes the resulting estimates overconfidenti.e., very small covariances meaning that too small gates and smallerKalman gains.

When Q = 0, Σijk = 0, i.e., no correlation when no process noise.


Correlation Illustration

Maneuvers make this problem more dominant and visible.

800 1000 1200 1400 1600 1800 2000 2200 2400 2600 2800200

400

600

800

1000

1200

1400

1600

1800

2000

2200σa = 0.3m/s2

x (m)

y (m

)

800 1000 1200 1400 1600 1800 2000 2200 2400 2600 2800200

400

600

800

1000

1200

1400

1600

1800

2000

2200σa = 0.3m/s2

x (m)

y (m

)

0 20 40 60 80 100 120 140 1600

20

40

60

80

100

120

time (s)

Pos

ition

Err

ors

(m)

0 20 40 60 80 100 120 140 1600

20

40

60

80

100

120

time (s)

Pos

ition

Err

ors

(m)


Track Association: Testing

Test for Track Association [Bar-Shalom (1995)]:

Two estimates xik|k, xjk|k and the covariances Σik|k, Σj

k|k are givenfrom ith and jth local systems.

We calculate the difference vector ∆ijk

∆ijk , xik|k − x

jk|k

Then we calculate covariance Γijk , E(∆ijk ∆ijT

k ) as

Γijk = Σik|k + Σj

k|k − Σijk − ΣijT

k

Then test statistics Dijk calculated as

Dijk = ∆ijT

k (Γijk )−1∆ijk ≶ γijk

can be used for checking track association.


Track Association cont’d

What about the cross covariance Σijk ?

Simple method is to set it Σijk = 0.

It can be calculated using Kalman gains if they are transmitted to the fusioncenter.

Approximation for cross covariance from [Bar-Shalom (1995)]:

The following cross-covariance approximation was proposed:

Σijk ≈ ρ

(Σik|k. ∗ Σj

k|k

). 12

where multiplication and power operations are to be done element-wise. Fornegative numbers, square root must be taken on the absolute value and signmust be kept.

The value of ρ must be adjusted experimentally. ρ = 0.4 was suggested for2D tracking.


Track Association: Assignment Problem

Method proposed by [Blackman (1999)] for two local agentsForm the assignment matrix:

Aij T j1 T j2 T j3 T j4 NA1 NA2 NA3

T i1 `11 `12 `13 `14 log βiNA × ×T i2 `21 `22 `23 `24 × log βiNA ×T i3 `31 `32 `33 `34 × × log βiNA

where `mn , logβTPi∈jPj∈iN (xm

k|k−xnk|k;0,Γmnk )

βjNA.

Then, use auction(Aij) to get track association decisions.


Track Association cont’d

Track association for more than two local agents.

One way is to solve multi dimensional assignment problem.

The simpler way is to do the so-called sequential pairwise trackassociation.

Suppose we have NL local agents whose tracks need to be fused. Then,we order the local agents according to some criteria e.g. accuracy, priority,etc.

Tracks ofLocal Agent 1


TrackAssociation &Track Fusion



Tracks ofLocal Agent NL


FinalFused Tracks


Track Fusion: Independence Assumption

Once we associate two tracks, we have to fuse them to obtain a fusedtrack. This is called as track fusion.Consider the track fusion at point A assuming tCR = tCT = t.

Independence assumption gives

(ΣAt )−1 =(ΣB

t )−1 + (ΣCt )−1

(ΣAt )−1xAt =(ΣB

t )−1xBt + (ΣCt )−1xCt

This is simplistic and expected to givevery bad results here.

This is also called as naive fusion.


Track Fusion cont’d

Illustration of Correlation Independent Schemes.

zT (ΣBt )−1z = 1

zT (ΣCt )−1z = 1

zT (ΣAt )−1z = 1

Largest EllipsoidAlgorithm

Covariance Intersection



Single extended target modeling

• Target state: kinematics and shape• Number of measurements per target• Motion modeling not covered. Typically possible to use point target

models.

Multiple extended target tracking

• Data association for extended targets• Measurement set partitioning



One measurement per target is often not valid, e.g.,

laser sensors, camera images, or automotive radar.

−10 −5 0 5 100

2

4

6

8

10

12

This leads us to the following definition:

Extended targets are targets that potentially give riseto more than one measurement per time step.



Multiple measurements per target ⇒ possible to estimate the target’sextension.

• The spatial extension, i.e. size and shape.• Hence the name extended target.

The extension estimate provides additional information, e.g. forclassification and identification of different target types:

• Human?• Bike?• Car?• etc...


Group targets

Closely related to extended targets is group targets.

Closely spaced point targets

Cannot be resolved

Treated as single group target

Presented extended target models applicable to group targets.


Single extended target

Airplane tracked with radar

Reflection points, or sources, 4spread across surface. Locationsand number vary with time.

Measurements zk © may fall“outside” target due to noise

Extended target state ξkcontains parameters for

• Kinematics• Extension

Not interested in sources, want to estimate the state ξk given the sets ofmeasurements

p(ξk|Zk

), Zk =

{z1k, . . . , z

mkk

}, Zk = {Z1, . . . ,Zk}


Target shape – single ellipse

Approximate shape as a single ellipse

Vector xk and positive semi-definite shape matrix Xk,

(zk − xk)T X−1

k (zk − xk) ≤ 1



“Extended Object and Group Tracking with Elliptic Random Hypersurface Models”, Baum et al.

Cholesky decomposition of shape matrix Xk = LkLTk

Lk =

[ak 0bk ck

]Extended state vector ξk =

[xTk ak bk ck

]TAssumed Gaussian models

p(ξk|Zk

)=N

(ξk ; mk|k, Pk|k

)p (zk|ξk) =N (zk ; hk (ξk) , Rk)



“Extended Object and Group Tracking with Elliptic Random Hypersurface Models”, Baum et al.

Simulation results (plots copied from paper)



“Bayesian Approach to Extended Object and Cluster Tracking using Random Matrices”, Koch.

State ξk is combination of kinematical vector xk and extension matrixXk, ξk = (xk, Xk). Gaussian inverse Wishart distributed,

p(ξk|Zk

)= N

(xk ; mk|k, Pk|k ⊗Xk

)IW

(Xk ; vk|k, Vk|k

)Called random matrix model

Measurement model

p (zk|ξk) = N (zk ; Hkxk, Xk) ,

i.e. the covariance is given by the extension matrix.



“Bayesian Approach to Extended Object and Cluster Tracking using Random Matrices”, Koch.

Pros: Linear measurement update equations for extension state.

Cons: Difficult to model sensor noise. The measurement model

p (zk|ξk) = N (zk ; Hkxk, Xk)

implies that sensor noise is much smaller than extension.

“Tracking of extended objects and group targets using random matrices”, Feldmann et al.

Alternative measurement model

p (zk|ξk) = N (zk ; Hkxk, sXk +Rk)


A simple random matrix model.

State ξk is combination of kinematical vector xk and extension matrixXk, ξk = (xk, Xk). Gaussian inverse Wishart distributed,

p(ξk|Zk

)= N

(xk ; xk|k, Pk|k

)IW

(Xk ; vk|k, Vk|k

)where,

IW (X ; v, V ) =2−

v−d−12

d|V |− v−d−12

Γd(v−d−1

2 )|X| v2exp tr(−1

2X−1V )

Measurement model





Random matrix model popular in the literature, see e.g.• “Probabilistic tracking of multiple extended targets using random matrices”,

Wieneke and Koch.• “A PHD filter for tracking multiple extended targets using random matrices”,

Granstrom and Orguner.• “Tracking of Extended Object or Target Group Using Random Matrix – Part I: New

Model and Approach”, Lan and Rong Li.

and the reference therein.

Single ellipse also used by Degerman et al in “Extended TargetTracking using Principal Components”.

• The shape matrix decomposed into rotation, major axis, and minor axis.


Target shape – multiple ellipses

A single ellipse can be a crude model, multiple ellipses gives betterapproximation, see e.g.

• Arbitrary number of ellipses, not same center point.“Tracking of Extended Object or Target Group Using Random Matrix – Part II:

Irregular Object”, Lan and Rong Li.


Taregt shape – Star convex

A set S(xk) is called star-convex if each line segment from the centerto any point is fully contained in S(xk).

−1 0 1

−1

0

1

θr

x-coordinate

y-co

ord

inat

e

0 π 2π0

1

r = f(θ)

Angle θ

Rad

iusr

Figure: Description of star-convex shapes using a radius function r = f(θ).


Target shape – general shapes

Single, or multiple, ellipses can be a crude model of the shape. Needfor more general shape model.

“Shape Tracking of Extended Objects and Group Targets with Star-Convex RHMs”,

Baum and Hanebeck.

The shape is given by a radial function r(θ) – distance from center oftarget to edge of shape at angle θ.

• Circle: r = f(θ)• Canonical ellipse: r(θ) =

√a2 cos(θ)2 + b2 sin(θ)2

• General case: truncated Fourier series expansion

r(θ) =a02

+

NF∑j=1

aj cos(jθ) + bj sin(jθ)

Shape parameter: p = [a0, a1, . . . , aNF , b1, . . . , bNF ]T


Target shape – general shapes

“Shape Tracking of Extended Objects and Group Targets with Star-Convex RHMs”,

Baum and Hanebeck.

Compare with ellipse (plots copied from paper):


Multiple extended target tracking

Assume that we have chosen the random matrix model for theextended targets.

How can we handle multiple extended targets, when there is clutterand detection uncertainty?

As with point target tracking, the data association problem must besolved.

Different because each target can generate multiple measurements.

• Which measurements were caused by clutter sources?• Which measurements were caused by targets? Of those, which were

caused by the same target?


Data association for extended targets

One approach: Partitioning of the measurement set

A partition p is a division of the set Zk into non-empty subsets, calledcells W .

Interpretation: One cell = one source.


Partitioning the measurements — example

Partition the measurement set Zk ={z

(1)k , z

(2)k , z

(3)k

}

−0.5 0 0.5 1 1.5−0.5

0

0.5

1

1.5

z(1)kz

(2)k

z(3)k

Z

x

y

−0.5 0 0.5 1 1.5−0.5

0

0.5

1

1.5

W 11

p1

x

y

−0.5 0 0.5 1 1.5−0.5

0

0.5

1

1.5

W 21

W 22

p2

x

y

−0.5 0 0.5 1 1.5−0.5

0

0.5

1

1.5

W 31

W 32

p3

x

y

−0.5 0 0.5 1 1.5−0.5

0

0.5

1

1.5

W 41

W 42

p4

x

y

−0.5 0 0.5 1 1.5−0.5

0

0.5

1

1.5

W 51

W 52

W 53

p5

x

y


Partitioning method

For optimality we must consider all possible partitions.

Number of possible partitions for n measurementsgiven by n:th Bell number Bn.

The Bell numbers Bn increase very fast when n increases,e.g. B3 = 5, B5 = 52 and B10 = 115975.

Computationally infeasible to consider all partitions.

Necessary to approximate the full set of partitionswith a subset of partitions.


Partitioning method – laser example

Intuition: Measurements are from same source if they are close, withrespect to some measure or distance.

−10 −5 0 5 100

2

4

6

8

10

12

−8 −7 −6 −5 −4 −3 −2 −1 0 1 2 31

2

3

4

5

6

7

8


Partitioning method – laser example

Intuition: Measurements are from same source if they are close, withrespect to some measure or distance.

−10 −5 0 5 100

2

4

6

8

10

12

−8 −7 −6 −5 −4 −3 −2 −1 0 1 2 31

2

3

4

5

6

7

8


Partitioning method

Method: Measurements are in same cell W if distance is “small”.

Distance between two cells Wj and Wi is measured as

min {d (zi , zj) : zi ∈Wi , zj ∈Wj}

Partitions pi where all pairwise cell distances is < di.

Limit to partitions for thresholds di that satisfy

dmin ≤ di < dmax

If possible, use scenario knowledge to choosedistance measure and to determine bounds.

Important to choose dmin and dmax conservatively.


Partitioning example

p1 ={W 1

1 ,W12 ,W

13

}

−8 −7 −6 −5 −4 −3 −2 −1 0 1 2 31

2

3

4

5

6

7

8

p2 ={W 2

1 ,W22 ,W

23 ,W

24

}

−8 −7 −6 −5 −4 −3 −2 −1 0 1 2 31

2

3

4

5

6

7

8

Reasonable to discard most partitions as highly unlikely.

Additional methods given in:“Extended Target Tracking using a Gaussian-Mixture PHD filter”, Granstrom et al.

“A PHD filter for tracking multiple extended targets using random matrices”, Granstrom

and Orguner


Extended Target Models

Extended target models applicable in scenarios where there aremultiple measurements per target.

Also applicable when multiple targets move in tight groups.

Shape models range from simple ellipse to general shapes.

Multiple extended target tracking possible using, e.g., the extendedtarget PHD filter.


Extended Multi-target Tracking- Recent Works

“A PHD filter for tracking multiple extended targets using random matrices”, Granstromand Orguner.

“A Multiple-detection joint probabilistic data association filter”, Habtemariam et al.

“A multiple hypothesis tracker for multitarget tracking with multiple simultaneousmeasurements”, Sathyan et al.


Extended Target Examples

Laser Pedestrian

Video Pedestrian

Laser - Mix

−20 −15 −10 −5 0 5 10 15 200

5

10

15

20

25

30

x [m]

y [m

]

−20 −15 −10 −5 0

4

6

8

10

12

14

x [m]

y [m

]


https://www.youtube.com/watch?v=P0rGlhFdxwQ

https://www.youtube.com/watch?v=jN-KXQqargE

References

M. E. Liggins II, Chee-Yee Chong, I. Kadar, M. G. Alford, V. Vannicola andV. Thomopoulos, “Distributed fusion architectures and algorithms for target tracking,”Proceedings of the IEEE, vol.85, no.1, pp. 95-107, Jan. 1997.

M. E. Liggins and Kuo-Chu Chang, “Distributed fusion architectures, algorithms, andperformance within a network-centric architecture,” Ch.17, Handbook of MultisensorData Fusion: Theory and Practice, Taylor & Francis, Second Edition, 2009.

K. C. Chang, Chee-Yee Chong and S. Mori, “On scalable distributed sensor fusion,”Proceedings of 11th International Conference on Information Fusion, Jul. 2008.


References

S. Blackman and R. Popoli, Design and Analysis of Modern Tracking Systems.Norwood, MA: Artech House, 1999.

Y. Bar-Shalom and X. R. Li, Multitarget-Multisensor Tracking: Principles, Techniques.Storrs, CT: YBS Publishing, 1995.

Granstrom et al., Extended Target Tracking using a Gaussian-Mixture PHD filter

Granstrom and Orguner, A PHD filter for tracking multiple extended targets usingrandom matrices


target tracking: lecture 6 multiple sensor issues and ... · \distributed fusion architectures,...

Documents