A Probabilistic Framework For Segmentation And Tracking Of Multiple Non Rigid Objects For

Video Surveillance

Aleksandar Ivanovic, Tomas S. HuangICIP 2004

Outline

• Introduction

• Content segmentation– Pixel probability model– Foreground probability model

• Object tracking method

• Object detection

• Experimental result

Introduction

• In video surveillance, reliable segmentation of moving objects is essential for successful event recognition.

• Tracking non-rigid objects presents several difficulties such as handling occlusion, disjoint objects and object detection.

• Park and Aggarwal proposed that the segmentation can be done on pixel, blob and object level.

Pixel Probability Model

• Use Lu*v* space

• Use a single Gaussian model the color distribution of each pixel p(x, y) at image coordinate (x, y)

• Use Mahalanobis distance Mb (x, y) for background segmentation

Foreground Probability Model

• F(x, y) : foreground label

• A(x, y) : feature vector– A(x, y) = [Mb (x, y), D(x, y), Ph (x, y))]

– D(x, y) : absolute distance• D(x, y)= |R(x, y) – Rmean(x, y)| + |G(x, y) – Gmean(x, y)|

+ |B(x, y) – Bmean(x, y)|

– Ph (x, y) : color similarity measure

Bayesian Network (BN) Modeling

Foreground Probability Model (cont.)

• P(A|F=0), P(A|F=1) :– Use Gaussian mixture model

• Gaussian mixture model :– v = [H, S, V]T, a random variable

Blob Formation

• Foreground pixels with the same color are labeled as being in the same class.

• Connected component analysis is used to relabel the disjoint blobs.– Adjacency criterion– Color similarity criterion– Small blob criterion– Skin blob criterion : especially for human

model

Connected Components Matching

• Connected components– 4-connected components– 8-connected components

• Tracking objects by matching the connected components to the foreground objects in the previous frame.– One-to-one match– Many-to-one match– One-to-Many match

Connected Components Matching (cont.)

• (f(i), c(j)) : (foreground, connected component)• k(t) : feature vector describing f(i), c(j)

– k(t) = [xs (t), ys (t), S(t), H(t), xc (t), yc (t)]• xs (t), ys (t) : horizontal and vertical size of bounding box• S(t) : size in pixel• H(t) : color histogram of object/connected component• xc(t), yc(t) : centroid of pixels of an object/connected component

• m(i, j) : information for matching f(i) to c(j)– m(i, j) = [SC(i, j), ED(i, j), HS(i, j), XC(i, j), YC(i, j)]

• SC(i, j) : size change, S(j) / S(i)• ED(i, j) : Euclidean distance between (xc(i), yc(i)) and (xc(j), yc(j))• HS(i, j) : similarity between H(i) and H(j)• XC(i, j), YC(i, j) : xs(j) / xs(i), ys(j) / ys(i)

Probability Model

• Use BN model matching from foreground object to connected components.

• M : match label (M = 1 if matched)

Probability Model (cont.)

• Case: occlusion group objects into one

• Case: similar to background match several objects at the same time

Object Detection

• The connected components not matched to any foreground object are considered to be new objects.

• Calculate the size of candidate– Doesn’t work very well with small objects

• Define feature T = [S, LC, SH, CS]

Experimental Results

d, g: segmented objects only background model

e, h: segmented objects using Pf of foreground

b: probability based only background model

c: Pf of foreground

Color Similarity Measure Ph (x, y)

• For all tracked objects: No. of pixels in bin that contains p(x, y)

Ph(x, y) = ─────────────────────────

No. of pixels in color histogram

0

20

40

60

80

100

0~15 16~31 32~47 … 240~255 value

pix

el

nu

mb

er

BN Model for Object Detection

• S : size of the connected component• LC : distance to the nearest location of an

appearance of a foreground object• SH : simple shape feature frequently used• CS : color similarity