background subtraction based on cooccurrence of image variations seki, wada, fujiwara & sumi -...

Background Subtractionbased on

Cooccurrence of Image VariationsSeki, Wada, Fujiwara & Sumi - 2003

Presented by: Alon Pakash & Gilad Karni

Motivation

Detecting foreground objectsin dynamic scenes

involving swaying trees andfluttering flags.

Dynamic Scenes

Background Subtraction so far:

Stationary background

Permissible range of image

variation

Dynamic update of the background

model

Cooccurrence…

Permissible range of image variation

Feature

Space

Input image

(as vector)

Background

model

Chosen Pixels,

DCT coefficients,

…

The Problem:

Training set

Background model + VARIANCE

BIG variance = Detection sensitivity decreases!

The Solution:

Dynamically narrow the permissible range…

By using the Cooccurrence.

“Cooccurrence”

• What is Cooccurrence?

Image variations at neighboring image blocks have strong correlation!

Permissible range with Cooccurrence

Input image

(as vector)

Cooccurrence DBof background

image variations

Feature

Space

Background model without considering cooccurrence

Narrowed background

model

Cooccurrence“Is it really that good”?

• Partition the image: NxN Blocks

• In time t, block u is represented by:i(u,t)

Example:Sunlight changes

Illustrating Principal Components Analysis

Our Goal:Revealing the internal structure of the data in a way which best explains the

variance in the data

Illustrating Principal Components Analysis

Illustrating Principal Components Analysis

Illustrating Principal Components Analysis

Example:Sunlight changes

N x N

1 x N2

1 x N2

e1

e2

Projection

Another Example:Tree sway

Block A Block B

Block A Block B

Cooccurrence – Cont’d

• Also stands for:– Higher dimension feature space– Other neighboring blocks in the picture– Fluttering flags

• Conclusion:Neighboring image blocks have strong

correlation!

Background Subtraction Method

The general idea:Narrow the background image variations by

estimating the background image in each block from the neighboring blocks in the input

image

e1

e2

e3

e1

e2

e3

(A,t1)(B,t1)

Z*

ZB(A,t2)

(B,t2)

(A,t3)(B,t3)ZA

e1

e2

e3

Z)B,t1(

Z*

ZB

Z)B,t2(

Z)B,t3(

Advantages

• Since the method utilizes the spatial property of background image variations, it is not affected by the quick image variations.

• The method can be applied not only to the background object motions, such as swaying tree leaves, but also to illumination variations.

Experiments

Difference Picture

The experiment procedure

• Number of dimensions?

• Number of neighbors?

Num. of Dimensions

• Determination of the dimensions of the eigen space: until more than 90% of the blocks are “effective”.

Num. of neighbors

• Determination of the number of neighbors: until the error (the Euclidean distance in the eigen space) is small enough.

Z(B,t1)

Z*

ZB

Z(B,t2)

Z)B,t3(

Comparison to other methods

• Method 1: Learning in the same features space for each block, background subtraction using Mahalanobis distances.

• Method 2: Doesn’t use “Cooccurence”, relies only on the input pattern in the focused block.

• Method 3: The proposed method.

Belief Propagation in a 3D Spatio-temporal MRF for Moving Object Detection

Yin & Collins - 2007

Dis\Similarity

Surroundings of an element is taken into consideration

Pixel Vs. Block

Problems solved in this method

• Objects camouflaged by similar appearance to the background

• Objects with uniform color

Markov Random Field+-+++-

-----+

+-++--

---++-

-+-+--

-++-++

+

+-

P)Xij = + | Xkm, k≠i, m≠j(

= P)Xij = + | Xkm, k=i±1, m=j±1(

• With the realization that each pixel influences neighboring pixels spatially and temporally in the video sequence we develop a 3D MRF (Markov Random Field) model to represent the system.

Frame i

Frame

i+1

Frame i-1

• Observed Data

• Hidden State

Hidden State & Observed Data

• Hidden state – represents the likelihood that a pixel contains object motion

• Observed data – represents the binary motion detection result

Relations between hidden to observed nodes

• If an observed node is “0” (no-motion), its corrsponding hidden node will contain a uniform distribution.

• Otherwise, it will contain an impulse distribution.

Φ j (s k

,d k)

sksk

Relations between hidden to hidden nodes

• Each hidden node encourages its neighboring nodes to have the same state. sk

sk

Ψjk

Belief PropagationIn a nutshell

A powerful algorithm for making approximate inferences over joint

distributions defined by MRF models

Message Update Schedule

• Different message passing schedules have different effects on the detection process.

To Conclude

• Copes with shape changes

• Not affected by speed changes of the moving object

• Handles low resolution videos (e.g. Thermal)