background subtraction based on cooccurrence of image variations seki, wada, fujiwara & sumi -...
TRANSCRIPT
Background Subtractionbased on
Cooccurrence of Image VariationsSeki, Wada, Fujiwara & Sumi - 2003
Presented by: Alon Pakash & Gilad Karni
Motivation
Detecting foreground objectsin dynamic scenes
involving swaying trees andfluttering flags.
Dynamic Scenes
Background Subtraction so far:
Stationary background
Permissible range of image
variation
Dynamic update of the background
model
Cooccurrence…
Permissible range of image variation
Feature
Space
Input image
(as vector)
Background
model
Chosen Pixels,
DCT coefficients,
…
The Problem:
Training set
Background model + VARIANCE
BIG variance = Detection sensitivity decreases!
The Solution:
Dynamically narrow the permissible range…
By using the Cooccurrence.
“Cooccurrence”
• What is Cooccurrence?
Image variations at neighboring image blocks have strong correlation!
Permissible range with Cooccurrence
Input image
(as vector)
Cooccurrence DBof background
image variations
Feature
Space
Background model without considering cooccurrence
Narrowed background
model
Cooccurrence“Is it really that good”?
• Partition the image: NxN Blocks
• In time t, block u is represented by:i(u,t)
Example:Sunlight changes
Illustrating Principal Components Analysis
Our Goal:Revealing the internal structure of the data in a way which best explains the
variance in the data
Illustrating Principal Components Analysis
Illustrating Principal Components Analysis
Illustrating Principal Components Analysis
Example:Sunlight changes
N x N
1 x N2
1 x N2
e1
e2
Projection
Another Example:Tree sway
Block A Block B
Block A Block B
Cooccurrence – Cont’d
• Also stands for:– Higher dimension feature space– Other neighboring blocks in the picture– Fluttering flags
• Conclusion:Neighboring image blocks have strong
correlation!
Background Subtraction Method
The general idea:Narrow the background image variations by
estimating the background image in each block from the neighboring blocks in the input
image
e1
e2
e3
e1
e2
e3
(A,t1)(B,t1)
Z*
ZB(A,t2)
(B,t2)
(A,t3)(B,t3)ZA
e1
e2
e3
Z)B,t1(
Z*
ZB
Z)B,t2(
Z)B,t3(
Advantages
• Since the method utilizes the spatial property of background image variations, it is not affected by the quick image variations.
• The method can be applied not only to the background object motions, such as swaying tree leaves, but also to illumination variations.
Experiments
Difference Picture
The experiment procedure
• Number of dimensions?
• Number of neighbors?
Num. of Dimensions
• Determination of the dimensions of the eigen space: until more than 90% of the blocks are “effective”.
Num. of neighbors
• Determination of the number of neighbors: until the error (the Euclidean distance in the eigen space) is small enough.
Z(B,t1)
Z*
ZB
Z(B,t2)
Z)B,t3(
Comparison to other methods
• Method 1: Learning in the same features space for each block, background subtraction using Mahalanobis distances.
• Method 2: Doesn’t use “Cooccurence”, relies only on the input pattern in the focused block.
• Method 3: The proposed method.
Belief Propagation in a 3D Spatio-temporal MRF for Moving Object Detection
Yin & Collins - 2007
Dis\Similarity
Surroundings of an element is taken into consideration
Pixel Vs. Block
Problems solved in this method
• Objects camouflaged by similar appearance to the background
• Objects with uniform color
Markov Random Field+-+++-
-----+
+-++--
---++-
-+-+--
-++-++
+
+-
P)Xij = + | Xkm, k≠i, m≠j(
= P)Xij = + | Xkm, k=i±1, m=j±1(
• With the realization that each pixel influences neighboring pixels spatially and temporally in the video sequence we develop a 3D MRF (Markov Random Field) model to represent the system.
Frame i
Frame
i+1
Frame i-1
• Observed Data
• Hidden State
Hidden State & Observed Data
• Hidden state – represents the likelihood that a pixel contains object motion
• Observed data – represents the binary motion detection result
Relations between hidden to observed nodes
• If an observed node is “0” (no-motion), its corrsponding hidden node will contain a uniform distribution.
• Otherwise, it will contain an impulse distribution.
Φ j (s k
,d k)
sksk
Relations between hidden to hidden nodes
• Each hidden node encourages its neighboring nodes to have the same state. sk
sk
Ψjk
Belief PropagationIn a nutshell
A powerful algorithm for making approximate inferences over joint
distributions defined by MRF models
Message Update Schedule
• Different message passing schedules have different effects on the detection process.
To Conclude
• Copes with shape changes
• Not affected by speed changes of the moving object
• Handles low resolution videos (e.g. Thermal)