background subtraction for urban traffic monitoring using webcams master graduation project final...
Post on 21-Dec-2015
214 views
TRANSCRIPT
Background Subtraction for Urban Traffic Monitoring
using Webcams
Master Graduation ProjectFinal Presentation
Supervisor: Rein van den Boomgaard
Mark Smids
December 12th 2006
Overview
• Introduction• Background Subtraction• Shadow Detection• Video Summarization • Demo’s
• Background Subtraction in action• Shadow Detector in action• Smart Surveillance using Video Summarization
• Evaluation • Conclusions
Introduction
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
• Traditional ways of traffic monitoring
• using magnetic loops
Introduction
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
• Traditional ways of traffic monitoring
• using magnetic loops
• Limitations:
• These systems only count, very costly
Introduction
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
• Traditional ways of traffic monitoring
• using magnetic loops
• Limitations:
• These systems only count, very costly
• For extended traffic monitoring we want to measure:
• road density, queue detection, vehicle speed, exact location of vehicles
Introduction
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
• Traditional ways of traffic monitoring
• using magnetic loops
• Limitations:
• These systems only count, very costly
• For extended traffic monitoring we want to measure:
• road density, queue detection, vehicle speed, exact location of vehicles
• Solution: use cameras to monitor traffic automatically
Introduction
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
• Why focus on an urban setting?
• Most research focused on a highway setting
• More challenging tasks
Introduction
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
• Why focus on an urban setting?
• Most research focused on a highway setting
• More challenging tasks
• Components of a vision based traffic monitoring system:
• cameras, calibration, background subtraction, tracking, shadow detection, parameter extraction, video summarization, …
Introduction
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
• Why focus on an urban setting?
• Most research focused on a highway setting
• More challenging tasks
• Components of a vision based traffic monitoring system:
• cameras, calibration, background subtraction, tracking, shadow detection, parameter extraction, video summarization, …
Background Subtraction
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
• Deterministic approach
• Create an initial background model from the first
N frames
• For each new frame, subtract it from the background model to obtain a binary mask for all x,y: if I(x,y) – B(x,y) > T then M(x,y) = 1 else M(x,y) = 0
• Update the background model: for all x,y: if M(x,y) = 0 then B(x,y) = I(x,y)
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
• Statistical approach
• Model each pixel in the background model by a mixture of Gaussians
),,(*)( ,,1
, titit
k
itit XXP
Background Subtraction
/
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
• Statistical approach
• Model each pixel in the background model by a mixture of Gaussians
• How to determine those components that model
the background?• Observation: these Gaussians have the most supporting evidence and lowest variances • Order the K distributions in the mixture by the value of• The first B distributions are chosen as the background model, where:
TBb
kkb
1
minarg
Background Subtraction
),,(*)( ,,1
, titit
k
itit XXP
Shadow Detection
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
• Shadows: cast and self shadows
Shadow Detection
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
• Shadows: cast and self shadows
• Elimination of cast shadows can improve background subtraction results very much…
Shadow Detection
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
• Shadows: cast and self shadows
• Elimination of cast shadows can improve background subtraction results very much…
Shadow Detection
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
• Consider the set of pixels classified as foreground pixels
Shadow Detection
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
• Consider the set of pixels classified as foreground pixels
• A pixel is a candidate shadow pixel when the pixel value has a significant lower value
than it’s corresponding background value
Shadow Detection
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
• Consider the set of pixels classified as foreground pixels
• A pixel is a candidate shadow pixel when the pixel value has a significant lower value
than it’s corresponding background value
• Extend this idea: let c = (R,G,B) and
Rate of similarity:
),,( bgr
c
SRGB
Shadow Detection
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
• Consider the set of pixels classified as foreground pixels
• A pixel is a candidate shadow pixel when the pixel value has a significant lower value
than it’s corresponding background value
• Extend this idea: let c = (R,G,B) and
Rate of similarity:
• If tau < < 1 then pixel is a shadow pixel
),,( bgr
c
SRGB
RGBS
Video Summarization
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
• Application: smart vision based surveillance system
Video Summarization
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
• Application: smart vision based surveillance system
• Record only frames which includes relevant foreground objects
Video Summarization
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
• Application: smart vision based surveillance system
• Record only frames which includes relevant foreground objects
How to guarantee that a full trajectory of a vehicle is recorded?
Demos
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
1. Shadow Detector in action – 1 | 2
2. Background Subtraction in action det 1 | stat 1 - det 2 | stat 2
3. Smart Surveillance using Video Sum. - 1
Evaluation
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
• Test videos: three different weather conditions (5 minutes each)
• Goal: test both background subtraction algorithms on these videos
• Limitation: no ground truth available!
Evaluation
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
• Evaluation on another level: using the video summarization component.
• A frame level ground truth is created
• For each algorithm a score can be computed
Evaluation
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
100)1( N
ES
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
Score S
(deterministicapproach)
Score S
(statisticalapproach)
Totalnumber of Frames
Video A(wind/cloudy)
85.6% 88.3% 4581
Video B(sunny)
88.5% 94.6% 4163
Video C(rain)
83.4% 93.4% 3024
Evaluation
Conclusions
• For all weather conditions: the statistical approach outperforms the deterministic approach (5-10%)
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
Conclusions
• For all weather conditions: the statistical approach outperforms the deterministic approach (5-10%)
• Wind is the hardest problem from both algorithms
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
Conclusions
• For all weather conditions: the statistical approach outperforms the deterministic approach (5-10%)
• Wind is the hardest problem from both algorithms
• Statistical approach performs much better in the sunny settings
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
Conclusions
• For all weather conditions: the statistical approach outperforms the deterministic approach (5-10%)
• Wind is the hardest problem from both algorithms
• Statistical approach performs much better in the sunny settings
• Future work: create a pixel-level ground truth and evaluate both algorithms
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
Questions?
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
• Questions?
http://www.science.uva.nl/~msmids/afstuderen/master
MoG details
• Update Equations:)( k
tkkk E
kktkkk E )/(
22kk ))(/( 2
kkTkk
tkE
Z. Zivkovic, “Improved Adaptive Gaussian Mixture Model for Background Subtraction”
ktk X
• MoG: ),,(*)( ,,1
, titit
k
itit XXP
)()(2
1
2
12
1
)2(
1),,(
ttT
tt XX
nt eX
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions