![Page 1: Background Subtraction for Urban Traffic Monitoring using Webcams Master Graduation Project Final Presentation Supervisor: Rein van den Boomgaard Mark](https://reader030.vdocument.in/reader030/viewer/2022032522/56649d615503460f94a43285/html5/thumbnails/1.jpg)
Background Subtraction for Urban Traffic Monitoring
using Webcams
Master Graduation ProjectFinal Presentation
Supervisor: Rein van den Boomgaard
Mark Smids
December 12th 2006
![Page 2: Background Subtraction for Urban Traffic Monitoring using Webcams Master Graduation Project Final Presentation Supervisor: Rein van den Boomgaard Mark](https://reader030.vdocument.in/reader030/viewer/2022032522/56649d615503460f94a43285/html5/thumbnails/2.jpg)
Overview
• Introduction• Background Subtraction• Shadow Detection• Video Summarization • Demo’s
• Background Subtraction in action• Shadow Detector in action• Smart Surveillance using Video Summarization
• Evaluation • Conclusions
![Page 3: Background Subtraction for Urban Traffic Monitoring using Webcams Master Graduation Project Final Presentation Supervisor: Rein van den Boomgaard Mark](https://reader030.vdocument.in/reader030/viewer/2022032522/56649d615503460f94a43285/html5/thumbnails/3.jpg)
Introduction
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
• Traditional ways of traffic monitoring
• using magnetic loops
![Page 4: Background Subtraction for Urban Traffic Monitoring using Webcams Master Graduation Project Final Presentation Supervisor: Rein van den Boomgaard Mark](https://reader030.vdocument.in/reader030/viewer/2022032522/56649d615503460f94a43285/html5/thumbnails/4.jpg)
Introduction
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
• Traditional ways of traffic monitoring
• using magnetic loops
• Limitations:
• These systems only count, very costly
![Page 5: Background Subtraction for Urban Traffic Monitoring using Webcams Master Graduation Project Final Presentation Supervisor: Rein van den Boomgaard Mark](https://reader030.vdocument.in/reader030/viewer/2022032522/56649d615503460f94a43285/html5/thumbnails/5.jpg)
Introduction
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
• Traditional ways of traffic monitoring
• using magnetic loops
• Limitations:
• These systems only count, very costly
• For extended traffic monitoring we want to measure:
• road density, queue detection, vehicle speed, exact location of vehicles
![Page 6: Background Subtraction for Urban Traffic Monitoring using Webcams Master Graduation Project Final Presentation Supervisor: Rein van den Boomgaard Mark](https://reader030.vdocument.in/reader030/viewer/2022032522/56649d615503460f94a43285/html5/thumbnails/6.jpg)
Introduction
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
• Traditional ways of traffic monitoring
• using magnetic loops
• Limitations:
• These systems only count, very costly
• For extended traffic monitoring we want to measure:
• road density, queue detection, vehicle speed, exact location of vehicles
• Solution: use cameras to monitor traffic automatically
![Page 7: Background Subtraction for Urban Traffic Monitoring using Webcams Master Graduation Project Final Presentation Supervisor: Rein van den Boomgaard Mark](https://reader030.vdocument.in/reader030/viewer/2022032522/56649d615503460f94a43285/html5/thumbnails/7.jpg)
Introduction
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
• Why focus on an urban setting?
• Most research focused on a highway setting
• More challenging tasks
![Page 8: Background Subtraction for Urban Traffic Monitoring using Webcams Master Graduation Project Final Presentation Supervisor: Rein van den Boomgaard Mark](https://reader030.vdocument.in/reader030/viewer/2022032522/56649d615503460f94a43285/html5/thumbnails/8.jpg)
Introduction
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
• Why focus on an urban setting?
• Most research focused on a highway setting
• More challenging tasks
• Components of a vision based traffic monitoring system:
• cameras, calibration, background subtraction, tracking, shadow detection, parameter extraction, video summarization, …
![Page 9: Background Subtraction for Urban Traffic Monitoring using Webcams Master Graduation Project Final Presentation Supervisor: Rein van den Boomgaard Mark](https://reader030.vdocument.in/reader030/viewer/2022032522/56649d615503460f94a43285/html5/thumbnails/9.jpg)
Introduction
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
• Why focus on an urban setting?
• Most research focused on a highway setting
• More challenging tasks
• Components of a vision based traffic monitoring system:
• cameras, calibration, background subtraction, tracking, shadow detection, parameter extraction, video summarization, …
![Page 10: Background Subtraction for Urban Traffic Monitoring using Webcams Master Graduation Project Final Presentation Supervisor: Rein van den Boomgaard Mark](https://reader030.vdocument.in/reader030/viewer/2022032522/56649d615503460f94a43285/html5/thumbnails/10.jpg)
Background Subtraction
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
• Deterministic approach
• Create an initial background model from the first
N frames
• For each new frame, subtract it from the background model to obtain a binary mask for all x,y: if I(x,y) – B(x,y) > T then M(x,y) = 1 else M(x,y) = 0
• Update the background model: for all x,y: if M(x,y) = 0 then B(x,y) = I(x,y)
![Page 11: Background Subtraction for Urban Traffic Monitoring using Webcams Master Graduation Project Final Presentation Supervisor: Rein van den Boomgaard Mark](https://reader030.vdocument.in/reader030/viewer/2022032522/56649d615503460f94a43285/html5/thumbnails/11.jpg)
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
• Statistical approach
• Model each pixel in the background model by a mixture of Gaussians
),,(*)( ,,1
, titit
k
itit XXP
Background Subtraction
![Page 12: Background Subtraction for Urban Traffic Monitoring using Webcams Master Graduation Project Final Presentation Supervisor: Rein van den Boomgaard Mark](https://reader030.vdocument.in/reader030/viewer/2022032522/56649d615503460f94a43285/html5/thumbnails/12.jpg)
/
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
• Statistical approach
• Model each pixel in the background model by a mixture of Gaussians
• How to determine those components that model
the background?• Observation: these Gaussians have the most supporting evidence and lowest variances • Order the K distributions in the mixture by the value of• The first B distributions are chosen as the background model, where:
TBb
kkb
1
minarg
Background Subtraction
),,(*)( ,,1
, titit
k
itit XXP
![Page 13: Background Subtraction for Urban Traffic Monitoring using Webcams Master Graduation Project Final Presentation Supervisor: Rein van den Boomgaard Mark](https://reader030.vdocument.in/reader030/viewer/2022032522/56649d615503460f94a43285/html5/thumbnails/13.jpg)
Shadow Detection
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
• Shadows: cast and self shadows
![Page 14: Background Subtraction for Urban Traffic Monitoring using Webcams Master Graduation Project Final Presentation Supervisor: Rein van den Boomgaard Mark](https://reader030.vdocument.in/reader030/viewer/2022032522/56649d615503460f94a43285/html5/thumbnails/14.jpg)
Shadow Detection
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
• Shadows: cast and self shadows
• Elimination of cast shadows can improve background subtraction results very much…
![Page 15: Background Subtraction for Urban Traffic Monitoring using Webcams Master Graduation Project Final Presentation Supervisor: Rein van den Boomgaard Mark](https://reader030.vdocument.in/reader030/viewer/2022032522/56649d615503460f94a43285/html5/thumbnails/15.jpg)
Shadow Detection
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
• Shadows: cast and self shadows
• Elimination of cast shadows can improve background subtraction results very much…
![Page 16: Background Subtraction for Urban Traffic Monitoring using Webcams Master Graduation Project Final Presentation Supervisor: Rein van den Boomgaard Mark](https://reader030.vdocument.in/reader030/viewer/2022032522/56649d615503460f94a43285/html5/thumbnails/16.jpg)
Shadow Detection
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
• Consider the set of pixels classified as foreground pixels
![Page 17: Background Subtraction for Urban Traffic Monitoring using Webcams Master Graduation Project Final Presentation Supervisor: Rein van den Boomgaard Mark](https://reader030.vdocument.in/reader030/viewer/2022032522/56649d615503460f94a43285/html5/thumbnails/17.jpg)
Shadow Detection
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
• Consider the set of pixels classified as foreground pixels
• A pixel is a candidate shadow pixel when the pixel value has a significant lower value
than it’s corresponding background value
![Page 18: Background Subtraction for Urban Traffic Monitoring using Webcams Master Graduation Project Final Presentation Supervisor: Rein van den Boomgaard Mark](https://reader030.vdocument.in/reader030/viewer/2022032522/56649d615503460f94a43285/html5/thumbnails/18.jpg)
Shadow Detection
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
• Consider the set of pixels classified as foreground pixels
• A pixel is a candidate shadow pixel when the pixel value has a significant lower value
than it’s corresponding background value
• Extend this idea: let c = (R,G,B) and
Rate of similarity:
),,( bgr
c
SRGB
![Page 19: Background Subtraction for Urban Traffic Monitoring using Webcams Master Graduation Project Final Presentation Supervisor: Rein van den Boomgaard Mark](https://reader030.vdocument.in/reader030/viewer/2022032522/56649d615503460f94a43285/html5/thumbnails/19.jpg)
Shadow Detection
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
• Consider the set of pixels classified as foreground pixels
• A pixel is a candidate shadow pixel when the pixel value has a significant lower value
than it’s corresponding background value
• Extend this idea: let c = (R,G,B) and
Rate of similarity:
• If tau < < 1 then pixel is a shadow pixel
),,( bgr
c
SRGB
RGBS
![Page 20: Background Subtraction for Urban Traffic Monitoring using Webcams Master Graduation Project Final Presentation Supervisor: Rein van den Boomgaard Mark](https://reader030.vdocument.in/reader030/viewer/2022032522/56649d615503460f94a43285/html5/thumbnails/20.jpg)
Video Summarization
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
• Application: smart vision based surveillance system
![Page 21: Background Subtraction for Urban Traffic Monitoring using Webcams Master Graduation Project Final Presentation Supervisor: Rein van den Boomgaard Mark](https://reader030.vdocument.in/reader030/viewer/2022032522/56649d615503460f94a43285/html5/thumbnails/21.jpg)
Video Summarization
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
• Application: smart vision based surveillance system
• Record only frames which includes relevant foreground objects
![Page 22: Background Subtraction for Urban Traffic Monitoring using Webcams Master Graduation Project Final Presentation Supervisor: Rein van den Boomgaard Mark](https://reader030.vdocument.in/reader030/viewer/2022032522/56649d615503460f94a43285/html5/thumbnails/22.jpg)
Video Summarization
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
• Application: smart vision based surveillance system
• Record only frames which includes relevant foreground objects
How to guarantee that a full trajectory of a vehicle is recorded?
![Page 23: Background Subtraction for Urban Traffic Monitoring using Webcams Master Graduation Project Final Presentation Supervisor: Rein van den Boomgaard Mark](https://reader030.vdocument.in/reader030/viewer/2022032522/56649d615503460f94a43285/html5/thumbnails/23.jpg)
Demos
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
1. Shadow Detector in action – 1 | 2
2. Background Subtraction in action det 1 | stat 1 - det 2 | stat 2
3. Smart Surveillance using Video Sum. - 1
![Page 24: Background Subtraction for Urban Traffic Monitoring using Webcams Master Graduation Project Final Presentation Supervisor: Rein van den Boomgaard Mark](https://reader030.vdocument.in/reader030/viewer/2022032522/56649d615503460f94a43285/html5/thumbnails/24.jpg)
Evaluation
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
• Test videos: three different weather conditions (5 minutes each)
• Goal: test both background subtraction algorithms on these videos
• Limitation: no ground truth available!
![Page 25: Background Subtraction for Urban Traffic Monitoring using Webcams Master Graduation Project Final Presentation Supervisor: Rein van den Boomgaard Mark](https://reader030.vdocument.in/reader030/viewer/2022032522/56649d615503460f94a43285/html5/thumbnails/25.jpg)
Evaluation
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
• Evaluation on another level: using the video summarization component.
• A frame level ground truth is created
• For each algorithm a score can be computed
![Page 26: Background Subtraction for Urban Traffic Monitoring using Webcams Master Graduation Project Final Presentation Supervisor: Rein van den Boomgaard Mark](https://reader030.vdocument.in/reader030/viewer/2022032522/56649d615503460f94a43285/html5/thumbnails/26.jpg)
Evaluation
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
100)1( N
ES
![Page 27: Background Subtraction for Urban Traffic Monitoring using Webcams Master Graduation Project Final Presentation Supervisor: Rein van den Boomgaard Mark](https://reader030.vdocument.in/reader030/viewer/2022032522/56649d615503460f94a43285/html5/thumbnails/27.jpg)
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
Score S
(deterministicapproach)
Score S
(statisticalapproach)
Totalnumber of Frames
Video A(wind/cloudy)
85.6% 88.3% 4581
Video B(sunny)
88.5% 94.6% 4163
Video C(rain)
83.4% 93.4% 3024
Evaluation
![Page 28: Background Subtraction for Urban Traffic Monitoring using Webcams Master Graduation Project Final Presentation Supervisor: Rein van den Boomgaard Mark](https://reader030.vdocument.in/reader030/viewer/2022032522/56649d615503460f94a43285/html5/thumbnails/28.jpg)
Conclusions
• For all weather conditions: the statistical approach outperforms the deterministic approach (5-10%)
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
![Page 29: Background Subtraction for Urban Traffic Monitoring using Webcams Master Graduation Project Final Presentation Supervisor: Rein van den Boomgaard Mark](https://reader030.vdocument.in/reader030/viewer/2022032522/56649d615503460f94a43285/html5/thumbnails/29.jpg)
Conclusions
• For all weather conditions: the statistical approach outperforms the deterministic approach (5-10%)
• Wind is the hardest problem from both algorithms
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
![Page 30: Background Subtraction for Urban Traffic Monitoring using Webcams Master Graduation Project Final Presentation Supervisor: Rein van den Boomgaard Mark](https://reader030.vdocument.in/reader030/viewer/2022032522/56649d615503460f94a43285/html5/thumbnails/30.jpg)
Conclusions
• For all weather conditions: the statistical approach outperforms the deterministic approach (5-10%)
• Wind is the hardest problem from both algorithms
• Statistical approach performs much better in the sunny settings
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
![Page 31: Background Subtraction for Urban Traffic Monitoring using Webcams Master Graduation Project Final Presentation Supervisor: Rein van den Boomgaard Mark](https://reader030.vdocument.in/reader030/viewer/2022032522/56649d615503460f94a43285/html5/thumbnails/31.jpg)
Conclusions
• For all weather conditions: the statistical approach outperforms the deterministic approach (5-10%)
• Wind is the hardest problem from both algorithms
• Statistical approach performs much better in the sunny settings
• Future work: create a pixel-level ground truth and evaluate both algorithms
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
![Page 32: Background Subtraction for Urban Traffic Monitoring using Webcams Master Graduation Project Final Presentation Supervisor: Rein van den Boomgaard Mark](https://reader030.vdocument.in/reader030/viewer/2022032522/56649d615503460f94a43285/html5/thumbnails/32.jpg)
Questions?
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions
• Questions?
http://www.science.uva.nl/~msmids/afstuderen/master
![Page 33: Background Subtraction for Urban Traffic Monitoring using Webcams Master Graduation Project Final Presentation Supervisor: Rein van den Boomgaard Mark](https://reader030.vdocument.in/reader030/viewer/2022032522/56649d615503460f94a43285/html5/thumbnails/33.jpg)
MoG details
• Update Equations:)( k
tkkk E
kktkkk E )/(
22kk ))(/( 2
kkTkk
tkE
Z. Zivkovic, “Improved Adaptive Gaussian Mixture Model for Background Subtraction”
ktk X
• MoG: ),,(*)( ,,1
, titit
k
itit XXP
)()(2
1
2
12
1
)2(
1),,(
ttT
tt XX
nt eX
• Introduction
• Background Subtraction
• Shadow Detection
• Video Summarization
• Demos
• Evaluation
• Conclusions