-
Michel AMAT PROBAYES SAS
3rd year ENSIMAG 345, rue Lavoisier - Inovallée
July 6th to Sept 15th 2009 38330 MONTBONNOT
WORK EXPERIENCE REPORT
TITLE: Design & Integration of computer vision algorithms in a video surveillance application.
ABSTRACT:
Open source softwares such as Zone Minder (http://www.zoneminder.com/) constitute complete video
surveillance systems but lack of any artificial intelligence component. On the contrary, Probayes SAS owns
many artificial intelligence libraries employed for computer vision, as well as for Bayesian analysis.
The topic of my training period was to plug existing softwares developed by Probayes in a complete video
surveillance solution: ZoneMinder.
First, I had to install and test ZoneMinder, the video surveillance software Probayes would like to employ.
For this purpose, a specific environment made of several USB & IP cameras was provided.
Second, existing libraries from Probayes allowing to do image and video analysis have been integrated in
this platform.
1
-
Contents
1 INTRODUCTION 3
1.1 About Probayes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2 Work Experience Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2 Background Subtraction Algorithms 4
2.1 Problematics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.2 Background Modelling with One Gaussian . . . . . . . . . . . . . . . . . . . . . . . . 4
2.3 Background Modelling with a Mixture of Gaussian . . . . . . . . . . . . . . . . . . . 5
2.4 Post processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.5 Software architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.6 Utilization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
3 ZONEMINDER 15
3.1 Installation of ZoneMinder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
3.2 Integration of the background subtraction algorithms in ZoneMinder . . . . . . . 17
4 CONCLUSION 18
Bibliography 19
2
-
1 INTRODUCTION
1.1 About Probayes
Probayes headquarters is located in France, 10km away from Grenoble. Grenoble is often referred as the
French capital of Alps and winter sports. The company is part of the 300 companies and laboratories of
the technopolis Inovallée. Probayes is a software editor specialized in Bayesian modelling, inference, and
learning. The experience of Probayes allows to propose:
• an unsurpassed know-how in implementing Bayesian solutions;
• an exclusive methodology to design such applications;
• a robust, fast, and easy-to-use computation engine;
• a set of patented software components ready to be integrated in any application.
The core technology ProBT c© extends the Bayesian Networks framework by providing a structured
programming language allowing the developers to increase their applications capabilities and robustness by
easily integrating Bayesian models.
Built upon a strong and rigorous theory, Bayesian techniques allow modelling complex real-world prob-
lems in a unified framework.
Bayesian inference and learning provide a powerful mathematical tool for:
• merging inaccurate/incomplete data with expert priors in a unified way;
• on-line learning to fit data evolution.
1.2 Work Experience Development
I worked within the R&D team, involved in important R&D projects like ProBT (http://www.probayes.com/products/
modelling-development-toolkits/probt/) or Prometheus (https://www.informationsystems.foi.se/∼prometheus-
fp7) whose responsible is my supervisor, Julien ROS, R&D engineer. However, my work was independent
of all the other projects and actually was pretty free in the directions to follow. It allows me to choose
to develop new competences interesting for my career. Julien ROS was very available, this gave me the
opportunity to learn very fast. Yet, he wasn’t at Probayes on the first week, and left on holidays for 2 weeks
in August. So, I also had to be autonomous in order to solve problems on my own.
3
-
2 Background Subtraction Algorithms
2.1 Problematics
Identifying moving objects in a video sequence is a fundamental and critical task in many computer vision
applications. Background subtraction techniques are commonly used to separate foreground moving objects
from the background [2].
Although different, most Background Subtraction techniques share a common framework: they make the
hypothesis that the observed video sequence I is made of a fixed background B in front of which moving
objects are observed. It is also based on the assumption that a moving object at time t has a color (or a
color distribution) different from the one observed in B.
In the algorithms presented in this report, the background B is modeled by a Gaussian (or a mixture of
K Gaussian) for each pixel. For each new frame It, the algorithm builds a mask of the new frame showing
which pixels are part of the foreground. Some post processing steps based on this mask permit to obtain a
better vision of the objects in the foreground (i-e: morphological steps, shadows removing, blob detection).
2.2 Background Modelling with One Gaussian
Principle Each pixel of the background is modeled separately by its average µ and its standard deviation
σ. For each new frame, those parameters of the background are updated as follow:
µt+1 = α ∗ Ft + (1 − α)µt (2.1)
σ2t+1 = α(Ft − µt)2 + (1 − α)σ2t (2.2)
The mask M is then obtained by comparison between the current background B and the current frame
It:
Mt = {|It − Bt| > Th ∗ σt} (2.3)
This algorithm has been implemented twice: once using RGB images, another time using grey images.
The results are equivalent except the program is quicker for grey images.
This algorithm has been applied on different video sequences and a result is shown on figure 1.
4
-
figure 1: Modelling of the background with one Gaussian
2.3 Background Modelling with a Mixture of Gaussian
In a lot of cases, the Background is not stable (camera vibrations, fluids, moving trees...). Consequently, the
distributions of the background pixels are not unimodal. In order to prevent the algorithm from considering
those pixels in the foreground, a mixture of Gaussian is used to model the background [4, 1].
Principle Each pixel of the background is modeled separately by a mixture of K Gaussian. The back-
ground is updated, before the foreground is detected, as follows:
1. If It matches component i, i.e., It is within λ standard deviations of µi,t, then the ith component is
updated as follows:
ωi,t = ωi,t−1 + α (2.4)
5
-
µi,t = µi,t−1 ∗ (1 −α
ωi,t) +
α
ωi,t∗ It (2.5)
σ2i,t = σ2i,t−1 ∗ (1 −
α
ωi,t) +
α
ωi,t∗ (It − µi,t)
2 (2.6)
2. If It does not match any component, then the least likely component is replaced with a new one initialized
with µi,t = It , σ0 large, and ωi,t low. After the update, the weights ωi,t are re-normalised. The foreground
is detected as follows. All components in the mixture are sorted into the decreasing order of ωi,t and the
components of higher importance are assumed to be the background.
If we consider
B = argminb(Σbi=1ωi,t > (1 − cf)) (2.7)
for some threshold cf. Then components 1. . .B are assumed to be background and if pixel does not match
one of these components, the pixel is marked as foreground.
This algorithm has been applied on different video sequences and a result is shown on figure 2.
figure 2: Modelling of the background with a mixture of Gaussian
6
-
2.4 Post processing
Mathematical morphology operations In order to improve the detection of the shapes of the object,
or to reduce noise, we use mathematical morphology operations:
• reducing noise is obtained by an opening (which consists in eroding and then dilating the mask),
• improving the shapes of the objects is obtained by a closing (which consists in dilating and then eroding
the mask).
Some results are shown on figure 3 and we can see that:
• the opening has reduced most of the noise of the mask.
• the closing has improved the shapes of the mask.
figure 3: Result of an opening
7
-
figure 4: Result of a closing
Shadows removing Foreground is commonly perturbed by shadow which need to be detected and
removed [3] . The shadows detection is based on the c1c2c3 model:
c1 = arctan(R
max(G,B)) (2.8)
c2 = arctan(G
max(R,B)) (2.9)
c3 = arctan(B
max(R,G)) (2.10)
This model is invariant to illumination moving and so allow to detect pixels of the foreground which are
not part of an object but of its shadow.
It is easy to see on figure 5 that this algorithm works quite well.
8
-
figure 5: Result of shadows removing
Blobs detection Blob detection permits to separate different objects of the foreground. It can be useful
to track, count or analyse behaviors. The blob detection algorithm we employed use connected component
analysis and we can see on figure 6 that it permits to effectively isolate objects thanks to the foreground
segmentation.
9
-
figure 6: Result of blob detection
2.5 Software architecture
The software developed has been written in C++ under Eclipse and the OpenCV and cvBlob libraries. It
is important to note that tools like valgrind and kcachegrind have been used to optimize it. Each algorithm
correspond to a sub-class of the CGeneric class allowing the library to be more generic.
10
-
figure 7: UML modelling of the software
11
-
2.6 Utilization
Both algorithms have been also implemented with grey images. The results are similar but the computing
times are smaller.
The main problem encountered with such algorithms concerns parameters tuning. It is strongly dependent
on the video to be processed. Launching the program with ”-h” displays the different parameters to tune as
shown on figure 8.
figure 8: Console
12
-
For saving computing, one can:
• use the grey one Gaussian algorithm
• reduce the video resolution using the -R option
• do not use the post processing options
• increase the threshold
Computing times for the station1.avi videoParameters: Time:-A 0 -R 4 -m 2 -S 1 -B 1 -T 1.8 44.387s-A 1 -R 4 -m 2 -S 1 -B 1 -T 1.8 56.088s-A 1 -R 4 -m 2 -S 1 -B 1 -T 4.0 30.854s-A 2 -m 0 -S 0 -R 4 -I 500 -T 1.0 -B 1 57.776s-A 2 -m 0 -S 0 -R 4 -I 500 -T 4.0 -B 1 26.222s-A 2 -m 0 -S 0 -B 0 -R 4 -I 500 -T 1.0 19.029s-A 2 -m 0 -S 0 -B 0 -R 4 -I 500 -T 4.0 19.025s-A 2 -m 0 -S 0 -B 0 -R 4 -I 500 -T 5.0 18.765s-A 2 -m 0 -S 0 -B 0 -R 4 -I 500 -T 5.0 -I 1000 18.001s-A 3 -R 4 -m 2 -S 1 -B 1 -cf 0.01 -K 3 -I 500 1m12.673s-A 3 -R 4 -m 0 -S 0 -B 0 -cf 0.01 -K 3 -I 500 1m01.728s-A 3 -R 4 -m 0 -S 0 -B 0 -cf 0.01 -K 5 -I 500 1m20.625s-A 4 -R 4 -m 0 -S 0 -B 0 -cf 0.01 -K 5 -I 500 54.699s-A 4 -R 4 -m 0 -S 0 -B 0 -cf 0.01 -K 3 -I 500 42.871s-A 4 -R 4 -m 0 -S 0 -B 0 -cf 0.05 -K 3 -I 500 44.531s-A 4 -R 4 -m 3 -S 1 -B 1 -cf 0.01 -K 3 -I 500 0m52.027s
13
-
figure 9: Result of a too low threshold
In the case of the figure 9, we can note that a lot of pixels of the background have been detected like
foreground. To improve the result, one can:
• increase the threshold Th (or cf in Mixture of Gaussian Algorithms) to detect less pixels in the fore-
ground;
• add more Gaussian to reduce the noise due to illumination moving;
• apply an opening to reduce the little isolated spots which shouldn’t be part of the foreground.
14
-
3 ZONEMINDER
ZoneMinder is one of the most employed software for video surveillance under Linux. Intended for use in
single or multi-camera video security applications, it supports capture, analysis, recording, and monitoring
of different cameras attached to a Linux system.
My work on ZoneMinder consisted of two steps:
• Installing and learning how to use ZoneMinder;
• inspect the code to integrate my background subtraction algorithms within of it;
• try to add parameters tuning.
3.1 Installation of ZoneMinder
The installation of ZoneMinder is different according the OS you are using. I followed the Ubuntu Installation
recommendations found on the wiki of the official ZoneMinder website.
First, I had to install an Apache server, a mySQL server, and a lot of others that ZoneMinder needs.
Then, I downloaded the SVN version of ZoneMinder (current development version of the software) before
compiling and installing the software.
ZoneMinder runs under a web interface allowing to see videos from the cameras and of the events recorded
as shown on figures 10 & 11.
15
-
figure 10: ZoneMinder main interface
figure 11: Events visualization
16
-
3.2 Integration of the background subtraction algorithms in ZoneMinder
For Probayes, being able to plug easily algorithms in such a software with a great interface could be very
interesting for demonstration purpose.
Though ZoneMinder code is pretty important, it has been rather quick to figure out how the software
was organized. So, I could quickly find where and how to plug my algorithms. Thus, I managed to apply
Background Subtraction to the frames captured by the cameras used by ZoneMinder, and the result was
convincing.
Among other things, I have been able to add different functions to the software like:
• using my Background Subtraction algorithms like detection mode;
• setting the score of alarmed frames to the number of blobs detected by my algorithms;
• allowing to resize the camera captures to accelerate the analysis;
• allowing the grey camera captures;
• allowing the use of inactive zone, and the writing of alarmed image.
figure 12: Background subtraction under ZoneMinder
17
-
4 CONCLUSION
This work experience was great. I acquired experience in several fields and for now on, I’ll have a
significant luggage to begin my professional life. Though very different than the student life, I immediately
fit in the working life, which is captivating, and, I have to acknowledge, absorbing.
More precisely, I have been able to
• improve my knowledge in C++ (polymorphism, architecture, templates);
• discover the integrated development environment of eclipse (how to install it, and use it with C++
plug-in);
• use the OpenCV Library (C++ library for image processing);
• increase my knowledge of Ubuntu (bash, installing new packages);
• using C++ libraries (installing them and use the right compilation options);
• employ valgrind and kcachegrind to optimize algorithms;
• design softwares with bouml;
• use cvs (version-management system);
• use cmake (to automate the creation of Makefiles);
• discover how works a company (and working with other people).
I wish to thank my instructor, Julien ROS, for his availability, his interest in my study and all his help.
And at last, I also thank all the Probayes team for its welcome and sympathy.
18
-
References
[1] Alan M. McIvor. Background subtraction techniques. pages 128–221.
[2] M. Piccardi. Background subtraction techniques: a review. In Proc. of IEEE SMC 2004 International
Conference on Systems, Man and Cybernetics, volume 4, pages 3099–3104, The Hague, The Netherlands,
Oct 2004.
[3] E. Salvador, A. Cavallaro, and T. Ebrahimi. Shadow identification and classification using invariant color
models. In ICASSP ’01: Proceedings of the Acoustics, Speech, and Signal Processing, 2001. on IEEE
International Conference, pages 1545–1548, Washington, DC, USA, 2001. IEEE Computer Society.
[4] Zoran Zivkovic. Improved adaptive gaussian mixture model for background subtraction. Pattern Recog-
nition, International Conference on, 2:28–31, 2004.
19