computer vision technology and expertise

Rhonda Software Computer Vision (CV) Experience

Video analytics methods and algorithms dedicated for continuous image processing

in fully-automated solutions

Computer Vision - Area of Expertise

Computer Vision is relatively new but most rapidly growing domain of Rhonda's expertise. Since 2007 Rhonda has been doing research and development in this area.

As the mainstream of CV R&D, Rhonda has released two Audience Measurement products dedicated for accurate people counting (myAudience-Count) and precise audience demographics studying (myAudience-Measure).

Rhonda Software also offers CV custom-oriented solutions in many other domains like:

• Object/pattern recognition

• Face detection

• Face landmarks automatic mark-up

• Human body detection & tracking

• Face recognition.

2© 2013 Rhonda Software: www.rhondasoftware.com

http://www.rhondasoftware.com/

Beholder Computer Vision Framework

• Cross-platform solution, active development under Linux and Windows

• Continuous integration with overnight builds and automatic testing

• Modular distributed architecture on top of ZeroMQ messaging platform and Google protocol buffers data serialization provides flexible allocation of system functions across system processing nodes (e.g. offloading CPU-consuming key-point extraction module from TI OMAP 3530 to Intel Atom-based notebook while keeping other modules on the same nodes was a matter of couple hours including re-building and testing)

• Designed to work on low performance platform

• Unit test available for all algorithms

• Process real-time and prerecorded video


Platform for Developing Applications in CV Domain

http://www.zeromq.org/

http://code.google.com/p/protobuf/


Beholder Computer Vision Framework

• Analysis stages can be transferred between different modules based on unique

solution requirements and hardware restraints.

• Combination of existing algorithms gives advantage to achieving result


Analysis Stages & Algorithms

Background model

Object tracking

Face detection

Feature Extraction

Face Recognition

Statistics and RDB

Media presentation

Background

Gaussian

Mean Shift

KDE

Object tracking

Color/Intensity based histograms

Mean shift/Cam shift

Key points/feature based tracking

Multiview tracking (Homography and

cooperative tracking)

Motion detection

Optical flow (Lucas–Kanade);

PTZ camera

Face detection

Viola and Jones

Face tracker

Feature Extraction

Statistical models: PCA/ICA

GMM

Morphological analysis

TVL1

Pattern maps

Laplacianfaces

Object recognition

Pattern-based recognition (Viola

and Jones)

Feature-based recognition

(SURF/SIFT)

Generalized Hough Transform, Hough

Transform

Contour-based detection

Barcode detection /recognition

Image Enhancement

Super resolution

Deconvolution

Capture using PTZ camera

Face Recognition/classification

Hidden Markov Models

Neural-networks

“3D” models

ASM/AAM

AdaBoost/AdaSVM

Gabor wavelets

Media presentation

IP stream

WEB

Linux/QT applications

Windows applications

Windows Media Center


Computer Vision Technologies

• Background subtraction – subtracting objects from complex/moving background as part of object tracking/recognition

• Overlapped object tracking – object tracking of overlapping objects• Multi-view tracking – tracking objects between different cameras• Object recognition – recognition of objects on complex background• Barcode detection and recognition – UPC-A Barcode recognition using video

stream from regular camera


CV Methods in Our Area of Expertise


Computer Vision Technologies

• Facial features extraction – locating facial features from video stream or still images

• Face detection, tracking and recognition – including attention recognition

• Demographic classification – age and gender recognition from video stream or still images

• People/object counting – using top side view camera or horizontally positioned camera

• Computer cluster for training algorithms – using dedicated cluster and MPI based applications to train and test algorithms


CV Methods in Our Area of Expertise (continue)


Computer Vision Projects

• Semiconductor Insight – electronic microscope image enhancement and de-blurring

• En-Vision America vision impaired helper, based on Freescale I.MX31 ARM11 board:• Barcode detection and barcode

recognition• Bank note recognition• Logo recognition• Face detection

• Traffic monitoring system for major US photo enforcement company – traffic lights phase detection under various illumination and weather conditions

• Rhonda audience measurement cross-platform solutions with support of attention detection, gender / age recognition: myAudience-Measure and myAudience-Count


Examples of Projects in CV Domain


Computer Vision Solutions

myAudience-Measure is an automated anonymous audience measurement product. It can be used both with Digital Signage vehicles and all kinds of static installations. This Computer Vision solution gathers statistical information about vehicle audience – i.e. it counts visitors in the camera’s field of view, recognizes gender, age group and attention of viewers.

myAudience-Count is a bi-directional people traffic counter SW product that utilizes top-mounted camera (USB and IP cams supported). myAudience-Count shares operational architecture, back-end server and software updates infrastructure with myAudience-Measure using same OS, data storage and online Reports Portal.


myAudience – Audience Measurement product details


http://www.myaudience.com/

Computer Vision Embedded

One of the newest directions in research and development activities conducted by Rhonda Software concerns FPGA embedded platform and its potential in acceleration of computational intensive algorithms of Computer Vision technology.

It was decided to start from porting the PC-based myAudience-Count SW product on Lattice HDR-60 board powered by ECP3 FPGA chip.

100% compatible solution was redesigned to fit on a single chip. Ported components include: Video capturing HDR-enabled module, HW-accelerated Video analytics pipeline, embedded Linux on new kernel, Web-server optimized for AJAX-requests and Ethernet controller implemented in programmable logic.


myAudience-Count ported to FPGA-based platform


Computer Vision Embedded

Most of Computer Vision algorithms are computationally intensive. Reduced CPU performance on compact and embedded systems brings developers to use DSP or GPU if available.

DSP and GPU implementations may have specific limitations but save a lot of CPU power at the expense of additional hardware and significant code redesign.

• Barcode Library on TI OMAP 3530 (Beagle Board), Angstrom Linux uses on-board DSP via DSP Bridge driver to preprocess source barcode pictures

• Viola and Jones Face Detector from OpenCV library was ported to nVidia CUDA GPGPU. Now it can run on GPU in parallel with other algorithms running on CPU

• Gaussian Background Subtraction algorithm was also ported to CUDA


DSP and GPGPU Development


Effective Computer Vision

Rhonda engineers optimize computationally intensive algorithms for speed:

• For the open source implementation of Viola and Jones Face Detector from

OpenCV it took 10 seconds to handle a 640x480 frame on I.MX31 ARM11. After the

optimization it takes 2 seconds for the same amount of input data, i.e. now it works 5

times faster

• The same code on Intel P4 handled 2.72 frames per second before the optimization.

The optimized code makes 15 FPS, i.e. 5.5 times faster

http://www.computer-vision-software.com/blog/2009/06/fastfurious-face-detection-with-opencv/

• It took 4 days for Haar Training algorithm to create a cascade for the Viola and Jones

classifier on Intel Core 2 Duo. Parallel version of the algorithm developed in Rhonda

to work on a cluster of 11 computers do the same job 6.8 times faster (21 hour)

http://www.computer-vision-software.com/blog/2009/06/parallel-world-of-opencv/

• For the open source implementation of SURF algorithm from OpenCV applied to bank

note recognition it took 4-12 seconds to handle a 640x480 frame on Intel P4. After the

optimization it takes 250-350 ms for the same amount of input data, i.e. now it works

16-34 times faster 11© 2013 Rhonda Software: www.rhondasoftware.com

CV Algorithms Optimization



http://www.computer-vision-software.com/blog/2009/06/parallel-world-of-opencv/


Andrey MischenkoCEO, Rhonda Software

Mobile: +7(423)257-1008

E-mail: [email protected]

Denis PomogaevVP of Technology Innovations

Mobile: +1(224)715-1154E-mail: [email protected]


Thank You!

mailto:[email protected]

mailto:[email protected]
