REAL TIME FACE DETECTION

Justin RillingPooja MhapsekarMoinuddin Sayed

Ogom J Obinor

2

IntroductionIntroduction

• The idea is to be able to detect faces appearing in an image.

• The faces can be of different sizes and orientations.

• The motivation behind implementing this on an FPGA is that image processing is inherently parallel in nature and lends itself well to an FPGA.

3

Viola and Jones’ DetectorViola and Jones’ Detector

Key Contributions

1. Integral Image – fast computation of features used by the detector.

2. Combines complex classifiers in “cascade” – focus on facial features.

4

Integral ImageIntegral Image

• Major contribution of the Viola and Jones detector.

• Summation of pixel values of the original image.

• Value at location (x,y) = sum of values of pixels above and to the left of (x,y).

• Whole image converted to integral image and a window buffer used to scan the entire image.

5

Haar ClassifiersHaar Classifiers

• Classifiers are composed of 2-3 rectangles.

• Weights and sizes associated with features – obtained through AdaBoost.

• Classifier sum = ∑ (area * weights)

• Integral image makes area calculation easy. Area = L4–L3–L2+L1.

• Several classifiers compose a stage.

6

StageStage

• Sum of all Haar feature classifiers compared with the stage threshold.

• Multiple stages, each stage has different number of classifiers.

• Threshold obtained from AdaBoost algorithm.

Face Detection Example

0.0Classifier Sum

< 0.0Classifier Threshold

0.0Right Value

0.0Left Value0.0

Stage Sum

+0.0Stage Threshold

<1

0

0Stage Number

0Classifier Number

Face Detection Example

426Classifier Sum

< 542Classifier Threshold

0.8378106Right Value

0.0337941Left Value0.0

Stage Sum

+0.8226894Stage Threshold

<1

0

0Stage Number

0Classifier Number

Face Detection Example

426Classifier Sum

< 542Classifier Threshold

0.8378106Right Value

0.0337941Left Value0.0337941

Stage Sum

+0.8226894Stage Threshold

<1

0

0Stage Number

0Classifier Number

Face Detection Example

887Classifier Sum

< 2045Classifier Threshold

0.7488812Right Value

0.1514132Left Value0.0337941

Stage Sum

+0.8226894Stage Threshold

<1

0

0Stage Number

1Classifier Number

Face Detection Example

887Classifier Sum

< 2045Classifier Threshold

0.7488812Right Value

0.1514132Left Value0.1852073

Stage Sum

+0.8226894Stage Threshold

<1

0

0Stage Number

1Classifier Number

Face Detection Example

349Classifier Sum

< 568Classifier Threshold

0.6374819Right Value

0.0900493Left Value0.1852073

Stage Sum

+0.8226894Stage Threshold

<1

0

0Stage Number

2Classifier Number

Face Detection Example

349Classifier Sum

< 568Classifier Threshold

0.6374819Right Value

0.0900493Left Value0.2752566

Stage Sum

+0.8226894Stage Threshold

<1

0

0Stage Number

2Classifier Number

Face Detection Example

349Classifier Sum

< 568Classifier Threshold

0.6374819Right Value

0.0900493Left Value0.2752566

Stage Sum

+0.8226894Stage Threshold

<1

0

0Stage Number

2Classifier Number

Window Buffer Failed Stage 0

Face Detection Example

0.0Classifier Sum

< 0.0Classifier Threshold

0.0Right Value

0.0Left Value0.0

Stage Sum

+0.0Stage Threshold

<1

0

0Stage Number

0Classifier Number

Face Detection Example

0.0Classifier Sum

< 0.0Classifier Threshold

0.0Right Value

0.0Left Value0.0

Stage Sum

+0.0Stage Threshold

<1

0

0Stage Number

0Classifier Number

Face Detection Example

0.0Classifier Sum

< 0.0Classifier Threshold

0.0Right Value

0.0Left Value0.0

Stage Sum

+0.0Stage Threshold

<1

0

0Stage Number

0Classifier Number

Face Detection Example

0.0Classifier Sum

< 0.0Classifier Threshold

0.0Right Value

0.0Left Value0.0

Stage Sum

+0.0Stage Threshold

<1

0

0Stage Number

0Classifier Number

Face Detection Example

0.0Classifier Sum

< 0.0Classifier Threshold

0.0Right Value

0.0Left Value0.0

Stage Sum

+0.0Stage Threshold

<1

0

0Stage Number

0Classifier Number

Face Detection Example

0.0Classifier Sum

< 0.0Classifier Threshold

0.0Right Value

0.0Left Value0.0

Stage Sum

+0.0Stage Threshold

<1

0

0Stage Number

0Classifier Number

Face Detection Example

0.0Classifier Sum

< 0.0Classifier Threshold

0.0Right Value

0.0Left Value0.0

Stage Sum

+0.0Stage Threshold

<1

0

0Stage Number

0Classifier Number

22

Top Level DesignTop Level Design

YES

Get ScaledImage

Get Integral Image

Window Buffer (21x21)

Face Detection PipelineEnd of

Image?

NO

Monitor

Draw RectangleIf face is det.

Get Image

Face Detection Pipeline

Original End Product

USB Webcam FPGA Display

USB DVI

Updated End Product

Digital Interface

1/3 Color Camera Mod C3188A-6018• Supports several standard image data

formats including YCrCb 4:2:2 • 640 x 480 resolution• 5 V signaling • 30 fps

FPGA Display DVI

3.3 VSignaling

Updated End Product

XC95144XLCPLD

Display

5 VSignaling

DVI1/3 Color Camera Mod C3188A-6018• Supports several standard image data

formats including YCrCb 4:2:2 • 640 x 480 resolution• 5 V signaling • 30 fps

FPGA

DVI Controller

Questions ???

References:

1. P. Viola and M. Jones, “Robust real-time object detection,” International Journal of Computer Vision, 57(2), 137-154, 2004.

2. Junguk Cho, Shahnam Mirzaei, Jason Oberg, Ryan Kastner, “FPGA-Based Face Detection System Using Haar Classifiers,” Proceeding of the ACM/SIGDA International Symposium on Field Programmable Gate Arrays, 103-112, 2009.

OpenCV Program – Lena.jpg

OpenCV Program – ER.jpg

OpenCV Program – ER.jpgWindow Buffer (2, 152, 122)Window Buffer (2, 152, 122)

Face Detection Pipeline

OpenCV Program – ER.jpgWindow Buffer (3, 165, 20)Window Buffer (3, 165, 20)

Face Detection Pipeline

Questions ???