(--this section does not print--) smileiden.fica ...€¦ · detected corners for non−smiling...

1
The difference between a ‘bad’ photo and a ‘good’ photo is often a matter of whether or not the person in the photo is smiling. With the help of feature recognition and corner detection, we can identify smiles in a photo, and determine whether or not it is good. Introduc4on Objec4ve Method Plots of mouth corner detection points Results Conclusion Using feature recognition and corner detection, we were able to successfully identify smiles that show teeth very accurately. We found closed mouth smiles were harder to detect. With our procedure closed mouth smiles were often categorized as inconclusive. Our system could prove to be a helpful application in digital photography, where it could be used to automatically select the best image in a set of similar images. Our system would be made even more useful if it were extended to work with video. Among possible video applications is marketing analysis of customer reaction. References M. Jones and P. Viola, “Robust real-time object detection,” Workshop on Statistical and Computational Theories …, 2001. S. Jianbo and C. Tomasi, “Good Features to Track,” 1994. M. Castrillón, O. Déniz, C. Guerra, and M. Hernández, “ENCARA2: Real-time detection of multiple faces at different resolutions in video streams,” Journal of Visual Communication and Image Representation, vol. 18, no. 2, pp. 130–140, Apr. 2007. Contact Informa4on Justin DeVito: [email protected] Amanda Meurer: [email protected] Daniel Volz: [email protected] Automatically identify the best photo of a person based on their smile. Average number of mouth corner detections and average concavity. Data collected from 200 subjects. Line of best fit for a smiling face Line of best fit for an unsmiling face Number of Mouth Corner Detec4ons Concavity of the Line of Best Fit Smiling Face 16.3 0.0124 Unsmiling Face 7.7 0.0016 Jus.n DeVito, Daniel Volz, Amanda Meurer Smile Iden.fica.on Via Feature Recogni.on and Corner Detec.on Background 310 320 330 340 350 360 370 380 390 315 310 305 300 295 290 285 280 275 270 265 260 280 300 320 340 360 380 400 400 390 380 370 360 350 340 Based on the concavity and corner density, decide whether the subject is smiling. Find the corner density of the mouth region. Calculate concavity of detected image corners. Detect mouth corners using ShiS Tomasi algorithm. Crop out the boVom third of the face. Find faces using a modified ViolaS Jones algorithm. Input faces Viola-Jones Feature Recognition Algorithm: Scan the image with Haar features. From their response to the image, determine where the face is. Shi-Tomasi Corner Detection Algorithm: I ~ Intensity of the Window R ~ Corner Significance Parameter ~ Eigenvalues of M Example Haar Features (Viola, 2001) Number (200) Total % Procedure Validity Correct Recogni4ons 121 61% 93% False Posi4ves 9 5% 7% Inconclusive 70 35% Statistical Analysis of 200 subjects. Each subject has one smiling and one non-smiling photo. 0 2 4 6 8 10 12 14 16 18 0 2 4 6 8 10 12 14 16 18 20 Detected Corners for Non-smiling Photos Number of Detected Corners Frequency (of 200) -0.015 -0.01 -0.005 0 0.005 0.01 0.015 0.02 0.025 0 0.5 1 1.5 2 2.5 3 3.5 4 Mouth Curvature for Non-smiling Photos Concavity Parameter Value Frequency (of 200) 0 5 10 15 20 25 30 35 40 0 2 4 6 8 10 12 14 16 18 20 Detected Corners for Smiling Photos Number of Detected Corners Frequency (of 200) -0.06 -0.04 -0.02 0 0.06 0 10 20 30 40 50 60 70 80 90 Mouth Curvature for Smiling Photos Concavity Parameter Value Frequency (of 200) Maximum Frequency for Non-Smiling Concavity is 4 Maximum Frequency for Smiling Concavity is 90 M = X x X y w(x, y) I 2 x I x I y I x I y I 2 y R = min(λ 1 , λ 2 ) λ 1 , λ 2

Upload: others

Post on 13-Oct-2020

0 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: (--THIS SECTION DOES NOT PRINT--) SmileIden.fica ...€¦ · Detected Corners for Non−smiling Photos Number of Detected Corners Frequency (of 200) −0.015 −0.01 −0.005 0 0.005

QUICK TIPS (--THIS SECTION DOES NOT PRINT--)

This PowerPoint template requires basic PowerPoint (version 2007 or newer) skills. Below is a list of commonly asked questions specific to this template. If you are using an older version of PowerPoint some template features may not work properly.

Using the template

Verifying the quality of your graphics Go to the VIEW menu and click on ZOOM to set your preferred magnification. This template is at 100% the size of the final poster. All text and graphics will be printed at 100% their size. To see what your poster will look like when printed, set the zoom to 100% and evaluate the quality of all your graphics before you submit your poster for printing. Using the placeholders To add text to this template click inside a placeholder and type in or paste your text. To move a placeholder, click on it once (to select it), place your cursor on its frame and your cursor will change to this symbol: Then, click once and drag it to its new location where you can resize it as needed. Additional placeholders can be found on the left side of this template. Modifying the layout This template has four different column layouts. Right-click your mouse on the background and click on “Layout” to see the layout options. The columns in the provided layouts are fixed and cannot be moved but advanced users can modify any layout by going to VIEW and then SLIDE MASTER. Importing text and graphics from external sources TEXT: Paste or type your text into a pre-existing placeholder or drag in a new placeholder from the left side of the template. Move it anywhere as needed. PHOTOS: Drag in a picture placeholder, size it first, click in it and insert a photo from the menu. TABLES: You can copy and paste a table from an external document onto this poster template. To adjust the way the text fits within the cells of a table that has been pasted, right-click on the table, click FORMAT SHAPE then click on TEXT BOX and change the INTERNAL MARGIN values to 0.25 Modifying the color scheme To change the color scheme of this template go to the “Design” menu and click on “Colors”. You can choose from the provide color combinations or you can create your own.

QUICK DESIGN GUIDE (--THIS SECTION DOES NOT PRINT--)

This PowerPoint 2007 template produces a 36”x48” professional poster. It will save you valuable time placing titles, subtitles, text, and graphics. Use it to create your presentation. Then send it to PosterPresentations.com for premium quality, same day affordable printing. We provide a series of online tutorials that will guide you through the poster design process and answer your poster production questions. View our online tutorials at: http://bit.ly/Poster_creation_help (copy and paste the link into your web browser). For assistance and to order your printed poster call PosterPresentations.com at 1.866.649.3004

Object Placeholders

Use the placeholders provided below to add new elements to your poster: Drag a placeholder onto the poster area, size it, and click it to edit. Section Header placeholder Move this preformatted section header placeholder to the poster area to add another section header. Use section headers to separate topics or concepts within your presentation. Text placeholder Move this preformatted text placeholder to the poster to add a new body of text. Picture placeholder Move this graphic placeholder onto your poster, size it first, and then click it to add a picture to the poster.

RESEARCH POSTER PRESENTATION DESIGN © 2012

www.PosterPresentations.com

©"2012"PosterPresenta.ons.com"""""2117"Fourth"Street","Unit"C"""""Berkeley"CA"94710"""""[email protected]

Student discounts are available on our Facebook page. Go to PosterPresentations.com and click on the FB icon.

The difference between a ‘bad’ photo and a ‘good’ photo is often a matter of whether or not the person in the photo is smiling. With the help of feature recognition and corner detection, we can identify smiles in a photo, and determine whether or not it is good.

Introduc4on0

Objec4ve0

Method0Plots of mouth corner detection points

Results0 Conclusion0

Using feature recognition and corner detection, we were able to successfully identify smiles that show teeth very accurately. We found closed mouth smiles were harder to detect. With our procedure closed mouth smiles were often categorized as inconclusive. Our system could prove to be a helpful application in digital photography, where it could be used to automatically select the best image in a set of similar images. Our system would be made even more useful if it were extended to work with video. Among possible video applications is marketing analysis of customer reaction.

References0 M. Jones and P. Viola, “Robust real-time object detection,” Workshop on Statistical and Computational Theories …, 2001.

S. Jianbo and C. Tomasi, “Good Features to Track,” 1994.

M. Castrillón, O. Déniz, C. Guerra, and M. Hernández, “ENCARA2: Real-time detection of multiple faces at different resolutions in video streams,” Journal of Visual Communication and Image Representation, vol. 18, no. 2, pp. 130–140, Apr. 2007.

Contact0Informa4on0Justin DeVito: [email protected] Amanda Meurer: [email protected] Daniel Volz: [email protected]

Automatically identify the best photo of a person based on their smile. Average number of mouth corner detections

and average concavity. Data collected from 200 subjects.

Line of best fit for a smiling face

Line of best fit for an unsmiling face

Number0of0Mouth0Corner0Detec4ons0

Concavity0of0the0Line0of0Best0Fit0

Smiling0Face0 16.3" 0.0124"

Unsmiling0Face0 7.7" 0.0016"

Jus.n"DeVito,"Daniel"Volz,"Amanda"Meurer"

Smile"Iden.fica.on"Via"Feature"Recogni.on"and"Corner"Detec.on"

Background0

310 320 330 340 350 360 370 380 390−315

−310

−305

−300

−295

−290

−285

−280

−275

−270

−265Detection of Curvature

260 280 300 320 340 360 380 400−400

−390

−380

−370

−360

−350

−340Detection of Curvature

Based"on"the"concavity"and"corner"density,"decide"whether"the"subject"is"smiling."

Find"the"corner"density"of"the"mouth"region."

Calculate"concavity"of"detected"image"corners."

Detect"mouth"corners"using"ShiSTomasi"algorithm."

Crop"out"the"boVom"third"of"the"face."

Find"faces"using"a"modified"ViolaSJones"algorithm."

Input"faces"

Viola-Jones Feature Recognition Algorithm: Scan the image with Haar features. From their response to the image, determine where the face is. Shi-Tomasi Corner Detection Algorithm:

I ~ Intensity of the Window R ~ Corner Significance Parameter ~ Eigenvalues of M

Example Haar Features (Viola, 2001)

Number00(200)" Total0%"

Procedure0Validity"

Correct0Recogni4ons" 121" 61%" 93%"

False0Posi4ves" 9" 5%" 7%"

Inconclusive" 70" 35%"

Statistical Analysis of 200 subjects. Each subject has one smiling and one non-smiling photo.

0 2 4 6 8 10 12 14 16 180

2

4

6

8

10

12

14

16

18

20

Detected Corners for Non!smiling Photos

Number of Detected Corners

Fre

qu

en

cy (

of

20

0)

!0.015 !0.01 !0.005 0 0.005 0.01 0.015 0.02 0.0250

0.5

1

1.5

2

2.5

3

3.5

4

Mouth Curvature for Non!smiling Photos

Concavity Parameter Value

Fre

quency

(of 200)

0 5 10 15 20 25 30 35 400

2

4

6

8

10

12

14

16

18

20

Detected Corners for Smiling Photos

Number of Detected Corners

Fre

qu

en

cy (

of

20

0)

!0.06 !0.04 !0.02 0 0.02 0.04 0.060

10

20

30

40

50

60

70

80

90

Mouth Curvature for Smiling Photos

Concavity Parameter Value

Fre

qu

en

cy (

of

20

0)

Maximum Frequency for Non-Smiling Concavity is 4 Maximum Frequency for Smiling Concavity is 90

Smile Recognition using Feature and Corner Detection AlgorithmsPurpose:We want to automatically detect a smiling subject in a picture. Our intended

application is for digital photography, where this algorithm can be applied toautomatically select the best image in a set of similar images. This would reducethe time a photographer would need to spend looking through images from aphoto shoot by automatically discarding images where the subject isn’t smiling.

First we detect the face using a Viola-Jones based algorithm. This algorithmis outlined by Lienhart, Kuranov, and Pisarevsky (Ref. XXXX)

asdfShi-Tomasi corner detection is based upon Harris-Stephens corner detection,

just with different threshold parameters.The Harris corner detector operator is defined as:

E(u, v) =X

x

X

y

w(x, y) [I (x+ u, y + v)� I (x, y)]2

• E - Sum of squared differences between the original and moved window

• u - x direction window displacement

• v - y direction window displacement

• w(x, y) - window at position (x,y)

• I(x+ u, y + v) - intensity of the moved window

• I(x, y) - intensity of the original window

We wish to maximize the E term. In order to do this, we use a first order Taylorseries approximation of I:

E(u, v) ⇡X

x

X

y

w(x, y) [I (x, y) + uI

x

+ vI

y

� I (x, y)]2

E(u, v) ⇡X

x

X

y

w(x, y)�u

2I

2x

+ 2uvIx

I

y

+ v

2I

2y

Then changing to a matrix representation gives:

E(u, v) ⇡⇥x y

⇤X

x

X

y

w(x, y)

I

2x

I

x

I

y

I

x

I

y

I

2y

� x

y

M is the structure tensor from above.

M =X

x

X

y

w(x, y)

I

2x

I

x

I

y

I

x

I

y

I

2y

The determination of R, which is the corner importance measurement isdone by taking the minimum of the two eigenvalues of this matrix.

1

R = min(�1,�2)

where �1,�2 are eigenvalues of M.

This is the Shi-Tomasi modification of the Harris and Stephens corner de-tection algorithm. (Good Features to Track)

2

R = min(�1,�2)

where �1,�2 are eigenvalues of M.

This is the Shi-Tomasi modification of the Harris and Stephens corner de-tection algorithm. (Good Features to Track)

2