aau summerschool - department of electronic...

7/28/15

1

AAU SUMMER SCHOOL

PROGRAMM ING SOC IAL ROBOTS FOR HUMAN INTERACTION

LECT URE 8 I MAG E PRO CESS I NG I I

1 . I n t ro du c t i o n t o Robo t Ope ra t i n g S y s t em (ROS )

2 . I n t ro du c t i o n t o i S o c i oB o t a nd NAO ro bo t , a n d d emos

3 . S o c i a l Robo t s a nd A pp l i c a t io n s

4 . Mac h i n e L ea rn i n g a nd P a t t e rn Re c ogn i t i o n

5 . S pee c h P ro c e s s i n g I : A c qu i si t io n o f S pee c h , Fea t u re E x t ra c t i o n a nd S pea k e r L o c a l i z a t i o n

6 . S pee c h P ro c e s s i n g I I : S pea k e r I d e n t i f i c a ti o n a nd S pee c h Rec ogn i ti o n

7 . Image P ro c e s s i n g I : Image A c qu i s i t io n , P re -p ro c e s s in g a nd Fea t u re E x t ra c t i o n

8 . Image P r ocess ing I I : Face De tec ti on and Face Recogni ti on

9 . Us e r Mode l l i n g

10 . Mu l t i moda l Human -Robo t I n t e ra c t i o n

COURSE OUTLINE

28. 07. 2015 AALBO RG UNI VERS I T Y 2

7/28/15

2

• To tel l whether faces ex i s t i n a image and f i nd the l ocat i on and s i ze of face i n the image.

• The detec ted face i s usual l y sur r ounded by a r ec tangl e.

• How to r eal i ze thi s?• C l ass i f i er• Sear chi ng method

FACE DETECTION


• C l ass i f i er

FACE DETECTION


Positive data

Negative data

Preprocessing Feature extraction Learning

Classifier

7/28/15

3

• T r ai ni ng data

FACE DETECTION


Negative data

Positive data

Images are fromPrince, Simon JD.”Computer v is ion: models, learning, and inference”

• Featur es• Sk i n col or• Rec tangl e featur es

• Lear ni ng method• SVM• Adaboos t

• Sear chi ng method• Res i ze the image to di f fer ent scal es• Sl i de a f i xed s i ze w i ndow acr oss al l the r es i zed image and eval uate the w i ndow at each l ocat i on us i ng the l ear ned c l ass i f i er

• Combi ne the over l apped pos i t i ve w i ndows

FACE DETECTION


7/28/15

4

• To f i nd the l ocat i on and s i ze of face i n the image.

• The detec ted face i s usual l y sur r ounded by a r ec tangl e.

FACE DETECTION

28. 07. 2015 AALBO RG UNI VERS I T Y 7Images are fromPrince, Simon JD.”Computer v is ion: models, learning, and inference”

• VIOLA, PAUL, AND M ICHAEL J . JONES. "ROBUST REAL- T IME FACE DETECT ION .” INTERNAT IONAL JOURNAL OF COMPUTER VISION 57.2 ( 2004) : 137- 154.

• VIOLA, PAUL, AND M ICHAEL JONES. "RAPID OBJECT DETECT ION USING A BOOSTED CASCADE OF SIMPLE FEATURES.” COMPUTER VISION AND PATTERN RECOGN IT ION , 2001. CVPR 2001. PROCEED INGS OF THE 2001 IEEE COMPUTER SOC IETY CONFERENCE ON . VOL. 1. IEEE, 2001.

• The face detec t i on method i n OpenCV i s fr om these two paper s .

REFERENCE FOR FACE DETECTION


7/28/15

5

• face_cascade = cv2.CascadeC l ass i f i er ( ' haar cascade_fr ont al fac e_ def aul t .xm l ' )

• face_cascade.detec tM ul t i Scal e( gr a y, 1.3, 5)

FACE DETECTION IN OPENCV


• Face Ident i f i cat i on• Gi ven a face image, i dent i fy who the per son i s based on the database of enr ol l ed user

• Face Ver i f i cat i on• Gi ven a pai r of face images , ver i fy whether they bel ong to the same per son

FACE RECOGNITION


7/28/15

6

• Chal l enges• Lar ge var i abi l i ty i n fac i al appear ance• H i ghl y compl ex nonl i near mani fol ds• H i gh dimens i onal i ty and smal l sampl e s i ze

FACE RECOGNITION


• Str uc tur e of face r ecogni t i on sys tem

FACE RECOGNITION


Face Detection

Face Alignment

Feature Extraction Classification

Database of enrolled users

7/28/15

7

• Featur e al i gnment• The detec ted face i s normal i zed w i th r espec t to geometr i cal pr oper t i es based on l ocated fac i al components such as nose, eyes , mouth…

FACE RECOGNITION


• Featur e ex tr ac t i on• Local bi nar y patter n ( LBP)• H i s togr am of gr adi ent ( HOG)• Scal e- i nvar i ant featur e tr ans form (SIFT )• Lear ned featur e us i ng deep l ear ni ng

FACE RECOGNITION


7/28/15

8

• C l ass i f i cat i on and database of enr ol l ed user s• Memor y - based l ear ni ng ( KNN )

• The database of enr ol l ed user s w i l l be the ex tr ac ted featur es of face images

• Model - based l ear ni ng ( SVM )• The database of enr ol l ed user s w i l l be the model l ear ned by SVM us i ng the tr ai ni ng data

FACE RECOGNITION


• JAIN , AN IL K., AND STAN Z . LI . HANDBOOK OF FACE RECOGN IT ION . VOL. 1. NEW YORK: SPR INGER , 2005.

• Face r ecogni t i on paper s fr om TPAM I, IJCV, CVPR , ICCV, ECCV, …

REFERENCE FOR FACE RECOGNITION


7/28/15

9

• Face detec t i on i n Py thon• http: / /docs .opencv .or g/m as ter / d7/ d8 b/t utor i al _ py_f ace _d etec t i on.htm l

• Face r ecogni t i on package i n Py thon• https :/ /gi thub.com /by t ef i sh/ facer ec

• Onl i ne tool s• Betaface: ht tp: / /betaface.c om /• Face++ : ht tp: / /www .facepl uspl us .com /

• Database• LFW : http: / /v i s - www .cs .umass .edu/ l fw /• YOUTUBE Faces : ht tp: / /www .cs .tau.ac . i l /~wol f /y t faces / i n dex . htm l

RESOURCE


• Resul ts on LFW

CURRENT PERFORMANCE


Methods Training data û ± SEBestImage (Tencent) ca. 1M 0.9965 ± 0.0025FaceNet (Google) between 100M-200M 0.9963 ± 0.0009DeepID3 (CUHK) ca. 300 thousand 0.9953 ± 0.0010DeepFace(Facebook) ca. 4M 0.9735 ± 0.0025

Mean classification accuracy û and standard error of the mean SE.All the results are from http://vis-www.cs.umass.edu/lfw/index.html

7/28/15

10

• Demo for s impl e face r ecogni t i on us i ng Py thon

FACE RECOGNITION


1 . I n t ro du c t i o n t o Robo t Ope ra t i n g S y s t em (ROS )

2 . I n t ro du c t i o n t o i S o c i oB o t a nd NAO ro bo t , a n d d emos

3 . S o c i a l Robo t s a nd A pp l i c a t io n s

4 . Mac h i n e L ea rn i n g a nd P a t t e rn Re c ogn i t i o n

5 . S pee c h P ro c e s s i n g I : A c qu i si t io n o f S pee c h , Fea t u re E x t ra c t i o n a nd S pea k e r L o c a l i z a t i o n

6 . S pee c h P ro c e s s i n g I I : S pea k e r I d e n t i f i c a ti o n a nd S pee c h Rec ogn i ti o n

7 . Image P ro c e s s i n g I : Image A c qu i s i t io n , P re -p ro c e s s in g a nd Fea t u re E x t ra c t i o n

8 . Image P ro c e s s i n g I I : Fa c e De t e c t i o n a nd Fa c e Rec ogn i t i o n

9 . Use r Mode l l i ng

10 . Mu l t i moda l Human -Robo t I n t e ra c t i o n

COURSE OUTLINE


aau summerschool - department of electronic...

Documents