aau summerschool - department of electronic...

7/28/15

1

AAU SUMMER  SCHOOL

PROGRAMM ING   SOC IAL   ROBOTS   FOR  HUMAN  INTERACTION

LECT URE 8   I MAG E  PRO CESS I NG  I I

1 .   I n t ro du c t i o n  t o  Robo t  Ope ra t i n g  S y s t em  (ROS )

2 .   I n t ro du c t i o n  t o  i S o c i oB o t a nd  NAO  ro bo t ,   a n d  d emos

3 .  S o c i a l  Robo t s  a nd  A pp l i c a t io n s

4 .  Mac h i n e  L ea rn i n g  a nd  P a t t e rn  Re c ogn i t i o n

5 .  S pee c h  P ro c e s s i n g  I :  A c qu i si t io n  o f  S pee c h ,  Fea t u re  E x t ra c t i o n  a nd  S pea k e r  L o c a l i z a t i o n

6 .  S pee c h  P ro c e s s i n g  I I :  S pea k e r  I d e n t i f i c a ti o n  a nd  S pee c h  Rec ogn i ti o n

7 .   Image  P ro c e s s i n g  I :  Image  A c qu i s i t io n ,  P re -­p ro c e s s in g  a nd  Fea t u re  E x t ra c t i o n

8 .   Image  P r ocess ing I I :   Face  De tec ti on   and  Face  Recogni ti on

9 .  Us e r  Mode l l i n g

10 .  Mu l t i moda l  Human -­Robo t  I n t e ra c t i o n

COURSE OUTLINE

28. 07. 2015 AALBO RG   UNI VERS I T Y 2

7/28/15

2

• To   tel l  whether   faces   ex i s t   i n  a   image   and   f i nd   the   l ocat i on   and  s i ze  of   face   i n   the   image.

• The   detec ted   face   i s  usual l y   sur r ounded   by   a  r ec tangl e.

• How  to   r eal i ze   thi s?• C l ass i f i er• Sear chi ng   method

FACE   DETECTION

28. 07. 2015 AALBO RG   UNI VERS I T Y 3

• C l ass i f i er

FACE   DETECTION

28. 07. 2015 AALBO RG   UNI VERS I T Y 4

Positive  data

Negative  data

Preprocessing Feature  extraction Learning

Classifier

7/28/15

3

• T r ai ni ng   data

FACE   DETECTION

28. 07. 2015 AALBO RG   UNI VERS I T Y 5

Negative  data

Positive  data

Images  are  fromPrince,  Simon  JD.”Computer  v is ion:  models,  learning,  and  inference”

• Featur es• Sk i n   col or• Rec tangl e   featur es

• Lear ni ng   method• SVM• Adaboos t

• Sear chi ng   method• Res i ze   the   image   to   di f fer ent   scal es• Sl i de   a   f i xed   s i ze  w i ndow   acr oss   al l   the   r es i zed   image   and  eval uate   the   w i ndow   at   each   l ocat i on   us i ng   the   l ear ned  c l ass i f i er

• Combi ne   the   over l apped   pos i t i ve   w i ndows

FACE   DETECTION

28. 07. 2015 AALBO RG   UNI VERS I T Y 6

7/28/15

4

• To   f i nd   the   l ocat i on   and   s i ze   of   face   i n   the   image.

• The   detec ted   face   i s  usual l y   sur r ounded   by   a  r ec tangl e.

FACE   DETECTION

28. 07. 2015 AALBO RG   UNI VERS I T Y 7Images  are  fromPrince,  Simon  JD.”Computer  v is ion:  models,  learning,  and  inference”

• VIOLA,   PAUL,   AND  M ICHAEL   J .  JONES.   "ROBUST   REAL-­ T IME  FACE  DETECT ION .”   INTERNAT IONAL   JOURNAL  OF  COMPUTER  VISION 57.2   ( 2004) :   137-­ 154.

• VIOLA,   PAUL,   AND  M ICHAEL   JONES.   "RAPID  OBJECT   DETECT ION  USING  A  BOOSTED   CASCADE  OF   SIMPLE   FEATURES.”  COMPUTER  VISION   AND  PATTERN   RECOGN IT ION ,   2001.   CVPR  2001.   PROCEED INGS   OF  THE  2001   IEEE   COMPUTER  SOC IETY  CONFERENCE  ON .   VOL.   1.   IEEE,   2001.

• The   face   detec t i on   method   i n  OpenCV i s  fr om   these   two   paper s .

REFERENCE  FOR  FACE  DETECTION

28. 07. 2015 AALBO RG   UNI VERS I T Y 8

7/28/15

5

• face_cascade =  cv2.CascadeC l ass i f i er ( ' haar cascade_fr ont al fac e_ def aul t .xm l ' )

• face_cascade.detec tM ul t i Scal e( gr a y,   1.3,   5)

FACE   DETECTION  IN  OPENCV

28. 07. 2015 AALBO RG   UNI VERS I T Y 9

• Face   Ident i f i cat i on• Gi ven   a   face   image,   i dent i fy   who   the   per son   i s  based   on   the  database   of   enr ol l ed   user

• Face   Ver i f i cat i on• Gi ven   a   pai r  of   face   images ,   ver i fy  whether   they   bel ong   to   the  same   per son

FACE   RECOGNITION

28. 07. 2015 AALBO RG   UNI VERS I T Y 10

7/28/15

6

• Chal l enges• Lar ge   var i abi l i ty   i n   fac i al   appear ance• H i ghl y  compl ex   nonl i near   mani fol ds• H i gh   dimens i onal i ty   and   smal l  sampl e   s i ze

FACE   RECOGNITION

28. 07. 2015 AALBO RG   UNI VERS I T Y 11

• Str uc tur e   of   face   r ecogni t i on   sys tem

FACE   RECOGNITION

28. 07. 2015 AALBO RG   UNI VERS I T Y 12

Face  Detection

Face  Alignment

Feature  Extraction Classification

Database  of  enrolled  users

7/28/15

7

• Featur e   al i gnment• The   detec ted   face   i s  normal i zed   w i th  r espec t   to   geometr i cal  pr oper t i es   based   on   l ocated   fac i al   components   such   as  nose,  eyes ,   mouth…

FACE   RECOGNITION

28. 07. 2015 AALBO RG   UNI VERS I T Y 13

• Featur e   ex tr ac t i on• Local   bi nar y   patter n   ( LBP)• H i s togr am   of   gr adi ent   ( HOG)• Scal e-­ i nvar i ant   featur e   tr ans form (SIFT )• Lear ned   featur e   us i ng   deep   l ear ni ng

FACE   RECOGNITION

28. 07. 2015 AALBO RG   UNI VERS I T Y 14

7/28/15

8

• C l ass i f i cat i on   and   database   of   enr ol l ed   user s• Memor y -­ based   l ear ni ng   ( KNN )

• The   database   of   enr ol l ed   user s   w i l l  be   the   ex tr ac ted  featur es   of   face   images

• Model -­ based   l ear ni ng   ( SVM )• The   database   of   enr ol l ed   user s   w i l l  be   the   model   l ear ned  by   SVM  us i ng   the   tr ai ni ng   data

FACE   RECOGNITION

28. 07. 2015 AALBO RG   UNI VERS I T Y 15

• JAIN ,  AN IL   K.,   AND  STAN   Z .  LI . HANDBOOK   OF  FACE  RECOGN IT ION .   VOL.   1.   NEW  YORK:   SPR INGER ,   2005.

• Face   r ecogni t i on   paper s   fr om  TPAM I,   IJCV,   CVPR ,  ICCV,   ECCV,   …

REFERENCE  FOR  FACE  RECOGNITION

28. 07. 2015 AALBO RG   UNI VERS I T Y 16

7/28/15

9

• Face   detec t i on   i n  Py thon• http: / /docs .opencv .or g/m as ter / d7/ d8 b/t utor i al _ py_f ace _d etec t i on.htm l

• Face   r ecogni t i on   package   i n  Py thon• https :/ /gi thub.com /by t ef i sh/ facer ec

• Onl i ne   tool s• Betaface:   ht tp: / /betaface.c om /• Face++ :   ht tp: / /www .facepl uspl us .com /

• Database• LFW :  http: / /v i s -­ www .cs .umass .edu/ l fw /• YOUTUBE   Faces :  ht tp: / /www .cs .tau.ac . i l /~wol f /y t faces / i n dex . htm l

RESOURCE

28. 07. 2015 AALBO RG   UNI VERS I T Y 17

• Resul ts   on   LFW

CURRENT   PERFORMANCE

28. 07. 2015 AALBO RG   UNI VERS I T Y 18

Methods Training data û ± SEBestImage (Tencent) ca.  1M 0.9965  ± 0.0025FaceNet (Google) between  100M-­200M 0.9963  ± 0.0009DeepID3  (CUHK) ca.  300  thousand 0.9953  ± 0.0010DeepFace(Facebook) ca. 4M 0.9735  ± 0.0025

Mean  classification  accuracy  û and  standard  error  of  the  mean  SE.All  the  results  are  from  http://vis-­www.cs.umass.edu/lfw/index.html

7/28/15

10

• Demo   for  s impl e   face   r ecogni t i on   us i ng   Py thon

FACE   RECOGNITION

28. 07. 2015 AALBO RG   UNI VERS I T Y 19

1 .   I n t ro du c t i o n  t o  Robo t  Ope ra t i n g  S y s t em  (ROS )

2 .   I n t ro du c t i o n  t o  i S o c i oB o t a nd  NAO  ro bo t ,   a n d  d emos

3 .  S o c i a l  Robo t s  a nd  A pp l i c a t io n s

4 .  Mac h i n e  L ea rn i n g  a nd  P a t t e rn  Re c ogn i t i o n

5 .  S pee c h  P ro c e s s i n g  I :  A c qu i si t io n  o f  S pee c h ,  Fea t u re  E x t ra c t i o n  a nd  S pea k e r  L o c a l i z a t i o n

6 .  S pee c h  P ro c e s s i n g  I I :  S pea k e r  I d e n t i f i c a ti o n  a nd  S pee c h  Rec ogn i ti o n

7 .   Image  P ro c e s s i n g  I :  Image  A c qu i s i t io n ,  P re -­p ro c e s s in g  a nd  Fea t u re  E x t ra c t i o n

8 .   Image  P ro c e s s i n g I I :  Fa c e  De t e c t i o n  a nd  Fa c e  Rec ogn i t i o n

9 .  Use r  Mode l l i ng

10 .  Mu l t i moda l  Human -­Robo t  I n t e ra c t i o n

COURSE OUTLINE

28. 07. 2015 AALBO RG   UNI VERS I T Y 20