facial expression analysis in mpeg-4 sequences

8/19/2019 FACIAL EXPRESSION ANALYSIS IN MPEG-4 SEQUENCES

http://slidepdf.com/reader/full/facial-expression-analysis-in-mpeg-4-sequences 1/4

FACIAL EXPRESSION ANALYSIS

IN MPEG-4 SEQUENCES

Amaryllis Raouzaiou, Kostas Karpouzis and Stefanos Kollias

Image, Video and Multimedia Systems Lab - Dept of Eletrial and !omputer Engineering

"ational #e$nial %ni&ersity of At$ens

'eroon (olyte$niou ), *+ .ograp$ou, /REE!E

#el01 23*4 556)* 7 8a91 23*4 556)5

email1 araouz:softlab0ee0ntua0gr

Abstract

;$ile t$e pre&ious M(E/ standards fous primarily on &ideo oding and transmission issues, M(E/-6

onentrates on $ybrid oding of natural and synt$eti data streams0 In t$is frame<or=, possible appliationsinlude teleonferening and entertainment appliations, <$ere an adaptable synt$eti agent substitutes t$e

atual user0 Su$ agents an interat <it$ ea$ ot$er, reei&ing input from multi-sensor data, and utilize $ig$-

le&el information, su$ as deteted emotions and e9pressions0 #$is greatly en$anes $uman-omputer

interation, by replaing single media representations <it$ dynami renderings, <$ile pro&iding feedba= on

t$e users> emotional status and reations0 Eduational en&ironments, &irtual ollaboration en&ironments and

online s$opping and entertainment appliations are e9peted to profit from t$is onept0 8aial e9pression

synt$esis and animation, in partiular, is gi&en mu$ attention <it$in t$e M(E/-6 frame<or=, <$ere $ig$er-

le&el, e9pliit 8aial Animation (arameters 28A(s4 $a&e been dediated to t$is purpose0 In t$is <or=, <e

employ general purpose 8A(s so as to redue t$e definition of faial e9pressions for synt$esis purposes, by

estimating t$e atual e9pression as a ombination of uni&ersal ones0 In addition, <e pro&ide e9pliit features,

as <ell as possible &alues for t$e 8A(s implementation, <$ile forming a relation bet<een 8A(s and t$eati&ation parameter proposed in lassi psy$ologial studies0

1. INTRODUCTION

#$e establis$ment of t$e M(E/-6 standard failitates an alternati&e <ay of analyzing and modeling faial

e9pressions and related emotions0 8aial Animation (arameters 28A(s4 are utilized in t$e frame<or= of

M(E/-6 for faial animation purposes, so as to enable effiient $ybrid oding of synt$eti ob?ets <it$

natural &ideo0 #$is enables animators to fous on loal or global ations on t$e fae, by means of @sripting

an animation seBuene0 8or e9ample, t$e animator an instrut t$e synt$eti model of a $uman fae to

@open mout$ or @lo<er eyebro< 2see 8igure *4C in essene, t$is instrution is passed to t$e M(E/-6

deoder, <$i$ in turn deforms t$e model by translating t$e &erties t$at orrespond to t$e area in Buestion

2see 8igure 540 ;$ile t$e standard does ater for t$e abstrat definition of e9pressions and emotion as aolletion of 8A(s and t$eir subseBuent interpolation into intermediate e9pression, t$is does not neessarily

mean t$at all possible e9pressions and emotions an be modeled t$is <ay *0 In general, faial e9pression

analysis $as been mainly onentrated on si9 e9pressions, termed as @uni&ersal0 #$is term means t$at

$umans aross different ultures an easily reognize e9pressions su$ as joy or disgust 50 Fne an

ombine different uni&ersal e9pressions to pro&ide intermediate ones, su$ as fa=e ?oy or upset 2see 8igure

4, or a number of emotional states, su$ as pain0

#$e re&erse problem, t$at is t$e identifiation of t$e uni&ersal e9pressions t$at must be ombined to result to

a gi&en intermediate e9pression is not al<ays lear-ut0 In t$e Buest of forming a lo<-dimensional spae in

<$i$ distane measures an be defined, notions su$ as t$e Feeltrace plane 2see 8igure 64, defined by t$e

activation and evaluation a9es may be used to di&ersify t$e proess of synt$esizing an intermediatee9pression0 #$ese notions originate from psy$ologial studies and an be e9ploited so as to mo&e from

features ompre$ensible by $umans to Buantitati&e measurements, su$ as 8A(s0 #$is an be



aomplis$ed by re&ersing of t$e desription of t$e si9 uni&ersal emotions <it$ M(E/-6 8A(s and use of a

priori =no<ledge t$at is embedded <it$in a fuzzy rule system0 Geause 8A(s do not orrespond to speifi

models or polygonal topologies, t$is s$eme an be e9tended to ot$er models or $araters, different from

t$e one t$at <as analyzed0

8igure *1 A fae model in itsneutral state

8igure 51 Deformation along t$eeyebro< area

8igure 1 #$e omplete definition of t$e emotion @upset

8igure 61 #$e 8eeltrae plane

2. MPEG-4 AND FACIAL ANIMATION

#$e definition parameters defined by t$e M(E/ /roup allo< for a detailed definition of bodyHfae s$ape, size

and te9ture, <$ile t$e animation parameters failitate t$e definition of faial e9pressions and body postures

60 #$ese parameters are designed to aommodate all natural possible e9pressions and postures, as <ell

as e9aggerated e9pressions and motions, t$us o&ering not only representation purposes, but also

entertainment as <ell0

As far as deoding is onerned, 8A(s manipulate ontrol points on a D model of t$e fae so to produe

animation of t$e $ead and faial features li=e t$e mout$, t$e eyes or t$e eyebro<s0 All t$e 8A( t$at in&ol&e

translation are e9pressed in terms of 8aial Animation (arameter %nits 28A(%40 #$ese units are defined <it$

respet to standard faial features, su$ as eye distane and allo< t$e interpretation of 8A(s on any faial

model in a onsistent, reasonable <ay0 #$is parameter set also ontains t<o $ig$ le&el parameters1 t$e

viseme parameter, <$i$ allo<s rendering of t$e &isual aspet of t$e lo<er part of t$e fae <it$out t$e needto e9press t$em in terms of ot$er parameters, and t$e expression parameter allo<s t$e definition of si9 $ig$

le&el faial e9pressions0

All t$ese animation @instrutions are fed to an M(E/-6 deoder0 #$is part of t$e system may animate a

speifi D model transmitted one as part of t$e ommuniation or use a generi faial model apable of

interpreting 8A(s0 If t$e appliation aims to repliate a gi&en $uman $ead, it may $oose to modify t$e s$ape

and appearane of t$e fae aordingly0 In t$is ase, t$e definition of its geometry and topology is neessary0

#$is definition is enoded in 8D(s, normally transmitted one per session, follo<ed by a stream of

ompressed 8A(s0 #$e distint 8D( fields are1

• 8eature(oints!oord 7 t$e atual D feature points for t$e alibration of t$e fae model

• #e9ture!oords 7 te9ture oordinates for t$e feature points

• #e9ture#ype 7 $ints to t$e deoder on t$e type of te9ture image0



• 8aeDef#ables - desribe t$e be$a&ior of 8A(s <0r0t t$e geometry deformation

• 8aeSene/rap$ - ontains t$e te9ture image or groups node for t$e model $ierar$y

. GOING FROM INTERMEDIATE EXPRESSIONS TO UNI!ERSAL

/rading of 8A(s is strongly related to t$e ati&ation parameter proposed by ;$issel +0 Sine t$is relation is

e9pressed in a different <ay for ea$ partiular e9pression, a fuzzy rule system seems appropriate for

mapping 8A(s to t$e ati&ation a9is0 As a general rule, one an define si9 general ategories, ea$ one

$araterized by a fundamental uni&ersal emotionC <it$in ea$ of t$ese ategories intermediate e9pressions

are desribed by different emotional and optial intensities, as <ell as minor &ariation in e9pression details0

8rom t$e synt$eti point of &ie<, e9pressions and emotions t$at belong in t$e same ategory an be

rendered by animating t$e same 8A(s in different intensities0 8or e9ample, t$e emotion group @fear also

ontains @<orry and @terrorC t$ese t<o emotions an be synt$esized by reduing or inreasing t$e

intensities of t$e employed 8A(s, respeti&ely0 #$e same rationale an also be applied in t$e group of

@disgust t$at also ontains @disdain and @repulsionC t$e fuzziness t$at is introdued by t$e &arying sale of

t$e $ange of 8A( intensity also pro&ides assistane in differentiating mildly t$e output in similar situations0

#$is ensures t$at t$e synt$esis <ill not render @robot-li=e animation, but drastially more realisti results0

During t$is proess, one an utilize fuzzy rules to go from t$e transmitted 8A(s to uni&ersal e9pressionsCt$ese rules stem from obser&ation and are fortified <it$ results from t$e analysis of atual &ideo

seBuenes0 #$e initial rules $a&e t$e form s$o<n in #able *, <$ere t$e symbol @J denotes t$e presene of a

partiular atomi ation in t$e $uman fae0 #$ese ations are on&erted to groups of 8A(s t$at map to t$e

faial area in BuestionC t$e &alues of t$ese 8A(s are initialized <it$ measurements from &ideo seBuenes

t$at orrespond to t$e analyzed e9pression 0 As a result, t$e representation of an e9pression as a

olletion of 8A(s may be redued to t$e estimation of t$e intensities of t<o uni&ersal e9pressions, <$i$

an be part of t$e lient deoder0

Rela9edLo<ered

RaisedInner ar$ Inner part

more less more less more less!ry J J

Almost rying J

Depressed J

Sadness J

#able *1 Rules for t$e mapping of eyebro< 8A(s to intermediate e9pressions

4. REFERENCES

* K0 Karpouzis, "0 #sapatsoulis, S0 Kollias, @Mo&ing to ontinuous faial e9pression spae using t$e

M(E/-6 faial definition parameter set, S(IE Eletroni Imaging 5333, anuary 5333, San ose, !A,

%SA

5 80 (ar=e and K0 ;aters, !omputer 8aial Animation, A K (eters, *))

R0 !o<ie R0 and E0 Douglas-!o<ie, @Automati statistial analysis of t$e signal and prosodi signs of

emotion in spee$, (ro0 of Intern0 !onferene on Spo=en Language (roessing, ($iladelp$ia, *))

6 ;0 S0 Lee, M0 Es$er, /0 Sannier, "0 Magnenat-#$almann, @M(E/-6 !ompatible 8aes from Frt$ogonal

($otos, !omputer Animation *))), /ene&a, S<itzerland, *)))

+ !0 M0 ;$issel, #$e ditionary of affet in language, R0 (lut$ni= and '0 Kellerman 2Eds4 @Emotion1

#$eory, resear$ and e9periene1 &ol 6, #$e measurement of emotions, Aademi (ress, "e< or=,

*))

/0 8aigin, N#$e ArtistOs !omplete /uide to 8aial E9pressionsN, ;atson-/uptill, "e< or=, *))3

facial expression analysis in mpeg-4 sequences

Documents