emotion recognition
DESCRIPTION
TRANSCRIPT
1
Multimodal emotion recognition and expressivity
2
Reference
S. Kollias, K. Karpouzis, “Multimodal emotion recognition and expressivity,” Multimedia and Expo, 2005. ICME 2005. IEEE International Conference on , 6-8 July 2005, p.p. 779- 783
3
Introduction
People express their emotions through multiple modalities Humans’ speech Facial expressions Body pose
Emotional feature and signs
4
Recognition of the user’s emotional state
Emotion analysis and recognition Audio Visual Physiological signal
Emotional psychological background
Human computer interaction (HCI)
5
Emotional speech analysis
Speech is a major channel for communicating emotion
Speech signal conveys Textual, lexical, emotional and gestural i
nformation The set of features in the speech signa
l Classification algorithm
6
Emotion recognition system
7
Paralinguistic speech analysis
Prosody is composed of Intonation Duration Intensity Speech quality
Voice quality is influenced by physiological factors
8
Feature extraction
Extracting information from Pitch contour, range, variance, mean,
jitter, intensity, shimmer Voice quality Duration : pauses, speaking rate Background information on the
speaker
9
Emotional facial analysis
Facial action coding system (FACS) Facial definition parameter (FDP) Facial animation parameter (FAP) MPEG-4 standard
10
Facial animation
11
Emotional Gesture Analysis Hand tracking systems Tracking the centroid of skin masks Estimates of user’s movements
12
Gesture recognition
13
Targeting Emotion Recognition Facial animation parameter from the
user’s face Future merging of different emotional
representations
14
Targeting Expressivity
Facial Expressivity Time-varying facial movements
Quantity and quality of movement Interaction Transition
Gesture Expressivity Speed, acceleration, direction variation
15
Physiologocal signal analysis Visceral differences between emotion
al states Heart rate Skin conductance level Finger temperature Muscle activity
16
Measurementwith physiological information
Biosensor The value of skin conductivity Electromyography (EMG) sensors for
muscle-activity
17
Multimodal emotion recognition
Define the processes and functions Visual, auditory and physiological modali
ties Identify different emotions in the reco
gnition processes Synchronization and temporal seque
nce in different modalities
18
Conclusions
Multimodal emotion recognition and expressivity analysis
Human computer interaction (HCI) Pattern recognition in combination wi
th different techniques