Evaluation of a Facial Expression Technique Portraying Deception

Sunee Holland

University of South AustraliaSchool of Computer and Information Science

Supervisor: Dr G Stewart Von Itzstein

Presentation Overview

What is Facial Expression Synthesis? Background Motivation Methodology

› Research Questions Implementation

› Why the Source SDK? › System Architecture› Script Design› Voice Recording› Source SDK – Face Poser› Source SDK – Hammer

User Study Results Conclusion

What is Facial Expression Synthesis?

Simulation of human facial communication into graphical form

The goal is to increase realism and enhance the user’s experience

Can extend facial expression synthesis techniques to show deception leakage in virtual faces

Background

Techniques for facial expression synthesis› Motion capture› Muscle based actuation› Deformations› Morphing or blending

Toolkits› Source SDK (Face Poser), Xface, BEAT, Face

Toolkit, Expression

Background

Embodied Conversational Agents (ECAs)› Realistic, virtual avatars› Gestures› Facial expressions› Speech

Psychology› Deceptive expressions in humans› Paul Ekman’s universal emotions› Micro expressions› Duchenne de Boulogne

Motivation

The aim of this research is to evaluate the means of communication between human and computer› Specifically, deception in virtual agents› Using micro expressions

Literature covers how facial expression synthesis is performed

We want to focus on what these systems can be used for

Methodology

Question the feasibility of deceptive facial expression synthesis

Develop a deceptive facial expression synthesis technique

Evaluate this technique through a user study

Research Questions

Is it feasible to create a facial expression synthesis technique that portrays deception?› Is the user able to detect and recognise

deception leakage on a computer generated model?

› What effect does the synthesised deception leakage of an animated 3D character have on the user's experience?

Research Questions

What are the challenges in implementing a technique for synthesising deception leakage in computer-generated characters?› What existing software is available for

synthesising facial deception leakage and what is the optimal choice for portraying deception?

› How can deception be implemented in facial expression synthesis with the tools currently available?

Implementation

Using the Source SDK Design the scenarios used in the

experiment Generate audio dialogue to create

choreographies in Face Poser Use those choreographies to make

maps in Hammer Map runs through the Source Engine in

Half Life 2

Why the Source SDK?

Other toolkits evaluated: › Xface, rFace, Expression

Need high level of detail so that subtlety can be accurately portrayed

Script Design

Need to create a script for the scenarios to be included in the experiment

As we want to evaluate the expressions only, must remove other confounding factors

Contextual bias could be a confounding factor› Bad: “I can’t wait to see your parents. It will be great”› Good: “I can’t wait to see my parents. It will be great”

Need to express emotions equally› Anger, disgust, fear, happiness, sadness and surprise,

Duchenne (genuine) smile and Pan American (non-genuine) smile

Scenarios need to be realistic

Voice Recording

Choosing voice actors› One male and one female

They were given a script and asked to voice the dialogue› Recorded using computer software› Encoded into a format Face Poser works with› WAV 16-bit PCM

The speech was required to be said in a neutral tone as to avoid bias induced in the voice

Source SDK – Face Poser

SDK tool that produces choreographed sequences

Audio files are imported› Phoneme extraction/lip synching

Choreographies are created using a timeline of events› Contains expressions and audio

Result is exported into Valve Choreography Data (VCD) file

Source SDK – Face Poser

Source SDK - Hammer

SDK tool to create, edit, and export maps for Source Engine games

Build the physical geometry Place entities in the map

› Player spawn location, triggers, map changes, choreographies, characters

Place an event on a generic character that links to a choreography file

Compile and run through Half Life 2

Source SDK - Hammer

System Architecture

User Study

In the process of finalising the experiment Participants

› Ideal population: over 30 participants› Students, researchers and staff

Training phase› Some people will have a predisposition to

recognising deception› Reduce this confounding factor by placing all

participants on a similar experience level

User Study

1. Evaluating the recognition of expressions› How well does the technique express

emotion?› User is shown an expression or scenario

and asked whether they thought it was genuine or deceptive, and/or what emotion they thought they were perceiving

User Study

2. Evaluating subjective user experience› What level of quality is the interaction

between computer and user?› Participants will be asked to fill out a

feedback form › Subjective feedback wrt how easily they

could identify deception and emotions

Results

Predict that there will be difficulty in accurately recognising deception› Similar with humans

However, it should show a recognition rate that is slightly greater than chance› This is all that is required to validate the

research

Conclusion

The main contribution of this research is the evaluation of facial expression synthesis portraying deception

The user study will answer the question regarding feasibility through recognition rate and subjective user experience

Question regarding challenges is answered in the implementation through selecting a pre existing solution and describing how the deception was implemented

Future Work

More complex facial expressions More ways of portraying deception

other than facially› Body leakage is important when detecting

deception Possible applications in gaming to

improve the immersive experience

Questions?

Experimental Software

Want a way to randomise the choreography files for every participant in the experiment

Solution: Custom software that inputs files from a location, randomises the ordering, then outputs in the Half Life 2 scenes directory

Written in C#